rcu/kvfree: Carefully reset number of objects in krcp
authorUladzislau Rezki (Sony) <urezki@gmail.com>
Wed, 14 Dec 2022 12:06:29 +0000 (13:06 +0100)
committerPaul E. McKenney <paulmck@kernel.org>
Wed, 4 Jan 2023 01:48:41 +0000 (17:48 -0800)
commit4c33464ae85e59cba3f8048a34d571edf229823a
treec748c0d8a218d6197d379db9047aeece1e72dcb6
parent9627456101ec9bb502daae7276e5141f66a9ddd1
rcu/kvfree: Carefully reset number of objects in krcp

The schedule_delayed_monitor_work() function relies on the count of
objects queued into any given kfree_rcu_cpu structure.  This count is
used to determine how quickly to schedule passing these objects to RCU.

There are three pipes where pointers can be placed.  When any pipe is
offloaded, the kfree_rcu_cpu structure's ->count counter is set to zero,
which is wrong because the other pipes might still be non-empty.

This commit therefore maintains per-pipe counters, and introduces a
krc_count() helper to access the aggregate value of those counters.

Signed-off-by: Uladzislau Rezki (Sony) <urezki@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
kernel/rcu/tree.c