kvfree_rcu: Directly allocate page for single-argument case
authorUladzislau Rezki (Sony) <urezki@gmail.com>
Wed, 20 Jan 2021 16:21:46 +0000 (17:21 +0100)
committerPaul E. McKenney <paulmck@kernel.org>
Mon, 8 Mar 2021 22:18:07 +0000 (14:18 -0800)
commit148e3731d124079a036b3acf780f3d35c1b9c0aa
tree4944f96134a9c4d3b4dfb6407e05249bb3dca680
parenta38fd8748464831584a19438cbb3082b5a2dab15
kvfree_rcu: Directly allocate page for single-argument case

Single-argument kvfree_rcu() must be invoked from sleepable contexts,
so we can directly allocate pages.  Furthermmore, the fallback in case
of page-allocation failure is the high-latency synchronize_rcu(), so it
makes sense to do these page allocations from the fastpath, and even to
permit limited sleeping within the allocator.

This commit therefore allocates if needed on the fastpath using
GFP_KERNEL|__GFP_RETRY_MAYFAIL.  This also has the beneficial effect
of leaving kvfree_rcu()'s per-CPU caches to the double-argument variant
of kvfree_rcu(), given that the double-argument variant cannot directly
invoke the allocator.

[ paulmck: Add add_ptr_to_bulk_krc_lock header comment per Michal Hocko. ]
Signed-off-by: Uladzislau Rezki (Sony) <urezki@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
kernel/rcu/tree.c