mm, bpf: Introduce free_pages_nolock()
authorAlexei Starovoitov <ast@kernel.org>
Sat, 22 Feb 2025 02:44:24 +0000 (18:44 -0800)
committerAlexei Starovoitov <ast@kernel.org>
Thu, 27 Feb 2025 17:36:18 +0000 (09:36 -0800)
commit8c57b687e8331eb80e302a2c528b18b966a9ac7a
tree4ce4359d87a9b34917d0e41359f2ed2dfa3b6e40
parent97769a53f117e2f33864c587d85992ee35194ecf
mm, bpf: Introduce free_pages_nolock()

Introduce free_pages_nolock() that can free pages without taking locks.
It relies on trylock and can be called from any context.
Since spin_trylock() cannot be used in PREEMPT_RT from hard IRQ or NMI
it uses lockless link list to stash the pages which will be freed
by subsequent free_pages() from good context.

Do not use llist unconditionally. BPF maps continuously
allocate/free, so we cannot unconditionally delay the freeing to
llist. When the memory becomes free make it available to the
kernel and BPF users right away if possible, and fallback to
llist as the last resort.

Acked-by: Vlastimil Babka <vbabka@suse.cz>
Acked-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Reviewed-by: Shakeel Butt <shakeel.butt@linux.dev>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/r/20250222024427.30294-4-alexei.starovoitov@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
include/linux/gfp.h
include/linux/mm_types.h
include/linux/mmzone.h
lib/stackdepot.c
mm/page_alloc.c
mm/page_owner.c