mm: move FOLL_PIN debug accounting under CONFIG_DEBUG_VM for-6.3/iov-extract
authorJens Axboe <axboe@kernel.dk>
Tue, 31 Jan 2023 15:28:53 +0000 (08:28 -0700)
committerJens Axboe <axboe@kernel.dk>
Tue, 14 Feb 2023 23:03:47 +0000 (16:03 -0700)
commita8fcff618f9e78836780176d48ae75df34b84ef4
treec83e316db7f03db941912c75725b7e3205b1e9f8
parent651d77d89c16b6b8deb487ff66dc894cd99ba888
mm: move FOLL_PIN debug accounting under CONFIG_DEBUG_VM

Using FOLL_PIN for mapping user pages caused a performance regression of
about 2.7%. Looking at profiles, we see:

+2.71%  [kernel.vmlinux]  [k] mod_node_page_state

which wasn't there before. The node page state counters are percpu, but
with a very low threshold. On my setup, every 108th update ends up
needing to punt to two atomic_lond_add()'s, which is causing this above
regression.

As these counters are purely for debug purposes, move them under
CONFIG_DEBUG_VM rather than do them unconditionally. Note that this
commit does not fix a real bug with the commits identified as being
fixed, rather it ensures that we don't regress on performance due to
those commits moving to using FOLL_PIN rather than FOLL_GET.

Fixes: 33f432039135 ("block: convert bio_map_user_iov to use iov_iter_extract_pages")
Fixes: b699de6806c1 ("block: Convert bio_iov_iter_get_pages to use iov_iter_extract_pages")
Link: https://lore.kernel.org/linux-block/f57ee72f-38e9-6afa-182f-2794638eadcb@kernel.dk/
Link: https://lore.kernel.org/all/54b0b07a-c178-9ffe-b5af-088f3c21696c@kernel.dk/
Acked-by: David Hildenbrand <david@redhat.com>
Reviewed-by: John Hubbard <jhubbard@nvidia.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
include/linux/mmzone.h
mm/gup.c
mm/vmstat.c