mm: enable page walking API to lock vmas during the walk
authorSuren Baghdasaryan <surenb@google.com>
Fri, 4 Aug 2023 15:27:19 +0000 (08:27 -0700)
committerAndrew Morton <akpm@linux-foundation.org>
Mon, 21 Aug 2023 20:07:20 +0000 (13:07 -0700)
commit49b0638502da097c15d46cd4e871dbaa022caf7c
treea36bae8bb4dd533dbe491ae25c3e506cd4dfb679
parent8b9c1cc0418a43196477083e7082568e7a4c9418
mm: enable page walking API to lock vmas during the walk

walk_page_range() and friends often operate under write-locked mmap_lock.
With introduction of vma locks, the vmas have to be locked as well during
such walks to prevent concurrent page faults in these areas.  Add an
additional member to mm_walk_ops to indicate locking requirements for the
walk.

The change ensures that page walks which prevent concurrent page faults
by write-locking mmap_lock, operate correctly after introduction of
per-vma locks.  With per-vma locks page faults can be handled under vma
lock without taking mmap_lock at all, so write locking mmap_lock would
not stop them.  The change ensures vmas are properly locked during such
walks.

A sample issue this solves is do_mbind() performing queue_pages_range()
to queue pages for migration.  Without this change a concurrent page
can be faulted into the area and be left out of migration.

Link: https://lkml.kernel.org/r/20230804152724.3090321-2-surenb@google.com
Signed-off-by: Suren Baghdasaryan <surenb@google.com>
Suggested-by: Linus Torvalds <torvalds@linuxfoundation.org>
Suggested-by: Jann Horn <jannh@google.com>
Cc: David Hildenbrand <david@redhat.com>
Cc: Davidlohr Bueso <dave@stgolabs.net>
Cc: Hugh Dickins <hughd@google.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Laurent Dufour <ldufour@linux.ibm.com>
Cc: Liam Howlett <liam.howlett@oracle.com>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Michel Lespinasse <michel@lespinasse.org>
Cc: Peter Xu <peterx@redhat.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
18 files changed:
arch/powerpc/mm/book3s64/subpage_prot.c
arch/riscv/mm/pageattr.c
arch/s390/mm/gmap.c
fs/proc/task_mmu.c
include/linux/pagewalk.h
mm/damon/vaddr.c
mm/hmm.c
mm/ksm.c
mm/madvise.c
mm/memcontrol.c
mm/memory-failure.c
mm/mempolicy.c
mm/migrate_device.c
mm/mincore.c
mm/mlock.c
mm/mprotect.c
mm/pagewalk.c
mm/vmscan.c