vfs: move d_lockref out of the area used by RCU lookup
authorMateusz Guzik <mjguzik@gmail.com>
Thu, 13 Jun 2024 00:12:15 +0000 (02:12 +0200)
committerChristian Brauner <brauner@kernel.org>
Wed, 3 Jul 2024 08:34:10 +0000 (10:34 +0200)
commit18a5daf0e4974ded74b2ed8a6aed1da49bde356d
tree79de3344b1322d61d55815ae85ce43fa776aa0e2
parent267574dee6ae0da68f5b454a30ff276d45976cf8
vfs: move d_lockref out of the area used by RCU lookup

Stock kernel scales worse than FreeBSD when doing a 20-way stat(2) on
the same tmpfs-backed file.

According to perf top:
  38.09%  lockref_put_return
  26.08%  lockref_get_not_dead
  25.60%  __d_lookup_rcu
   0.89%  clear_bhb_loop

__d_lookup_rcu is participating in cacheline ping pong due to the
embedded name sharing a cacheline with lockref.

Moving it out resolves the problem:
  41.50%  lockref_put_return
  41.03%  lockref_get_not_dead
   1.54%  clear_bhb_loop

benchmark (will-it-scale, Sapphire Rapids, tmpfs, ops/s):
FreeBSD:7219334
before: 5038006
after: 7842883 (+55%)

One minor remark: the 'after' result is unstable, fluctuating in the
range ~7.8 mln to ~9 mln during different runs.

Signed-off-by: Mateusz Guzik <mjguzik@gmail.com>
Link: https://lore.kernel.org/r/20240613001215.648829-3-mjguzik@gmail.com
Signed-off-by: Christian Brauner <brauner@kernel.org>
include/linux/dcache.h