block/mq-deadline: skip expensive merge lookups if contended
authorJens Axboe <axboe@kernel.dk>
Fri, 19 Jan 2024 15:16:36 +0000 (08:16 -0700)
committerJens Axboe <axboe@kernel.dk>
Mon, 1 Apr 2024 22:03:34 +0000 (16:03 -0600)
commiteba8080cf98c21e5d8713f6782b0fd2d9c8fd998
tree9997585d7c2b6eae0c3f34d56a98002706fbce04
parentea07e33ddbf5b41cafec9a7189bc2a7f079b3caf
block/mq-deadline: skip expensive merge lookups if contended

We do several stages of merging in the block layer - the most likely one
to work is also the cheap one, merging direct in the per-task plug when
IO is submitted. Getting merges outside of that is a lot less likely,
but IO schedulers may still maintain internal data structures to
facilitate merge lookups outside of the plug.

Make mq-deadline skip expensive merge lookups if the queue lock is
already contended. The likelihood of getting a merge here is not very
high, hence it should not be a problem skipping the attempt in the also
unlikely event that the queue is already contended.

Perf diff shows the difference between a random read/write workload
with 4 threads doing IO, with expensive merges turned on and off:

    25.00%    +61.94%  [kernel.kallsyms]  [k] queued_spin_lock_slowpath

where we almost quadruple the lock contention by attempting these
expensive merges.

Tested-by: Oleksandr Natalenko <oleksandr@natalenko.name>
Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
block/mq-deadline.c