io_uring/io-wq: move worker lists to struct io_wq_acct
authorMax Kellermann <max.kellermann@ionos.com>
Tue, 28 Jan 2025 13:39:22 +0000 (14:39 +0100)
committerJens Axboe <axboe@kernel.dk>
Mon, 17 Feb 2025 12:34:45 +0000 (05:34 -0700)
commit751eedc4b4b79332ecf1a78c0dbeb47d573a8f59
tree6686d9932955333662c6b6b5528bb5b415275bcf
parent3d3bafd35fb422eb36cfc5709473cef7400588e7
io_uring/io-wq: move worker lists to struct io_wq_acct

Have separate linked lists for bounded and unbounded workers.  This
way, io_acct_activate_free_worker() sees only workers relevant to it
and doesn't need to skip irrelevant ones.  This speeds up the
linked list traversal (under acct->lock).

The `io_wq.lock` field is moved to `io_wq_acct.workers_lock`.  It did
not actually protect "access to elements below", that is, not all of
them; it only protected access to the worker lists.  By having two
locks instead of one, contention on this lock is reduced.

Signed-off-by: Max Kellermann <max.kellermann@ionos.com>
Link: https://lore.kernel.org/r/20250128133927.3989681-4-max.kellermann@ionos.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
io_uring/io-wq.c