nvme-pci: handle completions outside of the queue lock
authorJens Axboe <axboe@kernel.dk>
Fri, 18 May 2018 21:29:22 +0000 (15:29 -0600)
committerJens Axboe <axboe@kernel.dk>
Fri, 18 May 2018 21:29:22 +0000 (15:29 -0600)
commite96e21d89ffe9e45978b540e7f396cf24a0d5ec4
tree7b4e56c4202db3fb3d92ec3672bf6b4791e292d3
parent59e8cedad1851f60fb59555cc7d599a5bb413d2e
nvme-pci: handle completions outside of the queue lock

Split the completion of events into a two part process:

1) Reap the events inside the queue lock
2) Complete the events outside the queue lock

Since we never wrap the queue, we can access it locklessly after we've
updated the completion queue head. This patch started off with batching
events on the stack, but with this trick we don't have to. Keith Busch
<keith.busch@intel.com> came up with that idea.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
Signed-off-by: Keith Busch <keith.busch@intel.com>
[hch: refactored, restored poll early exit optimization]
Signed-off-by: Christoph Hellwig <hch@lst.de>
drivers/nvme/host/pci.c