xprtrdma: Use xprt_pin_rqst in rpcrdma_reply_handler
authorChuck Lever <chuck.lever@oracle.com>
Wed, 23 Aug 2017 21:05:58 +0000 (17:05 -0400)
committerTrond Myklebust <trond.myklebust@primarydata.com>
Tue, 5 Sep 2017 22:27:07 +0000 (18:27 -0400)
commit9590d083c1bb1419b7992609d1a0a3e3517d3893
treeed285e1635c2232d66d432835fc13890ef209718
parentf9773b22a27a4234f436c9570afd62d905e00a13
xprtrdma: Use xprt_pin_rqst in rpcrdma_reply_handler

Adopt the use of xprt_pin_rqst to eliminate contention between
Call-side users of rb_lock and the use of rb_lock in
rpcrdma_reply_handler.

This replaces the mechanism introduced in 431af645cf66 ("xprtrdma:
Fix client lock-up after application signal fires").

Use recv_lock to quickly find the completing rqst, pin it, then
drop the lock. At that point invalidation and pull-up of the Reply
XDR can be done. Both are often expensive operations.

Finally, take recv_lock again to signal completion to the RPC
layer. It also protects adjustment of "cwnd".

This greatly reduces the amount of time a lock is held by the
reply handler. Comparing lock_stat results shows a marked decrease
in contention on rb_lock and recv_lock.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
[trond.myklebust@primarydata.com: Remove call to rpcrdma_buffer_put() from
   the "out_norqst:" path in rpcrdma_reply_handler.]
Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
net/sunrpc/xprt.c
net/sunrpc/xprtrdma/rpc_rdma.c
net/sunrpc/xprtrdma/transport.c
net/sunrpc/xprtrdma/verbs.c
net/sunrpc/xprtrdma/xprt_rdma.h