Merge branch 'optimize-busy-poll' of https://github.com/ketor/fio

Ketor writes:

Origin code use rbd_aio_is_complete to check if the io is complete.But
this function use mutex lock and unlock to get the complete result, this
will cause the thread context switch and lock wait latency. And also
because the lock and context switch latency is in the getevents thread,
will decrease the getevents speed and increase the test latency.

This patch go back to use an varible io_complete to check if the io is
complete. This variable is updated in the aiocb callback function and
will not influence the getevents speed.

In our practice, we run fio test jobs=1 iodepth=1 bs=4k busy_poll=1.
Before this patch: IOPS is 30930 After this patch: IOPS is 37300.  And
also after this patch we get avg 25us latency, is nearly equal to the
systemtap test result of the librbd randwrite process.

So I think this patch is realy needed.

diff --git a/engines/rbd.c b/engines/rbd.c
index 363be65bea58ded83e37eb7b867283d6aada67f8..52fc8c9f566e174b0c8d3173713233d25a66271e 100644 (file)
--- a/engines/rbd.c
+++ b/engines/rbd.c
@@ -13,6 +13,7 @@ struct fio_rbd_iou {
        struct io_u *io_u;
        rbd_completion_t completion;
        int io_seen;
+       int io_complete;
 };
 
 struct rbd_data {
@@ -186,6 +187,8 @@ static void _fio_rbd_finish_aiocb(rbd_completion_t comp, void *data)
         * a specific error. So we have to assume that it can't do
         * partial completions.
         */
+       fri->io_complete = 1;
+       
        ret = rbd_aio_get_return_value(fri->completion);
        if (ret < 0) {
                io_u->error = ret;
@@ -207,7 +210,7 @@ static inline int fri_check_complete(struct rbd_data *rbd_data,
 {
        struct fio_rbd_iou *fri = io_u->engine_data;
 
-       if (rbd_aio_is_complete(fri->completion)) {
+       if (fri->io_complete) {
                fri->io_seen = 1;
                rbd_data->aio_events[*events] = io_u;
                (*events)++;
@@ -283,6 +286,7 @@ static int fio_rbd_queue(struct thread_data *td, struct io_u *io_u)
        fio_ro_check(td, io_u);
 
        fri->io_seen = 0;
+       fri->io_complete = 0;
 
        r = rbd_aio_create_completion(fri, _fio_rbd_finish_aiocb,
                                                &fri->completion);