From: Bart Van Assche <bvanassche@acm.org>
Date: Sat, 4 Jul 2020 16:44:56 +0000 (-0700)
Subject: workqueue: Fix race conditions in the workqueue mechanism
X-Git-Tag: fio-3.21~14^2
X-Git-Url: https://git.kernel.dk/?a=commitdiff_plain;h=3d86aaaa12eb95934a1f9e9303008ff958da66c5;p=fio.git

workqueue: Fix race conditions in the workqueue mechanism

From https://pubs.opengroup.org/onlinepubs/9699919799/functions/pthread_cond_wait.html:
"The application shall ensure that these functions are called with mutex
locked by the calling thread; otherwise, an error (for
PTHREAD_MUTEX_ERRORCHECK and robust mutexes) or undefined behavior (for
other mutexes) results.

From https://pubs.opengroup.org/onlinepubs/9699919799/functions/pthread_cond_signal.html:
"however, if predictable scheduling behavior is required, then that mutex
shall be locked by the thread calling pthread_cond_broadcast() or
pthread_cond_signal()."

Hence always hold the associated mutex around pthread_cond_wait() and
pthread_signal() calls.

This patch fixes the hangs reported by Travis and Appveyor for test case
t0013.

Signed-off-by: Bart Van Assche <bvanassche@acm.org>
---

diff --git a/workqueue.c b/workqueue.c
index 254bf6b6..9e6c41ff 100644
--- a/workqueue.c
+++ b/workqueue.c
@@ -170,26 +170,32 @@ static void *worker_thread(void *data)
 				workqueue_pre_sleep(sw);
 				pthread_mutex_lock(&sw->lock);
 			}
-
-			/*
-			 * We dropped and reaquired the lock, check
-			 * state again.
-			 */
-			if (!flist_empty(&sw->work_list))
-				goto handle_work;
-
+		}
+		/*
+		 * We may have dropped and reaquired the lock, check state
+		 * again.
+		 */
+		if (flist_empty(&sw->work_list)) {
 			if (sw->flags & SW_F_EXIT) {
 				break;
-			} else if (!(sw->flags & SW_F_IDLE)) {
+			}
+			if (!(sw->flags & SW_F_IDLE)) {
 				sw->flags |= SW_F_IDLE;
 				wq->next_free_worker = sw->index;
+				pthread_mutex_unlock(&sw->lock);
+				pthread_mutex_lock(&wq->flush_lock);
 				if (wq->wake_idle)
 					pthread_cond_signal(&wq->flush_cond);
+				pthread_mutex_unlock(&wq->flush_lock);
+				pthread_mutex_lock(&sw->lock);
+			}
+		}
+		if (flist_empty(&sw->work_list)) {
+			if (sw->flags & SW_F_EXIT) {
+				break;
 			}
-
 			pthread_cond_wait(&sw->cond, &sw->lock);
 		} else {
-handle_work:
 			flist_splice_init(&sw->work_list, &local_list);
 		}
 		pthread_mutex_unlock(&sw->lock);