Revert "md: unlock mddev before reap sync_thread in action_store"
authorYu Kuai <yukuai3@huawei.com>
Mon, 29 May 2023 13:20:32 +0000 (21:20 +0800)
committerSong Liu <song@kernel.org>
Thu, 27 Jul 2023 07:13:28 +0000 (00:13 -0700)
commita865b96c513bcaeec49669010d67c40aa8e58619
treed92d7c89a1d249ab2386f83dfde9d1a70634b62c
parent51d74ec9b62f5813767a60226acaf943e26e7d7a
Revert "md: unlock mddev before reap sync_thread in action_store"

This reverts commit 9dfbdafda3b34e262e43e786077bab8e476a89d1.

Because it will introduce a defect that sync_thread can be running while
MD_RECOVERY_RUNNING is cleared, which will cause some unexpected problems,
for example:

list_add corruption. prev->next should be next (ffff0001ac1daba0), but was ffff0000ce1a02a0. (prev=ffff0000ce1a02a0).
Call trace:
 __list_add_valid+0xfc/0x140
 insert_work+0x78/0x1a0
 __queue_work+0x500/0xcf4
 queue_work_on+0xe8/0x12c
 md_check_recovery+0xa34/0xf30
 raid10d+0xb8/0x900 [raid10]
 md_thread+0x16c/0x2cc
 kthread+0x1a4/0x1ec
 ret_from_fork+0x10/0x18

This is because work is requeued while it's still inside workqueue:

t1: t2:
action_store
 mddev_lock
  if (mddev->sync_thread)
   mddev_unlock
   md_unregister_thread
   // first sync_thread is done
md_check_recovery
 mddev_try_lock
 /*
  * once MD_RECOVERY_DONE is set, new sync_thread
  * can start.
  */
 set_bit(MD_RECOVERY_RUNNING, &mddev->recovery)
 INIT_WORK(&mddev->del_work, md_start_sync)
 queue_work(md_misc_wq, &mddev->del_work)
  test_and_set_bit(WORK_STRUCT_PENDING_BIT, ...)
  // set pending bit
  insert_work
   list_add_tail
 mddev_unlock
   mddev_lock_nointr
   md_reap_sync_thread
   // MD_RECOVERY_RUNNING is cleared
 mddev_unlock

t3:

// before queued work started from t2
md_check_recovery
 // MD_RECOVERY_RUNNING is not set, a new sync_thread can be started
 INIT_WORK(&mddev->del_work, md_start_sync)
  work->data = 0
  // work pending bit is cleared
 queue_work(md_misc_wq, &mddev->del_work)
  insert_work
   list_add_tail
   // list is corrupted

The above commit is reverted to fix the problem, the deadlock this
commit tries to fix will be fixed in following patches.

Signed-off-by: Yu Kuai <yukuai3@huawei.com>
Signed-off-by: Song Liu <song@kernel.org>
Link: https://lore.kernel.org/r/20230529132037.2124527-2-yukuai1@huaweicloud.com
drivers/md/dm-raid.c
drivers/md/md.c