sched: Fix smp_call_function_single_async() usage for ILB
authorPeter Zijlstra <peterz@infradead.org>
Tue, 26 May 2020 16:10:58 +0000 (18:10 +0200)
committerIngo Molnar <mingo@kernel.org>
Thu, 28 May 2020 08:54:15 +0000 (10:54 +0200)
commit19a1f5ec699954d21be10f74ff71c2a7079e99ad
tree4ec037e6193e760846705d2a9bcb9948f03187aa
parent58ef57b16d9e91cce1c640a6fe8a21d53a85181d
sched: Fix smp_call_function_single_async() usage for ILB

The recent commit: 90b5363acd47 ("sched: Clean up scheduler_ipi()")
got smp_call_function_single_async() subtly wrong. Even though it will
return -EBUSY when trying to re-use a csd, that condition is not
atomic and still requires external serialization.

The change in kick_ilb() got this wrong.

While on first reading kick_ilb() has an atomic test-and-set that
appears to serialize the use, the matching 'release' is not in the
right place to actually guarantee this serialization.

Rework the nohz_idle_balance() trigger so that the release is in the
IPI callback and thus guarantees the required serialization for the
CSD.

Fixes: 90b5363acd47 ("sched: Clean up scheduler_ipi()")
Reported-by: Qian Cai <cai@lca.pw>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
Cc: mgorman@techsingularity.net
Link: https://lore.kernel.org/r/20200526161907.778543557@infradead.org
kernel/sched/core.c
kernel/sched/fair.c
kernel/sched/sched.h