rcu-tasks: Track blocked RCU Tasks Trace readers
authorPaul E. McKenney <paulmck@kernel.org>
Tue, 17 May 2022 18:30:32 +0000 (11:30 -0700)
committerPaul E. McKenney <paulmck@kernel.org>
Mon, 20 Jun 2022 16:22:28 +0000 (09:22 -0700)
commit0356d4e66214569de674ab2684f2e0b440a466ab
treefeb5195f3661d9dedb2c443f13d8d24285cde5e9
parent434c9eefb959c36331a93617ea95df903469b99f
rcu-tasks: Track blocked RCU Tasks Trace readers

This commit places any task that has ever blocked within its current
RCU Tasks Trace read-side critical section on a per-CPU list within the
rcu_tasks_percpu structure.  Tasks are removed from this list when they
exit by the exit_tasks_rcu_finish_trace() function.  The purpose of this
commit is to provide the information needed to eliminate the current
scan of the full task list.

This commit offsets the INT_MIN value for ->trc_reader_nesting with the
new nesting level in order to avoid queueing tasks that are exiting
their read-side critical sections.

[ paulmck: Apply kernel test robot feedback. ]
[ paulmck: Apply feedback from syzbot+9bb26e7c5e8e4fa7e641@syzkaller.appspotmail.com ]

Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Tested-by: syzbot <syzbot+9bb26e7c5e8e4fa7e641@syzkaller.appspotmail.com>
Tested-by: "Zhang, Qiang1" <qiang1.zhang@intel.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Neeraj Upadhyay <quic_neeraju@quicinc.com>
Cc: Eric Dumazet <edumazet@google.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Andrii Nakryiko <andrii@kernel.org>
Cc: Martin KaFai Lau <kafai@fb.com>
Cc: KP Singh <kpsingh@kernel.org>
include/linux/rcupdate.h
include/linux/rcupdate_trace.h
kernel/rcu/tasks.h