bpf: teach the verifier to enforce css_iter and task_iter in RCU CS
authorChuyi Zhou <zhouchuyi@bytedance.com>
Wed, 18 Oct 2023 06:17:43 +0000 (14:17 +0800)
committerAlexei Starovoitov <ast@kernel.org>
Fri, 20 Oct 2023 00:02:46 +0000 (17:02 -0700)
commitdfab99df147b0d364f0c199f832ff2aedfb2265a
tree6bbe3fa26b07cf39eaa8a0657f03079be79c85ef
parent7251d0905e7518bcb990c8e9a3615b1bb23c78f2
bpf: teach the verifier to enforce css_iter and task_iter in RCU CS

css_iter and task_iter should be used in rcu section. Specifically, in
sleepable progs explicit bpf_rcu_read_lock() is needed before use these
iters. In normal bpf progs that have implicit rcu_read_lock(), it's OK to
use them directly.

This patch adds a new a KF flag KF_RCU_PROTECTED for bpf_iter_task_new and
bpf_iter_css_new. It means the kfunc should be used in RCU CS. We check
whether we are in rcu cs before we want to invoke this kfunc. If the rcu
protection is guaranteed, we would let st->type = PTR_TO_STACK | MEM_RCU.
Once user do rcu_unlock during the iteration, state MEM_RCU of regs would
be cleared. is_iter_reg_valid_init() will reject if reg->type is UNTRUSTED.

It is worth noting that currently, bpf_rcu_read_unlock does not
clear the state of the STACK_ITER reg, since bpf_for_each_spilled_reg
only considers STACK_SPILL. This patch also let bpf_for_each_spilled_reg
search STACK_ITER.

Signed-off-by: Chuyi Zhou <zhouchuyi@bytedance.com>
Acked-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/r/20231018061746.111364-6-zhouchuyi@bytedance.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
include/linux/bpf_verifier.h
include/linux/btf.h
kernel/bpf/helpers.c
kernel/bpf/verifier.c