bpf: Refactor RCU enforcement in the verifier.
authorAlexei Starovoitov <ast@kernel.org>
Fri, 3 Mar 2023 04:14:46 +0000 (20:14 -0800)
committerDaniel Borkmann <daniel@iogearbox.net>
Fri, 3 Mar 2023 16:42:20 +0000 (17:42 +0100)
commit6fcd486b3a0a628c41f12b3a7329a18a2c74b351
treeac60957b666a5eac1e50dd7bb125bb7c106acb2f
parent0047d8343f6042c4feea24072ef254d47b8a33b3
bpf: Refactor RCU enforcement in the verifier.

bpf_rcu_read_lock/unlock() are only available in clang compiled kernels. Lack
of such key mechanism makes it impossible for sleepable bpf programs to use RCU
pointers.

Allow bpf_rcu_read_lock/unlock() in GCC compiled kernels (though GCC doesn't
support btf_type_tag yet) and allowlist certain field dereferences in important
data structures like tast_struct, cgroup, socket that are used by sleepable
programs either as RCU pointer or full trusted pointer (which is valid outside
of RCU CS). Use BTF_TYPE_SAFE_RCU and BTF_TYPE_SAFE_TRUSTED macros for such
tagging. They will be removed once GCC supports btf_type_tag.

With that refactor check_ptr_to_btf_access(). Make it strict in enforcing
PTR_TRUSTED and PTR_UNTRUSTED while deprecating old PTR_TO_BTF_ID without
modifier flags. There is a chance that this strict enforcement might break
existing programs (especially on GCC compiled kernels), but this cleanup has to
start sooner than later. Note PTR_TO_CTX access still yields old deprecated
PTR_TO_BTF_ID. Once it's converted to strict PTR_TRUSTED or PTR_UNTRUSTED the
kfuncs and helpers will be able to default to KF_TRUSTED_ARGS. KF_RCU will
remain as a weaker version of KF_TRUSTED_ARGS where obj refcnt could be 0.

Adjust rcu_read_lock selftest to run on gcc and clang compiled kernels.

Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: David Vernet <void@manifault.com>
Link: https://lore.kernel.org/bpf/20230303041446.3630-7-alexei.starovoitov@gmail.com
12 files changed:
include/linux/bpf.h
include/linux/bpf_verifier.h
kernel/bpf/btf.c
kernel/bpf/cpumask.c
kernel/bpf/verifier.c
tools/testing/selftests/bpf/prog_tests/cgrp_local_storage.c
tools/testing/selftests/bpf/prog_tests/rcu_read_lock.c
tools/testing/selftests/bpf/progs/cgrp_ls_sleepable.c
tools/testing/selftests/bpf/progs/cpumask_failure.c
tools/testing/selftests/bpf/progs/nested_trust_failure.c
tools/testing/selftests/bpf/progs/rcu_read_lock.c
tools/testing/selftests/bpf/verifier/calls.c