1 .. SPDX-License-Identifier: GPL-2.0
3 .. _kfuncs-header-label:
5 =============================
6 BPF Kernel Functions (kfuncs)
7 =============================
12 BPF Kernel Functions or more commonly known as kfuncs are functions in the Linux
13 kernel which are exposed for use by BPF programs. Unlike normal BPF helpers,
14 kfuncs do not have a stable interface and can change from one kernel release to
15 another. Hence, BPF programs need to be updated in response to changes in the
16 kernel. See :ref:`BPF_kfunc_lifecycle_expectations` for more information.
21 There are two ways to expose a kernel function to BPF programs, either make an
22 existing function in the kernel visible, or add a new wrapper for BPF. In both
23 cases, care must be taken that BPF program can only call such function in a
24 valid context. To enforce this, visibility of a kfunc can be per program type.
26 If you are not creating a BPF wrapper for existing kernel function, skip ahead
27 to :ref:`BPF_kfunc_nodef`.
29 2.1 Creating a wrapper kfunc
30 ----------------------------
32 When defining a wrapper kfunc, the wrapper function should have extern linkage.
33 This prevents the compiler from optimizing away dead code, as this wrapper kfunc
34 is not invoked anywhere in the kernel itself. It is not necessary to provide a
35 prototype in a header for the wrapper kfunc.
37 An example is given below::
39 /* Disables missing prototype warnings */
41 __diag_ignore_all("-Wmissing-prototypes",
42 "Global kfuncs as their definitions will be in BTF");
44 __bpf_kfunc struct task_struct *bpf_find_get_task_by_vpid(pid_t nr)
46 return find_get_task_by_vpid(nr);
51 A wrapper kfunc is often needed when we need to annotate parameters of the
52 kfunc. Otherwise one may directly make the kfunc visible to the BPF program by
53 registering it with the BPF subsystem. See :ref:`BPF_kfunc_nodef`.
55 2.2 Annotating kfunc parameters
56 -------------------------------
58 Similar to BPF helpers, there is sometime need for additional context required
59 by the verifier to make the usage of kernel functions safer and more useful.
60 Hence, we can annotate a parameter by suffixing the name of the argument of the
61 kfunc with a __tag, where tag may be one of the supported annotations.
66 This annotation is used to indicate a memory and size pair in the argument list.
67 An example is given below::
69 __bpf_kfunc void bpf_memzero(void *mem, int mem__sz)
74 Here, the verifier will treat first argument as a PTR_TO_MEM, and second
75 argument as its size. By default, without __sz annotation, the size of the type
76 of the pointer is used. Without __sz annotation, a kfunc cannot accept a void
82 This annotation is only understood for scalar arguments, where it indicates that
83 the verifier must check the scalar argument to be a known constant, which does
84 not indicate a size parameter, and the value of the constant is relevant to the
85 safety of the program.
87 An example is given below::
89 __bpf_kfunc void *bpf_obj_new(u32 local_type_id__k, ...)
94 Here, bpf_obj_new uses local_type_id argument to find out the size of that type
95 ID in program's BTF and return a sized pointer to it. Each type ID will have a
96 distinct size, hence it is crucial to treat each such call as distinct when
97 values don't match during verifier state pruning checks.
99 Hence, whenever a constant scalar argument is accepted by a kfunc which is not a
100 size parameter, and the value of the constant matters for program safety, __k
101 suffix should be used.
103 2.2.2 __uninit Annotation
104 -------------------------
106 This annotation is used to indicate that the argument will be treated as
109 An example is given below::
111 __bpf_kfunc int bpf_dynptr_from_skb(..., struct bpf_dynptr_kern *ptr__uninit)
116 Here, the dynptr will be treated as an uninitialized dynptr. Without this
117 annotation, the verifier will reject the program if the dynptr passed in is
122 2.3 Using an existing kernel function
123 -------------------------------------
125 When an existing function in the kernel is fit for consumption by BPF programs,
126 it can be directly registered with the BPF subsystem. However, care must still
127 be taken to review the context in which it will be invoked by the BPF program
128 and whether it is safe to do so.
130 2.4 Annotating kfuncs
131 ---------------------
133 In addition to kfuncs' arguments, verifier may need more information about the
134 type of kfunc(s) being registered with the BPF subsystem. To do so, we define
135 flags on a set of kfuncs as follows::
137 BTF_SET8_START(bpf_task_set)
138 BTF_ID_FLAGS(func, bpf_get_task_pid, KF_ACQUIRE | KF_RET_NULL)
139 BTF_ID_FLAGS(func, bpf_put_pid, KF_RELEASE)
140 BTF_SET8_END(bpf_task_set)
142 This set encodes the BTF ID of each kfunc listed above, and encodes the flags
143 along with it. Ofcourse, it is also allowed to specify no flags.
145 kfunc definitions should also always be annotated with the ``__bpf_kfunc``
146 macro. This prevents issues such as the compiler inlining the kfunc if it's a
147 static kernel function, or the function being elided in an LTO build as it's
148 not used in the rest of the kernel. Developers should not manually add
149 annotations to their kfunc to prevent these issues. If an annotation is
150 required to prevent such an issue with your kfunc, it is a bug and should be
151 added to the definition of the macro so that other kfuncs are similarly
152 protected. An example is given below::
154 __bpf_kfunc struct task_struct *bpf_get_task_pid(s32 pid)
159 2.4.1 KF_ACQUIRE flag
160 ---------------------
162 The KF_ACQUIRE flag is used to indicate that the kfunc returns a pointer to a
163 refcounted object. The verifier will then ensure that the pointer to the object
164 is eventually released using a release kfunc, or transferred to a map using a
165 referenced kptr (by invoking bpf_kptr_xchg). If not, the verifier fails the
166 loading of the BPF program until no lingering references remain in all possible
167 explored states of the program.
169 2.4.2 KF_RET_NULL flag
170 ----------------------
172 The KF_RET_NULL flag is used to indicate that the pointer returned by the kfunc
173 may be NULL. Hence, it forces the user to do a NULL check on the pointer
174 returned from the kfunc before making use of it (dereferencing or passing to
175 another helper). This flag is often used in pairing with KF_ACQUIRE flag, but
176 both are orthogonal to each other.
178 2.4.3 KF_RELEASE flag
179 ---------------------
181 The KF_RELEASE flag is used to indicate that the kfunc releases the pointer
182 passed in to it. There can be only one referenced pointer that can be passed
183 in. All copies of the pointer being released are invalidated as a result of
184 invoking kfunc with this flag. KF_RELEASE kfuncs automatically receive the
185 protection afforded by the KF_TRUSTED_ARGS flag described below.
187 2.4.4 KF_TRUSTED_ARGS flag
188 --------------------------
190 The KF_TRUSTED_ARGS flag is used for kfuncs taking pointer arguments. It
191 indicates that the all pointer arguments are valid, and that all pointers to
192 BTF objects have been passed in their unmodified form (that is, at a zero
193 offset, and without having been obtained from walking another pointer, with one
194 exception described below).
196 There are two types of pointers to kernel objects which are considered "valid":
198 1. Pointers which are passed as tracepoint or struct_ops callback arguments.
199 2. Pointers which were returned from a KF_ACQUIRE kfunc.
201 Pointers to non-BTF objects (e.g. scalar pointers) may also be passed to
202 KF_TRUSTED_ARGS kfuncs, and may have a non-zero offset.
204 The definition of "valid" pointers is subject to change at any time, and has
205 absolutely no ABI stability guarantees.
207 As mentioned above, a nested pointer obtained from walking a trusted pointer is
208 no longer trusted, with one exception. If a struct type has a field that is
209 guaranteed to be valid as long as its parent pointer is trusted, the
210 ``BTF_TYPE_SAFE_NESTED`` macro can be used to express that to the verifier as
215 BTF_TYPE_SAFE_NESTED(struct task_struct) {
216 const cpumask_t *cpus_ptr;
219 In other words, you must:
221 1. Wrap the trusted pointer type in the ``BTF_TYPE_SAFE_NESTED`` macro.
223 2. Specify the type and name of the trusted nested field. This field must match
224 the field in the original type definition exactly.
226 2.4.5 KF_SLEEPABLE flag
227 -----------------------
229 The KF_SLEEPABLE flag is used for kfuncs that may sleep. Such kfuncs can only
230 be called by sleepable BPF programs (BPF_F_SLEEPABLE).
232 2.4.6 KF_DESTRUCTIVE flag
233 --------------------------
235 The KF_DESTRUCTIVE flag is used to indicate functions calling which is
236 destructive to the system. For example such a call can result in system
237 rebooting or panicking. Due to this additional restrictions apply to these
238 calls. At the moment they only require CAP_SYS_BOOT capability, but more can be
244 The KF_RCU flag is a weaker version of KF_TRUSTED_ARGS. The kfuncs marked with
245 KF_RCU expect either PTR_TRUSTED or MEM_RCU arguments. The verifier guarantees
246 that the objects are valid and there is no use-after-free. The pointers are not
247 NULL, but the object's refcount could have reached zero. The kfuncs need to
248 consider doing refcnt != 0 check, especially when returning a KF_ACQUIRE
249 pointer. Note as well that a KF_ACQUIRE kfunc that is KF_RCU should very likely
252 .. _KF_deprecated_flag:
254 2.4.8 KF_DEPRECATED flag
255 ------------------------
257 The KF_DEPRECATED flag is used for kfuncs which are scheduled to be
258 changed or removed in a subsequent kernel release. A kfunc that is
259 marked with KF_DEPRECATED should also have any relevant information
260 captured in its kernel doc. Such information typically includes the
261 kfunc's expected remaining lifespan, a recommendation for new
262 functionality that can replace it if any is available, and possibly a
263 rationale for why it is being removed.
265 Note that while on some occasions, a KF_DEPRECATED kfunc may continue to be
266 supported and have its KF_DEPRECATED flag removed, it is likely to be far more
267 difficult to remove a KF_DEPRECATED flag after it's been added than it is to
268 prevent it from being added in the first place. As described in
269 :ref:`BPF_kfunc_lifecycle_expectations`, users that rely on specific kfuncs are
270 encouraged to make their use-cases known as early as possible, and participate
271 in upstream discussions regarding whether to keep, change, deprecate, or remove
272 those kfuncs if and when such discussions occur.
274 2.5 Registering the kfuncs
275 --------------------------
277 Once the kfunc is prepared for use, the final step to making it visible is
278 registering it with the BPF subsystem. Registration is done per BPF program
279 type. An example is shown below::
281 BTF_SET8_START(bpf_task_set)
282 BTF_ID_FLAGS(func, bpf_get_task_pid, KF_ACQUIRE | KF_RET_NULL)
283 BTF_ID_FLAGS(func, bpf_put_pid, KF_RELEASE)
284 BTF_SET8_END(bpf_task_set)
286 static const struct btf_kfunc_id_set bpf_task_kfunc_set = {
287 .owner = THIS_MODULE,
288 .set = &bpf_task_set,
291 static int init_subsystem(void)
293 return register_btf_kfunc_id_set(BPF_PROG_TYPE_TRACING, &bpf_task_kfunc_set);
295 late_initcall(init_subsystem);
297 2.6 Specifying no-cast aliases with ___init
298 --------------------------------------------
300 The verifier will always enforce that the BTF type of a pointer passed to a
301 kfunc by a BPF program, matches the type of pointer specified in the kfunc
302 definition. The verifier, does, however, allow types that are equivalent
303 according to the C standard to be passed to the same kfunc arg, even if their
306 For example, for the following type definition:
315 The verifier would allow a ``struct bpf_cpumask *`` to be passed to a kfunc
316 taking a ``cpumask_t *`` (which is a typedef of ``struct cpumask *``). For
317 instance, both ``struct cpumask *`` and ``struct bpf_cpmuask *`` can be passed
318 to bpf_cpumask_test_cpu().
320 In some cases, this type-aliasing behavior is not desired. ``struct
321 nf_conn___init`` is one such example:
325 struct nf_conn___init {
329 The C standard would consider these types to be equivalent, but it would not
330 always be safe to pass either type to a trusted kfunc. ``struct
331 nf_conn___init`` represents an allocated ``struct nf_conn`` object that has
332 *not yet been initialized*, so it would therefore be unsafe to pass a ``struct
333 nf_conn___init *`` to a kfunc that's expecting a fully initialized ``struct
334 nf_conn *`` (e.g. ``bpf_ct_change_timeout()``).
336 In order to accommodate such requirements, the verifier will enforce strict
337 PTR_TO_BTF_ID type matching if two types have the exact same name, with one
338 being suffixed with ``___init``.
340 .. _BPF_kfunc_lifecycle_expectations:
342 3. kfunc lifecycle expectations
343 ===============================
345 kfuncs provide a kernel <-> kernel API, and thus are not bound by any of the
346 strict stability restrictions associated with kernel <-> user UAPIs. This means
347 they can be thought of as similar to EXPORT_SYMBOL_GPL, and can therefore be
348 modified or removed by a maintainer of the subsystem they're defined in when
349 it's deemed necessary.
351 Like any other change to the kernel, maintainers will not change or remove a
352 kfunc without having a reasonable justification. Whether or not they'll choose
353 to change a kfunc will ultimately depend on a variety of factors, such as how
354 widely used the kfunc is, how long the kfunc has been in the kernel, whether an
355 alternative kfunc exists, what the norm is in terms of stability for the
356 subsystem in question, and of course what the technical cost is of continuing
357 to support the kfunc.
359 There are several implications of this:
361 a) kfuncs that are widely used or have been in the kernel for a long time will
362 be more difficult to justify being changed or removed by a maintainer. In
363 other words, kfuncs that are known to have a lot of users and provide
364 significant value provide stronger incentives for maintainers to invest the
365 time and complexity in supporting them. It is therefore important for
366 developers that are using kfuncs in their BPF programs to communicate and
367 explain how and why those kfuncs are being used, and to participate in
368 discussions regarding those kfuncs when they occur upstream.
370 b) Unlike regular kernel symbols marked with EXPORT_SYMBOL_GPL, BPF programs
371 that call kfuncs are generally not part of the kernel tree. This means that
372 refactoring cannot typically change callers in-place when a kfunc changes,
373 as is done for e.g. an upstreamed driver being updated in place when a
374 kernel symbol is changed.
376 Unlike with regular kernel symbols, this is expected behavior for BPF
377 symbols, and out-of-tree BPF programs that use kfuncs should be considered
378 relevant to discussions and decisions around modifying and removing those
379 kfuncs. The BPF community will take an active role in participating in
380 upstream discussions when necessary to ensure that the perspectives of such
381 users are taken into account.
383 c) A kfunc will never have any hard stability guarantees. BPF APIs cannot and
384 will not ever hard-block a change in the kernel purely for stability
385 reasons. That being said, kfuncs are features that are meant to solve
386 problems and provide value to users. The decision of whether to change or
387 remove a kfunc is a multivariate technical decision that is made on a
388 case-by-case basis, and which is informed by data points such as those
389 mentioned above. It is expected that a kfunc being removed or changed with
390 no warning will not be a common occurrence or take place without sound
391 justification, but it is a possibility that must be accepted if one is to
394 3.1 kfunc deprecation
395 ---------------------
397 As described above, while sometimes a maintainer may find that a kfunc must be
398 changed or removed immediately to accommodate some changes in their subsystem,
399 usually kfuncs will be able to accommodate a longer and more measured
400 deprecation process. For example, if a new kfunc comes along which provides
401 superior functionality to an existing kfunc, the existing kfunc may be
402 deprecated for some period of time to allow users to migrate their BPF programs
403 to use the new one. Or, if a kfunc has no known users, a decision may be made
404 to remove the kfunc (without providing an alternative API) after some
405 deprecation period so as to provide users with a window to notify the kfunc
406 maintainer if it turns out that the kfunc is actually being used.
408 It's expected that the common case will be that kfuncs will go through a
409 deprecation period rather than being changed or removed without warning. As
410 described in :ref:`KF_deprecated_flag`, the kfunc framework provides the
411 KF_DEPRECATED flag to kfunc developers to signal to users that a kfunc has been
412 deprecated. Once a kfunc has been marked with KF_DEPRECATED, the following
413 procedure is followed for removal:
415 1. Any relevant information for deprecated kfuncs is documented in the kfunc's
416 kernel docs. This documentation will typically include the kfunc's expected
417 remaining lifespan, a recommendation for new functionality that can replace
418 the usage of the deprecated function (or an explanation as to why no such
419 replacement exists), etc.
421 2. The deprecated kfunc is kept in the kernel for some period of time after it
422 was first marked as deprecated. This time period will be chosen on a
423 case-by-case basis, and will typically depend on how widespread the use of
424 the kfunc is, how long it has been in the kernel, and how hard it is to move
425 to alternatives. This deprecation time period is "best effort", and as
426 described :ref:`above<BPF_kfunc_lifecycle_expectations>`, circumstances may
427 sometimes dictate that the kfunc be removed before the full intended
428 deprecation period has elapsed.
430 3. After the deprecation period the kfunc will be removed. At this point, BPF
431 programs calling the kfunc will be rejected by the verifier.
436 The BPF subsystem provides a number of "core" kfuncs that are potentially
437 applicable to a wide variety of different possible use cases and programs.
438 Those kfuncs are documented here.
440 4.1 struct task_struct * kfuncs
441 -------------------------------
443 There are a number of kfuncs that allow ``struct task_struct *`` objects to be
446 .. kernel-doc:: kernel/bpf/helpers.c
447 :identifiers: bpf_task_acquire bpf_task_release
449 These kfuncs are useful when you want to acquire or release a reference to a
450 ``struct task_struct *`` that was passed as e.g. a tracepoint arg, or a
451 struct_ops callback arg. For example:
456 * A trivial example tracepoint program that shows how to
457 * acquire and release a struct task_struct * pointer.
459 SEC("tp_btf/task_newtask")
460 int BPF_PROG(task_acquire_release_example, struct task_struct *task, u64 clone_flags)
462 struct task_struct *acquired;
464 acquired = bpf_task_acquire(task);
467 * In a typical program you'd do something like store
468 * the task in a map, and the map will automatically
469 * release it later. Here, we release it manually.
471 bpf_task_release(acquired);
476 References acquired on ``struct task_struct *`` objects are RCU protected.
477 Therefore, when in an RCU read region, you can obtain a pointer to a task
478 embedded in a map value without having to acquire a reference:
482 #define private(name) SEC(".data." #name) __hidden __attribute__((aligned(8)))
483 private(TASK) static struct task_struct *global;
486 * A trivial example showing how to access a task stored
487 * in a map using RCU.
489 SEC("tp_btf/task_newtask")
490 int BPF_PROG(task_rcu_read_example, struct task_struct *task, u64 clone_flags)
492 struct task_struct *local_copy;
498 * We could also pass local_copy to kfuncs or helper functions here,
499 * as we're guaranteed that local_copy will be valid until we exit
500 * the RCU read region below.
502 bpf_printk("Global task %s is valid", local_copy->comm);
504 bpf_printk("No global task found");
505 bpf_rcu_read_unlock();
507 /* At this point we can no longer reference local_copy. */
514 A BPF program can also look up a task from a pid. This can be useful if the
515 caller doesn't have a trusted pointer to a ``struct task_struct *`` object that
516 it can acquire a reference on with bpf_task_acquire().
518 .. kernel-doc:: kernel/bpf/helpers.c
519 :identifiers: bpf_task_from_pid
521 Here is an example of it being used:
525 SEC("tp_btf/task_newtask")
526 int BPF_PROG(task_get_pid_example, struct task_struct *task, u64 clone_flags)
528 struct task_struct *lookup;
530 lookup = bpf_task_from_pid(task->pid);
532 /* A task should always be found, as %task is a tracepoint arg. */
535 if (lookup->pid != task->pid) {
536 /* bpf_task_from_pid() looks up the task via its
537 * globally-unique pid from the init_pid_ns. Thus,
538 * the pid of the lookup task should always be the
539 * same as the input task.
541 bpf_task_release(lookup);
545 /* bpf_task_from_pid() returns an acquired reference,
546 * so it must be dropped before returning from the
547 * tracepoint handler.
549 bpf_task_release(lookup);
553 4.2 struct cgroup * kfuncs
554 --------------------------
556 ``struct cgroup *`` objects also have acquire and release functions:
558 .. kernel-doc:: kernel/bpf/helpers.c
559 :identifiers: bpf_cgroup_acquire bpf_cgroup_release
561 These kfuncs are used in exactly the same manner as bpf_task_acquire() and
562 bpf_task_release() respectively, so we won't provide examples for them.
566 Other kfuncs available for interacting with ``struct cgroup *`` objects are
567 bpf_cgroup_ancestor() and bpf_cgroup_from_id(), allowing callers to access
568 the ancestor of a cgroup and find a cgroup by its ID, respectively. Both
569 return a cgroup kptr.
571 .. kernel-doc:: kernel/bpf/helpers.c
572 :identifiers: bpf_cgroup_ancestor
574 .. kernel-doc:: kernel/bpf/helpers.c
575 :identifiers: bpf_cgroup_from_id
577 Eventually, BPF should be updated to allow this to happen with a normal memory
578 load in the program itself. This is currently not possible without more work in
579 the verifier. bpf_cgroup_ancestor() can be used as follows:
584 * Simple tracepoint example that illustrates how a cgroup's
585 * ancestor can be accessed using bpf_cgroup_ancestor().
587 SEC("tp_btf/cgroup_mkdir")
588 int BPF_PROG(cgrp_ancestor_example, struct cgroup *cgrp, const char *path)
590 struct cgroup *parent;
592 /* The parent cgroup resides at the level before the current cgroup's level. */
593 parent = bpf_cgroup_ancestor(cgrp, cgrp->level - 1);
597 bpf_printk("Parent id is %d", parent->self.id);
599 /* Return the parent cgroup that was acquired above. */
600 bpf_cgroup_release(parent);
604 4.3 struct cpumask * kfuncs
605 ---------------------------
607 BPF provides a set of kfuncs that can be used to query, allocate, mutate, and
608 destroy struct cpumask * objects. Please refer to :ref:`cpumasks-header-label`