selftests/bpf: fix memory leak of lsm_cgroup
[linux-block.git] / Documentation / bpf / bpf_devel_QA.rst
CommitLineData
54222838
JDB
1=================================
2HOWTO interact with BPF subsystem
3=================================
4
34f15bf3
DB
5This document provides information for the BPF subsystem about various
6workflows related to reporting bugs, submitting patches, and queueing
7patches for stable kernels.
8
9For general information about submitting patches, please refer to
54222838 10`Documentation/process/`_. This document only describes additional specifics
34f15bf3
DB
11related to BPF.
12
54222838
JDB
13.. contents::
14 :local:
15 :depth: 2
34f15bf3 16
54222838
JDB
17Reporting bugs
18==============
34f15bf3 19
54222838
JDB
20Q: How do I report bugs for BPF kernel code?
21--------------------------------------------
34f15bf3 22A: Since all BPF kernel development as well as bpftool and iproute2 BPF
e42da4c6 23loader development happens through the bpf kernel mailing list,
54222838
JDB
24please report any found issues around BPF to the following mailing
25list:
34f15bf3 26
e42da4c6 27 bpf@vger.kernel.org
34f15bf3 28
54222838 29This may also include issues related to XDP, BPF tracing, etc.
34f15bf3 30
54222838 31Given netdev has a high volume of traffic, please also add the BPF
64ef3ddf 32maintainers to Cc (from kernel ``MAINTAINERS`` file):
34f15bf3 33
54222838
JDB
34* Alexei Starovoitov <ast@kernel.org>
35* Daniel Borkmann <daniel@iogearbox.net>
34f15bf3 36
54222838
JDB
37In case a buggy commit has already been identified, make sure to keep
38the actual commit authors in Cc as well for the report. They can
39typically be identified through the kernel's git tree.
34f15bf3 40
54222838
JDB
41**Please do NOT report BPF issues to bugzilla.kernel.org since it
42is a guarantee that the reported issue will be overlooked.**
34f15bf3 43
54222838
JDB
44Submitting patches
45==================
34f15bf3 46
26a9b433
DM
47Q: How do I run BPF CI on my changes before sending them out for review?
48------------------------------------------------------------------------
49A: BPF CI is GitHub based and hosted at https://github.com/kernel-patches/bpf.
50While GitHub also provides a CLI that can be used to accomplish the same
51results, here we focus on the UI based workflow.
52
53The following steps lay out how to start a CI run for your patches:
54- Create a fork of the aforementioned repository in your own account (one time
55 action)
56- Clone the fork locally, check out a new branch tracking either the bpf-next
57 or bpf branch, and apply your to-be-tested patches on top of it
58- Push the local branch to your fork and create a pull request against
59 kernel-patches/bpf's bpf-next_base or bpf_base branch, respectively
60
61Shortly after the pull request has been created, the CI workflow will run. Note
62that capacity is shared with patches submitted upstream being checked and so
63depending on utilization the run can take a while to finish.
64
65Note furthermore that both base branches (bpf-next_base and bpf_base) will be
66updated as patches are pushed to the respective upstream branches they track. As
67such, your patch set will automatically (be attempted to) be rebased as well.
68This behavior can result in a CI run being aborted and restarted with the new
69base line.
70
34f15bf3 71Q: To which mailing list do I need to submit my BPF patches?
54222838 72------------------------------------------------------------
e42da4c6 73A: Please submit your BPF patches to the bpf kernel mailing list:
34f15bf3 74
e42da4c6 75 bpf@vger.kernel.org
34f15bf3 76
54222838 77In case your patch has changes in various different subsystems (e.g.
e42da4c6 78networking, tracing, security, etc), make sure to Cc the related kernel mailing
54222838
JDB
79lists and maintainers from there as well, so they are able to review
80the changes and provide their Acked-by's to the patches.
34f15bf3
DB
81
82Q: Where can I find patches currently under discussion for BPF subsystem?
54222838 83-------------------------------------------------------------------------
34f15bf3 84A: All patches that are Cc'ed to netdev are queued for review under netdev
54222838 85patchwork project:
34f15bf3 86
ebb034b1 87 https://patchwork.kernel.org/project/netdevbpf/list/
34f15bf3 88
54222838
JDB
89Those patches which target BPF, are assigned to a 'bpf' delegate for
90further processing from BPF maintainers. The current queue with
91patches under review can be found at:
34f15bf3 92
ebb034b1 93 https://patchwork.kernel.org/project/netdevbpf/list/?delegate=121173
34f15bf3 94
54222838
JDB
95Once the patches have been reviewed by the BPF community as a whole
96and approved by the BPF maintainers, their status in patchwork will be
97changed to 'Accepted' and the submitter will be notified by mail. This
98means that the patches look good from a BPF perspective and have been
99applied to one of the two BPF kernel trees.
34f15bf3 100
54222838
JDB
101In case feedback from the community requires a respin of the patches,
102their status in patchwork will be set to 'Changes Requested', and purged
103from the current review queue. Likewise for cases where patches would
104get rejected or are not applicable to the BPF trees (but assigned to
105the 'bpf' delegate).
34f15bf3
DB
106
107Q: How do the changes make their way into Linux?
54222838 108------------------------------------------------
34f15bf3 109A: There are two BPF kernel trees (git repositories). Once patches have
54222838
JDB
110been accepted by the BPF maintainers, they will be applied to one
111of the two BPF trees:
34f15bf3 112
54222838
JDB
113 * https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/
114 * https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/
34f15bf3 115
54222838
JDB
116The bpf tree itself is for fixes only, whereas bpf-next for features,
117cleanups or other kind of improvements ("next-like" content). This is
118analogous to net and net-next trees for networking. Both bpf and
119bpf-next will only have a master branch in order to simplify against
120which branch patches should get rebased to.
34f15bf3 121
54222838
JDB
122Accumulated BPF patches in the bpf tree will regularly get pulled
123into the net kernel tree. Likewise, accumulated BPF patches accepted
124into the bpf-next tree will make their way into net-next tree. net and
125net-next are both run by David S. Miller. From there, they will go
126into the kernel mainline tree run by Linus Torvalds. To read up on the
127process of net and net-next being merged into the mainline tree, see
287f4fa9
TH
128the :ref:`netdev-FAQ`
129
34f15bf3 130
34f15bf3 131
54222838
JDB
132Occasionally, to prevent merge conflicts, we might send pull requests
133to other trees (e.g. tracing) with a small subset of the patches, but
134net and net-next are always the main trees targeted for integration.
34f15bf3 135
54222838
JDB
136The pull requests will contain a high-level summary of the accumulated
137patches and can be searched on netdev kernel mailing list through the
138following subject lines (``yyyy-mm-dd`` is the date of the pull
139request)::
34f15bf3 140
54222838
JDB
141 pull-request: bpf yyyy-mm-dd
142 pull-request: bpf-next yyyy-mm-dd
34f15bf3 143
54222838
JDB
144Q: How do I indicate which tree (bpf vs. bpf-next) my patch should be applied to?
145---------------------------------------------------------------------------------
34f15bf3 146
287f4fa9
TH
147A: The process is the very same as described in the :ref:`netdev-FAQ`,
148so please read up on it. The subject line must indicate whether the
54222838
JDB
149patch is a fix or rather "next-like" content in order to let the
150maintainers know whether it is targeted at bpf or bpf-next.
34f15bf3 151
54222838
JDB
152For fixes eventually landing in bpf -> net tree, the subject must
153look like::
34f15bf3 154
54222838 155 git format-patch --subject-prefix='PATCH bpf' start..finish
34f15bf3 156
54222838
JDB
157For features/improvements/etc that should eventually land in
158bpf-next -> net-next, the subject must look like::
34f15bf3 159
54222838 160 git format-patch --subject-prefix='PATCH bpf-next' start..finish
34f15bf3 161
54222838
JDB
162If unsure whether the patch or patch series should go into bpf
163or net directly, or bpf-next or net-next directly, it is not a
164problem either if the subject line says net or net-next as target.
165It is eventually up to the maintainers to do the delegation of
166the patches.
34f15bf3 167
54222838
JDB
168If it is clear that patches should go into bpf or bpf-next tree,
169please make sure to rebase the patches against those trees in
170order to reduce potential conflicts.
34f15bf3 171
54222838
JDB
172In case the patch or patch series has to be reworked and sent out
173again in a second or later revision, it is also required to add a
174version number (``v2``, ``v3``, ...) into the subject prefix::
34f15bf3 175
4d0d1673 176 git format-patch --subject-prefix='PATCH bpf-next v2' start..finish
34f15bf3 177
54222838
JDB
178When changes have been requested to the patch series, always send the
179whole patch series again with the feedback incorporated (never send
180individual diffs on top of the old series).
34f15bf3
DB
181
182Q: What does it mean when a patch gets applied to bpf or bpf-next tree?
54222838 183-----------------------------------------------------------------------
34f15bf3 184A: It means that the patch looks good for mainline inclusion from
54222838 185a BPF point of view.
34f15bf3 186
54222838
JDB
187Be aware that this is not a final verdict that the patch will
188automatically get accepted into net or net-next trees eventually:
189
e42da4c6 190On the bpf kernel mailing list reviews can come in at any point
54222838
JDB
191in time. If discussions around a patch conclude that they cannot
192get included as-is, we will either apply a follow-up fix or drop
193them from the trees entirely. Therefore, we also reserve to rebase
194the trees when deemed necessary. After all, the purpose of the tree
195is to:
196
197i) accumulate and stage BPF patches for integration into trees
198 like net and net-next, and
34f15bf3 199
54222838
JDB
200ii) run extensive BPF test suite and
201 workloads on the patches before they make their way any further.
202
203Once the BPF pull request was accepted by David S. Miller, then
204the patches end up in net or net-next tree, respectively, and
205make their way from there further into mainline. Again, see the
287f4fa9 206:ref:`netdev-FAQ` for additional information e.g. on how often they are
54222838
JDB
207merged to mainline.
208
209Q: How long do I need to wait for feedback on my BPF patches?
210-------------------------------------------------------------
34f15bf3 211A: We try to keep the latency low. The usual time to feedback will
54222838
JDB
212be around 2 or 3 business days. It may vary depending on the
213complexity of changes and current patch load.
34f15bf3 214
54222838
JDB
215Q: How often do you send pull requests to major kernel trees like net or net-next?
216----------------------------------------------------------------------------------
34f15bf3
DB
217
218A: Pull requests will be sent out rather often in order to not
54222838 219accumulate too many patches in bpf or bpf-next.
34f15bf3 220
54222838
JDB
221As a rule of thumb, expect pull requests for each tree regularly
222at the end of the week. In some cases pull requests could additionally
223come also in the middle of the week depending on the current patch
224load or urgency.
34f15bf3
DB
225
226Q: Are patches applied to bpf-next when the merge window is open?
54222838 227-----------------------------------------------------------------
34f15bf3 228A: For the time when the merge window is open, bpf-next will not be
54222838 229processed. This is roughly analogous to net-next patch processing,
287f4fa9 230so feel free to read up on the :ref:`netdev-FAQ` about further details.
34f15bf3 231
54222838
JDB
232During those two weeks of merge window, we might ask you to resend
233your patch series once bpf-next is open again. Once Linus released
234a ``v*-rc1`` after the merge window, we continue processing of bpf-next.
34f15bf3 235
54222838
JDB
236For non-subscribers to kernel mailing lists, there is also a status
237page run by David S. Miller on net-next that provides guidance:
34f15bf3 238
54222838 239 http://vger.kernel.org/~davem/net-next.html
34f15bf3 240
54222838
JDB
241Q: Verifier changes and test cases
242----------------------------------
34f15bf3 243Q: I made a BPF verifier change, do I need to add test cases for
54222838 244BPF kernel selftests_?
34f15bf3
DB
245
246A: If the patch has changes to the behavior of the verifier, then yes,
54222838
JDB
247it is absolutely necessary to add test cases to the BPF kernel
248selftests_ suite. If they are not present and we think they are
249needed, then we might ask for them before accepting any changes.
250
251In particular, test_verifier.c is tracking a high number of BPF test
252cases, including a lot of corner cases that LLVM BPF back end may
253generate out of the restricted C code. Thus, adding test cases is
254absolutely crucial to make sure future changes do not accidentally
255affect prior use-cases. Thus, treat those test cases as: verifier
256behavior that is not tracked in test_verifier.c could potentially
257be subject to change.
258
259Q: samples/bpf preference vs selftests?
260---------------------------------------
64ef3ddf
TY
261Q: When should I add code to ``samples/bpf/`` and when to BPF kernel
262selftests_?
54222838
JDB
263
264A: In general, we prefer additions to BPF kernel selftests_ rather than
64ef3ddf 265``samples/bpf/``. The rationale is very simple: kernel selftests are
54222838
JDB
266regularly run by various bots to test for kernel regressions.
267
268The more test cases we add to BPF selftests, the better the coverage
269and the less likely it is that those could accidentally break. It is
270not that BPF kernel selftests cannot demo how a specific feature can
271be used.
272
64ef3ddf 273That said, ``samples/bpf/`` may be a good place for people to get started,
54222838 274so it might be advisable that simple demos of features could go into
64ef3ddf 275``samples/bpf/``, but advanced functional and corner-case testing rather
54222838
JDB
276into kernel selftests.
277
278If your sample looks like a test case, then go for BPF kernel selftests
279instead!
34f15bf3
DB
280
281Q: When should I add code to the bpftool?
54222838 282-----------------------------------------
34f15bf3 283A: The main purpose of bpftool (under tools/bpf/bpftool/) is to provide
54222838
JDB
284a central user space tool for debugging and introspection of BPF programs
285and maps that are active in the kernel. If UAPI changes related to BPF
286enable for dumping additional information of programs or maps, then
287bpftool should be extended as well to support dumping them.
34f15bf3
DB
288
289Q: When should I add code to iproute2's BPF loader?
54222838
JDB
290---------------------------------------------------
291A: For UAPI changes related to the XDP or tc layer (e.g. ``cls_bpf``),
292the convention is that those control-path related changes are added to
293iproute2's BPF loader as well from user space side. This is not only
294useful to have UAPI changes properly designed to be usable, but also
295to make those changes available to a wider user base of major
296downstream distributions.
34f15bf3
DB
297
298Q: Do you accept patches as well for iproute2's BPF loader?
54222838 299-----------------------------------------------------------
34f15bf3
DB
300A: Patches for the iproute2's BPF loader have to be sent to:
301
54222838 302 netdev@vger.kernel.org
34f15bf3 303
54222838
JDB
304While those patches are not processed by the BPF kernel maintainers,
305please keep them in Cc as well, so they can be reviewed.
34f15bf3 306
54222838
JDB
307The official git repository for iproute2 is run by Stephen Hemminger
308and can be found at:
34f15bf3 309
54222838 310 https://git.kernel.org/pub/scm/linux/kernel/git/shemminger/iproute2.git/
34f15bf3 311
54222838
JDB
312The patches need to have a subject prefix of '``[PATCH iproute2
313master]``' or '``[PATCH iproute2 net-next]``'. '``master``' or
314'``net-next``' describes the target branch where the patch should be
315applied to. Meaning, if kernel changes went into the net-next kernel
316tree, then the related iproute2 changes need to go into the iproute2
317net-next branch, otherwise they can be targeted at master branch. The
318iproute2 net-next branch will get merged into the master branch after
319the current iproute2 version from master has been released.
34f15bf3 320
54222838
JDB
321Like BPF, the patches end up in patchwork under the netdev project and
322are delegated to 'shemminger' for further processing:
34f15bf3 323
54222838 324 http://patchwork.ozlabs.org/project/netdev/list/?delegate=389
34f15bf3
DB
325
326Q: What is the minimum requirement before I submit my BPF patches?
54222838 327------------------------------------------------------------------
34f15bf3 328A: When submitting patches, always take the time and properly test your
54222838
JDB
329patches *prior* to submission. Never rush them! If maintainers find
330that your patches have not been properly tested, it is a good way to
331get them grumpy. Testing patch submissions is a hard requirement!
332
333Note, fixes that go to bpf tree *must* have a ``Fixes:`` tag included.
334The same applies to fixes that target bpf-next, where the affected
335commit is in net-next (or in some cases bpf-next). The ``Fixes:`` tag is
336crucial in order to identify follow-up commits and tremendously helps
337for people having to do backporting, so it is a must have!
338
339We also don't accept patches with an empty commit message. Take your
340time and properly write up a high quality commit message, it is
341essential!
342
343Think about it this way: other developers looking at your code a month
344from now need to understand *why* a certain change has been done that
345way, and whether there have been flaws in the analysis or assumptions
346that the original author did. Thus providing a proper rationale and
347describing the use-case for the changes is a must.
348
349Patch submissions with >1 patch must have a cover letter which includes
350a high level description of the series. This high level summary will
351then be placed into the merge commit by the BPF maintainers such that
352it is also accessible from the git log for future reference.
353
354Q: Features changing BPF JIT and/or LLVM
355----------------------------------------
34f15bf3 356Q: What do I need to consider when adding a new instruction or feature
54222838 357that would require BPF JIT and/or LLVM integration as well?
34f15bf3
DB
358
359A: We try hard to keep all BPF JITs up to date such that the same user
54222838
JDB
360experience can be guaranteed when running BPF programs on different
361architectures without having the program punt to the less efficient
362interpreter in case the in-kernel BPF JIT is enabled.
34f15bf3 363
54222838
JDB
364If you are unable to implement or test the required JIT changes for
365certain architectures, please work together with the related BPF JIT
366developers in order to get the feature implemented in a timely manner.
367Please refer to the git log (``arch/*/net/``) to locate the necessary
368people for helping out.
34f15bf3 369
54222838
JDB
370Also always make sure to add BPF test cases (e.g. test_bpf.c and
371test_verifier.c) for new instructions, so that they can receive
372broad test coverage and help run-time testing the various BPF JITs.
34f15bf3 373
54222838
JDB
374In case of new BPF instructions, once the changes have been accepted
375into the Linux kernel, please implement support into LLVM's BPF back
376end. See LLVM_ section below for further information.
34f15bf3 377
54222838
JDB
378Stable submission
379=================
34f15bf3
DB
380
381Q: I need a specific BPF commit in stable kernels. What should I do?
54222838 382--------------------------------------------------------------------
34f15bf3 383A: In case you need a specific fix in stable kernels, first check whether
54222838 384the commit has already been applied in the related ``linux-*.y`` branches:
34f15bf3 385
54222838 386 https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/
34f15bf3 387
54222838
JDB
388If not the case, then drop an email to the BPF maintainers with the
389netdev kernel mailing list in Cc and ask for the fix to be queued up:
34f15bf3 390
54222838 391 netdev@vger.kernel.org
34f15bf3 392
54222838 393The process in general is the same as on netdev itself, see also the
287f4fa9 394:ref:`netdev-FAQ`.
34f15bf3
DB
395
396Q: Do you also backport to kernels not currently maintained as stable?
54222838 397----------------------------------------------------------------------
34f15bf3 398A: No. If you need a specific BPF commit in kernels that are currently not
54222838 399maintained by the stable maintainers, then you are on your own.
34f15bf3 400
54222838 401The current stable and longterm stable kernels are all listed here:
34f15bf3 402
54222838 403 https://www.kernel.org/
34f15bf3 404
54222838
JDB
405Q: The BPF patch I am about to submit needs to go to stable as well
406-------------------------------------------------------------------
407What should I do?
34f15bf3
DB
408
409A: The same rules apply as with netdev patch submissions in general, see
287f4fa9 410the :ref:`netdev-FAQ`.
34f15bf3 411
54222838
JDB
412Never add "``Cc: stable@vger.kernel.org``" to the patch description, but
413ask the BPF maintainers to queue the patches instead. This can be done
414with a note, for example, under the ``---`` part of the patch which does
415not go into the git log. Alternatively, this can be done as a simple
416request by mail instead.
34f15bf3 417
54222838
JDB
418Q: Queue stable patches
419-----------------------
34f15bf3 420Q: Where do I find currently queued BPF patches that will be submitted
54222838 421to stable?
34f15bf3
DB
422
423A: Once patches that fix critical bugs got applied into the bpf tree, they
54222838 424are queued up for stable submission under:
34f15bf3 425
54222838 426 http://patchwork.ozlabs.org/bundle/bpf/stable/?state=*
34f15bf3 427
54222838
JDB
428They will be on hold there at minimum until the related commit made its
429way into the mainline kernel tree.
34f15bf3 430
54222838
JDB
431After having been under broader exposure, the queued patches will be
432submitted by the BPF maintainers to the stable maintainers.
34f15bf3 433
54222838
JDB
434Testing patches
435===============
34f15bf3 436
b7a27c3a
JDB
437Q: How to run BPF selftests
438---------------------------
439A: After you have booted into the newly compiled kernel, navigate to
440the BPF selftests_ suite in order to test BPF functionality (current
441working directory points to the root of the cloned git tree)::
442
443 $ cd tools/testing/selftests/bpf/
444 $ make
445
446To run the verifier tests::
447
448 $ sudo ./test_verifier
449
450The verifier tests print out all the current checks being
451performed. The summary at the end of running all tests will dump
452information of test successes and failures::
453
454 Summary: 418 PASSED, 0 FAILED
455
456In order to run through all BPF selftests, the following command is
457needed::
458
459 $ sudo make run_tests
460
461See the kernels selftest `Documentation/dev-tools/kselftest.rst`_
462document for further documentation.
463
6736aa79
AM
464To maximize the number of tests passing, the .config of the kernel
465under test should match the config file fragment in
466tools/testing/selftests/bpf as closely as possible.
467
468Finally to ensure support for latest BPF Type Format features -
469discussed in `Documentation/bpf/btf.rst`_ - pahole version 1.16
470is required for kernels built with CONFIG_DEBUG_INFO_BTF=y.
471pahole is delivered in the dwarves package or can be built
472from source at
473
474https://github.com/acmel/dwarves
475
350a62ca
TY
476pahole starts to use libbpf definitions and APIs since v1.13 after the
477commit 21507cd3e97b ("pahole: add libbpf as submodule under lib/bpf").
478It works well with the git repository because the libbpf submodule will
479use "git submodule update --init --recursive" to update.
480
481Unfortunately, the default github release source code does not contain
482libbpf submodule source code and this will cause build issues, the tarball
483from https://git.kernel.org/pub/scm/devel/pahole/pahole.git/ is same with
484github, you can get the source tarball with corresponding libbpf submodule
485codes from
486
487https://fedorapeople.org/~acme/dwarves
488
6736aa79
AM
489Some distros have pahole version 1.16 packaged already, e.g.
490Fedora, Gentoo.
491
34f15bf3 492Q: Which BPF kernel selftests version should I run my kernel against?
54222838
JDB
493---------------------------------------------------------------------
494A: If you run a kernel ``xyz``, then always run the BPF kernel selftests
495from that kernel ``xyz`` as well. Do not expect that the BPF selftest
496from the latest mainline tree will pass all the time.
34f15bf3 497
54222838
JDB
498In particular, test_bpf.c and test_verifier.c have a large number of
499test cases and are constantly updated with new BPF test sequences, or
500existing ones are adapted to verifier changes e.g. due to verifier
501becoming smarter and being able to better track certain things.
34f15bf3 502
54222838
JDB
503LLVM
504====
34f15bf3
DB
505
506Q: Where do I find LLVM with BPF support?
54222838 507-----------------------------------------
34f15bf3
DB
508A: The BPF back end for LLVM is upstream in LLVM since version 3.7.1.
509
54222838
JDB
510All major distributions these days ship LLVM with BPF back end enabled,
511so for the majority of use-cases it is not required to compile LLVM by
512hand anymore, just install the distribution provided package.
34f15bf3 513
54222838
JDB
514LLVM's static compiler lists the supported targets through
515``llc --version``, make sure BPF targets are listed. Example::
34f15bf3
DB
516
517 $ llc --version
518 LLVM (http://llvm.org/):
4d0d1673 519 LLVM version 10.0.0
34f15bf3
DB
520 Optimized build.
521 Default target: x86_64-unknown-linux-gnu
522 Host CPU: skylake
523
524 Registered Targets:
4d0d1673
JL
525 aarch64 - AArch64 (little endian)
526 bpf - BPF (host endian)
527 bpfeb - BPF (big endian)
528 bpfel - BPF (little endian)
529 x86 - 32-bit X86: Pentium-Pro and above
530 x86-64 - 64-bit X86: EM64T and AMD64
34f15bf3 531
54222838
JDB
532For developers in order to utilize the latest features added to LLVM's
533BPF back end, it is advisable to run the latest LLVM releases. Support
534for new BPF kernel features such as additions to the BPF instruction
535set are often developed together.
34f15bf3 536
54222838 537All LLVM releases can be found at: http://releases.llvm.org/
34f15bf3
DB
538
539Q: Got it, so how do I build LLVM manually anyway?
54222838 540--------------------------------------------------
628add78
TY
541A: We recommend that developers who want the fastest incremental builds
542use the Ninja build system, you can find it in your system's package
543manager, usually the package is ninja or ninja-build.
544
545You need ninja, cmake and gcc-c++ as build requisites for LLVM. Once you
546have that set up, proceed with building the latest LLVM and clang version
54222838 547from the git repositories::
34f15bf3 548
e42da4c6 549 $ git clone https://github.com/llvm/llvm-project.git
628add78 550 $ mkdir -p llvm-project/llvm/build
e42da4c6
YS
551 $ cd llvm-project/llvm/build
552 $ cmake .. -G "Ninja" -DLLVM_TARGETS_TO_BUILD="BPF;X86" \
553 -DLLVM_ENABLE_PROJECTS="clang" \
34f15bf3
DB
554 -DCMAKE_BUILD_TYPE=Release \
555 -DLLVM_BUILD_RUNTIME=OFF
e42da4c6 556 $ ninja
34f15bf3 557
54222838
JDB
558The built binaries can then be found in the build/bin/ directory, where
559you can point the PATH variable to.
34f15bf3 560
4d0d1673
JL
561Set ``-DLLVM_TARGETS_TO_BUILD`` equal to the target you wish to build, you
562will find a full list of targets within the llvm-project/llvm/lib/Target
563directory.
564
54222838
JDB
565Q: Reporting LLVM BPF issues
566----------------------------
34f15bf3 567Q: Should I notify BPF kernel maintainers about issues in LLVM's BPF code
54222838
JDB
568generation back end or about LLVM generated code that the verifier
569refuses to accept?
570
571A: Yes, please do!
34f15bf3 572
54222838
JDB
573LLVM's BPF back end is a key piece of the whole BPF
574infrastructure and it ties deeply into verification of programs from the
575kernel side. Therefore, any issues on either side need to be investigated
576and fixed whenever necessary.
34f15bf3 577
54222838
JDB
578Therefore, please make sure to bring them up at netdev kernel mailing
579list and Cc BPF maintainers for LLVM and kernel bits:
34f15bf3 580
54222838
JDB
581* Yonghong Song <yhs@fb.com>
582* Alexei Starovoitov <ast@kernel.org>
583* Daniel Borkmann <daniel@iogearbox.net>
34f15bf3 584
54222838 585LLVM also has an issue tracker where BPF related bugs can be found:
34f15bf3 586
54222838 587 https://bugs.llvm.org/buglist.cgi?quicksearch=bpf
34f15bf3 588
54222838
JDB
589However, it is better to reach out through mailing lists with having
590maintainers in Cc.
34f15bf3 591
54222838
JDB
592Q: New BPF instruction for kernel and LLVM
593------------------------------------------
34f15bf3 594Q: I have added a new BPF instruction to the kernel, how can I integrate
54222838 595it into LLVM?
34f15bf3 596
54222838
JDB
597A: LLVM has a ``-mcpu`` selector for the BPF back end in order to allow
598the selection of BPF instruction set extensions. By default the
599``generic`` processor target is used, which is the base instruction set
600(v1) of BPF.
34f15bf3 601
54222838
JDB
602LLVM has an option to select ``-mcpu=probe`` where it will probe the host
603kernel for supported BPF instruction set extensions and selects the
604optimal set automatically.
34f15bf3 605
54222838 606For cross-compilation, a specific version can be select manually as well ::
34f15bf3
DB
607
608 $ llc -march bpf -mcpu=help
609 Available CPUs for this target:
610
611 generic - Select the generic processor.
612 probe - Select the probe processor.
613 v1 - Select the v1 processor.
614 v2 - Select the v2 processor.
615 [...]
616
54222838
JDB
617Newly added BPF instructions to the Linux kernel need to follow the same
618scheme, bump the instruction set version and implement probing for the
619extensions such that ``-mcpu=probe`` users can benefit from the
620optimization transparently when upgrading their kernels.
34f15bf3 621
54222838
JDB
622If you are unable to implement support for the newly added BPF instruction
623please reach out to BPF developers for help.
34f15bf3 624
54222838
JDB
625By the way, the BPF kernel selftests run with ``-mcpu=probe`` for better
626test coverage.
34f15bf3 627
54222838
JDB
628Q: clang flag for target bpf?
629-----------------------------
630Q: In some cases clang flag ``-target bpf`` is used but in other cases the
631default clang target, which matches the underlying architecture, is used.
632What is the difference and when I should use which?
6215ea6b
YS
633
634A: Although LLVM IR generation and optimization try to stay architecture
54222838
JDB
635independent, ``-target <arch>`` still has some impact on generated code:
636
637- BPF program may recursively include header file(s) with file scope
638 inline assembly codes. The default target can handle this well,
639 while ``bpf`` target may fail if bpf backend assembler does not
640 understand these assembly codes, which is true in most cases.
641
642- When compiled without ``-g``, additional elf sections, e.g.,
643 .eh_frame and .rela.eh_frame, may be present in the object file
644 with default target, but not with ``bpf`` target.
645
646- The default target may turn a C switch statement into a switch table
647 lookup and jump operation. Since the switch table is placed
648 in the global readonly section, the bpf program will fail to load.
649 The bpf target does not support switch table optimization.
650 The clang option ``-fno-jump-tables`` can be used to disable
651 switch table generation.
652
653- For clang ``-target bpf``, it is guaranteed that pointer or long /
654 unsigned long types will always have a width of 64 bit, no matter
655 whether underlying clang binary or default target (or kernel) is
656 32 bit. However, when native clang target is used, then it will
657 compile these types based on the underlying architecture's conventions,
658 meaning in case of 32 bit architecture, pointer or long / unsigned
659 long types e.g. in BPF context structure will have width of 32 bit
660 while the BPF LLVM back end still operates in 64 bit. The native
661 target is mostly needed in tracing for the case of walking ``pt_regs``
662 or other kernel structures where CPU's register width matters.
663 Otherwise, ``clang -target bpf`` is generally recommended.
664
665You should use default target when:
666
667- Your program includes a header file, e.g., ptrace.h, which eventually
668 pulls in some header files containing file scope host assembly codes.
669
670- You can add ``-fno-jump-tables`` to work around the switch table issue.
671
672Otherwise, you can use ``bpf`` target. Additionally, you *must* use bpf target
673when:
674
675- Your program uses data structures with pointer or long / unsigned long
676 types that interface with BPF helpers or context data structures. Access
677 into these structures is verified by the BPF verifier and may result
678 in verification failures if the native architecture is not aligned with
679 the BPF architecture, e.g. 64-bit. An example of this is
680 BPF_PROG_TYPE_SK_MSG require ``-target bpf``
681
682
683.. Links
684.. _Documentation/process/: https://www.kernel.org/doc/html/latest/process/
8df01363 685.. _netdev-FAQ: Documentation/process/maintainer-netdev.rst
64ef3ddf
TY
686.. _selftests:
687 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/tools/testing/selftests/bpf/
b7a27c3a
JDB
688.. _Documentation/dev-tools/kselftest.rst:
689 https://www.kernel.org/doc/html/latest/dev-tools/kselftest.html
009abf5b 690.. _Documentation/bpf/btf.rst: btf.rst
6215ea6b 691
34f15bf3 692Happy BPF hacking!