Commit | Line | Data |
---|---|---|
54222838 JDB |
1 | ================================= |
2 | HOWTO interact with BPF subsystem | |
3 | ================================= | |
4 | ||
34f15bf3 DB |
5 | This document provides information for the BPF subsystem about various |
6 | workflows related to reporting bugs, submitting patches, and queueing | |
7 | patches for stable kernels. | |
8 | ||
9 | For general information about submitting patches, please refer to | |
54222838 | 10 | `Documentation/process/`_. This document only describes additional specifics |
34f15bf3 DB |
11 | related to BPF. |
12 | ||
54222838 JDB |
13 | .. contents:: |
14 | :local: | |
15 | :depth: 2 | |
34f15bf3 | 16 | |
54222838 JDB |
17 | Reporting bugs |
18 | ============== | |
34f15bf3 | 19 | |
54222838 JDB |
20 | Q: How do I report bugs for BPF kernel code? |
21 | -------------------------------------------- | |
34f15bf3 | 22 | A: Since all BPF kernel development as well as bpftool and iproute2 BPF |
e42da4c6 | 23 | loader development happens through the bpf kernel mailing list, |
54222838 JDB |
24 | please report any found issues around BPF to the following mailing |
25 | list: | |
34f15bf3 | 26 | |
e42da4c6 | 27 | bpf@vger.kernel.org |
34f15bf3 | 28 | |
54222838 | 29 | This may also include issues related to XDP, BPF tracing, etc. |
34f15bf3 | 30 | |
54222838 | 31 | Given netdev has a high volume of traffic, please also add the BPF |
64ef3ddf | 32 | maintainers to Cc (from kernel ``MAINTAINERS`` file): |
34f15bf3 | 33 | |
54222838 JDB |
34 | * Alexei Starovoitov <ast@kernel.org> |
35 | * Daniel Borkmann <daniel@iogearbox.net> | |
34f15bf3 | 36 | |
54222838 JDB |
37 | In case a buggy commit has already been identified, make sure to keep |
38 | the actual commit authors in Cc as well for the report. They can | |
39 | typically be identified through the kernel's git tree. | |
34f15bf3 | 40 | |
54222838 JDB |
41 | **Please do NOT report BPF issues to bugzilla.kernel.org since it |
42 | is a guarantee that the reported issue will be overlooked.** | |
34f15bf3 | 43 | |
54222838 JDB |
44 | Submitting patches |
45 | ================== | |
34f15bf3 | 46 | |
26a9b433 DM |
47 | Q: How do I run BPF CI on my changes before sending them out for review? |
48 | ------------------------------------------------------------------------ | |
49 | A: BPF CI is GitHub based and hosted at https://github.com/kernel-patches/bpf. | |
50 | While GitHub also provides a CLI that can be used to accomplish the same | |
51 | results, here we focus on the UI based workflow. | |
52 | ||
53 | The following steps lay out how to start a CI run for your patches: | |
383f1a8d | 54 | |
26a9b433 DM |
55 | - Create a fork of the aforementioned repository in your own account (one time |
56 | action) | |
383f1a8d | 57 | |
26a9b433 DM |
58 | - Clone the fork locally, check out a new branch tracking either the bpf-next |
59 | or bpf branch, and apply your to-be-tested patches on top of it | |
383f1a8d | 60 | |
26a9b433 DM |
61 | - Push the local branch to your fork and create a pull request against |
62 | kernel-patches/bpf's bpf-next_base or bpf_base branch, respectively | |
63 | ||
64 | Shortly after the pull request has been created, the CI workflow will run. Note | |
65 | that capacity is shared with patches submitted upstream being checked and so | |
66 | depending on utilization the run can take a while to finish. | |
67 | ||
68 | Note furthermore that both base branches (bpf-next_base and bpf_base) will be | |
69 | updated as patches are pushed to the respective upstream branches they track. As | |
70 | such, your patch set will automatically (be attempted to) be rebased as well. | |
71 | This behavior can result in a CI run being aborted and restarted with the new | |
72 | base line. | |
73 | ||
34f15bf3 | 74 | Q: To which mailing list do I need to submit my BPF patches? |
54222838 | 75 | ------------------------------------------------------------ |
e42da4c6 | 76 | A: Please submit your BPF patches to the bpf kernel mailing list: |
34f15bf3 | 77 | |
e42da4c6 | 78 | bpf@vger.kernel.org |
34f15bf3 | 79 | |
54222838 | 80 | In case your patch has changes in various different subsystems (e.g. |
e42da4c6 | 81 | networking, tracing, security, etc), make sure to Cc the related kernel mailing |
54222838 JDB |
82 | lists and maintainers from there as well, so they are able to review |
83 | the changes and provide their Acked-by's to the patches. | |
34f15bf3 DB |
84 | |
85 | Q: Where can I find patches currently under discussion for BPF subsystem? | |
54222838 | 86 | ------------------------------------------------------------------------- |
34f15bf3 | 87 | A: All patches that are Cc'ed to netdev are queued for review under netdev |
54222838 | 88 | patchwork project: |
34f15bf3 | 89 | |
ebb034b1 | 90 | https://patchwork.kernel.org/project/netdevbpf/list/ |
34f15bf3 | 91 | |
54222838 JDB |
92 | Those patches which target BPF, are assigned to a 'bpf' delegate for |
93 | further processing from BPF maintainers. The current queue with | |
94 | patches under review can be found at: | |
34f15bf3 | 95 | |
ebb034b1 | 96 | https://patchwork.kernel.org/project/netdevbpf/list/?delegate=121173 |
34f15bf3 | 97 | |
54222838 JDB |
98 | Once the patches have been reviewed by the BPF community as a whole |
99 | and approved by the BPF maintainers, their status in patchwork will be | |
100 | changed to 'Accepted' and the submitter will be notified by mail. This | |
101 | means that the patches look good from a BPF perspective and have been | |
102 | applied to one of the two BPF kernel trees. | |
34f15bf3 | 103 | |
54222838 JDB |
104 | In case feedback from the community requires a respin of the patches, |
105 | their status in patchwork will be set to 'Changes Requested', and purged | |
106 | from the current review queue. Likewise for cases where patches would | |
107 | get rejected or are not applicable to the BPF trees (but assigned to | |
108 | the 'bpf' delegate). | |
34f15bf3 DB |
109 | |
110 | Q: How do the changes make their way into Linux? | |
54222838 | 111 | ------------------------------------------------ |
34f15bf3 | 112 | A: There are two BPF kernel trees (git repositories). Once patches have |
54222838 JDB |
113 | been accepted by the BPF maintainers, they will be applied to one |
114 | of the two BPF trees: | |
34f15bf3 | 115 | |
54222838 JDB |
116 | * https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/ |
117 | * https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/ | |
34f15bf3 | 118 | |
54222838 JDB |
119 | The bpf tree itself is for fixes only, whereas bpf-next for features, |
120 | cleanups or other kind of improvements ("next-like" content). This is | |
121 | analogous to net and net-next trees for networking. Both bpf and | |
122 | bpf-next will only have a master branch in order to simplify against | |
123 | which branch patches should get rebased to. | |
34f15bf3 | 124 | |
54222838 JDB |
125 | Accumulated BPF patches in the bpf tree will regularly get pulled |
126 | into the net kernel tree. Likewise, accumulated BPF patches accepted | |
127 | into the bpf-next tree will make their way into net-next tree. net and | |
128 | net-next are both run by David S. Miller. From there, they will go | |
129 | into the kernel mainline tree run by Linus Torvalds. To read up on the | |
130 | process of net and net-next being merged into the mainline tree, see | |
287f4fa9 TH |
131 | the :ref:`netdev-FAQ` |
132 | ||
34f15bf3 | 133 | |
34f15bf3 | 134 | |
54222838 JDB |
135 | Occasionally, to prevent merge conflicts, we might send pull requests |
136 | to other trees (e.g. tracing) with a small subset of the patches, but | |
137 | net and net-next are always the main trees targeted for integration. | |
34f15bf3 | 138 | |
54222838 JDB |
139 | The pull requests will contain a high-level summary of the accumulated |
140 | patches and can be searched on netdev kernel mailing list through the | |
141 | following subject lines (``yyyy-mm-dd`` is the date of the pull | |
142 | request):: | |
34f15bf3 | 143 | |
54222838 JDB |
144 | pull-request: bpf yyyy-mm-dd |
145 | pull-request: bpf-next yyyy-mm-dd | |
34f15bf3 | 146 | |
54222838 JDB |
147 | Q: How do I indicate which tree (bpf vs. bpf-next) my patch should be applied to? |
148 | --------------------------------------------------------------------------------- | |
34f15bf3 | 149 | |
287f4fa9 TH |
150 | A: The process is the very same as described in the :ref:`netdev-FAQ`, |
151 | so please read up on it. The subject line must indicate whether the | |
54222838 JDB |
152 | patch is a fix or rather "next-like" content in order to let the |
153 | maintainers know whether it is targeted at bpf or bpf-next. | |
34f15bf3 | 154 | |
54222838 JDB |
155 | For fixes eventually landing in bpf -> net tree, the subject must |
156 | look like:: | |
34f15bf3 | 157 | |
54222838 | 158 | git format-patch --subject-prefix='PATCH bpf' start..finish |
34f15bf3 | 159 | |
54222838 JDB |
160 | For features/improvements/etc that should eventually land in |
161 | bpf-next -> net-next, the subject must look like:: | |
34f15bf3 | 162 | |
54222838 | 163 | git format-patch --subject-prefix='PATCH bpf-next' start..finish |
34f15bf3 | 164 | |
54222838 JDB |
165 | If unsure whether the patch or patch series should go into bpf |
166 | or net directly, or bpf-next or net-next directly, it is not a | |
167 | problem either if the subject line says net or net-next as target. | |
168 | It is eventually up to the maintainers to do the delegation of | |
169 | the patches. | |
34f15bf3 | 170 | |
54222838 JDB |
171 | If it is clear that patches should go into bpf or bpf-next tree, |
172 | please make sure to rebase the patches against those trees in | |
173 | order to reduce potential conflicts. | |
34f15bf3 | 174 | |
54222838 JDB |
175 | In case the patch or patch series has to be reworked and sent out |
176 | again in a second or later revision, it is also required to add a | |
177 | version number (``v2``, ``v3``, ...) into the subject prefix:: | |
34f15bf3 | 178 | |
4d0d1673 | 179 | git format-patch --subject-prefix='PATCH bpf-next v2' start..finish |
34f15bf3 | 180 | |
54222838 JDB |
181 | When changes have been requested to the patch series, always send the |
182 | whole patch series again with the feedback incorporated (never send | |
183 | individual diffs on top of the old series). | |
34f15bf3 DB |
184 | |
185 | Q: What does it mean when a patch gets applied to bpf or bpf-next tree? | |
54222838 | 186 | ----------------------------------------------------------------------- |
34f15bf3 | 187 | A: It means that the patch looks good for mainline inclusion from |
54222838 | 188 | a BPF point of view. |
34f15bf3 | 189 | |
54222838 JDB |
190 | Be aware that this is not a final verdict that the patch will |
191 | automatically get accepted into net or net-next trees eventually: | |
192 | ||
e42da4c6 | 193 | On the bpf kernel mailing list reviews can come in at any point |
54222838 JDB |
194 | in time. If discussions around a patch conclude that they cannot |
195 | get included as-is, we will either apply a follow-up fix or drop | |
196 | them from the trees entirely. Therefore, we also reserve to rebase | |
197 | the trees when deemed necessary. After all, the purpose of the tree | |
198 | is to: | |
199 | ||
200 | i) accumulate and stage BPF patches for integration into trees | |
201 | like net and net-next, and | |
34f15bf3 | 202 | |
54222838 JDB |
203 | ii) run extensive BPF test suite and |
204 | workloads on the patches before they make their way any further. | |
205 | ||
206 | Once the BPF pull request was accepted by David S. Miller, then | |
207 | the patches end up in net or net-next tree, respectively, and | |
208 | make their way from there further into mainline. Again, see the | |
287f4fa9 | 209 | :ref:`netdev-FAQ` for additional information e.g. on how often they are |
54222838 JDB |
210 | merged to mainline. |
211 | ||
212 | Q: How long do I need to wait for feedback on my BPF patches? | |
213 | ------------------------------------------------------------- | |
34f15bf3 | 214 | A: We try to keep the latency low. The usual time to feedback will |
54222838 JDB |
215 | be around 2 or 3 business days. It may vary depending on the |
216 | complexity of changes and current patch load. | |
34f15bf3 | 217 | |
54222838 JDB |
218 | Q: How often do you send pull requests to major kernel trees like net or net-next? |
219 | ---------------------------------------------------------------------------------- | |
34f15bf3 DB |
220 | |
221 | A: Pull requests will be sent out rather often in order to not | |
54222838 | 222 | accumulate too many patches in bpf or bpf-next. |
34f15bf3 | 223 | |
54222838 JDB |
224 | As a rule of thumb, expect pull requests for each tree regularly |
225 | at the end of the week. In some cases pull requests could additionally | |
226 | come also in the middle of the week depending on the current patch | |
227 | load or urgency. | |
34f15bf3 DB |
228 | |
229 | Q: Are patches applied to bpf-next when the merge window is open? | |
54222838 | 230 | ----------------------------------------------------------------- |
34f15bf3 | 231 | A: For the time when the merge window is open, bpf-next will not be |
54222838 | 232 | processed. This is roughly analogous to net-next patch processing, |
287f4fa9 | 233 | so feel free to read up on the :ref:`netdev-FAQ` about further details. |
34f15bf3 | 234 | |
54222838 JDB |
235 | During those two weeks of merge window, we might ask you to resend |
236 | your patch series once bpf-next is open again. Once Linus released | |
237 | a ``v*-rc1`` after the merge window, we continue processing of bpf-next. | |
34f15bf3 | 238 | |
54222838 JDB |
239 | For non-subscribers to kernel mailing lists, there is also a status |
240 | page run by David S. Miller on net-next that provides guidance: | |
34f15bf3 | 241 | |
54222838 | 242 | http://vger.kernel.org/~davem/net-next.html |
34f15bf3 | 243 | |
54222838 JDB |
244 | Q: Verifier changes and test cases |
245 | ---------------------------------- | |
34f15bf3 | 246 | Q: I made a BPF verifier change, do I need to add test cases for |
54222838 | 247 | BPF kernel selftests_? |
34f15bf3 DB |
248 | |
249 | A: If the patch has changes to the behavior of the verifier, then yes, | |
54222838 JDB |
250 | it is absolutely necessary to add test cases to the BPF kernel |
251 | selftests_ suite. If they are not present and we think they are | |
252 | needed, then we might ask for them before accepting any changes. | |
253 | ||
254 | In particular, test_verifier.c is tracking a high number of BPF test | |
255 | cases, including a lot of corner cases that LLVM BPF back end may | |
256 | generate out of the restricted C code. Thus, adding test cases is | |
257 | absolutely crucial to make sure future changes do not accidentally | |
258 | affect prior use-cases. Thus, treat those test cases as: verifier | |
259 | behavior that is not tracked in test_verifier.c could potentially | |
260 | be subject to change. | |
261 | ||
262 | Q: samples/bpf preference vs selftests? | |
263 | --------------------------------------- | |
64ef3ddf TY |
264 | Q: When should I add code to ``samples/bpf/`` and when to BPF kernel |
265 | selftests_? | |
54222838 JDB |
266 | |
267 | A: In general, we prefer additions to BPF kernel selftests_ rather than | |
64ef3ddf | 268 | ``samples/bpf/``. The rationale is very simple: kernel selftests are |
54222838 JDB |
269 | regularly run by various bots to test for kernel regressions. |
270 | ||
271 | The more test cases we add to BPF selftests, the better the coverage | |
272 | and the less likely it is that those could accidentally break. It is | |
273 | not that BPF kernel selftests cannot demo how a specific feature can | |
274 | be used. | |
275 | ||
64ef3ddf | 276 | That said, ``samples/bpf/`` may be a good place for people to get started, |
54222838 | 277 | so it might be advisable that simple demos of features could go into |
64ef3ddf | 278 | ``samples/bpf/``, but advanced functional and corner-case testing rather |
54222838 JDB |
279 | into kernel selftests. |
280 | ||
281 | If your sample looks like a test case, then go for BPF kernel selftests | |
282 | instead! | |
34f15bf3 DB |
283 | |
284 | Q: When should I add code to the bpftool? | |
54222838 | 285 | ----------------------------------------- |
34f15bf3 | 286 | A: The main purpose of bpftool (under tools/bpf/bpftool/) is to provide |
54222838 JDB |
287 | a central user space tool for debugging and introspection of BPF programs |
288 | and maps that are active in the kernel. If UAPI changes related to BPF | |
289 | enable for dumping additional information of programs or maps, then | |
290 | bpftool should be extended as well to support dumping them. | |
34f15bf3 DB |
291 | |
292 | Q: When should I add code to iproute2's BPF loader? | |
54222838 JDB |
293 | --------------------------------------------------- |
294 | A: For UAPI changes related to the XDP or tc layer (e.g. ``cls_bpf``), | |
295 | the convention is that those control-path related changes are added to | |
296 | iproute2's BPF loader as well from user space side. This is not only | |
297 | useful to have UAPI changes properly designed to be usable, but also | |
298 | to make those changes available to a wider user base of major | |
299 | downstream distributions. | |
34f15bf3 DB |
300 | |
301 | Q: Do you accept patches as well for iproute2's BPF loader? | |
54222838 | 302 | ----------------------------------------------------------- |
34f15bf3 DB |
303 | A: Patches for the iproute2's BPF loader have to be sent to: |
304 | ||
54222838 | 305 | netdev@vger.kernel.org |
34f15bf3 | 306 | |
54222838 JDB |
307 | While those patches are not processed by the BPF kernel maintainers, |
308 | please keep them in Cc as well, so they can be reviewed. | |
34f15bf3 | 309 | |
54222838 JDB |
310 | The official git repository for iproute2 is run by Stephen Hemminger |
311 | and can be found at: | |
34f15bf3 | 312 | |
54222838 | 313 | https://git.kernel.org/pub/scm/linux/kernel/git/shemminger/iproute2.git/ |
34f15bf3 | 314 | |
54222838 JDB |
315 | The patches need to have a subject prefix of '``[PATCH iproute2 |
316 | master]``' or '``[PATCH iproute2 net-next]``'. '``master``' or | |
317 | '``net-next``' describes the target branch where the patch should be | |
318 | applied to. Meaning, if kernel changes went into the net-next kernel | |
319 | tree, then the related iproute2 changes need to go into the iproute2 | |
320 | net-next branch, otherwise they can be targeted at master branch. The | |
321 | iproute2 net-next branch will get merged into the master branch after | |
322 | the current iproute2 version from master has been released. | |
34f15bf3 | 323 | |
54222838 JDB |
324 | Like BPF, the patches end up in patchwork under the netdev project and |
325 | are delegated to 'shemminger' for further processing: | |
34f15bf3 | 326 | |
54222838 | 327 | http://patchwork.ozlabs.org/project/netdev/list/?delegate=389 |
34f15bf3 DB |
328 | |
329 | Q: What is the minimum requirement before I submit my BPF patches? | |
54222838 | 330 | ------------------------------------------------------------------ |
34f15bf3 | 331 | A: When submitting patches, always take the time and properly test your |
54222838 JDB |
332 | patches *prior* to submission. Never rush them! If maintainers find |
333 | that your patches have not been properly tested, it is a good way to | |
334 | get them grumpy. Testing patch submissions is a hard requirement! | |
335 | ||
336 | Note, fixes that go to bpf tree *must* have a ``Fixes:`` tag included. | |
337 | The same applies to fixes that target bpf-next, where the affected | |
338 | commit is in net-next (or in some cases bpf-next). The ``Fixes:`` tag is | |
339 | crucial in order to identify follow-up commits and tremendously helps | |
340 | for people having to do backporting, so it is a must have! | |
341 | ||
342 | We also don't accept patches with an empty commit message. Take your | |
343 | time and properly write up a high quality commit message, it is | |
344 | essential! | |
345 | ||
346 | Think about it this way: other developers looking at your code a month | |
347 | from now need to understand *why* a certain change has been done that | |
348 | way, and whether there have been flaws in the analysis or assumptions | |
349 | that the original author did. Thus providing a proper rationale and | |
350 | describing the use-case for the changes is a must. | |
351 | ||
352 | Patch submissions with >1 patch must have a cover letter which includes | |
353 | a high level description of the series. This high level summary will | |
354 | then be placed into the merge commit by the BPF maintainers such that | |
355 | it is also accessible from the git log for future reference. | |
356 | ||
357 | Q: Features changing BPF JIT and/or LLVM | |
358 | ---------------------------------------- | |
34f15bf3 | 359 | Q: What do I need to consider when adding a new instruction or feature |
54222838 | 360 | that would require BPF JIT and/or LLVM integration as well? |
34f15bf3 DB |
361 | |
362 | A: We try hard to keep all BPF JITs up to date such that the same user | |
54222838 JDB |
363 | experience can be guaranteed when running BPF programs on different |
364 | architectures without having the program punt to the less efficient | |
365 | interpreter in case the in-kernel BPF JIT is enabled. | |
34f15bf3 | 366 | |
54222838 JDB |
367 | If you are unable to implement or test the required JIT changes for |
368 | certain architectures, please work together with the related BPF JIT | |
369 | developers in order to get the feature implemented in a timely manner. | |
370 | Please refer to the git log (``arch/*/net/``) to locate the necessary | |
371 | people for helping out. | |
34f15bf3 | 372 | |
54222838 JDB |
373 | Also always make sure to add BPF test cases (e.g. test_bpf.c and |
374 | test_verifier.c) for new instructions, so that they can receive | |
375 | broad test coverage and help run-time testing the various BPF JITs. | |
34f15bf3 | 376 | |
54222838 JDB |
377 | In case of new BPF instructions, once the changes have been accepted |
378 | into the Linux kernel, please implement support into LLVM's BPF back | |
379 | end. See LLVM_ section below for further information. | |
34f15bf3 | 380 | |
54222838 JDB |
381 | Stable submission |
382 | ================= | |
34f15bf3 DB |
383 | |
384 | Q: I need a specific BPF commit in stable kernels. What should I do? | |
54222838 | 385 | -------------------------------------------------------------------- |
34f15bf3 | 386 | A: In case you need a specific fix in stable kernels, first check whether |
54222838 | 387 | the commit has already been applied in the related ``linux-*.y`` branches: |
34f15bf3 | 388 | |
54222838 | 389 | https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/ |
34f15bf3 | 390 | |
54222838 JDB |
391 | If not the case, then drop an email to the BPF maintainers with the |
392 | netdev kernel mailing list in Cc and ask for the fix to be queued up: | |
34f15bf3 | 393 | |
54222838 | 394 | netdev@vger.kernel.org |
34f15bf3 | 395 | |
54222838 | 396 | The process in general is the same as on netdev itself, see also the |
287f4fa9 | 397 | :ref:`netdev-FAQ`. |
34f15bf3 DB |
398 | |
399 | Q: Do you also backport to kernels not currently maintained as stable? | |
54222838 | 400 | ---------------------------------------------------------------------- |
34f15bf3 | 401 | A: No. If you need a specific BPF commit in kernels that are currently not |
54222838 | 402 | maintained by the stable maintainers, then you are on your own. |
34f15bf3 | 403 | |
54222838 | 404 | The current stable and longterm stable kernels are all listed here: |
34f15bf3 | 405 | |
54222838 | 406 | https://www.kernel.org/ |
34f15bf3 | 407 | |
54222838 JDB |
408 | Q: The BPF patch I am about to submit needs to go to stable as well |
409 | ------------------------------------------------------------------- | |
410 | What should I do? | |
34f15bf3 DB |
411 | |
412 | A: The same rules apply as with netdev patch submissions in general, see | |
287f4fa9 | 413 | the :ref:`netdev-FAQ`. |
34f15bf3 | 414 | |
54222838 JDB |
415 | Never add "``Cc: stable@vger.kernel.org``" to the patch description, but |
416 | ask the BPF maintainers to queue the patches instead. This can be done | |
417 | with a note, for example, under the ``---`` part of the patch which does | |
418 | not go into the git log. Alternatively, this can be done as a simple | |
419 | request by mail instead. | |
34f15bf3 | 420 | |
54222838 JDB |
421 | Q: Queue stable patches |
422 | ----------------------- | |
34f15bf3 | 423 | Q: Where do I find currently queued BPF patches that will be submitted |
54222838 | 424 | to stable? |
34f15bf3 DB |
425 | |
426 | A: Once patches that fix critical bugs got applied into the bpf tree, they | |
54222838 | 427 | are queued up for stable submission under: |
34f15bf3 | 428 | |
54222838 | 429 | http://patchwork.ozlabs.org/bundle/bpf/stable/?state=* |
34f15bf3 | 430 | |
54222838 JDB |
431 | They will be on hold there at minimum until the related commit made its |
432 | way into the mainline kernel tree. | |
34f15bf3 | 433 | |
54222838 JDB |
434 | After having been under broader exposure, the queued patches will be |
435 | submitted by the BPF maintainers to the stable maintainers. | |
34f15bf3 | 436 | |
54222838 JDB |
437 | Testing patches |
438 | =============== | |
34f15bf3 | 439 | |
b7a27c3a JDB |
440 | Q: How to run BPF selftests |
441 | --------------------------- | |
442 | A: After you have booted into the newly compiled kernel, navigate to | |
443 | the BPF selftests_ suite in order to test BPF functionality (current | |
444 | working directory points to the root of the cloned git tree):: | |
445 | ||
446 | $ cd tools/testing/selftests/bpf/ | |
447 | $ make | |
448 | ||
449 | To run the verifier tests:: | |
450 | ||
451 | $ sudo ./test_verifier | |
452 | ||
453 | The verifier tests print out all the current checks being | |
454 | performed. The summary at the end of running all tests will dump | |
455 | information of test successes and failures:: | |
456 | ||
457 | Summary: 418 PASSED, 0 FAILED | |
458 | ||
459 | In order to run through all BPF selftests, the following command is | |
460 | needed:: | |
461 | ||
462 | $ sudo make run_tests | |
463 | ||
464 | See the kernels selftest `Documentation/dev-tools/kselftest.rst`_ | |
465 | document for further documentation. | |
466 | ||
6736aa79 AM |
467 | To maximize the number of tests passing, the .config of the kernel |
468 | under test should match the config file fragment in | |
469 | tools/testing/selftests/bpf as closely as possible. | |
470 | ||
471 | Finally to ensure support for latest BPF Type Format features - | |
472 | discussed in `Documentation/bpf/btf.rst`_ - pahole version 1.16 | |
473 | is required for kernels built with CONFIG_DEBUG_INFO_BTF=y. | |
474 | pahole is delivered in the dwarves package or can be built | |
475 | from source at | |
476 | ||
477 | https://github.com/acmel/dwarves | |
478 | ||
350a62ca TY |
479 | pahole starts to use libbpf definitions and APIs since v1.13 after the |
480 | commit 21507cd3e97b ("pahole: add libbpf as submodule under lib/bpf"). | |
481 | It works well with the git repository because the libbpf submodule will | |
482 | use "git submodule update --init --recursive" to update. | |
483 | ||
484 | Unfortunately, the default github release source code does not contain | |
485 | libbpf submodule source code and this will cause build issues, the tarball | |
486 | from https://git.kernel.org/pub/scm/devel/pahole/pahole.git/ is same with | |
487 | github, you can get the source tarball with corresponding libbpf submodule | |
488 | codes from | |
489 | ||
490 | https://fedorapeople.org/~acme/dwarves | |
491 | ||
6736aa79 AM |
492 | Some distros have pahole version 1.16 packaged already, e.g. |
493 | Fedora, Gentoo. | |
494 | ||
34f15bf3 | 495 | Q: Which BPF kernel selftests version should I run my kernel against? |
54222838 JDB |
496 | --------------------------------------------------------------------- |
497 | A: If you run a kernel ``xyz``, then always run the BPF kernel selftests | |
498 | from that kernel ``xyz`` as well. Do not expect that the BPF selftest | |
499 | from the latest mainline tree will pass all the time. | |
34f15bf3 | 500 | |
54222838 JDB |
501 | In particular, test_bpf.c and test_verifier.c have a large number of |
502 | test cases and are constantly updated with new BPF test sequences, or | |
503 | existing ones are adapted to verifier changes e.g. due to verifier | |
504 | becoming smarter and being able to better track certain things. | |
34f15bf3 | 505 | |
54222838 JDB |
506 | LLVM |
507 | ==== | |
34f15bf3 DB |
508 | |
509 | Q: Where do I find LLVM with BPF support? | |
54222838 | 510 | ----------------------------------------- |
34f15bf3 DB |
511 | A: The BPF back end for LLVM is upstream in LLVM since version 3.7.1. |
512 | ||
54222838 JDB |
513 | All major distributions these days ship LLVM with BPF back end enabled, |
514 | so for the majority of use-cases it is not required to compile LLVM by | |
515 | hand anymore, just install the distribution provided package. | |
34f15bf3 | 516 | |
54222838 JDB |
517 | LLVM's static compiler lists the supported targets through |
518 | ``llc --version``, make sure BPF targets are listed. Example:: | |
34f15bf3 DB |
519 | |
520 | $ llc --version | |
521 | LLVM (http://llvm.org/): | |
4d0d1673 | 522 | LLVM version 10.0.0 |
34f15bf3 DB |
523 | Optimized build. |
524 | Default target: x86_64-unknown-linux-gnu | |
525 | Host CPU: skylake | |
526 | ||
527 | Registered Targets: | |
4d0d1673 JL |
528 | aarch64 - AArch64 (little endian) |
529 | bpf - BPF (host endian) | |
530 | bpfeb - BPF (big endian) | |
531 | bpfel - BPF (little endian) | |
532 | x86 - 32-bit X86: Pentium-Pro and above | |
533 | x86-64 - 64-bit X86: EM64T and AMD64 | |
34f15bf3 | 534 | |
54222838 JDB |
535 | For developers in order to utilize the latest features added to LLVM's |
536 | BPF back end, it is advisable to run the latest LLVM releases. Support | |
537 | for new BPF kernel features such as additions to the BPF instruction | |
538 | set are often developed together. | |
34f15bf3 | 539 | |
54222838 | 540 | All LLVM releases can be found at: http://releases.llvm.org/ |
34f15bf3 DB |
541 | |
542 | Q: Got it, so how do I build LLVM manually anyway? | |
54222838 | 543 | -------------------------------------------------- |
628add78 TY |
544 | A: We recommend that developers who want the fastest incremental builds |
545 | use the Ninja build system, you can find it in your system's package | |
546 | manager, usually the package is ninja or ninja-build. | |
547 | ||
548 | You need ninja, cmake and gcc-c++ as build requisites for LLVM. Once you | |
549 | have that set up, proceed with building the latest LLVM and clang version | |
54222838 | 550 | from the git repositories:: |
34f15bf3 | 551 | |
e42da4c6 | 552 | $ git clone https://github.com/llvm/llvm-project.git |
628add78 | 553 | $ mkdir -p llvm-project/llvm/build |
e42da4c6 YS |
554 | $ cd llvm-project/llvm/build |
555 | $ cmake .. -G "Ninja" -DLLVM_TARGETS_TO_BUILD="BPF;X86" \ | |
556 | -DLLVM_ENABLE_PROJECTS="clang" \ | |
34f15bf3 DB |
557 | -DCMAKE_BUILD_TYPE=Release \ |
558 | -DLLVM_BUILD_RUNTIME=OFF | |
e42da4c6 | 559 | $ ninja |
34f15bf3 | 560 | |
54222838 JDB |
561 | The built binaries can then be found in the build/bin/ directory, where |
562 | you can point the PATH variable to. | |
34f15bf3 | 563 | |
4d0d1673 JL |
564 | Set ``-DLLVM_TARGETS_TO_BUILD`` equal to the target you wish to build, you |
565 | will find a full list of targets within the llvm-project/llvm/lib/Target | |
566 | directory. | |
567 | ||
54222838 JDB |
568 | Q: Reporting LLVM BPF issues |
569 | ---------------------------- | |
34f15bf3 | 570 | Q: Should I notify BPF kernel maintainers about issues in LLVM's BPF code |
54222838 JDB |
571 | generation back end or about LLVM generated code that the verifier |
572 | refuses to accept? | |
573 | ||
574 | A: Yes, please do! | |
34f15bf3 | 575 | |
54222838 JDB |
576 | LLVM's BPF back end is a key piece of the whole BPF |
577 | infrastructure and it ties deeply into verification of programs from the | |
578 | kernel side. Therefore, any issues on either side need to be investigated | |
579 | and fixed whenever necessary. | |
34f15bf3 | 580 | |
54222838 JDB |
581 | Therefore, please make sure to bring them up at netdev kernel mailing |
582 | list and Cc BPF maintainers for LLVM and kernel bits: | |
34f15bf3 | 583 | |
54222838 JDB |
584 | * Yonghong Song <yhs@fb.com> |
585 | * Alexei Starovoitov <ast@kernel.org> | |
586 | * Daniel Borkmann <daniel@iogearbox.net> | |
34f15bf3 | 587 | |
54222838 | 588 | LLVM also has an issue tracker where BPF related bugs can be found: |
34f15bf3 | 589 | |
54222838 | 590 | https://bugs.llvm.org/buglist.cgi?quicksearch=bpf |
34f15bf3 | 591 | |
54222838 JDB |
592 | However, it is better to reach out through mailing lists with having |
593 | maintainers in Cc. | |
34f15bf3 | 594 | |
54222838 JDB |
595 | Q: New BPF instruction for kernel and LLVM |
596 | ------------------------------------------ | |
34f15bf3 | 597 | Q: I have added a new BPF instruction to the kernel, how can I integrate |
54222838 | 598 | it into LLVM? |
34f15bf3 | 599 | |
54222838 JDB |
600 | A: LLVM has a ``-mcpu`` selector for the BPF back end in order to allow |
601 | the selection of BPF instruction set extensions. By default the | |
602 | ``generic`` processor target is used, which is the base instruction set | |
603 | (v1) of BPF. | |
34f15bf3 | 604 | |
54222838 JDB |
605 | LLVM has an option to select ``-mcpu=probe`` where it will probe the host |
606 | kernel for supported BPF instruction set extensions and selects the | |
607 | optimal set automatically. | |
34f15bf3 | 608 | |
54222838 | 609 | For cross-compilation, a specific version can be select manually as well :: |
34f15bf3 DB |
610 | |
611 | $ llc -march bpf -mcpu=help | |
612 | Available CPUs for this target: | |
613 | ||
614 | generic - Select the generic processor. | |
615 | probe - Select the probe processor. | |
616 | v1 - Select the v1 processor. | |
617 | v2 - Select the v2 processor. | |
618 | [...] | |
619 | ||
54222838 JDB |
620 | Newly added BPF instructions to the Linux kernel need to follow the same |
621 | scheme, bump the instruction set version and implement probing for the | |
622 | extensions such that ``-mcpu=probe`` users can benefit from the | |
623 | optimization transparently when upgrading their kernels. | |
34f15bf3 | 624 | |
54222838 JDB |
625 | If you are unable to implement support for the newly added BPF instruction |
626 | please reach out to BPF developers for help. | |
34f15bf3 | 627 | |
54222838 JDB |
628 | By the way, the BPF kernel selftests run with ``-mcpu=probe`` for better |
629 | test coverage. | |
34f15bf3 | 630 | |
54222838 JDB |
631 | Q: clang flag for target bpf? |
632 | ----------------------------- | |
633 | Q: In some cases clang flag ``-target bpf`` is used but in other cases the | |
634 | default clang target, which matches the underlying architecture, is used. | |
635 | What is the difference and when I should use which? | |
6215ea6b YS |
636 | |
637 | A: Although LLVM IR generation and optimization try to stay architecture | |
54222838 JDB |
638 | independent, ``-target <arch>`` still has some impact on generated code: |
639 | ||
640 | - BPF program may recursively include header file(s) with file scope | |
641 | inline assembly codes. The default target can handle this well, | |
642 | while ``bpf`` target may fail if bpf backend assembler does not | |
643 | understand these assembly codes, which is true in most cases. | |
644 | ||
645 | - When compiled without ``-g``, additional elf sections, e.g., | |
646 | .eh_frame and .rela.eh_frame, may be present in the object file | |
647 | with default target, but not with ``bpf`` target. | |
648 | ||
649 | - The default target may turn a C switch statement into a switch table | |
650 | lookup and jump operation. Since the switch table is placed | |
651 | in the global readonly section, the bpf program will fail to load. | |
652 | The bpf target does not support switch table optimization. | |
653 | The clang option ``-fno-jump-tables`` can be used to disable | |
654 | switch table generation. | |
655 | ||
656 | - For clang ``-target bpf``, it is guaranteed that pointer or long / | |
657 | unsigned long types will always have a width of 64 bit, no matter | |
658 | whether underlying clang binary or default target (or kernel) is | |
659 | 32 bit. However, when native clang target is used, then it will | |
660 | compile these types based on the underlying architecture's conventions, | |
661 | meaning in case of 32 bit architecture, pointer or long / unsigned | |
662 | long types e.g. in BPF context structure will have width of 32 bit | |
663 | while the BPF LLVM back end still operates in 64 bit. The native | |
664 | target is mostly needed in tracing for the case of walking ``pt_regs`` | |
665 | or other kernel structures where CPU's register width matters. | |
666 | Otherwise, ``clang -target bpf`` is generally recommended. | |
667 | ||
668 | You should use default target when: | |
669 | ||
670 | - Your program includes a header file, e.g., ptrace.h, which eventually | |
671 | pulls in some header files containing file scope host assembly codes. | |
672 | ||
673 | - You can add ``-fno-jump-tables`` to work around the switch table issue. | |
674 | ||
675 | Otherwise, you can use ``bpf`` target. Additionally, you *must* use bpf target | |
676 | when: | |
677 | ||
678 | - Your program uses data structures with pointer or long / unsigned long | |
679 | types that interface with BPF helpers or context data structures. Access | |
680 | into these structures is verified by the BPF verifier and may result | |
681 | in verification failures if the native architecture is not aligned with | |
682 | the BPF architecture, e.g. 64-bit. An example of this is | |
683 | BPF_PROG_TYPE_SK_MSG require ``-target bpf`` | |
684 | ||
685 | ||
686 | .. Links | |
687 | .. _Documentation/process/: https://www.kernel.org/doc/html/latest/process/ | |
8df01363 | 688 | .. _netdev-FAQ: Documentation/process/maintainer-netdev.rst |
64ef3ddf TY |
689 | .. _selftests: |
690 | https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/tools/testing/selftests/bpf/ | |
b7a27c3a JDB |
691 | .. _Documentation/dev-tools/kselftest.rst: |
692 | https://www.kernel.org/doc/html/latest/dev-tools/kselftest.html | |
009abf5b | 693 | .. _Documentation/bpf/btf.rst: btf.rst |
6215ea6b | 694 | |
34f15bf3 | 695 | Happy BPF hacking! |