Commit | Line | Data |
---|---|---|
cef04cdc | 1 | ====================== |
80d03428 | 2 | Linux Kernel Selftests |
cef04cdc | 3 | ====================== |
80d03428 JK |
4 | |
5 | The kernel contains a set of "self tests" under the tools/testing/selftests/ | |
3409f9ab AA |
6 | directory. These are intended to be small tests to exercise individual code |
7 | paths in the kernel. Tests are intended to be run after building, installing | |
8 | and booting a kernel. | |
80d03428 | 9 | |
dede34b2 SK |
10 | Kselftest from mainline can be run on older stable kernels. Running tests |
11 | from mainline offers the best coverage. Several test rings run mainline | |
12 | kselftest suite on stable releases. The reason is that when a new test | |
13 | gets added to test existing code to regression test a bug, we should be | |
14 | able to run that test on an older kernel. Hence, it is important to keep | |
15 | code that can still test an older kernel and make sure it skips the test | |
16 | gracefully on newer releases. | |
17 | ||
9f436194 SK |
18 | You can find additional information on Kselftest framework, how to |
19 | write new tests using the framework on Kselftest wiki: | |
20 | ||
21 | https://kselftest.wiki.kernel.org/ | |
22 | ||
ddddda9b SK |
23 | On some systems, hot-plug tests could hang forever waiting for cpu and |
24 | memory to be ready to be offlined. A special hot-plug target is created | |
c284d428 | 25 | to run the full range of hot-plug tests. In default mode, hot-plug tests run |
ddddda9b SK |
26 | in safe mode with a limited scope. In limited mode, cpu-hotplug test is |
27 | run on a single cpu as opposed to all hotplug capable cpus, and memory | |
28 | hotplug test is run on 2% of hotplug capable memory instead of 10%. | |
29 | ||
eebf4dd4 TH |
30 | kselftest runs as a userspace process. Tests that can be written/run in |
31 | userspace may wish to use the `Test Harness`_. Tests that need to be | |
32 | run in kernel space may wish to use a `Test Module`_. | |
33 | ||
ddddda9b SK |
34 | Running the selftests (hotplug tests are run in limited mode) |
35 | ============================================================= | |
80d03428 | 36 | |
cef04cdc MS |
37 | To build the tests:: |
38 | ||
fde662d9 | 39 | $ make -C tools/testing/selftests |
cef04cdc MS |
40 | |
41 | To run the tests:: | |
80d03428 | 42 | |
fde662d9 | 43 | $ make -C tools/testing/selftests run_tests |
80d03428 | 44 | |
cef04cdc | 45 | To build and run the tests with a single command, use:: |
80d03428 | 46 | |
fde662d9 | 47 | $ make kselftest |
3c415707 | 48 | |
cef04cdc | 49 | Note that some tests will require root privileges. |
80d03428 | 50 | |
9f436194 SK |
51 | Kselftest supports saving output files in a separate directory and then |
52 | running tests. To locate output files in a separate directory two syntaxes | |
53 | are supported. In both cases the working directory must be the root of the | |
54 | kernel src. This is applicable to "Running a subset of selftests" section | |
55 | below. | |
56 | ||
57 | To build, save output files in a separate directory with O= :: | |
d5cdbb87 SK |
58 | |
59 | $ make O=/tmp/kselftest kselftest | |
60 | ||
9f436194 SK |
61 | To build, save output files in a separate directory with KBUILD_OUTPUT :: |
62 | ||
63 | $ export KBUILD_OUTPUT=/tmp/kselftest; make kselftest | |
d5cdbb87 | 64 | |
9f436194 SK |
65 | The O= assignment takes precedence over the KBUILD_OUTPUT environment |
66 | variable. | |
d5cdbb87 | 67 | |
9f436194 SK |
68 | The above commands by default run the tests and print full pass/fail report. |
69 | Kselftest supports "summary" option to make it easier to understand the test | |
70 | results. Please find the detailed individual test results for each test in | |
71 | /tmp/testname file(s) when summary option is specified. This is applicable | |
72 | to "Running a subset of selftests" section below. | |
73 | ||
74 | To run kselftest with summary option enabled :: | |
75 | ||
76 | $ make summary=1 kselftest | |
80d03428 | 77 | |
3c415707 | 78 | Running a subset of selftests |
cef04cdc MS |
79 | ============================= |
80 | ||
3c415707 TB |
81 | You can use the "TARGETS" variable on the make command line to specify |
82 | single test to run, or a list of tests to run. | |
83 | ||
cef04cdc MS |
84 | To run only tests targeted for a single subsystem:: |
85 | ||
fde662d9 | 86 | $ make -C tools/testing/selftests TARGETS=ptrace run_tests |
3c415707 | 87 | |
cef04cdc MS |
88 | You can specify multiple tests to build and run:: |
89 | ||
fde662d9 | 90 | $ make TARGETS="size timers" kselftest |
3c415707 | 91 | |
9f436194 | 92 | To build, save output files in a separate directory with O= :: |
d5cdbb87 SK |
93 | |
94 | $ make O=/tmp/kselftest TARGETS="size timers" kselftest | |
95 | ||
9f436194 | 96 | To build, save output files in a separate directory with KBUILD_OUTPUT :: |
d5cdbb87 | 97 | |
9f436194 | 98 | $ export KBUILD_OUTPUT=/tmp/kselftest; make TARGETS="size timers" kselftest |
d5cdbb87 | 99 | |
3a24f7f6 CM |
100 | Additionally you can use the "SKIP_TARGETS" variable on the make command |
101 | line to specify one or more targets to exclude from the TARGETS list. | |
102 | ||
103 | To run all tests but a single subsystem:: | |
104 | ||
105 | $ make -C tools/testing/selftests SKIP_TARGETS=ptrace run_tests | |
106 | ||
107 | You can specify multiple tests to skip:: | |
108 | ||
109 | $ make SKIP_TARGETS="size timers" kselftest | |
110 | ||
111 | You can also specify a restricted list of tests to run together with a | |
112 | dedicated skiplist:: | |
113 | ||
114 | $ make TARGETS="bpf breakpoints size timers" SKIP_TARGETS=bpf kselftest | |
115 | ||
3c415707 TB |
116 | See the top-level tools/testing/selftests/Makefile for the list of all |
117 | possible targets. | |
80d03428 | 118 | |
ddddda9b SK |
119 | Running the full range hotplug selftests |
120 | ======================================== | |
121 | ||
cef04cdc MS |
122 | To build the hotplug tests:: |
123 | ||
fde662d9 | 124 | $ make -C tools/testing/selftests hotplug |
cef04cdc MS |
125 | |
126 | To run the hotplug tests:: | |
ddddda9b | 127 | |
fde662d9 | 128 | $ make -C tools/testing/selftests run_hotplug |
ddddda9b | 129 | |
cef04cdc | 130 | Note that some tests will require root privileges. |
80d03428 | 131 | |
3c415707 | 132 | |
76caa6ca WL |
133 | Install selftests |
134 | ================= | |
135 | ||
4c851131 KC |
136 | You can use the "install" target of "make" (which calls the `kselftest_install.sh` |
137 | tool) to install selftests in the default location (`tools/testing/selftests/kselftest_install`), | |
138 | or in a user specified location via the `INSTALL_PATH` "make" variable. | |
76caa6ca | 139 | |
cef04cdc | 140 | To install selftests in default location:: |
76caa6ca | 141 | |
4c851131 | 142 | $ make -C tools/testing/selftests install |
cef04cdc MS |
143 | |
144 | To install selftests in a user specified location:: | |
145 | ||
4c851131 | 146 | $ make -C tools/testing/selftests install INSTALL_PATH=/some/other/path |
76caa6ca | 147 | |
78566cf1 SK |
148 | Running installed selftests |
149 | =========================== | |
150 | ||
4c851131 KC |
151 | Found in the install directory, as well as in the Kselftest tarball, |
152 | is a script named `run_kselftest.sh` to run the tests. | |
78566cf1 | 153 | |
c284d428 | 154 | You can simply do the following to run the installed Kselftests. Please |
fde662d9 | 155 | note some tests will require root privileges:: |
78566cf1 | 156 | |
4c851131 | 157 | $ cd kselftest_install |
fde662d9 | 158 | $ ./run_kselftest.sh |
76caa6ca | 159 | |
4c851131 KC |
160 | To see the list of available tests, the `-l` option can be used:: |
161 | ||
162 | $ ./run_kselftest.sh -l | |
163 | ||
164 | The `-c` option can be used to run all the tests from a test collection, or | |
165 | the `-t` option for specific single tests. Either can be used multiple times:: | |
166 | ||
167 | $ ./run_kselftest.sh -c bpf -c seccomp -t timers:posix_timers -t timer:nanosleep | |
168 | ||
169 | For other features see the script usage output, seen with the `-h` option. | |
170 | ||
a5f30467 VK |
171 | Packaging selftests |
172 | =================== | |
173 | ||
174 | In some cases packaging is desired, such as when tests need to run on a | |
175 | different system. To package selftests, run:: | |
176 | ||
177 | $ make -C tools/testing/selftests gen_tar | |
178 | ||
179 | This generates a tarball in the `INSTALL_PATH/kselftest-packages` directory. By | |
4c851131 KC |
180 | default, `.gz` format is used. The tar compression format can be overridden by |
181 | specifying a `FORMAT` make variable. Any value recognized by `tar's auto-compress`_ | |
182 | option is supported, such as:: | |
a5f30467 VK |
183 | |
184 | $ make -C tools/testing/selftests gen_tar FORMAT=.xz | |
185 | ||
186 | `make gen_tar` invokes `make install` so you can use it to package a subset of | |
187 | tests by using variables specified in `Running a subset of selftests`_ | |
188 | section:: | |
189 | ||
190 | $ make -C tools/testing/selftests gen_tar TARGETS="bpf" FORMAT=.xz | |
191 | ||
192 | .. _tar's auto-compress: https://www.gnu.org/software/tar/manual/html_node/gzip.html#auto_002dcompress | |
193 | ||
80d03428 JK |
194 | Contributing new tests |
195 | ====================== | |
196 | ||
4db1fa66 | 197 | In general, the rules for selftests are |
80d03428 JK |
198 | |
199 | * Do as much as you can if you're not root; | |
200 | ||
201 | * Don't take too long; | |
202 | ||
203 | * Don't break the build on any architecture, and | |
204 | ||
205 | * Don't cause the top-level "make run_tests" to fail if your feature is | |
206 | unconfigured. | |
88baa78d | 207 | |
cef04cdc MS |
208 | Contributing new tests (details) |
209 | ================================ | |
88baa78d | 210 | |
82063610 MUA |
211 | * In your Makefile, use facilities from lib.mk by including it instead of |
212 | reinventing the wheel. Specify flags and binaries generation flags on | |
213 | need basis before including lib.mk. :: | |
214 | ||
215 | CFLAGS = $(KHDR_INCLUDES) | |
216 | TEST_GEN_PROGS := close_range_test | |
217 | include ../lib.mk | |
218 | ||
88baa78d | 219 | * Use TEST_GEN_XXX if such binaries or files are generated during |
220 | compiling. | |
fde662d9 | 221 | |
d5cdbb87 | 222 | TEST_PROGS, TEST_GEN_PROGS mean it is the executable tested by |
88baa78d | 223 | default. |
fde662d9 | 224 | |
d5cdbb87 | 225 | TEST_CUSTOM_PROGS should be used by tests that require custom build |
c284d428 | 226 | rules and prevent common build rule use. |
d5cdbb87 SK |
227 | |
228 | TEST_PROGS are for test shell scripts. Please ensure shell script has | |
229 | its exec bit set. Otherwise, lib.mk run_tests will generate a warning. | |
230 | ||
231 | TEST_CUSTOM_PROGS and TEST_PROGS will be run by common run_tests. | |
232 | ||
88baa78d | 233 | TEST_PROGS_EXTENDED, TEST_GEN_PROGS_EXTENDED mean it is the |
234 | executable which is not tested by default. | |
235 | TEST_FILES, TEST_GEN_FILES mean it is the file which is used by | |
236 | test. | |
7e6a32ab | 237 | |
a997a703 AR |
238 | * First use the headers inside the kernel source and/or git repo, and then the |
239 | system headers. Headers for the kernel release as opposed to headers | |
240 | installed by the distro on the system should be the primary focus to be able | |
82063610 MUA |
241 | to find regressions. Use KHDR_INCLUDES in Makefile to include headers from |
242 | the kernel source. | |
a997a703 | 243 | |
8ab1416a AR |
244 | * If a test needs specific kernel config options enabled, add a config file in |
245 | the test directory to enable them. | |
246 | ||
0e0f0093 | 247 | e.g: tools/testing/selftests/android/config |
8ab1416a | 248 | |
82063610 MUA |
249 | * Create a .gitignore file inside test directory and add all generated objects |
250 | in it. | |
251 | ||
252 | * Add new test name in TARGETS in selftests/Makefile:: | |
253 | ||
254 | TARGETS += android | |
255 | ||
256 | * All changes should pass:: | |
257 | ||
258 | kselftest-{all,install,clean,gen_tar} | |
259 | kselftest-{all,install,clean,gen_tar} O=abo_path | |
260 | kselftest-{all,install,clean,gen_tar} O=rel_path | |
261 | make -C tools/testing/selftests {all,install,clean,gen_tar} | |
262 | make -C tools/testing/selftests {all,install,clean,gen_tar} O=abs_path | |
263 | make -C tools/testing/selftests {all,install,clean,gen_tar} O=rel_path | |
264 | ||
eebf4dd4 TH |
265 | Test Module |
266 | =========== | |
267 | ||
268 | Kselftest tests the kernel from userspace. Sometimes things need | |
269 | testing from within the kernel, one method of doing this is to create a | |
270 | test module. We can tie the module into the kselftest framework by | |
3a019636 | 271 | using a shell script test runner. ``kselftest/module.sh`` is designed |
eebf4dd4 TH |
272 | to facilitate this process. There is also a header file provided to |
273 | assist writing kernel modules that are for use with kselftest: | |
274 | ||
87f1c20e AT |
275 | - ``tools/testing/selftests/kselftest_module.h`` |
276 | - ``tools/testing/selftests/kselftest/module.sh`` | |
eebf4dd4 | 277 | |
8370b400 DG |
278 | Note that test modules should taint the kernel with TAINT_TEST. This will |
279 | happen automatically for modules which are in the ``tools/testing/`` | |
280 | directory, or for modules which use the ``kselftest_module.h`` header above. | |
281 | Otherwise, you'll need to add ``MODULE_INFO(test, "Y")`` to your module | |
282 | source. selftests which do not load modules typically should not taint the | |
283 | kernel, but in cases where a non-test module is loaded, TEST_TAINT can be | |
284 | applied from userspace by writing to ``/proc/sys/kernel/tainted``. | |
285 | ||
eebf4dd4 TH |
286 | How to use |
287 | ---------- | |
288 | ||
289 | Here we show the typical steps to create a test module and tie it into | |
290 | kselftest. We use kselftests for lib/ as an example. | |
291 | ||
292 | 1. Create the test module | |
293 | ||
294 | 2. Create the test script that will run (load/unload) the module | |
295 | e.g. ``tools/testing/selftests/lib/printf.sh`` | |
296 | ||
297 | 3. Add line to config file e.g. ``tools/testing/selftests/lib/config`` | |
298 | ||
299 | 4. Add test script to makefile e.g. ``tools/testing/selftests/lib/Makefile`` | |
300 | ||
301 | 5. Verify it works: | |
302 | ||
303 | .. code-block:: sh | |
304 | ||
305 | # Assumes you have booted a fresh build of this kernel tree | |
306 | cd /path/to/linux/tree | |
307 | make kselftest-merge | |
308 | make modules | |
309 | sudo make modules_install | |
310 | make TARGETS=lib kselftest | |
311 | ||
312 | Example Module | |
313 | -------------- | |
314 | ||
315 | A bare bones test module might look like this: | |
316 | ||
317 | .. code-block:: c | |
318 | ||
319 | // SPDX-License-Identifier: GPL-2.0+ | |
320 | ||
321 | #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt | |
322 | ||
83e14a57 | 323 | #include "../tools/testing/selftests/kselftest_module.h" |
eebf4dd4 TH |
324 | |
325 | KSTM_MODULE_GLOBALS(); | |
326 | ||
327 | /* | |
328 | * Kernel module for testing the foobinator | |
329 | */ | |
330 | ||
331 | static int __init test_function() | |
332 | { | |
333 | ... | |
334 | } | |
335 | ||
336 | static void __init selftest(void) | |
337 | { | |
338 | KSTM_CHECK_ZERO(do_test_case("", 0)); | |
339 | } | |
340 | ||
341 | KSTM_MODULE_LOADERS(test_foo); | |
342 | MODULE_AUTHOR("John Developer <jd@fooman.org>"); | |
343 | MODULE_LICENSE("GPL"); | |
8370b400 | 344 | MODULE_INFO(test, "Y"); |
eebf4dd4 TH |
345 | |
346 | Example test script | |
347 | ------------------- | |
348 | ||
349 | .. code-block:: sh | |
350 | ||
351 | #!/bin/bash | |
352 | # SPDX-License-Identifier: GPL-2.0+ | |
3a019636 | 353 | $(dirname $0)/../kselftest/module.sh "foo" test_foo |
eebf4dd4 TH |
354 | |
355 | ||
7e6a32ab MS |
356 | Test Harness |
357 | ============ | |
358 | ||
eebf4dd4 TH |
359 | The kselftest_harness.h file contains useful helpers to build tests. The |
360 | test harness is for userspace testing, for kernel space testing see `Test | |
361 | Module`_ above. | |
362 | ||
363 | The tests from tools/testing/selftests/seccomp/seccomp_bpf.c can be used as | |
364 | example. | |
7e6a32ab MS |
365 | |
366 | Example | |
367 | ------- | |
368 | ||
369 | .. kernel-doc:: tools/testing/selftests/kselftest_harness.h | |
370 | :doc: example | |
371 | ||
372 | ||
373 | Helpers | |
374 | ------- | |
375 | ||
376 | .. kernel-doc:: tools/testing/selftests/kselftest_harness.h | |
377 | :functions: TH_LOG TEST TEST_SIGNAL FIXTURE FIXTURE_DATA FIXTURE_SETUP | |
74bc7c97 JK |
378 | FIXTURE_TEARDOWN TEST_F TEST_HARNESS_MAIN FIXTURE_VARIANT |
379 | FIXTURE_VARIANT_ADD | |
7e6a32ab MS |
380 | |
381 | Operators | |
382 | --------- | |
383 | ||
384 | .. kernel-doc:: tools/testing/selftests/kselftest_harness.h | |
385 | :doc: operators | |
386 | ||
387 | .. kernel-doc:: tools/testing/selftests/kselftest_harness.h | |
388 | :functions: ASSERT_EQ ASSERT_NE ASSERT_LT ASSERT_LE ASSERT_GT ASSERT_GE | |
389 | ASSERT_NULL ASSERT_TRUE ASSERT_NULL ASSERT_TRUE ASSERT_FALSE | |
390 | ASSERT_STREQ ASSERT_STRNE EXPECT_EQ EXPECT_NE EXPECT_LT | |
391 | EXPECT_LE EXPECT_GT EXPECT_GE EXPECT_NULL EXPECT_TRUE | |
392 | EXPECT_FALSE EXPECT_STREQ EXPECT_STRNE |