Commit | Line | Data |
---|---|---|
10ffebbe | 1 | =========================================== |
de1ba09b AM |
2 | Fault injection capabilities infrastructure |
3 | =========================================== | |
4 | ||
1892ce4c | 5 | See also drivers/md/md-faulty.c and "every_nth" module option for scsi_debug. |
de1ba09b AM |
6 | |
7 | ||
8 | Available fault injection capabilities | |
9 | -------------------------------------- | |
10 | ||
10ffebbe | 11 | - failslab |
de1ba09b AM |
12 | |
13 | injects slab allocation failures. (kmalloc(), kmem_cache_alloc(), ...) | |
14 | ||
10ffebbe | 15 | - fail_page_alloc |
de1ba09b AM |
16 | |
17 | injects page allocation failures. (alloc_pages(), get_free_pages(), ...) | |
18 | ||
2c739ced AL |
19 | - fail_usercopy |
20 | ||
21 | injects failures in user memory access functions. (copy_from_user(), get_user(), ...) | |
22 | ||
10ffebbe | 23 | - fail_futex |
ab51fbab DB |
24 | |
25 | injects futex deadlock and uaddr fault errors. | |
26 | ||
400edd8c CL |
27 | - fail_sunrpc |
28 | ||
29 | injects kernel RPC client and server failures. | |
30 | ||
10ffebbe | 31 | - fail_make_request |
de1ba09b | 32 | |
5d0ffa2b | 33 | injects disk IO errors on devices permitted by setting |
de1ba09b | 34 | /sys/block/<device>/make-it-fail or |
ed00aabd | 35 | /sys/block/<device>/<partition>/make-it-fail. (submit_bio_noacct()) |
de1ba09b | 36 | |
10ffebbe | 37 | - fail_mmc_request |
1e4cb22b PF |
38 | |
39 | injects MMC data errors on devices permitted by setting | |
40 | debugfs entries under /sys/kernel/debug/mmc0/fail_mmc_request | |
41 | ||
10ffebbe | 42 | - fail_function |
4b1a29a7 MH |
43 | |
44 | injects error return on specific functions, which are marked by | |
45 | ALLOW_ERROR_INJECTION() macro, by setting debugfs entries | |
46 | under /sys/kernel/debug/fail_function. No boot option supported. | |
47 | ||
10ffebbe | 48 | - NVMe fault injection |
cf4182f3 TT |
49 | |
50 | inject NVMe status code and retry flag on devices permitted by setting | |
51 | debugfs entries under /sys/kernel/debug/nvme*/fault_inject. The default | |
52 | status code is NVME_SC_INVALID_OPCODE with no retry. The status code and | |
53 | retry flag can be set via the debugfs. | |
54 | ||
bb4c19e0 AM |
55 | - Null test block driver fault injection |
56 | ||
57 | inject IO timeouts by setting config items under | |
58 | /sys/kernel/config/nullb/<disk>/timeout_inject, | |
59 | inject requeue requests by setting config items under | |
60 | /sys/kernel/config/nullb/<disk>/requeue_inject, and | |
61 | inject init_hctx() errors by setting config items under | |
62 | /sys/kernel/config/nullb/<disk>/init_hctx_fault_inject. | |
cf4182f3 | 63 | |
de1ba09b AM |
64 | Configure fault-injection capabilities behavior |
65 | ----------------------------------------------- | |
66 | ||
10ffebbe MCC |
67 | debugfs entries |
68 | ^^^^^^^^^^^^^^^ | |
de1ba09b AM |
69 | |
70 | fault-inject-debugfs kernel module provides some debugfs entries for runtime | |
71 | configuration of fault-injection capabilities. | |
72 | ||
156f5a78 | 73 | - /sys/kernel/debug/fail*/probability: |
de1ba09b AM |
74 | |
75 | likelihood of failure injection, in percent. | |
10ffebbe | 76 | |
de1ba09b AM |
77 | Format: <percent> |
78 | ||
5d0ffa2b DM |
79 | Note that one-failure-per-hundred is a very high error rate |
80 | for some testcases. Consider setting probability=100 and configure | |
156f5a78 | 81 | /sys/kernel/debug/fail*/interval for such testcases. |
de1ba09b | 82 | |
156f5a78 | 83 | - /sys/kernel/debug/fail*/interval: |
de1ba09b AM |
84 | |
85 | specifies the interval between failures, for calls to | |
86 | should_fail() that pass all the other tests. | |
87 | ||
88 | Note that if you enable this, by setting interval>1, you will | |
89 | probably want to set probability=100. | |
90 | ||
156f5a78 | 91 | - /sys/kernel/debug/fail*/times: |
de1ba09b | 92 | |
00574752 | 93 | specifies how many times failures may happen at most. A value of -1 |
d472cf79 | 94 | means "no limit". |
de1ba09b | 95 | |
156f5a78 | 96 | - /sys/kernel/debug/fail*/space: |
de1ba09b AM |
97 | |
98 | specifies an initial resource "budget", decremented by "size" | |
99 | on each call to should_fail(,size). Failure injection is | |
100 | suppressed until "space" reaches zero. | |
101 | ||
156f5a78 | 102 | - /sys/kernel/debug/fail*/verbose |
de1ba09b AM |
103 | |
104 | Format: { 0 | 1 | 2 } | |
10ffebbe | 105 | |
5d0ffa2b DM |
106 | specifies the verbosity of the messages when failure is |
107 | injected. '0' means no messages; '1' will print only a single | |
108 | log line per failure; '2' will print a call trace too -- useful | |
109 | to debug the problems revealed by fault injection. | |
de1ba09b | 110 | |
156f5a78 | 111 | - /sys/kernel/debug/fail*/task-filter: |
de1ba09b | 112 | |
5d0ffa2b | 113 | Format: { 'Y' | 'N' } |
10ffebbe | 114 | |
5d0ffa2b | 115 | A value of 'N' disables filtering by process (default). |
de1ba09b AM |
116 | Any positive value limits failures to only processes indicated by |
117 | /proc/<pid>/make-it-fail==1. | |
118 | ||
10ffebbe MCC |
119 | - /sys/kernel/debug/fail*/require-start, |
120 | /sys/kernel/debug/fail*/require-end, | |
121 | /sys/kernel/debug/fail*/reject-start, | |
122 | /sys/kernel/debug/fail*/reject-end: | |
de1ba09b AM |
123 | |
124 | specifies the range of virtual addresses tested during | |
125 | stacktrace walking. Failure is injected only if some caller | |
329409ae AM |
126 | in the walked stacktrace lies within the required range, and |
127 | none lies within the rejected range. | |
128 | Default required range is [0,ULONG_MAX) (whole of virtual address space). | |
129 | Default rejected range is [0,0). | |
de1ba09b | 130 | |
156f5a78 | 131 | - /sys/kernel/debug/fail*/stacktrace-depth: |
de1ba09b AM |
132 | |
133 | specifies the maximum stacktrace depth walked during search | |
5d0ffa2b DM |
134 | for a caller within [require-start,require-end) OR |
135 | [reject-start,reject-end). | |
de1ba09b | 136 | |
156f5a78 | 137 | - /sys/kernel/debug/fail_page_alloc/ignore-gfp-highmem: |
de1ba09b | 138 | |
5d0ffa2b | 139 | Format: { 'Y' | 'N' } |
10ffebbe | 140 | |
bad3fbb2 DY |
141 | default is 'Y', setting it to 'N' will also inject failures into |
142 | highmem/user allocations (__GFP_HIGHMEM allocations). | |
de1ba09b | 143 | |
156f5a78 GL |
144 | - /sys/kernel/debug/failslab/ignore-gfp-wait: |
145 | - /sys/kernel/debug/fail_page_alloc/ignore-gfp-wait: | |
de1ba09b | 146 | |
5d0ffa2b | 147 | Format: { 'Y' | 'N' } |
10ffebbe | 148 | |
bad3fbb2 DY |
149 | default is 'Y', setting it to 'N' will also inject failures |
150 | into allocations that can sleep (__GFP_DIRECT_RECLAIM allocations). | |
de1ba09b | 151 | |
156f5a78 | 152 | - /sys/kernel/debug/fail_page_alloc/min-order: |
54114994 AM |
153 | |
154 | specifies the minimum page allocation order to be injected | |
155 | failures. | |
156 | ||
ab51fbab DB |
157 | - /sys/kernel/debug/fail_futex/ignore-private: |
158 | ||
159 | Format: { 'Y' | 'N' } | |
10ffebbe | 160 | |
ab51fbab DB |
161 | default is 'N', setting it to 'Y' will disable failure injections |
162 | when dealing with private (address space) futexes. | |
163 | ||
400edd8c CL |
164 | - /sys/kernel/debug/fail_sunrpc/ignore-client-disconnect: |
165 | ||
166 | Format: { 'Y' | 'N' } | |
167 | ||
168 | default is 'N', setting it to 'Y' will disable disconnect | |
169 | injection on the RPC client. | |
170 | ||
171 | - /sys/kernel/debug/fail_sunrpc/ignore-server-disconnect: | |
172 | ||
173 | Format: { 'Y' | 'N' } | |
174 | ||
175 | default is 'N', setting it to 'Y' will disable disconnect | |
176 | injection on the RPC server. | |
177 | ||
36f2ef2d CL |
178 | - /sys/kernel/debug/fail_sunrpc/ignore-cache-wait: |
179 | ||
180 | Format: { 'Y' | 'N' } | |
181 | ||
182 | default is 'N', setting it to 'Y' will disable cache wait | |
183 | injection on the RPC server. | |
184 | ||
4b1a29a7 MH |
185 | - /sys/kernel/debug/fail_function/inject: |
186 | ||
187 | Format: { 'function-name' | '!function-name' | '' } | |
10ffebbe | 188 | |
4b1a29a7 MH |
189 | specifies the target function of error injection by name. |
190 | If the function name leads '!' prefix, given function is | |
191 | removed from injection list. If nothing specified ('') | |
192 | injection list is cleared. | |
193 | ||
194 | - /sys/kernel/debug/fail_function/injectable: | |
195 | ||
196 | (read only) shows error injectable functions and what type of | |
197 | error values can be specified. The error type will be one of | |
198 | below; | |
199 | - NULL: retval must be 0. | |
200 | - ERRNO: retval must be -1 to -MAX_ERRNO (-4096). | |
201 | - ERR_NULL: retval must be 0 or -1 to -MAX_ERRNO (-4096). | |
202 | ||
00574752 | 203 | - /sys/kernel/debug/fail_function/<function-name>/retval: |
4b1a29a7 | 204 | |
00574752 WS |
205 | specifies the "error" return value to inject to the given function. |
206 | This will be created when the user specifies a new injection entry. | |
207 | Note that this file only accepts unsigned values. So, if you want to | |
208 | use a negative errno, you better use 'printf' instead of 'echo', e.g.: | |
209 | $ printf %#x -12 > retval | |
4b1a29a7 | 210 | |
10ffebbe MCC |
211 | Boot option |
212 | ^^^^^^^^^^^ | |
de1ba09b AM |
213 | |
214 | In order to inject faults while debugfs is not available (early boot time), | |
10ffebbe | 215 | use the boot option:: |
de1ba09b AM |
216 | |
217 | failslab= | |
218 | fail_page_alloc= | |
2c739ced | 219 | fail_usercopy= |
1e4cb22b | 220 | fail_make_request= |
ab51fbab | 221 | fail_futex= |
199e3f4b | 222 | mmc_core.fail_request=<interval>,<probability>,<space>,<times> |
de1ba09b | 223 | |
10ffebbe MCC |
224 | proc entries |
225 | ^^^^^^^^^^^^ | |
e41d5818 | 226 | |
10ffebbe MCC |
227 | - /proc/<pid>/fail-nth, |
228 | /proc/self/task/<tid>/fail-nth: | |
e41d5818 | 229 | |
9049f2f6 | 230 | Write to this file of integer N makes N-th call in the task fail. |
bfc74093 AM |
231 | Read from this file returns a integer value. A value of '0' indicates |
232 | that the fault setup with a previous write to this file was injected. | |
233 | A positive integer N indicates that the fault wasn't yet injected. | |
e41d5818 DV |
234 | Note that this file enables all types of faults (slab, futex, etc). |
235 | This setting takes precedence over all other generic debugfs settings | |
236 | like probability, interval, times, etc. But per-capability settings | |
237 | (e.g. fail_futex/ignore-private) take precedence over it. | |
238 | ||
239 | This feature is intended for systematic testing of faults in a single | |
240 | system call. See an example below. | |
241 | ||
bef7ec4e MHG |
242 | |
243 | Error Injectable Functions | |
244 | -------------------------- | |
245 | ||
246 | This part is for the kenrel developers considering to add a function to | |
247 | ALLOW_ERROR_INJECTION() macro. | |
248 | ||
249 | Requirements for the Error Injectable Functions | |
250 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
251 | ||
252 | Since the function-level error injection forcibly changes the code path | |
253 | and returns an error even if the input and conditions are proper, this can | |
254 | cause unexpected kernel crash if you allow error injection on the function | |
255 | which is NOT error injectable. Thus, you (and reviewers) must ensure; | |
256 | ||
257 | - The function returns an error code if it fails, and the callers must check | |
258 | it correctly (need to recover from it). | |
259 | ||
260 | - The function does not execute any code which can change any state before | |
261 | the first error return. The state includes global or local, or input | |
262 | variable. For example, clear output address storage (e.g. `*ret = NULL`), | |
263 | increments/decrements counter, set a flag, preempt/irq disable or get | |
264 | a lock (if those are recovered before returning error, that will be OK.) | |
265 | ||
266 | The first requirement is important, and it will result in that the release | |
267 | (free objects) functions are usually harder to inject errors than allocate | |
268 | functions. If errors of such release functions are not correctly handled | |
269 | it will cause a memory leak easily (the caller will confuse that the object | |
270 | has been released or corrupted.) | |
271 | ||
272 | The second one is for the caller which expects the function should always | |
273 | does something. Thus if the function error injection skips whole of the | |
274 | function, the expectation is betrayed and causes an unexpected error. | |
275 | ||
276 | Type of the Error Injectable Functions | |
277 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
278 | ||
279 | Each error injectable functions will have the error type specified by the | |
280 | ALLOW_ERROR_INJECTION() macro. You have to choose it carefully if you add | |
281 | a new error injectable function. If the wrong error type is chosen, the | |
282 | kernel may crash because it may not be able to handle the error. | |
283 | There are 4 types of errors defined in include/asm-generic/error-injection.h | |
284 | ||
285 | EI_ETYPE_NULL | |
286 | This function will return `NULL` if it fails. e.g. return an allocateed | |
287 | object address. | |
288 | ||
289 | EI_ETYPE_ERRNO | |
290 | This function will return an `-errno` error code if it fails. e.g. return | |
291 | -EINVAL if the input is wrong. This will include the functions which will | |
292 | return an address which encodes `-errno` by ERR_PTR() macro. | |
293 | ||
294 | EI_ETYPE_ERRNO_NULL | |
295 | This function will return an `-errno` or `NULL` if it fails. If the caller | |
296 | of this function checks the return value with IS_ERR_OR_NULL() macro, this | |
297 | type will be appropriate. | |
298 | ||
299 | EI_ETYPE_TRUE | |
300 | This function will return `true` (non-zero positive value) if it fails. | |
301 | ||
302 | If you specifies a wrong type, for example, EI_TYPE_ERRNO for the function | |
303 | which returns an allocated object, it may cause a problem because the returned | |
304 | value is not an object address and the caller can not access to the address. | |
305 | ||
306 | ||
de1ba09b AM |
307 | How to add new fault injection capability |
308 | ----------------------------------------- | |
309 | ||
10ffebbe | 310 | - #include <linux/fault-inject.h> |
de1ba09b | 311 | |
10ffebbe | 312 | - define the fault attributes |
de1ba09b | 313 | |
2d87948a | 314 | DECLARE_FAULT_ATTR(name); |
de1ba09b AM |
315 | |
316 | Please see the definition of struct fault_attr in fault-inject.h | |
317 | for details. | |
318 | ||
10ffebbe | 319 | - provide a way to configure fault attributes |
de1ba09b AM |
320 | |
321 | - boot option | |
322 | ||
323 | If you need to enable the fault injection capability from boot time, you can | |
5d0ffa2b | 324 | provide boot option to configure it. There is a helper function for it: |
de1ba09b | 325 | |
5d0ffa2b | 326 | setup_fault_attr(attr, str); |
de1ba09b AM |
327 | |
328 | - debugfs entries | |
329 | ||
2c739ced | 330 | failslab, fail_page_alloc, fail_usercopy, and fail_make_request use this way. |
5d0ffa2b | 331 | Helper functions: |
de1ba09b | 332 | |
dd48c085 | 333 | fault_create_debugfs_attr(name, parent, attr); |
de1ba09b AM |
334 | |
335 | - module parameters | |
336 | ||
337 | If the scope of the fault injection capability is limited to a | |
338 | single kernel module, it is better to provide module parameters to | |
339 | configure the fault attributes. | |
340 | ||
10ffebbe | 341 | - add a hook to insert failures |
de1ba09b | 342 | |
10ffebbe | 343 | Upon should_fail() returning true, client code should inject a failure: |
de1ba09b | 344 | |
5d0ffa2b | 345 | should_fail(attr, size); |
de1ba09b AM |
346 | |
347 | Application Examples | |
348 | -------------------- | |
349 | ||
10ffebbe | 350 | - Inject slab allocation failures into module init/exit code:: |
de1ba09b | 351 | |
10ffebbe | 352 | #!/bin/bash |
de1ba09b | 353 | |
10ffebbe MCC |
354 | FAILTYPE=failslab |
355 | echo Y > /sys/kernel/debug/$FAILTYPE/task-filter | |
356 | echo 10 > /sys/kernel/debug/$FAILTYPE/probability | |
357 | echo 100 > /sys/kernel/debug/$FAILTYPE/interval | |
d472cf79 | 358 | echo -1 > /sys/kernel/debug/$FAILTYPE/times |
10ffebbe MCC |
359 | echo 0 > /sys/kernel/debug/$FAILTYPE/space |
360 | echo 2 > /sys/kernel/debug/$FAILTYPE/verbose | |
bad3fbb2 | 361 | echo Y > /sys/kernel/debug/$FAILTYPE/ignore-gfp-wait |
de1ba09b | 362 | |
10ffebbe MCC |
363 | faulty_system() |
364 | { | |
18584870 | 365 | bash -c "echo 1 > /proc/self/make-it-fail && exec $*" |
10ffebbe | 366 | } |
de1ba09b | 367 | |
10ffebbe MCC |
368 | if [ $# -eq 0 ] |
369 | then | |
18584870 AM |
370 | echo "Usage: $0 modulename [ modulename ... ]" |
371 | exit 1 | |
10ffebbe | 372 | fi |
18584870 | 373 | |
10ffebbe MCC |
374 | for m in $* |
375 | do | |
18584870 AM |
376 | echo inserting $m... |
377 | faulty_system modprobe $m | |
de1ba09b | 378 | |
18584870 AM |
379 | echo removing $m... |
380 | faulty_system modprobe -r $m | |
10ffebbe | 381 | done |
de1ba09b AM |
382 | |
383 | ------------------------------------------------------------------------------ | |
384 | ||
10ffebbe | 385 | - Inject page allocation failures only for a specific module:: |
de1ba09b | 386 | |
10ffebbe | 387 | #!/bin/bash |
de1ba09b | 388 | |
10ffebbe MCC |
389 | FAILTYPE=fail_page_alloc |
390 | module=$1 | |
de1ba09b | 391 | |
10ffebbe MCC |
392 | if [ -z $module ] |
393 | then | |
18584870 AM |
394 | echo "Usage: $0 <modulename>" |
395 | exit 1 | |
10ffebbe | 396 | fi |
de1ba09b | 397 | |
10ffebbe | 398 | modprobe $module |
de1ba09b | 399 | |
10ffebbe MCC |
400 | if [ ! -d /sys/module/$module/sections ] |
401 | then | |
18584870 AM |
402 | echo Module $module is not loaded |
403 | exit 1 | |
10ffebbe | 404 | fi |
18584870 | 405 | |
10ffebbe MCC |
406 | cat /sys/module/$module/sections/.text > /sys/kernel/debug/$FAILTYPE/require-start |
407 | cat /sys/module/$module/sections/.data > /sys/kernel/debug/$FAILTYPE/require-end | |
18584870 | 408 | |
10ffebbe MCC |
409 | echo N > /sys/kernel/debug/$FAILTYPE/task-filter |
410 | echo 10 > /sys/kernel/debug/$FAILTYPE/probability | |
411 | echo 100 > /sys/kernel/debug/$FAILTYPE/interval | |
d472cf79 | 412 | echo -1 > /sys/kernel/debug/$FAILTYPE/times |
10ffebbe MCC |
413 | echo 0 > /sys/kernel/debug/$FAILTYPE/space |
414 | echo 2 > /sys/kernel/debug/$FAILTYPE/verbose | |
bad3fbb2 DY |
415 | echo Y > /sys/kernel/debug/$FAILTYPE/ignore-gfp-wait |
416 | echo Y > /sys/kernel/debug/$FAILTYPE/ignore-gfp-highmem | |
10ffebbe | 417 | echo 10 > /sys/kernel/debug/$FAILTYPE/stacktrace-depth |
18584870 | 418 | |
10ffebbe | 419 | trap "echo 0 > /sys/kernel/debug/$FAILTYPE/probability" SIGINT SIGTERM EXIT |
18584870 | 420 | |
10ffebbe MCC |
421 | echo "Injecting errors into the module $module... (interrupt to stop)" |
422 | sleep 1000000 | |
de1ba09b | 423 | |
4b1a29a7 MH |
424 | ------------------------------------------------------------------------------ |
425 | ||
10ffebbe MCC |
426 | - Inject open_ctree error while btrfs mount:: |
427 | ||
428 | #!/bin/bash | |
429 | ||
430 | rm -f testfile.img | |
431 | dd if=/dev/zero of=testfile.img bs=1M seek=1000 count=1 | |
432 | DEVICE=$(losetup --show -f testfile.img) | |
433 | mkfs.btrfs -f $DEVICE | |
434 | mkdir -p tmpmnt | |
435 | ||
436 | FAILTYPE=fail_function | |
437 | FAILFUNC=open_ctree | |
438 | echo $FAILFUNC > /sys/kernel/debug/$FAILTYPE/inject | |
00574752 | 439 | printf %#x -12 > /sys/kernel/debug/$FAILTYPE/$FAILFUNC/retval |
10ffebbe MCC |
440 | echo N > /sys/kernel/debug/$FAILTYPE/task-filter |
441 | echo 100 > /sys/kernel/debug/$FAILTYPE/probability | |
442 | echo 0 > /sys/kernel/debug/$FAILTYPE/interval | |
d472cf79 | 443 | echo -1 > /sys/kernel/debug/$FAILTYPE/times |
10ffebbe MCC |
444 | echo 0 > /sys/kernel/debug/$FAILTYPE/space |
445 | echo 1 > /sys/kernel/debug/$FAILTYPE/verbose | |
446 | ||
447 | mount -t btrfs $DEVICE tmpmnt | |
448 | if [ $? -ne 0 ] | |
449 | then | |
4b1a29a7 | 450 | echo "SUCCESS!" |
10ffebbe | 451 | else |
4b1a29a7 MH |
452 | echo "FAILED!" |
453 | umount tmpmnt | |
10ffebbe | 454 | fi |
4b1a29a7 | 455 | |
10ffebbe | 456 | echo > /sys/kernel/debug/$FAILTYPE/inject |
4b1a29a7 | 457 | |
10ffebbe MCC |
458 | rmdir tmpmnt |
459 | losetup -d $DEVICE | |
460 | rm testfile.img | |
4b1a29a7 MH |
461 | |
462 | ||
c24aa64d AM |
463 | Tool to run command with failslab or fail_page_alloc |
464 | ---------------------------------------------------- | |
465 | In order to make it easier to accomplish the tasks mentioned above, we can use | |
466 | tools/testing/fault-injection/failcmd.sh. Please run a command | |
467 | "./tools/testing/fault-injection/failcmd.sh --help" for more information and | |
468 | see the following examples. | |
469 | ||
470 | Examples: | |
471 | ||
472 | Run a command "make -C tools/testing/selftests/ run_tests" with injecting slab | |
10ffebbe | 473 | allocation failure:: |
c24aa64d AM |
474 | |
475 | # ./tools/testing/fault-injection/failcmd.sh \ | |
476 | -- make -C tools/testing/selftests/ run_tests | |
477 | ||
478 | Same as above except to specify 100 times failures at most instead of one time | |
10ffebbe | 479 | at most by default:: |
c24aa64d AM |
480 | |
481 | # ./tools/testing/fault-injection/failcmd.sh --times=100 \ | |
482 | -- make -C tools/testing/selftests/ run_tests | |
483 | ||
484 | Same as above except to inject page allocation failure instead of slab | |
10ffebbe | 485 | allocation failure:: |
c24aa64d AM |
486 | |
487 | # env FAILCMD_TYPE=fail_page_alloc \ | |
488 | ./tools/testing/fault-injection/failcmd.sh --times=100 \ | |
10ffebbe | 489 | -- make -C tools/testing/selftests/ run_tests |
e41d5818 DV |
490 | |
491 | Systematic faults using fail-nth | |
492 | --------------------------------- | |
493 | ||
494 | The following code systematically faults 0-th, 1-st, 2-nd and so on | |
10ffebbe MCC |
495 | capabilities in the socketpair() system call:: |
496 | ||
497 | #include <sys/types.h> | |
498 | #include <sys/stat.h> | |
499 | #include <sys/socket.h> | |
500 | #include <sys/syscall.h> | |
501 | #include <fcntl.h> | |
502 | #include <unistd.h> | |
503 | #include <string.h> | |
504 | #include <stdlib.h> | |
505 | #include <stdio.h> | |
506 | #include <errno.h> | |
507 | ||
508 | int main() | |
509 | { | |
e41d5818 DV |
510 | int i, err, res, fail_nth, fds[2]; |
511 | char buf[128]; | |
512 | ||
513 | system("echo N > /sys/kernel/debug/failslab/ignore-gfp-wait"); | |
514 | sprintf(buf, "/proc/self/task/%ld/fail-nth", syscall(SYS_gettid)); | |
515 | fail_nth = open(buf, O_RDWR); | |
9049f2f6 | 516 | for (i = 1;; i++) { |
e41d5818 DV |
517 | sprintf(buf, "%d", i); |
518 | write(fail_nth, buf, strlen(buf)); | |
519 | res = socketpair(AF_LOCAL, SOCK_STREAM, 0, fds); | |
520 | err = errno; | |
bfc74093 | 521 | pread(fail_nth, buf, sizeof(buf), 0); |
e41d5818 DV |
522 | if (res == 0) { |
523 | close(fds[0]); | |
524 | close(fds[1]); | |
525 | } | |
bfc74093 AM |
526 | printf("%d-th fault %c: res=%d/%d\n", i, atoi(buf) ? 'N' : 'Y', |
527 | res, err); | |
528 | if (atoi(buf)) | |
e41d5818 DV |
529 | break; |
530 | } | |
531 | return 0; | |
10ffebbe MCC |
532 | } |
533 | ||
534 | An example output:: | |
535 | ||
536 | 1-th fault Y: res=-1/23 | |
537 | 2-th fault Y: res=-1/23 | |
538 | 3-th fault Y: res=-1/12 | |
539 | 4-th fault Y: res=-1/12 | |
540 | 5-th fault Y: res=-1/23 | |
541 | 6-th fault Y: res=-1/23 | |
542 | 7-th fault Y: res=-1/23 | |
543 | 8-th fault Y: res=-1/12 | |
544 | 9-th fault Y: res=-1/12 | |
545 | 10-th fault Y: res=-1/12 | |
546 | 11-th fault Y: res=-1/12 | |
547 | 12-th fault Y: res=-1/12 | |
548 | 13-th fault Y: res=-1/12 | |
549 | 14-th fault Y: res=-1/12 | |
550 | 15-th fault Y: res=-1/12 | |
551 | 16-th fault N: res=0/12 |