Commit | Line | Data |
---|---|---|
0c14398b | 1 | ========================== |
35243421 | 2 | Short users guide for SLUB |
0c14398b | 3 | ========================== |
35243421 | 4 | |
35243421 CL |
5 | The basic philosophy of SLUB is very different from SLAB. SLAB |
6 | requires rebuilding the kernel to activate debug options for all | |
c1aee215 | 7 | slab caches. SLUB always includes full debugging but it is off by default. |
35243421 CL |
8 | SLUB can enable debugging only for selected slabs in order to avoid |
9 | an impact on overall system performance which may make a bug more | |
10 | difficult to find. | |
11 | ||
0c14398b | 12 | In order to switch debugging on one can add an option ``slub_debug`` |
35243421 CL |
13 | to the kernel command line. That will enable full debugging for |
14 | all slabs. | |
15 | ||
0c14398b MR |
16 | Typically one would then use the ``slabinfo`` command to get statistical |
17 | data and perform operation on the slabs. By default ``slabinfo`` only lists | |
35243421 | 18 | slabs that have data in them. See "slabinfo -h" for more options when |
0c14398b MR |
19 | running the command. ``slabinfo`` can be compiled with |
20 | :: | |
35243421 | 21 | |
799fb82a | 22 | gcc -o slabinfo tools/mm/slabinfo.c |
35243421 | 23 | |
0c14398b | 24 | Some of the modes of operation of ``slabinfo`` require that slub debugging |
35243421 CL |
25 | be enabled on the command line. F.e. no tracking information will be |
26 | available without debugging on and validation can only partially | |
27 | be performed if debugging was not switched on. | |
28 | ||
29 | Some more sophisticated uses of slub_debug: | |
30 | ------------------------------------------- | |
31 | ||
0c14398b | 32 | Parameters may be given to ``slub_debug``. If none is specified then full |
35243421 CL |
33 | debugging is enabled. Format: |
34 | ||
0c14398b MR |
35 | slub_debug=<Debug-Options> |
36 | Enable options for all slabs | |
0c14398b | 37 | |
c5fd3ca0 AT |
38 | slub_debug=<Debug-Options>,<slab name1>,<slab name2>,... |
39 | Enable options only for select slabs (no spaces | |
40 | after a comma) | |
0c14398b | 41 | |
e17f1dfb VB |
42 | Multiple blocks of options for all slabs or selected slabs can be given, with |
43 | blocks of options delimited by ';'. The last of "all slabs" blocks is applied | |
44 | to all slabs except those that match one of the "select slabs" block. Options | |
45 | of the first "select slabs" blocks that matches the slab's name are applied. | |
46 | ||
0c14398b | 47 | Possible debug options are:: |
35243421 | 48 | |
becfda68 LA |
49 | F Sanity checks on (enables SLAB_DEBUG_CONSISTENCY_CHECKS |
50 | Sorry SLAB legacy issues) | |
35243421 CL |
51 | Z Red zoning |
52 | P Poisoning (object and padding) | |
53 | U User tracking (free and alloc) | |
54 | T Trace (please only use on single slabs) | |
a3df6927 | 55 | A Enable failslab filter mark for the cache |
fa5ec8a1 DR |
56 | O Switch debugging off for caches that would have |
57 | caused higher minimum slab orders | |
f0630fff CL |
58 | - Switch all debugging off (useful if the kernel is |
59 | configured with CONFIG_SLUB_DEBUG_ON) | |
35243421 | 60 | |
0c14398b | 61 | F.e. in order to boot just with sanity checks and red zoning one would specify:: |
35243421 CL |
62 | |
63 | slub_debug=FZ | |
64 | ||
0c14398b | 65 | Trying to find an issue in the dentry cache? Try:: |
35243421 | 66 | |
989a7241 | 67 | slub_debug=,dentry |
35243421 | 68 | |
c5fd3ca0 AT |
69 | to only enable debugging on the dentry cache. You may use an asterisk at the |
70 | end of the slab name, in order to cover all slabs with the same prefix. For | |
71 | example, here's how you can poison the dentry cache as well as all kmalloc | |
11ede500 | 72 | slabs:: |
c5fd3ca0 AT |
73 | |
74 | slub_debug=P,kmalloc-*,dentry | |
35243421 CL |
75 | |
76 | Red zoning and tracking may realign the slab. We can just apply sanity checks | |
0c14398b | 77 | to the dentry cache with:: |
35243421 | 78 | |
989a7241 | 79 | slub_debug=F,dentry |
35243421 | 80 | |
fa5ec8a1 DR |
81 | Debugging options may require the minimum possible slab order to increase as |
82 | a result of storing the metadata (for example, caches with PAGE_SIZE object | |
83 | sizes). This has a higher liklihood of resulting in slab allocation errors | |
84 | in low memory situations or if there's high fragmentation of memory. To | |
0c14398b | 85 | switch off debugging for such caches by default, use:: |
fa5ec8a1 DR |
86 | |
87 | slub_debug=O | |
88 | ||
e17f1dfb VB |
89 | You can apply different options to different list of slab names, using blocks |
90 | of options. This will enable red zoning for dentry and user tracking for | |
91 | kmalloc. All other slabs will not get any debugging enabled:: | |
92 | ||
93 | slub_debug=Z,dentry;U,kmalloc-* | |
94 | ||
95 | You can also enable options (e.g. sanity checks and poisoning) for all caches | |
96 | except some that are deemed too performance critical and don't need to be | |
97 | debugged by specifying global debug options followed by a list of slab names | |
98 | with "-" as options:: | |
99 | ||
100 | slub_debug=FZ;-,zs_handle,zspage | |
101 | ||
ad38b5b1 VB |
102 | The state of each debug option for a slab can be found in the respective files |
103 | under:: | |
35243421 | 104 | |
0c14398b | 105 | /sys/kernel/slab/<slab name>/ |
35243421 | 106 | |
ad38b5b1 VB |
107 | If the file contains 1, the option is enabled, 0 means disabled. The debug |
108 | options from the ``slub_debug`` parameter translate to the following files:: | |
109 | ||
110 | F sanity_checks | |
111 | Z red_zone | |
112 | P poison | |
113 | U store_user | |
114 | T trace | |
115 | A failslab | |
116 | ||
7c82b3b3 AA |
117 | failslab file is writable, so writing 1 or 0 will enable or disable |
118 | the option at runtime. Write returns -EINVAL if cache is an alias. | |
060807f8 VB |
119 | Careful with tracing: It may spew out lots of information and never stop if |
120 | used on the wrong slab. | |
35243421 | 121 | |
c1aee215 | 122 | Slab merging |
0c14398b | 123 | ============ |
35243421 | 124 | |
c1aee215 | 125 | If no debug options are specified then SLUB may merge similar slabs together |
35243421 | 126 | in order to reduce overhead and increase cache hotness of objects. |
0c14398b | 127 | ``slabinfo -a`` displays which slabs were merged together. |
35243421 | 128 | |
c1aee215 | 129 | Slab validation |
0c14398b | 130 | =============== |
c1aee215 CL |
131 | |
132 | SLUB can validate all object if the kernel was booted with slub_debug. In | |
0c14398b MR |
133 | order to do so you must have the ``slabinfo`` tool. Then you can do |
134 | :: | |
c1aee215 | 135 | |
0c14398b | 136 | slabinfo -v |
c1aee215 CL |
137 | |
138 | which will test all objects. Output will be generated to the syslog. | |
139 | ||
140 | This also works in a more limited way if boot was without slab debug. | |
0c14398b | 141 | In that case ``slabinfo -v`` simply tests all reachable objects. Usually |
c1aee215 CL |
142 | these are in the cpu slabs and the partial slabs. Full slabs are not |
143 | tracked by SLUB in a non debug situation. | |
144 | ||
35243421 | 145 | Getting more performance |
0c14398b | 146 | ======================== |
35243421 CL |
147 | |
148 | To some degree SLUB's performance is limited by the need to take the | |
149 | list_lock once in a while to deal with partial slabs. That overhead is | |
150 | governed by the order of the allocation for each slab. The allocations | |
151 | can be influenced by kernel parameters: | |
152 | ||
0c14398b MR |
153 | .. slub_min_objects=x (default 4) |
154 | .. slub_min_order=x (default 0) | |
155 | .. slub_max_order=x (default 3 (PAGE_ALLOC_COSTLY_ORDER)) | |
156 | ||
157 | ``slub_min_objects`` | |
158 | allows to specify how many objects must at least fit into one | |
159 | slab in order for the allocation order to be acceptable. In | |
160 | general slub will be able to perform this number of | |
161 | allocations on a slab without consulting centralized resources | |
162 | (list_lock) where contention may occur. | |
163 | ||
164 | ``slub_min_order`` | |
358b6ba9 | 165 | specifies a minimum order of slabs. A similar effect like |
0c14398b MR |
166 | ``slub_min_objects``. |
167 | ||
168 | ``slub_max_order`` | |
169 | specified the order at which ``slub_min_objects`` should no | |
170 | longer be checked. This is useful to avoid SLUB trying to | |
171 | generate super large order pages to fit ``slub_min_objects`` | |
172 | of a slab cache with large object sizes into one high order | |
173 | page. Setting command line parameter | |
174 | ``debug_guardpage_minorder=N`` (N > 0), forces setting | |
175 | ``slub_max_order`` to 0, what cause minimum possible order of | |
176 | slabs allocation. | |
35243421 | 177 | |
c1aee215 | 178 | SLUB Debug output |
0c14398b MR |
179 | ================= |
180 | ||
181 | Here is a sample of slub debug output:: | |
182 | ||
183 | ==================================================================== | |
8669dbab | 184 | BUG kmalloc-8: Right Redzone overwritten |
0c14398b MR |
185 | -------------------------------------------------------------------- |
186 | ||
187 | INFO: 0xc90f6d28-0xc90f6d2b. First byte 0x00 instead of 0xcc | |
188 | INFO: Slab 0xc528c530 flags=0x400000c3 inuse=61 fp=0xc90f6d58 | |
189 | INFO: Object 0xc90f6d20 @offset=3360 fp=0xc90f6d58 | |
190 | INFO: Allocated in get_modalias+0x61/0xf5 age=53 cpu=1 pid=554 | |
191 | ||
8669dbab KC |
192 | Bytes b4 (0xc90f6d10): 00 00 00 00 00 00 00 00 5a 5a 5a 5a 5a 5a 5a 5a ........ZZZZZZZZ |
193 | Object (0xc90f6d20): 31 30 31 39 2e 30 30 35 1019.005 | |
194 | Redzone (0xc90f6d28): 00 cc cc cc . | |
195 | Padding (0xc90f6d50): 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZ | |
0c14398b MR |
196 | |
197 | [<c010523d>] dump_trace+0x63/0x1eb | |
198 | [<c01053df>] show_trace_log_lvl+0x1a/0x2f | |
199 | [<c010601d>] show_trace+0x12/0x14 | |
200 | [<c0106035>] dump_stack+0x16/0x18 | |
201 | [<c017e0fa>] object_err+0x143/0x14b | |
202 | [<c017e2cc>] check_object+0x66/0x234 | |
203 | [<c017eb43>] __slab_free+0x239/0x384 | |
204 | [<c017f446>] kfree+0xa6/0xc6 | |
205 | [<c02e2335>] get_modalias+0xb9/0xf5 | |
206 | [<c02e23b7>] dmi_dev_uevent+0x27/0x3c | |
207 | [<c027866a>] dev_uevent+0x1ad/0x1da | |
208 | [<c0205024>] kobject_uevent_env+0x20a/0x45b | |
209 | [<c020527f>] kobject_uevent+0xa/0xf | |
210 | [<c02779f1>] store_uevent+0x4f/0x58 | |
211 | [<c027758e>] dev_attr_store+0x29/0x2f | |
212 | [<c01bec4f>] sysfs_write_file+0x16e/0x19c | |
213 | [<c0183ba7>] vfs_write+0xd1/0x15a | |
214 | [<c01841d7>] sys_write+0x3d/0x72 | |
215 | [<c0104112>] sysenter_past_esp+0x5f/0x99 | |
216 | [<b7f7b410>] 0xb7f7b410 | |
217 | ======================= | |
218 | ||
219 | FIX kmalloc-8: Restoring Redzone 0xc90f6d28-0xc90f6d2b=0xcc | |
c1aee215 | 220 | |
24922684 CL |
221 | If SLUB encounters a corrupted object (full detection requires the kernel |
222 | to be booted with slub_debug) then the following output will be dumped | |
223 | into the syslog: | |
c1aee215 | 224 | |
24922684 | 225 | 1. Description of the problem encountered |
c1aee215 | 226 | |
0c14398b | 227 | This will be a message in the system log starting with:: |
c1aee215 | 228 | |
0c14398b MR |
229 | =============================================== |
230 | BUG <slab cache affected>: <What went wrong> | |
231 | ----------------------------------------------- | |
c1aee215 | 232 | |
0c14398b MR |
233 | INFO: <corruption start>-<corruption_end> <more info> |
234 | INFO: Slab <address> <slab information> | |
235 | INFO: Object <address> <object information> | |
236 | INFO: Allocated in <kernel function> age=<jiffies since alloc> cpu=<allocated by | |
24922684 | 237 | cpu> pid=<pid of the process> |
0c14398b MR |
238 | INFO: Freed in <kernel function> age=<jiffies since free> cpu=<freed by cpu> |
239 | pid=<pid of the process> | |
c1aee215 | 240 | |
0c14398b MR |
241 | (Object allocation / free information is only available if SLAB_STORE_USER is |
242 | set for the slab. slub_debug sets that option) | |
c1aee215 | 243 | |
24922684 | 244 | 2. The object contents if an object was involved. |
c1aee215 | 245 | |
0c14398b | 246 | Various types of lines can follow the BUG SLUB line: |
c1aee215 | 247 | |
0c14398b | 248 | Bytes b4 <address> : <bytes> |
24922684 | 249 | Shows a few bytes before the object where the problem was detected. |
c1aee215 CL |
250 | Can be useful if the corruption does not stop with the start of the |
251 | object. | |
252 | ||
0c14398b | 253 | Object <address> : <bytes> |
c1aee215 | 254 | The bytes of the object. If the object is inactive then the bytes |
24922684 | 255 | typically contain poison values. Any non-poison value shows a |
c1aee215 CL |
256 | corruption by a write after free. |
257 | ||
0c14398b | 258 | Redzone <address> : <bytes> |
24922684 | 259 | The Redzone following the object. The Redzone is used to detect |
c1aee215 CL |
260 | writes after the object. All bytes should always have the same |
261 | value. If there is any deviation then it is due to a write after | |
262 | the object boundary. | |
263 | ||
24922684 CL |
264 | (Redzone information is only available if SLAB_RED_ZONE is set. |
265 | slub_debug sets that option) | |
c1aee215 | 266 | |
0c14398b | 267 | Padding <address> : <bytes> |
c1aee215 CL |
268 | Unused data to fill up the space in order to get the next object |
269 | properly aligned. In the debug case we make sure that there are | |
24922684 | 270 | at least 4 bytes of padding. This allows the detection of writes |
c1aee215 CL |
271 | before the object. |
272 | ||
24922684 CL |
273 | 3. A stackdump |
274 | ||
0c14398b MR |
275 | The stackdump describes the location where the error was detected. The cause |
276 | of the corruption is may be more likely found by looking at the function that | |
277 | allocated or freed the object. | |
24922684 CL |
278 | |
279 | 4. Report on how the problem was dealt with in order to ensure the continued | |
0c14398b | 280 | operation of the system. |
24922684 | 281 | |
0c14398b | 282 | These are messages in the system log beginning with:: |
24922684 | 283 | |
0c14398b | 284 | FIX <slab cache affected>: <corrective action taken> |
24922684 | 285 | |
0c14398b MR |
286 | In the above sample SLUB found that the Redzone of an active object has |
287 | been overwritten. Here a string of 8 characters was written into a slab that | |
288 | has the length of 8 characters. However, a 8 character string needs a | |
289 | terminating 0. That zero has overwritten the first byte of the Redzone field. | |
290 | After reporting the details of the issue encountered the FIX SLUB message | |
291 | tells us that SLUB has restored the Redzone to its proper value and then | |
292 | system operations continue. | |
24922684 | 293 | |
0c14398b MR |
294 | Emergency operations |
295 | ==================== | |
24922684 | 296 | |
0c14398b | 297 | Minimal debugging (sanity checks alone) can be enabled by booting with:: |
24922684 CL |
298 | |
299 | slub_debug=F | |
300 | ||
301 | This will be generally be enough to enable the resiliency features of slub | |
302 | which will keep the system running even if a bad kernel component will | |
303 | keep corrupting objects. This may be important for production systems. | |
304 | Performance will be impacted by the sanity checks and there will be a | |
305 | continual stream of error messages to the syslog but no additional memory | |
306 | will be used (unlike full debugging). | |
307 | ||
308 | No guarantees. The kernel component still needs to be fixed. Performance | |
309 | may be optimized further by locating the slab that experiences corruption | |
310 | and enabling debugging only for that cache | |
311 | ||
0c14398b | 312 | I.e.:: |
24922684 CL |
313 | |
314 | slub_debug=F,dentry | |
315 | ||
316 | If the corruption occurs by writing after the end of the object then it | |
317 | may be advisable to enable a Redzone to avoid corrupting the beginning | |
0c14398b | 318 | of other objects:: |
24922684 CL |
319 | |
320 | slub_debug=FZ,dentry | |
c1aee215 | 321 | |
05be9617 | 322 | Extended slabinfo mode and plotting |
0c14398b | 323 | =================================== |
05be9617 | 324 | |
0c14398b | 325 | The ``slabinfo`` tool has a special 'extended' ('-X') mode that includes: |
05be9617 SS |
326 | - Slabcache Totals |
327 | - Slabs sorted by size (up to -N <num> slabs, default 1) | |
328 | - Slabs sorted by loss (up to -N <num> slabs, default 1) | |
329 | ||
0c14398b MR |
330 | Additionally, in this mode ``slabinfo`` does not dynamically scale |
331 | sizes (G/M/K) and reports everything in bytes (this functionality is | |
332 | also available to other slabinfo modes via '-B' option) which makes | |
333 | reporting more precise and accurate. Moreover, in some sense the `-X' | |
334 | mode also simplifies the analysis of slabs' behaviour, because its | |
335 | output can be plotted using the ``slabinfo-gnuplot.sh`` script. So it | |
336 | pushes the analysis from looking through the numbers (tons of numbers) | |
337 | to something easier -- visual analysis. | |
05be9617 SS |
338 | |
339 | To generate plots: | |
0c14398b MR |
340 | |
341 | a) collect slabinfo extended records, for example:: | |
342 | ||
343 | while [ 1 ]; do slabinfo -X >> FOO_STATS; sleep 1; done | |
344 | ||
345 | b) pass stats file(-s) to ``slabinfo-gnuplot.sh`` script:: | |
346 | ||
347 | slabinfo-gnuplot.sh FOO_STATS [FOO_STATS2 .. FOO_STATSN] | |
348 | ||
349 | The ``slabinfo-gnuplot.sh`` script will pre-processes the collected records | |
350 | and generates 3 png files (and 3 pre-processing cache files) per STATS | |
351 | file: | |
352 | - Slabcache Totals: FOO_STATS-totals.png | |
353 | - Slabs sorted by size: FOO_STATS-slabs-by-size.png | |
354 | - Slabs sorted by loss: FOO_STATS-slabs-by-loss.png | |
355 | ||
356 | Another use case, when ``slabinfo-gnuplot.sh`` can be useful, is when you | |
357 | need to compare slabs' behaviour "prior to" and "after" some code | |
358 | modification. To help you out there, ``slabinfo-gnuplot.sh`` script | |
359 | can 'merge' the `Slabcache Totals` sections from different | |
360 | measurements. To visually compare N plots: | |
361 | ||
362 | a) Collect as many STATS1, STATS2, .. STATSN files as you need:: | |
363 | ||
364 | while [ 1 ]; do slabinfo -X >> STATS<X>; sleep 1; done | |
365 | ||
366 | b) Pre-process those STATS files:: | |
367 | ||
368 | slabinfo-gnuplot.sh STATS1 STATS2 .. STATSN | |
369 | ||
370 | c) Execute ``slabinfo-gnuplot.sh`` in '-t' mode, passing all of the | |
371 | generated pre-processed \*-totals:: | |
372 | ||
373 | slabinfo-gnuplot.sh -t STATS1-totals STATS2-totals .. STATSN-totals | |
374 | ||
375 | This will produce a single plot (png file). | |
376 | ||
377 | Plots, expectedly, can be large so some fluctuations or small spikes | |
378 | can go unnoticed. To deal with that, ``slabinfo-gnuplot.sh`` has two | |
379 | options to 'zoom-in'/'zoom-out': | |
380 | ||
94ebdd28 | 381 | a) ``-s %d,%d`` -- overwrites the default image width and height |
0c14398b MR |
382 | b) ``-r %d,%d`` -- specifies a range of samples to use (for example, |
383 | in ``slabinfo -X >> FOO_STATS; sleep 1;`` case, using a ``-r | |
384 | 40,60`` range will plot only samples collected between 40th and | |
385 | 60th seconds). | |
05be9617 | 386 | |
9f04b55f OG |
387 | |
388 | DebugFS files for SLUB | |
389 | ====================== | |
390 | ||
391 | For more information about current state of SLUB caches with the user tracking | |
392 | debug option enabled, debugfs files are available, typically under | |
393 | /sys/kernel/debug/slab/<cache>/ (created only for caches with enabled user | |
394 | tracking). There are 2 types of these files with the following debug | |
395 | information: | |
396 | ||
397 | 1. alloc_traces:: | |
398 | ||
399 | Prints information about unique allocation traces of the currently | |
400 | allocated objects. The output is sorted by frequency of each trace. | |
401 | ||
402 | Information in the output: | |
6edf2576 FT |
403 | Number of objects, allocating function, possible memory wastage of |
404 | kmalloc objects(total/per-object), minimal/average/maximal jiffies | |
405 | since alloc, pid range of the allocating processes, cpu mask of | |
406 | allocating cpus, numa node mask of origins of memory, and stack trace. | |
9f04b55f OG |
407 | |
408 | Example::: | |
409 | ||
6edf2576 FT |
410 | 338 pci_alloc_dev+0x2c/0xa0 waste=521872/1544 age=290837/291891/293509 pid=1 cpus=106 nodes=0-1 |
411 | __kmem_cache_alloc_node+0x11f/0x4e0 | |
412 | kmalloc_trace+0x26/0xa0 | |
413 | pci_alloc_dev+0x2c/0xa0 | |
414 | pci_scan_single_device+0xd2/0x150 | |
415 | pci_scan_slot+0xf7/0x2d0 | |
416 | pci_scan_child_bus_extend+0x4e/0x360 | |
417 | acpi_pci_root_create+0x32e/0x3b0 | |
418 | pci_acpi_scan_root+0x2b9/0x2d0 | |
419 | acpi_pci_root_add.cold.11+0x110/0xb0a | |
420 | acpi_bus_attach+0x262/0x3f0 | |
421 | device_for_each_child+0xb7/0x110 | |
422 | acpi_dev_for_each_child+0x77/0xa0 | |
423 | acpi_bus_attach+0x108/0x3f0 | |
424 | device_for_each_child+0xb7/0x110 | |
425 | acpi_dev_for_each_child+0x77/0xa0 | |
426 | acpi_bus_attach+0x108/0x3f0 | |
9f04b55f OG |
427 | |
428 | 2. free_traces:: | |
429 | ||
430 | Prints information about unique freeing traces of the currently allocated | |
431 | objects. The freeing traces thus come from the previous life-cycle of the | |
432 | objects and are reported as not available for objects allocated for the first | |
433 | time. The output is sorted by frequency of each trace. | |
434 | ||
435 | Information in the output: | |
436 | Number of objects, freeing function, minimal/average/maximal jiffies since free, | |
437 | pid range of the freeing processes, cpu mask of freeing cpus, and stack trace. | |
438 | ||
439 | Example::: | |
440 | ||
441 | 1980 <not-available> age=4294912290 pid=0 cpus=0 | |
442 | 51 acpi_ut_update_ref_count+0x6a6/0x782 age=236886/237027/237772 pid=1 cpus=1 | |
443 | kfree+0x2db/0x420 | |
444 | acpi_ut_update_ref_count+0x6a6/0x782 | |
445 | acpi_ut_update_object_reference+0x1ad/0x234 | |
446 | acpi_ut_remove_reference+0x7d/0x84 | |
447 | acpi_rs_get_prt_method_data+0x97/0xd6 | |
448 | acpi_get_irq_routing_table+0x82/0xc4 | |
449 | acpi_pci_irq_find_prt_entry+0x8e/0x2e0 | |
450 | acpi_pci_irq_lookup+0x3a/0x1e0 | |
451 | acpi_pci_irq_enable+0x77/0x240 | |
452 | pcibios_enable_device+0x39/0x40 | |
453 | do_pci_enable_device.part.0+0x5d/0xe0 | |
454 | pci_enable_device_flags+0xfc/0x120 | |
455 | pci_enable_device+0x13/0x20 | |
456 | virtio_pci_probe+0x9e/0x170 | |
457 | local_pci_probe+0x48/0x80 | |
458 | pci_device_probe+0x105/0x1c0 | |
459 | ||
cde53535 | 460 | Christoph Lameter, May 30, 2007 |
05be9617 | 461 | Sergey Senozhatsky, October 23, 2015 |