Commit | Line | Data |
---|---|---|
73d98127 CD |
1 | ============= |
2 | Event Tracing | |
3 | ============= | |
abd41443 | 4 | |
73d98127 CD |
5 | :Author: Theodore Ts'o |
6 | :Updated: Li Zefan and Tom Zanussi | |
abd41443 | 7 | |
143c145e LZ |
8 | 1. Introduction |
9 | =============== | |
abd41443 | 10 | |
ec15872d | 11 | Tracepoints (see Documentation/trace/tracepoints.rst) can be used |
abd41443 TT |
12 | without creating custom kernel modules to register probe functions |
13 | using the event tracing infrastructure. | |
14 | ||
15 | Not all tracepoints can be traced using the event tracing system; | |
16 | the kernel developer must provide code snippets which define how the | |
17 | tracing information is saved into the tracing buffer, and how the | |
143c145e | 18 | tracing information should be printed. |
abd41443 | 19 | |
143c145e LZ |
20 | 2. Using Event Tracing |
21 | ====================== | |
22 | ||
23 | 2.1 Via the 'set_event' interface | |
24 | --------------------------------- | |
abd41443 TT |
25 | |
26 | The events which are available for tracing can be found in the file | |
52ad51e7 | 27 | /sys/kernel/debug/tracing/available_events. |
abd41443 TT |
28 | |
29 | To enable a particular event, such as 'sched_wakeup', simply echo it | |
73d98127 | 30 | to /sys/kernel/debug/tracing/set_event. For example:: |
abd41443 | 31 | |
52ad51e7 | 32 | # echo sched_wakeup >> /sys/kernel/debug/tracing/set_event |
abd41443 | 33 | |
73d98127 | 34 | .. Note:: '>>' is necessary, otherwise it will firstly disable all the events. |
abd41443 TT |
35 | |
36 | To disable an event, echo the event name to the set_event file prefixed | |
73d98127 | 37 | with an exclamation point:: |
abd41443 | 38 | |
52ad51e7 | 39 | # echo '!sched_wakeup' >> /sys/kernel/debug/tracing/set_event |
143c145e | 40 | |
73d98127 | 41 | To disable all events, echo an empty line to the set_event file:: |
143c145e | 42 | |
52ad51e7 | 43 | # echo > /sys/kernel/debug/tracing/set_event |
abd41443 | 44 | |
6234c7bd | 45 | To enable all events, echo ``*:*`` or ``*:`` to the set_event file:: |
abd41443 | 46 | |
52ad51e7 | 47 | # echo *:* > /sys/kernel/debug/tracing/set_event |
abd41443 TT |
48 | |
49 | The events are organized into subsystems, such as ext4, irq, sched, | |
50 | etc., and a full event name looks like this: <subsystem>:<event>. The | |
51 | subsystem name is optional, but it is displayed in the available_events | |
52 | file. All of the events in a subsystem can be specified via the syntax | |
6234c7bd | 53 | ``<subsystem>:*``; for example, to enable all irq events, you can use the |
73d98127 | 54 | command:: |
abd41443 | 55 | |
52ad51e7 | 56 | # echo 'irq:*' > /sys/kernel/debug/tracing/set_event |
143c145e LZ |
57 | |
58 | 2.2 Via the 'enable' toggle | |
59 | --------------------------- | |
60 | ||
52ad51e7 | 61 | The events available are also listed in /sys/kernel/debug/tracing/events/ hierarchy |
143c145e LZ |
62 | of directories. |
63 | ||
73d98127 | 64 | To enable event 'sched_wakeup':: |
143c145e | 65 | |
52ad51e7 | 66 | # echo 1 > /sys/kernel/debug/tracing/events/sched/sched_wakeup/enable |
143c145e | 67 | |
73d98127 | 68 | To disable it:: |
143c145e | 69 | |
52ad51e7 | 70 | # echo 0 > /sys/kernel/debug/tracing/events/sched/sched_wakeup/enable |
143c145e | 71 | |
73d98127 | 72 | To enable all events in sched subsystem:: |
143c145e | 73 | |
52ad51e7 | 74 | # echo 1 > /sys/kernel/debug/tracing/events/sched/enable |
143c145e | 75 | |
73d98127 | 76 | To enable all events:: |
143c145e | 77 | |
52ad51e7 | 78 | # echo 1 > /sys/kernel/debug/tracing/events/enable |
143c145e LZ |
79 | |
80 | When reading one of these enable files, there are four results: | |
81 | ||
73d98127 CD |
82 | - 0 - all events this file affects are disabled |
83 | - 1 - all events this file affects are enabled | |
84 | - X - there is a mixture of events enabled and disabled | |
85 | - ? - this file does not affect any event | |
143c145e | 86 | |
020e5f85 LZ |
87 | 2.3 Boot option |
88 | --------------- | |
89 | ||
73d98127 | 90 | In order to facilitate early boot debugging, use boot option:: |
020e5f85 LZ |
91 | |
92 | trace_event=[event-list] | |
93 | ||
03d646e6 LZ |
94 | event-list is a comma separated list of events. See section 2.1 for event |
95 | format. | |
020e5f85 | 96 | |
143c145e LZ |
97 | 3. Defining an event-enabled tracepoint |
98 | ======================================= | |
99 | ||
100 | See The example provided in samples/trace_events | |
101 | ||
95b69608 TZ |
102 | 4. Event formats |
103 | ================ | |
104 | ||
105 | Each trace event has a 'format' file associated with it that contains | |
106 | a description of each field in a logged event. This information can | |
107 | be used to parse the binary trace stream, and is also the place to | |
108 | find the field names that can be used in event filters (see section 5). | |
109 | ||
110 | It also displays the format string that will be used to print the | |
111 | event in text mode, along with the event name and ID used for | |
112 | profiling. | |
113 | ||
6234c7bd JC |
114 | Every event has a set of ``common`` fields associated with it; these are |
115 | the fields prefixed with ``common_``. The other fields vary between | |
95b69608 TZ |
116 | events and correspond to the fields defined in the TRACE_EVENT |
117 | definition for that event. | |
118 | ||
73d98127 | 119 | Each field in the format has the form:: |
95b69608 TZ |
120 | |
121 | field:field-type field-name; offset:N; size:N; | |
122 | ||
123 | where offset is the offset of the field in the trace record and size | |
124 | is the size of the data item, in bytes. | |
125 | ||
126 | For example, here's the information displayed for the 'sched_wakeup' | |
73d98127 | 127 | event:: |
95b69608 | 128 | |
73d98127 | 129 | # cat /sys/kernel/debug/tracing/events/sched/sched_wakeup/format |
95b69608 | 130 | |
73d98127 CD |
131 | name: sched_wakeup |
132 | ID: 60 | |
133 | format: | |
134 | field:unsigned short common_type; offset:0; size:2; | |
135 | field:unsigned char common_flags; offset:2; size:1; | |
136 | field:unsigned char common_preempt_count; offset:3; size:1; | |
137 | field:int common_pid; offset:4; size:4; | |
138 | field:int common_tgid; offset:8; size:4; | |
95b69608 | 139 | |
73d98127 CD |
140 | field:char comm[TASK_COMM_LEN]; offset:12; size:16; |
141 | field:pid_t pid; offset:28; size:4; | |
142 | field:int prio; offset:32; size:4; | |
143 | field:int success; offset:36; size:4; | |
144 | field:int cpu; offset:40; size:4; | |
95b69608 | 145 | |
73d98127 CD |
146 | print fmt: "task %s:%d [%d] success=%d [%03d]", REC->comm, REC->pid, |
147 | REC->prio, REC->success, REC->cpu | |
95b69608 TZ |
148 | |
149 | This event contains 10 fields, the first 5 common and the remaining 5 | |
150 | event-specific. All the fields for this event are numeric, except for | |
151 | 'comm' which is a string, a distinction important for event filtering. | |
152 | ||
153 | 5. Event filtering | |
154 | ================== | |
155 | ||
156 | Trace events can be filtered in the kernel by associating boolean | |
157 | 'filter expressions' with them. As soon as an event is logged into | |
158 | the trace buffer, its fields are checked against the filter expression | |
159 | associated with that event type. An event with field values that | |
160 | 'match' the filter will appear in the trace output, and an event whose | |
161 | values don't match will be discarded. An event with no filter | |
162 | associated with it matches everything, and is the default when no | |
163 | filter has been set for an event. | |
164 | ||
165 | 5.1 Expression syntax | |
166 | --------------------- | |
167 | ||
168 | A filter expression consists of one or more 'predicates' that can be | |
169 | combined using the logical operators '&&' and '||'. A predicate is | |
170 | simply a clause that compares the value of a field contained within a | |
171 | logged event with a constant value and returns either 0 or 1 depending | |
73d98127 | 172 | on whether the field value matched (1) or didn't match (0):: |
95b69608 TZ |
173 | |
174 | field-name relational-operator value | |
175 | ||
176 | Parentheses can be used to provide arbitrary logical groupings and | |
177 | double-quotes can be used to prevent the shell from interpreting | |
178 | operators as shell metacharacters. | |
179 | ||
180 | The field-names available for use in filters can be found in the | |
181 | 'format' files for trace events (see section 4). | |
182 | ||
183 | The relational-operators depend on the type of the field being tested: | |
184 | ||
185 | The operators available for numeric fields are: | |
186 | ||
1a891cf1 | 187 | ==, !=, <, <=, >, >=, & |
95b69608 TZ |
188 | |
189 | And for string fields they are: | |
190 | ||
c3e13c7c | 191 | ==, !=, ~ |
95b69608 | 192 | |
6234c7bd | 193 | The glob (~) accepts a wild card character (\*,?) and character classes |
73d98127 | 194 | ([). For example:: |
c3e13c7c SRRH |
195 | |
196 | prev_comm ~ "*sh" | |
197 | prev_comm ~ "sh*" | |
198 | prev_comm ~ "*sh*" | |
60f1d5e3 | 199 | prev_comm ~ "ba*sh" |
95b69608 | 200 | |
95b69608 TZ |
201 | 5.2 Setting filters |
202 | ------------------- | |
203 | ||
204 | A filter for an individual event is set by writing a filter expression | |
205 | to the 'filter' file for the given event. | |
206 | ||
73d98127 | 207 | For example:: |
95b69608 | 208 | |
73d98127 CD |
209 | # cd /sys/kernel/debug/tracing/events/sched/sched_wakeup |
210 | # echo "common_preempt_count > 4" > filter | |
95b69608 | 211 | |
73d98127 | 212 | A slightly more involved example:: |
95b69608 | 213 | |
73d98127 CD |
214 | # cd /sys/kernel/debug/tracing/events/signal/signal_generate |
215 | # echo "((sig >= 10 && sig < 15) || sig == 17) && comm != bash" > filter | |
95b69608 TZ |
216 | |
217 | If there is an error in the expression, you'll get an 'Invalid | |
218 | argument' error when setting it, and the erroneous string along with | |
73d98127 | 219 | an error message can be seen by looking at the filter e.g.:: |
95b69608 | 220 | |
73d98127 CD |
221 | # cd /sys/kernel/debug/tracing/events/signal/signal_generate |
222 | # echo "((sig >= 10 && sig < 15) || dsig == 17) && comm != bash" > filter | |
223 | -bash: echo: write error: Invalid argument | |
224 | # cat filter | |
225 | ((sig >= 10 && sig < 15) || dsig == 17) && comm != bash | |
226 | ^ | |
227 | parse_error: Field not found | |
95b69608 TZ |
228 | |
229 | Currently the caret ('^') for an error always appears at the beginning of | |
230 | the filter string; the error message should still be useful though | |
231 | even without more accurate position info. | |
232 | ||
233 | 5.3 Clearing filters | |
234 | -------------------- | |
235 | ||
236 | To clear the filter for an event, write a '0' to the event's filter | |
237 | file. | |
238 | ||
239 | To clear the filters for all events in a subsystem, write a '0' to the | |
240 | subsystem's filter file. | |
241 | ||
242 | 5.3 Subsystem filters | |
243 | --------------------- | |
244 | ||
245 | For convenience, filters for every event in a subsystem can be set or | |
246 | cleared as a group by writing a filter expression into the filter file | |
88393161 | 247 | at the root of the subsystem. Note however, that if a filter for any |
95b69608 TZ |
248 | event within the subsystem lacks a field specified in the subsystem |
249 | filter, or if the filter can't be applied for any other reason, the | |
250 | filter for that event will retain its previous setting. This can | |
251 | result in an unintended mixture of filters which could lead to | |
252 | confusing (to the user who might think different filters are in | |
253 | effect) trace output. Only filters that reference just the common | |
254 | fields can be guaranteed to propagate successfully to all events. | |
255 | ||
256 | Here are a few subsystem filter examples that also illustrate the | |
257 | above points: | |
258 | ||
73d98127 | 259 | Clear the filters on all events in the sched subsystem:: |
95b69608 | 260 | |
73d98127 CD |
261 | # cd /sys/kernel/debug/tracing/events/sched |
262 | # echo 0 > filter | |
263 | # cat sched_switch/filter | |
264 | none | |
265 | # cat sched_wakeup/filter | |
266 | none | |
95b69608 TZ |
267 | |
268 | Set a filter using only common fields for all events in the sched | |
73d98127 | 269 | subsystem (all events end up with the same filter):: |
95b69608 | 270 | |
73d98127 CD |
271 | # cd /sys/kernel/debug/tracing/events/sched |
272 | # echo common_pid == 0 > filter | |
273 | # cat sched_switch/filter | |
274 | common_pid == 0 | |
275 | # cat sched_wakeup/filter | |
276 | common_pid == 0 | |
95b69608 TZ |
277 | |
278 | Attempt to set a filter using a non-common field for all events in the | |
88393161 | 279 | sched subsystem (all events but those that have a prev_pid field retain |
73d98127 | 280 | their old filters):: |
95b69608 | 281 | |
73d98127 CD |
282 | # cd /sys/kernel/debug/tracing/events/sched |
283 | # echo prev_pid == 0 > filter | |
284 | # cat sched_switch/filter | |
285 | prev_pid == 0 | |
286 | # cat sched_wakeup/filter | |
287 | common_pid == 0 | |
ac38fb85 | 288 | |
627645fd SRRH |
289 | 5.4 PID filtering |
290 | ----------------- | |
291 | ||
292 | The set_event_pid file in the same directory as the top events directory | |
293 | exists, will filter all events from tracing any task that does not have the | |
294 | PID listed in the set_event_pid file. | |
73d98127 | 295 | :: |
627645fd | 296 | |
73d98127 CD |
297 | # cd /sys/kernel/debug/tracing |
298 | # echo $$ > set_event_pid | |
299 | # echo 1 > events/enable | |
627645fd SRRH |
300 | |
301 | Will only trace events for the current task. | |
302 | ||
303 | To add more PIDs without losing the PIDs already included, use '>>'. | |
73d98127 | 304 | :: |
627645fd | 305 | |
73d98127 | 306 | # echo 123 244 1 >> set_event_pid |
627645fd SRRH |
307 | |
308 | ||
ac38fb85 TZ |
309 | 6. Event triggers |
310 | ================= | |
311 | ||
312 | Trace events can be made to conditionally invoke trigger 'commands' | |
313 | which can take various forms and are described in detail below; | |
314 | examples would be enabling or disabling other trace events or invoking | |
315 | a stack trace whenever the trace event is hit. Whenever a trace event | |
316 | with attached triggers is invoked, the set of trigger commands | |
317 | associated with that event is invoked. Any given trigger can | |
318 | additionally have an event filter of the same form as described in | |
319 | section 5 (Event filtering) associated with it - the command will only | |
320 | be invoked if the event being invoked passes the associated filter. | |
321 | If no filter is associated with the trigger, it always passes. | |
322 | ||
323 | Triggers are added to and removed from a particular event by writing | |
324 | trigger expressions to the 'trigger' file for the given event. | |
325 | ||
326 | A given event can have any number of triggers associated with it, | |
327 | subject to any restrictions that individual commands may have in that | |
328 | regard. | |
329 | ||
330 | Event triggers are implemented on top of "soft" mode, which means that | |
331 | whenever a trace event has one or more triggers associated with it, | |
332 | the event is activated even if it isn't actually enabled, but is | |
333 | disabled in a "soft" mode. That is, the tracepoint will be called, | |
334 | but just will not be traced, unless of course it's actually enabled. | |
335 | This scheme allows triggers to be invoked even for events that aren't | |
336 | enabled, and also allows the current event filter implementation to be | |
337 | used for conditionally invoking triggers. | |
338 | ||
339 | The syntax for event triggers is roughly based on the syntax for | |
340 | set_ftrace_filter 'ftrace filter commands' (see the 'Filter commands' | |
d3439f9d | 341 | section of Documentation/trace/ftrace.rst), but there are major |
ac38fb85 TZ |
342 | differences and the implementation isn't currently tied to it in any |
343 | way, so beware about making generalizations between the two. | |
344 | ||
d3439f9d SRV |
345 | Note: Writing into trace_marker (See Documentation/trace/ftrace.rst) |
346 | can also enable triggers that are written into | |
347 | /sys/kernel/tracing/events/ftrace/print/trigger | |
348 | ||
ac38fb85 TZ |
349 | 6.1 Expression syntax |
350 | --------------------- | |
351 | ||
73d98127 | 352 | Triggers are added by echoing the command to the 'trigger' file:: |
ac38fb85 TZ |
353 | |
354 | # echo 'command[:count] [if filter]' > trigger | |
355 | ||
356 | Triggers are removed by echoing the same command but starting with '!' | |
73d98127 | 357 | to the 'trigger' file:: |
ac38fb85 TZ |
358 | |
359 | # echo '!command[:count] [if filter]' > trigger | |
360 | ||
361 | The [if filter] part isn't used in matching commands when removing, so | |
362 | leaving that off in a '!' command will accomplish the same thing as | |
363 | having it in. | |
364 | ||
365 | The filter syntax is the same as that described in the 'Event | |
366 | filtering' section above. | |
367 | ||
368 | For ease of use, writing to the trigger file using '>' currently just | |
369 | adds or removes a single trigger and there's no explicit '>>' support | |
370 | ('>' actually behaves like '>>') or truncation support to remove all | |
371 | triggers (you have to use '!' for each one added.) | |
372 | ||
373 | 6.2 Supported trigger commands | |
374 | ------------------------------ | |
375 | ||
376 | The following commands are supported: | |
377 | ||
378 | - enable_event/disable_event | |
379 | ||
380 | These commands can enable or disable another trace event whenever | |
381 | the triggering event is hit. When these commands are registered, | |
382 | the other trace event is activated, but disabled in a "soft" mode. | |
383 | That is, the tracepoint will be called, but just will not be traced. | |
384 | The event tracepoint stays in this mode as long as there's a trigger | |
385 | in effect that can trigger it. | |
386 | ||
387 | For example, the following trigger causes kmalloc events to be | |
388 | traced when a read system call is entered, and the :1 at the end | |
73d98127 | 389 | specifies that this enablement happens only once:: |
ac38fb85 | 390 | |
73d98127 CD |
391 | # echo 'enable_event:kmem:kmalloc:1' > \ |
392 | /sys/kernel/debug/tracing/events/syscalls/sys_enter_read/trigger | |
ac38fb85 TZ |
393 | |
394 | The following trigger causes kmalloc events to stop being traced | |
395 | when a read system call exits. This disablement happens on every | |
73d98127 | 396 | read system call exit:: |
ac38fb85 | 397 | |
73d98127 CD |
398 | # echo 'disable_event:kmem:kmalloc' > \ |
399 | /sys/kernel/debug/tracing/events/syscalls/sys_exit_read/trigger | |
ac38fb85 | 400 | |
73d98127 | 401 | The format is:: |
ac38fb85 TZ |
402 | |
403 | enable_event:<system>:<event>[:count] | |
404 | disable_event:<system>:<event>[:count] | |
405 | ||
73d98127 | 406 | To remove the above commands:: |
ac38fb85 | 407 | |
73d98127 CD |
408 | # echo '!enable_event:kmem:kmalloc:1' > \ |
409 | /sys/kernel/debug/tracing/events/syscalls/sys_enter_read/trigger | |
ac38fb85 | 410 | |
73d98127 CD |
411 | # echo '!disable_event:kmem:kmalloc' > \ |
412 | /sys/kernel/debug/tracing/events/syscalls/sys_exit_read/trigger | |
ac38fb85 TZ |
413 | |
414 | Note that there can be any number of enable/disable_event triggers | |
415 | per triggering event, but there can only be one trigger per | |
416 | triggered event. e.g. sys_enter_read can have triggers enabling both | |
417 | kmem:kmalloc and sched:sched_switch, but can't have two kmem:kmalloc | |
418 | versions such as kmem:kmalloc and kmem:kmalloc:1 or 'kmem:kmalloc if | |
419 | bytes_req == 256' and 'kmem:kmalloc if bytes_alloc == 256' (they | |
420 | could be combined into a single filter on kmem:kmalloc though). | |
421 | ||
422 | - stacktrace | |
423 | ||
424 | This command dumps a stacktrace in the trace buffer whenever the | |
425 | triggering event occurs. | |
426 | ||
427 | For example, the following trigger dumps a stacktrace every time the | |
73d98127 | 428 | kmalloc tracepoint is hit:: |
ac38fb85 | 429 | |
73d98127 CD |
430 | # echo 'stacktrace' > \ |
431 | /sys/kernel/debug/tracing/events/kmem/kmalloc/trigger | |
ac38fb85 TZ |
432 | |
433 | The following trigger dumps a stacktrace the first 5 times a kmalloc | |
73d98127 | 434 | request happens with a size >= 64K:: |
ac38fb85 | 435 | |
73d98127 CD |
436 | # echo 'stacktrace:5 if bytes_req >= 65536' > \ |
437 | /sys/kernel/debug/tracing/events/kmem/kmalloc/trigger | |
ac38fb85 | 438 | |
73d98127 | 439 | The format is:: |
ac38fb85 TZ |
440 | |
441 | stacktrace[:count] | |
442 | ||
73d98127 | 443 | To remove the above commands:: |
ac38fb85 | 444 | |
73d98127 CD |
445 | # echo '!stacktrace' > \ |
446 | /sys/kernel/debug/tracing/events/kmem/kmalloc/trigger | |
ac38fb85 | 447 | |
73d98127 CD |
448 | # echo '!stacktrace:5 if bytes_req >= 65536' > \ |
449 | /sys/kernel/debug/tracing/events/kmem/kmalloc/trigger | |
ac38fb85 TZ |
450 | |
451 | The latter can also be removed more simply by the following (without | |
73d98127 | 452 | the filter):: |
ac38fb85 | 453 | |
73d98127 CD |
454 | # echo '!stacktrace:5' > \ |
455 | /sys/kernel/debug/tracing/events/kmem/kmalloc/trigger | |
ac38fb85 TZ |
456 | |
457 | Note that there can be only one stacktrace trigger per triggering | |
458 | event. | |
459 | ||
460 | - snapshot | |
461 | ||
462 | This command causes a snapshot to be triggered whenever the | |
463 | triggering event occurs. | |
464 | ||
465 | The following command creates a snapshot every time a block request | |
466 | queue is unplugged with a depth > 1. If you were tracing a set of | |
467 | events or functions at the time, the snapshot trace buffer would | |
73d98127 | 468 | capture those events when the trigger event occurred:: |
ac38fb85 | 469 | |
73d98127 CD |
470 | # echo 'snapshot if nr_rq > 1' > \ |
471 | /sys/kernel/debug/tracing/events/block/block_unplug/trigger | |
ac38fb85 | 472 | |
73d98127 | 473 | To only snapshot once:: |
ac38fb85 | 474 | |
73d98127 CD |
475 | # echo 'snapshot:1 if nr_rq > 1' > \ |
476 | /sys/kernel/debug/tracing/events/block/block_unplug/trigger | |
ac38fb85 | 477 | |
73d98127 | 478 | To remove the above commands:: |
ac38fb85 | 479 | |
73d98127 CD |
480 | # echo '!snapshot if nr_rq > 1' > \ |
481 | /sys/kernel/debug/tracing/events/block/block_unplug/trigger | |
ac38fb85 | 482 | |
73d98127 CD |
483 | # echo '!snapshot:1 if nr_rq > 1' > \ |
484 | /sys/kernel/debug/tracing/events/block/block_unplug/trigger | |
ac38fb85 TZ |
485 | |
486 | Note that there can be only one snapshot trigger per triggering | |
487 | event. | |
488 | ||
489 | - traceon/traceoff | |
490 | ||
491 | These commands turn tracing on and off when the specified events are | |
492 | hit. The parameter determines how many times the tracing system is | |
493 | turned on and off. If unspecified, there is no limit. | |
494 | ||
495 | The following command turns tracing off the first time a block | |
496 | request queue is unplugged with a depth > 1. If you were tracing a | |
497 | set of events or functions at the time, you could then examine the | |
498 | trace buffer to see the sequence of events that led up to the | |
73d98127 | 499 | trigger event:: |
ac38fb85 | 500 | |
73d98127 CD |
501 | # echo 'traceoff:1 if nr_rq > 1' > \ |
502 | /sys/kernel/debug/tracing/events/block/block_unplug/trigger | |
ac38fb85 | 503 | |
73d98127 | 504 | To always disable tracing when nr_rq > 1:: |
ac38fb85 | 505 | |
73d98127 CD |
506 | # echo 'traceoff if nr_rq > 1' > \ |
507 | /sys/kernel/debug/tracing/events/block/block_unplug/trigger | |
ac38fb85 | 508 | |
73d98127 | 509 | To remove the above commands:: |
ac38fb85 | 510 | |
73d98127 CD |
511 | # echo '!traceoff:1 if nr_rq > 1' > \ |
512 | /sys/kernel/debug/tracing/events/block/block_unplug/trigger | |
ac38fb85 | 513 | |
73d98127 CD |
514 | # echo '!traceoff if nr_rq > 1' > \ |
515 | /sys/kernel/debug/tracing/events/block/block_unplug/trigger | |
ac38fb85 TZ |
516 | |
517 | Note that there can be only one traceon or traceoff trigger per | |
518 | triggering event. | |
0fc3813c TZ |
519 | |
520 | - hist | |
521 | ||
522 | This command aggregates event hits into a hash table keyed on one or | |
523 | more trace event format fields (or stacktrace) and a set of running | |
524 | totals derived from one or more trace event format fields and/or | |
525 | event counts (hitcount). | |
526 | ||
ea272257 | 527 | See Documentation/trace/histogram.rst for details and examples. |