Commit | Line | Data |
---|---|---|
d8ec9185 MH |
1 | Kprobe-based Event Tracer |
2 | ========================= | |
3 | ||
4 | Documentation is written by Masami Hiramatsu | |
5 | ||
6 | ||
7 | Overview | |
8 | -------- | |
9 | This tracer is similar to the events tracer which is based on Tracepoint | |
10 | infrastructure. Instead of Tracepoint, this tracer is based on kprobes(kprobe | |
11 | and kretprobe). It probes anywhere where kprobes can probe(this means, all | |
12 | functions body except for __kprobes functions). | |
13 | ||
14 | Unlike the function tracer, this tracer can probe instructions inside of | |
15 | kernel functions. It allows you to check which instruction has been executed. | |
16 | ||
17 | Unlike the Tracepoint based events tracer, this tracer can add and remove | |
18 | probe points on the fly. | |
19 | ||
20 | Similar to the events tracer, this tracer doesn't need to be activated via | |
21 | current_tracer, instead of that, just set probe points via | |
22 | /sys/kernel/debug/tracing/kprobe_events. And you can set filters on each | |
23 | probe events via /sys/kernel/debug/tracing/events/kprobes/<EVENT>/filter. | |
24 | ||
25 | ||
26 | Synopsis of kprobe_events | |
27 | ------------------------- | |
2fba0c88 MH |
28 | p[:EVENT] SYMBOL[+offs]|MEMADDR [FETCHARGS] : Set a probe |
29 | r[:EVENT] SYMBOL[+0] [FETCHARGS] : Set a return probe | |
d8ec9185 | 30 | |
2fba0c88 MH |
31 | EVENT : Event name. If omitted, the event name is generated |
32 | based on SYMBOL+offs or MEMADDR. | |
33 | SYMBOL[+offs] : Symbol+offset where the probe is inserted. | |
34 | MEMADDR : Address where the probe is inserted. | |
d8ec9185 | 35 | |
2fba0c88 | 36 | FETCHARGS : Arguments. Each probe can have up to 128 args. |
d8ec9185 MH |
37 | %REG : Fetch register REG |
38 | sN : Fetch Nth entry of stack (N >= 0) | |
39 | sa : Fetch stack address. | |
40 | @ADDR : Fetch memory at ADDR (ADDR should be in kernel) | |
41 | @SYM[+|-offs] : Fetch memory at SYM +|- offs (SYM should be a data symbol) | |
42 | aN : Fetch function argument. (N >= 0)(*) | |
43 | rv : Fetch return value.(**) | |
44 | ra : Fetch return address.(**) | |
eca0d916 MH |
45 | +|-offs(FETCHARG) : Fetch memory at FETCHARG +|- offs address.(***) |
46 | NAME=FETCHARG: Set NAME as the argument name of FETCHARG. | |
d8ec9185 MH |
47 | |
48 | (*) aN may not correct on asmlinkaged functions and at the middle of | |
49 | function body. | |
50 | (**) only for return probe. | |
51 | (***) this is useful for fetching a field of data structures. | |
52 | ||
53 | ||
54 | Per-Probe Event Filtering | |
55 | ------------------------- | |
56 | Per-probe event filtering feature allows you to set different filter on each | |
57 | probe and gives you what arguments will be shown in trace buffer. If an event | |
58 | name is specified right after 'p:' or 'r:' in kprobe_events, the tracer adds | |
59 | an event under tracing/events/kprobes/<EVENT>, at the directory you can see | |
60 | 'id', 'enabled', 'format' and 'filter'. | |
61 | ||
62 | enabled: | |
63 | You can enable/disable the probe by writing 1 or 0 on it. | |
64 | ||
65 | format: | |
eca0d916 | 66 | This shows the format of this probe event. |
d8ec9185 MH |
67 | |
68 | filter: | |
eca0d916 | 69 | You can write filtering rules of this event. |
d8ec9185 | 70 | |
e08d1c65 MH |
71 | id: |
72 | This shows the id of this probe event. | |
d8ec9185 | 73 | |
cd7e7bd5 MH |
74 | Event Profiling |
75 | --------------- | |
76 | You can check the total number of probe hits and probe miss-hits via | |
77 | /sys/kernel/debug/tracing/kprobe_profile. | |
78 | The first column is event name, the second is the number of probe hits, | |
79 | the third is the number of probe miss-hits. | |
80 | ||
81 | ||
d8ec9185 MH |
82 | Usage examples |
83 | -------------- | |
84 | To add a probe as a new event, write a new definition to kprobe_events | |
85 | as below. | |
86 | ||
eca0d916 | 87 | echo p:myprobe do_sys_open dfd=a0 filename=a1 flags=a2 mode=a3 > /sys/kernel/debug/tracing/kprobe_events |
d8ec9185 MH |
88 | |
89 | This sets a kprobe on the top of do_sys_open() function with recording | |
eca0d916 MH |
90 | 1st to 4th arguments as "myprobe" event. As this example shows, users can |
91 | choose more familiar names for each arguments. | |
d8ec9185 MH |
92 | |
93 | echo r:myretprobe do_sys_open rv ra >> /sys/kernel/debug/tracing/kprobe_events | |
94 | ||
95 | This sets a kretprobe on the return point of do_sys_open() function with | |
96 | recording return value and return address as "myretprobe" event. | |
97 | You can see the format of these events via | |
98 | /sys/kernel/debug/tracing/events/kprobes/<EVENT>/format. | |
99 | ||
100 | cat /sys/kernel/debug/tracing/events/kprobes/myprobe/format | |
101 | name: myprobe | |
eca0d916 | 102 | ID: 75 |
d8ec9185 MH |
103 | format: |
104 | field:unsigned short common_type; offset:0; size:2; | |
105 | field:unsigned char common_flags; offset:2; size:1; | |
106 | field:unsigned char common_preempt_count; offset:3; size:1; | |
107 | field:int common_pid; offset:4; size:4; | |
108 | field:int common_tgid; offset:8; size:4; | |
109 | ||
110 | field: unsigned long ip; offset:16;tsize:8; | |
111 | field: int nargs; offset:24;tsize:4; | |
eca0d916 MH |
112 | field: unsigned long dfd; offset:32;tsize:8; |
113 | field: unsigned long filename; offset:40;tsize:8; | |
114 | field: unsigned long flags; offset:48;tsize:8; | |
115 | field: unsigned long mode; offset:56;tsize:8; | |
d8ec9185 | 116 | |
6e9f23d1 | 117 | print fmt: "(%lx) dfd=%lx filename=%lx flags=%lx mode=%lx", REC->ip, REC->dfd, REC->filename, REC->flags, REC->mode |
d8ec9185 | 118 | |
d8ec9185 | 119 | |
eca0d916 | 120 | You can see that the event has 4 arguments as in the expressions you specified. |
d8ec9185 MH |
121 | |
122 | echo > /sys/kernel/debug/tracing/kprobe_events | |
123 | ||
124 | This clears all probe points. and you can see the traced information via | |
125 | /sys/kernel/debug/tracing/trace. | |
126 | ||
127 | cat /sys/kernel/debug/tracing/trace | |
128 | # tracer: nop | |
129 | # | |
130 | # TASK-PID CPU# TIMESTAMP FUNCTION | |
131 | # | | | | | | |
6e9f23d1 MH |
132 | <...>-1447 [001] 1038282.286875: myprobe: (do_sys_open+0x0/0xd6) dfd=3 filename=7fffd1ec4440 flags=8000 mode=0 |
133 | <...>-1447 [001] 1038282.286878: myretprobe: (sys_openat+0xc/0xe <- do_sys_open) rv=fffffffffffffffe ra=ffffffff81367a3a | |
134 | <...>-1447 [001] 1038282.286885: myprobe: (do_sys_open+0x0/0xd6) dfd=ffffff9c filename=40413c flags=8000 mode=1b6 | |
135 | <...>-1447 [001] 1038282.286915: myretprobe: (sys_open+0x1b/0x1d <- do_sys_open) rv=3 ra=ffffffff81367a3a | |
136 | <...>-1447 [001] 1038282.286969: myprobe: (do_sys_open+0x0/0xd6) dfd=ffffff9c filename=4041c6 flags=98800 mode=10 | |
137 | <...>-1447 [001] 1038282.286976: myretprobe: (sys_open+0x1b/0x1d <- do_sys_open) rv=3 ra=ffffffff81367a3a | |
d8ec9185 MH |
138 | |
139 | ||
6e9f23d1 | 140 | Each line shows when the kernel hits an event, and <- SYMBOL means kernel |
d8ec9185 MH |
141 | returns from SYMBOL(e.g. "sys_open+0x1b/0x1d <- do_sys_open" means kernel |
142 | returns from do_sys_open to sys_open+0x1b). | |
143 | ||
144 |