Commit | Line | Data |
---|---|---|
d8ec9185 MH |
1 | Kprobe-based Event Tracer |
2 | ========================= | |
3 | ||
4 | Documentation is written by Masami Hiramatsu | |
5 | ||
6 | ||
7 | Overview | |
8 | -------- | |
9 | This tracer is similar to the events tracer which is based on Tracepoint | |
10 | infrastructure. Instead of Tracepoint, this tracer is based on kprobes(kprobe | |
11 | and kretprobe). It probes anywhere where kprobes can probe(this means, all | |
12 | functions body except for __kprobes functions). | |
13 | ||
14 | Unlike the function tracer, this tracer can probe instructions inside of | |
15 | kernel functions. It allows you to check which instruction has been executed. | |
16 | ||
17 | Unlike the Tracepoint based events tracer, this tracer can add and remove | |
18 | probe points on the fly. | |
19 | ||
20 | Similar to the events tracer, this tracer doesn't need to be activated via | |
21 | current_tracer, instead of that, just set probe points via | |
22 | /sys/kernel/debug/tracing/kprobe_events. And you can set filters on each | |
23 | probe events via /sys/kernel/debug/tracing/events/kprobes/<EVENT>/filter. | |
24 | ||
25 | ||
26 | Synopsis of kprobe_events | |
27 | ------------------------- | |
2fba0c88 MH |
28 | p[:EVENT] SYMBOL[+offs]|MEMADDR [FETCHARGS] : Set a probe |
29 | r[:EVENT] SYMBOL[+0] [FETCHARGS] : Set a return probe | |
d8ec9185 | 30 | |
2fba0c88 MH |
31 | EVENT : Event name. If omitted, the event name is generated |
32 | based on SYMBOL+offs or MEMADDR. | |
33 | SYMBOL[+offs] : Symbol+offset where the probe is inserted. | |
34 | MEMADDR : Address where the probe is inserted. | |
d8ec9185 | 35 | |
2fba0c88 | 36 | FETCHARGS : Arguments. Each probe can have up to 128 args. |
d8ec9185 MH |
37 | %REG : Fetch register REG |
38 | sN : Fetch Nth entry of stack (N >= 0) | |
39 | sa : Fetch stack address. | |
40 | @ADDR : Fetch memory at ADDR (ADDR should be in kernel) | |
41 | @SYM[+|-offs] : Fetch memory at SYM +|- offs (SYM should be a data symbol) | |
42 | aN : Fetch function argument. (N >= 0)(*) | |
43 | rv : Fetch return value.(**) | |
44 | ra : Fetch return address.(**) | |
45 | +|-offs(FETCHARG) : fetch memory at FETCHARG +|- offs address.(***) | |
46 | ||
47 | (*) aN may not correct on asmlinkaged functions and at the middle of | |
48 | function body. | |
49 | (**) only for return probe. | |
50 | (***) this is useful for fetching a field of data structures. | |
51 | ||
52 | ||
53 | Per-Probe Event Filtering | |
54 | ------------------------- | |
55 | Per-probe event filtering feature allows you to set different filter on each | |
56 | probe and gives you what arguments will be shown in trace buffer. If an event | |
57 | name is specified right after 'p:' or 'r:' in kprobe_events, the tracer adds | |
58 | an event under tracing/events/kprobes/<EVENT>, at the directory you can see | |
59 | 'id', 'enabled', 'format' and 'filter'. | |
60 | ||
61 | enabled: | |
62 | You can enable/disable the probe by writing 1 or 0 on it. | |
63 | ||
64 | format: | |
e08d1c65 | 65 | This shows the format of this probe event. It also shows aliases of arguments |
d8ec9185 MH |
66 | which you specified to kprobe_events. |
67 | ||
68 | filter: | |
69 | You can write filtering rules of this event. And you can use both of aliase | |
70 | names and field names for describing filters. | |
71 | ||
e08d1c65 MH |
72 | id: |
73 | This shows the id of this probe event. | |
d8ec9185 | 74 | |
cd7e7bd5 MH |
75 | Event Profiling |
76 | --------------- | |
77 | You can check the total number of probe hits and probe miss-hits via | |
78 | /sys/kernel/debug/tracing/kprobe_profile. | |
79 | The first column is event name, the second is the number of probe hits, | |
80 | the third is the number of probe miss-hits. | |
81 | ||
82 | ||
d8ec9185 MH |
83 | Usage examples |
84 | -------------- | |
85 | To add a probe as a new event, write a new definition to kprobe_events | |
86 | as below. | |
87 | ||
88 | echo p:myprobe do_sys_open a0 a1 a2 a3 > /sys/kernel/debug/tracing/kprobe_events | |
89 | ||
90 | This sets a kprobe on the top of do_sys_open() function with recording | |
91 | 1st to 4th arguments as "myprobe" event. | |
92 | ||
93 | echo r:myretprobe do_sys_open rv ra >> /sys/kernel/debug/tracing/kprobe_events | |
94 | ||
95 | This sets a kretprobe on the return point of do_sys_open() function with | |
96 | recording return value and return address as "myretprobe" event. | |
97 | You can see the format of these events via | |
98 | /sys/kernel/debug/tracing/events/kprobes/<EVENT>/format. | |
99 | ||
100 | cat /sys/kernel/debug/tracing/events/kprobes/myprobe/format | |
101 | name: myprobe | |
102 | ID: 23 | |
103 | format: | |
104 | field:unsigned short common_type; offset:0; size:2; | |
105 | field:unsigned char common_flags; offset:2; size:1; | |
106 | field:unsigned char common_preempt_count; offset:3; size:1; | |
107 | field:int common_pid; offset:4; size:4; | |
108 | field:int common_tgid; offset:8; size:4; | |
109 | ||
110 | field: unsigned long ip; offset:16;tsize:8; | |
111 | field: int nargs; offset:24;tsize:4; | |
112 | field: unsigned long arg0; offset:32;tsize:8; | |
113 | field: unsigned long arg1; offset:40;tsize:8; | |
114 | field: unsigned long arg2; offset:48;tsize:8; | |
115 | field: unsigned long arg3; offset:56;tsize:8; | |
116 | ||
117 | alias: a0; original: arg0; | |
118 | alias: a1; original: arg1; | |
119 | alias: a2; original: arg2; | |
120 | alias: a3; original: arg3; | |
121 | ||
122 | print fmt: "%lx: 0x%lx 0x%lx 0x%lx 0x%lx", ip, arg0, arg1, arg2, arg3 | |
123 | ||
124 | ||
125 | You can see that the event has 4 arguments and alias expressions | |
126 | corresponding to it. | |
127 | ||
128 | echo > /sys/kernel/debug/tracing/kprobe_events | |
129 | ||
130 | This clears all probe points. and you can see the traced information via | |
131 | /sys/kernel/debug/tracing/trace. | |
132 | ||
133 | cat /sys/kernel/debug/tracing/trace | |
134 | # tracer: nop | |
135 | # | |
136 | # TASK-PID CPU# TIMESTAMP FUNCTION | |
137 | # | | | | | | |
138 | <...>-1447 [001] 1038282.286875: do_sys_open+0x0/0xd6: 0x3 0x7fffd1ec4440 0x8000 0x0 | |
139 | <...>-1447 [001] 1038282.286878: sys_openat+0xc/0xe <- do_sys_open: 0xfffffffffffffffe 0xffffffff81367a3a | |
140 | <...>-1447 [001] 1038282.286885: do_sys_open+0x0/0xd6: 0xffffff9c 0x40413c 0x8000 0x1b6 | |
141 | <...>-1447 [001] 1038282.286915: sys_open+0x1b/0x1d <- do_sys_open: 0x3 0xffffffff81367a3a | |
142 | <...>-1447 [001] 1038282.286969: do_sys_open+0x0/0xd6: 0xffffff9c 0x4041c6 0x98800 0x10 | |
143 | <...>-1447 [001] 1038282.286976: sys_open+0x1b/0x1d <- do_sys_open: 0x3 0xffffffff81367a3a | |
144 | ||
145 | ||
146 | Each line shows when the kernel hits a probe, and <- SYMBOL means kernel | |
147 | returns from SYMBOL(e.g. "sys_open+0x1b/0x1d <- do_sys_open" means kernel | |
148 | returns from do_sys_open to sys_open+0x1b). | |
149 | ||
150 |