Commit | Line | Data |
---|---|---|
76e7fd84 AB |
1 | .. _perf_security: |
2 | ||
3 | Perf Events and tool security | |
4 | ============================= | |
5 | ||
6 | Overview | |
7 | -------- | |
8 | ||
e85a198e AB |
9 | Usage of Performance Counters for Linux (perf_events) [1]_ , [2]_ , [3]_ |
10 | can impose a considerable risk of leaking sensitive data accessed by | |
11 | monitored processes. The data leakage is possible both in scenarios of | |
12 | direct usage of perf_events system call API [2]_ and over data files | |
13 | generated by Perf tool user mode utility (Perf) [3]_ , [4]_ . The risk | |
14 | depends on the nature of data that perf_events performance monitoring | |
15 | units (PMU) [2]_ and Perf collect and expose for performance analysis. | |
16 | Collected system and performance data may be split into several | |
17 | categories: | |
18 | ||
19 | 1. System hardware and software configuration data, for example: a CPU | |
20 | model and its cache configuration, an amount of available memory and | |
21 | its topology, used kernel and Perf versions, performance monitoring | |
22 | setup including experiment time, events configuration, Perf command | |
23 | line parameters, etc. | |
24 | ||
25 | 2. User and kernel module paths and their load addresses with sizes, | |
26 | process and thread names with their PIDs and TIDs, timestamps for | |
27 | captured hardware and software events. | |
28 | ||
29 | 3. Content of kernel software counters (e.g., for context switches, page | |
30 | faults, CPU migrations), architectural hardware performance counters | |
31 | (PMC) [8]_ and machine specific registers (MSR) [9]_ that provide | |
32 | execution metrics for various monitored parts of the system (e.g., | |
33 | memory controller (IMC), interconnect (QPI/UPI) or peripheral (PCIe) | |
34 | uncore counters) without direct attribution to any execution context | |
35 | state. | |
36 | ||
37 | 4. Content of architectural execution context registers (e.g., RIP, RSP, | |
38 | RBP on x86_64), process user and kernel space memory addresses and | |
39 | data, content of various architectural MSRs that capture data from | |
40 | this category. | |
41 | ||
42 | Data that belong to the fourth category can potentially contain | |
43 | sensitive process data. If PMUs in some monitoring modes capture values | |
44 | of execution context registers or data from process memory then access | |
45 | to such monitoring capabilities requires to be ordered and secured | |
46 | properly. So, perf_events/Perf performance monitoring is the subject for | |
47 | security access control management [5]_ . | |
76e7fd84 AB |
48 | |
49 | perf_events/Perf access control | |
50 | ------------------------------- | |
51 | ||
e85a198e AB |
52 | To perform security checks, the Linux implementation splits processes |
53 | into two categories [6]_ : a) privileged processes (whose effective user | |
54 | ID is 0, referred to as superuser or root), and b) unprivileged | |
55 | processes (whose effective UID is nonzero). Privileged processes bypass | |
56 | all kernel security permission checks so perf_events performance | |
57 | monitoring is fully available to privileged processes without access, | |
58 | scope and resource restrictions. | |
59 | ||
60 | Unprivileged processes are subject to a full security permission check | |
61 | based on the process's credentials [5]_ (usually: effective UID, | |
62 | effective GID, and supplementary group list). | |
63 | ||
64 | Linux divides the privileges traditionally associated with superuser | |
65 | into distinct units, known as capabilities [6]_ , which can be | |
66 | independently enabled and disabled on per-thread basis for processes and | |
67 | files of unprivileged users. | |
68 | ||
69 | Unprivileged processes with enabled CAP_SYS_ADMIN capability are treated | |
70 | as privileged processes with respect to perf_events performance | |
71 | monitoring and bypass *scope* permissions checks in the kernel. | |
72 | ||
73 | Unprivileged processes using perf_events system call API is also subject | |
74 | for PTRACE_MODE_READ_REALCREDS ptrace access mode check [7]_ , whose | |
75 | outcome determines whether monitoring is permitted. So unprivileged | |
76 | processes provided with CAP_SYS_PTRACE capability are effectively | |
77 | permitted to pass the check. | |
78 | ||
79 | Other capabilities being granted to unprivileged processes can | |
80 | effectively enable capturing of additional data required for later | |
81 | performance analysis of monitored processes or a system. For example, | |
82 | CAP_SYSLOG capability permits reading kernel space memory addresses from | |
83 | /proc/kallsyms file. | |
76e7fd84 | 84 | |
e152c7b7 AB |
85 | perf_events/Perf privileged users |
86 | --------------------------------- | |
87 | ||
e85a198e AB |
88 | Mechanisms of capabilities, privileged capability-dumb files [6]_ and |
89 | file system ACLs [10]_ can be used to create a dedicated group of | |
90 | perf_events/Perf privileged users who are permitted to execute | |
91 | performance monitoring without scope limits. The following steps can be | |
92 | taken to create such a group of privileged Perf users. | |
e152c7b7 | 93 | |
e85a198e AB |
94 | 1. Create perf_users group of privileged Perf users, assign perf_users |
95 | group to Perf tool executable and limit access to the executable for | |
96 | other users in the system who are not in the perf_users group: | |
e152c7b7 AB |
97 | |
98 | :: | |
99 | ||
100 | # groupadd perf_users | |
101 | # ls -alhF | |
102 | -rwxr-xr-x 2 root root 11M Oct 19 15:12 perf | |
103 | # chgrp perf_users perf | |
104 | # ls -alhF | |
105 | -rwxr-xr-x 2 root perf_users 11M Oct 19 15:12 perf | |
106 | # chmod o-rwx perf | |
107 | # ls -alhF | |
108 | -rwxr-x--- 2 root perf_users 11M Oct 19 15:12 perf | |
109 | ||
e85a198e AB |
110 | 2. Assign the required capabilities to the Perf tool executable file and |
111 | enable members of perf_users group with performance monitoring | |
112 | privileges [6]_ : | |
e152c7b7 AB |
113 | |
114 | :: | |
115 | ||
116 | # setcap "cap_sys_admin,cap_sys_ptrace,cap_syslog=ep" perf | |
117 | # setcap -v "cap_sys_admin,cap_sys_ptrace,cap_syslog=ep" perf | |
118 | perf: OK | |
119 | # getcap perf | |
120 | perf = cap_sys_ptrace,cap_sys_admin,cap_syslog+ep | |
121 | ||
e85a198e AB |
122 | As a result, members of perf_users group are capable of conducting |
123 | performance monitoring by using functionality of the configured Perf | |
124 | tool executable that, when executes, passes perf_events subsystem scope | |
125 | checks. | |
e152c7b7 | 126 | |
e85a198e AB |
127 | This specific access control management is only available to superuser |
128 | or root running processes with CAP_SETPCAP, CAP_SETFCAP [6]_ | |
129 | capabilities. | |
e152c7b7 | 130 | |
76e7fd84 AB |
131 | perf_events/Perf unprivileged users |
132 | ----------------------------------- | |
133 | ||
e85a198e AB |
134 | perf_events/Perf *scope* and *access* control for unprivileged processes |
135 | is governed by perf_event_paranoid [2]_ setting: | |
76e7fd84 AB |
136 | |
137 | -1: | |
e85a198e AB |
138 | Impose no *scope* and *access* restrictions on using perf_events |
139 | performance monitoring. Per-user per-cpu perf_event_mlock_kb [2]_ | |
140 | locking limit is ignored when allocating memory buffers for storing | |
141 | performance data. This is the least secure mode since allowed | |
142 | monitored *scope* is maximized and no perf_events specific limits | |
143 | are imposed on *resources* allocated for performance monitoring. | |
76e7fd84 AB |
144 | |
145 | >=0: | |
146 | *scope* includes per-process and system wide performance monitoring | |
e85a198e AB |
147 | but excludes raw tracepoints and ftrace function tracepoints |
148 | monitoring. CPU and system events happened when executing either in | |
149 | user or in kernel space can be monitored and captured for later | |
150 | analysis. Per-user per-cpu perf_event_mlock_kb locking limit is | |
151 | imposed but ignored for unprivileged processes with CAP_IPC_LOCK | |
152 | [6]_ capability. | |
76e7fd84 AB |
153 | |
154 | >=1: | |
e85a198e AB |
155 | *scope* includes per-process performance monitoring only and |
156 | excludes system wide performance monitoring. CPU and system events | |
157 | happened when executing either in user or in kernel space can be | |
158 | monitored and captured for later analysis. Per-user per-cpu | |
159 | perf_event_mlock_kb locking limit is imposed but ignored for | |
160 | unprivileged processes with CAP_IPC_LOCK capability. | |
76e7fd84 AB |
161 | |
162 | >=2: | |
e85a198e AB |
163 | *scope* includes per-process performance monitoring only. CPU and |
164 | system events happened when executing in user space only can be | |
165 | monitored and captured for later analysis. Per-user per-cpu | |
166 | perf_event_mlock_kb locking limit is imposed but ignored for | |
167 | unprivileged processes with CAP_IPC_LOCK capability. | |
76e7fd84 | 168 | |
9d87bbae AB |
169 | perf_events/Perf resource control |
170 | --------------------------------- | |
171 | ||
172 | Open file descriptors | |
173 | +++++++++++++++++++++ | |
174 | ||
e85a198e AB |
175 | The perf_events system call API [2]_ allocates file descriptors for |
176 | every configured PMU event. Open file descriptors are a per-process | |
177 | accountable resource governed by the RLIMIT_NOFILE [11]_ limit | |
178 | (ulimit -n), which is usually derived from the login shell process. When | |
179 | configuring Perf collection for a long list of events on a large server | |
180 | system, this limit can be easily hit preventing required monitoring | |
181 | configuration. RLIMIT_NOFILE limit can be increased on per-user basis | |
182 | modifying content of the limits.conf file [12]_ . Ordinarily, a Perf | |
183 | sampling session (perf record) requires an amount of open perf_event | |
184 | file descriptors that is not less than the number of monitored events | |
185 | multiplied by the number of monitored CPUs. | |
9d87bbae AB |
186 | |
187 | Memory allocation | |
188 | +++++++++++++++++ | |
189 | ||
e85a198e AB |
190 | The amount of memory available to user processes for capturing |
191 | performance monitoring data is governed by the perf_event_mlock_kb [2]_ | |
192 | setting. This perf_event specific resource setting defines overall | |
193 | per-cpu limits of memory allowed for mapping by the user processes to | |
194 | execute performance monitoring. The setting essentially extends the | |
195 | RLIMIT_MEMLOCK [11]_ limit, but only for memory regions mapped | |
196 | specifically for capturing monitored performance events and related data. | |
197 | ||
198 | For example, if a machine has eight cores and perf_event_mlock_kb limit | |
199 | is set to 516 KiB, then a user process is provided with 516 KiB * 8 = | |
200 | 4128 KiB of memory above the RLIMIT_MEMLOCK limit (ulimit -l) for | |
201 | perf_event mmap buffers. In particular, this means that, if the user | |
202 | wants to start two or more performance monitoring processes, the user is | |
203 | required to manually distribute the available 4128 KiB between the | |
204 | monitoring processes, for example, using the --mmap-pages Perf record | |
205 | mode option. Otherwise, the first started performance monitoring process | |
206 | allocates all available 4128 KiB and the other processes will fail to | |
207 | proceed due to the lack of memory. | |
208 | ||
209 | RLIMIT_MEMLOCK and perf_event_mlock_kb resource constraints are ignored | |
210 | for processes with the CAP_IPC_LOCK capability. Thus, perf_events/Perf | |
211 | privileged users can be provided with memory above the constraints for | |
212 | perf_events/Perf performance monitoring purpose by providing the Perf | |
213 | executable with CAP_IPC_LOCK capability. | |
9d87bbae | 214 | |
76e7fd84 AB |
215 | Bibliography |
216 | ------------ | |
217 | ||
218 | .. [1] `<https://lwn.net/Articles/337493/>`_ | |
219 | .. [2] `<http://man7.org/linux/man-pages/man2/perf_event_open.2.html>`_ | |
220 | .. [3] `<http://web.eece.maine.edu/~vweaver/projects/perf_events/>`_ | |
221 | .. [4] `<https://perf.wiki.kernel.org/index.php/Main_Page>`_ | |
222 | .. [5] `<https://www.kernel.org/doc/html/latest/security/credentials.html>`_ | |
223 | .. [6] `<http://man7.org/linux/man-pages/man7/capabilities.7.html>`_ | |
224 | .. [7] `<http://man7.org/linux/man-pages/man2/ptrace.2.html>`_ | |
68570ca0 AB |
225 | .. [8] `<https://en.wikipedia.org/wiki/Hardware_performance_counter>`_ |
226 | .. [9] `<https://en.wikipedia.org/wiki/Model-specific_register>`_ | |
e152c7b7 | 227 | .. [10] `<http://man7.org/linux/man-pages/man5/acl.5.html>`_ |
9d87bbae AB |
228 | .. [11] `<http://man7.org/linux/man-pages/man2/getrlimit.2.html>`_ |
229 | .. [12] `<http://man7.org/linux/man-pages/man5/limits.conf.5.html>`_ | |
76e7fd84 | 230 |