Commit | Line | Data |
---|---|---|
4d2e26a3 | 1 | ===================== |
c6bef2e9 | 2 | DAWR issues on POWER9 |
4d2e26a3 | 3 | ===================== |
c6bef2e9 | 4 | |
26b78c81 RA |
5 | On older POWER9 processors, the Data Address Watchpoint Register (DAWR) can |
6 | cause a checkstop if it points to cache inhibited (CI) memory. Currently Linux | |
7 | has no way to distinguish CI memory when configuring the DAWR, so on affected | |
8 | systems, the DAWR is disabled. | |
9 | ||
10 | Affected processor revisions | |
11 | ============================ | |
12 | ||
13 | This issue is only present on processors prior to v2.3. The revision can be | |
14 | found in /proc/cpuinfo:: | |
15 | ||
16 | processor : 0 | |
17 | cpu : POWER9, altivec supported | |
18 | clock : 3800.000000MHz | |
19 | revision : 2.3 (pvr 004e 1203) | |
20 | ||
21 | On a system with the issue, the DAWR is disabled as detailed below. | |
c6bef2e9 MN |
22 | |
23 | Technical Details: | |
4d2e26a3 | 24 | ================== |
c6bef2e9 MN |
25 | |
26 | DAWR has 6 different ways of being set. | |
27 | 1) ptrace | |
28 | 2) h_set_mode(DAWR) | |
29 | 3) h_set_dabr() | |
30 | 4) kvmppc_set_one_reg() | |
31 | 5) xmon | |
32 | ||
33 | For ptrace, we now advertise zero breakpoints on POWER9 via the | |
34 | PPC_PTRACE_GETHWDBGINFO call. This results in GDB falling back to | |
35 | software emulation of the watchpoint (which is slow). | |
36 | ||
37 | h_set_mode(DAWR) and h_set_dabr() will now return an error to the | |
38 | guest on a POWER9 host. Current Linux guests ignore this error, so | |
39 | they will silently not get the DAWR. | |
40 | ||
41 | kvmppc_set_one_reg() will store the value in the vcpu but won't | |
42 | actually set it on POWER9 hardware. This is done so we don't break | |
43 | migration from POWER8 to POWER9, at the cost of silently losing the | |
44 | DAWR on the migration. | |
45 | ||
46 | For xmon, the 'bd' command will return an error on P9. | |
47 | ||
48 | Consequences for users | |
4d2e26a3 | 49 | ====================== |
c6bef2e9 MN |
50 | |
51 | For GDB watchpoints (ie 'watch' command) on POWER9 bare metal , GDB | |
52 | will accept the command. Unfortunately since there is no hardware | |
53 | support for the watchpoint, GDB will software emulate the watchpoint | |
54 | making it run very slowly. | |
55 | ||
56 | The same will also be true for any guests started on a POWER9 | |
57 | host. The watchpoint will fail and GDB will fall back to software | |
58 | emulation. | |
59 | ||
60 | If a guest is started on a POWER8 host, GDB will accept the watchpoint | |
61 | and configure the hardware to use the DAWR. This will run at full | |
62 | speed since it can use the hardware emulation. Unfortunately if this | |
63 | guest is migrated to a POWER9 host, the watchpoint will be lost on the | |
64 | POWER9. Loads and stores to the watchpoint locations will not be | |
65 | trapped in GDB. The watchpoint is remembered, so if the guest is | |
66 | migrated back to the POWER8 host, it will start working again. | |
67 | ||
c1fe190c | 68 | Force enabling the DAWR |
4d2e26a3 MCC |
69 | ======================= |
70 | Kernels (since ~v5.2) have an option to force enable the DAWR via:: | |
c1fe190c MN |
71 | |
72 | echo Y > /sys/kernel/debug/powerpc/dawr_enable_dangerous | |
73 | ||
74 | This enables the DAWR even on POWER9. | |
75 | ||
76 | This is a dangerous setting, USE AT YOUR OWN RISK. | |
77 | ||
78 | Some users may not care about a bad user crashing their box | |
79 | (ie. single user/desktop systems) and really want the DAWR. This | |
80 | allows them to force enable DAWR. | |
81 | ||
82 | This flag can also be used to disable DAWR access. Once this is | |
83 | cleared, all DAWR access should be cleared immediately and your | |
84 | machine once again safe from crashing. | |
85 | ||
86 | Userspace may get confused by toggling this. If DAWR is force | |
87 | enabled/disabled between getting the number of breakpoints (via | |
88 | PTRACE_GETHWDBGINFO) and setting the breakpoint, userspace will get an | |
89 | inconsistent view of what's available. Similarly for guests. | |
90 | ||
91 | For the DAWR to be enabled in a KVM guest, the DAWR needs to be force | |
92 | enabled in the host AND the guest. For this reason, this won't work on | |
93 | POWERVM as it doesn't allow the HCALL to work. Writes of 'Y' to the | |
94 | dawr_enable_dangerous file will fail if the hypervisor doesn't support | |
95 | writing the DAWR. | |
96 | ||
97 | To double check the DAWR is working, run this kernel selftest: | |
4d2e26a3 | 98 | |
c1fe190c | 99 | tools/testing/selftests/powerpc/ptrace/ptrace-hwbreak.c |
4d2e26a3 | 100 | |
c1fe190c | 101 | Any errors/failures/skips mean something is wrong. |