Commit | Line | Data |
---|---|---|
4e37f055 CD |
1 | .. SPDX-License-Identifier: GPL-2.0 |
2 | .. include:: <isonum.txt> | |
47402400 | 3 | |
4e37f055 CD |
4 | =========================================================== |
5 | The PCI Express Advanced Error Reporting Driver Guide HOWTO | |
6 | =========================================================== | |
47402400 | 7 | |
4e37f055 CD |
8 | :Authors: - T. Long Nguyen <tom.l.nguyen@intel.com> |
9 | - Yanmin Zhang <yanmin.zhang@intel.com> | |
47402400 | 10 | |
4e37f055 CD |
11 | :Copyright: |copy| 2006 Intel Corporation |
12 | ||
13 | Overview | |
14 | =========== | |
15 | ||
16 | About this guide | |
17 | ---------------- | |
47402400 | 18 | |
11502fea | 19 | This guide describes the basics of the PCI Express (PCIe) Advanced Error |
47402400 | 20 | Reporting (AER) driver and provides information on how to use it, as |
11502fea BH |
21 | well as how to enable the drivers of Endpoint devices to conform with |
22 | the PCIe AER driver. | |
47402400 | 23 | |
47402400 | 24 | |
11502fea BH |
25 | What is the PCIe AER Driver? |
26 | ---------------------------- | |
47402400 | 27 | |
11502fea BH |
28 | PCIe error signaling can occur on the PCIe link itself |
29 | or on behalf of transactions initiated on the link. PCIe | |
47402400 ZY |
30 | defines two error reporting paradigms: the baseline capability and |
31 | the Advanced Error Reporting capability. The baseline capability is | |
11502fea | 32 | required of all PCIe components providing a minimum defined |
47402400 | 33 | set of error reporting requirements. Advanced Error Reporting |
11502fea | 34 | capability is implemented with a PCIe Advanced Error Reporting |
47402400 ZY |
35 | extended capability structure providing more robust error reporting. |
36 | ||
11502fea BH |
37 | The PCIe AER driver provides the infrastructure to support PCIe Advanced |
38 | Error Reporting capability. The PCIe AER driver provides three basic | |
39 | functions: | |
47402400 | 40 | |
4e37f055 CD |
41 | - Gathers the comprehensive error information if errors occurred. |
42 | - Reports error to the users. | |
43 | - Performs error recovery actions. | |
47402400 | 44 | |
11502fea BH |
45 | The AER driver only attaches to Root Ports and RCECs that support the PCIe |
46 | AER capability. | |
47402400 ZY |
47 | |
48 | ||
4e37f055 CD |
49 | User Guide |
50 | ========== | |
47402400 | 51 | |
11502fea BH |
52 | Include the PCIe AER Root Driver into the Linux Kernel |
53 | ------------------------------------------------------ | |
47402400 | 54 | |
11502fea BH |
55 | The PCIe AER driver is a Root Port service driver attached |
56 | via the PCIe Port Bus driver. If a user wants to use it, the driver | |
57 | must be compiled. It is enabled with CONFIG_PCIEAER, which | |
58 | depends on CONFIG_PCIEPORTBUS. | |
47402400 | 59 | |
11502fea BH |
60 | Load PCIe AER Root Driver |
61 | ------------------------- | |
7ece1417 BH |
62 | |
63 | Some systems have AER support in firmware. Enabling Linux AER support at | |
11502fea | 64 | the same time the firmware handles AER would result in unpredictable |
7ece1417 | 65 | behavior. Therefore, Linux does not handle AER events unless the firmware |
11502fea | 66 | grants AER control to the OS via the ACPI _OSC method. See the PCI Firmware |
7ece1417 | 67 | Specification for details regarding _OSC usage. |
47402400 | 68 | |
4e37f055 CD |
69 | AER error output |
70 | ---------------- | |
7ece1417 BH |
71 | |
72 | When a PCIe AER error is captured, an error message will be output to | |
11502fea | 73 | console. If it's a correctable error, it is output as an info message. |
47402400 ZY |
74 | Otherwise, it is printed as an error. So users could choose different |
75 | log level to filter out correctable error messages. | |
76 | ||
4e37f055 CD |
77 | Below shows an example:: |
78 | ||
79 | 0000:50:00.0: PCIe Bus Error: severity=Uncorrected (Fatal), type=Transaction Layer, id=0500(Requester ID) | |
80 | 0000:50:00.0: device [8086:0329] error status/mask=00100000/00000000 | |
81 | 0000:50:00.0: [20] Unsupported Request (First) | |
82 | 0000:50:00.0: TLP Header: 04000001 00200a03 05010000 00050100 | |
47402400 | 83 | |
11502fea BH |
84 | In the example, 'Requester ID' means the ID of the device that sent |
85 | the error message to the Root Port. Please refer to PCIe specs for other | |
86 | fields. | |
47402400 | 87 | |
4e37f055 CD |
88 | AER Statistics / Counters |
89 | ------------------------- | |
81aa5206 RJ |
90 | |
91 | When PCIe AER errors are captured, the counters / statistics are also exposed | |
92 | in the form of sysfs attributes which are documented at | |
93 | Documentation/ABI/testing/sysfs-bus-pci-devices-aer_stats | |
47402400 | 94 | |
4e37f055 CD |
95 | Developer Guide |
96 | =============== | |
47402400 | 97 | |
11502fea | 98 | To enable error recovery, a software driver must provide callbacks. |
47402400 | 99 | |
11502fea | 100 | To support AER better, developers need to understand how AER works. |
47402400 | 101 | |
11502fea BH |
102 | PCIe errors are classified into two types: correctable errors |
103 | and uncorrectable errors. This classification is based on the impact | |
47402400 ZY |
104 | of those errors, which may result in degraded performance or function |
105 | failure. | |
106 | ||
107 | Correctable errors pose no impacts on the functionality of the | |
11502fea | 108 | interface. The PCIe protocol can recover without any software |
47402400 | 109 | intervention or any loss of data. These errors are detected and |
11502fea BH |
110 | corrected by hardware. |
111 | ||
112 | Unlike correctable errors, uncorrectable | |
47402400 | 113 | errors impact functionality of the interface. Uncorrectable errors |
11502fea | 114 | can cause a particular transaction or a particular PCIe link |
47402400 ZY |
115 | to be unreliable. Depending on those error conditions, uncorrectable |
116 | errors are further classified into non-fatal errors and fatal errors. | |
117 | Non-fatal errors cause the particular transaction to be unreliable, | |
11502fea | 118 | but the PCIe link itself is fully functional. Fatal errors, on |
47402400 ZY |
119 | the other hand, cause the link to be unreliable. |
120 | ||
11502fea BH |
121 | When PCIe error reporting is enabled, a device will automatically send an |
122 | error message to the Root Port above it when it captures | |
47402400 | 123 | an error. The Root Port, upon receiving an error reporting message, |
11502fea BH |
124 | internally processes and logs the error message in its AER |
125 | Capability structure. Error information being logged includes storing | |
47402400 ZY |
126 | the error reporting agent's requestor ID into the Error Source |
127 | Identification Registers and setting the error bits of the Root Error | |
11502fea BH |
128 | Status Register accordingly. If AER error reporting is enabled in the Root |
129 | Error Command Register, the Root Port generates an interrupt when an | |
47402400 ZY |
130 | error is detected. |
131 | ||
11502fea | 132 | Note that the errors as described above are related to the PCIe |
47402400 ZY |
133 | hierarchy and links. These errors do not include any device specific |
134 | errors because device specific errors will still get sent directly to | |
135 | the device driver. | |
136 | ||
4e37f055 CD |
137 | Provide callbacks |
138 | ----------------- | |
47402400 | 139 | |
11502fea BH |
140 | callback reset_link to reset PCIe link |
141 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
47402400 | 142 | |
11502fea BH |
143 | This callback is used to reset the PCIe physical link when a |
144 | fatal error happens. The Root Port AER service driver provides a | |
145 | default reset_link function, but different Upstream Ports might | |
146 | have different specifications to reset the PCIe link, so | |
147 | Upstream Port drivers may provide their own reset_link functions. | |
47402400 | 148 | |
47402400 ZY |
149 | Section 3.2.2.2 provides more detailed info on when to call |
150 | reset_link. | |
151 | ||
4e37f055 CD |
152 | PCI error-recovery callbacks |
153 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
47402400 | 154 | |
11502fea | 155 | The PCIe AER Root driver uses error callbacks to coordinate |
47402400 ZY |
156 | with downstream device drivers associated with a hierarchy in question |
157 | when performing error recovery actions. | |
158 | ||
159 | Data struct pci_driver has a pointer, err_handler, to point to | |
160 | pci_error_handlers who consists of a couple of callback function | |
11502fea BH |
161 | pointers. The AER driver follows the rules defined in |
162 | pci-error-recovery.rst except PCIe-specific parts (e.g. | |
163 | reset_link). Please refer to pci-error-recovery.rst for detailed | |
47402400 ZY |
164 | definitions of the callbacks. |
165 | ||
11502fea | 166 | The sections below specify when to call the error callback functions. |
47402400 | 167 | |
4e37f055 CD |
168 | Correctable errors |
169 | ~~~~~~~~~~~~~~~~~~ | |
47402400 ZY |
170 | |
171 | Correctable errors pose no impacts on the functionality of | |
11502fea | 172 | the interface. The PCIe protocol can recover without any |
47402400 ZY |
173 | software intervention or any loss of data. These errors do not |
174 | require any recovery actions. The AER driver clears the device's | |
175 | correctable error status register accordingly and logs these errors. | |
176 | ||
4e37f055 CD |
177 | Non-correctable (non-fatal and fatal) errors |
178 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
47402400 ZY |
179 | |
180 | If an error message indicates a non-fatal error, performing link reset | |
181 | at upstream is not required. The AER driver calls error_detected(dev, | |
182 | pci_channel_io_normal) to all drivers associated within a hierarchy in | |
11502fea | 183 | question. For example:: |
4e37f055 | 184 | |
11502fea | 185 | Endpoint <==> Downstream Port B <==> Upstream Port A <==> Root Port |
4e37f055 | 186 | |
11502fea BH |
187 | If Upstream Port A captures an AER error, the hierarchy consists of |
188 | Downstream Port B and Endpoint. | |
47402400 ZY |
189 | |
190 | A driver may return PCI_ERS_RESULT_CAN_RECOVER, | |
191 | PCI_ERS_RESULT_DISCONNECT, or PCI_ERS_RESULT_NEED_RESET, depending on | |
192 | whether it can recover or the AER driver calls mmio_enabled as next. | |
193 | ||
194 | If an error message indicates a fatal error, kernel will broadcast | |
195 | error_detected(dev, pci_channel_io_frozen) to all drivers within | |
196 | a hierarchy in question. Then, performing link reset at upstream is | |
197 | necessary. As different kinds of devices might use different approaches | |
198 | to reset link, AER port service driver is required to provide the | |
b6cf1a42 KS |
199 | function to reset link via callback parameter of pcie_do_recovery() |
200 | function. If reset_link is not NULL, recovery function will use it | |
201 | to reset the link. If error_detected returns PCI_ERS_RESULT_CAN_RECOVER | |
202 | and reset_link returns PCI_ERS_RESULT_RECOVERED, the error handling goes | |
47402400 ZY |
203 | to mmio_enabled. |
204 | ||
4e37f055 CD |
205 | Frequent Asked Questions |
206 | ------------------------ | |
47402400 | 207 | |
4e37f055 | 208 | Q: |
11502fea | 209 | What happens if a PCIe device driver does not provide an |
4e37f055 | 210 | error recovery handler (pci_driver->err_handler is equal to NULL)? |
47402400 | 211 | |
4e37f055 CD |
212 | A: |
213 | The devices attached with the driver won't be recovered. If the | |
214 | error is fatal, kernel will print out warning messages. Please refer | |
215 | to section 3 for more information. | |
47402400 | 216 | |
4e37f055 CD |
217 | Q: |
218 | What happens if an upstream port service driver does not provide | |
219 | callback reset_link? | |
47402400 | 220 | |
4e37f055 CD |
221 | A: |
222 | Fatal error recovery will fail if the errors are reported by the | |
223 | upstream ports who are attached by the service driver. | |
47402400 | 224 | |
bfe5a740 | 225 | |
4e37f055 CD |
226 | Software error injection |
227 | ======================== | |
bfe5a740 | 228 | |
89713422 | 229 | Debugging PCIe AER error recovery code is quite difficult because it |
bfe5a740 | 230 | is hard to trigger real hardware errors. Software based error |
89713422 | 231 | injection can be used to fake various kinds of PCIe errors. |
bfe5a740 | 232 | |
89713422 | 233 | First you should enable PCIe AER software error injection in kernel |
bfe5a740 HY |
234 | configuration, that is, following item should be in your .config. |
235 | ||
236 | CONFIG_PCIEAER_INJECT=y or CONFIG_PCIEAER_INJECT=m | |
237 | ||
238 | After reboot with new kernel or insert the module, a device file named | |
239 | /dev/aer_inject should be created. | |
240 | ||
241 | Then, you need a user space tool named aer-inject, which can be gotten | |
242 | from: | |
4e37f055 | 243 | |
2eb6a4b2 | 244 | https://git.kernel.org/cgit/linux/kernel/git/gong.chen/aer-inject.git/ |
bfe5a740 | 245 | |
11502fea BH |
246 | More information about aer-inject can be found in the document in |
247 | its source code. |