Commit | Line | Data |
---|---|---|
cfb9b89f PP |
1 | .. Copyright 2020 DisplayLink (UK) Ltd. |
2 | ||
22554020 | 3 | =================== |
ca00c2b9 JN |
4 | Userland interfaces |
5 | =================== | |
6 | ||
7 | The DRM core exports several interfaces to applications, generally | |
8 | intended to be used through corresponding libdrm wrapper functions. In | |
9 | addition, drivers export device-specific interfaces for use by userspace | |
10 | drivers & device-aware applications through ioctls and sysfs files. | |
11 | ||
12 | External interfaces include: memory mapping, context management, DMA | |
13 | operations, AGP management, vblank control, fence management, memory | |
14 | management, and output management. | |
15 | ||
16 | Cover generic ioctls and sysfs layout here. We only need high-level | |
17 | info, since man pages should cover the rest. | |
18 | ||
a3257256 DV |
19 | libdrm Device Lookup |
20 | ==================== | |
21 | ||
22 | .. kernel-doc:: drivers/gpu/drm/drm_ioctl.c | |
23 | :doc: getunique and setversion story | |
24 | ||
3b96a0b1 | 25 | |
b93658f8 DV |
26 | .. _drm_primary_node: |
27 | ||
3b96a0b1 DV |
28 | Primary Nodes, DRM Master and Authentication |
29 | ============================================ | |
30 | ||
31 | .. kernel-doc:: drivers/gpu/drm/drm_auth.c | |
32 | :doc: master and authentication | |
33 | ||
34 | .. kernel-doc:: drivers/gpu/drm/drm_auth.c | |
35 | :export: | |
36 | ||
37 | .. kernel-doc:: include/drm/drm_auth.h | |
38 | :internal: | |
39 | ||
d793b8f7 DCZX |
40 | |
41 | .. _drm_leasing: | |
42 | ||
43 | DRM Display Resource Leasing | |
44 | ============================ | |
45 | ||
46 | .. kernel-doc:: drivers/gpu/drm/drm_lease.c | |
47 | :doc: drm leasing | |
48 | ||
bcb32b69 DV |
49 | Open-Source Userspace Requirements |
50 | ================================== | |
51 | ||
0d42204f DV |
52 | The DRM subsystem has stricter requirements than most other kernel subsystems on |
53 | what the userspace side for new uAPI needs to look like. This section here | |
54 | explains what exactly those requirements are, and why they exist. | |
55 | ||
56 | The short summary is that any addition of DRM uAPI requires corresponding | |
57 | open-sourced userspace patches, and those patches must be reviewed and ready for | |
58 | merging into a suitable and canonical upstream project. | |
59 | ||
60 | GFX devices (both display and render/GPU side) are really complex bits of | |
61 | hardware, with userspace and kernel by necessity having to work together really | |
62 | closely. The interfaces, for rendering and modesetting, must be extremely wide | |
63 | and flexible, and therefore it is almost always impossible to precisely define | |
64 | them for every possible corner case. This in turn makes it really practically | |
65 | infeasible to differentiate between behaviour that's required by userspace, and | |
66 | which must not be changed to avoid regressions, and behaviour which is only an | |
67 | accidental artifact of the current implementation. | |
68 | ||
69 | Without access to the full source code of all userspace users that means it | |
70 | becomes impossible to change the implementation details, since userspace could | |
71 | depend upon the accidental behaviour of the current implementation in minute | |
72 | details. And debugging such regressions without access to source code is pretty | |
73 | much impossible. As a consequence this means: | |
74 | ||
75 | - The Linux kernel's "no regression" policy holds in practice only for | |
76 | open-source userspace of the DRM subsystem. DRM developers are perfectly fine | |
77 | if closed-source blob drivers in userspace use the same uAPI as the open | |
78 | drivers, but they must do so in the exact same way as the open drivers. | |
79 | Creative (ab)use of the interfaces will, and in the past routinely has, lead | |
80 | to breakage. | |
81 | ||
82 | - Any new userspace interface must have an open-source implementation as | |
83 | demonstration vehicle. | |
84 | ||
85 | The other reason for requiring open-source userspace is uAPI review. Since the | |
86 | kernel and userspace parts of a GFX stack must work together so closely, code | |
87 | review can only assess whether a new interface achieves its goals by looking at | |
88 | both sides. Making sure that the interface indeed covers the use-case fully | |
89 | leads to a few additional requirements: | |
90 | ||
91 | - The open-source userspace must not be a toy/test application, but the real | |
92 | thing. Specifically it needs to handle all the usual error and corner cases. | |
93 | These are often the places where new uAPI falls apart and hence essential to | |
94 | assess the fitness of a proposed interface. | |
95 | ||
96 | - The userspace side must be fully reviewed and tested to the standards of that | |
97 | userspace project. For e.g. mesa this means piglit testcases and review on the | |
98 | mailing list. This is again to ensure that the new interface actually gets the | |
e33df4ca DV |
99 | job done. The userspace-side reviewer should also provide an Acked-by on the |
100 | kernel uAPI patch indicating that they believe the proposed uAPI is sound and | |
101 | sufficiently documented and validated for userspace's consumption. | |
0d42204f DV |
102 | |
103 | - The userspace patches must be against the canonical upstream, not some vendor | |
104 | fork. This is to make sure that no one cheats on the review and testing | |
105 | requirements by doing a quick fork. | |
106 | ||
107 | - The kernel patch can only be merged after all the above requirements are met, | |
3d42fca0 EA |
108 | but it **must** be merged to either drm-next or drm-misc-next **before** the |
109 | userspace patches land. uAPI always flows from the kernel, doing things the | |
110 | other way round risks divergence of the uAPI definitions and header files. | |
0d42204f DV |
111 | |
112 | These are fairly steep requirements, but have grown out from years of shared | |
113 | pain and experience with uAPI added hastily, and almost always regretted about | |
114 | just as fast. GFX devices change really fast, requiring a paradigm shift and | |
115 | entire new set of uAPI interfaces every few years at least. Together with the | |
116 | Linux kernel's guarantee to keep existing userspace running for 10+ years this | |
117 | is already rather painful for the DRM subsystem, with multiple different uAPIs | |
118 | for the same thing co-existing. If we add a few more complete mistakes into the | |
119 | mix every year it would be entirely unmanageable. | |
120 | ||
b93658f8 DV |
121 | .. _drm_render_node: |
122 | ||
ca00c2b9 | 123 | Render nodes |
22554020 | 124 | ============ |
ca00c2b9 JN |
125 | |
126 | DRM core provides multiple character-devices for user-space to use. | |
127 | Depending on which device is opened, user-space can perform a different | |
128 | set of operations (mainly ioctls). The primary node is always created | |
129 | and called card<num>. Additionally, a currently unused control node, | |
130 | called controlD<num> is also created. The primary node provides all | |
131 | legacy operations and historically was the only interface used by | |
132 | userspace. With KMS, the control node was introduced. However, the | |
133 | planned KMS control interface has never been written and so the control | |
134 | node stays unused to date. | |
135 | ||
136 | With the increased use of offscreen renderers and GPGPU applications, | |
137 | clients no longer require running compositors or graphics servers to | |
138 | make use of a GPU. But the DRM API required unprivileged clients to | |
139 | authenticate to a DRM-Master prior to getting GPU access. To avoid this | |
140 | step and to grant clients GPU access without authenticating, render | |
141 | nodes were introduced. Render nodes solely serve render clients, that | |
142 | is, no modesetting or privileged ioctls can be issued on render nodes. | |
143 | Only non-global rendering commands are allowed. If a driver supports | |
144 | render nodes, it must advertise it via the DRIVER_RENDER DRM driver | |
145 | capability. If not supported, the primary node must be used for render | |
146 | clients together with the legacy drmAuth authentication procedure. | |
147 | ||
148 | If a driver advertises render node support, DRM core will create a | |
149 | separate render node called renderD<num>. There will be one render node | |
150 | per device. No ioctls except PRIME-related ioctls will be allowed on | |
aa457ff1 JH |
151 | this node. Especially GEM_OPEN will be explicitly prohibited. For a |
152 | complete list of driver-independent ioctls that can be used on render | |
153 | nodes, see the ioctls marked DRM_RENDER_ALLOW in drm_ioctl.c Render | |
ca00c2b9 JN |
154 | nodes are designed to avoid the buffer-leaks, which occur if clients |
155 | guess the flink names or mmap offsets on the legacy interface. | |
156 | Additionally to this basic interface, drivers must mark their | |
157 | driver-dependent render-only ioctls as DRM_RENDER_ALLOW so render | |
158 | clients can use them. Driver authors must be careful not to allow any | |
159 | privileged ioctls on render nodes. | |
160 | ||
161 | With render nodes, user-space can now control access to the render node | |
162 | via basic file-system access-modes. A running graphics server which | |
163 | authenticates clients on the privileged primary/legacy node is no longer | |
164 | required. Instead, a client can open the render node and is immediately | |
165 | granted GPU access. Communication between clients (or servers) is done | |
166 | via PRIME. FLINK from render node to legacy node is not supported. New | |
167 | clients must not use the insecure FLINK interface. | |
168 | ||
169 | Besides dropping all modeset/global ioctls, render nodes also drop the | |
170 | DRM-Master concept. There is no reason to associate render clients with | |
171 | a DRM-Master as they are independent of any graphics server. Besides, | |
172 | they must work without any running master, anyway. Drivers must be able | |
173 | to run without a master object if they support render nodes. If, on the | |
174 | other hand, a driver requires shared state between clients which is | |
175 | visible to user-space and accessible beyond open-file boundaries, they | |
176 | cannot support render nodes. | |
177 | ||
cfb9b89f PP |
178 | Device Hot-Unplug |
179 | ================= | |
180 | ||
181 | .. note:: | |
182 | The following is the plan. Implementation is not there yet | |
183 | (2020 May). | |
184 | ||
185 | Graphics devices (display and/or render) may be connected via USB (e.g. | |
186 | display adapters or docking stations) or Thunderbolt (e.g. eGPU). An end | |
187 | user is able to hot-unplug this kind of devices while they are being | |
188 | used, and expects that the very least the machine does not crash. Any | |
189 | damage from hot-unplugging a DRM device needs to be limited as much as | |
190 | possible and userspace must be given the chance to handle it if it wants | |
191 | to. Ideally, unplugging a DRM device still lets a desktop continue to | |
192 | run, but that is going to need explicit support throughout the whole | |
193 | graphics stack: from kernel and userspace drivers, through display | |
194 | servers, via window system protocols, and in applications and libraries. | |
195 | ||
196 | Other scenarios that should lead to the same are: unrecoverable GPU | |
197 | crash, PCI device disappearing off the bus, or forced unbind of a driver | |
198 | from the physical device. | |
199 | ||
200 | In other words, from userspace perspective everything needs to keep on | |
201 | working more or less, until userspace stops using the disappeared DRM | |
202 | device and closes it completely. Userspace will learn of the device | |
203 | disappearance from the device removed uevent, ioctls returning ENODEV | |
204 | (or driver-specific ioctls returning driver-specific things), or open() | |
205 | returning ENXIO. | |
206 | ||
207 | Only after userspace has closed all relevant DRM device and dmabuf file | |
208 | descriptors and removed all mmaps, the DRM driver can tear down its | |
209 | instance for the device that no longer exists. If the same physical | |
210 | device somehow comes back in the mean time, it shall be a new DRM | |
211 | device. | |
212 | ||
213 | Similar to PIDs, chardev minor numbers are not recycled immediately. A | |
214 | new DRM device always picks the next free minor number compared to the | |
215 | previous one allocated, and wraps around when minor numbers are | |
216 | exhausted. | |
217 | ||
218 | The goal raises at least the following requirements for the kernel and | |
219 | drivers. | |
220 | ||
221 | Requirements for KMS UAPI | |
222 | ------------------------- | |
223 | ||
224 | - KMS connectors must change their status to disconnected. | |
225 | ||
226 | - Legacy modesets and pageflips, and atomic commits, both real and | |
227 | TEST_ONLY, and any other ioctls either fail with ENODEV or fake | |
228 | success. | |
229 | ||
230 | - Pending non-blocking KMS operations deliver the DRM events userspace | |
231 | is expecting. This applies also to ioctls that faked success. | |
232 | ||
233 | - open() on a device node whose underlying device has disappeared will | |
234 | fail with ENXIO. | |
235 | ||
236 | - Attempting to create a DRM lease on a disappeared DRM device will | |
237 | fail with ENODEV. Existing DRM leases remain and work as listed | |
238 | above. | |
239 | ||
240 | Requirements for Render and Cross-Device UAPI | |
241 | --------------------------------------------- | |
242 | ||
243 | - All GPU jobs that can no longer run must have their fences | |
244 | force-signalled to avoid inflicting hangs on userspace. | |
245 | The associated error code is ENODEV. | |
246 | ||
247 | - Some userspace APIs already define what should happen when the device | |
248 | disappears (OpenGL, GL ES: `GL_KHR_robustness`_; `Vulkan`_: | |
249 | VK_ERROR_DEVICE_LOST; etc.). DRM drivers are free to implement this | |
250 | behaviour the way they see best, e.g. returning failures in | |
251 | driver-specific ioctls and handling those in userspace drivers, or | |
252 | rely on uevents, and so on. | |
253 | ||
254 | - dmabuf which point to memory that has disappeared will either fail to | |
255 | import with ENODEV or continue to be successfully imported if it would | |
256 | have succeeded before the disappearance. See also about memory maps | |
257 | below for already imported dmabufs. | |
258 | ||
259 | - Attempting to import a dmabuf to a disappeared device will either fail | |
260 | with ENODEV or succeed if it would have succeeded without the | |
261 | disappearance. | |
262 | ||
263 | - open() on a device node whose underlying device has disappeared will | |
264 | fail with ENXIO. | |
265 | ||
266 | .. _GL_KHR_robustness: https://www.khronos.org/registry/OpenGL/extensions/KHR/KHR_robustness.txt | |
267 | .. _Vulkan: https://www.khronos.org/vulkan/ | |
268 | ||
269 | Requirements for Memory Maps | |
270 | ---------------------------- | |
271 | ||
272 | Memory maps have further requirements that apply to both existing maps | |
273 | and maps created after the device has disappeared. If the underlying | |
274 | memory disappears, the map is created or modified such that reads and | |
275 | writes will still complete successfully but the result is undefined. | |
276 | This applies to both userspace mmap()'d memory and memory pointed to by | |
277 | dmabuf which might be mapped to other devices (cross-device dmabuf | |
278 | imports). | |
279 | ||
280 | Raising SIGBUS is not an option, because userspace cannot realistically | |
281 | handle it. Signal handlers are global, which makes them extremely | |
282 | difficult to use correctly from libraries like those that Mesa produces. | |
283 | Signal handlers are not composable, you can't have different handlers | |
284 | for GPU1 and GPU2 from different vendors, and a third handler for | |
285 | mmapped regular files. Threads cause additional pain with signal | |
286 | handling as well. | |
287 | ||
bb2eaba6 DV |
288 | .. _drm_driver_ioctl: |
289 | ||
2640981f DV |
290 | IOCTL Support on Device Nodes |
291 | ============================= | |
292 | ||
293 | .. kernel-doc:: drivers/gpu/drm/drm_ioctl.c | |
294 | :doc: driver specific ioctls | |
295 | ||
371cadd8 DV |
296 | Recommended IOCTL Return Values |
297 | ------------------------------- | |
298 | ||
299 | In theory a driver's IOCTL callback is only allowed to return very few error | |
300 | codes. In practice it's good to abuse a few more. This section documents common | |
301 | practice within the DRM subsystem: | |
302 | ||
303 | ENOENT: | |
304 | Strictly this should only be used when a file doesn't exist e.g. when | |
305 | calling the open() syscall. We reuse that to signal any kind of object | |
306 | lookup failure, e.g. for unknown GEM buffer object handles, unknown KMS | |
307 | object handles and similar cases. | |
308 | ||
309 | ENOSPC: | |
310 | Some drivers use this to differentiate "out of kernel memory" from "out | |
311 | of VRAM". Sometimes also applies to other limited gpu resources used for | |
312 | rendering (e.g. when you have a special limited compression buffer). | |
313 | Sometimes resource allocation/reservation issues in command submission | |
314 | IOCTLs are also signalled through EDEADLK. | |
315 | ||
316 | Simply running out of kernel/system memory is signalled through ENOMEM. | |
317 | ||
cba8087d | 318 | EPERM/EACCES: |
371cadd8 DV |
319 | Returned for an operation that is valid, but needs more privileges. |
320 | E.g. root-only or much more common, DRM master-only operations return | |
bde952f9 | 321 | this when called by unpriviledged clients. There's no clear |
cba8087d | 322 | difference between EACCES and EPERM. |
371cadd8 DV |
323 | |
324 | ENODEV: | |
cfb9b89f | 325 | The device is not present anymore or is not yet fully initialized. |
9edb6a0b DV |
326 | |
327 | EOPNOTSUPP: | |
371cadd8 DV |
328 | Feature (like PRIME, modesetting, GEM) is not supported by the driver. |
329 | ||
330 | ENXIO: | |
331 | Remote failure, either a hardware transaction (like i2c), but also used | |
332 | when the exporting driver of a shared dma-buf or fence doesn't support a | |
333 | feature needed. | |
334 | ||
335 | EINTR: | |
336 | DRM drivers assume that userspace restarts all IOCTLs. Any DRM IOCTL can | |
337 | return EINTR and in such a case should be restarted with the IOCTL | |
338 | parameters left unchanged. | |
339 | ||
340 | EIO: | |
341 | The GPU died and couldn't be resurrected through a reset. Modesetting | |
342 | hardware failures are signalled through the "link status" connector | |
343 | property. | |
344 | ||
345 | EINVAL: | |
346 | Catch-all for anything that is an invalid argument combination which | |
347 | cannot work. | |
348 | ||
349 | IOCTL also use other error codes like ETIME, EFAULT, EBUSY, ENOTTY but their | |
350 | usage is in line with the common meanings. The above list tries to just document | |
351 | DRM specific patterns. Note that ENOTTY has the slightly unintuitive meaning of | |
352 | "this IOCTL does not exist", and is used exactly as such in DRM. | |
353 | ||
2640981f DV |
354 | .. kernel-doc:: include/drm/drm_ioctl.h |
355 | :internal: | |
356 | ||
357 | .. kernel-doc:: drivers/gpu/drm/drm_ioctl.c | |
358 | :export: | |
359 | ||
360 | .. kernel-doc:: drivers/gpu/drm/drm_ioc32.c | |
361 | :export: | |
a8182863 DV |
362 | |
363 | Testing and validation | |
364 | ====================== | |
365 | ||
badfa5be DV |
366 | Testing Requirements for userspace API |
367 | -------------------------------------- | |
368 | ||
369 | New cross-driver userspace interface extensions, like new IOCTL, new KMS | |
370 | properties, new files in sysfs or anything else that constitutes an API change | |
371 | should have driver-agnostic testcases in IGT for that feature, if such a test | |
372 | can be reasonably made using IGT for the target hardware. | |
373 | ||
75ac4953 | 374 | Validating changes with IGT |
a8182863 | 375 | --------------------------- |
75ac4953 TV |
376 | |
377 | There's a collection of tests that aims to cover the whole functionality of | |
378 | DRM drivers and that can be used to check that changes to DRM drivers or the | |
379 | core don't regress existing functionality. This test suite is called IGT and | |
8a537de0 LR |
380 | its code and instructions to build and run can be found in |
381 | https://gitlab.freedesktop.org/drm/igt-gpu-tools/. | |
75ac4953 | 382 | |
6f91f44d GB |
383 | Using VKMS to test DRM API |
384 | -------------------------- | |
385 | ||
386 | VKMS is a software-only model of a KMS driver that is useful for testing | |
387 | and for running compositors. VKMS aims to enable a virtual display without | |
388 | the need for a hardware display capability. These characteristics made VKMS | |
389 | a perfect tool for validating the DRM core behavior and also support the | |
390 | compositor developer. VKMS makes it possible to test DRM functions in a | |
391 | virtual machine without display, simplifying the validation of some of the | |
392 | core changes. | |
393 | ||
394 | To Validate changes in DRM API with VKMS, start setting the kernel: make | |
395 | sure to enable VKMS module; compile the kernel with the VKMS enabled and | |
396 | install it in the target machine. VKMS can be run in a Virtual Machine | |
397 | (QEMU, virtme or similar). It's recommended the use of KVM with the minimum | |
398 | of 1GB of RAM and four cores. | |
399 | ||
400 | It's possible to run the IGT-tests in a VM in two ways: | |
401 | ||
402 | 1. Use IGT inside a VM | |
403 | 2. Use IGT from the host machine and write the results in a shared directory. | |
404 | ||
405 | As follow, there is an example of using a VM with a shared directory with | |
406 | the host machine to run igt-tests. As an example it's used virtme:: | |
407 | ||
408 | $ virtme-run --rwdir /path/for/shared_dir --kdir=path/for/kernel/directory --mods=auto | |
409 | ||
410 | Run the igt-tests in the guest machine, as example it's ran the 'kms_flip' | |
411 | tests:: | |
412 | ||
413 | $ /path/for/igt-gpu-tools/scripts/run-tests.sh -p -s -t "kms_flip.*" -v | |
414 | ||
415 | In this example, instead of build the igt_runner, Piglit is used | |
416 | (-p option); it's created html summary of the tests results and it's saved | |
417 | in the folder "igt-gpu-tools/results"; it's executed only the igt-tests | |
418 | matching the -t option. | |
419 | ||
a8182863 DV |
420 | Display CRC Support |
421 | ------------------- | |
422 | ||
423 | .. kernel-doc:: drivers/gpu/drm/drm_debugfs_crc.c | |
424 | :doc: CRC ABI | |
425 | ||
760f71e7 DV |
426 | .. kernel-doc:: drivers/gpu/drm/drm_debugfs_crc.c |
427 | :export: | |
428 | ||
0cad7f71 DV |
429 | Debugfs Support |
430 | --------------- | |
431 | ||
432 | .. kernel-doc:: include/drm/drm_debugfs.h | |
433 | :internal: | |
434 | ||
435 | .. kernel-doc:: drivers/gpu/drm/drm_debugfs.c | |
436 | :export: | |
437 | ||
e2271704 DV |
438 | Sysfs Support |
439 | ============= | |
440 | ||
441 | .. kernel-doc:: drivers/gpu/drm/drm_sysfs.c | |
442 | :doc: overview | |
443 | ||
444 | .. kernel-doc:: drivers/gpu/drm/drm_sysfs.c | |
445 | :export: | |
446 | ||
447 | ||
ca00c2b9 | 448 | VBlank event handling |
22554020 | 449 | ===================== |
ca00c2b9 JN |
450 | |
451 | The DRM core exposes two vertical blank related ioctls: | |
452 | ||
453 | DRM_IOCTL_WAIT_VBLANK | |
454 | This takes a struct drm_wait_vblank structure as its argument, and | |
455 | it is used to block or request a signal when a specified vblank | |
456 | event occurs. | |
457 | ||
458 | DRM_IOCTL_MODESET_CTL | |
459 | This was only used for user-mode-settind drivers around modesetting | |
460 | changes to allow the kernel to update the vblank interrupt after | |
461 | mode setting, since on many devices the vertical blank counter is | |
462 | reset to 0 at some point during modeset. Modern drivers should not | |
463 | call this any more since with kernel mode setting it is a no-op. | |
309aa926 US |
464 | |
465 | Userspace API Structures | |
466 | ======================== | |
467 | ||
468 | .. kernel-doc:: include/uapi/drm/drm_mode.h | |
469 | :doc: overview | |
470 | ||
26594678 LR |
471 | .. _crtc_index: |
472 | ||
473 | CRTC index | |
474 | ---------- | |
475 | ||
476 | CRTC's have both an object ID and an index, and they are not the same thing. | |
477 | The index is used in cases where a densely packed identifier for a CRTC is | |
478 | needed, for instance a bitmask of CRTC's. The member possible_crtcs of struct | |
479 | drm_mode_get_plane is an example. | |
480 | ||
481 | DRM_IOCTL_MODE_GETRESOURCES populates a structure with an array of CRTC ID's, | |
482 | and the CRTC index is its position in this array. | |
483 | ||
cf9a4be4 SS |
484 | .. kernel-doc:: include/uapi/drm/drm.h |
485 | :internal: | |
486 | ||
309aa926 US |
487 | .. kernel-doc:: include/uapi/drm/drm_mode.h |
488 | :internal: |