Commit | Line | Data |
---|---|---|
868c97a8 DV |
1 | Buffer Sharing and Synchronization |
2 | ================================== | |
3 | ||
4 | The dma-buf subsystem provides the framework for sharing buffers for | |
5 | hardware (DMA) access across multiple device drivers and subsystems, and | |
6 | for synchronizing asynchronous hardware access. | |
7 | ||
8 | This is used, for example, by drm "prime" multi-GPU support, but is of | |
9 | course not limited to GPU use cases. | |
10 | ||
11 | The three main components of this are: (1) dma-buf, representing a | |
12 | sg_table and exposed to userspace as a file descriptor to allow passing | |
13 | between devices, (2) fence, which provides a mechanism to signal when | |
776d5882 | 14 | one device has finished access, and (3) reservation, which manages the |
868c97a8 DV |
15 | shared or exclusive fence(s) associated with the buffer. |
16 | ||
17 | Shared DMA Buffers | |
18 | ------------------ | |
19 | ||
2904a8c1 DV |
20 | This document serves as a guide to device-driver writers on what is the dma-buf |
21 | buffer sharing API, how to use it for exporting and using shared buffers. | |
22 | ||
23 | Any device driver which wishes to be a part of DMA buffer sharing, can do so as | |
24 | either the 'exporter' of buffers, or the 'user' or 'importer' of buffers. | |
25 | ||
26 | Say a driver A wants to use buffers created by driver B, then we call B as the | |
27 | exporter, and A as buffer-user/importer. | |
28 | ||
29 | The exporter | |
30 | ||
31 | - implements and manages operations in :c:type:`struct dma_buf_ops | |
32 | <dma_buf_ops>` for the buffer, | |
33 | - allows other users to share the buffer by using dma_buf sharing APIs, | |
776d5882 | 34 | - manages the details of buffer allocation, wrapped in a :c:type:`struct |
2904a8c1 DV |
35 | dma_buf <dma_buf>`, |
36 | - decides about the actual backing storage where this allocation happens, | |
37 | - and takes care of any migration of scatterlist - for all (shared) users of | |
38 | this buffer. | |
39 | ||
40 | The buffer-user | |
41 | ||
42 | - is one of (many) sharing users of the buffer. | |
43 | - doesn't need to worry about how the buffer is allocated, or where. | |
44 | - and needs a mechanism to get access to the scatterlist that makes up this | |
45 | buffer in memory, mapped into its own address space, so it can access the | |
46 | same area of memory. This interface is provided by :c:type:`struct | |
47 | dma_buf_attachment <dma_buf_attachment>`. | |
48 | ||
e7e21c72 DV |
49 | Any exporters or users of the dma-buf buffer sharing framework must have a |
50 | 'select DMA_SHARED_BUFFER' in their respective Kconfigs. | |
51 | ||
52 | Userspace Interface Notes | |
53 | ~~~~~~~~~~~~~~~~~~~~~~~~~ | |
54 | ||
55 | Mostly a DMA buffer file descriptor is simply an opaque object for userspace, | |
56 | and hence the generic interface exposed is very minimal. There's a few things to | |
57 | consider though: | |
58 | ||
59 | - Since kernel 3.12 the dma-buf FD supports the llseek system call, but only | |
60 | with offset=0 and whence=SEEK_END|SEEK_SET. SEEK_SET is supported to allow | |
61 | the usual size discover pattern size = SEEK_END(0); SEEK_SET(0). Every other | |
62 | llseek operation will report -EINVAL. | |
63 | ||
64 | If llseek on dma-buf FDs isn't support the kernel will report -ESPIPE for all | |
65 | cases. Userspace can use this to detect support for discovering the dma-buf | |
66 | size using llseek. | |
67 | ||
68 | - In order to avoid fd leaks on exec, the FD_CLOEXEC flag must be set | |
69 | on the file descriptor. This is not just a resource leak, but a | |
70 | potential security hole. It could give the newly exec'd application | |
71 | access to buffers, via the leaked fd, to which it should otherwise | |
72 | not be permitted access. | |
73 | ||
74 | The problem with doing this via a separate fcntl() call, versus doing it | |
75 | atomically when the fd is created, is that this is inherently racy in a | |
76 | multi-threaded app[3]. The issue is made worse when it is library code | |
77 | opening/creating the file descriptor, as the application may not even be | |
78 | aware of the fd's. | |
79 | ||
80 | To avoid this problem, userspace must have a way to request O_CLOEXEC | |
81 | flag be set when the dma-buf fd is created. So any API provided by | |
82 | the exporting driver to create a dmabuf fd must provide a way to let | |
83 | userspace control setting of O_CLOEXEC flag passed in to dma_buf_fd(). | |
84 | ||
85 | - Memory mapping the contents of the DMA buffer is also supported. See the | |
86 | discussion below on `CPU Access to DMA Buffer Objects`_ for the full details. | |
87 | ||
b899353d | 88 | - The DMA buffer FD is also pollable, see `Implicit Fence Poll Support`_ below for |
e7e21c72 DV |
89 | details. |
90 | ||
2904a8c1 DV |
91 | Basic Operation and Device DMA Access |
92 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
93 | ||
94 | .. kernel-doc:: drivers/dma-buf/dma-buf.c | |
95 | :doc: dma buf device access | |
96 | ||
0959a168 DV |
97 | CPU Access to DMA Buffer Objects |
98 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
99 | ||
100 | .. kernel-doc:: drivers/dma-buf/dma-buf.c | |
101 | :doc: cpu access | |
102 | ||
102514ec DV |
103 | Implicit Fence Poll Support |
104 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
e7e21c72 DV |
105 | |
106 | .. kernel-doc:: drivers/dma-buf/dma-buf.c | |
102514ec | 107 | :doc: implicit fence polling |
e7e21c72 | 108 | |
bdb8d06d HV |
109 | DMA-BUF statistics |
110 | ~~~~~~~~~~~~~~~~~~ | |
111 | .. kernel-doc:: drivers/dma-buf/dma-buf-sysfs-stats.c | |
112 | :doc: overview | |
113 | ||
2904a8c1 DV |
114 | Kernel Functions and Structures Reference |
115 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
116 | ||
868c97a8 DV |
117 | .. kernel-doc:: drivers/dma-buf/dma-buf.c |
118 | :export: | |
119 | ||
120 | .. kernel-doc:: include/linux/dma-buf.h | |
121 | :internal: | |
122 | ||
ccc22d41 TZ |
123 | Buffer Mapping Helpers |
124 | ~~~~~~~~~~~~~~~~~~~~~~ | |
125 | ||
126 | .. kernel-doc:: include/linux/dma-buf-map.h | |
127 | :doc: overview | |
128 | ||
129 | .. kernel-doc:: include/linux/dma-buf-map.h | |
130 | :internal: | |
131 | ||
868c97a8 DV |
132 | Reservation Objects |
133 | ------------------- | |
134 | ||
0f546217 | 135 | .. kernel-doc:: drivers/dma-buf/dma-resv.c |
868c97a8 DV |
136 | :doc: Reservation Object Overview |
137 | ||
0f546217 | 138 | .. kernel-doc:: drivers/dma-buf/dma-resv.c |
868c97a8 DV |
139 | :export: |
140 | ||
0f546217 | 141 | .. kernel-doc:: include/linux/dma-resv.h |
868c97a8 DV |
142 | :internal: |
143 | ||
144 | DMA Fences | |
145 | ---------- | |
146 | ||
4dd3cdb2 DV |
147 | .. kernel-doc:: drivers/dma-buf/dma-fence.c |
148 | :doc: DMA fences overview | |
149 | ||
d0b9a9ae DV |
150 | DMA Fence Cross-Driver Contract |
151 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
152 | ||
153 | .. kernel-doc:: drivers/dma-buf/dma-fence.c | |
154 | :doc: fence cross-driver contract | |
155 | ||
5fbff813 DV |
156 | DMA Fence Signalling Annotations |
157 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
158 | ||
159 | .. kernel-doc:: drivers/dma-buf/dma-fence.c | |
160 | :doc: fence signalling annotation | |
161 | ||
4dd3cdb2 DV |
162 | DMA Fences Functions Reference |
163 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
164 | ||
868c97a8 DV |
165 | .. kernel-doc:: drivers/dma-buf/dma-fence.c |
166 | :export: | |
167 | ||
168 | .. kernel-doc:: include/linux/dma-fence.h | |
169 | :internal: | |
170 | ||
171 | Seqno Hardware Fences | |
172 | ~~~~~~~~~~~~~~~~~~~~~ | |
173 | ||
868c97a8 DV |
174 | .. kernel-doc:: include/linux/seqno-fence.h |
175 | :internal: | |
176 | ||
177 | DMA Fence Array | |
178 | ~~~~~~~~~~~~~~~ | |
179 | ||
180 | .. kernel-doc:: drivers/dma-buf/dma-fence-array.c | |
181 | :export: | |
182 | ||
183 | .. kernel-doc:: include/linux/dma-fence-array.h | |
184 | :internal: | |
185 | ||
b970b8e9 DV |
186 | DMA Fence Chain |
187 | ~~~~~~~~~~~~~~~ | |
188 | ||
189 | .. kernel-doc:: drivers/dma-buf/dma-fence-chain.c | |
190 | :export: | |
191 | ||
192 | .. kernel-doc:: include/linux/dma-fence-chain.h | |
193 | :internal: | |
194 | ||
868c97a8 DV |
195 | DMA Fence uABI/Sync File |
196 | ~~~~~~~~~~~~~~~~~~~~~~~~ | |
197 | ||
198 | .. kernel-doc:: drivers/dma-buf/sync_file.c | |
199 | :export: | |
200 | ||
201 | .. kernel-doc:: include/linux/sync_file.h | |
202 | :internal: | |
203 | ||
72b6ede7 | 204 | Indefinite DMA Fences |
6546d28f | 205 | ~~~~~~~~~~~~~~~~~~~~~ |
72b6ede7 | 206 | |
26e08a6d | 207 | At various times struct dma_fence with an indefinite time until dma_fence_wait() |
72b6ede7 DV |
208 | finishes have been proposed. Examples include: |
209 | ||
210 | * Future fences, used in HWC1 to signal when a buffer isn't used by the display | |
211 | any longer, and created with the screen update that makes the buffer visible. | |
212 | The time this fence completes is entirely under userspace's control. | |
213 | ||
214 | * Proxy fences, proposed to handle &drm_syncobj for which the fence has not yet | |
215 | been set. Used to asynchronously delay command submission. | |
216 | ||
217 | * Userspace fences or gpu futexes, fine-grained locking within a command buffer | |
218 | that userspace uses for synchronization across engines or with the CPU, which | |
219 | are then imported as a DMA fence for integration into existing winsys | |
220 | protocols. | |
221 | ||
222 | * Long-running compute command buffers, while still using traditional end of | |
223 | batch DMA fences for memory management instead of context preemption DMA | |
224 | fences which get reattached when the compute job is rescheduled. | |
225 | ||
226 | Common to all these schemes is that userspace controls the dependencies of these | |
227 | fences and controls when they fire. Mixing indefinite fences with normal | |
228 | in-kernel DMA fences does not work, even when a fallback timeout is included to | |
229 | protect against malicious userspace: | |
230 | ||
231 | * Only the kernel knows about all DMA fence dependencies, userspace is not aware | |
232 | of dependencies injected due to memory management or scheduler decisions. | |
233 | ||
234 | * Only userspace knows about all dependencies in indefinite fences and when | |
235 | exactly they will complete, the kernel has no visibility. | |
236 | ||
237 | Furthermore the kernel has to be able to hold up userspace command submission | |
238 | for memory management needs, which means we must support indefinite fences being | |
239 | dependent upon DMA fences. If the kernel also support indefinite fences in the | |
240 | kernel like a DMA fence, like any of the above proposal would, there is the | |
241 | potential for deadlocks. | |
242 | ||
243 | .. kernel-render:: DOT | |
244 | :alt: Indefinite Fencing Dependency Cycle | |
245 | :caption: Indefinite Fencing Dependency Cycle | |
246 | ||
247 | digraph "Fencing Cycle" { | |
248 | node [shape=box bgcolor=grey style=filled] | |
249 | kernel [label="Kernel DMA Fences"] | |
250 | userspace [label="userspace controlled fences"] | |
251 | kernel -> userspace [label="memory management"] | |
252 | userspace -> kernel [label="Future fence, fence proxy, ..."] | |
253 | ||
254 | { rank=same; kernel userspace } | |
255 | } | |
256 | ||
257 | This means that the kernel might accidentally create deadlocks | |
258 | through memory management dependencies which userspace is unaware of, which | |
259 | randomly hangs workloads until the timeout kicks in. Workloads, which from | |
260 | userspace's perspective, do not contain a deadlock. In such a mixed fencing | |
261 | architecture there is no single entity with knowledge of all dependencies. | |
262 | Thefore preventing such deadlocks from within the kernel is not possible. | |
263 | ||
264 | The only solution to avoid dependencies loops is by not allowing indefinite | |
265 | fences in the kernel. This means: | |
266 | ||
267 | * No future fences, proxy fences or userspace fences imported as DMA fences, | |
268 | with or without a timeout. | |
269 | ||
270 | * No DMA fences that signal end of batchbuffer for command submission where | |
271 | userspace is allowed to use userspace fencing or long running compute | |
272 | workloads. This also means no implicit fencing for shared buffers in these | |
273 | cases. | |
8613385c DV |
274 | |
275 | Recoverable Hardware Page Faults Implications | |
276 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
277 | ||
278 | Modern hardware supports recoverable page faults, which has a lot of | |
279 | implications for DMA fences. | |
280 | ||
281 | First, a pending page fault obviously holds up the work that's running on the | |
282 | accelerator and a memory allocation is usually required to resolve the fault. | |
283 | But memory allocations are not allowed to gate completion of DMA fences, which | |
284 | means any workload using recoverable page faults cannot use DMA fences for | |
285 | synchronization. Synchronization fences controlled by userspace must be used | |
286 | instead. | |
287 | ||
288 | On GPUs this poses a problem, because current desktop compositor protocols on | |
289 | Linux rely on DMA fences, which means without an entirely new userspace stack | |
290 | built on top of userspace fences, they cannot benefit from recoverable page | |
291 | faults. Specifically this means implicit synchronization will not be possible. | |
292 | The exception is when page faults are only used as migration hints and never to | |
293 | on-demand fill a memory request. For now this means recoverable page | |
294 | faults on GPUs are limited to pure compute workloads. | |
295 | ||
296 | Furthermore GPUs usually have shared resources between the 3D rendering and | |
297 | compute side, like compute units or command submission engines. If both a 3D | |
298 | job with a DMA fence and a compute workload using recoverable page faults are | |
299 | pending they could deadlock: | |
300 | ||
301 | - The 3D workload might need to wait for the compute job to finish and release | |
302 | hardware resources first. | |
303 | ||
304 | - The compute workload might be stuck in a page fault, because the memory | |
305 | allocation is waiting for the DMA fence of the 3D workload to complete. | |
306 | ||
307 | There are a few options to prevent this problem, one of which drivers need to | |
308 | ensure: | |
309 | ||
310 | - Compute workloads can always be preempted, even when a page fault is pending | |
311 | and not yet repaired. Not all hardware supports this. | |
312 | ||
313 | - DMA fence workloads and workloads which need page fault handling have | |
314 | independent hardware resources to guarantee forward progress. This could be | |
315 | achieved through e.g. through dedicated engines and minimal compute unit | |
316 | reservations for DMA fence workloads. | |
317 | ||
318 | - The reservation approach could be further refined by only reserving the | |
319 | hardware resources for DMA fence workloads when they are in-flight. This must | |
320 | cover the time from when the DMA fence is visible to other threads up to | |
321 | moment when fence is completed through dma_fence_signal(). | |
322 | ||
323 | - As a last resort, if the hardware provides no useful reservation mechanics, | |
324 | all workloads must be flushed from the GPU when switching between jobs | |
325 | requiring DMA fences or jobs requiring page fault handling: This means all DMA | |
326 | fences must complete before a compute job with page fault handling can be | |
327 | inserted into the scheduler queue. And vice versa, before a DMA fence can be | |
328 | made visible anywhere in the system, all compute workloads must be preempted | |
329 | to guarantee all pending GPU page faults are flushed. | |
330 | ||
331 | - Only a fairly theoretical option would be to untangle these dependencies when | |
332 | allocating memory to repair hardware page faults, either through separate | |
333 | memory blocks or runtime tracking of the full dependency graph of all DMA | |
334 | fences. This results very wide impact on the kernel, since resolving the page | |
335 | on the CPU side can itself involve a page fault. It is much more feasible and | |
336 | robust to limit the impact of handling hardware page faults to the specific | |
337 | driver. | |
338 | ||
339 | Note that workloads that run on independent hardware like copy engines or other | |
340 | GPUs do not have any impact. This allows us to keep using DMA fences internally | |
341 | in the kernel even for resolving hardware page faults, e.g. by using copy | |
342 | engines to clear or copy memory needed to resolve the page fault. | |
343 | ||
344 | In some ways this page fault problem is a special case of the `Infinite DMA | |
345 | Fences` discussions: Infinite fences from compute workloads are allowed to | |
346 | depend on DMA fences, but not the other way around. And not even the page fault | |
347 | problem is new, because some other CPU thread in userspace might | |
348 | hit a page fault which holds up a userspace fence - supporting page faults on | |
349 | GPUs doesn't anything fundamentally new. |