Commit | Line | Data |
---|---|---|
094f3910 DS |
1 | ======================= |
2 | Linux UVC Gadget Driver | |
3 | ======================= | |
4 | ||
5 | Overview | |
6 | -------- | |
7 | The UVC Gadget driver is a driver for hardware on the *device* side of a USB | |
8 | connection. It is intended to run on a Linux system that has USB device-side | |
9 | hardware such as boards with an OTG port. | |
10 | ||
11 | On the device system, once the driver is bound it appears as a V4L2 device with | |
12 | the output capability. | |
13 | ||
14 | On the host side (once connected via USB cable), a device running the UVC Gadget | |
15 | driver *and controlled by an appropriate userspace program* should appear as a UVC | |
16 | specification compliant camera, and function appropriately with any program | |
17 | designed to handle them. The userspace program running on the device system can | |
18 | queue image buffers from a variety of sources to be transmitted via the USB | |
19 | connection. Typically this would mean forwarding the buffers from a camera sensor | |
20 | peripheral, but the source of the buffer is entirely dependent on the userspace | |
21 | companion program. | |
22 | ||
23 | Configuring the device kernel | |
24 | ----------------------------- | |
25 | The Kconfig options USB_CONFIGFS, USB_LIBCOMPOSITE, USB_CONFIGFS_F_UVC and | |
26 | USB_F_UVC must be selected to enable support for the UVC gadget. | |
27 | ||
28 | Configuring the gadget through configfs | |
29 | --------------------------------------- | |
30 | The UVC Gadget expects to be configured through configfs using the UVC function. | |
31 | This allows a significant degree of flexibility, as many of a UVC device's | |
32 | settings can be controlled this way. | |
33 | ||
34 | Not all of the available attributes are described here. For a complete enumeration | |
35 | see Documentation/ABI/testing/configfs-usb-gadget-uvc | |
36 | ||
37 | Assumptions | |
38 | ~~~~~~~~~~~ | |
39 | This section assumes that you have mounted configfs at `/sys/kernel/config` and | |
40 | created a gadget as `/sys/kernel/config/usb_gadget/g1`. | |
41 | ||
42 | The UVC Function | |
43 | ~~~~~~~~~~~~~~~~ | |
44 | ||
45 | The first step is to create the UVC function: | |
46 | ||
47 | .. code-block:: bash | |
48 | ||
49 | # These variables will be assumed throughout the rest of the document | |
50 | CONFIGFS="/sys/kernel/config" | |
51 | GADGET="$CONFIGFS/usb_gadget/g1" | |
52 | FUNCTION="$GADGET/functions/uvc.0" | |
53 | ||
54 | mkdir -p $FUNCTION | |
55 | ||
56 | Formats and Frames | |
57 | ~~~~~~~~~~~~~~~~~~ | |
58 | ||
59 | You must configure the gadget by telling it which formats you support, as well | |
60 | as the frame sizes and frame intervals that are supported for each format. In | |
61 | the current implementation there is no way for the gadget to refuse to set a | |
62 | format that the host instructs it to set, so it is important that this step is | |
63 | completed *accurately* to ensure that the host never asks for a format that | |
64 | can't be provided. | |
65 | ||
66 | Formats are created under the streaming/uncompressed and streaming/mjpeg configfs | |
67 | groups, with the framesizes created under the formats in the following | |
68 | structure: | |
69 | ||
70 | :: | |
71 | ||
72 | uvc.0 + | |
73 | | | |
74 | + streaming + | |
75 | | | |
76 | + mjpeg + | |
77 | | | | |
78 | | + mjpeg + | |
79 | | | | |
80 | | + 720p | |
81 | | | | |
82 | | + 1080p | |
83 | | | |
84 | + uncompressed + | |
85 | | | |
86 | + yuyv + | |
87 | | | |
88 | + 720p | |
89 | | | |
90 | + 1080p | |
91 | ||
92 | Each frame can then be configured with a width and height, plus the maximum | |
93 | buffer size required to store a single frame, and finally with the supported | |
94 | frame intervals for that format and framesize. Width and height are enumerated in | |
95 | units of pixels, frame interval in units of 100ns. To create the structure | |
96 | above with 2, 15 and 100 fps frameintervals for each framesize for example you | |
97 | might do: | |
98 | ||
99 | .. code-block:: bash | |
100 | ||
101 | create_frame() { | |
102 | # Example usage: | |
103 | # create_frame <width> <height> <group> <format name> | |
104 | ||
105 | WIDTH=$1 | |
106 | HEIGHT=$2 | |
107 | FORMAT=$3 | |
108 | NAME=$4 | |
109 | ||
110 | wdir=$FUNCTION/streaming/$FORMAT/$NAME/${HEIGHT}p | |
111 | ||
112 | mkdir -p $wdir | |
113 | echo $WIDTH > $wdir/wWidth | |
114 | echo $HEIGHT > $wdir/wHeight | |
115 | echo $(( $WIDTH * $HEIGHT * 2 )) > $wdir/dwMaxVideoFrameBufferSize | |
116 | cat <<EOF > $wdir/dwFrameInterval | |
117 | 666666 | |
118 | 100000 | |
119 | 5000000 | |
120 | EOF | |
121 | } | |
122 | ||
123 | create_frame 1280 720 mjpeg mjpeg | |
124 | create_frame 1920 1080 mjpeg mjpeg | |
125 | create_frame 1280 720 uncompressed yuyv | |
126 | create_frame 1920 1080 uncompressed yuyv | |
127 | ||
128 | The only uncompressed format currently supported is YUYV, which is detailed at | |
129 | Documentation/userspace-api/media/v4l/pixfmt-packed.yuv.rst. | |
130 | ||
131 | Color Matching Descriptors | |
132 | ~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
133 | It's possible to specify some colometry information for each format you create. | |
134 | This step is optional, and default information will be included if this step is | |
135 | skipped; those default values follow those defined in the Color Matching Descriptor | |
136 | section of the UVC specification. | |
137 | ||
138 | To create a Color Matching Descriptor, create a configfs item and set its three | |
139 | attributes to your desired settings and then link to it from the format you wish | |
140 | it to be associated with: | |
141 | ||
142 | .. code-block:: bash | |
143 | ||
144 | # Create a new Color Matching Descriptor | |
145 | ||
146 | mkdir $FUNCTION/streaming/color_matching/yuyv | |
147 | pushd $FUNCTION/streaming/color_matching/yuyv | |
148 | ||
149 | echo 1 > bColorPrimaries | |
150 | echo 1 > bTransferCharacteristics | |
151 | echo 4 > bMatrixCoefficients | |
152 | ||
153 | popd | |
154 | ||
155 | # Create a symlink to the Color Matching Descriptor from the format's config item | |
156 | ln -s $FUNCTION/streaming/color_matching/yuyv $FUNCTION/streaming/uncompressed/yuyv | |
157 | ||
158 | For details about the valid values, consult the UVC specification. Note that a | |
159 | default color matching descriptor exists and is used by any format which does | |
160 | not have a link to a different Color Matching Descriptor. It's possible to | |
161 | change the attribute settings for the default descriptor, so bear in mind that if | |
162 | you do that you are altering the defaults for any format that does not link to | |
163 | a different one. | |
164 | ||
165 | ||
166 | Header linking | |
167 | ~~~~~~~~~~~~~~ | |
168 | ||
169 | The UVC specification requires that Format and Frame descriptors be preceded by | |
170 | Headers detailing things such as the number and cumulative size of the different | |
171 | Format descriptors that follow. This and similar operations are acheived in | |
172 | configfs by linking between the configfs item representing the header and the | |
173 | config items representing those other descriptors, in this manner: | |
174 | ||
175 | .. code-block:: bash | |
176 | ||
177 | mkdir $FUNCTION/streaming/header/h | |
178 | ||
179 | # This section links the format descriptors and their associated frames | |
180 | # to the header | |
181 | cd $FUNCTION/streaming/header/h | |
182 | ln -s ../../uncompressed/yuyv | |
183 | ln -s ../../mjpeg/mjpeg | |
184 | ||
185 | # This section ensures that the header will be transmitted for each | |
186 | # speed's set of descriptors. If support for a particular speed is not | |
187 | # needed then it can be skipped here. | |
188 | cd ../../class/fs | |
189 | ln -s ../../header/h | |
190 | cd ../../class/hs | |
191 | ln -s ../../header/h | |
192 | cd ../../class/ss | |
193 | ln -s ../../header/h | |
194 | cd ../../../control | |
195 | mkdir header/h | |
196 | ln -s header/h class/fs | |
197 | ln -s header/h class/ss | |
198 | ||
199 | ||
200 | Extension Unit Support | |
201 | ~~~~~~~~~~~~~~~~~~~~~~ | |
202 | ||
203 | A UVC Extension Unit (XU) basically provides a distinct unit to which control set | |
204 | and get requests can be addressed. The meaning of those control requests is | |
205 | entirely implementation dependent, but may be used to control settings outside | |
206 | of the UVC specification (for example enabling or disabling video effects). An | |
207 | XU can be inserted into the UVC unit chain or left free-hanging. | |
208 | ||
209 | Configuring an extension unit involves creating an entry in the appropriate | |
210 | directory and setting its attributes appropriately, like so: | |
211 | ||
212 | .. code-block:: bash | |
213 | ||
214 | mkdir $FUNCTION/control/extensions/xu.0 | |
215 | pushd $FUNCTION/control/extensions/xu.0 | |
216 | ||
217 | # Set the bUnitID of the Processing Unit as the source for this | |
218 | # Extension Unit | |
219 | echo 2 > baSourceID | |
220 | ||
221 | # Set this XU as the source of the default output terminal. This inserts | |
222 | # the XU into the UVC chain between the PU and OT such that the final | |
223 | # chain is IT > PU > XU.0 > OT | |
224 | cat bUnitID > ../../terminal/output/default/baSourceID | |
225 | ||
226 | # Flag some controls as being available for use. The bmControl field is | |
227 | # a bitmap with each bit denoting the availability of a particular | |
228 | # control. For example to flag the 0th, 2nd and 3rd controls available: | |
229 | echo 0x0d > bmControls | |
230 | ||
231 | # Set the GUID; this is a vendor-specific code identifying the XU. | |
232 | echo -e -n "\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10" > guidExtensionCode | |
233 | ||
234 | popd | |
235 | ||
236 | The bmControls attribute and the baSourceID attribute are multi-value attributes. | |
237 | This means that you may write multiple newline separated values to them. For | |
238 | example to flag the 1st, 2nd, 9th and 10th controls as being available you would | |
239 | need to write two values to bmControls, like so: | |
240 | ||
241 | .. code-block:: bash | |
242 | ||
243 | cat << EOF > bmControls | |
244 | 0x03 | |
245 | 0x03 | |
246 | EOF | |
247 | ||
248 | The multi-value nature of the baSourceID attribute belies the fact that XUs can | |
249 | be multiple-input, though note that this currently has no significant effect. | |
250 | ||
251 | The bControlSize attribute reflects the size of the bmControls attribute, and | |
252 | similarly bNrInPins reflects the size of the baSourceID attributes. Both | |
253 | attributes are automatically increased / decreased as you set bmControls and | |
254 | baSourceID. It is also possible to manually increase or decrease bControlSize | |
255 | which has the effect of truncating entries to the new size, or padding entries | |
256 | out with 0x00, for example: | |
257 | ||
258 | :: | |
259 | ||
260 | $ cat bmControls | |
261 | 0x03 | |
262 | 0x05 | |
263 | ||
264 | $ cat bControlSize | |
265 | 2 | |
266 | ||
267 | $ echo 1 > bControlSize | |
268 | $ cat bmControls | |
269 | 0x03 | |
270 | ||
271 | $ echo 2 > bControlSize | |
272 | $ cat bmControls | |
273 | 0x03 | |
274 | 0x00 | |
275 | ||
276 | bNrInPins and baSourceID function in the same way. | |
277 | ||
7bf1c56a DS |
278 | Configuring Supported Controls for Camera Terminal and Processing Unit |
279 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
280 | ||
281 | The Camera Terminal and Processing Units in the UVC chain also have bmControls | |
282 | attributes which function similarly to the same field in an Extension Unit. | |
283 | Unlike XUs however, the meaning of the bitflag for these units is defined in | |
284 | the UVC specification; you should consult the "Camera Terminal Descriptor" and | |
285 | "Processing Unit Descriptor" sections for an enumeration of the flags. | |
286 | ||
287 | .. code-block:: bash | |
288 | ||
289 | # Set the Processing Unit's bmControls, flagging Brightness, Contrast | |
290 | # and Hue as available controls: | |
291 | echo 0x05 > $FUNCTION/control/processing/default/bmControls | |
292 | ||
293 | # Set the Camera Terminal's bmControls, flagging Focus Absolute and | |
294 | # Focus Relative as available controls: | |
295 | echo 0x60 > $FUNCTION/control/terminal/camera/default/bmControls | |
296 | ||
297 | If you do not set these fields then by default the Auto-Exposure Mode control | |
298 | for the Camera Terminal and the Brightness control for the Processing Unit will | |
299 | be flagged as available; if they are not supported you should set the field to | |
300 | 0x00. | |
301 | ||
302 | Note that the size of the bmControls field for a Camera Terminal or Processing | |
303 | Unit is fixed by the UVC specification, and so the bControlSize attribute is | |
304 | read-only here. | |
305 | ||
094f3910 DS |
306 | Custom Strings Support |
307 | ~~~~~~~~~~~~~~~~~~~~~~ | |
308 | ||
309 | String descriptors that provide a textual description for various parts of a | |
310 | USB device can be defined in the usual place within USB configfs, and may then | |
311 | be linked to from the UVC function root or from Extension Unit directories to | |
312 | assign those strings as descriptors: | |
313 | ||
314 | .. code-block:: bash | |
315 | ||
316 | # Create a string descriptor in us-EN and link to it from the function | |
317 | # root. The name of the link is significant here, as it declares this | |
318 | # descriptor to be intended for the Interface Association Descriptor. | |
319 | # Other significant link names at function root are vs0_desc and vs1_desc | |
320 | # For the VideoStreaming Interface 0/1 Descriptors. | |
321 | ||
322 | mkdir -p $GADGET/strings/0x409/iad_desc | |
323 | echo -n "Interface Associaton Descriptor" > $GADGET/strings/0x409/iad_desc/s | |
324 | ln -s $GADGET/strings/0x409/iad_desc $FUNCTION/iad_desc | |
325 | ||
326 | # Because the link to a String Descriptor from an Extension Unit clearly | |
327 | # associates the two, the name of this link is not significant and may | |
328 | # be set freely. | |
329 | ||
330 | mkdir -p $GADGET/strings/0x409/xu.0 | |
331 | echo -n "A Very Useful Extension Unit" > $GADGET/strings/0x409/xu.0/s | |
332 | ln -s $GADGET/strings/0x409/xu.0 $FUNCTION/control/extensions/xu.0 | |
333 | ||
334 | The interrupt endpoint | |
335 | ~~~~~~~~~~~~~~~~~~~~~~ | |
336 | ||
337 | The VideoControl interface has an optional interrupt endpoint which is by default | |
338 | disabled. This is intended to support delayed response control set requests for | |
339 | UVC (which should respond through the interrupt endpoint rather than tying up | |
340 | endpoint 0). At present support for sending data through this endpoint is missing | |
341 | and so it is left disabled to avoid confusion. If you wish to enable it you can | |
342 | do so through the configfs attribute: | |
343 | ||
344 | .. code-block:: bash | |
345 | ||
346 | echo 1 > $FUNCTION/control/enable_interrupt_ep | |
347 | ||
348 | Bandwidth configuration | |
349 | ~~~~~~~~~~~~~~~~~~~~~~~ | |
350 | ||
351 | There are three attributes which control the bandwidth of the USB connection. | |
352 | These live in the function root and can be set within limits: | |
353 | ||
354 | .. code-block:: bash | |
355 | ||
356 | # streaming_interval sets bInterval. Values range from 1..255 | |
357 | echo 1 > $FUNCTION/streaming_interval | |
358 | ||
359 | # streaming_maxpacket sets wMaxPacketSize. Valid values are 1024/2048/3072 | |
360 | echo 3072 > $FUNCTION/streaming_maxpacket | |
361 | ||
362 | # streaming_maxburst sets bMaxBurst. Valid values are 1..15 | |
363 | echo 1 > $FUNCTION/streaming_maxburst | |
364 | ||
365 | ||
366 | The values passed here will be clamped to valid values according to the UVC | |
367 | specification (which depend on the speed of the USB connection). To understand | |
368 | how the settings influence bandwidth you should consult the UVC specifications, | |
369 | but a rule of thumb is that increasing the streaming_maxpacket setting will | |
370 | improve bandwidth (and thus the maximum possible framerate), whilst the same is | |
371 | true for streaming_maxburst provided the USB connection is running at SuperSpeed. | |
372 | Increasing streaming_interval will reduce bandwidth and framerate. | |
373 | ||
374 | The userspace application | |
375 | ------------------------- | |
376 | By itself, the UVC Gadget driver cannot do anything particularly interesting. It | |
377 | must be paired with a userspace program that responds to UVC control requests and | |
378 | fills buffers to be queued to the V4L2 device that the driver creates. How those | |
379 | things are achieved is implementation dependent and beyond the scope of this | |
380 | document, but a reference application can be found at https://gitlab.freedesktop.org/camera/uvc-gadget |