Commit | Line | Data |
---|---|---|
e4cf8c58 SA |
1 | .. SPDX-License-Identifier: GPL-2.0 |
2 | ||
3 | Writing camera sensor drivers | |
4 | ============================= | |
5 | ||
6 | CSI-2 | |
7 | ----- | |
8 | ||
9 | Please see what is written on :ref:`MIPI_CSI_2`. | |
10 | ||
11 | Handling clocks | |
12 | --------------- | |
13 | ||
14 | Camera sensors have an internal clock tree including a PLL and a number of | |
15 | divisors. The clock tree is generally configured by the driver based on a few | |
16 | input parameters that are specific to the hardware:: the external clock frequency | |
17 | and the link frequency. The two parameters generally are obtained from system | |
2225cf44 | 18 | firmware. **No other frequencies should be used in any circumstances.** |
e4cf8c58 SA |
19 | |
20 | The reason why the clock frequencies are so important is that the clock signals | |
21 | come out of the SoC, and in many cases a specific frequency is designed to be | |
22 | used in the system. Using another frequency may cause harmful effects | |
23 | elsewhere. Therefore only the pre-determined frequencies are configurable by the | |
24 | user. | |
25 | ||
2225cf44 SA |
26 | ACPI |
27 | ~~~~ | |
28 | ||
29 | Read the "clock-frequency" _DSD property to denote the frequency. The driver can | |
30 | rely on this frequency being used. | |
31 | ||
32 | Devicetree | |
33 | ~~~~~~~~~~ | |
34 | ||
35 | The currently preferred way to achieve this is using "assigned-clock-rates" | |
36 | property. See Documentation/devicetree/bindings/clock/clock-bindings.txt for | |
37 | more information. The driver then gets the frequency using clk_get_rate(). | |
38 | ||
39 | This approach has the drawback that there's no guarantee that the frequency | |
40 | hasn't been modified directly or indirectly by another driver, or supported by | |
41 | the board's clock tree to begin with. Changes to the Common Clock Framework API | |
42 | are required to ensure reliability. | |
43 | ||
e4cf8c58 SA |
44 | Frame size |
45 | ---------- | |
46 | ||
47 | There are two distinct ways to configure the frame size produced by camera | |
48 | sensors. | |
49 | ||
50 | Freely configurable camera sensor drivers | |
51 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
52 | ||
53 | Freely configurable camera sensor drivers expose the device's internal | |
54 | processing pipeline as one or more sub-devices with different cropping and | |
55 | scaling configurations. The output size of the device is the result of a series | |
56 | of cropping and scaling operations from the device's pixel array's size. | |
57 | ||
58 | An example of such a driver is the smiapp driver (see drivers/media/i2c/smiapp). | |
59 | ||
60 | Register list based drivers | |
61 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
62 | ||
63 | Register list based drivers generally, instead of able to configure the device | |
64 | they control based on user requests, are limited to a number of preset | |
65 | configurations that combine a number of different parameters that on hardware | |
66 | level are independent. How a driver picks such configuration is based on the | |
67 | format set on a source pad at the end of the device's internal pipeline. | |
68 | ||
69 | Most sensor drivers are implemented this way, see e.g. | |
70 | drivers/media/i2c/imx319.c for an example. | |
71 | ||
72 | Frame interval configuration | |
73 | ---------------------------- | |
74 | ||
75 | There are two different methods for obtaining possibilities for different frame | |
76 | intervals as well as configuring the frame interval. Which one to implement | |
77 | depends on the type of the device. | |
78 | ||
79 | Raw camera sensors | |
80 | ~~~~~~~~~~~~~~~~~~ | |
81 | ||
82 | Instead of a high level parameter such as frame interval, the frame interval is | |
83 | a result of the configuration of a number of camera sensor implementation | |
84 | specific parameters. Luckily, these parameters tend to be the same for more or | |
85 | less all modern raw camera sensors. | |
86 | ||
87 | The frame interval is calculated using the following equation:: | |
88 | ||
89 | frame interval = (analogue crop width + horizontal blanking) * | |
90 | (analogue crop height + vertical blanking) / pixel rate | |
91 | ||
92 | The formula is bus independent and is applicable for raw timing parameters on | |
93 | large variety of devices beyond camera sensors. Devices that have no analogue | |
94 | crop, use the full source image size, i.e. pixel array size. | |
95 | ||
96 | Horizontal and vertical blanking are specified by ``V4L2_CID_HBLANK`` and | |
97 | ``V4L2_CID_VBLANK``, respectively. The unit of these controls are lines. The | |
98 | pixel rate is specified by ``V4L2_CID_PIXEL_RATE`` in the same sub-device. The | |
99 | unit of that control is Hz. | |
100 | ||
101 | Register list based drivers need to implement read-only sub-device nodes for the | |
102 | purpose. Devices that are not register list based need these to configure the | |
103 | device's internal processing pipeline. | |
104 | ||
105 | The first entity in the linear pipeline is the pixel array. The pixel array may | |
106 | be followed by other entities that are there to allow configuring binning, | |
107 | skipping, scaling or digital crop :ref:`v4l2-subdev-selections`. | |
108 | ||
109 | USB cameras etc. devices | |
110 | ~~~~~~~~~~~~~~~~~~~~~~~~ | |
111 | ||
112 | USB video class hardware, as well as many cameras offering a similar higher | |
113 | level interface natively, generally use the concept of frame interval (or frame | |
114 | rate) on device level in firmware or hardware. This means lower level controls | |
115 | implemented by raw cameras may not be used on uAPI (or even kAPI) to control the | |
116 | frame interval on these devices. | |
117 | ||
118 | Power management | |
119 | ---------------- | |
120 | ||
121 | Always use runtime PM to manage the power states of your device. Camera sensor | |
122 | drivers are in no way special in this respect: they are responsible for | |
123 | controlling the power state of the device they otherwise control as well. In | |
124 | general, the device must be powered on at least when its registers are being | |
125 | accessed and when it is streaming. | |
126 | ||
127 | Existing camera sensor drivers may rely on the old | |
128 | :c:type:`v4l2_subdev_core_ops`->s_power() callback for bridge or ISP drivers to | |
129 | manage their power state. This is however **deprecated**. If you feel you need | |
130 | to begin calling an s_power from an ISP or a bridge driver, instead please add | |
131 | runtime PM support to the sensor driver you are using. Likewise, new drivers | |
132 | should not use s_power. | |
133 | ||
134 | Please see examples in e.g. ``drivers/media/i2c/ov8856.c`` and | |
135 | ``drivers/media/i2c/smiapp/smiapp-core.c``. The two drivers work in both ACPI | |
136 | and DT based systems. | |
137 | ||
138 | Control framework | |
139 | ~~~~~~~~~~~~~~~~~ | |
140 | ||
141 | ``v4l2_ctrl_handler_setup()`` function may not be used in the device's runtime | |
142 | PM ``runtime_resume`` callback, as it has no way to figure out the power state | |
143 | of the device. This is because the power state of the device is only changed | |
6fcadfc7 | 144 | after the power state transition has taken place. The ``s_ctrl`` callback can be |
e4cf8c58 SA |
145 | used to obtain device's power state after the power state transition: |
146 | ||
f993b298 | 147 | .. c:function:: int pm_runtime_get_if_in_use(struct device *dev); |
e4cf8c58 SA |
148 | |
149 | The function returns a non-zero value if it succeeded getting the power count or | |
150 | runtime PM was disabled, in either of which cases the driver may proceed to | |
151 | access the device. | |
9d0a79e5 SA |
152 | |
153 | Controls | |
154 | -------- | |
155 | ||
156 | For camera sensors that are connected to a bus where transmitter and receiver | |
157 | require common configuration set by drivers, such as CSI-2 or parallel (BT.601 | |
158 | or BT.656) bus, the ``V4L2_CID_LINK_FREQ`` control is mandatory on transmitter | |
159 | drivers. Receiver drivers can use the ``V4L2_CID_LINK_FREQ`` to query the | |
160 | frequency used on the bus. | |
161 | ||
162 | The transmitter drivers should also implement ``V4L2_CID_PIXEL_RATE`` control in | |
163 | order to tell the maximum pixel rate to the receiver. This is required on raw | |
164 | camera sensors. |