Commit | Line | Data |
---|---|---|
15fc39ae TT |
1 | .. include:: <isonum.txt> |
2 | ||
3 | Qualcomm Camera Subsystem driver | |
4 | ================================ | |
5 | ||
6 | Introduction | |
7 | ------------ | |
8 | ||
9 | This file documents the Qualcomm Camera Subsystem driver located under | |
10 | drivers/media/platform/qcom/camss-8x16. | |
11 | ||
12 | The current version of the driver supports the Camera Subsystem found on | |
13 | Qualcomm MSM8916 and APQ8016 processors. | |
14 | ||
15 | The driver implements V4L2, Media controller and V4L2 subdev interfaces. | |
16 | Camera sensor using V4L2 subdev interface in the kernel is supported. | |
17 | ||
18 | The driver is implemented using as a reference the Qualcomm Camera Subsystem | |
19 | driver for Android as found in Code Aurora [#f1]_. | |
20 | ||
21 | ||
22 | Qualcomm Camera Subsystem hardware | |
23 | ---------------------------------- | |
24 | ||
25 | The Camera Subsystem hardware found on 8x16 processors and supported by the | |
26 | driver consists of: | |
27 | ||
28 | - 2 CSIPHY modules. They handle the Physical layer of the CSI2 receivers. | |
29 | A separate camera sensor can be connected to each of the CSIPHY module; | |
30 | - 2 CSID (CSI Decoder) modules. They handle the Protocol and Application layer | |
31 | of the CSI2 receivers. A CSID can decode data stream from any of the CSIPHY. | |
32 | Each CSID also contains a TG (Test Generator) block which can generate | |
33 | artificial input data for test purposes; | |
34 | - ISPIF (ISP Interface) module. Handles the routing of the data streams from | |
35 | the CSIDs to the inputs of the VFE; | |
36 | - VFE (Video Front End) module. Contains a pipeline of image processing hardware | |
55e6efe0 TT |
37 | blocks. The VFE has different input interfaces. The PIX (Pixel) input |
38 | interface feeds the input data to the image processing pipeline. The image | |
39 | processing pipeline contains also a scale and crop module at the end. Three | |
40 | RDI (Raw Dump Interface) input interfaces bypass the image processing | |
41 | pipeline. The VFE also contains the AXI bus interface which writes the output | |
42 | data to memory. | |
15fc39ae TT |
43 | |
44 | ||
45 | Supported functionality | |
46 | ----------------------- | |
47 | ||
48 | The current version of the driver supports: | |
49 | ||
d226efcf TT |
50 | - Input from camera sensor via CSIPHY; |
51 | - Generation of test input data by the TG in CSID; | |
52 | - RDI interface of VFE - raw dump of the input data to memory. | |
53 | ||
54 | Supported formats: | |
55 | ||
56 | - YUYV/UYVY/YVYU/VYUY (packed YUV 4:2:2 - V4L2_PIX_FMT_YUYV / | |
57 | V4L2_PIX_FMT_UYVY / V4L2_PIX_FMT_YVYU / V4L2_PIX_FMT_VYUY); | |
58 | - MIPI RAW8 (8bit Bayer RAW - V4L2_PIX_FMT_SRGGB8 / | |
59 | V4L2_PIX_FMT_SGRBG8 / V4L2_PIX_FMT_SGBRG8 / V4L2_PIX_FMT_SBGGR8); | |
60 | - MIPI RAW10 (10bit packed Bayer RAW - V4L2_PIX_FMT_SBGGR10P / | |
61 | V4L2_PIX_FMT_SGBRG10P / V4L2_PIX_FMT_SGRBG10P / V4L2_PIX_FMT_SRGGB10P); | |
62 | - MIPI RAW12 (12bit packed Bayer RAW - V4L2_PIX_FMT_SRGGB12P / | |
63 | V4L2_PIX_FMT_SGBRG12P / V4L2_PIX_FMT_SGRBG12P / V4L2_PIX_FMT_SRGGB12P). | |
64 | ||
65 | - PIX interface of VFE | |
66 | ||
67 | - Format conversion of the input data. | |
68 | ||
69 | Supported input formats: | |
70 | ||
71 | - YUYV/UYVY/YVYU/VYUY (packed YUV 4:2:2 - V4L2_PIX_FMT_YUYV / | |
72 | V4L2_PIX_FMT_UYVY / V4L2_PIX_FMT_YVYU / V4L2_PIX_FMT_VYUY). | |
73 | ||
74 | Supported output formats: | |
75 | ||
76 | - NV12/NV21 (two plane YUV 4:2:0 - V4L2_PIX_FMT_NV12 / V4L2_PIX_FMT_NV21); | |
77 | - NV16/NV61 (two plane YUV 4:2:2 - V4L2_PIX_FMT_NV16 / V4L2_PIX_FMT_NV61). | |
78 | ||
474a53d6 TT |
79 | - Scaling support. Configuration of the VFE Encoder Scale module |
80 | for downscalling with ratio up to 16x. | |
81 | ||
82 | - Cropping support. Configuration of the VFE Encoder Crop module. | |
83 | ||
d226efcf | 84 | - Concurrent and independent usage of two data inputs - could be camera sensors |
15fc39ae TT |
85 | and/or TG. |
86 | ||
87 | ||
88 | Driver Architecture and Design | |
89 | ------------------------------ | |
90 | ||
91 | The driver implements the V4L2 subdev interface. With the goal to model the | |
92 | hardware links between the modules and to expose a clean, logical and usable | |
93 | interface, the driver is split into V4L2 sub-devices as follows: | |
94 | ||
95 | - 2 CSIPHY sub-devices - each CSIPHY is represented by a single sub-device; | |
96 | - 2 CSID sub-devices - each CSID is represented by a single sub-device; | |
97 | - 2 ISPIF sub-devices - ISPIF is represented by a number of sub-devices equal | |
98 | to the number of CSID sub-devices; | |
d226efcf TT |
99 | - 4 VFE sub-devices - VFE is represented by a number of sub-devices equal to |
100 | the number of the input interfaces (3 RDI and 1 PIX). | |
15fc39ae TT |
101 | |
102 | The considerations to split the driver in this particular way are as follows: | |
103 | ||
104 | - representing CSIPHY and CSID modules by a separate sub-device for each module | |
105 | allows to model the hardware links between these modules; | |
d226efcf TT |
106 | - representing VFE by a separate sub-devices for each input interface allows |
107 | to use the input interfaces concurently and independently as this is | |
15fc39ae TT |
108 | supported by the hardware; |
109 | - representing ISPIF by a number of sub-devices equal to the number of CSID | |
110 | sub-devices allows to create linear media controller pipelines when using two | |
111 | cameras simultaneously. This avoids branches in the pipelines which otherwise | |
112 | will require a) userspace and b) media framework (e.g. power on/off | |
113 | operations) to make assumptions about the data flow from a sink pad to a | |
114 | source pad on a single media entity. | |
115 | ||
116 | Each VFE sub-device is linked to a separate video device node. | |
117 | ||
4dfbd3d9 TT |
118 | The media controller pipeline graph is as follows (with connected two OV5645 |
119 | camera sensors): | |
120 | ||
121 | .. _qcom_camss_graph: | |
122 | ||
123 | .. kernel-figure:: qcom_camss_graph.dot | |
124 | :alt: qcom_camss_graph.dot | |
125 | :align: center | |
126 | ||
127 | Media pipeline graph | |
15fc39ae TT |
128 | |
129 | ||
130 | Implementation | |
131 | -------------- | |
132 | ||
133 | Runtime configuration of the hardware (updating settings while streaming) is | |
134 | not required to implement the currently supported functionality. The complete | |
135 | configuration on each hardware module is applied on STREAMON ioctl based on | |
136 | the current active media links, formats and controls set. | |
137 | ||
474a53d6 TT |
138 | The output size of the scaler module in the VFE is configured with the actual |
139 | compose selection rectangle on the sink pad of the 'msm_vfe0_pix' entity. | |
140 | ||
141 | The crop output area of the crop module in the VFE is configured with the actual | |
142 | crop selection rectangle on the source pad of the 'msm_vfe0_pix' entity. | |
143 | ||
15fc39ae TT |
144 | |
145 | Documentation | |
146 | ------------- | |
147 | ||
148 | APQ8016 Specification: | |
149 | https://developer.qualcomm.com/download/sd410/snapdragon-410-processor-device-specification.pdf | |
150 | Referenced 2016-11-24. | |
151 | ||
152 | ||
153 | References | |
154 | ---------- | |
155 | ||
156 | .. [#f1] https://source.codeaurora.org/quic/la/kernel/msm-3.10/ |