Commit | Line | Data |
---|---|---|
e228b1e6 NP |
1 | Copyright 2010 Nicolas Palix <npalix@diku.dk> |
2 | Copyright 2010 Julia Lawall <julia@diku.dk> | |
3 | Copyright 2010 Gilles Muller <Gilles.Muller@lip6.fr> | |
4 | ||
5 | ||
6 | Getting Coccinelle | |
7 | ~~~~~~~~~~~~~~~~~~~~ | |
8 | ||
ec97946e NP |
9 | The semantic patches included in the kernel use features and options |
10 | which are provided by Coccinelle version 1.0.0-rc11 and above. | |
11 | Using earlier versions will fail as the option names used by | |
12 | the Coccinelle files and coccicheck have been updated. | |
e228b1e6 | 13 | |
ec97946e | 14 | Coccinelle is available through the package manager |
e228b1e6 NP |
15 | of many distributions, e.g. : |
16 | ||
ec97946e NP |
17 | - Debian |
18 | - Fedora | |
19 | - Ubuntu | |
e228b1e6 NP |
20 | - OpenSUSE |
21 | - Arch Linux | |
22 | - NetBSD | |
23 | - FreeBSD | |
24 | ||
25 | ||
26 | You can get the latest version released from the Coccinelle homepage at | |
27 | http://coccinelle.lip6.fr/ | |
28 | ||
32af0898 NP |
29 | Information and tips about Coccinelle are also provided on the wiki |
30 | pages at http://cocci.ekstranet.diku.dk/wiki/doku.php | |
31 | ||
e228b1e6 NP |
32 | Once you have it, run the following command: |
33 | ||
34 | ./configure | |
35 | make | |
36 | ||
37 | as a regular user, and install it with | |
38 | ||
39 | sudo make install | |
40 | ||
e228b1e6 NP |
41 | Using Coccinelle on the Linux kernel |
42 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
43 | ||
44 | A Coccinelle-specific target is defined in the top level | |
45 | Makefile. This target is named 'coccicheck' and calls the 'coccicheck' | |
46 | front-end in the 'scripts' directory. | |
47 | ||
78a95b9b | 48 | Four basic modes are defined: patch, report, context, and org. The mode to |
e228b1e6 NP |
49 | use is specified by setting the MODE variable with 'MODE=<mode>'. |
50 | ||
32af0898 NP |
51 | 'patch' proposes a fix, when possible. |
52 | ||
e228b1e6 NP |
53 | 'report' generates a list in the following format: |
54 | file:line:column-column: message | |
55 | ||
e228b1e6 NP |
56 | 'context' highlights lines of interest and their context in a |
57 | diff-like style.Lines of interest are indicated with '-'. | |
58 | ||
59 | 'org' generates a report in the Org mode format of Emacs. | |
60 | ||
32af0898 | 61 | Note that not all semantic patches implement all modes. For easy use |
78a95b9b | 62 | of Coccinelle, the default mode is "report". |
e228b1e6 | 63 | |
78a95b9b | 64 | Two other modes provide some common combinations of these modes. |
e228b1e6 | 65 | |
78a95b9b | 66 | 'chain' tries the previous modes in the order above until one succeeds. |
e228b1e6 | 67 | |
78a95b9b NP |
68 | 'rep+ctxt' runs successively the report mode and the context mode. |
69 | It should be used with the C option (described later) | |
70 | which checks the code on a file basis. | |
e228b1e6 | 71 | |
78a95b9b NP |
72 | Examples: |
73 | To make a report for every semantic patch, run the following command: | |
e228b1e6 | 74 | |
78a95b9b NP |
75 | make coccicheck MODE=report |
76 | ||
77 | To produce patches, run: | |
78 | ||
79 | make coccicheck MODE=patch | |
e228b1e6 NP |
80 | |
81 | ||
82 | The coccicheck target applies every semantic patch available in the | |
32af0898 | 83 | sub-directories of 'scripts/coccinelle' to the entire Linux kernel. |
e228b1e6 | 84 | |
32af0898 | 85 | For each semantic patch, a commit message is proposed. It gives a |
e228b1e6 NP |
86 | description of the problem being checked by the semantic patch, and |
87 | includes a reference to Coccinelle. | |
88 | ||
89 | As any static code analyzer, Coccinelle produces false | |
90 | positives. Thus, reports must be carefully checked, and patches | |
91 | reviewed. | |
92 | ||
26e56720 BS |
93 | To enable verbose messages set the V= variable, for example: |
94 | ||
95 | make coccicheck MODE=report V=1 | |
96 | ||
c930a1b2 LR |
97 | Coccinelle parallelization |
98 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
99 | ||
90d06a46 KC |
100 | By default, coccicheck tries to run as parallel as possible. To change |
101 | the parallelism, set the J= variable. For example, to run across 4 CPUs: | |
102 | ||
103 | make coccicheck MODE=report J=4 | |
104 | ||
c930a1b2 LR |
105 | As of Coccinelle 1.0.2 Coccinelle uses Ocaml parmap for parallelization, |
106 | if support for this is detected you will benefit from parmap parallelization. | |
107 | ||
108 | When parmap is enabled coccicheck will enable dynamic load balancing by using | |
109 | '--chunksize 1' argument, this ensures we keep feeding threads with work | |
110 | one by one, so that we avoid the situation where most work gets done by only | |
111 | a few threads. With dynamic load balancing, if a thread finishes early we keep | |
112 | feeding it more work. | |
113 | ||
114 | When parmap is enabled, if an error occurs in Coccinelle, this error | |
115 | value is propagated back, the return value of the 'make coccicheck' | |
116 | captures this return value. | |
e228b1e6 NP |
117 | |
118 | Using Coccinelle with a single semantic patch | |
119 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
120 | ||
121 | The optional make variable COCCI can be used to check a single | |
122 | semantic patch. In that case, the variable must be initialized with | |
123 | the name of the semantic patch to apply. | |
124 | ||
125 | For instance: | |
126 | ||
127 | make coccicheck COCCI=<my_SP.cocci> MODE=patch | |
128 | or | |
129 | make coccicheck COCCI=<my_SP.cocci> MODE=report | |
130 | ||
131 | ||
f95ab209 GD |
132 | Controlling Which Files are Processed by Coccinelle |
133 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
134 | By default the entire kernel source tree is checked. | |
135 | ||
136 | To apply Coccinelle to a specific directory, M= can be used. | |
137 | For example, to check drivers/net/wireless/ one may write: | |
32af0898 | 138 | |
f95ab209 | 139 | make coccicheck M=drivers/net/wireless/ |
ed621cc4 | 140 | |
32af0898 NP |
141 | To apply Coccinelle on a file basis, instead of a directory basis, the |
142 | following command may be used: | |
143 | ||
144 | make C=1 CHECK="scripts/coccicheck" | |
145 | ||
146 | To check only newly edited code, use the value 2 for the C flag, i.e. | |
147 | ||
148 | make C=2 CHECK="scripts/coccicheck" | |
149 | ||
78a95b9b NP |
150 | In these modes, which works on a file basis, there is no information |
151 | about semantic patches displayed, and no commit message proposed. | |
152 | ||
32af0898 NP |
153 | This runs every semantic patch in scripts/coccinelle by default. The |
154 | COCCI variable may additionally be used to only apply a single | |
155 | semantic patch as shown in the previous section. | |
156 | ||
78a95b9b | 157 | The "report" mode is the default. You can select another one with the |
32af0898 NP |
158 | MODE variable explained above. |
159 | ||
be1fa900 LR |
160 | Debugging Coccinelle SmPL patches |
161 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
162 | ||
163 | Using coccicheck is best as it provides in the spatch command line | |
164 | include options matching the options used when we compile the kernel. | |
165 | You can learn what these options are by using V=1, you could then | |
166 | manually run Coccinelle with debug options added. | |
167 | ||
168 | Alternatively you can debug running Coccinelle against SmPL patches | |
169 | by asking for stderr to be redirected to stderr, by default stderr | |
170 | is redirected to /dev/null, if you'd like to capture stderr you | |
171 | can specify the DEBUG_FILE="file.txt" option to coccicheck. For | |
172 | instance: | |
173 | ||
174 | rm -f cocci.err | |
175 | make coccicheck COCCI=scripts/coccinelle/free/kfree.cocci MODE=report DEBUG_FILE=cocci.err | |
176 | cat cocci.err | |
177 | ||
5c384dba LR |
178 | You can use SPFLAGS to add debugging flags, for instance you may want to |
179 | add both --profile --show-trying to SPFLAGS when debugging. For instance | |
180 | you may want to use: | |
181 | ||
182 | rm -f err.log | |
183 | export COCCI=scripts/coccinelle/misc/irqf_oneshot.cocci | |
184 | make coccicheck DEBUG_FILE="err.log" MODE=report SPFLAGS="--profile --show-trying" M=./drivers/mfd/arizona-irq.c | |
185 | ||
186 | err.log will now have the profiling information, while stdout will | |
187 | provide some progress information as Coccinelle moves forward with | |
188 | work. | |
189 | ||
be1fa900 LR |
190 | DEBUG_FILE support is only supported when using coccinelle >= 1.2. |
191 | ||
dd951fc1 LR |
192 | .cocciconfig support |
193 | ~~~~~~~~~~~~~~~~~~~~~~ | |
194 | ||
195 | Coccinelle supports reading .cocciconfig for default Coccinelle options that | |
196 | should be used every time spatch is spawned, the order of precedence for | |
197 | variables for .cocciconfig is as follows: | |
198 | ||
199 | o Your current user's home directory is processed first | |
200 | o Your directory from which spatch is called is processed next | |
201 | o The directory provided with the --dir option is processed last, if used | |
202 | ||
203 | Since coccicheck runs through make, it naturally runs from the kernel | |
204 | proper dir, as such the second rule above would be implied for picking up a | |
205 | .cocciconfig when using 'make coccicheck'. | |
206 | ||
207 | 'make coccicheck' also supports using M= targets.If you do not supply | |
208 | any M= target, it is assumed you want to target the entire kernel. | |
209 | The kernel coccicheck script has: | |
210 | ||
211 | if [ "$KBUILD_EXTMOD" = "" ] ; then | |
212 | OPTIONS="--dir $srctree $COCCIINCLUDE" | |
213 | else | |
214 | OPTIONS="--dir $KBUILD_EXTMOD $COCCIINCLUDE" | |
215 | fi | |
216 | ||
217 | KBUILD_EXTMOD is set when an explicit target with M= is used. For both cases | |
218 | the spatch --dir argument is used, as such third rule applies when whether M= | |
219 | is used or not, and when M= is used the target directory can have its own | |
220 | .cocciconfig file. When M= is not passed as an argument to coccicheck the | |
221 | target directory is the same as the directory from where spatch was called. | |
222 | ||
223 | If not using the kernel's coccicheck target, keep the above precedence | |
224 | order logic of .cocciconfig reading. If using the kernel's coccicheck target, | |
225 | override any of the kernel's .coccicheck's settings using SPFLAGS. | |
226 | ||
227 | We help Coccinelle when used against Linux with a set of sensible defaults | |
228 | options for Linux with our own Linux .cocciconfig. This hints to coccinelle | |
229 | git can be used for 'git grep' queries over coccigrep. A timeout of 200 | |
230 | seconds should suffice for now. | |
231 | ||
232 | The options picked up by coccinelle when reading a .cocciconfig do not appear | |
233 | as arguments to spatch processes running on your system, to confirm what | |
234 | options will be used by Coccinelle run: | |
235 | ||
236 | spatch --print-options-only | |
237 | ||
238 | You can override with your own preferred index option by using SPFLAGS. Take | |
239 | note that when there are conflicting options Coccinelle takes precedence for | |
240 | the last options passed. Using .cocciconfig is possible to use idutils, however | |
241 | given the order of precedence followed by Coccinelle, since the kernel now | |
242 | carries its own .cocciconfig, you will need to use SPFLAGS to use idutils if | |
243 | desired. See below section "Additional flags" for more details on how to use | |
244 | idutils. | |
245 | ||
ed621cc4 NP |
246 | Additional flags |
247 | ~~~~~~~~~~~~~~~~~~ | |
248 | ||
249 | Additional flags can be passed to spatch through the SPFLAGS | |
8e826ad5 LR |
250 | variable. This works as Coccinelle respects the last flags |
251 | given to it when options are in conflict. | |
ed621cc4 | 252 | |
78a95b9b | 253 | make SPFLAGS=--use-glimpse coccicheck |
dd951fc1 LR |
254 | |
255 | Coccinelle supports idutils as well but requires coccinelle >= 1.0.6. | |
256 | When no ID file is specified coccinelle assumes your ID database file | |
257 | is in the file .id-utils.index on the top level of the kernel, coccinelle | |
258 | carries a script scripts/idutils_index.sh which creates the database with | |
259 | ||
260 | mkid -i C --output .id-utils.index | |
261 | ||
262 | If you have another database filename you can also just symlink with this | |
263 | name. | |
264 | ||
78a95b9b | 265 | make SPFLAGS=--use-idutils coccicheck |
ed621cc4 | 266 | |
dd951fc1 LR |
267 | Alternatively you can specify the database filename explicitly, for |
268 | instance: | |
269 | ||
270 | make SPFLAGS="--use-idutils /full-path/to/ID" coccicheck | |
271 | ||
ed621cc4 | 272 | See spatch --help to learn more about spatch options. |
32af0898 | 273 | |
78a95b9b NP |
274 | Note that the '--use-glimpse' and '--use-idutils' options |
275 | require external tools for indexing the code. None of them is | |
276 | thus active by default. However, by indexing the code with | |
277 | one of these tools, and according to the cocci file used, | |
278 | spatch could proceed the entire code base more quickly. | |
279 | ||
a9e064c0 LR |
280 | SmPL patch specific options |
281 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
282 | ||
283 | SmPL patches can have their own requirements for options passed | |
284 | to Coccinelle. SmPL patch specific options can be provided by | |
285 | providing them at the top of the SmPL patch, for instance: | |
286 | ||
287 | // Options: --no-includes --include-headers | |
288 | ||
289 | SmPL patch Coccinelle requirements | |
290 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
291 | ||
292 | As Coccinelle features get added some more advanced SmPL patches | |
293 | may require newer versions of Coccinelle. If an SmPL patch requires | |
294 | at least a version of Coccinelle, this can be specified as follows, | |
295 | as an example if requiring at least Coccinelle >= 1.0.5: | |
296 | ||
297 | // Requires: 1.0.5 | |
298 | ||
e228b1e6 NP |
299 | Proposing new semantic patches |
300 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
301 | ||
302 | New semantic patches can be proposed and submitted by kernel | |
303 | developers. For sake of clarity, they should be organized in the | |
32af0898 | 304 | sub-directories of 'scripts/coccinelle/'. |
e228b1e6 NP |
305 | |
306 | ||
307 | Detailed description of the 'report' mode | |
308 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
309 | ||
310 | 'report' generates a list in the following format: | |
311 | file:line:column-column: message | |
312 | ||
313 | Example: | |
314 | ||
315 | Running | |
316 | ||
9dcf7990 | 317 | make coccicheck MODE=report COCCI=scripts/coccinelle/api/err_cast.cocci |
e228b1e6 NP |
318 | |
319 | will execute the following part of the SmPL script. | |
320 | ||
321 | <smpl> | |
322 | @r depends on !context && !patch && (org || report)@ | |
323 | expression x; | |
324 | position p; | |
325 | @@ | |
326 | ||
327 | ERR_PTR@p(PTR_ERR(x)) | |
328 | ||
329 | @script:python depends on report@ | |
330 | p << r.p; | |
331 | x << r.x; | |
332 | @@ | |
333 | ||
334 | msg="ERR_CAST can be used with %s" % (x) | |
335 | coccilib.report.print_report(p[0], msg) | |
336 | </smpl> | |
337 | ||
338 | This SmPL excerpt generates entries on the standard output, as | |
339 | illustrated below: | |
340 | ||
341 | /home/user/linux/crypto/ctr.c:188:9-16: ERR_CAST can be used with alg | |
342 | /home/user/linux/crypto/authenc.c:619:9-16: ERR_CAST can be used with auth | |
343 | /home/user/linux/crypto/xts.c:227:9-16: ERR_CAST can be used with alg | |
344 | ||
345 | ||
346 | Detailed description of the 'patch' mode | |
347 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
348 | ||
349 | When the 'patch' mode is available, it proposes a fix for each problem | |
350 | identified. | |
351 | ||
352 | Example: | |
353 | ||
354 | Running | |
9dcf7990 | 355 | make coccicheck MODE=patch COCCI=scripts/coccinelle/api/err_cast.cocci |
e228b1e6 NP |
356 | |
357 | will execute the following part of the SmPL script. | |
358 | ||
359 | <smpl> | |
360 | @ depends on !context && patch && !org && !report @ | |
361 | expression x; | |
362 | @@ | |
363 | ||
364 | - ERR_PTR(PTR_ERR(x)) | |
365 | + ERR_CAST(x) | |
366 | </smpl> | |
367 | ||
368 | This SmPL excerpt generates patch hunks on the standard output, as | |
369 | illustrated below: | |
370 | ||
371 | diff -u -p a/crypto/ctr.c b/crypto/ctr.c | |
372 | --- a/crypto/ctr.c 2010-05-26 10:49:38.000000000 +0200 | |
373 | +++ b/crypto/ctr.c 2010-06-03 23:44:49.000000000 +0200 | |
374 | @@ -185,7 +185,7 @@ static struct crypto_instance *crypto_ct | |
375 | alg = crypto_attr_alg(tb[1], CRYPTO_ALG_TYPE_CIPHER, | |
376 | CRYPTO_ALG_TYPE_MASK); | |
377 | if (IS_ERR(alg)) | |
378 | - return ERR_PTR(PTR_ERR(alg)); | |
379 | + return ERR_CAST(alg); | |
380 | ||
381 | /* Block size must be >= 4 bytes. */ | |
382 | err = -EINVAL; | |
383 | ||
384 | Detailed description of the 'context' mode | |
385 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
386 | ||
387 | 'context' highlights lines of interest and their context | |
388 | in a diff-like style. | |
389 | ||
390 | NOTE: The diff-like output generated is NOT an applicable patch. The | |
391 | intent of the 'context' mode is to highlight the important lines | |
392 | (annotated with minus, '-') and gives some surrounding context | |
393 | lines around. This output can be used with the diff mode of | |
394 | Emacs to review the code. | |
395 | ||
396 | Example: | |
397 | ||
398 | Running | |
9dcf7990 | 399 | make coccicheck MODE=context COCCI=scripts/coccinelle/api/err_cast.cocci |
e228b1e6 NP |
400 | |
401 | will execute the following part of the SmPL script. | |
402 | ||
403 | <smpl> | |
404 | @ depends on context && !patch && !org && !report@ | |
405 | expression x; | |
406 | @@ | |
407 | ||
408 | * ERR_PTR(PTR_ERR(x)) | |
409 | </smpl> | |
410 | ||
411 | This SmPL excerpt generates diff hunks on the standard output, as | |
412 | illustrated below: | |
413 | ||
414 | diff -u -p /home/user/linux/crypto/ctr.c /tmp/nothing | |
415 | --- /home/user/linux/crypto/ctr.c 2010-05-26 10:49:38.000000000 +0200 | |
416 | +++ /tmp/nothing | |
417 | @@ -185,7 +185,6 @@ static struct crypto_instance *crypto_ct | |
418 | alg = crypto_attr_alg(tb[1], CRYPTO_ALG_TYPE_CIPHER, | |
419 | CRYPTO_ALG_TYPE_MASK); | |
420 | if (IS_ERR(alg)) | |
421 | - return ERR_PTR(PTR_ERR(alg)); | |
422 | ||
423 | /* Block size must be >= 4 bytes. */ | |
424 | err = -EINVAL; | |
425 | ||
426 | Detailed description of the 'org' mode | |
427 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
428 | ||
429 | 'org' generates a report in the Org mode format of Emacs. | |
430 | ||
431 | Example: | |
432 | ||
433 | Running | |
9dcf7990 | 434 | make coccicheck MODE=org COCCI=scripts/coccinelle/api/err_cast.cocci |
e228b1e6 NP |
435 | |
436 | will execute the following part of the SmPL script. | |
437 | ||
438 | <smpl> | |
439 | @r depends on !context && !patch && (org || report)@ | |
440 | expression x; | |
441 | position p; | |
442 | @@ | |
443 | ||
444 | ERR_PTR@p(PTR_ERR(x)) | |
445 | ||
446 | @script:python depends on org@ | |
447 | p << r.p; | |
448 | x << r.x; | |
449 | @@ | |
450 | ||
451 | msg="ERR_CAST can be used with %s" % (x) | |
452 | msg_safe=msg.replace("[","@(").replace("]",")") | |
453 | coccilib.org.print_todo(p[0], msg_safe) | |
454 | </smpl> | |
455 | ||
456 | This SmPL excerpt generates Org entries on the standard output, as | |
457 | illustrated below: | |
458 | ||
459 | * TODO [[view:/home/user/linux/crypto/ctr.c::face=ovl-face1::linb=188::colb=9::cole=16][ERR_CAST can be used with alg]] | |
460 | * TODO [[view:/home/user/linux/crypto/authenc.c::face=ovl-face1::linb=619::colb=9::cole=16][ERR_CAST can be used with auth]] | |
461 | * TODO [[view:/home/user/linux/crypto/xts.c::face=ovl-face1::linb=227::colb=9::cole=16][ERR_CAST can be used with alg]] |