Merge tag 'sound-4.2-rc4' of git://git.kernel.org/pub/scm/linux/kernel/git/tiwai...
[linux-2.6-block.git] / Documentation / local_ops.txt
CommitLineData
f1f8810c
MD
1 Semantics and Behavior of Local Atomic Operations
2
3 Mathieu Desnoyers
4
5
6 This document explains the purpose of the local atomic operations, how
7to implement them for any given architecture and shows how they can be used
8properly. It also stresses on the precautions that must be taken when reading
9those local variables across CPUs when the order of memory writes matters.
10
7d94a82e
CL
11Note that local_t based operations are not recommended for general kernel use.
12Please use the this_cpu operations instead unless there is really a special purpose.
13Most uses of local_t in the kernel have been replaced by this_cpu operations.
14this_cpu operations combine the relocation with the local_t like semantics in
15a single instruction and yield more compact and faster executing code.
f1f8810c
MD
16
17
18* Purpose of local atomic operations
19
20Local atomic operations are meant to provide fast and highly reentrant per CPU
21counters. They minimize the performance cost of standard atomic operations by
22removing the LOCK prefix and memory barriers normally required to synchronize
23across CPUs.
24
25Having fast per CPU atomic counters is interesting in many cases : it does not
26require disabling interrupts to protect from interrupt handlers and it permits
27coherent counters in NMI handlers. It is especially useful for tracing purposes
28and for various performance monitoring counters.
29
30Local atomic operations only guarantee variable modification atomicity wrt the
31CPU which owns the data. Therefore, care must taken to make sure that only one
32CPU writes to the local_t data. This is done by using per cpu data and making
33sure that we modify it from within a preemption safe context. It is however
34permitted to read local_t data from any CPU : it will then appear to be written
0e1ccb96 35out of order wrt other memory writes by the owner CPU.
f1f8810c
MD
36
37
38* Implementation for a given architecture
39
40It can be done by slightly modifying the standard atomic operations : only
41their UP variant must be kept. It typically means removing LOCK prefix (on
19f59460 42i386 and x86_64) and any SMP synchronization barrier. If the architecture does
f1f8810c 43not have a different behavior between SMP and UP, including asm-generic/local.h
d9195881 44in your architecture's local.h is sufficient.
f1f8810c
MD
45
46The local_t type is defined as an opaque signed long by embedding an
47atomic_long_t inside a structure. This is made so a cast from this type to a
48long fails. The definition looks like :
49
50typedef struct { atomic_long_t a; } local_t;
51
52
74beb9db
MD
53* Rules to follow when using local atomic operations
54
55- Variables touched by local ops must be per cpu variables.
56- _Only_ the CPU owner of these variables must write to them.
57- This CPU can use local ops from any context (process, irq, softirq, nmi, ...)
58 to update its local_t variables.
59- Preemption (or interrupts) must be disabled when using local ops in
60 process context to make sure the process won't be migrated to a
61 different CPU between getting the per-cpu variable and doing the
62 actual local op.
63- When using local ops in interrupt context, no special care must be
64 taken on a mainline kernel, since they will run on the local CPU with
65 preemption already disabled. I suggest, however, to explicitly
66 disable preemption anyway to make sure it will still work correctly on
67 -rt kernels.
68- Reading the local cpu variable will provide the current copy of the
69 variable.
70- Reads of these variables can be done from any CPU, because updates to
71 "long", aligned, variables are always atomic. Since no memory
72 synchronization is done by the writer CPU, an outdated copy of the
73 variable can be read when reading some _other_ cpu's variables.
74
75
f1f8810c
MD
76* How to use local atomic operations
77
78#include <linux/percpu.h>
79#include <asm/local.h>
80
81static DEFINE_PER_CPU(local_t, counters) = LOCAL_INIT(0);
82
83
84* Counting
85
86Counting is done on all the bits of a signed long.
87
88In preemptible context, use get_cpu_var() and put_cpu_var() around local atomic
89operations : it makes sure that preemption is disabled around write access to
90the per cpu variable. For instance :
91
92 local_inc(&get_cpu_var(counters));
93 put_cpu_var(counters);
94
7d94a82e
CL
95If you are already in a preemption-safe context, you can use
96this_cpu_ptr() instead.
f1f8810c 97
7d94a82e 98 local_inc(this_cpu_ptr(&counters));
f1f8810c
MD
99
100
101
102* Reading the counters
103
104Those local counters can be read from foreign CPUs to sum the count. Note that
105the data seen by local_read across CPUs must be considered to be out of order
106relatively to other memory writes happening on the CPU that owns the data.
107
108 long sum = 0;
109 for_each_online_cpu(cpu)
110 sum += local_read(&per_cpu(counters, cpu));
111
112If you want to use a remote local_read to synchronize access to a resource
113between CPUs, explicit smp_wmb() and smp_rmb() memory barriers must be used
114respectively on the writer and the reader CPUs. It would be the case if you use
115the local_t variable as a counter of bytes written in a buffer : there should
116be a smp_wmb() between the buffer write and the counter increment and also a
117smp_rmb() between the counter read and the buffer read.
118
119
120Here is a sample module which implements a basic per cpu counter using local.h.
121
122--- BEGIN ---
123/* test-local.c
124 *
125 * Sample module for local.h usage.
126 */
127
128
129#include <asm/local.h>
130#include <linux/module.h>
131#include <linux/timer.h>
132
133static DEFINE_PER_CPU(local_t, counters) = LOCAL_INIT(0);
134
135static struct timer_list test_timer;
136
137/* IPI called on each CPU. */
138static void test_each(void *info)
139{
140 /* Increment the counter from a non preemptible context */
141 printk("Increment on cpu %d\n", smp_processor_id());
7d94a82e 142 local_inc(this_cpu_ptr(&counters));
f1f8810c
MD
143
144 /* This is what incrementing the variable would look like within a
145 * preemptible context (it disables preemption) :
146 *
147 * local_inc(&get_cpu_var(counters));
148 * put_cpu_var(counters);
149 */
150}
151
152static void do_test_timer(unsigned long data)
153{
154 int cpu;
155
156 /* Increment the counters */
02d43b1d 157 on_each_cpu(test_each, NULL, 1);
f1f8810c
MD
158 /* Read all the counters */
159 printk("Counters read from CPU %d\n", smp_processor_id());
160 for_each_online_cpu(cpu) {
161 printk("Read : CPU %d, count %ld\n", cpu,
162 local_read(&per_cpu(counters, cpu)));
163 }
164 del_timer(&test_timer);
165 test_timer.expires = jiffies + 1000;
166 add_timer(&test_timer);
167}
168
169static int __init test_init(void)
170{
171 /* initialize the timer that will increment the counter */
172 init_timer(&test_timer);
173 test_timer.function = do_test_timer;
174 test_timer.expires = jiffies + 1;
175 add_timer(&test_timer);
176
177 return 0;
178}
179
180static void __exit test_exit(void)
181{
182 del_timer_sync(&test_timer);
183}
184
185module_init(test_init);
186module_exit(test_exit);
187
188MODULE_LICENSE("GPL");
189MODULE_AUTHOR("Mathieu Desnoyers");
190MODULE_DESCRIPTION("Local Atomic Ops");
191--- END ---