Commit | Line | Data |
---|---|---|
faebe9fd PE |
1 | |
2 | The Resource Counter | |
3 | ||
4 | The resource counter, declared at include/linux/res_counter.h, | |
5 | is supposed to facilitate the resource management by controllers | |
6 | by providing common stuff for accounting. | |
7 | ||
8 | This "stuff" includes the res_counter structure and routines | |
9 | to work with it. | |
10 | ||
11 | ||
12 | ||
13 | 1. Crucial parts of the res_counter structure | |
14 | ||
15 | a. unsigned long long usage | |
16 | ||
17 | The usage value shows the amount of a resource that is consumed | |
18 | by a group at a given time. The units of measurement should be | |
19 | determined by the controller that uses this counter. E.g. it can | |
20 | be bytes, items or any other unit the controller operates on. | |
21 | ||
22 | b. unsigned long long max_usage | |
23 | ||
24 | The maximal value of the usage over time. | |
25 | ||
26 | This value is useful when gathering statistical information about | |
27 | the particular group, as it shows the actual resource requirements | |
28 | for a particular group, not just some usage snapshot. | |
29 | ||
30 | c. unsigned long long limit | |
31 | ||
32 | The maximal allowed amount of resource to consume by the group. In | |
33 | case the group requests for more resources, so that the usage value | |
34 | would exceed the limit, the resource allocation is rejected (see | |
35 | the next section). | |
36 | ||
37 | d. unsigned long long failcnt | |
38 | ||
39 | The failcnt stands for "failures counter". This is the number of | |
40 | resource allocation attempts that failed. | |
41 | ||
42 | c. spinlock_t lock | |
43 | ||
44 | Protects changes of the above values. | |
45 | ||
46 | ||
47 | ||
48 | 2. Basic accounting routines | |
49 | ||
50 | a. void res_counter_init(struct res_counter *rc) | |
51 | ||
52 | Initializes the resource counter. As usual, should be the first | |
53 | routine called for a new counter. | |
54 | ||
55 | b. int res_counter_charge[_locked] | |
56 | (struct res_counter *rc, unsigned long val) | |
57 | ||
58 | When a resource is about to be allocated it has to be accounted | |
59 | with the appropriate resource counter (controller should determine | |
60 | which one to use on its own). This operation is called "charging". | |
61 | ||
62 | This is not very important which operation - resource allocation | |
63 | or charging - is performed first, but | |
64 | * if the allocation is performed first, this may create a | |
65 | temporary resource over-usage by the time resource counter is | |
66 | charged; | |
67 | * if the charging is performed first, then it should be uncharged | |
68 | on error path (if the one is called). | |
69 | ||
70 | c. void res_counter_uncharge[_locked] | |
71 | (struct res_counter *rc, unsigned long val) | |
72 | ||
73 | When a resource is released (freed) it should be de-accounted | |
74 | from the resource counter it was accounted to. This is called | |
75 | "uncharging". | |
76 | ||
77 | The _locked routines imply that the res_counter->lock is taken. | |
78 | ||
79 | ||
80 | 2.1 Other accounting routines | |
81 | ||
82 | There are more routines that may help you with common needs, like | |
83 | checking whether the limit is reached or resetting the max_usage | |
84 | value. They are all declared in include/linux/res_counter.h. | |
85 | ||
86 | ||
87 | ||
88 | 3. Analyzing the resource counter registrations | |
89 | ||
90 | a. If the failcnt value constantly grows, this means that the counter's | |
91 | limit is too tight. Either the group is misbehaving and consumes too | |
92 | many resources, or the configuration is not suitable for the group | |
93 | and the limit should be increased. | |
94 | ||
95 | b. The max_usage value can be used to quickly tune the group. One may | |
96 | set the limits to maximal values and either load the container with | |
97 | a common pattern or leave one for a while. After this the max_usage | |
98 | value shows the amount of memory the container would require during | |
99 | its common activity. | |
100 | ||
101 | Setting the limit a bit above this value gives a pretty good | |
102 | configuration that works in most of the cases. | |
103 | ||
104 | c. If the max_usage is much less than the limit, but the failcnt value | |
105 | is growing, then the group tries to allocate a big chunk of resource | |
106 | at once. | |
107 | ||
108 | d. If the max_usage is much less than the limit, but the failcnt value | |
109 | is 0, then this group is given too high limit, that it does not | |
110 | require. It is better to lower the limit a bit leaving more resource | |
111 | for other groups. | |
112 | ||
113 | ||
114 | ||
115 | 4. Communication with the control groups subsystem (cgroups) | |
116 | ||
117 | All the resource controllers that are using cgroups and resource counters | |
118 | should provide files (in the cgroup filesystem) to work with the resource | |
119 | counter fields. They are recommended to adhere to the following rules: | |
120 | ||
121 | a. File names | |
122 | ||
123 | Field name File name | |
124 | --------------------------------------------------- | |
125 | usage usage_in_<unit_of_measurement> | |
126 | max_usage max_usage_in_<unit_of_measurement> | |
127 | limit limit_in_<unit_of_measurement> | |
128 | failcnt failcnt | |
129 | lock no file :) | |
130 | ||
131 | b. Reading from file should show the corresponding field value in the | |
132 | appropriate format. | |
133 | ||
134 | c. Writing to file | |
135 | ||
136 | Field Expected behavior | |
137 | ---------------------------------- | |
138 | usage prohibited | |
139 | max_usage reset to usage | |
140 | limit set the limit | |
141 | failcnt reset to zero | |
142 | ||
143 | ||
144 | ||
145 | 5. Usage example | |
146 | ||
147 | a. Declare a task group (take a look at cgroups subsystem for this) and | |
148 | fold a res_counter into it | |
149 | ||
150 | struct my_group { | |
151 | struct res_counter res; | |
152 | ||
153 | <other fields> | |
154 | } | |
155 | ||
156 | b. Put hooks in resource allocation/release paths | |
157 | ||
158 | int alloc_something(...) | |
159 | { | |
160 | if (res_counter_charge(res_counter_ptr, amount) < 0) | |
161 | return -ENOMEM; | |
162 | ||
163 | <allocate the resource and return to the caller> | |
164 | } | |
165 | ||
166 | void release_something(...) | |
167 | { | |
168 | res_counter_uncharge(res_counter_ptr, amount); | |
169 | ||
170 | <release the resource> | |
171 | } | |
172 | ||
173 | In order to keep the usage value self-consistent, both the | |
174 | "res_counter_ptr" and the "amount" in release_something() should be | |
175 | the same as they were in the alloc_something() when the releasing | |
176 | resource was allocated. | |
177 | ||
178 | c. Provide the way to read res_counter values and set them (the cgroups | |
179 | still can help with it). | |
180 | ||
181 | c. Compile and run :) |