Commit | Line | Data |
---|---|---|
9836d891 | 1 | Memory Resource Controller(Memcg) Implementation Memo. |
1080d7a3 DN |
2 | Last Updated: 2010/2 |
3 | Base Kernel Version: based on 2.6.33-rc7-mm(candidate for 34). | |
9836d891 KH |
4 | |
5 | Because VM is getting complex (one of reasons is memcg...), memcg's behavior | |
6 | is complex. This is a document for memcg's internal behavior. | |
7 | Please note that implementation details can be changed. | |
8 | ||
45ce80fb | 9 | (*) Topics on API should be in Documentation/cgroups/memory.txt) |
9836d891 KH |
10 | |
11 | 0. How to record usage ? | |
12 | 2 objects are used. | |
13 | ||
14 | page_cgroup ....an object per page. | |
15 | Allocated at boot or memory hotplug. Freed at memory hot removal. | |
16 | ||
17 | swap_cgroup ... an entry per swp_entry. | |
18 | Allocated at swapon(). Freed at swapoff(). | |
19 | ||
20 | The page_cgroup has USED bit and double count against a page_cgroup never | |
21 | occurs. swap_cgroup is used only when a charged page is swapped-out. | |
22 | ||
23 | 1. Charge | |
24 | ||
25 | a page/swp_entry may be charged (usage += PAGE_SIZE) at | |
26 | ||
00501b53 | 27 | mem_cgroup_try_charge() |
9836d891 KH |
28 | |
29 | 2. Uncharge | |
30 | a page/swp_entry may be uncharged (usage -= PAGE_SIZE) by | |
31 | ||
32 | mem_cgroup_uncharge_page() | |
33 | Called when an anonymous page is fully unmapped. I.e., mapcount goes | |
34 | to 0. If the page is SwapCache, uncharge is delayed until | |
35 | mem_cgroup_uncharge_swapcache(). | |
36 | ||
37 | mem_cgroup_uncharge_cache_page() | |
38 | Called when a page-cache is deleted from radix-tree. If the page is | |
39 | SwapCache, uncharge is delayed until mem_cgroup_uncharge_swapcache(). | |
40 | ||
41 | mem_cgroup_uncharge_swapcache() | |
42 | Called when SwapCache is removed from radix-tree. The charge itself | |
43 | is moved to swap_cgroup. (If mem+swap controller is disabled, no | |
44 | charge to swap occurs.) | |
45 | ||
46 | mem_cgroup_uncharge_swap() | |
47 | Called when swp_entry's refcnt goes down to 0. A charge against swap | |
48 | disappears. | |
49 | ||
50 | mem_cgroup_end_migration(old, new) | |
51 | At success of migration old is uncharged (if necessary), a charge | |
52 | to new page is committed. At failure, charge to old page is committed. | |
53 | ||
54 | 3. charge-commit-cancel | |
00501b53 JW |
55 | Memcg pages are charged in two steps: |
56 | mem_cgroup_try_charge() | |
57 | mem_cgroup_commit_charge() or mem_cgroup_cancel_charge() | |
9836d891 KH |
58 | |
59 | At try_charge(), there are no flags to say "this page is charged". | |
60 | at this point, usage += PAGE_SIZE. | |
61 | ||
00501b53 | 62 | At commit(), the page is associated with the memcg. |
9836d891 KH |
63 | |
64 | At cancel(), simply usage -= PAGE_SIZE. | |
65 | ||
66 | Under below explanation, we assume CONFIG_MEM_RES_CTRL_SWAP=y. | |
67 | ||
68 | 4. Anonymous | |
69 | Anonymous page is newly allocated at | |
70 | - page fault into MAP_ANONYMOUS mapping. | |
71 | - Copy-On-Write. | |
72 | It is charged right after it's allocated before doing any page table | |
73 | related operations. Of course, it's uncharged when another page is used | |
74 | for the fault address. | |
75 | ||
76 | At freeing anonymous page (by exit() or munmap()), zap_pte() is called | |
77 | and pages for ptes are freed one by one.(see mm/memory.c). Uncharges | |
78 | are done at page_remove_rmap() when page_mapcount() goes down to 0. | |
79 | ||
80 | Another page freeing is by page-reclaim (vmscan.c) and anonymous | |
81 | pages are swapped out. In this case, the page is marked as | |
82 | PageSwapCache(). uncharge() routine doesn't uncharge the page marked | |
83 | as SwapCache(). It's delayed until __delete_from_swap_cache(). | |
84 | ||
85 | 4.1 Swap-in. | |
86 | At swap-in, the page is taken from swap-cache. There are 2 cases. | |
87 | ||
88 | (a) If the SwapCache is newly allocated and read, it has no charges. | |
89 | (b) If the SwapCache has been mapped by processes, it has been | |
90 | charged already. | |
91 | ||
03f3c433 KH |
92 | This swap-in is one of the most complicated work. In do_swap_page(), |
93 | following events occur when pte is unchanged. | |
94 | ||
95 | (1) the page (SwapCache) is looked up. | |
96 | (2) lock_page() | |
97 | (3) try_charge_swapin() | |
98 | (4) reuse_swap_page() (may call delete_swap_cache()) | |
99 | (5) commit_charge_swapin() | |
100 | (6) swap_free(). | |
101 | ||
102 | Considering following situation for example. | |
103 | ||
104 | (A) The page has not been charged before (2) and reuse_swap_page() | |
105 | doesn't call delete_from_swap_cache(). | |
106 | (B) The page has not been charged before (2) and reuse_swap_page() | |
107 | calls delete_from_swap_cache(). | |
108 | (C) The page has been charged before (2) and reuse_swap_page() doesn't | |
109 | call delete_from_swap_cache(). | |
110 | (D) The page has been charged before (2) and reuse_swap_page() calls | |
111 | delete_from_swap_cache(). | |
112 | ||
113 | memory.usage/memsw.usage changes to this page/swp_entry will be | |
114 | Case (A) (B) (C) (D) | |
115 | Event | |
116 | Before (2) 0/ 1 0/ 1 1/ 1 1/ 1 | |
117 | =========================================== | |
118 | (3) +1/+1 +1/+1 +1/+1 +1/+1 | |
119 | (4) - 0/ 0 - -1/ 0 | |
120 | (5) 0/-1 0/ 0 -1/-1 0/ 0 | |
121 | (6) - 0/-1 - 0/-1 | |
122 | =========================================== | |
123 | Result 1/ 1 1/ 1 1/ 1 1/ 1 | |
124 | ||
125 | In any cases, charges to this page should be 1/ 1. | |
9836d891 KH |
126 | |
127 | 4.2 Swap-out. | |
128 | At swap-out, typical state transition is below. | |
129 | ||
130 | (a) add to swap cache. (marked as SwapCache) | |
131 | swp_entry's refcnt += 1. | |
132 | (b) fully unmapped. | |
133 | swp_entry's refcnt += # of ptes. | |
134 | (c) write back to swap. | |
135 | (d) delete from swap cache. (remove from SwapCache) | |
136 | swp_entry's refcnt -= 1. | |
137 | ||
138 | ||
139 | At (b), the page is marked as SwapCache and not uncharged. | |
140 | At (d), the page is removed from SwapCache and a charge in page_cgroup | |
141 | is moved to swap_cgroup. | |
142 | ||
143 | Finally, at task exit, | |
144 | (e) zap_pte() is called and swp_entry's refcnt -=1 -> 0. | |
145 | Here, a charge in swap_cgroup disappears. | |
146 | ||
147 | 5. Page Cache | |
148 | Page Cache is charged at | |
149 | - add_to_page_cache_locked(). | |
150 | ||
151 | uncharged at | |
152 | - __remove_from_page_cache(). | |
153 | ||
154 | The logic is very clear. (About migration, see below) | |
155 | Note: __remove_from_page_cache() is called by remove_from_page_cache() | |
156 | and __remove_mapping(). | |
157 | ||
158 | 6. Shmem(tmpfs) Page Cache | |
159 | Memcg's charge/uncharge have special handlers of shmem. The best way | |
160 | to understand shmem's page state transition is to read mm/shmem.c. | |
161 | But brief explanation of the behavior of memcg around shmem will be | |
162 | helpful to understand the logic. | |
163 | ||
164 | Shmem's page (just leaf page, not direct/indirect block) can be on | |
165 | - radix-tree of shmem's inode. | |
166 | - SwapCache. | |
167 | - Both on radix-tree and SwapCache. This happens at swap-in | |
168 | and swap-out, | |
169 | ||
170 | It's charged when... | |
171 | - A new page is added to shmem's radix-tree. | |
172 | - A swp page is read. (move a charge from swap_cgroup to page_cgroup) | |
173 | It's uncharged when | |
174 | - A page is removed from radix-tree and not SwapCache. | |
175 | - When SwapCache is removed, a charge is moved to swap_cgroup. | |
176 | - When swp_entry's refcnt goes down to 0, a charge in swap_cgroup | |
177 | disappears. | |
178 | ||
179 | 7. Page Migration | |
180 | One of the most complicated functions is page-migration-handler. | |
181 | Memcg has 2 routines. Assume that we are migrating a page's contents | |
182 | from OLDPAGE to NEWPAGE. | |
183 | ||
184 | Usual migration logic is.. | |
185 | (a) remove the page from LRU. | |
186 | (b) allocate NEWPAGE (migration target) | |
187 | (c) lock by lock_page(). | |
188 | (d) unmap all mappings. | |
189 | (e-1) If necessary, replace entry in radix-tree. | |
190 | (e-2) move contents of a page. | |
191 | (f) map all mappings again. | |
192 | (g) pushback the page to LRU. | |
193 | (-) OLDPAGE will be freed. | |
194 | ||
195 | Before (g), memcg should complete all necessary charge/uncharge to | |
196 | NEWPAGE/OLDPAGE. | |
197 | ||
198 | The point is.... | |
199 | - If OLDPAGE is anonymous, all charges will be dropped at (d) because | |
200 | try_to_unmap() drops all mapcount and the page will not be | |
201 | SwapCache. | |
202 | ||
203 | - If OLDPAGE is SwapCache, charges will be kept at (g) because | |
204 | __delete_from_swap_cache() isn't called at (e-1) | |
205 | ||
206 | - If OLDPAGE is page-cache, charges will be kept at (g) because | |
207 | remove_from_swap_cache() isn't called at (e-1) | |
208 | ||
209 | memcg provides following hooks. | |
210 | ||
211 | - mem_cgroup_prepare_migration(OLDPAGE) | |
212 | Called after (b) to account a charge (usage += PAGE_SIZE) against | |
213 | memcg which OLDPAGE belongs to. | |
214 | ||
215 | - mem_cgroup_end_migration(OLDPAGE, NEWPAGE) | |
216 | Called after (f) before (g). | |
217 | If OLDPAGE is used, commit OLDPAGE again. If OLDPAGE is already | |
218 | charged, a charge by prepare_migration() is automatically canceled. | |
219 | If NEWPAGE is used, commit NEWPAGE and uncharge OLDPAGE. | |
220 | ||
221 | But zap_pte() (by exit or munmap) can be called while migration, | |
222 | we have to check if OLDPAGE/NEWPAGE is a valid page after commit(). | |
223 | ||
224 | 8. LRU | |
a33f3224 | 225 | Each memcg has its own private LRU. Now, its handling is under global |
9836d891 KH |
226 | VM's control (means that it's handled under global zone->lru_lock). |
227 | Almost all routines around memcg's LRU is called by global LRU's | |
228 | list management functions under zone->lru_lock(). | |
229 | ||
230 | A special function is mem_cgroup_isolate_pages(). This scans | |
231 | memcg's private LRU and call __isolate_lru_page() to extract a page | |
232 | from LRU. | |
233 | (By __isolate_lru_page(), the page is removed from both of global and | |
234 | private LRU.) | |
235 | ||
236 | ||
237 | 9. Typical Tests. | |
238 | ||
239 | Tests for racy cases. | |
240 | ||
241 | 9.1 Small limit to memcg. | |
242 | When you do test to do racy case, it's good test to set memcg's limit | |
243 | to be very small rather than GB. Many races found in the test under | |
244 | xKB or xxMB limits. | |
245 | (Memory behavior under GB and Memory behavior under MB shows very | |
246 | different situation.) | |
247 | ||
248 | 9.2 Shmem | |
249 | Historically, memcg's shmem handling was poor and we saw some amount | |
250 | of troubles here. This is because shmem is page-cache but can be | |
251 | SwapCache. Test with shmem/tmpfs is always good test. | |
252 | ||
253 | 9.3 Migration | |
254 | For NUMA, migration is an another special case. To do easy test, cpuset | |
255 | is useful. Following is a sample script to do migration. | |
256 | ||
257 | mount -t cgroup -o cpuset none /opt/cpuset | |
258 | ||
259 | mkdir /opt/cpuset/01 | |
260 | echo 1 > /opt/cpuset/01/cpuset.cpus | |
261 | echo 0 > /opt/cpuset/01/cpuset.mems | |
262 | echo 1 > /opt/cpuset/01/cpuset.memory_migrate | |
263 | mkdir /opt/cpuset/02 | |
264 | echo 1 > /opt/cpuset/02/cpuset.cpus | |
265 | echo 1 > /opt/cpuset/02/cpuset.mems | |
266 | echo 1 > /opt/cpuset/02/cpuset.memory_migrate | |
267 | ||
268 | In above set, when you moves a task from 01 to 02, page migration to | |
269 | node 0 to node 1 will occur. Following is a script to migrate all | |
270 | under cpuset. | |
271 | -- | |
272 | move_task() | |
273 | { | |
274 | for pid in $1 | |
275 | do | |
276 | /bin/echo $pid >$2/tasks 2>/dev/null | |
277 | echo -n $pid | |
278 | echo -n " " | |
279 | done | |
280 | echo END | |
281 | } | |
282 | ||
283 | G1_TASK=`cat ${G1}/tasks` | |
284 | G2_TASK=`cat ${G2}/tasks` | |
285 | move_task "${G1_TASK}" ${G2} & | |
286 | -- | |
287 | 9.4 Memory hotplug. | |
288 | memory hotplug test is one of good test. | |
289 | to offline memory, do following. | |
290 | # echo offline > /sys/devices/system/memory/memoryXXX/state | |
291 | (XXX is the place of memory) | |
292 | This is an easy way to test page migration, too. | |
293 | ||
294 | 9.5 mkdir/rmdir | |
295 | When using hierarchy, mkdir/rmdir test should be done. | |
296 | Use tests like the following. | |
297 | ||
298 | echo 1 >/opt/cgroup/01/memory/use_hierarchy | |
299 | mkdir /opt/cgroup/01/child_a | |
300 | mkdir /opt/cgroup/01/child_b | |
301 | ||
302 | set limit to 01. | |
303 | add limit to 01/child_b | |
304 | run jobs under child_a and child_b | |
305 | ||
306 | create/delete following groups at random while jobs are running. | |
307 | /opt/cgroup/01/child_a/child_aa | |
308 | /opt/cgroup/01/child_b/child_bb | |
309 | /opt/cgroup/01/child_c | |
310 | ||
311 | running new jobs in new group is also good. | |
312 | ||
313 | 9.6 Mount with other subsystems. | |
314 | Mounting with other subsystems is a good test because there is a | |
315 | race and lock dependency with other cgroup subsystems. | |
316 | ||
317 | example) | |
0263c12c | 318 | # mount -t cgroup none /cgroup -o cpuset,memory,cpu,devices |
9836d891 KH |
319 | |
320 | and do task move, mkdir, rmdir etc...under this. | |
8d50d369 KH |
321 | |
322 | 9.7 swapoff. | |
323 | Besides management of swap is one of complicated parts of memcg, | |
324 | call path of swap-in at swapoff is not same as usual swap-in path.. | |
325 | It's worth to be tested explicitly. | |
326 | ||
327 | For example, test like following is good. | |
328 | (Shell-A) | |
0263c12c | 329 | # mount -t cgroup none /cgroup -o memory |
8d50d369 KH |
330 | # mkdir /cgroup/test |
331 | # echo 40M > /cgroup/test/memory.limit_in_bytes | |
332 | # echo 0 > /cgroup/test/tasks | |
333 | Run malloc(100M) program under this. You'll see 60M of swaps. | |
334 | (Shell-B) | |
335 | # move all tasks in /cgroup/test to /cgroup | |
336 | # /sbin/swapoff -a | |
6d5e147d | 337 | # rmdir /cgroup/test |
8d50d369 KH |
338 | # kill malloc task. |
339 | ||
340 | Of course, tmpfs v.s. swapoff test should be tested, too. | |
0b7f569e KH |
341 | |
342 | 9.8 OOM-Killer | |
343 | Out-of-memory caused by memcg's limit will kill tasks under | |
344 | the memcg. When hierarchy is used, a task under hierarchy | |
345 | will be killed by the kernel. | |
346 | In this case, panic_on_oom shouldn't be invoked and tasks | |
347 | in other groups shouldn't be killed. | |
348 | ||
349 | It's not difficult to cause OOM under memcg as following. | |
350 | Case A) when you can swapoff | |
351 | #swapoff -a | |
352 | #echo 50M > /memory.limit_in_bytes | |
353 | run 51M of malloc | |
354 | ||
355 | Case B) when you use mem+swap limitation. | |
356 | #echo 50M > memory.limit_in_bytes | |
357 | #echo 50M > memory.memsw.limit_in_bytes | |
358 | run 51M of malloc | |
1080d7a3 DN |
359 | |
360 | 9.9 Move charges at task migration | |
361 | Charges associated with a task can be moved along with task migration. | |
362 | ||
363 | (Shell-A) | |
364 | #mkdir /cgroup/A | |
365 | #echo $$ >/cgroup/A/tasks | |
366 | run some programs which uses some amount of memory in /cgroup/A. | |
367 | ||
368 | (Shell-B) | |
369 | #mkdir /cgroup/B | |
370 | #echo 1 >/cgroup/B/memory.move_charge_at_immigrate | |
371 | #echo "pid of the program running in group A" >/cgroup/B/tasks | |
372 | ||
373 | You can see charges have been moved by reading *.usage_in_bytes or | |
374 | memory.stat of both A and B. | |
375 | See 8.2 of Documentation/cgroups/memory.txt to see what value should be | |
376 | written to move_charge_at_immigrate. | |
1e111452 KS |
377 | |
378 | 9.10 Memory thresholds | |
b595076a | 379 | Memory controller implements memory thresholds using cgroups notification |
92e015b1 | 380 | API. You can use tools/cgroup/cgroup_event_listener.c to test it. |
1e111452 KS |
381 | |
382 | (Shell-A) Create cgroup and run event listener | |
383 | # mkdir /cgroup/A | |
384 | # ./cgroup_event_listener /cgroup/A/memory.usage_in_bytes 5M | |
385 | ||
386 | (Shell-B) Add task to cgroup and try to allocate and free memory | |
387 | # echo $$ >/cgroup/A/tasks | |
388 | # a="$(dd if=/dev/zero bs=1M count=10)" | |
389 | # a= | |
390 | ||
391 | You will see message from cgroup_event_listener every time you cross | |
392 | the thresholds. | |
393 | ||
394 | Use /cgroup/A/memory.memsw.usage_in_bytes to test memsw thresholds. | |
395 | ||
396 | It's good idea to test root cgroup as well. |