1 Memory Resource Controller
3 NOTE: The Memory Resource Controller has generically been referred to as the
4 memory controller in this document. Do not confuse memory controller
5 used here with the memory controller that is used in hardware.
9 When we mention a cgroup (cgroupfs's directory) with memory controller,
10 we call it "memory cgroup". When you see git-log and source code, you'll
11 see patch's title and function names tend to use "memcg".
12 In this document, we avoid using it.
14 Benefits and Purpose of the memory controller
16 The memory controller isolates the memory behaviour of a group of tasks
17 from the rest of the system. The article on LWN [12] mentions some probable
18 uses of the memory controller. The memory controller can be used to
20 a. Isolate an application or a group of applications
21 Memory hungry applications can be isolated and limited to a smaller
23 b. Create a cgroup with limited amount of memory, this can be used
24 as a good alternative to booting with mem=XXXX.
25 c. Virtualization solutions can control the amount of memory they want
26 to assign to a virtual machine instance.
27 d. A CD/DVD burner could control the amount of memory used by the
28 rest of the system to ensure that burning does not fail due to lack
30 e. There are several other use cases, find one or use the controller just
31 for fun (to learn and hack on the VM subsystem).
33 Current Status: linux-2.6.34-mmotm(development version of 2010/April)
36 - accounting anonymous pages, file caches, swap caches usage and limiting them.
37 - private LRU and reclaim routine. (system's global LRU and private LRU
38 work independently from each other)
39 - optionally, memory+swap usage can be accounted and limited.
40 - hierarchical accounting
42 - moving(recharging) account at moving a task is selectable.
43 - usage threshold notifier
44 - oom-killer disable knob and oom-notifier
45 - Root cgroup has no limit controls.
47 Hugepages is not under control yet. We just manage pages on LRU. To add more
48 controls, we have to take care of performance. Kernel memory support is work
49 in progress, and the current version provides basically functionality.
51 Brief summary of control files.
53 tasks # attach a task(thread) and show list of threads
54 cgroup.procs # show list of processes
55 cgroup.event_control # an interface for event_fd()
56 memory.usage_in_bytes # show current res_counter usage for memory
58 memory.memsw.usage_in_bytes # show current res_counter usage for memory+Swap
60 memory.kmem.usage_in_bytes # show current res_counter usage for kmem only.
62 memory.limit_in_bytes # set/show limit of memory usage
63 memory.memsw.limit_in_bytes # set/show limit of memory+Swap usage
64 memory.kmem.limit_in_bytes # if allowed, set/show limit of kernel memory
65 memory.failcnt # show the number of memory usage hits limits
66 memory.memsw.failcnt # show the number of memory+Swap hits limits
67 memory.max_usage_in_bytes # show max memory usage recorded
68 memory.memsw.usage_in_bytes # show max memory+Swap usage recorded
69 memory.soft_limit_in_bytes # set/show soft limit of memory usage
70 memory.stat # show various statistics
71 memory.use_hierarchy # set/show hierarchical account enabled
72 memory.force_empty # trigger forced move charge to parent
73 memory.swappiness # set/show swappiness parameter of vmscan
74 (See sysctl's vm.swappiness)
75 memory.move_charge_at_immigrate # set/show controls of moving charges
76 memory.oom_control # set/show oom controls.
77 memory.numa_stat # show the number of memory usage per numa node
79 memory.independent_kmem_limit # select whether or not kernel memory limits are
80 independent of user limits
81 memory.kmem.tcp.limit_in_bytes # set/show hard limit for tcp buf memory
82 memory.kmem.tcp.usage_in_bytes # show current tcp buf memory allocation
86 The memory controller has a long history. A request for comments for the memory
87 controller was posted by Balbir Singh [1]. At the time the RFC was posted
88 there were several implementations for memory control. The goal of the
89 RFC was to build consensus and agreement for the minimal features required
90 for memory control. The first RSS controller was posted by Balbir Singh[2]
91 in Feb 2007. Pavel Emelianov [3][4][5] has since posted three versions of the
92 RSS controller. At OLS, at the resource management BoF, everyone suggested
93 that we handle both page cache and RSS together. Another request was raised
94 to allow user space handling of OOM. The current memory controller is
95 at version 6; it combines both mapped (RSS) and unmapped Page
100 Memory is a unique resource in the sense that it is present in a limited
101 amount. If a task requires a lot of CPU processing, the task can spread
102 its processing over a period of hours, days, months or years, but with
103 memory, the same physical memory needs to be reused to accomplish the task.
105 The memory controller implementation has been divided into phases. These
109 2. mlock(2) controller
110 3. Kernel user memory accounting and slab control
111 4. user mappings length controller
113 The memory controller is the first controller developed.
117 The core of the design is a counter called the res_counter. The res_counter
118 tracks the current memory usage and limit of the group of processes associated
119 with the controller. Each cgroup has a memory controller specific data
120 structure (mem_cgroup) associated with it.
124 +--------------------+
127 +--------------------+
130 +---------------+ | +---------------+
131 | mm_struct | |.... | mm_struct |
133 +---------------+ | +---------------+
137 +---------------+ +------+--------+
138 | page +----------> page_cgroup|
140 +---------------+ +---------------+
142 (Figure 1: Hierarchy of Accounting)
145 Figure 1 shows the important aspects of the controller
147 1. Accounting happens per cgroup
148 2. Each mm_struct knows about which cgroup it belongs to
149 3. Each page has a pointer to the page_cgroup, which in turn knows the
152 The accounting is done as follows: mem_cgroup_charge() is invoked to setup
153 the necessary data structures and check if the cgroup that is being charged
154 is over its limit. If it is then reclaim is invoked on the cgroup.
155 More details can be found in the reclaim section of this document.
156 If everything goes well, a page meta-data-structure called page_cgroup is
157 updated. page_cgroup has its own LRU on cgroup.
158 (*) page_cgroup structure is allocated at boot/memory-hotplug time.
160 2.2.1 Accounting details
162 All mapped anon pages (RSS) and cache pages (Page Cache) are accounted.
163 Some pages which are never reclaimable and will not be on the global LRU
164 are not accounted. We just account pages under usual VM management.
166 RSS pages are accounted at page_fault unless they've already been accounted
167 for earlier. A file page will be accounted for as Page Cache when it's
168 inserted into inode (radix-tree). While it's mapped into the page tables of
169 processes, duplicate accounting is carefully avoided.
171 A RSS page is unaccounted when it's fully unmapped. A PageCache page is
172 unaccounted when it's removed from radix-tree. Even if RSS pages are fully
173 unmapped (by kswapd), they may exist as SwapCache in the system until they
174 are really freed. Such SwapCaches also also accounted.
175 A swapped-in page is not accounted until it's mapped.
177 Note: The kernel does swapin-readahead and read multiple swaps at once.
178 This means swapped-in pages may contain pages for other tasks than a task
179 causing page fault. So, we avoid accounting at swap-in I/O.
181 At page migration, accounting information is kept.
183 Note: we just account pages-on-LRU because our purpose is to control amount
184 of used pages; not-on-LRU pages tend to be out-of-control from VM view.
186 2.3 Shared Page Accounting
188 Shared pages are accounted on the basis of the first touch approach. The
189 cgroup that first touches a page is accounted for the page. The principle
190 behind this approach is that a cgroup that aggressively uses a shared
191 page will eventually get charged for it (once it is uncharged from
192 the cgroup that brought it in -- this will happen on memory pressure).
194 Exception: If CONFIG_CGROUP_CGROUP_MEM_RES_CTLR_SWAP is not used.
195 When you do swapoff and make swapped-out pages of shmem(tmpfs) to
196 be backed into memory in force, charges for pages are accounted against the
197 caller of swapoff rather than the users of shmem.
200 2.4 Swap Extension (CONFIG_CGROUP_MEM_RES_CTLR_SWAP)
202 Swap Extension allows you to record charge for swap. A swapped-in page is
203 charged back to original page allocator if possible.
205 When swap is accounted, following files are added.
206 - memory.memsw.usage_in_bytes.
207 - memory.memsw.limit_in_bytes.
209 memsw means memory+swap. Usage of memory+swap is limited by
210 memsw.limit_in_bytes.
212 Example: Assume a system with 4G of swap. A task which allocates 6G of memory
213 (by mistake) under 2G memory limitation will use all swap.
214 In this case, setting memsw.limit_in_bytes=3G will prevent bad use of swap.
215 By using memsw limit, you can avoid system OOM which can be caused by swap
218 * why 'memory+swap' rather than swap.
219 The global LRU(kswapd) can swap out arbitrary pages. Swap-out means
220 to move account from memory to swap...there is no change in usage of
221 memory+swap. In other words, when we want to limit the usage of swap without
222 affecting global LRU, memory+swap limit is better than just limiting swap from
225 * What happens when a cgroup hits memory.memsw.limit_in_bytes
226 When a cgroup hits memory.memsw.limit_in_bytes, it's useless to do swap-out
227 in this cgroup. Then, swap-out will not be done by cgroup routine and file
228 caches are dropped. But as mentioned above, global LRU can do swapout memory
229 from it for sanity of the system's memory management state. You can't forbid
234 Each cgroup maintains a per cgroup LRU which has the same structure as
235 global VM. When a cgroup goes over its limit, we first try
236 to reclaim memory from the cgroup so as to make space for the new
237 pages that the cgroup has touched. If the reclaim is unsuccessful,
238 an OOM routine is invoked to select and kill the bulkiest task in the
239 cgroup. (See 10. OOM Control below.)
241 The reclaim algorithm has not been modified for cgroups, except that
242 pages that are selected for reclaiming come from the per cgroup LRU
245 NOTE: Reclaim does not work for the root cgroup, since we cannot set any
246 limits on the root cgroup.
248 Note2: When panic_on_oom is set to "2", the whole system will panic.
250 When oom event notifier is registered, event will be delivered.
251 (See oom_control section)
255 lock_page_cgroup()/unlock_page_cgroup() should not be called under
258 Other lock order is following:
263 In many cases, just lock_page_cgroup() is called.
264 per-zone-per-cgroup LRU (cgroup's private LRU) is just guarded by
265 zone->lru_lock, it has no lock of its own.
267 2.7 Kernel Memory Extension (CONFIG_CGROUP_MEM_RES_CTLR_KMEM)
269 With the Kernel memory extension, the Memory Controller is able to limit
270 the amount of kernel memory used by the system. Kernel memory is fundamentally
271 different than user memory, since it can't be swapped out, which makes it
272 possible to DoS the system by consuming too much of this precious resource.
274 Some kernel memory resources may be accounted and limited separately from the
275 main "kmem" resource. For instance, a slab cache that is considered important
276 enough to be limited separately may have its own knobs.
278 Kernel memory limits are not imposed for the root cgroup. Usage for the root
279 cgroup may or may not be accounted.
281 Memory limits as specified by the standard Memory Controller may or may not
282 take kernel memory into consideration. This is achieved through the file
283 memory.independent_kmem_limit. A Value different than 0 will allow for kernel
284 memory to be controlled separately.
286 When kernel memory limits are not independent, the limit values set in
287 memory.kmem files are ignored.
289 Currently no soft limit is implemented for kernel memory. It is future work
290 to trigger slab reclaim when those limits are reached.
292 2.7.1 Current Kernel Memory resources accounted
294 * sockets memory pressure: some sockets protocols have memory pressure
295 thresholds. The Memory Controller allows them to be controlled individually
296 per cgroup, instead of globally.
298 * tcp memory pressure: sockets memory pressure for the tcp protocol.
304 a. Enable CONFIG_CGROUPS
305 b. Enable CONFIG_RESOURCE_COUNTERS
306 c. Enable CONFIG_CGROUP_MEM_RES_CTLR
307 d. Enable CONFIG_CGROUP_MEM_RES_CTLR_SWAP (to use swap extension)
309 1. Prepare the cgroups (see cgroups.txt, Why are cgroups needed?)
310 # mount -t tmpfs none /sys/fs/cgroup
311 # mkdir /sys/fs/cgroup/memory
312 # mount -t cgroup none /sys/fs/cgroup/memory -o memory
314 2. Make the new group and move bash into it
315 # mkdir /sys/fs/cgroup/memory/0
316 # echo $$ > /sys/fs/cgroup/memory/0/tasks
318 Since now we're in the 0 cgroup, we can alter the memory limit:
319 # echo 4M > /sys/fs/cgroup/memory/0/memory.limit_in_bytes
321 NOTE: We can use a suffix (k, K, m, M, g or G) to indicate values in kilo,
322 mega or gigabytes. (Here, Kilo, Mega, Giga are Kibibytes, Mebibytes, Gibibytes.)
324 NOTE: We can write "-1" to reset the *.limit_in_bytes(unlimited).
325 NOTE: We cannot set limits on the root cgroup any more.
327 # cat /sys/fs/cgroup/memory/0/memory.limit_in_bytes
330 We can check the usage:
331 # cat /sys/fs/cgroup/memory/0/memory.usage_in_bytes
334 A successful write to this file does not guarantee a successful set of
335 this limit to the value written into the file. This can be due to a
336 number of factors, such as rounding up to page boundaries or the total
337 availability of memory on the system. The user is required to re-read
338 this file after a write to guarantee the value committed by the kernel.
340 # echo 1 > memory.limit_in_bytes
341 # cat memory.limit_in_bytes
344 The memory.failcnt field gives the number of times that the cgroup limit was
347 The memory.stat file gives accounting information. Now, the number of
348 caches, RSS and Active pages/Inactive pages are shown.
352 For testing features and implementation, see memcg_test.txt.
354 Performance test is also important. To see pure memory controller's overhead,
355 testing on tmpfs will give you good numbers of small overheads.
356 Example: do kernel make on tmpfs.
358 Page-fault scalability is also important. At measuring parallel
359 page fault test, multi-process test may be better than multi-thread
360 test because it has noise of shared objects/status.
362 But the above two are testing extreme situations.
363 Trying usual test under memory controller is always helpful.
367 Sometimes a user might find that the application under a cgroup is
368 terminated by OOM killer. There are several causes for this:
370 1. The cgroup limit is too low (just too low to do anything useful)
371 2. The user is using anonymous memory and swap is turned off or too low
373 A sync followed by echo 1 > /proc/sys/vm/drop_caches will help get rid of
374 some of the pages cached in the cgroup (page cache pages).
376 To know what happens, disable OOM_Kill by 10. OOM Control(see below) and
377 seeing what happens will be helpful.
381 When a task migrates from one cgroup to another, its charge is not
382 carried forward by default. The pages allocated from the original cgroup still
383 remain charged to it, the charge is dropped when the page is freed or
386 You can move charges of a task along with task migration.
387 See 8. "Move charges at task migration"
389 4.3 Removing a cgroup
391 A cgroup can be removed by rmdir, but as discussed in sections 4.1 and 4.2, a
392 cgroup might have some charge associated with it, even though all
393 tasks have migrated away from it. (because we charge against pages, not
396 Such charges are freed or moved to their parent. At moving, both of RSS
397 and CACHES are moved to parent.
398 rmdir() may return -EBUSY if freeing/moving fails. See 5.1 also.
400 Charges recorded in swap information is not updated at removal of cgroup.
401 Recorded information is discarded and a cgroup which uses swap (swapcache)
402 will be charged as a new owner of it.
408 memory.force_empty interface is provided to make cgroup's memory usage empty.
409 You can use this interface only when the cgroup has no tasks.
410 When writing anything to this
412 # echo 0 > memory.force_empty
414 Almost all pages tracked by this memory cgroup will be unmapped and freed.
415 Some pages cannot be freed because they are locked or in-use. Such pages are
416 moved to parent and this cgroup will be empty. This may return -EBUSY if
417 VM is too busy to free/move all pages immediately.
419 Typical use case of this interface is that calling this before rmdir().
420 Because rmdir() moves all pages to parent, some out-of-use page caches can be
421 moved to the parent. If you want to avoid that, force_empty will be useful.
425 memory.stat file includes following statistics
427 # per-memory cgroup local status
428 cache - # of bytes of page cache memory.
429 rss - # of bytes of anonymous and swap cache memory.
430 mapped_file - # of bytes of mapped file (includes tmpfs/shmem)
431 pgpgin - # of pages paged in (equivalent to # of charging events).
432 pgpgout - # of pages paged out (equivalent to # of uncharging events).
433 swap - # of bytes of swap usage
434 inactive_anon - # of bytes of anonymous memory and swap cache memory on
436 active_anon - # of bytes of anonymous and swap cache memory on active
438 inactive_file - # of bytes of file-backed memory on inactive LRU list.
439 active_file - # of bytes of file-backed memory on active LRU list.
440 unevictable - # of bytes of memory that cannot be reclaimed (mlocked etc).
442 # status considering hierarchy (see memory.use_hierarchy settings)
444 hierarchical_memory_limit - # of bytes of memory limit with regard to hierarchy
445 under which the memory cgroup is
446 hierarchical_memsw_limit - # of bytes of memory+swap limit with regard to
447 hierarchy under which memory cgroup is.
449 total_cache - sum of all children's "cache"
450 total_rss - sum of all children's "rss"
451 total_mapped_file - sum of all children's "cache"
452 total_pgpgin - sum of all children's "pgpgin"
453 total_pgpgout - sum of all children's "pgpgout"
454 total_swap - sum of all children's "swap"
455 total_inactive_anon - sum of all children's "inactive_anon"
456 total_active_anon - sum of all children's "active_anon"
457 total_inactive_file - sum of all children's "inactive_file"
458 total_active_file - sum of all children's "active_file"
459 total_unevictable - sum of all children's "unevictable"
461 # The following additional stats are dependent on CONFIG_DEBUG_VM.
463 recent_rotated_anon - VM internal parameter. (see mm/vmscan.c)
464 recent_rotated_file - VM internal parameter. (see mm/vmscan.c)
465 recent_scanned_anon - VM internal parameter. (see mm/vmscan.c)
466 recent_scanned_file - VM internal parameter. (see mm/vmscan.c)
469 recent_rotated means recent frequency of LRU rotation.
470 recent_scanned means recent # of scans to LRU.
471 showing for better debug please see the code for meanings.
474 Only anonymous and swap cache memory is listed as part of 'rss' stat.
475 This should not be confused with the true 'resident set size' or the
476 amount of physical memory used by the cgroup.
477 'rss + file_mapped" will give you resident set size of cgroup.
478 (Note: file and shmem may be shared among other cgroups. In that case,
479 file_mapped is accounted only when the memory cgroup is owner of page
484 Similar to /proc/sys/vm/swappiness, but affecting a hierarchy of groups only.
486 Following cgroups' swappiness can't be changed.
487 - root cgroup (uses /proc/sys/vm/swappiness).
488 - a cgroup which uses hierarchy and it has other cgroup(s) below it.
489 - a cgroup which uses hierarchy and not the root of hierarchy.
493 A memory cgroup provides memory.failcnt and memory.memsw.failcnt files.
494 This failcnt(== failure count) shows the number of times that a usage counter
495 hit its limit. When a memory cgroup hits a limit, failcnt increases and
496 memory under it will be reclaimed.
498 You can reset failcnt by writing 0 to failcnt file.
499 # echo 0 > .../memory.failcnt
503 For efficiency, as other kernel components, memory cgroup uses some optimization
504 to avoid unnecessary cacheline false sharing. usage_in_bytes is affected by the
505 method and doesn't show 'exact' value of memory(and swap) usage, it's an fuzz
506 value for efficient access. (Of course, when necessary, it's synchronized.)
507 If you want to know more exact memory usage, you should use RSS+CACHE(+SWAP)
508 value in memory.stat(see 5.2).
512 This is similar to numa_maps but operates on a per-memcg basis. This is
513 useful for providing visibility into the numa locality information within
514 an memcg since the pages are allowed to be allocated from any physical
515 node. One of the usecases is evaluating application performance by
516 combining this information with the application's cpu allocation.
518 We export "total", "file", "anon" and "unevictable" pages per-node for
519 each memcg. The ouput format of memory.numa_stat is:
521 total=<total pages> N0=<node 0 pages> N1=<node 1 pages> ...
522 file=<total file pages> N0=<node 0 pages> N1=<node 1 pages> ...
523 anon=<total anon pages> N0=<node 0 pages> N1=<node 1 pages> ...
524 unevictable=<total anon pages> N0=<node 0 pages> N1=<node 1 pages> ...
526 And we have total = file + anon + unevictable.
530 The memory controller supports a deep hierarchy and hierarchical accounting.
531 The hierarchy is created by creating the appropriate cgroups in the
532 cgroup filesystem. Consider for example, the following cgroup filesystem
543 In the diagram above, with hierarchical accounting enabled, all memory
544 usage of e, is accounted to its ancestors up until the root (i.e, c and root),
545 that has memory.use_hierarchy enabled. If one of the ancestors goes over its
546 limit, the reclaim algorithm reclaims from the tasks in the ancestor and the
547 children of the ancestor.
549 6.1 Enabling hierarchical accounting and reclaim
551 A memory cgroup by default disables the hierarchy feature. Support
552 can be enabled by writing 1 to memory.use_hierarchy file of the root cgroup
554 # echo 1 > memory.use_hierarchy
556 The feature can be disabled by
558 # echo 0 > memory.use_hierarchy
560 NOTE1: Enabling/disabling will fail if either the cgroup already has other
561 cgroups created below it, or if the parent cgroup has use_hierarchy
564 NOTE2: When panic_on_oom is set to "2", the whole system will panic in
565 case of an OOM event in any cgroup.
569 Soft limits allow for greater sharing of memory. The idea behind soft limits
570 is to allow control groups to use as much of the memory as needed, provided
572 a. There is no memory contention
573 b. They do not exceed their hard limit
575 When the system detects memory contention or low memory, control groups
576 are pushed back to their soft limits. If the soft limit of each control
577 group is very high, they are pushed back as much as possible to make
578 sure that one control group does not starve the others of memory.
580 Please note that soft limits is a best effort feature, it comes with
581 no guarantees, but it does its best to make sure that when memory is
582 heavily contended for, memory is allocated based on the soft limit
583 hints/setup. Currently soft limit based reclaim is setup such that
584 it gets invoked from balance_pgdat (kswapd).
588 Soft limits can be setup by using the following commands (in this example we
589 assume a soft limit of 256 MiB)
591 # echo 256M > memory.soft_limit_in_bytes
593 If we want to change this to 1G, we can at any time use
595 # echo 1G > memory.soft_limit_in_bytes
597 NOTE1: Soft limits take effect over a long period of time, since they involve
598 reclaiming memory for balancing between memory cgroups
599 NOTE2: It is recommended to set the soft limit always below the hard limit,
600 otherwise the hard limit will take precedence.
602 8. Move charges at task migration
604 Users can move charges associated with a task along with task migration, that
605 is, uncharge task's pages from the old cgroup and charge them to the new cgroup.
606 This feature is not supported in !CONFIG_MMU environments because of lack of
611 This feature is disabled by default. It can be enabled(and disabled again) by
612 writing to memory.move_charge_at_immigrate of the destination cgroup.
614 If you want to enable it:
616 # echo (some positive value) > memory.move_charge_at_immigrate
618 Note: Each bits of move_charge_at_immigrate has its own meaning about what type
619 of charges should be moved. See 8.2 for details.
620 Note: Charges are moved only when you move mm->owner, IOW, a leader of a thread
622 Note: If we cannot find enough space for the task in the destination cgroup, we
623 try to make space by reclaiming memory. Task migration may fail if we
624 cannot make enough space.
625 Note: It can take several seconds if you move charges much.
627 And if you want disable it again:
629 # echo 0 > memory.move_charge_at_immigrate
631 8.2 Type of charges which can be move
633 Each bits of move_charge_at_immigrate has its own meaning about what type of
634 charges should be moved. But in any cases, it must be noted that an account of
635 a page or a swap can be moved only when it is charged to the task's current(old)
638 bit | what type of charges would be moved ?
639 -----+------------------------------------------------------------------------
640 0 | A charge of an anonymous page(or swap of it) used by the target task.
641 | Those pages and swaps must be used only by the target task. You must
642 | enable Swap Extension(see 2.4) to enable move of swap charges.
643 -----+------------------------------------------------------------------------
644 1 | A charge of file pages(normal file, tmpfs file(e.g. ipc shared memory)
645 | and swaps of tmpfs file) mmapped by the target task. Unlike the case of
646 | anonymous pages, file pages(and swaps) in the range mmapped by the task
647 | will be moved even if the task hasn't done page fault, i.e. they might
648 | not be the task's "RSS", but other task's "RSS" that maps the same file.
649 | And mapcount of the page is ignored(the page can be moved even if
650 | page_mapcount(page) > 1). You must enable Swap Extension(see 2.4) to
651 | enable move of swap charges.
655 - Implement madvise(2) to let users decide the vma to be moved or not to be
657 - All of moving charge operations are done under cgroup_mutex. It's not good
658 behavior to hold the mutex too long, so we may need some trick.
662 Memory cgroup implements memory thresholds using cgroups notification
663 API (see cgroups.txt). It allows to register multiple memory and memsw
664 thresholds and gets notifications when it crosses.
666 To register a threshold application need:
667 - create an eventfd using eventfd(2);
668 - open memory.usage_in_bytes or memory.memsw.usage_in_bytes;
669 - write string like "<event_fd> <fd of memory.usage_in_bytes> <threshold>" to
670 cgroup.event_control.
672 Application will be notified through eventfd when memory usage crosses
673 threshold in any direction.
675 It's applicable for root and non-root cgroup.
679 memory.oom_control file is for OOM notification and other controls.
681 Memory cgroup implements OOM notifier using cgroup notification
682 API (See cgroups.txt). It allows to register multiple OOM notification
683 delivery and gets notification when OOM happens.
685 To register a notifier, application need:
686 - create an eventfd using eventfd(2)
687 - open memory.oom_control file
688 - write string like "<event_fd> <fd of memory.oom_control>" to
691 Application will be notified through eventfd when OOM happens.
692 OOM notification doesn't work for root cgroup.
694 You can disable OOM-killer by writing "1" to memory.oom_control file, as:
696 #echo 1 > memory.oom_control
698 This operation is only allowed to the top cgroup of sub-hierarchy.
699 If OOM-killer is disabled, tasks under cgroup will hang/sleep
700 in memory cgroup's OOM-waitqueue when they request accountable memory.
702 For running them, you have to relax the memory cgroup's OOM status by
703 * enlarge limit or reduce usage.
706 * move some tasks to other group with account migration.
707 * remove some files (on tmpfs?)
709 Then, stopped tasks will work again.
711 At reading, current status of OOM is shown.
712 oom_kill_disable 0 or 1 (if 1, oom-killer is disabled)
713 under_oom 0 or 1 (if 1, the memory cgroup is under OOM, tasks may
718 1. Add support for accounting huge pages (as a separate controller)
719 2. Make per-cgroup scanner reclaim not-shared pages first
720 3. Teach controller to account for shared-pages
721 4. Start reclamation in the background when the limit is
722 not yet hit but the usage is getting closer
726 Overall, the memory controller has been a stable controller and has been
727 commented and discussed quite extensively in the community.
731 1. Singh, Balbir. RFC: Memory Controller, http://lwn.net/Articles/206697/
732 2. Singh, Balbir. Memory Controller (RSS Control),
733 http://lwn.net/Articles/222762/
734 3. Emelianov, Pavel. Resource controllers based on process cgroups
735 http://lkml.org/lkml/2007/3/6/198
736 4. Emelianov, Pavel. RSS controller based on process cgroups (v2)
737 http://lkml.org/lkml/2007/4/9/78
738 5. Emelianov, Pavel. RSS controller based on process cgroups (v3)
739 http://lkml.org/lkml/2007/5/30/244
740 6. Menage, Paul. Control Groups v10, http://lwn.net/Articles/236032/
741 7. Vaidyanathan, Srinivasan, Control Groups: Pagecache accounting and control
742 subsystem (v3), http://lwn.net/Articles/235534/
743 8. Singh, Balbir. RSS controller v2 test results (lmbench),
744 http://lkml.org/lkml/2007/5/17/232
745 9. Singh, Balbir. RSS controller v2 AIM9 results
746 http://lkml.org/lkml/2007/5/18/1
747 10. Singh, Balbir. Memory controller v6 test results,
748 http://lkml.org/lkml/2007/8/19/36
749 11. Singh, Balbir. Memory controller introduction (v6),
750 http://lkml.org/lkml/2007/8/17/69
751 12. Corbet, Jonathan, Controlling memory use in cgroups,
752 http://lwn.net/Articles/243795/