[linux-2.6-block.git] / Documentation / vm / overcommit-accounting.rst

.. _overcommit_accounting:

=====================
Overcommit Accounting
=====================

The Linux kernel supports the following overcommit handling modes

0
	Heuristic overcommit handling. Obvious overcommits of address
	space are refused. Used for a typical system. It ensures a
	seriously wild allocation fails while allowing overcommit to
	reduce swap usage.  root is allowed to allocate slightly more
	memory in this mode. This is the default.

1
	Always overcommit. Appropriate for some scientific
	applications. Classic example is code using sparse arrays and
	just relying on the virtual memory consisting almost entirely
	of zero pages.

2
	Don't overcommit. The total address space commit for the
	system is not permitted to exceed swap + a configurable amount
	(default is 50%) of physical RAM.  Depending on the amount you
	use, in most situations this means a process will not be
	killed while accessing pages but will receive errors on memory
	allocation as appropriate.

	Useful for applications that want to guarantee their memory
	allocations will be available in the future without having to
	initialize every page.

The overcommit policy is set via the sysctl ``vm.overcommit_memory``.

The overcommit amount can be set via ``vm.overcommit_ratio`` (percentage)
or ``vm.overcommit_kbytes`` (absolute value).

The current overcommit limit and amount committed are viewable in
``/proc/meminfo`` as CommitLimit and Committed_AS respectively.

Gotchas
=======

The C language stack growth does an implicit mremap. If you want absolute
guarantees and run close to the edge you MUST mmap your stack for the
largest size you think you will need. For typical stack usage this does
not matter much but it's a corner case if you really really care

In mode 2 the MAP_NORESERVE flag is ignored.


How It Works
============

The overcommit is based on the following rules

For a file backed map
	| SHARED or READ-only	-	0 cost (the file is the map not swap)
	| PRIVATE WRITABLE	-	size of mapping per instance

For an anonymous or ``/dev/zero`` map
	| SHARED			-	size of mapping
	| PRIVATE READ-only	-	0 cost (but of little use)
	| PRIVATE WRITABLE	-	size of mapping per instance

Additional accounting
	| Pages made writable copies by mmap
	| shmfs memory drawn from the same pool

Status
======

*	We account mmap memory mappings
*	We account mprotect changes in commit
*	We account mremap changes in size
*	We account brk
*	We account munmap
*	We report the commit status in /proc
*	Account and check on fork
*	Review stack handling/building on exec
*	SHMfs accounting
*	Implement actual limit enforcement

To Do
=====
*	Account ptrace pages (this is hard)
Commit	Line	Data
8d83d826 MR	1	.. _overcommit_accounting:
	2
	3	=====================
	4	Overcommit Accounting
	5	=====================
	6
1da177e4 LT	7	The Linux kernel supports the following overcommit handling modes
1da177e4 LT	8
8d83d826 MR	9	0
	10	Heuristic overcommit handling. Obvious overcommits of address
	11	space are refused. Used for a typical system. It ensures a
	12	seriously wild allocation fails while allowing overcommit to
	13	reduce swap usage. root is allowed to allocate slightly more
	14	memory in this mode. This is the default.
1da177e4	15
8d83d826 MR	16	1
	17	Always overcommit. Appropriate for some scientific
	18	applications. Classic example is code using sparse arrays and
	19	just relying on the virtual memory consisting almost entirely
	20	of zero pages.
1da177e4	21
8d83d826 MR	22	2
	23	Don't overcommit. The total address space commit for the
	24	system is not permitted to exceed swap + a configurable amount
	25	(default is 50%) of physical RAM. Depending on the amount you
	26	use, in most situations this means a process will not be
	27	killed while accessing pages but will receive errors on memory
	28	allocation as appropriate.
1da177e4	29
8d83d826 MR	30	Useful for applications that want to guarantee their memory
	31	allocations will be available in the future without having to
	32	initialize every page.
c9b1d098	33
8d83d826	34	The overcommit policy is set via the sysctl ``vm.overcommit_memory``.
1da177e4	35
8d83d826 MR	36	The overcommit amount can be set via ``vm.overcommit_ratio`` (percentage)
8d83d826 MR	37	or ``vm.overcommit_kbytes`` (absolute value).
1da177e4 LT	38
1da177e4 LT	39	The current overcommit limit and amount committed are viewable in
8d83d826	40	``/proc/meminfo`` as CommitLimit and Committed_AS respectively.
1da177e4 LT	41
1da177e4 LT	42	Gotchas
8d83d826	43	=======
1da177e4 LT	44
1da177e4 LT	45	The C language stack growth does an implicit mremap. If you want absolute
8d83d826	46	guarantees and run close to the edge you MUST mmap your stack for the
1da177e4 LT	47	largest size you think you will need. For typical stack usage this does
	48	not matter much but it's a corner case if you really really care
	49
8d83d826	50	In mode 2 the MAP_NORESERVE flag is ignored.
1da177e4 LT	51
	52
	53	How It Works
8d83d826	54	============
1da177e4 LT	55
	56	The overcommit is based on the following rules
	57
	58	For a file backed map
8d83d826 MR	59	\| SHARED or READ-only - 0 cost (the file is the map not swap)
8d83d826 MR	60	\| PRIVATE WRITABLE - size of mapping per instance
1da177e4	61
8d83d826 MR	62	For an anonymous or ``/dev/zero`` map
	63	\| SHARED - size of mapping
	64	\| PRIVATE READ-only - 0 cost (but of little use)
	65	\| PRIVATE WRITABLE - size of mapping per instance
1da177e4 LT	66
1da177e4 LT	67	Additional accounting
8d83d826 MR	68	\| Pages made writable copies by mmap
8d83d826 MR	69	\| shmfs memory drawn from the same pool
1da177e4 LT	70
1da177e4 LT	71	Status
8d83d826 MR	72	======
	73
	74	* We account mmap memory mappings
	75	* We account mprotect changes in commit
	76	* We account mremap changes in size
	77	* We account brk
	78	* We account munmap
	79	* We report the commit status in /proc
	80	* Account and check on fork
	81	* Review stack handling/building on exec
	82	* SHMfs accounting
	83	* Implement actual limit enforcement
1da177e4 LT	84
1da177e4 LT	85	To Do
8d83d826 MR	86	=====
8d83d826 MR	87	* Account ptrace pages (this is hard)