Commit | Line | Data |
---|---|---|
148723f7 MR |
1 | ============= |
2 | HugeTLB Pages | |
3 | ============= | |
4 | ||
5 | Overview | |
6 | ======== | |
1da177e4 LT |
7 | |
8 | The intent of this file is to give a brief summary of hugetlbpage support in | |
9 | the Linux kernel. This support is built on top of multiple page size support | |
c0d7305c MI |
10 | that is provided by most modern architectures. For example, x86 CPUs normally |
11 | support 4K and 2M (1G if architecturally supported) page sizes, ia64 | |
1da177e4 LT |
12 | architecture supports multiple page sizes 4K, 8K, 64K, 256K, 1M, 4M, 16M, |
13 | 256M and ppc64 supports 4K and 16M. A TLB is a cache of virtual-to-physical | |
14 | translations. Typically this is a very scarce resource on processor. | |
15 | Operating systems try to make best use of limited number of TLB resources. | |
16 | This optimization is more critical now as bigger and bigger physical memories | |
17 | (several GBs) are more readily available. | |
18 | ||
19 | Users can use the huge page support in Linux kernel by either using the mmap | |
267b4c28 | 20 | system call or standard SYSV shared memory system calls (shmget, shmat). |
1da177e4 | 21 | |
5c7ad510 MBY |
22 | First the Linux kernel needs to be built with the CONFIG_HUGETLBFS |
23 | (present under "File systems") and CONFIG_HUGETLB_PAGE (selected | |
24 | automatically when CONFIG_HUGETLBFS is selected) configuration | |
25 | options. | |
1da177e4 | 26 | |
148723f7 | 27 | The ``/proc/meminfo`` file provides information about the total number of |
267b4c28 | 28 | persistent hugetlb pages in the kernel's huge page pool. It also displays |
fcb2b0c5 RG |
29 | default huge page size and information about the number of free, reserved |
30 | and surplus huge pages in the pool of huge pages of default size. | |
31 | The huge page size is needed for generating the proper alignment and | |
32 | size of the arguments to system calls that map huge page regions. | |
1da177e4 | 33 | |
148723f7 | 34 | The output of ``cat /proc/meminfo`` will include lines like:: |
1da177e4 | 35 | |
148723f7 MR |
36 | HugePages_Total: uuu |
37 | HugePages_Free: vvv | |
38 | HugePages_Rsvd: www | |
39 | HugePages_Surp: xxx | |
40 | Hugepagesize: yyy kB | |
41 | Hugetlb: zzz kB | |
5e122271 RD |
42 | |
43 | where: | |
148723f7 MR |
44 | |
45 | HugePages_Total | |
46 | is the size of the pool of huge pages. | |
47 | HugePages_Free | |
48 | is the number of huge pages in the pool that are not yet | |
49 | allocated. | |
50 | HugePages_Rsvd | |
51 | is short for "reserved," and is the number of huge pages for | |
52 | which a commitment to allocate from the pool has been made, | |
53 | but no allocation has yet been made. Reserved huge pages | |
54 | guarantee that an application will be able to allocate a | |
55 | huge page from the pool of huge pages at fault time. | |
56 | HugePages_Surp | |
57 | is short for "surplus," and is the number of huge pages in | |
58 | the pool above the value in ``/proc/sys/vm/nr_hugepages``. The | |
59 | maximum number of surplus huge pages is controlled by | |
60 | ``/proc/sys/vm/nr_overcommit_hugepages``. | |
ad2fa371 MS |
61 | Note: When the feature of freeing unused vmemmap pages associated |
62 | with each hugetlb page is enabled, the number of surplus huge pages | |
63 | may be temporarily larger than the maximum number of surplus huge | |
64 | pages when the system is under memory pressure. | |
148723f7 | 65 | Hugepagesize |
16461c66 | 66 | is the default hugepage size (in kB). |
148723f7 MR |
67 | Hugetlb |
68 | is the total amount of memory (in kB), consumed by huge | |
69 | pages of all sizes. | |
70 | If huge pages of different sizes are in use, this number | |
71 | will exceed HugePages_Total \* Hugepagesize. To get more | |
72 | detailed information, please, refer to | |
73 | ``/sys/kernel/mm/hugepages`` (described below). | |
74 | ||
75 | ||
76 | ``/proc/filesystems`` should also show a filesystem of type "hugetlbfs" | |
77 | configured in the kernel. | |
78 | ||
79 | ``/proc/sys/vm/nr_hugepages`` indicates the current number of "persistent" huge | |
267b4c28 LS |
80 | pages in the kernel's huge page pool. "Persistent" huge pages will be |
81 | returned to the huge page pool when freed by a task. A user with root | |
82 | privileges can dynamically allocate more or free some persistent huge pages | |
148723f7 | 83 | by increasing or decreasing the value of ``nr_hugepages``. |
1da177e4 | 84 | |
ad2fa371 MS |
85 | Note: When the feature of freeing unused vmemmap pages associated with each |
86 | hugetlb page is enabled, we can fail to free the huge pages triggered by | |
dbeb56fe | 87 | the user when the system is under memory pressure. Please try again later. |
ad2fa371 | 88 | |
267b4c28 LS |
89 | Pages that are used as huge pages are reserved inside the kernel and cannot |
90 | be used for other purposes. Huge pages cannot be swapped out under | |
91 | memory pressure. | |
1da177e4 | 92 | |
267b4c28 LS |
93 | Once a number of huge pages have been pre-allocated to the kernel huge page |
94 | pool, a user with appropriate privilege can use either the mmap system call | |
95 | or shared memory system calls to use the huge pages. See the discussion of | |
fde7917f | 96 | :ref:`Using Huge Pages <using_huge_pages>`, below. |
1da177e4 | 97 | |
267b4c28 LS |
98 | The administrator can allocate persistent huge pages on the kernel boot |
99 | command line by specifying the "hugepages=N" parameter, where 'N' = the | |
100 | number of huge pages requested. This is the most reliable method of | |
101 | allocating huge pages as memory has not yet become fragmented. | |
41a25e7e | 102 | |
267b4c28 | 103 | Some platforms support multiple huge page sizes. To allocate huge pages |
25985edc | 104 | of a specific size, one must precede the huge pages boot command parameters |
41a25e7e LS |
105 | with a huge page size selection parameter "hugepagesz=<size>". <size> must |
106 | be specified in bytes with optional scale suffix [kKmMgG]. The default huge | |
107 | page size may be selected with the "default_hugepagesz=<size>" boot parameter. | |
108 | ||
282f4214 | 109 | Hugetlb boot command line parameter semantics |
72a3e3e2 MCC |
110 | |
111 | hugepagesz | |
112 | Specify a huge page size. Used in conjunction with hugepages | |
282f4214 MK |
113 | parameter to preallocate a number of huge pages of the specified |
114 | size. Hence, hugepagesz and hugepages are typically specified in | |
72a3e3e2 MCC |
115 | pairs such as:: |
116 | ||
282f4214 | 117 | hugepagesz=2M hugepages=512 |
72a3e3e2 | 118 | |
282f4214 MK |
119 | hugepagesz can only be specified once on the command line for a |
120 | specific huge page size. Valid huge page sizes are architecture | |
121 | dependent. | |
72a3e3e2 MCC |
122 | hugepages |
123 | Specify the number of huge pages to preallocate. This typically | |
282f4214 MK |
124 | follows a valid hugepagesz or default_hugepagesz parameter. However, |
125 | if hugepages is the first or only hugetlb command line parameter it | |
126 | implicitly specifies the number of huge pages of default size to | |
127 | allocate. If the number of huge pages of default size is implicitly | |
128 | specified, it can not be overwritten by a hugepagesz,hugepages | |
b5389086 ZY |
129 | parameter pair for the default size. This parameter also has a |
130 | node format. The node format specifies the number of huge pages | |
131 | to allocate on specific nodes. | |
72a3e3e2 MCC |
132 | |
133 | For example, on an architecture with 2M default huge page size:: | |
134 | ||
282f4214 | 135 | hugepages=256 hugepagesz=2M hugepages=512 |
72a3e3e2 | 136 | |
282f4214 MK |
137 | will result in 256 2M huge pages being allocated and a warning message |
138 | indicating that the hugepages=512 parameter is ignored. If a hugepages | |
139 | parameter is preceded by an invalid hugepagesz parameter, it will | |
140 | be ignored. | |
b5389086 ZY |
141 | |
142 | Node format example:: | |
143 | ||
144 | hugepagesz=2M hugepages=0:1,1:2 | |
145 | ||
146 | It will allocate 1 2M hugepage on node0 and 2 2M hugepages on node1. | |
147 | If the node number is invalid, the parameter will be ignored. | |
148 | ||
72a3e3e2 | 149 | default_hugepagesz |
540809be | 150 | Specify the default huge page size. This parameter can |
282f4214 MK |
151 | only be specified once on the command line. default_hugepagesz can |
152 | optionally be followed by the hugepages parameter to preallocate a | |
153 | specific number of huge pages of default size. The number of default | |
154 | sized huge pages to preallocate can also be implicitly specified as | |
155 | mentioned in the hugepages section above. Therefore, on an | |
72a3e3e2 MCC |
156 | architecture with 2M default huge page size:: |
157 | ||
282f4214 MK |
158 | hugepages=256 |
159 | default_hugepagesz=2M hugepages=256 | |
160 | hugepages=256 default_hugepagesz=2M | |
72a3e3e2 | 161 | |
282f4214 MK |
162 | will all result in 256 2M huge pages being allocated. Valid default |
163 | huge page size is architecture dependent. | |
e9fdff87 | 164 | hugetlb_free_vmemmap |
dff03381 MS |
165 | When CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP is set, this enables HugeTLB |
166 | Vmemmap Optimization (HVO). | |
282f4214 | 167 | |
148723f7 | 168 | When multiple huge page sizes are supported, ``/proc/sys/vm/nr_hugepages`` |
267b4c28 LS |
169 | indicates the current number of pre-allocated huge pages of the default size. |
170 | Thus, one can use the following command to dynamically allocate/deallocate | |
148723f7 | 171 | default sized persistent huge pages:: |
1da177e4 LT |
172 | |
173 | echo 20 > /proc/sys/vm/nr_hugepages | |
174 | ||
267b4c28 LS |
175 | This command will try to adjust the number of default sized huge pages in the |
176 | huge page pool to 20, allocating or freeing huge pages, as required. | |
177 | ||
41a25e7e | 178 | On a NUMA platform, the kernel will attempt to distribute the huge page pool |
267b4c28 | 179 | over all the set of allowed nodes specified by the NUMA memory policy of the |
148723f7 | 180 | task that modifies ``nr_hugepages``. The default for the allowed nodes--when the |
9b5e5d0f LS |
181 | task has default memory policy--is all on-line nodes with memory. Allowed |
182 | nodes with insufficient available, contiguous memory for a huge page will be | |
fde7917f MR |
183 | silently skipped when allocating persistent huge pages. See the |
184 | :ref:`discussion below <mem_policy_and_hp_alloc>` | |
185 | of the interaction of task memory policy, cpusets and per node attributes | |
9b5e5d0f | 186 | with the allocation and freeing of persistent huge pages. |
41a25e7e LS |
187 | |
188 | The success or failure of huge page allocation depends on the amount of | |
267b4c28 | 189 | physically contiguous memory that is present in system at the time of the |
41a25e7e LS |
190 | allocation attempt. If the kernel is unable to allocate huge pages from |
191 | some nodes in a NUMA system, it will attempt to make up the difference by | |
192 | allocating extra pages on other nodes with sufficient available contiguous | |
193 | memory, if any. | |
194 | ||
267b4c28 LS |
195 | System administrators may want to put this command in one of the local rc |
196 | init files. This will enable the kernel to allocate huge pages early in | |
197 | the boot process when the possibility of getting physical contiguous pages | |
198 | is still very high. Administrators can verify the number of huge pages | |
199 | actually allocated by checking the sysctl or meminfo. To check the per node | |
148723f7 | 200 | distribution of huge pages in a NUMA system, use:: |
41a25e7e LS |
201 | |
202 | cat /sys/devices/system/node/node*/meminfo | fgrep Huge | |
203 | ||
148723f7 MR |
204 | ``/proc/sys/vm/nr_overcommit_hugepages`` specifies how large the pool of |
205 | huge pages can grow, if more huge pages than ``/proc/sys/vm/nr_hugepages`` are | |
41a25e7e | 206 | requested by applications. Writing any non-zero value into this file |
267b4c28 LS |
207 | indicates that the hugetlb subsystem is allowed to try to obtain that |
208 | number of "surplus" huge pages from the kernel's normal page pool, when the | |
209 | persistent huge page pool is exhausted. As these surplus huge pages become | |
210 | unused, they are freed back to the kernel's normal page pool. | |
d5dbac87 | 211 | |
148723f7 MR |
212 | When increasing the huge page pool size via ``nr_hugepages``, any existing |
213 | surplus pages will first be promoted to persistent huge pages. Then, additional | |
41a25e7e | 214 | huge pages will be allocated, if necessary and if possible, to fulfill |
267b4c28 | 215 | the new persistent huge page pool size. |
41a25e7e | 216 | |
267b4c28 | 217 | The administrator may shrink the pool of persistent huge pages for |
148723f7 | 218 | the default huge page size by setting the ``nr_hugepages`` sysctl to a |
41a25e7e | 219 | smaller value. The kernel will attempt to balance the freeing of huge pages |
148723f7 | 220 | across all nodes in the memory policy of the task modifying ``nr_hugepages``. |
267b4c28 LS |
221 | Any free huge pages on the selected nodes will be freed back to the kernel's |
222 | normal page pool. | |
223 | ||
148723f7 | 224 | Caveat: Shrinking the persistent huge page pool via ``nr_hugepages`` such that |
267b4c28 LS |
225 | it becomes less than the number of huge pages in use will convert the balance |
226 | of the in-use huge pages to surplus huge pages. This will occur even if | |
fde7917f | 227 | the number of surplus pages would exceed the overcommit value. As long as |
148723f7 | 228 | this condition holds--that is, until ``nr_hugepages+nr_overcommit_hugepages`` is |
267b4c28 LS |
229 | increased sufficiently, or the surplus huge pages go out of use and are freed-- |
230 | no more surplus huge pages will be allowed to be allocated. | |
1da177e4 | 231 | |
41a25e7e | 232 | With support for multiple huge page pools at run-time available, much of |
148723f7 MR |
233 | the huge page userspace interface in ``/proc/sys/vm`` has been duplicated in |
234 | sysfs. | |
235 | The ``/proc`` interfaces discussed above have been retained for backwards | |
236 | compatibility. The root huge page control directory in sysfs is:: | |
a3437870 NA |
237 | |
238 | /sys/kernel/mm/hugepages | |
239 | ||
41a25e7e | 240 | For each huge page size supported by the running kernel, a subdirectory |
148723f7 | 241 | will exist, of the form:: |
a3437870 NA |
242 | |
243 | hugepages-${size}kB | |
244 | ||
79dfc695 MK |
245 | Inside each of these directories, the set of files contained in ``/proc`` |
246 | will exist. In addition, two additional interfaces for demoting huge | |
247 | pages may exist:: | |
a3437870 | 248 | |
79dfc695 MK |
249 | demote |
250 | demote_size | |
a3437870 | 251 | nr_hugepages |
267b4c28 | 252 | nr_hugepages_mempolicy |
a3437870 NA |
253 | nr_overcommit_hugepages |
254 | free_hugepages | |
255 | resv_hugepages | |
256 | surplus_hugepages | |
257 | ||
79dfc695 MK |
258 | The demote interfaces provide the ability to split a huge page into |
259 | smaller huge pages. For example, the x86 architecture supports both | |
260 | 1GB and 2MB huge pages sizes. A 1GB huge page can be split into 512 | |
261 | 2MB huge pages. Demote interfaces are not available for the smallest | |
262 | huge page size. The demote interfaces are: | |
263 | ||
264 | demote_size | |
265 | is the size of demoted pages. When a page is demoted a corresponding | |
266 | number of huge pages of demote_size will be created. By default, | |
267 | demote_size is set to the next smaller huge page size. If there are | |
268 | multiple smaller huge page sizes, demote_size can be set to any of | |
269 | these smaller sizes. Only huge page sizes less than the current huge | |
270 | pages size are allowed. | |
271 | ||
272 | demote | |
273 | is used to demote a number of huge pages. A user with root privileges | |
274 | can write to this file. It may not be possible to demote the | |
275 | requested number of huge pages. To determine how many pages were | |
276 | actually demoted, compare the value of nr_hugepages before and after | |
277 | writing to the demote interface. demote is a write only interface. | |
278 | ||
279 | The interfaces which are the same as in ``/proc`` (all except demote and | |
280 | demote_size) function as described above for the default huge page-sized case. | |
a3437870 | 281 | |
fde7917f | 282 | .. _mem_policy_and_hp_alloc: |
267b4c28 LS |
283 | |
284 | Interaction of Task Memory Policy with Huge Page Allocation/Freeing | |
15610c86 | 285 | =================================================================== |
267b4c28 | 286 | |
148723f7 MR |
287 | Whether huge pages are allocated and freed via the ``/proc`` interface or |
288 | the ``/sysfs`` interface using the ``nr_hugepages_mempolicy`` attribute, the | |
289 | NUMA nodes from which huge pages are allocated or freed are controlled by the | |
290 | NUMA memory policy of the task that modifies the ``nr_hugepages_mempolicy`` | |
291 | sysctl or attribute. When the ``nr_hugepages`` attribute is used, mempolicy | |
267b4c28 LS |
292 | is ignored. |
293 | ||
294 | The recommended method to allocate or free huge pages to/from the kernel | |
148723f7 | 295 | huge page pool, using the ``nr_hugepages`` example above, is:: |
267b4c28 LS |
296 | |
297 | numactl --interleave <node-list> echo 20 \ | |
298 | >/proc/sys/vm/nr_hugepages_mempolicy | |
299 | ||
148723f7 | 300 | or, more succinctly:: |
267b4c28 LS |
301 | |
302 | numactl -m <node-list> echo 20 >/proc/sys/vm/nr_hugepages_mempolicy | |
303 | ||
148723f7 | 304 | This will allocate or free ``abs(20 - nr_hugepages)`` to or from the nodes |
267b4c28 LS |
305 | specified in <node-list>, depending on whether number of persistent huge pages |
306 | is initially less than or greater than 20, respectively. No huge pages will be | |
307 | allocated nor freed on any node not included in the specified <node-list>. | |
308 | ||
148723f7 | 309 | When adjusting the persistent hugepage count via ``nr_hugepages_mempolicy``, any |
267b4c28 LS |
310 | memory policy mode--bind, preferred, local or interleave--may be used. The |
311 | resulting effect on persistent huge page allocation is as follows: | |
312 | ||
e27a20f1 | 313 | #. Regardless of mempolicy mode [see |
00cba6b6 | 314 | Documentation/admin-guide/mm/numa_memory_policy.rst], |
267b4c28 LS |
315 | persistent huge pages will be distributed across the node or nodes |
316 | specified in the mempolicy as if "interleave" had been specified. | |
317 | However, if a node in the policy does not contain sufficient contiguous | |
318 | memory for a huge page, the allocation will not "fallback" to the nearest | |
319 | neighbor node with sufficient contiguous memory. To do this would cause | |
320 | undesirable imbalance in the distribution of the huge page pool, or | |
321 | possibly, allocation of persistent huge pages on nodes not allowed by | |
322 | the task's memory policy. | |
323 | ||
148723f7 | 324 | #. One or more nodes may be specified with the bind or interleave policy. |
267b4c28 LS |
325 | If more than one node is specified with the preferred policy, only the |
326 | lowest numeric id will be used. Local policy will select the node where | |
327 | the task is running at the time the nodes_allowed mask is constructed. | |
328 | For local policy to be deterministic, the task must be bound to a cpu or | |
329 | cpus in a single node. Otherwise, the task could be migrated to some | |
330 | other node at any time after launch and the resulting node will be | |
331 | indeterminate. Thus, local policy is not very useful for this purpose. | |
332 | Any of the other mempolicy modes may be used to specify a single node. | |
333 | ||
148723f7 | 334 | #. The nodes allowed mask will be derived from any non-default task mempolicy, |
267b4c28 LS |
335 | whether this policy was set explicitly by the task itself or one of its |
336 | ancestors, such as numactl. This means that if the task is invoked from a | |
337 | shell with non-default policy, that policy will be used. One can specify a | |
338 | node list of "all" with numactl --interleave or --membind [-m] to achieve | |
339 | interleaving over all nodes in the system or cpuset. | |
340 | ||
148723f7 | 341 | #. Any task mempolicy specified--e.g., using numactl--will be constrained by |
267b4c28 LS |
342 | the resource limits of any cpuset in which the task runs. Thus, there will |
343 | be no way for a task with non-default policy running in a cpuset with a | |
344 | subset of the system nodes to allocate huge pages outside the cpuset | |
345 | without first moving to a cpuset that contains all of the desired nodes. | |
346 | ||
148723f7 | 347 | #. Boot-time huge page allocation attempts to distribute the requested number |
9b5e5d0f | 348 | of huge pages over all on-lines nodes with memory. |
267b4c28 LS |
349 | |
350 | Per Node Hugepages Attributes | |
15610c86 | 351 | ============================= |
267b4c28 LS |
352 | |
353 | A subset of the contents of the root huge page control directory in sysfs, | |
4faf8d95 | 354 | described above, will be replicated under each the system device of each |
148723f7 | 355 | NUMA node with memory in:: |
267b4c28 LS |
356 | |
357 | /sys/devices/system/node/node[0-9]*/hugepages/ | |
358 | ||
359 | Under this directory, the subdirectory for each supported huge page size | |
148723f7 | 360 | contains the following attribute files:: |
267b4c28 LS |
361 | |
362 | nr_hugepages | |
363 | free_hugepages | |
364 | surplus_hugepages | |
365 | ||
148723f7 | 366 | The free\_' and surplus\_' attribute files are read-only. They return the number |
267b4c28 LS |
367 | of free and surplus [overcommitted] huge pages, respectively, on the parent |
368 | node. | |
369 | ||
148723f7 | 370 | The ``nr_hugepages`` attribute returns the total number of huge pages on the |
267b4c28 LS |
371 | specified node. When this attribute is written, the number of persistent huge |
372 | pages on the parent node will be adjusted to the specified value, if sufficient | |
373 | resources exist, regardless of the task's mempolicy or cpuset constraints. | |
374 | ||
375 | Note that the number of overcommit and reserve pages remain global quantities, | |
376 | as we don't know until fault time, when the faulting task's mempolicy is | |
377 | applied, from which node the huge page allocation will be attempted. | |
378 | ||
fde7917f | 379 | .. _using_huge_pages: |
267b4c28 LS |
380 | |
381 | Using Huge Pages | |
15610c86 | 382 | ================ |
267b4c28 | 383 | |
41a25e7e | 384 | If the user applications are going to request huge pages using mmap system |
1da177e4 | 385 | call, then it is required that system administrator mount a file system of |
148723f7 | 386 | type hugetlbfs:: |
1da177e4 | 387 | |
e73a75fa | 388 | mount -t hugetlbfs \ |
8c9b9703 MK |
389 | -o uid=<value>,gid=<value>,mode=<value>,pagesize=<value>,size=<value>,\ |
390 | min_size=<value>,nr_inodes=<value> none /mnt/huge | |
1da177e4 LT |
391 | |
392 | This command mounts a (pseudo) filesystem of type hugetlbfs on the directory | |
fde7917f | 393 | ``/mnt/huge``. Any file created on ``/mnt/huge`` uses huge pages. |
148723f7 MR |
394 | |
395 | The ``uid`` and ``gid`` options sets the owner and group of the root of the | |
396 | file system. By default the ``uid`` and ``gid`` of the current process | |
397 | are taken. | |
398 | ||
399 | The ``mode`` option sets the mode of root of file system to value & 01777. | |
400 | This value is given in octal. By default the value 0755 is picked. | |
401 | ||
402 | If the platform supports multiple huge page sizes, the ``pagesize`` option can | |
403 | be used to specify the huge page size and associated pool. ``pagesize`` | |
404 | is specified in bytes. If ``pagesize`` is not specified the platform's | |
405 | default huge page size and associated pool will be used. | |
406 | ||
407 | The ``size`` option sets the maximum value of memory (huge pages) allowed | |
408 | for that filesystem (``/mnt/huge``). The ``size`` option can be specified | |
409 | in bytes, or as a percentage of the specified huge page pool (``nr_hugepages``). | |
410 | The size is rounded down to HPAGE_SIZE boundary. | |
411 | ||
412 | The ``min_size`` option sets the minimum value of memory (huge pages) allowed | |
413 | for the filesystem. ``min_size`` can be specified in the same way as ``size``, | |
414 | either bytes or a percentage of the huge page pool. | |
415 | At mount time, the number of huge pages specified by ``min_size`` are reserved | |
416 | for use by the filesystem. | |
417 | If there are not enough free huge pages available, the mount will fail. | |
418 | As huge pages are allocated to the filesystem and freed, the reserve count | |
419 | is adjusted so that the sum of allocated and reserved huge pages is always | |
420 | at least ``min_size``. | |
421 | ||
422 | The option ``nr_inodes`` sets the maximum number of inodes that ``/mnt/huge`` | |
423 | can use. | |
424 | ||
425 | If the ``size``, ``min_size`` or ``nr_inodes`` option is not provided on | |
426 | command line then no limits are set. | |
427 | ||
428 | For ``pagesize``, ``size``, ``min_size`` and ``nr_inodes`` options, you can | |
429 | use [G|g]/[M|m]/[K|k] to represent giga/mega/kilo. | |
430 | For example, size=2K has the same meaning as size=2048. | |
1da177e4 | 431 | |
d5dbac87 NA |
432 | While read system calls are supported on files that reside on hugetlb |
433 | file systems, write system calls are not. | |
1da177e4 | 434 | |
21a26d49 | 435 | Regular chown, chgrp, and chmod commands (with right permissions) could be |
1da177e4 LT |
436 | used to change the file attributes on hugetlbfs. |
437 | ||
80d6b94b | 438 | Also, it is important to note that no such mount command is required if |
94bf5cea | 439 | applications are going to use only shmat/shmget system calls or mmap with |
148723f7 MR |
440 | MAP_HUGETLB. For an example of how to use mmap with MAP_HUGETLB see |
441 | :ref:`map_hugetlb <map_hugetlb>` below. | |
80d6b94b | 442 | |
fde7917f MR |
443 | Users who wish to use hugetlb memory via shared memory segment should be |
444 | members of a supplementary group and system admin needs to configure that gid | |
148723f7 | 445 | into ``/proc/sys/vm/hugetlb_shm_group``. It is possible for same or different |
80d6b94b DR |
446 | applications to use any combination of mmaps and shm* calls, though the mount of |
447 | filesystem will be required for using mmap calls without MAP_HUGETLB. | |
448 | ||
449 | Syscalls that operate on memory backed by hugetlb pages only have their lengths | |
450 | aligned to the native page size of the processor; they will normally fail with | |
451 | errno set to EINVAL or exclude hugetlb pages that extend beyond the length if | |
452 | not hugepage aligned. For example, munmap(2) will fail if memory is backed by | |
453 | a hugetlb page and the length is smaller than the hugepage size. | |
454 | ||
1da177e4 | 455 | |
15610c86 DB |
456 | Examples |
457 | ======== | |
1da177e4 | 458 | |
148723f7 MR |
459 | .. _map_hugetlb: |
460 | ||
461 | ``map_hugetlb`` | |
baa489fa | 462 | see tools/testing/selftests/mm/map_hugetlb.c |
148723f7 MR |
463 | |
464 | ``hugepage-shm`` | |
baa489fa | 465 | see tools/testing/selftests/mm/hugepage-shm.c |
1da177e4 | 466 | |
148723f7 | 467 | ``hugepage-mmap`` |
baa489fa | 468 | see tools/testing/selftests/mm/hugepage-mmap.c |
1da177e4 | 469 | |
148723f7 MR |
470 | The `libhugetlbfs`_ library provides a wide range of userspace tools |
471 | to help with huge page usability, environment setup, and control. | |
d46f3d86 | 472 | |
148723f7 | 473 | .. _libhugetlbfs: https://github.com/libhugetlbfs/libhugetlbfs |