Commit | Line | Data |
---|---|---|
eeb8a642 MR |
1 | ==================== |
2 | High Memory Handling | |
3 | ==================== | |
d65bfacb PZ |
4 | |
5 | By: Peter Zijlstra <a.p.zijlstra@chello.nl> | |
6 | ||
eeb8a642 | 7 | .. contents:: :local: |
d65bfacb | 8 | |
eeb8a642 | 9 | What Is High Memory? |
d65bfacb PZ |
10 | ==================== |
11 | ||
12 | High memory (highmem) is used when the size of physical memory approaches or | |
13 | exceeds the maximum size of virtual memory. At that point it becomes | |
14 | impossible for the kernel to keep all of the available physical memory mapped | |
15 | at all times. This means the kernel needs to start using temporary mappings of | |
16 | the pieces of physical memory that it wants to access. | |
17 | ||
18 | The part of (physical) memory not covered by a permanent mapping is what we | |
19 | refer to as 'highmem'. There are various architecture dependent constraints on | |
20 | where exactly that border lies. | |
21 | ||
22 | In the i386 arch, for example, we choose to map the kernel into every process's | |
23 | VM space so that we don't have to pay the full TLB invalidation costs for | |
24 | kernel entry/exit. This means the available virtual memory space (4GiB on | |
25 | i386) has to be divided between user and kernel space. | |
26 | ||
27 | The traditional split for architectures using this approach is 3:1, 3GiB for | |
eeb8a642 | 28 | userspace and the top 1GiB for kernel space:: |
d65bfacb PZ |
29 | |
30 | +--------+ 0xffffffff | |
31 | | Kernel | | |
32 | +--------+ 0xc0000000 | |
33 | | | | |
34 | | User | | |
35 | | | | |
36 | +--------+ 0x00000000 | |
37 | ||
38 | This means that the kernel can at most map 1GiB of physical memory at any one | |
39 | time, but because we need virtual address space for other things - including | |
40 | temporary maps to access the rest of the physical memory - the actual direct | |
41 | map will typically be less (usually around ~896MiB). | |
42 | ||
43 | Other architectures that have mm context tagged TLBs can have separate kernel | |
44 | and user maps. Some hardware (like some ARMs), however, have limited virtual | |
45 | space when they use mm context tags. | |
46 | ||
47 | ||
eeb8a642 | 48 | Temporary Virtual Mappings |
d65bfacb PZ |
49 | ========================== |
50 | ||
110bf7a5 FDF |
51 | The kernel contains several ways of creating temporary mappings. The following |
52 | list shows them in order of preference of use. | |
d65bfacb | 53 | |
e4c9eabc FDF |
54 | * kmap_local_page(), kmap_local_folio() - These functions are used to create |
55 | short term mappings. They can be invoked from any context (including | |
56 | interrupts) but the mappings can only be used in the context which acquired | |
57 | them. The only differences between them consist in the first taking a pointer | |
58 | to a struct page and the second taking a pointer to struct folio and the byte | |
59 | offset within the folio which identifies the page. | |
60 | ||
61 | These functions should always be used, whereas kmap_atomic() and kmap() have | |
92b64bd0 | 62 | been deprecated. |
d65bfacb | 63 | |
110bf7a5 | 64 | These mappings are thread-local and CPU-local, meaning that the mapping |
a9e9c939 FDF |
65 | can only be accessed from within this thread and the thread is bound to the |
66 | CPU while the mapping is active. Although preemption is never disabled by | |
67 | this function, the CPU can not be unplugged from the system via | |
68 | CPU-hotplug until the mapping is disposed. | |
110bf7a5 FDF |
69 | |
70 | It's valid to take pagefaults in a local kmap region, unless the context | |
71 | in which the local mapping is acquired does not allow it for other reasons. | |
72 | ||
a9e9c939 FDF |
73 | As said, pagefaults and preemption are never disabled. There is no need to |
74 | disable preemption because, when context switches to a different task, the | |
75 | maps of the outgoing task are saved and those of the incoming one are | |
76 | restored. | |
77 | ||
e4c9eabc FDF |
78 | kmap_local_page(), as well as kmap_local_folio() always returns valid virtual |
79 | kernel addresses and it is assumed that kunmap_local() will never fail. | |
110bf7a5 | 80 | |
e4c9eabc | 81 | On CONFIG_HIGHMEM=n kernels and for low memory pages they return the |
516ea046 FDF |
82 | virtual address of the direct mapping. Only real highmem pages are |
83 | temporarily mapped. Therefore, users may call a plain page_address() | |
84 | for pages which are known to not come from ZONE_HIGHMEM. However, it is | |
e4c9eabc | 85 | always safe to use kmap_local_{page,folio}() / kunmap_local(). |
516ea046 | 86 | |
e4c9eabc FDF |
87 | While they are significantly faster than kmap(), for the highmem case they |
88 | come with restrictions about the pointers validity. Contrary to kmap() | |
6b3afe2e FDF |
89 | mappings, the local mappings are only valid in the context of the caller |
90 | and cannot be handed to other contexts. This implies that users must | |
91 | be absolutely sure to keep the use of the return address local to the | |
92 | thread which mapped it. | |
93 | ||
84b86f60 FDF |
94 | Most code can be designed to use thread local mappings. User should |
95 | therefore try to design their code to avoid the use of kmap() by mapping | |
96 | pages in the same thread the address will be used and prefer | |
e4c9eabc | 97 | kmap_local_page() or kmap_local_folio(). |
84b86f60 | 98 | |
110bf7a5 FDF |
99 | Nesting kmap_local_page() and kmap_atomic() mappings is allowed to a certain |
100 | extent (up to KMAP_TYPE_NR) but their invocations have to be strictly ordered | |
101 | because the map implementation is stack based. See kmap_local_page() kdocs | |
102 | (included in the "Functions" section) for details on how to manage nested | |
103 | mappings. | |
d65bfacb | 104 | |
92b64bd0 FDF |
105 | * kmap_atomic(). This function has been deprecated; use kmap_local_page(). |
106 | ||
107 | NOTE: Conversions to kmap_local_page() must take care to follow the mapping | |
108 | restrictions imposed on kmap_local_page(). Furthermore, the code between | |
109 | calls to kmap_atomic() and kunmap_atomic() may implicitly depend on the side | |
110 | effects of atomic mappings, i.e. disabling page faults or preemption, or both. | |
111 | In that case, explicit calls to pagefault_disable() or preempt_disable() or | |
112 | both must be made in conjunction with the use of kmap_local_page(). | |
113 | ||
114 | [Legacy documentation] | |
115 | ||
116 | This permits a very short duration mapping of a single page. Since the | |
117 | mapping is restricted to the CPU that issued it, it performs well, but | |
118 | the issuing task is therefore required to stay on that CPU until it has | |
119 | finished, lest some other task displace its mappings. | |
d65bfacb | 120 | |
110bf7a5 FDF |
121 | kmap_atomic() may also be used by interrupt contexts, since it does not |
122 | sleep and the callers too may not sleep until after kunmap_atomic() is | |
123 | called. | |
124 | ||
125 | Each call of kmap_atomic() in the kernel creates a non-preemptible section | |
126 | and disable pagefaults. This could be a source of unwanted latency. Therefore | |
127 | users should prefer kmap_local_page() instead of kmap_atomic(). | |
d65bfacb | 128 | |
110bf7a5 FDF |
129 | It is assumed that k[un]map_atomic() won't fail. |
130 | ||
92b64bd0 FDF |
131 | * kmap(). This function has been deprecated; use kmap_local_page(). |
132 | ||
133 | NOTE: Conversions to kmap_local_page() must take care to follow the mapping | |
134 | restrictions imposed on kmap_local_page(). In particular, it is necessary to | |
135 | make sure that the kernel virtual memory pointer is only valid in the thread | |
136 | that obtained it. | |
137 | ||
138 | [Legacy documentation] | |
139 | ||
140 | This should be used to make short duration mapping of a single page with no | |
141 | restrictions on preemption or migration. It comes with an overhead as mapping | |
142 | space is restricted and protected by a global lock for synchronization. When | |
143 | mapping is no longer needed, the address that the page was mapped to must be | |
144 | released with kunmap(). | |
110bf7a5 FDF |
145 | |
146 | Mapping changes must be propagated across all the CPUs. kmap() also | |
147 | requires global TLB invalidation when the kmap's pool wraps and it might | |
148 | block when the mapping space is fully utilized until a slot becomes | |
149 | available. Therefore, kmap() is only callable from preemptible context. | |
150 | ||
151 | All the above work is necessary if a mapping must last for a relatively | |
152 | long time but the bulk of high-memory mappings in the kernel are | |
153 | short-lived and only used in one place. This means that the cost of | |
154 | kmap() is mostly wasted in such cases. kmap() was not intended for long | |
155 | term mappings but it has morphed in that direction and its use is | |
156 | strongly discouraged in newer code and the set of the preceding functions | |
157 | should be preferred. | |
158 | ||
159 | On 64-bit systems, calls to kmap_local_page(), kmap_atomic() and kmap() have | |
160 | no real work to do because a 64-bit address space is more than sufficient to | |
161 | address all the physical memory whose pages are permanently mapped. | |
162 | ||
163 | * vmap(). This can be used to make a long duration mapping of multiple | |
164 | physical pages into a contiguous virtual space. It needs global | |
165 | synchronization to unmap. | |
d65bfacb PZ |
166 | |
167 | ||
eeb8a642 | 168 | Cost of Temporary Mappings |
d65bfacb PZ |
169 | ========================== |
170 | ||
171 | The cost of creating temporary mappings can be quite high. The arch has to | |
172 | manipulate the kernel's page tables, the data TLB and/or the MMU's registers. | |
173 | ||
174 | If CONFIG_HIGHMEM is not set, then the kernel will try and create a mapping | |
175 | simply with a bit of arithmetic that will convert the page struct address into | |
176 | a pointer to the page contents rather than juggling mappings about. In such a | |
177 | case, the unmap operation may be a null operation. | |
178 | ||
179 | If CONFIG_MMU is not set, then there can be no temporary mappings and no | |
180 | highmem. In such a case, the arithmetic approach will also be used. | |
181 | ||
182 | ||
d65bfacb PZ |
183 | i386 PAE |
184 | ======== | |
185 | ||
186 | The i386 arch, under some circumstances, will permit you to stick up to 64GiB | |
187 | of RAM into your 32-bit machine. This has a number of consequences: | |
188 | ||
eeb8a642 MR |
189 | * Linux needs a page-frame structure for each page in the system and the |
190 | pageframes need to live in the permanent mapping, which means: | |
d65bfacb | 191 | |
eeb8a642 MR |
192 | * you can have 896M/sizeof(struct page) page-frames at most; with struct |
193 | page being 32-bytes that would end up being something in the order of 112G | |
194 | worth of pages; the kernel, however, needs to store more than just | |
195 | page-frames in that memory... | |
d65bfacb | 196 | |
eeb8a642 MR |
197 | * PAE makes your page tables larger - which slows the system down as more |
198 | data has to be accessed to traverse in TLB fills and the like. One | |
199 | advantage is that PAE has more PTE bits and can provide advanced features | |
200 | like NX and PAT. | |
d65bfacb PZ |
201 | |
202 | The general recommendation is that you don't use more than 8GiB on a 32-bit | |
203 | machine - although more might work for you and your workload, you're pretty | |
204 | much on your own - don't expect kernel developers to really care much if things | |
205 | come apart. | |
174270c2 FDF |
206 | |
207 | ||
208 | Functions | |
209 | ========= | |
210 | ||
211 | .. kernel-doc:: include/linux/highmem.h | |
61ff748b | 212 | .. kernel-doc:: mm/highmem.c |
174270c2 | 213 | .. kernel-doc:: include/linux/highmem-internal.h |