Commit | Line | Data |
---|---|---|
8979fc9a MCC |
1 | .. SPDX-License-Identifier: GPL-2.0 |
2 | ||
3 | =========================== | |
4 | Ramfs, rootfs and initramfs | |
5 | =========================== | |
6 | ||
7f46a240 | 7 | October 17, 2005 |
8979fc9a | 8 | |
7f46a240 RL |
9 | Rob Landley <rob@landley.net> |
10 | ============================= | |
11 | ||
12 | What is ramfs? | |
13 | -------------- | |
14 | ||
15 | Ramfs is a very simple filesystem that exports Linux's disk caching | |
16 | mechanisms (the page cache and dentry cache) as a dynamically resizable | |
1810732e | 17 | RAM-based filesystem. |
7f46a240 RL |
18 | |
19 | Normally all files are cached in memory by Linux. Pages of data read from | |
20 | backing store (usually the block device the filesystem is mounted on) are kept | |
21 | around in case it's needed again, but marked as clean (freeable) in case the | |
22 | Virtual Memory system needs the memory for something else. Similarly, data | |
23 | written to files is marked clean as soon as it has been written to backing | |
24 | store, but kept around for caching purposes until the VM reallocates the | |
25 | memory. A similar mechanism (the dentry cache) greatly speeds up access to | |
26 | directories. | |
27 | ||
28 | With ramfs, there is no backing store. Files written into ramfs allocate | |
29 | dentries and page cache as usual, but there's nowhere to write them to. | |
30 | This means the pages are never marked clean, so they can't be freed by the | |
31 | VM when it's looking to recycle memory. | |
32 | ||
33 | The amount of code required to implement ramfs is tiny, because all the | |
34 | work is done by the existing Linux caching infrastructure. Basically, | |
35 | you're mounting the disk cache as a filesystem. Because of this, ramfs is not | |
36 | an optional component removable via menuconfig, since there would be negligible | |
37 | space savings. | |
38 | ||
39 | ramfs and ramdisk: | |
40 | ------------------ | |
41 | ||
42 | The older "ram disk" mechanism created a synthetic block device out of | |
1810732e | 43 | an area of RAM and used it as backing store for a filesystem. This block |
7f46a240 RL |
44 | device was of fixed size, so the filesystem mounted on it was of fixed |
45 | size. Using a ram disk also required unnecessarily copying memory from the | |
46 | fake block device into the page cache (and copying changes back out), as well | |
47 | as creating and destroying dentries. Plus it needed a filesystem driver | |
48 | (such as ext2) to format and interpret this data. | |
49 | ||
50 | Compared to ramfs, this wastes memory (and memory bus bandwidth), creates | |
51 | unnecessary work for the CPU, and pollutes the CPU caches. (There are tricks | |
52 | to avoid this copying by playing with the page tables, but they're unpleasantly | |
53 | complicated and turn out to be about as expensive as the copying anyway.) | |
54 | More to the point, all the work ramfs is doing has to happen _anyway_, | |
1810732e RD |
55 | since all file access goes through the page and dentry caches. The RAM |
56 | disk is simply unnecessary; ramfs is internally much simpler. | |
7f46a240 RL |
57 | |
58 | Another reason ramdisks are semi-obsolete is that the introduction of | |
59 | loopback devices offered a more flexible and convenient way to create | |
60 | synthetic block devices, now from files instead of from chunks of memory. | |
61 | See losetup (8) for details. | |
62 | ||
63 | ramfs and tmpfs: | |
64 | ---------------- | |
65 | ||
66 | One downside of ramfs is you can keep writing data into it until you fill | |
67 | up all memory, and the VM can't free it because the VM thinks that files | |
68 | should get written to backing store (rather than swap space), but ramfs hasn't | |
69 | got any backing store. Because of this, only root (or a trusted user) should | |
70 | be allowed write access to a ramfs mount. | |
71 | ||
72 | A ramfs derivative called tmpfs was created to add size limits, and the ability | |
73 | to write the data to swap space. Normal users can be allowed write access to | |
0c1bc6b8 | 74 | tmpfs mounts. See Documentation/filesystems/tmpfs.rst for more information. |
7f46a240 RL |
75 | |
76 | What is rootfs? | |
77 | --------------- | |
78 | ||
e7b69055 RL |
79 | Rootfs is a special instance of ramfs (or tmpfs, if that's enabled), which is |
80 | always present in 2.6 systems. You can't unmount rootfs for approximately the | |
81 | same reason you can't kill the init process; rather than having special code | |
82 | to check for and handle an empty list, it's smaller and simpler for the kernel | |
83 | to just make sure certain lists can't become empty. | |
7f46a240 | 84 | |
e7b69055 | 85 | Most systems just mount another filesystem over rootfs and ignore it. The |
7f46a240 RL |
86 | amount of space an empty instance of ramfs takes up is tiny. |
87 | ||
6e19eded RL |
88 | If CONFIG_TMPFS is enabled, rootfs will use tmpfs instead of ramfs by |
89 | default. To force ramfs, add "rootfstype=ramfs" to the kernel command | |
90 | line. | |
91 | ||
7f46a240 RL |
92 | What is initramfs? |
93 | ------------------ | |
94 | ||
95 | All 2.6 Linux kernels contain a gzipped "cpio" format archive, which is | |
96 | extracted into rootfs when the kernel boots up. After extracting, the kernel | |
97 | checks to see if rootfs contains a file "init", and if so it executes it as PID | |
98 | 1. If found, this init process is responsible for bringing the system the | |
99 | rest of the way up, including locating and mounting the real root device (if | |
100 | any). If rootfs does not contain an init program after the embedded cpio | |
101 | archive is extracted into it, the kernel will fall through to the older code | |
102 | to locate and mount a root partition, then exec some variant of /sbin/init | |
103 | out of that. | |
104 | ||
105 | All this differs from the old initrd in several ways: | |
106 | ||
e7b69055 | 107 | - The old initrd was always a separate file, while the initramfs archive is |
8979fc9a MCC |
108 | linked into the linux kernel image. (The directory ``linux-*/usr`` is |
109 | devoted to generating this archive during the build.) | |
7f46a240 RL |
110 | |
111 | - The old initrd file was a gzipped filesystem image (in some file format, | |
e7b69055 | 112 | such as ext2, that needed a driver built into the kernel), while the new |
7f46a240 | 113 | initramfs archive is a gzipped cpio archive (like tar only simpler, |
8979fc9a MCC |
114 | see cpio(1) and Documentation/driver-api/early-userspace/buffer-format.rst). |
115 | The kernel's cpio extraction code is not only extremely small, it's also | |
1810732e | 116 | __init text and data that can be discarded during the boot process. |
7f46a240 RL |
117 | |
118 | - The program run by the old initrd (which was called /initrd, not /init) did | |
119 | some setup and then returned to the kernel, while the init program from | |
120 | initramfs is not expected to return to the kernel. (If /init needs to hand | |
121 | off control it can overmount / with a new root device and exec another init | |
122 | program. See the switch_root utility, below.) | |
123 | ||
124 | - When switching another root device, initrd would pivot_root and then | |
125 | umount the ramdisk. But initramfs is rootfs: you can neither pivot_root | |
126 | rootfs, nor unmount it. Instead delete everything out of rootfs to | |
127 | free up the space (find -xdev / -exec rm '{}' ';'), overmount rootfs | |
128 | with the new root (cd /newmount; mount --move . /; chroot .), attach | |
129 | stdin/stdout/stderr to the new /dev/console, and exec the new init. | |
130 | ||
33b13025 | 131 | Since this is a remarkably persnickety process (and involves deleting |
7f46a240 RL |
132 | commands before you can run them), the klibc package introduced a helper |
133 | program (utils/run_init.c) to do all this for you. Most other packages | |
134 | (such as busybox) have named this command "switch_root". | |
135 | ||
136 | Populating initramfs: | |
137 | --------------------- | |
138 | ||
139 | The 2.6 kernel build process always creates a gzipped cpio format initramfs | |
140 | archive and links it into the resulting kernel binary. By default, this | |
e7b69055 RL |
141 | archive is empty (consuming 134 bytes on x86). |
142 | ||
1838e392 | 143 | The config option CONFIG_INITRAMFS_SOURCE (in General Setup in menuconfig, |
144 | and living in usr/Kconfig) can be used to specify a source for the | |
145 | initramfs archive, which will automatically be incorporated into the | |
146 | resulting binary. This option can point to an existing gzipped cpio | |
147 | archive, a directory containing files to be archived, or a text file | |
8979fc9a | 148 | specification such as the following example:: |
7f46a240 RL |
149 | |
150 | dir /dev 755 0 0 | |
151 | nod /dev/console 644 0 0 c 5 1 | |
152 | nod /dev/loop0 644 0 0 b 7 0 | |
153 | dir /bin 755 1000 1000 | |
154 | slink /bin/sh busybox 777 0 0 | |
155 | file /bin/busybox initramfs/busybox 755 0 0 | |
156 | dir /proc 755 0 0 | |
157 | dir /sys 755 0 0 | |
158 | dir /mnt 755 0 0 | |
159 | file /init initramfs/init.sh 755 0 0 | |
160 | ||
99aef427 RL |
161 | Run "usr/gen_init_cpio" (after the kernel build) to get a usage message |
162 | documenting the above file format. | |
163 | ||
e7b69055 | 164 | One advantage of the configuration file is that root access is not required to |
7f46a240 RL |
165 | set permissions or create device nodes in the new archive. (Note that those |
166 | two example "file" entries expect to find files named "init.sh" and "busybox" in | |
167 | a directory called "initramfs", under the linux-2.6.* directory. See | |
ec4b78a0 | 168 | Documentation/driver-api/early-userspace/early_userspace_support.rst for more details.) |
7f46a240 | 169 | |
e7b69055 RL |
170 | The kernel does not depend on external cpio tools. If you specify a |
171 | directory instead of a configuration file, the kernel's build infrastructure | |
172 | creates a configuration file from that directory (usr/Makefile calls | |
f6f57a46 | 173 | usr/gen_initramfs_list.sh), and proceeds to package up that directory |
e7b69055 RL |
174 | using the config file (by feeding it to usr/gen_init_cpio, which is created |
175 | from usr/gen_init_cpio.c). The kernel's build-time cpio creation code is | |
176 | entirely self-contained, and the kernel's boot-time extractor is also | |
177 | (obviously) self-contained. | |
178 | ||
179 | The one thing you might need external cpio utilities installed for is creating | |
180 | or extracting your own preprepared cpio files to feed to the kernel build | |
181 | (instead of a config file or directory). | |
182 | ||
183 | The following command line can extract a cpio image (either by the above script | |
8979fc9a | 184 | or by the kernel build) back into its component files:: |
99aef427 RL |
185 | |
186 | cpio -i -d -H newc -F initramfs_data.cpio --no-absolute-filenames | |
187 | ||
e7b69055 | 188 | The following shell script can create a prebuilt cpio archive you can |
8979fc9a | 189 | use in place of the above config file:: |
e7b69055 RL |
190 | |
191 | #!/bin/sh | |
192 | ||
193 | # Copyright 2006 Rob Landley <rob@landley.net> and TimeSys Corporation. | |
194 | # Licensed under GPL version 2 | |
195 | ||
196 | if [ $# -ne 2 ] | |
197 | then | |
198 | echo "usage: mkinitramfs directory imagename.cpio.gz" | |
199 | exit 1 | |
200 | fi | |
201 | ||
202 | if [ -d "$1" ] | |
203 | then | |
204 | echo "creating $2 from $1" | |
205 | (cd "$1"; find . | cpio -o -H newc | gzip) > "$2" | |
206 | else | |
207 | echo "First argument must be a directory" | |
208 | exit 1 | |
209 | fi | |
210 | ||
8979fc9a MCC |
211 | .. Note:: |
212 | ||
213 | The cpio man page contains some bad advice that will break your initramfs | |
214 | archive if you follow it. It says "A typical way to generate the list | |
215 | of filenames is with the find command; you should give find the -depth | |
216 | option to minimize problems with permissions on directories that are | |
217 | unwritable or not searchable." Don't do this when creating | |
218 | initramfs.cpio.gz images, it won't work. The Linux kernel cpio extractor | |
219 | won't create files in a directory that doesn't exist, so the directory | |
220 | entries must go before the files that go in those directories. | |
221 | The above script gets them in the right order. | |
e7b69055 RL |
222 | |
223 | External initramfs images: | |
224 | -------------------------- | |
225 | ||
226 | If the kernel has initrd support enabled, an external cpio.gz archive can also | |
227 | be passed into a 2.6 kernel in place of an initrd. In this case, the kernel | |
228 | will autodetect the type (initramfs, not initrd) and extract the external cpio | |
229 | archive into rootfs before trying to run /init. | |
230 | ||
231 | This has the memory efficiency advantages of initramfs (no ramdisk block | |
232 | device) but the separate packaging of initrd (which is nice if you have | |
233 | non-GPL code you'd like to run from initramfs, without conflating it with | |
234 | the GPL licensed Linux kernel binary). | |
235 | ||
1810732e | 236 | It can also be used to supplement the kernel's built-in initramfs image. The |
e7b69055 RL |
237 | files in the external archive will overwrite any conflicting files in |
238 | the built-in initramfs archive. Some distributors also prefer to customize | |
239 | a single kernel image with task-specific initramfs images, without recompiling. | |
240 | ||
99aef427 RL |
241 | Contents of initramfs: |
242 | ---------------------- | |
243 | ||
e7b69055 | 244 | An initramfs archive is a complete self-contained root filesystem for Linux. |
7f46a240 RL |
245 | If you don't already understand what shared libraries, devices, and paths |
246 | you need to get a minimal root filesystem up and running, here are some | |
247 | references: | |
8979fc9a | 248 | |
c69f22f2 AK |
249 | - https://www.tldp.org/HOWTO/Bootdisk-HOWTO/ |
250 | - https://www.tldp.org/HOWTO/From-PowerUp-To-Bash-Prompt-HOWTO.html | |
8979fc9a | 251 | - http://www.linuxfromscratch.org/lfs/view/stable/ |
7f46a240 | 252 | |
c69f22f2 | 253 | The "klibc" package (https://www.kernel.org/pub/linux/libs/klibc) is |
7f46a240 RL |
254 | designed to be a tiny C library to statically link early userspace |
255 | code against, along with some related utilities. It is BSD licensed. | |
256 | ||
c69f22f2 | 257 | I use uClibc (https://www.uclibc.org) and busybox (https://www.busybox.net) |
99aef427 | 258 | myself. These are LGPL and GPL, respectively. (A self-contained initramfs |
e7b69055 | 259 | package is planned for the busybox 1.3 release.) |
7f46a240 RL |
260 | |
261 | In theory you could use glibc, but that's not well suited for small embedded | |
262 | uses like this. (A "hello world" program statically linked against glibc is | |
263 | over 400k. With uClibc it's 7k. Also note that glibc dlopens libnss to do | |
264 | name lookups, even when otherwise statically linked.) | |
265 | ||
e7b69055 RL |
266 | A good first step is to get initramfs to run a statically linked "hello world" |
267 | program as init, and test it under an emulator like qemu (www.qemu.org) or | |
8979fc9a | 268 | User Mode Linux, like so:: |
e7b69055 RL |
269 | |
270 | cat > hello.c << EOF | |
271 | #include <stdio.h> | |
272 | #include <unistd.h> | |
273 | ||
274 | int main(int argc, char *argv[]) | |
275 | { | |
276 | printf("Hello world!\n"); | |
277 | sleep(999999999); | |
278 | } | |
279 | EOF | |
dd1c53a6 | 280 | gcc -static hello.c -o init |
e7b69055 RL |
281 | echo init | cpio -o -H newc | gzip > test.cpio.gz |
282 | # Testing external initramfs using the initrd loading mechanism. | |
283 | qemu -kernel /boot/vmlinuz -initrd test.cpio.gz /dev/zero | |
284 | ||
285 | When debugging a normal root filesystem, it's nice to be able to boot with | |
286 | "init=/bin/sh". The initramfs equivalent is "rdinit=/bin/sh", and it's | |
287 | just as useful. | |
288 | ||
99aef427 RL |
289 | Why cpio rather than tar? |
290 | ------------------------- | |
291 | ||
292 | This decision was made back in December, 2001. The discussion started here: | |
293 | ||
294 | http://www.uwsg.iu.edu/hypermail/linux/kernel/0112.2/1538.html | |
295 | ||
296 | And spawned a second thread (specifically on tar vs cpio), starting here: | |
297 | ||
298 | http://www.uwsg.iu.edu/hypermail/linux/kernel/0112.2/1587.html | |
299 | ||
300 | The quick and dirty summary version (which is no substitute for reading | |
301 | the above threads) is: | |
302 | ||
303 | 1) cpio is a standard. It's decades old (from the AT&T days), and already | |
304 | widely used on Linux (inside RPM, Red Hat's device driver disks). Here's | |
305 | a Linux Journal article about it from 1996: | |
306 | ||
307 | http://www.linuxjournal.com/article/1213 | |
308 | ||
309 | It's not as popular as tar because the traditional cpio command line tools | |
310 | require _truly_hideous_ command line arguments. But that says nothing | |
311 | either way about the archive format, and there are alternative tools, | |
312 | such as: | |
313 | ||
1f8ee46b | 314 | http://freecode.com/projects/afio |
99aef427 RL |
315 | |
316 | 2) The cpio archive format chosen by the kernel is simpler and cleaner (and | |
317 | thus easier to create and parse) than any of the (literally dozens of) | |
318 | various tar archive formats. The complete initramfs archive format is | |
319 | explained in buffer-format.txt, created in usr/gen_init_cpio.c, and | |
320 | extracted in init/initramfs.c. All three together come to less than 26k | |
321 | total of human-readable text. | |
322 | ||
323 | 3) The GNU project standardizing on tar is approximately as relevant as | |
324 | Windows standardizing on zip. Linux is not part of either, and is free | |
325 | to make its own technical decisions. | |
326 | ||
327 | 4) Since this is a kernel internal format, it could easily have been | |
328 | something brand new. The kernel provides its own tools to create and | |
329 | extract this format anyway. Using an existing standard was preferable, | |
330 | but not essential. | |
331 | ||
332 | 5) Al Viro made the decision (quote: "tar is ugly as hell and not going to be | |
333 | supported on the kernel side"): | |
334 | ||
335 | http://www.uwsg.iu.edu/hypermail/linux/kernel/0112.2/1540.html | |
336 | ||
337 | explained his reasoning: | |
338 | ||
8979fc9a MCC |
339 | - http://www.uwsg.iu.edu/hypermail/linux/kernel/0112.2/1550.html |
340 | - http://www.uwsg.iu.edu/hypermail/linux/kernel/0112.2/1638.html | |
99aef427 RL |
341 | |
342 | and, most importantly, designed and implemented the initramfs code. | |
343 | ||
7f46a240 RL |
344 | Future directions: |
345 | ------------------ | |
346 | ||
e7b69055 | 347 | Today (2.6.16), initramfs is always compiled in, but not always used. The |
7f46a240 RL |
348 | kernel falls back to legacy boot code that is reached only if initramfs does |
349 | not contain an /init program. The fallback is legacy code, there to ensure a | |
350 | smooth transition and allowing early boot functionality to gradually move to | |
351 | "early userspace" (I.E. initramfs). | |
352 | ||
353 | The move to early userspace is necessary because finding and mounting the real | |
354 | root device is complex. Root partitions can span multiple devices (raid or | |
355 | separate journal). They can be out on the network (requiring dhcp, setting a | |
1810732e | 356 | specific MAC address, logging into a server, etc). They can live on removable |
7f46a240 RL |
357 | media, with dynamically allocated major/minor numbers and persistent naming |
358 | issues requiring a full udev implementation to sort out. They can be | |
359 | compressed, encrypted, copy-on-write, loopback mounted, strangely partitioned, | |
360 | and so on. | |
361 | ||
362 | This kind of complexity (which inevitably includes policy) is rightly handled | |
363 | in userspace. Both klibc and busybox/uClibc are working on simple initramfs | |
e7b69055 | 364 | packages to drop into a kernel build. |
7f46a240 | 365 | |
e7b69055 RL |
366 | The klibc package has now been accepted into Andrew Morton's 2.6.17-mm tree. |
367 | The kernel's current early boot code (partition detection, etc) will probably | |
368 | be migrated into a default initramfs, automatically created and used by the | |
369 | kernel build. |