projects / linux-2.6-block.git / blame
| Commit | Line | Data |
|---|---|---|
| 98e4da8c JK |
1 | ================================================================================ |
| 2 | WHAT IS Flash-Friendly File System (F2FS)? | |
| 3 | ================================================================================ | |
| 4 | ||
| 5 | NAND flash memory-based storage devices, such as SSD, eMMC, and SD cards, have | |
| 6 | been equipped on a variety systems ranging from mobile to server systems. Since | |
| 7 | they are known to have different characteristics from the conventional rotating | |
| 8 | disks, a file system, an upper layer to the storage device, should adapt to the | |
| 9 | changes from the sketch in the design level. | |
| 10 | ||
| 11 | F2FS is a file system exploiting NAND flash memory-based storage devices, which | |
| 12 | is based on Log-structured File System (LFS). The design has been focused on | |
| 13 | addressing the fundamental issues in LFS, which are snowball effect of wandering | |
| 14 | tree and high cleaning overhead. | |
| 15 | ||
| 16 | Since a NAND flash memory-based storage device shows different characteristic | |
| 17 | according to its internal geometry or flash memory management scheme, namely FTL, | |
| 18 | F2FS and its tools support various parameters not only for configuring on-disk | |
| 19 | layout, but also for selecting allocation and cleaning algorithms. | |
| 20 | ||
| d51a7fba CL |
21 | The following git tree provides the file system formatting tool (mkfs.f2fs), |
| 22 | a consistency checking tool (fsck.f2fs), and a debugging tool (dump.f2fs). | |
| 5bb446a2 JK |
23 | >> git://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs-tools.git |
| 24 | ||
| 25 | For reporting bugs and sending patches, please use the following mailing list: | |
| 26 | >> linux-f2fs-devel@lists.sourceforge.net | |
| 98e4da8c JK |
27 | |
| 28 | ================================================================================ | |
| 29 | BACKGROUND AND DESIGN ISSUES | |
| 30 | ================================================================================ | |
| 31 | ||
| 32 | Log-structured File System (LFS) | |
| 33 | -------------------------------- | |
| 34 | "A log-structured file system writes all modifications to disk sequentially in | |
| 35 | a log-like structure, thereby speeding up both file writing and crash recovery. | |
| 36 | The log is the only structure on disk; it contains indexing information so that | |
| 37 | files can be read back from the log efficiently. In order to maintain large free | |
| 38 | areas on disk for fast writing, we divide the log into segments and use a | |
| 39 | segment cleaner to compress the live information from heavily fragmented | |
| 40 | segments." from Rosenblum, M. and Ousterhout, J. K., 1992, "The design and | |
| 41 | implementation of a log-structured file system", ACM Trans. Computer Systems | |
| 42 | 10, 1, 26–52. | |
| 43 | ||
| 44 | Wandering Tree Problem | |
| 45 | ---------------------- | |
| 46 | In LFS, when a file data is updated and written to the end of log, its direct | |
| 47 | pointer block is updated due to the changed location. Then the indirect pointer | |
| 48 | block is also updated due to the direct pointer block update. In this manner, | |
| 49 | the upper index structures such as inode, inode map, and checkpoint block are | |
| 50 | also updated recursively. This problem is called as wandering tree problem [1], | |
| 51 | and in order to enhance the performance, it should eliminate or relax the update | |
| 52 | propagation as much as possible. | |
| 53 | ||
| 54 | [1] Bityutskiy, A. 2005. JFFS3 design issues. http://www.linux-mtd.infradead.org/ | |
| 55 | ||
| 56 | Cleaning Overhead | |
| 57 | ----------------- | |
| 58 | Since LFS is based on out-of-place writes, it produces so many obsolete blocks | |
| 59 | scattered across the whole storage. In order to serve new empty log space, it | |
| 60 | needs to reclaim these obsolete blocks seamlessly to users. This job is called | |
| 61 | as a cleaning process. | |
| 62 | ||
| 63 | The process consists of three operations as follows. | |
| 64 | 1. A victim segment is selected through referencing segment usage table. | |
| 65 | 2. It loads parent index structures of all the data in the victim identified by | |
| 66 | segment summary blocks. | |
| 67 | 3. It checks the cross-reference between the data and its parent index structure. | |
| 68 | 4. It moves valid data selectively. | |
| 69 | ||
| 70 | This cleaning job may cause unexpected long delays, so the most important goal | |
| 71 | is to hide the latencies to users. And also definitely, it should reduce the | |
| 72 | amount of valid data to be moved, and move them quickly as well. | |
| 73 | ||
| 74 | ================================================================================ | |
| 75 | KEY FEATURES | |
| 76 | ================================================================================ | |
| 77 | ||
| 78 | Flash Awareness | |
| 79 | --------------- | |
| 80 | - Enlarge the random write area for better performance, but provide the high | |
| 81 | spatial locality | |
| 82 | - Align FS data structures to the operational units in FTL as best efforts | |
| 83 | ||
| 84 | Wandering Tree Problem | |
| 85 | ---------------------- | |
| 86 | - Use a term, “node”, that represents inodes as well as various pointer blocks | |
| 87 | - Introduce Node Address Table (NAT) containing the locations of all the “node” | |
| 88 | blocks; this will cut off the update propagation. | |
| 89 | ||
| 90 | Cleaning Overhead | |
| 91 | ----------------- | |
| 92 | - Support a background cleaning process | |
| 93 | - Support greedy and cost-benefit algorithms for victim selection policies | |
| 94 | - Support multi-head logs for static/dynamic hot and cold data separation | |
| 95 | - Introduce adaptive logging for efficient block allocation | |
| 96 | ||
| 97 | ================================================================================ | |
| 98 | MOUNT OPTIONS | |
| 99 | ================================================================================ | |
| 100 | ||
| 696c018c NJ |
101 | background_gc=%s Turn on/off cleaning operations, namely garbage |
| 102 | collection, triggered in background when I/O subsystem is | |
| 103 | idle. If background_gc=on, it will turn on the garbage | |
| 104 | collection and if background_gc=off, garbage collection | |
| 4bb9998d | 105 | will be turned off. If background_gc=sync, it will turn |
| 6aefd93b | 106 | on synchronous garbage collection running in background. |
| 696c018c NJ |
107 | Default value for this option is on. So garbage |
| 108 | collection is on by default. | |
| 98e4da8c | 109 | disable_roll_forward Disable the roll-forward recovery routine |
| 2d834bf9 JK |
110 | norecovery Disable the roll-forward recovery routine, mounted read- |
| 111 | only (i.e., -o ro,disable_roll_forward) | |
| 64058be9 CY |
112 | discard/nodiscard Enable/disable real-time discard in f2fs, if discard is |
| 113 | enabled, f2fs will issue discard/TRIM commands when a | |
| 114 | segment is cleaned. | |
| 98e4da8c JK |
115 | no_heap Disable heap-style segment allocation which finds free |
| 116 | segments for data from the beginning of main area, while | |
| 117 | for node from the end of main area. | |
| 118 | nouser_xattr Disable Extended User Attributes. Note: xattr is enabled | |
| 119 | by default if CONFIG_F2FS_FS_XATTR is selected. | |
| 120 | noacl Disable POSIX Access Control List. Note: acl is enabled | |
| 121 | by default if CONFIG_F2FS_FS_POSIX_ACL is selected. | |
| 122 | active_logs=%u Support configuring the number of active logs. In the | |
| 123 | current design, f2fs supports only 2, 4, and 6 logs. | |
| 124 | Default number is 6. | |
| 125 | disable_ext_identify Disable the extension list configured by mkfs, so f2fs | |
| 126 | does not aware of cold files such as media files. | |
| 66e960c6 | 127 | inline_xattr Enable the inline xattrs feature. |
| 23cf7212 | 128 | noinline_xattr Disable the inline xattrs feature. |
| 7321dd97 CY |
129 | inline_xattr_size=%u Support configuring inline xattr size, it depends on |
| 130 | flexible inline xattr feature. | |
| e4024e86 HL |
131 | inline_data Enable the inline data feature: New created small(<~3.4k) |
| 132 | files can be written into inode block. | |
| d37a868f CY |
133 | inline_dentry Enable the inline dir feature: data in new created |
| 134 | directory entries can be written into inode block. The | |
| 135 | space of inode block which is used to store inline | |
| 136 | dentries is limited to ~3.4k. | |
| 04b9a5f0 | 137 | noinline_dentry Disable the inline dentry feature. |
| 6b4afdd7 JK |
138 | flush_merge Merge concurrent cache_flush commands as much as possible |
| 139 | to eliminate redundant command issues. If the underlying | |
| 140 | device handles the cache_flush command relatively slowly, | |
| 141 | recommend to enable this option. | |
| 0f7b2abd JK |
142 | nobarrier This option can be used if underlying storage guarantees |
| 143 | its cached data should be written to the novolatile area. | |
| 144 | If this option is set, no cache_flush commands are issued | |
| 145 | but f2fs still guarantees the write ordering of all the | |
| 146 | data writes. | |
| d5053a34 JK |
147 | fastboot This option is used when a system wants to reduce mount |
| 148 | time as much as possible, even though normal performance | |
| 149 | can be sacrificed. | |
| 89672159 CY |
150 | extent_cache Enable an extent cache based on rb-tree, it can cache |
| 151 | as many as extent which map between contiguous logical | |
| 152 | address and physical address per inode, resulting in | |
| 7daaea25 | 153 | increasing the cache hit ratio. Set by default. |
| 4bb9998d | 154 | noextent_cache Disable an extent cache based on rb-tree explicitly, see |
| 7daaea25 | 155 | the above extent_cache mount option. |
| 75342797 WL |
156 | noinline_data Disable the inline data feature, inline data feature is |
| 157 | enabled by default. | |
| 343f40f0 CY |
158 | data_flush Enable data flushing before checkpoint in order to |
| 159 | persist data of regular and symlink. | |
| d0995b53 CY |
160 | reserve_root=%d Support configuring reserved space which is used for |
| 161 | allocation from a privileged user with specified uid or | |
| 162 | gid, unit: 4KB, the default limit is 0.2% of user blocks. | |
| 163 | resuid=%d The user ID which may use the reserved blocks. | |
| 164 | resgid=%d The group ID which may use the reserved blocks. | |
| 56412894 CY |
165 | fault_injection=%d Enable fault injection in all supported types with |
| 166 | specified injection rate. | |
| d494500a CY |
167 | fault_type=%d Support configuring fault injection type, should be |
| 168 | enabled with fault_injection option, fault type value | |
| 169 | is shown below, it supports single or combined type. | |
| 170 | Type_Name Type_Value | |
| 171 | FAULT_KMALLOC 0x000000001 | |
| 172 | FAULT_KVMALLOC 0x000000002 | |
| 173 | FAULT_PAGE_ALLOC 0x000000004 | |
| 174 | FAULT_PAGE_GET 0x000000008 | |
| 175 | FAULT_ALLOC_BIO 0x000000010 | |
| 176 | FAULT_ALLOC_NID 0x000000020 | |
| 177 | FAULT_ORPHAN 0x000000040 | |
| 178 | FAULT_BLOCK 0x000000080 | |
| 179 | FAULT_DIR_DEPTH 0x000000100 | |
| 180 | FAULT_EVICT_INODE 0x000000200 | |
| 181 | FAULT_TRUNCATE 0x000000400 | |
| 6f5c2ed0 | 182 | FAULT_READ_IO 0x000000800 |
| d494500a CY |
183 | FAULT_CHECKPOINT 0x000001000 |
| 184 | FAULT_DISCARD 0x000002000 | |
| 6f5c2ed0 | 185 | FAULT_WRITE_IO 0x000004000 |
| 36abef4e JK |
186 | mode=%s Control block allocation mode which supports "adaptive" |
| 187 | and "lfs". In "lfs" mode, there should be no random | |
| 188 | writes towards main area. | |
| ec91538d JK |
189 | io_bits=%u Set the bit size of write IO requests. It should be set |
| 190 | with "mode=lfs". | |
| 0abd675e CY |
191 | usrquota Enable plain user disk quota accounting. |
| 192 | grpquota Enable plain group disk quota accounting. | |
| 5c57132e | 193 | prjquota Enable plain project quota accounting. |
| 4b2414d0 CY |
194 | usrjquota=<file> Appoint specified file and type during mount, so that quota |
| 195 | grpjquota=<file> information can be properly updated during recovery flow, | |
| 196 | prjjquota=<file> <quota file>: must be in root directory; | |
| 197 | jqfmt=<quota type> <quota type>: [vfsold,vfsv0,vfsv1]. | |
| 198 | offusrjquota Turn off user journelled quota. | |
| 199 | offgrpjquota Turn off group journelled quota. | |
| 200 | offprjjquota Turn off project journelled quota. | |
| 201 | quota Enable plain user disk quota accounting. | |
| 202 | noquota Disable all plain disk quota option. | |
| 8b3a0ca0 HL |
203 | whint_mode=%s Control which write hints are passed down to block |
| 204 | layer. This supports "off", "user-based", and | |
| 205 | "fs-based". In "off" mode (default), f2fs does not pass | |
| 206 | down hints. In "user-based" mode, f2fs tries to pass | |
| 207 | down hints given by users. And in "fs-based" mode, f2fs | |
| 208 | passes down hints with its policy. | |
| 07939627 JK |
209 | alloc_mode=%s Adjust block allocation policy, which supports "reuse" |
| 210 | and "default". | |
| d6290814 JK |
211 | fsync_mode=%s Control the policy of fsync. Currently supports "posix", |
| 212 | "strict", and "nobarrier". In "posix" mode, which is | |
| 213 | default, fsync will follow POSIX semantics and does a | |
| 214 | light operation to improve the filesystem performance. | |
| 215 | In "strict" mode, fsync will be heavy and behaves in line | |
| 216 | with xfs, ext4 and btrfs, where xfstest generic/342 will | |
| 217 | pass, but the performance will regress. "nobarrier" is | |
| 218 | based on "posix", but doesn't issue flush command for | |
| 219 | non-atomic files likewise "nobarrier" mount option. | |
| ff62af20 SY |
220 | test_dummy_encryption Enable dummy encryption, which provides a fake fscrypt |
| 221 | context. The fake fscrypt context is used by xfstests. | |
| 4d3aed70 | 222 | checkpoint=%s[:%u[%]] Set to "disable" to turn off checkpointing. Set to "enable" |
| 4354994f DR |
223 | to reenable checkpointing. Is enabled by default. While |
| 224 | disabled, any unmounting or unexpected shutdowns will cause | |
| 225 | the filesystem contents to appear as they did when the | |
| 226 | filesystem was mounted with that option. | |
| 4d3aed70 DR |
227 | While mounting with checkpoint=disabled, the filesystem must |
| 228 | run garbage collection to ensure that all available space can | |
| 229 | be used. If this takes too much time, the mount may return | |
| 230 | EAGAIN. You may optionally add a value to indicate how much | |
| 231 | of the disk you would be willing to temporarily give up to | |
| 232 | avoid additional garbage collection. This can be given as a | |
| 233 | number of blocks, or as a percent. For instance, mounting | |
| 234 | with checkpoint=disable:100% would always succeed, but it may | |
| 235 | hide up to all remaining free space. The actual space that | |
| 236 | would be unusable can be viewed at /sys/fs/f2fs/<disk>/unusable | |
| 237 | This space is reclaimed once checkpoint=enable. | |
| 98e4da8c JK |
238 | |
| 239 | ================================================================================ | |
| 240 | DEBUGFS ENTRIES | |
| 241 | ================================================================================ | |
| 242 | ||
| 243 | /sys/kernel/debug/f2fs/ contains information about all the partitions mounted as | |
| 244 | f2fs. Each file shows the whole f2fs information. | |
| 245 | ||
| 246 | /sys/kernel/debug/f2fs/status includes: | |
| 247 | - major file system information managed by f2fs currently | |
| 248 | - average SIT information about whole segments | |
| 249 | - current memory footprint consumed by f2fs. | |
| 250 | ||
| b59d0bae NJ |
251 | ================================================================================ |
| 252 | SYSFS ENTRIES | |
| 253 | ================================================================================ | |
| 254 | ||
| 6de3f12e | 255 | Information about mounted f2fs file systems can be found in |
| b59d0bae NJ |
256 | /sys/fs/f2fs. Each mounted filesystem will have a directory in |
| 257 | /sys/fs/f2fs based on its device name (i.e., /sys/fs/f2fs/sda). | |
| 258 | The files in each per-device directory are shown in table below. | |
| 259 | ||
| 260 | Files in /sys/fs/f2fs/<devname> | |
| 261 | (see also Documentation/ABI/testing/sysfs-fs-f2fs) | |
| 262 | .............................................................................. | |
| 263 | File Content | |
| 264 | ||
| 4d11d13e JK |
265 | gc_urgent_sleep_time This parameter controls sleep time for gc_urgent. |
| 266 | 500 ms is set by default. See above gc_urgent. | |
| 267 | ||
| 268 | gc_min_sleep_time This tuning parameter controls the minimum sleep | |
| b59d0bae NJ |
269 | time for the garbage collection thread. Time is |
| 270 | in milliseconds. | |
| 271 | ||
| 4d11d13e | 272 | gc_max_sleep_time This tuning parameter controls the maximum sleep |
| b59d0bae NJ |
273 | time for the garbage collection thread. Time is |
| 274 | in milliseconds. | |
| 275 | ||
| 276 | gc_no_gc_sleep_time This tuning parameter controls the default sleep | |
| 277 | time for the garbage collection thread. Time is | |
| 278 | in milliseconds. | |
| 279 | ||
| d2dc095f NJ |
280 | gc_idle This parameter controls the selection of victim |
| 281 | policy for garbage collection. Setting gc_idle = 0 | |
| 282 | (default) will disable this option. Setting | |
| 283 | gc_idle = 1 will select the Cost Benefit approach | |
| 4bb9998d | 284 | & setting gc_idle = 2 will select the greedy approach. |
| d2dc095f | 285 | |
| d9872a69 JK |
286 | gc_urgent This parameter controls triggering background GCs |
| 287 | urgently or not. Setting gc_urgent = 0 [default] | |
| 288 | makes back to default behavior, while if it is set | |
| 289 | to 1, background thread starts to do GC by given | |
| 290 | gc_urgent_sleep_time interval. | |
| 291 | ||
| ea91e9b0 JK |
292 | reclaim_segments This parameter controls the number of prefree |
| 293 | segments to be reclaimed. If the number of prefree | |
| 58c41035 JK |
294 | segments is larger than the number of segments |
| 295 | in the proportion to the percentage over total | |
| 296 | volume size, f2fs tries to conduct checkpoint to | |
| 297 | reclaim the prefree segments to free segments. | |
| 298 | By default, 5% over total # of segments. | |
| ea91e9b0 | 299 | |
| ba0697ec JK |
300 | max_small_discards This parameter controls the number of discard |
| 301 | commands that consist small blocks less than 2MB. | |
| 302 | The candidates to be discarded are cached until | |
| 303 | checkpoint is triggered, and issued during the | |
| 304 | checkpoint. By default, it is disabled with 0. | |
| 305 | ||
| 4d11d13e JK |
306 | discard_granularity This parameter controls the granularity of discard |
| 307 | command size. It will issue discard commands iif | |
| 308 | the size is larger than given granularity. Its | |
| 309 | unit size is 4KB, and 4 (=16KB) is set by default. | |
| 310 | The maximum value is 128 (=512KB). | |
| 311 | ||
| 312 | reserved_blocks This parameter indicates the number of blocks that | |
| 313 | f2fs reserves internally for root. | |
| 314 | ||
| 315 | batched_trim_sections This parameter controls the number of sections | |
| bba681cb JK |
316 | to be trimmed out in batch mode when FITRIM |
| 317 | conducts. 32 sections is set by default. | |
| 318 | ||
| 216fbd64 JK |
319 | ipu_policy This parameter controls the policy of in-place |
| 320 | updates in f2fs. There are five policies: | |
| 9b5f136f JK |
321 | 0x01: F2FS_IPU_FORCE, 0x02: F2FS_IPU_SSR, |
| 322 | 0x04: F2FS_IPU_UTIL, 0x08: F2FS_IPU_SSR_UTIL, | |
| 323 | 0x10: F2FS_IPU_FSYNC. | |
| 216fbd64 JK |
324 | |
| 325 | min_ipu_util This parameter controls the threshold to trigger | |
| 326 | in-place-updates. The number indicates percentage | |
| 327 | of the filesystem utilization, and used by | |
| 328 | F2FS_IPU_UTIL and F2FS_IPU_SSR_UTIL policies. | |
| 329 | ||
| c1ce1b02 JK |
330 | min_fsync_blocks This parameter controls the threshold to trigger |
| 331 | in-place-updates when F2FS_IPU_FSYNC mode is set. | |
| 332 | The number indicates the number of dirty pages | |
| 333 | when fsync needs to flush on its call path. If | |
| 334 | the number is less than this value, it triggers | |
| 335 | in-place-updates. | |
| 336 | ||
| 4d11d13e JK |
337 | min_seq_blocks This parameter controls the threshold to serialize |
| 338 | write IOs issued by multiple threads in parallel. | |
| 339 | ||
| 340 | min_hot_blocks This parameter controls the threshold to allocate | |
| 341 | a hot data log for pending data blocks to write. | |
| 342 | ||
| 343 | min_ssr_sections This parameter adds the threshold when deciding | |
| 344 | SSR block allocation. If this is large, SSR mode | |
| 345 | will be enabled early. | |
| 346 | ||
| 347 | ram_thresh This parameter controls the memory footprint used | |
| 348 | by free nids and cached nat entries. By default, | |
| 349 | 10 is set, which indicates 10 MB / 1 GB RAM. | |
| 350 | ||
| 351 | ra_nid_pages When building free nids, F2FS reads NAT blocks | |
| 352 | ahead for speed up. Default is 0. | |
| 353 | ||
| 354 | dirty_nats_ratio Given dirty ratio of cached nat entries, F2FS | |
| 355 | determines flushing them in background. | |
| 356 | ||
| 3bac380c JK |
357 | max_victim_search This parameter controls the number of trials to |
| 358 | find a victim segment when conducting SSR and | |
| 359 | cleaning operations. The default value is 4096 | |
| 360 | which covers 8GB block address range. | |
| 361 | ||
| 4d11d13e JK |
362 | migration_granularity For large-sized sections, F2FS can stop GC given |
| 363 | this granularity instead of reclaiming entire | |
| 364 | section. | |
| 365 | ||
| ab9fa662 JK |
366 | dir_level This parameter controls the directory level to |
| 367 | support large directory. If a directory has a | |
| 368 | number of files, it can reduce the file lookup | |
| 369 | latency by increasing this dir_level value. | |
| 370 | Otherwise, it needs to decrease this value to | |
| 371 | reduce the space overhead. The default value is 0. | |
| 372 | ||
| 4d11d13e JK |
373 | cp_interval F2FS tries to do checkpoint periodically, 60 secs |
| 374 | by default. | |
| 375 | ||
| 376 | idle_interval F2FS detects system is idle, if there's no F2FS | |
| 377 | operations during given interval, 5 secs by | |
| 378 | default. | |
| 379 | ||
| 380 | discard_idle_interval F2FS detects the discard thread is idle, given | |
| 381 | time interval. Default is 5 secs. | |
| 382 | ||
| 383 | gc_idle_interval F2FS detects the GC thread is idle, given time | |
| 384 | interval. Default is 5 secs. | |
| 385 | ||
| 386 | umount_discard_timeout When unmounting the disk, F2FS waits for finishing | |
| 387 | queued discard commands which can take huge time. | |
| 388 | This gives time out for it, 5 secs by default. | |
| 389 | ||
| 390 | iostat_enable This controls to enable/disable iostat in F2FS. | |
| 391 | ||
| 392 | readdir_ra This enables/disabled readahead of inode blocks | |
| 393 | in readdir, and default is enabled. | |
| 394 | ||
| 395 | gc_pin_file_thresh This indicates how many GC can be failed for the | |
| 396 | pinned file. If it exceeds this, F2FS doesn't | |
| 397 | guarantee its pinning state. 2048 trials is set | |
| 398 | by default. | |
| 399 | ||
| 400 | extension_list This enables to change extension_list for hot/cold | |
| 401 | files in runtime. | |
| 402 | ||
| 403 | inject_rate This controls injection rate of arbitrary faults. | |
| 404 | ||
| 405 | inject_type This controls injection type of arbitrary faults. | |
| 406 | ||
| 407 | dirty_segments This shows # of dirty segments. | |
| 408 | ||
| 409 | lifetime_write_kbytes This shows # of data written to the disk. | |
| 410 | ||
| 411 | features This shows current features enabled on F2FS. | |
| 412 | ||
| 413 | current_reserved_blocks This shows # of blocks currently reserved. | |
| cdfc41c1 | 414 | |
| 4d3aed70 DR |
415 | unusable If checkpoint=disable, this shows the number of |
| 416 | blocks that are unusable. | |
| 417 | If checkpoint=enable it shows the number of blocks | |
| 418 | that would be unusable if checkpoint=disable were | |
| 419 | to be set. | |
| 420 | ||
| 5aba5430 DR |
421 | encoding This shows the encoding used for casefolding. |
| 422 | If casefolding is not enabled, returns (none) | |
| 423 | ||
| 98e4da8c JK |
424 | ================================================================================ |
| 425 | USAGE | |
| 426 | ================================================================================ | |
| 427 | ||
| 428 | 1. Download userland tools and compile them. | |
| 429 | ||
| 430 | 2. Skip, if f2fs was compiled statically inside kernel. | |
| 431 | Otherwise, insert the f2fs.ko module. | |
| 432 | # insmod f2fs.ko | |
| 433 | ||
| 434 | 3. Create a directory trying to mount | |
| 435 | # mkdir /mnt/f2fs | |
| 436 | ||
| 437 | 4. Format the block device, and then mount as f2fs | |
| 438 | # mkfs.f2fs -l label /dev/block_device | |
| 439 | # mount -t f2fs /dev/block_device /mnt/f2fs | |
| 440 | ||
| d51a7fba CL |
441 | mkfs.f2fs |
| 442 | --------- | |
| 443 | The mkfs.f2fs is for the use of formatting a partition as the f2fs filesystem, | |
| 444 | which builds a basic on-disk layout. | |
| 445 | ||
| 446 | The options consist of: | |
| 1571f84a | 447 | -l [label] : Give a volume label, up to 512 unicode name. |
| 98e4da8c JK |
448 | -a [0 or 1] : Split start location of each area for heap-based allocation. |
| 449 | 1 is set by default, which performs this. | |
| 450 | -o [int] : Set overprovision ratio in percent over volume size. | |
| 451 | 5 is set by default. | |
| 452 | -s [int] : Set the number of segments per section. | |
| 453 | 1 is set by default. | |
| 454 | -z [int] : Set the number of sections per zone. | |
| 455 | 1 is set by default. | |
| 456 | -e [str] : Set basic extension list. e.g. "mp3,gif,mov" | |
| 1571f84a CL |
457 | -t [0 or 1] : Disable discard command or not. |
| 458 | 1 is set by default, which conducts discard. | |
| 98e4da8c | 459 | |
| d51a7fba CL |
460 | fsck.f2fs |
| 461 | --------- | |
| 462 | The fsck.f2fs is a tool to check the consistency of an f2fs-formatted | |
| 463 | partition, which examines whether the filesystem metadata and user-made data | |
| 464 | are cross-referenced correctly or not. | |
| 465 | Note that, initial version of the tool does not fix any inconsistency. | |
| 466 | ||
| 467 | The options consist of: | |
| 468 | -d debug level [default:0] | |
| 469 | ||
| 470 | dump.f2fs | |
| 471 | --------- | |
| 472 | The dump.f2fs shows the information of specific inode and dumps SSA and SIT to | |
| 473 | file. Each file is dump_ssa and dump_sit. | |
| 474 | ||
| 475 | The dump.f2fs is used to debug on-disk data structures of the f2fs filesystem. | |
| 4bb9998d | 476 | It shows on-disk inode information recognized by a given inode number, and is |
| d51a7fba CL |
477 | able to dump all the SSA and SIT entries into predefined files, ./dump_ssa and |
| 478 | ./dump_sit respectively. | |
| 479 | ||
| 480 | The options consist of: | |
| 481 | -d debug level [default:0] | |
| 482 | -i inode no (hex) | |
| 483 | -s [SIT dump segno from #1~#2 (decimal), for all 0~-1] | |
| 484 | -a [SSA dump segno from #1~#2 (decimal), for all 0~-1] | |
| 485 | ||
| 486 | Examples: | |
| 487 | # dump.f2fs -i [ino] /dev/sdx | |
| 488 | # dump.f2fs -s 0~-1 /dev/sdx (SIT dump) | |
| 489 | # dump.f2fs -a 0~-1 /dev/sdx (SSA dump) | |
| 490 | ||
| 98e4da8c JK |
491 | ================================================================================ |
| 492 | DESIGN | |
| 493 | ================================================================================ | |
| 494 | ||
| 495 | On-disk Layout | |
| 496 | -------------- | |
| 497 | ||
| 498 | F2FS divides the whole volume into a number of segments, each of which is fixed | |
| 499 | to 2MB in size. A section is composed of consecutive segments, and a zone | |
| 500 | consists of a set of sections. By default, section and zone sizes are set to one | |
| 501 | segment size identically, but users can easily modify the sizes by mkfs. | |
| 502 | ||
| 503 | F2FS splits the entire volume into six areas, and all the areas except superblock | |
| 504 | consists of multiple segments as described below. | |
| 505 | ||
| 506 | align with the zone size <-| | |
| 507 | |-> align with the segment size | |
| 508 | _________________________________________________________________________ | |
| 9268cc35 HL |
509 | | | | Segment | Node | Segment | | |
| 510 | | Superblock | Checkpoint | Info. | Address | Summary | Main | | |
| 511 | | (SB) | (CP) | Table (SIT) | Table (NAT) | Area (SSA) | | | |
| 98e4da8c JK |
512 | |____________|_____2______|______N______|______N______|______N_____|__N___| |
| 513 | . . | |
| 514 | . . | |
| 515 | . . | |
| 516 | ._________________________________________. | |
| 517 | |_Segment_|_..._|_Segment_|_..._|_Segment_| | |
| 518 | . . | |
| 519 | ._________._________ | |
| 520 | |_section_|__...__|_ | |
| 521 | . . | |
| 522 | .________. | |
| 523 | |__zone__| | |
| 524 | ||
| 525 | - Superblock (SB) | |
| 526 | : It is located at the beginning of the partition, and there exist two copies | |
| 527 | to avoid file system crash. It contains basic partition information and some | |
| 528 | default parameters of f2fs. | |
| 529 | ||
| 530 | - Checkpoint (CP) | |
| 531 | : It contains file system information, bitmaps for valid NAT/SIT sets, orphan | |
| 532 | inode lists, and summary entries of current active segments. | |
| 533 | ||
| 98e4da8c JK |
534 | - Segment Information Table (SIT) |
| 535 | : It contains segment information such as valid block count and bitmap for the | |
| 536 | validity of all the blocks. | |
| 537 | ||
| 9268cc35 HL |
538 | - Node Address Table (NAT) |
| 539 | : It is composed of a block address table for all the node blocks stored in | |
| 540 | Main area. | |
| 541 | ||
| 98e4da8c JK |
542 | - Segment Summary Area (SSA) |
| 543 | : It contains summary entries which contains the owner information of all the | |
| 544 | data and node blocks stored in Main area. | |
| 545 | ||
| 546 | - Main Area | |
| 547 | : It contains file and directory data including their indices. | |
| 548 | ||
| 549 | In order to avoid misalignment between file system and flash-based storage, F2FS | |
| 550 | aligns the start block address of CP with the segment size. Also, it aligns the | |
| 551 | start block address of Main area with the zone size by reserving some segments | |
| 552 | in SSA area. | |
| 553 | ||
| 554 | Reference the following survey for additional technical details. | |
| 555 | https://wiki.linaro.org/WorkingGroups/Kernel/Projects/FlashCardSurvey | |
| 556 | ||
| 557 | File System Metadata Structure | |
| 558 | ------------------------------ | |
| 559 | ||
| 560 | F2FS adopts the checkpointing scheme to maintain file system consistency. At | |
| 561 | mount time, F2FS first tries to find the last valid checkpoint data by scanning | |
| 562 | CP area. In order to reduce the scanning time, F2FS uses only two copies of CP. | |
| 563 | One of them always indicates the last valid data, which is called as shadow copy | |
| 564 | mechanism. In addition to CP, NAT and SIT also adopt the shadow copy mechanism. | |
| 565 | ||
| 566 | For file system consistency, each CP points to which NAT and SIT copies are | |
| 567 | valid, as shown as below. | |
| 568 | ||
| 569 | +--------+----------+---------+ | |
| 9268cc35 | 570 | | CP | SIT | NAT | |
| 98e4da8c JK |
571 | +--------+----------+---------+ |
| 572 | . . . . | |
| 573 | . . . . | |
| 574 | . . . . | |
| 575 | +-------+-------+--------+--------+--------+--------+ | |
| 9268cc35 | 576 | | CP #0 | CP #1 | SIT #0 | SIT #1 | NAT #0 | NAT #1 | |
| 98e4da8c JK |
577 | +-------+-------+--------+--------+--------+--------+ |
| 578 | | ^ ^ | |
| 579 | | | | | |
| 580 | `----------------------------------------' | |
| 581 | ||
| 582 | Index Structure | |
| 583 | --------------- | |
| 584 | ||
| 585 | The key data structure to manage the data locations is a "node". Similar to | |
| 586 | traditional file structures, F2FS has three types of node: inode, direct node, | |
| d08ab08d | 587 | indirect node. F2FS assigns 4KB to an inode block which contains 923 data block |
| 98e4da8c JK |
588 | indices, two direct node pointers, two indirect node pointers, and one double |
| 589 | indirect node pointer as described below. One direct node block contains 1018 | |
| 590 | data blocks, and one indirect node block contains also 1018 node blocks. Thus, | |
| 591 | one inode block (i.e., a file) covers: | |
| 592 | ||
| 593 | 4KB * (923 + 2 * 1018 + 2 * 1018 * 1018 + 1018 * 1018 * 1018) := 3.94TB. | |
| 594 | ||
| 595 | Inode block (4KB) | |
| 596 | |- data (923) | |
| 597 | |- direct node (2) | |
| 598 | | `- data (1018) | |
| 599 | |- indirect node (2) | |
| 600 | | `- direct node (1018) | |
| 601 | | `- data (1018) | |
| 602 | `- double indirect node (1) | |
| 603 | `- indirect node (1018) | |
| 604 | `- direct node (1018) | |
| 605 | `- data (1018) | |
| 606 | ||
| 607 | Note that, all the node blocks are mapped by NAT which means the location of | |
| 608 | each node is translated by the NAT table. In the consideration of the wandering | |
| 609 | tree problem, F2FS is able to cut off the propagation of node updates caused by | |
| 610 | leaf data writes. | |
| 611 | ||
| 612 | Directory Structure | |
| 613 | ------------------- | |
| 614 | ||
| 615 | A directory entry occupies 11 bytes, which consists of the following attributes. | |
| 616 | ||
| 617 | - hash hash value of the file name | |
| 618 | - ino inode number | |
| 619 | - len the length of file name | |
| 620 | - type file type such as directory, symlink, etc | |
| 621 | ||
| 622 | A dentry block consists of 214 dentry slots and file names. Therein a bitmap is | |
| 623 | used to represent whether each dentry is valid or not. A dentry block occupies | |
| 624 | 4KB with the following composition. | |
| 625 | ||
| 626 | Dentry Block(4 K) = bitmap (27 bytes) + reserved (3 bytes) + | |
| 627 | dentries(11 * 214 bytes) + file name (8 * 214 bytes) | |
| 628 | ||
| 629 | [Bucket] | |
| 630 | +--------------------------------+ | |
| 631 | |dentry block 1 | dentry block 2 | | |
| 632 | +--------------------------------+ | |
| 633 | . . | |
| 634 | . . | |
| 635 | . [Dentry Block Structure: 4KB] . | |
| 636 | +--------+----------+----------+------------+ | |
| 637 | | bitmap | reserved | dentries | file names | | |
| 638 | +--------+----------+----------+------------+ | |
| 639 | [Dentry Block: 4KB] . . | |
| 640 | . . | |
| 641 | . . | |
| 642 | +------+------+-----+------+ | |
| 643 | | hash | ino | len | type | | |
| 644 | +------+------+-----+------+ | |
| 645 | [Dentry Structure: 11 bytes] | |
| 646 | ||
| 647 | F2FS implements multi-level hash tables for directory structure. Each level has | |
| 648 | a hash table with dedicated number of hash buckets as shown below. Note that | |
| 649 | "A(2B)" means a bucket includes 2 data blocks. | |
| 650 | ||
| 651 | ---------------------- | |
| 652 | A : bucket | |
| 653 | B : block | |
| 654 | N : MAX_DIR_HASH_DEPTH | |
| 655 | ---------------------- | |
| 656 | ||
| 657 | level #0 | A(2B) | |
| 658 | | | |
| 659 | level #1 | A(2B) - A(2B) | |
| 660 | | | |
| 661 | level #2 | A(2B) - A(2B) - A(2B) - A(2B) | |
| 662 | . | . . . . | |
| 663 | level #N/2 | A(2B) - A(2B) - A(2B) - A(2B) - A(2B) - ... - A(2B) | |
| 664 | . | . . . . | |
| 665 | level #N | A(4B) - A(4B) - A(4B) - A(4B) - A(4B) - ... - A(4B) | |
| 666 | ||
| 667 | The number of blocks and buckets are determined by, | |
| 668 | ||
| 669 | ,- 2, if n < MAX_DIR_HASH_DEPTH / 2, | |
| 670 | # of blocks in level #n = | | |
| 671 | `- 4, Otherwise | |
| 672 | ||
| bfec07d0 CY |
673 | ,- 2^(n + dir_level), |
| 674 | | if n + dir_level < MAX_DIR_HASH_DEPTH / 2, | |
| 98e4da8c | 675 | # of buckets in level #n = | |
| bfec07d0 CY |
676 | `- 2^((MAX_DIR_HASH_DEPTH / 2) - 1), |
| 677 | Otherwise | |
| 98e4da8c JK |
678 | |
| 679 | When F2FS finds a file name in a directory, at first a hash value of the file | |
| 680 | name is calculated. Then, F2FS scans the hash table in level #0 to find the | |
| 681 | dentry consisting of the file name and its inode number. If not found, F2FS | |
| 682 | scans the next hash table in level #1. In this way, F2FS scans hash tables in | |
| 683 | each levels incrementally from 1 to N. In each levels F2FS needs to scan only | |
| 684 | one bucket determined by the following equation, which shows O(log(# of files)) | |
| 685 | complexity. | |
| 686 | ||
| 687 | bucket number to scan in level #n = (hash value) % (# of buckets in level #n) | |
| 688 | ||
| 689 | In the case of file creation, F2FS finds empty consecutive slots that cover the | |
| 690 | file name. F2FS searches the empty slots in the hash tables of whole levels from | |
| 691 | 1 to N in the same way as the lookup operation. | |
| 692 | ||
| 693 | The following figure shows an example of two cases holding children. | |
| 694 | --------------> Dir <-------------- | |
| 695 | | | | |
| 696 | child child | |
| 697 | ||
| 698 | child - child [hole] - child | |
| 699 | ||
| 700 | child - child - child [hole] - [hole] - child | |
| 701 | ||
| 702 | Case 1: Case 2: | |
| 703 | Number of children = 6, Number of children = 3, | |
| 704 | File size = 7 File size = 7 | |
| 705 | ||
| 706 | Default Block Allocation | |
| 707 | ------------------------ | |
| 708 | ||
| 709 | At runtime, F2FS manages six active logs inside "Main" area: Hot/Warm/Cold node | |
| 710 | and Hot/Warm/Cold data. | |
| 711 | ||
| 712 | - Hot node contains direct node blocks of directories. | |
| 713 | - Warm node contains direct node blocks except hot node blocks. | |
| 714 | - Cold node contains indirect node blocks | |
| 715 | - Hot data contains dentry blocks | |
| 716 | - Warm data contains data blocks except hot and cold data blocks | |
| 717 | - Cold data contains multimedia data or migrated data blocks | |
| 718 | ||
| 719 | LFS has two schemes for free space management: threaded log and copy-and-compac- | |
| 720 | tion. The copy-and-compaction scheme which is known as cleaning, is well-suited | |
| 721 | for devices showing very good sequential write performance, since free segments | |
| 722 | are served all the time for writing new data. However, it suffers from cleaning | |
| 723 | overhead under high utilization. Contrarily, the threaded log scheme suffers | |
| 724 | from random writes, but no cleaning process is needed. F2FS adopts a hybrid | |
| 725 | scheme where the copy-and-compaction scheme is adopted by default, but the | |
| 726 | policy is dynamically changed to the threaded log scheme according to the file | |
| 727 | system status. | |
| 728 | ||
| 729 | In order to align F2FS with underlying flash-based storage, F2FS allocates a | |
| 730 | segment in a unit of section. F2FS expects that the section size would be the | |
| 731 | same as the unit size of garbage collection in FTL. Furthermore, with respect | |
| 732 | to the mapping granularity in FTL, F2FS allocates each section of the active | |
| 733 | logs from different zones as much as possible, since FTL can write the data in | |
| 734 | the active logs into one allocation unit according to its mapping granularity. | |
| 735 | ||
| 736 | Cleaning process | |
| 737 | ---------------- | |
| 738 | ||
| 739 | F2FS does cleaning both on demand and in the background. On-demand cleaning is | |
| 740 | triggered when there are not enough free segments to serve VFS calls. Background | |
| 741 | cleaner is operated by a kernel thread, and triggers the cleaning job when the | |
| 742 | system is idle. | |
| 743 | ||
| 744 | F2FS supports two victim selection policies: greedy and cost-benefit algorithms. | |
| 745 | In the greedy algorithm, F2FS selects a victim segment having the smallest number | |
| 746 | of valid blocks. In the cost-benefit algorithm, F2FS selects a victim segment | |
| 747 | according to the segment age and the number of valid blocks in order to address | |
| 748 | log block thrashing problem in the greedy algorithm. F2FS adopts the greedy | |
| 749 | algorithm for on-demand cleaner, while background cleaner adopts cost-benefit | |
| 750 | algorithm. | |
| 751 | ||
| 752 | In order to identify whether the data in the victim segment are valid or not, | |
| 753 | F2FS manages a bitmap. Each bit represents the validity of a block, and the | |
| 754 | bitmap is composed of a bit stream covering whole blocks in main area. | |
| 8b3a0ca0 HL |
755 | |
| 756 | Write-hint Policy | |
| 757 | ----------------- | |
| 758 | ||
| 759 | 1) whint_mode=off. F2FS only passes down WRITE_LIFE_NOT_SET. | |
| 760 | ||
| 761 | 2) whint_mode=user-based. F2FS tries to pass down hints given by | |
| 762 | users. | |
| 763 | ||
| 764 | User F2FS Block | |
| 765 | ---- ---- ----- | |
| 766 | META WRITE_LIFE_NOT_SET | |
| 767 | HOT_NODE " | |
| 768 | WARM_NODE " | |
| 769 | COLD_NODE " | |
| 770 | *ioctl(COLD) COLD_DATA WRITE_LIFE_EXTREME | |
| 771 | *extension list " " | |
| 772 | ||
| 773 | -- buffered io | |
| 774 | WRITE_LIFE_EXTREME COLD_DATA WRITE_LIFE_EXTREME | |
| 775 | WRITE_LIFE_SHORT HOT_DATA WRITE_LIFE_SHORT | |
| 776 | WRITE_LIFE_NOT_SET WARM_DATA WRITE_LIFE_NOT_SET | |
| 777 | WRITE_LIFE_NONE " " | |
| 778 | WRITE_LIFE_MEDIUM " " | |
| 779 | WRITE_LIFE_LONG " " | |
| 780 | ||
| 781 | -- direct io | |
| 782 | WRITE_LIFE_EXTREME COLD_DATA WRITE_LIFE_EXTREME | |
| 783 | WRITE_LIFE_SHORT HOT_DATA WRITE_LIFE_SHORT | |
| 784 | WRITE_LIFE_NOT_SET WARM_DATA WRITE_LIFE_NOT_SET | |
| 785 | WRITE_LIFE_NONE " WRITE_LIFE_NONE | |
| 786 | WRITE_LIFE_MEDIUM " WRITE_LIFE_MEDIUM | |
| 787 | WRITE_LIFE_LONG " WRITE_LIFE_LONG | |
| 788 | ||
| 789 | 3) whint_mode=fs-based. F2FS passes down hints with its policy. | |
| 790 | ||
| 791 | User F2FS Block | |
| 792 | ---- ---- ----- | |
| 793 | META WRITE_LIFE_MEDIUM; | |
| 794 | HOT_NODE WRITE_LIFE_NOT_SET | |
| 795 | WARM_NODE " | |
| 796 | COLD_NODE WRITE_LIFE_NONE | |
| 797 | ioctl(COLD) COLD_DATA WRITE_LIFE_EXTREME | |
| 798 | extension list " " | |
| 799 | ||
| 800 | -- buffered io | |
| 801 | WRITE_LIFE_EXTREME COLD_DATA WRITE_LIFE_EXTREME | |
| 802 | WRITE_LIFE_SHORT HOT_DATA WRITE_LIFE_SHORT | |
| 803 | WRITE_LIFE_NOT_SET WARM_DATA WRITE_LIFE_LONG | |
| 804 | WRITE_LIFE_NONE " " | |
| 805 | WRITE_LIFE_MEDIUM " " | |
| 806 | WRITE_LIFE_LONG " " | |
| 807 | ||
| 808 | -- direct io | |
| 809 | WRITE_LIFE_EXTREME COLD_DATA WRITE_LIFE_EXTREME | |
| 810 | WRITE_LIFE_SHORT HOT_DATA WRITE_LIFE_SHORT | |
| 811 | WRITE_LIFE_NOT_SET WARM_DATA WRITE_LIFE_NOT_SET | |
| 812 | WRITE_LIFE_NONE " WRITE_LIFE_NONE | |
| 813 | WRITE_LIFE_MEDIUM " WRITE_LIFE_MEDIUM | |
| 814 | WRITE_LIFE_LONG " WRITE_LIFE_LONG | |
| cad3836f JK |
815 | |
| 816 | Fallocate(2) Policy | |
| 817 | ------------------- | |
| 818 | ||
| 819 | The default policy follows the below posix rule. | |
| 820 | ||
| 821 | Allocating disk space | |
| 822 | The default operation (i.e., mode is zero) of fallocate() allocates | |
| 823 | the disk space within the range specified by offset and len. The | |
| 824 | file size (as reported by stat(2)) will be changed if offset+len is | |
| 825 | greater than the file size. Any subregion within the range specified | |
| 826 | by offset and len that did not contain data before the call will be | |
| 827 | initialized to zero. This default behavior closely resembles the | |
| 828 | behavior of the posix_fallocate(3) library function, and is intended | |
| 829 | as a method of optimally implementing that function. | |
| 830 | ||
| 831 | However, once F2FS receives ioctl(fd, F2FS_IOC_SET_PIN_FILE) in prior to | |
| 832 | fallocate(fd, DEFAULT_MODE), it allocates on-disk blocks addressess having | |
| 833 | zero or random data, which is useful to the below scenario where: | |
| 834 | 1. create(fd) | |
| 835 | 2. ioctl(fd, F2FS_IOC_SET_PIN_FILE) | |
| 836 | 3. fallocate(fd, 0, 0, size) | |
| 837 | 4. address = fibmap(fd, offset) | |
| 838 | 5. open(blkdev) | |
| 839 | 6. write(blkdev, address) |