[linux-2.6-block.git] / Documentation / filesystems / nilfs2.txt

NILFS2
------

NILFS2 is a log-structured file system (LFS) supporting continuous
snapshotting.  In addition to versioning capability of the entire file
system, users can even restore files mistakenly overwritten or
destroyed just a few seconds ago.  Since NILFS2 can keep consistency
like conventional LFS, it achieves quick recovery after system
crashes.

NILFS2 creates a number of checkpoints every few seconds or per
synchronous write basis (unless there is no change).  Users can select
significant versions among continuously created checkpoints, and can
change them into snapshots which will be preserved until they are
changed back to checkpoints.

There is no limit on the number of snapshots until the volume gets
full.  Each snapshot is mountable as a read-only file system
concurrently with its writable mount, and this feature is convenient
for online backup.

The userland tools are included in nilfs-utils package, which is
available from the following download page.  At least "mkfs.nilfs2",
"mount.nilfs2", "umount.nilfs2", and "nilfs_cleanerd" (so called
cleaner or garbage collector) are required.  Details on the tools are
described in the man pages included in the package.

Project web page:    https://nilfs.sourceforge.io/
Download page:       https://nilfs.sourceforge.io/en/download.html
List info:           http://vger.kernel.org/vger-lists.html#linux-nilfs

Caveats
=======

Features which NILFS2 does not support yet:

	- atime
	- extended attributes
	- POSIX ACLs
	- quotas
	- fsck
	- defragmentation

Mount options
=============

NILFS2 supports the following mount options:
(*) == default

barrier(*)		This enables/disables the use of write barriers.  This
nobarrier		requires an IO stack which can support barriers, and
			if nilfs gets an error on a barrier write, it will
			disable again with a warning.
errors=continue		Keep going on a filesystem error.
errors=remount-ro(*)	Remount the filesystem read-only on an error.
errors=panic		Panic and halt the machine if an error occurs.
cp=n			Specify the checkpoint-number of the snapshot to be
			mounted.  Checkpoints and snapshots are listed by lscp
			user command.  Only the checkpoints marked as snapshot
			are mountable with this option.  Snapshot is read-only,
			so a read-only mount option must be specified together.
order=relaxed(*)	Apply relaxed order semantics that allows modified data
			blocks to be written to disk without making a
			checkpoint if no metadata update is going.  This mode
			is equivalent to the ordered data mode of the ext3
			filesystem except for the updates on data blocks still
			conserve atomicity.  This will improve synchronous
			write performance for overwriting.
order=strict		Apply strict in-order semantics that preserves sequence
			of all file operations including overwriting of data
			blocks.  That means, it is guaranteed that no
			overtaking of events occurs in the recovered file
			system after a crash.
norecovery		Disable recovery of the filesystem on mount.
			This disables every write access on the device for
			read-only mounts or snapshots.  This option will fail
			for r/w mounts on an unclean volume.
discard			This enables/disables the use of discard/TRIM commands.
nodiscard(*)		The discard/TRIM commands are sent to the underlying
			block device when blocks are freed.  This is useful
			for SSD devices and sparse/thinly-provisioned LUNs.

Ioctls
======

There is some NILFS2 specific functionality which can be accessed by applications
through the system call interfaces. The list of all NILFS2 specific ioctls are
shown in the table below.

Table of NILFS2 specific ioctls
..............................................................................
 Ioctl			        Description
 NILFS_IOCTL_CHANGE_CPMODE      Change mode of given checkpoint between
			        checkpoint and snapshot state. This ioctl is
			        used in chcp and mkcp utilities.

 NILFS_IOCTL_DELETE_CHECKPOINT  Remove checkpoint from NILFS2 file system.
			        This ioctl is used in rmcp utility.

 NILFS_IOCTL_GET_CPINFO         Return info about requested checkpoints. This
			        ioctl is used in lscp utility and by
			        nilfs_cleanerd daemon.

 NILFS_IOCTL_GET_CPSTAT         Return checkpoints statistics. This ioctl is
			        used by lscp, rmcp utilities and by
			        nilfs_cleanerd daemon.

 NILFS_IOCTL_GET_SUINFO         Return segment usage info about requested
			        segments. This ioctl is used in lssu,
			        nilfs_resize utilities and by nilfs_cleanerd
			        daemon.

 NILFS_IOCTL_SET_SUINFO         Modify segment usage info of requested
				segments. This ioctl is used by
				nilfs_cleanerd daemon to skip unnecessary
				cleaning operation of segments and reduce
				performance penalty or wear of flash device
				due to redundant move of in-use blocks.

 NILFS_IOCTL_GET_SUSTAT         Return segment usage statistics. This ioctl
			        is used in lssu, nilfs_resize utilities and
			        by nilfs_cleanerd daemon.

 NILFS_IOCTL_GET_VINFO          Return information on virtual block addresses.
			        This ioctl is used by nilfs_cleanerd daemon.

 NILFS_IOCTL_GET_BDESCS         Return information about descriptors of disk
			        block numbers. This ioctl is used by
			        nilfs_cleanerd daemon.

 NILFS_IOCTL_CLEAN_SEGMENTS     Do garbage collection operation in the
			        environment of requested parameters from
			        userspace. This ioctl is used by
			        nilfs_cleanerd daemon.

 NILFS_IOCTL_SYNC               Make a checkpoint. This ioctl is used in
			        mkcp utility.

 NILFS_IOCTL_RESIZE             Resize NILFS2 volume. This ioctl is used
			        by nilfs_resize utility.

 NILFS_IOCTL_SET_ALLOC_RANGE    Define lower limit of segments in bytes and
			        upper limit of segments in bytes. This ioctl
			        is used by nilfs_resize utility.

NILFS2 usage
============

To use nilfs2 as a local file system, simply:

 # mkfs -t nilfs2 /dev/block_device
 # mount -t nilfs2 /dev/block_device /dir

This will also invoke the cleaner through the mount helper program
(mount.nilfs2).

Checkpoints and snapshots are managed by the following commands.
Their manpages are included in the nilfs-utils package above.

  lscp     list checkpoints or snapshots.
  mkcp     make a checkpoint or a snapshot.
  chcp     change an existing checkpoint to a snapshot or vice versa.
  rmcp     invalidate specified checkpoint(s).

To mount a snapshot,

 # mount -t nilfs2 -r -o cp=<cno> /dev/block_device /snap_dir

where <cno> is the checkpoint number of the snapshot.

To unmount the NILFS2 mount point or snapshot, simply:

 # umount /dir

Then, the cleaner daemon is automatically shut down by the umount
helper program (umount.nilfs2).

Disk format
===========

A nilfs2 volume is equally divided into a number of segments except
for the super block (SB) and segment #0.  A segment is the container
of logs.  Each log is composed of summary information blocks, payload
blocks, and an optional super root block (SR):

   ______________________________________________________
  | |SB| | Segment | Segment | Segment | ... | Segment | |
  |_|__|_|____0____|____1____|____2____|_____|____N____|_|
  0 +1K +4K       +8M       +16M      +24M  +(8MB x N)
       .             .            (Typical offsets for 4KB-block)
    .                  .
  .______________________.
  | log | log |... | log |
  |__1__|__2__|____|__m__|
        .       .
      .               .
    .                       .
  .______________________________.
  | Summary | Payload blocks  |SR|
  |_blocks__|_________________|__|

The payload blocks are organized per file, and each file consists of
data blocks and B-tree node blocks:

    |<---       File-A        --->|<---       File-B        --->|
   _______________________________________________________________
    | Data blocks | B-tree blocks | Data blocks | B-tree blocks | ...
   _|_____________|_______________|_____________|_______________|_


Since only the modified blocks are written in the log, it may have
files without data blocks or B-tree node blocks.

The organization of the blocks is recorded in the summary information
blocks, which contains a header structure (nilfs_segment_summary), per
file structures (nilfs_finfo), and per block structures (nilfs_binfo):

  _________________________________________________________________________
 | Summary | finfo | binfo | ... | binfo | finfo | binfo | ... | binfo |...
 |_blocks__|___A___|_(A,1)_|_____|(A,Na)_|___B___|_(B,1)_|_____|(B,Nb)_|___


The logs include regular files, directory files, symbolic link files
and several meta data files.  The mata data files are the files used
to maintain file system meta data.  The current version of NILFS2 uses
the following meta data files:

 1) Inode file (ifile)             -- Stores on-disk inodes
 2) Checkpoint file (cpfile)       -- Stores checkpoints
 3) Segment usage file (sufile)    -- Stores allocation state of segments
 4) Data address translation file  -- Maps virtual block numbers to usual
    (DAT)                             block numbers.  This file serves to
                                      make on-disk blocks relocatable.

The following figure shows a typical organization of the logs:

  _________________________________________________________________________
 | Summary | regular file | file  | ... | ifile | cpfile | sufile | DAT |SR|
 |_blocks__|_or_directory_|_______|_____|_______|________|________|_____|__|


To stride over segment boundaries, this sequence of files may be split
into multiple logs.  The sequence of logs that should be treated as
logically one log, is delimited with flags marked in the segment
summary.  The recovery code of nilfs2 looks this boundary information
to ensure atomicity of updates.

The super root block is inserted for every checkpoints.  It includes
three special inodes, inodes for the DAT, cpfile, and sufile.  Inodes
of regular files, directories, symlinks and other special files, are
included in the ifile.  The inode of ifile itself is included in the
corresponding checkpoint entry in the cpfile.  Thus, the hierarchy
among NILFS2 files can be depicted as follows:

  Super block (SB)
       |
       v
  Super root block (the latest cno=xx)
       |-- DAT
       |-- sufile
       `-- cpfile
              |-- ifile (cno=c1)
              |-- ifile (cno=c2) ---- file (ino=i1)
              :        :          |-- file (ino=i2)
              `-- ifile (cno=xx)  |-- file (ino=i3)
                                  :        :
                                  `-- file (ino=yy)
                                    ( regular file, directory, or symlink )

For detail on the format of each file, please see nilfs2_ondisk.h
located at include/uapi/linux directory.

There are no patents or other intellectual property that we protect
with regard to the design of NILFS2.  It is allowed to replicate the
design in hopes that other operating systems could share (mount, read,
write, etc.) data stored in this format.
Commit	Line	Data
962281a7 RK	1	NILFS2
	2	------
	3
	4	NILFS2 is a log-structured file system (LFS) supporting continuous
	5	snapshotting. In addition to versioning capability of the entire file
	6	system, users can even restore files mistakenly overwritten or
	7	destroyed just a few seconds ago. Since NILFS2 can keep consistency
	8	like conventional LFS, it achieves quick recovery after system
	9	crashes.
	10
	11	NILFS2 creates a number of checkpoints every few seconds or per
	12	synchronous write basis (unless there is no change). Users can select
	13	significant versions among continuously created checkpoints, and can
	14	change them into snapshots which will be preserved until they are
	15	changed back to checkpoints.
	16
	17	There is no limit on the number of snapshots until the volume gets
	18	full. Each snapshot is mountable as a read-only file system
	19	concurrently with its writable mount, and this feature is convenient
	20	for online backup.
	21
	22	The userland tools are included in nilfs-utils package, which is
	23	available from the following download page. At least "mkfs.nilfs2",
	24	"mount.nilfs2", "umount.nilfs2", and "nilfs_cleanerd" (so called
	25	cleaner or garbage collector) are required. Details on the tools are
	26	described in the man pages included in the package.
	27
bed6760c RK	28	Project web page: https://nilfs.sourceforge.io/
bed6760c RK	29	Download page: https://nilfs.sourceforge.io/en/download.html
6aff43f8	30	List info: http://vger.kernel.org/vger-lists.html#linux-nilfs
962281a7 RK	31
	32	Caveats
	33	=======
	34
	35	Features which NILFS2 does not support yet:
	36
	37	- atime
	38	- extended attributes
	39	- POSIX ACLs
	40	- quotas
fb6e7113	41	- fsck
962281a7 RK	42	- defragmentation
	43
	44	Mount options
	45	=============
	46
	47	NILFS2 supports the following mount options:
	48	(*) == default
	49
773bc4f3 RK	50	barrier(*) This enables/disables the use of write barriers. This
	51	nobarrier requires an IO stack which can support barriers, and
	52	if nilfs gets an error on a barrier write, it will
	53	disable again with a warning.
277a6a34 RK	54	errors=continue Keep going on a filesystem error.
277a6a34 RK	55	errors=remount-ro(*) Remount the filesystem read-only on an error.
962281a7 RK	56	errors=panic Panic and halt the machine if an error occurs.
	57	cp=n Specify the checkpoint-number of the snapshot to be
	58	mounted. Checkpoints and snapshots are listed by lscp
	59	user command. Only the checkpoints marked as snapshot
	60	are mountable with this option. Snapshot is read-only,
	61	so a read-only mount option must be specified together.
	62	order=relaxed(*) Apply relaxed order semantics that allows modified data
	63	blocks to be written to disk without making a
	64	checkpoint if no metadata update is going. This mode
	65	is equivalent to the ordered data mode of the ext3
	66	filesystem except for the updates on data blocks still
	67	conserve atomicity. This will improve synchronous
	68	write performance for overwriting.
	69	order=strict Apply strict in-order semantics that preserves sequence
	70	of all file operations including overwriting of data
	71	blocks. That means, it is guaranteed that no
	72	overtaking of events occurs in the recovered file
	73	system after a crash.
0234576d RK	74	norecovery Disable recovery of the filesystem on mount.
	75	This disables every write access on the device for
	76	read-only mounts or snapshots. This option will fail
	77	for r/w mounts on an unclean volume.
802d3177 RK	78	discard This enables/disables the use of discard/TRIM commands.
	79	nodiscard(*) The discard/TRIM commands are sent to the underlying
	80	block device when blocks are freed. This is useful
	81	for SSD devices and sparse/thinly-provisioned LUNs.
962281a7	82
d623a942 VD	83	Ioctls
	84	======
	85
	86	There is some NILFS2 specific functionality which can be accessed by applications
	87	through the system call interfaces. The list of all NILFS2 specific ioctls are
	88	shown in the table below.
	89
	90	Table of NILFS2 specific ioctls
	91	..............................................................................
	92	Ioctl Description
	93	NILFS_IOCTL_CHANGE_CPMODE Change mode of given checkpoint between
	94	checkpoint and snapshot state. This ioctl is
	95	used in chcp and mkcp utilities.
	96
	97	NILFS_IOCTL_DELETE_CHECKPOINT Remove checkpoint from NILFS2 file system.
	98	This ioctl is used in rmcp utility.
	99
	100	NILFS_IOCTL_GET_CPINFO Return info about requested checkpoints. This
	101	ioctl is used in lscp utility and by
	102	nilfs_cleanerd daemon.
	103
	104	NILFS_IOCTL_GET_CPSTAT Return checkpoints statistics. This ioctl is
	105	used by lscp, rmcp utilities and by
	106	nilfs_cleanerd daemon.
	107
	108	NILFS_IOCTL_GET_SUINFO Return segment usage info about requested
	109	segments. This ioctl is used in lssu,
	110	nilfs_resize utilities and by nilfs_cleanerd
	111	daemon.
	112
2cc88f3a AR	113	NILFS_IOCTL_SET_SUINFO Modify segment usage info of requested
	114	segments. This ioctl is used by
	115	nilfs_cleanerd daemon to skip unnecessary
	116	cleaning operation of segments and reduce
	117	performance penalty or wear of flash device
	118	due to redundant move of in-use blocks.
	119
d623a942 VD	120	NILFS_IOCTL_GET_SUSTAT Return segment usage statistics. This ioctl
	121	is used in lssu, nilfs_resize utilities and
	122	by nilfs_cleanerd daemon.
	123
	124	NILFS_IOCTL_GET_VINFO Return information on virtual block addresses.
	125	This ioctl is used by nilfs_cleanerd daemon.
	126
	127	NILFS_IOCTL_GET_BDESCS Return information about descriptors of disk
	128	block numbers. This ioctl is used by
	129	nilfs_cleanerd daemon.
	130
	131	NILFS_IOCTL_CLEAN_SEGMENTS Do garbage collection operation in the
	132	environment of requested parameters from
	133	userspace. This ioctl is used by
	134	nilfs_cleanerd daemon.
	135
	136	NILFS_IOCTL_SYNC Make a checkpoint. This ioctl is used in
	137	mkcp utility.
	138
	139	NILFS_IOCTL_RESIZE Resize NILFS2 volume. This ioctl is used
	140	by nilfs_resize utility.
	141
	142	NILFS_IOCTL_SET_ALLOC_RANGE Define lower limit of segments in bytes and
	143	upper limit of segments in bytes. This ioctl
	144	is used by nilfs_resize utility.
	145
962281a7 RK	146	NILFS2 usage
	147	============
	148
	149	To use nilfs2 as a local file system, simply:
	150
	151	# mkfs -t nilfs2 /dev/block_device
	152	# mount -t nilfs2 /dev/block_device /dir
	153
	154	This will also invoke the cleaner through the mount helper program
	155	(mount.nilfs2).
	156
	157	Checkpoints and snapshots are managed by the following commands.
	158	Their manpages are included in the nilfs-utils package above.
	159
	160	lscp list checkpoints or snapshots.
	161	mkcp make a checkpoint or a snapshot.
	162	chcp change an existing checkpoint to a snapshot or vice versa.
	163	rmcp invalidate specified checkpoint(s).
	164
	165	To mount a snapshot,
	166
	167	# mount -t nilfs2 -r -o cp=<cno> /dev/block_device /snap_dir
	168
	169	where <cno> is the checkpoint number of the snapshot.
	170
	171	To unmount the NILFS2 mount point or snapshot, simply:
	172
	173	# umount /dir
	174
	175	Then, the cleaner daemon is automatically shut down by the umount
	176	helper program (umount.nilfs2).
	177
	178	Disk format
	179	===========
	180
	181	A nilfs2 volume is equally divided into a number of segments except
	182	for the super block (SB) and segment #0. A segment is the container
	183	of logs. Each log is composed of summary information blocks, payload
	184	blocks, and an optional super root block (SR):
	185
	186	______________________________________________________
	187	\| \|SB\| \| Segment \| Segment \| Segment \| ... \| Segment \| \|
	188	\|_\|__\|_\|____0____\|____1____\|____2____\|_____\|____N____\|_\|
	189	0 +1K +4K +8M +16M +24M +(8MB x N)
	190	. . (Typical offsets for 4KB-block)
	191	. .
	192	.______________________.
	193	\| log \| log \|... \| log \|
	194	\|__1__\|__2__\|____\|__m__\|
	195	. .
	196	. .
	197	. .
	198	.______________________________.
	199	\| Summary \| Payload blocks \|SR\|
	200	\|_blocks__\|_________________\|__\|
	201
	202	The payload blocks are organized per file, and each file consists of
	203	data blocks and B-tree node blocks:
	204
	205	\|<--- File-A --->\|<--- File-B --->\|
	206	_______________________________________________________________
	207	\| Data blocks \| B-tree blocks \| Data blocks \| B-tree blocks \| ...
	208	_\|_____________\|_______________\|_____________\|_______________\|_
	209
210
211	Since only the modified blocks are written in the log, it may have
212	files without data blocks or B-tree node blocks.
213
214	The organization of the blocks is recorded in the summary information
215	blocks, which contains a header structure (nilfs_segment_summary), per
216	file structures (nilfs_finfo), and per block structures (nilfs_binfo):
217
218	_________________________________________________________________________
219	\| Summary \| finfo \| binfo \| ... \| binfo \| finfo \| binfo \| ... \| binfo \|...
220	\|_blocks__\|___A___\|_(A,1)_\|_____\|(A,Na)_\|___B___\|_(B,1)_\|_____\|(B,Nb)_\|___
221
222
223	The logs include regular files, directory files, symbolic link files
224	and several meta data files. The mata data files are the files used
225	to maintain file system meta data. The current version of NILFS2 uses
226	the following meta data files:
227
228	1) Inode file (ifile) -- Stores on-disk inodes
229	2) Checkpoint file (cpfile) -- Stores checkpoints
230	3) Segment usage file (sufile) -- Stores allocation state of segments
231	4) Data address translation file -- Maps virtual block numbers to usual
232	(DAT) block numbers. This file serves to
233	make on-disk blocks relocatable.
962281a7 RK	234
	235	The following figure shows a typical organization of the logs:
	236
	237	_________________________________________________________________________
	238	\| Summary \| regular file \| file \| ... \| ifile \| cpfile \| sufile \| DAT \|SR\|
	239	\|_blocks__\|_or_directory_\|_______\|_____\|_______\|________\|________\|_____\|__\|
	240
	241
	242	To stride over segment boundaries, this sequence of files may be split
	243	into multiple logs. The sequence of logs that should be treated as
	244	logically one log, is delimited with flags marked in the segment
	245	summary. The recovery code of nilfs2 looks this boundary information
	246	to ensure atomicity of updates.
	247
	248	The super root block is inserted for every checkpoints. It includes
	249	three special inodes, inodes for the DAT, cpfile, and sufile. Inodes
	250	of regular files, directories, symlinks and other special files, are
	251	included in the ifile. The inode of ifile itself is included in the
	252	corresponding checkpoint entry in the cpfile. Thus, the hierarchy
	253	among NILFS2 files can be depicted as follows:
	254
	255	Super block (SB)
	256	\|
	257	v
	258	Super root block (the latest cno=xx)
	259	\|-- DAT
	260	\|-- sufile
	261	`-- cpfile
	262	\|-- ifile (cno=c1)
	263	\|-- ifile (cno=c2) ---- file (ino=i1)
	264	: : \|-- file (ino=i2)
	265	`-- ifile (cno=xx) \|-- file (ino=i3)
	266	: :
	267	`-- file (ino=yy)
	268	( regular file, directory, or symlink )
	269
e63e88bc RK	270	For detail on the format of each file, please see nilfs2_ondisk.h
e63e88bc RK	271	located at include/uapi/linux directory.
756cbdb3 RK	272
	273	There are no patents or other intellectual property that we protect
	274	with regard to the design of NILFS2. It is allowed to replicate the
	275	design in hopes that other operating systems could share (mount, read,
	276	write, etc.) data stored in this format.