defined by \fIioengine\fR. If no \fIioengine\fR is given, list all
available ioengines.
.TP
-.BI \-\-showcmd \fR=\fPjobfile
-Convert \fIjobfile\fR to a set of command\-line options.
+.BI \-\-showcmd
+Convert given \fIjobfile\fRs to a set of command\-line options.
.TP
.BI \-\-readonly
Turn on safety read\-only checks, preventing writes and trims. The \fB\-\-readonly\fR
.RS
.P
.PD 0
-z means Zone
+z means Zone
.P
.PD
.RE
.SS "Time related parameters"
.TP
.BI runtime \fR=\fPtime
-Tell fio to terminate processing after the specified period of time. It
-can be quite hard to determine for how long a specified job will run, so
-this parameter is handy to cap the total runtime to a given time. When
-the unit is omitted, the value is interpreted in seconds.
+Limit runtime. The test will run until it completes the configured I/O
+workload or until it has run for this specified amount of time, whichever
+occurs first. It can be quite hard to determine for how long a specified
+job will run, so this parameter is handy to cap the total runtime to a
+given time. When the unit is omitted, the value is interpreted in
+seconds.
.TP
.BI time_based
If set, fio will run for the duration of the \fBruntime\fR specified
explicit size is specified by \fBfilesize\fR.
.RS
.P
-Each colon in the wanted path must be escaped with a '\\'
+Each colon in the wanted path must be escaped with a '\e'
character. For instance, if the path is `/dev/dsk/foo@3,0:c' then you
would use `filename=/dev/dsk/foo@3,0\\:c' and if the path is
`F:\\filename' then you would use `filename=F\\:\\filename'.
so. Default: false.
.TP
.BI max_open_zones \fR=\fPint
-When running a random write test across an entire drive many more zones will be
-open than in a typical application workload. Hence this command line option
-that allows to limit the number of open zones. The number of open zones is
-defined as the number of zones to which write commands are issued by all
-threads/processes.
+A zone of a zoned block device is in the open state when it is partially written
+(i.e. not all sectors of the zone have been written). Zoned block devices may
+have limit a on the total number of zones that can be simultaneously in the
+open state, that is, the number of zones that can be written to simultaneously.
+The \fBmax_open_zones\fR parameter limits the number of zones to which write
+commands are issued by all fio jobs, that is, limits the number of zones that
+will be in the open state. This parameter is relevant only if the
+\fBzonemode=zbd\fR is used. The default value is always equal to maximum number
+of open zones of the target zoned block device and a value higher than this
+limit cannot be specified by users unless the option \fBignore_zone_limits\fR is
+specified. When \fBignore_zone_limits\fR is specified or the target device has
+no limit on the number of zones that can be in an open state,
+\fBmax_open_zones\fR can specify 0 to disable any limit on the number of zones
+that can be simultaneously written to by all jobs.
.TP
.BI job_max_open_zones \fR=\fPint
-Limit on the number of simultaneously opened zones per single thread/process.
+In the same manner as \fBmax_open_zones\fR, limit the number of open zones per
+fio job, that is, the number of zones that a single job can simultaneously write
+to. A value of zero indicates no limit. Default: zero.
.TP
.BI ignore_zone_limits \fR=\fPbool
If this option is used, fio will ignore the maximum number of open zones limit
value to be larger than the device reported limit. Default: false.
.TP
.BI zone_reset_threshold \fR=\fPfloat
-A number between zero and one that indicates the ratio of logical blocks with
-data to the total number of logical blocks in the test above which zones
-should be reset periodically.
+A number between zero and one that indicates the ratio of written bytes in the
+zones with write pointers in the IO range to the size of the IO range. When
+current ratio is above this ratio, zones are reset periodically as
+\fBzone_reset_frequency\fR specifies. If there are multiple jobs when using this
+option, the IO range for all write jobs has to be the same.
.TP
.BI zone_reset_frequency \fR=\fPfloat
A number between zero and one that indicates how often a zone reset should be
OpenBSD and ZFS on Solaris don't support direct I/O. On Windows the synchronous
ioengines don't support direct I/O. Default: false.
.TP
-.BI atomic \fR=\fPbool
-If value is true, attempt to use atomic direct I/O. Atomic writes are
-guaranteed to be stable once acknowledged by the operating system. Only
-Linux supports O_ATOMIC right now.
-.TP
.BI buffered \fR=\fPbool
If value is true, use buffered I/O. This is the opposite of the
\fBdirect\fR option. Defaults to true.
.TP
.B trimwrite
Sequential trim+write sequences. Blocks will be trimmed first,
-then the same blocks will be written to.
+then the same blocks will be written to. So if `io_size=64K' is specified,
+Fio will trim a total of 64K bytes and also write 64K bytes on the same
+trimmed blocks. This behaviour will be consistent with `number_ios' or
+other Fio options limiting the total bytes or number of I/O's.
+.TP
+.B randtrimwrite
+Like
+.B trimwrite ,
+but uses random offsets rather than sequential writes.
.RE
.P
Fio defaults to read if the option is not specified. For the mixed I/O
.P
\fBsequential\fR is only useful for random I/O, where fio would normally
generate a new random offset for every I/O. If you append e.g. 8 to randread,
-you would get a new random offset for every 8 I/Os. The result would be a
-seek for only every 8 I/Os, instead of for every I/O. Use `rw=randread:8'
-to specify that. As sequential I/O is already sequential, setting
-\fBsequential\fR for that would not result in any differences. \fBidentical\fR
-behaves in a similar fashion, except it sends the same offset 8 number of
-times before generating a new offset.
+i.e. `rw=randread:8' you would get a new random offset for every 8 I/Os. The
+result would be a sequence of 8 sequential offsets with a random starting
+point. However this behavior may change if a sequential I/O reaches end of the
+file. As sequential I/O is already sequential, setting \fBsequential\fR for
+that would not result in any difference. \fBidentical\fR behaves in a similar
+fashion, except it sends the same offset 8 number of times before generating a
+new offset.
+.P
+.P
+Example #1:
+.RS
+.P
+.PD 0
+rw=randread:8
+.P
+rw_sequencer=sequential
+.P
+bs=4k
+.PD
+.RE
+.P
+The generated sequence of offsets will look like this:
+4k, 8k, 12k, 16k, 20k, 24k, 28k, 32k, 92k, 96k, 100k, 104k, 108k, 112k, 116k,
+120k, 48k, 52k ...
+.P
+.P
+Example #2:
+.RS
+.P
+.PD 0
+rw=randread:8
+.P
+rw_sequencer=identical
+.P
+bs=4k
+.PD
+.RE
+.P
+The generated sequence of offsets will look like this:
+4k, 4k, 4k, 4k, 4k, 4k, 4k, 4k, 92k, 92k, 92k, 92k, 92k, 92k, 92k, 92k, 48k,
+48k, 48k ...
.RE
.TP
.BI unified_rw_reporting \fR=\fPstr
.RE
.TP
.BI randrepeat \fR=\fPbool
-Seed the random number generator used for random I/O patterns in a
-predictable way so the pattern is repeatable across runs. Default: true.
+Seed all random number generators in a predictable way so the pattern is
+repeatable across runs. Default: true.
.TP
.BI allrandrepeat \fR=\fPbool
-Seed all random number generators in a predictable way so results are
-repeatable across runs. Default: false.
+Alias for \fBrandrepeat\fR. Default: true.
.TP
.BI randseed \fR=\fPint
Seed the random number generators based on this seed value, to be able to
.TP
.B random
Advise using FADV_RANDOM.
+.TP
+.B noreuse
+Advise using FADV_NOREUSE. This may be a no-op on older Linux
+kernels. Since Linux 6.3, it provides a hint to the LRU algorithm.
+See the \fBposix_fadvise\fR\|(2) man page.
.RE
.RE
.TP
effectively caps the file size at `real_size \- offset'. Can be combined with
\fBsize\fR to constrain the start and end range of the I/O workload.
A percentage can be specified by a number between 1 and 100 followed by '%',
-for example, `offset=20%' to specify 20%. In ZBD mode, value can be set as
+for example, `offset=20%' to specify 20%. In ZBD mode, value can be set as
number of zones using 'z'.
.TP
.BI offset_align \fR=\fPint
intended to operate on a file in parallel disjoint segments, with even
spacing between the starting points. Percentages can be used for this option.
If a percentage is given, the generated offset will be aligned to the minimum
-\fBblocksize\fR or to the value of \fBoffset_align\fR if provided.In ZBD mode, value
+\fBblocksize\fR or to the value of \fBoffset_align\fR if provided.In ZBD mode, value
can be set as number of zones using 'z'.
.TP
.BI number_ios \fR=\fPint
supplied as a value between 0 and 100.
.P
The second, optional float is allowed for \fBpareto\fR, \fBzipf\fR and \fBnormal\fR
-distributions. It allows to set base of distribution in non-default place, giving
+distributions. It allows one to set base of distribution in non-default place, giving
more control over most probable outcome. This value is in range [0-1] which maps linearly to
range of possible random values.
Defaults are: random for \fBpareto\fR and \fBzipf\fR, and 0.5 for \fBnormal\fR.
.TP
.BI size \fR=\fPint[%|z]
The total size of file I/O for each thread of this job. Fio will run until
-this many bytes has been transferred, unless runtime is limited by other options
-(such as \fBruntime\fR, for instance, or increased/decreased by \fBio_size\fR).
+this many bytes has been transferred, unless runtime is altered by other means
+such as (1) \fBruntime\fR, (2) \fBio_size\fR, (3) \fBnumber_ios\fR, (4)
+gaps/holes while doing I/O's such as `rw=read:16K', or (5) sequential I/O
+reaching end of the file which is possible when \fBpercentage_random\fR is
+less than 100.
Fio will divide this size between the available files determined by options
such as \fBnrfiles\fR, \fBfilename\fR, unless \fBfilesize\fR is
specified by the job. If the result of division happens to be 0, the size is
files or devices. If the files do not exist, size must be given. It is also
possible to give size as a percentage between 1 and 100. If `size=20%' is
given, fio will use 20% of the full size of the given files or devices. In ZBD mode,
-size can be given in units of number of zones using 'z'. Can be combined with \fBoffset\fR to
+size can be given in units of number of zones using 'z'. Can be combined with \fBoffset\fR to
constrain the start and end range that I/O will be done within.
.TP
.BI io_size \fR=\fPint[%|z] "\fR,\fB io_limit" \fR=\fPint[%|z]
.BI filesize \fR=\fPirange(int)
Individual file sizes. May be a range, in which case fio will select sizes
for files at random within the given range. If not given, each created file
-is the same size. This option overrides \fBsize\fR in terms of file size,
+is the same size. This option overrides \fBsize\fR in terms of file size,
i.e. \fBsize\fR becomes merely the default for \fBio_size\fR (and
has no effect it all if \fBio_size\fR is set explicitly).
.TP
before overwriting. The \fBtrimwrite\fR mode works well for this
constraint.
.TP
-.B pmemblk
-Read and write using filesystem DAX to a file on a filesystem
-mounted with DAX on a persistent memory device through the PMDK
-libpmemblk library.
-.TP
.B dev\-dax
Read and write using device DAX to a persistent memory device (e.g.,
/dev/dax0.0) through the PMDK libpmem library.
flexibility to access GNU/Linux Kernel NVMe driver via libaio, IOCTLs, io_uring,
the SPDK NVMe driver, or your own custom NVMe driver. The xnvme engine includes
engine specific options. (See \fIhttps://xnvme.io/\fR).
+.TP
+.B libblkio
+Use the libblkio library (\fIhttps://gitlab.com/libblkio/libblkio\fR). The
+specific driver to use must be set using \fBlibblkio_driver\fR. If
+\fBmem\fR/\fBiomem\fR is not specified, memory allocation is delegated to
+libblkio (and so is guaranteed to work with the selected driver). One libblkio
+instance is used per process, so all jobs setting option \fBthread\fR will share
+a single instance (with one queue per thread) and must specify compatible
+options. Note that some drivers don't allow several instances to access the same
+device or file simultaneously, but allow it for threads.
.SS "I/O engine specific parameters"
In addition, there are some parameters which are only valid when a specific
\fBioengine\fR is in use. These are used identically to normal parameters,
before IO is started. This eliminates the need to map and release for each IO.
This is more efficient, and reduces the IO latency as well.
.TP
-.BI (io_uring,io_uring_cmd)nonvectored
+.BI (io_uring,io_uring_cmd)nonvectored \fR=\fPint
With this option, fio will use non-vectored read/write commands, where address
must contain the address directly. Default is -1.
.TP
Normally fio will submit IO by issuing a system call to notify the kernel of
available items in the SQ ring. If this option is set, the act of submitting IO
will be done by a polling thread in the kernel. This frees up cycles for fio, at
-the cost of using more CPU in the system.
+the cost of using more CPU in the system. As submission is just the time it
+takes to fill in the sqe entries and any syscall required to wake up the idle
+kernel thread, fio will not report submission latencies.
.TP
-.BI (io_uring,io_uring_cmd)sqthread_poll_cpu
+.BI (io_uring,io_uring_cmd)sqthread_poll_cpu \fR=\fPint
When `sqthread_poll` is set, this option provides a way to define which CPU
should be used for the polling thread.
.TP
When hipri is set this determines the probability of a pvsync2 I/O being high
priority. The default is 100%.
.TP
-.BI (pvsync2,libaio,io_uring)nowait
+.BI (pvsync2,libaio,io_uring,io_uring_cmd)nowait \fR=\fPbool
By default if a request cannot be executed immediately (e.g. resource starvation,
waiting on locks) it is queued and the initiating process will be blocked until
the required resource becomes free.
For direct I/O, requests will only succeed if cache invalidation isn't required,
file blocks are fully allocated and the disk request could be issued immediately.
.TP
+.BI (io_uring_cmd)fdp \fR=\fPbool
+Enable Flexible Data Placement mode for write commands.
+.TP
+.BI (io_uring_cmd)fdp_pli \fR=\fPstr
+Select which Placement ID Index/Indicies this job is allowed to use for writes.
+By default, the job will cycle through all available Placement IDs, so use this
+to isolate these identifiers to specific jobs. If you want fio to use placement
+identifier only at indices 0, 2 and 5 specify, you would set `fdp_pli=0,2,5`.
+.TP
.BI (cpuio)cpuload \fR=\fPint
Attempt to use the specified percentage of CPU cycles. This is a mandatory
option when using cpuio I/O engine.
Specify the label or UUID of the DAOS container to open.
.TP
.BI (dfs)chunk_size
-Specificy a different chunk size (in bytes) for the dfs file.
+Specify a different chunk size (in bytes) for the dfs file.
Use DAOS container's chunk size by default.
.TP
.BI (dfs)object_class
-Specificy a different object class for the dfs file.
+Specify a different object class for the dfs file.
Use DAOS container's object class by default.
.TP
.BI (nfs)nfs_url
.BI (exec)grace_time\fR=\fPint
Defines the time between the SIGTERM and SIGKILL signals. Default is 1 second.
.TP
-.BI (exec)std_redirect\fR=\fbool
+.BI (exec)std_redirect\fR=\fPbool
If set, stdout and stderr streams are redirected to files named from the job name. Default is true.
.TP
.BI (xnvme)xnvme_async\fR=\fPstr
Use the posix asynchronous I/O interface to perform one or more I/O operations
asynchronously.
.TP
+.BI vfio
+Use the user-space VFIO-based backend, implemented using libvfn instead of
+SPDK.
+.TP
.BI nil
Do not transfer any data; just pretend to. This is mainly used for
introspective performance evaluation.
.RE
.TP
.BI (xnvme)xnvme_dev_nsid\fR=\fPint
-xnvme namespace identifier for userspace NVMe driver such as SPDK.
+xnvme namespace identifier for userspace NVMe driver SPDK or vfio.
+.TP
+.BI (xnvme)xnvme_dev_subnqn\fR=\fPstr
+Sets the subsystem NQN for fabrics. This is for xNVMe to utilize a fabrics
+target with multiple systems.
+.TP
+.BI (xnvme)xnvme_mem\fR=\fPstr
+Select the xnvme memory backend. This can take these values.
+.RS
+.RS
+.TP
+.B posix
+This is the default posix memory backend for linux NVMe driver.
+.TP
+.BI hugepage
+Use hugepages, instead of existing posix memory backend. The memory backend
+uses hugetlbfs. This require users to allocate hugepages, mount hugetlbfs and
+set an enviornment variable for XNVME_HUGETLB_PATH.
+.TP
+.BI spdk
+Uses SPDK's memory allocator.
+.TP
+.BI vfio
+Uses libvfn's memory allocator. This also specifies the use of libvfn backend
+instead of SPDK.
+.RE
+.RE
.TP
.BI (xnvme)xnvme_iovec
If this option is set, xnvme will use vectored read/write commands.
+.TP
+.BI (libblkio)libblkio_driver \fR=\fPstr
+The libblkio driver to use. Different drivers access devices through different
+underlying interfaces. Available drivers depend on the libblkio version in use
+and are listed at \fIhttps://libblkio.gitlab.io/libblkio/blkio.html#drivers\fR
+.TP
+.BI (libblkio)libblkio_path \fR=\fPstr
+Sets the value of the driver-specific "path" property before connecting the
+libblkio instance, which identifies the target device or file on which to
+perform I/O. Its exact semantics are driver-dependent and not all drivers may
+support it; see \fIhttps://libblkio.gitlab.io/libblkio/blkio.html#drivers\fR
+.TP
+.BI (libblkio)libblkio_pre_connect_props \fR=\fPstr
+A colon-separated list of additional libblkio properties to be set after
+creating but before connecting the libblkio instance. Each property must have
+the format \fB<name>=<value>\fR. Colons can be escaped as \fB\\:\fR. These are
+set after the engine sets any other properties, so those can be overriden.
+Available properties depend on the libblkio version in use and are listed at
+\fIhttps://libblkio.gitlab.io/libblkio/blkio.html#properties\fR
+.TP
+.BI (libblkio)libblkio_num_entries \fR=\fPint
+Sets the value of the driver-specific "num-entries" property before starting the
+libblkio instance. Its exact semantics are driver-dependent and not all drivers
+may support it; see \fIhttps://libblkio.gitlab.io/libblkio/blkio.html#drivers\fR
+.TP
+.BI (libblkio)libblkio_queue_size \fR=\fPint
+Sets the value of the driver-specific "queue-size" property before starting the
+libblkio instance. Its exact semantics are driver-dependent and not all drivers
+may support it; see \fIhttps://libblkio.gitlab.io/libblkio/blkio.html#drivers\fR
+.TP
+.BI (libblkio)libblkio_pre_start_props \fR=\fPstr
+A colon-separated list of additional libblkio properties to be set after
+connecting but before starting the libblkio instance. Each property must have
+the format \fB<name>=<value>\fR. Colons can be escaped as \fB\\:\fR. These are
+set after the engine sets any other properties, so those can be overriden.
+Available properties depend on the libblkio version in use and are listed at
+\fIhttps://libblkio.gitlab.io/libblkio/blkio.html#properties\fR
+.TP
+.BI (libblkio)hipri
+Use poll queues. This is incompatible with \fBlibblkio_wait_mode=eventfd\fR and
+\fBlibblkio_force_enable_completion_eventfd\fR.
+.TP
+.BI (libblkio)libblkio_vectored
+Submit vectored read and write requests.
+.TP
+.BI (libblkio)libblkio_write_zeroes_on_trim
+Submit trims as "write zeroes" requests instead of discard requests.
+.TP
+.BI (libblkio)libblkio_wait_mode \fR=\fPstr
+How to wait for completions:
+.RS
+.RS
+.TP
+.B block \fR(default)
+Use a blocking call to \fBblkioq_do_io()\fR.
+.TP
+.B eventfd
+Use a blocking call to \fBread()\fR on the completion eventfd.
+.TP
+.B loop
+Use a busy loop with a non-blocking call to \fBblkioq_do_io()\fR.
+.RE
+.RE
+.TP
+.BI (libblkio)libblkio_force_enable_completion_eventfd
+Enable the queue's completion eventfd even when unused. This may impact
+performance. The default is to enable it only if
+\fBlibblkio_wait_mode=eventfd\fR.
.SS "I/O depth"
.TP
.BI iodepth \fR=\fPint
Trim this number of I/O blocks.
.TP
.BI experimental_verify \fR=\fPbool
-Enable experimental verification.
+Enable experimental verification. Standard verify records I/O metadata for
+later use during the verification phase. Experimental verify instead resets the
+file after the write phase and then replays I/Os for the verification phase.
.SS "Steady state"
.TP
.BI steadystate \fR=\fPstr:float "\fR,\fP ss" \fR=\fPstr:float
.TP
.BI steadystate_duration \fR=\fPtime "\fR,\fP ss_dur" \fR=\fPtime
A rolling window of this duration will be used to judge whether steady state
-has been reached. Data will be collected once per second. The default is 0
-which disables steady state detection. When the unit is omitted, the
-value is interpreted in seconds.
+has been reached. Data will be collected every \fBss_interval\fR. The default
+is 0 which disables steady state detection. When the unit is omitted, the value
+is interpreted in seconds.
.TP
.BI steadystate_ramp_time \fR=\fPtime "\fR,\fP ss_ramp" \fR=\fPtime
Allow the job to run for the specified duration before beginning data
collection for checking the steady state job termination criterion. The
default is 0. When the unit is omitted, the value is interpreted in seconds.
+.TP
+.BI steadystate_check_interval \fR=\fPtime "\fR,\fP ss_interval" \fR=\fPtime
+The values suring the rolling window will be collected with a period of this
+value. If \fBss_interval\fR is 30s and \fBss_dur\fR is 300s, 10 measurements
+will be taken. Default is 1s but that might not converge, especially for slower
+devices, so set this accordingly. When the unit is omitted, the value is
+interpreted in seconds.
.SS "Measurements and reporting"
.TP
.BI per_job_logs \fR=\fPbool
completed. If this option is used, there are two more stats that are
appended, the total error count and the first error. The error field given
in the stats is the first error that was hit during the run.
+.RS
+.P
+Note: a write error from the device may go unnoticed by fio when using buffered
+IO, as the write() (or similar) system call merely dirties the kernel pages,
+unless `sync' or `direct' is used. Device IO errors occur when the dirty data is
+actually written out to disk. If fully sync writes aren't desirable, `fsync' or
+`fdatasync' can be used as well. This is specific to writes, as reads are always
+synchronous.
+.RS
+.P
The allowed values are:
.RS
.RS
.TP
.B Trace file format v2
The second version of the trace file format was added in fio version 1.17. It
-allows to access more then one file per trace and has a bigger set of possible
+allows one to access more than one file per trace and has a bigger set of possible
file actions.
.RS
.P
projects / fio.git / blobdiff