parsed and used.
.TP
.BI \-\-alloc\-size \fR=\fPkb
-Set the internal smalloc pool size to \fIkb\fR in KiB. The
-\fB\-\-alloc\-size\fR switch allows one to use a larger pool size for smalloc.
+Allocate additional internal smalloc pools of size \fIkb\fR in KiB. The
+\fB\-\-alloc\-size\fR option increases shared memory set aside for use by fio.
If running large jobs with randommap enabled, fio can run out of memory.
Smalloc is an internal allocator for shared structures from a fixed size
memory pool and can grow to 16 pools. The pool size defaults to 16MiB.
unless otherwise specified.
.P
With `kb_base=1000', fio follows international standards for unit
-prefixes. To specify power\-of\-10 decimal values defined in the
+prefixes. To specify power-of-10 decimal values defined in the
International System of Units (SI):
.RS
.P
.PD
.RE
.P
-To specify power\-of\-2 binary values defined in IEC 80000\-13:
+To specify power-of-2 binary values defined in IEC 80000-13:
.RS
.P
.PD 0
.RE
.P
With `kb_base=1024' (the default), the unit prefixes are opposite
-from those specified in the SI and IEC 80000\-13 standards to provide
+from those specified in the SI and IEC 80000-13 standards to provide
compatibility with old scripts. For example, 4k means 4096.
.P
For quantities of data, an optional unit of 'B' may be included
.RS
.TP
.B 1000
-Inputs comply with IEC 80000\-13 and the International
+Inputs comply with IEC 80000-13 and the International
System of Units (SI). Use:
.RS
.P
.PD 0
-\- power\-of\-2 values with IEC prefixes (e.g., KiB)
+\- power-of-2 values with IEC prefixes (e.g., KiB)
.P
-\- power\-of\-10 values with SI prefixes (e.g., kB)
+\- power-of-10 values with SI prefixes (e.g., kB)
.PD
.RE
.TP
.P
.RS
.PD 0
-\- power\-of\-2 values with SI prefixes
+\- power-of-2 values with SI prefixes
.P
-\- power\-of\-10 values with IEC prefixes
+\- power-of-10 values with IEC prefixes
.PD
.RE
.RE
See \fBbs\fR for more details on input parameters.
.P
Outputs always use correct prefixes. Most outputs include both
-side\-by\-side, like:
+side-by-side, like:
.P
.RS
bw=2383.3kB/s (2327.4KiB/s)
.RS
.TP
.B 0
-Use auto\-detection (default).
+Use auto-detection (default).
.TP
.B 8
Byte based.
specified, but lets all clones use the same file if set).
.RS
.P
-See the \fBfilename\fR option for information on how to escape ':' and '\\'
+See the \fBfilename\fR option for information on how to escape ':'
characters within the directory path itself.
.P
Note: To control the directory fio will use for internal state files
explicit size is specified by \fBfilesize\fR.
.RS
.P
-Each colon and backslash in the wanted path must be escaped with a '\\'
+Each colon in the wanted path must be escaped with a '\\'
character. For instance, if the path is `/dev/dsk/foo@3,0:c' then you
would use `filename=/dev/dsk/foo@3,0\\:c' and if the path is
-`F:\\filename' then you would use `filename=F\\:\\\\filename'.
+`F:\\filename' then you would use `filename=F\\:\\filename'.
.P
On Windows, disk devices are accessed as `\\\\.\\PhysicalDrive0' for
the first device, `\\\\.\\PhysicalDrive1' for the second etc.
Note: Windows and FreeBSD prevent write access to areas
-of the disk containing in\-use data (e.g. filesystems).
+of the disk containing in-use data (e.g. filesystems).
.P
The filename `\-' is a reserved name, meaning *stdin* or *stdout*. Which
of the two depends on the read/write direction set.
For \fBrandom\fR, \fBroundrobin\fR, and \fBsequential\fR, a postfix can be appended to
tell fio how many I/Os to issue before switching to a new file. For example,
specifying `file_service_type=random:8' would cause fio to issue
-8 I/Os before selecting a new file at random. For the non\-uniform
+8 I/Os before selecting a new file at random. For the non-uniform
distributions, a floating point postfix can be given to influence how the
distribution is skewed. See \fBrandom_distribution\fR for a description
of how that would work.
\fBfsync\fR\|(2) the data file after creation. This is the default.
.TP
.BI create_on_open \fR=\fPbool
-If true, don't pre\-create files but allow the job's open() to create a file
-when it's time to do I/O. Default: false \-\- pre\-create all necessary files
+If true, don't pre-create files but allow the job's open() to create a file
+when it's time to do I/O. Default: false \-\- pre-create all necessary files
when the job starts.
.TP
.BI create_only \fR=\fPbool
writing against a mounted device regardless of this option. Default: false.
.TP
.BI pre_read \fR=\fPbool
-If this is given, files will be pre\-read into memory before starting the
+If this is given, files will be pre-read into memory before starting the
given I/O operation. This will also clear the \fBinvalidate\fR flag,
-since it is pointless to pre\-read and then drop the cache. This will only
-work for I/O engines that are seek\-able, since they allow you to read the
-same data multiple times. Thus it will not work on non\-seekable I/O engines
+since it is pointless to pre-read and then drop the cache. This will only
+work for I/O engines that are seek-able, since they allow you to read the
+same data multiple times. Thus it will not work on non-seekable I/O engines
(e.g. network, splice). Default: false.
.TP
.BI unlink \fR=\fPbool
.RE
.TP
.BI zonerange \fR=\fPint
-Size of a single zone. See also \fBzonesize\fR and \fBzoneskip\fR.
+For \fBzonemode\fR=strided, this is the size of a single zone. See also
+\fBzonesize\fR and \fBzoneskip\fR.
+
+For \fBzonemode\fR=zbd, this parameter is ignored.
.TP
.BI zonesize \fR=\fPint
For \fBzonemode\fR=strided, this is the number of bytes to transfer before
will be accessed. If this parameter is larger than \fBzonerange\fR then each
zone will be accessed multiple times before skipping to the next zone.
-For \fBzonemode\fR=zbd, this is the size of a single zone. The \fBzonerange\fR
-parameter is ignored in this mode.
+For \fBzonemode\fR=zbd, this is the size of a single zone. The
+\fBzonerange\fR parameter is ignored in this mode. For a job accessing a
+zoned block device, the specified \fBzonesize\fR must be 0 or equal to the
+device zone size. For a regular block device or file, the specified
+\fBzonesize\fR must be at least 512B.
.TP
.BI zoneskip \fR=\fPint
For \fBzonemode\fR=strided, the number of bytes to skip after \fBzonesize\fR
-bytes of data have been transferred. This parameter must be zero for
-\fBzonemode\fR=zbd.
+bytes of data have been transferred.
+
+For \fBzonemode\fR=zbd, the \fBzonesize\fR aligned number of bytes to skip
+once a zone is fully written (write workloads) or all written data in the
+zone have been read (read workloads). This parameter is valid only for
+sequential workloads and ignored for random workloads. For read workloads,
+see also \fBread_beyond_wp\fR.
.TP
.BI read_beyond_wp \fR=\fPbool
zone has a type, e.g. conventional or sequential. A conventional zone can be
written at any offset that is a multiple of the block size. Sequential zones
must be written sequentially. The position at which a write must occur is
-called the write pointer. A zoned block device can be either drive
-managed, host managed or host aware. For host managed devices the host must
-ensure that writes happen sequentially. Fio recognizes host managed devices
-and serializes writes to sequential zones for these devices.
+called the write pointer. A zoned block device can be either host managed or
+host aware. For host managed devices the host must ensure that writes happen
+sequentially. Fio recognizes host managed devices and serializes writes to
+sequential zones for these devices.
If a read occurs in a sequential zone beyond the write pointer then the zoned
block device will complete the read without reading any data from the storage
When running a random write test across an entire drive many more zones will be
open than in a typical application workload. Hence this command line option
that allows to limit the number of open zones. The number of open zones is
-defined as the number of zones to which write commands are issued.
+defined as the number of zones to which write commands are issued by all
+threads/processes.
+.TP
+.BI job_max_open_zones \fR=\fPint
+Limit on the number of simultaneously opened zones per single thread/process.
.TP
.BI zone_reset_threshold \fR=\fPfloat
A number between zero and one that indicates the ratio of logical blocks with
.SS "I/O type"
.TP
.BI direct \fR=\fPbool
-If value is true, use non\-buffered I/O. This is usually O_DIRECT. Note that
+If value is true, use non-buffered I/O. This is usually O_DIRECT. Note that
OpenBSD and ZFS on Solaris don't support direct I/O. On Windows the synchronous
ioengines don't support direct I/O. Default: false.
.TP
sequence depends on the \fBrandrepeat\fR setting.
.TP
.BI fallocate \fR=\fPstr
-Whether pre\-allocation is performed when laying down files.
+Whether pre-allocation is performed when laying down files.
Accepted values are:
.RS
.RS
.TP
.B none
-Do not pre\-allocate space.
+Do not pre-allocate space.
.TP
.B native
-Use a platform's native pre\-allocation call but fall back to
+Use a platform's native pre-allocation call but fall back to
\fBnone\fR behavior if it fails/is not implemented.
.TP
.B posix
-Pre\-allocate via \fBposix_fallocate\fR\|(3).
+Pre-allocate via \fBposix_fallocate\fR\|(3).
.TP
.B keep
-Pre\-allocate via \fBfallocate\fR\|(2) with
+Pre-allocate via \fBfallocate\fR\|(2) with
FALLOC_FL_KEEP_SIZE set.
.TP
+.B truncate
+Extend file to final size using \fBftruncate\fR|(2)
+instead of allocating.
+.TP
.B 0
-Backward\-compatible alias for \fBnone\fR.
+Backward-compatible alias for \fBnone\fR.
.TP
.B 1
-Backward\-compatible alias for \fBposix\fR.
+Backward-compatible alias for \fBposix\fR.
.RE
.P
May not be available on all supported platforms. \fBkeep\fR is only available
on Linux. If using ZFS on Solaris this cannot be set to \fBposix\fR
-because ZFS doesn't support pre\-allocation. Default: \fBnative\fR if any
-pre\-allocation methods are available, \fBnone\fR if not.
+because ZFS doesn't support pre-allocation. Default: \fBnative\fR if any
+pre-allocation methods except \fBtruncate\fR are available, \fBnone\fR if not.
+.P
+Note that using \fBtruncate\fR on Windows will interact surprisingly
+with non-sequential write patterns. When writing to a file that has
+been extended by setting the end-of-file information, Windows will
+backfill the unwritten portion of the file up to that offset with
+zeroes before issuing the new write. This means that a single small
+write to the end of an extended file will stall until the entire
+file has been filled with zeroes.
.RE
.TP
.BI fadvise_hint \fR=\fPstr
.BI offset_increment \fR=\fPint
If this is provided, then the real offset becomes `\fBoffset\fR + \fBoffset_increment\fR
* thread_number', where the thread number is a counter that starts at 0 and
-is incremented for each sub\-job (i.e. when \fBnumjobs\fR option is
+is incremented for each sub-job (i.e. when \fBnumjobs\fR option is
specified). This option is useful if there are several jobs which are
intended to operate on a file in parallel disjoint segments, with even
-spacing between the starting points.
+spacing between the starting points. Percentages can be used for this option.
+If a percentage is given, the generated offset will be aligned to the minimum
+\fBblocksize\fR or to the value of \fBoffset_align\fR if provided.
.TP
.BI number_ios \fR=\fPint
Fio will normally perform I/Os until it has exhausted the size of the region
the number of I/Os to perform. When fio reaches this number, it will exit
normally and report status. Note that this does not extend the amount of I/O
that will be done, it will only stop fio if this condition is met before
-other end\-of\-job criteria.
+other end-of-job criteria.
.TP
.BI fsync \fR=\fPint
If writing to a file, issue an \fBfsync\fR\|(2) (or its equivalent) of
the dirty data for every number of blocks given. For example, if you give 32
as a parameter, fio will sync the file after every 32 writes issued. If fio is
-using non\-buffered I/O, we may not sync the file. The exception is the sg
+using non-buffered I/O, we may not sync the file. The exception is the sg
I/O engine, which synchronizes the disk cache anyway. Defaults to 0, which
means fio does not periodically issue and wait for a sync to complete. Also
see \fBend_fsync\fR and \fBfsync_on_close\fR.
.TP
.BI fdatasync \fR=\fPint
Like \fBfsync\fR but uses \fBfdatasync\fR\|(2) to only sync data and
-not metadata blocks. In Windows, FreeBSD, and DragonFlyBSD there is no
+not metadata blocks. In Windows, FreeBSD, DragonFlyBSD or OSX there is no
\fBfdatasync\fR\|(2) so this falls back to using \fBfsync\fR\|(2).
Defaults to 0, which means fio does not periodically issue and wait for a
-data\-only sync to complete.
+data-only sync to complete.
.TP
.BI write_barrier \fR=\fPint
Make every N\-th write a barrier write.
defaults to 100%, in which case the workload is fully random. It can be set
from anywhere from 0 to 100. Setting it to 0 would make the workload fully
sequential. Any setting in between will result in a random mix of sequential
-and random I/O, at the given percentages. Comma\-separated values may be
+and random I/O, at the given percentages. Comma-separated values may be
specified for reads, writes, and trims as described in \fBblocksize\fR.
.TP
.BI norandommap
at past I/O history. This means that some blocks may not be read or written,
and that some blocks may be read/written more than once. If this option is
used with \fBverify\fR and multiple blocksizes (via \fBbsrange\fR),
-only intact blocks are verified, i.e., partially\-overwritten blocks are
+only intact blocks are verified, i.e., partially-overwritten blocks are
ignored. With an async I/O engine and an I/O depth > 1, it is possible for
the same block to be overwritten, which can cause verification errors. Either
do not use norandommap in this case, or also use the lfsr random generator.
.TP
.BI blocksize \fR=\fPint[,int][,int] "\fR,\fB bs" \fR=\fPint[,int][,int]
The block size in bytes used for I/O units. Default: 4096. A single value
-applies to reads, writes, and trims. Comma\-separated values may be
+applies to reads, writes, and trims. Comma-separated values may be
specified for reads, writes, and trims. A value not terminated in a comma
applies to subsequent types. Examples:
.RS
A range of block sizes in bytes for I/O units. The issued I/O unit will
always be a multiple of the minimum size, unless
\fBblocksize_unaligned\fR is set.
-Comma\-separated ranges may be specified for reads, writes, and trims as
+Comma-separated ranges may be specified for reads, writes, and trims as
described in \fBblocksize\fR. Example:
.RS
.RS
would have 50% 4k ios, and 25% 1k and 32k ios. The percentages always add up
to 100, if bssplit is given a range that adds up to more, it will error out.
.P
-Comma\-separated values may be specified for reads, writes, and trims as
+Comma-separated values may be specified for reads, writes, and trims as
described in \fBblocksize\fR.
.P
If you want a workload that has 50% 2k reads and 50% 4k reads, while having
\fBblocksize\fR. Minimum alignment is typically 512b for using direct
I/O, though it usually depends on the hardware block size. This option is
mutually exclusive with using a random map for files, so it will turn off
-that option. Comma\-separated values may be specified for reads, writes, and
+that option. Comma-separated values may be specified for reads, writes, and
trims as described in \fBblocksize\fR.
.SS "Buffers and memory"
.TP
.BI scramble_buffers \fR=\fPbool
If \fBrefill_buffers\fR is too costly and the target is using data
deduplication, then setting this option will slightly modify the I/O buffer
-contents to defeat normal de\-dupe attempts. This is not enough to defeat
+contents to defeat normal de-dupe attempts. This is not enough to defeat
more clever block compression attempts, but it will stop naive dedupe of
blocks. Default: true.
.TP
given job file, add up the I/O depth of all jobs (normally one unless
\fBiodepth\fR is used) and multiply by the maximum bs set. Then divide
that number by the huge page size. You can see the size of the huge pages in
-`/proc/meminfo'. If no huge pages are allocated by having a non\-zero
+`/proc/meminfo'. If no huge pages are allocated by having a non-zero
number in `nr_hugepages', using \fBmmaphuge\fR or \fBshmhuge\fR will fail. Also
see \fBhugepage\-size\fR.
.P
Defines the size of a huge page. Must at least be equal to the system
setting, see `/proc/meminfo'. Defaults to 4MiB. Should probably
always be a multiple of megabytes, so using `hugepage\-size=Xm' is the
-preferred way to set this to avoid setting a non\-pow\-2 bad value.
+preferred way to set this to avoid setting a non-pow-2 bad value.
.TP
.BI lockmem \fR=\fPint
Pin the specified amount of memory with \fBmlock\fR\|(2). Can be used to
Perform I/O after the end of the file. Normally fio will operate within the
size of a file. If this option is set, then fio will append to the file
instead. This has identical behavior to setting \fBoffset\fR to the size
-of a file. This option is ignored on non\-regular files.
+of a file. This option is ignored on non-regular files.
.TP
.BI fill_device \fR=\fPbool "\fR,\fB fill_fs" \fR=\fPbool
Sets size to something really large and waits for ENOSPC (no space left on
write. For a read workload, the mount point will be filled first then I/O
started on the result. This option doesn't make sense if operating on a raw
device node, since the size of that is already known by the file system.
-Additionally, writing beyond end\-of\-device will not return ENOSPC there.
+Additionally, writing beyond end-of-device will not return ENOSPC there.
.SS "I/O engine"
.TP
.BI ioengine \fR=\fPstr
.TP
.B libaio
Linux native asynchronous I/O. Note that Linux may only support
-queued behavior with non\-buffered I/O (set `direct=1' or
+queued behavior with non-buffered I/O (set `direct=1' or
`buffered=0').
This engine defines engine specific options.
.TP
character devices. This engine supports trim operations. The
sg engine includes engine specific options.
.TP
+.B libzbc
+Synchronous I/O engine for SMR hard-disks using the \fBlibzbc\fR
+library. The target can be either an sg character device or
+a block device file. This engine supports the zonemode=zbd zone
+operations.
+.TP
.B null
Doesn't transfer any data, just pretends to. This is mainly used to
exercise fio itself and for debugging/testing purposes.
of the CPU. In case of SMP machines, use `numjobs=<nr_of_cpu>'
to get desired CPU usage, as the cpuload only loads a
single CPU at the desired rate. A job never finishes unless there is
-at least one non\-cpuio job.
+at least one non-cpuio job.
.TP
.B guasi
The GUASI I/O engine is the Generic Userspace Asynchronous Syscall
-Interface approach to async I/O. See \fIhttp://www.xmailserver.org/guasi\-lib.html\fR
+Interface approach to async I/O. See \fIhttp://www.xmailserver.org/guasi-lib.html\fR
for more info on GUASI.
.TP
.B rdma
\fBfilesize\fR so that all the accounting still occurs, but no actual I/O will be
done other than creating the file.
.TP
+.B filestat
+Simply do stat() and do no I/O to the file. You need to set 'filesize'
+and 'nrfiles', so that files will be created.
+This engine is to measure file lookup and meta data access.
+.TP
.B libpmem
Read and write using mmap I/O to a file on a filesystem
mounted with DAX on a persistent memory device through the PMDK
.TP
.B libiscsi
Read and write iscsi lun with libiscsi.
+.TP
+.B nbd
+Synchronous read and write a Network Block Device (NBD).
.SS "I/O engine specific parameters"
In addition, there are some parameters which are only valid when a specific
\fBioengine\fR is in use. These are used identically to normal parameters,
with the caveat that when used on the command line, they must come after the
\fBioengine\fR that defines them is selected.
.TP
+.BI (io_uring, libaio)cmdprio_percentage \fR=\fPint
+Set the percentage of I/O that will be issued with higher priority by setting
+the priority bit. Non-read I/O is likely unaffected by ``cmdprio_percentage``.
+This option cannot be used with the `prio` or `prioclass` options. For this
+option to set the priority bit properly, NCQ priority must be supported and
+enabled and `direct=1' option must be used. fio must also be run as the root
+user.
+.TP
+.BI (io_uring)fixedbufs
+If fio is asked to do direct IO, then Linux will map pages for each IO call, and
+release them when IO is done. If this option is set, the pages are pre-mapped
+before IO is started. This eliminates the need to map and release for each IO.
+This is more efficient, and reduces the IO latency as well.
+.TP
+.BI (io_uring)hipri
+If this option is set, fio will attempt to use polled IO completions. Normal IO
+completions generate interrupts to signal the completion of IO, polled
+completions do not. Hence they are require active reaping by the application.
+The benefits are more efficient IO for high IOPS scenarios, and lower latencies
+for low queue depth IO.
+.TP
+.BI (io_uring)registerfiles
+With this option, fio registers the set of files being used with the kernel.
+This avoids the overhead of managing file counts in the kernel, making the
+submission and completion part more lightweight. Required for the below
+sqthread_poll option.
+.TP
+.BI (io_uring)sqthread_poll
+Normally fio will submit IO by issuing a system call to notify the kernel of
+available items in the SQ ring. If this option is set, the act of submitting IO
+will be done by a polling thread in the kernel. This frees up cycles for fio, at
+the cost of using more CPU in the system.
+.TP
+.BI (io_uring)sqthread_poll_cpu
+When `sqthread_poll` is set, this option provides a way to define which CPU
+should be used for the polling thread.
+.TP
.BI (libaio)userspace_reap
Normally, with the libaio engine in use, fio will use the
\fBio_getevents\fR\|(3) system call to reap newly returned events. With
-this flag turned on, the AIO ring will be read directly from user\-space to
+this flag turned on, the AIO ring will be read directly from user-space to
reap events. The reaping mode is only enabled when polling for a minimum of
0 events (e.g. when `iodepth_batch_complete=0').
.TP
function. This can be useful when multiple paths exist between the
client and the server or in certain loopback configurations.
.TP
+.BI (filestat)stat_type \fR=\fPstr
+Specify stat system call type to measure lookup/getattr performance.
+Default is \fBstat\fR for \fBstat\fR\|(2).
+.TP
.BI (sg)readfua \fR=\fPbool
With readfua option set to 1, read operations include the force
unit access (fua) flag. Default: 0.
generate 8k of data for each command butonly the first 512 bytes will
be used and transferred to the device. The writefua option is ignored
with this selection.
+.RE
+.RE
+.TP
+.BI (nbd)uri \fR=\fPstr
+Specify the NBD URI of the server to test.
+The string is a standard NBD URI (see
+\fIhttps://github.com/NetworkBlockDevice/nbd/tree/master/doc\fR).
+Example URIs:
+.RS
+.RS
+.TP
+\fInbd://localhost:10809\fR
+.TP
+\fInbd+unix:///?socket=/tmp/socket\fR
+.TP
+\fInbds://tlshost/exportname\fR
.SS "I/O depth"
.TP
\fBthinktime_blocks\fR and \fBthinktime_spin\fR.
.TP
.BI thinktime_spin \fR=\fPtime
-Only valid if \fBthinktime\fR is set \- pretend to spend CPU time doing
+Only valid if \fBthinktime\fR is set - pretend to spend CPU time doing
something with the data received, before falling back to sleeping for the
rest of the period specified by \fBthinktime\fR. When the unit is
omitted, the value is interpreted in microseconds.
.TP
.BI thinktime_blocks \fR=\fPint
-Only valid if \fBthinktime\fR is set \- control how many blocks to issue,
+Only valid if \fBthinktime\fR is set - control how many blocks to issue,
before waiting \fBthinktime\fR usecs. If not set, defaults to 1 which will make
fio wait \fBthinktime\fR usecs after every block. This effectively makes any
queue depth setting redundant, since no more than 1 I/O will be queued
.TP
.BI rate \fR=\fPint[,int][,int]
Cap the bandwidth used by this job. The number is in bytes/sec, the normal
-suffix rules apply. Comma\-separated values may be specified for reads,
+suffix rules apply. Comma-separated values may be specified for reads,
writes, and trims as described in \fBblocksize\fR.
.RS
.P
.TP
.BI rate_min \fR=\fPint[,int][,int]
Tell fio to do whatever it can to maintain at least this bandwidth. Failing
-to meet this requirement will cause the job to exit. Comma\-separated values
+to meet this requirement will cause the job to exit. Comma-separated values
may be specified for reads, writes, and trims as described in
\fBblocksize\fR.
.TP
Cap the bandwidth to this number of IOPS. Basically the same as
\fBrate\fR, just specified independently of bandwidth. If the job is
given a block size range instead of a fixed value, the smallest block size
-is used as the metric. Comma\-separated values may be specified for reads,
+is used as the metric. Comma-separated values may be specified for reads,
writes, and trims as described in \fBblocksize\fR.
.TP
.BI rate_iops_min \fR=\fPint[,int][,int]
If fio doesn't meet this rate of I/O, it will cause the job to exit.
-Comma\-separated values may be specified for reads, writes, and trims as
+Comma-separated values may be specified for reads, writes, and trims as
described in \fBblocksize\fR.
.TP
.BI rate_process \fR=\fPstr
defaults to 100.0, meaning that all I/Os must be equal or below to the value
set by \fBlatency_target\fR.
.TP
+.BI latency_run \fR=\fPbool
+Used with \fBlatency_target\fR. If false (default), fio will find the highest
+queue depth that meets \fBlatency_target\fR and exit. If true, fio will continue
+running and try to meet \fBlatency_target\fR by adjusting queue depth.
+.TP
.BI max_latency \fR=\fPtime
If set, fio will exit the job with an ETIMEDOUT error if it exceeds this
maximum latency. When the unit is omitted, the value is interpreted in
replay, the file needs to be turned into a blkparse binary data file first
(`blkparse <device> \-o /dev/null \-d file_for_fio.bin').
You can specify a number of files by separating the names with a ':' character.
-See the \fBfilename\fR option for information on how to escape ':' and '\'
+See the \fBfilename\fR option for information on how to escape ':'
characters within the file names. These files will be sequentially assigned to
job clones created by \fBnumjobs\fR.
.TP
Set the I/O priority value of this job. Linux limits us to a positive value
between 0 and 7, with 0 being the highest. See man
\fBionice\fR\|(1). Refer to an appropriate manpage for other operating
-systems since meaning of priority may differ.
+systems since meaning of priority may differ. For per-command priority
+setting, see I/O engine specific `cmdprio_percentage` and `hipri_percentage`
+options.
.TP
.BI prioclass \fR=\fPint
-Set the I/O priority class. See man \fBionice\fR\|(1).
+Set the I/O priority class. See man \fBionice\fR\|(1). For per-command
+priority setting, see I/O engine specific `cmdprio_percentage` and `hipri_percent`
+options.
.TP
.BI cpus_allowed \fR=\fPstr
Controls the same options as \fBcpumask\fR, but accepts a textual
.RE
.P
\fBshared\fR is the default behavior, if the option isn't specified. If
-\fBsplit\fR is specified, then fio will will assign one cpu per job. If not
+\fBsplit\fR is specified, then fio will assign one cpu per job. If not
enough CPUs are given for the jobs listed, then fio will roundrobin the CPUs
in the set.
.RE
flow. See \fBflow\fR.
.TP
.BI flow \fR=\fPint
-Weight in token\-based flow control. If this value is used, then there is
+Weight in token-based flow control. If this value is used, then there is
a 'flow counter' which is used to regulate the proportion of activity between
two or more jobs. Fio attempts to keep this flow counter near zero. The
\fBflow\fR parameter stands for how much should be added or subtracted to the
\fBgroup_reporting\fR.
.TP
.BI exitall
-By default, fio will continue running all other jobs when one job finishes
-but sometimes this is not the desired action. Setting \fBexitall\fR will
-instead make fio terminate all other jobs when one job finishes.
+By default, fio will continue running all other jobs when one job finishes.
+Sometimes this is not the desired action. Setting \fBexitall\fR will instead
+make fio terminate all jobs in the same group, as soon as one job of that
+group finishes.
+.TP
+.BI exit_what
+By default, fio will continue running all other jobs when one job finishes.
+Sometimes this is not the desired action. Setting \fBexit_all\fR will instead
+make fio terminate all jobs in the same group. The option \fBexit_what\fR
+allows to control which jobs get terminated when \fBexitall\fR is enabled. The
+default is \fBgroup\fR and does not change the behaviour of \fBexitall\fR. The
+setting \fBall\fR terminates all jobs. The setting \fBstonewall\fR terminates
+all currently running jobs across all groups and continues execution with the
+next stonewalled group.
.TP
.BI exec_prerun \fR=\fPstr
Before running this job, issue the command specified through
It may sometimes be interesting to display statistics for groups of jobs as
a whole instead of for each individual job. This is especially true if
\fBnumjobs\fR is used; looking at individual thread/process output
-quickly becomes unwieldy. To see the final report per\-group instead of
-per\-job, use \fBgroup_reporting\fR. Jobs in a file will be part of the
+quickly becomes unwieldy. To see the final report per-group instead of
+per-job, use \fBgroup_reporting\fR. Jobs in a file will be part of the
same reporting group, unless if separated by a \fBstonewall\fR, or by
using \fBnew_group\fR.
.TP
.TP
.BI log_unix_epoch \fR=\fPbool
If set, fio will log Unix timestamps to the log files produced by enabling
-write_type_log for each log type, instead of the default zero\-based
+write_type_log for each log type, instead of the default zero-based
timestamps.
.TP
.BI block_error_percentiles \fR=\fPbool
-If set, record errors in trim block\-sized units from writes and trims and
+If set, record errors in trim block-sized units from writes and trims and
output a histogram of how many trims it took to get to errors, and what kind
of error was encountered.
.TP
Disable measurements of throughput/bandwidth numbers. See
\fBdisable_lat\fR.
.TP
+.BI slat_percentiles \fR=\fPbool
+Report submission latency percentiles. Submission latency is not recorded
+for synchronous ioengines.
+.TP
.BI clat_percentiles \fR=\fPbool
-Enable the reporting of percentiles of completion latencies. This option is
-mutually exclusive with \fBlat_percentiles\fR.
+Report completion latency percentiles.
.TP
.BI lat_percentiles \fR=\fPbool
-Enable the reporting of percentiles of I/O latencies. This is similar to
-\fBclat_percentiles\fR, except that this includes the submission latency.
-This option is mutually exclusive with \fBclat_percentiles\fR.
+Report total latency percentiles. Total latency is the sum of submission
+latency and completion latency.
.TP
.BI percentile_list \fR=\fPfloat_list
-Overwrite the default list of percentiles for completion latencies and the
-block error histogram. Each number is a floating number in the range
+Overwrite the default list of percentiles for latencies and the
+block error histogram. Each number is a floating point number in the range
(0,100], and the maximum length of the list is 20. Use ':' to separate the
-numbers, and list the numbers in ascending order. For example,
-`\-\-percentile_list=99.5:99.9' will cause fio to report the values of
-completion latency below which 99.5% and 99.9% of the observed latencies
-fell, respectively.
+numbers. For example, `\-\-percentile_list=99.5:99.9' will cause fio to
+report the latency durations below which 99.5% and 99.9% of the observed
+latencies fell, respectively.
.TP
.BI significant_figures \fR=\fPint
If using \fB\-\-output\-format\fR of `normal', set the significant figures
.TP
.BI continue_on_error \fR=\fPstr
Normally fio will exit the job on the first observed failure. If this option
-is set, fio will continue the job when there is a 'non\-fatal error' (EIO or
+is set, fio will continue the job when there is a 'non-fatal error' (EIO or
EILSEQ) until the runtime is exceeded or the I/O size specified is
completed. If this option is used, there are two more stats that are
appended, the total error count and the first error. The error field given
Continue on all errors.
.TP
.B 0
-Backward\-compatible alias for 'none'.
+Backward-compatible alias for 'none'.
.TP
.B 1
-Backward\-compatible alias for 'all'.
+Backward-compatible alias for 'all'.
.RE
.RE
.TP
.BI ignore_error \fR=\fPstr
Sometimes you want to ignore some errors during test in that case you can
specify error list for each error type, instead of only being able to
-ignore the default 'non\-fatal error' using \fBcontinue_on_error\fR.
+ignore the default 'non-fatal error' using \fBcontinue_on_error\fR.
`ignore_error=READ_ERR_LIST,WRITE_ERR_LIST,VERIFY_ERR_LIST' errors for
given error type is separated with ':'. Error may be symbol ('ENOSPC', 'ENOMEM')
or integer. Example:
Thread initialized, waiting or generating necessary data.
.TP
.B p
-Thread running pre\-reading file(s).
+Thread running pre-reading file(s).
.TP
.B /
Thread is in ramp period.
.B bw
Aggregate bandwidth of threads in this group followed by the
minimum and maximum bandwidth of all the threads in this group.
-Values outside of brackets are power\-of\-2 format and those
-within are the equivalent value in a power\-of\-10 format.
+Values outside of brackets are power-of-2 format and those
+within are the equivalent value in a power-of-10 format.
.TP
.B io
Aggregate I/O performed of all threads in this group. The
100,864ns to complete, and 7529 I/Os required 107,008ns to complete.
.P
Also included with fio is a Python script \fBfio_jsonplus_clat2csv\fR that takes
-json+ output and generates CSV\-formatted latency data suitable for plotting.
+json+ output and generates CSV-formatted latency data suitable for plotting.
.P
The latency durations actually represent the midpoints of latency intervals.
For details refer to `stat.h' in the fio source.
its values in a separate row. Further, when using windowed logging the `block
size' and `offset' entries will always contain 0.
.SH CLIENT / SERVER
-Normally fio is invoked as a stand\-alone application on the machine where the
+Normally fio is invoked as a stand-alone application on the machine where the
I/O workload should be generated. However, the backend and frontend of fio can
be run separately i.e., the fio server can generate an I/O workload on the "Device
Under Test" while being controlled by a client on another machine.
$ fio \-\-client=host.list <job file(s)>
.RE
.P
-In this mode, you cannot input server\-specific parameters or job files \-\- all
+In this mode, you cannot input server-specific parameters or job files \-\- all
servers receive the same job file.
.P
In order to let `fio \-\-client' runs use a shared filesystem from multiple
projects / fio.git / blobdiff