Write output to \fIfilename\fR.
.TP
.BI \-\-output-format \fR=\fPformat
-Set the reporting format to \fInormal\fR, \fIterse\fR, or \fIjosn\fR.
+Set the reporting format to \fInormal\fR, \fIterse\fR, or \fIjson\fR.
.TP
.BI \-\-runtime \fR=\fPruntime
Limit run time to \fIruntime\fR seconds.
Background a fio server, writing the pid to the given pid file.
.TP
.BI \-\-client \fR=\fPhost
-Instead of running the jobs locally, send and run them on the given host.
+Instead of running the jobs locally, send and run them on the given host or set of hosts. See client/server section.
.TP
.BI \-\-idle\-prof \fR=\fPoption
Report cpu idleness on a system or percpu basis (\fIoption\fP=system,percpu) or run unit work calibration only (\fIoption\fP=calibrate).
.TP
.B randrw
Mixed random reads and writes.
+.TP
+.B trimwrite
+Trim and write mixed workload. Blocks will be trimmed first, then the same
+blocks will be written to.
.RE
.P
For mixed I/O, the default split is 50/50. For certain types of io the result
Use \fBposix_fadvise\fR\|(2) to advise the kernel what I/O patterns
are likely to be issued. Default: true.
.TP
+.BI fadvise_stream \fR=\fPint
+Use \fBposix_fadvise\fR\|(2) to advise the kernel what stream ID the
+writes issued belong to. Only supported on Linux. Note, this option
+may change going forward.
+.TP
.BI size \fR=\fPint
Total size of I/O for this job. \fBfio\fR will run until this many bytes have
been transferred, unless limited by other options (\fBruntime\fR, for instance,
.TP
.B zero_buffers
Initialize buffers with all zeros. Default: fill buffers with random data.
-The resulting IO buffers will not be completely zeroed, unless
-\fPscramble_buffers\fR is also turned off.
.TP
.B refill_buffers
If this option is given, fio will refill the IO buffers on every submit. The
example job file to create such files, use rw=write option). Please note, you
might want to set necessary environment variables to work with hdfs/libhdfs
properly.
+.TP
+.B mtd
+Read, write and erase an MTD character device (e.g., /dev/mtd0). Discards are
+treated as erases. Depending on the underlying device type, the I/O may have
+to go in a certain pattern, e.g., on NAND, writing sequentially to erase blocks
+and discarding before overwriting. The writetrim mode works well for this
+constraint.
.RE
.P
.RE
Low watermark indicating when to start filling the queue again. Default:
\fBiodepth\fR.
.TP
+.BI io_submit_mode \fR=\fPstr
+This option controls how fio submits the IO to the IO engine. The default is
+\fBinline\fR, which means that the fio job threads submit and reap IO directly.
+If set to \fBoffload\fR, the job threads will offload IO submission to a
+dedicated pool of IO threads. This requires some coordination and thus has a
+bit of extra overhead, especially for lower queue depth IO where it can
+increase latencies. The benefit is that fio can manage submission rates
+independently of the device completion rates. This avoids skewed latency
+reporting if IO gets back up on the device side (the coordinated omission
+problem).
+.TP
.BI direct \fR=\fPbool
If true, use non-buffered I/O (usually O_DIRECT). Default: false.
.TP
\fB\-\-inflate-log\fR command line parameter. The files will be stored with a
\fB\.fz\fR suffix.
.TP
+.BI block_error_percentiles \fR=\fPbool
+If set, record errors in trim block-sized units from writes and trims and output
+a histogram of how many trims it took to get to errors, and what kind of error
+was encountered.
+.TP
.BI disable_lat \fR=\fPbool
Disable measurements of total latency numbers. Useful only for cutting
back the number of calls to \fBgettimeofday\fR\|(2), as that does impact performance at
Enable the reporting of percentiles of completion latencies.
.TP
.BI percentile_list \fR=\fPfloat_list
-Overwrite the default list of percentiles for completion
-latencies. Each number is a floating number in the range (0,100], and
-the maximum length of the list is 20. Use ':' to separate the
+Overwrite the default list of percentiles for completion latencies and the
+block error histogram. Each number is a floating number in the range (0,100],
+and the maximum length of the list is 20. Use ':' to separate the
numbers. For example, \-\-percentile_list=99.5:99.9 will cause fio to
report the values of completion latency below which 99.5% and 99.9% of
the observed latencies fell, respectively.
.TP
.BI (rbd)clientname \fR=\fPstr
Specifies the username (without the 'client.' prefix) used to access the Ceph cluster.
+.TP
+.BI (mtd)skipbad \fR=\fPbool
+Skip operations against known bad blocks.
.SH OUTPUT
While running, \fBfio\fR will display the status of the created jobs. For
example:
fio \-\-client=server \-\-remote-config /path/to/file.fio
-Then the fio serer will open this local (to the server) job file instead
+Then fio will open this local (to the server) job file instead
of being passed one from the client.
+
+If you have many servers (example: 100 VMs/containers), you can input a pathname
+of a file containing host IPs/names as the parameter value for the \-\-client option.
+For example, here is an example "host.list" file containing 2 hostnames:
+
+host1.your.dns.domain
+.br
+host2.your.dns.domain
+
+The fio command would then be:
+
+fio \-\-client=host.list <job file>
+
+In this mode, you cannot input server-specific parameters or job files, and all
+servers receive the same job file.
+
+In order to enable fio \-\-client runs utilizing a shared filesystem from multiple hosts,
+fio \-\-client now prepends the IP address of the server to the filename. For example,
+if fio is using directory /mnt/nfs/fio and is writing filename fileio.tmp,
+with a \-\-client hostfile
+containing two hostnames h1 and h2 with IP addresses 192.168.10.120 and 192.168.10.121, then
+fio will create two files:
+
+/mnt/nfs/fio/192.168.10.120.fileio.tmp
+.br
+/mnt/nfs/fio/192.168.10.121.fileio.tmp
+
.SH AUTHORS
.B fio