Write output to \fIfilename\fR.
.TP
.BI \-\-output-format \fR=\fPformat
-Set the reporting format to \fInormal\fR, \fIterse\fR, or \fIjson\fR.
+Set the reporting format to \fInormal\fR, \fIterse\fR, \fIjson\fR, or
+\fIjson+\fR. Multiple formats can be selected, separate by a comma. \fIterse\fR
+is a CSV based format. \fIjson+\fR is like \fIjson\fR, except it adds a full
+dump of the latency buckets.
.TP
.BI \-\-runtime \fR=\fPruntime
Limit run time to \fIruntime\fR seconds.
.TP
.B \-\-append-terse
Print statistics in selected mode AND terse, semicolon-delimited format.
+Deprecated, use \-\-output-format instead to select multiple formats.
.TP
.B \-\-version
Display version information and exit.
Background a fio server, writing the pid to the given pid file.
.TP
.BI \-\-client \fR=\fPhost
-Instead of running the jobs locally, send and run them on the given host.
+Instead of running the jobs locally, send and run them on the given host or set of hosts. See client/server section.
.TP
.BI \-\-idle\-prof \fR=\fPoption
Report cpu idleness on a system or percpu basis (\fIoption\fP=system,percpu) or run unit work calibration only (\fIoption\fP=calibrate).
May be used to override the job name. On the command line, this parameter
has the special purpose of signalling the start of a new job.
.TP
+.BI wait_for \fR=\fPstr
+Specifies the name of the already defined job to wait for. Single waitee name
+only may be specified. If set, the job won't be started until all workers of
+the waitee job are done. Wait_for operates on the job name basis, so there are
+a few limitations. First, the waitee must be defined prior to the waiter job
+(meaning no forward references). Second, if a job is being referenced as a
+waitee, it must have a unique name (no duplicate waitees).
+.TP
.BI description \fR=\fPstr
Human-readable description of the job. It is printed when the job is run, but
otherwise has no special purpose.
.B rw, readwrite
Mixed sequential reads and writes.
.TP
-.B randrw
+.B randrw
Mixed random reads and writes.
.TP
.B trimwrite
of IO buffers is defined by the other options related to buffer contents. The
setting can be any pattern of bytes, and can be prefixed with 0x for hex
values. It may also be a string, where the string must then be wrapped with
-"".
+"", e.g.:
+.RS
+.RS
+\fBbuffer_pattern\fR="abcd"
+.RS
+or
+.RE
+\fBbuffer_pattern\fR=-12
+.RS
+or
+.RE
+\fBbuffer_pattern\fR=0xdeadface
+.RE
+.LP
+Also you can combine everything together in any order:
+.LP
+.RS
+\fBbuffer_pattern\fR=0xdeadface"abcd"-12
+.RE
+.RE
.TP
.BI dedupe_percentage \fR=\fPint
If set, fio will generate this percentage of identical buffers when writing.
request to DDIR_WRITE event
.TP
.B rbd
-IO engine supporting direct access to Ceph Rados Block Devices (RBD) via librbd
-without the need to use the kernel rbd driver. This ioengine defines engine specific
+IO engine supporting direct access to Ceph Rados Block Devices (RBD) via librbd
+without the need to use the kernel rbd driver. This ioengine defines engine specific
options.
.TP
.B gfapi
not async on that OS. Keep an eye on the IO depth distribution in the
fio output to verify that the achieved depth is as expected. Default: 1.
.TP
-.BI iodepth_batch \fR=\fPint
-Number of I/Os to submit at once. Default: \fBiodepth\fR.
+.BI iodepth_batch \fR=\fPint "\fR,\fP iodepth_batch_submit" \fR=\fPint
+This defines how many pieces of IO to submit at once. It defaults to 1
+which means that we submit each IO as soon as it is available, but can
+be raised to submit bigger batches of IO at the time. If it is set to 0
+the \fBiodepth\fR value will be used.
.TP
-.BI iodepth_batch_complete \fR=\fPint
+.BI iodepth_batch_complete_min \fR=\fPint "\fR,\fP iodepth_batch_complete" \fR=\fPint
This defines how many pieces of IO to retrieve at once. It defaults to 1 which
means that we'll ask for a minimum of 1 IO in the retrieval process from the
kernel. The IO retrieval will go on until we hit the limit set by
completed events before queuing more IO. This helps reduce IO latency, at the
cost of more retrieval system calls.
.TP
+.BI iodepth_batch_complete_max \fR=\fPint
+This defines maximum pieces of IO to
+retrieve at once. This variable should be used along with
+\fBiodepth_batch_complete_min\fR=int variable, specifying the range
+of min and max amount of IO which should be retrieved. By default
+it is equal to \fBiodepth_batch_complete_min\fR value.
+
+Example #1:
+.RS
+.RS
+\fBiodepth_batch_complete_min\fR=1
+.LP
+\fBiodepth_batch_complete_max\fR=<iodepth>
+.RE
+
+which means that we will retrieve at leat 1 IO and up to the
+whole submitted queue depth. If none of IO has been completed
+yet, we will wait.
+
+Example #2:
+.RS
+\fBiodepth_batch_complete_min\fR=0
+.LP
+\fBiodepth_batch_complete_max\fR=<iodepth>
+.RE
+
+which means that we can retrieve up to the whole submitted
+queue depth, but if none of IO has been completed yet, we will
+NOT wait and immediately exit the system call. In this example
+we simply do polling.
+.RE
+.TP
.BI iodepth_low \fR=\fPint
Low watermark indicating when to start filling the queue again. Default:
-\fBiodepth\fR.
+\fBiodepth\fR.
+.TP
+.BI io_submit_mode \fR=\fPstr
+This option controls how fio submits the IO to the IO engine. The default is
+\fBinline\fR, which means that the fio job threads submit and reap IO directly.
+If set to \fBoffload\fR, the job threads will offload IO submission to a
+dedicated pool of IO threads. This requires some coordination and thus has a
+bit of extra overhead, especially for lower queue depth IO where it can
+increase latencies. The benefit is that fio can manage submission rates
+independently of the device completion rates. This avoids skewed latency
+reporting if IO gets back up on the device side (the coordinated omission
+problem).
.TP
.BI direct \fR=\fPbool
If true, use non-buffered I/O (usually O_DIRECT). Default: false.
.B lfsr
Linear feedback shift register generator
.TP
+.B tausworthe64
+Strong 64-bit 2^258 cycle random number generator
+.TP
.RE
.P
Tausworthe is a strong random number generator, but it requires tracking on the
can be done with \fBrate\fR=,500k or \fBrate\fR=500k,. The former will only
limit writes (to 500KB/sec), the latter will only limit reads.
.TP
-.BI ratemin \fR=\fPint
+.BI rate_min \fR=\fPint
Tell \fBfio\fR to do whatever it can to maintain at least the given bandwidth.
Failing to meet this requirement will cause the job to exit. The same format
as \fBrate\fR is used for read vs write separation.
If this rate of I/O is not met, the job will exit. The same format as \fBrate\fR
is used for read vs write separation.
.TP
-.BI ratecycle \fR=\fPint
-Average bandwidth for \fBrate\fR and \fBratemin\fR over this number of
+.BI rate_process \fR=\fPstr
+This option controls how fio manages rated IO submissions. The default is
+\fBlinear\fR, which submits IO in a linear fashion with fixed delays between
+IOs that gets adjusted based on IO completion rates. If this is set to
+\fBpoisson\fR, fio will submit IO based on a more real world random request
+flow, known as the Poisson process
+(https://en.wikipedia.org/wiki/Poisson_process). The lambda will be
+10^6 / IOPS for the given workload.
+.TP
+.BI rate_cycle \fR=\fPint
+Average bandwidth for \fBrate\fR and \fBrate_min\fR over this number of
milliseconds. Default: 1000ms.
.TP
.BI latency_target \fR=\fPint
.TP
.B mmaphuge
Same as \fBmmap\fR, but use huge files as backing.
+.TP
+.B mmapshared
+Same as \fBmmap\fR, but use a MMAP_SHARED mapping.
.RE
.P
The amount of memory allocated is the maximum allowed \fBblocksize\fR for the
.B exitall
Terminate all jobs when one finishes. Default: wait for each job to finish.
.TP
+.B exitall_on_error \fR=\fPbool
+Terminate all jobs if one job finishes in error. Default: wait for each job
+to finish.
+.TP
.BI bwavgtime \fR=\fPint
Average bandwidth calculations over the given time in milliseconds. Default:
500ms.
laid out or updated on disk, only that will be done. The actual job contents
are not executed.
.TP
+.BI allow_file_create \fR=\fPbool
+If true, fio is permitted to create files as part of its workload. This is
+the default behavior. If this option is false, then fio will error out if the
+files it needs to use don't already exist. Default: true.
+.TP
+.BI allow_mounted_write \fR=\fPbool
+If this isn't set, fio will abort jobs that are destructive (eg that write)
+to what appears to be a mounted device or partition. This should help catch
+creating inadvertently destructive tests, not realizing that the test will
+destroy data on the mounted file system. Default: false.
+.TP
.BI pre_read \fR=\fPbool
If this is given, files will be pre-read into memory before starting the given
IO operation. This will also clear the \fR \fBinvalidate\fR flag, since it is
Default: true.
.TP
.BI verify \fR=\fPstr
-Method of verifying file contents after each iteration of the job. Allowed
-values are:
+Method of verifying file contents after each iteration of the job. Each
+verification method also implies verification of special header, which is
+written to the beginning of each block. This header also includes meta
+information, like offset of the block, block number, timestamp when block
+was written, etc. \fBverify\fR=str can be combined with \fBverify_pattern\fR=str
+option. The allowed values are:
.RS
.RS
.TP
not supported by the system.
.TP
.B meta
-Write extra information about each I/O (timestamp, block number, etc.). The
-block number is verified. See \fBverify_pattern\fR as well.
+This option is deprecated, since now meta information is included in generic
+verification header and meta verification happens by default. For detailed
+information see the description of the \fBverify\fR=str setting. This option
+is kept because of compatibility's sake with old configurations. Do not use it.
+.TP
+.B pattern
+Verify a strict pattern. Normally fio includes a header with some basic
+information and checksumming, but if this option is set, only the
+specific pattern set with \fBverify_pattern\fR is verified.
.TP
.B null
Pretend to verify. Used for testing internals.
fio will fill 1/2/3/4 bytes of the buffer at the time(it can be either a
decimal or a hex number). The verify_pattern if larger than a 32-bit quantity
has to be a hex number that starts with either "0x" or "0X". Use with
-\fBverify\fP=meta.
+\fBverify\fP=str. Also, verify_pattern supports %o format, which means that for
+each block offset will be written and then verifyied back, e.g.:
+.RS
+.RS
+\fBverify_pattern\fR=%o
+.RE
+Or use combination of everything:
+.LP
+.RS
+\fBverify_pattern\fR=0xff%o"abcd"-21
+.RE
+.RE
.TP
.BI verify_fatal \fR=\fPbool
If true, exit the job on the first observed verification failure. Default:
.BI verify_backlog_batch \fR=\fPint
Control how many blocks fio will verify if verify_backlog is set. If not set,
will default to the value of \fBverify_backlog\fR (meaning the entire queue is
-read back and verified). If \fBverify_backlog_batch\fR is less than
-\fBverify_backlog\fR then not all blocks will be verified, if
+read back and verified). If \fBverify_backlog_batch\fR is less than
+\fBverify_backlog\fR then not all blocks will be verified, if
\fBverify_backlog_batch\fR is larger than \fBverify_backlog\fR, some blocks
will be verified more than once.
.TP
of the same reporting group, unless separated by a stonewall.
.TP
.BI numjobs \fR=\fPint
-Number of clones (processes/threads performing the same workload) of this job.
+Number of clones (processes/threads performing the same workload) of this job.
Default: 1.
.TP
.B group_reporting
from. Setting \fBreplay_redirect\fR causes all IOPS to be replayed onto the
single specified device regardless of the device it was recorded from.
.TP
+.BI replay_align \fR=\fPint
+Force alignment of IO offsets and lengths in a trace to this power of 2 value.
+.TP
+.BI replay_scale \fR=\fPint
+Scale sector offsets down by this factor when replaying traces.
+.TP
+.BI per_job_logs \fR=\fPbool
+If set, this generates bw/clat/iops log with per file private filenames. If
+not set, jobs with identical names will share the log filename. Default: true.
+.TP
.BI write_bw_log \fR=\fPstr
If given, write a bandwidth log of the jobs in this job file. Can be used to
store data of the bandwidth of the jobs in their lifetime. The included
fio_generate_plots script uses gnuplot to turn these text files into nice
graphs. See \fBwrite_lat_log\fR for behaviour of given filename. For this
option, the postfix is _bw.x.log, where x is the index of the job (1..N,
-where N is the number of jobs)
+where N is the number of jobs). If \fBper_job_logs\fR is false, then the
+filename will not include the job index.
.TP
.BI write_lat_log \fR=\fPstr
Same as \fBwrite_bw_log\fR, but writes I/O completion latencies. If no
filename is given with this option, the default filename of
"jobname_type.x.log" is used, where x is the index of the job (1..N, where
N is the number of jobs). Even if the filename is given, fio will still
-append the type of log.
+append the type of log. If \fBper_job_logs\fR is false, then the filename will
+not include the job index.
.TP
.BI write_iops_log \fR=\fPstr
Same as \fBwrite_bw_log\fR, but writes IOPS. If no filename is given with this
option, the default filename of "jobname_type.x.log" is used, where x is the
index of the job (1..N, where N is the number of jobs). Even if the filename
-is given, fio will still append the type of log.
+is given, fio will still append the type of log. If \fBper_job_logs\fR is false,
+then the filename will not include the job index.
.TP
.BI log_avg_msec \fR=\fPint
By default, fio will log an entry in the iops, latency, or bw log for every
normally at the end of a run, by decompressing the chunks and storing them
in the specified log file. This feature depends on the availability of zlib.
.TP
+.BI log_compression_cpus \fR=\fPstr
+Define the set of CPUs that are allowed to handle online log compression
+for the IO jobs. This can provide better isolation between performance
+sensitive jobs, and background compression work.
+.TP
.BI log_store_compressed \fR=\fPbool
-If set, and \fBlog\fR_compression is also set, fio will store the log files in
-a compressed format. They can be decompressed with fio, using the
-\fB\-\-inflate-log\fR command line parameter. The files will be stored with a
-\fB\.fz\fR suffix.
+If set, fio will store the log files in a compressed format. They can be
+decompressed with fio, using the \fB\-\-inflate-log\fR command line parameter.
+The files will be stored with a \fB\.fz\fR suffix.
.TP
.BI block_error_percentiles \fR=\fPbool
If set, record errors in trim block-sized units from writes and trims and output
Error may be symbol ('ENOSPC', 'ENOMEM') or an integer.
.br
Example: ignore_error=EAGAIN,ENOSPC:122 .
-.br
-This option will ignore EAGAIN from READ, and ENOSPC and 122(EDQUOT) from WRITE.
+.br
+This option will ignore EAGAIN from READ, and ENOSPC and 122(EDQUOT) from WRITE.
.TP
.BI error_dump \fR=\fPbool
If set dump every error even if it is non fatal, true by default. If disabled
File will be used as a block donor (swap extents between files)
.TP
.BI (e4defrag,inplace) \fR=\fPint
-Configure donor file block allocation strategy
+Configure donor file block allocation strategy
.RS
.BI 0(default) :
Preallocate donor's file on init
.P
Error Info (dependent on continue_on_error, default off):
.RS
-.B total # errors, first error code
+.B total # errors, first error code
.RE
.P
.B text description (if provided in config - appears on newline)
fio \-\-client=server \-\-remote-config /path/to/file.fio
-Then the fio serer will open this local (to the server) job file instead
+Then fio will open this local (to the server) job file instead
of being passed one from the client.
+
+If you have many servers (example: 100 VMs/containers), you can input a pathname
+of a file containing host IPs/names as the parameter value for the \-\-client option.
+For example, here is an example "host.list" file containing 2 hostnames:
+
+host1.your.dns.domain
+.br
+host2.your.dns.domain
+
+The fio command would then be:
+
+fio \-\-client=host.list <job file>
+
+In this mode, you cannot input server-specific parameters or job files, and all
+servers receive the same job file.
+
+In order to enable fio \-\-client runs utilizing a shared filesystem from multiple hosts,
+fio \-\-client now prepends the IP address of the server to the filename. For example,
+if fio is using directory /mnt/nfs/fio and is writing filename fileio.tmp,
+with a \-\-client hostfile
+containing two hostnames h1 and h2 with IP addresses 192.168.10.120 and 192.168.10.121, then
+fio will create two files:
+
+/mnt/nfs/fio/192.168.10.120.fileio.tmp
+.br
+/mnt/nfs/fio/192.168.10.121.fileio.tmp
+
.SH AUTHORS
.B fio
For further documentation see \fBHOWTO\fR and \fBREADME\fR.
.br
Sample jobfiles are available in the \fBexamples\fR directory.
-
projects / fio.git / blobdiff