is a CSV based format. `json+' is like `json', except it adds a full
dump of the latency buckets.
.TP
-.BI \-\-runtime \fR=\fPruntime
-Limit run time to \fIruntime\fR seconds.
-.TP
.BI \-\-bandwidth\-log
Generate aggregate bandwidth logs.
.TP
the value is interpreted in seconds.
.TP
.BI \-\-status\-interval \fR=\fPtime
-Force full status dump every \fItime\fR period passed. When the unit is omitted,
-the value is interpreted in seconds.
+Force a full status dump of cumulative (from job start) values at \fItime\fR
+intervals. This option does *not* provide per-period measurements. So
+values such as bandwidth are running averages. When the time unit is omitted,
+\fItime\fR is interpreted in seconds.
.TP
.BI \-\-section \fR=\fPname
Only run specified section \fIname\fR in job file. Multiple sections can be specified.
.TP
.BI \-\-max\-jobs \fR=\fPnr
Set the maximum number of threads/processes to support to \fInr\fR.
+NOTE: On Linux, it may be necessary to increase the shared-memory limit
+(`/proc/sys/kernel/shmmax') if fio runs into errors while creating jobs.
.TP
.BI \-\-server \fR=\fPargs
Start a backend server, with \fIargs\fR specifying what to listen to.
.RS
.P
.PD 0
-Ki means kilo (K) or 1000
+K means kilo (K) or 1000
.P
-Mi means mega (M) or 1000**2
+M means mega (M) or 1000**2
.P
-Gi means giga (G) or 1000**3
+G means giga (G) or 1000**3
.P
-Ti means tera (T) or 1000**4
+T means tera (T) or 1000**4
.P
-Pi means peta (P) or 1000**5
+P means peta (P) or 1000**5
.PD
.RE
.P
.RS
.P
.PD 0
-K means kibi (Ki) or 1024
+Ki means kibi (Ki) or 1024
.P
-M means mebi (Mi) or 1024**2
+Mi means mebi (Mi) or 1024**2
.P
-G means gibi (Gi) or 1024**3
+Gi means gibi (Gi) or 1024**3
.P
-T means tebi (Ti) or 1024**4
+Ti means tebi (Ti) or 1024**4
.P
-P means pebi (Pi) or 1024**5
+Pi means pebi (Pi) or 1024**5
.PD
.RE
.P
`testfiles.$filenum' is specified, file number 4 for any job will be
named `testfiles.4'. The default of `$jobname.$jobnum.$filenum'
will be used if no other format specifier is given.
+.P
+If you specify a path then the directories will be created up to the main
+directory for the file. So for example if you specify `a/b/c/$jobnum` then the
+directories a/b/c will be created before the file setup part of the job. If you
+specify \fBdirectory\fR then the path will be relative that directory, otherwise
+it is treated as the absolute path.
.RE
.TP
.BI unique_filename \fR=\fPbool
.TP
.BI direct \fR=\fPbool
If value is true, use non\-buffered I/O. This is usually O_DIRECT. Note that
-ZFS on Solaris doesn't support direct I/O. On Windows the synchronous
+OpenBSD and ZFS on Solaris don't support direct I/O. On Windows the synchronous
ioengines don't support direct I/O. Default: false.
.TP
.BI atomic \fR=\fPbool
.TP
.BI offset \fR=\fPint
Start I/O at the provided offset in the file, given as either a fixed size in
-bytes or a percentage. If a percentage is given, the next \fBblockalign\fR\-ed
-offset will be used. Data before the given offset will not be touched. This
+bytes or a percentage. If a percentage is given, the generated offset will be
+aligned to the minimum \fBblocksize\fR or to the value of \fBoffset_align\fR if
+provided. Data before the given offset will not be touched. This
effectively caps the file size at `real_size \- offset'. Can be combined with
\fBsize\fR to constrain the start and end range of the I/O workload.
A percentage can be specified by a number between 1 and 100 followed by '%',
for example, `offset=20%' to specify 20%.
.TP
+.BI offset_align \fR=\fPint
+If set to non-zero value, the byte offset generated by a percentage \fBoffset\fR
+is aligned upwards to this value. Defaults to 0 meaning that a percentage
+offset is aligned to the minimum block size.
+.TP
.BI offset_increment \fR=\fPint
If this is provided, then the real offset becomes `\fBoffset\fR + \fBoffset_increment\fR
* thread_number', where the thread number is a counter that starts at 0 and
.TP
.B zoned
Zoned random distribution
+.B zoned_abs
+Zoned absolute random distribution
.RE
.P
When using a \fBzipf\fR or \fBpareto\fR distribution, an input value is also
random_distribution=zoned:60/10:30/20:8/30:2/40
.RE
.P
-similarly to how \fBbssplit\fR works for setting ranges and percentages
+A \fBzoned_abs\fR distribution works exactly like the\fBzoned\fR, except that
+it takes absolute sizes. For example, let's say you wanted to define access
+according to the following criteria:
+.RS
+.P
+.PD 0
+60% of accesses should be to the first 20G
+.P
+30% of accesses should be to the next 100G
+.P
+10% of accesses should be to the next 500G
+.PD
+.RE
+.P
+we can define an absolute zoning distribution with:
+.RS
+.P
+random_distribution=zoned:60/10:30/20:8/30:2/40
+.RE
+.P
+For both \fBzoned\fR and \fBzoned_abs\fR, fio supports defining up to 256
+separate zones.
+.P
+Similarly to how \fBbssplit\fR works for setting ranges and percentages
of block sizes. Like \fBbssplit\fR, it's possible to specify separate
zones for reads, writes, and trims. If just one set is given, it'll apply to
all of them.
.P
bssplit=2k/50:4k/50,4k/90,8k/10
.RE
+.P
+Fio supports defining up to 64 different weights for each data direction.
.RE
.TP
.BI blocksize_unaligned "\fR,\fB bs_unaligned"
blocks. Default: true.
.TP
.BI buffer_compress_percentage \fR=\fPint
-If this is set, then fio will attempt to provide I/O buffer content (on
-WRITEs) that compresses to the specified level. Fio does this by providing a
-mix of random data and a fixed pattern. The fixed pattern is either zeros,
-or the pattern specified by \fBbuffer_pattern\fR. If the pattern option
-is used, it might skew the compression ratio slightly. Note that this is per
-block size unit, for file/disk wide compression level that matches this
-setting, you'll also want to set \fBrefill_buffers\fR.
+If this is set, then fio will attempt to provide I/O buffer content
+(on WRITEs) that compresses to the specified level. Fio does this by
+providing a mix of random data followed by fixed pattern data. The
+fixed pattern is either zeros, or the pattern specified by
+\fBbuffer_pattern\fR. If the \fBbuffer_pattern\fR option is used, it
+might skew the compression ratio slightly. Setting
+\fBbuffer_compress_percentage\fR to a value other than 100 will also
+enable \fBrefill_buffers\fR in order to reduce the likelihood that
+adjacent blocks are so similar that they over compress when seen
+together. See \fBbuffer_compress_chunk\fR for how to set a finer or
+coarser granularity of the random/fixed data regions. Defaults to unset
+i.e., buffer data will not adhere to any compression level.
.TP
.BI buffer_compress_chunk \fR=\fPint
-See \fBbuffer_compress_percentage\fR. This setting allows fio to manage
-how big the ranges of random data and zeroed data is. Without this set, fio
-will provide \fBbuffer_compress_percentage\fR of blocksize random data,
-followed by the remaining zeroed. With this set to some chunk size smaller
-than the block size, fio can alternate random and zeroed data throughout the
-I/O buffer.
+This setting allows fio to manage how big the random/fixed data region
+is when using \fBbuffer_compress_percentage\fR. When
+\fBbuffer_compress_chunk\fR is set to some non-zero value smaller than the
+block size, fio can repeat the random/fixed region throughout the I/O
+buffer at the specified interval (which particularly useful when
+bigger block sizes are used for a job). When set to 0, fio will use a
+chunk size that matches the block size resulting in a single
+random/fixed region within the I/O buffer. Defaults to 512. When the
+unit is omitted, the value is interpreted in bytes.
.TP
.BI buffer_pattern \fR=\fPstr
If set, fio will fill the I/O buffers with this pattern or with the contents
writing. These buffers will be naturally dedupable. The contents of the
buffers depend on what other buffer compression settings have been set. It's
possible to have the individual buffers either fully compressible, or not at
-all. This option only controls the distribution of unique buffers.
+all \-\- this option only controls the distribution of unique buffers. Setting
+this option will also enable \fBrefill_buffers\fR to prevent every buffer
+being identical.
.TP
.BI invalidate \fR=\fPbool
Invalidate the buffer/page cache parts of the files to be used prior to
.B rdma
The RDMA I/O engine supports both RDMA memory semantics
(RDMA_WRITE/RDMA_READ) and channel semantics (Send/Recv) for the
-InfiniBand, RoCE and iWARP protocols.
+InfiniBand, RoCE and iWARP protocols. This engine defines engine
+specific options.
.TP
.B falloc
I/O engine that does regular fallocate to simulate data transfer as
.B external
Prefix to specify loading an external I/O engine object file. Append
the engine filename, e.g. `ioengine=external:/tmp/foo.o' to load
-ioengine `foo.o' in `/tmp'.
+ioengine `foo.o' in `/tmp'. The path can be either
+absolute or relative. See `engines/skeleton_external.c' in the fio source for
+details of writing an external I/O engine.
+.TP
+.B filecreate
+Simply create the files and do no I/O to them. You still need to set
+\fBfilesize\fR so that all the accounting still occurs, but no actual I/O will be
+done other than creating the file.
+.TP
+.B libpmem
+Read and write using mmap I/O to a file on a filesystem
+mounted with DAX on a persistent memory device through the NVML
+libpmem library.
.SS "I/O engine specific parameters"
In addition, there are some parameters which are only valid when a specific
\fBioengine\fR is in use. These are used identically to normal parameters,
this will be the starting port number since fio will use a range of
ports.
.TP
-.BI (netsplice,net)hostname \fR=\fPstr
-The hostname or IP address to use for TCP or UDP based I/O. If the job is
-a TCP listener or UDP reader, the hostname is not used and must be omitted
-unless it is a valid UDP multicast address.
+.BI (rdma)port
+The port to use for RDMA-CM communication. This should be the same
+value on the client and the server side.
+.TP
+.BI (netsplice,net, rdma)hostname \fR=\fPstr
+The hostname or IP address to use for TCP, UDP or RDMA-CM based I/O.
+If the job is a TCP listener or UDP reader, the hostname is not used
+and must be omitted unless it is a valid UDP multicast address.
.TP
.BI (netsplice,net)interface \fR=\fPstr
The IP address of the network interface used to send or receive UDP
.TP
.BI (libhdfs)chunk_size
The size of the chunk to use for each file.
+.TP
+.BI (rdma)verb \fR=\fPstr
+The RDMA verb to use on this side of the RDMA ioengine
+connection. Valid values are write, read, send and recv. These
+correspond to the equivalent RDMA verbs (e.g. write = rdma_write
+etc.). Note that this only needs to be specified on the client side of
+the connection. See the examples folder.
+.TP
+.BI (rdma)bindname \fR=\fPstr
+The name to use to bind the local RDMA-CM connection to a local RDMA
+device. This could be a hostname or an IPv4 or IPv6 address. On the
+server side this will be passed into the rdma_bind_addr() function and
+on the client site it will be used in the rdma_resolve_add()
+function. This can be useful when multiple paths exist between the
+client and the server or in certain loopback configurations.
.SS "I/O depth"
.TP
.BI iodepth \fR=\fPint
flow, known as the Poisson process
(\fIhttps://en.wikipedia.org/wiki/Poisson_point_process\fR). The lambda will be
10^6 / IOPS for the given workload.
+.TP
+.BI rate_ignore_thinktime \fR=\fPbool
+By default, fio will attempt to catch up to the specified rate setting, if any
+kind of thinktime setting was used. If this option is set, then fio will
+ignore the thinktime and continue doing IO at the specified rate, instead of
+entering a catch-up mode after thinktime is done.
.SS "I/O latency"
.TP
.BI latency_target \fR=\fPtime
.TP
.BI write_bw_log \fR=\fPstr
If given, write a bandwidth log for this job. Can be used to store data of
-the bandwidth of the jobs in their lifetime. The included
-\fBfio_generate_plots\fR script uses gnuplot to turn these
-text files into nice graphs. See \fBwrite_lat_log\fR for behavior of
-given filename. For this option, the postfix is `_bw.x.log', where `x'
-is the index of the job (1..N, where N is the number of jobs). If
-\fBper_job_logs\fR is false, then the filename will not include the job
-index. See \fBLOG FILE FORMATS\fR section.
-.TP
-.BI write_lat_log \fR=\fPstr
-Same as \fBwrite_bw_log\fR, except that this option stores I/O
-submission, completion, and total latencies instead. If no filename is given
-with this option, the default filename of `jobname_type.log' is
-used. Even if the filename is given, fio will still append the type of
-log. So if one specifies:
+the bandwidth of the jobs in their lifetime.
.RS
+.P
+If no str argument is given, the default filename of
+`jobname_type.x.log' is used. Even when the argument is given, fio
+will still append the type of log. So if one specifies:
.RS
.P
-write_lat_log=foo
+write_bw_log=foo
.RE
.P
-The actual log names will be `foo_slat.x.log', `foo_clat.x.log',
-and `foo_lat.x.log', where `x' is the index of the job (1..N, where N
-is the number of jobs). This helps \fBfio_generate_plots\fR find the
-logs automatically. If \fBper_job_logs\fR is false, then the filename
-will not include the job index. See \fBLOG FILE FORMATS\fR section.
+The actual log name will be `foo_bw.x.log' where `x' is the index
+of the job (1..N, where N is the number of jobs). If
+\fBper_job_logs\fR is false, then the filename will not include the
+`.x` job index.
+.P
+The included \fBfio_generate_plots\fR script uses gnuplot to turn these
+text files into nice graphs. See the \fBLOG FILE FORMATS\fR section for how data is
+structured within the file.
.RE
.TP
+.BI write_lat_log \fR=\fPstr
+Same as \fBwrite_bw_log\fR, except this option creates I/O
+submission (e.g., `name_slat.x.log'), completion (e.g.,
+`name_clat.x.log'), and total (e.g., `name_lat.x.log') latency
+files instead. See \fBwrite_bw_log\fR for details about the
+filename format and the \fBLOG FILE FORMATS\fR section for how data is structured
+within the files.
+.TP
.BI write_hist_log \fR=\fPstr
-Same as \fBwrite_lat_log\fR, but writes I/O completion latency
-histograms. If no filename is given with this option, the default filename
-of `jobname_clat_hist.x.log' is used, where `x' is the index of the
-job (1..N, where N is the number of jobs). Even if the filename is given,
-fio will still append the type of log. If \fBper_job_logs\fR is false,
-then the filename will not include the job index. See \fBLOG FILE FORMATS\fR section.
+Same as \fBwrite_bw_log\fR but writes an I/O completion latency
+histogram file (e.g., `name_hist.x.log') instead. Note that this
+file will be empty unless \fBlog_hist_msec\fR has also been set.
+See \fBwrite_bw_log\fR for details about the filename format and
+the \fBLOG FILE FORMATS\fR section for how data is structured
+within the file.
.TP
.BI write_iops_log \fR=\fPstr
-Same as \fBwrite_bw_log\fR, but writes IOPS. If no filename is given
-with this option, the default filename of `jobname_type.x.log' is
-used, where `x' is the index of the job (1..N, where N is the number of
-jobs). Even if the filename is given, fio will still append the type of
-log. If \fBper_job_logs\fR is false, then the filename will not include
-the job index. See \fBLOG FILE FORMATS\fR section.
+Same as \fBwrite_bw_log\fR, but writes an IOPS file (e.g.
+`name_iops.x.log') instead. See \fBwrite_bw_log\fR for
+details about the filename format and the \fBLOG FILE FORMATS\fR section for how data
+is structured within the file.
.TP
.BI log_avg_msec \fR=\fPint
By default, fio will log an entry in the iops, latency, or bw log for every
\fBlog_avg_msec\fR is inaccurate. Setting this option makes fio log
histogram entries over the specified period of time, reducing log sizes for
high IOPS devices while retaining percentile accuracy. See
-\fBlog_hist_coarseness\fR as well. Defaults to 0, meaning histogram
-logging is disabled.
+\fBlog_hist_coarseness\fR and \fBwrite_hist_log\fR as well.
+Defaults to 0, meaning histogram logging is disabled.
.TP
.BI log_hist_coarseness \fR=\fPint
Integer ranging from 0 to 6, defining the coarseness of the resolution of
\fBdisable_lat\fR.
.TP
.BI clat_percentiles \fR=\fPbool
-Enable the reporting of percentiles of completion latencies.
+Enable the reporting of percentiles of completion latencies. This option is
+mutually exclusive with \fBlat_percentiles\fR.
+.TP
+.BI lat_percentiles \fR=\fPbool
+Enable the reporting of percentiles of I/O latencies. This is similar to
+\fBclat_percentiles\fR, except that this includes the submission latency.
+This option is mutually exclusive with \fBclat_percentiles\fR.
.TP
.BI percentile_list \fR=\fPfloat_list
Overwrite the default list of percentiles for completion latencies and the
`\-\-percentile_list=99.5:99.9' will cause fio to report the values of
completion latency below which 99.5% and 99.9% of the observed latencies
fell, respectively.
+.TP
+.BI significant_figures \fR=\fPint
+If using \fB\-\-output\-format\fR of `normal', set the significant figures
+to this value. Higher values will yield more precise IOPS and throughput
+units, while lower values will round. Requires a minimum value of 1 and a
+maximum value of 10. Defaults to 4.
.SS "Error handling"
.TP
.BI exitall_on_error
.B I
Thread initialized, waiting or generating necessary data.
.TP
-.B P
+.B p
Thread running pre\-reading file(s).
.TP
.B /
short or dropped.
.TP
.B IO latency
-These values are for \fBlatency-target\fR and related options. When
+These values are for \fBlatency_target\fR and related options. When
these options are engaged, this section describes the I/O depth required
to meet the specified latency target.
.RE
.nf
terse_version_3;fio_version;jobname;groupid;error;read_kb;read_bandwidth;read_iops;read_runtime_ms;read_slat_min;read_slat_max;read_slat_mean;read_slat_dev;read_clat_min;read_clat_max;read_clat_mean;read_clat_dev;read_clat_pct01;read_clat_pct02;read_clat_pct03;read_clat_pct04;read_clat_pct05;read_clat_pct06;read_clat_pct07;read_clat_pct08;read_clat_pct09;read_clat_pct10;read_clat_pct11;read_clat_pct12;read_clat_pct13;read_clat_pct14;read_clat_pct15;read_clat_pct16;read_clat_pct17;read_clat_pct18;read_clat_pct19;read_clat_pct20;read_tlat_min;read_lat_max;read_lat_mean;read_lat_dev;read_bw_min;read_bw_max;read_bw_agg_pct;read_bw_mean;read_bw_dev;write_kb;write_bandwidth;write_iops;write_runtime_ms;write_slat_min;write_slat_max;write_slat_mean;write_slat_dev;write_clat_min;write_clat_max;write_clat_mean;write_clat_dev;write_clat_pct01;write_clat_pct02;write_clat_pct03;write_clat_pct04;write_clat_pct05;write_clat_pct06;write_clat_pct07;write_clat_pct08;write_clat_pct09;write_clat_pct10;write_clat_pct11;write_clat_pct12;write_clat_pct13;write_clat_pct14;write_clat_pct15;write_clat_pct16;write_clat_pct17;write_clat_pct18;write_clat_pct19;write_clat_pct20;write_tlat_min;write_lat_max;write_lat_mean;write_lat_dev;write_bw_min;write_bw_max;write_bw_agg_pct;write_bw_mean;write_bw_dev;cpu_user;cpu_sys;cpu_csw;cpu_mjf;cpu_minf;iodepth_1;iodepth_2;iodepth_4;iodepth_8;iodepth_16;iodepth_32;iodepth_64;lat_2us;lat_4us;lat_10us;lat_20us;lat_50us;lat_100us;lat_250us;lat_500us;lat_750us;lat_1000us;lat_2ms;lat_4ms;lat_10ms;lat_20ms;lat_50ms;lat_100ms;lat_250ms;lat_500ms;lat_750ms;lat_1000ms;lat_2000ms;lat_over_2000ms;disk_name;disk_read_iops;disk_write_iops;disk_read_merges;disk_write_merges;disk_read_ticks;write_ticks;disk_queue_time;disk_util
.fi
+.SH JSON OUTPUT
+The \fBjson\fR output format is intended to be both human readable and convenient
+for automated parsing. For the most part its sections mirror those of the
+\fBnormal\fR output. The \fBruntime\fR value is reported in msec and the \fBbw\fR value is
+reported in 1024 bytes per second units.
+.fi
.SH JSON+ OUTPUT
The \fBjson+\fR output format is identical to the \fBjson\fR output format except that it
adds a full dump of the completion latency bins. Each \fBbins\fR object contains a
.RS
.TP
.B Latency log
-Value is latency in usecs
+Value is latency in nsecs
.TP
.B Bandwidth log
Value is in KiB/sec
projects / fio.git / blobdiff