is a CSV based format. `json+' is like `json', except it adds a full
dump of the latency buckets.
.TP
-.BI \-\-runtime \fR=\fPruntime
-Limit run time to \fIruntime\fR seconds.
-.TP
.BI \-\-bandwidth\-log
Generate aggregate bandwidth logs.
.TP
the value is interpreted in seconds.
.TP
.BI \-\-status\-interval \fR=\fPtime
-Force full status dump every \fItime\fR period passed. When the unit is omitted,
-the value is interpreted in seconds.
+Force a full status dump of cumulative (from job start) values at \fItime\fR
+intervals. This option does *not* provide per-period measurements. So
+values such as bandwidth are running averages. When the time unit is omitted,
+\fItime\fR is interpreted in seconds.
.TP
.BI \-\-section \fR=\fPname
Only run specified section \fIname\fR in job file. Multiple sections can be specified.
.RS
.P
.PD 0
-Ki means kilo (K) or 1000
+K means kilo (K) or 1000
.P
-Mi means mega (M) or 1000**2
+M means mega (M) or 1000**2
.P
-Gi means giga (G) or 1000**3
+G means giga (G) or 1000**3
.P
-Ti means tera (T) or 1000**4
+T means tera (T) or 1000**4
.P
-Pi means peta (P) or 1000**5
+P means peta (P) or 1000**5
.PD
.RE
.P
.RS
.P
.PD 0
-K means kibi (Ki) or 1024
+Ki means kibi (Ki) or 1024
.P
-M means mebi (Mi) or 1024**2
+Mi means mebi (Mi) or 1024**2
.P
-G means gibi (Gi) or 1024**3
+Gi means gibi (Gi) or 1024**3
.P
-T means tebi (Ti) or 1024**4
+Ti means tebi (Ti) or 1024**4
.P
-P means pebi (Pi) or 1024**5
+Pi means pebi (Pi) or 1024**5
.PD
.RE
.P
.TP
.BI direct \fR=\fPbool
If value is true, use non\-buffered I/O. This is usually O_DIRECT. Note that
-ZFS on Solaris doesn't support direct I/O. On Windows the synchronous
+OpenBSD and ZFS on Solaris don't support direct I/O. On Windows the synchronous
ioengines don't support direct I/O. Default: false.
.TP
.BI atomic \fR=\fPbool
.B external
Prefix to specify loading an external I/O engine object file. Append
the engine filename, e.g. `ioengine=external:/tmp/foo.o' to load
-ioengine `foo.o' in `/tmp'.
+ioengine `foo.o' in `/tmp'. The path can be either
+absolute or relative. See `engines/skeleton_external.c' in the fio source for
+details of writing an external I/O engine.
.SS "I/O engine specific parameters"
In addition, there are some parameters which are only valid when a specific
\fBioengine\fR is in use. These are used identically to normal parameters,
\fBdisable_lat\fR.
.TP
.BI clat_percentiles \fR=\fPbool
-Enable the reporting of percentiles of completion latencies.
+Enable the reporting of percentiles of completion latencies. This option is
+mutually exclusive with \fBlat_percentiles\fR.
+.TP
+.BI lat_percentiles \fR=\fPbool
+Enable the reporting of percentiles of IO latencies. This is similar to
+\fBclat_percentiles\fR, except that this includes the submission latency.
+This option is mutually exclusive with \fBclat_percentiles\fR.
.TP
.BI percentile_list \fR=\fPfloat_list
Overwrite the default list of percentiles for completion latencies and the
| 99.99th=[78119]
bw ( KiB/s): min= 532, max= 686, per=0.10%, avg=622.87, stdev=24.82, samples= 100
iops : min= 76, max= 98, avg=88.98, stdev= 3.54, samples= 100
- lat (usec) : 250=0.04%, 500=64.11%, 750=4.81%, 1000=2.79%
- lat (msec) : 2=4.16%, 4=1.84%, 10=4.90%, 20=11.33%, 50=5.37%
- lat (msec) : 100=0.65%
+ lat (usec) : 250=0.04%, 500=64.11%, 750=4.81%, 1000=2.79%
+ lat (msec) : 2=4.16%, 4=1.84%, 10=4.90%, 20=11.33%, 50=5.37%
+ lat (msec) : 100=0.65%
cpu : usr=0.27%, sys=0.18%, ctx=12072, majf=0, minf=21
IO depths : 1=85.0%, 2=13.1%, 4=1.8%, 8=0.1%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete is basically just CPU time (I/O has already been done, see slat
explanation).
.TP
+.B lat
+Total latency. Same names as slat and clat, this denotes the time from
+when fio created the I/O unit to completion of the I/O operation.
+.TP
.B bw
Bandwidth statistics based on samples. Same names as the xlat stats,
but also includes the number of samples taken (\fIsamples\fR) and an
.B iops
IOPS statistics based on samples. Same names as \fBbw\fR.
.TP
+.B lat (nsec/usec/msec)
+The distribution of I/O completion latencies. This is the time from when
+I/O leaves fio and when it gets completed. Unlike the separate
+read/write/trim sections above, the data here and in the remaining
+sections apply to all I/Os for the reporting group. 250=0.04% means that
+0.04% of the I/Os completed in under 250us. 500=64.11% means that 64.11%
+of the I/Os required 250 to 499us for completion.
+.TP
.B cpu
CPU usage. User and system time, along with the number of context
switches this thread went through, usage of system and user time, and
The number of \fBread/write/trim\fR requests issued, and how many of them were
short or dropped.
.TP
-.B IO latencies
-The distribution of I/O completion latencies. This is the time from when
-I/O leaves fio and when it gets completed. The numbers follow the same
-pattern as the I/O \fBdepths\fR, meaning that 2=1.6% means that 1.6% of the
-I/O completed within 2 msecs, 20=12.8% means that 12.8% of the I/O took
-more than 10 msecs, but less than (or equal to) 20 msecs.
+.B IO latency
+These values are for \fBlatency-target\fR and related options. When
+these options are engaged, this section describes the I/O depth required
+to meet the specified latency target.
.RE
.P
After each client has been listed, the group statistics are printed. They
.nf
terse_version_3;fio_version;jobname;groupid;error;read_kb;read_bandwidth;read_iops;read_runtime_ms;read_slat_min;read_slat_max;read_slat_mean;read_slat_dev;read_clat_min;read_clat_max;read_clat_mean;read_clat_dev;read_clat_pct01;read_clat_pct02;read_clat_pct03;read_clat_pct04;read_clat_pct05;read_clat_pct06;read_clat_pct07;read_clat_pct08;read_clat_pct09;read_clat_pct10;read_clat_pct11;read_clat_pct12;read_clat_pct13;read_clat_pct14;read_clat_pct15;read_clat_pct16;read_clat_pct17;read_clat_pct18;read_clat_pct19;read_clat_pct20;read_tlat_min;read_lat_max;read_lat_mean;read_lat_dev;read_bw_min;read_bw_max;read_bw_agg_pct;read_bw_mean;read_bw_dev;write_kb;write_bandwidth;write_iops;write_runtime_ms;write_slat_min;write_slat_max;write_slat_mean;write_slat_dev;write_clat_min;write_clat_max;write_clat_mean;write_clat_dev;write_clat_pct01;write_clat_pct02;write_clat_pct03;write_clat_pct04;write_clat_pct05;write_clat_pct06;write_clat_pct07;write_clat_pct08;write_clat_pct09;write_clat_pct10;write_clat_pct11;write_clat_pct12;write_clat_pct13;write_clat_pct14;write_clat_pct15;write_clat_pct16;write_clat_pct17;write_clat_pct18;write_clat_pct19;write_clat_pct20;write_tlat_min;write_lat_max;write_lat_mean;write_lat_dev;write_bw_min;write_bw_max;write_bw_agg_pct;write_bw_mean;write_bw_dev;cpu_user;cpu_sys;cpu_csw;cpu_mjf;cpu_minf;iodepth_1;iodepth_2;iodepth_4;iodepth_8;iodepth_16;iodepth_32;iodepth_64;lat_2us;lat_4us;lat_10us;lat_20us;lat_50us;lat_100us;lat_250us;lat_500us;lat_750us;lat_1000us;lat_2ms;lat_4ms;lat_10ms;lat_20ms;lat_50ms;lat_100ms;lat_250ms;lat_500ms;lat_750ms;lat_1000ms;lat_2000ms;lat_over_2000ms;disk_name;disk_read_iops;disk_write_iops;disk_read_merges;disk_write_merges;disk_read_ticks;write_ticks;disk_queue_time;disk_util
.fi
+.SH JSON OUTPUT
+The \fBjson\fR output format is intended to be both human readable and convenient
+for automated parsing. For the most part its sections mirror those of the
+\fBnormal\fR output. The \fBruntime\fR value is reported in msec and the \fBbw\fR value is
+reported in 1024 bytes per second units.
+.fi
.SH JSON+ OUTPUT
The \fBjson+\fR output format is identical to the \fBjson\fR output format except that it
adds a full dump of the completion latency bins. Each \fBbins\fR object contains a
.RS
.TP
.B Latency log
-Value is latency in usecs
+Value is latency in nsecs
.TP
.B Bandwidth log
Value is in KiB/sec