Add exit_on_io_done option to the CPU IO engine
[fio.git] / fio.1
CommitLineData
65f3c785 1.TH fio 1 "October 2013" "User Manual"
d60e92d1
AC
2.SH NAME
3fio \- flexible I/O tester
4.SH SYNOPSIS
5.B fio
6[\fIoptions\fR] [\fIjobfile\fR]...
7.SH DESCRIPTION
8.B fio
9is a tool that will spawn a number of threads or processes doing a
10particular type of I/O action as specified by the user.
11The typical use of fio is to write a job file matching the I/O load
12one wants to simulate.
13.SH OPTIONS
14.TP
49da1240
JA
15.BI \-\-debug \fR=\fPtype
16Enable verbose tracing of various fio actions. May be `all' for all types
17or individual types separated by a comma (eg \-\-debug=io,file). `help' will
18list all available tracing options.
19.TP
d60e92d1
AC
20.BI \-\-output \fR=\fPfilename
21Write output to \fIfilename\fR.
22.TP
b2cecdc2 23.BI \-\-runtime \fR=\fPruntime
24Limit run time to \fIruntime\fR seconds.
d60e92d1
AC
25.TP
26.B \-\-latency\-log
27Generate per-job latency logs.
28.TP
29.B \-\-bandwidth\-log
30Generate per-job bandwidth logs.
31.TP
32.B \-\-minimal
d1429b5c 33Print statistics in a terse, semicolon-delimited format.
d60e92d1 34.TP
2b8c71b0
CE
35.B \-\-append-terse
36Print statistics in selected mode AND terse, semicolon-delimited format.
37.TP
49da1240
JA
38.B \-\-version
39Display version information and exit.
40.TP
065248bf 41.BI \-\-terse\-version \fR=\fPversion
4d658652 42Set terse version output format (Current version 3, or older version 2).
49da1240
JA
43.TP
44.B \-\-help
45Display usage information and exit.
46.TP
fec0f21c
JA
47.B \-\-cpuclock-test
48Perform test and validation of internal CPU clock
49.TP
50.BI \-\-crctest[\fR=\fPtest]
51Test the speed of the builtin checksumming functions. If no argument is given,
52all of them are tested. Or a comma separated list can be passed, in which
53case the given ones are tested.
54.TP
49da1240
JA
55.BI \-\-cmdhelp \fR=\fPcommand
56Print help information for \fIcommand\fR. May be `all' for all commands.
57.TP
de890a1e
SL
58.BI \-\-enghelp \fR=\fPioengine[,command]
59List all commands defined by \fIioengine\fR, or print help for \fIcommand\fR defined by \fIioengine\fR.
60.TP
d60e92d1
AC
61.BI \-\-showcmd \fR=\fPjobfile
62Convert \fIjobfile\fR to a set of command-line options.
63.TP
d60e92d1
AC
64.BI \-\-eta \fR=\fPwhen
65Specifies when real-time ETA estimate should be printed. \fIwhen\fR may
66be one of `always', `never' or `auto'.
67.TP
30b5d57f
JA
68.BI \-\-eta\-newline \fR=\fPtime
69Force an ETA newline for every `time` period passed.
70.TP
71.BI \-\-status\-interval \fR=\fPtime
72Report full output status every `time` period passed.
73.TP
49da1240
JA
74.BI \-\-readonly
75Turn on safety read-only checks, preventing any attempted write.
76.TP
c0a5d35e 77.BI \-\-section \fR=\fPsec
49da1240 78Only run section \fIsec\fR from job file. Multiple of these options can be given, adding more sections to run.
c0a5d35e 79.TP
49da1240
JA
80.BI \-\-alloc\-size \fR=\fPkb
81Set the internal smalloc pool size to \fIkb\fP kilobytes.
d60e92d1 82.TP
49da1240
JA
83.BI \-\-warnings\-fatal
84All fio parser warnings are fatal, causing fio to exit with an error.
9183788d 85.TP
49da1240 86.BI \-\-max\-jobs \fR=\fPnr
57e118a2 87Set the maximum allowed number of jobs (threads/processes) to support.
d60e92d1 88.TP
49da1240
JA
89.BI \-\-server \fR=\fPargs
90Start a backend server, with \fIargs\fP specifying what to listen to. See client/server section.
f57a9c59 91.TP
49da1240
JA
92.BI \-\-daemonize \fR=\fPpidfile
93Background a fio server, writing the pid to the given pid file.
94.TP
95.BI \-\-client \fR=\fPhost
96Instead of running the jobs locally, send and run them on the given host.
f2a2ce0e
HL
97.TP
98.BI \-\-idle\-prof \fR=\fPoption
99Report cpu idleness on a system or percpu basis (\fIoption\fP=system,percpu) or run unit work calibration only (\fIoption\fP=calibrate).
d60e92d1
AC
100.SH "JOB FILE FORMAT"
101Job files are in `ini' format. They consist of one or more
102job definitions, which begin with a job name in square brackets and
103extend to the next job name. The job name can be any ASCII string
104except `global', which has a special meaning. Following the job name is
105a sequence of zero or more parameters, one per line, that define the
106behavior of the job. Any line starting with a `;' or `#' character is
d1429b5c 107considered a comment and ignored.
d9956b64
AC
108.P
109If \fIjobfile\fR is specified as `-', the job file will be read from
110standard input.
d60e92d1
AC
111.SS "Global Section"
112The global section contains default parameters for jobs specified in the
113job file. A job is only affected by global sections residing above it,
114and there may be any number of global sections. Specific job definitions
115may override any parameter set in global sections.
116.SH "JOB PARAMETERS"
117.SS Types
118Some parameters may take arguments of a specific type. The types used are:
119.TP
120.I str
121String: a sequence of alphanumeric characters.
122.TP
123.I int
d60e92d1 124SI integer: a whole number, possibly containing a suffix denoting the base unit
b09da8fa
JA
125of the value. Accepted suffixes are `k', 'M', 'G', 'T', and 'P', denoting
126kilo (1024), mega (1024^2), giga (1024^3), tera (1024^4), and peta (1024^5)
74454ce4
CE
127respectively. If prefixed with '0x', the value is assumed to be base 16
128(hexadecimal). A suffix may include a trailing 'b', for instance 'kb' is
129identical to 'k'. You can specify a base 10 value by using 'KiB', 'MiB','GiB',
130etc. This is useful for disk drives where values are often given in base 10
131values. Specifying '30GiB' will get you 30*1000^3 bytes.
132When specifying times the default suffix meaning changes, still denoting the
133base unit of the value, but accepted suffixes are 'D' (days), 'H' (hours), 'M'
0de5b26f
JA
134(minutes), 'S' Seconds, 'ms' (or msec) milli seconds, 'us' (or 'usec') micro
135seconds. Time values without a unit specify seconds.
74454ce4 136The suffixes are not case sensitive.
d60e92d1
AC
137.TP
138.I bool
139Boolean: a true or false value. `0' denotes false, `1' denotes true.
140.TP
141.I irange
142Integer range: a range of integers specified in the format
d1429b5c
AC
143\fIlower\fR:\fIupper\fR or \fIlower\fR\-\fIupper\fR. \fIlower\fR and
144\fIupper\fR may contain a suffix as described above. If an option allows two
145sets of ranges, they are separated with a `,' or `/' character. For example:
146`8\-8k/8M\-4G'.
83349190
YH
147.TP
148.I float_list
149List of floating numbers: A list of floating numbers, separated by
cecbfd47 150a ':' character.
d60e92d1
AC
151.SS "Parameter List"
152.TP
153.BI name \fR=\fPstr
d9956b64 154May be used to override the job name. On the command line, this parameter
d60e92d1
AC
155has the special purpose of signalling the start of a new job.
156.TP
157.BI description \fR=\fPstr
158Human-readable description of the job. It is printed when the job is run, but
159otherwise has no special purpose.
160.TP
161.BI directory \fR=\fPstr
162Prefix filenames with this directory. Used to place files in a location other
163than `./'.
bcbfeefa
CE
164You can specify a number of directories by separating the names with a ':'
165character. These directories will be assigned equally distributed to job clones
166creates with \fInumjobs\fR as long as they are using generated filenames.
167If specific \fIfilename(s)\fR are set fio will use the first listed directory,
168and thereby matching the \fIfilename\fR semantic which generates a file each
67445b63
JA
169clone if not specified, but let all clones use the same if set. See
170\fIfilename\fR for considerations regarding escaping certain characters on
171some platforms.
d60e92d1
AC
172.TP
173.BI filename \fR=\fPstr
174.B fio
175normally makes up a file name based on the job name, thread number, and file
d1429b5c 176number. If you want to share files between threads in a job or several jobs,
de890a1e
SL
177specify a \fIfilename\fR for each of them to override the default.
178If the I/O engine is file-based, you can specify
d1429b5c
AC
179a number of files by separating the names with a `:' character. `\-' is a
180reserved name, meaning stdin or stdout, depending on the read/write direction
67445b63
JA
181set. On Windows, disk devices are accessed as \\.\PhysicalDrive0 for the first
182device, \\.\PhysicalDrive1 for the second etc. Note: Windows and FreeBSD
183prevent write access to areas of the disk containing in-use data
184(e.g. filesystems). If the wanted filename does need to include a colon, then
185escape that with a '\' character. For instance, if the filename is
186"/dev/dsk/foo@3,0:c", then you would use filename="/dev/dsk/foo@3,0\:c".
d60e92d1 187.TP
de98bd30 188.BI filename_format \fR=\fPstr
ce594fbe 189If sharing multiple files between jobs, it is usually necessary to have
de98bd30
JA
190fio generate the exact names that you want. By default, fio will name a file
191based on the default file format specification of
192\fBjobname.jobnumber.filenumber\fP. With this option, that can be
193customized. Fio will recognize and replace the following keywords in this
194string:
195.RS
196.RS
197.TP
198.B $jobname
199The name of the worker thread or process.
200.TP
201.B $jobnum
202The incremental number of the worker thread or process.
203.TP
204.B $filenum
205The incremental number of the file for that worker thread or process.
206.RE
207.P
208To have dependent jobs share a set of files, this option can be set to
209have fio generate filenames that are shared between the two. For instance,
210if \fBtestfiles.$filenum\fR is specified, file number 4 for any job will
211be named \fBtestfiles.4\fR. The default of \fB$jobname.$jobnum.$filenum\fR
212will be used if no other format specifier is given.
213.RE
214.P
215.TP
3ce9dcaf
JA
216.BI lockfile \fR=\fPstr
217Fio defaults to not locking any files before it does IO to them. If a file or
218file descriptor is shared, fio can serialize IO to that file to make the end
219result consistent. This is usual for emulating real workloads that share files.
220The lock modes are:
221.RS
222.RS
223.TP
224.B none
225No locking. This is the default.
226.TP
227.B exclusive
228Only one thread or process may do IO at the time, excluding all others.
229.TP
230.B readwrite
231Read-write locking on the file. Many readers may access the file at the same
232time, but writes get exclusive access.
233.RE
ce594fbe 234.RE
3ce9dcaf 235.P
d60e92d1
AC
236.BI opendir \fR=\fPstr
237Recursively open any files below directory \fIstr\fR.
238.TP
239.BI readwrite \fR=\fPstr "\fR,\fP rw" \fR=\fPstr
240Type of I/O pattern. Accepted values are:
241.RS
242.RS
243.TP
244.B read
d1429b5c 245Sequential reads.
d60e92d1
AC
246.TP
247.B write
d1429b5c 248Sequential writes.
d60e92d1 249.TP
fa769d44
SW
250.B trim
251Sequential trim (Linux block devices only).
252.TP
d60e92d1 253.B randread
d1429b5c 254Random reads.
d60e92d1
AC
255.TP
256.B randwrite
d1429b5c 257Random writes.
d60e92d1 258.TP
fa769d44
SW
259.B randtrim
260Random trim (Linux block devices only).
261.TP
10b023db 262.B rw, readwrite
d1429b5c 263Mixed sequential reads and writes.
d60e92d1
AC
264.TP
265.B randrw
d1429b5c 266Mixed random reads and writes.
d60e92d1
AC
267.RE
268.P
38dad62d
JA
269For mixed I/O, the default split is 50/50. For certain types of io the result
270may still be skewed a bit, since the speed may be different. It is possible to
3b7fa9ec 271specify a number of IO's to do before getting a new offset, this is done by
38dad62d
JA
272appending a `:\fI<nr>\fR to the end of the string given. For a random read, it
273would look like \fBrw=randread:8\fR for passing in an offset modifier with a
059b0802
JA
274value of 8. If the postfix is used with a sequential IO pattern, then the value
275specified will be added to the generated offset for each IO. For instance,
276using \fBrw=write:4k\fR will skip 4k for every write. It turns sequential IO
277into sequential IO with holes. See the \fBrw_sequencer\fR option.
d60e92d1
AC
278.RE
279.TP
38dad62d
JA
280.BI rw_sequencer \fR=\fPstr
281If an offset modifier is given by appending a number to the \fBrw=<str>\fR line,
282then this option controls how that number modifies the IO offset being
283generated. Accepted values are:
284.RS
285.RS
286.TP
287.B sequential
288Generate sequential offset
289.TP
290.B identical
291Generate the same offset
292.RE
293.P
294\fBsequential\fR is only useful for random IO, where fio would normally
295generate a new random offset for every IO. If you append eg 8 to randread, you
296would get a new random offset for every 8 IO's. The result would be a seek for
297only every 8 IO's, instead of for every IO. Use \fBrw=randread:8\fR to specify
298that. As sequential IO is already sequential, setting \fBsequential\fR for that
299would not result in any differences. \fBidentical\fR behaves in a similar
300fashion, except it sends the same offset 8 number of times before generating a
301new offset.
302.RE
303.P
304.TP
90fef2d1
JA
305.BI kb_base \fR=\fPint
306The base unit for a kilobyte. The defacto base is 2^10, 1024. Storage
307manufacturers like to use 10^3 or 1000 as a base ten unit instead, for obvious
5c9323fb 308reasons. Allowed values are 1024 or 1000, with 1024 being the default.
90fef2d1 309.TP
771e58be
JA
310.BI unified_rw_reporting \fR=\fPbool
311Fio normally reports statistics on a per data direction basis, meaning that
312read, write, and trim are accounted and reported separately. If this option is
313set, the fio will sum the results and report them as "mixed" instead.
314.TP
d60e92d1 315.BI randrepeat \fR=\fPbool
56e2a5fc
CE
316Seed the random number generator used for random I/O patterns in a predictable
317way so the pattern is repeatable across runs. Default: true.
318.TP
319.BI allrandrepeat \fR=\fPbool
320Seed all random number generators in a predictable way so results are
321repeatable across runs. Default: false.
d60e92d1 322.TP
04778baf
JA
323.BI randseed \fR=\fPint
324Seed the random number generators based on this seed value, to be able to
325control what sequence of output is being generated. If not set, the random
326sequence depends on the \fBrandrepeat\fR setting.
327.TP
2615cc4b
JA
328.BI use_os_rand \fR=\fPbool
329Fio can either use the random generator supplied by the OS to generator random
330offsets, or it can use it's own internal generator (based on Tausworthe).
331Default is to use the internal generator, which is often of better quality and
332faster. Default: false.
333.TP
a596f047
EG
334.BI fallocate \fR=\fPstr
335Whether pre-allocation is performed when laying down files. Accepted values
336are:
337.RS
338.RS
339.TP
340.B none
341Do not pre-allocate space.
342.TP
343.B posix
ccc2b328 344Pre-allocate via \fBposix_fallocate\fR\|(3).
a596f047
EG
345.TP
346.B keep
ccc2b328 347Pre-allocate via \fBfallocate\fR\|(2) with FALLOC_FL_KEEP_SIZE set.
a596f047
EG
348.TP
349.B 0
350Backward-compatible alias for 'none'.
351.TP
352.B 1
353Backward-compatible alias for 'posix'.
354.RE
355.P
356May not be available on all supported platforms. 'keep' is only
357available on Linux. If using ZFS on Solaris this must be set to 'none'
358because ZFS doesn't support it. Default: 'posix'.
359.RE
7bc8c2cf 360.TP
d60e92d1 361.BI fadvise_hint \fR=\fPbool
ccc2b328 362Use of \fBposix_fadvise\fR\|(2) to advise the kernel what I/O patterns
d1429b5c 363are likely to be issued. Default: true.
d60e92d1 364.TP
f7fa2653 365.BI size \fR=\fPint
d60e92d1 366Total size of I/O for this job. \fBfio\fR will run until this many bytes have
ca45881f 367been transferred, unless limited by other options (\fBruntime\fR, for instance).
d7c8be03 368Unless \fBnrfiles\fR and \fBfilesize\fR options are given, this amount will be
d6667268 369divided between the available files for the job. If not set, fio will use the
cecbfd47 370full size of the given files or devices. If the files do not exist, size
7bb59102
JA
371must be given. It is also possible to give size as a percentage between 1 and
372100. If size=20% is given, fio will use 20% of the full size of the given files
373or devices.
d60e92d1 374.TP
74586c1e 375.BI fill_device \fR=\fPbool "\fR,\fB fill_fs" \fR=\fPbool
3ce9dcaf
JA
376Sets size to something really large and waits for ENOSPC (no space left on
377device) as the terminating condition. Only makes sense with sequential write.
378For a read workload, the mount point will be filled first then IO started on
4f12432e
JA
379the result. This option doesn't make sense if operating on a raw device node,
380since the size of that is already known by the file system. Additionally,
381writing beyond end-of-device will not return ENOSPC there.
3ce9dcaf 382.TP
d60e92d1
AC
383.BI filesize \fR=\fPirange
384Individual file sizes. May be a range, in which case \fBfio\fR will select sizes
d1429b5c
AC
385for files at random within the given range, limited to \fBsize\fR in total (if
386that is given). If \fBfilesize\fR is not specified, each created file is the
387same size.
d60e92d1 388.TP
bedc9dc2
JA
389.BI file_append \fR=\fPbool
390Perform IO after the end of the file. Normally fio will operate within the
391size of a file. If this option is set, then fio will append to the file
392instead. This has identical behavior to setting \fRoffset\fP to the size
0aae4ce7 393of a file. This option is ignored on non-regular files.
bedc9dc2 394.TP
f7fa2653 395.BI blocksize \fR=\fPint[,int] "\fR,\fB bs" \fR=\fPint[,int]
d9472271
JA
396Block size for I/O units. Default: 4k. Values for reads, writes, and trims
397can be specified separately in the format \fIread\fR,\fIwrite\fR,\fItrim\fR
398either of which may be empty to leave that value at its default. If a trailing
399comma isn't given, the remainder will inherit the last value set.
d60e92d1 400.TP
9183788d 401.BI blocksize_range \fR=\fPirange[,irange] "\fR,\fB bsrange" \fR=\fPirange[,irange]
d1429b5c
AC
402Specify a range of I/O block sizes. The issued I/O unit will always be a
403multiple of the minimum size, unless \fBblocksize_unaligned\fR is set. Applies
9183788d 404to both reads and writes if only one range is given, but can be specified
de8f6de9 405separately with a comma separating the values. Example: bsrange=1k-4k,2k-8k.
9183788d
JA
406Also (see \fBblocksize\fR).
407.TP
408.BI bssplit \fR=\fPstr
409This option allows even finer grained control of the block sizes issued,
410not just even splits between them. With this option, you can weight various
411block sizes for exact control of the issued IO for a job that has mixed
412block sizes. The format of the option is bssplit=blocksize/percentage,
5982a925 413optionally adding as many definitions as needed separated by a colon.
9183788d 414Example: bssplit=4k/10:64k/50:32k/40 would issue 50% 64k blocks, 10% 4k
c83cdd3e
JA
415blocks and 40% 32k blocks. \fBbssplit\fR also supports giving separate
416splits to reads and writes. The format is identical to what the
417\fBbs\fR option accepts, the read and write parts are separated with a
418comma.
d60e92d1
AC
419.TP
420.B blocksize_unaligned\fR,\fP bs_unaligned
d1429b5c
AC
421If set, any size in \fBblocksize_range\fR may be used. This typically won't
422work with direct I/O, as that normally requires sector alignment.
d60e92d1 423.TP
2b7a01d0 424.BI blockalign \fR=\fPint[,int] "\fR,\fB ba" \fR=\fPint[,int]
639ce0f3
MS
425At what boundary to align random IO offsets. Defaults to the same as 'blocksize'
426the minimum blocksize given. Minimum alignment is typically 512b
2b7a01d0
JA
427for using direct IO, though it usually depends on the hardware block size.
428This option is mutually exclusive with using a random map for files, so it
429will turn off that option.
43602667 430.TP
6aca9b3d
JA
431.BI bs_is_seq_rand \fR=\fPbool
432If this option is set, fio will use the normal read,write blocksize settings as
433sequential,random instead. Any random read or write will use the WRITE
434blocksize settings, and any sequential read or write will use the READ
435blocksize setting.
436.TP
d60e92d1
AC
437.B zero_buffers
438Initialise buffers with all zeros. Default: fill buffers with random data.
7750aac4
JA
439The resulting IO buffers will not be completely zeroed, unless
440\fPscramble_buffers\fR is also turned off.
d60e92d1 441.TP
901bb994
JA
442.B refill_buffers
443If this option is given, fio will refill the IO buffers on every submit. The
444default is to only fill it at init time and reuse that data. Only makes sense
445if zero_buffers isn't specified, naturally. If data verification is enabled,
446refill_buffers is also automatically enabled.
447.TP
fd68418e
JA
448.BI scramble_buffers \fR=\fPbool
449If \fBrefill_buffers\fR is too costly and the target is using data
450deduplication, then setting this option will slightly modify the IO buffer
451contents to defeat normal de-dupe attempts. This is not enough to defeat
452more clever block compression attempts, but it will stop naive dedupe
453of blocks. Default: true.
454.TP
c5751c62
JA
455.BI buffer_compress_percentage \fR=\fPint
456If this is set, then fio will attempt to provide IO buffer content (on WRITEs)
457that compress to the specified level. Fio does this by providing a mix of
458random data and zeroes. Note that this is per block size unit, for file/disk
459wide compression level that matches this setting, you'll also want to set
460\fBrefill_buffers\fR.
461.TP
462.BI buffer_compress_chunk \fR=\fPint
463See \fBbuffer_compress_percentage\fR. This setting allows fio to manage how
464big the ranges of random data and zeroed data is. Without this set, fio will
465provide \fBbuffer_compress_percentage\fR of blocksize random data, followed by
466the remaining zeroed. With this set to some chunk size smaller than the block
467size, fio can alternate random and zeroed data throughout the IO buffer.
468.TP
ce35b1ec
JA
469.BI buffer_pattern \fR=\fPstr
470If set, fio will fill the io buffers with this pattern. If not set, the contents
471of io buffers is defined by the other options related to buffer contents. The
472setting can be any pattern of bytes, and can be prefixed with 0x for hex
473values.
474.TP
d60e92d1
AC
475.BI nrfiles \fR=\fPint
476Number of files to use for this job. Default: 1.
477.TP
478.BI openfiles \fR=\fPint
479Number of files to keep open at the same time. Default: \fBnrfiles\fR.
480.TP
481.BI file_service_type \fR=\fPstr
482Defines how files to service are selected. The following types are defined:
483.RS
484.RS
485.TP
486.B random
5c9323fb 487Choose a file at random.
d60e92d1
AC
488.TP
489.B roundrobin
490Round robin over open files (default).
5c9323fb 491.TP
6b7f6851
JA
492.B sequential
493Do each file in the set sequentially.
d60e92d1
AC
494.RE
495.P
496The number of I/Os to issue before switching a new file can be specified by
497appending `:\fIint\fR' to the service type.
498.RE
499.TP
500.BI ioengine \fR=\fPstr
501Defines how the job issues I/O. The following types are defined:
502.RS
503.RS
504.TP
505.B sync
ccc2b328 506Basic \fBread\fR\|(2) or \fBwrite\fR\|(2) I/O. \fBfseek\fR\|(2) is used to
d60e92d1
AC
507position the I/O location.
508.TP
a31041ea 509.B psync
ccc2b328 510Basic \fBpread\fR\|(2) or \fBpwrite\fR\|(2) I/O.
a31041ea 511.TP
9183788d 512.B vsync
ccc2b328 513Basic \fBreadv\fR\|(2) or \fBwritev\fR\|(2) I/O. Will emulate queuing by
cecbfd47 514coalescing adjacent IOs into a single submission.
9183788d 515.TP
a46c5e01 516.B pvsync
ccc2b328 517Basic \fBpreadv\fR\|(2) or \fBpwritev\fR\|(2) I/O.
a46c5e01 518.TP
d60e92d1 519.B libaio
de890a1e 520Linux native asynchronous I/O. This ioengine defines engine specific options.
d60e92d1
AC
521.TP
522.B posixaio
ccc2b328 523POSIX asynchronous I/O using \fBaio_read\fR\|(3) and \fBaio_write\fR\|(3).
03e20d68
BC
524.TP
525.B solarisaio
526Solaris native asynchronous I/O.
527.TP
528.B windowsaio
529Windows native asynchronous I/O.
d60e92d1
AC
530.TP
531.B mmap
ccc2b328
SW
532File is memory mapped with \fBmmap\fR\|(2) and data copied using
533\fBmemcpy\fR\|(3).
d60e92d1
AC
534.TP
535.B splice
ccc2b328 536\fBsplice\fR\|(2) is used to transfer the data and \fBvmsplice\fR\|(2) to
d1429b5c 537transfer data from user-space to the kernel.
d60e92d1
AC
538.TP
539.B syslet-rw
540Use the syslet system calls to make regular read/write asynchronous.
541.TP
542.B sg
543SCSI generic sg v3 I/O. May be either synchronous using the SG_IO ioctl, or if
ccc2b328
SW
544the target is an sg character device, we use \fBread\fR\|(2) and
545\fBwrite\fR\|(2) for asynchronous I/O.
d60e92d1
AC
546.TP
547.B null
548Doesn't transfer any data, just pretends to. Mainly used to exercise \fBfio\fR
549itself and for debugging and testing purposes.
550.TP
551.B net
de890a1e
SL
552Transfer over the network. The protocol to be used can be defined with the
553\fBprotocol\fR parameter. Depending on the protocol, \fBfilename\fR,
554\fBhostname\fR, \fBport\fR, or \fBlisten\fR must be specified.
555This ioengine defines engine specific options.
d60e92d1
AC
556.TP
557.B netsplice
ccc2b328 558Like \fBnet\fR, but uses \fBsplice\fR\|(2) and \fBvmsplice\fR\|(2) to map data
de890a1e 559and send/receive. This ioengine defines engine specific options.
d60e92d1 560.TP
53aec0a4 561.B cpuio
d60e92d1
AC
562Doesn't transfer any data, but burns CPU cycles according to \fBcpuload\fR and
563\fBcpucycles\fR parameters.
564.TP
565.B guasi
566The GUASI I/O engine is the Generic Userspace Asynchronous Syscall Interface
cecbfd47 567approach to asynchronous I/O.
d1429b5c
AC
568.br
569See <http://www.xmailserver.org/guasi\-lib.html>.
d60e92d1 570.TP
21b8aee8 571.B rdma
85286c5c
BVA
572The RDMA I/O engine supports both RDMA memory semantics (RDMA_WRITE/RDMA_READ)
573and channel semantics (Send/Recv) for the InfiniBand, RoCE and iWARP protocols.
21b8aee8 574.TP
d60e92d1
AC
575.B external
576Loads an external I/O engine object file. Append the engine filename as
577`:\fIenginepath\fR'.
d54fce84
DM
578.TP
579.B falloc
cecbfd47 580 IO engine that does regular linux native fallocate call to simulate data
d54fce84
DM
581transfer as fio ioengine
582.br
583 DDIR_READ does fallocate(,mode = FALLOC_FL_KEEP_SIZE,)
584.br
0981fd71 585 DIR_WRITE does fallocate(,mode = 0)
d54fce84
DM
586.br
587 DDIR_TRIM does fallocate(,mode = FALLOC_FL_KEEP_SIZE|FALLOC_FL_PUNCH_HOLE)
588.TP
589.B e4defrag
590IO engine that does regular EXT4_IOC_MOVE_EXT ioctls to simulate defragment activity
591request to DDIR_WRITE event
0d978694
DAG
592.TP
593.B rbd
594IO engine supporting direct access to Ceph Rados Block Devices (RBD) via librbd
595without the need to use the kernel rbd driver. This ioengine defines engine specific
596options.
d60e92d1 597.RE
595e1734 598.P
d60e92d1
AC
599.RE
600.TP
601.BI iodepth \fR=\fPint
8489dae4
SK
602Number of I/O units to keep in flight against the file. Note that increasing
603iodepth beyond 1 will not affect synchronous ioengines (except for small
ee72ca09
JA
604degress when verify_async is in use). Even async engines my impose OS
605restrictions causing the desired depth not to be achieved. This may happen on
606Linux when using libaio and not setting \fBdirect\fR=1, since buffered IO is
607not async on that OS. Keep an eye on the IO depth distribution in the
608fio output to verify that the achieved depth is as expected. Default: 1.
d60e92d1
AC
609.TP
610.BI iodepth_batch \fR=\fPint
611Number of I/Os to submit at once. Default: \fBiodepth\fR.
612.TP
3ce9dcaf
JA
613.BI iodepth_batch_complete \fR=\fPint
614This defines how many pieces of IO to retrieve at once. It defaults to 1 which
615 means that we'll ask for a minimum of 1 IO in the retrieval process from the
616kernel. The IO retrieval will go on until we hit the limit set by
617\fBiodepth_low\fR. If this variable is set to 0, then fio will always check for
618completed events before queuing more IO. This helps reduce IO latency, at the
619cost of more retrieval system calls.
620.TP
d60e92d1
AC
621.BI iodepth_low \fR=\fPint
622Low watermark indicating when to start filling the queue again. Default:
623\fBiodepth\fR.
624.TP
625.BI direct \fR=\fPbool
626If true, use non-buffered I/O (usually O_DIRECT). Default: false.
627.TP
d01612f3
CM
628.BI atomic \fR=\fPbool
629If value is true, attempt to use atomic direct IO. Atomic writes are guaranteed
630to be stable once acknowledged by the operating system. Only Linux supports
631O_ATOMIC right now.
632.TP
d60e92d1
AC
633.BI buffered \fR=\fPbool
634If true, use buffered I/O. This is the opposite of the \fBdirect\fR parameter.
635Default: true.
636.TP
f7fa2653 637.BI offset \fR=\fPint
d60e92d1
AC
638Offset in the file to start I/O. Data before the offset will not be touched.
639.TP
591e9e06
JA
640.BI offset_increment \fR=\fPint
641If this is provided, then the real offset becomes the
642offset + offset_increment * thread_number, where the thread number is a counter
643that starts at 0 and is incremented for each job. This option is useful if
644there are several jobs which are intended to operate on a file in parallel in
645disjoint segments, with even spacing between the starting points.
646.TP
ddf24e42
JA
647.BI number_ios \fR=\fPint
648Fio will normally perform IOs until it has exhausted the size of the region
649set by \fBsize\fR, or if it exhaust the allocated time (or hits an error
650condition). With this setting, the range/size can be set independently of
651the number of IOs to perform. When fio reaches this number, it will exit
652normally and report status.
653.TP
d60e92d1 654.BI fsync \fR=\fPint
d1429b5c
AC
655How many I/Os to perform before issuing an \fBfsync\fR\|(2) of dirty data. If
6560, don't sync. Default: 0.
d60e92d1 657.TP
5f9099ea
JA
658.BI fdatasync \fR=\fPint
659Like \fBfsync\fR, but uses \fBfdatasync\fR\|(2) instead to only sync the
660data parts of the file. Default: 0.
661.TP
fa769d44
SW
662.BI write_barrier \fR=\fPint
663Make every Nth write a barrier write.
664.TP
e76b1da4 665.BI sync_file_range \fR=\fPstr:int
ccc2b328
SW
666Use \fBsync_file_range\fR\|(2) for every \fRval\fP number of write operations. Fio will
667track range of writes that have happened since the last \fBsync_file_range\fR\|(2) call.
e76b1da4
JA
668\fRstr\fP can currently be one or more of:
669.RS
670.TP
671.B wait_before
672SYNC_FILE_RANGE_WAIT_BEFORE
673.TP
674.B write
675SYNC_FILE_RANGE_WRITE
676.TP
677.B wait_after
678SYNC_FILE_RANGE_WRITE
679.TP
680.RE
681.P
682So if you do sync_file_range=wait_before,write:8, fio would use
683\fBSYNC_FILE_RANGE_WAIT_BEFORE | SYNC_FILE_RANGE_WRITE\fP for every 8 writes.
ccc2b328 684Also see the \fBsync_file_range\fR\|(2) man page. This option is Linux specific.
e76b1da4 685.TP
d60e92d1 686.BI overwrite \fR=\fPbool
d1429b5c 687If writing, setup the file first and do overwrites. Default: false.
d60e92d1
AC
688.TP
689.BI end_fsync \fR=\fPbool
dbd11ead 690Sync file contents when a write stage has completed. Default: false.
d60e92d1
AC
691.TP
692.BI fsync_on_close \fR=\fPbool
693If true, sync file contents on close. This differs from \fBend_fsync\fR in that
d1429b5c 694it will happen on every close, not just at the end of the job. Default: false.
d60e92d1 695.TP
d60e92d1
AC
696.BI rwmixread \fR=\fPint
697Percentage of a mixed workload that should be reads. Default: 50.
698.TP
699.BI rwmixwrite \fR=\fPint
d1429b5c 700Percentage of a mixed workload that should be writes. If \fBrwmixread\fR and
c35dd7a6
JA
701\fBrwmixwrite\fR are given and do not sum to 100%, the latter of the two
702overrides the first. This may interfere with a given rate setting, if fio is
703asked to limit reads or writes to a certain rate. If that is the case, then
704the distribution may be skewed. Default: 50.
d60e92d1 705.TP
92d42d69
JA
706.BI random_distribution \fR=\fPstr:float
707By default, fio will use a completely uniform random distribution when asked
708to perform random IO. Sometimes it is useful to skew the distribution in
709specific ways, ensuring that some parts of the data is more hot than others.
710Fio includes the following distribution models:
711.RS
712.TP
713.B random
714Uniform random distribution
715.TP
716.B zipf
717Zipf distribution
718.TP
719.B pareto
720Pareto distribution
721.TP
722.RE
723.P
724When using a zipf or pareto distribution, an input value is also needed to
725define the access pattern. For zipf, this is the zipf theta. For pareto,
726it's the pareto power. Fio includes a test program, genzipf, that can be
727used visualize what the given input values will yield in terms of hit rates.
728If you wanted to use zipf with a theta of 1.2, you would use
729random_distribution=zipf:1.2 as the option. If a non-uniform model is used,
730fio will disable use of the random map.
731.TP
211c9b89
JA
732.BI percentage_random \fR=\fPint
733For a random workload, set how big a percentage should be random. This defaults
734to 100%, in which case the workload is fully random. It can be set from
735anywhere from 0 to 100. Setting it to 0 would make the workload fully
d9472271
JA
736sequential. It is possible to set different values for reads, writes, and
737trim. To do so, simply use a comma separated list. See \fBblocksize\fR.
211c9b89 738.TP
d60e92d1
AC
739.B norandommap
740Normally \fBfio\fR will cover every block of the file when doing random I/O. If
741this parameter is given, a new offset will be chosen without looking at past
742I/O history. This parameter is mutually exclusive with \fBverify\fR.
743.TP
744492c9 744.BI softrandommap \fR=\fPbool
3ce9dcaf
JA
745See \fBnorandommap\fR. If fio runs with the random block map enabled and it
746fails to allocate the map, if this option is set it will continue without a
747random block map. As coverage will not be as complete as with random maps, this
748option is disabled by default.
749.TP
e8b1961d
JA
750.BI random_generator \fR=\fPstr
751Fio supports the following engines for generating IO offsets for random IO:
752.RS
753.TP
754.B tausworthe
755Strong 2^88 cycle random number generator
756.TP
757.B lfsr
758Linear feedback shift register generator
759.TP
760.RE
761.P
762Tausworthe is a strong random number generator, but it requires tracking on the
763side if we want to ensure that blocks are only read or written once. LFSR
764guarantees that we never generate the same offset twice, and it's also less
765computationally expensive. It's not a true random generator, however, though
766for IO purposes it's typically good enough. LFSR only works with single block
767sizes, not with workloads that use multiple block sizes. If used with such a
768workload, fio may read or write some blocks multiple times.
769.TP
d60e92d1 770.BI nice \fR=\fPint
ccc2b328 771Run job with given nice value. See \fBnice\fR\|(2).
d60e92d1
AC
772.TP
773.BI prio \fR=\fPint
774Set I/O priority value of this job between 0 (highest) and 7 (lowest). See
ccc2b328 775\fBionice\fR\|(1).
d60e92d1
AC
776.TP
777.BI prioclass \fR=\fPint
ccc2b328 778Set I/O priority class. See \fBionice\fR\|(1).
d60e92d1
AC
779.TP
780.BI thinktime \fR=\fPint
781Stall job for given number of microseconds between issuing I/Os.
782.TP
783.BI thinktime_spin \fR=\fPint
784Pretend to spend CPU time for given number of microseconds, sleeping the rest
785of the time specified by \fBthinktime\fR. Only valid if \fBthinktime\fR is set.
786.TP
787.BI thinktime_blocks \fR=\fPint
4d01ece6
JA
788Only valid if thinktime is set - control how many blocks to issue, before
789waiting \fBthinktime\fR microseconds. If not set, defaults to 1 which will
790make fio wait \fBthinktime\fR microseconds after every block. This
791effectively makes any queue depth setting redundant, since no more than 1 IO
792will be queued before we have to complete it and do our thinktime. In other
793words, this setting effectively caps the queue depth if the latter is larger.
d60e92d1
AC
794Default: 1.
795.TP
796.BI rate \fR=\fPint
c35dd7a6
JA
797Cap bandwidth used by this job. The number is in bytes/sec, the normal postfix
798rules apply. You can use \fBrate\fR=500k to limit reads and writes to 500k each,
799or you can specify read and writes separately. Using \fBrate\fR=1m,500k would
800limit reads to 1MB/sec and writes to 500KB/sec. Capping only reads or writes
801can be done with \fBrate\fR=,500k or \fBrate\fR=500k,. The former will only
802limit writes (to 500KB/sec), the latter will only limit reads.
d60e92d1
AC
803.TP
804.BI ratemin \fR=\fPint
805Tell \fBfio\fR to do whatever it can to maintain at least the given bandwidth.
c35dd7a6
JA
806Failing to meet this requirement will cause the job to exit. The same format
807as \fBrate\fR is used for read vs write separation.
d60e92d1
AC
808.TP
809.BI rate_iops \fR=\fPint
c35dd7a6
JA
810Cap the bandwidth to this number of IOPS. Basically the same as rate, just
811specified independently of bandwidth. The same format as \fBrate\fR is used for
de8f6de9 812read vs write separation. If \fBblocksize\fR is a range, the smallest block
c35dd7a6 813size is used as the metric.
d60e92d1
AC
814.TP
815.BI rate_iops_min \fR=\fPint
c35dd7a6 816If this rate of I/O is not met, the job will exit. The same format as \fBrate\fR
de8f6de9 817is used for read vs write separation.
d60e92d1
AC
818.TP
819.BI ratecycle \fR=\fPint
820Average bandwidth for \fBrate\fR and \fBratemin\fR over this number of
821milliseconds. Default: 1000ms.
822.TP
3e260a46
JA
823.BI latency_target \fR=\fPint
824If set, fio will attempt to find the max performance point that the given
825workload will run at while maintaining a latency below this target. The
826values is given in microseconds. See \fBlatency_window\fR and
827\fBlatency_percentile\fR.
828.TP
829.BI latency_window \fR=\fPint
830Used with \fBlatency_target\fR to specify the sample window that the job
831is run at varying queue depths to test the performance. The value is given
832in microseconds.
833.TP
834.BI latency_percentile \fR=\fPfloat
835The percentage of IOs that must fall within the criteria specified by
836\fBlatency_target\fR and \fBlatency_window\fR. If not set, this defaults
837to 100.0, meaning that all IOs must be equal or below to the value set
838by \fBlatency_target\fR.
839.TP
15501535
JA
840.BI max_latency \fR=\fPint
841If set, fio will exit the job if it exceeds this maximum latency. It will exit
842with an ETIME error.
843.TP
d60e92d1
AC
844.BI cpumask \fR=\fPint
845Set CPU affinity for this job. \fIint\fR is a bitmask of allowed CPUs the job
846may run on. See \fBsched_setaffinity\fR\|(2).
847.TP
848.BI cpus_allowed \fR=\fPstr
849Same as \fBcpumask\fR, but allows a comma-delimited list of CPU numbers.
850.TP
c2acfbac
JA
851.BI cpus_allowed_policy \fR=\fPstr
852Set the policy of how fio distributes the CPUs specified by \fBcpus_allowed\fR
853or \fBcpumask\fR. Two policies are supported:
854.RS
855.RS
856.TP
857.B shared
858All jobs will share the CPU set specified.
859.TP
860.B split
861Each job will get a unique CPU from the CPU set.
862.RE
863.P
864\fBshared\fR is the default behaviour, if the option isn't specified. If
ada083cd
JA
865\fBsplit\fR is specified, then fio will assign one cpu per job. If not enough
866CPUs are given for the jobs listed, then fio will roundrobin the CPUs in
867the set.
c2acfbac
JA
868.RE
869.P
870.TP
d0b937ed 871.BI numa_cpu_nodes \fR=\fPstr
cecbfd47 872Set this job running on specified NUMA nodes' CPUs. The arguments allow
d0b937ed
YR
873comma delimited list of cpu numbers, A-B ranges, or 'all'.
874.TP
875.BI numa_mem_policy \fR=\fPstr
876Set this job's memory policy and corresponding NUMA nodes. Format of
cecbfd47 877the arguments:
d0b937ed
YR
878.RS
879.TP
880.B <mode>[:<nodelist>]
881.TP
882.B mode
883is one of the following memory policy:
884.TP
885.B default, prefer, bind, interleave, local
886.TP
887.RE
888For \fBdefault\fR and \fBlocal\fR memory policy, no \fBnodelist\fR is
889needed to be specified. For \fBprefer\fR, only one node is
890allowed. For \fBbind\fR and \fBinterleave\fR, \fBnodelist\fR allows
891comma delimited list of numbers, A-B ranges, or 'all'.
892.TP
23ed19b0
CE
893.BI startdelay \fR=\fPirange
894Delay start of job for the specified number of seconds. Supports all time
895suffixes to allow specification of hours, minutes, seconds and
896milliseconds - seconds are the default if a unit is ommited.
897Can be given as a range which causes each thread to choose randomly out of the
898range.
d60e92d1
AC
899.TP
900.BI runtime \fR=\fPint
901Terminate processing after the specified number of seconds.
902.TP
903.B time_based
904If given, run for the specified \fBruntime\fR duration even if the files are
905completely read or written. The same workload will be repeated as many times
906as \fBruntime\fR allows.
907.TP
901bb994
JA
908.BI ramp_time \fR=\fPint
909If set, fio will run the specified workload for this amount of time before
910logging any performance numbers. Useful for letting performance settle before
911logging results, thus minimizing the runtime required for stable results. Note
c35dd7a6
JA
912that the \fBramp_time\fR is considered lead in time for a job, thus it will
913increase the total runtime if a special timeout or runtime is specified.
901bb994 914.TP
d60e92d1
AC
915.BI invalidate \fR=\fPbool
916Invalidate buffer-cache for the file prior to starting I/O. Default: true.
917.TP
918.BI sync \fR=\fPbool
919Use synchronous I/O for buffered writes. For the majority of I/O engines,
d1429b5c 920this means using O_SYNC. Default: false.
d60e92d1
AC
921.TP
922.BI iomem \fR=\fPstr "\fR,\fP mem" \fR=\fPstr
923Allocation method for I/O unit buffer. Allowed values are:
924.RS
925.RS
926.TP
927.B malloc
ccc2b328 928Allocate memory with \fBmalloc\fR\|(3).
d60e92d1
AC
929.TP
930.B shm
ccc2b328 931Use shared memory buffers allocated through \fBshmget\fR\|(2).
d60e92d1
AC
932.TP
933.B shmhuge
934Same as \fBshm\fR, but use huge pages as backing.
935.TP
936.B mmap
ccc2b328 937Use \fBmmap\fR\|(2) for allocation. Uses anonymous memory unless a filename
d60e92d1
AC
938is given after the option in the format `:\fIfile\fR'.
939.TP
940.B mmaphuge
941Same as \fBmmap\fR, but use huge files as backing.
942.RE
943.P
944The amount of memory allocated is the maximum allowed \fBblocksize\fR for the
945job multiplied by \fBiodepth\fR. For \fBshmhuge\fR or \fBmmaphuge\fR to work,
946the system must have free huge pages allocated. \fBmmaphuge\fR also needs to
2e266ba6
JA
947have hugetlbfs mounted, and \fIfile\fR must point there. At least on Linux,
948huge pages must be manually allocated. See \fB/proc/sys/vm/nr_hugehages\fR
949and the documentation for that. Normally you just need to echo an appropriate
950number, eg echoing 8 will ensure that the OS has 8 huge pages ready for
951use.
d60e92d1
AC
952.RE
953.TP
d392365e 954.BI iomem_align \fR=\fPint "\fR,\fP mem_align" \fR=\fPint
cecbfd47 955This indicates the memory alignment of the IO memory buffers. Note that the
d529ee19
JA
956given alignment is applied to the first IO unit buffer, if using \fBiodepth\fR
957the alignment of the following buffers are given by the \fBbs\fR used. In
958other words, if using a \fBbs\fR that is a multiple of the page sized in the
959system, all buffers will be aligned to this value. If using a \fBbs\fR that
960is not page aligned, the alignment of subsequent IO memory buffers is the
961sum of the \fBiomem_align\fR and \fBbs\fR used.
962.TP
f7fa2653 963.BI hugepage\-size \fR=\fPint
d60e92d1 964Defines the size of a huge page. Must be at least equal to the system setting.
b22989b9 965Should be a multiple of 1MB. Default: 4MB.
d60e92d1
AC
966.TP
967.B exitall
968Terminate all jobs when one finishes. Default: wait for each job to finish.
969.TP
970.BI bwavgtime \fR=\fPint
971Average bandwidth calculations over the given time in milliseconds. Default:
972500ms.
973.TP
c8eeb9df
JA
974.BI iopsavgtime \fR=\fPint
975Average IOPS calculations over the given time in milliseconds. Default:
976500ms.
977.TP
d60e92d1 978.BI create_serialize \fR=\fPbool
d1429b5c 979If true, serialize file creation for the jobs. Default: true.
d60e92d1
AC
980.TP
981.BI create_fsync \fR=\fPbool
ccc2b328 982\fBfsync\fR\|(2) data file after creation. Default: true.
d60e92d1 983.TP
6b7f6851
JA
984.BI create_on_open \fR=\fPbool
985If true, the files are not created until they are opened for IO by the job.
986.TP
25460cf6
JA
987.BI create_only \fR=\fPbool
988If true, fio will only run the setup phase of the job. If files need to be
989laid out or updated on disk, only that will be done. The actual job contents
990are not executed.
991.TP
e9f48479
JA
992.BI pre_read \fR=\fPbool
993If this is given, files will be pre-read into memory before starting the given
994IO operation. This will also clear the \fR \fBinvalidate\fR flag, since it is
9c0d2241
JA
995pointless to pre-read and then drop the cache. This will only work for IO
996engines that are seekable, since they allow you to read the same data
997multiple times. Thus it will not work on eg network or splice IO.
e9f48479 998.TP
d60e92d1
AC
999.BI unlink \fR=\fPbool
1000Unlink job files when done. Default: false.
1001.TP
1002.BI loops \fR=\fPint
1003Specifies the number of iterations (runs of the same workload) of this job.
1004Default: 1.
1005.TP
5e4c7118
JA
1006.BI verify_only \fR=\fPbool
1007Do not perform the specified workload, only verify data still matches previous
1008invocation of this workload. This option allows one to check data multiple
1009times at a later date without overwriting it. This option makes sense only for
1010workloads that write data, and does not support workloads with the
1011\fBtime_based\fR option set.
1012.TP
d60e92d1
AC
1013.BI do_verify \fR=\fPbool
1014Run the verify phase after a write phase. Only valid if \fBverify\fR is set.
1015Default: true.
1016.TP
1017.BI verify \fR=\fPstr
1018Method of verifying file contents after each iteration of the job. Allowed
1019values are:
1020.RS
1021.RS
1022.TP
844ea602 1023.B md5 crc16 crc32 crc32c crc32c-intel crc64 crc7 sha256 sha512 sha1 xxhash
0539d758
JA
1024Store appropriate checksum in the header of each block. crc32c-intel is
1025hardware accelerated SSE4.2 driven, falls back to regular crc32c if
1026not supported by the system.
d60e92d1
AC
1027.TP
1028.B meta
1029Write extra information about each I/O (timestamp, block number, etc.). The
996093bb 1030block number is verified. See \fBverify_pattern\fR as well.
d60e92d1
AC
1031.TP
1032.B null
1033Pretend to verify. Used for testing internals.
1034.RE
b892dc08
JA
1035
1036This option can be used for repeated burn-in tests of a system to make sure
1037that the written data is also correctly read back. If the data direction given
1038is a read or random read, fio will assume that it should verify a previously
1039written file. If the data direction includes any form of write, the verify will
1040be of the newly written data.
d60e92d1
AC
1041.RE
1042.TP
5c9323fb 1043.BI verifysort \fR=\fPbool
d60e92d1
AC
1044If true, written verify blocks are sorted if \fBfio\fR deems it to be faster to
1045read them back in a sorted manner. Default: true.
1046.TP
fa769d44
SW
1047.BI verifysort_nr \fR=\fPint
1048Pre-load and sort verify blocks for a read workload.
1049.TP
f7fa2653 1050.BI verify_offset \fR=\fPint
d60e92d1 1051Swap the verification header with data somewhere else in the block before
d1429b5c 1052writing. It is swapped back before verifying.
d60e92d1 1053.TP
f7fa2653 1054.BI verify_interval \fR=\fPint
d60e92d1
AC
1055Write the verification header for this number of bytes, which should divide
1056\fBblocksize\fR. Default: \fBblocksize\fR.
1057.TP
996093bb
JA
1058.BI verify_pattern \fR=\fPstr
1059If set, fio will fill the io buffers with this pattern. Fio defaults to filling
1060with totally random bytes, but sometimes it's interesting to fill with a known
1061pattern for io verification purposes. Depending on the width of the pattern,
1062fio will fill 1/2/3/4 bytes of the buffer at the time(it can be either a
1063decimal or a hex number). The verify_pattern if larger than a 32-bit quantity
1064has to be a hex number that starts with either "0x" or "0X". Use with
1065\fBverify\fP=meta.
1066.TP
d60e92d1
AC
1067.BI verify_fatal \fR=\fPbool
1068If true, exit the job on the first observed verification failure. Default:
1069false.
1070.TP
b463e936
JA
1071.BI verify_dump \fR=\fPbool
1072If set, dump the contents of both the original data block and the data block we
1073read off disk to files. This allows later analysis to inspect just what kind of
ef71e317 1074data corruption occurred. Off by default.
b463e936 1075.TP
e8462bd8
JA
1076.BI verify_async \fR=\fPint
1077Fio will normally verify IO inline from the submitting thread. This option
1078takes an integer describing how many async offload threads to create for IO
1079verification instead, causing fio to offload the duty of verifying IO contents
c85c324c
JA
1080to one or more separate threads. If using this offload option, even sync IO
1081engines can benefit from using an \fBiodepth\fR setting higher than 1, as it
1082allows them to have IO in flight while verifies are running.
e8462bd8
JA
1083.TP
1084.BI verify_async_cpus \fR=\fPstr
1085Tell fio to set the given CPU affinity on the async IO verification threads.
1086See \fBcpus_allowed\fP for the format used.
1087.TP
6f87418f
JA
1088.BI verify_backlog \fR=\fPint
1089Fio will normally verify the written contents of a job that utilizes verify
1090once that job has completed. In other words, everything is written then
1091everything is read back and verified. You may want to verify continually
1092instead for a variety of reasons. Fio stores the meta data associated with an
1093IO block in memory, so for large verify workloads, quite a bit of memory would
092f707f
DN
1094be used up holding this meta data. If this option is enabled, fio will write
1095only N blocks before verifying these blocks.
6f87418f
JA
1096.TP
1097.BI verify_backlog_batch \fR=\fPint
1098Control how many blocks fio will verify if verify_backlog is set. If not set,
1099will default to the value of \fBverify_backlog\fR (meaning the entire queue is
092f707f
DN
1100read back and verified). If \fBverify_backlog_batch\fR is less than
1101\fBverify_backlog\fR then not all blocks will be verified, if
1102\fBverify_backlog_batch\fR is larger than \fBverify_backlog\fR, some blocks
1103will be verified more than once.
6f87418f 1104.TP
fa769d44
SW
1105.BI trim_percentage \fR=\fPint
1106Number of verify blocks to discard/trim.
1107.TP
1108.BI trim_verify_zero \fR=\fPbool
1109Verify that trim/discarded blocks are returned as zeroes.
1110.TP
1111.BI trim_backlog \fR=\fPint
1112Trim after this number of blocks are written.
1113.TP
1114.BI trim_backlog_batch \fR=\fPint
1115Trim this number of IO blocks.
1116.TP
1117.BI experimental_verify \fR=\fPbool
1118Enable experimental verification.
1119.TP
d392365e 1120.B stonewall "\fR,\fP wait_for_previous"
5982a925 1121Wait for preceding jobs in the job file to exit before starting this one.
d60e92d1
AC
1122\fBstonewall\fR implies \fBnew_group\fR.
1123.TP
1124.B new_group
1125Start a new reporting group. If not given, all jobs in a file will be part
1126of the same reporting group, unless separated by a stonewall.
1127.TP
1128.BI numjobs \fR=\fPint
1129Number of clones (processes/threads performing the same workload) of this job.
1130Default: 1.
1131.TP
1132.B group_reporting
1133If set, display per-group reports instead of per-job when \fBnumjobs\fR is
1134specified.
1135.TP
1136.B thread
1137Use threads created with \fBpthread_create\fR\|(3) instead of processes created
1138with \fBfork\fR\|(2).
1139.TP
f7fa2653 1140.BI zonesize \fR=\fPint
d60e92d1
AC
1141Divide file into zones of the specified size in bytes. See \fBzoneskip\fR.
1142.TP
fa769d44
SW
1143.BI zonerange \fR=\fPint
1144Give size of an IO zone. See \fBzoneskip\fR.
1145.TP
f7fa2653 1146.BI zoneskip \fR=\fPint
d1429b5c 1147Skip the specified number of bytes when \fBzonesize\fR bytes of data have been
d60e92d1
AC
1148read.
1149.TP
1150.BI write_iolog \fR=\fPstr
5b42a488
SH
1151Write the issued I/O patterns to the specified file. Specify a separate file
1152for each job, otherwise the iologs will be interspersed and the file may be
1153corrupt.
d60e92d1
AC
1154.TP
1155.BI read_iolog \fR=\fPstr
1156Replay the I/O patterns contained in the specified file generated by
1157\fBwrite_iolog\fR, or may be a \fBblktrace\fR binary file.
1158.TP
64bbb865
DN
1159.BI replay_no_stall \fR=\fPint
1160While replaying I/O patterns using \fBread_iolog\fR the default behavior
1161attempts to respect timing information between I/Os. Enabling
1162\fBreplay_no_stall\fR causes I/Os to be replayed as fast as possible while
1163still respecting ordering.
1164.TP
d1c46c04
DN
1165.BI replay_redirect \fR=\fPstr
1166While replaying I/O patterns using \fBread_iolog\fR the default behavior
1167is to replay the IOPS onto the major/minor device that each IOP was recorded
1168from. Setting \fBreplay_redirect\fR causes all IOPS to be replayed onto the
1169single specified device regardless of the device it was recorded from.
1170.TP
836bad52 1171.BI write_bw_log \fR=\fPstr
901bb994
JA
1172If given, write a bandwidth log of the jobs in this job file. Can be used to
1173store data of the bandwidth of the jobs in their lifetime. The included
1174fio_generate_plots script uses gnuplot to turn these text files into nice
26b26fca 1175graphs. See \fBwrite_lat_log\fR for behaviour of given filename. For this
901bb994 1176option, the postfix is _bw.log.
d60e92d1 1177.TP
836bad52 1178.BI write_lat_log \fR=\fPstr
901bb994
JA
1179Same as \fBwrite_bw_log\fR, but writes I/O completion latencies. If no
1180filename is given with this option, the default filename of "jobname_type.log"
1181is used. Even if the filename is given, fio will still append the type of log.
1182.TP
c8eeb9df
JA
1183.BI write_iops_log \fR=\fPstr
1184Same as \fBwrite_bw_log\fR, but writes IOPS. If no filename is given with this
1185option, the default filename of "jobname_type.log" is used. Even if the
1186filename is given, fio will still append the type of log.
1187.TP
b8bc8cba
JA
1188.BI log_avg_msec \fR=\fPint
1189By default, fio will log an entry in the iops, latency, or bw log for every
1190IO that completes. When writing to the disk log, that can quickly grow to a
1191very large size. Setting this option makes fio average the each log entry
1192over the specified period of time, reducing the resolution of the log.
1193Defaults to 0.
1194.TP
836bad52 1195.BI disable_lat \fR=\fPbool
02af0988 1196Disable measurements of total latency numbers. Useful only for cutting
ccc2b328 1197back the number of calls to \fBgettimeofday\fR\|(2), as that does impact performance at
901bb994
JA
1198really high IOPS rates. Note that to really get rid of a large amount of these
1199calls, this option must be used with disable_slat and disable_bw as well.
1200.TP
836bad52 1201.BI disable_clat \fR=\fPbool
c95f9daf 1202Disable measurements of completion latency numbers. See \fBdisable_lat\fR.
02af0988 1203.TP
836bad52 1204.BI disable_slat \fR=\fPbool
02af0988 1205Disable measurements of submission latency numbers. See \fBdisable_lat\fR.
901bb994 1206.TP
836bad52 1207.BI disable_bw_measurement \fR=\fPbool
02af0988 1208Disable measurements of throughput/bandwidth numbers. See \fBdisable_lat\fR.
d60e92d1 1209.TP
f7fa2653 1210.BI lockmem \fR=\fPint
d60e92d1 1211Pin the specified amount of memory with \fBmlock\fR\|(2). Can be used to
81c6b6cd 1212simulate a smaller amount of memory. The amount specified is per worker.
d60e92d1
AC
1213.TP
1214.BI exec_prerun \fR=\fPstr
1215Before running the job, execute the specified command with \fBsystem\fR\|(3).
ce486495
EV
1216.RS
1217Output is redirected in a file called \fBjobname.prerun.txt\fR
1218.RE
d60e92d1
AC
1219.TP
1220.BI exec_postrun \fR=\fPstr
1221Same as \fBexec_prerun\fR, but the command is executed after the job completes.
ce486495
EV
1222.RS
1223Output is redirected in a file called \fBjobname.postrun.txt\fR
1224.RE
d60e92d1
AC
1225.TP
1226.BI ioscheduler \fR=\fPstr
1227Attempt to switch the device hosting the file to the specified I/O scheduler.
1228.TP
d60e92d1 1229.BI disk_util \fR=\fPbool
d1429b5c 1230Generate disk utilization statistics if the platform supports it. Default: true.
901bb994 1231.TP
23893646
JA
1232.BI clocksource \fR=\fPstr
1233Use the given clocksource as the base of timing. The supported options are:
1234.RS
1235.TP
1236.B gettimeofday
ccc2b328 1237\fBgettimeofday\fR\|(2)
23893646
JA
1238.TP
1239.B clock_gettime
ccc2b328 1240\fBclock_gettime\fR\|(2)
23893646
JA
1241.TP
1242.B cpu
1243Internal CPU clock source
1244.TP
1245.RE
1246.P
1247\fBcpu\fR is the preferred clocksource if it is reliable, as it is very fast
1248(and fio is heavy on time calls). Fio will automatically use this clocksource
1249if it's supported and considered reliable on the system it is running on,
1250unless another clocksource is specifically set. For x86/x86-64 CPUs, this
1251means supporting TSC Invariant.
1252.TP
901bb994 1253.BI gtod_reduce \fR=\fPbool
ccc2b328 1254Enable all of the \fBgettimeofday\fR\|(2) reducing options (disable_clat, disable_slat,
901bb994 1255disable_bw) plus reduce precision of the timeout somewhat to really shrink the
ccc2b328 1256\fBgettimeofday\fR\|(2) call count. With this option enabled, we only do about 0.4% of
901bb994
JA
1257the gtod() calls we would have done if all time keeping was enabled.
1258.TP
1259.BI gtod_cpu \fR=\fPint
1260Sometimes it's cheaper to dedicate a single thread of execution to just getting
1261the current time. Fio (and databases, for instance) are very intensive on
ccc2b328 1262\fBgettimeofday\fR\|(2) calls. With this option, you can set one CPU aside for doing
901bb994
JA
1263nothing but logging current time to a shared memory location. Then the other
1264threads/processes that run IO workloads need only copy that segment, instead of
ccc2b328 1265entering the kernel with a \fBgettimeofday\fR\|(2) call. The CPU set aside for doing
901bb994
JA
1266these time calls will be excluded from other uses. Fio will manually clear it
1267from the CPU mask of other jobs.
f2bba182 1268.TP
8b28bd41
DM
1269.BI ignore_error \fR=\fPstr
1270Sometimes you want to ignore some errors during test in that case you can specify
1271error list for each error type.
1272.br
1273ignore_error=READ_ERR_LIST,WRITE_ERR_LIST,VERIFY_ERR_LIST
1274.br
1275errors for given error type is separated with ':'.
1276Error may be symbol ('ENOSPC', 'ENOMEM') or an integer.
1277.br
1278Example: ignore_error=EAGAIN,ENOSPC:122 .
1279.br
1280This option will ignore EAGAIN from READ, and ENOSPC and 122(EDQUOT) from WRITE.
1281.TP
1282.BI error_dump \fR=\fPbool
1283If set dump every error even if it is non fatal, true by default. If disabled
1284only fatal error will be dumped
1285.TP
fa769d44
SW
1286.BI profile \fR=\fPstr
1287Select a specific builtin performance test.
1288.TP
a696fa2a
JA
1289.BI cgroup \fR=\fPstr
1290Add job to this control group. If it doesn't exist, it will be created.
6adb38a1
JA
1291The system must have a mounted cgroup blkio mount point for this to work. If
1292your system doesn't have it mounted, you can do so with:
1293
5982a925 1294# mount \-t cgroup \-o blkio none /cgroup
a696fa2a
JA
1295.TP
1296.BI cgroup_weight \fR=\fPint
1297Set the weight of the cgroup to this value. See the documentation that comes
1298with the kernel, allowed values are in the range of 100..1000.
e0b0d892 1299.TP
7de87099
VG
1300.BI cgroup_nodelete \fR=\fPbool
1301Normally fio will delete the cgroups it has created after the job completion.
1302To override this behavior and to leave cgroups around after the job completion,
1303set cgroup_nodelete=1. This can be useful if one wants to inspect various
1304cgroup files after job completion. Default: false
1305.TP
e0b0d892
JA
1306.BI uid \fR=\fPint
1307Instead of running as the invoking user, set the user ID to this value before
1308the thread/process does any work.
1309.TP
1310.BI gid \fR=\fPint
1311Set group ID, see \fBuid\fR.
83349190 1312.TP
fa769d44
SW
1313.BI unit_base \fR=\fPint
1314Base unit for reporting. Allowed values are:
1315.RS
1316.TP
1317.B 0
1318Use auto-detection (default).
1319.TP
1320.B 8
1321Byte based.
1322.TP
1323.B 1
1324Bit based.
1325.RE
1326.P
1327.TP
9e684a49
DE
1328.BI flow_id \fR=\fPint
1329The ID of the flow. If not specified, it defaults to being a global flow. See
1330\fBflow\fR.
1331.TP
1332.BI flow \fR=\fPint
1333Weight in token-based flow control. If this value is used, then there is a
1334\fBflow counter\fR which is used to regulate the proportion of activity between
1335two or more jobs. fio attempts to keep this flow counter near zero. The
1336\fBflow\fR parameter stands for how much should be added or subtracted to the
1337flow counter on each iteration of the main I/O loop. That is, if one job has
1338\fBflow=8\fR and another job has \fBflow=-1\fR, then there will be a roughly
13391:8 ratio in how much one runs vs the other.
1340.TP
1341.BI flow_watermark \fR=\fPint
1342The maximum value that the absolute value of the flow counter is allowed to
1343reach before the job must wait for a lower value of the counter.
1344.TP
1345.BI flow_sleep \fR=\fPint
1346The period of time, in microseconds, to wait after the flow watermark has been
1347exceeded before retrying operations
1348.TP
83349190
YH
1349.BI clat_percentiles \fR=\fPbool
1350Enable the reporting of percentiles of completion latencies.
1351.TP
1352.BI percentile_list \fR=\fPfloat_list
1353Overwrite the default list of percentiles for completion
1354latencies. Each number is a floating number in the range (0,100], and
1355the maximum length of the list is 20. Use ':' to separate the
3eb07285 1356numbers. For example, \-\-percentile_list=99.5:99.9 will cause fio to
83349190
YH
1357report the values of completion latency below which 99.5% and 99.9% of
1358the observed latencies fell, respectively.
de890a1e
SL
1359.SS "Ioengine Parameters List"
1360Some parameters are only valid when a specific ioengine is in use. These are
1361used identically to normal parameters, with the caveat that when used on the
1362command line, the must come after the ioengine that defines them is selected.
1363.TP
e4585935
JA
1364.BI (cpu)cpuload \fR=\fPint
1365Attempt to use the specified percentage of CPU cycles.
1366.TP
1367.BI (cpu)cpuchunks \fR=\fPint
1368Split the load into cycles of the given time. In microseconds.
1369.TP
046395d7
JA
1370.BI (cpu)exit_on_io_done \fR=\fPbool
1371Detect when IO threads are done, then exit.
1372.TP
de890a1e
SL
1373.BI (libaio)userspace_reap
1374Normally, with the libaio engine in use, fio will use
1375the io_getevents system call to reap newly returned events.
1376With this flag turned on, the AIO ring will be read directly
1377from user-space to reap events. The reaping mode is only
1378enabled when polling for a minimum of 0 events (eg when
1379iodepth_batch_complete=0).
1380.TP
1381.BI (net,netsplice)hostname \fR=\fPstr
1382The host name or IP address to use for TCP or UDP based IO.
1383If the job is a TCP listener or UDP reader, the hostname is not
b511c9aa 1384used and must be omitted unless it is a valid UDP multicast address.
de890a1e
SL
1385.TP
1386.BI (net,netsplice)port \fR=\fPint
1387The TCP or UDP port to bind to or connect to.
1388.TP
b93b6a2e
SB
1389.BI (net,netsplice)interface \fR=\fPstr
1390The IP address of the network interface used to send or receive UDP multicast
1391packets.
1392.TP
d3a623de
SB
1393.BI (net,netsplice)ttl \fR=\fPint
1394Time-to-live value for outgoing UDP multicast packets. Default: 1
1395.TP
1d360ffb
JA
1396.BI (net,netsplice)nodelay \fR=\fPbool
1397Set TCP_NODELAY on TCP connections.
1398.TP
de890a1e
SL
1399.BI (net,netsplice)protocol \fR=\fPstr "\fR,\fP proto" \fR=\fPstr
1400The network protocol to use. Accepted values are:
1401.RS
1402.RS
1403.TP
1404.B tcp
1405Transmission control protocol
1406.TP
49ccb8c1
JA
1407.B tcpv6
1408Transmission control protocol V6
1409.TP
de890a1e 1410.B udp
f5cc3d0e 1411User datagram protocol
de890a1e 1412.TP
49ccb8c1
JA
1413.B udpv6
1414User datagram protocol V6
1415.TP
de890a1e
SL
1416.B unix
1417UNIX domain socket
1418.RE
1419.P
1420When the protocol is TCP or UDP, the port must also be given,
1421as well as the hostname if the job is a TCP listener or UDP
1422reader. For unix sockets, the normal filename option should be
1423used and the port is invalid.
1424.RE
1425.TP
1426.BI (net,netsplice)listen
1427For TCP network connections, tell fio to listen for incoming
1428connections rather than initiating an outgoing connection. The
1429hostname must be omitted if this option is used.
d54fce84 1430.TP
7aeb1e94 1431.BI (net, pingpong) \fR=\fPbool
cecbfd47 1432Normally a network writer will just continue writing data, and a network reader
7aeb1e94
JA
1433will just consume packages. If pingpong=1 is set, a writer will send its normal
1434payload to the reader, then wait for the reader to send the same payload back.
1435This allows fio to measure network latencies. The submission and completion
1436latencies then measure local time spent sending or receiving, and the
1437completion latency measures how long it took for the other end to receive and
b511c9aa
SB
1438send back. For UDP multicast traffic pingpong=1 should only be set for a single
1439reader when multiple readers are listening to the same address.
7aeb1e94 1440.TP
d54fce84
DM
1441.BI (e4defrag,donorname) \fR=\fPstr
1442File will be used as a block donor (swap extents between files)
1443.TP
1444.BI (e4defrag,inplace) \fR=\fPint
1445Configure donor file block allocation strategy
1446.RS
1447.BI 0(default) :
1448Preallocate donor's file on init
1449.TP
1450.BI 1:
cecbfd47 1451allocate space immediately inside defragment event, and free right after event
d54fce84 1452.RE
0d978694
DAG
1453.TP
1454.BI (rbd)rbdname \fR=\fPstr
1455Specifies the name of the RBD.
1456.TP
1457.BI (rbd)pool \fR=\fPstr
1458Specifies the name of the Ceph pool containing the RBD.
1459.TP
1460.BI (rbd)clientname \fR=\fPstr
1461Specifies the username (without the 'client.' prefix) used to access the Ceph cluster.
d60e92d1 1462.SH OUTPUT
d1429b5c
AC
1463While running, \fBfio\fR will display the status of the created jobs. For
1464example:
d60e92d1 1465.RS
d1429b5c 1466.P
d60e92d1
AC
1467Threads: 1: [_r] [24.8% done] [ 13509/ 8334 kb/s] [eta 00h:01m:31s]
1468.RE
1469.P
d1429b5c
AC
1470The characters in the first set of brackets denote the current status of each
1471threads. The possible values are:
1472.P
1473.PD 0
d60e92d1
AC
1474.RS
1475.TP
1476.B P
1477Setup but not started.
1478.TP
1479.B C
1480Thread created.
1481.TP
1482.B I
1483Initialized, waiting.
1484.TP
1485.B R
1486Running, doing sequential reads.
1487.TP
1488.B r
1489Running, doing random reads.
1490.TP
1491.B W
1492Running, doing sequential writes.
1493.TP
1494.B w
1495Running, doing random writes.
1496.TP
1497.B M
1498Running, doing mixed sequential reads/writes.
1499.TP
1500.B m
1501Running, doing mixed random reads/writes.
1502.TP
1503.B F
1504Running, currently waiting for \fBfsync\fR\|(2).
1505.TP
1506.B V
1507Running, verifying written data.
1508.TP
1509.B E
1510Exited, not reaped by main thread.
1511.TP
1512.B \-
1513Exited, thread reaped.
1514.RE
d1429b5c 1515.PD
d60e92d1
AC
1516.P
1517The second set of brackets shows the estimated completion percentage of
1518the current group. The third set shows the read and write I/O rate,
1519respectively. Finally, the estimated run time of the job is displayed.
1520.P
1521When \fBfio\fR completes (or is interrupted by Ctrl-C), it will show data
1522for each thread, each group of threads, and each disk, in that order.
1523.P
1524Per-thread statistics first show the threads client number, group-id, and
1525error code. The remaining figures are as follows:
1526.RS
d60e92d1
AC
1527.TP
1528.B io
1529Number of megabytes of I/O performed.
1530.TP
1531.B bw
1532Average data rate (bandwidth).
1533.TP
1534.B runt
1535Threads run time.
1536.TP
1537.B slat
1538Submission latency minimum, maximum, average and standard deviation. This is
1539the time it took to submit the I/O.
1540.TP
1541.B clat
1542Completion latency minimum, maximum, average and standard deviation. This
1543is the time between submission and completion.
1544.TP
1545.B bw
1546Bandwidth minimum, maximum, percentage of aggregate bandwidth received, average
1547and standard deviation.
1548.TP
1549.B cpu
1550CPU usage statistics. Includes user and system time, number of context switches
1551this thread went through and number of major and minor page faults.
1552.TP
1553.B IO depths
1554Distribution of I/O depths. Each depth includes everything less than (or equal)
1555to it, but greater than the previous depth.
1556.TP
1557.B IO issued
1558Number of read/write requests issued, and number of short read/write requests.
1559.TP
1560.B IO latencies
1561Distribution of I/O completion latencies. The numbers follow the same pattern
1562as \fBIO depths\fR.
1563.RE
d60e92d1
AC
1564.P
1565The group statistics show:
d1429b5c 1566.PD 0
d60e92d1
AC
1567.RS
1568.TP
1569.B io
1570Number of megabytes I/O performed.
1571.TP
1572.B aggrb
1573Aggregate bandwidth of threads in the group.
1574.TP
1575.B minb
1576Minimum average bandwidth a thread saw.
1577.TP
1578.B maxb
1579Maximum average bandwidth a thread saw.
1580.TP
1581.B mint
d1429b5c 1582Shortest runtime of threads in the group.
d60e92d1
AC
1583.TP
1584.B maxt
1585Longest runtime of threads in the group.
1586.RE
d1429b5c 1587.PD
d60e92d1
AC
1588.P
1589Finally, disk statistics are printed with reads first:
d1429b5c 1590.PD 0
d60e92d1
AC
1591.RS
1592.TP
1593.B ios
1594Number of I/Os performed by all groups.
1595.TP
1596.B merge
1597Number of merges in the I/O scheduler.
1598.TP
1599.B ticks
1600Number of ticks we kept the disk busy.
1601.TP
1602.B io_queue
1603Total time spent in the disk queue.
1604.TP
1605.B util
1606Disk utilization.
1607.RE
d1429b5c 1608.PD
8423bd11
JA
1609.P
1610It is also possible to get fio to dump the current output while it is
1611running, without terminating the job. To do that, send fio the \fBUSR1\fR
1612signal.
d60e92d1 1613.SH TERSE OUTPUT
2b8c71b0
CE
1614If the \fB\-\-minimal\fR / \fB\-\-append-terse\fR options are given, the
1615results will be printed/appended in a semicolon-delimited format suitable for
1616scripted use.
1617A job description (if provided) follows on a new line. Note that the first
525c2bfa
JA
1618number in the line is the version number. If the output has to be changed
1619for some reason, this number will be incremented by 1 to signify that
1620change. The fields are:
d60e92d1
AC
1621.P
1622.RS
5e726d0a 1623.B terse version, fio version, jobname, groupid, error
d60e92d1
AC
1624.P
1625Read status:
1626.RS
312b4af2 1627.B Total I/O \fR(KB)\fP, bandwidth \fR(KB/s)\fP, IOPS, runtime \fR(ms)\fP
d60e92d1
AC
1628.P
1629Submission latency:
1630.RS
1631.B min, max, mean, standard deviation
1632.RE
1633Completion latency:
1634.RS
1635.B min, max, mean, standard deviation
1636.RE
1db92cb6
JA
1637Completion latency percentiles (20 fields):
1638.RS
1639.B Xth percentile=usec
1640.RE
525c2bfa
JA
1641Total latency:
1642.RS
1643.B min, max, mean, standard deviation
1644.RE
d60e92d1
AC
1645Bandwidth:
1646.RS
1647.B min, max, aggregate percentage of total, mean, standard deviation
1648.RE
1649.RE
1650.P
1651Write status:
1652.RS
312b4af2 1653.B Total I/O \fR(KB)\fP, bandwidth \fR(KB/s)\fP, IOPS, runtime \fR(ms)\fP
d60e92d1
AC
1654.P
1655Submission latency:
1656.RS
1657.B min, max, mean, standard deviation
1658.RE
1659Completion latency:
1660.RS
1661.B min, max, mean, standard deviation
1662.RE
1db92cb6
JA
1663Completion latency percentiles (20 fields):
1664.RS
1665.B Xth percentile=usec
1666.RE
525c2bfa
JA
1667Total latency:
1668.RS
1669.B min, max, mean, standard deviation
1670.RE
d60e92d1
AC
1671Bandwidth:
1672.RS
1673.B min, max, aggregate percentage of total, mean, standard deviation
1674.RE
1675.RE
1676.P
d1429b5c 1677CPU usage:
d60e92d1 1678.RS
bd2626f0 1679.B user, system, context switches, major page faults, minor page faults
d60e92d1
AC
1680.RE
1681.P
1682IO depth distribution:
1683.RS
1684.B <=1, 2, 4, 8, 16, 32, >=64
1685.RE
1686.P
562c2d2f 1687IO latency distribution:
d60e92d1 1688.RS
562c2d2f
DN
1689Microseconds:
1690.RS
1691.B <=2, 4, 10, 20, 50, 100, 250, 500, 750, 1000
1692.RE
1693Milliseconds:
1694.RS
1695.B <=2, 4, 10, 20, 50, 100, 250, 500, 750, 1000, 2000, >=2000
1696.RE
1697.RE
1698.P
f2f788dd
JA
1699Disk utilization (1 for each disk used):
1700.RS
1701.B name, read ios, write ios, read merges, write merges, read ticks, write ticks, read in-queue time, write in-queue time, disk utilization percentage
1702.RE
1703.P
5982a925 1704Error Info (dependent on continue_on_error, default off):
562c2d2f
DN
1705.RS
1706.B total # errors, first error code
d60e92d1
AC
1707.RE
1708.P
562c2d2f 1709.B text description (if provided in config - appears on newline)
d60e92d1 1710.RE
49da1240
JA
1711.SH CLIENT / SERVER
1712Normally you would run fio as a stand-alone application on the machine
1713where the IO workload should be generated. However, it is also possible to
1714run the frontend and backend of fio separately. This makes it possible to
1715have a fio server running on the machine(s) where the IO workload should
1716be running, while controlling it from another machine.
1717
1718To start the server, you would do:
1719
1720\fBfio \-\-server=args\fR
1721
1722on that machine, where args defines what fio listens to. The arguments
811826be 1723are of the form 'type:hostname or IP:port'. 'type' is either 'ip' (or ip4)
20c67f10
MS
1724for TCP/IP v4, 'ip6' for TCP/IP v6, or 'sock' for a local unix domain
1725socket. 'hostname' is either a hostname or IP address, and 'port' is the port to
811826be 1726listen to (only valid for TCP/IP, not a local socket). Some examples:
49da1240 1727
e01e9745 17281) fio \-\-server
49da1240
JA
1729
1730 Start a fio server, listening on all interfaces on the default port (8765).
1731
e01e9745 17322) fio \-\-server=ip:hostname,4444
49da1240
JA
1733
1734 Start a fio server, listening on IP belonging to hostname and on port 4444.
1735
e01e9745 17363) fio \-\-server=ip6:::1,4444
811826be
JA
1737
1738 Start a fio server, listening on IPv6 localhost ::1 and on port 4444.
1739
e01e9745 17404) fio \-\-server=,4444
49da1240
JA
1741
1742 Start a fio server, listening on all interfaces on port 4444.
1743
e01e9745 17445) fio \-\-server=1.2.3.4
49da1240
JA
1745
1746 Start a fio server, listening on IP 1.2.3.4 on the default port.
1747
e01e9745 17486) fio \-\-server=sock:/tmp/fio.sock
49da1240
JA
1749
1750 Start a fio server, listening on the local socket /tmp/fio.sock.
1751
1752When a server is running, you can connect to it from a client. The client
1753is run with:
1754
e01e9745 1755fio \-\-local-args \-\-client=server \-\-remote-args <job file(s)>
49da1240 1756
e01e9745
MS
1757where \-\-local-args are arguments that are local to the client where it is
1758running, 'server' is the connect string, and \-\-remote-args and <job file(s)>
49da1240
JA
1759are sent to the server. The 'server' string follows the same format as it
1760does on the server side, to allow IP/hostname/socket and port strings.
1761You can connect to multiple clients as well, to do that you could run:
1762
e01e9745 1763fio \-\-client=server2 \-\-client=server2 <job file(s)>
d60e92d1 1764.SH AUTHORS
49da1240 1765
d60e92d1 1766.B fio
aa58d252
JA
1767was written by Jens Axboe <jens.axboe@oracle.com>,
1768now Jens Axboe <jaxboe@fusionio.com>.
d1429b5c
AC
1769.br
1770This man page was written by Aaron Carroll <aaronc@cse.unsw.edu.au> based
d60e92d1
AC
1771on documentation by Jens Axboe.
1772.SH "REPORTING BUGS"
482900c9 1773Report bugs to the \fBfio\fR mailing list <fio@vger.kernel.org>.
d1429b5c 1774See \fBREADME\fR.
d60e92d1 1775.SH "SEE ALSO"
d1429b5c
AC
1776For further documentation see \fBHOWTO\fR and \fBREADME\fR.
1777.br
1778Sample jobfiles are available in the \fBexamples\fR directory.
d60e92d1 1779