fio: allow to combine terse output with any selected output type
[fio.git] / fio.1
CommitLineData
65f3c785 1.TH fio 1 "October 2013" "User Manual"
d60e92d1
AC
2.SH NAME
3fio \- flexible I/O tester
4.SH SYNOPSIS
5.B fio
6[\fIoptions\fR] [\fIjobfile\fR]...
7.SH DESCRIPTION
8.B fio
9is a tool that will spawn a number of threads or processes doing a
10particular type of I/O action as specified by the user.
11The typical use of fio is to write a job file matching the I/O load
12one wants to simulate.
13.SH OPTIONS
14.TP
49da1240
JA
15.BI \-\-debug \fR=\fPtype
16Enable verbose tracing of various fio actions. May be `all' for all types
17or individual types separated by a comma (eg \-\-debug=io,file). `help' will
18list all available tracing options.
19.TP
d60e92d1
AC
20.BI \-\-output \fR=\fPfilename
21Write output to \fIfilename\fR.
22.TP
b2cecdc2 23.BI \-\-runtime \fR=\fPruntime
24Limit run time to \fIruntime\fR seconds.
d60e92d1
AC
25.TP
26.B \-\-latency\-log
27Generate per-job latency logs.
28.TP
29.B \-\-bandwidth\-log
30Generate per-job bandwidth logs.
31.TP
32.B \-\-minimal
d1429b5c 33Print statistics in a terse, semicolon-delimited format.
d60e92d1 34.TP
2b8c71b0
CE
35.B \-\-append-terse
36Print statistics in selected mode AND terse, semicolon-delimited format.
37.TP
49da1240
JA
38.B \-\-version
39Display version information and exit.
40.TP
065248bf 41.BI \-\-terse\-version \fR=\fPversion
4d658652 42Set terse version output format (Current version 3, or older version 2).
49da1240
JA
43.TP
44.B \-\-help
45Display usage information and exit.
46.TP
fec0f21c
JA
47.B \-\-cpuclock-test
48Perform test and validation of internal CPU clock
49.TP
50.BI \-\-crctest[\fR=\fPtest]
51Test the speed of the builtin checksumming functions. If no argument is given,
52all of them are tested. Or a comma separated list can be passed, in which
53case the given ones are tested.
54.TP
49da1240
JA
55.BI \-\-cmdhelp \fR=\fPcommand
56Print help information for \fIcommand\fR. May be `all' for all commands.
57.TP
de890a1e
SL
58.BI \-\-enghelp \fR=\fPioengine[,command]
59List all commands defined by \fIioengine\fR, or print help for \fIcommand\fR defined by \fIioengine\fR.
60.TP
d60e92d1
AC
61.BI \-\-showcmd \fR=\fPjobfile
62Convert \fIjobfile\fR to a set of command-line options.
63.TP
d60e92d1
AC
64.BI \-\-eta \fR=\fPwhen
65Specifies when real-time ETA estimate should be printed. \fIwhen\fR may
66be one of `always', `never' or `auto'.
67.TP
30b5d57f
JA
68.BI \-\-eta\-newline \fR=\fPtime
69Force an ETA newline for every `time` period passed.
70.TP
71.BI \-\-status\-interval \fR=\fPtime
72Report full output status every `time` period passed.
73.TP
49da1240
JA
74.BI \-\-readonly
75Turn on safety read-only checks, preventing any attempted write.
76.TP
c0a5d35e 77.BI \-\-section \fR=\fPsec
49da1240 78Only run section \fIsec\fR from job file. Multiple of these options can be given, adding more sections to run.
c0a5d35e 79.TP
49da1240
JA
80.BI \-\-alloc\-size \fR=\fPkb
81Set the internal smalloc pool size to \fIkb\fP kilobytes.
d60e92d1 82.TP
49da1240
JA
83.BI \-\-warnings\-fatal
84All fio parser warnings are fatal, causing fio to exit with an error.
9183788d 85.TP
49da1240 86.BI \-\-max\-jobs \fR=\fPnr
57e118a2 87Set the maximum allowed number of jobs (threads/processes) to support.
d60e92d1 88.TP
49da1240
JA
89.BI \-\-server \fR=\fPargs
90Start a backend server, with \fIargs\fP specifying what to listen to. See client/server section.
f57a9c59 91.TP
49da1240
JA
92.BI \-\-daemonize \fR=\fPpidfile
93Background a fio server, writing the pid to the given pid file.
94.TP
95.BI \-\-client \fR=\fPhost
96Instead of running the jobs locally, send and run them on the given host.
f2a2ce0e
HL
97.TP
98.BI \-\-idle\-prof \fR=\fPoption
99Report cpu idleness on a system or percpu basis (\fIoption\fP=system,percpu) or run unit work calibration only (\fIoption\fP=calibrate).
d60e92d1
AC
100.SH "JOB FILE FORMAT"
101Job files are in `ini' format. They consist of one or more
102job definitions, which begin with a job name in square brackets and
103extend to the next job name. The job name can be any ASCII string
104except `global', which has a special meaning. Following the job name is
105a sequence of zero or more parameters, one per line, that define the
106behavior of the job. Any line starting with a `;' or `#' character is
d1429b5c 107considered a comment and ignored.
d9956b64
AC
108.P
109If \fIjobfile\fR is specified as `-', the job file will be read from
110standard input.
d60e92d1
AC
111.SS "Global Section"
112The global section contains default parameters for jobs specified in the
113job file. A job is only affected by global sections residing above it,
114and there may be any number of global sections. Specific job definitions
115may override any parameter set in global sections.
116.SH "JOB PARAMETERS"
117.SS Types
118Some parameters may take arguments of a specific type. The types used are:
119.TP
120.I str
121String: a sequence of alphanumeric characters.
122.TP
123.I int
d60e92d1 124SI integer: a whole number, possibly containing a suffix denoting the base unit
b09da8fa
JA
125of the value. Accepted suffixes are `k', 'M', 'G', 'T', and 'P', denoting
126kilo (1024), mega (1024^2), giga (1024^3), tera (1024^4), and peta (1024^5)
74454ce4
CE
127respectively. If prefixed with '0x', the value is assumed to be base 16
128(hexadecimal). A suffix may include a trailing 'b', for instance 'kb' is
129identical to 'k'. You can specify a base 10 value by using 'KiB', 'MiB','GiB',
130etc. This is useful for disk drives where values are often given in base 10
131values. Specifying '30GiB' will get you 30*1000^3 bytes.
132When specifying times the default suffix meaning changes, still denoting the
133base unit of the value, but accepted suffixes are 'D' (days), 'H' (hours), 'M'
134(minutes), 'S' Seconds, 'ms' milli seconds. Time values without a unit specify
135seconds.
136The suffixes are not case sensitive.
d60e92d1
AC
137.TP
138.I bool
139Boolean: a true or false value. `0' denotes false, `1' denotes true.
140.TP
141.I irange
142Integer range: a range of integers specified in the format
d1429b5c
AC
143\fIlower\fR:\fIupper\fR or \fIlower\fR\-\fIupper\fR. \fIlower\fR and
144\fIupper\fR may contain a suffix as described above. If an option allows two
145sets of ranges, they are separated with a `,' or `/' character. For example:
146`8\-8k/8M\-4G'.
83349190
YH
147.TP
148.I float_list
149List of floating numbers: A list of floating numbers, separated by
cecbfd47 150a ':' character.
d60e92d1
AC
151.SS "Parameter List"
152.TP
153.BI name \fR=\fPstr
d9956b64 154May be used to override the job name. On the command line, this parameter
d60e92d1
AC
155has the special purpose of signalling the start of a new job.
156.TP
157.BI description \fR=\fPstr
158Human-readable description of the job. It is printed when the job is run, but
159otherwise has no special purpose.
160.TP
161.BI directory \fR=\fPstr
162Prefix filenames with this directory. Used to place files in a location other
163than `./'.
bcbfeefa
CE
164You can specify a number of directories by separating the names with a ':'
165character. These directories will be assigned equally distributed to job clones
166creates with \fInumjobs\fR as long as they are using generated filenames.
167If specific \fIfilename(s)\fR are set fio will use the first listed directory,
168and thereby matching the \fIfilename\fR semantic which generates a file each
169clone if not specified, but let all clones use the same if set.
d60e92d1
AC
170.TP
171.BI filename \fR=\fPstr
172.B fio
173normally makes up a file name based on the job name, thread number, and file
d1429b5c 174number. If you want to share files between threads in a job or several jobs,
de890a1e
SL
175specify a \fIfilename\fR for each of them to override the default.
176If the I/O engine is file-based, you can specify
d1429b5c
AC
177a number of files by separating the names with a `:' character. `\-' is a
178reserved name, meaning stdin or stdout, depending on the read/write direction
179set.
d60e92d1 180.TP
de98bd30 181.BI filename_format \fR=\fPstr
ce594fbe 182If sharing multiple files between jobs, it is usually necessary to have
de98bd30
JA
183fio generate the exact names that you want. By default, fio will name a file
184based on the default file format specification of
185\fBjobname.jobnumber.filenumber\fP. With this option, that can be
186customized. Fio will recognize and replace the following keywords in this
187string:
188.RS
189.RS
190.TP
191.B $jobname
192The name of the worker thread or process.
193.TP
194.B $jobnum
195The incremental number of the worker thread or process.
196.TP
197.B $filenum
198The incremental number of the file for that worker thread or process.
199.RE
200.P
201To have dependent jobs share a set of files, this option can be set to
202have fio generate filenames that are shared between the two. For instance,
203if \fBtestfiles.$filenum\fR is specified, file number 4 for any job will
204be named \fBtestfiles.4\fR. The default of \fB$jobname.$jobnum.$filenum\fR
205will be used if no other format specifier is given.
206.RE
207.P
208.TP
3ce9dcaf
JA
209.BI lockfile \fR=\fPstr
210Fio defaults to not locking any files before it does IO to them. If a file or
211file descriptor is shared, fio can serialize IO to that file to make the end
212result consistent. This is usual for emulating real workloads that share files.
213The lock modes are:
214.RS
215.RS
216.TP
217.B none
218No locking. This is the default.
219.TP
220.B exclusive
221Only one thread or process may do IO at the time, excluding all others.
222.TP
223.B readwrite
224Read-write locking on the file. Many readers may access the file at the same
225time, but writes get exclusive access.
226.RE
ce594fbe 227.RE
3ce9dcaf 228.P
d60e92d1
AC
229.BI opendir \fR=\fPstr
230Recursively open any files below directory \fIstr\fR.
231.TP
232.BI readwrite \fR=\fPstr "\fR,\fP rw" \fR=\fPstr
233Type of I/O pattern. Accepted values are:
234.RS
235.RS
236.TP
237.B read
d1429b5c 238Sequential reads.
d60e92d1
AC
239.TP
240.B write
d1429b5c 241Sequential writes.
d60e92d1 242.TP
fa769d44
SW
243.B trim
244Sequential trim (Linux block devices only).
245.TP
d60e92d1 246.B randread
d1429b5c 247Random reads.
d60e92d1
AC
248.TP
249.B randwrite
d1429b5c 250Random writes.
d60e92d1 251.TP
fa769d44
SW
252.B randtrim
253Random trim (Linux block devices only).
254.TP
10b023db 255.B rw, readwrite
d1429b5c 256Mixed sequential reads and writes.
d60e92d1
AC
257.TP
258.B randrw
d1429b5c 259Mixed random reads and writes.
d60e92d1
AC
260.RE
261.P
38dad62d
JA
262For mixed I/O, the default split is 50/50. For certain types of io the result
263may still be skewed a bit, since the speed may be different. It is possible to
3b7fa9ec 264specify a number of IO's to do before getting a new offset, this is done by
38dad62d
JA
265appending a `:\fI<nr>\fR to the end of the string given. For a random read, it
266would look like \fBrw=randread:8\fR for passing in an offset modifier with a
059b0802
JA
267value of 8. If the postfix is used with a sequential IO pattern, then the value
268specified will be added to the generated offset for each IO. For instance,
269using \fBrw=write:4k\fR will skip 4k for every write. It turns sequential IO
270into sequential IO with holes. See the \fBrw_sequencer\fR option.
d60e92d1
AC
271.RE
272.TP
38dad62d
JA
273.BI rw_sequencer \fR=\fPstr
274If an offset modifier is given by appending a number to the \fBrw=<str>\fR line,
275then this option controls how that number modifies the IO offset being
276generated. Accepted values are:
277.RS
278.RS
279.TP
280.B sequential
281Generate sequential offset
282.TP
283.B identical
284Generate the same offset
285.RE
286.P
287\fBsequential\fR is only useful for random IO, where fio would normally
288generate a new random offset for every IO. If you append eg 8 to randread, you
289would get a new random offset for every 8 IO's. The result would be a seek for
290only every 8 IO's, instead of for every IO. Use \fBrw=randread:8\fR to specify
291that. As sequential IO is already sequential, setting \fBsequential\fR for that
292would not result in any differences. \fBidentical\fR behaves in a similar
293fashion, except it sends the same offset 8 number of times before generating a
294new offset.
295.RE
296.P
297.TP
90fef2d1
JA
298.BI kb_base \fR=\fPint
299The base unit for a kilobyte. The defacto base is 2^10, 1024. Storage
300manufacturers like to use 10^3 or 1000 as a base ten unit instead, for obvious
5c9323fb 301reasons. Allowed values are 1024 or 1000, with 1024 being the default.
90fef2d1 302.TP
771e58be
JA
303.BI unified_rw_reporting \fR=\fPbool
304Fio normally reports statistics on a per data direction basis, meaning that
305read, write, and trim are accounted and reported separately. If this option is
306set, the fio will sum the results and report them as "mixed" instead.
307.TP
d60e92d1 308.BI randrepeat \fR=\fPbool
56e2a5fc
CE
309Seed the random number generator used for random I/O patterns in a predictable
310way so the pattern is repeatable across runs. Default: true.
311.TP
312.BI allrandrepeat \fR=\fPbool
313Seed all random number generators in a predictable way so results are
314repeatable across runs. Default: false.
d60e92d1 315.TP
04778baf
JA
316.BI randseed \fR=\fPint
317Seed the random number generators based on this seed value, to be able to
318control what sequence of output is being generated. If not set, the random
319sequence depends on the \fBrandrepeat\fR setting.
320.TP
2615cc4b
JA
321.BI use_os_rand \fR=\fPbool
322Fio can either use the random generator supplied by the OS to generator random
323offsets, or it can use it's own internal generator (based on Tausworthe).
324Default is to use the internal generator, which is often of better quality and
325faster. Default: false.
326.TP
a596f047
EG
327.BI fallocate \fR=\fPstr
328Whether pre-allocation is performed when laying down files. Accepted values
329are:
330.RS
331.RS
332.TP
333.B none
334Do not pre-allocate space.
335.TP
336.B posix
ccc2b328 337Pre-allocate via \fBposix_fallocate\fR\|(3).
a596f047
EG
338.TP
339.B keep
ccc2b328 340Pre-allocate via \fBfallocate\fR\|(2) with FALLOC_FL_KEEP_SIZE set.
a596f047
EG
341.TP
342.B 0
343Backward-compatible alias for 'none'.
344.TP
345.B 1
346Backward-compatible alias for 'posix'.
347.RE
348.P
349May not be available on all supported platforms. 'keep' is only
350available on Linux. If using ZFS on Solaris this must be set to 'none'
351because ZFS doesn't support it. Default: 'posix'.
352.RE
7bc8c2cf 353.TP
d60e92d1 354.BI fadvise_hint \fR=\fPbool
ccc2b328 355Use of \fBposix_fadvise\fR\|(2) to advise the kernel what I/O patterns
d1429b5c 356are likely to be issued. Default: true.
d60e92d1 357.TP
f7fa2653 358.BI size \fR=\fPint
d60e92d1 359Total size of I/O for this job. \fBfio\fR will run until this many bytes have
ca45881f 360been transferred, unless limited by other options (\fBruntime\fR, for instance).
d7c8be03 361Unless \fBnrfiles\fR and \fBfilesize\fR options are given, this amount will be
d6667268 362divided between the available files for the job. If not set, fio will use the
cecbfd47 363full size of the given files or devices. If the files do not exist, size
7bb59102
JA
364must be given. It is also possible to give size as a percentage between 1 and
365100. If size=20% is given, fio will use 20% of the full size of the given files
366or devices.
d60e92d1 367.TP
74586c1e 368.BI fill_device \fR=\fPbool "\fR,\fB fill_fs" \fR=\fPbool
3ce9dcaf
JA
369Sets size to something really large and waits for ENOSPC (no space left on
370device) as the terminating condition. Only makes sense with sequential write.
371For a read workload, the mount point will be filled first then IO started on
4f12432e
JA
372the result. This option doesn't make sense if operating on a raw device node,
373since the size of that is already known by the file system. Additionally,
374writing beyond end-of-device will not return ENOSPC there.
3ce9dcaf 375.TP
d60e92d1
AC
376.BI filesize \fR=\fPirange
377Individual file sizes. May be a range, in which case \fBfio\fR will select sizes
d1429b5c
AC
378for files at random within the given range, limited to \fBsize\fR in total (if
379that is given). If \fBfilesize\fR is not specified, each created file is the
380same size.
d60e92d1 381.TP
f7fa2653 382.BI blocksize \fR=\fPint[,int] "\fR,\fB bs" \fR=\fPint[,int]
d9472271
JA
383Block size for I/O units. Default: 4k. Values for reads, writes, and trims
384can be specified separately in the format \fIread\fR,\fIwrite\fR,\fItrim\fR
385either of which may be empty to leave that value at its default. If a trailing
386comma isn't given, the remainder will inherit the last value set.
d60e92d1 387.TP
9183788d 388.BI blocksize_range \fR=\fPirange[,irange] "\fR,\fB bsrange" \fR=\fPirange[,irange]
d1429b5c
AC
389Specify a range of I/O block sizes. The issued I/O unit will always be a
390multiple of the minimum size, unless \fBblocksize_unaligned\fR is set. Applies
9183788d 391to both reads and writes if only one range is given, but can be specified
de8f6de9 392separately with a comma separating the values. Example: bsrange=1k-4k,2k-8k.
9183788d
JA
393Also (see \fBblocksize\fR).
394.TP
395.BI bssplit \fR=\fPstr
396This option allows even finer grained control of the block sizes issued,
397not just even splits between them. With this option, you can weight various
398block sizes for exact control of the issued IO for a job that has mixed
399block sizes. The format of the option is bssplit=blocksize/percentage,
5982a925 400optionally adding as many definitions as needed separated by a colon.
9183788d 401Example: bssplit=4k/10:64k/50:32k/40 would issue 50% 64k blocks, 10% 4k
c83cdd3e
JA
402blocks and 40% 32k blocks. \fBbssplit\fR also supports giving separate
403splits to reads and writes. The format is identical to what the
404\fBbs\fR option accepts, the read and write parts are separated with a
405comma.
d60e92d1
AC
406.TP
407.B blocksize_unaligned\fR,\fP bs_unaligned
d1429b5c
AC
408If set, any size in \fBblocksize_range\fR may be used. This typically won't
409work with direct I/O, as that normally requires sector alignment.
d60e92d1 410.TP
2b7a01d0 411.BI blockalign \fR=\fPint[,int] "\fR,\fB ba" \fR=\fPint[,int]
639ce0f3
MS
412At what boundary to align random IO offsets. Defaults to the same as 'blocksize'
413the minimum blocksize given. Minimum alignment is typically 512b
2b7a01d0
JA
414for using direct IO, though it usually depends on the hardware block size.
415This option is mutually exclusive with using a random map for files, so it
416will turn off that option.
43602667 417.TP
6aca9b3d
JA
418.BI bs_is_seq_rand \fR=\fPbool
419If this option is set, fio will use the normal read,write blocksize settings as
420sequential,random instead. Any random read or write will use the WRITE
421blocksize settings, and any sequential read or write will use the READ
422blocksize setting.
423.TP
d60e92d1
AC
424.B zero_buffers
425Initialise buffers with all zeros. Default: fill buffers with random data.
426.TP
901bb994
JA
427.B refill_buffers
428If this option is given, fio will refill the IO buffers on every submit. The
429default is to only fill it at init time and reuse that data. Only makes sense
430if zero_buffers isn't specified, naturally. If data verification is enabled,
431refill_buffers is also automatically enabled.
432.TP
fd68418e
JA
433.BI scramble_buffers \fR=\fPbool
434If \fBrefill_buffers\fR is too costly and the target is using data
435deduplication, then setting this option will slightly modify the IO buffer
436contents to defeat normal de-dupe attempts. This is not enough to defeat
437more clever block compression attempts, but it will stop naive dedupe
438of blocks. Default: true.
439.TP
c5751c62
JA
440.BI buffer_compress_percentage \fR=\fPint
441If this is set, then fio will attempt to provide IO buffer content (on WRITEs)
442that compress to the specified level. Fio does this by providing a mix of
443random data and zeroes. Note that this is per block size unit, for file/disk
444wide compression level that matches this setting, you'll also want to set
445\fBrefill_buffers\fR.
446.TP
447.BI buffer_compress_chunk \fR=\fPint
448See \fBbuffer_compress_percentage\fR. This setting allows fio to manage how
449big the ranges of random data and zeroed data is. Without this set, fio will
450provide \fBbuffer_compress_percentage\fR of blocksize random data, followed by
451the remaining zeroed. With this set to some chunk size smaller than the block
452size, fio can alternate random and zeroed data throughout the IO buffer.
453.TP
ce35b1ec
JA
454.BI buffer_pattern \fR=\fPstr
455If set, fio will fill the io buffers with this pattern. If not set, the contents
456of io buffers is defined by the other options related to buffer contents. The
457setting can be any pattern of bytes, and can be prefixed with 0x for hex
458values.
459.TP
d60e92d1
AC
460.BI nrfiles \fR=\fPint
461Number of files to use for this job. Default: 1.
462.TP
463.BI openfiles \fR=\fPint
464Number of files to keep open at the same time. Default: \fBnrfiles\fR.
465.TP
466.BI file_service_type \fR=\fPstr
467Defines how files to service are selected. The following types are defined:
468.RS
469.RS
470.TP
471.B random
5c9323fb 472Choose a file at random.
d60e92d1
AC
473.TP
474.B roundrobin
475Round robin over open files (default).
5c9323fb 476.TP
6b7f6851
JA
477.B sequential
478Do each file in the set sequentially.
d60e92d1
AC
479.RE
480.P
481The number of I/Os to issue before switching a new file can be specified by
482appending `:\fIint\fR' to the service type.
483.RE
484.TP
485.BI ioengine \fR=\fPstr
486Defines how the job issues I/O. The following types are defined:
487.RS
488.RS
489.TP
490.B sync
ccc2b328 491Basic \fBread\fR\|(2) or \fBwrite\fR\|(2) I/O. \fBfseek\fR\|(2) is used to
d60e92d1
AC
492position the I/O location.
493.TP
a31041ea 494.B psync
ccc2b328 495Basic \fBpread\fR\|(2) or \fBpwrite\fR\|(2) I/O.
a31041ea 496.TP
9183788d 497.B vsync
ccc2b328 498Basic \fBreadv\fR\|(2) or \fBwritev\fR\|(2) I/O. Will emulate queuing by
cecbfd47 499coalescing adjacent IOs into a single submission.
9183788d 500.TP
a46c5e01 501.B pvsync
ccc2b328 502Basic \fBpreadv\fR\|(2) or \fBpwritev\fR\|(2) I/O.
a46c5e01 503.TP
d60e92d1 504.B libaio
de890a1e 505Linux native asynchronous I/O. This ioengine defines engine specific options.
d60e92d1
AC
506.TP
507.B posixaio
ccc2b328 508POSIX asynchronous I/O using \fBaio_read\fR\|(3) and \fBaio_write\fR\|(3).
03e20d68
BC
509.TP
510.B solarisaio
511Solaris native asynchronous I/O.
512.TP
513.B windowsaio
514Windows native asynchronous I/O.
d60e92d1
AC
515.TP
516.B mmap
ccc2b328
SW
517File is memory mapped with \fBmmap\fR\|(2) and data copied using
518\fBmemcpy\fR\|(3).
d60e92d1
AC
519.TP
520.B splice
ccc2b328 521\fBsplice\fR\|(2) is used to transfer the data and \fBvmsplice\fR\|(2) to
d1429b5c 522transfer data from user-space to the kernel.
d60e92d1
AC
523.TP
524.B syslet-rw
525Use the syslet system calls to make regular read/write asynchronous.
526.TP
527.B sg
528SCSI generic sg v3 I/O. May be either synchronous using the SG_IO ioctl, or if
ccc2b328
SW
529the target is an sg character device, we use \fBread\fR\|(2) and
530\fBwrite\fR\|(2) for asynchronous I/O.
d60e92d1
AC
531.TP
532.B null
533Doesn't transfer any data, just pretends to. Mainly used to exercise \fBfio\fR
534itself and for debugging and testing purposes.
535.TP
536.B net
de890a1e
SL
537Transfer over the network. The protocol to be used can be defined with the
538\fBprotocol\fR parameter. Depending on the protocol, \fBfilename\fR,
539\fBhostname\fR, \fBport\fR, or \fBlisten\fR must be specified.
540This ioengine defines engine specific options.
d60e92d1
AC
541.TP
542.B netsplice
ccc2b328 543Like \fBnet\fR, but uses \fBsplice\fR\|(2) and \fBvmsplice\fR\|(2) to map data
de890a1e 544and send/receive. This ioengine defines engine specific options.
d60e92d1 545.TP
53aec0a4 546.B cpuio
d60e92d1
AC
547Doesn't transfer any data, but burns CPU cycles according to \fBcpuload\fR and
548\fBcpucycles\fR parameters.
549.TP
550.B guasi
551The GUASI I/O engine is the Generic Userspace Asynchronous Syscall Interface
cecbfd47 552approach to asynchronous I/O.
d1429b5c
AC
553.br
554See <http://www.xmailserver.org/guasi\-lib.html>.
d60e92d1 555.TP
21b8aee8 556.B rdma
85286c5c
BVA
557The RDMA I/O engine supports both RDMA memory semantics (RDMA_WRITE/RDMA_READ)
558and channel semantics (Send/Recv) for the InfiniBand, RoCE and iWARP protocols.
21b8aee8 559.TP
d60e92d1
AC
560.B external
561Loads an external I/O engine object file. Append the engine filename as
562`:\fIenginepath\fR'.
d54fce84
DM
563.TP
564.B falloc
cecbfd47 565 IO engine that does regular linux native fallocate call to simulate data
d54fce84
DM
566transfer as fio ioengine
567.br
568 DDIR_READ does fallocate(,mode = FALLOC_FL_KEEP_SIZE,)
569.br
0981fd71 570 DIR_WRITE does fallocate(,mode = 0)
d54fce84
DM
571.br
572 DDIR_TRIM does fallocate(,mode = FALLOC_FL_KEEP_SIZE|FALLOC_FL_PUNCH_HOLE)
573.TP
574.B e4defrag
575IO engine that does regular EXT4_IOC_MOVE_EXT ioctls to simulate defragment activity
576request to DDIR_WRITE event
0d978694
DAG
577.TP
578.B rbd
579IO engine supporting direct access to Ceph Rados Block Devices (RBD) via librbd
580without the need to use the kernel rbd driver. This ioengine defines engine specific
581options.
d60e92d1 582.RE
595e1734 583.P
d60e92d1
AC
584.RE
585.TP
586.BI iodepth \fR=\fPint
8489dae4
SK
587Number of I/O units to keep in flight against the file. Note that increasing
588iodepth beyond 1 will not affect synchronous ioengines (except for small
ee72ca09
JA
589degress when verify_async is in use). Even async engines my impose OS
590restrictions causing the desired depth not to be achieved. This may happen on
591Linux when using libaio and not setting \fBdirect\fR=1, since buffered IO is
592not async on that OS. Keep an eye on the IO depth distribution in the
593fio output to verify that the achieved depth is as expected. Default: 1.
d60e92d1
AC
594.TP
595.BI iodepth_batch \fR=\fPint
596Number of I/Os to submit at once. Default: \fBiodepth\fR.
597.TP
3ce9dcaf
JA
598.BI iodepth_batch_complete \fR=\fPint
599This defines how many pieces of IO to retrieve at once. It defaults to 1 which
600 means that we'll ask for a minimum of 1 IO in the retrieval process from the
601kernel. The IO retrieval will go on until we hit the limit set by
602\fBiodepth_low\fR. If this variable is set to 0, then fio will always check for
603completed events before queuing more IO. This helps reduce IO latency, at the
604cost of more retrieval system calls.
605.TP
d60e92d1
AC
606.BI iodepth_low \fR=\fPint
607Low watermark indicating when to start filling the queue again. Default:
608\fBiodepth\fR.
609.TP
610.BI direct \fR=\fPbool
611If true, use non-buffered I/O (usually O_DIRECT). Default: false.
612.TP
d01612f3
CM
613.BI atomic \fR=\fPbool
614If value is true, attempt to use atomic direct IO. Atomic writes are guaranteed
615to be stable once acknowledged by the operating system. Only Linux supports
616O_ATOMIC right now.
617.TP
d60e92d1
AC
618.BI buffered \fR=\fPbool
619If true, use buffered I/O. This is the opposite of the \fBdirect\fR parameter.
620Default: true.
621.TP
f7fa2653 622.BI offset \fR=\fPint
d60e92d1
AC
623Offset in the file to start I/O. Data before the offset will not be touched.
624.TP
591e9e06
JA
625.BI offset_increment \fR=\fPint
626If this is provided, then the real offset becomes the
627offset + offset_increment * thread_number, where the thread number is a counter
628that starts at 0 and is incremented for each job. This option is useful if
629there are several jobs which are intended to operate on a file in parallel in
630disjoint segments, with even spacing between the starting points.
631.TP
ddf24e42
JA
632.BI number_ios \fR=\fPint
633Fio will normally perform IOs until it has exhausted the size of the region
634set by \fBsize\fR, or if it exhaust the allocated time (or hits an error
635condition). With this setting, the range/size can be set independently of
636the number of IOs to perform. When fio reaches this number, it will exit
637normally and report status.
638.TP
d60e92d1 639.BI fsync \fR=\fPint
d1429b5c
AC
640How many I/Os to perform before issuing an \fBfsync\fR\|(2) of dirty data. If
6410, don't sync. Default: 0.
d60e92d1 642.TP
5f9099ea
JA
643.BI fdatasync \fR=\fPint
644Like \fBfsync\fR, but uses \fBfdatasync\fR\|(2) instead to only sync the
645data parts of the file. Default: 0.
646.TP
fa769d44
SW
647.BI write_barrier \fR=\fPint
648Make every Nth write a barrier write.
649.TP
e76b1da4 650.BI sync_file_range \fR=\fPstr:int
ccc2b328
SW
651Use \fBsync_file_range\fR\|(2) for every \fRval\fP number of write operations. Fio will
652track range of writes that have happened since the last \fBsync_file_range\fR\|(2) call.
e76b1da4
JA
653\fRstr\fP can currently be one or more of:
654.RS
655.TP
656.B wait_before
657SYNC_FILE_RANGE_WAIT_BEFORE
658.TP
659.B write
660SYNC_FILE_RANGE_WRITE
661.TP
662.B wait_after
663SYNC_FILE_RANGE_WRITE
664.TP
665.RE
666.P
667So if you do sync_file_range=wait_before,write:8, fio would use
668\fBSYNC_FILE_RANGE_WAIT_BEFORE | SYNC_FILE_RANGE_WRITE\fP for every 8 writes.
ccc2b328 669Also see the \fBsync_file_range\fR\|(2) man page. This option is Linux specific.
e76b1da4 670.TP
d60e92d1 671.BI overwrite \fR=\fPbool
d1429b5c 672If writing, setup the file first and do overwrites. Default: false.
d60e92d1
AC
673.TP
674.BI end_fsync \fR=\fPbool
dbd11ead 675Sync file contents when a write stage has completed. Default: false.
d60e92d1
AC
676.TP
677.BI fsync_on_close \fR=\fPbool
678If true, sync file contents on close. This differs from \fBend_fsync\fR in that
d1429b5c 679it will happen on every close, not just at the end of the job. Default: false.
d60e92d1 680.TP
d60e92d1
AC
681.BI rwmixread \fR=\fPint
682Percentage of a mixed workload that should be reads. Default: 50.
683.TP
684.BI rwmixwrite \fR=\fPint
d1429b5c 685Percentage of a mixed workload that should be writes. If \fBrwmixread\fR and
c35dd7a6
JA
686\fBrwmixwrite\fR are given and do not sum to 100%, the latter of the two
687overrides the first. This may interfere with a given rate setting, if fio is
688asked to limit reads or writes to a certain rate. If that is the case, then
689the distribution may be skewed. Default: 50.
d60e92d1 690.TP
92d42d69
JA
691.BI random_distribution \fR=\fPstr:float
692By default, fio will use a completely uniform random distribution when asked
693to perform random IO. Sometimes it is useful to skew the distribution in
694specific ways, ensuring that some parts of the data is more hot than others.
695Fio includes the following distribution models:
696.RS
697.TP
698.B random
699Uniform random distribution
700.TP
701.B zipf
702Zipf distribution
703.TP
704.B pareto
705Pareto distribution
706.TP
707.RE
708.P
709When using a zipf or pareto distribution, an input value is also needed to
710define the access pattern. For zipf, this is the zipf theta. For pareto,
711it's the pareto power. Fio includes a test program, genzipf, that can be
712used visualize what the given input values will yield in terms of hit rates.
713If you wanted to use zipf with a theta of 1.2, you would use
714random_distribution=zipf:1.2 as the option. If a non-uniform model is used,
715fio will disable use of the random map.
716.TP
211c9b89
JA
717.BI percentage_random \fR=\fPint
718For a random workload, set how big a percentage should be random. This defaults
719to 100%, in which case the workload is fully random. It can be set from
720anywhere from 0 to 100. Setting it to 0 would make the workload fully
d9472271
JA
721sequential. It is possible to set different values for reads, writes, and
722trim. To do so, simply use a comma separated list. See \fBblocksize\fR.
211c9b89 723.TP
d60e92d1
AC
724.B norandommap
725Normally \fBfio\fR will cover every block of the file when doing random I/O. If
726this parameter is given, a new offset will be chosen without looking at past
727I/O history. This parameter is mutually exclusive with \fBverify\fR.
728.TP
744492c9 729.BI softrandommap \fR=\fPbool
3ce9dcaf
JA
730See \fBnorandommap\fR. If fio runs with the random block map enabled and it
731fails to allocate the map, if this option is set it will continue without a
732random block map. As coverage will not be as complete as with random maps, this
733option is disabled by default.
734.TP
e8b1961d
JA
735.BI random_generator \fR=\fPstr
736Fio supports the following engines for generating IO offsets for random IO:
737.RS
738.TP
739.B tausworthe
740Strong 2^88 cycle random number generator
741.TP
742.B lfsr
743Linear feedback shift register generator
744.TP
745.RE
746.P
747Tausworthe is a strong random number generator, but it requires tracking on the
748side if we want to ensure that blocks are only read or written once. LFSR
749guarantees that we never generate the same offset twice, and it's also less
750computationally expensive. It's not a true random generator, however, though
751for IO purposes it's typically good enough. LFSR only works with single block
752sizes, not with workloads that use multiple block sizes. If used with such a
753workload, fio may read or write some blocks multiple times.
754.TP
d60e92d1 755.BI nice \fR=\fPint
ccc2b328 756Run job with given nice value. See \fBnice\fR\|(2).
d60e92d1
AC
757.TP
758.BI prio \fR=\fPint
759Set I/O priority value of this job between 0 (highest) and 7 (lowest). See
ccc2b328 760\fBionice\fR\|(1).
d60e92d1
AC
761.TP
762.BI prioclass \fR=\fPint
ccc2b328 763Set I/O priority class. See \fBionice\fR\|(1).
d60e92d1
AC
764.TP
765.BI thinktime \fR=\fPint
766Stall job for given number of microseconds between issuing I/Os.
767.TP
768.BI thinktime_spin \fR=\fPint
769Pretend to spend CPU time for given number of microseconds, sleeping the rest
770of the time specified by \fBthinktime\fR. Only valid if \fBthinktime\fR is set.
771.TP
772.BI thinktime_blocks \fR=\fPint
4d01ece6
JA
773Only valid if thinktime is set - control how many blocks to issue, before
774waiting \fBthinktime\fR microseconds. If not set, defaults to 1 which will
775make fio wait \fBthinktime\fR microseconds after every block. This
776effectively makes any queue depth setting redundant, since no more than 1 IO
777will be queued before we have to complete it and do our thinktime. In other
778words, this setting effectively caps the queue depth if the latter is larger.
d60e92d1
AC
779Default: 1.
780.TP
781.BI rate \fR=\fPint
c35dd7a6
JA
782Cap bandwidth used by this job. The number is in bytes/sec, the normal postfix
783rules apply. You can use \fBrate\fR=500k to limit reads and writes to 500k each,
784or you can specify read and writes separately. Using \fBrate\fR=1m,500k would
785limit reads to 1MB/sec and writes to 500KB/sec. Capping only reads or writes
786can be done with \fBrate\fR=,500k or \fBrate\fR=500k,. The former will only
787limit writes (to 500KB/sec), the latter will only limit reads.
d60e92d1
AC
788.TP
789.BI ratemin \fR=\fPint
790Tell \fBfio\fR to do whatever it can to maintain at least the given bandwidth.
c35dd7a6
JA
791Failing to meet this requirement will cause the job to exit. The same format
792as \fBrate\fR is used for read vs write separation.
d60e92d1
AC
793.TP
794.BI rate_iops \fR=\fPint
c35dd7a6
JA
795Cap the bandwidth to this number of IOPS. Basically the same as rate, just
796specified independently of bandwidth. The same format as \fBrate\fR is used for
de8f6de9 797read vs write separation. If \fBblocksize\fR is a range, the smallest block
c35dd7a6 798size is used as the metric.
d60e92d1
AC
799.TP
800.BI rate_iops_min \fR=\fPint
c35dd7a6 801If this rate of I/O is not met, the job will exit. The same format as \fBrate\fR
de8f6de9 802is used for read vs write separation.
d60e92d1
AC
803.TP
804.BI ratecycle \fR=\fPint
805Average bandwidth for \fBrate\fR and \fBratemin\fR over this number of
806milliseconds. Default: 1000ms.
807.TP
3e260a46
JA
808.BI latency_target \fR=\fPint
809If set, fio will attempt to find the max performance point that the given
810workload will run at while maintaining a latency below this target. The
811values is given in microseconds. See \fBlatency_window\fR and
812\fBlatency_percentile\fR.
813.TP
814.BI latency_window \fR=\fPint
815Used with \fBlatency_target\fR to specify the sample window that the job
816is run at varying queue depths to test the performance. The value is given
817in microseconds.
818.TP
819.BI latency_percentile \fR=\fPfloat
820The percentage of IOs that must fall within the criteria specified by
821\fBlatency_target\fR and \fBlatency_window\fR. If not set, this defaults
822to 100.0, meaning that all IOs must be equal or below to the value set
823by \fBlatency_target\fR.
824.TP
15501535
JA
825.BI max_latency \fR=\fPint
826If set, fio will exit the job if it exceeds this maximum latency. It will exit
827with an ETIME error.
828.TP
d60e92d1
AC
829.BI cpumask \fR=\fPint
830Set CPU affinity for this job. \fIint\fR is a bitmask of allowed CPUs the job
831may run on. See \fBsched_setaffinity\fR\|(2).
832.TP
833.BI cpus_allowed \fR=\fPstr
834Same as \fBcpumask\fR, but allows a comma-delimited list of CPU numbers.
835.TP
d0b937ed 836.BI numa_cpu_nodes \fR=\fPstr
cecbfd47 837Set this job running on specified NUMA nodes' CPUs. The arguments allow
d0b937ed
YR
838comma delimited list of cpu numbers, A-B ranges, or 'all'.
839.TP
840.BI numa_mem_policy \fR=\fPstr
841Set this job's memory policy and corresponding NUMA nodes. Format of
cecbfd47 842the arguments:
d0b937ed
YR
843.RS
844.TP
845.B <mode>[:<nodelist>]
846.TP
847.B mode
848is one of the following memory policy:
849.TP
850.B default, prefer, bind, interleave, local
851.TP
852.RE
853For \fBdefault\fR and \fBlocal\fR memory policy, no \fBnodelist\fR is
854needed to be specified. For \fBprefer\fR, only one node is
855allowed. For \fBbind\fR and \fBinterleave\fR, \fBnodelist\fR allows
856comma delimited list of numbers, A-B ranges, or 'all'.
857.TP
23ed19b0
CE
858.BI startdelay \fR=\fPirange
859Delay start of job for the specified number of seconds. Supports all time
860suffixes to allow specification of hours, minutes, seconds and
861milliseconds - seconds are the default if a unit is ommited.
862Can be given as a range which causes each thread to choose randomly out of the
863range.
d60e92d1
AC
864.TP
865.BI runtime \fR=\fPint
866Terminate processing after the specified number of seconds.
867.TP
868.B time_based
869If given, run for the specified \fBruntime\fR duration even if the files are
870completely read or written. The same workload will be repeated as many times
871as \fBruntime\fR allows.
872.TP
901bb994
JA
873.BI ramp_time \fR=\fPint
874If set, fio will run the specified workload for this amount of time before
875logging any performance numbers. Useful for letting performance settle before
876logging results, thus minimizing the runtime required for stable results. Note
c35dd7a6
JA
877that the \fBramp_time\fR is considered lead in time for a job, thus it will
878increase the total runtime if a special timeout or runtime is specified.
901bb994 879.TP
d60e92d1
AC
880.BI invalidate \fR=\fPbool
881Invalidate buffer-cache for the file prior to starting I/O. Default: true.
882.TP
883.BI sync \fR=\fPbool
884Use synchronous I/O for buffered writes. For the majority of I/O engines,
d1429b5c 885this means using O_SYNC. Default: false.
d60e92d1
AC
886.TP
887.BI iomem \fR=\fPstr "\fR,\fP mem" \fR=\fPstr
888Allocation method for I/O unit buffer. Allowed values are:
889.RS
890.RS
891.TP
892.B malloc
ccc2b328 893Allocate memory with \fBmalloc\fR\|(3).
d60e92d1
AC
894.TP
895.B shm
ccc2b328 896Use shared memory buffers allocated through \fBshmget\fR\|(2).
d60e92d1
AC
897.TP
898.B shmhuge
899Same as \fBshm\fR, but use huge pages as backing.
900.TP
901.B mmap
ccc2b328 902Use \fBmmap\fR\|(2) for allocation. Uses anonymous memory unless a filename
d60e92d1
AC
903is given after the option in the format `:\fIfile\fR'.
904.TP
905.B mmaphuge
906Same as \fBmmap\fR, but use huge files as backing.
907.RE
908.P
909The amount of memory allocated is the maximum allowed \fBblocksize\fR for the
910job multiplied by \fBiodepth\fR. For \fBshmhuge\fR or \fBmmaphuge\fR to work,
911the system must have free huge pages allocated. \fBmmaphuge\fR also needs to
2e266ba6
JA
912have hugetlbfs mounted, and \fIfile\fR must point there. At least on Linux,
913huge pages must be manually allocated. See \fB/proc/sys/vm/nr_hugehages\fR
914and the documentation for that. Normally you just need to echo an appropriate
915number, eg echoing 8 will ensure that the OS has 8 huge pages ready for
916use.
d60e92d1
AC
917.RE
918.TP
d392365e 919.BI iomem_align \fR=\fPint "\fR,\fP mem_align" \fR=\fPint
cecbfd47 920This indicates the memory alignment of the IO memory buffers. Note that the
d529ee19
JA
921given alignment is applied to the first IO unit buffer, if using \fBiodepth\fR
922the alignment of the following buffers are given by the \fBbs\fR used. In
923other words, if using a \fBbs\fR that is a multiple of the page sized in the
924system, all buffers will be aligned to this value. If using a \fBbs\fR that
925is not page aligned, the alignment of subsequent IO memory buffers is the
926sum of the \fBiomem_align\fR and \fBbs\fR used.
927.TP
f7fa2653 928.BI hugepage\-size \fR=\fPint
d60e92d1 929Defines the size of a huge page. Must be at least equal to the system setting.
b22989b9 930Should be a multiple of 1MB. Default: 4MB.
d60e92d1
AC
931.TP
932.B exitall
933Terminate all jobs when one finishes. Default: wait for each job to finish.
934.TP
935.BI bwavgtime \fR=\fPint
936Average bandwidth calculations over the given time in milliseconds. Default:
937500ms.
938.TP
c8eeb9df
JA
939.BI iopsavgtime \fR=\fPint
940Average IOPS calculations over the given time in milliseconds. Default:
941500ms.
942.TP
d60e92d1 943.BI create_serialize \fR=\fPbool
d1429b5c 944If true, serialize file creation for the jobs. Default: true.
d60e92d1
AC
945.TP
946.BI create_fsync \fR=\fPbool
ccc2b328 947\fBfsync\fR\|(2) data file after creation. Default: true.
d60e92d1 948.TP
6b7f6851
JA
949.BI create_on_open \fR=\fPbool
950If true, the files are not created until they are opened for IO by the job.
951.TP
25460cf6
JA
952.BI create_only \fR=\fPbool
953If true, fio will only run the setup phase of the job. If files need to be
954laid out or updated on disk, only that will be done. The actual job contents
955are not executed.
956.TP
e9f48479
JA
957.BI pre_read \fR=\fPbool
958If this is given, files will be pre-read into memory before starting the given
959IO operation. This will also clear the \fR \fBinvalidate\fR flag, since it is
9c0d2241
JA
960pointless to pre-read and then drop the cache. This will only work for IO
961engines that are seekable, since they allow you to read the same data
962multiple times. Thus it will not work on eg network or splice IO.
e9f48479 963.TP
d60e92d1
AC
964.BI unlink \fR=\fPbool
965Unlink job files when done. Default: false.
966.TP
967.BI loops \fR=\fPint
968Specifies the number of iterations (runs of the same workload) of this job.
969Default: 1.
970.TP
5e4c7118
JA
971.BI verify_only \fR=\fPbool
972Do not perform the specified workload, only verify data still matches previous
973invocation of this workload. This option allows one to check data multiple
974times at a later date without overwriting it. This option makes sense only for
975workloads that write data, and does not support workloads with the
976\fBtime_based\fR option set.
977.TP
d60e92d1
AC
978.BI do_verify \fR=\fPbool
979Run the verify phase after a write phase. Only valid if \fBverify\fR is set.
980Default: true.
981.TP
982.BI verify \fR=\fPstr
983Method of verifying file contents after each iteration of the job. Allowed
984values are:
985.RS
986.RS
987.TP
b892dc08 988.B md5 crc16 crc32 crc32c crc32c-intel crc64 crc7 sha256 sha512 sha1
0539d758
JA
989Store appropriate checksum in the header of each block. crc32c-intel is
990hardware accelerated SSE4.2 driven, falls back to regular crc32c if
991not supported by the system.
d60e92d1
AC
992.TP
993.B meta
994Write extra information about each I/O (timestamp, block number, etc.). The
996093bb 995block number is verified. See \fBverify_pattern\fR as well.
d60e92d1
AC
996.TP
997.B null
998Pretend to verify. Used for testing internals.
999.RE
b892dc08
JA
1000
1001This option can be used for repeated burn-in tests of a system to make sure
1002that the written data is also correctly read back. If the data direction given
1003is a read or random read, fio will assume that it should verify a previously
1004written file. If the data direction includes any form of write, the verify will
1005be of the newly written data.
d60e92d1
AC
1006.RE
1007.TP
5c9323fb 1008.BI verifysort \fR=\fPbool
d60e92d1
AC
1009If true, written verify blocks are sorted if \fBfio\fR deems it to be faster to
1010read them back in a sorted manner. Default: true.
1011.TP
fa769d44
SW
1012.BI verifysort_nr \fR=\fPint
1013Pre-load and sort verify blocks for a read workload.
1014.TP
f7fa2653 1015.BI verify_offset \fR=\fPint
d60e92d1 1016Swap the verification header with data somewhere else in the block before
d1429b5c 1017writing. It is swapped back before verifying.
d60e92d1 1018.TP
f7fa2653 1019.BI verify_interval \fR=\fPint
d60e92d1
AC
1020Write the verification header for this number of bytes, which should divide
1021\fBblocksize\fR. Default: \fBblocksize\fR.
1022.TP
996093bb
JA
1023.BI verify_pattern \fR=\fPstr
1024If set, fio will fill the io buffers with this pattern. Fio defaults to filling
1025with totally random bytes, but sometimes it's interesting to fill with a known
1026pattern for io verification purposes. Depending on the width of the pattern,
1027fio will fill 1/2/3/4 bytes of the buffer at the time(it can be either a
1028decimal or a hex number). The verify_pattern if larger than a 32-bit quantity
1029has to be a hex number that starts with either "0x" or "0X". Use with
1030\fBverify\fP=meta.
1031.TP
d60e92d1
AC
1032.BI verify_fatal \fR=\fPbool
1033If true, exit the job on the first observed verification failure. Default:
1034false.
1035.TP
b463e936
JA
1036.BI verify_dump \fR=\fPbool
1037If set, dump the contents of both the original data block and the data block we
1038read off disk to files. This allows later analysis to inspect just what kind of
ef71e317 1039data corruption occurred. Off by default.
b463e936 1040.TP
e8462bd8
JA
1041.BI verify_async \fR=\fPint
1042Fio will normally verify IO inline from the submitting thread. This option
1043takes an integer describing how many async offload threads to create for IO
1044verification instead, causing fio to offload the duty of verifying IO contents
c85c324c
JA
1045to one or more separate threads. If using this offload option, even sync IO
1046engines can benefit from using an \fBiodepth\fR setting higher than 1, as it
1047allows them to have IO in flight while verifies are running.
e8462bd8
JA
1048.TP
1049.BI verify_async_cpus \fR=\fPstr
1050Tell fio to set the given CPU affinity on the async IO verification threads.
1051See \fBcpus_allowed\fP for the format used.
1052.TP
6f87418f
JA
1053.BI verify_backlog \fR=\fPint
1054Fio will normally verify the written contents of a job that utilizes verify
1055once that job has completed. In other words, everything is written then
1056everything is read back and verified. You may want to verify continually
1057instead for a variety of reasons. Fio stores the meta data associated with an
1058IO block in memory, so for large verify workloads, quite a bit of memory would
092f707f
DN
1059be used up holding this meta data. If this option is enabled, fio will write
1060only N blocks before verifying these blocks.
6f87418f
JA
1061.TP
1062.BI verify_backlog_batch \fR=\fPint
1063Control how many blocks fio will verify if verify_backlog is set. If not set,
1064will default to the value of \fBverify_backlog\fR (meaning the entire queue is
092f707f
DN
1065read back and verified). If \fBverify_backlog_batch\fR is less than
1066\fBverify_backlog\fR then not all blocks will be verified, if
1067\fBverify_backlog_batch\fR is larger than \fBverify_backlog\fR, some blocks
1068will be verified more than once.
6f87418f 1069.TP
fa769d44
SW
1070.BI trim_percentage \fR=\fPint
1071Number of verify blocks to discard/trim.
1072.TP
1073.BI trim_verify_zero \fR=\fPbool
1074Verify that trim/discarded blocks are returned as zeroes.
1075.TP
1076.BI trim_backlog \fR=\fPint
1077Trim after this number of blocks are written.
1078.TP
1079.BI trim_backlog_batch \fR=\fPint
1080Trim this number of IO blocks.
1081.TP
1082.BI experimental_verify \fR=\fPbool
1083Enable experimental verification.
1084.TP
d392365e 1085.B stonewall "\fR,\fP wait_for_previous"
5982a925 1086Wait for preceding jobs in the job file to exit before starting this one.
d60e92d1
AC
1087\fBstonewall\fR implies \fBnew_group\fR.
1088.TP
1089.B new_group
1090Start a new reporting group. If not given, all jobs in a file will be part
1091of the same reporting group, unless separated by a stonewall.
1092.TP
1093.BI numjobs \fR=\fPint
1094Number of clones (processes/threads performing the same workload) of this job.
1095Default: 1.
1096.TP
1097.B group_reporting
1098If set, display per-group reports instead of per-job when \fBnumjobs\fR is
1099specified.
1100.TP
1101.B thread
1102Use threads created with \fBpthread_create\fR\|(3) instead of processes created
1103with \fBfork\fR\|(2).
1104.TP
f7fa2653 1105.BI zonesize \fR=\fPint
d60e92d1
AC
1106Divide file into zones of the specified size in bytes. See \fBzoneskip\fR.
1107.TP
fa769d44
SW
1108.BI zonerange \fR=\fPint
1109Give size of an IO zone. See \fBzoneskip\fR.
1110.TP
f7fa2653 1111.BI zoneskip \fR=\fPint
d1429b5c 1112Skip the specified number of bytes when \fBzonesize\fR bytes of data have been
d60e92d1
AC
1113read.
1114.TP
1115.BI write_iolog \fR=\fPstr
5b42a488
SH
1116Write the issued I/O patterns to the specified file. Specify a separate file
1117for each job, otherwise the iologs will be interspersed and the file may be
1118corrupt.
d60e92d1
AC
1119.TP
1120.BI read_iolog \fR=\fPstr
1121Replay the I/O patterns contained in the specified file generated by
1122\fBwrite_iolog\fR, or may be a \fBblktrace\fR binary file.
1123.TP
64bbb865
DN
1124.BI replay_no_stall \fR=\fPint
1125While replaying I/O patterns using \fBread_iolog\fR the default behavior
1126attempts to respect timing information between I/Os. Enabling
1127\fBreplay_no_stall\fR causes I/Os to be replayed as fast as possible while
1128still respecting ordering.
1129.TP
d1c46c04
DN
1130.BI replay_redirect \fR=\fPstr
1131While replaying I/O patterns using \fBread_iolog\fR the default behavior
1132is to replay the IOPS onto the major/minor device that each IOP was recorded
1133from. Setting \fBreplay_redirect\fR causes all IOPS to be replayed onto the
1134single specified device regardless of the device it was recorded from.
1135.TP
836bad52 1136.BI write_bw_log \fR=\fPstr
901bb994
JA
1137If given, write a bandwidth log of the jobs in this job file. Can be used to
1138store data of the bandwidth of the jobs in their lifetime. The included
1139fio_generate_plots script uses gnuplot to turn these text files into nice
26b26fca 1140graphs. See \fBwrite_lat_log\fR for behaviour of given filename. For this
901bb994 1141option, the postfix is _bw.log.
d60e92d1 1142.TP
836bad52 1143.BI write_lat_log \fR=\fPstr
901bb994
JA
1144Same as \fBwrite_bw_log\fR, but writes I/O completion latencies. If no
1145filename is given with this option, the default filename of "jobname_type.log"
1146is used. Even if the filename is given, fio will still append the type of log.
1147.TP
c8eeb9df
JA
1148.BI write_iops_log \fR=\fPstr
1149Same as \fBwrite_bw_log\fR, but writes IOPS. If no filename is given with this
1150option, the default filename of "jobname_type.log" is used. Even if the
1151filename is given, fio will still append the type of log.
1152.TP
b8bc8cba
JA
1153.BI log_avg_msec \fR=\fPint
1154By default, fio will log an entry in the iops, latency, or bw log for every
1155IO that completes. When writing to the disk log, that can quickly grow to a
1156very large size. Setting this option makes fio average the each log entry
1157over the specified period of time, reducing the resolution of the log.
1158Defaults to 0.
1159.TP
836bad52 1160.BI disable_lat \fR=\fPbool
02af0988 1161Disable measurements of total latency numbers. Useful only for cutting
ccc2b328 1162back the number of calls to \fBgettimeofday\fR\|(2), as that does impact performance at
901bb994
JA
1163really high IOPS rates. Note that to really get rid of a large amount of these
1164calls, this option must be used with disable_slat and disable_bw as well.
1165.TP
836bad52 1166.BI disable_clat \fR=\fPbool
c95f9daf 1167Disable measurements of completion latency numbers. See \fBdisable_lat\fR.
02af0988 1168.TP
836bad52 1169.BI disable_slat \fR=\fPbool
02af0988 1170Disable measurements of submission latency numbers. See \fBdisable_lat\fR.
901bb994 1171.TP
836bad52 1172.BI disable_bw_measurement \fR=\fPbool
02af0988 1173Disable measurements of throughput/bandwidth numbers. See \fBdisable_lat\fR.
d60e92d1 1174.TP
f7fa2653 1175.BI lockmem \fR=\fPint
d60e92d1 1176Pin the specified amount of memory with \fBmlock\fR\|(2). Can be used to
81c6b6cd 1177simulate a smaller amount of memory. The amount specified is per worker.
d60e92d1
AC
1178.TP
1179.BI exec_prerun \fR=\fPstr
1180Before running the job, execute the specified command with \fBsystem\fR\|(3).
ce486495
EV
1181.RS
1182Output is redirected in a file called \fBjobname.prerun.txt\fR
1183.RE
d60e92d1
AC
1184.TP
1185.BI exec_postrun \fR=\fPstr
1186Same as \fBexec_prerun\fR, but the command is executed after the job completes.
ce486495
EV
1187.RS
1188Output is redirected in a file called \fBjobname.postrun.txt\fR
1189.RE
d60e92d1
AC
1190.TP
1191.BI ioscheduler \fR=\fPstr
1192Attempt to switch the device hosting the file to the specified I/O scheduler.
1193.TP
1194.BI cpuload \fR=\fPint
1195If the job is a CPU cycle-eater, attempt to use the specified percentage of
1196CPU cycles.
1197.TP
1198.BI cpuchunks \fR=\fPint
1199If the job is a CPU cycle-eater, split the load into cycles of the
1200given time in milliseconds.
1201.TP
1202.BI disk_util \fR=\fPbool
d1429b5c 1203Generate disk utilization statistics if the platform supports it. Default: true.
901bb994 1204.TP
23893646
JA
1205.BI clocksource \fR=\fPstr
1206Use the given clocksource as the base of timing. The supported options are:
1207.RS
1208.TP
1209.B gettimeofday
ccc2b328 1210\fBgettimeofday\fR\|(2)
23893646
JA
1211.TP
1212.B clock_gettime
ccc2b328 1213\fBclock_gettime\fR\|(2)
23893646
JA
1214.TP
1215.B cpu
1216Internal CPU clock source
1217.TP
1218.RE
1219.P
1220\fBcpu\fR is the preferred clocksource if it is reliable, as it is very fast
1221(and fio is heavy on time calls). Fio will automatically use this clocksource
1222if it's supported and considered reliable on the system it is running on,
1223unless another clocksource is specifically set. For x86/x86-64 CPUs, this
1224means supporting TSC Invariant.
1225.TP
901bb994 1226.BI gtod_reduce \fR=\fPbool
ccc2b328 1227Enable all of the \fBgettimeofday\fR\|(2) reducing options (disable_clat, disable_slat,
901bb994 1228disable_bw) plus reduce precision of the timeout somewhat to really shrink the
ccc2b328 1229\fBgettimeofday\fR\|(2) call count. With this option enabled, we only do about 0.4% of
901bb994
JA
1230the gtod() calls we would have done if all time keeping was enabled.
1231.TP
1232.BI gtod_cpu \fR=\fPint
1233Sometimes it's cheaper to dedicate a single thread of execution to just getting
1234the current time. Fio (and databases, for instance) are very intensive on
ccc2b328 1235\fBgettimeofday\fR\|(2) calls. With this option, you can set one CPU aside for doing
901bb994
JA
1236nothing but logging current time to a shared memory location. Then the other
1237threads/processes that run IO workloads need only copy that segment, instead of
ccc2b328 1238entering the kernel with a \fBgettimeofday\fR\|(2) call. The CPU set aside for doing
901bb994
JA
1239these time calls will be excluded from other uses. Fio will manually clear it
1240from the CPU mask of other jobs.
f2bba182 1241.TP
8b28bd41
DM
1242.BI ignore_error \fR=\fPstr
1243Sometimes you want to ignore some errors during test in that case you can specify
1244error list for each error type.
1245.br
1246ignore_error=READ_ERR_LIST,WRITE_ERR_LIST,VERIFY_ERR_LIST
1247.br
1248errors for given error type is separated with ':'.
1249Error may be symbol ('ENOSPC', 'ENOMEM') or an integer.
1250.br
1251Example: ignore_error=EAGAIN,ENOSPC:122 .
1252.br
1253This option will ignore EAGAIN from READ, and ENOSPC and 122(EDQUOT) from WRITE.
1254.TP
1255.BI error_dump \fR=\fPbool
1256If set dump every error even if it is non fatal, true by default. If disabled
1257only fatal error will be dumped
1258.TP
fa769d44
SW
1259.BI profile \fR=\fPstr
1260Select a specific builtin performance test.
1261.TP
a696fa2a
JA
1262.BI cgroup \fR=\fPstr
1263Add job to this control group. If it doesn't exist, it will be created.
6adb38a1
JA
1264The system must have a mounted cgroup blkio mount point for this to work. If
1265your system doesn't have it mounted, you can do so with:
1266
5982a925 1267# mount \-t cgroup \-o blkio none /cgroup
a696fa2a
JA
1268.TP
1269.BI cgroup_weight \fR=\fPint
1270Set the weight of the cgroup to this value. See the documentation that comes
1271with the kernel, allowed values are in the range of 100..1000.
e0b0d892 1272.TP
7de87099
VG
1273.BI cgroup_nodelete \fR=\fPbool
1274Normally fio will delete the cgroups it has created after the job completion.
1275To override this behavior and to leave cgroups around after the job completion,
1276set cgroup_nodelete=1. This can be useful if one wants to inspect various
1277cgroup files after job completion. Default: false
1278.TP
e0b0d892
JA
1279.BI uid \fR=\fPint
1280Instead of running as the invoking user, set the user ID to this value before
1281the thread/process does any work.
1282.TP
1283.BI gid \fR=\fPint
1284Set group ID, see \fBuid\fR.
83349190 1285.TP
fa769d44
SW
1286.BI unit_base \fR=\fPint
1287Base unit for reporting. Allowed values are:
1288.RS
1289.TP
1290.B 0
1291Use auto-detection (default).
1292.TP
1293.B 8
1294Byte based.
1295.TP
1296.B 1
1297Bit based.
1298.RE
1299.P
1300.TP
9e684a49
DE
1301.BI flow_id \fR=\fPint
1302The ID of the flow. If not specified, it defaults to being a global flow. See
1303\fBflow\fR.
1304.TP
1305.BI flow \fR=\fPint
1306Weight in token-based flow control. If this value is used, then there is a
1307\fBflow counter\fR which is used to regulate the proportion of activity between
1308two or more jobs. fio attempts to keep this flow counter near zero. The
1309\fBflow\fR parameter stands for how much should be added or subtracted to the
1310flow counter on each iteration of the main I/O loop. That is, if one job has
1311\fBflow=8\fR and another job has \fBflow=-1\fR, then there will be a roughly
13121:8 ratio in how much one runs vs the other.
1313.TP
1314.BI flow_watermark \fR=\fPint
1315The maximum value that the absolute value of the flow counter is allowed to
1316reach before the job must wait for a lower value of the counter.
1317.TP
1318.BI flow_sleep \fR=\fPint
1319The period of time, in microseconds, to wait after the flow watermark has been
1320exceeded before retrying operations
1321.TP
83349190
YH
1322.BI clat_percentiles \fR=\fPbool
1323Enable the reporting of percentiles of completion latencies.
1324.TP
1325.BI percentile_list \fR=\fPfloat_list
1326Overwrite the default list of percentiles for completion
1327latencies. Each number is a floating number in the range (0,100], and
1328the maximum length of the list is 20. Use ':' to separate the
3eb07285 1329numbers. For example, \-\-percentile_list=99.5:99.9 will cause fio to
83349190
YH
1330report the values of completion latency below which 99.5% and 99.9% of
1331the observed latencies fell, respectively.
de890a1e
SL
1332.SS "Ioengine Parameters List"
1333Some parameters are only valid when a specific ioengine is in use. These are
1334used identically to normal parameters, with the caveat that when used on the
1335command line, the must come after the ioengine that defines them is selected.
1336.TP
e4585935
JA
1337.BI (cpu)cpuload \fR=\fPint
1338Attempt to use the specified percentage of CPU cycles.
1339.TP
1340.BI (cpu)cpuchunks \fR=\fPint
1341Split the load into cycles of the given time. In microseconds.
1342.TP
de890a1e
SL
1343.BI (libaio)userspace_reap
1344Normally, with the libaio engine in use, fio will use
1345the io_getevents system call to reap newly returned events.
1346With this flag turned on, the AIO ring will be read directly
1347from user-space to reap events. The reaping mode is only
1348enabled when polling for a minimum of 0 events (eg when
1349iodepth_batch_complete=0).
1350.TP
1351.BI (net,netsplice)hostname \fR=\fPstr
1352The host name or IP address to use for TCP or UDP based IO.
1353If the job is a TCP listener or UDP reader, the hostname is not
b511c9aa 1354used and must be omitted unless it is a valid UDP multicast address.
de890a1e
SL
1355.TP
1356.BI (net,netsplice)port \fR=\fPint
1357The TCP or UDP port to bind to or connect to.
1358.TP
b93b6a2e
SB
1359.BI (net,netsplice)interface \fR=\fPstr
1360The IP address of the network interface used to send or receive UDP multicast
1361packets.
1362.TP
d3a623de
SB
1363.BI (net,netsplice)ttl \fR=\fPint
1364Time-to-live value for outgoing UDP multicast packets. Default: 1
1365.TP
1d360ffb
JA
1366.BI (net,netsplice)nodelay \fR=\fPbool
1367Set TCP_NODELAY on TCP connections.
1368.TP
de890a1e
SL
1369.BI (net,netsplice)protocol \fR=\fPstr "\fR,\fP proto" \fR=\fPstr
1370The network protocol to use. Accepted values are:
1371.RS
1372.RS
1373.TP
1374.B tcp
1375Transmission control protocol
1376.TP
49ccb8c1
JA
1377.B tcpv6
1378Transmission control protocol V6
1379.TP
de890a1e 1380.B udp
f5cc3d0e 1381User datagram protocol
de890a1e 1382.TP
49ccb8c1
JA
1383.B udpv6
1384User datagram protocol V6
1385.TP
de890a1e
SL
1386.B unix
1387UNIX domain socket
1388.RE
1389.P
1390When the protocol is TCP or UDP, the port must also be given,
1391as well as the hostname if the job is a TCP listener or UDP
1392reader. For unix sockets, the normal filename option should be
1393used and the port is invalid.
1394.RE
1395.TP
1396.BI (net,netsplice)listen
1397For TCP network connections, tell fio to listen for incoming
1398connections rather than initiating an outgoing connection. The
1399hostname must be omitted if this option is used.
d54fce84 1400.TP
7aeb1e94 1401.BI (net, pingpong) \fR=\fPbool
cecbfd47 1402Normally a network writer will just continue writing data, and a network reader
7aeb1e94
JA
1403will just consume packages. If pingpong=1 is set, a writer will send its normal
1404payload to the reader, then wait for the reader to send the same payload back.
1405This allows fio to measure network latencies. The submission and completion
1406latencies then measure local time spent sending or receiving, and the
1407completion latency measures how long it took for the other end to receive and
b511c9aa
SB
1408send back. For UDP multicast traffic pingpong=1 should only be set for a single
1409reader when multiple readers are listening to the same address.
7aeb1e94 1410.TP
d54fce84
DM
1411.BI (e4defrag,donorname) \fR=\fPstr
1412File will be used as a block donor (swap extents between files)
1413.TP
1414.BI (e4defrag,inplace) \fR=\fPint
1415Configure donor file block allocation strategy
1416.RS
1417.BI 0(default) :
1418Preallocate donor's file on init
1419.TP
1420.BI 1:
cecbfd47 1421allocate space immediately inside defragment event, and free right after event
d54fce84 1422.RE
0d978694
DAG
1423.TP
1424.BI (rbd)rbdname \fR=\fPstr
1425Specifies the name of the RBD.
1426.TP
1427.BI (rbd)pool \fR=\fPstr
1428Specifies the name of the Ceph pool containing the RBD.
1429.TP
1430.BI (rbd)clientname \fR=\fPstr
1431Specifies the username (without the 'client.' prefix) used to access the Ceph cluster.
d60e92d1 1432.SH OUTPUT
d1429b5c
AC
1433While running, \fBfio\fR will display the status of the created jobs. For
1434example:
d60e92d1 1435.RS
d1429b5c 1436.P
d60e92d1
AC
1437Threads: 1: [_r] [24.8% done] [ 13509/ 8334 kb/s] [eta 00h:01m:31s]
1438.RE
1439.P
d1429b5c
AC
1440The characters in the first set of brackets denote the current status of each
1441threads. The possible values are:
1442.P
1443.PD 0
d60e92d1
AC
1444.RS
1445.TP
1446.B P
1447Setup but not started.
1448.TP
1449.B C
1450Thread created.
1451.TP
1452.B I
1453Initialized, waiting.
1454.TP
1455.B R
1456Running, doing sequential reads.
1457.TP
1458.B r
1459Running, doing random reads.
1460.TP
1461.B W
1462Running, doing sequential writes.
1463.TP
1464.B w
1465Running, doing random writes.
1466.TP
1467.B M
1468Running, doing mixed sequential reads/writes.
1469.TP
1470.B m
1471Running, doing mixed random reads/writes.
1472.TP
1473.B F
1474Running, currently waiting for \fBfsync\fR\|(2).
1475.TP
1476.B V
1477Running, verifying written data.
1478.TP
1479.B E
1480Exited, not reaped by main thread.
1481.TP
1482.B \-
1483Exited, thread reaped.
1484.RE
d1429b5c 1485.PD
d60e92d1
AC
1486.P
1487The second set of brackets shows the estimated completion percentage of
1488the current group. The third set shows the read and write I/O rate,
1489respectively. Finally, the estimated run time of the job is displayed.
1490.P
1491When \fBfio\fR completes (or is interrupted by Ctrl-C), it will show data
1492for each thread, each group of threads, and each disk, in that order.
1493.P
1494Per-thread statistics first show the threads client number, group-id, and
1495error code. The remaining figures are as follows:
1496.RS
d60e92d1
AC
1497.TP
1498.B io
1499Number of megabytes of I/O performed.
1500.TP
1501.B bw
1502Average data rate (bandwidth).
1503.TP
1504.B runt
1505Threads run time.
1506.TP
1507.B slat
1508Submission latency minimum, maximum, average and standard deviation. This is
1509the time it took to submit the I/O.
1510.TP
1511.B clat
1512Completion latency minimum, maximum, average and standard deviation. This
1513is the time between submission and completion.
1514.TP
1515.B bw
1516Bandwidth minimum, maximum, percentage of aggregate bandwidth received, average
1517and standard deviation.
1518.TP
1519.B cpu
1520CPU usage statistics. Includes user and system time, number of context switches
1521this thread went through and number of major and minor page faults.
1522.TP
1523.B IO depths
1524Distribution of I/O depths. Each depth includes everything less than (or equal)
1525to it, but greater than the previous depth.
1526.TP
1527.B IO issued
1528Number of read/write requests issued, and number of short read/write requests.
1529.TP
1530.B IO latencies
1531Distribution of I/O completion latencies. The numbers follow the same pattern
1532as \fBIO depths\fR.
1533.RE
d60e92d1
AC
1534.P
1535The group statistics show:
d1429b5c 1536.PD 0
d60e92d1
AC
1537.RS
1538.TP
1539.B io
1540Number of megabytes I/O performed.
1541.TP
1542.B aggrb
1543Aggregate bandwidth of threads in the group.
1544.TP
1545.B minb
1546Minimum average bandwidth a thread saw.
1547.TP
1548.B maxb
1549Maximum average bandwidth a thread saw.
1550.TP
1551.B mint
d1429b5c 1552Shortest runtime of threads in the group.
d60e92d1
AC
1553.TP
1554.B maxt
1555Longest runtime of threads in the group.
1556.RE
d1429b5c 1557.PD
d60e92d1
AC
1558.P
1559Finally, disk statistics are printed with reads first:
d1429b5c 1560.PD 0
d60e92d1
AC
1561.RS
1562.TP
1563.B ios
1564Number of I/Os performed by all groups.
1565.TP
1566.B merge
1567Number of merges in the I/O scheduler.
1568.TP
1569.B ticks
1570Number of ticks we kept the disk busy.
1571.TP
1572.B io_queue
1573Total time spent in the disk queue.
1574.TP
1575.B util
1576Disk utilization.
1577.RE
d1429b5c 1578.PD
8423bd11
JA
1579.P
1580It is also possible to get fio to dump the current output while it is
1581running, without terminating the job. To do that, send fio the \fBUSR1\fR
1582signal.
d60e92d1 1583.SH TERSE OUTPUT
2b8c71b0
CE
1584If the \fB\-\-minimal\fR / \fB\-\-append-terse\fR options are given, the
1585results will be printed/appended in a semicolon-delimited format suitable for
1586scripted use.
1587A job description (if provided) follows on a new line. Note that the first
525c2bfa
JA
1588number in the line is the version number. If the output has to be changed
1589for some reason, this number will be incremented by 1 to signify that
1590change. The fields are:
d60e92d1
AC
1591.P
1592.RS
5e726d0a 1593.B terse version, fio version, jobname, groupid, error
d60e92d1
AC
1594.P
1595Read status:
1596.RS
312b4af2 1597.B Total I/O \fR(KB)\fP, bandwidth \fR(KB/s)\fP, IOPS, runtime \fR(ms)\fP
d60e92d1
AC
1598.P
1599Submission latency:
1600.RS
1601.B min, max, mean, standard deviation
1602.RE
1603Completion latency:
1604.RS
1605.B min, max, mean, standard deviation
1606.RE
1db92cb6
JA
1607Completion latency percentiles (20 fields):
1608.RS
1609.B Xth percentile=usec
1610.RE
525c2bfa
JA
1611Total latency:
1612.RS
1613.B min, max, mean, standard deviation
1614.RE
d60e92d1
AC
1615Bandwidth:
1616.RS
1617.B min, max, aggregate percentage of total, mean, standard deviation
1618.RE
1619.RE
1620.P
1621Write status:
1622.RS
312b4af2 1623.B Total I/O \fR(KB)\fP, bandwidth \fR(KB/s)\fP, IOPS, runtime \fR(ms)\fP
d60e92d1
AC
1624.P
1625Submission latency:
1626.RS
1627.B min, max, mean, standard deviation
1628.RE
1629Completion latency:
1630.RS
1631.B min, max, mean, standard deviation
1632.RE
1db92cb6
JA
1633Completion latency percentiles (20 fields):
1634.RS
1635.B Xth percentile=usec
1636.RE
525c2bfa
JA
1637Total latency:
1638.RS
1639.B min, max, mean, standard deviation
1640.RE
d60e92d1
AC
1641Bandwidth:
1642.RS
1643.B min, max, aggregate percentage of total, mean, standard deviation
1644.RE
1645.RE
1646.P
d1429b5c 1647CPU usage:
d60e92d1 1648.RS
bd2626f0 1649.B user, system, context switches, major page faults, minor page faults
d60e92d1
AC
1650.RE
1651.P
1652IO depth distribution:
1653.RS
1654.B <=1, 2, 4, 8, 16, 32, >=64
1655.RE
1656.P
562c2d2f 1657IO latency distribution:
d60e92d1 1658.RS
562c2d2f
DN
1659Microseconds:
1660.RS
1661.B <=2, 4, 10, 20, 50, 100, 250, 500, 750, 1000
1662.RE
1663Milliseconds:
1664.RS
1665.B <=2, 4, 10, 20, 50, 100, 250, 500, 750, 1000, 2000, >=2000
1666.RE
1667.RE
1668.P
f2f788dd
JA
1669Disk utilization (1 for each disk used):
1670.RS
1671.B name, read ios, write ios, read merges, write merges, read ticks, write ticks, read in-queue time, write in-queue time, disk utilization percentage
1672.RE
1673.P
5982a925 1674Error Info (dependent on continue_on_error, default off):
562c2d2f
DN
1675.RS
1676.B total # errors, first error code
d60e92d1
AC
1677.RE
1678.P
562c2d2f 1679.B text description (if provided in config - appears on newline)
d60e92d1 1680.RE
49da1240
JA
1681.SH CLIENT / SERVER
1682Normally you would run fio as a stand-alone application on the machine
1683where the IO workload should be generated. However, it is also possible to
1684run the frontend and backend of fio separately. This makes it possible to
1685have a fio server running on the machine(s) where the IO workload should
1686be running, while controlling it from another machine.
1687
1688To start the server, you would do:
1689
1690\fBfio \-\-server=args\fR
1691
1692on that machine, where args defines what fio listens to. The arguments
811826be 1693are of the form 'type:hostname or IP:port'. 'type' is either 'ip' (or ip4)
20c67f10
MS
1694for TCP/IP v4, 'ip6' for TCP/IP v6, or 'sock' for a local unix domain
1695socket. 'hostname' is either a hostname or IP address, and 'port' is the port to
811826be 1696listen to (only valid for TCP/IP, not a local socket). Some examples:
49da1240 1697
e01e9745 16981) fio \-\-server
49da1240
JA
1699
1700 Start a fio server, listening on all interfaces on the default port (8765).
1701
e01e9745 17022) fio \-\-server=ip:hostname,4444
49da1240
JA
1703
1704 Start a fio server, listening on IP belonging to hostname and on port 4444.
1705
e01e9745 17063) fio \-\-server=ip6:::1,4444
811826be
JA
1707
1708 Start a fio server, listening on IPv6 localhost ::1 and on port 4444.
1709
e01e9745 17104) fio \-\-server=,4444
49da1240
JA
1711
1712 Start a fio server, listening on all interfaces on port 4444.
1713
e01e9745 17145) fio \-\-server=1.2.3.4
49da1240
JA
1715
1716 Start a fio server, listening on IP 1.2.3.4 on the default port.
1717
e01e9745 17186) fio \-\-server=sock:/tmp/fio.sock
49da1240
JA
1719
1720 Start a fio server, listening on the local socket /tmp/fio.sock.
1721
1722When a server is running, you can connect to it from a client. The client
1723is run with:
1724
e01e9745 1725fio \-\-local-args \-\-client=server \-\-remote-args <job file(s)>
49da1240 1726
e01e9745
MS
1727where \-\-local-args are arguments that are local to the client where it is
1728running, 'server' is the connect string, and \-\-remote-args and <job file(s)>
49da1240
JA
1729are sent to the server. The 'server' string follows the same format as it
1730does on the server side, to allow IP/hostname/socket and port strings.
1731You can connect to multiple clients as well, to do that you could run:
1732
e01e9745 1733fio \-\-client=server2 \-\-client=server2 <job file(s)>
d60e92d1 1734.SH AUTHORS
49da1240 1735
d60e92d1 1736.B fio
aa58d252
JA
1737was written by Jens Axboe <jens.axboe@oracle.com>,
1738now Jens Axboe <jaxboe@fusionio.com>.
d1429b5c
AC
1739.br
1740This man page was written by Aaron Carroll <aaronc@cse.unsw.edu.au> based
d60e92d1
AC
1741on documentation by Jens Axboe.
1742.SH "REPORTING BUGS"
482900c9 1743Report bugs to the \fBfio\fR mailing list <fio@vger.kernel.org>.
d1429b5c 1744See \fBREADME\fR.
d60e92d1 1745.SH "SEE ALSO"
d1429b5c
AC
1746For further documentation see \fBHOWTO\fR and \fBREADME\fR.
1747.br
1748Sample jobfiles are available in the \fBexamples\fR directory.
d60e92d1 1749