configure: add TCP_NODELAY check
[fio.git] / fio.1
CommitLineData
d60e92d1
AC
1.TH fio 1 "September 2007" "User Manual"
2.SH NAME
3fio \- flexible I/O tester
4.SH SYNOPSIS
5.B fio
6[\fIoptions\fR] [\fIjobfile\fR]...
7.SH DESCRIPTION
8.B fio
9is a tool that will spawn a number of threads or processes doing a
10particular type of I/O action as specified by the user.
11The typical use of fio is to write a job file matching the I/O load
12one wants to simulate.
13.SH OPTIONS
14.TP
49da1240
JA
15.BI \-\-debug \fR=\fPtype
16Enable verbose tracing of various fio actions. May be `all' for all types
17or individual types separated by a comma (eg \-\-debug=io,file). `help' will
18list all available tracing options.
19.TP
d60e92d1
AC
20.BI \-\-output \fR=\fPfilename
21Write output to \fIfilename\fR.
22.TP
b2cecdc2 23.BI \-\-runtime \fR=\fPruntime
24Limit run time to \fIruntime\fR seconds.
d60e92d1
AC
25.TP
26.B \-\-latency\-log
27Generate per-job latency logs.
28.TP
29.B \-\-bandwidth\-log
30Generate per-job bandwidth logs.
31.TP
32.B \-\-minimal
d1429b5c 33Print statistics in a terse, semicolon-delimited format.
d60e92d1 34.TP
49da1240
JA
35.B \-\-version
36Display version information and exit.
37.TP
065248bf 38.BI \-\-terse\-version \fR=\fPversion
4d658652 39Set terse version output format (Current version 3, or older version 2).
49da1240
JA
40.TP
41.B \-\-help
42Display usage information and exit.
43.TP
44.BI \-\-cmdhelp \fR=\fPcommand
45Print help information for \fIcommand\fR. May be `all' for all commands.
46.TP
de890a1e
SL
47.BI \-\-enghelp \fR=\fPioengine[,command]
48List all commands defined by \fIioengine\fR, or print help for \fIcommand\fR defined by \fIioengine\fR.
49.TP
d60e92d1
AC
50.BI \-\-showcmd \fR=\fPjobfile
51Convert \fIjobfile\fR to a set of command-line options.
52.TP
d60e92d1
AC
53.BI \-\-eta \fR=\fPwhen
54Specifies when real-time ETA estimate should be printed. \fIwhen\fR may
55be one of `always', `never' or `auto'.
56.TP
49da1240
JA
57.BI \-\-readonly
58Turn on safety read-only checks, preventing any attempted write.
59.TP
c0a5d35e 60.BI \-\-section \fR=\fPsec
49da1240 61Only run section \fIsec\fR from job file. Multiple of these options can be given, adding more sections to run.
c0a5d35e 62.TP
49da1240
JA
63.BI \-\-alloc\-size \fR=\fPkb
64Set the internal smalloc pool size to \fIkb\fP kilobytes.
d60e92d1 65.TP
49da1240
JA
66.BI \-\-warnings\-fatal
67All fio parser warnings are fatal, causing fio to exit with an error.
9183788d 68.TP
49da1240 69.BI \-\-max\-jobs \fR=\fPnr
57e118a2 70Set the maximum allowed number of jobs (threads/processes) to support.
d60e92d1 71.TP
49da1240
JA
72.BI \-\-server \fR=\fPargs
73Start a backend server, with \fIargs\fP specifying what to listen to. See client/server section.
f57a9c59 74.TP
49da1240
JA
75.BI \-\-daemonize \fR=\fPpidfile
76Background a fio server, writing the pid to the given pid file.
77.TP
78.BI \-\-client \fR=\fPhost
79Instead of running the jobs locally, send and run them on the given host.
f2a2ce0e
HL
80.TP
81.BI \-\-idle\-prof \fR=\fPoption
82Report cpu idleness on a system or percpu basis (\fIoption\fP=system,percpu) or run unit work calibration only (\fIoption\fP=calibrate).
d60e92d1
AC
83.SH "JOB FILE FORMAT"
84Job files are in `ini' format. They consist of one or more
85job definitions, which begin with a job name in square brackets and
86extend to the next job name. The job name can be any ASCII string
87except `global', which has a special meaning. Following the job name is
88a sequence of zero or more parameters, one per line, that define the
89behavior of the job. Any line starting with a `;' or `#' character is
d1429b5c 90considered a comment and ignored.
d9956b64
AC
91.P
92If \fIjobfile\fR is specified as `-', the job file will be read from
93standard input.
d60e92d1
AC
94.SS "Global Section"
95The global section contains default parameters for jobs specified in the
96job file. A job is only affected by global sections residing above it,
97and there may be any number of global sections. Specific job definitions
98may override any parameter set in global sections.
99.SH "JOB PARAMETERS"
100.SS Types
101Some parameters may take arguments of a specific type. The types used are:
102.TP
103.I str
104String: a sequence of alphanumeric characters.
105.TP
106.I int
d60e92d1 107SI integer: a whole number, possibly containing a suffix denoting the base unit
b09da8fa
JA
108of the value. Accepted suffixes are `k', 'M', 'G', 'T', and 'P', denoting
109kilo (1024), mega (1024^2), giga (1024^3), tera (1024^4), and peta (1024^5)
110respectively. The suffix is not case sensitive. If prefixed with '0x', the
5982a925
MS
111value is assumed to be base 16 (hexadecimal). A suffix may include a trailing 'b',
112for instance 'kb' is identical to 'k'. You can specify a base 10 value
57fc29fa
JA
113by using 'KiB', 'MiB', 'GiB', etc. This is useful for disk drives where
114values are often given in base 10 values. Specifying '30GiB' will get you
11530*1000^3 bytes.
d60e92d1
AC
116.TP
117.I bool
118Boolean: a true or false value. `0' denotes false, `1' denotes true.
119.TP
120.I irange
121Integer range: a range of integers specified in the format
d1429b5c
AC
122\fIlower\fR:\fIupper\fR or \fIlower\fR\-\fIupper\fR. \fIlower\fR and
123\fIupper\fR may contain a suffix as described above. If an option allows two
124sets of ranges, they are separated with a `,' or `/' character. For example:
125`8\-8k/8M\-4G'.
83349190
YH
126.TP
127.I float_list
128List of floating numbers: A list of floating numbers, separated by
129a ':' charcater.
d60e92d1
AC
130.SS "Parameter List"
131.TP
132.BI name \fR=\fPstr
d9956b64 133May be used to override the job name. On the command line, this parameter
d60e92d1
AC
134has the special purpose of signalling the start of a new job.
135.TP
136.BI description \fR=\fPstr
137Human-readable description of the job. It is printed when the job is run, but
138otherwise has no special purpose.
139.TP
140.BI directory \fR=\fPstr
141Prefix filenames with this directory. Used to place files in a location other
142than `./'.
143.TP
144.BI filename \fR=\fPstr
145.B fio
146normally makes up a file name based on the job name, thread number, and file
d1429b5c 147number. If you want to share files between threads in a job or several jobs,
de890a1e
SL
148specify a \fIfilename\fR for each of them to override the default.
149If the I/O engine is file-based, you can specify
d1429b5c
AC
150a number of files by separating the names with a `:' character. `\-' is a
151reserved name, meaning stdin or stdout, depending on the read/write direction
152set.
d60e92d1 153.TP
3ce9dcaf
JA
154.BI lockfile \fR=\fPstr
155Fio defaults to not locking any files before it does IO to them. If a file or
156file descriptor is shared, fio can serialize IO to that file to make the end
157result consistent. This is usual for emulating real workloads that share files.
158The lock modes are:
159.RS
160.RS
161.TP
162.B none
163No locking. This is the default.
164.TP
165.B exclusive
166Only one thread or process may do IO at the time, excluding all others.
167.TP
168.B readwrite
169Read-write locking on the file. Many readers may access the file at the same
170time, but writes get exclusive access.
171.RE
172.P
173The option may be post-fixed with a lock batch number. If set, then each
174thread/process may do that amount of IOs to the file before giving up the lock.
175Since lock acquisition is expensive, batching the lock/unlocks will speed up IO.
176.RE
177.P
d60e92d1
AC
178.BI opendir \fR=\fPstr
179Recursively open any files below directory \fIstr\fR.
180.TP
181.BI readwrite \fR=\fPstr "\fR,\fP rw" \fR=\fPstr
182Type of I/O pattern. Accepted values are:
183.RS
184.RS
185.TP
186.B read
d1429b5c 187Sequential reads.
d60e92d1
AC
188.TP
189.B write
d1429b5c 190Sequential writes.
d60e92d1
AC
191.TP
192.B randread
d1429b5c 193Random reads.
d60e92d1
AC
194.TP
195.B randwrite
d1429b5c 196Random writes.
d60e92d1 197.TP
10b023db 198.B rw, readwrite
d1429b5c 199Mixed sequential reads and writes.
d60e92d1
AC
200.TP
201.B randrw
d1429b5c 202Mixed random reads and writes.
d60e92d1
AC
203.RE
204.P
38dad62d
JA
205For mixed I/O, the default split is 50/50. For certain types of io the result
206may still be skewed a bit, since the speed may be different. It is possible to
3b7fa9ec 207specify a number of IO's to do before getting a new offset, this is done by
38dad62d
JA
208appending a `:\fI<nr>\fR to the end of the string given. For a random read, it
209would look like \fBrw=randread:8\fR for passing in an offset modifier with a
059b0802
JA
210value of 8. If the postfix is used with a sequential IO pattern, then the value
211specified will be added to the generated offset for each IO. For instance,
212using \fBrw=write:4k\fR will skip 4k for every write. It turns sequential IO
213into sequential IO with holes. See the \fBrw_sequencer\fR option.
d60e92d1
AC
214.RE
215.TP
38dad62d
JA
216.BI rw_sequencer \fR=\fPstr
217If an offset modifier is given by appending a number to the \fBrw=<str>\fR line,
218then this option controls how that number modifies the IO offset being
219generated. Accepted values are:
220.RS
221.RS
222.TP
223.B sequential
224Generate sequential offset
225.TP
226.B identical
227Generate the same offset
228.RE
229.P
230\fBsequential\fR is only useful for random IO, where fio would normally
231generate a new random offset for every IO. If you append eg 8 to randread, you
232would get a new random offset for every 8 IO's. The result would be a seek for
233only every 8 IO's, instead of for every IO. Use \fBrw=randread:8\fR to specify
234that. As sequential IO is already sequential, setting \fBsequential\fR for that
235would not result in any differences. \fBidentical\fR behaves in a similar
236fashion, except it sends the same offset 8 number of times before generating a
237new offset.
238.RE
239.P
240.TP
90fef2d1
JA
241.BI kb_base \fR=\fPint
242The base unit for a kilobyte. The defacto base is 2^10, 1024. Storage
243manufacturers like to use 10^3 or 1000 as a base ten unit instead, for obvious
244reasons. Allow values are 1024 or 1000, with 1024 being the default.
245.TP
771e58be
JA
246.BI unified_rw_reporting \fR=\fPbool
247Fio normally reports statistics on a per data direction basis, meaning that
248read, write, and trim are accounted and reported separately. If this option is
249set, the fio will sum the results and report them as "mixed" instead.
250.TP
d60e92d1
AC
251.BI randrepeat \fR=\fPbool
252Seed the random number generator in a predictable way so results are repeatable
d1429b5c 253across runs. Default: true.
d60e92d1 254.TP
2615cc4b
JA
255.BI use_os_rand \fR=\fPbool
256Fio can either use the random generator supplied by the OS to generator random
257offsets, or it can use it's own internal generator (based on Tausworthe).
258Default is to use the internal generator, which is often of better quality and
259faster. Default: false.
260.TP
a596f047
EG
261.BI fallocate \fR=\fPstr
262Whether pre-allocation is performed when laying down files. Accepted values
263are:
264.RS
265.RS
266.TP
267.B none
268Do not pre-allocate space.
269.TP
270.B posix
271Pre-allocate via posix_fallocate().
272.TP
273.B keep
274Pre-allocate via fallocate() with FALLOC_FL_KEEP_SIZE set.
275.TP
276.B 0
277Backward-compatible alias for 'none'.
278.TP
279.B 1
280Backward-compatible alias for 'posix'.
281.RE
282.P
283May not be available on all supported platforms. 'keep' is only
284available on Linux. If using ZFS on Solaris this must be set to 'none'
285because ZFS doesn't support it. Default: 'posix'.
286.RE
7bc8c2cf 287.TP
d60e92d1 288.BI fadvise_hint \fR=\fPbool
23a7b043 289Use of \fIposix_fadvise\fR\|(2) to advise the kernel what I/O patterns
d1429b5c 290are likely to be issued. Default: true.
d60e92d1 291.TP
f7fa2653 292.BI size \fR=\fPint
d60e92d1
AC
293Total size of I/O for this job. \fBfio\fR will run until this many bytes have
294been transfered, unless limited by other options (\fBruntime\fR, for instance).
d7c8be03 295Unless \fBnrfiles\fR and \fBfilesize\fR options are given, this amount will be
d6667268
JA
296divided between the available files for the job. If not set, fio will use the
297full size of the given files or devices. If the the files do not exist, size
7bb59102
JA
298must be given. It is also possible to give size as a percentage between 1 and
299100. If size=20% is given, fio will use 20% of the full size of the given files
300or devices.
d60e92d1 301.TP
74586c1e 302.BI fill_device \fR=\fPbool "\fR,\fB fill_fs" \fR=\fPbool
3ce9dcaf
JA
303Sets size to something really large and waits for ENOSPC (no space left on
304device) as the terminating condition. Only makes sense with sequential write.
305For a read workload, the mount point will be filled first then IO started on
4f12432e
JA
306the result. This option doesn't make sense if operating on a raw device node,
307since the size of that is already known by the file system. Additionally,
308writing beyond end-of-device will not return ENOSPC there.
3ce9dcaf 309.TP
d60e92d1
AC
310.BI filesize \fR=\fPirange
311Individual file sizes. May be a range, in which case \fBfio\fR will select sizes
d1429b5c
AC
312for files at random within the given range, limited to \fBsize\fR in total (if
313that is given). If \fBfilesize\fR is not specified, each created file is the
314same size.
d60e92d1 315.TP
f7fa2653 316.BI blocksize \fR=\fPint[,int] "\fR,\fB bs" \fR=\fPint[,int]
d60e92d1 317Block size for I/O units. Default: 4k. Values for reads and writes can be
656ebab7 318specified separately in the format \fIread\fR,\fIwrite\fR, either of
d60e92d1
AC
319which may be empty to leave that value at its default.
320.TP
9183788d 321.BI blocksize_range \fR=\fPirange[,irange] "\fR,\fB bsrange" \fR=\fPirange[,irange]
d1429b5c
AC
322Specify a range of I/O block sizes. The issued I/O unit will always be a
323multiple of the minimum size, unless \fBblocksize_unaligned\fR is set. Applies
9183788d 324to both reads and writes if only one range is given, but can be specified
656ebab7 325separately with a comma seperating the values. Example: bsrange=1k-4k,2k-8k.
9183788d
JA
326Also (see \fBblocksize\fR).
327.TP
328.BI bssplit \fR=\fPstr
329This option allows even finer grained control of the block sizes issued,
330not just even splits between them. With this option, you can weight various
331block sizes for exact control of the issued IO for a job that has mixed
332block sizes. The format of the option is bssplit=blocksize/percentage,
5982a925 333optionally adding as many definitions as needed separated by a colon.
9183788d 334Example: bssplit=4k/10:64k/50:32k/40 would issue 50% 64k blocks, 10% 4k
c83cdd3e
JA
335blocks and 40% 32k blocks. \fBbssplit\fR also supports giving separate
336splits to reads and writes. The format is identical to what the
337\fBbs\fR option accepts, the read and write parts are separated with a
338comma.
d60e92d1
AC
339.TP
340.B blocksize_unaligned\fR,\fP bs_unaligned
d1429b5c
AC
341If set, any size in \fBblocksize_range\fR may be used. This typically won't
342work with direct I/O, as that normally requires sector alignment.
d60e92d1 343.TP
2b7a01d0 344.BI blockalign \fR=\fPint[,int] "\fR,\fB ba" \fR=\fPint[,int]
639ce0f3
MS
345At what boundary to align random IO offsets. Defaults to the same as 'blocksize'
346the minimum blocksize given. Minimum alignment is typically 512b
2b7a01d0
JA
347for using direct IO, though it usually depends on the hardware block size.
348This option is mutually exclusive with using a random map for files, so it
349will turn off that option.
43602667 350.TP
d60e92d1
AC
351.B zero_buffers
352Initialise buffers with all zeros. Default: fill buffers with random data.
353.TP
901bb994
JA
354.B refill_buffers
355If this option is given, fio will refill the IO buffers on every submit. The
356default is to only fill it at init time and reuse that data. Only makes sense
357if zero_buffers isn't specified, naturally. If data verification is enabled,
358refill_buffers is also automatically enabled.
359.TP
fd68418e
JA
360.BI scramble_buffers \fR=\fPbool
361If \fBrefill_buffers\fR is too costly and the target is using data
362deduplication, then setting this option will slightly modify the IO buffer
363contents to defeat normal de-dupe attempts. This is not enough to defeat
364more clever block compression attempts, but it will stop naive dedupe
365of blocks. Default: true.
366.TP
c5751c62
JA
367.BI buffer_compress_percentage \fR=\fPint
368If this is set, then fio will attempt to provide IO buffer content (on WRITEs)
369that compress to the specified level. Fio does this by providing a mix of
370random data and zeroes. Note that this is per block size unit, for file/disk
371wide compression level that matches this setting, you'll also want to set
372\fBrefill_buffers\fR.
373.TP
374.BI buffer_compress_chunk \fR=\fPint
375See \fBbuffer_compress_percentage\fR. This setting allows fio to manage how
376big the ranges of random data and zeroed data is. Without this set, fio will
377provide \fBbuffer_compress_percentage\fR of blocksize random data, followed by
378the remaining zeroed. With this set to some chunk size smaller than the block
379size, fio can alternate random and zeroed data throughout the IO buffer.
380.TP
d60e92d1
AC
381.BI nrfiles \fR=\fPint
382Number of files to use for this job. Default: 1.
383.TP
384.BI openfiles \fR=\fPint
385Number of files to keep open at the same time. Default: \fBnrfiles\fR.
386.TP
387.BI file_service_type \fR=\fPstr
388Defines how files to service are selected. The following types are defined:
389.RS
390.RS
391.TP
392.B random
393Choose a file at random
394.TP
395.B roundrobin
396Round robin over open files (default).
6b7f6851
JA
397.B sequential
398Do each file in the set sequentially.
d60e92d1
AC
399.RE
400.P
401The number of I/Os to issue before switching a new file can be specified by
402appending `:\fIint\fR' to the service type.
403.RE
404.TP
405.BI ioengine \fR=\fPstr
406Defines how the job issues I/O. The following types are defined:
407.RS
408.RS
409.TP
410.B sync
411Basic \fIread\fR\|(2) or \fIwrite\fR\|(2) I/O. \fIfseek\fR\|(2) is used to
412position the I/O location.
413.TP
a31041ea 414.B psync
415Basic \fIpread\fR\|(2) or \fIpwrite\fR\|(2) I/O.
416.TP
9183788d
JA
417.B vsync
418Basic \fIreadv\fR\|(2) or \fIwritev\fR\|(2) I/O. Will emulate queuing by
419coalescing adjacents IOs into a single submission.
420.TP
d60e92d1 421.B libaio
de890a1e 422Linux native asynchronous I/O. This ioengine defines engine specific options.
d60e92d1
AC
423.TP
424.B posixaio
03e20d68
BC
425POSIX asynchronous I/O using \fIaio_read\fR\|(3) and \fIaio_write\fR\|(3).
426.TP
427.B solarisaio
428Solaris native asynchronous I/O.
429.TP
430.B windowsaio
431Windows native asynchronous I/O.
d60e92d1
AC
432.TP
433.B mmap
d1429b5c
AC
434File is memory mapped with \fImmap\fR\|(2) and data copied using
435\fImemcpy\fR\|(3).
d60e92d1
AC
436.TP
437.B splice
d1429b5c
AC
438\fIsplice\fR\|(2) is used to transfer the data and \fIvmsplice\fR\|(2) to
439transfer data from user-space to the kernel.
d60e92d1
AC
440.TP
441.B syslet-rw
442Use the syslet system calls to make regular read/write asynchronous.
443.TP
444.B sg
445SCSI generic sg v3 I/O. May be either synchronous using the SG_IO ioctl, or if
d1429b5c
AC
446the target is an sg character device, we use \fIread\fR\|(2) and
447\fIwrite\fR\|(2) for asynchronous I/O.
d60e92d1
AC
448.TP
449.B null
450Doesn't transfer any data, just pretends to. Mainly used to exercise \fBfio\fR
451itself and for debugging and testing purposes.
452.TP
453.B net
de890a1e
SL
454Transfer over the network. The protocol to be used can be defined with the
455\fBprotocol\fR parameter. Depending on the protocol, \fBfilename\fR,
456\fBhostname\fR, \fBport\fR, or \fBlisten\fR must be specified.
457This ioengine defines engine specific options.
d60e92d1
AC
458.TP
459.B netsplice
460Like \fBnet\fR, but uses \fIsplice\fR\|(2) and \fIvmsplice\fR\|(2) to map data
de890a1e 461and send/receive. This ioengine defines engine specific options.
d60e92d1 462.TP
53aec0a4 463.B cpuio
d60e92d1
AC
464Doesn't transfer any data, but burns CPU cycles according to \fBcpuload\fR and
465\fBcpucycles\fR parameters.
466.TP
467.B guasi
468The GUASI I/O engine is the Generic Userspace Asynchronous Syscall Interface
469approach to asycnronous I/O.
d1429b5c
AC
470.br
471See <http://www.xmailserver.org/guasi\-lib.html>.
d60e92d1 472.TP
21b8aee8 473.B rdma
85286c5c
BVA
474The RDMA I/O engine supports both RDMA memory semantics (RDMA_WRITE/RDMA_READ)
475and channel semantics (Send/Recv) for the InfiniBand, RoCE and iWARP protocols.
21b8aee8 476.TP
d60e92d1
AC
477.B external
478Loads an external I/O engine object file. Append the engine filename as
479`:\fIenginepath\fR'.
d54fce84
DM
480.TP
481.B falloc
482 IO engine that does regular linux native fallocate callt to simulate data
483transfer as fio ioengine
484.br
485 DDIR_READ does fallocate(,mode = FALLOC_FL_KEEP_SIZE,)
486.br
0981fd71 487 DIR_WRITE does fallocate(,mode = 0)
d54fce84
DM
488.br
489 DDIR_TRIM does fallocate(,mode = FALLOC_FL_KEEP_SIZE|FALLOC_FL_PUNCH_HOLE)
490.TP
491.B e4defrag
492IO engine that does regular EXT4_IOC_MOVE_EXT ioctls to simulate defragment activity
493request to DDIR_WRITE event
d60e92d1 494.RE
595e1734 495.P
d60e92d1
AC
496.RE
497.TP
498.BI iodepth \fR=\fPint
8489dae4
SK
499Number of I/O units to keep in flight against the file. Note that increasing
500iodepth beyond 1 will not affect synchronous ioengines (except for small
ee72ca09
JA
501degress when verify_async is in use). Even async engines my impose OS
502restrictions causing the desired depth not to be achieved. This may happen on
503Linux when using libaio and not setting \fBdirect\fR=1, since buffered IO is
504not async on that OS. Keep an eye on the IO depth distribution in the
505fio output to verify that the achieved depth is as expected. Default: 1.
d60e92d1
AC
506.TP
507.BI iodepth_batch \fR=\fPint
508Number of I/Os to submit at once. Default: \fBiodepth\fR.
509.TP
3ce9dcaf
JA
510.BI iodepth_batch_complete \fR=\fPint
511This defines how many pieces of IO to retrieve at once. It defaults to 1 which
512 means that we'll ask for a minimum of 1 IO in the retrieval process from the
513kernel. The IO retrieval will go on until we hit the limit set by
514\fBiodepth_low\fR. If this variable is set to 0, then fio will always check for
515completed events before queuing more IO. This helps reduce IO latency, at the
516cost of more retrieval system calls.
517.TP
d60e92d1
AC
518.BI iodepth_low \fR=\fPint
519Low watermark indicating when to start filling the queue again. Default:
520\fBiodepth\fR.
521.TP
522.BI direct \fR=\fPbool
523If true, use non-buffered I/O (usually O_DIRECT). Default: false.
524.TP
525.BI buffered \fR=\fPbool
526If true, use buffered I/O. This is the opposite of the \fBdirect\fR parameter.
527Default: true.
528.TP
f7fa2653 529.BI offset \fR=\fPint
d60e92d1
AC
530Offset in the file to start I/O. Data before the offset will not be touched.
531.TP
591e9e06
JA
532.BI offset_increment \fR=\fPint
533If this is provided, then the real offset becomes the
534offset + offset_increment * thread_number, where the thread number is a counter
535that starts at 0 and is incremented for each job. This option is useful if
536there are several jobs which are intended to operate on a file in parallel in
537disjoint segments, with even spacing between the starting points.
538.TP
d60e92d1 539.BI fsync \fR=\fPint
d1429b5c
AC
540How many I/Os to perform before issuing an \fBfsync\fR\|(2) of dirty data. If
5410, don't sync. Default: 0.
d60e92d1 542.TP
5f9099ea
JA
543.BI fdatasync \fR=\fPint
544Like \fBfsync\fR, but uses \fBfdatasync\fR\|(2) instead to only sync the
545data parts of the file. Default: 0.
546.TP
e76b1da4
JA
547.BI sync_file_range \fR=\fPstr:int
548Use sync_file_range() for every \fRval\fP number of write operations. Fio will
549track range of writes that have happened since the last sync_file_range() call.
550\fRstr\fP can currently be one or more of:
551.RS
552.TP
553.B wait_before
554SYNC_FILE_RANGE_WAIT_BEFORE
555.TP
556.B write
557SYNC_FILE_RANGE_WRITE
558.TP
559.B wait_after
560SYNC_FILE_RANGE_WRITE
561.TP
562.RE
563.P
564So if you do sync_file_range=wait_before,write:8, fio would use
565\fBSYNC_FILE_RANGE_WAIT_BEFORE | SYNC_FILE_RANGE_WRITE\fP for every 8 writes.
566Also see the sync_file_range(2) man page. This option is Linux specific.
567.TP
d60e92d1 568.BI overwrite \fR=\fPbool
d1429b5c 569If writing, setup the file first and do overwrites. Default: false.
d60e92d1
AC
570.TP
571.BI end_fsync \fR=\fPbool
dbd11ead 572Sync file contents when a write stage has completed. Default: false.
d60e92d1
AC
573.TP
574.BI fsync_on_close \fR=\fPbool
575If true, sync file contents on close. This differs from \fBend_fsync\fR in that
d1429b5c 576it will happen on every close, not just at the end of the job. Default: false.
d60e92d1 577.TP
d60e92d1
AC
578.BI rwmixread \fR=\fPint
579Percentage of a mixed workload that should be reads. Default: 50.
580.TP
581.BI rwmixwrite \fR=\fPint
d1429b5c 582Percentage of a mixed workload that should be writes. If \fBrwmixread\fR and
c35dd7a6
JA
583\fBrwmixwrite\fR are given and do not sum to 100%, the latter of the two
584overrides the first. This may interfere with a given rate setting, if fio is
585asked to limit reads or writes to a certain rate. If that is the case, then
586the distribution may be skewed. Default: 50.
d60e92d1 587.TP
92d42d69
JA
588.BI random_distribution \fR=\fPstr:float
589By default, fio will use a completely uniform random distribution when asked
590to perform random IO. Sometimes it is useful to skew the distribution in
591specific ways, ensuring that some parts of the data is more hot than others.
592Fio includes the following distribution models:
593.RS
594.TP
595.B random
596Uniform random distribution
597.TP
598.B zipf
599Zipf distribution
600.TP
601.B pareto
602Pareto distribution
603.TP
604.RE
605.P
606When using a zipf or pareto distribution, an input value is also needed to
607define the access pattern. For zipf, this is the zipf theta. For pareto,
608it's the pareto power. Fio includes a test program, genzipf, that can be
609used visualize what the given input values will yield in terms of hit rates.
610If you wanted to use zipf with a theta of 1.2, you would use
611random_distribution=zipf:1.2 as the option. If a non-uniform model is used,
612fio will disable use of the random map.
613.TP
d60e92d1
AC
614.B norandommap
615Normally \fBfio\fR will cover every block of the file when doing random I/O. If
616this parameter is given, a new offset will be chosen without looking at past
617I/O history. This parameter is mutually exclusive with \fBverify\fR.
618.TP
744492c9 619.BI softrandommap \fR=\fPbool
3ce9dcaf
JA
620See \fBnorandommap\fR. If fio runs with the random block map enabled and it
621fails to allocate the map, if this option is set it will continue without a
622random block map. As coverage will not be as complete as with random maps, this
623option is disabled by default.
624.TP
e8b1961d
JA
625.BI random_generator \fR=\fPstr
626Fio supports the following engines for generating IO offsets for random IO:
627.RS
628.TP
629.B tausworthe
630Strong 2^88 cycle random number generator
631.TP
632.B lfsr
633Linear feedback shift register generator
634.TP
635.RE
636.P
637Tausworthe is a strong random number generator, but it requires tracking on the
638side if we want to ensure that blocks are only read or written once. LFSR
639guarantees that we never generate the same offset twice, and it's also less
640computationally expensive. It's not a true random generator, however, though
641for IO purposes it's typically good enough. LFSR only works with single block
642sizes, not with workloads that use multiple block sizes. If used with such a
643workload, fio may read or write some blocks multiple times.
644.TP
d60e92d1
AC
645.BI nice \fR=\fPint
646Run job with given nice value. See \fInice\fR\|(2).
647.TP
648.BI prio \fR=\fPint
649Set I/O priority value of this job between 0 (highest) and 7 (lowest). See
650\fIionice\fR\|(1).
651.TP
652.BI prioclass \fR=\fPint
653Set I/O priority class. See \fIionice\fR\|(1).
654.TP
655.BI thinktime \fR=\fPint
656Stall job for given number of microseconds between issuing I/Os.
657.TP
658.BI thinktime_spin \fR=\fPint
659Pretend to spend CPU time for given number of microseconds, sleeping the rest
660of the time specified by \fBthinktime\fR. Only valid if \fBthinktime\fR is set.
661.TP
662.BI thinktime_blocks \fR=\fPint
663Number of blocks to issue before waiting \fBthinktime\fR microseconds.
664Default: 1.
665.TP
666.BI rate \fR=\fPint
c35dd7a6
JA
667Cap bandwidth used by this job. The number is in bytes/sec, the normal postfix
668rules apply. You can use \fBrate\fR=500k to limit reads and writes to 500k each,
669or you can specify read and writes separately. Using \fBrate\fR=1m,500k would
670limit reads to 1MB/sec and writes to 500KB/sec. Capping only reads or writes
671can be done with \fBrate\fR=,500k or \fBrate\fR=500k,. The former will only
672limit writes (to 500KB/sec), the latter will only limit reads.
d60e92d1
AC
673.TP
674.BI ratemin \fR=\fPint
675Tell \fBfio\fR to do whatever it can to maintain at least the given bandwidth.
c35dd7a6
JA
676Failing to meet this requirement will cause the job to exit. The same format
677as \fBrate\fR is used for read vs write separation.
d60e92d1
AC
678.TP
679.BI rate_iops \fR=\fPint
c35dd7a6
JA
680Cap the bandwidth to this number of IOPS. Basically the same as rate, just
681specified independently of bandwidth. The same format as \fBrate\fR is used for
682read vs write seperation. If \fBblocksize\fR is a range, the smallest block
683size is used as the metric.
d60e92d1
AC
684.TP
685.BI rate_iops_min \fR=\fPint
c35dd7a6
JA
686If this rate of I/O is not met, the job will exit. The same format as \fBrate\fR
687is used for read vs write seperation.
d60e92d1
AC
688.TP
689.BI ratecycle \fR=\fPint
690Average bandwidth for \fBrate\fR and \fBratemin\fR over this number of
691milliseconds. Default: 1000ms.
692.TP
15501535
JA
693.BI max_latency \fR=\fPint
694If set, fio will exit the job if it exceeds this maximum latency. It will exit
695with an ETIME error.
696.TP
d60e92d1
AC
697.BI cpumask \fR=\fPint
698Set CPU affinity for this job. \fIint\fR is a bitmask of allowed CPUs the job
699may run on. See \fBsched_setaffinity\fR\|(2).
700.TP
701.BI cpus_allowed \fR=\fPstr
702Same as \fBcpumask\fR, but allows a comma-delimited list of CPU numbers.
703.TP
d0b937ed
YR
704.BI numa_cpu_nodes \fR=\fPstr
705Set this job running on spcified NUMA nodes' CPUs. The arguments allow
706comma delimited list of cpu numbers, A-B ranges, or 'all'.
707.TP
708.BI numa_mem_policy \fR=\fPstr
709Set this job's memory policy and corresponding NUMA nodes. Format of
710the argements:
711.RS
712.TP
713.B <mode>[:<nodelist>]
714.TP
715.B mode
716is one of the following memory policy:
717.TP
718.B default, prefer, bind, interleave, local
719.TP
720.RE
721For \fBdefault\fR and \fBlocal\fR memory policy, no \fBnodelist\fR is
722needed to be specified. For \fBprefer\fR, only one node is
723allowed. For \fBbind\fR and \fBinterleave\fR, \fBnodelist\fR allows
724comma delimited list of numbers, A-B ranges, or 'all'.
725.TP
d60e92d1
AC
726.BI startdelay \fR=\fPint
727Delay start of job for the specified number of seconds.
728.TP
729.BI runtime \fR=\fPint
730Terminate processing after the specified number of seconds.
731.TP
732.B time_based
733If given, run for the specified \fBruntime\fR duration even if the files are
734completely read or written. The same workload will be repeated as many times
735as \fBruntime\fR allows.
736.TP
901bb994
JA
737.BI ramp_time \fR=\fPint
738If set, fio will run the specified workload for this amount of time before
739logging any performance numbers. Useful for letting performance settle before
740logging results, thus minimizing the runtime required for stable results. Note
c35dd7a6
JA
741that the \fBramp_time\fR is considered lead in time for a job, thus it will
742increase the total runtime if a special timeout or runtime is specified.
901bb994 743.TP
d60e92d1
AC
744.BI invalidate \fR=\fPbool
745Invalidate buffer-cache for the file prior to starting I/O. Default: true.
746.TP
747.BI sync \fR=\fPbool
748Use synchronous I/O for buffered writes. For the majority of I/O engines,
d1429b5c 749this means using O_SYNC. Default: false.
d60e92d1
AC
750.TP
751.BI iomem \fR=\fPstr "\fR,\fP mem" \fR=\fPstr
752Allocation method for I/O unit buffer. Allowed values are:
753.RS
754.RS
755.TP
756.B malloc
757Allocate memory with \fImalloc\fR\|(3).
758.TP
759.B shm
760Use shared memory buffers allocated through \fIshmget\fR\|(2).
761.TP
762.B shmhuge
763Same as \fBshm\fR, but use huge pages as backing.
764.TP
765.B mmap
766Use \fImmap\fR\|(2) for allocation. Uses anonymous memory unless a filename
767is given after the option in the format `:\fIfile\fR'.
768.TP
769.B mmaphuge
770Same as \fBmmap\fR, but use huge files as backing.
771.RE
772.P
773The amount of memory allocated is the maximum allowed \fBblocksize\fR for the
774job multiplied by \fBiodepth\fR. For \fBshmhuge\fR or \fBmmaphuge\fR to work,
775the system must have free huge pages allocated. \fBmmaphuge\fR also needs to
2e266ba6
JA
776have hugetlbfs mounted, and \fIfile\fR must point there. At least on Linux,
777huge pages must be manually allocated. See \fB/proc/sys/vm/nr_hugehages\fR
778and the documentation for that. Normally you just need to echo an appropriate
779number, eg echoing 8 will ensure that the OS has 8 huge pages ready for
780use.
d60e92d1
AC
781.RE
782.TP
d392365e 783.BI iomem_align \fR=\fPint "\fR,\fP mem_align" \fR=\fPint
d529ee19
JA
784This indiciates the memory alignment of the IO memory buffers. Note that the
785given alignment is applied to the first IO unit buffer, if using \fBiodepth\fR
786the alignment of the following buffers are given by the \fBbs\fR used. In
787other words, if using a \fBbs\fR that is a multiple of the page sized in the
788system, all buffers will be aligned to this value. If using a \fBbs\fR that
789is not page aligned, the alignment of subsequent IO memory buffers is the
790sum of the \fBiomem_align\fR and \fBbs\fR used.
791.TP
f7fa2653 792.BI hugepage\-size \fR=\fPint
d60e92d1 793Defines the size of a huge page. Must be at least equal to the system setting.
b22989b9 794Should be a multiple of 1MB. Default: 4MB.
d60e92d1
AC
795.TP
796.B exitall
797Terminate all jobs when one finishes. Default: wait for each job to finish.
798.TP
799.BI bwavgtime \fR=\fPint
800Average bandwidth calculations over the given time in milliseconds. Default:
801500ms.
802.TP
c8eeb9df
JA
803.BI iopsavgtime \fR=\fPint
804Average IOPS calculations over the given time in milliseconds. Default:
805500ms.
806.TP
d60e92d1 807.BI create_serialize \fR=\fPbool
d1429b5c 808If true, serialize file creation for the jobs. Default: true.
d60e92d1
AC
809.TP
810.BI create_fsync \fR=\fPbool
811\fIfsync\fR\|(2) data file after creation. Default: true.
812.TP
6b7f6851
JA
813.BI create_on_open \fR=\fPbool
814If true, the files are not created until they are opened for IO by the job.
815.TP
25460cf6
JA
816.BI create_only \fR=\fPbool
817If true, fio will only run the setup phase of the job. If files need to be
818laid out or updated on disk, only that will be done. The actual job contents
819are not executed.
820.TP
e9f48479
JA
821.BI pre_read \fR=\fPbool
822If this is given, files will be pre-read into memory before starting the given
823IO operation. This will also clear the \fR \fBinvalidate\fR flag, since it is
9c0d2241
JA
824pointless to pre-read and then drop the cache. This will only work for IO
825engines that are seekable, since they allow you to read the same data
826multiple times. Thus it will not work on eg network or splice IO.
e9f48479 827.TP
d60e92d1
AC
828.BI unlink \fR=\fPbool
829Unlink job files when done. Default: false.
830.TP
831.BI loops \fR=\fPint
832Specifies the number of iterations (runs of the same workload) of this job.
833Default: 1.
834.TP
835.BI do_verify \fR=\fPbool
836Run the verify phase after a write phase. Only valid if \fBverify\fR is set.
837Default: true.
838.TP
839.BI verify \fR=\fPstr
840Method of verifying file contents after each iteration of the job. Allowed
841values are:
842.RS
843.RS
844.TP
b892dc08 845.B md5 crc16 crc32 crc32c crc32c-intel crc64 crc7 sha256 sha512 sha1
0539d758
JA
846Store appropriate checksum in the header of each block. crc32c-intel is
847hardware accelerated SSE4.2 driven, falls back to regular crc32c if
848not supported by the system.
d60e92d1
AC
849.TP
850.B meta
851Write extra information about each I/O (timestamp, block number, etc.). The
996093bb 852block number is verified. See \fBverify_pattern\fR as well.
d60e92d1
AC
853.TP
854.B null
855Pretend to verify. Used for testing internals.
856.RE
b892dc08
JA
857
858This option can be used for repeated burn-in tests of a system to make sure
859that the written data is also correctly read back. If the data direction given
860is a read or random read, fio will assume that it should verify a previously
861written file. If the data direction includes any form of write, the verify will
862be of the newly written data.
d60e92d1
AC
863.RE
864.TP
865.BI verify_sort \fR=\fPbool
866If true, written verify blocks are sorted if \fBfio\fR deems it to be faster to
867read them back in a sorted manner. Default: true.
868.TP
f7fa2653 869.BI verify_offset \fR=\fPint
d60e92d1 870Swap the verification header with data somewhere else in the block before
d1429b5c 871writing. It is swapped back before verifying.
d60e92d1 872.TP
f7fa2653 873.BI verify_interval \fR=\fPint
d60e92d1
AC
874Write the verification header for this number of bytes, which should divide
875\fBblocksize\fR. Default: \fBblocksize\fR.
876.TP
996093bb
JA
877.BI verify_pattern \fR=\fPstr
878If set, fio will fill the io buffers with this pattern. Fio defaults to filling
879with totally random bytes, but sometimes it's interesting to fill with a known
880pattern for io verification purposes. Depending on the width of the pattern,
881fio will fill 1/2/3/4 bytes of the buffer at the time(it can be either a
882decimal or a hex number). The verify_pattern if larger than a 32-bit quantity
883has to be a hex number that starts with either "0x" or "0X". Use with
884\fBverify\fP=meta.
885.TP
d60e92d1
AC
886.BI verify_fatal \fR=\fPbool
887If true, exit the job on the first observed verification failure. Default:
888false.
889.TP
b463e936
JA
890.BI verify_dump \fR=\fPbool
891If set, dump the contents of both the original data block and the data block we
892read off disk to files. This allows later analysis to inspect just what kind of
ef71e317 893data corruption occurred. Off by default.
b463e936 894.TP
e8462bd8
JA
895.BI verify_async \fR=\fPint
896Fio will normally verify IO inline from the submitting thread. This option
897takes an integer describing how many async offload threads to create for IO
898verification instead, causing fio to offload the duty of verifying IO contents
c85c324c
JA
899to one or more separate threads. If using this offload option, even sync IO
900engines can benefit from using an \fBiodepth\fR setting higher than 1, as it
901allows them to have IO in flight while verifies are running.
e8462bd8
JA
902.TP
903.BI verify_async_cpus \fR=\fPstr
904Tell fio to set the given CPU affinity on the async IO verification threads.
905See \fBcpus_allowed\fP for the format used.
906.TP
6f87418f
JA
907.BI verify_backlog \fR=\fPint
908Fio will normally verify the written contents of a job that utilizes verify
909once that job has completed. In other words, everything is written then
910everything is read back and verified. You may want to verify continually
911instead for a variety of reasons. Fio stores the meta data associated with an
912IO block in memory, so for large verify workloads, quite a bit of memory would
092f707f
DN
913be used up holding this meta data. If this option is enabled, fio will write
914only N blocks before verifying these blocks.
6f87418f
JA
915.TP
916.BI verify_backlog_batch \fR=\fPint
917Control how many blocks fio will verify if verify_backlog is set. If not set,
918will default to the value of \fBverify_backlog\fR (meaning the entire queue is
092f707f
DN
919read back and verified). If \fBverify_backlog_batch\fR is less than
920\fBverify_backlog\fR then not all blocks will be verified, if
921\fBverify_backlog_batch\fR is larger than \fBverify_backlog\fR, some blocks
922will be verified more than once.
6f87418f 923.TP
d392365e 924.B stonewall "\fR,\fP wait_for_previous"
5982a925 925Wait for preceding jobs in the job file to exit before starting this one.
d60e92d1
AC
926\fBstonewall\fR implies \fBnew_group\fR.
927.TP
928.B new_group
929Start a new reporting group. If not given, all jobs in a file will be part
930of the same reporting group, unless separated by a stonewall.
931.TP
932.BI numjobs \fR=\fPint
933Number of clones (processes/threads performing the same workload) of this job.
934Default: 1.
935.TP
936.B group_reporting
937If set, display per-group reports instead of per-job when \fBnumjobs\fR is
938specified.
939.TP
940.B thread
941Use threads created with \fBpthread_create\fR\|(3) instead of processes created
942with \fBfork\fR\|(2).
943.TP
f7fa2653 944.BI zonesize \fR=\fPint
d60e92d1
AC
945Divide file into zones of the specified size in bytes. See \fBzoneskip\fR.
946.TP
f7fa2653 947.BI zoneskip \fR=\fPint
d1429b5c 948Skip the specified number of bytes when \fBzonesize\fR bytes of data have been
d60e92d1
AC
949read.
950.TP
951.BI write_iolog \fR=\fPstr
5b42a488
SH
952Write the issued I/O patterns to the specified file. Specify a separate file
953for each job, otherwise the iologs will be interspersed and the file may be
954corrupt.
d60e92d1
AC
955.TP
956.BI read_iolog \fR=\fPstr
957Replay the I/O patterns contained in the specified file generated by
958\fBwrite_iolog\fR, or may be a \fBblktrace\fR binary file.
959.TP
64bbb865
DN
960.BI replay_no_stall \fR=\fPint
961While replaying I/O patterns using \fBread_iolog\fR the default behavior
962attempts to respect timing information between I/Os. Enabling
963\fBreplay_no_stall\fR causes I/Os to be replayed as fast as possible while
964still respecting ordering.
965.TP
d1c46c04
DN
966.BI replay_redirect \fR=\fPstr
967While replaying I/O patterns using \fBread_iolog\fR the default behavior
968is to replay the IOPS onto the major/minor device that each IOP was recorded
969from. Setting \fBreplay_redirect\fR causes all IOPS to be replayed onto the
970single specified device regardless of the device it was recorded from.
971.TP
836bad52 972.BI write_bw_log \fR=\fPstr
901bb994
JA
973If given, write a bandwidth log of the jobs in this job file. Can be used to
974store data of the bandwidth of the jobs in their lifetime. The included
975fio_generate_plots script uses gnuplot to turn these text files into nice
976graphs. See \fBwrite_log_log\fR for behaviour of given filename. For this
977option, the postfix is _bw.log.
d60e92d1 978.TP
836bad52 979.BI write_lat_log \fR=\fPstr
901bb994
JA
980Same as \fBwrite_bw_log\fR, but writes I/O completion latencies. If no
981filename is given with this option, the default filename of "jobname_type.log"
982is used. Even if the filename is given, fio will still append the type of log.
983.TP
c8eeb9df
JA
984.BI write_iops_log \fR=\fPstr
985Same as \fBwrite_bw_log\fR, but writes IOPS. If no filename is given with this
986option, the default filename of "jobname_type.log" is used. Even if the
987filename is given, fio will still append the type of log.
988.TP
b8bc8cba
JA
989.BI log_avg_msec \fR=\fPint
990By default, fio will log an entry in the iops, latency, or bw log for every
991IO that completes. When writing to the disk log, that can quickly grow to a
992very large size. Setting this option makes fio average the each log entry
993over the specified period of time, reducing the resolution of the log.
994Defaults to 0.
995.TP
836bad52 996.BI disable_lat \fR=\fPbool
02af0988 997Disable measurements of total latency numbers. Useful only for cutting
901bb994
JA
998back the number of calls to gettimeofday, as that does impact performance at
999really high IOPS rates. Note that to really get rid of a large amount of these
1000calls, this option must be used with disable_slat and disable_bw as well.
1001.TP
836bad52 1002.BI disable_clat \fR=\fPbool
c95f9daf 1003Disable measurements of completion latency numbers. See \fBdisable_lat\fR.
02af0988 1004.TP
836bad52 1005.BI disable_slat \fR=\fPbool
02af0988 1006Disable measurements of submission latency numbers. See \fBdisable_lat\fR.
901bb994 1007.TP
836bad52 1008.BI disable_bw_measurement \fR=\fPbool
02af0988 1009Disable measurements of throughput/bandwidth numbers. See \fBdisable_lat\fR.
d60e92d1 1010.TP
f7fa2653 1011.BI lockmem \fR=\fPint
d60e92d1
AC
1012Pin the specified amount of memory with \fBmlock\fR\|(2). Can be used to
1013simulate a smaller amount of memory.
1014.TP
1015.BI exec_prerun \fR=\fPstr
1016Before running the job, execute the specified command with \fBsystem\fR\|(3).
1017.TP
1018.BI exec_postrun \fR=\fPstr
1019Same as \fBexec_prerun\fR, but the command is executed after the job completes.
1020.TP
1021.BI ioscheduler \fR=\fPstr
1022Attempt to switch the device hosting the file to the specified I/O scheduler.
1023.TP
1024.BI cpuload \fR=\fPint
1025If the job is a CPU cycle-eater, attempt to use the specified percentage of
1026CPU cycles.
1027.TP
1028.BI cpuchunks \fR=\fPint
1029If the job is a CPU cycle-eater, split the load into cycles of the
1030given time in milliseconds.
1031.TP
1032.BI disk_util \fR=\fPbool
d1429b5c 1033Generate disk utilization statistics if the platform supports it. Default: true.
901bb994 1034.TP
23893646
JA
1035.BI clocksource \fR=\fPstr
1036Use the given clocksource as the base of timing. The supported options are:
1037.RS
1038.TP
1039.B gettimeofday
1040gettimeofday(2)
1041.TP
1042.B clock_gettime
1043clock_gettime(2)
1044.TP
1045.B cpu
1046Internal CPU clock source
1047.TP
1048.RE
1049.P
1050\fBcpu\fR is the preferred clocksource if it is reliable, as it is very fast
1051(and fio is heavy on time calls). Fio will automatically use this clocksource
1052if it's supported and considered reliable on the system it is running on,
1053unless another clocksource is specifically set. For x86/x86-64 CPUs, this
1054means supporting TSC Invariant.
1055.TP
901bb994
JA
1056.BI gtod_reduce \fR=\fPbool
1057Enable all of the gettimeofday() reducing options (disable_clat, disable_slat,
1058disable_bw) plus reduce precision of the timeout somewhat to really shrink the
1059gettimeofday() call count. With this option enabled, we only do about 0.4% of
1060the gtod() calls we would have done if all time keeping was enabled.
1061.TP
1062.BI gtod_cpu \fR=\fPint
1063Sometimes it's cheaper to dedicate a single thread of execution to just getting
1064the current time. Fio (and databases, for instance) are very intensive on
1065gettimeofday() calls. With this option, you can set one CPU aside for doing
1066nothing but logging current time to a shared memory location. Then the other
1067threads/processes that run IO workloads need only copy that segment, instead of
1068entering the kernel with a gettimeofday() call. The CPU set aside for doing
1069these time calls will be excluded from other uses. Fio will manually clear it
1070from the CPU mask of other jobs.
f2bba182 1071.TP
8b28bd41
DM
1072.BI ignore_error \fR=\fPstr
1073Sometimes you want to ignore some errors during test in that case you can specify
1074error list for each error type.
1075.br
1076ignore_error=READ_ERR_LIST,WRITE_ERR_LIST,VERIFY_ERR_LIST
1077.br
1078errors for given error type is separated with ':'.
1079Error may be symbol ('ENOSPC', 'ENOMEM') or an integer.
1080.br
1081Example: ignore_error=EAGAIN,ENOSPC:122 .
1082.br
1083This option will ignore EAGAIN from READ, and ENOSPC and 122(EDQUOT) from WRITE.
1084.TP
1085.BI error_dump \fR=\fPbool
1086If set dump every error even if it is non fatal, true by default. If disabled
1087only fatal error will be dumped
1088.TP
a696fa2a
JA
1089.BI cgroup \fR=\fPstr
1090Add job to this control group. If it doesn't exist, it will be created.
6adb38a1
JA
1091The system must have a mounted cgroup blkio mount point for this to work. If
1092your system doesn't have it mounted, you can do so with:
1093
5982a925 1094# mount \-t cgroup \-o blkio none /cgroup
a696fa2a
JA
1095.TP
1096.BI cgroup_weight \fR=\fPint
1097Set the weight of the cgroup to this value. See the documentation that comes
1098with the kernel, allowed values are in the range of 100..1000.
e0b0d892 1099.TP
7de87099
VG
1100.BI cgroup_nodelete \fR=\fPbool
1101Normally fio will delete the cgroups it has created after the job completion.
1102To override this behavior and to leave cgroups around after the job completion,
1103set cgroup_nodelete=1. This can be useful if one wants to inspect various
1104cgroup files after job completion. Default: false
1105.TP
e0b0d892
JA
1106.BI uid \fR=\fPint
1107Instead of running as the invoking user, set the user ID to this value before
1108the thread/process does any work.
1109.TP
1110.BI gid \fR=\fPint
1111Set group ID, see \fBuid\fR.
83349190 1112.TP
9e684a49
DE
1113.BI flow_id \fR=\fPint
1114The ID of the flow. If not specified, it defaults to being a global flow. See
1115\fBflow\fR.
1116.TP
1117.BI flow \fR=\fPint
1118Weight in token-based flow control. If this value is used, then there is a
1119\fBflow counter\fR which is used to regulate the proportion of activity between
1120two or more jobs. fio attempts to keep this flow counter near zero. The
1121\fBflow\fR parameter stands for how much should be added or subtracted to the
1122flow counter on each iteration of the main I/O loop. That is, if one job has
1123\fBflow=8\fR and another job has \fBflow=-1\fR, then there will be a roughly
11241:8 ratio in how much one runs vs the other.
1125.TP
1126.BI flow_watermark \fR=\fPint
1127The maximum value that the absolute value of the flow counter is allowed to
1128reach before the job must wait for a lower value of the counter.
1129.TP
1130.BI flow_sleep \fR=\fPint
1131The period of time, in microseconds, to wait after the flow watermark has been
1132exceeded before retrying operations
1133.TP
83349190
YH
1134.BI clat_percentiles \fR=\fPbool
1135Enable the reporting of percentiles of completion latencies.
1136.TP
1137.BI percentile_list \fR=\fPfloat_list
1138Overwrite the default list of percentiles for completion
1139latencies. Each number is a floating number in the range (0,100], and
1140the maximum length of the list is 20. Use ':' to separate the
3eb07285 1141numbers. For example, \-\-percentile_list=99.5:99.9 will cause fio to
83349190
YH
1142report the values of completion latency below which 99.5% and 99.9% of
1143the observed latencies fell, respectively.
de890a1e
SL
1144.SS "Ioengine Parameters List"
1145Some parameters are only valid when a specific ioengine is in use. These are
1146used identically to normal parameters, with the caveat that when used on the
1147command line, the must come after the ioengine that defines them is selected.
1148.TP
1149.BI (libaio)userspace_reap
1150Normally, with the libaio engine in use, fio will use
1151the io_getevents system call to reap newly returned events.
1152With this flag turned on, the AIO ring will be read directly
1153from user-space to reap events. The reaping mode is only
1154enabled when polling for a minimum of 0 events (eg when
1155iodepth_batch_complete=0).
1156.TP
1157.BI (net,netsplice)hostname \fR=\fPstr
1158The host name or IP address to use for TCP or UDP based IO.
1159If the job is a TCP listener or UDP reader, the hostname is not
1160used and must be omitted.
1161.TP
1162.BI (net,netsplice)port \fR=\fPint
1163The TCP or UDP port to bind to or connect to.
1164.TP
1165.BI (net,netsplice)protocol \fR=\fPstr "\fR,\fP proto" \fR=\fPstr
1166The network protocol to use. Accepted values are:
1167.RS
1168.RS
1169.TP
1170.B tcp
1171Transmission control protocol
1172.TP
1173.B udp
f5cc3d0e 1174User datagram protocol
de890a1e
SL
1175.TP
1176.B unix
1177UNIX domain socket
1178.RE
1179.P
1180When the protocol is TCP or UDP, the port must also be given,
1181as well as the hostname if the job is a TCP listener or UDP
1182reader. For unix sockets, the normal filename option should be
1183used and the port is invalid.
1184.RE
1185.TP
1186.BI (net,netsplice)listen
1187For TCP network connections, tell fio to listen for incoming
1188connections rather than initiating an outgoing connection. The
1189hostname must be omitted if this option is used.
d54fce84 1190.TP
7aeb1e94
JA
1191.BI (net, pingpong) \fR=\fPbool
1192Normal a network writer will just continue writing data, and a network reader
1193will just consume packages. If pingpong=1 is set, a writer will send its normal
1194payload to the reader, then wait for the reader to send the same payload back.
1195This allows fio to measure network latencies. The submission and completion
1196latencies then measure local time spent sending or receiving, and the
1197completion latency measures how long it took for the other end to receive and
1198send back.
1199.TP
d54fce84
DM
1200.BI (e4defrag,donorname) \fR=\fPstr
1201File will be used as a block donor (swap extents between files)
1202.TP
1203.BI (e4defrag,inplace) \fR=\fPint
1204Configure donor file block allocation strategy
1205.RS
1206.BI 0(default) :
1207Preallocate donor's file on init
1208.TP
1209.BI 1:
1210allocate space immidietly inside defragment event, and free right after event
1211.RE
1212.TP
d60e92d1 1213.SH OUTPUT
d1429b5c
AC
1214While running, \fBfio\fR will display the status of the created jobs. For
1215example:
d60e92d1 1216.RS
d1429b5c 1217.P
d60e92d1
AC
1218Threads: 1: [_r] [24.8% done] [ 13509/ 8334 kb/s] [eta 00h:01m:31s]
1219.RE
1220.P
d1429b5c
AC
1221The characters in the first set of brackets denote the current status of each
1222threads. The possible values are:
1223.P
1224.PD 0
d60e92d1
AC
1225.RS
1226.TP
1227.B P
1228Setup but not started.
1229.TP
1230.B C
1231Thread created.
1232.TP
1233.B I
1234Initialized, waiting.
1235.TP
1236.B R
1237Running, doing sequential reads.
1238.TP
1239.B r
1240Running, doing random reads.
1241.TP
1242.B W
1243Running, doing sequential writes.
1244.TP
1245.B w
1246Running, doing random writes.
1247.TP
1248.B M
1249Running, doing mixed sequential reads/writes.
1250.TP
1251.B m
1252Running, doing mixed random reads/writes.
1253.TP
1254.B F
1255Running, currently waiting for \fBfsync\fR\|(2).
1256.TP
1257.B V
1258Running, verifying written data.
1259.TP
1260.B E
1261Exited, not reaped by main thread.
1262.TP
1263.B \-
1264Exited, thread reaped.
1265.RE
d1429b5c 1266.PD
d60e92d1
AC
1267.P
1268The second set of brackets shows the estimated completion percentage of
1269the current group. The third set shows the read and write I/O rate,
1270respectively. Finally, the estimated run time of the job is displayed.
1271.P
1272When \fBfio\fR completes (or is interrupted by Ctrl-C), it will show data
1273for each thread, each group of threads, and each disk, in that order.
1274.P
1275Per-thread statistics first show the threads client number, group-id, and
1276error code. The remaining figures are as follows:
1277.RS
d60e92d1
AC
1278.TP
1279.B io
1280Number of megabytes of I/O performed.
1281.TP
1282.B bw
1283Average data rate (bandwidth).
1284.TP
1285.B runt
1286Threads run time.
1287.TP
1288.B slat
1289Submission latency minimum, maximum, average and standard deviation. This is
1290the time it took to submit the I/O.
1291.TP
1292.B clat
1293Completion latency minimum, maximum, average and standard deviation. This
1294is the time between submission and completion.
1295.TP
1296.B bw
1297Bandwidth minimum, maximum, percentage of aggregate bandwidth received, average
1298and standard deviation.
1299.TP
1300.B cpu
1301CPU usage statistics. Includes user and system time, number of context switches
1302this thread went through and number of major and minor page faults.
1303.TP
1304.B IO depths
1305Distribution of I/O depths. Each depth includes everything less than (or equal)
1306to it, but greater than the previous depth.
1307.TP
1308.B IO issued
1309Number of read/write requests issued, and number of short read/write requests.
1310.TP
1311.B IO latencies
1312Distribution of I/O completion latencies. The numbers follow the same pattern
1313as \fBIO depths\fR.
1314.RE
d60e92d1
AC
1315.P
1316The group statistics show:
d1429b5c 1317.PD 0
d60e92d1
AC
1318.RS
1319.TP
1320.B io
1321Number of megabytes I/O performed.
1322.TP
1323.B aggrb
1324Aggregate bandwidth of threads in the group.
1325.TP
1326.B minb
1327Minimum average bandwidth a thread saw.
1328.TP
1329.B maxb
1330Maximum average bandwidth a thread saw.
1331.TP
1332.B mint
d1429b5c 1333Shortest runtime of threads in the group.
d60e92d1
AC
1334.TP
1335.B maxt
1336Longest runtime of threads in the group.
1337.RE
d1429b5c 1338.PD
d60e92d1
AC
1339.P
1340Finally, disk statistics are printed with reads first:
d1429b5c 1341.PD 0
d60e92d1
AC
1342.RS
1343.TP
1344.B ios
1345Number of I/Os performed by all groups.
1346.TP
1347.B merge
1348Number of merges in the I/O scheduler.
1349.TP
1350.B ticks
1351Number of ticks we kept the disk busy.
1352.TP
1353.B io_queue
1354Total time spent in the disk queue.
1355.TP
1356.B util
1357Disk utilization.
1358.RE
d1429b5c 1359.PD
8423bd11
JA
1360.P
1361It is also possible to get fio to dump the current output while it is
1362running, without terminating the job. To do that, send fio the \fBUSR1\fR
1363signal.
d60e92d1
AC
1364.SH TERSE OUTPUT
1365If the \fB\-\-minimal\fR option is given, the results will be printed in a
562c2d2f
DN
1366semicolon-delimited format suitable for scripted use - a job description
1367(if provided) follows on a new line. Note that the first
525c2bfa
JA
1368number in the line is the version number. If the output has to be changed
1369for some reason, this number will be incremented by 1 to signify that
1370change. The fields are:
d60e92d1
AC
1371.P
1372.RS
5e726d0a 1373.B terse version, fio version, jobname, groupid, error
d60e92d1
AC
1374.P
1375Read status:
1376.RS
312b4af2 1377.B Total I/O \fR(KB)\fP, bandwidth \fR(KB/s)\fP, IOPS, runtime \fR(ms)\fP
d60e92d1
AC
1378.P
1379Submission latency:
1380.RS
1381.B min, max, mean, standard deviation
1382.RE
1383Completion latency:
1384.RS
1385.B min, max, mean, standard deviation
1386.RE
1db92cb6
JA
1387Completion latency percentiles (20 fields):
1388.RS
1389.B Xth percentile=usec
1390.RE
525c2bfa
JA
1391Total latency:
1392.RS
1393.B min, max, mean, standard deviation
1394.RE
d60e92d1
AC
1395Bandwidth:
1396.RS
1397.B min, max, aggregate percentage of total, mean, standard deviation
1398.RE
1399.RE
1400.P
1401Write status:
1402.RS
312b4af2 1403.B Total I/O \fR(KB)\fP, bandwidth \fR(KB/s)\fP, IOPS, runtime \fR(ms)\fP
d60e92d1
AC
1404.P
1405Submission latency:
1406.RS
1407.B min, max, mean, standard deviation
1408.RE
1409Completion latency:
1410.RS
1411.B min, max, mean, standard deviation
1412.RE
1db92cb6
JA
1413Completion latency percentiles (20 fields):
1414.RS
1415.B Xth percentile=usec
1416.RE
525c2bfa
JA
1417Total latency:
1418.RS
1419.B min, max, mean, standard deviation
1420.RE
d60e92d1
AC
1421Bandwidth:
1422.RS
1423.B min, max, aggregate percentage of total, mean, standard deviation
1424.RE
1425.RE
1426.P
d1429b5c 1427CPU usage:
d60e92d1 1428.RS
bd2626f0 1429.B user, system, context switches, major page faults, minor page faults
d60e92d1
AC
1430.RE
1431.P
1432IO depth distribution:
1433.RS
1434.B <=1, 2, 4, 8, 16, 32, >=64
1435.RE
1436.P
562c2d2f 1437IO latency distribution:
d60e92d1 1438.RS
562c2d2f
DN
1439Microseconds:
1440.RS
1441.B <=2, 4, 10, 20, 50, 100, 250, 500, 750, 1000
1442.RE
1443Milliseconds:
1444.RS
1445.B <=2, 4, 10, 20, 50, 100, 250, 500, 750, 1000, 2000, >=2000
1446.RE
1447.RE
1448.P
f2f788dd
JA
1449Disk utilization (1 for each disk used):
1450.RS
1451.B name, read ios, write ios, read merges, write merges, read ticks, write ticks, read in-queue time, write in-queue time, disk utilization percentage
1452.RE
1453.P
5982a925 1454Error Info (dependent on continue_on_error, default off):
562c2d2f
DN
1455.RS
1456.B total # errors, first error code
d60e92d1
AC
1457.RE
1458.P
562c2d2f 1459.B text description (if provided in config - appears on newline)
d60e92d1 1460.RE
49da1240
JA
1461.SH CLIENT / SERVER
1462Normally you would run fio as a stand-alone application on the machine
1463where the IO workload should be generated. However, it is also possible to
1464run the frontend and backend of fio separately. This makes it possible to
1465have a fio server running on the machine(s) where the IO workload should
1466be running, while controlling it from another machine.
1467
1468To start the server, you would do:
1469
1470\fBfio \-\-server=args\fR
1471
1472on that machine, where args defines what fio listens to. The arguments
811826be 1473are of the form 'type:hostname or IP:port'. 'type' is either 'ip' (or ip4)
20c67f10
MS
1474for TCP/IP v4, 'ip6' for TCP/IP v6, or 'sock' for a local unix domain
1475socket. 'hostname' is either a hostname or IP address, and 'port' is the port to
811826be 1476listen to (only valid for TCP/IP, not a local socket). Some examples:
49da1240 1477
e01e9745 14781) fio \-\-server
49da1240
JA
1479
1480 Start a fio server, listening on all interfaces on the default port (8765).
1481
e01e9745 14822) fio \-\-server=ip:hostname,4444
49da1240
JA
1483
1484 Start a fio server, listening on IP belonging to hostname and on port 4444.
1485
e01e9745 14863) fio \-\-server=ip6:::1,4444
811826be
JA
1487
1488 Start a fio server, listening on IPv6 localhost ::1 and on port 4444.
1489
e01e9745 14904) fio \-\-server=,4444
49da1240
JA
1491
1492 Start a fio server, listening on all interfaces on port 4444.
1493
e01e9745 14945) fio \-\-server=1.2.3.4
49da1240
JA
1495
1496 Start a fio server, listening on IP 1.2.3.4 on the default port.
1497
e01e9745 14986) fio \-\-server=sock:/tmp/fio.sock
49da1240
JA
1499
1500 Start a fio server, listening on the local socket /tmp/fio.sock.
1501
1502When a server is running, you can connect to it from a client. The client
1503is run with:
1504
e01e9745 1505fio \-\-local-args \-\-client=server \-\-remote-args <job file(s)>
49da1240 1506
e01e9745
MS
1507where \-\-local-args are arguments that are local to the client where it is
1508running, 'server' is the connect string, and \-\-remote-args and <job file(s)>
49da1240
JA
1509are sent to the server. The 'server' string follows the same format as it
1510does on the server side, to allow IP/hostname/socket and port strings.
1511You can connect to multiple clients as well, to do that you could run:
1512
e01e9745 1513fio \-\-client=server2 \-\-client=server2 <job file(s)>
d60e92d1 1514.SH AUTHORS
49da1240 1515
d60e92d1 1516.B fio
aa58d252
JA
1517was written by Jens Axboe <jens.axboe@oracle.com>,
1518now Jens Axboe <jaxboe@fusionio.com>.
d1429b5c
AC
1519.br
1520This man page was written by Aaron Carroll <aaronc@cse.unsw.edu.au> based
d60e92d1
AC
1521on documentation by Jens Axboe.
1522.SH "REPORTING BUGS"
482900c9 1523Report bugs to the \fBfio\fR mailing list <fio@vger.kernel.org>.
d1429b5c 1524See \fBREADME\fR.
d60e92d1 1525.SH "SEE ALSO"
d1429b5c
AC
1526For further documentation see \fBHOWTO\fR and \fBREADME\fR.
1527.br
1528Sample jobfiles are available in the \fBexamples\fR directory.
d60e92d1 1529