gitignore: ignore configure script outputs
[fio.git] / fio.1
CommitLineData
d60e92d1
AC
1.TH fio 1 "September 2007" "User Manual"
2.SH NAME
3fio \- flexible I/O tester
4.SH SYNOPSIS
5.B fio
6[\fIoptions\fR] [\fIjobfile\fR]...
7.SH DESCRIPTION
8.B fio
9is a tool that will spawn a number of threads or processes doing a
10particular type of I/O action as specified by the user.
11The typical use of fio is to write a job file matching the I/O load
12one wants to simulate.
13.SH OPTIONS
14.TP
49da1240
JA
15.BI \-\-debug \fR=\fPtype
16Enable verbose tracing of various fio actions. May be `all' for all types
17or individual types separated by a comma (eg \-\-debug=io,file). `help' will
18list all available tracing options.
19.TP
d60e92d1
AC
20.BI \-\-output \fR=\fPfilename
21Write output to \fIfilename\fR.
22.TP
b2cecdc2 23.BI \-\-runtime \fR=\fPruntime
24Limit run time to \fIruntime\fR seconds.
d60e92d1
AC
25.TP
26.B \-\-latency\-log
27Generate per-job latency logs.
28.TP
29.B \-\-bandwidth\-log
30Generate per-job bandwidth logs.
31.TP
32.B \-\-minimal
d1429b5c 33Print statistics in a terse, semicolon-delimited format.
d60e92d1 34.TP
49da1240
JA
35.B \-\-version
36Display version information and exit.
37.TP
065248bf 38.BI \-\-terse\-version \fR=\fPversion
4d658652 39Set terse version output format (Current version 3, or older version 2).
49da1240
JA
40.TP
41.B \-\-help
42Display usage information and exit.
43.TP
44.BI \-\-cmdhelp \fR=\fPcommand
45Print help information for \fIcommand\fR. May be `all' for all commands.
46.TP
de890a1e
SL
47.BI \-\-enghelp \fR=\fPioengine[,command]
48List all commands defined by \fIioengine\fR, or print help for \fIcommand\fR defined by \fIioengine\fR.
49.TP
d60e92d1
AC
50.BI \-\-showcmd \fR=\fPjobfile
51Convert \fIjobfile\fR to a set of command-line options.
52.TP
d60e92d1
AC
53.BI \-\-eta \fR=\fPwhen
54Specifies when real-time ETA estimate should be printed. \fIwhen\fR may
55be one of `always', `never' or `auto'.
56.TP
49da1240
JA
57.BI \-\-readonly
58Turn on safety read-only checks, preventing any attempted write.
59.TP
c0a5d35e 60.BI \-\-section \fR=\fPsec
49da1240 61Only run section \fIsec\fR from job file. Multiple of these options can be given, adding more sections to run.
c0a5d35e 62.TP
49da1240
JA
63.BI \-\-alloc\-size \fR=\fPkb
64Set the internal smalloc pool size to \fIkb\fP kilobytes.
d60e92d1 65.TP
49da1240
JA
66.BI \-\-warnings\-fatal
67All fio parser warnings are fatal, causing fio to exit with an error.
9183788d 68.TP
49da1240 69.BI \-\-max\-jobs \fR=\fPnr
57e118a2 70Set the maximum allowed number of jobs (threads/processes) to support.
d60e92d1 71.TP
49da1240
JA
72.BI \-\-server \fR=\fPargs
73Start a backend server, with \fIargs\fP specifying what to listen to. See client/server section.
f57a9c59 74.TP
49da1240
JA
75.BI \-\-daemonize \fR=\fPpidfile
76Background a fio server, writing the pid to the given pid file.
77.TP
78.BI \-\-client \fR=\fPhost
79Instead of running the jobs locally, send and run them on the given host.
f2a2ce0e
HL
80.TP
81.BI \-\-idle\-prof \fR=\fPoption
82Report cpu idleness on a system or percpu basis (\fIoption\fP=system,percpu) or run unit work calibration only (\fIoption\fP=calibrate).
d60e92d1
AC
83.SH "JOB FILE FORMAT"
84Job files are in `ini' format. They consist of one or more
85job definitions, which begin with a job name in square brackets and
86extend to the next job name. The job name can be any ASCII string
87except `global', which has a special meaning. Following the job name is
88a sequence of zero or more parameters, one per line, that define the
89behavior of the job. Any line starting with a `;' or `#' character is
d1429b5c 90considered a comment and ignored.
d9956b64
AC
91.P
92If \fIjobfile\fR is specified as `-', the job file will be read from
93standard input.
d60e92d1
AC
94.SS "Global Section"
95The global section contains default parameters for jobs specified in the
96job file. A job is only affected by global sections residing above it,
97and there may be any number of global sections. Specific job definitions
98may override any parameter set in global sections.
99.SH "JOB PARAMETERS"
100.SS Types
101Some parameters may take arguments of a specific type. The types used are:
102.TP
103.I str
104String: a sequence of alphanumeric characters.
105.TP
106.I int
d60e92d1 107SI integer: a whole number, possibly containing a suffix denoting the base unit
b09da8fa
JA
108of the value. Accepted suffixes are `k', 'M', 'G', 'T', and 'P', denoting
109kilo (1024), mega (1024^2), giga (1024^3), tera (1024^4), and peta (1024^5)
110respectively. The suffix is not case sensitive. If prefixed with '0x', the
5982a925
MS
111value is assumed to be base 16 (hexadecimal). A suffix may include a trailing 'b',
112for instance 'kb' is identical to 'k'. You can specify a base 10 value
57fc29fa
JA
113by using 'KiB', 'MiB', 'GiB', etc. This is useful for disk drives where
114values are often given in base 10 values. Specifying '30GiB' will get you
11530*1000^3 bytes.
d60e92d1
AC
116.TP
117.I bool
118Boolean: a true or false value. `0' denotes false, `1' denotes true.
119.TP
120.I irange
121Integer range: a range of integers specified in the format
d1429b5c
AC
122\fIlower\fR:\fIupper\fR or \fIlower\fR\-\fIupper\fR. \fIlower\fR and
123\fIupper\fR may contain a suffix as described above. If an option allows two
124sets of ranges, they are separated with a `,' or `/' character. For example:
125`8\-8k/8M\-4G'.
83349190
YH
126.TP
127.I float_list
128List of floating numbers: A list of floating numbers, separated by
129a ':' charcater.
d60e92d1
AC
130.SS "Parameter List"
131.TP
132.BI name \fR=\fPstr
d9956b64 133May be used to override the job name. On the command line, this parameter
d60e92d1
AC
134has the special purpose of signalling the start of a new job.
135.TP
136.BI description \fR=\fPstr
137Human-readable description of the job. It is printed when the job is run, but
138otherwise has no special purpose.
139.TP
140.BI directory \fR=\fPstr
141Prefix filenames with this directory. Used to place files in a location other
142than `./'.
143.TP
144.BI filename \fR=\fPstr
145.B fio
146normally makes up a file name based on the job name, thread number, and file
d1429b5c 147number. If you want to share files between threads in a job or several jobs,
de890a1e
SL
148specify a \fIfilename\fR for each of them to override the default.
149If the I/O engine is file-based, you can specify
d1429b5c
AC
150a number of files by separating the names with a `:' character. `\-' is a
151reserved name, meaning stdin or stdout, depending on the read/write direction
152set.
d60e92d1 153.TP
3ce9dcaf
JA
154.BI lockfile \fR=\fPstr
155Fio defaults to not locking any files before it does IO to them. If a file or
156file descriptor is shared, fio can serialize IO to that file to make the end
157result consistent. This is usual for emulating real workloads that share files.
158The lock modes are:
159.RS
160.RS
161.TP
162.B none
163No locking. This is the default.
164.TP
165.B exclusive
166Only one thread or process may do IO at the time, excluding all others.
167.TP
168.B readwrite
169Read-write locking on the file. Many readers may access the file at the same
170time, but writes get exclusive access.
171.RE
172.P
d60e92d1
AC
173.BI opendir \fR=\fPstr
174Recursively open any files below directory \fIstr\fR.
175.TP
176.BI readwrite \fR=\fPstr "\fR,\fP rw" \fR=\fPstr
177Type of I/O pattern. Accepted values are:
178.RS
179.RS
180.TP
181.B read
d1429b5c 182Sequential reads.
d60e92d1
AC
183.TP
184.B write
d1429b5c 185Sequential writes.
d60e92d1
AC
186.TP
187.B randread
d1429b5c 188Random reads.
d60e92d1
AC
189.TP
190.B randwrite
d1429b5c 191Random writes.
d60e92d1 192.TP
10b023db 193.B rw, readwrite
d1429b5c 194Mixed sequential reads and writes.
d60e92d1
AC
195.TP
196.B randrw
d1429b5c 197Mixed random reads and writes.
d60e92d1
AC
198.RE
199.P
38dad62d
JA
200For mixed I/O, the default split is 50/50. For certain types of io the result
201may still be skewed a bit, since the speed may be different. It is possible to
3b7fa9ec 202specify a number of IO's to do before getting a new offset, this is done by
38dad62d
JA
203appending a `:\fI<nr>\fR to the end of the string given. For a random read, it
204would look like \fBrw=randread:8\fR for passing in an offset modifier with a
059b0802
JA
205value of 8. If the postfix is used with a sequential IO pattern, then the value
206specified will be added to the generated offset for each IO. For instance,
207using \fBrw=write:4k\fR will skip 4k for every write. It turns sequential IO
208into sequential IO with holes. See the \fBrw_sequencer\fR option.
d60e92d1
AC
209.RE
210.TP
38dad62d
JA
211.BI rw_sequencer \fR=\fPstr
212If an offset modifier is given by appending a number to the \fBrw=<str>\fR line,
213then this option controls how that number modifies the IO offset being
214generated. Accepted values are:
215.RS
216.RS
217.TP
218.B sequential
219Generate sequential offset
220.TP
221.B identical
222Generate the same offset
223.RE
224.P
225\fBsequential\fR is only useful for random IO, where fio would normally
226generate a new random offset for every IO. If you append eg 8 to randread, you
227would get a new random offset for every 8 IO's. The result would be a seek for
228only every 8 IO's, instead of for every IO. Use \fBrw=randread:8\fR to specify
229that. As sequential IO is already sequential, setting \fBsequential\fR for that
230would not result in any differences. \fBidentical\fR behaves in a similar
231fashion, except it sends the same offset 8 number of times before generating a
232new offset.
233.RE
234.P
235.TP
90fef2d1
JA
236.BI kb_base \fR=\fPint
237The base unit for a kilobyte. The defacto base is 2^10, 1024. Storage
238manufacturers like to use 10^3 or 1000 as a base ten unit instead, for obvious
239reasons. Allow values are 1024 or 1000, with 1024 being the default.
240.TP
771e58be
JA
241.BI unified_rw_reporting \fR=\fPbool
242Fio normally reports statistics on a per data direction basis, meaning that
243read, write, and trim are accounted and reported separately. If this option is
244set, the fio will sum the results and report them as "mixed" instead.
245.TP
d60e92d1
AC
246.BI randrepeat \fR=\fPbool
247Seed the random number generator in a predictable way so results are repeatable
d1429b5c 248across runs. Default: true.
d60e92d1 249.TP
2615cc4b
JA
250.BI use_os_rand \fR=\fPbool
251Fio can either use the random generator supplied by the OS to generator random
252offsets, or it can use it's own internal generator (based on Tausworthe).
253Default is to use the internal generator, which is often of better quality and
254faster. Default: false.
255.TP
a596f047
EG
256.BI fallocate \fR=\fPstr
257Whether pre-allocation is performed when laying down files. Accepted values
258are:
259.RS
260.RS
261.TP
262.B none
263Do not pre-allocate space.
264.TP
265.B posix
266Pre-allocate via posix_fallocate().
267.TP
268.B keep
269Pre-allocate via fallocate() with FALLOC_FL_KEEP_SIZE set.
270.TP
271.B 0
272Backward-compatible alias for 'none'.
273.TP
274.B 1
275Backward-compatible alias for 'posix'.
276.RE
277.P
278May not be available on all supported platforms. 'keep' is only
279available on Linux. If using ZFS on Solaris this must be set to 'none'
280because ZFS doesn't support it. Default: 'posix'.
281.RE
7bc8c2cf 282.TP
d60e92d1 283.BI fadvise_hint \fR=\fPbool
23a7b043 284Use of \fIposix_fadvise\fR\|(2) to advise the kernel what I/O patterns
d1429b5c 285are likely to be issued. Default: true.
d60e92d1 286.TP
f7fa2653 287.BI size \fR=\fPint
d60e92d1
AC
288Total size of I/O for this job. \fBfio\fR will run until this many bytes have
289been transfered, unless limited by other options (\fBruntime\fR, for instance).
d7c8be03 290Unless \fBnrfiles\fR and \fBfilesize\fR options are given, this amount will be
d6667268
JA
291divided between the available files for the job. If not set, fio will use the
292full size of the given files or devices. If the the files do not exist, size
7bb59102
JA
293must be given. It is also possible to give size as a percentage between 1 and
294100. If size=20% is given, fio will use 20% of the full size of the given files
295or devices.
d60e92d1 296.TP
74586c1e 297.BI fill_device \fR=\fPbool "\fR,\fB fill_fs" \fR=\fPbool
3ce9dcaf
JA
298Sets size to something really large and waits for ENOSPC (no space left on
299device) as the terminating condition. Only makes sense with sequential write.
300For a read workload, the mount point will be filled first then IO started on
4f12432e
JA
301the result. This option doesn't make sense if operating on a raw device node,
302since the size of that is already known by the file system. Additionally,
303writing beyond end-of-device will not return ENOSPC there.
3ce9dcaf 304.TP
d60e92d1
AC
305.BI filesize \fR=\fPirange
306Individual file sizes. May be a range, in which case \fBfio\fR will select sizes
d1429b5c
AC
307for files at random within the given range, limited to \fBsize\fR in total (if
308that is given). If \fBfilesize\fR is not specified, each created file is the
309same size.
d60e92d1 310.TP
f7fa2653 311.BI blocksize \fR=\fPint[,int] "\fR,\fB bs" \fR=\fPint[,int]
d60e92d1 312Block size for I/O units. Default: 4k. Values for reads and writes can be
656ebab7 313specified separately in the format \fIread\fR,\fIwrite\fR, either of
d60e92d1
AC
314which may be empty to leave that value at its default.
315.TP
9183788d 316.BI blocksize_range \fR=\fPirange[,irange] "\fR,\fB bsrange" \fR=\fPirange[,irange]
d1429b5c
AC
317Specify a range of I/O block sizes. The issued I/O unit will always be a
318multiple of the minimum size, unless \fBblocksize_unaligned\fR is set. Applies
9183788d 319to both reads and writes if only one range is given, but can be specified
656ebab7 320separately with a comma seperating the values. Example: bsrange=1k-4k,2k-8k.
9183788d
JA
321Also (see \fBblocksize\fR).
322.TP
323.BI bssplit \fR=\fPstr
324This option allows even finer grained control of the block sizes issued,
325not just even splits between them. With this option, you can weight various
326block sizes for exact control of the issued IO for a job that has mixed
327block sizes. The format of the option is bssplit=blocksize/percentage,
5982a925 328optionally adding as many definitions as needed separated by a colon.
9183788d 329Example: bssplit=4k/10:64k/50:32k/40 would issue 50% 64k blocks, 10% 4k
c83cdd3e
JA
330blocks and 40% 32k blocks. \fBbssplit\fR also supports giving separate
331splits to reads and writes. The format is identical to what the
332\fBbs\fR option accepts, the read and write parts are separated with a
333comma.
d60e92d1
AC
334.TP
335.B blocksize_unaligned\fR,\fP bs_unaligned
d1429b5c
AC
336If set, any size in \fBblocksize_range\fR may be used. This typically won't
337work with direct I/O, as that normally requires sector alignment.
d60e92d1 338.TP
2b7a01d0 339.BI blockalign \fR=\fPint[,int] "\fR,\fB ba" \fR=\fPint[,int]
639ce0f3
MS
340At what boundary to align random IO offsets. Defaults to the same as 'blocksize'
341the minimum blocksize given. Minimum alignment is typically 512b
2b7a01d0
JA
342for using direct IO, though it usually depends on the hardware block size.
343This option is mutually exclusive with using a random map for files, so it
344will turn off that option.
43602667 345.TP
d60e92d1
AC
346.B zero_buffers
347Initialise buffers with all zeros. Default: fill buffers with random data.
348.TP
901bb994
JA
349.B refill_buffers
350If this option is given, fio will refill the IO buffers on every submit. The
351default is to only fill it at init time and reuse that data. Only makes sense
352if zero_buffers isn't specified, naturally. If data verification is enabled,
353refill_buffers is also automatically enabled.
354.TP
fd68418e
JA
355.BI scramble_buffers \fR=\fPbool
356If \fBrefill_buffers\fR is too costly and the target is using data
357deduplication, then setting this option will slightly modify the IO buffer
358contents to defeat normal de-dupe attempts. This is not enough to defeat
359more clever block compression attempts, but it will stop naive dedupe
360of blocks. Default: true.
361.TP
c5751c62
JA
362.BI buffer_compress_percentage \fR=\fPint
363If this is set, then fio will attempt to provide IO buffer content (on WRITEs)
364that compress to the specified level. Fio does this by providing a mix of
365random data and zeroes. Note that this is per block size unit, for file/disk
366wide compression level that matches this setting, you'll also want to set
367\fBrefill_buffers\fR.
368.TP
369.BI buffer_compress_chunk \fR=\fPint
370See \fBbuffer_compress_percentage\fR. This setting allows fio to manage how
371big the ranges of random data and zeroed data is. Without this set, fio will
372provide \fBbuffer_compress_percentage\fR of blocksize random data, followed by
373the remaining zeroed. With this set to some chunk size smaller than the block
374size, fio can alternate random and zeroed data throughout the IO buffer.
375.TP
d60e92d1
AC
376.BI nrfiles \fR=\fPint
377Number of files to use for this job. Default: 1.
378.TP
379.BI openfiles \fR=\fPint
380Number of files to keep open at the same time. Default: \fBnrfiles\fR.
381.TP
382.BI file_service_type \fR=\fPstr
383Defines how files to service are selected. The following types are defined:
384.RS
385.RS
386.TP
387.B random
388Choose a file at random
389.TP
390.B roundrobin
391Round robin over open files (default).
6b7f6851
JA
392.B sequential
393Do each file in the set sequentially.
d60e92d1
AC
394.RE
395.P
396The number of I/Os to issue before switching a new file can be specified by
397appending `:\fIint\fR' to the service type.
398.RE
399.TP
400.BI ioengine \fR=\fPstr
401Defines how the job issues I/O. The following types are defined:
402.RS
403.RS
404.TP
405.B sync
406Basic \fIread\fR\|(2) or \fIwrite\fR\|(2) I/O. \fIfseek\fR\|(2) is used to
407position the I/O location.
408.TP
a31041ea 409.B psync
410Basic \fIpread\fR\|(2) or \fIpwrite\fR\|(2) I/O.
411.TP
9183788d
JA
412.B vsync
413Basic \fIreadv\fR\|(2) or \fIwritev\fR\|(2) I/O. Will emulate queuing by
414coalescing adjacents IOs into a single submission.
415.TP
d60e92d1 416.B libaio
de890a1e 417Linux native asynchronous I/O. This ioengine defines engine specific options.
d60e92d1
AC
418.TP
419.B posixaio
03e20d68
BC
420POSIX asynchronous I/O using \fIaio_read\fR\|(3) and \fIaio_write\fR\|(3).
421.TP
422.B solarisaio
423Solaris native asynchronous I/O.
424.TP
425.B windowsaio
426Windows native asynchronous I/O.
d60e92d1
AC
427.TP
428.B mmap
d1429b5c
AC
429File is memory mapped with \fImmap\fR\|(2) and data copied using
430\fImemcpy\fR\|(3).
d60e92d1
AC
431.TP
432.B splice
d1429b5c
AC
433\fIsplice\fR\|(2) is used to transfer the data and \fIvmsplice\fR\|(2) to
434transfer data from user-space to the kernel.
d60e92d1
AC
435.TP
436.B syslet-rw
437Use the syslet system calls to make regular read/write asynchronous.
438.TP
439.B sg
440SCSI generic sg v3 I/O. May be either synchronous using the SG_IO ioctl, or if
d1429b5c
AC
441the target is an sg character device, we use \fIread\fR\|(2) and
442\fIwrite\fR\|(2) for asynchronous I/O.
d60e92d1
AC
443.TP
444.B null
445Doesn't transfer any data, just pretends to. Mainly used to exercise \fBfio\fR
446itself and for debugging and testing purposes.
447.TP
448.B net
de890a1e
SL
449Transfer over the network. The protocol to be used can be defined with the
450\fBprotocol\fR parameter. Depending on the protocol, \fBfilename\fR,
451\fBhostname\fR, \fBport\fR, or \fBlisten\fR must be specified.
452This ioengine defines engine specific options.
d60e92d1
AC
453.TP
454.B netsplice
455Like \fBnet\fR, but uses \fIsplice\fR\|(2) and \fIvmsplice\fR\|(2) to map data
de890a1e 456and send/receive. This ioengine defines engine specific options.
d60e92d1 457.TP
53aec0a4 458.B cpuio
d60e92d1
AC
459Doesn't transfer any data, but burns CPU cycles according to \fBcpuload\fR and
460\fBcpucycles\fR parameters.
461.TP
462.B guasi
463The GUASI I/O engine is the Generic Userspace Asynchronous Syscall Interface
464approach to asycnronous I/O.
d1429b5c
AC
465.br
466See <http://www.xmailserver.org/guasi\-lib.html>.
d60e92d1 467.TP
21b8aee8 468.B rdma
85286c5c
BVA
469The RDMA I/O engine supports both RDMA memory semantics (RDMA_WRITE/RDMA_READ)
470and channel semantics (Send/Recv) for the InfiniBand, RoCE and iWARP protocols.
21b8aee8 471.TP
d60e92d1
AC
472.B external
473Loads an external I/O engine object file. Append the engine filename as
474`:\fIenginepath\fR'.
d54fce84
DM
475.TP
476.B falloc
477 IO engine that does regular linux native fallocate callt to simulate data
478transfer as fio ioengine
479.br
480 DDIR_READ does fallocate(,mode = FALLOC_FL_KEEP_SIZE,)
481.br
0981fd71 482 DIR_WRITE does fallocate(,mode = 0)
d54fce84
DM
483.br
484 DDIR_TRIM does fallocate(,mode = FALLOC_FL_KEEP_SIZE|FALLOC_FL_PUNCH_HOLE)
485.TP
486.B e4defrag
487IO engine that does regular EXT4_IOC_MOVE_EXT ioctls to simulate defragment activity
488request to DDIR_WRITE event
d60e92d1 489.RE
595e1734 490.P
d60e92d1
AC
491.RE
492.TP
493.BI iodepth \fR=\fPint
8489dae4
SK
494Number of I/O units to keep in flight against the file. Note that increasing
495iodepth beyond 1 will not affect synchronous ioengines (except for small
ee72ca09
JA
496degress when verify_async is in use). Even async engines my impose OS
497restrictions causing the desired depth not to be achieved. This may happen on
498Linux when using libaio and not setting \fBdirect\fR=1, since buffered IO is
499not async on that OS. Keep an eye on the IO depth distribution in the
500fio output to verify that the achieved depth is as expected. Default: 1.
d60e92d1
AC
501.TP
502.BI iodepth_batch \fR=\fPint
503Number of I/Os to submit at once. Default: \fBiodepth\fR.
504.TP
3ce9dcaf
JA
505.BI iodepth_batch_complete \fR=\fPint
506This defines how many pieces of IO to retrieve at once. It defaults to 1 which
507 means that we'll ask for a minimum of 1 IO in the retrieval process from the
508kernel. The IO retrieval will go on until we hit the limit set by
509\fBiodepth_low\fR. If this variable is set to 0, then fio will always check for
510completed events before queuing more IO. This helps reduce IO latency, at the
511cost of more retrieval system calls.
512.TP
d60e92d1
AC
513.BI iodepth_low \fR=\fPint
514Low watermark indicating when to start filling the queue again. Default:
515\fBiodepth\fR.
516.TP
517.BI direct \fR=\fPbool
518If true, use non-buffered I/O (usually O_DIRECT). Default: false.
519.TP
520.BI buffered \fR=\fPbool
521If true, use buffered I/O. This is the opposite of the \fBdirect\fR parameter.
522Default: true.
523.TP
f7fa2653 524.BI offset \fR=\fPint
d60e92d1
AC
525Offset in the file to start I/O. Data before the offset will not be touched.
526.TP
591e9e06
JA
527.BI offset_increment \fR=\fPint
528If this is provided, then the real offset becomes the
529offset + offset_increment * thread_number, where the thread number is a counter
530that starts at 0 and is incremented for each job. This option is useful if
531there are several jobs which are intended to operate on a file in parallel in
532disjoint segments, with even spacing between the starting points.
533.TP
d60e92d1 534.BI fsync \fR=\fPint
d1429b5c
AC
535How many I/Os to perform before issuing an \fBfsync\fR\|(2) of dirty data. If
5360, don't sync. Default: 0.
d60e92d1 537.TP
5f9099ea
JA
538.BI fdatasync \fR=\fPint
539Like \fBfsync\fR, but uses \fBfdatasync\fR\|(2) instead to only sync the
540data parts of the file. Default: 0.
541.TP
e76b1da4
JA
542.BI sync_file_range \fR=\fPstr:int
543Use sync_file_range() for every \fRval\fP number of write operations. Fio will
544track range of writes that have happened since the last sync_file_range() call.
545\fRstr\fP can currently be one or more of:
546.RS
547.TP
548.B wait_before
549SYNC_FILE_RANGE_WAIT_BEFORE
550.TP
551.B write
552SYNC_FILE_RANGE_WRITE
553.TP
554.B wait_after
555SYNC_FILE_RANGE_WRITE
556.TP
557.RE
558.P
559So if you do sync_file_range=wait_before,write:8, fio would use
560\fBSYNC_FILE_RANGE_WAIT_BEFORE | SYNC_FILE_RANGE_WRITE\fP for every 8 writes.
561Also see the sync_file_range(2) man page. This option is Linux specific.
562.TP
d60e92d1 563.BI overwrite \fR=\fPbool
d1429b5c 564If writing, setup the file first and do overwrites. Default: false.
d60e92d1
AC
565.TP
566.BI end_fsync \fR=\fPbool
dbd11ead 567Sync file contents when a write stage has completed. Default: false.
d60e92d1
AC
568.TP
569.BI fsync_on_close \fR=\fPbool
570If true, sync file contents on close. This differs from \fBend_fsync\fR in that
d1429b5c 571it will happen on every close, not just at the end of the job. Default: false.
d60e92d1 572.TP
d60e92d1
AC
573.BI rwmixread \fR=\fPint
574Percentage of a mixed workload that should be reads. Default: 50.
575.TP
576.BI rwmixwrite \fR=\fPint
d1429b5c 577Percentage of a mixed workload that should be writes. If \fBrwmixread\fR and
c35dd7a6
JA
578\fBrwmixwrite\fR are given and do not sum to 100%, the latter of the two
579overrides the first. This may interfere with a given rate setting, if fio is
580asked to limit reads or writes to a certain rate. If that is the case, then
581the distribution may be skewed. Default: 50.
d60e92d1 582.TP
92d42d69
JA
583.BI random_distribution \fR=\fPstr:float
584By default, fio will use a completely uniform random distribution when asked
585to perform random IO. Sometimes it is useful to skew the distribution in
586specific ways, ensuring that some parts of the data is more hot than others.
587Fio includes the following distribution models:
588.RS
589.TP
590.B random
591Uniform random distribution
592.TP
593.B zipf
594Zipf distribution
595.TP
596.B pareto
597Pareto distribution
598.TP
599.RE
600.P
601When using a zipf or pareto distribution, an input value is also needed to
602define the access pattern. For zipf, this is the zipf theta. For pareto,
603it's the pareto power. Fio includes a test program, genzipf, that can be
604used visualize what the given input values will yield in terms of hit rates.
605If you wanted to use zipf with a theta of 1.2, you would use
606random_distribution=zipf:1.2 as the option. If a non-uniform model is used,
607fio will disable use of the random map.
608.TP
d60e92d1
AC
609.B norandommap
610Normally \fBfio\fR will cover every block of the file when doing random I/O. If
611this parameter is given, a new offset will be chosen without looking at past
612I/O history. This parameter is mutually exclusive with \fBverify\fR.
613.TP
744492c9 614.BI softrandommap \fR=\fPbool
3ce9dcaf
JA
615See \fBnorandommap\fR. If fio runs with the random block map enabled and it
616fails to allocate the map, if this option is set it will continue without a
617random block map. As coverage will not be as complete as with random maps, this
618option is disabled by default.
619.TP
e8b1961d
JA
620.BI random_generator \fR=\fPstr
621Fio supports the following engines for generating IO offsets for random IO:
622.RS
623.TP
624.B tausworthe
625Strong 2^88 cycle random number generator
626.TP
627.B lfsr
628Linear feedback shift register generator
629.TP
630.RE
631.P
632Tausworthe is a strong random number generator, but it requires tracking on the
633side if we want to ensure that blocks are only read or written once. LFSR
634guarantees that we never generate the same offset twice, and it's also less
635computationally expensive. It's not a true random generator, however, though
636for IO purposes it's typically good enough. LFSR only works with single block
637sizes, not with workloads that use multiple block sizes. If used with such a
638workload, fio may read or write some blocks multiple times.
639.TP
d60e92d1
AC
640.BI nice \fR=\fPint
641Run job with given nice value. See \fInice\fR\|(2).
642.TP
643.BI prio \fR=\fPint
644Set I/O priority value of this job between 0 (highest) and 7 (lowest). See
645\fIionice\fR\|(1).
646.TP
647.BI prioclass \fR=\fPint
648Set I/O priority class. See \fIionice\fR\|(1).
649.TP
650.BI thinktime \fR=\fPint
651Stall job for given number of microseconds between issuing I/Os.
652.TP
653.BI thinktime_spin \fR=\fPint
654Pretend to spend CPU time for given number of microseconds, sleeping the rest
655of the time specified by \fBthinktime\fR. Only valid if \fBthinktime\fR is set.
656.TP
657.BI thinktime_blocks \fR=\fPint
658Number of blocks to issue before waiting \fBthinktime\fR microseconds.
659Default: 1.
660.TP
661.BI rate \fR=\fPint
c35dd7a6
JA
662Cap bandwidth used by this job. The number is in bytes/sec, the normal postfix
663rules apply. You can use \fBrate\fR=500k to limit reads and writes to 500k each,
664or you can specify read and writes separately. Using \fBrate\fR=1m,500k would
665limit reads to 1MB/sec and writes to 500KB/sec. Capping only reads or writes
666can be done with \fBrate\fR=,500k or \fBrate\fR=500k,. The former will only
667limit writes (to 500KB/sec), the latter will only limit reads.
d60e92d1
AC
668.TP
669.BI ratemin \fR=\fPint
670Tell \fBfio\fR to do whatever it can to maintain at least the given bandwidth.
c35dd7a6
JA
671Failing to meet this requirement will cause the job to exit. The same format
672as \fBrate\fR is used for read vs write separation.
d60e92d1
AC
673.TP
674.BI rate_iops \fR=\fPint
c35dd7a6
JA
675Cap the bandwidth to this number of IOPS. Basically the same as rate, just
676specified independently of bandwidth. The same format as \fBrate\fR is used for
677read vs write seperation. If \fBblocksize\fR is a range, the smallest block
678size is used as the metric.
d60e92d1
AC
679.TP
680.BI rate_iops_min \fR=\fPint
c35dd7a6
JA
681If this rate of I/O is not met, the job will exit. The same format as \fBrate\fR
682is used for read vs write seperation.
d60e92d1
AC
683.TP
684.BI ratecycle \fR=\fPint
685Average bandwidth for \fBrate\fR and \fBratemin\fR over this number of
686milliseconds. Default: 1000ms.
687.TP
15501535
JA
688.BI max_latency \fR=\fPint
689If set, fio will exit the job if it exceeds this maximum latency. It will exit
690with an ETIME error.
691.TP
d60e92d1
AC
692.BI cpumask \fR=\fPint
693Set CPU affinity for this job. \fIint\fR is a bitmask of allowed CPUs the job
694may run on. See \fBsched_setaffinity\fR\|(2).
695.TP
696.BI cpus_allowed \fR=\fPstr
697Same as \fBcpumask\fR, but allows a comma-delimited list of CPU numbers.
698.TP
d0b937ed
YR
699.BI numa_cpu_nodes \fR=\fPstr
700Set this job running on spcified NUMA nodes' CPUs. The arguments allow
701comma delimited list of cpu numbers, A-B ranges, or 'all'.
702.TP
703.BI numa_mem_policy \fR=\fPstr
704Set this job's memory policy and corresponding NUMA nodes. Format of
705the argements:
706.RS
707.TP
708.B <mode>[:<nodelist>]
709.TP
710.B mode
711is one of the following memory policy:
712.TP
713.B default, prefer, bind, interleave, local
714.TP
715.RE
716For \fBdefault\fR and \fBlocal\fR memory policy, no \fBnodelist\fR is
717needed to be specified. For \fBprefer\fR, only one node is
718allowed. For \fBbind\fR and \fBinterleave\fR, \fBnodelist\fR allows
719comma delimited list of numbers, A-B ranges, or 'all'.
720.TP
d60e92d1
AC
721.BI startdelay \fR=\fPint
722Delay start of job for the specified number of seconds.
723.TP
724.BI runtime \fR=\fPint
725Terminate processing after the specified number of seconds.
726.TP
727.B time_based
728If given, run for the specified \fBruntime\fR duration even if the files are
729completely read or written. The same workload will be repeated as many times
730as \fBruntime\fR allows.
731.TP
901bb994
JA
732.BI ramp_time \fR=\fPint
733If set, fio will run the specified workload for this amount of time before
734logging any performance numbers. Useful for letting performance settle before
735logging results, thus minimizing the runtime required for stable results. Note
c35dd7a6
JA
736that the \fBramp_time\fR is considered lead in time for a job, thus it will
737increase the total runtime if a special timeout or runtime is specified.
901bb994 738.TP
d60e92d1
AC
739.BI invalidate \fR=\fPbool
740Invalidate buffer-cache for the file prior to starting I/O. Default: true.
741.TP
742.BI sync \fR=\fPbool
743Use synchronous I/O for buffered writes. For the majority of I/O engines,
d1429b5c 744this means using O_SYNC. Default: false.
d60e92d1
AC
745.TP
746.BI iomem \fR=\fPstr "\fR,\fP mem" \fR=\fPstr
747Allocation method for I/O unit buffer. Allowed values are:
748.RS
749.RS
750.TP
751.B malloc
752Allocate memory with \fImalloc\fR\|(3).
753.TP
754.B shm
755Use shared memory buffers allocated through \fIshmget\fR\|(2).
756.TP
757.B shmhuge
758Same as \fBshm\fR, but use huge pages as backing.
759.TP
760.B mmap
761Use \fImmap\fR\|(2) for allocation. Uses anonymous memory unless a filename
762is given after the option in the format `:\fIfile\fR'.
763.TP
764.B mmaphuge
765Same as \fBmmap\fR, but use huge files as backing.
766.RE
767.P
768The amount of memory allocated is the maximum allowed \fBblocksize\fR for the
769job multiplied by \fBiodepth\fR. For \fBshmhuge\fR or \fBmmaphuge\fR to work,
770the system must have free huge pages allocated. \fBmmaphuge\fR also needs to
2e266ba6
JA
771have hugetlbfs mounted, and \fIfile\fR must point there. At least on Linux,
772huge pages must be manually allocated. See \fB/proc/sys/vm/nr_hugehages\fR
773and the documentation for that. Normally you just need to echo an appropriate
774number, eg echoing 8 will ensure that the OS has 8 huge pages ready for
775use.
d60e92d1
AC
776.RE
777.TP
d392365e 778.BI iomem_align \fR=\fPint "\fR,\fP mem_align" \fR=\fPint
d529ee19
JA
779This indiciates the memory alignment of the IO memory buffers. Note that the
780given alignment is applied to the first IO unit buffer, if using \fBiodepth\fR
781the alignment of the following buffers are given by the \fBbs\fR used. In
782other words, if using a \fBbs\fR that is a multiple of the page sized in the
783system, all buffers will be aligned to this value. If using a \fBbs\fR that
784is not page aligned, the alignment of subsequent IO memory buffers is the
785sum of the \fBiomem_align\fR and \fBbs\fR used.
786.TP
f7fa2653 787.BI hugepage\-size \fR=\fPint
d60e92d1 788Defines the size of a huge page. Must be at least equal to the system setting.
b22989b9 789Should be a multiple of 1MB. Default: 4MB.
d60e92d1
AC
790.TP
791.B exitall
792Terminate all jobs when one finishes. Default: wait for each job to finish.
793.TP
794.BI bwavgtime \fR=\fPint
795Average bandwidth calculations over the given time in milliseconds. Default:
796500ms.
797.TP
c8eeb9df
JA
798.BI iopsavgtime \fR=\fPint
799Average IOPS calculations over the given time in milliseconds. Default:
800500ms.
801.TP
d60e92d1 802.BI create_serialize \fR=\fPbool
d1429b5c 803If true, serialize file creation for the jobs. Default: true.
d60e92d1
AC
804.TP
805.BI create_fsync \fR=\fPbool
806\fIfsync\fR\|(2) data file after creation. Default: true.
807.TP
6b7f6851
JA
808.BI create_on_open \fR=\fPbool
809If true, the files are not created until they are opened for IO by the job.
810.TP
25460cf6
JA
811.BI create_only \fR=\fPbool
812If true, fio will only run the setup phase of the job. If files need to be
813laid out or updated on disk, only that will be done. The actual job contents
814are not executed.
815.TP
e9f48479
JA
816.BI pre_read \fR=\fPbool
817If this is given, files will be pre-read into memory before starting the given
818IO operation. This will also clear the \fR \fBinvalidate\fR flag, since it is
9c0d2241
JA
819pointless to pre-read and then drop the cache. This will only work for IO
820engines that are seekable, since they allow you to read the same data
821multiple times. Thus it will not work on eg network or splice IO.
e9f48479 822.TP
d60e92d1
AC
823.BI unlink \fR=\fPbool
824Unlink job files when done. Default: false.
825.TP
826.BI loops \fR=\fPint
827Specifies the number of iterations (runs of the same workload) of this job.
828Default: 1.
829.TP
830.BI do_verify \fR=\fPbool
831Run the verify phase after a write phase. Only valid if \fBverify\fR is set.
832Default: true.
833.TP
834.BI verify \fR=\fPstr
835Method of verifying file contents after each iteration of the job. Allowed
836values are:
837.RS
838.RS
839.TP
b892dc08 840.B md5 crc16 crc32 crc32c crc32c-intel crc64 crc7 sha256 sha512 sha1
0539d758
JA
841Store appropriate checksum in the header of each block. crc32c-intel is
842hardware accelerated SSE4.2 driven, falls back to regular crc32c if
843not supported by the system.
d60e92d1
AC
844.TP
845.B meta
846Write extra information about each I/O (timestamp, block number, etc.). The
996093bb 847block number is verified. See \fBverify_pattern\fR as well.
d60e92d1
AC
848.TP
849.B null
850Pretend to verify. Used for testing internals.
851.RE
b892dc08
JA
852
853This option can be used for repeated burn-in tests of a system to make sure
854that the written data is also correctly read back. If the data direction given
855is a read or random read, fio will assume that it should verify a previously
856written file. If the data direction includes any form of write, the verify will
857be of the newly written data.
d60e92d1
AC
858.RE
859.TP
860.BI verify_sort \fR=\fPbool
861If true, written verify blocks are sorted if \fBfio\fR deems it to be faster to
862read them back in a sorted manner. Default: true.
863.TP
f7fa2653 864.BI verify_offset \fR=\fPint
d60e92d1 865Swap the verification header with data somewhere else in the block before
d1429b5c 866writing. It is swapped back before verifying.
d60e92d1 867.TP
f7fa2653 868.BI verify_interval \fR=\fPint
d60e92d1
AC
869Write the verification header for this number of bytes, which should divide
870\fBblocksize\fR. Default: \fBblocksize\fR.
871.TP
996093bb
JA
872.BI verify_pattern \fR=\fPstr
873If set, fio will fill the io buffers with this pattern. Fio defaults to filling
874with totally random bytes, but sometimes it's interesting to fill with a known
875pattern for io verification purposes. Depending on the width of the pattern,
876fio will fill 1/2/3/4 bytes of the buffer at the time(it can be either a
877decimal or a hex number). The verify_pattern if larger than a 32-bit quantity
878has to be a hex number that starts with either "0x" or "0X". Use with
879\fBverify\fP=meta.
880.TP
d60e92d1
AC
881.BI verify_fatal \fR=\fPbool
882If true, exit the job on the first observed verification failure. Default:
883false.
884.TP
b463e936
JA
885.BI verify_dump \fR=\fPbool
886If set, dump the contents of both the original data block and the data block we
887read off disk to files. This allows later analysis to inspect just what kind of
ef71e317 888data corruption occurred. Off by default.
b463e936 889.TP
e8462bd8
JA
890.BI verify_async \fR=\fPint
891Fio will normally verify IO inline from the submitting thread. This option
892takes an integer describing how many async offload threads to create for IO
893verification instead, causing fio to offload the duty of verifying IO contents
c85c324c
JA
894to one or more separate threads. If using this offload option, even sync IO
895engines can benefit from using an \fBiodepth\fR setting higher than 1, as it
896allows them to have IO in flight while verifies are running.
e8462bd8
JA
897.TP
898.BI verify_async_cpus \fR=\fPstr
899Tell fio to set the given CPU affinity on the async IO verification threads.
900See \fBcpus_allowed\fP for the format used.
901.TP
6f87418f
JA
902.BI verify_backlog \fR=\fPint
903Fio will normally verify the written contents of a job that utilizes verify
904once that job has completed. In other words, everything is written then
905everything is read back and verified. You may want to verify continually
906instead for a variety of reasons. Fio stores the meta data associated with an
907IO block in memory, so for large verify workloads, quite a bit of memory would
092f707f
DN
908be used up holding this meta data. If this option is enabled, fio will write
909only N blocks before verifying these blocks.
6f87418f
JA
910.TP
911.BI verify_backlog_batch \fR=\fPint
912Control how many blocks fio will verify if verify_backlog is set. If not set,
913will default to the value of \fBverify_backlog\fR (meaning the entire queue is
092f707f
DN
914read back and verified). If \fBverify_backlog_batch\fR is less than
915\fBverify_backlog\fR then not all blocks will be verified, if
916\fBverify_backlog_batch\fR is larger than \fBverify_backlog\fR, some blocks
917will be verified more than once.
6f87418f 918.TP
d392365e 919.B stonewall "\fR,\fP wait_for_previous"
5982a925 920Wait for preceding jobs in the job file to exit before starting this one.
d60e92d1
AC
921\fBstonewall\fR implies \fBnew_group\fR.
922.TP
923.B new_group
924Start a new reporting group. If not given, all jobs in a file will be part
925of the same reporting group, unless separated by a stonewall.
926.TP
927.BI numjobs \fR=\fPint
928Number of clones (processes/threads performing the same workload) of this job.
929Default: 1.
930.TP
931.B group_reporting
932If set, display per-group reports instead of per-job when \fBnumjobs\fR is
933specified.
934.TP
935.B thread
936Use threads created with \fBpthread_create\fR\|(3) instead of processes created
937with \fBfork\fR\|(2).
938.TP
f7fa2653 939.BI zonesize \fR=\fPint
d60e92d1
AC
940Divide file into zones of the specified size in bytes. See \fBzoneskip\fR.
941.TP
f7fa2653 942.BI zoneskip \fR=\fPint
d1429b5c 943Skip the specified number of bytes when \fBzonesize\fR bytes of data have been
d60e92d1
AC
944read.
945.TP
946.BI write_iolog \fR=\fPstr
5b42a488
SH
947Write the issued I/O patterns to the specified file. Specify a separate file
948for each job, otherwise the iologs will be interspersed and the file may be
949corrupt.
d60e92d1
AC
950.TP
951.BI read_iolog \fR=\fPstr
952Replay the I/O patterns contained in the specified file generated by
953\fBwrite_iolog\fR, or may be a \fBblktrace\fR binary file.
954.TP
64bbb865
DN
955.BI replay_no_stall \fR=\fPint
956While replaying I/O patterns using \fBread_iolog\fR the default behavior
957attempts to respect timing information between I/Os. Enabling
958\fBreplay_no_stall\fR causes I/Os to be replayed as fast as possible while
959still respecting ordering.
960.TP
d1c46c04
DN
961.BI replay_redirect \fR=\fPstr
962While replaying I/O patterns using \fBread_iolog\fR the default behavior
963is to replay the IOPS onto the major/minor device that each IOP was recorded
964from. Setting \fBreplay_redirect\fR causes all IOPS to be replayed onto the
965single specified device regardless of the device it was recorded from.
966.TP
836bad52 967.BI write_bw_log \fR=\fPstr
901bb994
JA
968If given, write a bandwidth log of the jobs in this job file. Can be used to
969store data of the bandwidth of the jobs in their lifetime. The included
970fio_generate_plots script uses gnuplot to turn these text files into nice
971graphs. See \fBwrite_log_log\fR for behaviour of given filename. For this
972option, the postfix is _bw.log.
d60e92d1 973.TP
836bad52 974.BI write_lat_log \fR=\fPstr
901bb994
JA
975Same as \fBwrite_bw_log\fR, but writes I/O completion latencies. If no
976filename is given with this option, the default filename of "jobname_type.log"
977is used. Even if the filename is given, fio will still append the type of log.
978.TP
c8eeb9df
JA
979.BI write_iops_log \fR=\fPstr
980Same as \fBwrite_bw_log\fR, but writes IOPS. If no filename is given with this
981option, the default filename of "jobname_type.log" is used. Even if the
982filename is given, fio will still append the type of log.
983.TP
b8bc8cba
JA
984.BI log_avg_msec \fR=\fPint
985By default, fio will log an entry in the iops, latency, or bw log for every
986IO that completes. When writing to the disk log, that can quickly grow to a
987very large size. Setting this option makes fio average the each log entry
988over the specified period of time, reducing the resolution of the log.
989Defaults to 0.
990.TP
836bad52 991.BI disable_lat \fR=\fPbool
02af0988 992Disable measurements of total latency numbers. Useful only for cutting
901bb994
JA
993back the number of calls to gettimeofday, as that does impact performance at
994really high IOPS rates. Note that to really get rid of a large amount of these
995calls, this option must be used with disable_slat and disable_bw as well.
996.TP
836bad52 997.BI disable_clat \fR=\fPbool
c95f9daf 998Disable measurements of completion latency numbers. See \fBdisable_lat\fR.
02af0988 999.TP
836bad52 1000.BI disable_slat \fR=\fPbool
02af0988 1001Disable measurements of submission latency numbers. See \fBdisable_lat\fR.
901bb994 1002.TP
836bad52 1003.BI disable_bw_measurement \fR=\fPbool
02af0988 1004Disable measurements of throughput/bandwidth numbers. See \fBdisable_lat\fR.
d60e92d1 1005.TP
f7fa2653 1006.BI lockmem \fR=\fPint
d60e92d1
AC
1007Pin the specified amount of memory with \fBmlock\fR\|(2). Can be used to
1008simulate a smaller amount of memory.
1009.TP
1010.BI exec_prerun \fR=\fPstr
1011Before running the job, execute the specified command with \fBsystem\fR\|(3).
1012.TP
1013.BI exec_postrun \fR=\fPstr
1014Same as \fBexec_prerun\fR, but the command is executed after the job completes.
1015.TP
1016.BI ioscheduler \fR=\fPstr
1017Attempt to switch the device hosting the file to the specified I/O scheduler.
1018.TP
1019.BI cpuload \fR=\fPint
1020If the job is a CPU cycle-eater, attempt to use the specified percentage of
1021CPU cycles.
1022.TP
1023.BI cpuchunks \fR=\fPint
1024If the job is a CPU cycle-eater, split the load into cycles of the
1025given time in milliseconds.
1026.TP
1027.BI disk_util \fR=\fPbool
d1429b5c 1028Generate disk utilization statistics if the platform supports it. Default: true.
901bb994 1029.TP
23893646
JA
1030.BI clocksource \fR=\fPstr
1031Use the given clocksource as the base of timing. The supported options are:
1032.RS
1033.TP
1034.B gettimeofday
1035gettimeofday(2)
1036.TP
1037.B clock_gettime
1038clock_gettime(2)
1039.TP
1040.B cpu
1041Internal CPU clock source
1042.TP
1043.RE
1044.P
1045\fBcpu\fR is the preferred clocksource if it is reliable, as it is very fast
1046(and fio is heavy on time calls). Fio will automatically use this clocksource
1047if it's supported and considered reliable on the system it is running on,
1048unless another clocksource is specifically set. For x86/x86-64 CPUs, this
1049means supporting TSC Invariant.
1050.TP
901bb994
JA
1051.BI gtod_reduce \fR=\fPbool
1052Enable all of the gettimeofday() reducing options (disable_clat, disable_slat,
1053disable_bw) plus reduce precision of the timeout somewhat to really shrink the
1054gettimeofday() call count. With this option enabled, we only do about 0.4% of
1055the gtod() calls we would have done if all time keeping was enabled.
1056.TP
1057.BI gtod_cpu \fR=\fPint
1058Sometimes it's cheaper to dedicate a single thread of execution to just getting
1059the current time. Fio (and databases, for instance) are very intensive on
1060gettimeofday() calls. With this option, you can set one CPU aside for doing
1061nothing but logging current time to a shared memory location. Then the other
1062threads/processes that run IO workloads need only copy that segment, instead of
1063entering the kernel with a gettimeofday() call. The CPU set aside for doing
1064these time calls will be excluded from other uses. Fio will manually clear it
1065from the CPU mask of other jobs.
f2bba182 1066.TP
8b28bd41
DM
1067.BI ignore_error \fR=\fPstr
1068Sometimes you want to ignore some errors during test in that case you can specify
1069error list for each error type.
1070.br
1071ignore_error=READ_ERR_LIST,WRITE_ERR_LIST,VERIFY_ERR_LIST
1072.br
1073errors for given error type is separated with ':'.
1074Error may be symbol ('ENOSPC', 'ENOMEM') or an integer.
1075.br
1076Example: ignore_error=EAGAIN,ENOSPC:122 .
1077.br
1078This option will ignore EAGAIN from READ, and ENOSPC and 122(EDQUOT) from WRITE.
1079.TP
1080.BI error_dump \fR=\fPbool
1081If set dump every error even if it is non fatal, true by default. If disabled
1082only fatal error will be dumped
1083.TP
a696fa2a
JA
1084.BI cgroup \fR=\fPstr
1085Add job to this control group. If it doesn't exist, it will be created.
6adb38a1
JA
1086The system must have a mounted cgroup blkio mount point for this to work. If
1087your system doesn't have it mounted, you can do so with:
1088
5982a925 1089# mount \-t cgroup \-o blkio none /cgroup
a696fa2a
JA
1090.TP
1091.BI cgroup_weight \fR=\fPint
1092Set the weight of the cgroup to this value. See the documentation that comes
1093with the kernel, allowed values are in the range of 100..1000.
e0b0d892 1094.TP
7de87099
VG
1095.BI cgroup_nodelete \fR=\fPbool
1096Normally fio will delete the cgroups it has created after the job completion.
1097To override this behavior and to leave cgroups around after the job completion,
1098set cgroup_nodelete=1. This can be useful if one wants to inspect various
1099cgroup files after job completion. Default: false
1100.TP
e0b0d892
JA
1101.BI uid \fR=\fPint
1102Instead of running as the invoking user, set the user ID to this value before
1103the thread/process does any work.
1104.TP
1105.BI gid \fR=\fPint
1106Set group ID, see \fBuid\fR.
83349190 1107.TP
9e684a49
DE
1108.BI flow_id \fR=\fPint
1109The ID of the flow. If not specified, it defaults to being a global flow. See
1110\fBflow\fR.
1111.TP
1112.BI flow \fR=\fPint
1113Weight in token-based flow control. If this value is used, then there is a
1114\fBflow counter\fR which is used to regulate the proportion of activity between
1115two or more jobs. fio attempts to keep this flow counter near zero. The
1116\fBflow\fR parameter stands for how much should be added or subtracted to the
1117flow counter on each iteration of the main I/O loop. That is, if one job has
1118\fBflow=8\fR and another job has \fBflow=-1\fR, then there will be a roughly
11191:8 ratio in how much one runs vs the other.
1120.TP
1121.BI flow_watermark \fR=\fPint
1122The maximum value that the absolute value of the flow counter is allowed to
1123reach before the job must wait for a lower value of the counter.
1124.TP
1125.BI flow_sleep \fR=\fPint
1126The period of time, in microseconds, to wait after the flow watermark has been
1127exceeded before retrying operations
1128.TP
83349190
YH
1129.BI clat_percentiles \fR=\fPbool
1130Enable the reporting of percentiles of completion latencies.
1131.TP
1132.BI percentile_list \fR=\fPfloat_list
1133Overwrite the default list of percentiles for completion
1134latencies. Each number is a floating number in the range (0,100], and
1135the maximum length of the list is 20. Use ':' to separate the
3eb07285 1136numbers. For example, \-\-percentile_list=99.5:99.9 will cause fio to
83349190
YH
1137report the values of completion latency below which 99.5% and 99.9% of
1138the observed latencies fell, respectively.
de890a1e
SL
1139.SS "Ioengine Parameters List"
1140Some parameters are only valid when a specific ioengine is in use. These are
1141used identically to normal parameters, with the caveat that when used on the
1142command line, the must come after the ioengine that defines them is selected.
1143.TP
1144.BI (libaio)userspace_reap
1145Normally, with the libaio engine in use, fio will use
1146the io_getevents system call to reap newly returned events.
1147With this flag turned on, the AIO ring will be read directly
1148from user-space to reap events. The reaping mode is only
1149enabled when polling for a minimum of 0 events (eg when
1150iodepth_batch_complete=0).
1151.TP
1152.BI (net,netsplice)hostname \fR=\fPstr
1153The host name or IP address to use for TCP or UDP based IO.
1154If the job is a TCP listener or UDP reader, the hostname is not
1155used and must be omitted.
1156.TP
1157.BI (net,netsplice)port \fR=\fPint
1158The TCP or UDP port to bind to or connect to.
1159.TP
1d360ffb
JA
1160.BI (net,netsplice)nodelay \fR=\fPbool
1161Set TCP_NODELAY on TCP connections.
1162.TP
de890a1e
SL
1163.BI (net,netsplice)protocol \fR=\fPstr "\fR,\fP proto" \fR=\fPstr
1164The network protocol to use. Accepted values are:
1165.RS
1166.RS
1167.TP
1168.B tcp
1169Transmission control protocol
1170.TP
1171.B udp
f5cc3d0e 1172User datagram protocol
de890a1e
SL
1173.TP
1174.B unix
1175UNIX domain socket
1176.RE
1177.P
1178When the protocol is TCP or UDP, the port must also be given,
1179as well as the hostname if the job is a TCP listener or UDP
1180reader. For unix sockets, the normal filename option should be
1181used and the port is invalid.
1182.RE
1183.TP
1184.BI (net,netsplice)listen
1185For TCP network connections, tell fio to listen for incoming
1186connections rather than initiating an outgoing connection. The
1187hostname must be omitted if this option is used.
d54fce84 1188.TP
7aeb1e94
JA
1189.BI (net, pingpong) \fR=\fPbool
1190Normal a network writer will just continue writing data, and a network reader
1191will just consume packages. If pingpong=1 is set, a writer will send its normal
1192payload to the reader, then wait for the reader to send the same payload back.
1193This allows fio to measure network latencies. The submission and completion
1194latencies then measure local time spent sending or receiving, and the
1195completion latency measures how long it took for the other end to receive and
1196send back.
1197.TP
d54fce84
DM
1198.BI (e4defrag,donorname) \fR=\fPstr
1199File will be used as a block donor (swap extents between files)
1200.TP
1201.BI (e4defrag,inplace) \fR=\fPint
1202Configure donor file block allocation strategy
1203.RS
1204.BI 0(default) :
1205Preallocate donor's file on init
1206.TP
1207.BI 1:
1208allocate space immidietly inside defragment event, and free right after event
1209.RE
1210.TP
d60e92d1 1211.SH OUTPUT
d1429b5c
AC
1212While running, \fBfio\fR will display the status of the created jobs. For
1213example:
d60e92d1 1214.RS
d1429b5c 1215.P
d60e92d1
AC
1216Threads: 1: [_r] [24.8% done] [ 13509/ 8334 kb/s] [eta 00h:01m:31s]
1217.RE
1218.P
d1429b5c
AC
1219The characters in the first set of brackets denote the current status of each
1220threads. The possible values are:
1221.P
1222.PD 0
d60e92d1
AC
1223.RS
1224.TP
1225.B P
1226Setup but not started.
1227.TP
1228.B C
1229Thread created.
1230.TP
1231.B I
1232Initialized, waiting.
1233.TP
1234.B R
1235Running, doing sequential reads.
1236.TP
1237.B r
1238Running, doing random reads.
1239.TP
1240.B W
1241Running, doing sequential writes.
1242.TP
1243.B w
1244Running, doing random writes.
1245.TP
1246.B M
1247Running, doing mixed sequential reads/writes.
1248.TP
1249.B m
1250Running, doing mixed random reads/writes.
1251.TP
1252.B F
1253Running, currently waiting for \fBfsync\fR\|(2).
1254.TP
1255.B V
1256Running, verifying written data.
1257.TP
1258.B E
1259Exited, not reaped by main thread.
1260.TP
1261.B \-
1262Exited, thread reaped.
1263.RE
d1429b5c 1264.PD
d60e92d1
AC
1265.P
1266The second set of brackets shows the estimated completion percentage of
1267the current group. The third set shows the read and write I/O rate,
1268respectively. Finally, the estimated run time of the job is displayed.
1269.P
1270When \fBfio\fR completes (or is interrupted by Ctrl-C), it will show data
1271for each thread, each group of threads, and each disk, in that order.
1272.P
1273Per-thread statistics first show the threads client number, group-id, and
1274error code. The remaining figures are as follows:
1275.RS
d60e92d1
AC
1276.TP
1277.B io
1278Number of megabytes of I/O performed.
1279.TP
1280.B bw
1281Average data rate (bandwidth).
1282.TP
1283.B runt
1284Threads run time.
1285.TP
1286.B slat
1287Submission latency minimum, maximum, average and standard deviation. This is
1288the time it took to submit the I/O.
1289.TP
1290.B clat
1291Completion latency minimum, maximum, average and standard deviation. This
1292is the time between submission and completion.
1293.TP
1294.B bw
1295Bandwidth minimum, maximum, percentage of aggregate bandwidth received, average
1296and standard deviation.
1297.TP
1298.B cpu
1299CPU usage statistics. Includes user and system time, number of context switches
1300this thread went through and number of major and minor page faults.
1301.TP
1302.B IO depths
1303Distribution of I/O depths. Each depth includes everything less than (or equal)
1304to it, but greater than the previous depth.
1305.TP
1306.B IO issued
1307Number of read/write requests issued, and number of short read/write requests.
1308.TP
1309.B IO latencies
1310Distribution of I/O completion latencies. The numbers follow the same pattern
1311as \fBIO depths\fR.
1312.RE
d60e92d1
AC
1313.P
1314The group statistics show:
d1429b5c 1315.PD 0
d60e92d1
AC
1316.RS
1317.TP
1318.B io
1319Number of megabytes I/O performed.
1320.TP
1321.B aggrb
1322Aggregate bandwidth of threads in the group.
1323.TP
1324.B minb
1325Minimum average bandwidth a thread saw.
1326.TP
1327.B maxb
1328Maximum average bandwidth a thread saw.
1329.TP
1330.B mint
d1429b5c 1331Shortest runtime of threads in the group.
d60e92d1
AC
1332.TP
1333.B maxt
1334Longest runtime of threads in the group.
1335.RE
d1429b5c 1336.PD
d60e92d1
AC
1337.P
1338Finally, disk statistics are printed with reads first:
d1429b5c 1339.PD 0
d60e92d1
AC
1340.RS
1341.TP
1342.B ios
1343Number of I/Os performed by all groups.
1344.TP
1345.B merge
1346Number of merges in the I/O scheduler.
1347.TP
1348.B ticks
1349Number of ticks we kept the disk busy.
1350.TP
1351.B io_queue
1352Total time spent in the disk queue.
1353.TP
1354.B util
1355Disk utilization.
1356.RE
d1429b5c 1357.PD
8423bd11
JA
1358.P
1359It is also possible to get fio to dump the current output while it is
1360running, without terminating the job. To do that, send fio the \fBUSR1\fR
1361signal.
d60e92d1
AC
1362.SH TERSE OUTPUT
1363If the \fB\-\-minimal\fR option is given, the results will be printed in a
562c2d2f
DN
1364semicolon-delimited format suitable for scripted use - a job description
1365(if provided) follows on a new line. Note that the first
525c2bfa
JA
1366number in the line is the version number. If the output has to be changed
1367for some reason, this number will be incremented by 1 to signify that
1368change. The fields are:
d60e92d1
AC
1369.P
1370.RS
5e726d0a 1371.B terse version, fio version, jobname, groupid, error
d60e92d1
AC
1372.P
1373Read status:
1374.RS
312b4af2 1375.B Total I/O \fR(KB)\fP, bandwidth \fR(KB/s)\fP, IOPS, runtime \fR(ms)\fP
d60e92d1
AC
1376.P
1377Submission latency:
1378.RS
1379.B min, max, mean, standard deviation
1380.RE
1381Completion latency:
1382.RS
1383.B min, max, mean, standard deviation
1384.RE
1db92cb6
JA
1385Completion latency percentiles (20 fields):
1386.RS
1387.B Xth percentile=usec
1388.RE
525c2bfa
JA
1389Total latency:
1390.RS
1391.B min, max, mean, standard deviation
1392.RE
d60e92d1
AC
1393Bandwidth:
1394.RS
1395.B min, max, aggregate percentage of total, mean, standard deviation
1396.RE
1397.RE
1398.P
1399Write status:
1400.RS
312b4af2 1401.B Total I/O \fR(KB)\fP, bandwidth \fR(KB/s)\fP, IOPS, runtime \fR(ms)\fP
d60e92d1
AC
1402.P
1403Submission latency:
1404.RS
1405.B min, max, mean, standard deviation
1406.RE
1407Completion latency:
1408.RS
1409.B min, max, mean, standard deviation
1410.RE
1db92cb6
JA
1411Completion latency percentiles (20 fields):
1412.RS
1413.B Xth percentile=usec
1414.RE
525c2bfa
JA
1415Total latency:
1416.RS
1417.B min, max, mean, standard deviation
1418.RE
d60e92d1
AC
1419Bandwidth:
1420.RS
1421.B min, max, aggregate percentage of total, mean, standard deviation
1422.RE
1423.RE
1424.P
d1429b5c 1425CPU usage:
d60e92d1 1426.RS
bd2626f0 1427.B user, system, context switches, major page faults, minor page faults
d60e92d1
AC
1428.RE
1429.P
1430IO depth distribution:
1431.RS
1432.B <=1, 2, 4, 8, 16, 32, >=64
1433.RE
1434.P
562c2d2f 1435IO latency distribution:
d60e92d1 1436.RS
562c2d2f
DN
1437Microseconds:
1438.RS
1439.B <=2, 4, 10, 20, 50, 100, 250, 500, 750, 1000
1440.RE
1441Milliseconds:
1442.RS
1443.B <=2, 4, 10, 20, 50, 100, 250, 500, 750, 1000, 2000, >=2000
1444.RE
1445.RE
1446.P
f2f788dd
JA
1447Disk utilization (1 for each disk used):
1448.RS
1449.B name, read ios, write ios, read merges, write merges, read ticks, write ticks, read in-queue time, write in-queue time, disk utilization percentage
1450.RE
1451.P
5982a925 1452Error Info (dependent on continue_on_error, default off):
562c2d2f
DN
1453.RS
1454.B total # errors, first error code
d60e92d1
AC
1455.RE
1456.P
562c2d2f 1457.B text description (if provided in config - appears on newline)
d60e92d1 1458.RE
49da1240
JA
1459.SH CLIENT / SERVER
1460Normally you would run fio as a stand-alone application on the machine
1461where the IO workload should be generated. However, it is also possible to
1462run the frontend and backend of fio separately. This makes it possible to
1463have a fio server running on the machine(s) where the IO workload should
1464be running, while controlling it from another machine.
1465
1466To start the server, you would do:
1467
1468\fBfio \-\-server=args\fR
1469
1470on that machine, where args defines what fio listens to. The arguments
811826be 1471are of the form 'type:hostname or IP:port'. 'type' is either 'ip' (or ip4)
20c67f10
MS
1472for TCP/IP v4, 'ip6' for TCP/IP v6, or 'sock' for a local unix domain
1473socket. 'hostname' is either a hostname or IP address, and 'port' is the port to
811826be 1474listen to (only valid for TCP/IP, not a local socket). Some examples:
49da1240 1475
e01e9745 14761) fio \-\-server
49da1240
JA
1477
1478 Start a fio server, listening on all interfaces on the default port (8765).
1479
e01e9745 14802) fio \-\-server=ip:hostname,4444
49da1240
JA
1481
1482 Start a fio server, listening on IP belonging to hostname and on port 4444.
1483
e01e9745 14843) fio \-\-server=ip6:::1,4444
811826be
JA
1485
1486 Start a fio server, listening on IPv6 localhost ::1 and on port 4444.
1487
e01e9745 14884) fio \-\-server=,4444
49da1240
JA
1489
1490 Start a fio server, listening on all interfaces on port 4444.
1491
e01e9745 14925) fio \-\-server=1.2.3.4
49da1240
JA
1493
1494 Start a fio server, listening on IP 1.2.3.4 on the default port.
1495
e01e9745 14966) fio \-\-server=sock:/tmp/fio.sock
49da1240
JA
1497
1498 Start a fio server, listening on the local socket /tmp/fio.sock.
1499
1500When a server is running, you can connect to it from a client. The client
1501is run with:
1502
e01e9745 1503fio \-\-local-args \-\-client=server \-\-remote-args <job file(s)>
49da1240 1504
e01e9745
MS
1505where \-\-local-args are arguments that are local to the client where it is
1506running, 'server' is the connect string, and \-\-remote-args and <job file(s)>
49da1240
JA
1507are sent to the server. The 'server' string follows the same format as it
1508does on the server side, to allow IP/hostname/socket and port strings.
1509You can connect to multiple clients as well, to do that you could run:
1510
e01e9745 1511fio \-\-client=server2 \-\-client=server2 <job file(s)>
d60e92d1 1512.SH AUTHORS
49da1240 1513
d60e92d1 1514.B fio
aa58d252
JA
1515was written by Jens Axboe <jens.axboe@oracle.com>,
1516now Jens Axboe <jaxboe@fusionio.com>.
d1429b5c
AC
1517.br
1518This man page was written by Aaron Carroll <aaronc@cse.unsw.edu.au> based
d60e92d1
AC
1519on documentation by Jens Axboe.
1520.SH "REPORTING BUGS"
482900c9 1521Report bugs to the \fBfio\fR mailing list <fio@vger.kernel.org>.
d1429b5c 1522See \fBREADME\fR.
d60e92d1 1523.SH "SEE ALSO"
d1429b5c
AC
1524For further documentation see \fBHOWTO\fR and \fBREADME\fR.
1525.br
1526Sample jobfiles are available in the \fBexamples\fR directory.
d60e92d1 1527