Avoid segfault if verify header appears longer than io_u
[fio.git] / fio.1
CommitLineData
d60e92d1
AC
1.TH fio 1 "September 2007" "User Manual"
2.SH NAME
3fio \- flexible I/O tester
4.SH SYNOPSIS
5.B fio
6[\fIoptions\fR] [\fIjobfile\fR]...
7.SH DESCRIPTION
8.B fio
9is a tool that will spawn a number of threads or processes doing a
10particular type of I/O action as specified by the user.
11The typical use of fio is to write a job file matching the I/O load
12one wants to simulate.
13.SH OPTIONS
14.TP
49da1240
JA
15.BI \-\-debug \fR=\fPtype
16Enable verbose tracing of various fio actions. May be `all' for all types
17or individual types separated by a comma (eg \-\-debug=io,file). `help' will
18list all available tracing options.
19.TP
d60e92d1
AC
20.BI \-\-output \fR=\fPfilename
21Write output to \fIfilename\fR.
22.TP
23.BI \-\-timeout \fR=\fPtimeout
24Limit run time to \fItimeout\fR seconds.
25.TP
26.B \-\-latency\-log
27Generate per-job latency logs.
28.TP
29.B \-\-bandwidth\-log
30Generate per-job bandwidth logs.
31.TP
32.B \-\-minimal
d1429b5c 33Print statistics in a terse, semicolon-delimited format.
d60e92d1 34.TP
49da1240
JA
35.B \-\-version
36Display version information and exit.
37.TP
065248bf 38.BI \-\-terse\-version \fR=\fPversion
4d658652 39Set terse version output format (Current version 3, or older version 2).
49da1240
JA
40.TP
41.B \-\-help
42Display usage information and exit.
43.TP
44.BI \-\-cmdhelp \fR=\fPcommand
45Print help information for \fIcommand\fR. May be `all' for all commands.
46.TP
de890a1e
SL
47.BI \-\-enghelp \fR=\fPioengine[,command]
48List all commands defined by \fIioengine\fR, or print help for \fIcommand\fR defined by \fIioengine\fR.
49.TP
d60e92d1
AC
50.BI \-\-showcmd \fR=\fPjobfile
51Convert \fIjobfile\fR to a set of command-line options.
52.TP
53.B \-\-readonly
54Enable read-only safety checks.
55.TP
56.BI \-\-eta \fR=\fPwhen
57Specifies when real-time ETA estimate should be printed. \fIwhen\fR may
58be one of `always', `never' or `auto'.
59.TP
49da1240
JA
60.BI \-\-readonly
61Turn on safety read-only checks, preventing any attempted write.
62.TP
c0a5d35e 63.BI \-\-section \fR=\fPsec
49da1240 64Only run section \fIsec\fR from job file. Multiple of these options can be given, adding more sections to run.
c0a5d35e 65.TP
49da1240
JA
66.BI \-\-alloc\-size \fR=\fPkb
67Set the internal smalloc pool size to \fIkb\fP kilobytes.
d60e92d1 68.TP
49da1240
JA
69.BI \-\-warnings\-fatal
70All fio parser warnings are fatal, causing fio to exit with an error.
9183788d 71.TP
49da1240
JA
72.BI \-\-max\-jobs \fR=\fPnr
73Set the maximum allowed number of jobs (threads/processes) to suport.
d60e92d1 74.TP
49da1240
JA
75.BI \-\-server \fR=\fPargs
76Start a backend server, with \fIargs\fP specifying what to listen to. See client/server section.
f57a9c59 77.TP
49da1240
JA
78.BI \-\-daemonize \fR=\fPpidfile
79Background a fio server, writing the pid to the given pid file.
80.TP
81.BI \-\-client \fR=\fPhost
82Instead of running the jobs locally, send and run them on the given host.
d60e92d1
AC
83.SH "JOB FILE FORMAT"
84Job files are in `ini' format. They consist of one or more
85job definitions, which begin with a job name in square brackets and
86extend to the next job name. The job name can be any ASCII string
87except `global', which has a special meaning. Following the job name is
88a sequence of zero or more parameters, one per line, that define the
89behavior of the job. Any line starting with a `;' or `#' character is
d1429b5c 90considered a comment and ignored.
d9956b64
AC
91.P
92If \fIjobfile\fR is specified as `-', the job file will be read from
93standard input.
d60e92d1
AC
94.SS "Global Section"
95The global section contains default parameters for jobs specified in the
96job file. A job is only affected by global sections residing above it,
97and there may be any number of global sections. Specific job definitions
98may override any parameter set in global sections.
99.SH "JOB PARAMETERS"
100.SS Types
101Some parameters may take arguments of a specific type. The types used are:
102.TP
103.I str
104String: a sequence of alphanumeric characters.
105.TP
106.I int
d60e92d1 107SI integer: a whole number, possibly containing a suffix denoting the base unit
b09da8fa
JA
108of the value. Accepted suffixes are `k', 'M', 'G', 'T', and 'P', denoting
109kilo (1024), mega (1024^2), giga (1024^3), tera (1024^4), and peta (1024^5)
110respectively. The suffix is not case sensitive. If prefixed with '0x', the
5982a925
MS
111value is assumed to be base 16 (hexadecimal). A suffix may include a trailing 'b',
112for instance 'kb' is identical to 'k'. You can specify a base 10 value
57fc29fa
JA
113by using 'KiB', 'MiB', 'GiB', etc. This is useful for disk drives where
114values are often given in base 10 values. Specifying '30GiB' will get you
11530*1000^3 bytes.
d60e92d1
AC
116.TP
117.I bool
118Boolean: a true or false value. `0' denotes false, `1' denotes true.
119.TP
120.I irange
121Integer range: a range of integers specified in the format
d1429b5c
AC
122\fIlower\fR:\fIupper\fR or \fIlower\fR\-\fIupper\fR. \fIlower\fR and
123\fIupper\fR may contain a suffix as described above. If an option allows two
124sets of ranges, they are separated with a `,' or `/' character. For example:
125`8\-8k/8M\-4G'.
83349190
YH
126.TP
127.I float_list
128List of floating numbers: A list of floating numbers, separated by
129a ':' charcater.
d60e92d1
AC
130.SS "Parameter List"
131.TP
132.BI name \fR=\fPstr
d9956b64 133May be used to override the job name. On the command line, this parameter
d60e92d1
AC
134has the special purpose of signalling the start of a new job.
135.TP
136.BI description \fR=\fPstr
137Human-readable description of the job. It is printed when the job is run, but
138otherwise has no special purpose.
139.TP
140.BI directory \fR=\fPstr
141Prefix filenames with this directory. Used to place files in a location other
142than `./'.
143.TP
144.BI filename \fR=\fPstr
145.B fio
146normally makes up a file name based on the job name, thread number, and file
d1429b5c 147number. If you want to share files between threads in a job or several jobs,
de890a1e
SL
148specify a \fIfilename\fR for each of them to override the default.
149If the I/O engine is file-based, you can specify
d1429b5c
AC
150a number of files by separating the names with a `:' character. `\-' is a
151reserved name, meaning stdin or stdout, depending on the read/write direction
152set.
d60e92d1 153.TP
3ce9dcaf
JA
154.BI lockfile \fR=\fPstr
155Fio defaults to not locking any files before it does IO to them. If a file or
156file descriptor is shared, fio can serialize IO to that file to make the end
157result consistent. This is usual for emulating real workloads that share files.
158The lock modes are:
159.RS
160.RS
161.TP
162.B none
163No locking. This is the default.
164.TP
165.B exclusive
166Only one thread or process may do IO at the time, excluding all others.
167.TP
168.B readwrite
169Read-write locking on the file. Many readers may access the file at the same
170time, but writes get exclusive access.
171.RE
172.P
173The option may be post-fixed with a lock batch number. If set, then each
174thread/process may do that amount of IOs to the file before giving up the lock.
175Since lock acquisition is expensive, batching the lock/unlocks will speed up IO.
176.RE
177.P
d60e92d1
AC
178.BI opendir \fR=\fPstr
179Recursively open any files below directory \fIstr\fR.
180.TP
181.BI readwrite \fR=\fPstr "\fR,\fP rw" \fR=\fPstr
182Type of I/O pattern. Accepted values are:
183.RS
184.RS
185.TP
186.B read
d1429b5c 187Sequential reads.
d60e92d1
AC
188.TP
189.B write
d1429b5c 190Sequential writes.
d60e92d1
AC
191.TP
192.B randread
d1429b5c 193Random reads.
d60e92d1
AC
194.TP
195.B randwrite
d1429b5c 196Random writes.
d60e92d1
AC
197.TP
198.B rw
d1429b5c 199Mixed sequential reads and writes.
d60e92d1
AC
200.TP
201.B randrw
d1429b5c 202Mixed random reads and writes.
d60e92d1
AC
203.RE
204.P
38dad62d
JA
205For mixed I/O, the default split is 50/50. For certain types of io the result
206may still be skewed a bit, since the speed may be different. It is possible to
207specify a number of IO's to do before getting a new offset, this is one by
208appending a `:\fI<nr>\fR to the end of the string given. For a random read, it
209would look like \fBrw=randread:8\fR for passing in an offset modifier with a
059b0802
JA
210value of 8. If the postfix is used with a sequential IO pattern, then the value
211specified will be added to the generated offset for each IO. For instance,
212using \fBrw=write:4k\fR will skip 4k for every write. It turns sequential IO
213into sequential IO with holes. See the \fBrw_sequencer\fR option.
d60e92d1
AC
214.RE
215.TP
38dad62d
JA
216.BI rw_sequencer \fR=\fPstr
217If an offset modifier is given by appending a number to the \fBrw=<str>\fR line,
218then this option controls how that number modifies the IO offset being
219generated. Accepted values are:
220.RS
221.RS
222.TP
223.B sequential
224Generate sequential offset
225.TP
226.B identical
227Generate the same offset
228.RE
229.P
230\fBsequential\fR is only useful for random IO, where fio would normally
231generate a new random offset for every IO. If you append eg 8 to randread, you
232would get a new random offset for every 8 IO's. The result would be a seek for
233only every 8 IO's, instead of for every IO. Use \fBrw=randread:8\fR to specify
234that. As sequential IO is already sequential, setting \fBsequential\fR for that
235would not result in any differences. \fBidentical\fR behaves in a similar
236fashion, except it sends the same offset 8 number of times before generating a
237new offset.
238.RE
239.P
240.TP
90fef2d1
JA
241.BI kb_base \fR=\fPint
242The base unit for a kilobyte. The defacto base is 2^10, 1024. Storage
243manufacturers like to use 10^3 or 1000 as a base ten unit instead, for obvious
244reasons. Allow values are 1024 or 1000, with 1024 being the default.
245.TP
d60e92d1
AC
246.BI randrepeat \fR=\fPbool
247Seed the random number generator in a predictable way so results are repeatable
d1429b5c 248across runs. Default: true.
d60e92d1 249.TP
2615cc4b
JA
250.BI use_os_rand \fR=\fPbool
251Fio can either use the random generator supplied by the OS to generator random
252offsets, or it can use it's own internal generator (based on Tausworthe).
253Default is to use the internal generator, which is often of better quality and
254faster. Default: false.
255.TP
a596f047
EG
256.BI fallocate \fR=\fPstr
257Whether pre-allocation is performed when laying down files. Accepted values
258are:
259.RS
260.RS
261.TP
262.B none
263Do not pre-allocate space.
264.TP
265.B posix
266Pre-allocate via posix_fallocate().
267.TP
268.B keep
269Pre-allocate via fallocate() with FALLOC_FL_KEEP_SIZE set.
270.TP
271.B 0
272Backward-compatible alias for 'none'.
273.TP
274.B 1
275Backward-compatible alias for 'posix'.
276.RE
277.P
278May not be available on all supported platforms. 'keep' is only
279available on Linux. If using ZFS on Solaris this must be set to 'none'
280because ZFS doesn't support it. Default: 'posix'.
281.RE
7bc8c2cf 282.TP
d60e92d1 283.BI fadvise_hint \fR=\fPbool
23a7b043 284Use of \fIposix_fadvise\fR\|(2) to advise the kernel what I/O patterns
d1429b5c 285are likely to be issued. Default: true.
d60e92d1 286.TP
f7fa2653 287.BI size \fR=\fPint
d60e92d1
AC
288Total size of I/O for this job. \fBfio\fR will run until this many bytes have
289been transfered, unless limited by other options (\fBruntime\fR, for instance).
d7c8be03 290Unless \fBnrfiles\fR and \fBfilesize\fR options are given, this amount will be
d6667268
JA
291divided between the available files for the job. If not set, fio will use the
292full size of the given files or devices. If the the files do not exist, size
7bb59102
JA
293must be given. It is also possible to give size as a percentage between 1 and
294100. If size=20% is given, fio will use 20% of the full size of the given files
295or devices.
d60e92d1 296.TP
74586c1e 297.BI fill_device \fR=\fPbool "\fR,\fB fill_fs" \fR=\fPbool
3ce9dcaf
JA
298Sets size to something really large and waits for ENOSPC (no space left on
299device) as the terminating condition. Only makes sense with sequential write.
300For a read workload, the mount point will be filled first then IO started on
4f12432e
JA
301the result. This option doesn't make sense if operating on a raw device node,
302since the size of that is already known by the file system. Additionally,
303writing beyond end-of-device will not return ENOSPC there.
3ce9dcaf 304.TP
d60e92d1
AC
305.BI filesize \fR=\fPirange
306Individual file sizes. May be a range, in which case \fBfio\fR will select sizes
d1429b5c
AC
307for files at random within the given range, limited to \fBsize\fR in total (if
308that is given). If \fBfilesize\fR is not specified, each created file is the
309same size.
d60e92d1 310.TP
f7fa2653 311.BI blocksize \fR=\fPint[,int] "\fR,\fB bs" \fR=\fPint[,int]
d60e92d1 312Block size for I/O units. Default: 4k. Values for reads and writes can be
656ebab7 313specified separately in the format \fIread\fR,\fIwrite\fR, either of
d60e92d1
AC
314which may be empty to leave that value at its default.
315.TP
9183788d 316.BI blocksize_range \fR=\fPirange[,irange] "\fR,\fB bsrange" \fR=\fPirange[,irange]
d1429b5c
AC
317Specify a range of I/O block sizes. The issued I/O unit will always be a
318multiple of the minimum size, unless \fBblocksize_unaligned\fR is set. Applies
9183788d 319to both reads and writes if only one range is given, but can be specified
656ebab7 320separately with a comma seperating the values. Example: bsrange=1k-4k,2k-8k.
9183788d
JA
321Also (see \fBblocksize\fR).
322.TP
323.BI bssplit \fR=\fPstr
324This option allows even finer grained control of the block sizes issued,
325not just even splits between them. With this option, you can weight various
326block sizes for exact control of the issued IO for a job that has mixed
327block sizes. The format of the option is bssplit=blocksize/percentage,
5982a925 328optionally adding as many definitions as needed separated by a colon.
9183788d 329Example: bssplit=4k/10:64k/50:32k/40 would issue 50% 64k blocks, 10% 4k
c83cdd3e
JA
330blocks and 40% 32k blocks. \fBbssplit\fR also supports giving separate
331splits to reads and writes. The format is identical to what the
332\fBbs\fR option accepts, the read and write parts are separated with a
333comma.
d60e92d1
AC
334.TP
335.B blocksize_unaligned\fR,\fP bs_unaligned
d1429b5c
AC
336If set, any size in \fBblocksize_range\fR may be used. This typically won't
337work with direct I/O, as that normally requires sector alignment.
d60e92d1 338.TP
2b7a01d0 339.BI blockalign \fR=\fPint[,int] "\fR,\fB ba" \fR=\fPint[,int]
639ce0f3
MS
340At what boundary to align random IO offsets. Defaults to the same as 'blocksize'
341the minimum blocksize given. Minimum alignment is typically 512b
2b7a01d0
JA
342for using direct IO, though it usually depends on the hardware block size.
343This option is mutually exclusive with using a random map for files, so it
344will turn off that option.
43602667 345.TP
d60e92d1
AC
346.B zero_buffers
347Initialise buffers with all zeros. Default: fill buffers with random data.
348.TP
901bb994
JA
349.B refill_buffers
350If this option is given, fio will refill the IO buffers on every submit. The
351default is to only fill it at init time and reuse that data. Only makes sense
352if zero_buffers isn't specified, naturally. If data verification is enabled,
353refill_buffers is also automatically enabled.
354.TP
fd68418e
JA
355.BI scramble_buffers \fR=\fPbool
356If \fBrefill_buffers\fR is too costly and the target is using data
357deduplication, then setting this option will slightly modify the IO buffer
358contents to defeat normal de-dupe attempts. This is not enough to defeat
359more clever block compression attempts, but it will stop naive dedupe
360of blocks. Default: true.
361.TP
d60e92d1
AC
362.BI nrfiles \fR=\fPint
363Number of files to use for this job. Default: 1.
364.TP
365.BI openfiles \fR=\fPint
366Number of files to keep open at the same time. Default: \fBnrfiles\fR.
367.TP
368.BI file_service_type \fR=\fPstr
369Defines how files to service are selected. The following types are defined:
370.RS
371.RS
372.TP
373.B random
374Choose a file at random
375.TP
376.B roundrobin
377Round robin over open files (default).
6b7f6851
JA
378.B sequential
379Do each file in the set sequentially.
d60e92d1
AC
380.RE
381.P
382The number of I/Os to issue before switching a new file can be specified by
383appending `:\fIint\fR' to the service type.
384.RE
385.TP
386.BI ioengine \fR=\fPstr
387Defines how the job issues I/O. The following types are defined:
388.RS
389.RS
390.TP
391.B sync
392Basic \fIread\fR\|(2) or \fIwrite\fR\|(2) I/O. \fIfseek\fR\|(2) is used to
393position the I/O location.
394.TP
a31041ea 395.B psync
396Basic \fIpread\fR\|(2) or \fIpwrite\fR\|(2) I/O.
397.TP
9183788d
JA
398.B vsync
399Basic \fIreadv\fR\|(2) or \fIwritev\fR\|(2) I/O. Will emulate queuing by
400coalescing adjacents IOs into a single submission.
401.TP
d60e92d1 402.B libaio
de890a1e 403Linux native asynchronous I/O. This ioengine defines engine specific options.
d60e92d1
AC
404.TP
405.B posixaio
03e20d68
BC
406POSIX asynchronous I/O using \fIaio_read\fR\|(3) and \fIaio_write\fR\|(3).
407.TP
408.B solarisaio
409Solaris native asynchronous I/O.
410.TP
411.B windowsaio
412Windows native asynchronous I/O.
d60e92d1
AC
413.TP
414.B mmap
d1429b5c
AC
415File is memory mapped with \fImmap\fR\|(2) and data copied using
416\fImemcpy\fR\|(3).
d60e92d1
AC
417.TP
418.B splice
d1429b5c
AC
419\fIsplice\fR\|(2) is used to transfer the data and \fIvmsplice\fR\|(2) to
420transfer data from user-space to the kernel.
d60e92d1
AC
421.TP
422.B syslet-rw
423Use the syslet system calls to make regular read/write asynchronous.
424.TP
425.B sg
426SCSI generic sg v3 I/O. May be either synchronous using the SG_IO ioctl, or if
d1429b5c
AC
427the target is an sg character device, we use \fIread\fR\|(2) and
428\fIwrite\fR\|(2) for asynchronous I/O.
d60e92d1
AC
429.TP
430.B null
431Doesn't transfer any data, just pretends to. Mainly used to exercise \fBfio\fR
432itself and for debugging and testing purposes.
433.TP
434.B net
de890a1e
SL
435Transfer over the network. The protocol to be used can be defined with the
436\fBprotocol\fR parameter. Depending on the protocol, \fBfilename\fR,
437\fBhostname\fR, \fBport\fR, or \fBlisten\fR must be specified.
438This ioengine defines engine specific options.
d60e92d1
AC
439.TP
440.B netsplice
441Like \fBnet\fR, but uses \fIsplice\fR\|(2) and \fIvmsplice\fR\|(2) to map data
de890a1e 442and send/receive. This ioengine defines engine specific options.
d60e92d1 443.TP
53aec0a4 444.B cpuio
d60e92d1
AC
445Doesn't transfer any data, but burns CPU cycles according to \fBcpuload\fR and
446\fBcpucycles\fR parameters.
447.TP
448.B guasi
449The GUASI I/O engine is the Generic Userspace Asynchronous Syscall Interface
450approach to asycnronous I/O.
d1429b5c
AC
451.br
452See <http://www.xmailserver.org/guasi\-lib.html>.
d60e92d1 453.TP
21b8aee8 454.B rdma
85286c5c
BVA
455The RDMA I/O engine supports both RDMA memory semantics (RDMA_WRITE/RDMA_READ)
456and channel semantics (Send/Recv) for the InfiniBand, RoCE and iWARP protocols.
21b8aee8 457.TP
d60e92d1
AC
458.B external
459Loads an external I/O engine object file. Append the engine filename as
460`:\fIenginepath\fR'.
461.RE
462.RE
463.TP
464.BI iodepth \fR=\fPint
8489dae4
SK
465Number of I/O units to keep in flight against the file. Note that increasing
466iodepth beyond 1 will not affect synchronous ioengines (except for small
ee72ca09
JA
467degress when verify_async is in use). Even async engines my impose OS
468restrictions causing the desired depth not to be achieved. This may happen on
469Linux when using libaio and not setting \fBdirect\fR=1, since buffered IO is
470not async on that OS. Keep an eye on the IO depth distribution in the
471fio output to verify that the achieved depth is as expected. Default: 1.
d60e92d1
AC
472.TP
473.BI iodepth_batch \fR=\fPint
474Number of I/Os to submit at once. Default: \fBiodepth\fR.
475.TP
3ce9dcaf
JA
476.BI iodepth_batch_complete \fR=\fPint
477This defines how many pieces of IO to retrieve at once. It defaults to 1 which
478 means that we'll ask for a minimum of 1 IO in the retrieval process from the
479kernel. The IO retrieval will go on until we hit the limit set by
480\fBiodepth_low\fR. If this variable is set to 0, then fio will always check for
481completed events before queuing more IO. This helps reduce IO latency, at the
482cost of more retrieval system calls.
483.TP
d60e92d1
AC
484.BI iodepth_low \fR=\fPint
485Low watermark indicating when to start filling the queue again. Default:
486\fBiodepth\fR.
487.TP
488.BI direct \fR=\fPbool
489If true, use non-buffered I/O (usually O_DIRECT). Default: false.
490.TP
491.BI buffered \fR=\fPbool
492If true, use buffered I/O. This is the opposite of the \fBdirect\fR parameter.
493Default: true.
494.TP
f7fa2653 495.BI offset \fR=\fPint
d60e92d1
AC
496Offset in the file to start I/O. Data before the offset will not be touched.
497.TP
498.BI fsync \fR=\fPint
d1429b5c
AC
499How many I/Os to perform before issuing an \fBfsync\fR\|(2) of dirty data. If
5000, don't sync. Default: 0.
d60e92d1 501.TP
5f9099ea
JA
502.BI fdatasync \fR=\fPint
503Like \fBfsync\fR, but uses \fBfdatasync\fR\|(2) instead to only sync the
504data parts of the file. Default: 0.
505.TP
e76b1da4
JA
506.BI sync_file_range \fR=\fPstr:int
507Use sync_file_range() for every \fRval\fP number of write operations. Fio will
508track range of writes that have happened since the last sync_file_range() call.
509\fRstr\fP can currently be one or more of:
510.RS
511.TP
512.B wait_before
513SYNC_FILE_RANGE_WAIT_BEFORE
514.TP
515.B write
516SYNC_FILE_RANGE_WRITE
517.TP
518.B wait_after
519SYNC_FILE_RANGE_WRITE
520.TP
521.RE
522.P
523So if you do sync_file_range=wait_before,write:8, fio would use
524\fBSYNC_FILE_RANGE_WAIT_BEFORE | SYNC_FILE_RANGE_WRITE\fP for every 8 writes.
525Also see the sync_file_range(2) man page. This option is Linux specific.
526.TP
d60e92d1 527.BI overwrite \fR=\fPbool
d1429b5c 528If writing, setup the file first and do overwrites. Default: false.
d60e92d1
AC
529.TP
530.BI end_fsync \fR=\fPbool
d1429b5c 531Sync file contents when job exits. Default: false.
d60e92d1
AC
532.TP
533.BI fsync_on_close \fR=\fPbool
534If true, sync file contents on close. This differs from \fBend_fsync\fR in that
d1429b5c 535it will happen on every close, not just at the end of the job. Default: false.
d60e92d1
AC
536.TP
537.BI rwmixcycle \fR=\fPint
538How many milliseconds before switching between reads and writes for a mixed
539workload. Default: 500ms.
540.TP
541.BI rwmixread \fR=\fPint
542Percentage of a mixed workload that should be reads. Default: 50.
543.TP
544.BI rwmixwrite \fR=\fPint
d1429b5c 545Percentage of a mixed workload that should be writes. If \fBrwmixread\fR and
c35dd7a6
JA
546\fBrwmixwrite\fR are given and do not sum to 100%, the latter of the two
547overrides the first. This may interfere with a given rate setting, if fio is
548asked to limit reads or writes to a certain rate. If that is the case, then
549the distribution may be skewed. Default: 50.
d60e92d1
AC
550.TP
551.B norandommap
552Normally \fBfio\fR will cover every block of the file when doing random I/O. If
553this parameter is given, a new offset will be chosen without looking at past
554I/O history. This parameter is mutually exclusive with \fBverify\fR.
555.TP
744492c9 556.BI softrandommap \fR=\fPbool
3ce9dcaf
JA
557See \fBnorandommap\fR. If fio runs with the random block map enabled and it
558fails to allocate the map, if this option is set it will continue without a
559random block map. As coverage will not be as complete as with random maps, this
560option is disabled by default.
561.TP
d60e92d1
AC
562.BI nice \fR=\fPint
563Run job with given nice value. See \fInice\fR\|(2).
564.TP
565.BI prio \fR=\fPint
566Set I/O priority value of this job between 0 (highest) and 7 (lowest). See
567\fIionice\fR\|(1).
568.TP
569.BI prioclass \fR=\fPint
570Set I/O priority class. See \fIionice\fR\|(1).
571.TP
572.BI thinktime \fR=\fPint
573Stall job for given number of microseconds between issuing I/Os.
574.TP
575.BI thinktime_spin \fR=\fPint
576Pretend to spend CPU time for given number of microseconds, sleeping the rest
577of the time specified by \fBthinktime\fR. Only valid if \fBthinktime\fR is set.
578.TP
579.BI thinktime_blocks \fR=\fPint
580Number of blocks to issue before waiting \fBthinktime\fR microseconds.
581Default: 1.
582.TP
583.BI rate \fR=\fPint
c35dd7a6
JA
584Cap bandwidth used by this job. The number is in bytes/sec, the normal postfix
585rules apply. You can use \fBrate\fR=500k to limit reads and writes to 500k each,
586or you can specify read and writes separately. Using \fBrate\fR=1m,500k would
587limit reads to 1MB/sec and writes to 500KB/sec. Capping only reads or writes
588can be done with \fBrate\fR=,500k or \fBrate\fR=500k,. The former will only
589limit writes (to 500KB/sec), the latter will only limit reads.
d60e92d1
AC
590.TP
591.BI ratemin \fR=\fPint
592Tell \fBfio\fR to do whatever it can to maintain at least the given bandwidth.
c35dd7a6
JA
593Failing to meet this requirement will cause the job to exit. The same format
594as \fBrate\fR is used for read vs write separation.
d60e92d1
AC
595.TP
596.BI rate_iops \fR=\fPint
c35dd7a6
JA
597Cap the bandwidth to this number of IOPS. Basically the same as rate, just
598specified independently of bandwidth. The same format as \fBrate\fR is used for
599read vs write seperation. If \fBblocksize\fR is a range, the smallest block
600size is used as the metric.
d60e92d1
AC
601.TP
602.BI rate_iops_min \fR=\fPint
c35dd7a6
JA
603If this rate of I/O is not met, the job will exit. The same format as \fBrate\fR
604is used for read vs write seperation.
d60e92d1
AC
605.TP
606.BI ratecycle \fR=\fPint
607Average bandwidth for \fBrate\fR and \fBratemin\fR over this number of
608milliseconds. Default: 1000ms.
609.TP
610.BI cpumask \fR=\fPint
611Set CPU affinity for this job. \fIint\fR is a bitmask of allowed CPUs the job
612may run on. See \fBsched_setaffinity\fR\|(2).
613.TP
614.BI cpus_allowed \fR=\fPstr
615Same as \fBcpumask\fR, but allows a comma-delimited list of CPU numbers.
616.TP
617.BI startdelay \fR=\fPint
618Delay start of job for the specified number of seconds.
619.TP
620.BI runtime \fR=\fPint
621Terminate processing after the specified number of seconds.
622.TP
623.B time_based
624If given, run for the specified \fBruntime\fR duration even if the files are
625completely read or written. The same workload will be repeated as many times
626as \fBruntime\fR allows.
627.TP
901bb994
JA
628.BI ramp_time \fR=\fPint
629If set, fio will run the specified workload for this amount of time before
630logging any performance numbers. Useful for letting performance settle before
631logging results, thus minimizing the runtime required for stable results. Note
c35dd7a6
JA
632that the \fBramp_time\fR is considered lead in time for a job, thus it will
633increase the total runtime if a special timeout or runtime is specified.
901bb994 634.TP
d60e92d1
AC
635.BI invalidate \fR=\fPbool
636Invalidate buffer-cache for the file prior to starting I/O. Default: true.
637.TP
638.BI sync \fR=\fPbool
639Use synchronous I/O for buffered writes. For the majority of I/O engines,
d1429b5c 640this means using O_SYNC. Default: false.
d60e92d1
AC
641.TP
642.BI iomem \fR=\fPstr "\fR,\fP mem" \fR=\fPstr
643Allocation method for I/O unit buffer. Allowed values are:
644.RS
645.RS
646.TP
647.B malloc
648Allocate memory with \fImalloc\fR\|(3).
649.TP
650.B shm
651Use shared memory buffers allocated through \fIshmget\fR\|(2).
652.TP
653.B shmhuge
654Same as \fBshm\fR, but use huge pages as backing.
655.TP
656.B mmap
657Use \fImmap\fR\|(2) for allocation. Uses anonymous memory unless a filename
658is given after the option in the format `:\fIfile\fR'.
659.TP
660.B mmaphuge
661Same as \fBmmap\fR, but use huge files as backing.
662.RE
663.P
664The amount of memory allocated is the maximum allowed \fBblocksize\fR for the
665job multiplied by \fBiodepth\fR. For \fBshmhuge\fR or \fBmmaphuge\fR to work,
666the system must have free huge pages allocated. \fBmmaphuge\fR also needs to
2e266ba6
JA
667have hugetlbfs mounted, and \fIfile\fR must point there. At least on Linux,
668huge pages must be manually allocated. See \fB/proc/sys/vm/nr_hugehages\fR
669and the documentation for that. Normally you just need to echo an appropriate
670number, eg echoing 8 will ensure that the OS has 8 huge pages ready for
671use.
d60e92d1
AC
672.RE
673.TP
d392365e 674.BI iomem_align \fR=\fPint "\fR,\fP mem_align" \fR=\fPint
d529ee19
JA
675This indiciates the memory alignment of the IO memory buffers. Note that the
676given alignment is applied to the first IO unit buffer, if using \fBiodepth\fR
677the alignment of the following buffers are given by the \fBbs\fR used. In
678other words, if using a \fBbs\fR that is a multiple of the page sized in the
679system, all buffers will be aligned to this value. If using a \fBbs\fR that
680is not page aligned, the alignment of subsequent IO memory buffers is the
681sum of the \fBiomem_align\fR and \fBbs\fR used.
682.TP
f7fa2653 683.BI hugepage\-size \fR=\fPint
d60e92d1 684Defines the size of a huge page. Must be at least equal to the system setting.
b22989b9 685Should be a multiple of 1MB. Default: 4MB.
d60e92d1
AC
686.TP
687.B exitall
688Terminate all jobs when one finishes. Default: wait for each job to finish.
689.TP
690.BI bwavgtime \fR=\fPint
691Average bandwidth calculations over the given time in milliseconds. Default:
692500ms.
693.TP
c8eeb9df
JA
694.BI iopsavgtime \fR=\fPint
695Average IOPS calculations over the given time in milliseconds. Default:
696500ms.
697.TP
d60e92d1 698.BI create_serialize \fR=\fPbool
d1429b5c 699If true, serialize file creation for the jobs. Default: true.
d60e92d1
AC
700.TP
701.BI create_fsync \fR=\fPbool
702\fIfsync\fR\|(2) data file after creation. Default: true.
703.TP
6b7f6851
JA
704.BI create_on_open \fR=\fPbool
705If true, the files are not created until they are opened for IO by the job.
706.TP
e9f48479
JA
707.BI pre_read \fR=\fPbool
708If this is given, files will be pre-read into memory before starting the given
709IO operation. This will also clear the \fR \fBinvalidate\fR flag, since it is
9c0d2241
JA
710pointless to pre-read and then drop the cache. This will only work for IO
711engines that are seekable, since they allow you to read the same data
712multiple times. Thus it will not work on eg network or splice IO.
e9f48479 713.TP
d60e92d1
AC
714.BI unlink \fR=\fPbool
715Unlink job files when done. Default: false.
716.TP
717.BI loops \fR=\fPint
718Specifies the number of iterations (runs of the same workload) of this job.
719Default: 1.
720.TP
721.BI do_verify \fR=\fPbool
722Run the verify phase after a write phase. Only valid if \fBverify\fR is set.
723Default: true.
724.TP
725.BI verify \fR=\fPstr
726Method of verifying file contents after each iteration of the job. Allowed
727values are:
728.RS
729.RS
730.TP
b892dc08 731.B md5 crc16 crc32 crc32c crc32c-intel crc64 crc7 sha256 sha512 sha1
0539d758
JA
732Store appropriate checksum in the header of each block. crc32c-intel is
733hardware accelerated SSE4.2 driven, falls back to regular crc32c if
734not supported by the system.
d60e92d1
AC
735.TP
736.B meta
737Write extra information about each I/O (timestamp, block number, etc.). The
996093bb 738block number is verified. See \fBverify_pattern\fR as well.
d60e92d1
AC
739.TP
740.B null
741Pretend to verify. Used for testing internals.
742.RE
b892dc08
JA
743
744This option can be used for repeated burn-in tests of a system to make sure
745that the written data is also correctly read back. If the data direction given
746is a read or random read, fio will assume that it should verify a previously
747written file. If the data direction includes any form of write, the verify will
748be of the newly written data.
d60e92d1
AC
749.RE
750.TP
751.BI verify_sort \fR=\fPbool
752If true, written verify blocks are sorted if \fBfio\fR deems it to be faster to
753read them back in a sorted manner. Default: true.
754.TP
f7fa2653 755.BI verify_offset \fR=\fPint
d60e92d1 756Swap the verification header with data somewhere else in the block before
d1429b5c 757writing. It is swapped back before verifying.
d60e92d1 758.TP
f7fa2653 759.BI verify_interval \fR=\fPint
d60e92d1
AC
760Write the verification header for this number of bytes, which should divide
761\fBblocksize\fR. Default: \fBblocksize\fR.
762.TP
996093bb
JA
763.BI verify_pattern \fR=\fPstr
764If set, fio will fill the io buffers with this pattern. Fio defaults to filling
765with totally random bytes, but sometimes it's interesting to fill with a known
766pattern for io verification purposes. Depending on the width of the pattern,
767fio will fill 1/2/3/4 bytes of the buffer at the time(it can be either a
768decimal or a hex number). The verify_pattern if larger than a 32-bit quantity
769has to be a hex number that starts with either "0x" or "0X". Use with
770\fBverify\fP=meta.
771.TP
d60e92d1
AC
772.BI verify_fatal \fR=\fPbool
773If true, exit the job on the first observed verification failure. Default:
774false.
775.TP
b463e936
JA
776.BI verify_dump \fR=\fPbool
777If set, dump the contents of both the original data block and the data block we
778read off disk to files. This allows later analysis to inspect just what kind of
ef71e317 779data corruption occurred. Off by default.
b463e936 780.TP
e8462bd8
JA
781.BI verify_async \fR=\fPint
782Fio will normally verify IO inline from the submitting thread. This option
783takes an integer describing how many async offload threads to create for IO
784verification instead, causing fio to offload the duty of verifying IO contents
c85c324c
JA
785to one or more separate threads. If using this offload option, even sync IO
786engines can benefit from using an \fBiodepth\fR setting higher than 1, as it
787allows them to have IO in flight while verifies are running.
e8462bd8
JA
788.TP
789.BI verify_async_cpus \fR=\fPstr
790Tell fio to set the given CPU affinity on the async IO verification threads.
791See \fBcpus_allowed\fP for the format used.
792.TP
6f87418f
JA
793.BI verify_backlog \fR=\fPint
794Fio will normally verify the written contents of a job that utilizes verify
795once that job has completed. In other words, everything is written then
796everything is read back and verified. You may want to verify continually
797instead for a variety of reasons. Fio stores the meta data associated with an
798IO block in memory, so for large verify workloads, quite a bit of memory would
092f707f
DN
799be used up holding this meta data. If this option is enabled, fio will write
800only N blocks before verifying these blocks.
6f87418f
JA
801.TP
802.BI verify_backlog_batch \fR=\fPint
803Control how many blocks fio will verify if verify_backlog is set. If not set,
804will default to the value of \fBverify_backlog\fR (meaning the entire queue is
092f707f
DN
805read back and verified). If \fBverify_backlog_batch\fR is less than
806\fBverify_backlog\fR then not all blocks will be verified, if
807\fBverify_backlog_batch\fR is larger than \fBverify_backlog\fR, some blocks
808will be verified more than once.
6f87418f 809.TP
d392365e 810.B stonewall "\fR,\fP wait_for_previous"
5982a925 811Wait for preceding jobs in the job file to exit before starting this one.
d60e92d1
AC
812\fBstonewall\fR implies \fBnew_group\fR.
813.TP
814.B new_group
815Start a new reporting group. If not given, all jobs in a file will be part
816of the same reporting group, unless separated by a stonewall.
817.TP
818.BI numjobs \fR=\fPint
819Number of clones (processes/threads performing the same workload) of this job.
820Default: 1.
821.TP
822.B group_reporting
823If set, display per-group reports instead of per-job when \fBnumjobs\fR is
824specified.
825.TP
826.B thread
827Use threads created with \fBpthread_create\fR\|(3) instead of processes created
828with \fBfork\fR\|(2).
829.TP
f7fa2653 830.BI zonesize \fR=\fPint
d60e92d1
AC
831Divide file into zones of the specified size in bytes. See \fBzoneskip\fR.
832.TP
f7fa2653 833.BI zoneskip \fR=\fPint
d1429b5c 834Skip the specified number of bytes when \fBzonesize\fR bytes of data have been
d60e92d1
AC
835read.
836.TP
837.BI write_iolog \fR=\fPstr
5b42a488
SH
838Write the issued I/O patterns to the specified file. Specify a separate file
839for each job, otherwise the iologs will be interspersed and the file may be
840corrupt.
d60e92d1
AC
841.TP
842.BI read_iolog \fR=\fPstr
843Replay the I/O patterns contained in the specified file generated by
844\fBwrite_iolog\fR, or may be a \fBblktrace\fR binary file.
845.TP
64bbb865
DN
846.BI replay_no_stall \fR=\fPint
847While replaying I/O patterns using \fBread_iolog\fR the default behavior
848attempts to respect timing information between I/Os. Enabling
849\fBreplay_no_stall\fR causes I/Os to be replayed as fast as possible while
850still respecting ordering.
851.TP
d1c46c04
DN
852.BI replay_redirect \fR=\fPstr
853While replaying I/O patterns using \fBread_iolog\fR the default behavior
854is to replay the IOPS onto the major/minor device that each IOP was recorded
855from. Setting \fBreplay_redirect\fR causes all IOPS to be replayed onto the
856single specified device regardless of the device it was recorded from.
857.TP
836bad52 858.BI write_bw_log \fR=\fPstr
901bb994
JA
859If given, write a bandwidth log of the jobs in this job file. Can be used to
860store data of the bandwidth of the jobs in their lifetime. The included
861fio_generate_plots script uses gnuplot to turn these text files into nice
862graphs. See \fBwrite_log_log\fR for behaviour of given filename. For this
863option, the postfix is _bw.log.
d60e92d1 864.TP
836bad52 865.BI write_lat_log \fR=\fPstr
901bb994
JA
866Same as \fBwrite_bw_log\fR, but writes I/O completion latencies. If no
867filename is given with this option, the default filename of "jobname_type.log"
868is used. Even if the filename is given, fio will still append the type of log.
869.TP
c8eeb9df
JA
870.BI write_iops_log \fR=\fPstr
871Same as \fBwrite_bw_log\fR, but writes IOPS. If no filename is given with this
872option, the default filename of "jobname_type.log" is used. Even if the
873filename is given, fio will still append the type of log.
874.TP
b8bc8cba
JA
875.BI log_avg_msec \fR=\fPint
876By default, fio will log an entry in the iops, latency, or bw log for every
877IO that completes. When writing to the disk log, that can quickly grow to a
878very large size. Setting this option makes fio average the each log entry
879over the specified period of time, reducing the resolution of the log.
880Defaults to 0.
881.TP
836bad52 882.BI disable_lat \fR=\fPbool
02af0988 883Disable measurements of total latency numbers. Useful only for cutting
901bb994
JA
884back the number of calls to gettimeofday, as that does impact performance at
885really high IOPS rates. Note that to really get rid of a large amount of these
886calls, this option must be used with disable_slat and disable_bw as well.
887.TP
836bad52 888.BI disable_clat \fR=\fPbool
c95f9daf 889Disable measurements of completion latency numbers. See \fBdisable_lat\fR.
02af0988 890.TP
836bad52 891.BI disable_slat \fR=\fPbool
02af0988 892Disable measurements of submission latency numbers. See \fBdisable_lat\fR.
901bb994 893.TP
836bad52 894.BI disable_bw_measurement \fR=\fPbool
02af0988 895Disable measurements of throughput/bandwidth numbers. See \fBdisable_lat\fR.
d60e92d1 896.TP
f7fa2653 897.BI lockmem \fR=\fPint
d60e92d1
AC
898Pin the specified amount of memory with \fBmlock\fR\|(2). Can be used to
899simulate a smaller amount of memory.
900.TP
901.BI exec_prerun \fR=\fPstr
902Before running the job, execute the specified command with \fBsystem\fR\|(3).
903.TP
904.BI exec_postrun \fR=\fPstr
905Same as \fBexec_prerun\fR, but the command is executed after the job completes.
906.TP
907.BI ioscheduler \fR=\fPstr
908Attempt to switch the device hosting the file to the specified I/O scheduler.
909.TP
910.BI cpuload \fR=\fPint
911If the job is a CPU cycle-eater, attempt to use the specified percentage of
912CPU cycles.
913.TP
914.BI cpuchunks \fR=\fPint
915If the job is a CPU cycle-eater, split the load into cycles of the
916given time in milliseconds.
917.TP
918.BI disk_util \fR=\fPbool
d1429b5c 919Generate disk utilization statistics if the platform supports it. Default: true.
901bb994
JA
920.TP
921.BI gtod_reduce \fR=\fPbool
922Enable all of the gettimeofday() reducing options (disable_clat, disable_slat,
923disable_bw) plus reduce precision of the timeout somewhat to really shrink the
924gettimeofday() call count. With this option enabled, we only do about 0.4% of
925the gtod() calls we would have done if all time keeping was enabled.
926.TP
927.BI gtod_cpu \fR=\fPint
928Sometimes it's cheaper to dedicate a single thread of execution to just getting
929the current time. Fio (and databases, for instance) are very intensive on
930gettimeofday() calls. With this option, you can set one CPU aside for doing
931nothing but logging current time to a shared memory location. Then the other
932threads/processes that run IO workloads need only copy that segment, instead of
933entering the kernel with a gettimeofday() call. The CPU set aside for doing
934these time calls will be excluded from other uses. Fio will manually clear it
935from the CPU mask of other jobs.
f2bba182 936.TP
a696fa2a
JA
937.BI cgroup \fR=\fPstr
938Add job to this control group. If it doesn't exist, it will be created.
6adb38a1
JA
939The system must have a mounted cgroup blkio mount point for this to work. If
940your system doesn't have it mounted, you can do so with:
941
5982a925 942# mount \-t cgroup \-o blkio none /cgroup
a696fa2a
JA
943.TP
944.BI cgroup_weight \fR=\fPint
945Set the weight of the cgroup to this value. See the documentation that comes
946with the kernel, allowed values are in the range of 100..1000.
e0b0d892 947.TP
7de87099
VG
948.BI cgroup_nodelete \fR=\fPbool
949Normally fio will delete the cgroups it has created after the job completion.
950To override this behavior and to leave cgroups around after the job completion,
951set cgroup_nodelete=1. This can be useful if one wants to inspect various
952cgroup files after job completion. Default: false
953.TP
e0b0d892
JA
954.BI uid \fR=\fPint
955Instead of running as the invoking user, set the user ID to this value before
956the thread/process does any work.
957.TP
958.BI gid \fR=\fPint
959Set group ID, see \fBuid\fR.
83349190 960.TP
9e684a49
DE
961.BI flow_id \fR=\fPint
962The ID of the flow. If not specified, it defaults to being a global flow. See
963\fBflow\fR.
964.TP
965.BI flow \fR=\fPint
966Weight in token-based flow control. If this value is used, then there is a
967\fBflow counter\fR which is used to regulate the proportion of activity between
968two or more jobs. fio attempts to keep this flow counter near zero. The
969\fBflow\fR parameter stands for how much should be added or subtracted to the
970flow counter on each iteration of the main I/O loop. That is, if one job has
971\fBflow=8\fR and another job has \fBflow=-1\fR, then there will be a roughly
9721:8 ratio in how much one runs vs the other.
973.TP
974.BI flow_watermark \fR=\fPint
975The maximum value that the absolute value of the flow counter is allowed to
976reach before the job must wait for a lower value of the counter.
977.TP
978.BI flow_sleep \fR=\fPint
979The period of time, in microseconds, to wait after the flow watermark has been
980exceeded before retrying operations
981.TP
83349190
YH
982.BI clat_percentiles \fR=\fPbool
983Enable the reporting of percentiles of completion latencies.
984.TP
985.BI percentile_list \fR=\fPfloat_list
986Overwrite the default list of percentiles for completion
987latencies. Each number is a floating number in the range (0,100], and
988the maximum length of the list is 20. Use ':' to separate the
3eb07285 989numbers. For example, \-\-percentile_list=99.5:99.9 will cause fio to
83349190
YH
990report the values of completion latency below which 99.5% and 99.9% of
991the observed latencies fell, respectively.
de890a1e
SL
992.SS "Ioengine Parameters List"
993Some parameters are only valid when a specific ioengine is in use. These are
994used identically to normal parameters, with the caveat that when used on the
995command line, the must come after the ioengine that defines them is selected.
996.TP
997.BI (libaio)userspace_reap
998Normally, with the libaio engine in use, fio will use
999the io_getevents system call to reap newly returned events.
1000With this flag turned on, the AIO ring will be read directly
1001from user-space to reap events. The reaping mode is only
1002enabled when polling for a minimum of 0 events (eg when
1003iodepth_batch_complete=0).
1004.TP
1005.BI (net,netsplice)hostname \fR=\fPstr
1006The host name or IP address to use for TCP or UDP based IO.
1007If the job is a TCP listener or UDP reader, the hostname is not
1008used and must be omitted.
1009.TP
1010.BI (net,netsplice)port \fR=\fPint
1011The TCP or UDP port to bind to or connect to.
1012.TP
1013.BI (net,netsplice)protocol \fR=\fPstr "\fR,\fP proto" \fR=\fPstr
1014The network protocol to use. Accepted values are:
1015.RS
1016.RS
1017.TP
1018.B tcp
1019Transmission control protocol
1020.TP
1021.B udp
1022Unreliable datagram protocol
1023.TP
1024.B unix
1025UNIX domain socket
1026.RE
1027.P
1028When the protocol is TCP or UDP, the port must also be given,
1029as well as the hostname if the job is a TCP listener or UDP
1030reader. For unix sockets, the normal filename option should be
1031used and the port is invalid.
1032.RE
1033.TP
1034.BI (net,netsplice)listen
1035For TCP network connections, tell fio to listen for incoming
1036connections rather than initiating an outgoing connection. The
1037hostname must be omitted if this option is used.
d60e92d1 1038.SH OUTPUT
d1429b5c
AC
1039While running, \fBfio\fR will display the status of the created jobs. For
1040example:
d60e92d1 1041.RS
d1429b5c 1042.P
d60e92d1
AC
1043Threads: 1: [_r] [24.8% done] [ 13509/ 8334 kb/s] [eta 00h:01m:31s]
1044.RE
1045.P
d1429b5c
AC
1046The characters in the first set of brackets denote the current status of each
1047threads. The possible values are:
1048.P
1049.PD 0
d60e92d1
AC
1050.RS
1051.TP
1052.B P
1053Setup but not started.
1054.TP
1055.B C
1056Thread created.
1057.TP
1058.B I
1059Initialized, waiting.
1060.TP
1061.B R
1062Running, doing sequential reads.
1063.TP
1064.B r
1065Running, doing random reads.
1066.TP
1067.B W
1068Running, doing sequential writes.
1069.TP
1070.B w
1071Running, doing random writes.
1072.TP
1073.B M
1074Running, doing mixed sequential reads/writes.
1075.TP
1076.B m
1077Running, doing mixed random reads/writes.
1078.TP
1079.B F
1080Running, currently waiting for \fBfsync\fR\|(2).
1081.TP
1082.B V
1083Running, verifying written data.
1084.TP
1085.B E
1086Exited, not reaped by main thread.
1087.TP
1088.B \-
1089Exited, thread reaped.
1090.RE
d1429b5c 1091.PD
d60e92d1
AC
1092.P
1093The second set of brackets shows the estimated completion percentage of
1094the current group. The third set shows the read and write I/O rate,
1095respectively. Finally, the estimated run time of the job is displayed.
1096.P
1097When \fBfio\fR completes (or is interrupted by Ctrl-C), it will show data
1098for each thread, each group of threads, and each disk, in that order.
1099.P
1100Per-thread statistics first show the threads client number, group-id, and
1101error code. The remaining figures are as follows:
1102.RS
d60e92d1
AC
1103.TP
1104.B io
1105Number of megabytes of I/O performed.
1106.TP
1107.B bw
1108Average data rate (bandwidth).
1109.TP
1110.B runt
1111Threads run time.
1112.TP
1113.B slat
1114Submission latency minimum, maximum, average and standard deviation. This is
1115the time it took to submit the I/O.
1116.TP
1117.B clat
1118Completion latency minimum, maximum, average and standard deviation. This
1119is the time between submission and completion.
1120.TP
1121.B bw
1122Bandwidth minimum, maximum, percentage of aggregate bandwidth received, average
1123and standard deviation.
1124.TP
1125.B cpu
1126CPU usage statistics. Includes user and system time, number of context switches
1127this thread went through and number of major and minor page faults.
1128.TP
1129.B IO depths
1130Distribution of I/O depths. Each depth includes everything less than (or equal)
1131to it, but greater than the previous depth.
1132.TP
1133.B IO issued
1134Number of read/write requests issued, and number of short read/write requests.
1135.TP
1136.B IO latencies
1137Distribution of I/O completion latencies. The numbers follow the same pattern
1138as \fBIO depths\fR.
1139.RE
d60e92d1
AC
1140.P
1141The group statistics show:
d1429b5c 1142.PD 0
d60e92d1
AC
1143.RS
1144.TP
1145.B io
1146Number of megabytes I/O performed.
1147.TP
1148.B aggrb
1149Aggregate bandwidth of threads in the group.
1150.TP
1151.B minb
1152Minimum average bandwidth a thread saw.
1153.TP
1154.B maxb
1155Maximum average bandwidth a thread saw.
1156.TP
1157.B mint
d1429b5c 1158Shortest runtime of threads in the group.
d60e92d1
AC
1159.TP
1160.B maxt
1161Longest runtime of threads in the group.
1162.RE
d1429b5c 1163.PD
d60e92d1
AC
1164.P
1165Finally, disk statistics are printed with reads first:
d1429b5c 1166.PD 0
d60e92d1
AC
1167.RS
1168.TP
1169.B ios
1170Number of I/Os performed by all groups.
1171.TP
1172.B merge
1173Number of merges in the I/O scheduler.
1174.TP
1175.B ticks
1176Number of ticks we kept the disk busy.
1177.TP
1178.B io_queue
1179Total time spent in the disk queue.
1180.TP
1181.B util
1182Disk utilization.
1183.RE
d1429b5c 1184.PD
d60e92d1
AC
1185.SH TERSE OUTPUT
1186If the \fB\-\-minimal\fR option is given, the results will be printed in a
562c2d2f
DN
1187semicolon-delimited format suitable for scripted use - a job description
1188(if provided) follows on a new line. Note that the first
525c2bfa
JA
1189number in the line is the version number. If the output has to be changed
1190for some reason, this number will be incremented by 1 to signify that
1191change. The fields are:
d60e92d1
AC
1192.P
1193.RS
5e726d0a 1194.B terse version, fio version, jobname, groupid, error
d60e92d1
AC
1195.P
1196Read status:
1197.RS
312b4af2 1198.B Total I/O \fR(KB)\fP, bandwidth \fR(KB/s)\fP, IOPS, runtime \fR(ms)\fP
d60e92d1
AC
1199.P
1200Submission latency:
1201.RS
1202.B min, max, mean, standard deviation
1203.RE
1204Completion latency:
1205.RS
1206.B min, max, mean, standard deviation
1207.RE
1db92cb6
JA
1208Completion latency percentiles (20 fields):
1209.RS
1210.B Xth percentile=usec
1211.RE
525c2bfa
JA
1212Total latency:
1213.RS
1214.B min, max, mean, standard deviation
1215.RE
d60e92d1
AC
1216Bandwidth:
1217.RS
1218.B min, max, aggregate percentage of total, mean, standard deviation
1219.RE
1220.RE
1221.P
1222Write status:
1223.RS
312b4af2 1224.B Total I/O \fR(KB)\fP, bandwidth \fR(KB/s)\fP, IOPS, runtime \fR(ms)\fP
d60e92d1
AC
1225.P
1226Submission latency:
1227.RS
1228.B min, max, mean, standard deviation
1229.RE
1230Completion latency:
1231.RS
1232.B min, max, mean, standard deviation
1233.RE
1db92cb6
JA
1234Completion latency percentiles (20 fields):
1235.RS
1236.B Xth percentile=usec
1237.RE
525c2bfa
JA
1238Total latency:
1239.RS
1240.B min, max, mean, standard deviation
1241.RE
d60e92d1
AC
1242Bandwidth:
1243.RS
1244.B min, max, aggregate percentage of total, mean, standard deviation
1245.RE
1246.RE
1247.P
d1429b5c 1248CPU usage:
d60e92d1 1249.RS
bd2626f0 1250.B user, system, context switches, major page faults, minor page faults
d60e92d1
AC
1251.RE
1252.P
1253IO depth distribution:
1254.RS
1255.B <=1, 2, 4, 8, 16, 32, >=64
1256.RE
1257.P
562c2d2f 1258IO latency distribution:
d60e92d1 1259.RS
562c2d2f
DN
1260Microseconds:
1261.RS
1262.B <=2, 4, 10, 20, 50, 100, 250, 500, 750, 1000
1263.RE
1264Milliseconds:
1265.RS
1266.B <=2, 4, 10, 20, 50, 100, 250, 500, 750, 1000, 2000, >=2000
1267.RE
1268.RE
1269.P
f2f788dd
JA
1270Disk utilization (1 for each disk used):
1271.RS
1272.B name, read ios, write ios, read merges, write merges, read ticks, write ticks, read in-queue time, write in-queue time, disk utilization percentage
1273.RE
1274.P
5982a925 1275Error Info (dependent on continue_on_error, default off):
562c2d2f
DN
1276.RS
1277.B total # errors, first error code
d60e92d1
AC
1278.RE
1279.P
562c2d2f 1280.B text description (if provided in config - appears on newline)
d60e92d1 1281.RE
49da1240
JA
1282.SH CLIENT / SERVER
1283Normally you would run fio as a stand-alone application on the machine
1284where the IO workload should be generated. However, it is also possible to
1285run the frontend and backend of fio separately. This makes it possible to
1286have a fio server running on the machine(s) where the IO workload should
1287be running, while controlling it from another machine.
1288
1289To start the server, you would do:
1290
1291\fBfio \-\-server=args\fR
1292
1293on that machine, where args defines what fio listens to. The arguments
811826be
JA
1294are of the form 'type:hostname or IP:port'. 'type' is either 'ip' (or ip4)
1295for TCP/IP v4, 'ip6' for TCP/IP v6, or 'sock' for a local unix domain socket.
1296'hostname' is either a hostname or IP address, and 'port' is the port to
1297listen to (only valid for TCP/IP, not a local socket). Some examples:
49da1240
JA
1298
12991) fio --server
1300
1301 Start a fio server, listening on all interfaces on the default port (8765).
1302
811826be 13032) fio --server=ip:hostname,4444
49da1240
JA
1304
1305 Start a fio server, listening on IP belonging to hostname and on port 4444.
1306
811826be
JA
13073) fio --server=ip6:::1,4444
1308
1309 Start a fio server, listening on IPv6 localhost ::1 and on port 4444.
1310
13114) fio --server=,4444
49da1240
JA
1312
1313 Start a fio server, listening on all interfaces on port 4444.
1314
811826be 13155) fio --server=1.2.3.4
49da1240
JA
1316
1317 Start a fio server, listening on IP 1.2.3.4 on the default port.
1318
811826be 13196) fio --server=sock:/tmp/fio.sock
49da1240
JA
1320
1321 Start a fio server, listening on the local socket /tmp/fio.sock.
1322
1323When a server is running, you can connect to it from a client. The client
1324is run with:
1325
1326fio --local-args --client=server --remote-args <job file(s)>
1327
1328where --local-args are arguments that are local to the client where it is
1329running, 'server' is the connect string, and --remote-args and <job file(s)>
1330are sent to the server. The 'server' string follows the same format as it
1331does on the server side, to allow IP/hostname/socket and port strings.
1332You can connect to multiple clients as well, to do that you could run:
1333
1334fio --client=server2 --client=server2 <job file(s)>
d60e92d1 1335.SH AUTHORS
49da1240 1336
d60e92d1 1337.B fio
aa58d252
JA
1338was written by Jens Axboe <jens.axboe@oracle.com>,
1339now Jens Axboe <jaxboe@fusionio.com>.
d1429b5c
AC
1340.br
1341This man page was written by Aaron Carroll <aaronc@cse.unsw.edu.au> based
d60e92d1
AC
1342on documentation by Jens Axboe.
1343.SH "REPORTING BUGS"
482900c9 1344Report bugs to the \fBfio\fR mailing list <fio@vger.kernel.org>.
d1429b5c 1345See \fBREADME\fR.
d60e92d1 1346.SH "SEE ALSO"
d1429b5c
AC
1347For further documentation see \fBHOWTO\fR and \fBREADME\fR.
1348.br
1349Sample jobfiles are available in the \fBexamples\fR directory.
d60e92d1 1350