Add rw_sequencer option
[fio.git] / fio.1
CommitLineData
d60e92d1
AC
1.TH fio 1 "September 2007" "User Manual"
2.SH NAME
3fio \- flexible I/O tester
4.SH SYNOPSIS
5.B fio
6[\fIoptions\fR] [\fIjobfile\fR]...
7.SH DESCRIPTION
8.B fio
9is a tool that will spawn a number of threads or processes doing a
10particular type of I/O action as specified by the user.
11The typical use of fio is to write a job file matching the I/O load
12one wants to simulate.
13.SH OPTIONS
14.TP
15.BI \-\-output \fR=\fPfilename
16Write output to \fIfilename\fR.
17.TP
18.BI \-\-timeout \fR=\fPtimeout
19Limit run time to \fItimeout\fR seconds.
20.TP
21.B \-\-latency\-log
22Generate per-job latency logs.
23.TP
24.B \-\-bandwidth\-log
25Generate per-job bandwidth logs.
26.TP
27.B \-\-minimal
d1429b5c 28Print statistics in a terse, semicolon-delimited format.
d60e92d1
AC
29.TP
30.BI \-\-showcmd \fR=\fPjobfile
31Convert \fIjobfile\fR to a set of command-line options.
32.TP
33.B \-\-readonly
34Enable read-only safety checks.
35.TP
36.BI \-\-eta \fR=\fPwhen
37Specifies when real-time ETA estimate should be printed. \fIwhen\fR may
38be one of `always', `never' or `auto'.
39.TP
c0a5d35e
AC
40.BI \-\-section \fR=\fPsec
41Only run section \fIsec\fR from job file.
42.TP
d60e92d1
AC
43.BI \-\-cmdhelp \fR=\fPcommand
44Print help information for \fIcommand\fR. May be `all' for all commands.
45.TP
9183788d
JA
46.BI \-\-debug \fR=\fPtype
47Enable verbose tracing of various fio actions. May be `all' for all types
c6e13ea5 48or individual types seperated by a comma (eg \-\-debug=io,file). `help' will
9183788d
JA
49list all available tracing options.
50.TP
d60e92d1
AC
51.B \-\-help
52Display usage information and exit.
53.TP
54.B \-\-version
55Display version information and exit.
56.SH "JOB FILE FORMAT"
57Job files are in `ini' format. They consist of one or more
58job definitions, which begin with a job name in square brackets and
59extend to the next job name. The job name can be any ASCII string
60except `global', which has a special meaning. Following the job name is
61a sequence of zero or more parameters, one per line, that define the
62behavior of the job. Any line starting with a `;' or `#' character is
d1429b5c 63considered a comment and ignored.
d9956b64
AC
64.P
65If \fIjobfile\fR is specified as `-', the job file will be read from
66standard input.
d60e92d1
AC
67.SS "Global Section"
68The global section contains default parameters for jobs specified in the
69job file. A job is only affected by global sections residing above it,
70and there may be any number of global sections. Specific job definitions
71may override any parameter set in global sections.
72.SH "JOB PARAMETERS"
73.SS Types
74Some parameters may take arguments of a specific type. The types used are:
75.TP
76.I str
77String: a sequence of alphanumeric characters.
78.TP
79.I int
d60e92d1 80SI integer: a whole number, possibly containing a suffix denoting the base unit
b09da8fa
JA
81of the value. Accepted suffixes are `k', 'M', 'G', 'T', and 'P', denoting
82kilo (1024), mega (1024^2), giga (1024^3), tera (1024^4), and peta (1024^5)
83respectively. The suffix is not case sensitive. If prefixed with '0x', the
57fc29fa
JA
84value is assumed to be base 16 (hexadecimal). A suffix may include a trailing
85'b', for instance 'kb' is identical to 'k'. You can specify a base 10 value
86by using 'KiB', 'MiB', 'GiB', etc. This is useful for disk drives where
87values are often given in base 10 values. Specifying '30GiB' will get you
8830*1000^3 bytes.
d60e92d1
AC
89.TP
90.I bool
91Boolean: a true or false value. `0' denotes false, `1' denotes true.
92.TP
93.I irange
94Integer range: a range of integers specified in the format
d1429b5c
AC
95\fIlower\fR:\fIupper\fR or \fIlower\fR\-\fIupper\fR. \fIlower\fR and
96\fIupper\fR may contain a suffix as described above. If an option allows two
97sets of ranges, they are separated with a `,' or `/' character. For example:
98`8\-8k/8M\-4G'.
d60e92d1
AC
99.SS "Parameter List"
100.TP
101.BI name \fR=\fPstr
d9956b64 102May be used to override the job name. On the command line, this parameter
d60e92d1
AC
103has the special purpose of signalling the start of a new job.
104.TP
105.BI description \fR=\fPstr
106Human-readable description of the job. It is printed when the job is run, but
107otherwise has no special purpose.
108.TP
109.BI directory \fR=\fPstr
110Prefix filenames with this directory. Used to place files in a location other
111than `./'.
112.TP
113.BI filename \fR=\fPstr
114.B fio
115normally makes up a file name based on the job name, thread number, and file
d1429b5c
AC
116number. If you want to share files between threads in a job or several jobs,
117specify a \fIfilename\fR for each of them to override the default. If the I/O
118engine used is `net', \fIfilename\fR is the host and port to connect to in the
119format \fIhost\fR/\fIport\fR. If the I/O engine is file-based, you can specify
120a number of files by separating the names with a `:' character. `\-' is a
121reserved name, meaning stdin or stdout, depending on the read/write direction
122set.
d60e92d1 123.TP
3ce9dcaf
JA
124.BI lockfile \fR=\fPstr
125Fio defaults to not locking any files before it does IO to them. If a file or
126file descriptor is shared, fio can serialize IO to that file to make the end
127result consistent. This is usual for emulating real workloads that share files.
128The lock modes are:
129.RS
130.RS
131.TP
132.B none
133No locking. This is the default.
134.TP
135.B exclusive
136Only one thread or process may do IO at the time, excluding all others.
137.TP
138.B readwrite
139Read-write locking on the file. Many readers may access the file at the same
140time, but writes get exclusive access.
141.RE
142.P
143The option may be post-fixed with a lock batch number. If set, then each
144thread/process may do that amount of IOs to the file before giving up the lock.
145Since lock acquisition is expensive, batching the lock/unlocks will speed up IO.
146.RE
147.P
d60e92d1
AC
148.BI opendir \fR=\fPstr
149Recursively open any files below directory \fIstr\fR.
150.TP
151.BI readwrite \fR=\fPstr "\fR,\fP rw" \fR=\fPstr
152Type of I/O pattern. Accepted values are:
153.RS
154.RS
155.TP
156.B read
d1429b5c 157Sequential reads.
d60e92d1
AC
158.TP
159.B write
d1429b5c 160Sequential writes.
d60e92d1
AC
161.TP
162.B randread
d1429b5c 163Random reads.
d60e92d1
AC
164.TP
165.B randwrite
d1429b5c 166Random writes.
d60e92d1
AC
167.TP
168.B rw
d1429b5c 169Mixed sequential reads and writes.
d60e92d1
AC
170.TP
171.B randrw
d1429b5c 172Mixed random reads and writes.
d60e92d1
AC
173.RE
174.P
38dad62d
JA
175For mixed I/O, the default split is 50/50. For certain types of io the result
176may still be skewed a bit, since the speed may be different. It is possible to
177specify a number of IO's to do before getting a new offset, this is one by
178appending a `:\fI<nr>\fR to the end of the string given. For a random read, it
179would look like \fBrw=randread:8\fR for passing in an offset modifier with a
180value of 8. See the \fBrw_sequencer\fR option.
d60e92d1
AC
181.RE
182.TP
38dad62d
JA
183.BI rw_sequencer \fR=\fPstr
184If an offset modifier is given by appending a number to the \fBrw=<str>\fR line,
185then this option controls how that number modifies the IO offset being
186generated. Accepted values are:
187.RS
188.RS
189.TP
190.B sequential
191Generate sequential offset
192.TP
193.B identical
194Generate the same offset
195.RE
196.P
197\fBsequential\fR is only useful for random IO, where fio would normally
198generate a new random offset for every IO. If you append eg 8 to randread, you
199would get a new random offset for every 8 IO's. The result would be a seek for
200only every 8 IO's, instead of for every IO. Use \fBrw=randread:8\fR to specify
201that. As sequential IO is already sequential, setting \fBsequential\fR for that
202would not result in any differences. \fBidentical\fR behaves in a similar
203fashion, except it sends the same offset 8 number of times before generating a
204new offset.
205.RE
206.P
207.TP
90fef2d1
JA
208.BI kb_base \fR=\fPint
209The base unit for a kilobyte. The defacto base is 2^10, 1024. Storage
210manufacturers like to use 10^3 or 1000 as a base ten unit instead, for obvious
211reasons. Allow values are 1024 or 1000, with 1024 being the default.
212.TP
d60e92d1
AC
213.BI randrepeat \fR=\fPbool
214Seed the random number generator in a predictable way so results are repeatable
d1429b5c 215across runs. Default: true.
d60e92d1 216.TP
7bc8c2cf
JA
217.BI fallocate \fR=\fPbool
218By default, fio will use fallocate() to advise the system of the size of the
219file we are going to write. This can be turned off with fallocate=0. May not
220be available on all supported platforms.
221.TP
d60e92d1 222.BI fadvise_hint \fR=\fPbool
d1429b5c
AC
223Disable use of \fIposix_fadvise\fR\|(2) to advise the kernel what I/O patterns
224are likely to be issued. Default: true.
d60e92d1 225.TP
f7fa2653 226.BI size \fR=\fPint
d60e92d1
AC
227Total size of I/O for this job. \fBfio\fR will run until this many bytes have
228been transfered, unless limited by other options (\fBruntime\fR, for instance).
229Unless \fBnr_files\fR and \fBfilesize\fR options are given, this amount will be
d6667268
JA
230divided between the available files for the job. If not set, fio will use the
231full size of the given files or devices. If the the files do not exist, size
232must be given.
d60e92d1 233.TP
3ce9dcaf
JA
234.BI fill_device \fR=\fPbool
235Sets size to something really large and waits for ENOSPC (no space left on
236device) as the terminating condition. Only makes sense with sequential write.
237For a read workload, the mount point will be filled first then IO started on
238the result.
239.TP
d60e92d1
AC
240.BI filesize \fR=\fPirange
241Individual file sizes. May be a range, in which case \fBfio\fR will select sizes
d1429b5c
AC
242for files at random within the given range, limited to \fBsize\fR in total (if
243that is given). If \fBfilesize\fR is not specified, each created file is the
244same size.
d60e92d1 245.TP
f7fa2653 246.BI blocksize \fR=\fPint[,int] "\fR,\fB bs" \fR=\fPint[,int]
d60e92d1 247Block size for I/O units. Default: 4k. Values for reads and writes can be
656ebab7 248specified separately in the format \fIread\fR,\fIwrite\fR, either of
d60e92d1
AC
249which may be empty to leave that value at its default.
250.TP
9183788d 251.BI blocksize_range \fR=\fPirange[,irange] "\fR,\fB bsrange" \fR=\fPirange[,irange]
d1429b5c
AC
252Specify a range of I/O block sizes. The issued I/O unit will always be a
253multiple of the minimum size, unless \fBblocksize_unaligned\fR is set. Applies
9183788d 254to both reads and writes if only one range is given, but can be specified
656ebab7 255separately with a comma seperating the values. Example: bsrange=1k-4k,2k-8k.
9183788d
JA
256Also (see \fBblocksize\fR).
257.TP
258.BI bssplit \fR=\fPstr
259This option allows even finer grained control of the block sizes issued,
260not just even splits between them. With this option, you can weight various
261block sizes for exact control of the issued IO for a job that has mixed
262block sizes. The format of the option is bssplit=blocksize/percentage,
263optionally adding as many definitions as needed seperated by a colon.
264Example: bssplit=4k/10:64k/50:32k/40 would issue 50% 64k blocks, 10% 4k
c83cdd3e
JA
265blocks and 40% 32k blocks. \fBbssplit\fR also supports giving separate
266splits to reads and writes. The format is identical to what the
267\fBbs\fR option accepts, the read and write parts are separated with a
268comma.
d60e92d1
AC
269.TP
270.B blocksize_unaligned\fR,\fP bs_unaligned
d1429b5c
AC
271If set, any size in \fBblocksize_range\fR may be used. This typically won't
272work with direct I/O, as that normally requires sector alignment.
d60e92d1 273.TP
2b7a01d0 274.BI blockalign \fR=\fPint[,int] "\fR,\fB ba" \fR=\fPint[,int]
639ce0f3
MS
275At what boundary to align random IO offsets. Defaults to the same as 'blocksize'
276the minimum blocksize given. Minimum alignment is typically 512b
2b7a01d0
JA
277for using direct IO, though it usually depends on the hardware block size.
278This option is mutually exclusive with using a random map for files, so it
279will turn off that option.
43602667 280.TP
d60e92d1
AC
281.B zero_buffers
282Initialise buffers with all zeros. Default: fill buffers with random data.
283.TP
901bb994
JA
284.B refill_buffers
285If this option is given, fio will refill the IO buffers on every submit. The
286default is to only fill it at init time and reuse that data. Only makes sense
287if zero_buffers isn't specified, naturally. If data verification is enabled,
288refill_buffers is also automatically enabled.
289.TP
d60e92d1
AC
290.BI nrfiles \fR=\fPint
291Number of files to use for this job. Default: 1.
292.TP
293.BI openfiles \fR=\fPint
294Number of files to keep open at the same time. Default: \fBnrfiles\fR.
295.TP
296.BI file_service_type \fR=\fPstr
297Defines how files to service are selected. The following types are defined:
298.RS
299.RS
300.TP
301.B random
302Choose a file at random
303.TP
304.B roundrobin
305Round robin over open files (default).
6b7f6851
JA
306.B sequential
307Do each file in the set sequentially.
d60e92d1
AC
308.RE
309.P
310The number of I/Os to issue before switching a new file can be specified by
311appending `:\fIint\fR' to the service type.
312.RE
313.TP
314.BI ioengine \fR=\fPstr
315Defines how the job issues I/O. The following types are defined:
316.RS
317.RS
318.TP
319.B sync
320Basic \fIread\fR\|(2) or \fIwrite\fR\|(2) I/O. \fIfseek\fR\|(2) is used to
321position the I/O location.
322.TP
a31041ea 323.B psync
324Basic \fIpread\fR\|(2) or \fIpwrite\fR\|(2) I/O.
325.TP
9183788d
JA
326.B vsync
327Basic \fIreadv\fR\|(2) or \fIwritev\fR\|(2) I/O. Will emulate queuing by
328coalescing adjacents IOs into a single submission.
329.TP
d60e92d1
AC
330.B libaio
331Linux native asynchronous I/O.
332.TP
333.B posixaio
334glibc POSIX asynchronous I/O using \fIaio_read\fR\|(3) and \fIaio_write\fR\|(3).
335.TP
336.B mmap
d1429b5c
AC
337File is memory mapped with \fImmap\fR\|(2) and data copied using
338\fImemcpy\fR\|(3).
d60e92d1
AC
339.TP
340.B splice
d1429b5c
AC
341\fIsplice\fR\|(2) is used to transfer the data and \fIvmsplice\fR\|(2) to
342transfer data from user-space to the kernel.
d60e92d1
AC
343.TP
344.B syslet-rw
345Use the syslet system calls to make regular read/write asynchronous.
346.TP
347.B sg
348SCSI generic sg v3 I/O. May be either synchronous using the SG_IO ioctl, or if
d1429b5c
AC
349the target is an sg character device, we use \fIread\fR\|(2) and
350\fIwrite\fR\|(2) for asynchronous I/O.
d60e92d1
AC
351.TP
352.B null
353Doesn't transfer any data, just pretends to. Mainly used to exercise \fBfio\fR
354itself and for debugging and testing purposes.
355.TP
356.B net
357Transfer over the network. \fBfilename\fR must be set appropriately to
358`\fIhost\fR/\fIport\fR' regardless of data direction. If receiving, only the
359\fIport\fR argument is used.
360.TP
361.B netsplice
362Like \fBnet\fR, but uses \fIsplice\fR\|(2) and \fIvmsplice\fR\|(2) to map data
363and send/receive.
364.TP
53aec0a4 365.B cpuio
d60e92d1
AC
366Doesn't transfer any data, but burns CPU cycles according to \fBcpuload\fR and
367\fBcpucycles\fR parameters.
368.TP
369.B guasi
370The GUASI I/O engine is the Generic Userspace Asynchronous Syscall Interface
371approach to asycnronous I/O.
d1429b5c
AC
372.br
373See <http://www.xmailserver.org/guasi\-lib.html>.
d60e92d1
AC
374.TP
375.B external
376Loads an external I/O engine object file. Append the engine filename as
377`:\fIenginepath\fR'.
378.RE
379.RE
380.TP
381.BI iodepth \fR=\fPint
382Number of I/O units to keep in flight against the file. Default: 1.
383.TP
384.BI iodepth_batch \fR=\fPint
385Number of I/Os to submit at once. Default: \fBiodepth\fR.
386.TP
3ce9dcaf
JA
387.BI iodepth_batch_complete \fR=\fPint
388This defines how many pieces of IO to retrieve at once. It defaults to 1 which
389 means that we'll ask for a minimum of 1 IO in the retrieval process from the
390kernel. The IO retrieval will go on until we hit the limit set by
391\fBiodepth_low\fR. If this variable is set to 0, then fio will always check for
392completed events before queuing more IO. This helps reduce IO latency, at the
393cost of more retrieval system calls.
394.TP
d60e92d1
AC
395.BI iodepth_low \fR=\fPint
396Low watermark indicating when to start filling the queue again. Default:
397\fBiodepth\fR.
398.TP
399.BI direct \fR=\fPbool
400If true, use non-buffered I/O (usually O_DIRECT). Default: false.
401.TP
402.BI buffered \fR=\fPbool
403If true, use buffered I/O. This is the opposite of the \fBdirect\fR parameter.
404Default: true.
405.TP
f7fa2653 406.BI offset \fR=\fPint
d60e92d1
AC
407Offset in the file to start I/O. Data before the offset will not be touched.
408.TP
409.BI fsync \fR=\fPint
d1429b5c
AC
410How many I/Os to perform before issuing an \fBfsync\fR\|(2) of dirty data. If
4110, don't sync. Default: 0.
d60e92d1 412.TP
5f9099ea
JA
413.BI fdatasync \fR=\fPint
414Like \fBfsync\fR, but uses \fBfdatasync\fR\|(2) instead to only sync the
415data parts of the file. Default: 0.
416.TP
e76b1da4
JA
417.BI sync_file_range \fR=\fPstr:int
418Use sync_file_range() for every \fRval\fP number of write operations. Fio will
419track range of writes that have happened since the last sync_file_range() call.
420\fRstr\fP can currently be one or more of:
421.RS
422.TP
423.B wait_before
424SYNC_FILE_RANGE_WAIT_BEFORE
425.TP
426.B write
427SYNC_FILE_RANGE_WRITE
428.TP
429.B wait_after
430SYNC_FILE_RANGE_WRITE
431.TP
432.RE
433.P
434So if you do sync_file_range=wait_before,write:8, fio would use
435\fBSYNC_FILE_RANGE_WAIT_BEFORE | SYNC_FILE_RANGE_WRITE\fP for every 8 writes.
436Also see the sync_file_range(2) man page. This option is Linux specific.
437.TP
d60e92d1 438.BI overwrite \fR=\fPbool
d1429b5c 439If writing, setup the file first and do overwrites. Default: false.
d60e92d1
AC
440.TP
441.BI end_fsync \fR=\fPbool
d1429b5c 442Sync file contents when job exits. Default: false.
d60e92d1
AC
443.TP
444.BI fsync_on_close \fR=\fPbool
445If true, sync file contents on close. This differs from \fBend_fsync\fR in that
d1429b5c 446it will happen on every close, not just at the end of the job. Default: false.
d60e92d1
AC
447.TP
448.BI rwmixcycle \fR=\fPint
449How many milliseconds before switching between reads and writes for a mixed
450workload. Default: 500ms.
451.TP
452.BI rwmixread \fR=\fPint
453Percentage of a mixed workload that should be reads. Default: 50.
454.TP
455.BI rwmixwrite \fR=\fPint
d1429b5c 456Percentage of a mixed workload that should be writes. If \fBrwmixread\fR and
c35dd7a6
JA
457\fBrwmixwrite\fR are given and do not sum to 100%, the latter of the two
458overrides the first. This may interfere with a given rate setting, if fio is
459asked to limit reads or writes to a certain rate. If that is the case, then
460the distribution may be skewed. Default: 50.
d60e92d1
AC
461.TP
462.B norandommap
463Normally \fBfio\fR will cover every block of the file when doing random I/O. If
464this parameter is given, a new offset will be chosen without looking at past
465I/O history. This parameter is mutually exclusive with \fBverify\fR.
466.TP
3ce9dcaf
JA
467.B softrandommap
468See \fBnorandommap\fR. If fio runs with the random block map enabled and it
469fails to allocate the map, if this option is set it will continue without a
470random block map. As coverage will not be as complete as with random maps, this
471option is disabled by default.
472.TP
d60e92d1
AC
473.BI nice \fR=\fPint
474Run job with given nice value. See \fInice\fR\|(2).
475.TP
476.BI prio \fR=\fPint
477Set I/O priority value of this job between 0 (highest) and 7 (lowest). See
478\fIionice\fR\|(1).
479.TP
480.BI prioclass \fR=\fPint
481Set I/O priority class. See \fIionice\fR\|(1).
482.TP
483.BI thinktime \fR=\fPint
484Stall job for given number of microseconds between issuing I/Os.
485.TP
486.BI thinktime_spin \fR=\fPint
487Pretend to spend CPU time for given number of microseconds, sleeping the rest
488of the time specified by \fBthinktime\fR. Only valid if \fBthinktime\fR is set.
489.TP
490.BI thinktime_blocks \fR=\fPint
491Number of blocks to issue before waiting \fBthinktime\fR microseconds.
492Default: 1.
493.TP
494.BI rate \fR=\fPint
c35dd7a6
JA
495Cap bandwidth used by this job. The number is in bytes/sec, the normal postfix
496rules apply. You can use \fBrate\fR=500k to limit reads and writes to 500k each,
497or you can specify read and writes separately. Using \fBrate\fR=1m,500k would
498limit reads to 1MB/sec and writes to 500KB/sec. Capping only reads or writes
499can be done with \fBrate\fR=,500k or \fBrate\fR=500k,. The former will only
500limit writes (to 500KB/sec), the latter will only limit reads.
d60e92d1
AC
501.TP
502.BI ratemin \fR=\fPint
503Tell \fBfio\fR to do whatever it can to maintain at least the given bandwidth.
c35dd7a6
JA
504Failing to meet this requirement will cause the job to exit. The same format
505as \fBrate\fR is used for read vs write separation.
d60e92d1
AC
506.TP
507.BI rate_iops \fR=\fPint
c35dd7a6
JA
508Cap the bandwidth to this number of IOPS. Basically the same as rate, just
509specified independently of bandwidth. The same format as \fBrate\fR is used for
510read vs write seperation. If \fBblocksize\fR is a range, the smallest block
511size is used as the metric.
d60e92d1
AC
512.TP
513.BI rate_iops_min \fR=\fPint
c35dd7a6
JA
514If this rate of I/O is not met, the job will exit. The same format as \fBrate\fR
515is used for read vs write seperation.
d60e92d1
AC
516.TP
517.BI ratecycle \fR=\fPint
518Average bandwidth for \fBrate\fR and \fBratemin\fR over this number of
519milliseconds. Default: 1000ms.
520.TP
521.BI cpumask \fR=\fPint
522Set CPU affinity for this job. \fIint\fR is a bitmask of allowed CPUs the job
523may run on. See \fBsched_setaffinity\fR\|(2).
524.TP
525.BI cpus_allowed \fR=\fPstr
526Same as \fBcpumask\fR, but allows a comma-delimited list of CPU numbers.
527.TP
528.BI startdelay \fR=\fPint
529Delay start of job for the specified number of seconds.
530.TP
531.BI runtime \fR=\fPint
532Terminate processing after the specified number of seconds.
533.TP
534.B time_based
535If given, run for the specified \fBruntime\fR duration even if the files are
536completely read or written. The same workload will be repeated as many times
537as \fBruntime\fR allows.
538.TP
901bb994
JA
539.BI ramp_time \fR=\fPint
540If set, fio will run the specified workload for this amount of time before
541logging any performance numbers. Useful for letting performance settle before
542logging results, thus minimizing the runtime required for stable results. Note
c35dd7a6
JA
543that the \fBramp_time\fR is considered lead in time for a job, thus it will
544increase the total runtime if a special timeout or runtime is specified.
901bb994 545.TP
d60e92d1
AC
546.BI invalidate \fR=\fPbool
547Invalidate buffer-cache for the file prior to starting I/O. Default: true.
548.TP
549.BI sync \fR=\fPbool
550Use synchronous I/O for buffered writes. For the majority of I/O engines,
d1429b5c 551this means using O_SYNC. Default: false.
d60e92d1
AC
552.TP
553.BI iomem \fR=\fPstr "\fR,\fP mem" \fR=\fPstr
554Allocation method for I/O unit buffer. Allowed values are:
555.RS
556.RS
557.TP
558.B malloc
559Allocate memory with \fImalloc\fR\|(3).
560.TP
561.B shm
562Use shared memory buffers allocated through \fIshmget\fR\|(2).
563.TP
564.B shmhuge
565Same as \fBshm\fR, but use huge pages as backing.
566.TP
567.B mmap
568Use \fImmap\fR\|(2) for allocation. Uses anonymous memory unless a filename
569is given after the option in the format `:\fIfile\fR'.
570.TP
571.B mmaphuge
572Same as \fBmmap\fR, but use huge files as backing.
573.RE
574.P
575The amount of memory allocated is the maximum allowed \fBblocksize\fR for the
576job multiplied by \fBiodepth\fR. For \fBshmhuge\fR or \fBmmaphuge\fR to work,
577the system must have free huge pages allocated. \fBmmaphuge\fR also needs to
2e266ba6
JA
578have hugetlbfs mounted, and \fIfile\fR must point there. At least on Linux,
579huge pages must be manually allocated. See \fB/proc/sys/vm/nr_hugehages\fR
580and the documentation for that. Normally you just need to echo an appropriate
581number, eg echoing 8 will ensure that the OS has 8 huge pages ready for
582use.
d60e92d1
AC
583.RE
584.TP
d529ee19
JA
585.BI iomem_align \fR=\fPint
586This indiciates the memory alignment of the IO memory buffers. Note that the
587given alignment is applied to the first IO unit buffer, if using \fBiodepth\fR
588the alignment of the following buffers are given by the \fBbs\fR used. In
589other words, if using a \fBbs\fR that is a multiple of the page sized in the
590system, all buffers will be aligned to this value. If using a \fBbs\fR that
591is not page aligned, the alignment of subsequent IO memory buffers is the
592sum of the \fBiomem_align\fR and \fBbs\fR used.
593.TP
f7fa2653 594.BI hugepage\-size \fR=\fPint
d60e92d1 595Defines the size of a huge page. Must be at least equal to the system setting.
b22989b9 596Should be a multiple of 1MB. Default: 4MB.
d60e92d1
AC
597.TP
598.B exitall
599Terminate all jobs when one finishes. Default: wait for each job to finish.
600.TP
601.BI bwavgtime \fR=\fPint
602Average bandwidth calculations over the given time in milliseconds. Default:
603500ms.
604.TP
605.BI create_serialize \fR=\fPbool
d1429b5c 606If true, serialize file creation for the jobs. Default: true.
d60e92d1
AC
607.TP
608.BI create_fsync \fR=\fPbool
609\fIfsync\fR\|(2) data file after creation. Default: true.
610.TP
6b7f6851
JA
611.BI create_on_open \fR=\fPbool
612If true, the files are not created until they are opened for IO by the job.
613.TP
e9f48479
JA
614.BI pre_read \fR=\fPbool
615If this is given, files will be pre-read into memory before starting the given
616IO operation. This will also clear the \fR \fBinvalidate\fR flag, since it is
9c0d2241
JA
617pointless to pre-read and then drop the cache. This will only work for IO
618engines that are seekable, since they allow you to read the same data
619multiple times. Thus it will not work on eg network or splice IO.
e9f48479 620.TP
d60e92d1
AC
621.BI unlink \fR=\fPbool
622Unlink job files when done. Default: false.
623.TP
624.BI loops \fR=\fPint
625Specifies the number of iterations (runs of the same workload) of this job.
626Default: 1.
627.TP
628.BI do_verify \fR=\fPbool
629Run the verify phase after a write phase. Only valid if \fBverify\fR is set.
630Default: true.
631.TP
632.BI verify \fR=\fPstr
633Method of verifying file contents after each iteration of the job. Allowed
634values are:
635.RS
636.RS
637.TP
b892dc08 638.B md5 crc16 crc32 crc32c crc32c-intel crc64 crc7 sha256 sha512 sha1
0539d758
JA
639Store appropriate checksum in the header of each block. crc32c-intel is
640hardware accelerated SSE4.2 driven, falls back to regular crc32c if
641not supported by the system.
d60e92d1
AC
642.TP
643.B meta
644Write extra information about each I/O (timestamp, block number, etc.). The
996093bb 645block number is verified. See \fBverify_pattern\fR as well.
d60e92d1
AC
646.TP
647.B null
648Pretend to verify. Used for testing internals.
649.RE
b892dc08
JA
650
651This option can be used for repeated burn-in tests of a system to make sure
652that the written data is also correctly read back. If the data direction given
653is a read or random read, fio will assume that it should verify a previously
654written file. If the data direction includes any form of write, the verify will
655be of the newly written data.
d60e92d1
AC
656.RE
657.TP
658.BI verify_sort \fR=\fPbool
659If true, written verify blocks are sorted if \fBfio\fR deems it to be faster to
660read them back in a sorted manner. Default: true.
661.TP
f7fa2653 662.BI verify_offset \fR=\fPint
d60e92d1 663Swap the verification header with data somewhere else in the block before
d1429b5c 664writing. It is swapped back before verifying.
d60e92d1 665.TP
f7fa2653 666.BI verify_interval \fR=\fPint
d60e92d1
AC
667Write the verification header for this number of bytes, which should divide
668\fBblocksize\fR. Default: \fBblocksize\fR.
669.TP
996093bb
JA
670.BI verify_pattern \fR=\fPstr
671If set, fio will fill the io buffers with this pattern. Fio defaults to filling
672with totally random bytes, but sometimes it's interesting to fill with a known
673pattern for io verification purposes. Depending on the width of the pattern,
674fio will fill 1/2/3/4 bytes of the buffer at the time(it can be either a
675decimal or a hex number). The verify_pattern if larger than a 32-bit quantity
676has to be a hex number that starts with either "0x" or "0X". Use with
677\fBverify\fP=meta.
678.TP
d60e92d1
AC
679.BI verify_fatal \fR=\fPbool
680If true, exit the job on the first observed verification failure. Default:
681false.
682.TP
e8462bd8
JA
683.BI verify_async \fR=\fPint
684Fio will normally verify IO inline from the submitting thread. This option
685takes an integer describing how many async offload threads to create for IO
686verification instead, causing fio to offload the duty of verifying IO contents
c85c324c
JA
687to one or more separate threads. If using this offload option, even sync IO
688engines can benefit from using an \fBiodepth\fR setting higher than 1, as it
689allows them to have IO in flight while verifies are running.
e8462bd8
JA
690.TP
691.BI verify_async_cpus \fR=\fPstr
692Tell fio to set the given CPU affinity on the async IO verification threads.
693See \fBcpus_allowed\fP for the format used.
694.TP
6f87418f
JA
695.BI verify_backlog \fR=\fPint
696Fio will normally verify the written contents of a job that utilizes verify
697once that job has completed. In other words, everything is written then
698everything is read back and verified. You may want to verify continually
699instead for a variety of reasons. Fio stores the meta data associated with an
700IO block in memory, so for large verify workloads, quite a bit of memory would
701be used up holding this meta data. If this option is enabled, fio will verify
702the previously written blocks before continuing to write new ones.
703.TP
704.BI verify_backlog_batch \fR=\fPint
705Control how many blocks fio will verify if verify_backlog is set. If not set,
706will default to the value of \fBverify_backlog\fR (meaning the entire queue is
707read back and verified).
708.TP
d60e92d1 709.B stonewall
d1429b5c 710Wait for preceeding jobs in the job file to exit before starting this one.
d60e92d1
AC
711\fBstonewall\fR implies \fBnew_group\fR.
712.TP
713.B new_group
714Start a new reporting group. If not given, all jobs in a file will be part
715of the same reporting group, unless separated by a stonewall.
716.TP
717.BI numjobs \fR=\fPint
718Number of clones (processes/threads performing the same workload) of this job.
719Default: 1.
720.TP
721.B group_reporting
722If set, display per-group reports instead of per-job when \fBnumjobs\fR is
723specified.
724.TP
725.B thread
726Use threads created with \fBpthread_create\fR\|(3) instead of processes created
727with \fBfork\fR\|(2).
728.TP
f7fa2653 729.BI zonesize \fR=\fPint
d60e92d1
AC
730Divide file into zones of the specified size in bytes. See \fBzoneskip\fR.
731.TP
f7fa2653 732.BI zoneskip \fR=\fPint
d1429b5c 733Skip the specified number of bytes when \fBzonesize\fR bytes of data have been
d60e92d1
AC
734read.
735.TP
736.BI write_iolog \fR=\fPstr
737Write the issued I/O patterns to the specified file.
738.TP
739.BI read_iolog \fR=\fPstr
740Replay the I/O patterns contained in the specified file generated by
741\fBwrite_iolog\fR, or may be a \fBblktrace\fR binary file.
742.TP
901bb994
JA
743.B write_bw_log \fR=\fPstr
744If given, write a bandwidth log of the jobs in this job file. Can be used to
745store data of the bandwidth of the jobs in their lifetime. The included
746fio_generate_plots script uses gnuplot to turn these text files into nice
747graphs. See \fBwrite_log_log\fR for behaviour of given filename. For this
748option, the postfix is _bw.log.
d60e92d1
AC
749.TP
750.B write_lat_log
901bb994
JA
751Same as \fBwrite_bw_log\fR, but writes I/O completion latencies. If no
752filename is given with this option, the default filename of "jobname_type.log"
753is used. Even if the filename is given, fio will still append the type of log.
754.TP
02af0988
JA
755.B disable_lat \fR=\fPbool
756Disable measurements of total latency numbers. Useful only for cutting
901bb994
JA
757back the number of calls to gettimeofday, as that does impact performance at
758really high IOPS rates. Note that to really get rid of a large amount of these
759calls, this option must be used with disable_slat and disable_bw as well.
760.TP
02af0988
JA
761.B disable_clat \fR=\fPbool
762Disable measurements of submission latency numbers. See \fBdisable_lat\fR.
763.TP
901bb994 764.B disable_slat \fR=\fPbool
02af0988 765Disable measurements of submission latency numbers. See \fBdisable_lat\fR.
901bb994
JA
766.TP
767.B disable_bw_measurement \fR=\fPbool
02af0988 768Disable measurements of throughput/bandwidth numbers. See \fBdisable_lat\fR.
d60e92d1 769.TP
f7fa2653 770.BI lockmem \fR=\fPint
d60e92d1
AC
771Pin the specified amount of memory with \fBmlock\fR\|(2). Can be used to
772simulate a smaller amount of memory.
773.TP
774.BI exec_prerun \fR=\fPstr
775Before running the job, execute the specified command with \fBsystem\fR\|(3).
776.TP
777.BI exec_postrun \fR=\fPstr
778Same as \fBexec_prerun\fR, but the command is executed after the job completes.
779.TP
780.BI ioscheduler \fR=\fPstr
781Attempt to switch the device hosting the file to the specified I/O scheduler.
782.TP
783.BI cpuload \fR=\fPint
784If the job is a CPU cycle-eater, attempt to use the specified percentage of
785CPU cycles.
786.TP
787.BI cpuchunks \fR=\fPint
788If the job is a CPU cycle-eater, split the load into cycles of the
789given time in milliseconds.
790.TP
791.BI disk_util \fR=\fPbool
d1429b5c 792Generate disk utilization statistics if the platform supports it. Default: true.
901bb994
JA
793.TP
794.BI gtod_reduce \fR=\fPbool
795Enable all of the gettimeofday() reducing options (disable_clat, disable_slat,
796disable_bw) plus reduce precision of the timeout somewhat to really shrink the
797gettimeofday() call count. With this option enabled, we only do about 0.4% of
798the gtod() calls we would have done if all time keeping was enabled.
799.TP
800.BI gtod_cpu \fR=\fPint
801Sometimes it's cheaper to dedicate a single thread of execution to just getting
802the current time. Fio (and databases, for instance) are very intensive on
803gettimeofday() calls. With this option, you can set one CPU aside for doing
804nothing but logging current time to a shared memory location. Then the other
805threads/processes that run IO workloads need only copy that segment, instead of
806entering the kernel with a gettimeofday() call. The CPU set aside for doing
807these time calls will be excluded from other uses. Fio will manually clear it
808from the CPU mask of other jobs.
f2bba182 809.TP
a696fa2a
JA
810.BI cgroup \fR=\fPstr
811Add job to this control group. If it doesn't exist, it will be created.
6adb38a1
JA
812The system must have a mounted cgroup blkio mount point for this to work. If
813your system doesn't have it mounted, you can do so with:
814
815# mount -t cgroup -o blkio none /cgroup
a696fa2a
JA
816.TP
817.BI cgroup_weight \fR=\fPint
818Set the weight of the cgroup to this value. See the documentation that comes
819with the kernel, allowed values are in the range of 100..1000.
e0b0d892 820.TP
7de87099
VG
821.BI cgroup_nodelete \fR=\fPbool
822Normally fio will delete the cgroups it has created after the job completion.
823To override this behavior and to leave cgroups around after the job completion,
824set cgroup_nodelete=1. This can be useful if one wants to inspect various
825cgroup files after job completion. Default: false
826.TP
e0b0d892
JA
827.BI uid \fR=\fPint
828Instead of running as the invoking user, set the user ID to this value before
829the thread/process does any work.
830.TP
831.BI gid \fR=\fPint
832Set group ID, see \fBuid\fR.
d60e92d1 833.SH OUTPUT
d1429b5c
AC
834While running, \fBfio\fR will display the status of the created jobs. For
835example:
d60e92d1 836.RS
d1429b5c 837.P
d60e92d1
AC
838Threads: 1: [_r] [24.8% done] [ 13509/ 8334 kb/s] [eta 00h:01m:31s]
839.RE
840.P
d1429b5c
AC
841The characters in the first set of brackets denote the current status of each
842threads. The possible values are:
843.P
844.PD 0
d60e92d1
AC
845.RS
846.TP
847.B P
848Setup but not started.
849.TP
850.B C
851Thread created.
852.TP
853.B I
854Initialized, waiting.
855.TP
856.B R
857Running, doing sequential reads.
858.TP
859.B r
860Running, doing random reads.
861.TP
862.B W
863Running, doing sequential writes.
864.TP
865.B w
866Running, doing random writes.
867.TP
868.B M
869Running, doing mixed sequential reads/writes.
870.TP
871.B m
872Running, doing mixed random reads/writes.
873.TP
874.B F
875Running, currently waiting for \fBfsync\fR\|(2).
876.TP
877.B V
878Running, verifying written data.
879.TP
880.B E
881Exited, not reaped by main thread.
882.TP
883.B \-
884Exited, thread reaped.
885.RE
d1429b5c 886.PD
d60e92d1
AC
887.P
888The second set of brackets shows the estimated completion percentage of
889the current group. The third set shows the read and write I/O rate,
890respectively. Finally, the estimated run time of the job is displayed.
891.P
892When \fBfio\fR completes (or is interrupted by Ctrl-C), it will show data
893for each thread, each group of threads, and each disk, in that order.
894.P
895Per-thread statistics first show the threads client number, group-id, and
896error code. The remaining figures are as follows:
897.RS
d60e92d1
AC
898.TP
899.B io
900Number of megabytes of I/O performed.
901.TP
902.B bw
903Average data rate (bandwidth).
904.TP
905.B runt
906Threads run time.
907.TP
908.B slat
909Submission latency minimum, maximum, average and standard deviation. This is
910the time it took to submit the I/O.
911.TP
912.B clat
913Completion latency minimum, maximum, average and standard deviation. This
914is the time between submission and completion.
915.TP
916.B bw
917Bandwidth minimum, maximum, percentage of aggregate bandwidth received, average
918and standard deviation.
919.TP
920.B cpu
921CPU usage statistics. Includes user and system time, number of context switches
922this thread went through and number of major and minor page faults.
923.TP
924.B IO depths
925Distribution of I/O depths. Each depth includes everything less than (or equal)
926to it, but greater than the previous depth.
927.TP
928.B IO issued
929Number of read/write requests issued, and number of short read/write requests.
930.TP
931.B IO latencies
932Distribution of I/O completion latencies. The numbers follow the same pattern
933as \fBIO depths\fR.
934.RE
d60e92d1
AC
935.P
936The group statistics show:
d1429b5c 937.PD 0
d60e92d1
AC
938.RS
939.TP
940.B io
941Number of megabytes I/O performed.
942.TP
943.B aggrb
944Aggregate bandwidth of threads in the group.
945.TP
946.B minb
947Minimum average bandwidth a thread saw.
948.TP
949.B maxb
950Maximum average bandwidth a thread saw.
951.TP
952.B mint
d1429b5c 953Shortest runtime of threads in the group.
d60e92d1
AC
954.TP
955.B maxt
956Longest runtime of threads in the group.
957.RE
d1429b5c 958.PD
d60e92d1
AC
959.P
960Finally, disk statistics are printed with reads first:
d1429b5c 961.PD 0
d60e92d1
AC
962.RS
963.TP
964.B ios
965Number of I/Os performed by all groups.
966.TP
967.B merge
968Number of merges in the I/O scheduler.
969.TP
970.B ticks
971Number of ticks we kept the disk busy.
972.TP
973.B io_queue
974Total time spent in the disk queue.
975.TP
976.B util
977Disk utilization.
978.RE
d1429b5c 979.PD
d60e92d1
AC
980.SH TERSE OUTPUT
981If the \fB\-\-minimal\fR option is given, the results will be printed in a
525c2bfa
JA
982semicolon-delimited format suitable for scripted use. Note that the first
983number in the line is the version number. If the output has to be changed
984for some reason, this number will be incremented by 1 to signify that
985change. The fields are:
d60e92d1
AC
986.P
987.RS
525c2bfa 988.B version, jobname, groupid, error
d60e92d1
AC
989.P
990Read status:
991.RS
b22989b9 992.B KB I/O, bandwidth \fR(KB/s)\fP, runtime \fR(ms)\fP
d60e92d1
AC
993.P
994Submission latency:
995.RS
996.B min, max, mean, standard deviation
997.RE
998Completion latency:
999.RS
1000.B min, max, mean, standard deviation
1001.RE
525c2bfa
JA
1002Total latency:
1003.RS
1004.B min, max, mean, standard deviation
1005.RE
d60e92d1
AC
1006Bandwidth:
1007.RS
1008.B min, max, aggregate percentage of total, mean, standard deviation
1009.RE
1010.RE
1011.P
1012Write status:
1013.RS
b22989b9 1014.B KB I/O, bandwidth \fR(KB/s)\fP, runtime \fR(ms)\fP
d60e92d1
AC
1015.P
1016Submission latency:
1017.RS
1018.B min, max, mean, standard deviation
1019.RE
1020Completion latency:
1021.RS
1022.B min, max, mean, standard deviation
1023.RE
525c2bfa
JA
1024Total latency:
1025.RS
1026.B min, max, mean, standard deviation
1027.RE
d60e92d1
AC
1028Bandwidth:
1029.RS
1030.B min, max, aggregate percentage of total, mean, standard deviation
1031.RE
1032.RE
1033.P
d1429b5c 1034CPU usage:
d60e92d1 1035.RS
bd2626f0 1036.B user, system, context switches, major page faults, minor page faults
d60e92d1
AC
1037.RE
1038.P
1039IO depth distribution:
1040.RS
1041.B <=1, 2, 4, 8, 16, 32, >=64
1042.RE
1043.P
1044IO latency distribution (ms):
1045.RS
1046.B <=2, 4, 10, 20, 50, 100, 250, 500, 750, 1000, >=2000
1047.RE
1048.P
1049.B text description
1050.RE
1051.SH AUTHORS
1052.B fio
aa58d252
JA
1053was written by Jens Axboe <jens.axboe@oracle.com>,
1054now Jens Axboe <jaxboe@fusionio.com>.
d1429b5c
AC
1055.br
1056This man page was written by Aaron Carroll <aaronc@cse.unsw.edu.au> based
d60e92d1
AC
1057on documentation by Jens Axboe.
1058.SH "REPORTING BUGS"
482900c9 1059Report bugs to the \fBfio\fR mailing list <fio@vger.kernel.org>.
d1429b5c 1060See \fBREADME\fR.
d60e92d1 1061.SH "SEE ALSO"
d1429b5c
AC
1062For further documentation see \fBHOWTO\fR and \fBREADME\fR.
1063.br
1064Sample jobfiles are available in the \fBexamples\fR directory.
d60e92d1 1065