Clear f on error get_next_file_rr()

[fio.git] / fio.1

diff --git a/fio.1 b/fio.1
index f491c533272dc85ff685c6cbc74a5e092d905b6a..10ff4b776a10c58d5692f6ea175296b1d7482fed 100644 (file)
--- a/fio.1
+++ b/fio.1
@@ -25,7 +25,7 @@ Generate per-job latency logs.
 Generate per-job bandwidth logs.
 .TP
 .B \-\-minimal
 Generate per-job bandwidth logs.
 .TP
 .B \-\-minimal

-Print statistics in a terse, semicolon\-delimited format.
+Print statistics in a terse, semicolon-delimited format.

 .TP
 .BI \-\-showcmd \fR=\fPjobfile
 Convert \fIjobfile\fR to a set of command-line options.
 .TP
 .BI \-\-showcmd \fR=\fPjobfile
 Convert \fIjobfile\fR to a set of command-line options.


@@ -37,9 +37,17 @@ Enable read-only safety checks.
 Specifies when real-time ETA estimate should be printed.  \fIwhen\fR may
 be one of `always', `never' or `auto'.
 .TP
 Specifies when real-time ETA estimate should be printed.  \fIwhen\fR may
 be one of `always', `never' or `auto'.
 .TP

+.BI \-\-section \fR=\fPsec
+Only run section \fIsec\fR from job file.
+.TP

 .BI \-\-cmdhelp \fR=\fPcommand
 Print help information for \fIcommand\fR.  May be `all' for all commands.
 .TP
 .BI \-\-cmdhelp \fR=\fPcommand
 Print help information for \fIcommand\fR.  May be `all' for all commands.
 .TP

+.BI \-\-debug \fR=\fPtype
+Enable verbose tracing of various fio actions. May be `all' for all types
+or individual types seperated by a comma (eg --debug=io,file). `help' will
+list all available tracing options.
+.TP

 .B \-\-help
 Display usage information and exit.
 .TP
 .B \-\-help
 Display usage information and exit.
 .TP


@@ -52,8 +60,10 @@ extend to the next job name.  The job name can be any ASCII string
 except `global', which has a special meaning.  Following the job name is
 a sequence of zero or more parameters, one per line, that define the
 behavior of the job.  Any line starting with a `;' or `#' character is
 except `global', which has a special meaning.  Following the job name is
 a sequence of zero or more parameters, one per line, that define the
 behavior of the job.  Any line starting with a `;' or `#' character is

-considered a comment and ignored.  See section EXAMPLES for sample
-job files.
+considered a comment and ignored.
+.P
+If \fIjobfile\fR is specified as `-', the job file will be read from
+standard input.

 .SS "Global Section"
 The global section contains default parameters for jobs specified in the
 job file.  A job is only affected by global sections residing above it,
 .SS "Global Section"
 The global section contains default parameters for jobs specified in the
 job file.  A job is only affected by global sections residing above it,


@@ -67,26 +77,24 @@ Some parameters may take arguments of a specific type.  The types used are:
 String: a sequence of alphanumeric characters.
 .TP
 .I int
 String: a sequence of alphanumeric characters.
 .TP
 .I int

-Integer: a whole number, possibly negative.  If prefixed with `0x', the value
-is assumed to be base 16 (hexadecimal).
-.TP
-.I siint

 SI integer: a whole number, possibly containing a suffix denoting the base unit
 of the value.  Accepted suffixes are `k', 'M' and 'G', denoting kilo (1024),
 SI integer: a whole number, possibly containing a suffix denoting the base unit
 of the value.  Accepted suffixes are `k', 'M' and 'G', denoting kilo (1024),

-mega (1024*1024) and giga (1024*1024*1024) respectively.
+mega (1024*1024) and giga (1024*1024*1024) respectively. If prefixed with '0x',
+the value is assumed to be base 16 (hexadecimal).

 .TP
 .I bool
 Boolean: a true or false value. `0' denotes false, `1' denotes true.
 .TP
 .I irange
 Integer range: a range of integers specified in the format
 .TP
 .I bool
 Boolean: a true or false value. `0' denotes false, `1' denotes true.
 .TP
 .I irange
 Integer range: a range of integers specified in the format

-\fIlower\fR:\fIupper\fR or \fIlower\fR-\fIupper\fR. \fIlower\fR and \fIupper\fR
-may contain a suffix as described above.  If an option allows two sets of ranges,
-they are separated with a `,' or `/' character. For example: `8-8k/8M-4G'.
+\fIlower\fR:\fIupper\fR or \fIlower\fR\-\fIupper\fR. \fIlower\fR and
+\fIupper\fR may contain a suffix as described above.  If an option allows two
+sets of ranges, they are separated with a `,' or `/' character. For example:
+`8\-8k/8M\-4G'.

 .SS "Parameter List"
 .TP
 .BI name \fR=\fPstr
 .SS "Parameter List"
 .TP
 .BI name \fR=\fPstr

-May be used to override the job name.  On the command line, this paramter
+May be used to override the job name.  On the command line, this parameter

 has the special purpose of signalling the start of a new job.
 .TP
 .BI description \fR=\fPstr
 has the special purpose of signalling the start of a new job.
 .TP
 .BI description \fR=\fPstr


@@ -100,12 +108,13 @@ than `./'.
 .BI filename \fR=\fPstr
 .B fio
 normally makes up a file name based on the job name, thread number, and file
 .BI filename \fR=\fPstr
 .B fio
 normally makes up a file name based on the job name, thread number, and file

-number. If you want to share files between threads in a job or several jobs, specify
-a \fIfilename\fR for each of them to override the default. If the I/O engine used is
-`net', \fIfilename\fR is the host and port to connect to in the format
-\fIhost\fR/\fIport\fR. If the I/O engine is file-based, you can specify a number of
-files by separating the names with a `:' character. `-' is a reserved name, meaning
-stdin or stdout, depending on the read/write direction set.
+number. If you want to share files between threads in a job or several jobs,
+specify a \fIfilename\fR for each of them to override the default. If the I/O
+engine used is `net', \fIfilename\fR is the host and port to connect to in the
+format \fIhost\fR/\fIport\fR. If the I/O engine is file-based, you can specify
+a number of files by separating the names with a `:' character. `\-' is a
+reserved name, meaning stdin or stdout, depending on the read/write direction
+set.

 .TP
 .BI opendir \fR=\fPstr
 Recursively open any files below directory \fIstr\fR.
 .TP
 .BI opendir \fR=\fPstr
 Recursively open any files below directory \fIstr\fR.


@@ -116,38 +125,38 @@ Type of I/O pattern.  Accepted values are:
 .RS
 .TP
 .B read
 .RS
 .TP
 .B read

-Sequential reads
+Sequential reads.

 .TP
 .B write
 .TP
 .B write

-Sequential writes
+Sequential writes.

 .TP
 .B randread
 .TP
 .B randread

-Random reads
+Random reads.

 .TP
 .B randwrite
 .TP
 .B randwrite

-Random writes
+Random writes.

 .TP
 .B rw
 .TP
 .B rw

-Mixed sequential reads and writes
+Mixed sequential reads and writes.

 .TP
 .B randrw 
 .TP
 .B randrw 

-Mixed random reads and writes
+Mixed random reads and writes.

 .RE
 .P
 .RE
 .P

-For mixed I/O, the default split is 50/50.  For random I/O, the number of I/Os to
-perform before getting a new offset can be specified by appending `:\fIint\fR' to
-the pattern type.  The default is 1.
+For mixed I/O, the default split is 50/50.  For random I/O, the number of I/Os
+to perform before getting a new offset can be specified by appending
+`:\fIint\fR' to the pattern type.  The default is 1.

 .RE
 .TP
 .BI randrepeat \fR=\fPbool
 Seed the random number generator in a predictable way so results are repeatable
 .RE
 .TP
 .BI randrepeat \fR=\fPbool
 Seed the random number generator in a predictable way so results are repeatable

-across runs.
+across runs.  Default: true.

 .TP
 .BI fadvise_hint \fR=\fPbool
 .TP
 .BI fadvise_hint \fR=\fPbool

-Disable use of \fIposix_fadvise\fR\|(2) to advise the kernel what I/O patters are
-likely to be issued. Default: true.
+Disable use of \fIposix_fadvise\fR\|(2) to advise the kernel what I/O patterns
+are likely to be issued. Default: true.

 .TP
 .TP

-.BI size \fR=\fPsiint
+.BI size \fR=\fPint

 Total size of I/O for this job.  \fBfio\fR will run until this many bytes have
 been transfered, unless limited by other options (\fBruntime\fR, for instance).
 Unless \fBnr_files\fR and \fBfilesize\fR options are given, this amount will be
 Total size of I/O for this job.  \fBfio\fR will run until this many bytes have
 been transfered, unless limited by other options (\fBruntime\fR, for instance).
 Unless \fBnr_files\fR and \fBfilesize\fR options are given, this amount will be


@@ -155,26 +164,54 @@ divided between the available files for the job.
 .TP
 .BI filesize \fR=\fPirange
 Individual file sizes. May be a range, in which case \fBfio\fR will select sizes
 .TP
 .BI filesize \fR=\fPirange
 Individual file sizes. May be a range, in which case \fBfio\fR will select sizes

-for files at random within the given range, limited to \fBsize\fR in total (if that
-is given). If \fBfilesize\fR is not specified, each created file is the same size.
+for files at random within the given range, limited to \fBsize\fR in total (if
+that is given). If \fBfilesize\fR is not specified, each created file is the
+same size.

 .TP
 .TP

-.BI blocksize \fR=\fPsiint "\fR,\fB bs" \fR=\fPsiint
+.BI blocksize \fR=\fPint[,int] "\fR,\fB bs" \fR=\fPint[,int]

 Block size for I/O units.  Default: 4k.  Values for reads and writes can be
 specified seperately in the format \fIread\fR,\fIwrite\fR, either of
 which may be empty to leave that value at its default.
 .TP
 Block size for I/O units.  Default: 4k.  Values for reads and writes can be
 specified seperately in the format \fIread\fR,\fIwrite\fR, either of
 which may be empty to leave that value at its default.
 .TP

-.BI blocksize_range \fR=\fPirange "\fR,\fB bsrange" \fR=\fPirange
-Specify a range of I/O block sizes.  The issued I/O unit will always be a multiple
-of the minimum size, unless \fBblocksize_unaligned\fR is set.  Applied to both reads
-and writes, but can be specified seperately (see \fBblocksize\fR).
+.BI blocksize_range \fR=\fPirange[,irange] "\fR,\fB bsrange" \fR=\fPirange[,irange]
+Specify a range of I/O block sizes.  The issued I/O unit will always be a
+multiple of the minimum size, unless \fBblocksize_unaligned\fR is set.  Applies
+to both reads and writes if only one range is given, but can be specified
+seperately with a comma seperating the values. Example: bsrange=1k-4k,2k-8k.
+Also (see \fBblocksize\fR).
+.TP
+.BI bssplit \fR=\fPstr
+This option allows even finer grained control of the block sizes issued,
+not just even splits between them. With this option, you can weight various
+block sizes for exact control of the issued IO for a job that has mixed
+block sizes. The format of the option is bssplit=blocksize/percentage,
+optionally adding as many definitions as needed seperated by a colon.
+Example: bssplit=4k/10:64k/50:32k/40 would issue 50% 64k blocks, 10% 4k
+blocks and 40% 32k blocks. \fBbssplit\fR also supports giving separate
+splits to reads and writes. The format is identical to what the
+\fBbs\fR option accepts, the read and write parts are separated with a
+comma.

 .TP
 .B blocksize_unaligned\fR,\fP bs_unaligned
 .TP
 .B blocksize_unaligned\fR,\fP bs_unaligned

-If set, any size in \fBblocksize_range\fR may be used.  This typically won't work
-with direct I/O, as that normally requires sector alignment.
+If set, any size in \fBblocksize_range\fR may be used.  This typically won't
+work with direct I/O, as that normally requires sector alignment.
+.TP
+.BI blockalign \fR=\fPint[,int] "\fR,\fB ba" \fR=\fPint[,int]
+At what boundary to align random IO offsets. Defaults to the same as
+'blocksize' the minimum blocksize given.  Minimum alignment is typically 512b
+for using direct IO, though it usually depends on the hardware block size.
+This option is mutually exclusive with using a random map for files, so it
+will turn off that option.

 .TP
 .B zero_buffers
 Initialise buffers with all zeros. Default: fill buffers with random data.
 .TP
 .TP
 .B zero_buffers
 Initialise buffers with all zeros. Default: fill buffers with random data.
 .TP

+.B refill_buffers
+If this option is given, fio will refill the IO buffers on every submit. The
+default is to only fill it at init time and reuse that data. Only makes sense
+if zero_buffers isn't specified, naturally. If data verification is enabled,
+refill_buffers is also automatically enabled.
+.TP

 .BI nrfiles \fR=\fPint
 Number of files to use for this job.  Default: 1.
 .TP
 .BI nrfiles \fR=\fPint
 Number of files to use for this job.  Default: 1.
 .TP


@@ -191,6 +228,8 @@ Choose a file at random
 .TP
 .B roundrobin
 Round robin over open files (default).
 .TP
 .B roundrobin
 Round robin over open files (default).

+.B sequential
+Do each file in the set sequentially.

 .RE
 .P
 The number of I/Os to issue before switching a new file can be specified by
 .RE
 .P
 The number of I/Os to issue before switching a new file can be specified by


@@ -206,6 +245,13 @@ Defines how the job issues I/O.  The following types are defined:
 Basic \fIread\fR\|(2) or \fIwrite\fR\|(2) I/O.  \fIfseek\fR\|(2) is used to
 position the I/O location.
 .TP
 Basic \fIread\fR\|(2) or \fIwrite\fR\|(2) I/O.  \fIfseek\fR\|(2) is used to
 position the I/O location.
 .TP

+.B psync
+Basic \fIpread\fR\|(2) or \fIpwrite\fR\|(2) I/O.
+.TP
+.B vsync
+Basic \fIreadv\fR\|(2) or \fIwritev\fR\|(2) I/O. Will emulate queuing by
+coalescing adjacents IOs into a single submission.
+.TP

 .B libaio
 Linux native asynchronous I/O.
 .TP
 .B libaio
 Linux native asynchronous I/O.
 .TP


@@ -213,19 +259,20 @@ Linux native asynchronous I/O.
 glibc POSIX asynchronous I/O using \fIaio_read\fR\|(3) and \fIaio_write\fR\|(3).
 .TP
 .B mmap
 glibc POSIX asynchronous I/O using \fIaio_read\fR\|(3) and \fIaio_write\fR\|(3).
 .TP
 .B mmap

-File is memory mapped with \fImmap\fR\|(2) and data coped using \fImemcpy\fR\|(3).
+File is memory mapped with \fImmap\fR\|(2) and data copied using
+\fImemcpy\fR\|(3).

 .TP
 .B splice
 .TP
 .B splice

-\fIsplice\fR\|(2) is used to transfer the data and \fIvmsplice\fR\|(2) to transfer
-data from user-space to the kernel.
+\fIsplice\fR\|(2) is used to transfer the data and \fIvmsplice\fR\|(2) to
+transfer data from user-space to the kernel.

 .TP
 .B syslet-rw
 Use the syslet system calls to make regular read/write asynchronous.
 .TP
 .B sg
 SCSI generic sg v3 I/O. May be either synchronous using the SG_IO ioctl, or if
 .TP
 .B syslet-rw
 Use the syslet system calls to make regular read/write asynchronous.
 .TP
 .B sg
 SCSI generic sg v3 I/O. May be either synchronous using the SG_IO ioctl, or if

-the target is an sg character device, we use \fIread\fR\|(2) and \fIwrite\fR\|(2)
-for asynchronous I/O.
+the target is an sg character device, we use \fIread\fR\|(2) and
+\fIwrite\fR\|(2) for asynchronous I/O.

 .TP
 .B null
 Doesn't transfer any data, just pretends to.  Mainly used to exercise \fBfio\fR
 .TP
 .B null
 Doesn't transfer any data, just pretends to.  Mainly used to exercise \fBfio\fR


@@ -240,15 +287,15 @@ Transfer over the network.  \fBfilename\fR must be set appropriately to
 Like \fBnet\fR, but uses \fIsplice\fR\|(2) and \fIvmsplice\fR\|(2) to map data
 and send/receive.
 .TP
 Like \fBnet\fR, but uses \fIsplice\fR\|(2) and \fIvmsplice\fR\|(2) to map data
 and send/receive.
 .TP

-.B cpu
+.B cpuio

 Doesn't transfer any data, but burns CPU cycles according to \fBcpuload\fR and
 \fBcpucycles\fR parameters.
 .TP
 .B guasi
 The GUASI I/O engine is the Generic Userspace Asynchronous Syscall Interface
 approach to asycnronous I/O.
 Doesn't transfer any data, but burns CPU cycles according to \fBcpuload\fR and
 \fBcpucycles\fR parameters.
 .TP
 .B guasi
 The GUASI I/O engine is the Generic Userspace Asynchronous Syscall Interface
 approach to asycnronous I/O.

-
-See <http://www.xmailserver.org/guasi-lib.html>.
+.br
+See <http://www.xmailserver.org/guasi\-lib.html>.

 .TP
 .B external
 Loads an external I/O engine object file.  Append the engine filename as
 .TP
 .B external
 Loads an external I/O engine object file.  Append the engine filename as


@@ -273,22 +320,22 @@ If true, use non-buffered I/O (usually O_DIRECT).  Default: false.
 If true, use buffered I/O.  This is the opposite of the \fBdirect\fR parameter.
 Default: true.
 .TP
 If true, use buffered I/O.  This is the opposite of the \fBdirect\fR parameter.
 Default: true.
 .TP

-.BI offset \fR=\fPsiint
+.BI offset \fR=\fPint

 Offset in the file to start I/O. Data before the offset will not be touched.
 .TP
 .BI fsync \fR=\fPint
 Offset in the file to start I/O. Data before the offset will not be touched.
 .TP
 .BI fsync \fR=\fPint

-How many I/Os to perform before issuing an \fBfsync\fR\|(2) of dirty data.  If 0, don't
-sync.  Default: 0.
+How many I/Os to perform before issuing an \fBfsync\fR\|(2) of dirty data.  If
+0, don't sync.  Default: 0.

 .TP
 .BI overwrite \fR=\fPbool
 .TP
 .BI overwrite \fR=\fPbool

-If writing, setup the file first and do overwrites.
+If writing, setup the file first and do overwrites.  Default: false.

 .TP
 .BI end_fsync \fR=\fPbool
 .TP
 .BI end_fsync \fR=\fPbool

-If true, sync file contents when job exits.
+Sync file contents when job exits.  Default: false.

 .TP
 .BI fsync_on_close \fR=\fPbool
 If true, sync file contents on close.  This differs from \fBend_fsync\fR in that
 .TP
 .BI fsync_on_close \fR=\fPbool
 If true, sync file contents on close.  This differs from \fBend_fsync\fR in that

-it will happen on every close, not just at the end of the job.
+it will happen on every close, not just at the end of the job.  Default: false.

 .TP
 .BI rwmixcycle \fR=\fPint
 How many milliseconds before switching between reads and writes for a mixed
 .TP
 .BI rwmixcycle \fR=\fPint
 How many milliseconds before switching between reads and writes for a mixed


@@ -298,9 +345,9 @@ workload. Default: 500ms.
 Percentage of a mixed workload that should be reads. Default: 50.
 .TP
 .BI rwmixwrite \fR=\fPint
 Percentage of a mixed workload that should be reads. Default: 50.
 .TP
 .BI rwmixwrite \fR=\fPint

-Percentage of a mixed workload that would be writes.  If \fBrwmixread\fR and
+Percentage of a mixed workload that should be writes.  If \fBrwmixread\fR and

 \fBwrmixwrite\fR are given and do not sum to 100%, the latter of the two
 \fBwrmixwrite\fR are given and do not sum to 100%, the latter of the two

-overrides the first.
+overrides the first.  Default: 50.

 .TP
 .B norandommap
 Normally \fBfio\fR will cover every block of the file when doing random I/O. If
 .TP
 .B norandommap
 Normally \fBfio\fR will cover every block of the file when doing random I/O. If


@@ -364,12 +411,19 @@ If given, run for the specified \fBruntime\fR duration even if the files are
 completely read or written. The same workload will be repeated as many times
 as \fBruntime\fR allows.
 .TP
 completely read or written. The same workload will be repeated as many times
 as \fBruntime\fR allows.
 .TP

+.BI ramp_time \fR=\fPint
+If set, fio will run the specified workload for this amount of time before
+logging any performance numbers. Useful for letting performance settle before
+logging results, thus minimizing the runtime required for stable results. Note
+that the ramp_time is considered lead in time for a job, thus it will increase
+the total runtime if a special timeout or runtime is specified.
+.TP

 .BI invalidate \fR=\fPbool
 Invalidate buffer-cache for the file prior to starting I/O.  Default: true.
 .TP
 .BI sync \fR=\fPbool
 Use synchronous I/O for buffered writes.  For the majority of I/O engines,
 .BI invalidate \fR=\fPbool
 Invalidate buffer-cache for the file prior to starting I/O.  Default: true.
 .TP
 .BI sync \fR=\fPbool
 Use synchronous I/O for buffered writes.  For the majority of I/O engines,

-this means using O_SYNC.
+this means using O_SYNC.  Default: false.

 .TP
 .BI iomem \fR=\fPstr "\fR,\fP mem" \fR=\fPstr
 Allocation method for I/O unit buffer.  Allowed values are:
 .TP
 .BI iomem \fR=\fPstr "\fR,\fP mem" \fR=\fPstr
 Allocation method for I/O unit buffer.  Allowed values are:


@@ -399,7 +453,7 @@ the system must have free huge pages allocated.  \fBmmaphuge\fR also needs to
 have hugetlbfs mounted, and \fIfile\fR must point there.
 .RE
 .TP
 have hugetlbfs mounted, and \fIfile\fR must point there.
 .RE
 .TP

-.BI hugepage-size \fR=\fPsiint
+.BI hugepage\-size \fR=\fPint

 Defines the size of a huge page.  Must be at least equal to the system setting.
 Should be a multiple of 1MiB. Default: 4MiB.
 .TP
 Defines the size of a huge page.  Must be at least equal to the system setting.
 Should be a multiple of 1MiB. Default: 4MiB.
 .TP


@@ -411,11 +465,14 @@ Average bandwidth calculations over the given time in milliseconds.  Default:
 500ms.
 .TP
 .BI create_serialize \fR=\fPbool
 500ms.
 .TP
 .BI create_serialize \fR=\fPbool

-If true, serialize file creation for the jobs.
+If true, serialize file creation for the jobs.  Default: true.

 .TP
 .BI create_fsync \fR=\fPbool
 \fIfsync\fR\|(2) data file after creation.  Default: true.
 .TP
 .TP
 .BI create_fsync \fR=\fPbool
 \fIfsync\fR\|(2) data file after creation.  Default: true.
 .TP

+.BI create_on_open \fR=\fPbool
+If true, the files are not created until they are opened for IO by the job.
+.TP

 .BI unlink \fR=\fPbool
 Unlink job files when done.  Default: false.
 .TP
 .BI unlink \fR=\fPbool
 Unlink job files when done.  Default: false.
 .TP


@@ -454,11 +511,11 @@ Pretend to verify.  Used for testing internals.
 If true, written verify blocks are sorted if \fBfio\fR deems it to be faster to
 read them back in a sorted manner.  Default: true.
 .TP
 If true, written verify blocks are sorted if \fBfio\fR deems it to be faster to
 read them back in a sorted manner.  Default: true.
 .TP

-.BI verify_offset \fR=\fPsiint
+.BI verify_offset \fR=\fPint

 Swap the verification header with data somewhere else in the block before
 Swap the verification header with data somewhere else in the block before

-writing.  It it swapped back before verifying.
+writing.  It is swapped back before verifying.

 .TP
 .TP

-.BI verify_interval \fR=\fPsiint
+.BI verify_interval \fR=\fPint

 Write the verification header for this number of bytes, which should divide
 \fBblocksize\fR.  Default: \fBblocksize\fR.
 .TP
 Write the verification header for this number of bytes, which should divide
 \fBblocksize\fR.  Default: \fBblocksize\fR.
 .TP


@@ -467,7 +524,7 @@ If true, exit the job on the first observed verification failure.  Default:
 false.
 .TP
 .B stonewall
 false.
 .TP
 .B stonewall

-Wait for precedding jobs in the job file to exit before starting this one.
+Wait for preceeding jobs in the job file to exit before starting this one.

 \fBstonewall\fR implies \fBnew_group\fR.
 .TP
 .B new_group
 \fBstonewall\fR implies \fBnew_group\fR.
 .TP
 .B new_group


@@ -486,11 +543,11 @@ specified.
 Use threads created with \fBpthread_create\fR\|(3) instead of processes created
 with \fBfork\fR\|(2).
 .TP
 Use threads created with \fBpthread_create\fR\|(3) instead of processes created
 with \fBfork\fR\|(2).
 .TP

-.BI zonesize \fR=\fPsiint
+.BI zonesize \fR=\fPint

 Divide file into zones of the specified size in bytes.  See \fBzoneskip\fR.
 .TP
 Divide file into zones of the specified size in bytes.  See \fBzoneskip\fR.
 .TP

-.BI zoneskip \fR=\fPsiint
-Skip the specified number of bytes when \fBzonesize\fR bytes of data has been
+.BI zoneskip \fR=\fPint
+Skip the specified number of bytes when \fBzonesize\fR bytes of data have been

 read.
 .TP
 .BI write_iolog \fR=\fPstr
 read.
 .TP
 .BI write_iolog \fR=\fPstr


@@ -500,13 +557,31 @@ Write the issued I/O patterns to the specified file.
 Replay the I/O patterns contained in the specified file generated by
 \fBwrite_iolog\fR, or may be a \fBblktrace\fR binary file.
 .TP
 Replay the I/O patterns contained in the specified file generated by
 \fBwrite_iolog\fR, or may be a \fBblktrace\fR binary file.
 .TP

-.B write_bw_log
-If given, write bandwidth logs of the jobs in this file.
+.B write_bw_log \fR=\fPstr
+If given, write a bandwidth log of the jobs in this job file. Can be used to
+store data of the bandwidth of the jobs in their lifetime. The included
+fio_generate_plots script uses gnuplot to turn these text files into nice
+graphs. See \fBwrite_log_log\fR for behaviour of given filename. For this
+option, the postfix is _bw.log.

 .TP
 .B write_lat_log
 .TP
 .B write_lat_log

-Same as \fBwrite_bw_log\fR, but writes I/O completion latencies.
+Same as \fBwrite_bw_log\fR, but writes I/O completion latencies.  If no
+filename is given with this option, the default filename of "jobname_type.log"
+is used. Even if the filename is given, fio will still append the type of log.
+.TP
+.B disable_clat \fR=\fPbool
+Disable measurements of completion latency numbers. Useful only for cutting
+back the number of calls to gettimeofday, as that does impact performance at
+really high IOPS rates.  Note that to really get rid of a large amount of these
+calls, this option must be used with disable_slat and disable_bw as well.

 .TP
 .TP

-.BI lockmem \fR=\fPsiint
+.B disable_slat \fR=\fPbool
+Disable measurements of submission latency numbers. See \fBdisable_clat\fR.
+.TP
+.B disable_bw_measurement \fR=\fPbool
+Disable measurements of throughput/bandwidth numbers. See \fBdisable_clat\fR.
+.TP
+.BI lockmem \fR=\fPint

 Pin the specified amount of memory with \fBmlock\fR\|(2).  Can be used to
 simulate a smaller amount of memory.
 .TP
 Pin the specified amount of memory with \fBmlock\fR\|(2).  Can be used to
 simulate a smaller amount of memory.
 .TP


@@ -528,16 +603,35 @@ If the job is a CPU cycle-eater, split the load into cycles of the
 given time in milliseconds.
 .TP
 .BI disk_util \fR=\fPbool
 given time in milliseconds.
 .TP
 .BI disk_util \fR=\fPbool

-Generate disk utilization statistics if the platform supports it.  Default: true.
+Generate disk utilization statistics if the platform supports it. Default: true.
+.TP
+.BI gtod_reduce \fR=\fPbool
+Enable all of the gettimeofday() reducing options (disable_clat, disable_slat,
+disable_bw) plus reduce precision of the timeout somewhat to really shrink the
+gettimeofday() call count. With this option enabled, we only do about 0.4% of
+the gtod() calls we would have done if all time keeping was enabled.
+.TP
+.BI gtod_cpu \fR=\fPint
+Sometimes it's cheaper to dedicate a single thread of execution to just getting
+the current time. Fio (and databases, for instance) are very intensive on
+gettimeofday() calls. With this option, you can set one CPU aside for doing
+nothing but logging current time to a shared memory location. Then the other
+threads/processes that run IO workloads need only copy that segment, instead of
+entering the kernel with a gettimeofday() call. The CPU set aside for doing
+these time calls will be excluded from other uses. Fio will manually clear it
+from the CPU mask of other jobs.

 .SH OUTPUT
 .SH OUTPUT

-While running, \fBfio\fR will display the status of the created jobs.  For example:
+While running, \fBfio\fR will display the status of the created jobs.  For
+example:

 .RS
 .RS

+.P

 Threads: 1: [_r] [24.8% done] [ 13509/  8334 kb/s] [eta 00h:01m:31s]
 .RE
 .P
 Threads: 1: [_r] [24.8% done] [ 13509/  8334 kb/s] [eta 00h:01m:31s]
 .RE
 .P

-The characters in the first set of brackets denote the current status of each threads.
-The possible values are:
-.RS
+The characters in the first set of brackets denote the current status of each
+threads.  The possible values are:
+.P
+.PD 0

 .RS
 .TP
 .B P
 .RS
 .TP
 .B P


@@ -579,7 +673,7 @@ Exited, not reaped by main thread.
 .B \-
 Exited, thread reaped.
 .RE
 .B \-
 Exited, thread reaped.
 .RE

-.RE
+.PD

 .P
 The second set of brackets shows the estimated completion percentage of
 the current group.  The third set shows the read and write I/O rate,
 .P
 The second set of brackets shows the estimated completion percentage of
 the current group.  The third set shows the read and write I/O rate,


@@ -591,7 +685,6 @@ for each thread, each group of threads, and each disk, in that order.
 Per-thread statistics first show the threads client number, group-id, and
 error code.  The remaining figures are as follows:
 .RS
 Per-thread statistics first show the threads client number, group-id, and
 error code.  The remaining figures are as follows:
 .RS

-.RS

 .TP
 .B io
 Number of megabytes of I/O performed.
 .TP
 .B io
 Number of megabytes of I/O performed.


@@ -629,10 +722,9 @@ Number of read/write requests issued, and number of short read/write requests.
 Distribution of I/O completion latencies.  The numbers follow the same pattern
 as \fBIO depths\fR.
 .RE
 Distribution of I/O completion latencies.  The numbers follow the same pattern
 as \fBIO depths\fR.
 .RE

-.RE

 .P
 The group statistics show:
 .P
 The group statistics show:

-.RS
+.PD 0

 .RS
 .TP
 .B io
 .RS
 .TP
 .B io


@@ -648,15 +740,15 @@ Minimum average bandwidth a thread saw.
 Maximum average bandwidth a thread saw.
 .TP
 .B mint
 Maximum average bandwidth a thread saw.
 .TP
 .B mint

-Smallest runtime of threads in the group.
+Shortest runtime of threads in the group.

 .TP
 .B maxt
 Longest runtime of threads in the group.
 .RE
 .TP
 .B maxt
 Longest runtime of threads in the group.
 .RE

-.RE
+.PD

 .P
 Finally, disk statistics are printed with reads first:
 .P
 Finally, disk statistics are printed with reads first:

-.RS
+.PD 0

 .RS
 .TP
 .B ios
 .RS
 .TP
 .B ios


@@ -674,7 +766,7 @@ Total time spent in the disk queue.
 .B util
 Disk utilization.
 .RE
 .B util
 Disk utilization.
 .RE

-.RE
+.PD

 .SH TERSE OUTPUT
 If the \fB\-\-minimal\fR option is given, the results will be printed in a
 semicolon-delimited format suitable for scripted use.  The fields are:
 .SH TERSE OUTPUT
 If the \fB\-\-minimal\fR option is given, the results will be printed in a
 semicolon-delimited format suitable for scripted use.  The fields are:


@@ -718,9 +810,9 @@ Bandwidth:
 .RE
 .RE
 .P
 .RE
 .RE
 .P

-CPU usage;
+CPU usage:

 .RS
 .RS

-.B user, system, context switches
+.B user, system, context switches, major page faults, minor page faults

 .RE
 .P
 IO depth distribution:
 .RE
 .P
 IO depth distribution:


@@ -738,12 +830,14 @@ IO latency distribution (ms):
 .SH AUTHORS
 .B fio
 was written by Jens Axboe <jens.axboe@oracle.com>.
 .SH AUTHORS
 .B fio
 was written by Jens Axboe <jens.axboe@oracle.com>.

-This man page was
-written by Aaron Carroll <aaronc@cse.unsw.edu.au> based
+.br
+This man page was written by Aaron Carroll <aaronc@cse.unsw.edu.au> based

 on documentation by Jens Axboe.
 .SH "REPORTING BUGS"
 on documentation by Jens Axboe.
 .SH "REPORTING BUGS"

-Report bugs to the \fBfio\fR mailing list <fio-devel@kernel.dk>.  See \fBREADME\fR.
+Report bugs to the \fBfio\fR mailing list <fio-devel@kernel.dk>.
+See \fBREADME\fR.

 .SH "SEE ALSO"
 .SH "SEE ALSO"

-Further documentation is available in \fBfio\fR's \fBHOWTO\fR and \fBREADME\fR.
-Sample jobfiles are available in \fBfio\fR's \fBexamples/\fR directory.
+For further documentation see \fBHOWTO\fR and \fBREADME\fR.
+.br
+Sample jobfiles are available in the \fBexamples\fR directory.