.. option:: --debug=type
- Enable verbose tracing of various fio actions. May be ``all`` for all types
- or individual types separated by a comma (e.g. ``--debug=file,mem`` will
- enable file and memory debugging). Currently, additional logging is
- available for:
+ Enable verbose tracing `type` of various fio actions. May be ``all`` for all types
+ or individual types separated by a comma (e.g. ``--debug=file,mem`` will
+ enable file and memory debugging). Currently, additional logging is
+ available for:
- *process*
+ *process*
Dump info related to processes.
- *file*
+ *file*
Dump info related to file actions.
- *io*
+ *io*
Dump info related to I/O queuing.
- *mem*
+ *mem*
Dump info related to memory allocations.
- *blktrace*
+ *blktrace*
Dump info related to blktrace setup.
- *verify*
+ *verify*
Dump info related to I/O verification.
- *all*
+ *all*
Enable all debug options.
- *random*
+ *random*
Dump info related to random offset generation.
- *parse*
+ *parse*
Dump info related to option matching and parsing.
- *diskutil*
+ *diskutil*
Dump info related to disk utilization updates.
- *job:x*
+ *job:x*
Dump info only related to job number x.
- *mutex*
+ *mutex*
Dump info only related to mutex up/down ops.
- *profile*
+ *profile*
Dump info related to profile extensions.
- *time*
+ *time*
Dump info related to internal time keeping.
- *net*
+ *net*
Dump info related to networking connections.
- *rate*
+ *rate*
Dump info related to I/O rate switching.
- *compress*
+ *compress*
Dump info related to log compress/decompress.
- *?* or *help*
+ *?* or *help*
Show available debug options.
.. option:: --parse-only
- Parse options only, don\'t start any I/O.
+ Parse options only, don't start any I/O.
.. option:: --output=filename
Write output to file `filename`.
+.. option:: --output-format=format
+
+ Set the reporting `format` to `normal`, `terse`, `json`, or `json+`. Multiple
+ formats can be selected, separated by a comma. `terse` is a CSV based
+ format. `json+` is like `json`, except it adds a full dump of the latency
+ buckets.
+
.. option:: --bandwidth-log
Generate aggregate bandwidth logs.
.. option:: --append-terse
- Print statistics in selected mode AND terse, semicolon-delimited format.
- **deprecated**, use :option:`--output-format` instead to select multiple
- formats.
+ Print statistics in selected mode AND terse, semicolon-delimited format.
+ **Deprecated**, use :option:`--output-format` instead to select multiple
+ formats.
-.. option:: --output-format=type
-
- Set the reporting format to `normal`, `terse`, `json`, or `json+`. Multiple
- formats can be selected, separated by a comma. `terse` is a CSV based
- format. `json+` is like `json`, except it adds a full dump of the latency
- buckets.
+.. option:: --terse-version=version
-.. option:: --terse-version=type
-
- Set terse version output format (default 3, or 2 or 4 or 5).
+ Set terse `version` output format (default 3, or 2 or 4 or 5).
.. option:: --version
- Print version info and exit.
+ Print version information and exit.
.. option:: --help
.. option:: --crctest=[test]
- Test the speed of the built-in checksumming functions. If no argument is
- given all of them are tested. Alternatively, a comma separated list can be passed, in
- which case the given ones are tested.
+ Test the speed of the built-in checksumming functions. If no argument is
+ given, all of them are tested. Alternatively, a comma separated list can
+ be passed, in which case the given ones are tested.
.. option:: --cmdhelp=command
.. option:: --enghelp=[ioengine[,command]]
- List all commands defined by :option:`ioengine`, or print help for `command`
- defined by :option:`ioengine`. If no :option:`ioengine` is given, list all
- available ioengines.
+ List all commands defined by `ioengine`, or print help for `command`
+ defined by `ioengine`. If no `ioengine` is given, list all
+ available ioengines.
.. option:: --showcmd=jobfile
- Turn a job file into command line options.
+ Convert `jobfile` to a set of command-line options.
.. option:: --readonly
- Turn on safety read-only checks, preventing writes. The ``--readonly``
- option is an extra safety guard to prevent users from accidentally starting
- a write workload when that is not desired. Fio will only write if
- `rw=write/randwrite/rw/randrw` is given. This extra safety net can be used
- as an extra precaution as ``--readonly`` will also enable a write check in
- the I/O engine core to prevent writes due to unknown user space bug(s).
+ Turn on safety read-only checks, preventing writes. The ``--readonly``
+ option is an extra safety guard to prevent users from accidentally starting
+ a write workload when that is not desired. Fio will only write if
+ `rw=write/randwrite/rw/randrw` is given. This extra safety net can be used
+ as an extra precaution as ``--readonly`` will also enable a write check in
+ the I/O engine core to prevent writes due to unknown user space bug(s).
.. option:: --eta=when
- When real-time ETA estimate should be printed. May be `always`, `never` or
- `auto`.
+ Specifies when real-time ETA estimate should be printed. `when` may be
+ `always`, `never` or `auto`.
.. option:: --eta-newline=time
.. option:: --status-interval=time
- Force full status dump every `time` period passed. When the unit is
- omitted, the value is interpreted in seconds.
+ Force a full status dump of cumulative (from job start) values at `time`
+ intervals. This option does *not* provide per-period measurements. So
+ values such as bandwidth are running averages. When the time unit is omitted,
+ `time` is interpreted in seconds.
.. option:: --section=name
- Only run specified section in job file. Multiple sections can be specified.
- The ``--section`` option allows one to combine related jobs into one file.
- E.g. one job file could define light, moderate, and heavy sections. Tell
- fio to run only the "heavy" section by giving ``--section=heavy``
- command line option. One can also specify the "write" operations in one
- section and "verify" operation in another section. The ``--section`` option
- only applies to job sections. The reserved *global* section is always
- parsed and used.
+ Only run specified section `name` in job file. Multiple sections can be specified.
+ The ``--section`` option allows one to combine related jobs into one file.
+ E.g. one job file could define light, moderate, and heavy sections. Tell
+ fio to run only the "heavy" section by giving ``--section=heavy``
+ command line option. One can also specify the "write" operations in one
+ section and "verify" operation in another section. The ``--section`` option
+ only applies to job sections. The reserved *global* section is always
+ parsed and used.
.. option:: --alloc-size=kb
- Set the internal smalloc pool to this size in KiB. The
- ``--alloc-size`` switch allows one to use a larger pool size for smalloc.
- If running large jobs with randommap enabled, fio can run out of memory.
- Smalloc is an internal allocator for shared structures from a fixed size
- memory pool and can grow to 16 pools. The pool size defaults to 16MiB.
+ Set the internal smalloc pool size to `kb` in KiB. The
+ ``--alloc-size`` switch allows one to use a larger pool size for smalloc.
+ If running large jobs with randommap enabled, fio can run out of memory.
+ Smalloc is an internal allocator for shared structures from a fixed size
+ memory pool and can grow to 16 pools. The pool size defaults to 16MiB.
- NOTE: While running :file:`.fio_smalloc.*` backing store files are visible
- in :file:`/tmp`.
+ NOTE: While running :file:`.fio_smalloc.*` backing store files are visible
+ in :file:`/tmp`.
.. option:: --warnings-fatal
- All fio parser warnings are fatal, causing fio to exit with an
- error.
+ All fio parser warnings are fatal, causing fio to exit with an
+ error.
.. option:: --max-jobs=nr
- Maximum number of threads/processes to support.
+ Set the maximum number of threads/processes to support to `nr`.
.. option:: --server=args
- Start a backend server, with `args` specifying what to listen to.
- See `Client/Server`_ section.
+ Start a backend server, with `args` specifying what to listen to.
+ See `Client/Server`_ section.
.. option:: --daemonize=pidfile
- Background a fio server, writing the pid to the given `pidfile` file.
+ Background a fio server, writing the pid to the given `pidfile` file.
.. option:: --client=hostname
- Instead of running the jobs locally, send and run them on the given host or
- set of hosts. See `Client/Server`_ section.
+ Instead of running the jobs locally, send and run them on the given `hostname`
+ or set of `hostname`s. See `Client/Server`_ section.
.. option:: --remote-config=file
- Tell fio server to load this local file.
+ Tell fio server to load this local `file`.
.. option:: --idle-prof=option
- Report CPU idleness. *option* is one of the following:
+ Report CPU idleness. `option` is one of the following:
**calibrate**
Run unit work calibration only and exit.
.. option:: --inflate-log=log
- Inflate and output compressed log.
+ Inflate and output compressed `log`.
.. option:: --trigger-file=file
- Execute trigger cmd when file exists.
+ Execute trigger command when `file` exists.
-.. option:: --trigger-timeout=t
+.. option:: --trigger-timeout=time
- Execute trigger at this time.
+ Execute trigger at this `time`.
-.. option:: --trigger=cmd
+.. option:: --trigger=command
- Set this command as local trigger.
+ Set this `command` as local trigger.
-.. option:: --trigger-remote=cmd
+.. option:: --trigger-remote=command
- Set this command as remote trigger.
+ Set this `command` as remote trigger.
.. option:: --aux-path=path
- Use this path for fio state generated files.
+ Use this `path` for fio state generated files.
Any parameters following the options will be assumed to be job files, unless
they match a job file parameter. Multiple job files can be listed and each job
*global* sections if so desired. A job is only affected by a *global* section
residing above it.
-The :option:`--cmdhelp` option also lists all options. If used with an `option`
-argument, :option:`--cmdhelp` will detail the given `option`.
+The :option:`--cmdhelp` option also lists all options. If used with a `command`
+argument, :option:`--cmdhelp` will detail the given `command`.
See the `examples/` directory for inspiration on how to write job files. Note
the copyright and license requirements currently apply to `examples/` files.
run. Simple math is also supported on these keywords, so you can perform actions
like::
- size=8*$mb_memory
+ size=8*$mb_memory
and get that properly expanded to 8 times the size of memory in the machine.
~~~~~~~~~~~~~~~
**str**
- String. This is a sequence of alpha characters.
+ String: A sequence of alphanumeric characters.
**time**
Integer with possible time suffix. Without a unit value is interpreted as
Integer. A whole number value, which may contain an integer prefix
and an integer suffix:
- [*integer prefix*] **number** [*integer suffix*]
+ [*integer prefix*] **number** [*integer suffix*]
The optional *integer prefix* specifies the number's base. The default
is decimal. *0x* specifies hexadecimal.
prefixes. To specify power-of-10 decimal values defined in the
International System of Units (SI):
- * *Ki* -- means kilo (K) or 1000
- * *Mi* -- means mega (M) or 1000**2
- * *Gi* -- means giga (G) or 1000**3
- * *Ti* -- means tera (T) or 1000**4
- * *Pi* -- means peta (P) or 1000**5
+ * *K* -- means kilo (K) or 1000
+ * *M* -- means mega (M) or 1000**2
+ * *G* -- means giga (G) or 1000**3
+ * *T* -- means tera (T) or 1000**4
+ * *P* -- means peta (P) or 1000**5
To specify power-of-2 binary values defined in IEC 80000-13:
- * *k* -- means kibi (Ki) or 1024
- * *M* -- means mebi (Mi) or 1024**2
- * *G* -- means gibi (Gi) or 1024**3
- * *T* -- means tebi (Ti) or 1024**4
- * *P* -- means pebi (Pi) or 1024**5
+ * *Ki* -- means kibi (Ki) or 1024
+ * *Mi* -- means mebi (Mi) or 1024**2
+ * *Gi* -- means gibi (Gi) or 1024**3
+ * *Ti* -- means tebi (Ti) or 1024**4
+ * *Pi* -- means pebi (Pi) or 1024**5
With :option:`kb_base`\=1024 (the default), the unit prefixes are opposite
from those specified in the SI and IEC 80000-13 standards to provide
compatibility with old scripts. For example, 4k means 4096.
For quantities of data, an optional unit of 'B' may be included
- (e.g., 'kB' is the same as 'k').
+ (e.g., 'kB' is the same as 'k').
The *integer suffix* is not case sensitive (e.g., m/mi mean mebi/mega,
not milli). 'b' and 'B' both mean byte, not bit.
**float_list**
A list of floating point numbers, separated by a ':' character.
+With the above in mind, here follows the complete list of fio job parameters.
+
Units
~~~~~
Bit based.
-With the above in mind, here follows the complete list of fio job parameters.
-
-
Job description
~~~~~~~~~~~~~~~
**sequential**
Finish one file before moving on to the next. Multiple files can
- still be open depending on 'openfiles'.
+ still be open depending on :option:`openfiles`.
**zipf**
Use a *Zipf* distribution to decide what file to access.
**pareto**
Use a *Pareto* distribution to decide what file to access.
- **gauss**
+ **normal**
Use a *Gaussian* (normal) distribution to decide what file to
access.
+ **gauss**
+ Alias for normal.
+
For *random*, *roundrobin*, and *sequential*, a postfix can be appended to
tell fio how many I/Os to issue before switching to a new file. For example,
specifying ``file_service_type=random:8`` would cause fio to issue
.. option:: direct=bool
If value is true, use non-buffered I/O. This is usually O_DIRECT. Note that
- ZFS on Solaris doesn't support direct I/O. On Windows the synchronous
+ OpenBSD and ZFS on Solaris don't support direct I/O. On Windows the synchronous
ioengines don't support direct I/O. Default: false.
.. option:: atomic=bool
``sequential`` is only useful for random I/O, where fio would normally
generate a new random offset for every I/O. If you append e.g. 8 to randread,
- you would get a new random offset for every 8 I/O's. The result would be a
- seek for only every 8 I/O's, instead of for every I/O. Use ``rw=randread:8``
+ you would get a new random offset for every 8 I/Os. The result would be a
+ seek for only every 8 I/Os, instead of for every I/O. Use ``rw=randread:8``
to specify that. As sequential I/O is already sequential, setting
``sequential`` for that would not result in any differences. ``identical``
behaves in a similar fashion, except it sends the same offset 8 number of
**none**
Do not pre-allocate space.
+ **native**
+ Use a platform's native pre-allocation call but fall back to
+ **none** behavior if it fails/is not implemented.
+
**posix**
Pre-allocate via :manpage:`posix_fallocate(3)`.
Backward-compatible alias for **posix**.
May not be available on all supported platforms. **keep** is only available
- on Linux. If using ZFS on Solaris this must be set to **none** because ZFS
- doesn't support it. Default: **posix**.
+ on Linux. If using ZFS on Solaris this cannot be set to **posix**
+ because ZFS doesn't support pre-allocation. Default: **native** if any
+ pre-allocation methods are available, **none** if not.
.. option:: fadvise_hint=str
**random**
Advise using **FADV_RANDOM**.
-.. option:: fadvise_stream=int
+.. option:: write_hint=str
+
+ Use :manpage:`fcntl(2)` to advise the kernel what life time to expect
+ from a write. Only supported on Linux, as of version 4.13. Accepted
+ values are:
+
+ **none**
+ No particular life time associated with this file.
+
+ **short**
+ Data written to this file has a short life time.
+
+ **medium**
+ Data written to this file has a medium life time.
+
+ **long**
+ Data written to this file has a long life time.
- Use :manpage:`posix_fadvise(2)` to advise the kernel what stream ID the
- writes issued belong to. Only supported on Linux. Note, this option may
- change going forward.
+ **extreme**
+ Data written to this file has a very long life time.
+
+ The values are all relative to each other, and no absolute meaning
+ should be associated with them.
.. option:: offset=int
- Start I/O at the provided offset in the file, given as either a fixed size or
- a percentage. If a percentage is given, the next ``blockalign``-ed offset
- will be used. Data before the given offset will not be touched. This
+ Start I/O at the provided offset in the file, given as either a fixed size in
+ bytes or a percentage. If a percentage is given, the next ``blockalign``-ed
+ offset will be used. Data before the given offset will not be touched. This
effectively caps the file size at `real_size - offset`. Can be combined with
:option:`size` to constrain the start and end range of the I/O workload.
+ A percentage can be specified by a number between 1 and 100 followed by '%',
+ for example, ``offset=20%`` to specify 20%.
.. option:: offset_increment=int
Make every `N-th` write a barrier write.
-.. option:: sync_file_range=str:val
+.. option:: sync_file_range=str:int
- Use :manpage:`sync_file_range(2)` for every `val` number of write
+ Use :manpage:`sync_file_range(2)` for every `int` number of write
operations. Fio will track range of writes that have happened since the last
:manpage:`sync_file_range(2)` call. `str` can currently be one or more of:
**pareto**
Pareto distribution
- **gauss**
+ **normal**
Normal (Gaussian) distribution
**zoned**
Zoned random distribution
When using a **zipf** or **pareto** distribution, an input value is also
- needed to define the access pattern. For **zipf**, this is the `zipf
+ needed to define the access pattern. For **zipf**, this is the `Zipf
theta`. For **pareto**, it's the `Pareto power`. Fio includes a test
- program, :command:`genzipf`, that can be used visualize what the given input
+ program, :command:`fio-genzipf`, that can be used visualize what the given input
values will yield in terms of hit rates. If you wanted to use **zipf** with
a `theta` of 1.2, you would use ``random_distribution=zipf:1.2`` as the
option. If a non-uniform model is used, fio will disable use of the random
- map. For the **gauss** distribution, a normal deviation is supplied as a
- value between 0 and 100.
+ map. For the **normal** distribution, a normal (Gaussian) deviation is
+ supplied as a value between 0 and 100.
For a **zoned** distribution, fio supports specifying percentages of I/O
access that should fall within what range of the file or device. For
example, given a criteria of:
- * 60% of accesses should be to the first 10%
- * 30% of accesses should be to the next 20%
- * 8% of accesses should be to to the next 30%
- * 2% of accesses should be to the next 40%
+ * 60% of accesses should be to the first 10%
+ * 30% of accesses should be to the next 20%
+ * 8% of accesses should be to the next 30%
+ * 2% of accesses should be to the next 40%
we can define that through zoning of the random accesses. For the above
example, the user would do::
.. option:: random_generator=str
- Fio supports the following engines for generating
- I/O offsets for random I/O:
+ Fio supports the following engines for generating I/O offsets for random I/O:
**tausworthe**
- Strong 2^88 cycle random number generator
+ Strong 2^88 cycle random number generator.
**lfsr**
- Linear feedback shift register generator
+ Linear feedback shift register generator.
**tausworthe64**
- Strong 64-bit 2^258 cycle random number generator
+ Strong 64-bit 2^258 cycle random number generator.
**tausworthe** is a strong random number generator, but it requires tracking
on the side if we want to ensure that blocks are only read or written
- once. **LFSR** guarantees that we never generate the same offset twice, and
+ once. **lfsr** guarantees that we never generate the same offset twice, and
it's also less computationally expensive. It's not a true random generator,
- however, though for I/O purposes it's typically good enough. **LFSR** only
+ however, though for I/O purposes it's typically good enough. **lfsr** only
works with single block sizes, not with workloads that use multiple block
sizes. If used with such a workload, fio may read or write some blocks
multiple times. The default value is **tausworthe**, unless the required
typically won't work with direct I/O, as that normally requires sector
alignment.
-.. option:: bs_is_seq_rand
+.. option:: bs_is_seq_rand=bool
If this option is set, fio will use the normal read,write blocksize settings
as sequential,random blocksize settings instead. Any random read or write
**cudamalloc**
Use GPU memory as the buffers for GPUDirect RDMA benchmark.
+ The :option:`ioengine` must be `rdma`.
The area allocated is a function of the maximum allowed bs size for the job,
multiplied by the I/O depth given. Note that for **shmhuge** and
should point there. So if it's mounted in :file:`/huge`, you would use
`mem=mmaphuge:/huge/somefile`.
-.. option:: iomem_align=int
+.. option:: iomem_align=int, mem_align=int
This indicates the memory alignment of the I/O memory buffers. Note that
the given alignment is applied to the first I/O unit buffer, if using
SCSI generic sg v3 I/O. May either be synchronous using the SG_IO
ioctl, or if the target is an sg character device we use
:manpage:`read(2)` and :manpage:`write(2)` for asynchronous
- I/O. Requires filename option to specify either block or character
- devices.
+ I/O. Requires :option:`filename` option to specify either block or
+ character devices.
**null**
Doesn't transfer any data, just pretends to. This is mainly used to
Doesn't transfer any data, but burns CPU cycles according to the
:option:`cpuload` and :option:`cpuchunks` options. Setting
:option:`cpuload`\=85 will cause that job to do nothing but burn 85%
- of the CPU. In case of SMP machines, use :option:`numjobs`
- =<no_of_cpu> to get desired CPU usage, as the cpuload only loads a
+ of the CPU. In case of SMP machines, use :option:`numjobs`=<nr_of_cpu>
+ to get desired CPU usage, as the cpuload only loads a
single CPU at the desired rate. A job never finishes unless there is
at least one non-cpuio job.
**ftruncate**
I/O engine that sends :manpage:`ftruncate(2)` operations in response
to write (DDIR_WRITE) events. Each ftruncate issued sets the file's
- size to the current block offset. Block size is ignored.
+ size to the current block offset. :option:`blocksize` is ignored.
**e4defrag**
I/O engine that does regular EXT4_IOC_MOVE_EXT ioctls to simulate
defines engine specific options.
**libhdfs**
- Read and write through Hadoop (HDFS). The :file:`filename` option
+ Read and write through Hadoop (HDFS). The :option:`filename` option
is used to specify host,port of the hdfs name-node to connect. This
engine interprets offsets a little differently. In HDFS, files once
created cannot be modified so random writes are not possible. To
**external**
Prefix to specify loading an external I/O engine object file. Append
the engine filename, e.g. ``ioengine=external:/tmp/foo.o`` to load
- ioengine :file:`foo.o` in :file:`/tmp`.
+ ioengine :file:`foo.o` in :file:`/tmp`. The path can be either
+ absolute or relative. See :file:`engines/skeleton_external.c` for
+ details of writing an external I/O engine.
I/O engine specific parameters
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In addition, there are some parameters which are only valid when a specific
-ioengine is in use. These are used identically to normal parameters, with the
-caveat that when used on the command line, they must come after the
+:option:`ioengine` is in use. These are used identically to normal parameters,
+with the caveat that when used on the command line, they must come after the
:option:`ioengine` that defines them is selected.
.. option:: userspace_reap : [libaio]
Set RWF_HIPRI on I/O, indicating to the kernel that it's of higher priority
than normal.
+.. option:: hipri_percentage : [pvsync2]
+
+ When hipri is set this determines the probability of a pvsync2 I/O being high
+ priority. The default is 100%.
+
.. option:: cpuload=int : [cpuio]
Attempt to use the specified percentage of CPU cycles. This is a mandatory
Detect when I/O threads are done, then exit.
-.. option:: hostname=str : [netsplice] [net]
-
- The hostname or IP address to use for TCP or UDP based I/O. If the job is
- a TCP listener or UDP reader, the hostname is not used and must be omitted
- unless it is a valid UDP multicast address.
-
.. option:: namenode=str : [libhdfs]
The hostname or IP address of a HDFS cluster namenode to contact.
.. option:: port=int
+ [libhdfs]
+
+ The listening port of the HFDS cluster namenode.
+
[netsplice], [net]
The TCP or UDP port to bind to or connect to. If this is used with
this will be the starting port number since fio will use a range of
ports.
- [libhdfs]
+.. option:: hostname=str : [netsplice] [net]
- the listening port of the HFDS cluster namenode.
+ The hostname or IP address to use for TCP or UDP based I/O. If the job is
+ a TCP listener or UDP reader, the hostname is not used and must be omitted
+ unless it is a valid UDP multicast address.
.. option:: interface=str : [netsplice] [net]
Set TCP_NODELAY on TCP connections.
-.. option:: protocol=str : [netsplice] [net]
-
-.. option:: proto=str : [netsplice] [net]
+.. option:: protocol=str, proto=str : [netsplice] [net]
The network protocol to use. Accepted values are:
When the protocol is TCP or UDP, the port must also be given, as well as the
hostname if the job is a TCP listener or UDP reader. For unix sockets, the
- normal filename option should be used and the port is invalid.
+ normal :option:`filename` option should be used and the port is invalid.
-.. option:: listen : [net]
+.. option:: listen : [netsplice] [net]
For TCP network connections, tell fio to listen for incoming connections
rather than initiating an outgoing connection. The :option:`hostname` must
be omitted if this option is used.
-.. option:: pingpong : [net]
+.. option:: pingpong : [netsplice] [net]
Normally a network writer will just continue writing data, and a network
reader will just consume packages. If ``pingpong=1`` is set, a writer will
``pingpong=1`` should only be set for a single reader when multiple readers
are listening to the same address.
-.. option:: window_size : [net]
+.. option:: window_size : [netsplice] [net]
Set the desired socket buffer size for the connection.
-.. option:: mss : [net]
+.. option:: mss : [netsplice] [net]
Set the TCP maximum segment size (TCP_MAXSEG).
**0**
Default. Preallocate donor's file on init.
**1**
- Allocate space immediately inside defragment event, and free right
+ Allocate space immediately inside defragment event, and free right
after event.
.. option:: clustername=str : [rbd]
.. option:: chunk_size : [libhdfs]
- the size of the chunk to use for each file.
+ The size of the chunk to use for each file.
I/O depth
16 requests, it will let the depth drain down to 4 before starting to fill
it again.
+.. option:: serialize_overlap=bool
+
+ Serialize in-flight I/Os that might otherwise cause or suffer from data races.
+ When two or more I/Os are submitted simultaneously, there is no guarantee that
+ the I/Os will be processed or completed in the submitted order. Further, if
+ two or more of those I/Os are writes, any overlapping region between them can
+ become indeterminate/undefined on certain storage. These issues can cause
+ verification to fail erratically when at least one of the racing I/Os is
+ changing data and the overlapping region has a non-zero size. Setting
+ ``serialize_overlap`` tells fio to avoid provoking this behavior by explicitly
+ serializing in-flight I/Os that have a non-zero overlap. Note that setting
+ this option can reduce both performance and the `:option:iodepth` achieved.
+ Additionally this option does not work when :option:`io_submit_mode` is set to
+ offload. Default: false.
+
.. option:: io_submit_mode=str
This option controls how fio submits the I/O to the I/O engine. The default
.. option:: thinktime_blocks=int
Only valid if :option:`thinktime` is set - control how many blocks to issue,
- before waiting `thinktime` usecs. If not set, defaults to 1 which will make
- fio wait `thinktime` usecs after every block. This effectively makes any
+ before waiting :option:`thinktime` usecs. If not set, defaults to 1 which will make
+ fio wait :option:`thinktime` usecs after every block. This effectively makes any
queue depth setting redundant, since no more than 1 I/O will be queued
- before we have to complete it and do our thinktime. In other words, this
+ before we have to complete it and do our :option:`thinktime`. In other words, this
setting effectively caps the queue depth if the latter is larger.
.. option:: rate=int[,int][,int]
replay, the file needs to be turned into a blkparse binary data file first
(``blkparse <device> -o /dev/null -d file_for_fio.bin``).
-.. option:: replay_no_stall=int
+.. option:: replay_no_stall=bool
When replaying I/O with :option:`read_iolog` the default behavior is to
attempt to respect the timestamps within the log and replay them with the
.. option:: verifysort_nr=int
- Pre-load and sort verify blocks for a read workload.
+ Pre-load and sort verify blocks for a read workload.
.. option:: verify_offset=int
state is loaded for the verify read phase. The format of the filename is,
roughly::
- <type>-<jobname>-<jobindex>-verify.state.
+ <type>-<jobname>-<jobindex>-verify.state.
<type> is "local" for a local run, "sock" for a client/server socket
connection, and "ip" (192.168.0.1, for instance) for a networked
.. option:: trim_backlog=int
- Verify that trim/discarded blocks are returned as zeros.
+ Trim after this number of blocks are written.
.. option:: trim_backlog_batch=int
Enable experimental verification.
-
Steady state
~~~~~~~~~~~~
all jobs in a file will be part of the same reporting group, unless
separated by a :option:`stonewall`.
-.. option:: stats
+.. option:: stats=bool
By default, fio collects and shows final output results for all jobs
that run. If this option is set to 0, then fio will ignore it in
write_lat_log=foo
The actual log names will be :file:`foo_slat.x.log`, :file:`foo_clat.x.log`,
- and :file:`foo_lat.x.log`, where `x` is the index of the job (1..N, where N
- is the number of jobs). This helps :command:`fio_generate_plot` find the
+ and :file:`foo_lat.x.log`, where `x` is the index of the job (`1..N`, where `N`
+ is the number of jobs). This helps :command:`fio_generate_plots` find the
logs automatically. If :option:`per_job_logs` is false, then the filename
will not include the job index. See `Log File Formats`_.
Same as :option:`write_lat_log`, but writes I/O completion latency
histograms. If no filename is given with this option, the default filename
of :file:`jobname_clat_hist.x.log` is used, where `x` is the index of the
- job (1..N, where `N` is the number of jobs). Even if the filename is given,
+ job (`1..N`, where `N` is the number of jobs). Even if the filename is given,
fio will still append the type of log. If :option:`per_job_logs` is false,
then the filename will not include the job index. See `Log File Formats`_.
Same as :option:`write_bw_log`, but writes IOPS. If no filename is given
with this option, the default filename of :file:`jobname_type.x.log` is
- used,where `x` is the index of the job (1..N, where `N` is the number of
+ used, where `x` is the index of the job (`1..N`, where `N` is the number of
jobs). Even if the filename is given, fio will still append the type of
log. If :option:`per_job_logs` is false, then the filename will not include
the job index. See `Log File Formats`_.
you instead want to log the maximum value, set this option to 1. Defaults to
0, meaning that averaged values are logged.
-.. option:: log_offset=int
+.. option:: log_offset=bool
If this is set, the iolog options will include the byte offset for the I/O
- entry as well as the other data values.
+ entry as well as the other data values. Defaults to 0 meaning that
+ offsets are not present in logs. Also see `Log File Formats`_.
.. option:: log_compression=int
.. option:: disable_slat=bool
Disable measurements of submission latency numbers. See
- :option:`disable_slat`.
+ :option:`disable_lat`.
.. option:: disable_bw_measurement=bool, disable_bw=bool
.. option:: clat_percentiles=bool
- Enable the reporting of percentiles of completion latencies.
+ Enable the reporting of percentiles of completion latencies. This
+ option is mutually exclusive with :option:`lat_percentiles`.
+
+.. option:: lat_percentiles=bool
+
+ Enable the reporting of percentiles of IO latencies. This is similar
+ to :option:`clat_percentiles`, except that this includes the
+ submission latency. This option is mutually exclusive with
+ :option:`clat_percentiles`.
.. option:: percentile_list=float_list
To view a profile's additional options use :option:`--cmdhelp` after specifying
the profile. For example::
-$ fio --profile=act --cmdhelp
+ $ fio --profile=act --cmdhelp
Act profile options
~~~~~~~~~~~~~~~~~~~
.. option:: threads-per-queue=int
:noindex:
- Number of read IO threads per device. Default: 8.
+ Number of read I/O threads per device. Default: 8.
.. option:: read-req-num-512-blocks=int
:noindex:
.. option:: size=str
:noindex:
- Size in MiB
+ Size in MiB.
.. option:: block=int
:noindex:
The other values are fairly self explanatory -- number of threads currently
running and doing I/O, the number of currently open files (f=), the estimated
completion percentage, the rate of I/O since last check (read speed listed first,
-then write speed and optionally trim speed) in terms of bandwidth and IOPS, and time to completion for the current
-running group. It's impossible to estimate runtime of the following groups (if
-any).
+then write speed and optionally trim speed) in terms of bandwidth and IOPS,
+and time to completion for the current running group. It's impossible to estimate
+runtime of the following groups (if any).
..
Example output was based on the following:
| 99.99th=[78119]
bw ( KiB/s): min= 532, max= 686, per=0.10%, avg=622.87, stdev=24.82, samples= 100
iops : min= 76, max= 98, avg=88.98, stdev= 3.54, samples= 100
- lat (usec) : 250=0.04%, 500=64.11%, 750=4.81%, 1000=2.79%
- lat (msec) : 2=4.16%, 4=1.84%, 10=4.90%, 20=11.33%, 50=5.37%
- lat (msec) : 100=0.65%
+ lat (usec) : 250=0.04%, 500=64.11%, 750=4.81%, 1000=2.79%
+ lat (msec) : 2=4.16%, 4=1.84%, 10=4.90%, 20=11.33%, 50=5.37%
+ lat (msec) : 100=0.65%
cpu : usr=0.27%, sys=0.18%, ctx=12072, majf=0, minf=21
IO depths : 1=85.0%, 2=13.1%, 4=1.8%, 8=0.1%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete is basically just CPU time (I/O has already been done, see slat
explanation).
+**lat**
+ Total latency. Same names as slat and clat, this denotes the time from
+ when fio created the I/O unit to completion of the I/O operation.
+
**bw**
Bandwidth statistics based on samples. Same names as the xlat stats,
but also includes the number of samples taken (**samples**) and an
**iops**
IOPS statistics based on samples. Same names as bw.
+**lat (nsec/usec/msec)**
+ The distribution of I/O completion latencies. This is the time from when
+ I/O leaves fio and when it gets completed. Unlike the separate
+ read/write/trim sections above, the data here and in the remaining
+ sections apply to all I/Os for the reporting group. 250=0.04% means that
+ 0.04% of the I/Os completed in under 250us. 500=64.11% means that 64.11%
+ of the I/Os required 250 to 499us for completion.
+
**cpu**
CPU usage. User and system time, along with the number of context
switches this thread went through, usage of system and user time, and
The number of read/write/trim requests issued, and how many of them were
short or dropped.
-**IO latencies**
- The distribution of I/O completion latencies. This is the time from when
- I/O leaves fio and when it gets completed. The numbers follow the same
- pattern as the I/O depths, meaning that 2=1.6% means that 1.6% of the
- I/O completed within 2 msecs, 20=12.8% means that 12.8% of the I/O took
- more than 10 msecs, but less than (or equal to) 20 msecs.
+**IO latency**
+ These values are for `--latency-target` and related options. When
+ these options are engaged, this section describes the I/O depth required
+ to meet the specified latency target.
..
Example output was based on the following:
**run**
The smallest and longest runtimes of the threads in this group.
-And finally, the disk statistics are printed. They will look like this::
+And finally, the disk statistics are printed. This is Linux specific. They will look like this::
Disk stats (read/write):
sda: ios=16398/16511, merge=30/162, ticks=6853/819634, in_queue=826487, util=100.00%
**ios**
Number of I/Os performed by all groups.
**merge**
- Number of merges I/O the I/O scheduler.
+ Number of merges performed by the I/O scheduler.
**ticks**
Number of ticks we kept the disk busy.
**in_queue**
change.
Split up, the format is as follows (comments in brackets denote when a
-field was introduced or whether its specific to some terse version):
+field was introduced or whether it's specific to some terse version):
::
- terse version, fio version [v3], jobname, groupid, error
+ terse version, fio version [v3], jobname, groupid, error
READ status::
Completion latency: min, max, mean, stdev (usec)
Completion latency percentiles: 20 fields (see below)
Total latency: min, max, mean, stdev (usec)
- Bw (KiB/s): min, max, aggregate percentage of total, mean, stdev, number of samples [v5]
- IOPS [v5]: min, max, mean, stdev, number of samples
+ Bw (KiB/s): min, max, aggregate percentage of total, mean, stdev, number of samples [v5]
+ IOPS [v5]: min, max, mean, stdev, number of samples
WRITE status:
Completion latency: min, max, mean, stdev (usec)
Completion latency percentiles: 20 fields (see below)
Total latency: min, max, mean, stdev (usec)
- Bw (KiB/s): min, max, aggregate percentage of total, mean, stdev, number of samples [v5]
- IOPS [v5]: min, max, mean, stdev, number of samples
+ Bw (KiB/s): min, max, aggregate percentage of total, mean, stdev, number of samples [v5]
+ IOPS [v5]: min, max, mean, stdev, number of samples
TRIM status [all but version 3]:
- Fields are similar to READ/WRITE status.
+ Fields are similar to READ/WRITE status.
CPU usage::
Disk utilization [v3]::
- Disk name, Read ios, write ios,
- Read merges, write merges,
- Read ticks, write ticks,
- Time spent in queue, disk utilization percentage
+ disk name, read ios, write ios, read merges, write merges, read ticks, write ticks,
+ time spent in queue, disk utilization percentage
Additional Info (dependent on continue_on_error, default off)::
Completion latency percentiles can be a grouping of up to 20 sets, so for the
terse output fio writes all of them. Each field will look like this::
- 1.00%=6112
+ 1.00%=6112
which is the Xth percentile, and the `usec` latency associated with it.
-For disk utilization, all disks used by fio are shown. So for each disk there
+For `Disk utilization`, all disks used by fio are shown. So for each disk there
will be a disk utilization section.
Below is a single line containing short names for each of the fields in the
minimal output v3, separated by semicolons::
- terse_version_3;fio_version;jobname;groupid;error;read_kb;read_bandwidth;read_iops;read_runtime_ms;read_slat_min;read_slat_max;read_slat_mean;read_slat_dev;read_clat_max;read_clat_min;read_clat_mean;read_clat_dev;read_clat_pct01;read_clat_pct02;read_clat_pct03;read_clat_pct04;read_clat_pct05;read_clat_pct06;read_clat_pct07;read_clat_pct08;read_clat_pct09;read_clat_pct10;read_clat_pct11;read_clat_pct12;read_clat_pct13;read_clat_pct14;read_clat_pct15;read_clat_pct16;read_clat_pct17;read_clat_pct18;read_clat_pct19;read_clat_pct20;read_tlat_min;read_lat_max;read_lat_mean;read_lat_dev;read_bw_min;read_bw_max;read_bw_agg_pct;read_bw_mean;read_bw_dev;write_kb;write_bandwidth;write_iops;write_runtime_ms;write_slat_min;write_slat_max;write_slat_mean;write_slat_dev;write_clat_max;write_clat_min;write_clat_mean;write_clat_dev;write_clat_pct01;write_clat_pct02;write_clat_pct03;write_clat_pct04;write_clat_pct05;write_clat_pct06;write_clat_pct07;write_clat_pct08;write_clat_pct09;write_clat_pct10;write_clat_pct11;write_clat_pct12;write_clat_pct13;write_clat_pct14;write_clat_pct15;write_clat_pct16;write_clat_pct17;write_clat_pct18;write_clat_pct19;write_clat_pct20;write_tlat_min;write_lat_max;write_lat_mean;write_lat_dev;write_bw_min;write_bw_max;write_bw_agg_pct;write_bw_mean;write_bw_dev;cpu_user;cpu_sys;cpu_csw;cpu_mjf;pu_minf;iodepth_1;iodepth_2;iodepth_4;iodepth_8;iodepth_16;iodepth_32;iodepth_64;lat_2us;lat_4us;lat_10us;lat_20us;lat_50us;lat_100us;lat_250us;lat_500us;lat_750us;lat_1000us;lat_2ms;lat_4ms;lat_10ms;lat_20ms;lat_50ms;lat_100ms;lat_250ms;lat_500ms;lat_750ms;lat_1000ms;lat_2000ms;lat_over_2000ms;disk_name;disk_read_iops;disk_write_iops;disk_read_merges;disk_write_merges;disk_read_ticks;write_ticks;disk_queue_time;disk_util
+ terse_version_3;fio_version;jobname;groupid;error;read_kb;read_bandwidth;read_iops;read_runtime_ms;read_slat_min;read_slat_max;read_slat_mean;read_slat_dev;read_clat_min;read_clat_max;read_clat_mean;read_clat_dev;read_clat_pct01;read_clat_pct02;read_clat_pct03;read_clat_pct04;read_clat_pct05;read_clat_pct06;read_clat_pct07;read_clat_pct08;read_clat_pct09;read_clat_pct10;read_clat_pct11;read_clat_pct12;read_clat_pct13;read_clat_pct14;read_clat_pct15;read_clat_pct16;read_clat_pct17;read_clat_pct18;read_clat_pct19;read_clat_pct20;read_tlat_min;read_lat_max;read_lat_mean;read_lat_dev;read_bw_min;read_bw_max;read_bw_agg_pct;read_bw_mean;read_bw_dev;write_kb;write_bandwidth;write_iops;write_runtime_ms;write_slat_min;write_slat_max;write_slat_mean;write_slat_dev;write_clat_min;write_clat_max;write_clat_mean;write_clat_dev;write_clat_pct01;write_clat_pct02;write_clat_pct03;write_clat_pct04;write_clat_pct05;write_clat_pct06;write_clat_pct07;write_clat_pct08;write_clat_pct09;write_clat_pct10;write_clat_pct11;write_clat_pct12;write_clat_pct13;write_clat_pct14;write_clat_pct15;write_clat_pct16;write_clat_pct17;write_clat_pct18;write_clat_pct19;write_clat_pct20;write_tlat_min;write_lat_max;write_lat_mean;write_lat_dev;write_bw_min;write_bw_max;write_bw_agg_pct;write_bw_mean;write_bw_dev;cpu_user;cpu_sys;cpu_csw;cpu_mjf;cpu_minf;iodepth_1;iodepth_2;iodepth_4;iodepth_8;iodepth_16;iodepth_32;iodepth_64;lat_2us;lat_4us;lat_10us;lat_20us;lat_50us;lat_100us;lat_250us;lat_500us;lat_750us;lat_1000us;lat_2ms;lat_4ms;lat_10ms;lat_20ms;lat_50ms;lat_100ms;lat_250ms;lat_500ms;lat_750ms;lat_1000ms;lat_2000ms;lat_over_2000ms;disk_name;disk_read_iops;disk_write_iops;disk_read_merges;disk_write_merges;disk_read_ticks;write_ticks;disk_queue_time;disk_util
+
+
+JSON output
+------------
+
+The `json` output format is intended to be both human readable and convenient
+for automated parsing. For the most part its sections mirror those of the
+`normal` output. The `runtime` value is reported in msec and the `bw` value is
+reported in 1024 bytes per second units.
+
+
+JSON+ output
+------------
+
+The `json+` output format is identical to the `json` output format except that it
+adds a full dump of the completion latency bins. Each `bins` object contains a
+set of (key, value) pairs where keys are latency durations and values count how
+many I/Os had completion latencies of the corresponding duration. For example,
+consider:
+
+ "bins" : { "87552" : 1, "89600" : 1, "94720" : 1, "96768" : 1, "97792" : 1, "99840" : 1, "100864" : 2, "103936" : 6, "104960" : 534, "105984" : 5995, "107008" : 7529, ... }
+
+This data indicates that one I/O required 87,552ns to complete, two I/Os required
+100,864ns to complete, and 7529 I/Os required 107,008ns to complete.
+
+Also included with fio is a Python script `fio_jsonplus_clat2csv` that takes
+json+ output and generates CSV-formatted latency data suitable for plotting.
+
+The latency durations actually represent the midpoints of latency intervals.
+For details refer to :file:`stat.h`.
Trace file format
rw, offset, length
-where `rw=0/1` for read/write, and the offset and length entries being in bytes.
+where `rw=0/1` for read/write, and the `offset` and `length` entries being in bytes.
This format is not supported in fio versions >= 1.20-rc3.
filename action
-The filename is given as an absolute path. The action can be one of these:
+The `filename` is given as an absolute path. The `action` can be one of these:
**add**
- Add the given filename to the trace.
+ Add the given `filename` to the trace.
**open**
- Open the file with the given filename. The filename has to have
+ Open the file with the given `filename`. The `filename` has to have
been added with the **add** action before.
**close**
- Close the file with the given filename. The file has to have been
+ Close the file with the given `filename`. The file has to have been
opened before.
Verification trigger example
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-Let's say we want to run a powercut test on the remote machine 'server'. Our
-write workload is in :file:`write-test.fio`. We want to cut power to 'server' at
+Let's say we want to run a powercut test on the remote Linux machine 'server'.
+Our write workload is in :file:`write-test.fio`. We want to cut power to 'server' at
some point during the run, and we'll run this test from the safety or our local
machine, 'localbox'. On the server, we'll start the fio backend normally::
Fio supports a variety of log file formats, for logging latencies, bandwidth,
and IOPS. The logs share a common format, which looks like this:
- *time* (`msec`), *value*, *data direction*, *offset*
+ *time* (`msec`), *value*, *data direction*, *block size* (`bytes`),
+ *offset* (`bytes`)
-Time for the log entry is always in milliseconds. The *value* logged depends
+*Time* for the log entry is always in milliseconds. The *value* logged depends
on the type of log, it will be one of the following:
**Latency log**
- Value is latency in usecs
+ Value is latency in nsecs
**Bandwidth log**
Value is in KiB/sec
**IOPS log**
**2**
I/O is a TRIM
-The *offset* is the offset, in bytes, from the start of the file, for that
-particular I/O. The logging of the offset can be toggled with
-:option:`log_offset`.
+The entry's *block size* is always in bytes. The *offset* is the offset, in bytes,
+from the start of the file, for that particular I/O. The logging of the offset can be
+toggled with :option:`log_offset`.
Fio defaults to logging every individual I/O. When IOPS are logged for individual
-I/Os the value entry will always be 1. If windowed logging is enabled through
+I/Os the *value* entry will always be 1. If windowed logging is enabled through
:option:`log_avg_msec`, fio logs the average values over the specified period of time.
If windowed logging is enabled and :option:`log_max_value` is set, then fio logs
-maximum values in that window instead of averages. Since 'data direction' and
-'offset' are per-I/O values, they aren't applicable if windowed logging is enabled.
+maximum values in that window instead of averages. Since *data direction*, *block
+size* and *offset* are per-I/O values, if windowed logging is enabled they
+aren't applicable and will be 0.
-Client/server
+Client/Server
-------------
Normally fio is invoked as a stand-alone application on the machine where the
Start the server on the machine which has access to the storage DUT::
- fio --server=args
+ $ fio --server=args
where `args` defines what fio listens to. The arguments are of the form
``type,hostname`` or ``IP,port``. *type* is either ``ip`` (or ip4) for TCP/IP
projects / fio.git / blobdiff