.. option:: --debug=type
- Enable verbose tracing of various fio actions. May be ``all`` for all types
+ Enable verbose tracing `type` of various fio actions. May be ``all`` for all types
or individual types separated by a comma (e.g. ``--debug=file,mem`` will
enable file and memory debugging). Currently, additional logging is
available for:
Write output to file `filename`.
-.. option:: --output-format=type
+.. option:: --output-format=format
- Set the reporting format to `normal`, `terse`, `json`, or `json+`. Multiple
+ Set the reporting `format` to `normal`, `terse`, `json`, or `json+`. Multiple
formats can be selected, separated by a comma. `terse` is a CSV based
format. `json+` is like `json`, except it adds a full dump of the latency
buckets.
-.. option:: --runtime
- Limit run time to runtime seconds.
-
.. option:: --bandwidth-log
Generate aggregate bandwidth logs.
**Deprecated**, use :option:`--output-format` instead to select multiple
formats.
-.. option:: --terse-version=type
+.. option:: --terse-version=version
- Set terse version output format (default 3, or 2 or 4 or 5).
+ Set terse `version` output format (default 3, or 2 or 4 or 5).
.. option:: --version
.. option:: --enghelp=[ioengine[,command]]
- List all commands defined by :option:`ioengine`, or print help for `command`
- defined by :option:`ioengine`. If no :option:`ioengine` is given, list all
+ List all commands defined by `ioengine`, or print help for `command`
+ defined by `ioengine`. If no `ioengine` is given, list all
available ioengines.
.. option:: --showcmd=jobfile
.. option:: --status-interval=time
- Force full status dump every `time` period passed. When the unit is
- omitted, the value is interpreted in seconds.
+ Force a full status dump of cumulative (from job start) values at `time`
+ intervals. This option does *not* provide per-period measurements. So
+ values such as bandwidth are running averages. When the time unit is omitted,
+ `time` is interpreted in seconds.
.. option:: --section=name
.. option:: --max-jobs=nr
- Set the maximum number of threads/processes to support.
+ Set the maximum number of threads/processes to support to `nr`.
+ NOTE: On Linux, it may be necessary to increase the shared-memory
+ limit (:file:`/proc/sys/kernel/shmmax`) if fio runs into errors while
+ creating jobs.
.. option:: --server=args
.. option:: --client=hostname
- Instead of running the jobs locally, send and run them on the given host or
- set of hosts. See `Client/Server`_ section.
+ Instead of running the jobs locally, send and run them on the given `hostname`
+ or set of `hostname`\s. See `Client/Server`_ section.
.. option:: --remote-config=file
- Tell fio server to load this local file.
+ Tell fio server to load this local `file`.
.. option:: --idle-prof=option
.. option:: --inflate-log=log
- Inflate and output compressed log.
+ Inflate and output compressed `log`.
.. option:: --trigger-file=file
- Execute trigger cmd when file exists.
+ Execute trigger command when `file` exists.
-.. option:: --trigger-timeout=t
+.. option:: --trigger-timeout=time
- Execute trigger at this time.
+ Execute trigger at this `time`.
-.. option:: --trigger=cmd
+.. option:: --trigger=command
- Set this command as local trigger.
+ Set this `command` as local trigger.
-.. option:: --trigger-remote=cmd
+.. option:: --trigger-remote=command
- Set this command as remote trigger.
+ Set this `command` as remote trigger.
.. option:: --aux-path=path
- Use this path for fio state generated files.
+ Use this `path` for fio state generated files.
Any parameters following the options will be assumed to be job files, unless
they match a job file parameter. Multiple job files can be listed and each job
*global* sections if so desired. A job is only affected by a *global* section
residing above it.
-The :option:`--cmdhelp` option also lists all options. If used with an `option`
-argument, :option:`--cmdhelp` will detail the given `option`.
+The :option:`--cmdhelp` option also lists all options. If used with a `command`
+argument, :option:`--cmdhelp` will detail the given `command`.
See the `examples/` directory for inspiration on how to write job files. Note
the copyright and license requirements currently apply to `examples/` files.
prefixes. To specify power-of-10 decimal values defined in the
International System of Units (SI):
- * *Ki* -- means kilo (K) or 1000
- * *Mi* -- means mega (M) or 1000**2
- * *Gi* -- means giga (G) or 1000**3
- * *Ti* -- means tera (T) or 1000**4
- * *Pi* -- means peta (P) or 1000**5
+ * *K* -- means kilo (K) or 1000
+ * *M* -- means mega (M) or 1000**2
+ * *G* -- means giga (G) or 1000**3
+ * *T* -- means tera (T) or 1000**4
+ * *P* -- means peta (P) or 1000**5
To specify power-of-2 binary values defined in IEC 80000-13:
- * *k* -- means kibi (Ki) or 1024
- * *M* -- means mebi (Mi) or 1024**2
- * *G* -- means gibi (Gi) or 1024**3
- * *T* -- means tebi (Ti) or 1024**4
- * *P* -- means pebi (Pi) or 1024**5
+ * *Ki* -- means kibi (Ki) or 1024
+ * *Mi* -- means mebi (Mi) or 1024**2
+ * *Gi* -- means gibi (Gi) or 1024**3
+ * *Ti* -- means tebi (Ti) or 1024**4
+ * *Pi* -- means pebi (Pi) or 1024**5
With :option:`kb_base`\=1024 (the default), the unit prefixes are opposite
from those specified in the SI and IEC 80000-13 standards to provide
**float_list**
A list of floating point numbers, separated by a ':' character.
+With the above in mind, here follows the complete list of fio job parameters.
+
Units
~~~~~
Bit based.
-With the above in mind, here follows the complete list of fio job parameters.
-
-
Job description
~~~~~~~~~~~~~~~
named :file:`testfiles.4`. The default of :file:`$jobname.$jobnum.$filenum`
will be used if no other format specifier is given.
+ If you specify a path then the directories will be created up to the
+ main directory for the file. So for example if you specify
+ ``filename_format=a/b/c/$jobnum`` then the directories a/b/c will be
+ created before the file setup part of the job. If you specify
+ :option:`directory` then the path will be relative that directory,
+ otherwise it is treated as the absolute path.
+
.. option:: unique_filename=bool
To avoid collisions between networked clients, fio defaults to prefixing any
**sequential**
Finish one file before moving on to the next. Multiple files can
- still be open depending on 'openfiles'.
+ still be open depending on :option:`openfiles`.
**zipf**
Use a *Zipf* distribution to decide what file to access.
.. option:: direct=bool
If value is true, use non-buffered I/O. This is usually O_DIRECT. Note that
- ZFS on Solaris doesn't support direct I/O. On Windows the synchronous
+ OpenBSD and ZFS on Solaris don't support direct I/O. On Windows the synchronous
ioengines don't support direct I/O. Default: false.
.. option:: atomic=bool
.. option:: offset=int
Start I/O at the provided offset in the file, given as either a fixed size in
- bytes or a percentage. If a percentage is given, the next ``blockalign``-ed
- offset will be used. Data before the given offset will not be touched. This
+ bytes or a percentage. If a percentage is given, the generated offset will be
+ aligned to the minimum ``blocksize`` or to the value of ``offset_align`` if
+ provided. Data before the given offset will not be touched. This
effectively caps the file size at `real_size - offset`. Can be combined with
:option:`size` to constrain the start and end range of the I/O workload.
A percentage can be specified by a number between 1 and 100 followed by '%',
for example, ``offset=20%`` to specify 20%.
+.. option:: offset_align=int
+
+ If set to non-zero value, the byte offset generated by a percentage ``offset``
+ is aligned upwards to this value. Defaults to 0 meaning that a percentage
+ offset is aligned to the minimum block size.
+
.. option:: offset_increment=int
If this is provided, then the real offset becomes `offset + offset_increment
Make every `N-th` write a barrier write.
-.. option:: sync_file_range=str:val
+.. option:: sync_file_range=str:int
- Use :manpage:`sync_file_range(2)` for every `val` number of write
+ Use :manpage:`sync_file_range(2)` for every `int` number of write
operations. Fio will track range of writes that have happened since the last
:manpage:`sync_file_range(2)` call. `str` can currently be one or more of:
**zoned**
Zoned random distribution
+ **zoned_abs**
+ Zone absolute random distribution
+
When using a **zipf** or **pareto** distribution, an input value is also
- needed to define the access pattern. For **zipf**, this is the `zipf
+ needed to define the access pattern. For **zipf**, this is the `Zipf
theta`. For **pareto**, it's the `Pareto power`. Fio includes a test
- program, :command:`genzipf`, that can be used visualize what the given input
+ program, :command:`fio-genzipf`, that can be used visualize what the given input
values will yield in terms of hit rates. If you wanted to use **zipf** with
a `theta` of 1.2, you would use ``random_distribution=zipf:1.2`` as the
option. If a non-uniform model is used, fio will disable use of the random
access that should fall within what range of the file or device. For
example, given a criteria of:
- * 60% of accesses should be to the first 10%
- * 30% of accesses should be to the next 20%
- * 8% of accesses should be to the next 30%
- * 2% of accesses should be to the next 40%
+ * 60% of accesses should be to the first 10%
+ * 30% of accesses should be to the next 20%
+ * 8% of accesses should be to the next 30%
+ * 2% of accesses should be to the next 40%
we can define that through zoning of the random accesses. For the above
example, the user would do::
random_distribution=zoned:60/10:30/20:8/30:2/40
- similarly to how :option:`bssplit` works for setting ranges and percentages
- of block sizes. Like :option:`bssplit`, it's possible to specify separate
- zones for reads, writes, and trims. If just one set is given, it'll apply to
- all of them.
+ A **zoned_abs** distribution works exactly like the **zoned**, except
+ that it takes absolute sizes. For example, let's say you wanted to
+ define access according to the following criteria:
+
+ * 60% of accesses should be to the first 20G
+ * 30% of accesses should be to the next 100G
+ * 10% of accesses should be to the next 500G
+
+ we can define an absolute zoning distribution with:
+
+ random_distribution=zoned_abs=60/20G:30/100G:10/500g
+
+ Similarly to how :option:`bssplit` works for setting ranges and
+ percentages of block sizes. Like :option:`bssplit`, it's possible to
+ specify separate zones for reads, writes, and trims. If just one set
+ is given, it'll apply to all of them. This goes for both **zoned**
+ **zoned_abs** distributions.
.. option:: percentage_random=int[,int][,int]
.. option:: random_generator=str
- Fio supports the following engines for generating
- I/O offsets for random I/O:
+ Fio supports the following engines for generating I/O offsets for random I/O:
**tausworthe**
- Strong 2^88 cycle random number generator
+ Strong 2^88 cycle random number generator.
**lfsr**
- Linear feedback shift register generator
+ Linear feedback shift register generator.
**tausworthe64**
- Strong 64-bit 2^258 cycle random number generator
+ Strong 64-bit 2^258 cycle random number generator.
**tausworthe** is a strong random number generator, but it requires tracking
on the side if we want to ensure that blocks are only read or written
- once. **LFSR** guarantees that we never generate the same offset twice, and
+ once. **lfsr** guarantees that we never generate the same offset twice, and
it's also less computationally expensive. It's not a true random generator,
- however, though for I/O purposes it's typically good enough. **LFSR** only
+ however, though for I/O purposes it's typically good enough. **lfsr** only
works with single block sizes, not with workloads that use multiple block
sizes. If used with such a workload, fio may read or write some blocks
multiple times. The default value is **tausworthe**, unless the required
typically won't work with direct I/O, as that normally requires sector
alignment.
-.. option:: bs_is_seq_rand
+.. option:: bs_is_seq_rand=bool
If this option is set, fio will use the normal read,write blocksize settings
as sequential,random blocksize settings instead. Any random read or write
.. option:: refill_buffers
If this option is given, fio will refill the I/O buffers on every
- submit. The default is to only fill it at init time and reuse that
- data. Only makes sense if zero_buffers isn't specified, naturally. If data
- verification is enabled, `refill_buffers` is also automatically enabled.
+ submit. Only makes sense if :option:`zero_buffers` isn't specified,
+ naturally. Defaults to being unset i.e., the buffer is only filled at
+ init time and the data in it is reused when possible but if any of
+ :option:`verify`, :option:`buffer_compress_percentage` or
+ :option:`dedupe_percentage` are enabled then `refill_buffers` is also
+ automatically enabled.
.. option:: scramble_buffers=bool
.. option:: buffer_compress_percentage=int
- If this is set, then fio will attempt to provide I/O buffer content (on
- WRITEs) that compresses to the specified level. Fio does this by providing a
- mix of random data and a fixed pattern. The fixed pattern is either zeros,
- or the pattern specified by :option:`buffer_pattern`. If the pattern option
- is used, it might skew the compression ratio slightly. Note that this is per
- block size unit, for file/disk wide compression level that matches this
- setting, you'll also want to set :option:`refill_buffers`.
+ If this is set, then fio will attempt to provide I/O buffer content
+ (on WRITEs) that compresses to the specified level. Fio does this by
+ providing a mix of random data followed by fixed pattern data. The
+ fixed pattern is either zeros, or the pattern specified by
+ :option:`buffer_pattern`. If the `buffer_pattern` option is used, it
+ might skew the compression ratio slightly. Setting
+ `buffer_compress_percentage` to a value other than 100 will also
+ enable :option:`refill_buffers` in order to reduce the likelihood that
+ adjacent blocks are so similar that they over compress when seen
+ together. See :option:`buffer_compress_chunk` for how to set a finer or
+ coarser granularity for the random/fixed data region. Defaults to unset
+ i.e., buffer data will not adhere to any compression level.
.. option:: buffer_compress_chunk=int
- See :option:`buffer_compress_percentage`. This setting allows fio to manage
- how big the ranges of random data and zeroed data is. Without this set, fio
- will provide :option:`buffer_compress_percentage` of blocksize random data,
- followed by the remaining zeroed. With this set to some chunk size smaller
- than the block size, fio can alternate random and zeroed data throughout the
- I/O buffer.
+ This setting allows fio to manage how big the random/fixed data region
+ is when using :option:`buffer_compress_percentage`. When
+ `buffer_compress_chunk` is set to some non-zero value smaller than the
+ block size, fio can repeat the random/fixed region throughout the I/O
+ buffer at the specified interval (which particularly useful when
+ bigger block sizes are used for a job). When set to 0, fio will use a
+ chunk size that matches the block size resulting in a single
+ random/fixed region within the I/O buffer. Defaults to 512. When the
+ unit is omitted, the value is interpreted in bytes.
.. option:: buffer_pattern=str
writing. These buffers will be naturally dedupable. The contents of the
buffers depend on what other buffer compression settings have been set. It's
possible to have the individual buffers either fully compressible, or not at
- all. This option only controls the distribution of unique buffers.
+ all -- this option only controls the distribution of unique buffers. Setting
+ this option will also enable :option:`refill_buffers` to prevent every buffer
+ being identical.
.. option:: invalidate=bool
**cudamalloc**
Use GPU memory as the buffers for GPUDirect RDMA benchmark.
+ The :option:`ioengine` must be `rdma`.
The area allocated is a function of the maximum allowed bs size for the job,
multiplied by the I/O depth given. Note that for **shmhuge** and
should point there. So if it's mounted in :file:`/huge`, you would use
`mem=mmaphuge:/huge/somefile`.
-.. option:: iomem_align=int
+.. option:: iomem_align=int, mem_align=int
This indicates the memory alignment of the I/O memory buffers. Note that
the given alignment is applied to the first I/O unit buffer, if using
SCSI generic sg v3 I/O. May either be synchronous using the SG_IO
ioctl, or if the target is an sg character device we use
:manpage:`read(2)` and :manpage:`write(2)` for asynchronous
- I/O. Requires filename option to specify either block or character
- devices.
+ I/O. Requires :option:`filename` option to specify either block or
+ character devices.
**null**
Doesn't transfer any data, just pretends to. This is mainly used to
Doesn't transfer any data, but burns CPU cycles according to the
:option:`cpuload` and :option:`cpuchunks` options. Setting
:option:`cpuload`\=85 will cause that job to do nothing but burn 85%
- of the CPU. In case of SMP machines, use :option:`numjobs`
- =<no_of_cpu> to get desired CPU usage, as the cpuload only loads a
+ of the CPU. In case of SMP machines, use :option:`numjobs`\=<nr_of_cpu>
+ to get desired CPU usage, as the cpuload only loads a
single CPU at the desired rate. A job never finishes unless there is
at least one non-cpuio job.
**rdma**
The RDMA I/O engine supports both RDMA memory semantics
(RDMA_WRITE/RDMA_READ) and channel semantics (Send/Recv) for the
- InfiniBand, RoCE and iWARP protocols.
+ InfiniBand, RoCE and iWARP protocols. This engine defines engine
+ specific options.
**falloc**
I/O engine that does regular fallocate to simulate data transfer as
**ftruncate**
I/O engine that sends :manpage:`ftruncate(2)` operations in response
to write (DDIR_WRITE) events. Each ftruncate issued sets the file's
- size to the current block offset. Block size is ignored.
+ size to the current block offset. :option:`blocksize` is ignored.
**e4defrag**
I/O engine that does regular EXT4_IOC_MOVE_EXT ioctls to simulate
defines engine specific options.
**libhdfs**
- Read and write through Hadoop (HDFS). The :file:`filename` option
+ Read and write through Hadoop (HDFS). The :option:`filename` option
is used to specify host,port of the hdfs name-node to connect. This
engine interprets offsets a little differently. In HDFS, files once
created cannot be modified so random writes are not possible. To
**external**
Prefix to specify loading an external I/O engine object file. Append
the engine filename, e.g. ``ioengine=external:/tmp/foo.o`` to load
- ioengine :file:`foo.o` in :file:`/tmp`.
+ ioengine :file:`foo.o` in :file:`/tmp`. The path can be either
+ absolute or relative. See :file:`engines/skeleton_external.c` for
+ details of writing an external I/O engine.
+ **filecreate**
+ Simply create the files and do no I/O to them. You still need to
+ set `filesize` so that all the accounting still occurs, but no
+ actual I/O will be done other than creating the file.
+
+ **libpmem**
+ Read and write using mmap I/O to a file on a filesystem
+ mounted with DAX on a persistent memory device through the NVML
+ libpmem library.
I/O engine specific parameters
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In addition, there are some parameters which are only valid when a specific
-ioengine is in use. These are used identically to normal parameters, with the
-caveat that when used on the command line, they must come after the
+:option:`ioengine` is in use. These are used identically to normal parameters,
+with the caveat that when used on the command line, they must come after the
:option:`ioengine` that defines them is selected.
.. option:: userspace_reap : [libaio]
Set RWF_HIPRI on I/O, indicating to the kernel that it's of higher priority
than normal.
+.. option:: hipri_percentage : [pvsync2]
+
+ When hipri is set this determines the probability of a pvsync2 I/O being high
+ priority. The default is 100%.
+
.. option:: cpuload=int : [cpuio]
Attempt to use the specified percentage of CPU cycles. This is a mandatory
Detect when I/O threads are done, then exit.
-.. option:: hostname=str : [netsplice] [net]
-
- The hostname or IP address to use for TCP or UDP based I/O. If the job is
- a TCP listener or UDP reader, the hostname is not used and must be omitted
- unless it is a valid UDP multicast address.
-
.. option:: namenode=str : [libhdfs]
The hostname or IP address of a HDFS cluster namenode to contact.
.. option:: port=int
+ [libhdfs]
+
+ The listening port of the HFDS cluster namenode.
+
[netsplice], [net]
The TCP or UDP port to bind to or connect to. If this is used with
this will be the starting port number since fio will use a range of
ports.
- [libhdfs]
+ [rdma]
- the listening port of the HFDS cluster namenode.
+ The port to use for RDMA-CM communication. This should be the same value
+ on the client and the server side.
+
+.. option:: hostname=str : [netsplice] [net] [rdma]
+
+ The hostname or IP address to use for TCP, UDP or RDMA-CM based I/O. If the job
+ is a TCP listener or UDP reader, the hostname is not used and must be omitted
+ unless it is a valid UDP multicast address.
.. option:: interface=str : [netsplice] [net]
Set TCP_NODELAY on TCP connections.
-.. option:: protocol=str : [netsplice] [net]
-
-.. option:: proto=str : [netsplice] [net]
+.. option:: protocol=str, proto=str : [netsplice] [net]
The network protocol to use. Accepted values are:
When the protocol is TCP or UDP, the port must also be given, as well as the
hostname if the job is a TCP listener or UDP reader. For unix sockets, the
- normal filename option should be used and the port is invalid.
+ normal :option:`filename` option should be used and the port is invalid.
.. option:: listen : [netsplice] [net]
**0**
Default. Preallocate donor's file on init.
**1**
- Allocate space immediately inside defragment event, and free right
+ Allocate space immediately inside defragment event, and free right
after event.
.. option:: clustername=str : [rbd]
.. option:: chunk_size : [libhdfs]
- the size of the chunk to use for each file.
+ The size of the chunk to use for each file.
+
+.. option:: verb=str : [rdma]
+ The RDMA verb to use on this side of the RDMA ioengine connection. Valid
+ values are write, read, send and recv. These correspond to the equivalent
+ RDMA verbs (e.g. write = rdma_write etc.). Note that this only needs to be
+ specified on the client side of the connection. See the examples folder.
+
+.. option:: bindname=str : [rdma]
+
+ The name to use to bind the local RDMA-CM connection to a local RDMA device.
+ This could be a hostname or an IPv4 or IPv6 address. On the server side this
+ will be passed into the rdma_bind_addr() function and on the client site it
+ will be used in the rdma_resolve_add() function. This can be useful when
+ multiple paths exist between the client and the server or in certain loopback
+ configurations.
I/O depth
~~~~~~~~~
16 requests, it will let the depth drain down to 4 before starting to fill
it again.
+.. option:: serialize_overlap=bool
+
+ Serialize in-flight I/Os that might otherwise cause or suffer from data races.
+ When two or more I/Os are submitted simultaneously, there is no guarantee that
+ the I/Os will be processed or completed in the submitted order. Further, if
+ two or more of those I/Os are writes, any overlapping region between them can
+ become indeterminate/undefined on certain storage. These issues can cause
+ verification to fail erratically when at least one of the racing I/Os is
+ changing data and the overlapping region has a non-zero size. Setting
+ ``serialize_overlap`` tells fio to avoid provoking this behavior by explicitly
+ serializing in-flight I/Os that have a non-zero overlap. Note that setting
+ this option can reduce both performance and the :option:`iodepth` achieved.
+ Additionally this option does not work when :option:`io_submit_mode` is set to
+ offload. Default: false.
+
.. option:: io_submit_mode=str
This option controls how fio submits the I/O to the I/O engine. The default
.. option:: thinktime_blocks=int
Only valid if :option:`thinktime` is set - control how many blocks to issue,
- before waiting `thinktime` usecs. If not set, defaults to 1 which will make
- fio wait `thinktime` usecs after every block. This effectively makes any
+ before waiting :option:`thinktime` usecs. If not set, defaults to 1 which will make
+ fio wait :option:`thinktime` usecs after every block. This effectively makes any
queue depth setting redundant, since no more than 1 I/O will be queued
- before we have to complete it and do our thinktime. In other words, this
+ before we have to complete it and do our :option:`thinktime`. In other words, this
setting effectively caps the queue depth if the latter is larger.
.. option:: rate=int[,int][,int]
replay, the file needs to be turned into a blkparse binary data file first
(``blkparse <device> -o /dev/null -d file_for_fio.bin``).
-.. option:: replay_no_stall=int
+.. option:: replay_no_stall=bool
When replaying I/O with :option:`read_iolog` the default behavior is to
attempt to respect the timestamps within the log and replay them with the
.. option:: verifysort_nr=int
- Pre-load and sort verify blocks for a read workload.
+ Pre-load and sort verify blocks for a read workload.
.. option:: verify_offset=int
state is loaded for the verify read phase. The format of the filename is,
roughly::
- <type>-<jobname>-<jobindex>-verify.state.
+ <type>-<jobname>-<jobindex>-verify.state.
<type> is "local" for a local run, "sock" for a client/server socket
connection, and "ip" (192.168.0.1, for instance) for a networked
.. option:: trim_backlog=int
- Verify that trim/discarded blocks are returned as zeros.
+ Trim after this number of blocks are written.
.. option:: trim_backlog_batch=int
Enable experimental verification.
-
Steady state
~~~~~~~~~~~~
all jobs in a file will be part of the same reporting group, unless
separated by a :option:`stonewall`.
-.. option:: stats
+.. option:: stats=bool
By default, fio collects and shows final output results for all jobs
that run. If this option is set to 0, then fio will ignore it in
.. option:: write_bw_log=str
If given, write a bandwidth log for this job. Can be used to store data of
- the bandwidth of the jobs in their lifetime. The included
- :command:`fio_generate_plots` script uses :command:`gnuplot` to turn these
- text files into nice graphs. See :option:`write_lat_log` for behavior of
- given filename. For this option, the postfix is :file:`_bw.x.log`, where `x`
- is the index of the job (`1..N`, where `N` is the number of jobs). If
- :option:`per_job_logs` is false, then the filename will not include the job
- index. See `Log File Formats`_.
+ the bandwidth of the jobs in their lifetime.
-.. option:: write_lat_log=str
+ If no str argument is given, the default filename of
+ :file:`jobname_type.x.log` is used. Even when the argument is given, fio
+ will still append the type of log. So if one specifies::
- Same as :option:`write_bw_log`, except that this option stores I/O
- submission, completion, and total latencies instead. If no filename is given
- with this option, the default filename of :file:`jobname_type.log` is
- used. Even if the filename is given, fio will still append the type of
- log. So if one specifies::
+ write_bw_log=foo
- write_lat_log=foo
+ The actual log name will be :file:`foo_bw.x.log` where `x` is the index
+ of the job (`1..N`, where `N` is the number of jobs). If
+ :option:`per_job_logs` is false, then the filename will not include the
+ `.x` job index.
- The actual log names will be :file:`foo_slat.x.log`, :file:`foo_clat.x.log`,
- and :file:`foo_lat.x.log`, where `x` is the index of the job (1..N, where N
- is the number of jobs). This helps :command:`fio_generate_plot` find the
- logs automatically. If :option:`per_job_logs` is false, then the filename
- will not include the job index. See `Log File Formats`_.
+ The included :command:`fio_generate_plots` script uses :command:`gnuplot` to turn these
+ text files into nice graphs. See `Log File Formats`_ for how data is
+ structured within the file.
+
+.. option:: write_lat_log=str
+
+ Same as :option:`write_bw_log`, except this option creates I/O
+ submission (e.g., :file:`name_slat.x.log`), completion (e.g.,
+ :file:`name_clat.x.log`), and total (e.g., :file:`name_lat.x.log`)
+ latency files instead. See :option:`write_bw_log` for details about
+ the filename format and `Log File Formats`_ for how data is structured
+ within the files.
.. option:: write_hist_log=str
- Same as :option:`write_lat_log`, but writes I/O completion latency
- histograms. If no filename is given with this option, the default filename
- of :file:`jobname_clat_hist.x.log` is used, where `x` is the index of the
- job (1..N, where `N` is the number of jobs). Even if the filename is given,
- fio will still append the type of log. If :option:`per_job_logs` is false,
- then the filename will not include the job index. See `Log File Formats`_.
+ Same as :option:`write_bw_log` but writes an I/O completion latency
+ histogram file (e.g., :file:`name_hist.x.log`) instead. Note that this
+ file will be empty unless :option:`log_hist_msec` has also been set.
+ See :option:`write_bw_log` for details about the filename format and
+ `Log File Formats`_ for how data is structured within the file.
.. option:: write_iops_log=str
- Same as :option:`write_bw_log`, but writes IOPS. If no filename is given
- with this option, the default filename of :file:`jobname_type.x.log` is
- used,where `x` is the index of the job (1..N, where `N` is the number of
- jobs). Even if the filename is given, fio will still append the type of
- log. If :option:`per_job_logs` is false, then the filename will not include
- the job index. See `Log File Formats`_.
+ Same as :option:`write_bw_log`, but writes an IOPS file (e.g.
+ :file:`name_iops.x.log`) instead. See :option:`write_bw_log` for
+ details about the filename format and `Log File Formats`_ for how data
+ is structured within the file.
.. option:: log_avg_msec=int
:option:`log_avg_msec` is inaccurate. Setting this option makes fio log
histogram entries over the specified period of time, reducing log sizes for
high IOPS devices while retaining percentile accuracy. See
- :option:`log_hist_coarseness` as well. Defaults to 0, meaning histogram
- logging is disabled.
+ :option:`log_hist_coarseness` and :option:`write_hist_log` as well.
+ Defaults to 0, meaning histogram logging is disabled.
.. option:: log_hist_coarseness=int
Integer ranging from 0 to 6, defining the coarseness of the resolution of
the histogram logs enabled with :option:`log_hist_msec`. For each increment
in coarseness, fio outputs half as many bins. Defaults to 0, for which
- histogram logs contain 1216 latency bins. See `Log File Formats`_.
+ histogram logs contain 1216 latency bins. See :option:`write_hist_log`
+ and `Log File Formats`_.
.. option:: log_max_value=bool
you instead want to log the maximum value, set this option to 1. Defaults to
0, meaning that averaged values are logged.
-.. option:: log_offset=int
+.. option:: log_offset=bool
If this is set, the iolog options will include the byte offset for the I/O
entry as well as the other data values. Defaults to 0 meaning that
.. option:: disable_slat=bool
Disable measurements of submission latency numbers. See
- :option:`disable_slat`.
+ :option:`disable_lat`.
.. option:: disable_bw_measurement=bool, disable_bw=bool
.. option:: clat_percentiles=bool
- Enable the reporting of percentiles of completion latencies.
+ Enable the reporting of percentiles of completion latencies. This
+ option is mutually exclusive with :option:`lat_percentiles`.
+
+.. option:: lat_percentiles=bool
+
+ Enable the reporting of percentiles of I/O latencies. This is similar
+ to :option:`clat_percentiles`, except that this includes the
+ submission latency. This option is mutually exclusive with
+ :option:`clat_percentiles`.
.. option:: percentile_list=float_list
completion latency below which 99.5% and 99.9% of the observed latencies
fell, respectively.
+.. option:: significant_figures=int
+
+ If using :option:`--output-format` of `normal`, set the significant figures
+ to this value. Higher values will yield more precise IOPS and throughput
+ units, while lower values will round. Requires a minimum value of 1 and a
+ maximum value of 10. Defaults to 4.
+
Error handling
~~~~~~~~~~~~~~
To view a profile's additional options use :option:`--cmdhelp` after specifying
the profile. For example::
-$ fio --profile=act --cmdhelp
+ $ fio --profile=act --cmdhelp
Act profile options
~~~~~~~~~~~~~~~~~~~
.. option:: threads-per-queue=int
:noindex:
- Number of read IO threads per device. Default: 8.
+ Number of read I/O threads per device. Default: 8.
.. option:: read-req-num-512-blocks=int
:noindex:
.. option:: size=str
:noindex:
- Size in MiB
+ Size in MiB.
.. option:: block=int
:noindex:
The other values are fairly self explanatory -- number of threads currently
running and doing I/O, the number of currently open files (f=), the estimated
completion percentage, the rate of I/O since last check (read speed listed first,
-then write speed and optionally trim speed) in terms of bandwidth and IOPS, and time to completion for the current
-running group. It's impossible to estimate runtime of the following groups (if
-any).
+then write speed and optionally trim speed) in terms of bandwidth and IOPS,
+and time to completion for the current running group. It's impossible to estimate
+runtime of the following groups (if any).
..
Example output was based on the following:
| 99.99th=[78119]
bw ( KiB/s): min= 532, max= 686, per=0.10%, avg=622.87, stdev=24.82, samples= 100
iops : min= 76, max= 98, avg=88.98, stdev= 3.54, samples= 100
- lat (usec) : 250=0.04%, 500=64.11%, 750=4.81%, 1000=2.79%
- lat (msec) : 2=4.16%, 4=1.84%, 10=4.90%, 20=11.33%, 50=5.37%
- lat (msec) : 100=0.65%
+ lat (usec) : 250=0.04%, 500=64.11%, 750=4.81%, 1000=2.79%
+ lat (msec) : 2=4.16%, 4=1.84%, 10=4.90%, 20=11.33%, 50=5.37%
+ lat (msec) : 100=0.65%
cpu : usr=0.27%, sys=0.18%, ctx=12072, majf=0, minf=21
IO depths : 1=85.0%, 2=13.1%, 4=1.8%, 8=0.1%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete is basically just CPU time (I/O has already been done, see slat
explanation).
+**lat**
+ Total latency. Same names as slat and clat, this denotes the time from
+ when fio created the I/O unit to completion of the I/O operation.
+
**bw**
Bandwidth statistics based on samples. Same names as the xlat stats,
but also includes the number of samples taken (**samples**) and an
**iops**
IOPS statistics based on samples. Same names as bw.
+**lat (nsec/usec/msec)**
+ The distribution of I/O completion latencies. This is the time from when
+ I/O leaves fio and when it gets completed. Unlike the separate
+ read/write/trim sections above, the data here and in the remaining
+ sections apply to all I/Os for the reporting group. 250=0.04% means that
+ 0.04% of the I/Os completed in under 250us. 500=64.11% means that 64.11%
+ of the I/Os required 250 to 499us for completion.
+
**cpu**
CPU usage. User and system time, along with the number of context
switches this thread went through, usage of system and user time, and
The number of read/write/trim requests issued, and how many of them were
short or dropped.
-**IO latencies**
- The distribution of I/O completion latencies. This is the time from when
- I/O leaves fio and when it gets completed. The numbers follow the same
- pattern as the I/O depths, meaning that 2=1.6% means that 1.6% of the
- I/O completed within 2 msecs, 20=12.8% means that 12.8% of the I/O took
- more than 10 msecs, but less than (or equal to) 20 msecs.
+**IO latency**
+ These values are for :option:`latency_target` and related options. When
+ these options are engaged, this section describes the I/O depth required
+ to meet the specified latency target.
..
Example output was based on the following:
**run**
The smallest and longest runtimes of the threads in this group.
-And finally, the disk statistics are printed. They will look like this::
+And finally, the disk statistics are printed. This is Linux specific. They will look like this::
Disk stats (read/write):
sda: ios=16398/16511, merge=30/162, ticks=6853/819634, in_queue=826487, util=100.00%
::
- terse version, fio version [v3], jobname, groupid, error
+ terse version, fio version [v3], jobname, groupid, error
READ status::
Completion latency: min, max, mean, stdev (usec)
Completion latency percentiles: 20 fields (see below)
Total latency: min, max, mean, stdev (usec)
- Bw (KiB/s): min, max, aggregate percentage of total, mean, stdev, number of samples [v5]
- IOPS [v5]: min, max, mean, stdev, number of samples
+ Bw (KiB/s): min, max, aggregate percentage of total, mean, stdev, number of samples [v5]
+ IOPS [v5]: min, max, mean, stdev, number of samples
WRITE status:
Completion latency: min, max, mean, stdev (usec)
Completion latency percentiles: 20 fields (see below)
Total latency: min, max, mean, stdev (usec)
- Bw (KiB/s): min, max, aggregate percentage of total, mean, stdev, number of samples [v5]
- IOPS [v5]: min, max, mean, stdev, number of samples
+ Bw (KiB/s): min, max, aggregate percentage of total, mean, stdev, number of samples [v5]
+ IOPS [v5]: min, max, mean, stdev, number of samples
TRIM status [all but version 3]:
- Fields are similar to READ/WRITE status.
+ Fields are similar to READ/WRITE status.
CPU usage::
Disk utilization [v3]::
- Disk name, Read ios, write ios,
- Read merges, write merges,
- Read ticks, write ticks,
- Time spent in queue, disk utilization percentage
+ disk name, read ios, write ios, read merges, write merges, read ticks, write ticks,
+ time spent in queue, disk utilization percentage
Additional Info (dependent on continue_on_error, default off)::
Completion latency percentiles can be a grouping of up to 20 sets, so for the
terse output fio writes all of them. Each field will look like this::
- 1.00%=6112
+ 1.00%=6112
which is the Xth percentile, and the `usec` latency associated with it.
-For disk utilization, all disks used by fio are shown. So for each disk there
+For `Disk utilization`, all disks used by fio are shown. So for each disk there
will be a disk utilization section.
Below is a single line containing short names for each of the fields in the
minimal output v3, separated by semicolons::
- terse_version_3;fio_version;jobname;groupid;error;read_kb;read_bandwidth;read_iops;read_runtime_ms;read_slat_min;read_slat_max;read_slat_mean;read_slat_dev;read_clat_min;read_clat_max;read_clat_mean;read_clat_dev;read_clat_pct01;read_clat_pct02;read_clat_pct03;read_clat_pct04;read_clat_pct05;read_clat_pct06;read_clat_pct07;read_clat_pct08;read_clat_pct09;read_clat_pct10;read_clat_pct11;read_clat_pct12;read_clat_pct13;read_clat_pct14;read_clat_pct15;read_clat_pct16;read_clat_pct17;read_clat_pct18;read_clat_pct19;read_clat_pct20;read_tlat_min;read_lat_max;read_lat_mean;read_lat_dev;read_bw_min;read_bw_max;read_bw_agg_pct;read_bw_mean;read_bw_dev;write_kb;write_bandwidth;write_iops;write_runtime_ms;write_slat_min;write_slat_max;write_slat_mean;write_slat_dev;write_clat_min;write_clat_max;write_clat_mean;write_clat_dev;write_clat_pct01;write_clat_pct02;write_clat_pct03;write_clat_pct04;write_clat_pct05;write_clat_pct06;write_clat_pct07;write_clat_pct08;write_clat_pct09;write_clat_pct10;write_clat_pct11;write_clat_pct12;write_clat_pct13;write_clat_pct14;write_clat_pct15;write_clat_pct16;write_clat_pct17;write_clat_pct18;write_clat_pct19;write_clat_pct20;write_tlat_min;write_lat_max;write_lat_mean;write_lat_dev;write_bw_min;write_bw_max;write_bw_agg_pct;write_bw_mean;write_bw_dev;cpu_user;cpu_sys;cpu_csw;cpu_mjf;cpu_minf;iodepth_1;iodepth_2;iodepth_4;iodepth_8;iodepth_16;iodepth_32;iodepth_64;lat_2us;lat_4us;lat_10us;lat_20us;lat_50us;lat_100us;lat_250us;lat_500us;lat_750us;lat_1000us;lat_2ms;lat_4ms;lat_10ms;lat_20ms;lat_50ms;lat_100ms;lat_250ms;lat_500ms;lat_750ms;lat_1000ms;lat_2000ms;lat_over_2000ms;disk_name;disk_read_iops;disk_write_iops;disk_read_merges;disk_write_merges;disk_read_ticks;write_ticks;disk_queue_time;disk_util
+ terse_version_3;fio_version;jobname;groupid;error;read_kb;read_bandwidth;read_iops;read_runtime_ms;read_slat_min;read_slat_max;read_slat_mean;read_slat_dev;read_clat_min;read_clat_max;read_clat_mean;read_clat_dev;read_clat_pct01;read_clat_pct02;read_clat_pct03;read_clat_pct04;read_clat_pct05;read_clat_pct06;read_clat_pct07;read_clat_pct08;read_clat_pct09;read_clat_pct10;read_clat_pct11;read_clat_pct12;read_clat_pct13;read_clat_pct14;read_clat_pct15;read_clat_pct16;read_clat_pct17;read_clat_pct18;read_clat_pct19;read_clat_pct20;read_tlat_min;read_lat_max;read_lat_mean;read_lat_dev;read_bw_min;read_bw_max;read_bw_agg_pct;read_bw_mean;read_bw_dev;write_kb;write_bandwidth;write_iops;write_runtime_ms;write_slat_min;write_slat_max;write_slat_mean;write_slat_dev;write_clat_min;write_clat_max;write_clat_mean;write_clat_dev;write_clat_pct01;write_clat_pct02;write_clat_pct03;write_clat_pct04;write_clat_pct05;write_clat_pct06;write_clat_pct07;write_clat_pct08;write_clat_pct09;write_clat_pct10;write_clat_pct11;write_clat_pct12;write_clat_pct13;write_clat_pct14;write_clat_pct15;write_clat_pct16;write_clat_pct17;write_clat_pct18;write_clat_pct19;write_clat_pct20;write_tlat_min;write_lat_max;write_lat_mean;write_lat_dev;write_bw_min;write_bw_max;write_bw_agg_pct;write_bw_mean;write_bw_dev;cpu_user;cpu_sys;cpu_csw;cpu_mjf;cpu_minf;iodepth_1;iodepth_2;iodepth_4;iodepth_8;iodepth_16;iodepth_32;iodepth_64;lat_2us;lat_4us;lat_10us;lat_20us;lat_50us;lat_100us;lat_250us;lat_500us;lat_750us;lat_1000us;lat_2ms;lat_4ms;lat_10ms;lat_20ms;lat_50ms;lat_100ms;lat_250ms;lat_500ms;lat_750ms;lat_1000ms;lat_2000ms;lat_over_2000ms;disk_name;disk_read_iops;disk_write_iops;disk_read_merges;disk_write_merges;disk_read_ticks;write_ticks;disk_queue_time;disk_util
+
+
+JSON output
+------------
+
+The `json` output format is intended to be both human readable and convenient
+for automated parsing. For the most part its sections mirror those of the
+`normal` output. The `runtime` value is reported in msec and the `bw` value is
+reported in 1024 bytes per second units.
+
+
+JSON+ output
+------------
+
+The `json+` output format is identical to the `json` output format except that it
+adds a full dump of the completion latency bins. Each `bins` object contains a
+set of (key, value) pairs where keys are latency durations and values count how
+many I/Os had completion latencies of the corresponding duration. For example,
+consider:
+
+ "bins" : { "87552" : 1, "89600" : 1, "94720" : 1, "96768" : 1, "97792" : 1, "99840" : 1, "100864" : 2, "103936" : 6, "104960" : 534, "105984" : 5995, "107008" : 7529, ... }
+
+This data indicates that one I/O required 87,552ns to complete, two I/Os required
+100,864ns to complete, and 7529 I/Os required 107,008ns to complete.
+
+Also included with fio is a Python script `fio_jsonplus_clat2csv` that takes
+json+ output and generates CSV-formatted latency data suitable for plotting.
+
+The latency durations actually represent the midpoints of latency intervals.
+For details refer to :file:`stat.h`.
Trace file format
rw, offset, length
-where `rw=0/1` for read/write, and the offset and length entries being in bytes.
+where `rw=0/1` for read/write, and the `offset` and `length` entries being in bytes.
This format is not supported in fio versions >= 1.20-rc3.
filename action
-The filename is given as an absolute path. The action can be one of these:
+The `filename` is given as an absolute path. The `action` can be one of these:
**add**
- Add the given filename to the trace.
+ Add the given `filename` to the trace.
**open**
- Open the file with the given filename. The filename has to have
+ Open the file with the given `filename`. The `filename` has to have
been added with the **add** action before.
**close**
- Close the file with the given filename. The file has to have been
+ Close the file with the given `filename`. The file has to have been
opened before.
Verification trigger example
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-Let's say we want to run a powercut test on the remote machine 'server'. Our
-write workload is in :file:`write-test.fio`. We want to cut power to 'server' at
+Let's say we want to run a powercut test on the remote Linux machine 'server'.
+Our write workload is in :file:`write-test.fio`. We want to cut power to 'server' at
some point during the run, and we'll run this test from the safety or our local
machine, 'localbox'. On the server, we'll start the fio backend normally::
on the type of log, it will be one of the following:
**Latency log**
- Value is latency in usecs
+ Value is latency in nsecs
**Bandwidth log**
Value is in KiB/sec
**IOPS log**
Start the server on the machine which has access to the storage DUT::
- fio --server=args
+ $ fio --server=args
where `args` defines what fio listens to. The arguments are of the form
``type,hostname`` or ``IP,port``. *type* is either ``ip`` (or ip4) for TCP/IP
projects / fio.git / blobdiff