.. option:: --output-format=type
Set the reporting format to `normal`, `terse`, `json`, or `json+`. Multiple
- formats can be selected, separate by a comma. `terse` is a CSV based
+ formats can be selected, separated by a comma. `terse` is a CSV based
format. `json+` is like `json`, except it adds a full dump of the latency
buckets.
.. option:: --terse-version=type
- Set terse version output format (default 3, or 2 or 4).
+ Set terse version output format (default 3, or 2 or 4 or 5).
.. option:: --version
.. option:: --eta-newline=time
- Force a new line for every `time` period passed.
+ Force a new line for every `time` period passed. When the unit is omitted,
+ the value is interpreted in seconds.
.. option:: --status-interval=time
- Force full status dump every `time` period passed.
+ Force full status dump every `time` period passed. When the unit is
+ omitted, the value is interpreted in seconds.
.. option:: --section=name
The optional *integer suffix* specifies the number's units, and includes an
optional unit prefix and an optional unit. For quantities of data, the
- default unit is bytes. For quantities of time, the default unit is seconds.
+ default unit is bytes. For quantities of time, the default unit is seconds
+ unless otherwise specified.
With :option:`kb_base` =1000, fio follows international standards for unit
prefixes. To specify power-of-10 decimal values defined in the
* *D* -- means days
* *H* -- means hours
- * *M* -- mean minutes
+ * *M* -- means minutes
* *s* -- or sec means seconds (default)
* *ms* -- or *msec* means milliseconds
* *us* -- or *usec* means microseconds
If the option accepts an upper and lower range, use a colon ':' or
minus '-' to separate such values. See :ref:`irange <irange>`.
+ If the lower value specified happens to be larger than the upper value
+ the two values are swapped.
.. _bool:
larger number of threads/processes doing the same thing. Each thread is
reported separately; to see statistics for all clones as a whole, use
:option:`group_reporting` in conjunction with :option:`new_group`.
- See :option:`--max-jobs`.
+ See :option:`--max-jobs`. Default: 1.
Time related parameters
Tell fio to terminate processing after the specified period of time. It
can be quite hard to determine for how long a specified job will run, so
this parameter is handy to cap the total runtime to a given time. When
- the unit is omitted, the value is given in seconds.
+ the unit is omitted, the value is intepreted in seconds.
.. option:: time_based
.. option:: startdelay=irange(time)
- Delay start of job for the specified number of seconds. Supports all time
- suffixes to allow specification of hours, minutes, seconds and milliseconds
- -- seconds are the default if a unit is omitted. Can be given as a range
- which causes each thread to choose randomly out of the range.
+ Delay the start of job for the specified amount of time. Can be a single
+ value or a range. When given as a range, each thread will choose a value
+ randomly from within the range. Value is in seconds if a unit is omitted.
.. option:: ramp_time=time
If true, serialize the file creation for the jobs. This may be handy to
avoid interleaving of data files, which may greatly depend on the filesystem
- used and even the number of processors in the system.
+ used and even the number of processors in the system. Default: true.
.. option:: create_fsync=bool
.. option:: create_on_open=bool
Don't pre-setup the files for I/O, just create open() when it's time to do
- I/O to that file.
+ I/O to that file. Default: false.
.. option:: create_only=bool
If true, fio will only run the setup phase of the job. If files need to be
- laid out or updated on disk, only that will be done. The actual job contents
- are not executed.
+ laid out or updated on disk, only that will be done -- the actual job contents
+ are not executed. Default: false.
.. option:: allow_file_create=bool
If this isn't set, fio will abort jobs that are destructive (e.g. that write)
to what appears to be a mounted device or partition. This should help catch
creating inadvertently destructive tests, not realizing that the test will
- destroy data on the mounted file system. Default: false.
+ destroy data on the mounted file system. Note that some platforms don't allow
+ writing against a mounted device regardless of this option. Default: false.
.. option:: pre_read=bool
given I/O operation. This will also clear the :option:`invalidate` flag,
since it is pointless to pre-read and then drop the cache. This will only
work for I/O engines that are seek-able, since they allow you to read the
- same data multiple times. Thus it will not work on e.g. network or splice I/O.
+ same data multiple times. Thus it will not work on non-seekable I/O engines
+ (e.g. network, splice). Default: false.
.. option:: unlink=bool
Unlink the job files when done. Not the default, as repeated runs of that
- job would then waste time recreating the file set again and again.
+ job would then waste time recreating the file set again and again. Default:
+ false.
.. option:: unlink_each_loop=bool
- Unlink job files after each iteration or loop.
+ Unlink job files after each iteration or loop. Default: false.
.. option:: zonesize=int
.. option:: offset=int
- Start I/O at the given offset in the file. The data before the given offset
- will not be touched. This effectively caps the file size at `real_size -
- offset`. Can be combined with :option:`size` to constrain the start and
- end range that I/O will be done within.
+ Start I/O at the provided offset in the file, given as either a fixed size or
+ a percentage. If a percentage is given, the next ``blockalign``-ed offset
+ will be used. Data before the given offset will not be touched. This
+ effectively caps the file size at `real_size - offset`. Can be combined with
+ :option:`size` to constrain the start and end range of the I/O workload.
.. option:: offset_increment=int
blocks given. For example, if you give 32 as a parameter, fio will sync the
file for every 32 writes issued. If fio is using non-buffered I/O, we may
not sync the file. The exception is the sg I/O engine, which synchronizes
- the disk cache anyway.
+ the disk cache anyway. Defaults to 0, which means no sync every certain
+ number of writes.
.. option:: fdatasync=int
Like :option:`fsync` but uses :manpage:`fdatasync(2)` to only sync data and
not metadata blocks. In Windows, FreeBSD, and DragonFlyBSD there is no
:manpage:`fdatasync(2)`, this falls back to using :manpage:`fsync(2)`.
+ Defaults to 0, which means no sync data every certain number of writes.
.. option:: write_barrier=int
If true, writes to a file will always overwrite existing data. If the file
doesn't already exist, it will be created before the write phase begins. If
the file exists and is large enough for the specified write phase, nothing
- will be done.
+ will be done. Default: false.
.. option:: end_fsync=bool
- If true, fsync file contents when a write stage has completed.
+ If true, :manpage:`fsync(2)` file contents when a write stage has completed.
+ Default: false.
.. option:: fsync_on_close=bool
If true, fio will :manpage:`fsync(2)` a dirty file on close. This differs
- from end_fsync in that it will happen on every file close, not just at the
- end of the job.
+ from :option:`end_fsync` in that it will happen on every file close, not
+ just at the end of the job. Default: false.
.. option:: rwmixread=int
means default for reads, 8k for writes and trims.
**bs=,8k,**
- means default for reads, 8k for writes, and default for writes.
+ means default for reads, 8k for writes, and default for trims.
.. option:: blocksize_range=irange[,irange][,irange], bsrange=irange[,irange][,irange]
.. option:: buffer_pattern=str
- If set, fio will fill the I/O buffers with this pattern. If not set, the
- contents of I/O buffers is defined by the other options related to buffer
- contents. The setting can be any pattern of bytes, and can be prefixed with
- 0x for hex values. It may also be a string, where the string must then be
- wrapped with ``""``, e.g.::
+ If set, fio will fill the I/O buffers with this pattern or with the contents
+ of a file. If not set, the contents of I/O buffers are defined by the other
+ options related to buffer contents. The setting can be any pattern of bytes,
+ and can be prefixed with 0x for hex values. It may also be a string, where
+ the string must then be wrapped with ``""``. Or it may also be a filename,
+ where the filename must be wrapped with ``''`` in which case the file is
+ opened and read. Note that not all the file contents will be read if that
+ would cause the buffers to overflow. So, for example::
+
+ buffer_pattern='filename'
+
+ or::
buffer_pattern="abcd"
Also you can combine everything together in any order::
- buffer_pattern=0xdeadface"abcd"-12
+ buffer_pattern=0xdeadface"abcd"-12'filename'
.. option:: dedupe_percentage=int
.. option:: invalidate=bool
Invalidate the buffer/page cache parts for this file prior to starting
- I/O. Defaults to true.
+ I/O if the platform and file type support it. Defaults to true.
+ This will be ignored if :option:`pre_read` is also specified for the
+ same job.
.. option:: sync=bool
**mmapshared**
Same as mmap, but use a MMAP_SHARED mapping.
+ **cudamalloc**
+ Use GPU memory as the buffers for GPUDirect RDMA benchmark.
+
The area allocated is a function of the maximum allowed bs size for the job,
multiplied by the I/O depth given. Note that for **shmhuge** and
**mmaphuge** to work, the system must have free huge pages allocated. This
**sync**
Basic :manpage:`read(2)` or :manpage:`write(2)`
I/O. :manpage:`lseek(2)` is used to position the I/O location.
+ See :option:`fsync` and :option:`fdatasync` for syncing write I/Os.
**psync**
Basic :manpage:`pread(2)` or :manpage:`pwrite(2)` I/O. Default on
DDIR_TRIM
does fallocate(,mode = FALLOC_FL_KEEP_SIZE|FALLOC_FL_PUNCH_HOLE).
+ **ftruncate**
+ I/O engine that sends :manpage:`ftruncate(2)` operations in response
+ to write (DDIR_WRITE) events. Each ftruncate issued sets the file's
+ size to the current block offset. Block size is ignored.
+
**e4defrag**
I/O engine that does regular EXT4_IOC_MOVE_EXT ioctls to simulate
defragment activity in request to DDIR_WRITE event.
.. option:: cpuload=int : [cpuio]
- Attempt to use the specified percentage of CPU cycles.
+ Attempt to use the specified percentage of CPU cycles. This is a mandatory
+ option when using cpuio I/O engine.
.. option:: cpuchunks=int : [cpuio]
Stall the job for the specified period of time after an I/O has completed before issuing the
next. May be used to simulate processing being done by an application.
- When the unit is omitted, the value is given in microseconds. See
+ When the unit is omitted, the value is interpreted in microseconds. See
:option:`thinktime_blocks` and :option:`thinktime_spin`.
.. option:: thinktime_spin=time
Only valid if :option:`thinktime` is set - pretend to spend CPU time doing
something with the data received, before falling back to sleeping for the
rest of the period specified by :option:`thinktime`. When the unit is
- omitted, the value is given in microseconds.
+ omitted, the value is interpreted in microseconds.
.. option:: thinktime_blocks=int
If set, fio will attempt to find the max performance point that the given
workload will run at while maintaining a latency below this target. When
- the unit is omitted, the value is given in microseconds. See
+ the unit is omitted, the value is interpreted in microseconds. See
:option:`latency_window` and :option:`latency_percentile`.
.. option:: latency_window=time
Used with :option:`latency_target` to specify the sample window that the job
is run at varying queue depths to test the performance. When the unit is
- omitted, the value is given in microseconds.
+ omitted, the value is interpreted in microseconds.
.. option:: latency_percentile=float
.. option:: max_latency=time
If set, fio will exit the job with an ETIMEDOUT error if it exceeds this
- maximum latency. When the unit is omitted, the value is given in
+ maximum latency. When the unit is omitted, the value is interpreted in
microseconds.
.. option:: rate_cycle=int
Average bandwidth for :option:`rate` and :option:`rate_min` over this number
- of milliseconds.
+ of milliseconds. Defaults to 1000.
I/O replay
contents to one or more separate threads. If using this offload option, even
sync I/O engines can benefit from using an :option:`iodepth` setting higher
than 1, as it allows them to have I/O in flight while verifies are running.
+ Defaults to 0 async threads, i.e. verification is not asynchronous.
.. option:: verify_async_cpus=str
<type> is "local" for a local run, "sock" for a client/server socket
connection, and "ip" (192.168.0.1, for instance) for a networked
- client/server connection.
+ client/server connection. Defaults to true.
.. option:: verify_state_load=bool
If a verify termination trigger was used, fio stores the current write state
of each thread. This can be used at verification time so that fio knows how
far it should verify. Without this information, fio will run a full
- verification pass, according to the settings in the job file used.
+ verification pass, according to the settings in the job file used. Default
+ false.
.. option:: trim_percentage=int
A rolling window of this duration will be used to judge whether steady state
has been reached. Data will be collected once per second. The default is 0
which disables steady state detection. When the unit is omitted, the
- value is given in seconds.
+ value is interpreted in seconds.
.. option:: steadystate_ramp_time=time, ss_ramp=time
Allow the job to run for the specified duration before beginning data
collection for checking the steady state job termination criterion. The
- default is 0. When the unit is omitted, the value is given in seconds.
+ default is 0. When the unit is omitted, the value is interpreted in seconds.
Measurements and reporting
all jobs in a file will be part of the same reporting group, unless
separated by a :option:`stonewall`.
+.. option:: stats
+
+ By default, fio collects and shows final output results for all jobs
+ that run. If this option is set to 0, then fio will ignore it in
+ the final stat output.
+
.. option:: write_bw_log=str
If given, write a bandwidth log for this job. Can be used to store data of
.. option:: ignore_error=str
Sometimes you want to ignore some errors during test in that case you can
- specify error list for each error type.
+ specify error list for each error type, instead of only being able to
+ ignore the default 'non-fatal error' using :option:`continue_on_error`.
``ignore_error=READ_ERR_LIST,WRITE_ERR_LIST,VERIFY_ERR_LIST`` errors for
given error type is separated with ':'. Error may be symbol ('ENOSPC',
'ENOMEM') or integer. Example::
ignore_error=EAGAIN,ENOSPC:122
This option will ignore EAGAIN from READ, and ENOSPC and 122(EDQUOT) from
- WRITE.
+ WRITE. This option works by overriding :option:`continue_on_error` with
+ the list of errors for each error type if any.
.. option:: error_dump=bool
.. option:: test-duration=time
:noindex:
- How long the entire test takes to run. Default: 24h.
+ How long the entire test takes to run. When the unit is omitted, the value
+ is given in seconds. Default: 24h.
.. option:: threads-per-queue=int
:noindex:
changed for some reason, this number will be incremented by 1 to signify that
change.
-Split up, the format is as follows:
+Split up, the format is as follows (comments in brackets denote when a
+field was introduced or whether its specific to some terse version):
::
- terse version, fio version, jobname, groupid, error
+ terse version, fio version [v3], jobname, groupid, error
READ status::
Completion latency: min, max, mean, stdev (usec)
Completion latency percentiles: 20 fields (see below)
Total latency: min, max, mean, stdev (usec)
- Bw (KiB/s): min, max, aggregate percentage of total, mean, stdev
+ Bw (KiB/s): min, max, aggregate percentage of total, mean, stdev, number of samples [v5]
+ IOPS [v5]: min, max, mean, stdev, number of samples
WRITE status:
Total IO (KiB), bandwidth (KiB/sec), IOPS, runtime (msec)
Submission latency: min, max, mean, stdev (usec)
- Completion latency: min, max, mean, stdev(usec)
+ Completion latency: min, max, mean, stdev (usec)
Completion latency percentiles: 20 fields (see below)
Total latency: min, max, mean, stdev (usec)
- Bw (KiB/s): min, max, aggregate percentage of total, mean, stdev
+ Bw (KiB/s): min, max, aggregate percentage of total, mean, stdev, number of samples [v5]
+ IOPS [v5]: min, max, mean, stdev, number of samples
+
+ TRIM status [all but version 3]:
+
+ Fields are similar to READ/WRITE status.
CPU usage::
<=2, 4, 10, 20, 50, 100, 250, 500, 750, 1000, 2000, >=2000
- Disk utilization::
+ Disk utilization [v3]::
Disk name, Read ios, write ios,
Read merges, write merges,
For disk utilization, all disks used by fio are shown. So for each disk there
will be a disk utilization section.
+Below is a single line containing short names for each of the fields in the
+minimal output v3, separated by semicolons:
+
+terse_version_3;fio_version;jobname;groupid;error;read_kb;read_bandwidth;read_iops;read_runtime_ms;read_slat_min;read_slat_max;read_slat_mean;read_slat_dev;read_clat_max;read_clat_min;read_clat_mean;read_clat_dev;read_clat_pct01;read_clat_pct02;read_clat_pct03;read_clat_pct04;read_clat_pct05;read_clat_pct06;read_clat_pct07;read_clat_pct08;read_clat_pct09;read_clat_pct10;read_clat_pct11;read_clat_pct12;read_clat_pct13;read_clat_pct14;read_clat_pct15;read_clat_pct16;read_clat_pct17;read_clat_pct18;read_clat_pct19;read_clat_pct20;read_tlat_min;read_lat_max;read_lat_mean;read_lat_dev;read_bw_min;read_bw_max;read_bw_agg_pct;read_bw_mean;read_bw_dev;write_kb;write_bandwidth;write_iops;write_runtime_ms;write_slat_min;write_slat_max;write_slat_mean;write_slat_dev;write_clat_max;write_clat_min;write_clat_mean;write_clat_dev;write_clat_pct01;write_clat_pct02;write_clat_pct03;write_clat_pct04;write_clat_pct05;write_clat_pct06;write_clat_pct07;write_clat_pct08;write_clat_pct09;write_clat_pct10;write_clat_pct11;write_clat_pct12;write_clat_pct13;write_clat_pct14;write_clat_pct15;write_clat_pct16;write_clat_pct17;write_clat_pct18;write_clat_pct19;write_clat_pct20;write_tlat_min;write_lat_max;write_lat_mean;write_lat_dev;write_bw_min;write_bw_max;write_bw_agg_pct;write_bw_mean;write_bw_dev;cpu_user;cpu_sys;cpu_csw;cpu_mjf;pu_minf;iodepth_1;iodepth_2;iodepth_4;iodepth_8;iodepth_16;iodepth_32;iodepth_64;lat_2us;lat_4us;lat_10us;lat_20us;lat_50us;lat_100us;lat_250us;lat_500us;lat_750us;lat_1000us;lat_2ms;lat_4ms;lat_10ms;lat_20ms;lat_50ms;lat_100ms;lat_250ms;lat_500ms;lat_750ms;lat_1000ms;lat_2000ms;lat_over_2000ms;disk_name;disk_read_iops;disk_write_iops;disk_read_merges;disk_write_merges;disk_read_ticks;write_ticks;disk_queue_time;disk_util
+
Trace file format
-----------------
Verification trigger example
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-Lets say we want to run a powercut test on the remote machine 'server'. Our
+Let's say we want to run a powercut test on the remote machine 'server'. Our
write workload is in :file:`write-test.fio`. We want to cut power to 'server' at
some point during the run, and we'll run this test from the safety or our local
machine, 'localbox'. On the server, we'll start the fio backend normally::
will work, but it's not **really** cutting power to the server, it's merely
abruptly rebooting it. If we have a remote way of cutting power to the server
through IPMI or similar, we could do that through a local trigger command
-instead. Lets assume we have a script that does IPMI reboot of a given hostname,
+instead. Let's assume we have a script that does IPMI reboot of a given hostname,
ipmi-reboot. On localbox, we could then have run fio with a local trigger
instead::
Loading verify state
~~~~~~~~~~~~~~~~~~~~
-To load store write state, read verification job file must contain the
+To load stored write state, a read verification job file must contain the
:option:`verify_state_load` option. If that is set, fio will load the previously
stored state. For a local fio run this is done by loading the files directly,
and on a client/server run, the server backend will ask the client to send the
In order to let ``fio --client`` runs use a shared filesystem from multiple
hosts, ``fio --client`` now prepends the IP address of the server to the
-filename. For example, if fio is using directory :file:`/mnt/nfs/fio` and is
+filename. For example, if fio is using the directory :file:`/mnt/nfs/fio` and is
writing filename :file:`fileio.tmp`, with a :option:`--client` `hostfile`
containing two hostnames ``h1`` and ``h2`` with IP addresses 192.168.10.120 and
192.168.10.121, then fio will create two files::
projects / fio.git / blobdiff