calls will be excluded from other uses. Fio will manually clear it from the
CPU mask of other jobs.
+.. option:: job_start_clock_id=int
+
+ The clock_id passed to the call to `clock_gettime` used to record
+ job_start in the `json` output format. Default is 0, or CLOCK_REALTIME.
+
Target file/device
~~~~~~~~~~~~~~~~~~
On Windows, disk devices are accessed as :file:`\\\\.\\PhysicalDrive0` for
the first device, :file:`\\\\.\\PhysicalDrive1` for the second etc.
- Note: Windows and FreeBSD prevent write access to areas
+ Note: Windows and FreeBSD (refer to geom(4)) prevent write access to areas
of the disk containing in-use data (e.g. filesystems).
The filename "`-`" is a reserved name, meaning *stdin* or *stdout*. Which
.. option:: unlink=bool
- Unlink the job files when done. Not the default, as repeated runs of that
+ Unlink (delete) the job files when done. Not the default, as repeated runs of that
job would then waste time recreating the file set again and again. Default:
false.
.. option:: unlink_each_loop=bool
- Unlink job files after each iteration or loop. Default: false.
+ Unlink (delete) job files after each iteration or loop. Default: false.
.. option:: zonemode=str
**none**
The :option:`zonerange`, :option:`zonesize`,
- :option `zonecapacity` and option:`zoneskip`
+ :option:`zonecapacity` and :option:`zoneskip`
parameters are ignored.
**strided**
I/O happens in a single zone until
:option:`zonesize` bytes have been transferred.
After that number of bytes has been
transferred processing of the next zone
- starts. :option `zonecapacity` is ignored.
+ starts. :option:`zonecapacity` is ignored.
**zbd**
Zoned block device mode. I/O happens
sequentially in each zone, even if random I/O
.. option:: max_open_zones=int
- A zone of a zoned block device is in the open state when it is partially
- written (i.e. not all sectors of the zone have been written). Zoned
- block devices may have a limit on the total number of zones that can
- be simultaneously in the open state, that is, the number of zones that
- can be written to simultaneously. The :option:`max_open_zones` parameter
- limits the number of zones to which write commands are issued by all fio
- jobs, that is, limits the number of zones that will be in the open
- state. This parameter is relevant only if the :option:`zonemode` =zbd is
- used. The default value is always equal to maximum number of open zones
- of the target zoned block device and a value higher than this limit
- cannot be specified by users unless the option
- :option:`ignore_zone_limits` is specified. When
- :option:`ignore_zone_limits` is specified or the target device has no
- limit on the number of zones that can be in an open state,
- :option:`max_open_zones` can specify 0 to disable any limit on the
- number of zones that can be simultaneously written to by all jobs.
+ When a zone of a zoned block device is partially written (i.e. not all
+ sectors of the zone have been written), the zone is in one of three
+ conditions: 'implicit open', 'explicit open' or 'closed'. Zoned block
+ devices may have a limit called 'max_open_zones' (same name as the
+ parameter) on the total number of zones that can simultaneously be in
+ the 'implicit open' or 'explicit open' conditions. Zoned block devices
+ may have another limit called 'max_active_zones', on the total number of
+ zones that can simultaneously be in the three conditions. The
+ :option:`max_open_zones` parameter limits the number of zones to which
+ write commands are issued by all fio jobs, that is, limits the number of
+ zones that will be in the conditions. When the device has the
+ max_open_zones limit and does not have the max_active_zones limit, the
+ :option:`max_open_zones` parameter limits the number of zones in the two
+ open conditions up to the limit. In this case, fio includes zones in the
+ two open conditions to the write target zones at fio start. When the
+ device has both the max_open_zones and the max_active_zones limits, the
+ :option:`max_open_zones` parameter limits the number of zones in the
+ three conditions up to the limit. In this case, fio includes zones in
+ the three conditions to the write target zones at fio start.
+
+ This parameter is relevant only if the :option:`zonemode` =zbd is used.
+ The default value is always equal to the max_open_zones limit of the
+ target zoned block device and a value higher than this limit cannot be
+ specified by users unless the option :option:`ignore_zone_limits` is
+ specified. When :option:`ignore_zone_limits` is specified or the target
+ device does not have the max_open_zones limit, :option:`max_open_zones`
+ can specify 0 to disable any limit on the number of zones that can be
+ simultaneously written to by all jobs.
.. option:: job_max_open_zones=int
Comma-separated ranges may be specified for reads, writes, and trims as
described in :option:`blocksize`.
- Example: ``bsrange=1k-4k,2k-8k``.
+ Example: ``bsrange=1k-4k,2k-8k`` also the ':' delimiter ``bsrange=1k:4k,2k:8k``.
.. option:: bssplit=str[,str][,str]
.. option:: ioengine=str
- Defines how the job issues I/O to the file. The following types are defined:
+ fio supports 2 kinds of performance measurement: I/O and file/directory operation.
+
+ I/O engines define how the job issues I/O to the file. The following types are defined:
**sync**
Basic :manpage:`read(2)` or :manpage:`write(2)`
absolute or relative. See :file:`engines/skeleton_external.c` for
details of writing an external I/O engine.
- **filecreate**
- Simply create the files and do no I/O to them. You still need to
- set `filesize` so that all the accounting still occurs, but no
- actual I/O will be done other than creating the file.
-
- **filestat**
- Simply do stat() and do no I/O to the file. You need to set 'filesize'
- and 'nrfiles', so that files will be created.
- This engine is to measure file lookup and meta data access.
-
- **filedelete**
- Simply delete the files by unlink() and do no I/O to them. You need to set 'filesize'
- and 'nrfiles', so that the files will be created.
- This engine is to measure file delete.
-
**libpmem**
Read and write using mmap I/O to a file on a filesystem
mounted with DAX on a persistent memory device through the PMDK
several instances to access the same device or file
simultaneously, but allow it for threads.
+ File/directory operation engines define how the job operates file or directory. The
+ following types are defined:
+
+ **filecreate**
+ Simply create the files and do no I/O to them. You still need to
+ set `filesize` so that all the accounting still occurs, but no
+ actual I/O will be done other than creating the file.
+ Example job file: filecreate-ioengine.fio.
+
+ **filestat**
+ Simply do stat() and do no I/O to the file. You need to set 'filesize'
+ and 'nrfiles', so that files will be created.
+ This engine is to measure file lookup and meta data access.
+ Example job file: filestat-ioengine.fio.
+
+ **filedelete**
+ Simply delete the files by unlink() and do no I/O to them. You need to set 'filesize'
+ and 'nrfiles', so that the files will be created.
+ This engine is to measure file delete.
+ Example job file: filedelete-ioengine.fio.
+
+ **dircreate**
+ Simply create the directories and do no I/O to them. You still need to
+ set `filesize` so that all the accounting still occurs, but no
+ actual I/O will be done other than creating the directories.
+ Example job file: dircreate-ioengine.fio.
+
+ **dirstat**
+ Simply do stat() and do no I/O to the directories. You need to set 'filesize'
+ and 'nrfiles', so that directories will be created.
+ This engine is to measure directory lookup and meta data access.
+ Example job file: dirstat-ioengine.fio.
+
+ **dirdelete**
+ Simply delete the directories by rmdir() and do no I/O to them. You need to set 'filesize'
+ and 'nrfiles', so that the directories will be created.
+ This engine is to measure directory delete.
+ Example job file: dirdelete-ioengine.fio.
+
+ For file and directory operation engines, there is no I/O throughput, then the
+ statistics data in report have different meanings. The meaningful output indexes are: 'iops' and 'clat'.
+ 'bw' is meaningless. Refer to section: "Interpreting the output" for more details.
+
+
I/O engine specific parameters
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
reads and writes. See :manpage:`ionice(1)`. See also the
:option:`prioclass` option.
+.. option:: cmdprio_hint=int[,int] : [io_uring] [libaio]
+
+ Set the I/O priority hint to use for I/Os that must be issued with
+ a priority when :option:`cmdprio_percentage` or
+ :option:`cmdprio_bssplit` is set. If not specified when
+ :option:`cmdprio_percentage` or :option:`cmdprio_bssplit` is set,
+ this defaults to 0 (no hint). A single value applies to reads and
+ writes. Comma-separated values may be specified for reads and writes.
+ See also the :option:`priohint` option.
+
.. option:: cmdprio=int[,int] : [io_uring] [libaio]
Set the I/O priority value to use for I/Os that must be issued with
cmdprio_bssplit=blocksize/percentage:blocksize/percentage
- In this case, each entry will use the priority class and priority
- level defined by the options :option:`cmdprio_class` and
- :option:`cmdprio` respectively.
+ In this case, each entry will use the priority class, priority hint
+ and priority level defined by the options :option:`cmdprio_class`,
+ :option:`cmdprio` and :option:`cmdprio_hint` respectively.
The second accepted format for this option is:
accepted format does not restrict all entries to have the same priority
class and priority level.
- For both formats, only the read and write data directions are supported,
+ The third accepted format for this option is:
+
+ cmdprio_bssplit=blocksize/percentage/class/level/hint:...
+
+ This is an extension of the second accepted format that allows one to
+ also specify a priority hint.
+
+ For all formats, only the read and write data directions are supported,
values for trim IOs are ignored. This option is mutually exclusive with
the :option:`cmdprio_percentage` option.
Enable Flexible Data Placement mode for write commands.
-.. option:: fdp_pli_select=str : [io_uring_cmd] [xnvme]
+.. option:: dataplacement=str : [io_uring_cmd] [xnvme]
+
+ Specifies the data placement directive type to use for write commands.
+ The following types are supported:
+
+ **none**
+ Do not use a data placement directive. This is the
+ default.
+
+ **fdp**
+ Use Flexible Data Placement directives for write
+ commands. This is equivalent to specifying
+ :option:`fdp` =1.
+
+ **streams**
+ Use Streams directives for write commands.
+
+.. option:: plid_select=str, fdp_pli_select=str : [io_uring_cmd] [xnvme]
Defines how fio decides which placement ID to use next. The following
types are defined:
Round robin over available placement IDs. This is the
default.
- The available placement ID index/indices is defined by the option
- :option:`fdp_pli`.
+ The available placement ID (indices) are defined by the option
+ :option:`plids`.
+
+.. option:: plids=str, fdp_pli=str : [io_uring_cmd] [xnvme]
+
+ Select which Placement IDs (streams) or Placement ID Indices (FDP) this
+ job is allowed to use for writes. For FDP by default, the job will
+ cycle through all available Placement IDs, so use this to isolate these
+ identifiers to specific jobs. If you want fio to use FDP placement
+ identifiers only at indices 0, 2 and 5 specify ``plids=0,2,5``. For
+ streams this should be a comma-separated list of Stream IDs.
+
+.. option:: md_per_io_size=int : [io_uring_cmd] [xnvme]
+
+ Size in bytes for separate metadata buffer per IO. Default: 0.
+
+.. option:: pi_act=int : [io_uring_cmd] [xnvme]
+
+ Action to take when nvme namespace is formatted with protection
+ information. If this is set to 1 and namespace is formatted with
+ metadata size equal to protection information size, fio won't use
+ separate metadata buffer or extended logical block. If this is set to
+ 1 and namespace is formatted with metadata size greater than protection
+ information size, fio will not generate or verify the protection
+ information portion of metadata for write or read case respectively.
+ If this is set to 0, fio generates protection information for
+ write case and verifies for read case. Default: 1.
+
+ For 16 bit CRC generation fio will use isa-l if available otherwise
+ it will use the default slower generator.
+ (see: https://github.com/intel/isa-l)
+
+.. option:: pi_chk=str[,str][,str] : [io_uring_cmd] [xnvme]
+
+ Controls the protection information check. This can take one or more
+ of these values. Default: none.
+
+ **GUARD**
+ Enables protection information checking of guard field.
+ **REFTAG**
+ Enables protection information checking of logical block
+ reference tag field.
+ **APPTAG**
+ Enables protection information checking of application tag field.
+
+.. option:: apptag=int : [io_uring_cmd] [xnvme]
+
+ Specifies logical block application tag value, if namespace is
+ formatted to use end to end protection information. Default: 0x1234.
-.. option:: fdp_pli=str : [io_uring_cmd] [xnvme]
+.. option:: apptag_mask=int : [io_uring_cmd] [xnvme]
- Select which Placement ID Index/Indicies this job is allowed to use for
- writes. By default, the job will cycle through all available Placement
- IDs, so use this to isolate these identifiers to specific jobs. If you
- want fio to use placement identifier only at indices 0, 2 and 5 specify
- ``fdp_pli=0,2,5``.
+ Specifies logical block application tag mask value, if namespace is
+ formatted to use end to end protection information. Default: 0xffff.
+
+.. option:: num_range=int : [io_uring_cmd]
+
+ For trim command this will be the number of ranges to trim per I/O
+ request. The number of logical blocks per range is determined by the
+ :option:`bs` option which should be a multiple of logical block size.
+ This cannot be used with read or write. Note that setting this
+ option > 1, :option:`log_offset` will not be able to log all the
+ offsets. Default: 1.
.. option:: cpuload=int : [cpuio]
User datagram protocol V6.
**unix**
UNIX domain socket.
+ **vsock**
+ VSOCK protocol.
- When the protocol is TCP or UDP, the port must also be given, as well as the
- hostname if the job is a TCP listener or UDP reader. For unix sockets, the
+ When the protocol is TCP, UDP or VSOCK, the port must also be given, as well as the
+ hostname if the job is a TCP or VSOCK listener or UDP reader. For unix sockets, the
normal :option:`filename` option should be used and the port is invalid.
+ When the protocol is VSOCK, the :option:`hostname` is the CID of the remote VM.
.. option:: listen : [netsplice] [net]
.. option:: sg_write_mode=str : [sg]
- Specify the type of write commands to issue. This option can take three values:
+ Specify the type of write commands to issue. This option can take ten values:
**write**
This is the default where write opcodes are issued as usual.
**hugepage**
Use hugepages, instead of existing posix memory backend. The
memory backend uses hugetlbfs. This require users to allocate
- hugepages, mount hugetlbfs and set an enviornment variable for
+ hugepages, mount hugetlbfs and set an environment variable for
XNVME_HUGETLB_PATH.
**spdk**
Uses SPDK's memory allocator.
creating but before connecting the libblkio instance. Each property must
have the format ``<name>=<value>``. Colons can be escaped as ``\:``.
These are set after the engine sets any other properties, so those can
- be overriden. Available properties depend on the libblkio version in use
+ be overridden. Available properties depend on the libblkio version in use
and are listed at
https://libblkio.gitlab.io/libblkio/blkio.html#properties
connecting but before starting the libblkio instance. Each property must
have the format ``<name>=<value>``. Colons can be escaped as ``\:``.
These are set after the engine sets any other properties, so those can
- be overriden. Available properties depend on the libblkio version in use
+ be overridden. Available properties depend on the libblkio version in use
and are listed at
https://libblkio.gitlab.io/libblkio/blkio.html#properties
I/O rate
~~~~~~~~
+.. option:: thinkcycles=int
+
+ Stall the job for the specified number of cycles after an I/O has completed before
+ issuing the next. May be used to simulate processing being done by an application.
+ This is not taken into account for the time to be waited on for :option:`thinktime`.
+ Might not have any effect on some platforms, this can be checked by trying a setting
+ a high enough amount of thinkcycles.
+
.. option:: thinktime=time
Stall the job for the specified period of time after an I/O has completed before issuing the
.. option:: rate_cycle=int
- Average bandwidth for :option:`rate` and :option:`rate_min` over this number
- of milliseconds. Defaults to 1000.
+ Average bandwidth for :option:`rate_min` and :option:`rate_iops_min`
+ over this number of milliseconds. Defaults to 1000.
I/O latency
priority setting, see I/O engine specific :option:`cmdprio_percentage`
and :option:`cmdprio_class` options.
+.. option:: priohint=int
+
+ Set the I/O priority hint. This is only applicable to platforms that
+ support I/O priority classes and to devices with features controlled
+ through priority hints, e.g. block devices supporting command duration
+ limits, or CDL. CDL is a way to indicate the desired maximum latency
+ of I/Os so that the device can optimize its internal command scheduling
+ according to the latency limits indicated by the user.
+
+ For per-I/O priority hint setting, see the I/O engine specific
+ :option:`cmdprio_hint` option.
+
.. option:: cpus_allowed=str
Controls the same options as :option:`cpumask`, but accepts a textual
By default, fio will continue running all other jobs when one job finishes.
Sometimes this is not the desired action. Setting ``exitall`` will
instead make fio terminate all jobs in the same group. The option
- ``exit_what`` allows to control which jobs get terminated when ``exitall`` is
- enabled. The default is ``group`` and does not change the behaviour of
+ ``exit_what`` allows one to control which jobs get terminated when ``exitall``
+ is enabled. The default is ``group`` and does not change the behaviour of
``exitall``. The setting ``all`` terminates all jobs. The setting ``stonewall``
terminates all currently running jobs across all groups and continues execution
with the next stonewalled group.
.. option:: per_job_logs=bool
- If set, this generates bw/clat/iops log with per file private filenames. If
- not set, jobs with identical names will share the log filename. Default:
- true.
+ If set to true, fio generates bw/clat/iops logs with per job unique
+ filenames. If set to false, jobs with identical names will share a log
+ filename. Note that when this option is set to false log files will be
+ opened in append mode and if log files already exist the previous
+ contents will not be overwritten. Default: true.
.. option:: group_reporting
same reporting group, unless if separated by a :option:`stonewall`, or by
using :option:`new_group`.
+ NOTE: When :option:`group_reporting` is used along with `json` output,
+ there are certain per-job properties which can be different between jobs
+ but do not have a natural group-level equivalent. Examples include
+ `kb_base`, `unit_base`, `sig_figs`, `thread_number`, `pid`, and
+ `job_start`. For these properties, the values for the first job are
+ recorded for the group.
+
.. option:: new_group
Start a new reporting group. See: :option:`group_reporting`. If not given,
.. option:: log_avg_msec=int
- By default, fio will log an entry in the iops, latency, or bw log for every
- I/O that completes. When writing to the disk log, that can quickly grow to a
- very large size. Setting this option makes fio average the each log entry
- over the specified period of time, reducing the resolution of the log. See
- :option:`log_max_value` as well. Defaults to 0, logging all entries.
- Also see `Log File Formats`_.
+ By default, fio will log an entry in the iops, latency, or bw log for
+ every I/O that completes. When writing to the disk log, that can
+ quickly grow to a very large size. Setting this option directs fio to
+ instead record an average over the specified duration for each log
+ entry, reducing the resolution of the log. When the job completes, fio
+ will flush any accumulated latency log data, so the final log interval
+ may not match the value specified by this option and there may even be
+ duplicate timestamps. See :option:`log_window_value` as well. Defaults
+ to 0, logging entries for each I/O. Also see `Log File Formats`_.
.. option:: log_hist_msec=int
histogram logs contain 1216 latency bins. See :option:`write_hist_log`
and `Log File Formats`_.
-.. option:: log_max_value=bool
+.. option:: log_window_value=str, log_max_value=str
+
+ If :option:`log_avg_msec` is set, fio by default logs the average over that
+ window. This option determines whether fio logs the average, maximum or
+ both the values over the window. This only affects the latency logging,
+ as both average and maximum values for iops or bw log will be same.
+ Accepted values are:
+
+ **avg**
+ Log average value over the window. The default.
+
+ **max**
+ Log maximum value in the window.
- If :option:`log_avg_msec` is set, fio logs the average over that window. If
- you instead want to log the maximum value, set this option to 1. Defaults to
- 0, meaning that averaged values are logged.
+ **both**
+ Log both average and maximum value over the window.
+
+ **0**
+ Backward-compatible alias for **avg**.
+
+ **1**
+ Backward-compatible alias for **max**.
.. option:: log_offset=bool
.. option:: log_unix_epoch=bool
- If set, fio will log Unix timestamps to the log files produced by enabling
- write_type_log for each log type, instead of the default zero-based
- timestamps.
+ Backwards compatible alias for log_alternate_epoch.
.. option:: log_alternate_epoch=bool
.. option:: log_alternate_epoch_clock_id=int
- Specifies the clock_id to be used by clock_gettime to obtain the alternate epoch
- if either log_unix_epoch or log_alternate_epoch are true. Otherwise has no
- effect. Default value is 0, or CLOCK_REALTIME.
+ Specifies the clock_id to be used by clock_gettime to obtain the alternate
+ epoch if log_alternate_epoch is true. Otherwise has no effect. Default
+ value is 0, or CLOCK_REALTIME.
.. option:: block_error_percentiles=bool
commit if available) functions were completed to when the I/O's
completion was reaped by fio.
+ For file and directory operation engines, **clat** denotes the time
+ to complete one file or directory operation.
+
+ **filecreate engine**:the time cost to create a new file
+
+ **filestat engine**: the time cost to look up an existing file
+
+ **filedelete engine**:the time cost to delete a file
+
+ **dircreate engine**: the time cost to create a new directory
+
+ **dirstat engine**: the time cost to look up an existing directory
+
+ **dirdelete engine**: the time cost to delete a directory
+
**lat**
Total latency. Same names as slat and clat, this denotes the time from
when fio created the I/O unit to completion of the I/O operation.
are on the same disk, since they are then competing for disk
access.
+ For file and directory operation engines, **bw** is meaningless.
+
**iops**
IOPS statistics based on measurements from discrete intervals.
For details see the description for bw above. See
:option:`iopsavgtime` to control the duration of the intervals.
Same values reported here as for bw except for percentage.
+ For file and directory operation engines, **iops** is the most
+ fundamental index to denote the performance.
+ It means how many files or directories can be operated per second.
+
+ **filecreate engine**:number of files can be created per second
+
+ **filestat engine**: number of files can be looked up per second
+
+ **filedelete engine**:number of files can be deleted per second
+
+ **dircreate engine**: number of directories can be created per second
+
+ **dirstat engine**: number of directories can be looked up per second
+
+ **dirdelete engine**: number of directories can be deleted per second
+
**lat (nsec/usec/msec)**
The distribution of I/O completion latencies. This is the time from when
I/O leaves fio and when it gets completed. Unlike the separate
by the ioengine specific :option:`cmdprio_percentage`.
Fio defaults to logging every individual I/O but when windowed logging is set
-through :option:`log_avg_msec`, either the average (by default) or the maximum
-(:option:`log_max_value` is set) *value* seen over the specified period of time
-is recorded. Each *data direction* seen within the window period will aggregate
-its values in a separate row. Further, when using windowed logging the *block
-size* and *offset* entries will always contain 0.
+through :option:`log_avg_msec`, either the average (by default), the maximum
+(:option:`log_window_value` is set to max) *value* seen over the specified period
+of time, or both the average *value* and maximum *value1* (:option:`log_window_value`
+is set to both) is recorded. The log file format when both the values are reported
+takes this form:
+
+ *time* (`msec`), *value*, *value1*, *data direction*, *block size* (`bytes`),
+ *offset* (`bytes`), *command priority*
+
+
+Each *data direction* seen within the window period will aggregate its values in a
+separate row. Further, when using windowed logging the *block size* and *offset*
+entries will always contain 0.
Client/Server
server. The `server` string follows the same format as it does on the server
side, to allow IP/hostname/socket and port strings.
+Note that all job options must be defined in job files when running fio as a
+client. Any job options specified in `remote-args` will be ignored.
+
Fio can connect to multiple servers this way::
fio --client=<server1> <job file(s)> --client=<server2> <job file(s)>
projects / fio.git / blobdiff