7. Terse output
8. Trace file format
9. CPU idleness profiling
+10. Verification and triggers
+11. Log File Formats
+
1.0 Overview and history
------------------------
IO engine How do we issue io? We could be memory mapping the
file, we could be using regular read/write, we
- could be using splice, async io, syslet, or even
- SG (SCSI generic sg).
+ could be using splice, async io, or even SG
+ (SCSI generic sg).
IO depth If the io engine is async, how large a queuing
depth do we want to maintain?
special purpose of also signaling the start of a new
job.
+wait_for=str Specifies the name of the already defined job to wait
+ for. Single waitee name only may be specified. If set, the job
+ won't be started until all workers of the waitee job are done.
+
+ Wait_for operates on the job name basis, so there are a few
+ limitations. First, the waitee must be defined prior to the
+ waiter job (meaning no forward references). Second, if a job
+ is being referenced as a waitee, it must have a unique name
+ (no duplicate waitees).
+
description=str Text description of the job. Doesn't do anything except
dump this text description when this job is run. It's
not parsed.
default of $jobname.$jobnum.$filenum will be used if
no other format specifier is given.
+unique_filename=bool To avoid collisions between networked clients, fio
+ defaults to prefixing any generated filenames (with a directory
+ specified) with the source of the client connecting. To disable
+ this behavior, set this option to 0.
+
opendir=str Tell fio to recursively add any file it can find in this
directory and down the file system tree.
the next. Multiple files can still be
open depending on 'openfiles'.
- The string can have a number appended, indicating how
- often to switch to a new file. So if option random:4 is
- given, fio will switch to a new random file after 4 ios
- have been issued.
+ zipf Use a zipfian distribution to decide what file
+ to access.
+
+ pareto Use a pareto distribution to decide what file
+ to access.
+
+ gauss Use a gaussian (normal) distribution to decide
+ what file to access.
+
+ For random, roundrobin, and sequential, a postfix can be
+ appended to tell fio how many I/Os to issue before switching
+ to a new file. For example, specifying
+ 'file_service_type=random:8' would cause fio to issue 8 I/Os
+ before selecting a new file at random. For the non-uniform
+ distributions, a floating point postfix can be given to
+ influence how the distribution is skewed. See
+ 'random_distribution' for a description of how that would work.
ioengine=str Defines how the job issues io to the file. The following
types are defined:
vsync Basic readv(2) or writev(2) IO.
- psyncv Basic preadv(2) or pwritev(2) IO.
+ pvsync Basic preadv(2) or pwritev(2) IO.
+
+ psync2 Basic preadv2(2) or pwritev2(2) IO.
libaio Linux native asynchronous io. Note that Linux
may only support queued behaviour with
vmsplice(2) to transfer data from user
space to the kernel.
- syslet-rw Use the syslet system calls to make
- regular read/write async.
-
sg SCSI generic sg v3 io. May either be
synchronous using the SG_IO ioctl, or if
the target is an sg character device
defines engine specific options.
libhdfs Read and write through Hadoop (HDFS).
- The 'filename' option is used to specify host,
- port of the hdfs name-node to connect. This
- engine interprets offsets a little
+ This engine interprets offsets a little
differently. In HDFS, files once created
cannot be modified. So random writes are not
possible. To imitate this, libhdfs engine
- expects bunch of small files to be created
- over HDFS, and engine will randomly pick a
- file out of those files based on the offset
- generated by fio backend. (see the example
- job file to create such files, use rw=write
- option). Please note, you might want to set
- necessary environment variables to work with
- hdfs/libhdfs properly.
+ creates bunch of small files, and engine will
+ pick a file out of those files based on the
+ offset enerated by fio backend. Each jobs uses
+ it's own connection to HDFS.
mtd Read, write and erase an MTD character device
(e.g., /dev/mtd0). Discards are treated as
overwriting. The writetrim mode works well
for this constraint.
+ pmemblk Read and write through the NVML libpmemblk
+ interface.
+
external Prefix to specify loading an external
IO engine object file. Append the engine
filename, eg ioengine=external:/tmp/foo.o
random Uniform random distribution
zipf Zipf distribution
pareto Pareto distribution
+ gauss Normal (guassian) distribution
+ zoned Zoned random distribution
When using a zipf or pareto distribution, an input value
is also needed to define the access pattern. For zipf, this
what the given input values will yield in terms of hit rates.
If you wanted to use zipf with a theta of 1.2, you would use
random_distribution=zipf:1.2 as the option. If a non-uniform
- model is used, fio will disable use of the random map.
+ model is used, fio will disable use of the random map. For
+ the gauss distribution, a normal deviation is supplied as
+ a value between 0 and 100.
+
+ For a zoned distribution, fio supports specifying percentages
+ of IO access that should fall within what range of the file or
+ device. For example, given a criteria of:
+
+ 60% of accesses should be to the first 10%
+ 30% of accesses should be to the next 20%
+ 8% of accesses should be to to the next 30%
+ 2% of accesses should be to the next 40%
+
+ we can define that through zoning of the random accesses. For
+ the above example, the user would do:
+
+ random_distribution=zoned:60/10:30/20:8/30:2/40
+
+ similarly to how bssplit works for setting ranges and
+ percentages of block sizes. Like bssplit, it's possible to
+ specify separate zones for reads, writes, and trims. If just
+ one set is given, it'll apply to all of them.
percentage_random=int For a random workload, set how big a percentage should
be random. This defaults to 100%, in which case the workload
typically good enough. LFSR only works with single
block sizes, not with workloads that use multiple block
sizes. If used with such a workload, fio may read or write
- some blocks multiple times.
+ some blocks multiple times. The default value is tausworthe,
+ unless the required space exceeds 2^32 blocks. If it does,
+ then tausworthe64 is selected automatically.
nice=int Run the job with the given nice value. See man nice(2).
the job to exit. The same format as rate is used for read vs
write separation.
-rate_poisson=bool Real world random request flow follows Poisson process
- (https://en.wikipedia.org/wiki/Poisson_process). With
- rate_poisson=1, fio tries to simulate request flow under Poisson
- process (instead of even distribution, which is the default).
- Default: false.
+rate_process=str This option controls how fio manages rated IO
+ submissions. The default is 'linear', which submits IO in a
+ linear fashion with fixed delays between IOs that gets
+ adjusted based on IO completion rates. If this is set to
+ 'poisson', fio will submit IO based on a more real world
+ random request flow, known as the Poisson process
+ (https://en.wikipedia.org/wiki/Poisson_process). The lambda
+ will be 10^6 / IOPS for the given workload.
latency_target=int If set, fio will attempt to find the max performance
point that the given workload will run at while maintaining a
to wait for each job to finish, sometimes that is not the
desired action.
+exitall_on_error When one job finishes in error, terminate the rest. The
+ default is to wait for each job to finish.
+
bwavgtime=int Average the calculated bandwidth over the given time. Value
- is specified in milliseconds.
+ is specified in milliseconds. If the job also does bandwidth
+ logging through 'write_bw_log', then the minimum of this option
+ and 'log_avg_msec' will be used. Default: 500ms.
iopsavgtime=int Average the calculated IOPS over the given time. Value
- is specified in milliseconds.
+ is specified in milliseconds. If the job also does IOPS logging
+ through 'write_iops_log', then the minimum of this option and
+ 'log_avg_msec' will be used. Default: 500ms.
create_serialize=bool If true, serialize the file creating for the jobs.
This may be handy to avoid interleaving of data
filename. For this option, the suffix is _bw.x.log, where
x is the index of the job (1..N, where N is the number of
jobs). If 'per_job_logs' is false, then the filename will not
- include the job index.
+ include the job index. See 'Log File Formats'.
write_lat_log=str Same as write_bw_log, except that this option stores io
submission, completion, and total latencies instead. If no
and foo_lat.x.log, where x is the index of the job (1..N,
where N is the number of jobs). This helps fio_generate_plot
fine the logs automatically. If 'per_job_logs' is false, then
- the filename will not include the job index.
-
+ the filename will not include the job index. See 'Log File
+ Formats'.
write_iops_log=str Same as write_bw_log, but writes IOPS. If no filename is
given with this option, the default filename of
(1..N, where N is the number of jobs). Even if the filename
is given, fio will still append the type of log. If
'per_job_logs' is false, then the filename will not include
- the job index.
+ the job index. See 'Log File Formats'.
log_avg_msec=int By default, fio will log an entry in the iops, latency,
or bw log for every IO that completes. When writing to the
disk log, that can quickly grow to a very large size. Setting
this option makes fio average the each log entry over the
specified period of time, reducing the resolution of the log.
- Defaults to 0.
+ See log_max_value as well. Defaults to 0, logging all entries.
+
+log_max_value=bool If log_avg_msec is set, fio logs the average over that
+ window. If you instead want to log the maximum value, set this
+ option to 1. Defaults to 0, meaning that averaged values are
+ logged.
log_offset=int If this is set, the iolog options will include the byte
offset for the IO entry as well as the other data values.
in the specified log file. This feature depends on the
availability of zlib.
-log_store_compressed=bool If set, and log_compression is also set,
- fio will store the log files in a compressed format. They
- can be decompressed with fio, using the --inflate-log
- command line parameter. The files will be stored with a
- .fz suffix.
+log_compression_cpus=str Define the set of CPUs that are allowed to
+ handle online log compression for the IO jobs. This can
+ provide better isolation between performance sensitive jobs,
+ and background compression work.
+
+log_store_compressed=bool If set, fio will store the log files in a
+ compressed format. They can be decompressed with fio, using
+ the --inflate-log command line parameter. The files will be
+ stored with a .fz suffix.
block_error_percentiles=bool If set, record errors in trim block-sized
units from writes and trims and output a histogram of
enabled when polling for a minimum of 0 events (eg when
iodepth_batch_complete=0).
+[psyncv2] hipri Set RWF_HIPRI on IO, indicating to the kernel that
+ it's of higher priority than normal.
+
[cpu] cpuload=int Attempt to use the specified percentage of CPU cycles.
[cpu] cpuchunks=int Split the load into cycles of the given time. In
If the job is a TCP listener or UDP reader, the hostname is not
used and must be omitted unless it is a valid UDP multicast
address.
+[libhdfs] namenode=str The host name or IP address of a HDFS cluster namenode to contact.
[netsplice] port=int
[net] port=int The TCP or UDP port to bind to or connect to. If this is used
with numjobs to spawn multiple instances of the same job type, then this will
be the starting port number since fio will use a range of ports.
+[libhdfs] port=int the listening port of the HFDS cluster namenode.
[netsplice] interface=str
[net] interface=str The IP address of the network interface used to send or
1 : allocate space immidietly inside defragment event,
and free right after event
+[rbd] clustername=str Specifies the name of the Ceph cluster.
+[rbd] rbdname=str Specifies the name of the RBD.
+[rbd] pool=str Specifies the naem of the Ceph pool containing RBD.
+[rbd] clientname=str Specifies the username (without the 'client.' prefix)
+ used to access the Ceph cluster. If the clustername is
+ specified, the clientmae shall be the full type.id
+ string. If no type. prefix is given, fio will add
+ 'client.' by default.
+
[mtd] skip_bad=bool Skip operations against known bad blocks.
+[libhdfs] hdfsdirectory libhdfs will create chunk in this HDFS directory
+[libhdfs] chunck_size the size of the chunck to use for each file.
+
6.0 Interpreting the output
---------------------------
cpu= CPU usage. User and system time, along with the number
of context switches this thread went through, usage of
system and user time, and finally the number of major
- and minor page faults.
+ and minor page faults. The CPU utilization numbers are
+ averages for the jobs in that reporting group, while the
+ context and fault counters are summed.
IO depths= The distribution of io depths over the job life time. The
numbers are divided into powers of 2, so for example the
16= entries includes depths up to that value but higher
For this case, fio would wait for the server to send us the write state,
then execute 'ipmi-reboot server' when that happened.
-10.1 Loading verify state
+10.2 Loading verify state
-------------------------
To load store write state, read verification job file must contain
the verify_state_load option. If that is set, fio will load the previously
stored state. For a local fio run this is done by loading the files directly,
and on a client/server run, the server backend will ask the client to send
the files over and load them from there.
+
+
+11.0 Log File Formats
+---------------------
+
+Fio supports a variety of log file formats, for logging latencies, bandwidth,
+and IOPS. The logs share a common format, which looks like this:
+
+time (msec), value, data direction, offset
+
+Time for the log entry is always in milliseconds. The value logged depends
+on the type of log, it will be one of the following:
+
+ Latency log Value is latency in usecs
+ Bandwidth log Value is in KB/sec
+ IOPS log Value is IOPS
+
+Data direction is one of the following:
+
+ 0 IO is a READ
+ 1 IO is a WRITE
+ 2 IO is a TRIM
+
+The offset is the offset, in bytes, from the start of the file, for that
+particular IO. The logging of the offset can be toggled with 'log_offset'.
+
+If windowed logging is enabled though 'log_avg_msec', then fio doesn't log
+individual IOs. Instead of logs the average values over the specified
+period of time. Since 'data direction' and 'offset' are per-IO values,
+they aren't applicable if windowed logging is enabled. If windowed logging
+is enabled and 'log_max_value' is set, then fio logs maximum values in
+that window instead of averages.
+
projects / fio.git / blobdiff