.. option:: offset=int
- Start I/O at the provided offset in the file, given as either a fixed size or
- a percentage. If a percentage is given, the next ``blockalign``-ed offset
- will be used. Data before the given offset will not be touched. This
+ Start I/O at the provided offset in the file, given as either a fixed size in
+ bytes or a percentage. If a percentage is given, the next ``blockalign``-ed
+ offset will be used. Data before the given offset will not be touched. This
effectively caps the file size at `real_size - offset`. Can be combined with
:option:`size` to constrain the start and end range of the I/O workload.
+ A percentage can be specified by the percentage number plus 1 with preceding '-'.
+ For example, -1 is parsed as 0%, -10 is parsed as 9%, -101 is parsed as 100%.
.. option:: offset_increment=int
and :option:`io_size` is set to 40GiB, then fio will do 40GiB of I/O within
the 0..20GiB region.
-.. option:: filesize=int
+.. option:: filesize=irange(int)
Individual file sizes. May be a range, in which case fio will select sizes
for files at random within the given range and limited to :option:`size` in
Read and write through Hadoop (HDFS). The :file:`filename` option
is used to specify host,port of the hdfs name-node to connect. This
engine interprets offsets a little differently. In HDFS, files once
- created cannot be modified. So random writes are not possible. To
- imitate this, libhdfs engine expects bunch of small files to be
- created over HDFS, and engine will randomly pick a file out of those
- files based on the offset generated by fio backend. (see the example
+ created cannot be modified so random writes are not possible. To
+ imitate this the libhdfs engine expects a bunch of small files to be
+ created over HDFS and will randomly pick a file from them
+ based on the offset generated by fio backend (see the example
job file to create such files, use ``rw=write`` option). Please
- note, you might want to set necessary environment variables to work
- with hdfs/libhdfs properly. Each job uses its own connection to
+ note, it may be necessary to set environment variables to work
+ with HDFS/libhdfs properly. Each job uses its own connection to
HDFS.
**mtd**
:file:`/dev/mtd0`). Discards are treated as erases. Depending on the
underlying device type, the I/O may have to go in a certain pattern,
e.g., on NAND, writing sequentially to erase blocks and discarding
- before overwriting. The writetrim mode works well for this
+ before overwriting. The `trimwrite` mode works well for this
constraint.
**pmemblk**
suffix rules apply. Comma-separated values may be specified for reads,
writes, and trims as described in :option:`blocksize`.
+ For example, using `rate=1m,500k` would limit reads to 1MiB/sec and writes to
+ 500KiB/sec. Capping only reads or writes can be done with `rate=,500k` or
+ `rate=500k,` where the former will only limit writes (to 500KiB/sec) and the
+ latter will only limit reads.
+
.. option:: rate_min=int[,int][,int]
Tell fio to do whatever it can to maintain at least this bandwidth. Failing
header of each block.
**crc32c**
- Use a crc32c sum of the data area and store it in the header of each
- block.
+ Use a crc32c sum of the data area and store it in the header of
+ each block. This will automatically use hardware acceleration
+ (e.g. SSE4.2 on an x86 or CRC crypto extensions on ARM64) but will
+ fall back to software crc32c if none is found. Generally the
+ fatest checksum fio supports when hardware accelerated.
**crc32c-intel**
- Use hardware assisted crc32c calculation provided on SSE4.2 enabled
- processors. Falls back to regular software crc32c, if not supported
- by the system.
+ Synonym for crc32c.
**crc32**
Use a crc32 sum of the data area and store it in the header of each
very large size. Setting this option makes fio average the each log entry
over the specified period of time, reducing the resolution of the log. See
:option:`log_max_value` as well. Defaults to 0, logging all entries.
+ Also see `Log File Formats`_.
.. option:: log_hist_msec=int
Interpreting the output
-----------------------
+..
+ Example output was based on the following:
+ TZ=UTC fio --iodepth=8 --ioengine=null --size=100M --time_based \
+ --rate=1256k --bs=14K --name=quick --runtime=1s --name=mixed \
+ --runtime=2m --rw=rw
+
Fio spits out a lot of output. While running, fio will display the status of the
jobs created. An example of that would be::
Jobs: 1 (f=1): [_(1),M(1)][24.8%][r=20.5MiB/s,w=23.5MiB/s][r=82,w=94 IOPS][eta 01m:31s]
-The characters inside the square brackets denote the current status of each
-thread. The possible values (in typical life cycle order) are:
+The characters inside the first set of square brackets denote the current status of
+each thread. The first character is the first job defined in the job file, and so
+forth. The possible values (in typical life cycle order) are:
+------+-----+-----------------------------------------------------------+
| Idle | Run | |
+------+-----+-----------------------------------------------------------+
| | p | Thread running pre-reading file(s). |
+------+-----+-----------------------------------------------------------+
+| | / | Thread is in ramp period. |
++------+-----+-----------------------------------------------------------+
| | R | Running, doing sequential reads. |
+------+-----+-----------------------------------------------------------+
| | r | Running, doing random reads. |
+------+-----+-----------------------------------------------------------+
| | m | Running, doing mixed random reads/writes. |
+------+-----+-----------------------------------------------------------+
-| | F | Running, currently waiting for :manpage:`fsync(2)` |
+| | D | Running, doing sequential trims. |
++------+-----+-----------------------------------------------------------+
+| | d | Running, doing random trims. |
++------+-----+-----------------------------------------------------------+
+| | F | Running, currently waiting for :manpage:`fsync(2)`. |
+------+-----+-----------------------------------------------------------+
| | V | Running, doing verification of written data. |
+------+-----+-----------------------------------------------------------+
+| f | | Thread finishing. |
++------+-----+-----------------------------------------------------------+
| E | | Thread exited, not reaped by main thread yet. |
+------+-----+-----------------------------------------------------------+
-| _ | | Thread reaped, or |
+| _ | | Thread reaped. |
+------+-----+-----------------------------------------------------------+
| X | | Thread reaped, exited with an error. |
+------+-----+-----------------------------------------------------------+
| K | | Thread reaped, exited due to signal. |
+------+-----+-----------------------------------------------------------+
+..
+ Example output was based on the following:
+ TZ=UTC fio --iodepth=8 --ioengine=null --size=100M --runtime=58m \
+ --time_based --rate=2512k --bs=256K --numjobs=10 \
+ --name=readers --rw=read --name=writers --rw=write
+
Fio will condense the thread string as not to take up more space on the command
-line as is needed. For instance, if you have 10 readers and 10 writers running,
+line than needed. For instance, if you have 10 readers and 10 writers running,
the output would look like this::
Jobs: 20 (f=20): [R(10),W(10)][4.0%][r=20.5MiB/s,w=23.5MiB/s][r=82,w=94 IOPS][eta 57m:36s]
-Fio will still maintain the ordering, though. So the above means that jobs 1..10
-are readers, and 11..20 are writers.
+Note that the status string is displayed in order, so it's possible to tell which of
+the jobs are currently doing what. In the example above this means that jobs 1--10
+are readers and 11--20 are writers.
The other values are fairly self explanatory -- number of threads currently
-running and doing I/O, the number of currently open files (f=), the rate of I/O
-since last check (read speed listed first, then write speed and optionally trim
-speed), and the estimated completion percentage and time for the current
+running and doing I/O, the number of currently open files (f=), the estimated
+completion percentage, the rate of I/O since last check (read speed listed first,
+then write speed and optionally trim speed) in terms of bandwidth and IOPS, and time to completion for the current
running group. It's impossible to estimate runtime of the following groups (if
-any). Note that the string is displayed in order, so it's possible to tell which
-of the jobs are currently doing what. The first character is the first job
-defined in the job file, and so forth.
-
-When fio is done (or interrupted by :kbd:`ctrl-c`), it will show the data for
-each thread, group of threads, and disks in that order. For each data direction,
-the output looks like::
-
- Client1 (g=0): err= 0:
- write: io= 32MiB, bw= 666KiB/s, iops=89 , runt= 50320msec
- slat (msec): min= 0, max= 136, avg= 0.03, stdev= 1.92
- clat (msec): min= 0, max= 631, avg=48.50, stdev=86.82
- bw (KiB/s) : min= 0, max= 1196, per=51.00%, avg=664.02, stdev=681.68
- cpu : usr=1.49%, sys=0.25%, ctx=7969, majf=0, minf=17
- IO depths : 1=0.1%, 2=0.3%, 4=0.5%, 8=99.0%, 16=0.0%, 32=0.0%, >32=0.0%
- submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
- complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
- issued r/w: total=0/32768, short=0/0
- lat (msec): 2=1.6%, 4=0.0%, 10=3.2%, 20=12.8%, 50=38.4%, 100=24.8%,
- lat (msec): 250=15.2%, 500=0.0%, 750=0.0%, 1000=0.0%, >=2048=0.0%
-
-The client number is printed, along with the group id and error of that
-thread. Below is the I/O statistics, here for writes. In the order listed, they
-denote:
-
-**io**
- Number of megabytes I/O performed.
-
-**bw**
- Average bandwidth rate.
-
-**iops**
- Average I/Os performed per second.
-
-**runt**
- The runtime of that thread.
+any).
+
+..
+ Example output was based on the following:
+ TZ=UTC fio --iodepth=16 --ioengine=posixaio --filename=/tmp/fiofile \
+ --direct=1 --size=100M --time_based --runtime=50s --rate_iops=89 \
+ --bs=7K --name=Client1 --rw=write
+
+When fio is done (or interrupted by :kbd:`Ctrl-C`), it will show the data for
+each thread, group of threads, and disks in that order. For each overall thread (or
+group) the output looks like::
+
+ Client1: (groupid=0, jobs=1): err= 0: pid=16109: Sat Jun 24 12:07:54 2017
+ write: IOPS=88, BW=623KiB/s (638kB/s)(30.4MiB/50032msec)
+ slat (nsec): min=500, max=145500, avg=8318.00, stdev=4781.50
+ clat (usec): min=170, max=78367, avg=4019.02, stdev=8293.31
+ lat (usec): min=174, max=78375, avg=4027.34, stdev=8291.79
+ clat percentiles (usec):
+ | 1.00th=[ 302], 5.00th=[ 326], 10.00th=[ 343], 20.00th=[ 363],
+ | 30.00th=[ 392], 40.00th=[ 404], 50.00th=[ 416], 60.00th=[ 445],
+ | 70.00th=[ 816], 80.00th=[ 6718], 90.00th=[12911], 95.00th=[21627],
+ | 99.00th=[43779], 99.50th=[51643], 99.90th=[68682], 99.95th=[72877],
+ | 99.99th=[78119]
+ bw ( KiB/s): min= 532, max= 686, per=0.10%, avg=622.87, stdev=24.82, samples= 100
+ iops : min= 76, max= 98, avg=88.98, stdev= 3.54, samples= 100
+ lat (usec) : 250=0.04%, 500=64.11%, 750=4.81%, 1000=2.79%
+ lat (msec) : 2=4.16%, 4=1.84%, 10=4.90%, 20=11.33%, 50=5.37%
+ lat (msec) : 100=0.65%
+ cpu : usr=0.27%, sys=0.18%, ctx=12072, majf=0, minf=21
+ IO depths : 1=85.0%, 2=13.1%, 4=1.8%, 8=0.1%, 16=0.0%, 32=0.0%, >=64=0.0%
+ submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
+ complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
+ issued rwt: total=0,4450,0, short=0,0,0, dropped=0,0,0
+ latency : target=0, window=0, percentile=100.00%, depth=8
+
+The job name (or first job's name when using :option:`group_reporting`) is printed,
+along with the group id, count of jobs being aggregated, last error id seen (which
+is 0 when there are no errors), pid/tid of that thread and the time the job/group
+completed. Below are the I/O statistics for each data direction performed (showing
+writes in the example above). In the order listed, they denote:
+
+**read/write/trim**
+ The string before the colon shows the I/O direction the statistics
+ are for. **IOPS** is the average I/Os performed per second. **BW**
+ is the average bandwidth rate shown as: value in power of 2 format
+ (value in power of 10 format). The last two values show: (**total
+ I/O performed** in power of 2 format / **runtime** of that thread).
**slat**
- Submission latency (avg being the average, stdev being the standard
- deviation). This is the time it took to submit the I/O. For sync I/O,
- the slat is really the completion latency, since queue/complete is one
- operation there. This value can be in milliseconds or microseconds, fio
- will choose the most appropriate base and print that. In the example
- above, milliseconds is the best scale. Note: in :option:`--minimal` mode
+ Submission latency (**min** being the minimum, **max** being the
+ maximum, **avg** being the average, **stdev** being the standard
+ deviation). This is the time it took to submit the I/O. For
+ sync I/O this row is not displayed as the slat is really the
+ completion latency (since queue/complete is one operation there).
+ This value can be in nanoseconds, microseconds or milliseconds ---
+ fio will choose the most appropriate base and print that (in the
+ example above nanoseconds was the best scale). Note: in :option:`--minimal` mode
latencies are always expressed in microseconds.
**clat**
explanation).
**bw**
- Bandwidth. Same names as the xlat stats, but also includes an
- approximate percentage of total aggregate bandwidth this thread received
- in this group. This last value is only really useful if the threads in
- this group are on the same disk, since they are then competing for disk
- access.
+ Bandwidth statistics based on samples. Same names as the xlat stats,
+ but also includes the number of samples taken (**samples**) and an
+ approximate percentage of total aggregate bandwidth this thread
+ received in its group (**per**). This last value is only really
+ useful if the threads in this group are on the same disk, since they
+ are then competing for disk access.
+
+**iops**
+ IOPS statistics based on samples. Same names as bw.
**cpu**
CPU usage. User and system time, along with the number of context
context and fault counters are summed.
**IO depths**
- The distribution of I/O depths over the job life time. The numbers are
- divided into powers of 2, so for example the 16= entries includes depths
- up to that value but higher than the previous entry. In other words, it
- covers the range from 16 to 31.
+ The distribution of I/O depths over the job lifetime. The numbers are
+ divided into powers of 2 and each entry covers depths from that value
+ up to those that are lower than the next entry -- e.g., 16= covers
+ depths from 16 to 31. Note that the range covered by a depth
+ distribution entry can be different to the range covered by the
+ equivalent submit/complete distribution entry.
**IO submit**
How many pieces of I/O were submitting in a single submit call. Each
entry denotes that amount and below, until the previous entry -- e.g.,
- 8=100% mean that we submitted anywhere in between 5-8 I/Os per submit
- call.
+ 16=100% means that we submitted anywhere between 9 to 16 I/Os per submit
+ call. Note that the range covered by a submit distribution entry can
+ be different to the range covered by the equivalent depth distribution
+ entry.
**IO complete**
Like the above submit number, but for completions instead.
-**IO issued**
- The number of read/write requests issued, and how many of them were
- short.
+**IO issued rwt**
+ The number of read/write/trim requests issued, and how many of them were
+ short or dropped.
**IO latencies**
The distribution of I/O completion latencies. This is the time from when
I/O completed within 2 msecs, 20=12.8% means that 12.8% of the I/O took
more than 10 msecs, but less than (or equal to) 20 msecs.
+..
+ Example output was based on the following:
+ TZ=UTC fio --ioengine=null --iodepth=2 --size=100M --numjobs=2 \
+ --rate_process=poisson --io_limit=32M --name=read --bs=128k \
+ --rate=11M --name=write --rw=write --bs=2k --rate=700k
+
After each client has been listed, the group statistics are printed. They
will look like this::
Run status group 0 (all jobs):
- READ: io=64MB, aggrb=22178, minb=11355, maxb=11814, mint=2840msec, maxt=2955msec
- WRITE: io=64MB, aggrb=1302, minb=666, maxb=669, mint=50093msec, maxt=50320msec
+ READ: bw=20.9MiB/s (21.9MB/s), 10.4MiB/s-10.8MiB/s (10.9MB/s-11.3MB/s), io=64.0MiB (67.1MB), run=2973-3069msec
+ WRITE: bw=1231KiB/s (1261kB/s), 616KiB/s-621KiB/s (630kB/s-636kB/s), io=64.0MiB (67.1MB), run=52747-53223msec
-For each data direction, it prints:
+For each data direction it prints:
+**bw**
+ Aggregate bandwidth of threads in this group followed by the
+ minimum and maximum bandwidth of all the threads in this group.
+ Values outside of brackets are power-of-2 format and those
+ within are the equivalent value in a power-of-10 format.
**io**
- Number of megabytes I/O performed.
-**aggrb**
- Aggregate bandwidth of threads in this group.
-**minb**
- The minimum average bandwidth a thread saw.
-**maxb**
- The maximum average bandwidth a thread saw.
-**mint**
- The smallest runtime of the threads in that group.
-**maxt**
- The longest runtime of the threads in that group.
+ Aggregate I/O performed of all threads in this group. The
+ format is the same as bw.
+**run**
+ The smallest and longest runtimes of the threads in this group.
And finally, the disk statistics are printed. They will look like this::
Number of merges I/O the I/O scheduler.
**ticks**
Number of ticks we kept the disk busy.
-**io_queue**
+**in_queue**
Total time spent in the disk queue.
**util**
The disk utilization. A value of 100% means we kept the disk
The job description (if provided) follows on a second line.
-To enable terse output, use the :option:`--minimal` command line option. The
+To enable terse output, use the :option:`--minimal` or
+:option:`--output-format`\=terse command line options. The
first value is the version of the terse output format. If the output has to be
changed for some reason, this number will be incremented by 1 to signify that
change.
Below is a single line containing short names for each of the fields in the
minimal output v3, separated by semicolons::
- terse_version_3;fio_version;jobname;groupid;error;read_kb;read_bandwidth;read_iops;read_runtime_ms;read_slat_min;read_slat_max;read_slat_mean;read_slat_dev;read_clat_max;read_clat_min;read_clat_mean;read_clat_dev;read_clat_pct01;read_clat_pct02;read_clat_pct03;read_clat_pct04;read_clat_pct05;read_clat_pct06;read_clat_pct07;read_clat_pct08;read_clat_pct09;read_clat_pct10;read_clat_pct11;read_clat_pct12;read_clat_pct13;read_clat_pct14;read_clat_pct15;read_clat_pct16;read_clat_pct17;read_clat_pct18;read_clat_pct19;read_clat_pct20;read_tlat_min;read_lat_max;read_lat_mean;read_lat_dev;read_bw_min;read_bw_max;read_bw_agg_pct;read_bw_mean;read_bw_dev;write_kb;write_bandwidth;write_iops;write_runtime_ms;write_slat_min;write_slat_max;write_slat_mean;write_slat_dev;write_clat_max;write_clat_min;write_clat_mean;write_clat_dev;write_clat_pct01;write_clat_pct02;write_clat_pct03;write_clat_pct04;write_clat_pct05;write_clat_pct06;write_clat_pct07;write_clat_pct08;write_clat_pct09;write_clat_pct10;write_clat_pct11;write_clat_pct12;write_clat_pct13;write_clat_pct14;write_clat_pct15;write_clat_pct16;write_clat_pct17;write_clat_pct18;write_clat_pct19;write_clat_pct20;write_tlat_min;write_lat_max;write_lat_mean;write_lat_dev;write_bw_min;write_bw_max;write_bw_agg_pct;write_bw_mean;write_bw_dev;cpu_user;cpu_sys;cpu_csw;cpu_mjf;pu_minf;iodepth_1;iodepth_2;iodepth_4;iodepth_8;iodepth_16;iodepth_32;iodepth_64;lat_2us;lat_4us;lat_10us;lat_20us;lat_50us;lat_100us;lat_250us;lat_500us;lat_750us;lat_1000us;lat_2ms;lat_4ms;lat_10ms;lat_20ms;lat_50ms;lat_100ms;lat_250ms;lat_500ms;lat_750ms;lat_1000ms;lat_2000ms;lat_over_2000ms;disk_name;disk_read_iops;disk_write_iops;disk_read_merges;disk_write_merges;disk_read_ticks;write_ticks;disk_queue_time;disk_util
+terse_version_3;fio_version;jobname;groupid;error;read_kb;read_bandwidth;read_iops;read_runtime_ms;read_slat_min;read_slat_max;read_slat_mean;read_slat_dev;read_clat_min;read_clat_max;read_clat_mean;read_clat_dev;read_clat_pct01;read_clat_pct02;read_clat_pct03;read_clat_pct04;read_clat_pct05;read_clat_pct06;read_clat_pct07;read_clat_pct08;read_clat_pct09;read_clat_pct10;read_clat_pct11;read_clat_pct12;read_clat_pct13;read_clat_pct14;read_clat_pct15;read_clat_pct16;read_clat_pct17;read_clat_pct18;read_clat_pct19;read_clat_pct20;read_tlat_min;read_lat_max;read_lat_mean;read_lat_dev;read_bw_min;read_bw_max;read_bw_agg_pct;read_bw_mean;read_bw_dev;write_kb;write_bandwidth;write_iops;write_runtime_ms;write_slat_min;write_slat_max;write_slat_mean;write_slat_dev;write_clat_min;write_clat_max;write_clat_mean;write_clat_dev;write_clat_pct01;write_clat_pct02;write_clat_pct03;write_clat_pct04;write_clat_pct05;write_clat_pct06;write_clat_pct07;write_clat_pct08;write_clat_pct09;write_clat_pct10;write_clat_pct11;write_clat_pct12;write_clat_pct13;write_clat_pct14;write_clat_pct15;write_clat_pct16;write_clat_pct17;write_clat_pct18;write_clat_pct19;write_clat_pct20;write_tlat_min;write_lat_max;write_lat_mean;write_lat_dev;write_bw_min;write_bw_max;write_bw_agg_pct;write_bw_mean;write_bw_dev;cpu_user;cpu_sys;cpu_csw;cpu_mjf;cpu_minf;iodepth_1;iodepth_2;iodepth_4;iodepth_8;iodepth_16;iodepth_32;iodepth_64;lat_2us;lat_4us;lat_10us;lat_20us;lat_50us;lat_100us;lat_250us;lat_500us;lat_750us;lat_1000us;lat_2ms;lat_4ms;lat_10ms;lat_20ms;lat_50ms;lat_100ms;lat_250ms;lat_500ms;lat_750ms;lat_1000ms;lat_2000ms;lat_over_2000ms;disk_name;disk_read_iops;disk_write_iops;disk_read_merges;disk_write_merges;disk_read_ticks;write_ticks;disk_queue_time;disk_util
Trace file format
particular I/O. The logging of the offset can be toggled with
:option:`log_offset`.
-If windowed logging is enabled through :option:`log_avg_msec` then fio doesn't
-log individual I/Os. Instead of logs the average values over the specified period
-of time. Since 'data direction' and 'offset' are per-I/O values, they aren't
-applicable if windowed logging is enabled. If windowed logging is enabled and
-:option:`log_max_value` is set, then fio logs maximum values in that window
-instead of averages.
-
+Fio defaults to logging every individual I/O. When IOPS are logged for individual
+I/Os the value entry will always be 1. If windowed logging is enabled through
+:option:`log_avg_msec`, fio logs the average values over the specified period of time.
+If windowed logging is enabled and :option:`log_max_value` is set, then fio logs
+maximum values in that window instead of averages. Since 'data direction' and
+'offset' are per-I/O values, they aren't applicable if windowed logging is enabled.
Client/server
-------------
Normally fio is invoked as a stand-alone application on the machine where the
-I/O workload should be generated. However, the frontend and backend of fio can
-be run separately. Ie the fio server can generate an I/O workload on the "Device
-Under Test" while being controlled from another machine.
+I/O workload should be generated. However, the backend and frontend of fio can
+be run separately i.e., the fio server can generate an I/O workload on the "Device
+Under Test" while being controlled by a client on another machine.
Start the server on the machine which has access to the storage DUT::
fio --server=args
-where args defines what fio listens to. The arguments are of the form
+where `args` defines what fio listens to. The arguments are of the form
``type,hostname`` or ``IP,port``. *type* is either ``ip`` (or ip4) for TCP/IP
v4, ``ip6`` for TCP/IP v6, or ``sock`` for a local unix domain socket.
*hostname* is either a hostname or IP address, and *port* is the port to listen
6) ``fio --server=sock:/tmp/fio.sock``
- Start a fio server, listening on the local socket /tmp/fio.sock.
+ Start a fio server, listening on the local socket :file:`/tmp/fio.sock`.
Once a server is running, a "client" can connect to the fio server with::
projects / fio.git / blobdiff