X-Git-Url: https://git.kernel.dk/?p=fio.git;a=blobdiff_plain;f=HOWTO;h=e544634009ef0907dbd2ad0345af45c5326008f9;hp=a098a771189f34b2354cad1ccf6616fc4bcf1a96;hb=8f4b9f2475a4524067b6a0662aff006783919922;hpb=362147307296bc78ef3758ad977300f6903221d7 diff --git a/HOWTO b/HOWTO index a098a771..e5446340 100644 --- a/HOWTO +++ b/HOWTO @@ -54,56 +54,66 @@ Command line options .. option:: --debug=type - Enable verbose tracing of various fio actions. May be ``all`` for all types - or individual types separated by a comma (e.g. ``--debug=file,mem`` will - enable file and memory debugging). Currently, additional logging is - available for: + Enable verbose tracing of various fio actions. May be ``all`` for all types + or individual types separated by a comma (e.g. ``--debug=file,mem`` will + enable file and memory debugging). Currently, additional logging is + available for: - *process* + *process* Dump info related to processes. - *file* + *file* Dump info related to file actions. - *io* + *io* Dump info related to I/O queuing. - *mem* + *mem* Dump info related to memory allocations. - *blktrace* + *blktrace* Dump info related to blktrace setup. - *verify* + *verify* Dump info related to I/O verification. - *all* + *all* Enable all debug options. - *random* + *random* Dump info related to random offset generation. - *parse* + *parse* Dump info related to option matching and parsing. - *diskutil* + *diskutil* Dump info related to disk utilization updates. - *job:x* + *job:x* Dump info only related to job number x. - *mutex* + *mutex* Dump info only related to mutex up/down ops. - *profile* + *profile* Dump info related to profile extensions. - *time* + *time* Dump info related to internal time keeping. - *net* + *net* Dump info related to networking connections. - *rate* + *rate* Dump info related to I/O rate switching. - *compress* + *compress* Dump info related to log compress/decompress. - *?* or *help* + *?* or *help* Show available debug options. .. option:: --parse-only - Parse options only, don\'t start any I/O. + Parse options only, don\'t start any I/O. .. option:: --output=filename Write output to file `filename`. +.. option:: --output-format=type + + Set the reporting format to `normal`, `terse`, `json`, or `json+`. Multiple + formats can be selected, separated by a comma. `terse` is a CSV based + format. `json+` is like `json`, except it adds a full dump of the latency + buckets. + +.. option:: --runtime + Limit run time to runtime seconds. + .. option:: --bandwidth-log Generate aggregate bandwidth logs. @@ -114,16 +124,9 @@ Command line options .. option:: --append-terse - Print statistics in selected mode AND terse, semicolon-delimited format. - **deprecated**, use :option:`--output-format` instead to select multiple - formats. - -.. option:: --output-format=type - - Set the reporting format to `normal`, `terse`, `json`, or `json+`. Multiple - formats can be selected, separated by a comma. `terse` is a CSV based - format. `json+` is like `json`, except it adds a full dump of the latency - buckets. + Print statistics in selected mode AND terse, semicolon-delimited format. + **Deprecated**, use :option:`--output-format` instead to select multiple + formats. .. option:: --terse-version=type @@ -131,7 +134,7 @@ Command line options .. option:: --version - Print version info and exit. + Print version information and exit. .. option:: --help @@ -143,9 +146,9 @@ Command line options .. option:: --crctest=[test] - Test the speed of the built-in checksumming functions. If no argument is - given all of them are tested. Alternatively, a comma separated list can be passed, in - which case the given ones are tested. + Test the speed of the built-in checksumming functions. If no argument is + given, all of them are tested. Alternatively, a comma separated list can + be passed, in which case the given ones are tested. .. option:: --cmdhelp=command @@ -153,27 +156,27 @@ Command line options .. option:: --enghelp=[ioengine[,command]] - List all commands defined by :option:`ioengine`, or print help for `command` - defined by :option:`ioengine`. If no :option:`ioengine` is given, list all - available ioengines. + List all commands defined by :option:`ioengine`, or print help for `command` + defined by :option:`ioengine`. If no :option:`ioengine` is given, list all + available ioengines. .. option:: --showcmd=jobfile - Turn a job file into command line options. + Convert `jobfile` to a set of command-line options. .. option:: --readonly - Turn on safety read-only checks, preventing writes. The ``--readonly`` - option is an extra safety guard to prevent users from accidentally starting - a write workload when that is not desired. Fio will only write if - `rw=write/randwrite/rw/randrw` is given. This extra safety net can be used - as an extra precaution as ``--readonly`` will also enable a write check in - the I/O engine core to prevent writes due to unknown user space bug(s). + Turn on safety read-only checks, preventing writes. The ``--readonly`` + option is an extra safety guard to prevent users from accidentally starting + a write workload when that is not desired. Fio will only write if + `rw=write/randwrite/rw/randrw` is given. This extra safety net can be used + as an extra precaution as ``--readonly`` will also enable a write check in + the I/O engine core to prevent writes due to unknown user space bug(s). .. option:: --eta=when - When real-time ETA estimate should be printed. May be `always`, `never` or - `auto`. + Specifies when real-time ETA estimate should be printed. `when` may be + `always`, `never` or `auto`. .. option:: --eta-newline=time @@ -187,48 +190,48 @@ Command line options .. option:: --section=name - Only run specified section in job file. Multiple sections can be specified. - The ``--section`` option allows one to combine related jobs into one file. - E.g. one job file could define light, moderate, and heavy sections. Tell - fio to run only the "heavy" section by giving ``--section=heavy`` - command line option. One can also specify the "write" operations in one - section and "verify" operation in another section. The ``--section`` option - only applies to job sections. The reserved *global* section is always - parsed and used. + Only run specified section `name` in job file. Multiple sections can be specified. + The ``--section`` option allows one to combine related jobs into one file. + E.g. one job file could define light, moderate, and heavy sections. Tell + fio to run only the "heavy" section by giving ``--section=heavy`` + command line option. One can also specify the "write" operations in one + section and "verify" operation in another section. The ``--section`` option + only applies to job sections. The reserved *global* section is always + parsed and used. .. option:: --alloc-size=kb - Set the internal smalloc pool to this size in KiB. The - ``--alloc-size`` switch allows one to use a larger pool size for smalloc. - If running large jobs with randommap enabled, fio can run out of memory. - Smalloc is an internal allocator for shared structures from a fixed size - memory pool and can grow to 16 pools. The pool size defaults to 16MiB. + Set the internal smalloc pool size to `kb` in KiB. The + ``--alloc-size`` switch allows one to use a larger pool size for smalloc. + If running large jobs with randommap enabled, fio can run out of memory. + Smalloc is an internal allocator for shared structures from a fixed size + memory pool and can grow to 16 pools. The pool size defaults to 16MiB. - NOTE: While running :file:`.fio_smalloc.*` backing store files are visible - in :file:`/tmp`. + NOTE: While running :file:`.fio_smalloc.*` backing store files are visible + in :file:`/tmp`. .. option:: --warnings-fatal - All fio parser warnings are fatal, causing fio to exit with an - error. + All fio parser warnings are fatal, causing fio to exit with an + error. .. option:: --max-jobs=nr - Maximum number of threads/processes to support. + Set the maximum number of threads/processes to support. .. option:: --server=args - Start a backend server, with `args` specifying what to listen to. - See `Client/Server`_ section. + Start a backend server, with `args` specifying what to listen to. + See `Client/Server`_ section. .. option:: --daemonize=pidfile - Background a fio server, writing the pid to the given `pidfile` file. + Background a fio server, writing the pid to the given `pidfile` file. .. option:: --client=hostname - Instead of running the jobs locally, send and run them on the given host or - set of hosts. See `Client/Server`_ section. + Instead of running the jobs locally, send and run them on the given host or + set of hosts. See `Client/Server`_ section. .. option:: --remote-config=file @@ -236,7 +239,7 @@ Command line options .. option:: --idle-prof=option - Report CPU idleness. *option* is one of the following: + Report CPU idleness. `option` is one of the following: **calibrate** Run unit work calibration only and exit. @@ -445,7 +448,7 @@ automatically substituted with the current system values when the job is run. Simple math is also supported on these keywords, so you can perform actions like:: - size=8*$mb_memory + size=8*$mb_memory and get that properly expanded to 8 times the size of memory in the machine. @@ -474,7 +477,7 @@ Parameter types ~~~~~~~~~~~~~~~ **str** - String. This is a sequence of alpha characters. + String: A sequence of alphanumeric characters. **time** Integer with possible time suffix. Without a unit value is interpreted as @@ -488,7 +491,7 @@ Parameter types Integer. A whole number value, which may contain an integer prefix and an integer suffix: - [*integer prefix*] **number** [*integer suffix*] + [*integer prefix*] **number** [*integer suffix*] The optional *integer prefix* specifies the number's base. The default is decimal. *0x* specifies hexadecimal. @@ -521,7 +524,7 @@ Parameter types compatibility with old scripts. For example, 4k means 4096. For quantities of data, an optional unit of 'B' may be included - (e.g., 'kB' is the same as 'k'). + (e.g., 'kB' is the same as 'k'). The *integer suffix* is not case sensitive (e.g., m/mi mean mebi/mega, not milli). 'b' and 'B' both mean byte, not bit. @@ -853,10 +856,13 @@ Target file/device **pareto** Use a *Pareto* distribution to decide what file to access. - **gauss** + **normal** Use a *Gaussian* (normal) distribution to decide what file to access. + **gauss** + Alias for normal. + For *random*, *roundrobin*, and *sequential*, a postfix can be appended to tell fio how many I/Os to issue before switching to a new file. For example, specifying ``file_service_type=random:8`` would cause fio to issue @@ -1046,6 +1052,10 @@ I/O type **none** Do not pre-allocate space. + **native** + Use a platform's native pre-allocation call but fall back to + **none** behavior if it fails/is not implemented. + **posix** Pre-allocate via :manpage:`posix_fallocate(3)`. @@ -1060,8 +1070,9 @@ I/O type Backward-compatible alias for **posix**. May not be available on all supported platforms. **keep** is only available - on Linux. If using ZFS on Solaris this must be set to **none** because ZFS - doesn't support it. Default: **posix**. + on Linux. If using ZFS on Solaris this cannot be set to **posix** + because ZFS doesn't support pre-allocation. Default: **native** if any + pre-allocation methods are available, **none** if not. .. option:: fadvise_hint=str @@ -1082,19 +1093,39 @@ I/O type **random** Advise using **FADV_RANDOM**. -.. option:: fadvise_stream=int +.. option:: write_hint=str + + Use :manpage:`fcntl(2)` to advise the kernel what life time to expect + from a write. Only supported on Linux, as of version 4.13. Accepted + values are: + + **none** + No particular life time associated with this file. + + **short** + Data written to this file has a short life time. + + **medium** + Data written to this file has a medium life time. + + **long** + Data written to this file has a long life time. - Use :manpage:`posix_fadvise(2)` to advise the kernel what stream ID the - writes issued belong to. Only supported on Linux. Note, this option may - change going forward. + **extreme** + Data written to this file has a very long life time. + + The values are all relative to each other, and no absolute meaning + should be associated with them. .. option:: offset=int - Start I/O at the provided offset in the file, given as either a fixed size or - a percentage. If a percentage is given, the next ``blockalign``-ed offset - will be used. Data before the given offset will not be touched. This + Start I/O at the provided offset in the file, given as either a fixed size in + bytes or a percentage. If a percentage is given, the next ``blockalign``-ed + offset will be used. Data before the given offset will not be touched. This effectively caps the file size at `real_size - offset`. Can be combined with :option:`size` to constrain the start and end range of the I/O workload. + A percentage can be specified by a number between 1 and 100 followed by '%', + for example, ``offset=20%`` to specify 20%. .. option:: offset_increment=int @@ -1202,7 +1233,7 @@ I/O type **pareto** Pareto distribution - **gauss** + **normal** Normal (Gaussian) distribution **zoned** @@ -1215,8 +1246,8 @@ I/O type values will yield in terms of hit rates. If you wanted to use **zipf** with a `theta` of 1.2, you would use ``random_distribution=zipf:1.2`` as the option. If a non-uniform model is used, fio will disable use of the random - map. For the **gauss** distribution, a normal deviation is supplied as a - value between 0 and 100. + map. For the **normal** distribution, a normal (Gaussian) deviation is + supplied as a value between 0 and 100. For a **zoned** distribution, fio supports specifying percentages of I/O access that should fall within what range of the file or device. For @@ -1749,7 +1780,7 @@ I/O engine :file:`/dev/mtd0`). Discards are treated as erases. Depending on the underlying device type, the I/O may have to go in a certain pattern, e.g., on NAND, writing sequentially to erase blocks and discarding - before overwriting. The writetrim mode works well for this + before overwriting. The `trimwrite` mode works well for this constraint. **pmemblk** @@ -2373,13 +2404,14 @@ Verification header of each block. **crc32c** - Use a crc32c sum of the data area and store it in the header of each - block. + Use a crc32c sum of the data area and store it in the header of + each block. This will automatically use hardware acceleration + (e.g. SSE4.2 on an x86 or CRC crypto extensions on ARM64) but will + fall back to software crc32c if none is found. Generally the + fatest checksum fio supports when hardware accelerated. **crc32c-intel** - Use hardware assisted crc32c calculation provided on SSE4.2 enabled - processors. Falls back to regular software crc32c, if not supported - by the system. + Synonym for crc32c. **crc32** Use a crc32 sum of the data area and store it in the header of each @@ -3322,7 +3354,7 @@ will be a disk utilization section. Below is a single line containing short names for each of the fields in the minimal output v3, separated by semicolons:: - terse_version_3;fio_version;jobname;groupid;error;read_kb;read_bandwidth;read_iops;read_runtime_ms;read_slat_min;read_slat_max;read_slat_mean;read_slat_dev;read_clat_max;read_clat_min;read_clat_mean;read_clat_dev;read_clat_pct01;read_clat_pct02;read_clat_pct03;read_clat_pct04;read_clat_pct05;read_clat_pct06;read_clat_pct07;read_clat_pct08;read_clat_pct09;read_clat_pct10;read_clat_pct11;read_clat_pct12;read_clat_pct13;read_clat_pct14;read_clat_pct15;read_clat_pct16;read_clat_pct17;read_clat_pct18;read_clat_pct19;read_clat_pct20;read_tlat_min;read_lat_max;read_lat_mean;read_lat_dev;read_bw_min;read_bw_max;read_bw_agg_pct;read_bw_mean;read_bw_dev;write_kb;write_bandwidth;write_iops;write_runtime_ms;write_slat_min;write_slat_max;write_slat_mean;write_slat_dev;write_clat_max;write_clat_min;write_clat_mean;write_clat_dev;write_clat_pct01;write_clat_pct02;write_clat_pct03;write_clat_pct04;write_clat_pct05;write_clat_pct06;write_clat_pct07;write_clat_pct08;write_clat_pct09;write_clat_pct10;write_clat_pct11;write_clat_pct12;write_clat_pct13;write_clat_pct14;write_clat_pct15;write_clat_pct16;write_clat_pct17;write_clat_pct18;write_clat_pct19;write_clat_pct20;write_tlat_min;write_lat_max;write_lat_mean;write_lat_dev;write_bw_min;write_bw_max;write_bw_agg_pct;write_bw_mean;write_bw_dev;cpu_user;cpu_sys;cpu_csw;cpu_mjf;pu_minf;iodepth_1;iodepth_2;iodepth_4;iodepth_8;iodepth_16;iodepth_32;iodepth_64;lat_2us;lat_4us;lat_10us;lat_20us;lat_50us;lat_100us;lat_250us;lat_500us;lat_750us;lat_1000us;lat_2ms;lat_4ms;lat_10ms;lat_20ms;lat_50ms;lat_100ms;lat_250ms;lat_500ms;lat_750ms;lat_1000ms;lat_2000ms;lat_over_2000ms;disk_name;disk_read_iops;disk_write_iops;disk_read_merges;disk_write_merges;disk_read_ticks;write_ticks;disk_queue_time;disk_util + terse_version_3;fio_version;jobname;groupid;error;read_kb;read_bandwidth;read_iops;read_runtime_ms;read_slat_min;read_slat_max;read_slat_mean;read_slat_dev;read_clat_min;read_clat_max;read_clat_mean;read_clat_dev;read_clat_pct01;read_clat_pct02;read_clat_pct03;read_clat_pct04;read_clat_pct05;read_clat_pct06;read_clat_pct07;read_clat_pct08;read_clat_pct09;read_clat_pct10;read_clat_pct11;read_clat_pct12;read_clat_pct13;read_clat_pct14;read_clat_pct15;read_clat_pct16;read_clat_pct17;read_clat_pct18;read_clat_pct19;read_clat_pct20;read_tlat_min;read_lat_max;read_lat_mean;read_lat_dev;read_bw_min;read_bw_max;read_bw_agg_pct;read_bw_mean;read_bw_dev;write_kb;write_bandwidth;write_iops;write_runtime_ms;write_slat_min;write_slat_max;write_slat_mean;write_slat_dev;write_clat_min;write_clat_max;write_clat_mean;write_clat_dev;write_clat_pct01;write_clat_pct02;write_clat_pct03;write_clat_pct04;write_clat_pct05;write_clat_pct06;write_clat_pct07;write_clat_pct08;write_clat_pct09;write_clat_pct10;write_clat_pct11;write_clat_pct12;write_clat_pct13;write_clat_pct14;write_clat_pct15;write_clat_pct16;write_clat_pct17;write_clat_pct18;write_clat_pct19;write_clat_pct20;write_tlat_min;write_lat_max;write_lat_mean;write_lat_dev;write_bw_min;write_bw_max;write_bw_agg_pct;write_bw_mean;write_bw_dev;cpu_user;cpu_sys;cpu_csw;cpu_mjf;cpu_minf;iodepth_1;iodepth_2;iodepth_4;iodepth_8;iodepth_16;iodepth_32;iodepth_64;lat_2us;lat_4us;lat_10us;lat_20us;lat_50us;lat_100us;lat_250us;lat_500us;lat_750us;lat_1000us;lat_2ms;lat_4ms;lat_10ms;lat_20ms;lat_50ms;lat_100ms;lat_250ms;lat_500ms;lat_750ms;lat_1000ms;lat_2000ms;lat_over_2000ms;disk_name;disk_read_iops;disk_write_iops;disk_read_merges;disk_write_merges;disk_read_ticks;write_ticks;disk_queue_time;disk_util Trace file format @@ -3531,7 +3563,7 @@ If windowed logging is enabled and :option:`log_max_value` is set, then fio logs maximum values in that window instead of averages. Since 'data direction' and 'offset' are per-I/O values, they aren't applicable if windowed logging is enabled. -Client/server +Client/Server ------------- Normally fio is invoked as a stand-alone application on the machine where the