X-Git-Url: https://git.kernel.dk/?a=blobdiff_plain;f=HOWTO;h=e510e18c4c782a3335f6edaf29fed89799b9445c;hb=08bdeb50f763aa3cb8b910a4cff3b14f17f076a6;hp=bfaa054a5de0e84e1d914d2b2375c041c00eb57b;hpb=b599759ba565e7f2f573af364e6da4fe6d556a90;p=fio.git diff --git a/HOWTO b/HOWTO index bfaa054a..e510e18c 100644 --- a/HOWTO +++ b/HOWTO @@ -182,8 +182,10 @@ Command line options .. option:: --status-interval=time - Force full status dump every `time` period passed. When the unit is - omitted, the value is interpreted in seconds. + Force a full status dump of cumulative (from job start) values at `time` + intervals. This option does *not* provide per-period measurements. So + values such as bandwidth are running averages. When the time unit is omitted, + `time` is interpreted in seconds. .. option:: --section=name @@ -215,6 +217,9 @@ Command line options .. option:: --max-jobs=nr Set the maximum number of threads/processes to support to `nr`. + NOTE: On Linux, it may be necessary to increase the shared-memory + limit (:file:`/proc/sys/kernel/shmmax`) if fio runs into errors while + creating jobs. .. option:: --server=args @@ -228,7 +233,7 @@ Command line options .. option:: --client=hostname Instead of running the jobs locally, send and run them on the given `hostname` - or set of `hostname`s. See `Client/Server`_ section. + or set of `hostname`\s. See `Client/Server`_ section. .. option:: --remote-config=file @@ -790,6 +795,13 @@ Target file/device named :file:`testfiles.4`. The default of :file:`$jobname.$jobnum.$filenum` will be used if no other format specifier is given. + If you specify a path then the directories will be created up to the + main directory for the file. So for example if you specify + ``filename_format=a/b/c/$jobnum`` then the directories a/b/c will be + created before the file setup part of the job. If you specify + :option:`directory` then the path will be relative that directory, + otherwise it is treated as the absolute path. + .. option:: unique_filename=bool To avoid collisions between networked clients, fio defaults to prefixing any @@ -1116,13 +1128,20 @@ I/O type .. option:: offset=int Start I/O at the provided offset in the file, given as either a fixed size in - bytes or a percentage. If a percentage is given, the next ``blockalign``-ed - offset will be used. Data before the given offset will not be touched. This + bytes or a percentage. If a percentage is given, the generated offset will be + aligned to the minimum ``blocksize`` or to the value of ``offset_align`` if + provided. Data before the given offset will not be touched. This effectively caps the file size at `real_size - offset`. Can be combined with :option:`size` to constrain the start and end range of the I/O workload. A percentage can be specified by a number between 1 and 100 followed by '%', for example, ``offset=20%`` to specify 20%. +.. option:: offset_align=int + + If set to non-zero value, the byte offset generated by a percentage ``offset`` + is aligned upwards to this value. Defaults to 0 meaning that a percentage + offset is aligned to the minimum block size. + .. option:: offset_increment=int If this is provided, then the real offset becomes `offset + offset_increment @@ -1434,8 +1453,8 @@ Buffers and memory mix of random data and a fixed pattern. The fixed pattern is either zeros, or the pattern specified by :option:`buffer_pattern`. If the pattern option is used, it might skew the compression ratio slightly. Note that this is per - block size unit, for file/disk wide compression level that matches this - setting, you'll also want to set :option:`refill_buffers`. + block size unit, see :option:`buffer_compress_chunk` for setting a finer + granularity of compression regions. .. option:: buffer_compress_chunk=int @@ -1444,7 +1463,8 @@ Buffers and memory will provide :option:`buffer_compress_percentage` of blocksize random data, followed by the remaining zeroed. With this set to some chunk size smaller than the block size, fio can alternate random and zeroed data throughout the - I/O buffer. + I/O buffer. This is particularly useful when bigger block sizes are used + for a job. .. option:: buffer_pattern=str @@ -1703,7 +1723,7 @@ I/O engine Doesn't transfer any data, but burns CPU cycles according to the :option:`cpuload` and :option:`cpuchunks` options. Setting :option:`cpuload`\=85 will cause that job to do nothing but burn 85% - of the CPU. In case of SMP machines, use :option:`numjobs`= + of the CPU. In case of SMP machines, use :option:`numjobs`\= to get desired CPU usage, as the cpuload only loads a single CPU at the desired rate. A job never finishes unless there is at least one non-cpuio job. @@ -1795,6 +1815,10 @@ I/O engine absolute or relative. See :file:`engines/skeleton_external.c` for details of writing an external I/O engine. + **filecreate** + Simply create the files and do no I/O to them. You still need to + set `filesize` so that all the accounting still occurs, but no + actual I/O will be done other than creating the file. I/O engine specific parameters ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -2037,7 +2061,7 @@ I/O depth changing data and the overlapping region has a non-zero size. Setting ``serialize_overlap`` tells fio to avoid provoking this behavior by explicitly serializing in-flight I/Os that have a non-zero overlap. Note that setting - this option can reduce both performance and the `:option:iodepth` achieved. + this option can reduce both performance and the :option:`iodepth` achieved. Additionally this option does not work when :option:`io_submit_mode` is set to offload. Default: false. @@ -2699,47 +2723,46 @@ Measurements and reporting .. option:: write_bw_log=str If given, write a bandwidth log for this job. Can be used to store data of - the bandwidth of the jobs in their lifetime. The included - :command:`fio_generate_plots` script uses :command:`gnuplot` to turn these - text files into nice graphs. See :option:`write_lat_log` for behavior of - given filename. For this option, the postfix is :file:`_bw.x.log`, where `x` - is the index of the job (`1..N`, where `N` is the number of jobs). If - :option:`per_job_logs` is false, then the filename will not include the job - index. See `Log File Formats`_. + the bandwidth of the jobs in their lifetime. -.. option:: write_lat_log=str + If no str argument is given, the default filename of + :file:`jobname_type.x.log` is used. Even when the argument is given, fio + will still append the type of log. So if one specifies:: + + write_bw_log=foo - Same as :option:`write_bw_log`, except that this option stores I/O - submission, completion, and total latencies instead. If no filename is given - with this option, the default filename of :file:`jobname_type.log` is - used. Even if the filename is given, fio will still append the type of - log. So if one specifies:: + The actual log name will be :file:`foo_bw.x.log` where `x` is the index + of the job (`1..N`, where `N` is the number of jobs). If + :option:`per_job_logs` is false, then the filename will not include the + `.x` job index. - write_lat_log=foo + The included :command:`fio_generate_plots` script uses :command:`gnuplot` to turn these + text files into nice graphs. See `Log File Formats`_ for how data is + structured within the file. - The actual log names will be :file:`foo_slat.x.log`, :file:`foo_clat.x.log`, - and :file:`foo_lat.x.log`, where `x` is the index of the job (`1..N`, where `N` - is the number of jobs). This helps :command:`fio_generate_plots` find the - logs automatically. If :option:`per_job_logs` is false, then the filename - will not include the job index. See `Log File Formats`_. +.. option:: write_lat_log=str + + Same as :option:`write_bw_log`, except this option creates I/O + submission (e.g., `file:`name_slat.x.log`), completion (e.g., + `file:`name_clat.x.log`), and total (e.g., `file:`name_lat.x.log`) + latency files instead. See :option:`write_bw_log` for details about + the filename format and `Log File Formats`_ for how data is structured + within the files. .. option:: write_hist_log=str - Same as :option:`write_lat_log`, but writes I/O completion latency - histograms. If no filename is given with this option, the default filename - of :file:`jobname_clat_hist.x.log` is used, where `x` is the index of the - job (`1..N`, where `N` is the number of jobs). Even if the filename is given, - fio will still append the type of log. If :option:`per_job_logs` is false, - then the filename will not include the job index. See `Log File Formats`_. + Same as :option:`write_bw_log` but writes an I/O completion latency + histogram file (e.g., `file:`name_hist.x.log`) instead. Note that this + file will be empty unless :option:`log_hist_msec` has also been set. + See :option:`write_bw_log` for details about the filename format and + `Log File Formats`_ for how data is structured within the file. .. option:: write_iops_log=str - Same as :option:`write_bw_log`, but writes IOPS. If no filename is given - with this option, the default filename of :file:`jobname_type.x.log` is - used, where `x` is the index of the job (`1..N`, where `N` is the number of - jobs). Even if the filename is given, fio will still append the type of - log. If :option:`per_job_logs` is false, then the filename will not include - the job index. See `Log File Formats`_. + Same as :option:`write_bw_log`, but writes an IOPS file (e.g. + `file:`name_iops.x.log`) instead. See :option:`write_bw_log` for + details about the filename format and `Log File Formats`_ for how data + is structured within the file. .. option:: log_avg_msec=int @@ -2757,15 +2780,16 @@ Measurements and reporting :option:`log_avg_msec` is inaccurate. Setting this option makes fio log histogram entries over the specified period of time, reducing log sizes for high IOPS devices while retaining percentile accuracy. See - :option:`log_hist_coarseness` as well. Defaults to 0, meaning histogram - logging is disabled. + :option:`log_hist_coarseness` and :option:`write_hist_log` as well. + Defaults to 0, meaning histogram logging is disabled. .. option:: log_hist_coarseness=int Integer ranging from 0 to 6, defining the coarseness of the resolution of the histogram logs enabled with :option:`log_hist_msec`. For each increment in coarseness, fio outputs half as many bins. Defaults to 0, for which - histogram logs contain 1216 latency bins. See `Log File Formats`_. + histogram logs contain 1216 latency bins. See :option:`write_hist_log` + and `Log File Formats`_. .. option:: log_max_value=bool @@ -2865,7 +2889,7 @@ Measurements and reporting .. option:: lat_percentiles=bool - Enable the reporting of percentiles of IO latencies. This is similar + Enable the reporting of percentiles of I/O latencies. This is similar to :option:`clat_percentiles`, except that this includes the submission latency. This option is mutually exclusive with :option:`clat_percentiles`. @@ -3235,7 +3259,7 @@ writes in the example above). In the order listed, they denote: short or dropped. **IO latency** - These values are for `--latency-target` and related options. When + These values are for :option:`latency_target` and related options. When these options are engaged, this section describes the I/O depth required to meet the specified latency target. @@ -3389,6 +3413,15 @@ minimal output v3, separated by semicolons:: terse_version_3;fio_version;jobname;groupid;error;read_kb;read_bandwidth;read_iops;read_runtime_ms;read_slat_min;read_slat_max;read_slat_mean;read_slat_dev;read_clat_min;read_clat_max;read_clat_mean;read_clat_dev;read_clat_pct01;read_clat_pct02;read_clat_pct03;read_clat_pct04;read_clat_pct05;read_clat_pct06;read_clat_pct07;read_clat_pct08;read_clat_pct09;read_clat_pct10;read_clat_pct11;read_clat_pct12;read_clat_pct13;read_clat_pct14;read_clat_pct15;read_clat_pct16;read_clat_pct17;read_clat_pct18;read_clat_pct19;read_clat_pct20;read_tlat_min;read_lat_max;read_lat_mean;read_lat_dev;read_bw_min;read_bw_max;read_bw_agg_pct;read_bw_mean;read_bw_dev;write_kb;write_bandwidth;write_iops;write_runtime_ms;write_slat_min;write_slat_max;write_slat_mean;write_slat_dev;write_clat_min;write_clat_max;write_clat_mean;write_clat_dev;write_clat_pct01;write_clat_pct02;write_clat_pct03;write_clat_pct04;write_clat_pct05;write_clat_pct06;write_clat_pct07;write_clat_pct08;write_clat_pct09;write_clat_pct10;write_clat_pct11;write_clat_pct12;write_clat_pct13;write_clat_pct14;write_clat_pct15;write_clat_pct16;write_clat_pct17;write_clat_pct18;write_clat_pct19;write_clat_pct20;write_tlat_min;write_lat_max;write_lat_mean;write_lat_dev;write_bw_min;write_bw_max;write_bw_agg_pct;write_bw_mean;write_bw_dev;cpu_user;cpu_sys;cpu_csw;cpu_mjf;cpu_minf;iodepth_1;iodepth_2;iodepth_4;iodepth_8;iodepth_16;iodepth_32;iodepth_64;lat_2us;lat_4us;lat_10us;lat_20us;lat_50us;lat_100us;lat_250us;lat_500us;lat_750us;lat_1000us;lat_2ms;lat_4ms;lat_10ms;lat_20ms;lat_50ms;lat_100ms;lat_250ms;lat_500ms;lat_750ms;lat_1000ms;lat_2000ms;lat_over_2000ms;disk_name;disk_read_iops;disk_write_iops;disk_read_merges;disk_write_merges;disk_read_ticks;write_ticks;disk_queue_time;disk_util +JSON output +------------ + +The `json` output format is intended to be both human readable and convenient +for automated parsing. For the most part its sections mirror those of the +`normal` output. The `runtime` value is reported in msec and the `bw` value is +reported in 1024 bytes per second units. + + JSON+ output ------------