X-Git-Url: https://git.kernel.dk/?a=blobdiff_plain;f=HOWTO;h=dce96bcd8103f075d56b21173659f3061f6600b1;hb=ae0db592368ce686a3ec9e00c57b4a1a0da0b9ed;hp=342bad803ac56adc062a153a74543a5d77f85f7c;hpb=44c82dba75d26d3445757d25d59fe87a056a5409;p=fio.git diff --git a/HOWTO b/HOWTO index 342bad80..dce96bcd 100644 --- a/HOWTO +++ b/HOWTO @@ -217,6 +217,9 @@ Command line options .. option:: --max-jobs=nr Set the maximum number of threads/processes to support to `nr`. + NOTE: On Linux, it may be necessary to increase the shared-memory + limit (:file:`/proc/sys/kernel/shmmax`) if fio runs into errors while + creating jobs. .. option:: --server=args @@ -230,7 +233,7 @@ Command line options .. option:: --client=hostname Instead of running the jobs locally, send and run them on the given `hostname` - or set of `hostname`s. See `Client/Server`_ section. + or set of `hostname`\s. See `Client/Server`_ section. .. option:: --remote-config=file @@ -792,6 +795,13 @@ Target file/device named :file:`testfiles.4`. The default of :file:`$jobname.$jobnum.$filenum` will be used if no other format specifier is given. + If you specify a path then the directories will be created up to the + main directory for the file. So for example if you specify + ``filename_format=a/b/c/$jobnum`` then the directories a/b/c will be + created before the file setup part of the job. If you specify + :option:`directory` then the path will be relative that directory, + otherwise it is treated as the absolute path. + .. option:: unique_filename=bool To avoid collisions between networked clients, fio defaults to prefixing any @@ -1118,13 +1128,20 @@ I/O type .. option:: offset=int Start I/O at the provided offset in the file, given as either a fixed size in - bytes or a percentage. If a percentage is given, the next ``blockalign``-ed - offset will be used. Data before the given offset will not be touched. This + bytes or a percentage. If a percentage is given, the generated offset will be + aligned to the minimum ``blocksize`` or to the value of ``offset_align`` if + provided. Data before the given offset will not be touched. This effectively caps the file size at `real_size - offset`. Can be combined with :option:`size` to constrain the start and end range of the I/O workload. A percentage can be specified by a number between 1 and 100 followed by '%', for example, ``offset=20%`` to specify 20%. +.. option:: offset_align=int + + If set to non-zero value, the byte offset generated by a percentage ``offset`` + is aligned upwards to this value. Defaults to 0 meaning that a percentage + offset is aligned to the minimum block size. + .. option:: offset_increment=int If this is provided, then the real offset becomes `offset + offset_increment @@ -1436,8 +1453,8 @@ Buffers and memory mix of random data and a fixed pattern. The fixed pattern is either zeros, or the pattern specified by :option:`buffer_pattern`. If the pattern option is used, it might skew the compression ratio slightly. Note that this is per - block size unit, for file/disk wide compression level that matches this - setting, you'll also want to set :option:`refill_buffers`. + block size unit, see :option:`buffer_compress_chunk` for setting a finer + granularity of compression regions. .. option:: buffer_compress_chunk=int @@ -1446,7 +1463,8 @@ Buffers and memory will provide :option:`buffer_compress_percentage` of blocksize random data, followed by the remaining zeroed. With this set to some chunk size smaller than the block size, fio can alternate random and zeroed data throughout the - I/O buffer. + I/O buffer. This is particularly useful when bigger block sizes are used + for a job. Defaults to 512. .. option:: buffer_pattern=str @@ -1705,7 +1723,7 @@ I/O engine Doesn't transfer any data, but burns CPU cycles according to the :option:`cpuload` and :option:`cpuchunks` options. Setting :option:`cpuload`\=85 will cause that job to do nothing but burn 85% - of the CPU. In case of SMP machines, use :option:`numjobs`= + of the CPU. In case of SMP machines, use :option:`numjobs`\= to get desired CPU usage, as the cpuload only loads a single CPU at the desired rate. A job never finishes unless there is at least one non-cpuio job. @@ -1797,6 +1815,15 @@ I/O engine absolute or relative. See :file:`engines/skeleton_external.c` for details of writing an external I/O engine. + **filecreate** + Simply create the files and do no I/O to them. You still need to + set `filesize` so that all the accounting still occurs, but no + actual I/O will be done other than creating the file. + + **libpmem** + Read and write using mmap I/O to a file on a filesystem + mounted with DAX on a persistent memory device through the NVML + libpmem library. I/O engine specific parameters ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -2039,7 +2066,7 @@ I/O depth changing data and the overlapping region has a non-zero size. Setting ``serialize_overlap`` tells fio to avoid provoking this behavior by explicitly serializing in-flight I/Os that have a non-zero overlap. Note that setting - this option can reduce both performance and the `:option:iodepth` achieved. + this option can reduce both performance and the :option:`iodepth` achieved. Additionally this option does not work when :option:`io_submit_mode` is set to offload. Default: false. @@ -2701,47 +2728,46 @@ Measurements and reporting .. option:: write_bw_log=str If given, write a bandwidth log for this job. Can be used to store data of - the bandwidth of the jobs in their lifetime. The included - :command:`fio_generate_plots` script uses :command:`gnuplot` to turn these - text files into nice graphs. See :option:`write_lat_log` for behavior of - given filename. For this option, the postfix is :file:`_bw.x.log`, where `x` - is the index of the job (`1..N`, where `N` is the number of jobs). If - :option:`per_job_logs` is false, then the filename will not include the job - index. See `Log File Formats`_. + the bandwidth of the jobs in their lifetime. -.. option:: write_lat_log=str + If no str argument is given, the default filename of + :file:`jobname_type.x.log` is used. Even when the argument is given, fio + will still append the type of log. So if one specifies:: - Same as :option:`write_bw_log`, except that this option stores I/O - submission, completion, and total latencies instead. If no filename is given - with this option, the default filename of :file:`jobname_type.log` is - used. Even if the filename is given, fio will still append the type of - log. So if one specifies:: + write_bw_log=foo - write_lat_log=foo + The actual log name will be :file:`foo_bw.x.log` where `x` is the index + of the job (`1..N`, where `N` is the number of jobs). If + :option:`per_job_logs` is false, then the filename will not include the + `.x` job index. - The actual log names will be :file:`foo_slat.x.log`, :file:`foo_clat.x.log`, - and :file:`foo_lat.x.log`, where `x` is the index of the job (`1..N`, where `N` - is the number of jobs). This helps :command:`fio_generate_plots` find the - logs automatically. If :option:`per_job_logs` is false, then the filename - will not include the job index. See `Log File Formats`_. + The included :command:`fio_generate_plots` script uses :command:`gnuplot` to turn these + text files into nice graphs. See `Log File Formats`_ for how data is + structured within the file. + +.. option:: write_lat_log=str + + Same as :option:`write_bw_log`, except this option creates I/O + submission (e.g., `file:`name_slat.x.log`), completion (e.g., + `file:`name_clat.x.log`), and total (e.g., `file:`name_lat.x.log`) + latency files instead. See :option:`write_bw_log` for details about + the filename format and `Log File Formats`_ for how data is structured + within the files. .. option:: write_hist_log=str - Same as :option:`write_lat_log`, but writes I/O completion latency - histograms. If no filename is given with this option, the default filename - of :file:`jobname_clat_hist.x.log` is used, where `x` is the index of the - job (`1..N`, where `N` is the number of jobs). Even if the filename is given, - fio will still append the type of log. If :option:`per_job_logs` is false, - then the filename will not include the job index. See `Log File Formats`_. + Same as :option:`write_bw_log` but writes an I/O completion latency + histogram file (e.g., `file:`name_hist.x.log`) instead. Note that this + file will be empty unless :option:`log_hist_msec` has also been set. + See :option:`write_bw_log` for details about the filename format and + `Log File Formats`_ for how data is structured within the file. .. option:: write_iops_log=str - Same as :option:`write_bw_log`, but writes IOPS. If no filename is given - with this option, the default filename of :file:`jobname_type.x.log` is - used, where `x` is the index of the job (`1..N`, where `N` is the number of - jobs). Even if the filename is given, fio will still append the type of - log. If :option:`per_job_logs` is false, then the filename will not include - the job index. See `Log File Formats`_. + Same as :option:`write_bw_log`, but writes an IOPS file (e.g. + `file:`name_iops.x.log`) instead. See :option:`write_bw_log` for + details about the filename format and `Log File Formats`_ for how data + is structured within the file. .. option:: log_avg_msec=int @@ -2759,15 +2785,16 @@ Measurements and reporting :option:`log_avg_msec` is inaccurate. Setting this option makes fio log histogram entries over the specified period of time, reducing log sizes for high IOPS devices while retaining percentile accuracy. See - :option:`log_hist_coarseness` as well. Defaults to 0, meaning histogram - logging is disabled. + :option:`log_hist_coarseness` and :option:`write_hist_log` as well. + Defaults to 0, meaning histogram logging is disabled. .. option:: log_hist_coarseness=int Integer ranging from 0 to 6, defining the coarseness of the resolution of the histogram logs enabled with :option:`log_hist_msec`. For each increment in coarseness, fio outputs half as many bins. Defaults to 0, for which - histogram logs contain 1216 latency bins. See `Log File Formats`_. + histogram logs contain 1216 latency bins. See :option:`write_hist_log` + and `Log File Formats`_. .. option:: log_max_value=bool @@ -2862,7 +2889,15 @@ Measurements and reporting .. option:: clat_percentiles=bool - Enable the reporting of percentiles of completion latencies. + Enable the reporting of percentiles of completion latencies. This + option is mutually exclusive with :option:`lat_percentiles`. + +.. option:: lat_percentiles=bool + + Enable the reporting of percentiles of I/O latencies. This is similar + to :option:`clat_percentiles`, except that this includes the + submission latency. This option is mutually exclusive with + :option:`clat_percentiles`. .. option:: percentile_list=float_list @@ -3229,7 +3264,7 @@ writes in the example above). In the order listed, they denote: short or dropped. **IO latency** - These values are for `--latency-target` and related options. When + These values are for :option:`latency_target` and related options. When these options are engaged, this section describes the I/O depth required to meet the specified latency target.