X-Git-Url: https://git.kernel.dk/?p=fio.git;a=blobdiff_plain;f=HOWTO;h=181e94ca99a68c39c9f082916c89e0034fbfba62;hp=40233bd90e5c1cf43a47541b8367506a46327dc5;hb=169c098dd3e2081679cf42978ea926e8dac62801;hpb=82407585a3b3b10281428e4ca9c2d3e7dfca7392 diff --git a/HOWTO b/HOWTO index 40233bd9..181e94ca 100644 --- a/HOWTO +++ b/HOWTO @@ -10,6 +10,9 @@ Table of contents 7. Terse output 8. Trace file format 9. CPU idleness profiling +10. Verification and triggers +11. Log File Formats + 1.0 Overview and history ------------------------ @@ -50,8 +53,8 @@ bottom, it contains the following basic parameters: IO engine How do we issue io? We could be memory mapping the file, we could be using regular read/write, we - could be using splice, async io, syslet, or even - SG (SCSI generic sg). + could be using splice, async io, or even SG + (SCSI generic sg). IO depth If the io engine is async, how large a queuing depth do we want to maintain? @@ -294,7 +297,7 @@ irange Integer range with suffix. Allows value range to be given, such 1k:4k. If the option allows two sets of ranges, they can be specified with a ',' or '/' delimiter: 1k-4k/8k-32k. Also see int. -float_list A list of floating numbers, separated by a ':' character. +float_list A list of floating point numbers, separated by a ':' character. With the above in mind, here follows the complete list of fio job parameters. @@ -305,6 +308,16 @@ name=str ASCII name of the job. This may be used to override the special purpose of also signaling the start of a new job. +wait_for=str Specifies the name of the already defined job to wait + for. Single waitee name only may be specified. If set, the job + won't be started until all workers of the waitee job are done. + + Wait_for operates on the job name basis, so there are a few + limitations. First, the waitee must be defined prior to the + waiter job (meaning no forward references). Second, if a job + is being referenced as a waitee, it must have a unique name + (no duplicate waitees). + description=str Text description of the job. Doesn't do anything except dump this text description when this job is run. It's not parsed. @@ -316,18 +329,15 @@ directory=str Prefix filenames with this directory. Used to place files filename=str Fio normally makes up a filename based on the job name, thread number, and file number. If you want to share files between threads in a job or several jobs, specify - a filename for each of them to override the default. If - the ioengine used is 'net', the filename is the host, port, - and protocol to use in the format of =host,port,protocol. - See ioengine=net for more. If the ioengine is file based, you - can specify a number of files by separating the names with a - ':' colon. So if you wanted a job to open /dev/sda and /dev/sdb - as the two working files, you would use - filename=/dev/sda:/dev/sdb. On Windows, disk devices are - accessed as \\.\PhysicalDrive0 for the first device, - \\.\PhysicalDrive1 for the second etc. Note: Windows and - FreeBSD prevent write access to areas of the disk containing - in-use data (e.g. filesystems). + a filename for each of them to override the default. + If the ioengine is file based, you can specify a number of + files by separating the names with a ':' colon. So if you + wanted a job to open /dev/sda and /dev/sdb as the two working + files, you would use filename=/dev/sda:/dev/sdb. On Windows, + disk devices are accessed as \\.\PhysicalDrive0 for the first + device, \\.\PhysicalDrive1 for the second etc. Note: Windows + and FreeBSD prevent write access to areas of the disk + containing in-use data (e.g. filesystems). If the wanted filename does need to include a colon, then escape that with a '\' character. For instance, if the filename is "/dev/dsk/foo@3,0:c", then you would use @@ -361,6 +371,11 @@ filename_format=str default of $jobname.$jobnum.$filenum will be used if no other format specifier is given. +unique_filename=bool To avoid collisions between networked clients, fio + defaults to prefixing any generated filenames (with a directory + specified) with the source of the client connecting. To disable + this behavior, set this option to 0. + opendir=str Tell fio to recursively add any file it can find in this directory and down the file system tree. @@ -383,13 +398,15 @@ rw=str Type of io pattern. Accepted values are: read Sequential reads write Sequential writes + trim Sequential trims randwrite Random writes randread Random reads + randtrim Random trims rw,readwrite Sequential mixed reads and writes randrw Random mixed reads and writes - trimwrite Mixed trims and writes. Blocks will be - trimmed first, then written to. + trimwrite Sequential trim+write sequences + Fio defaults to read if the option is not specified. For the mixed io types, the default is to split them 50/50. For certain types of io the result may still be skewed a bit, since the speed may be different. It is possible to specify @@ -428,7 +445,7 @@ kb_base=int The base unit for a kilobyte. The defacto base is 2^10, 1024. 1024 or 1000, with 1024 being the default. unified_rw_reporting=bool Fio normally reports statistics on a per - data direction basis, meaning that read, write, and trim are + data direction basis, meaning that reads, writes, and trims are accounted and reported separately. If this option is set, the fio will sum the results and report them as "mixed" instead. @@ -460,8 +477,17 @@ fadvise_hint=bool By default, fio will use fadvise() to advise the kernel on what IO patterns it is likely to issue. Sometimes you want to test specific IO patterns without telling the kernel about it, in which case you can disable this option. - If set, fio will use POSIX_FADV_SEQUENTIAL for sequential - IO and POSIX_FADV_RANDOM for random IO. + The following options are supported: + + sequential Use FADV_SEQUENTIAL + random Use FADV_RANDOM + 1 Backwards-compatible hint for basing + the hint on the fio workload. Will use + FADV_SEQUENTIAL for a sequential + workload, and FADV_RANDOM for a random + workload. + 0 Backwards-compatible setting for not + issing a fadvise hint. fadvise_stream=int Notify the kernel what write stream ID to place these writes under. Only supported on Linux. Note, this option @@ -660,10 +686,23 @@ file_service_type=str Defines how fio decides which file from a job to the next. Multiple files can still be open depending on 'openfiles'. - The string can have a number appended, indicating how - often to switch to a new file. So if option random:4 is - given, fio will switch to a new random file after 4 ios - have been issued. + zipf Use a zipfian distribution to decide what file + to access. + + pareto Use a pareto distribution to decide what file + to access. + + gauss Use a gaussian (normal) distribution to decide + what file to access. + + For random, roundrobin, and sequential, a postfix can be + appended to tell fio how many I/Os to issue before switching + to a new file. For example, specifying + 'file_service_type=random:8' would cause fio to issue 8 I/Os + before selecting a new file at random. For the non-uniform + distributions, a floating point postfix can be given to + influence how the distribution is skewed. See + 'random_distribution' for a description of how that would work. ioengine=str Defines how the job issues io to the file. The following types are defined: @@ -671,11 +710,14 @@ ioengine=str Defines how the job issues io to the file. The following sync Basic read(2) or write(2) io. lseek(2) is used to position the io location. - psync Basic pread(2) or pwrite(2) io. + psync Basic pread(2) or pwrite(2) io. Default on all + supported operating systems except for Windows. vsync Basic readv(2) or writev(2) IO. - psyncv Basic preadv(2) or pwritev(2) IO. + pvsync Basic preadv(2) or pwritev(2) IO. + + pvsync2 Basic preadv2(2) or pwritev2(2) IO. libaio Linux native asynchronous io. Note that Linux may only support queued behaviour with @@ -687,6 +729,7 @@ ioengine=str Defines how the job issues io to the file. The following solarisaio Solaris native asynchronous io. windowsaio Windows native asynchronous io. + Default on Windows. mmap File is memory mapped and data copied to/from using memcpy(3). @@ -695,9 +738,6 @@ ioengine=str Defines how the job issues io to the file. The following vmsplice(2) to transfer data from user space to the kernel. - syslet-rw Use the syslet system calls to make - regular read/write async. - sg SCSI generic sg v3 io. May either be synchronous using the SG_IO ioctl, or if the target is an sg character device @@ -722,12 +762,13 @@ ioengine=str Defines how the job issues io to the file. The following cpuio Doesn't transfer any data, but burns CPU cycles according to the cpuload= and - cpucycle= options. Setting cpuload=85 + cpuchunks= options. Setting cpuload=85 will cause that job to do nothing but burn 85% of the CPU. In case of SMP machines, use numjobs= to get desired CPU usage, as the cpuload only loads a single - CPU at the desired rate. + CPU at the desired rate. A job never finishes + unless there is at least one non-cpuio job. guasi The GUASI IO engine is the Generic Userspace Asyncronous Syscall Interface approach @@ -767,20 +808,14 @@ ioengine=str Defines how the job issues io to the file. The following defines engine specific options. libhdfs Read and write through Hadoop (HDFS). - The 'filename' option is used to specify host, - port of the hdfs name-node to connect. This - engine interprets offsets a little + This engine interprets offsets a little differently. In HDFS, files once created cannot be modified. So random writes are not possible. To imitate this, libhdfs engine - expects bunch of small files to be created - over HDFS, and engine will randomly pick a - file out of those files based on the offset - generated by fio backend. (see the example - job file to create such files, use rw=write - option). Please note, you might want to set - necessary environment variables to work with - hdfs/libhdfs properly. + creates bunch of small files, and engine will + pick a file out of those files based on the + offset generated by fio backend. Each jobs uses + it's own connection to HDFS. mtd Read, write and erase an MTD character device (e.g., /dev/mtd0). Discards are treated as @@ -791,6 +826,12 @@ ioengine=str Defines how the job issues io to the file. The following overwriting. The writetrim mode works well for this constraint. + pmemblk Read and write through the NVML libpmemblk + interface. + + dev-dax Read and write through a DAX device exposed + from persistent memory. + external Prefix to specify loading an external IO engine object file. Append the engine filename, eg ioengine=external:/tmp/foo.o @@ -836,7 +877,7 @@ iodepth_batch_complete_max=int This defines maximum pieces of IO to iodepth_batch_complete_min=1 iodepth_batch_complete_max= - which means that we will retrieve at leat 1 IO and up to the + which means that we will retrieve at least 1 IO and up to the whole submitted queue depth. If none of IO has been completed yet, we will wait. @@ -958,6 +999,8 @@ random_distribution=str:float By default, fio will use a completely uniform random Uniform random distribution zipf Zipf distribution pareto Pareto distribution + gauss Normal (gaussian) distribution + zoned Zoned random distribution When using a zipf or pareto distribution, an input value is also needed to define the access pattern. For zipf, this @@ -966,7 +1009,28 @@ random_distribution=str:float By default, fio will use a completely uniform what the given input values will yield in terms of hit rates. If you wanted to use zipf with a theta of 1.2, you would use random_distribution=zipf:1.2 as the option. If a non-uniform - model is used, fio will disable use of the random map. + model is used, fio will disable use of the random map. For + the gauss distribution, a normal deviation is supplied as + a value between 0 and 100. + + For a zoned distribution, fio supports specifying percentages + of IO access that should fall within what range of the file or + device. For example, given a criteria of: + + 60% of accesses should be to the first 10% + 30% of accesses should be to the next 20% + 8% of accesses should be to to the next 30% + 2% of accesses should be to the next 40% + + we can define that through zoning of the random accesses. For + the above example, the user would do: + + random_distribution=zoned:60/10:30/20:8/30:2/40 + + similarly to how bssplit works for setting ranges and + percentages of block sizes. Like bssplit, it's possible to + specify separate zones for reads, writes, and trims. If just + one set is given, it'll apply to all of them. percentage_random=int For a random workload, set how big a percentage should be random. This defaults to 100%, in which case the workload @@ -976,7 +1040,7 @@ percentage_random=int For a random workload, set how big a percentage should and random IO, at the given percentages. It is possible to set different values for reads, writes, and trim. To do so, simply use a comma separated list. See blocksize. - + norandommap Normally fio will cover every block of the file when doing random IO. If this option is given, fio will just get a new random offset without looking at past io history. This @@ -1009,13 +1073,20 @@ random_generator=str Fio supports the following engines for generating typically good enough. LFSR only works with single block sizes, not with workloads that use multiple block sizes. If used with such a workload, fio may read or write - some blocks multiple times. + some blocks multiple times. The default value is tausworthe, + unless the required space exceeds 2^32 blocks. If it does, + then tausworthe64 is selected automatically. nice=int Run the job with the given nice value. See man nice(2). + On Windows, values less than -15 set the process class to "High"; + -1 through -15 set "Above Normal"; 1 through 15 "Below Normal"; + and above 15 "Idle" priority class. + prio=int Set the io priority value of this job. Linux limits us to a positive value between 0 and 7, with 0 being the highest. - See man ionice(1). + See man ionice(1). Refer to an appropriate manpage for + other operating systems since meaning of priority may differ. prioclass=int Set the io priority class. See man ionice(1). @@ -1049,7 +1120,7 @@ rate=int Cap the bandwidth used by this job. The number is in bytes/sec, will only limit writes (to 500KB/sec), the latter will only limit reads. -ratemin=int Tell fio to do whatever it can to maintain at least this +rate_min=int Tell fio to do whatever it can to maintain at least this bandwidth. Failing to meet this requirement, will cause the job to exit. The same format as rate is used for read vs write separation. @@ -1064,6 +1135,15 @@ rate_iops_min=int If fio doesn't meet this rate of IO, it will cause the job to exit. The same format as rate is used for read vs write separation. +rate_process=str This option controls how fio manages rated IO + submissions. The default is 'linear', which submits IO in a + linear fashion with fixed delays between IOs that gets + adjusted based on IO completion rates. If this is set to + 'poisson', fio will submit IO based on a more real world + random request flow, known as the Poisson process + (https://en.wikipedia.org/wiki/Poisson_process). The lambda + will be 10^6 / IOPS for the given workload. + latency_target=int If set, fio will attempt to find the max performance point that the given workload will run at while maintaining a latency below this target. The values is given in microseconds. @@ -1081,7 +1161,7 @@ latency_percentile=float The percentage of IOs that must fall within the max_latency=int If set, fio will exit the job if it exceeds this maximum latency. It will exit with an ETIME error. -ratecycle=int Average bandwidth for 'rate' and 'ratemin' over this number +rate_cycle=int Average bandwidth for 'rate' and 'rate_min' over this number of milliseconds. cpumask=int Set the CPU affinity of this job. The parameter given is a @@ -1112,13 +1192,13 @@ cpus_allowed_policy=str Set the policy of how fio distributes the CPUs one cpu per job. If not enough CPUs are given for the jobs listed, then fio will roundrobin the CPUs in the set. -numa_cpu_nodes=str Set this job running on spcified NUMA nodes' CPUs. The +numa_cpu_nodes=str Set this job running on specified NUMA nodes' CPUs. The arguments allow comma delimited list of cpu numbers, A-B ranges, or 'all'. Note, to enable numa options support, fio must be built on a system with libnuma-dev(el) installed. numa_mem_policy=str Set this job's memory policy and corresponding NUMA - nodes. Format of the argements: + nodes. Format of the arguments: [:] `mode' is one of the following memory policy: default, prefer, bind, interleave, local @@ -1151,6 +1231,48 @@ ramp_time=time If set, fio will run the specified workload for this amount thus it will increase the total runtime if a special timeout or runtime is specified. +steadystate=str:float +ss=str:float Define the criterion and limit for assessing steady state + performance. The first parameter designates the criterion + whereas the second parameter sets the threshold. When the + criterion falls below the threshold for the specified duration, + the job will stop. For example, iops_slope:0.1% will direct fio + to terminate the job when the least squares regression slope + falls below 0.1% of the mean IOPS. If group_reporting is + enabled this will apply to all jobs in the group. Below is the + list of available steady state assessment criteria. All + assessments are carried out using only data from the rolling + collection window. Threshold limits can be expressed as a fixed + value or as a percentage of the mean in the collection window. + iops Collect IOPS data. Stop the job if all + individual IOPS measurements are within the + specified limit of the mean IOPS (e.g., iops:2 + means that all individual IOPS values must be + within 2 of the mean, whereas iops:0.2% means + that all individual IOPS values must be within + 0.2% of the mean IOPS to terminate the job). + iops_slope + Collect IOPS data and calculate the least + squares regression slope. Stop the job if the + slope falls below the specified limit. + bw Collect bandwidth data. Stop the job if all + individual bandwidth measurements are within + the specified limit of the mean bandwidth. + bw_slope + Collect bandwidth data and calculate the least + squares regression slope. Stop the job if the + slope falls below the specified limit. + +steadystate_duration=time +ss_dur=time A rolling window of this duration will be used to judge whether + steady state has been reached. Data will be collected once per + second. The default is 0 which disables steady state detection. + +steadystate_ramp_time=time +ss_ramp=time Allow the job to run for the specified duration before + beginning data collection for checking the steady state job + termination criterion. The default is 0. + invalidate=bool Invalidate the buffer/page cache parts for this file prior to starting io. Defaults to true. @@ -1162,6 +1284,7 @@ mem=str Fio can use various types of memory as the io unit buffer. The allowed values are: malloc Use memory from malloc(3) as the buffers. + Default memory type. shm Use shared memory as the buffers. Allocated through shmget(2). @@ -1177,6 +1300,9 @@ mem=str Fio can use various types of memory as the io unit buffer. backing. Append filename after mmaphuge, ala mem=mmaphuge:/hugetlbfs/file + mmapshared Same as mmap, but use a MMAP_SHARED + mapping. + The area allocated is a function of the maximum allowed bs size for the job, multiplied by the io depth given. Note that for shmhuge and mmaphuge to work, the system must have @@ -1195,7 +1321,7 @@ mem=str Fio can use various types of memory as the io unit buffer. location should point there. So if it's mounted in /huge, you would use mem=mmaphuge:/huge/somefile. -iomem_align=int This indiciates the memory alignment of the IO memory buffers. +iomem_align=int This indicates the memory alignment of the IO memory buffers. Note that the given alignment is applied to the first IO unit buffer, if using iodepth the alignment of the following buffers are given by the bs used. In other words, if using a bs that is @@ -1215,13 +1341,20 @@ exitall When one job finishes, terminate the rest. The default is to wait for each job to finish, sometimes that is not the desired action. +exitall_on_error When one job finishes in error, terminate the rest. The + default is to wait for each job to finish. + bwavgtime=int Average the calculated bandwidth over the given time. Value - is specified in milliseconds. + is specified in milliseconds. If the job also does bandwidth + logging through 'write_bw_log', then the minimum of this option + and 'log_avg_msec' will be used. Default: 500ms. iopsavgtime=int Average the calculated IOPS over the given time. Value - is specified in milliseconds. + is specified in milliseconds. If the job also does IOPS logging + through 'write_iops_log', then the minimum of this option and + 'log_avg_msec' will be used. Default: 500ms. -create_serialize=bool If true, serialize the file creating for the jobs. +create_serialize=bool If true, serialize the file creation for the jobs. This may be handy to avoid interleaving of data files, which may greatly depend on the filesystem used and even the number of processors in the system. @@ -1252,7 +1385,7 @@ pre_read=bool If this is given, files will be pre-read into memory before starting the given IO operation. This will also clear the 'invalidate' flag, since it is pointless to pre-read and then drop the cache. This will only work for IO engines - that are seekable, since they allow you to read the same data + that are seek-able, since they allow you to read the same data multiple times. Thus it will not work on eg network or splice IO. @@ -1260,6 +1393,8 @@ unlink=bool Unlink the job files when done. Not the default, as repeated runs of that job would then waste time recreating the file set again and again. +unlink_each_loop=bool Unlink job files after each iteration or loop. + loops=int Run the specified number of iterations of this job. Used to repeat the same workload a given number of times. Defaults to 1. @@ -1292,7 +1427,7 @@ verify=str If writing to a file, fio can verify the file contents crc32c Use a crc32c sum of the data area and store it in the header of each block. - crc32c-intel Use hardware assisted crc32c calcuation + crc32c-intel Use hardware assisted crc32c calculation provided on SSE4.2 enabled processors. Falls back to regular software crc32c, if not supported by the system. @@ -1367,7 +1502,7 @@ verify_pattern=str If set, fio will fill the io buffers with this be a hex number that starts with either "0x" or "0X". Use with verify=str. Also, verify_pattern supports %o format, which means that for each block offset will be written and - then verifyied back, e.g.: + then verified back, e.g.: verify_pattern=%o @@ -1478,10 +1613,10 @@ read_iolog=str Open an iolog with the specified file name and replay the replay_no_stall=int When replaying I/O with read_iolog the default behavior is to attempt to respect the time stamps within the log and - replay them with the appropriate delay between IOPS. By + replay them with the appropriate delay between IOPS. By setting this variable fio will not respect the timestamps and attempt to replay them as fast as possible while still - respecting ordering. The result is the same I/O pattern to a + respecting ordering. The result is the same I/O pattern to a given device, but different timings. replay_redirect=str While replaying I/O patterns using read_iolog the @@ -1493,13 +1628,14 @@ replay_redirect=str While replaying I/O patterns using read_iolog the mapping. Replay_redirect causes all IOPS to be replayed onto the single specified device regardless of the device it was recorded from. i.e. replay_redirect=/dev/sdc would cause all - IO in the blktrace to be replayed onto /dev/sdc. This means - multiple devices will be replayed onto a single, if the trace - contains multiple devices. If you want multiple devices to be - replayed concurrently to multiple redirected devices you must - blkparse your trace into separate traces and replay them with - independent fio invocations. Unfortuantely this also breaks - the strict time ordering between multiple device accesses. + IO in the blktrace or iolog to be replayed onto /dev/sdc. + This means multiple devices will be replayed onto a single + device, if the trace contains multiple devices. If you want + multiple devices to be replayed concurrently to multiple + redirected devices you must blkparse your trace into separate + traces and replay them with independent fio invocations. + Unfortunately this also breaks the strict time ordering + between multiple device accesses. replay_align=int Force alignment of IO offsets and lengths in a trace to this power of 2 value. @@ -1519,7 +1655,7 @@ write_bw_log=str If given, write a bandwidth log of the jobs in this job filename. For this option, the suffix is _bw.x.log, where x is the index of the job (1..N, where N is the number of jobs). If 'per_job_logs' is false, then the filename will not - include the job index. + include the job index. See 'Log File Formats'. write_lat_log=str Same as write_bw_log, except that this option stores io submission, completion, and total latencies instead. If no @@ -1532,9 +1668,17 @@ write_lat_log=str Same as write_bw_log, except that this option stores io The actual log names will be foo_slat.x.log, foo_clat.x.log, and foo_lat.x.log, where x is the index of the job (1..N, where N is the number of jobs). This helps fio_generate_plot - fine the logs automatically. If 'per_job_logs' is false, then - the filename will not include the job index. - + find the logs automatically. If 'per_job_logs' is false, then + the filename will not include the job index. See 'Log File + Formats'. + +write_hist_log=str Same as write_lat_log, but writes I/O completion + latency histograms. If no filename is given with this option, the + default filename of "jobname_clat_hist.x.log" is used, where x is + the index of the job (1..N, where N is the number of jobs). Even + if the filename is given, fio will still append the type of log. + If per_job_logs is false, then the filename will not include the + job index. See 'Log File Formats'. write_iops_log=str Same as write_bw_log, but writes IOPS. If no filename is given with this option, the default filename of @@ -1542,14 +1686,33 @@ write_iops_log=str Same as write_bw_log, but writes IOPS. If no filename is (1..N, where N is the number of jobs). Even if the filename is given, fio will still append the type of log. If 'per_job_logs' is false, then the filename will not include - the job index. + the job index. See 'Log File Formats'. log_avg_msec=int By default, fio will log an entry in the iops, latency, or bw log for every IO that completes. When writing to the disk log, that can quickly grow to a very large size. Setting this option makes fio average the each log entry over the specified period of time, reducing the resolution of the log. - Defaults to 0. + See log_max_value as well. Defaults to 0, logging all entries. + +log_hist_msec=int Same as log_avg_msec, but logs entries for completion + latency histograms. Computing latency percentiles from averages of + intervals using log_avg_msec is innacurate. Setting this option makes + fio log histogram entries over the specified period of time, reducing + log sizes for high IOPS devices while retaining percentile accuracy. + See log_hist_coarseness as well. Defaults to 0, meaning histogram + logging is disabled. + +log_hist_coarseness=int Integer ranging from 0 to 6, defining the coarseness + of the resolution of the histogram logs enabled with log_hist_msec. For + each increment in coarseness, fio outputs half as many bins. Defaults to + 0, for which histogram logs contain 1216 latency bins. See + 'Log File Formats'. + +log_max_value=bool If log_avg_msec is set, fio logs the average over that + window. If you instead want to log the maximum value, set this + option to 1. Defaults to 0, meaning that averaged values are + logged. log_offset=int If this is set, the iolog options will include the byte offset for the IO entry as well as the other data values. @@ -1568,11 +1731,19 @@ log_compression=int If this is set, fio will compress the IO logs as in the specified log file. This feature depends on the availability of zlib. -log_store_compressed=bool If set, and log_compression is also set, - fio will store the log files in a compressed format. They - can be decompressed with fio, using the --inflate-log - command line parameter. The files will be stored with a - .fz suffix. +log_compression_cpus=str Define the set of CPUs that are allowed to + handle online log compression for the IO jobs. This can + provide better isolation between performance sensitive jobs, + and background compression work. + +log_store_compressed=bool If set, fio will store the log files in a + compressed format. They can be decompressed with fio, using + the --inflate-log command line parameter. The files will be + stored with a .fz suffix. + +log_unix_epoch=bool If set, fio will log Unix timestamps to the log + files produced by enabling write_type_log for each log type, instead + of the default zero-based timestamps. block_error_percentiles=bool If set, record errors in trim block-sized units from writes and trims and output a histogram of @@ -1754,23 +1925,28 @@ that defines them is selected. enabled when polling for a minimum of 0 events (eg when iodepth_batch_complete=0). -[cpu] cpuload=int Attempt to use the specified percentage of CPU cycles. +[psyncv2] hipri Set RWF_HIPRI on IO, indicating to the kernel that + it's of higher priority than normal. + +[cpuio] cpuload=int Attempt to use the specified percentage of CPU cycles. -[cpu] cpuchunks=int Split the load into cycles of the given time. In +[cpuio] cpuchunks=int Split the load into cycles of the given time. In microseconds. -[cpu] exit_on_io_done=bool Detect when IO threads are done, then exit. +[cpuio] exit_on_io_done=bool Detect when IO threads are done, then exit. [netsplice] hostname=str [net] hostname=str The host name or IP address to use for TCP or UDP based IO. If the job is a TCP listener or UDP reader, the hostname is not used and must be omitted unless it is a valid UDP multicast address. +[libhdfs] namenode=str The host name or IP address of a HDFS cluster namenode to contact. [netsplice] port=int [net] port=int The TCP or UDP port to bind to or connect to. If this is used with numjobs to spawn multiple instances of the same job type, then this will be the starting port number since fio will use a range of ports. +[libhdfs] port=int the listening port of the HFDS cluster namenode. [netsplice] interface=str [net] interface=str The IP address of the network interface used to send or @@ -1803,7 +1979,7 @@ be the starting port number since fio will use a range of ports. connections rather than initiating an outgoing connection. The hostname must be omitted if this option is used. -[net] pingpong Normaly a network writer will just continue writing data, and +[net] pingpong Normally a network writer will just continue writing data, and a network reader will just consume packages. If pingpong=1 is set, a writer will send its normal payload to the reader, then wait for the reader to send the same payload back. This @@ -1824,11 +2000,23 @@ be the starting port number since fio will use a range of ports. [e4defrag] inplace=int Configure donor file blocks allocation strategy 0(default): Preallocate donor's file on init - 1 : allocate space immidietly inside defragment event, + 1 : allocate space immediately inside defragment event, and free right after event +[rbd] clustername=str Specifies the name of the Ceph cluster. +[rbd] rbdname=str Specifies the name of the RBD. +[rbd] pool=str Specifies the name of the Ceph pool containing RBD. +[rbd] clientname=str Specifies the username (without the 'client.' prefix) + used to access the Ceph cluster. If the clustername is + specified, the clientname shall be the full type.id + string. If no type. prefix is given, fio will add + 'client.' by default. + [mtd] skip_bad=bool Skip operations against known bad blocks. +[libhdfs] hdfsdirectory libhdfs will create chunk in this HDFS directory +[libhdfs] chunk_size the size of the chunk to use for each file. + 6.0 Interpreting the output --------------------------- @@ -1924,7 +2112,9 @@ runt= The runtime of that thread cpu= CPU usage. User and system time, along with the number of context switches this thread went through, usage of system and user time, and finally the number of major - and minor page faults. + and minor page faults. The CPU utilization numbers are + averages for the jobs in that reporting group, while the + context and fault counters are summed. IO depths= The distribution of io depths over the job life time. The numbers are divided into powers of 2, so for example the 16= entries includes depths up to that value but higher @@ -2185,10 +2375,42 @@ localbox$ fio --client=server --trigger-file=/tmp/my-trigger --trigger="ipmi-reb For this case, fio would wait for the server to send us the write state, then execute 'ipmi-reboot server' when that happened. -10.1 Loading verify state +10.2 Loading verify state ------------------------- To load store write state, read verification job file must contain the verify_state_load option. If that is set, fio will load the previously stored state. For a local fio run this is done by loading the files directly, and on a client/server run, the server backend will ask the client to send the files over and load them from there. + + +11.0 Log File Formats +--------------------- + +Fio supports a variety of log file formats, for logging latencies, bandwidth, +and IOPS. The logs share a common format, which looks like this: + +time (msec), value, data direction, offset + +Time for the log entry is always in milliseconds. The value logged depends +on the type of log, it will be one of the following: + + Latency log Value is latency in usecs + Bandwidth log Value is in KB/sec + IOPS log Value is IOPS + +Data direction is one of the following: + + 0 IO is a READ + 1 IO is a WRITE + 2 IO is a TRIM + +The offset is the offset, in bytes, from the start of the file, for that +particular IO. The logging of the offset can be toggled with 'log_offset'. + +If windowed logging is enabled through 'log_avg_msec', then fio doesn't log +individual IOs. Instead of logs the average values over the specified +period of time. Since 'data direction' and 'offset' are per-IO values, +they aren't applicable if windowed logging is enabled. If windowed logging +is enabled and 'log_max_value' is set, then fio logs maximum values in +that window instead of averages.