X-Git-Url: https://git.kernel.dk/?p=fio.git;a=blobdiff_plain;f=HOWTO;h=e18eadb11811879ac20dd1272041c95d5bf2cbde;hp=e94d09f334d66fa58fe4cd76391a8f6e7f03e382;hb=7b38411c07fefcc3a79ed66014a8c502d4cf8335;hpb=aee2ab6775d96609a4632703827c409a7f9abcca diff --git a/HOWTO b/HOWTO index e94d09f3..e18eadb1 100644 --- a/HOWTO +++ b/HOWTO @@ -159,6 +159,41 @@ specify: $ fio --name=random-writers --ioengine=libaio --iodepth=4 --rw=randwrite --bs=32k --direct=0 --size=64m --numjobs=4 +When fio is utilized as a basis of any reasonably large test suite, it might be +desirable to share a set of standardized settings across multiple job files. +Instead of copy/pasting such settings, any section may pull in an external +.fio file with 'include filename' directive, as in the following example: + +; -- start job file including.fio -- +[global] +filename=/tmp/test +filesize=1m +include glob-include.fio + +[test] +rw=randread +bs=4k +time_based=1 +runtime=10 +include test-include.fio +; -- end job file including.fio -- + +; -- start job file glob-include.fio -- +thread=1 +group_reporting=1 +; -- end job file glob-include.fio -- + +; -- start job file test-include.fio -- +ioengine=libaio +iodepth=4 +; -- end job file test-include.fio -- + +Settings pulled into a section apply to that section only (except global +section). Include directives may be nested in that any included file may +contain further include directive(s). Include files may not contain [] +sections. + + 4.1 Environment variables ------------------------- @@ -218,7 +253,20 @@ machine. This section describes in details each parameter associated with a job. Some parameters take an option of a given type, such as an integer or -a string. The following types are used: +a string. Anywhere a numeric value is required, an arithmetic expression +may be used, provided it is surrounded by parentheses. Supported operators +are: + + addition (+) + subtraction (-) + multiplication (*) + division (/) + modulus (%) + exponentiation (^) + +For time values in expressions, units are microseconds by default. This is +different than for time values not in expressions (not enclosed in +parentheses). The following types are used: str String. This is a sequence of alpha characters. time Integer with possible time suffix. In seconds unless otherwise @@ -344,7 +392,7 @@ rw=str Type of io pattern. Accepted values are: For certain types of io the result may still be skewed a bit, since the speed may be different. It is possible to specify a number of IO's to do before getting a new offset, this is - one by appending a ':' to the end of the string given. + done by appending a ':' to the end of the string given. For a random read, it would look like 'rw=randread:8' for passing in an offset modifier with a value of 8. If the suffix is used with a sequential IO pattern, then the value @@ -519,7 +567,7 @@ bssplit=str Sometimes you want even finer grained control of the while having 90% 4k writes and 10% 8k writes, you would specify: - bssplit=2k/50:4k/50,4k/90,8k/10 + bssplit=2k/50:4k/50,4k/90:8k/10 blocksize_unaligned bs_unaligned If this option is given, any byte size value within bsrange @@ -565,10 +613,20 @@ buffer_compress_chunk=int See buffer_compress_percentage. This alternate random and zeroed data throughout the IO buffer. -buffer_pattern=str If set, fio will fill the io buffers with this pattern. - If not set, the contents of io buffers is defined by the other - options related to buffer contents. The setting can be any - pattern of bytes, and can be prefixed with 0x for hex values. +buffer_pattern=str If set, fio will fill the io buffers with this + pattern. If not set, the contents of io buffers is defined by + the other options related to buffer contents. The setting can + be any pattern of bytes, and can be prefixed with 0x for hex + values. It may also be a string, where the string must then + be wrapped with "". + +dedupe_percentage=int If set, fio will generate this percentage of + identical buffers when writing. These buffers will be + naturally dedupable. The contents of the buffers depend on + what other buffer compression settings have been set. It's + possible to have the individual buffers either fully + compressible, or not at all. This option only controls the + distribution of unique buffers. nrfiles=int Number of files to use for this job. Defaults to 1. @@ -670,15 +728,45 @@ ioengine=str Defines how the job issues io to the file. The following channel semantics (Send/Recv) for the InfiniBand, RoCE and iWARP protocols. - falloc IO engine that does regular fallocate to - simulate data transfer as fio ioengine. - DDIR_READ does fallocate(,mode = keep_size,) - DDIR_WRITE does fallocate(,mode = 0) - DDIR_TRIM does fallocate(,mode = punch_hole) + falloc IO engine that does regular fallocate to + simulate data transfer as fio ioengine. + DDIR_READ does fallocate(,mode = keep_size,) + DDIR_WRITE does fallocate(,mode = 0) + DDIR_TRIM does fallocate(,mode = punch_hole) e4defrag IO engine that does regular EXT4_IOC_MOVE_EXT - ioctls to simulate defragment activity in - request to DDIR_WRITE event + ioctls to simulate defragment activity in + request to DDIR_WRITE event + + rbd IO engine supporting direct access to Ceph + Rados Block Devices (RBD) via librbd without + the need to use the kernel rbd driver. This + ioengine defines engine specific options. + + gfapi Using Glusterfs libgfapi sync interface to + direct access to Glusterfs volumes without + options. + + gfapi_async Using Glusterfs libgfapi async interface + to direct access to Glusterfs volumes without + having to go through FUSE. This ioengine + defines engine specific options. + + libhdfs Read and write through Hadoop (HDFS). + The 'filename' option is used to specify host, + port of the hdfs name-node to connect. This + engine interprets offsets a little + differently. In HDFS, files once created + cannot be modified. So random writes are not + possible. To imitate this, libhdfs engine + expects bunch of small files to be created + over HDFS, and engine will randomly pick a + file out of those files based on the offset + generated by fio backend. (see the example + job file to create such files, use rw=write + option). Please note, you might want to set + necessary environment variables to work with + hdfs/libhdfs properly. external Prefix to specify loading an external IO engine object file. Append the engine @@ -736,18 +824,21 @@ offset=int Start io at the given offset in the file. The data before caps the file size at real_size - offset. offset_increment=int If this is provided, then the real offset becomes - the offset + offset_increment * thread_number, where the - thread number is a counter that starts at 0 and is incremented - for each job. This option is useful if there are several jobs - which are intended to operate on a file in parallel in disjoint - segments, with even spacing between the starting points. + offset + offset_increment * thread_number, where the thread + number is a counter that starts at 0 and is incremented for + each sub-job (i.e. when numjobs option is specified). This + option is useful if there are several jobs which are intended + to operate on a file in parallel disjoint segments, with + even spacing between the starting points. number_ios=int Fio will normally perform IOs until it has exhausted the size of the region set by size=, or if it exhaust the allocated time (or hits an error condition). With this setting, the range/size can be set independently of the number of IOs to perform. When fio reaches this number, it will exit normally - and report status. + and report status. Note that this does not extend the amount + of IO that will be done, it will only stop fio if this + condition is met before other end-of-job criteria. fsync=int If writing to a file, issue a sync of the dirty data for every number of blocks given. For example, if you give @@ -1307,7 +1398,9 @@ write_bw_log=str If given, write a bandwidth log of the jobs in this job jobs in their lifetime. The included fio_generate_plots script uses gnuplot to turn these text files into nice graphs. See write_lat_log for behaviour of given - filename. For this option, the suffix is _bw.log. + filename. For this option, the suffix is _bw.x.log, where + x is the index of the job (1..N, where N is the number of + jobs). write_lat_log=str Same as write_bw_log, except that this option stores io submission, completion, and total latencies instead. If no @@ -1317,14 +1410,16 @@ write_lat_log=str Same as write_bw_log, except that this option stores io write_lat_log=foo - The actual log names will be foo_slat.log, foo_clat.log, - and foo_lat.log. This helps fio_generate_plot fine the logs - automatically. + The actual log names will be foo_slat.x.log, foo_clat.x.log, + and foo_lat.x.log, where x is the index of the job (1..N, + where N is the number of jobs). This helps fio_generate_plot + fine the logs automatically. write_iops_log=str Same as write_bw_log, but writes IOPS. If no filename is given with this option, the default filename of - "jobname_type.log" is used. Even if the filename is given, - fio will still append the type of log. + "jobname_type.x.log" is used,where x is the index of the job + (1..N, where N is the number of jobs). Even if the filename + is given, fio will still append the type of log. log_avg_msec=int By default, fio will log an entry in the iops, latency, or bw log for every IO that completes. When writing to the @@ -1350,6 +1445,12 @@ log_compression=int If this is set, fio will compress the IO logs as in the specified log file. This feature depends on the availability of zlib. +log_store_compressed=bool If set, and log_compression is also set, + fio will store the log files in a compressed format. They + can be decompressed with fio, using the --inflate-log + command line parameter. The files will be stored with a + .fz suffix. + lockmem=int Pin down the specified amount of memory with mlock(2). Can potentially be used instead of removing memory or booting with less memory to simulate a smaller amount of memory. @@ -1539,7 +1640,9 @@ that defines them is selected. address. [netsplice] port=int -[net] port=int The TCP or UDP port to bind to or connect to. +[net] port=int The TCP or UDP port to bind to or connect to. If this is used +with numjobs to spawn multiple instances of the same job type, then this will +be the starting port number since fio will use a range of ports. [netsplice] interface=str [net] interface=str The IP address of the network interface used to send or @@ -1571,6 +1674,7 @@ that defines them is selected. [net] listen For TCP network connections, tell fio to listen for incoming connections rather than initiating an outgoing connection. The hostname must be omitted if this option is used. + [net] pingpong Normaly a network writer will just continue writing data, and a network reader will just consume packages. If pingpong=1 is set, a writer will send its normal payload to the reader, @@ -1583,6 +1687,10 @@ that defines them is selected. single reader when multiple readers are listening to the same address. +[net] window_size Set the desired socket buffer size for the connection. + +[net] mss Set the TCP maximum segment size (TCP_MAXSEG). + [e4defrag] donorname=str File will be used as a block donor(swap extents between files) [e4defrag] inplace=int