Merge branch 'offload-flags-fix' of https://github.com/vincentkfu/fio

[fio.git] / HOWTO

diff --git a/HOWTO b/HOWTO
index 4c117c2473d6693158f1b3f346387d342b8d55bd..0c5b7109df30c55c35bbc2d8441be0ee97d3ecb8 100644 (file)
--- a/HOWTO
+++ b/HOWTO
@@ -194,7 +194,10 @@ Command line options
        Force a full status dump of cumulative (from job start) values at `time`
        intervals. This option does *not* provide per-period measurements. So
        values such as bandwidth are running averages. When the time unit is omitted,
        Force a full status dump of cumulative (from job start) values at `time`
        intervals. This option does *not* provide per-period measurements. So
        values such as bandwidth are running averages. When the time unit is omitted,

-       `time` is interpreted in seconds.
+       `time` is interpreted in seconds. Note that using this option with
+       ``--output-format=json`` will yield output that technically isn't valid
+       json, since the output will be collated sets of valid json. It will need
+       to be split into valid sets of json after the run.

 
 .. option:: --section=name
 
 
 .. option:: --section=name
 


@@ -952,18 +955,92 @@ Target file/device
 
        Unlink job files after each iteration or loop.  Default: false.
 
 
        Unlink job files after each iteration or loop.  Default: false.
 

-.. option:: zonesize=int
+.. option:: zonemode=str
+
+       Accepted values are:

 
 

-       Divide a file into zones of the specified size. See :option:`zoneskip`.
+               **none**
+                               The :option:`zonerange`, :option:`zonesize` and
+                               :option:`zoneskip` parameters are ignored.
+               **strided**
+                               I/O happens in a single zone until
+                               :option:`zonesize` bytes have been transferred.
+                               After that number of bytes has been
+                               transferred processing of the next zone
+                               starts.
+               **zbd**
+                               Zoned block device mode. I/O happens
+                               sequentially in each zone, even if random I/O
+                               has been selected. Random I/O happens across
+                               all zones instead of being restricted to a
+                               single zone. The :option:`zoneskip` parameter
+                               is ignored. :option:`zonerange` and
+                               :option:`zonesize` must be identical.

 
 .. option:: zonerange=int
 
 
 .. option:: zonerange=int
 

-       Give size of an I/O zone.  See :option:`zoneskip`.
+       Size of a single zone. See also :option:`zonesize` and
+       :option:`zoneskip`.
+
+.. option:: zonesize=int
+
+       For :option:`zonemode` =strided, this is the number of bytes to
+       transfer before skipping :option:`zoneskip` bytes. If this parameter
+       is smaller than :option:`zonerange` then only a fraction of each zone
+       with :option:`zonerange` bytes will be accessed.  If this parameter is
+       larger than :option:`zonerange` then each zone will be accessed
+       multiple times before skipping to the next zone.
+
+       For :option:`zonemode` =zbd, this is the size of a single zone. The
+       :option:`zonerange` parameter is ignored in this mode.

 
 .. option:: zoneskip=int
 
 
 .. option:: zoneskip=int
 

-       Skip the specified number of bytes when :option:`zonesize` data has been
-       read. The two zone options can be used to only do I/O on zones of a file.
+       For :option:`zonemode` =strided, the number of bytes to skip after
+       :option:`zonesize` bytes of data have been transferred. This parameter
+       must be zero for :option:`zonemode` =zbd.
+
+.. option:: read_beyond_wp=bool
+
+       This parameter applies to :option:`zonemode` =zbd only.
+
+       Zoned block devices are block devices that consist of multiple zones.
+       Each zone has a type, e.g. conventional or sequential. A conventional
+       zone can be written at any offset that is a multiple of the block
+       size. Sequential zones must be written sequentially. The position at
+       which a write must occur is called the write pointer. A zoned block
+       device can be either drive managed, host managed or host aware. For
+       host managed devices the host must ensure that writes happen
+       sequentially. Fio recognizes host managed devices and serializes
+       writes to sequential zones for these devices.
+
+       If a read occurs in a sequential zone beyond the write pointer then
+       the zoned block device will complete the read without reading any data
+       from the storage medium. Since such reads lead to unrealistically high
+       bandwidth and IOPS numbers fio only reads beyond the write pointer if
+       explicitly told to do so. Default: false.
+
+.. option:: max_open_zones=int
+
+       When running a random write test across an entire drive many more
+       zones will be open than in a typical application workload. Hence this
+       command line option that allows to limit the number of open zones. The
+       number of open zones is defined as the number of zones to which write
+       commands are issued.
+
+.. option:: zone_reset_threshold=float
+
+       A number between zero and one that indicates the ratio of logical
+       blocks with data to the total number of logical blocks in the test
+       above which zones should be reset periodically.
+
+.. option:: zone_reset_frequency=float
+
+       A number between zero and one that indicates how often a zone reset
+       should be issued if the zone reset threshold has been exceeded. A zone
+       reset is submitted after each (1 / zone_reset_frequency) write
+       requests. This and the previous parameter can be used to simulate
+       garbage collection activity.

 
 
 I/O type
 
 
 I/O type


@@ -1829,6 +1906,15 @@ I/O engine
                        (RBD) via librbd without the need to use the kernel rbd driver. This
                        ioengine defines engine specific options.
 
                        (RBD) via librbd without the need to use the kernel rbd driver. This
                        ioengine defines engine specific options.
 

+               **http**
+                       I/O engine supporting GET/PUT requests over HTTP(S) with libcurl to
+                       a WebDAV or S3 endpoint.  This ioengine defines engine specific options.
+
+                       This engine only supports direct IO of iodepth=1; you need to scale this
+                       via numjobs. blocksize defines the size of the objects to be created.
+
+                       TRIM is translated to object deletion.
+

                **gfapi**
                        Using GlusterFS libgfapi sync interface to direct access to
                        GlusterFS volumes without having to go through FUSE.  This ioengine
                **gfapi**
                        Using GlusterFS libgfapi sync interface to direct access to
                        GlusterFS volumes without having to go through FUSE.  This ioengine


@@ -1886,6 +1972,22 @@ I/O engine
                        mounted with DAX on a persistent memory device through the PMDK
                        libpmem library.
 
                        mounted with DAX on a persistent memory device through the PMDK
                        libpmem library.
 

+               **ime_psync**
+                       Synchronous read and write using DDN's Infinite Memory Engine (IME).
+                       This engine is very basic and issues calls to IME whenever an IO is
+                       queued.
+
+               **ime_psyncv**
+                       Synchronous read and write using DDN's Infinite Memory Engine (IME).
+                       This engine uses iovecs and will try to stack as much IOs as possible
+                       (if the IOs are "contiguous" and the IO depth is not exceeded)
+                       before issuing a call to IME.
+
+               **ime_aio**
+                       Asynchronous read and write using DDN's Infinite Memory Engine (IME).
+                       This engine will try to stack as much IOs as possible by creating
+                       requests for IME. FIO will then decide when to commit these requests.
+

 I/O engine specific parameters
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 I/O engine specific parameters
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 


@@ -2109,6 +2211,54 @@ with the caveat that when used on the command line, they must come after the
                transferred to the device. The writefua option is ignored with this
                selection.
 
                transferred to the device. The writefua option is ignored with this
                selection.
 

+.. option:: http_host=str : [http]
+
+       Hostname to connect to. For S3, this could be the bucket hostname.
+       Default is **localhost**
+
+.. option:: http_user=str : [http]
+
+       Username for HTTP authentication.
+
+.. option:: http_pass=str : [http]
+
+       Password for HTTP authentication.
+
+.. option:: https=str : [http]
+
+       Enable HTTPS instead of http. *on* enables HTTPS; *insecure*
+       will enable HTTPS, but disable SSL peer verification (use with
+       caution!). Default is **off**
+
+.. option:: http_mode=str : [http]
+
+       Which HTTP access mode to use: *webdav*, *swift*, or *s3*.
+       Default is **webdav**
+
+.. option:: http_s3_region=str : [http]
+
+       The S3 region/zone string.
+       Default is **us-east-1**
+
+.. option:: http_s3_key=str : [http]
+
+       The S3 secret key.
+
+.. option:: http_s3_keyid=str : [http]
+
+       The S3 key/access id.
+
+.. option:: http_swift_auth_token=str : [http]
+
+       The Swift auth token. See the example configuration file on how
+       to retrieve this.
+
+.. option:: http_verbose=int : [http]
+
+       Enable verbose requests from libcurl. Useful for debugging. 1
+       turns on verbose logging from libcurl, 2 additionally enables
+       HTTP IO tracing. Default is **0**
+

 I/O depth
 ~~~~~~~~~
 
 I/O depth
 ~~~~~~~~~
 


@@ -2330,6 +2480,10 @@ I/O replay
        :manpage:`blktrace(8)` for how to capture such logging data. For blktrace
        replay, the file needs to be turned into a blkparse binary data file first
        (``blkparse <device> -o /dev/null -d file_for_fio.bin``).
        :manpage:`blktrace(8)` for how to capture such logging data. For blktrace
        replay, the file needs to be turned into a blkparse binary data file first
        (``blkparse <device> -o /dev/null -d file_for_fio.bin``).

+       You can specify a number of files by separating the names with a ':'
+       character. See the :option:`filename` option for information on how to
+       escape ':' and '\' characters within the file names. These files will
+       be sequentially assigned to job clones created by :option:`numjobs`.

 
 .. option:: read_iolog_chunked=bool
 
 
 .. option:: read_iolog_chunked=bool