From: Sitsofe Wheeler Date: Wed, 22 Nov 2017 22:58:10 +0000 (+0000) Subject: doc: reword buffer_compress_percentage, buffer_compress_chunk, dedupe_percentage X-Git-Tag: fio-3.3~67^2 X-Git-Url: https://git.kernel.dk/?p=fio.git;a=commitdiff_plain;h=725927803e15a56fafc4952a78c6810efae6cebb doc: reword buffer_compress_percentage, buffer_compress_chunk, dedupe_percentage Make the documentation for these options better reflect how buffer compression and dedupe tuning can be used since commit 1de80624466405bccdbc4607d71cd249320da3f1 ("Default buffer_compress_chunk to 512"). Signed-off-by: Sitsofe Wheeler --- diff --git a/HOWTO b/HOWTO index 70f93724..4d3a8c8c 100644 --- a/HOWTO +++ b/HOWTO @@ -1434,9 +1434,12 @@ Buffers and memory .. option:: refill_buffers If this option is given, fio will refill the I/O buffers on every - submit. The default is to only fill it at init time and reuse that - data. Only makes sense if zero_buffers isn't specified, naturally. If data - verification is enabled, `refill_buffers` is also automatically enabled. + submit. Only makes sense if :option:`zero_buffers` isn't specified, + naturally. Defaults to being unset i.e., the buffer is only filled at + init time and the data in it is reused when possible but if any of + :option:`verify`, :option:`buffer_compress_percentage` or + :option:`dedupe_percentage` are enabled then `refill_buffers` is also + automatically enabled. .. option:: scramble_buffers=bool @@ -1448,23 +1451,30 @@ Buffers and memory .. option:: buffer_compress_percentage=int - If this is set, then fio will attempt to provide I/O buffer content (on - WRITEs) that compresses to the specified level. Fio does this by providing a - mix of random data and a fixed pattern. The fixed pattern is either zeros, - or the pattern specified by :option:`buffer_pattern`. If the pattern option - is used, it might skew the compression ratio slightly. Note that this is per - block size unit, see :option:`buffer_compress_chunk` for setting a finer - granularity of compression regions. + If this is set, then fio will attempt to provide I/O buffer content + (on WRITEs) that compresses to the specified level. Fio does this by + providing a mix of random data followed by fixed pattern data. The + fixed pattern is either zeros, or the pattern specified by + :option:`buffer_pattern`. If the `buffer_pattern` option is used, it + might skew the compression ratio slightly. Setting + `buffer_compress_percentage` to a value other than 100 will also + enable :option:`refill_buffers` in order to reduce the likelihood that + adjacent blocks are so similar that they over compress when seen + together. See :option:`buffer_compress_chunk` for how to set a finer or + coarser granularity for the random/fixed data region. Defaults to unset + i.e., buffer data will not adhere to any compression level. .. option:: buffer_compress_chunk=int - See :option:`buffer_compress_percentage`. This setting allows fio to manage - how big the ranges of random data and zeroed data is. Without this set, fio - will provide :option:`buffer_compress_percentage` of blocksize random data, - followed by the remaining zeroed. With this set to some chunk size smaller - than the block size, fio can alternate random and zeroed data throughout the - I/O buffer. This is particularly useful when bigger block sizes are used - for a job. Defaults to 512. + This setting allows fio to manage how big the random/fixed data region + is when using :option:`buffer_compress_percentage`. When + `buffer_compress_chunk` is set to some non-zero value smaller than the + block size, fio can repeat the random/fixed region throughout the I/O + buffer at the specified interval (which particularly useful when + bigger block sizes are used for a job). When set to 0, fio will use a + chunk size that matches the block size resulting in a single + random/fixed region within the I/O buffer. Defaults to 512. When the + unit is omitted, the value is interpreted in bytes. .. option:: buffer_pattern=str @@ -1501,7 +1511,9 @@ Buffers and memory writing. These buffers will be naturally dedupable. The contents of the buffers depend on what other buffer compression settings have been set. It's possible to have the individual buffers either fully compressible, or not at - all. This option only controls the distribution of unique buffers. + all -- this option only controls the distribution of unique buffers. Setting + this option will also enable :option:`refill_buffers` to prevent every buffer + being identical. .. option:: invalidate=bool diff --git a/fio.1 b/fio.1 index 2be64d28..3224e9a1 100644 --- a/fio.1 +++ b/fio.1 @@ -1237,22 +1237,29 @@ more clever block compression attempts, but it will stop naive dedupe of blocks. Default: true. .TP .BI buffer_compress_percentage \fR=\fPint -If this is set, then fio will attempt to provide I/O buffer content (on -WRITEs) that compresses to the specified level. Fio does this by providing a -mix of random data and a fixed pattern. The fixed pattern is either zeros, -or the pattern specified by \fBbuffer_pattern\fR. If the pattern option -is used, it might skew the compression ratio slightly. Note that this is per -block size unit, see \fBbuffer_compress_chunk\fR for setting a finer granularity -of compressible regions. +If this is set, then fio will attempt to provide I/O buffer content +(on WRITEs) that compresses to the specified level. Fio does this by +providing a mix of random data followed by fixed pattern data. The +fixed pattern is either zeros, or the pattern specified by +\fBbuffer_pattern\fR. If the \fBbuffer_pattern\fR option is used, it +might skew the compression ratio slightly. Setting +\fBbuffer_compress_percentage\fR to a value other than 100 will also +enable \fBrefill_buffers\fR in order to reduce the likelihood that +adjacent blocks are so similar that they over compress when seen +together. See \fBbuffer_compress_chunk\fR for how to set a finer or +coarser granularity of the random/fixed data regions. Defaults to unset +i.e., buffer data will not adhere to any compression level. .TP .BI buffer_compress_chunk \fR=\fPint -See \fBbuffer_compress_percentage\fR. This setting allows fio to manage -how big the ranges of random data and zeroed data is. Without this set, fio -will provide \fBbuffer_compress_percentage\fR of blocksize random data, -followed by the remaining zeroed. With this set to some chunk size smaller -than the block size, fio can alternate random and zeroed data throughout the -I/O buffer. This is particularly useful when bigger block sizes are used -for a job. Defaults to 512. +This setting allows fio to manage how big the random/fixed data region +is when using \fBbuffer_compress_percentage\fR. When +\fBbuffer_compress_chunk\fR is set to some non-zero value smaller than the +block size, fio can repeat the random/fixed region throughout the I/O +buffer at the specified interval (which particularly useful when +bigger block sizes are used for a job). When set to 0, fio will use a +chunk size that matches the block size resulting in a single +random/fixed region within the I/O buffer. Defaults to 512. When the +unit is omitted, the value is interpreted in bytes. .TP .BI buffer_pattern \fR=\fPstr If set, fio will fill the I/O buffers with this pattern or with the contents @@ -1295,7 +1302,9 @@ If set, fio will generate this percentage of identical buffers when writing. These buffers will be naturally dedupable. The contents of the buffers depend on what other buffer compression settings have been set. It's possible to have the individual buffers either fully compressible, or not at -all. This option only controls the distribution of unique buffers. +all \-\- this option only controls the distribution of unique buffers. Setting +this option will also enable \fBrefill_buffers\fR to prevent every buffer +being identical. .TP .BI invalidate \fR=\fPbool Invalidate the buffer/page cache parts of the files to be used prior to