path: root/HOWTO
diff options
authorBar David <>2021-06-17 15:39:58 +0300
committerBar David <>2021-07-15 08:55:15 +0300
commit0d71aa983a4dce75a088b3a4831d5b217df066fb (patch)
tree81b63a555ce0f7353067e3ad0e040fe80c9ac894 /HOWTO
parent77c72e0f504364adf6a0e8f1155fdf3fd68ef248 (diff)
dedupe: allow to generate dedupe buffers from working set
This commit introduced new dedupe generation mode "working_set". Working set mode simulates a more realistic approach to deduped data, in which deduped buffers are generated from pre-existing working set - % size of the device or file. In other words, dedupe is not usually expected to be close in time with the source buffer, as well as source buffers are usually composed of small subset of the entire file or device. Signed-off-by: Bar David <>
Diffstat (limited to 'HOWTO')
1 files changed, 30 insertions, 0 deletions
diff --git a/HOWTO b/HOWTO
index 86fb2964..a12bccba 100644
--- a/HOWTO
+++ b/HOWTO
@@ -1705,6 +1705,36 @@ Buffers and memory
this option will also enable :option:`refill_buffers` to prevent every buffer
being identical.
+.. option:: dedupe_mode=str
+ If ``dedupe_percentage=<int>`` is given, then this option controls how fio
+ generates the dedupe buffers.
+ **repeat**
+ Generate dedupe buffers by repeating previous writes
+ **working_set**
+ Generate dedupe buffers from working set
+ ``repeat`` is the default option for fio. Dedupe buffers are generated
+ by repeating previous unique write.
+ ``working_set`` is a more realistic workload.
+ With ``working_set``, ``dedupe_working_set_percentage=<int>`` should be provided.
+ Given that, fio will use the initial unique write buffers as its working set.
+ Upon deciding to dedupe, fio will randomly choose a buffer from the working set.
+ Note that by using ``working_set`` the dedupe percentage will converge
+ to the desired over time while ``repeat`` maintains the desired percentage
+ throughout the job.
+.. option:: dedupe_working_set_percentage=int
+ If ``dedupe_mode=<str>`` is set to ``working_set``, then this controls
+ the percentage of size of the file or device used as the buffers
+ fio will choose to generate the dedupe buffers from
+ Note that size needs to be explicitly provided and only 1 file per
+ job is supported
.. option:: invalidate=bool
Invalidate the buffer/page cache parts of the files to be used prior to