trim: add support for multiple ranges NVMe specification allow multiple ranges for the dataset management commands. Currently the block ioctl only allows a single range for trim, however multiple ranges can be specified using nvme character device. Add an option num_range to send multiple range per trim request, which only works if the data direction is solely trim i.e. trim or randtrim. Add FIO_MULTI_RANGE_TRIM as the ioengine flag, to restrict the usage of this new option. For multi range trim request this modifies the way IO buffers are used. The buffer length will depend on number of trim ranges and the actual buffer will contains start and length of each range entry. This increases fio server version (FIO_SERVER_VER) to 103. Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com> Link: https://lore.kernel.org/r/20240215151812.138370-2-ankit.kumar@samsung.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
init: don't adjust time units again for subjobs We adjust max_latency and latency_target values to be nsec internally. Make sure we do this only once for the parent job and don't do it a second time for a subjob. Fixes: https://github.com/axboe/fio/issues/1582 Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
fio: replace malloc+memset with calloc Clean up the code base by replacing malloc+memset with calloc. This patch was generated from the Coccinelle script below. The script below is inspired by similar scripts used elsewhere: https://lore.kernel.org/linux-btrfs/cover.1443546000.git.silvio.fricke@gmail.com/ https://github.com/coccinelle/coccinellery/blob/master/simple_kzalloc/simple_kzalloc1.cocci @@ expression x,y; statement s; type T; @@ -x = malloc(y * sizeof(T)); +x = calloc(y, sizeof(T)); ( if (!x) s | if (x == NULL) s | ) -memset(x, 0, y * sizeof(T)); @@ expression x,y,z; statement s; @@ -x = malloc(y * sizeof(z)); +x = calloc(y, sizeof(z)); ( if (!x) s | if (x == NULL) s | ) -memset(x, 0, y * sizeof(z)); @@ expression e,x; statement s; @@ -x = malloc(e); +x = calloc(1, e); ( if (!x) s | if (x == NULL) s | ) -memset(x, 0, e); Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
init: clean up random seed options - make allrandrepeat a synonym of randrepeat. allrandrepeat is superfluous because the seeds set by randrepeat already encompass random number generators beyond the one used for random offsets. - allow randseed to override [all]randrepeat: this is what the documentation implies but was not previously the case This is a breaking change for users relying on the values of fio's default random seeds. Link: https://github.com/axboe/fio/pull/1546 Fixes: https://github.com/axboe/fio/issues/1502 Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
init: refactor random seed setting td->rand_seed was modified in three different places. Put all this code in setup_random_seeds() to make it easier to understand and more maintanable. Also put setup_random_seeds() next to the other random-seed-related functions in init.c. init_rand_seed() was called in three different places for fio's main random number generators. Also put these three sets of invocations in the same place. Always initialize all of fio's main set of random states instead of skipping some for sequential workloads. This makes debugging easier. No functional change. Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
fio: steadystate: allow for custom check interval Allow for a different steady state check interval than 1s with a new --ss_interval parameter. Steady state is reached when the steady state condition (like slope) is true when comparing the last windows (set with --ss_dur). The actual values for this comparison is currently calculated for a 1s interval during the window. This is especially problematic for slow random devices, where the values do not converge for such a fine granularity. Letting the user set this solves this problem, although requires them figuring out an appropriate value themselves. --ss=iops:5% --ss_dur=120s should reproduce this for many (slower) devices. Then adding like --ss_interval=20s may let it converge. Signed-off-by: Christian Loehle <cloehle@posteo.de>
Refactor for_each_td() to catch inappropriate td ptr reuse I recently introduced a bug caused by reusing a struct thread_data *td after the end of a for_each_td() loop construct. Link: https://github.com/axboe/fio/pull/1521#issuecomment-1448591102 To prevent others from making this same mistake, this commit refactors for_each_td() so that both the struct thread_data * and the loop index variable are placed inside their own scope for the loop. This will cause any reference to those variables outside the for_each_td() to produce an undeclared identifier error, provided the outer scope doesn't already reuse those same variable names for other code within the routine (which is fine because the scopes are separate). Because C/C++ doesn't let you declare two different variable types within the scope of a for() loop initializer, creating a scope for both struct thread_data * and the loop index required explicitly declaring a scope with a curly brace. This means for_each_td() includes an opening curly brace to create the scope, which means all uses of for_each_td() must now end with an invocation of a new macro named end_for_each() to emit an ending curly brace to match the scope brace created by for_each_td(): for_each_td(td) { while (td->runstate < TD_EXITED) sleep(1); } end_for_each(); The alternative is to end every for_each_td() construct with an inline curly brace, which is off-putting since the implementation of an extra opening curly brace is abstracted in for_each_td(): for_each_td(td) { while (td->runstate < TD_EXITED) sleep(1); }} Most fio logic only declares "struct thread_data *td" and "int i" for use in for_each_td(), which means those declarations will now cause -Wunused-variable warnings since they're not used outside the scope of the refactored for_each_td(). Those declarations have been removed. Implementing this change caught a latent bug in eta.c::calc_thread_status() that accesses the ending value of struct thread_data *td after the end of for_each_td(), now manifesting as a compile error, so working as designed :) Signed-off-by: Adam Horshack (horshack@live.com)
init: return error incase an invalid value is passed as option Currently, fio exits incase an invalid value is passed as fio option, but continues even if an invalid value is passed for I/O engine specific options. Exit in that scenario. Signed-off-by: Anuj Gupta <anuj20.g@samsung.com> Link: https://lore.kernel.org/r/20220531133155.17493-4-ankit.kumar@samsung.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
Introducing support for generation of dedup buffers across jobs. The dedup buffers are spread evenly between the jobs that enabled the dedupe_global option Note only dedupe_mode=working_set is supported. Note compression is supported with the global dedup enabled Signed-off-by: Bar David <bardavvid@gmail.com>
Fix :<nr> suffix with random read/write causing 0 initial offset When using the :<nr> suffix with random reads or writes, the initial offset would be set to 0 for the first nr-1 operations. This happened because td->ddir_seq_nr was initialized to the specified option value, when it needs to always be initialized to 1, so that the first call to get_next_offset leads to choosing a new random offset for the first nr operations. Signed-off-by: Nick Neumann nick@pcpartpicker.com
init: verify option lat_percentiles consistency for all jobs in group lat_percentiles is used to control if the high/low latency statistics (which are saved in ts->io_u_plat_high_prio/ts->io_u_plat_low_prio) should collect and display total latencies instead of completion latencies. When doing group reporting, stat.c:__show_run_stats() happily overwrites the dst ts with the setting of each job, which means that the summing can take total lat samples for some jobs, and clat samples for some jobs, while adding samples into the same group result. The output summary will claim that the results are of whatever type the final job in the group is set to. To make sure that this cannot happen, verify that the option lat_percentiles is consistent for all jobs in group. Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com> Reviewed-by: Damien Le Moal <damien.lemoal@opensource.wdc.com> Link: https://lore.kernel.org/r/20220203192814.18552-2-Niklas.Cassel@wdc.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
init: do not create lat logs when not needed When any of the options disable_lat, disable_slat and disable_clat are used, there is no need to create the lat log associated with the disabled latency. In addition, when write_lat_log is also specified, this change avoids the creation of empty latency log files. Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com> Reviewed-by: Niklas Cassel <niklas.cassel@wdc.com> Link: https://lore.kernel.org/r/20220117021127.9259-1-damien.lemoal@wdc.com Signed-off-by: Jens Axboe <axboe@kernel.dk>