path: root/lib
AgeCommit message (Collapse)Author
2023-01-31lib/pattern: fix formattingVincent Fu
The fix I committed was not formatted nicely. Make the code look nicer. Signed-off-by: Vincent Fu <>
2023-01-30lib/pattern: Fix seg fault when calculating pattern lengthVincent Fu
When --buffer_pattern or --verify_pattern has multiple elements (0x110x22 or 0xdeadface"abcd"-12'filename') calculating the length produces a segmentation fault in parse_and_fill_pattern() because it increments out when out is passed to the parse_* routines it calls. This patch uses the fix provided in the GitHub issue. Fixes: Fixes: 6c9397396eb83a6ce64a998795e7a50552e4337e "lib/pattern: Support NULL output buffer in parse_and_fill_pattern()" Signed-off-by: Vincent Fu <>
2022-11-18lib/pattern: Support binary pattern buffers on windowsLogan Gunthorpe
On windows, binary files used as pattern buffers may be mangled or truncated seeing the files are openned in text mode. Fix this by passing O_BINARY on windows when openning the file. Suggested-by: Vincent Fu <> Signed-off-by: Logan Gunthorpe <> Signed-off-by: Vincent Fu <>
2022-11-18options: Support arbitrarily long pattern buffersLogan Gunthorpe
Dynamically allocate the pattern buffer to remove the 512B length restriction. To accomplish this, store a pointer instead of a fixed block of memory for the buffers in the thread_options structure. Then introduce and use the function parse_and_fill_pattern_alloc() which will calculate the approprite size of the buffer and allocate it before filling it. The buffers will be freed, along with a number of string buffers in free_thread_options_to_cpu(). They will also be reallocated (if necessary) when receiving them over the wire with convert_thread_options_to_cpu(). This allows for specifying real world compressible data (eg. The Calgary Corpus) for the buffer_pattern option. Signed-off-by: Logan Gunthorpe <> Signed-off-by: Vincent Fu <>
2022-11-18lib/pattern: Support short repeated read calls when loading from fileLogan Gunthorpe
Once a pattern file can be much larger, it will be possible that kernel will return a short read while loading the file and thus may randomly only load part of the file. Fix this by putting the read in a loop so the entire file will be read. Signed-off-by: Logan Gunthorpe <> Signed-off-by: Vincent Fu <>
2022-11-18lib/pattern: Support NULL output buffer in parse_and_fill_pattern()Logan Gunthorpe
Support passing a NULL output buffer to parse_and_fill_pattern(). Each formatting function simply needs to avoid accessing the buffer when it is NULL. This allows calculating the required size of the buffer before one might be allocated. This will be useful in a subsequent patch for dynamically allocating the pattern buffers. Signed-off-by: Logan Gunthorpe <> Signed-off-by: Vincent Fu <>
2022-08-10lib/rand: get rid of unused MAX_SEED_BUCKETSJens Axboe
It's only used to size the array, we don't need it. Signed-off-by: Jens Axboe <>
2022-08-10lib/rand: Enhance __fill_random_buf using the multi random seedSungup Moon
The __fill_random_buf fills a buffer using the random 8byte integer to write. But, this mechanism is depend on the CPU performance and could not reach the max performance on the PCIe Gen5 devices. I have tested 128KB single worker sequential write on PCIe Gen5 NVMe, but it cannot reach write throughput 6.0GB/s. So, I have reviewed the __fill_random_buf and focused the multiplier dependency to generate the random number. So, I have changed __fill_random_buf using the multiple-random-seed to reduce the dependencies in the small data filling loop. I'll attach detail analysis result in the PR of this branch. Signed-off-by: Sungup Moon <>
2022-07-01hash: cleanupsJens Axboe
- Use __hash_u64() for __fill_random_buffer() - Convert rdma to use GOLDEN_RATIO_64 That's the last user of GOLDEN_RATIO_PRIME, which due to bit sparseness isn't really useful for our purposes. Signed-off-by: Jens Axboe <>
2022-07-01lib/rand: improve __fill_random_buf()Jens Axboe
This won't be equivalent to what we have, but I _think_ the randomness is good enough for this purpose. This improves performance by about 30% for me, tested on both aarch64 and x86-64. Signed-off-by: Jens Axboe <>
2022-07-01Simplify and optimize __fill_random_bufGeorg Sauthoff
This reduces the number of source lines and the code size. For example, when compiling with GCC 12.1 (-O3 -march=skylake), the resulting assembly shrinks from 33 to 27 instructions and the number of jump instructions is reduced from 4 to 3. NB: GCC is able to eliminate the memcpy() call. NB: Even if a compiler doesn't eliminate the memcpy() call, it's very unlikely to ever get called since the buffer sizes are expected to be powers of two (>= 8), usually. Signed-off-by: Georg Sauthoff <>
2022-03-30Rename 'fallthrough' attribute to 'fio_fallthrough'Jens Axboe
fallthrough is reserved in C++, so this causes issues with C++ programs pulling in the fio.h -> compiler.h header. Rename it to something fio specific instead. Signed-off-by: Jens Axboe <>
2022-02-20Spelling and grammar fixesVille Skyttä
Signed-off-by: Ville Skyttä <>
2021-11-19t/io_uring: use internal random generatorJens Axboe
Instead of using lrand48_r, use the internal fio random number generator. Signed-off-by: Jens Axboe <>
2021-09-08lib/fls.h: Remove unused variable assignmentErwan Velu
x is modified just before the last set of r but x is never used again. Let's remove this useless assignment. Signed-off-by: Erwan Velu <>
2021-08-26arch,lib/seqlock: implement seqlock with C++ atomic if compiled with C++Kefu Chai
because some functions declared by <stdatomic.h> share the same names with those declared by <atomic>, for instance `kill_dependency()` is defined as a macro by <stdatomic.h>, while it is defined as a template function in <atomic>. this renders it impossible to compile an ioengine written in C++ if its source file includes both <atomic> and <fio.h>. the latter includes <stdatomic.h> via arch/arch.h. the compile error would look like: In file included from ../src/test/fio/ In file included from src/fio/fio.h:18: In file included from src/fio/thread_options.h:8: In file included from src/fio/gettime.h:7: src/fio/lib/seqlock.h:21:9: error: expected ')' seq = atomic_load_acquire(&s->sequence); ^ src/fio/arch/../lib/../arch/arch.h:47:32: note: expanded from macro 'atomic_load_acquire' atomic_load_explicit((_Atomic typeof(*(p)) *)(p), \ ^ src/fio/lib/seqlock.h:21:9: note: to match this '(' to address this issue, instead of using the functions in <stdatomic.h> to implement seqlock, use the primitives offered by C++ standard library if the source code is compiled using a C++ compiler. Signed-off-by: Kefu Chai <>
2021-07-15dedupe: allow to generate dedupe buffers from working setBar David
This commit introduced new dedupe generation mode "working_set". Working set mode simulates a more realistic approach to deduped data, in which deduped buffers are generated from pre-existing working set - % size of the device or file. In other words, dedupe is not usually expected to be close in time with the source buffer, as well as source buffers are usually composed of small subset of the entire file or device. Signed-off-by: Bar David <>
2021-01-11distibutions: Extend flexibility of non-uniform random distributionsAdam Kupczyk
This change affects options random_distribution and file_service_type. For pareto, zipf and gauss distribution a contept of `center` is implemented. It allows to fix in place a value that is most probable to access. Example: fio --randseed=1 --ioengine=libaio --rw=randwrite --nrfiles=16 --bs=4k \ --size=256m --allow_file_create=1 --write_iolog=log.txt \ --file_service_type=gauss:10:0.1 --filename_format=object.\$filenum --name=x cat log.txt |grep write |cut -f 1 -d " " |sort |uniq -c | sort -n | \ sed "s/[.]/ /" | while read a b c; do echo $c $b $a; done |sort -n 0 object 13429 1 object 17928 2 object 14724 3 object 7845 4 object 2476 5 object 468 6 object 44 7 object 3 12 object 24 13 object 318 14 object 1795 15 object 6482 Signed-off-by: Adam Kupczyk <>
2021-01-09Merge branch 'num2str-patch' of Axboe
* 'num2str-patch' of num2str: fix precision loss bug when the fractional part is close to 1
2021-01-06Change ARRAY_SIZE to FIO_ARRAY_SIZEJens Axboe
Ensures that we don't clash with external symbols/macros. Signed-off-by: Jens Axboe <>
2020-12-21num2str: fix precision loss bug when the fractional part is close to 1gloit042
example: The result of num2str(11999999999999, 4, 1, 0, N2S_NONE) should be "12.0G", but current result is "11.0G". Signed-off-by: Jiahao Li <>
2020-08-28Use fallthrough attributeJens Axboe
Fio currently triggers a bunch of fall through errors on clang 10, since it doesn't work with the /* fall through */ method of indicating fallthrough. Normalize this a bit and use the correct compiler attribute for this. Signed-off-by: Jens Axboe <>
2020-07-27Add roundup_pow2() as a generic helperJens Axboe
Signed-off-by: Jens Axboe <>
2020-07-02num2str(): Add the E (exa) prefixBart Van Assche
This change guarantees that all 64-bit integers that should be assigned a multiplier are assigned a multiplier. Signed-off-by: Bart Van Assche <>
2020-07-02num2str(): Fix overflow handlingBart Van Assche
This patch changes the output for the arguments (UINT64_MAX, 4, 1, 0, N2S_NONE, "18.4") from "18.4(null)" into "18.4". This patch fixes the following Coverity complaint: CID 169307 (#1 of 1): Out-of-bounds read (OVERRUN) 27. overrun-local: Overrunning array of 6 8-byte elements at element index 6 (byte offset 55) by dereferencing pointer unitprefix + post_index. Signed-off-by: Bart Van Assche <>
2020-07-02num2str(): Remove the fmt[] arrayBart Van Assche
This patch does not change any functionality. Signed-off-by: Bart Van Assche <>
2020-07-02num2str(): Use asprintf() instead of malloc()Bart Van Assche
This patch reduces the size of the buffer allocated by num2str(). Signed-off-by: Bart Van Assche <>
2020-06-23Changed signedness of seqlock.sequence fixing comparison-warningSimon A. F. Lund
Signed-off-by: Simon A. F. Lund <>
2020-06-21Optimize the seqlock implementationBart Van Assche
Use atomic_load_acquire() and atomic_store_release() instead of barriers where appropriate. Signed-off-by: Bart Van Assche <>
2020-06-21Make __rand_0_1() compatible with clangBart Van Assche
This patch fixes the following clang compiler error: crc/../arch/../lib/rand.h:109:25: error: implicit conversion from 'unsigned long long' to 'double' changes value from 18446744073709551615 to 18446744073709551616 [-Werror,-Wimplicit-int-float-conversion] return (val + 1.0) / (FRAND64_MAX + 1.0); ^~~~~~~~~~~ ~ crc/../arch/../lib/rand.h:9:22: note: expanded from macro 'FRAND64_MAX' #define FRAND64_MAX (-1ULL) ^~~~~ Fixes: e7b240474543 ("Fixups for poisson rate") Signed-off-by: Bart Van Assche <>
2020-05-23Do not read past the end of fmt_desc[]Bart Van Assche
Callers of parse_format() pass a size in bytes while the parse_format() function itself expects a number of elements. Fix this by making the fmt_desc[] array NULL-terminated. This patch fixes the following Coverity complaint: CID 300986 (#1 of 1): Out-of-bounds access (OVERRUN) overrun-buffer-arg: Overrunning array fmt_desc of 1 24-byte elements by passing it to a function which accesses it at element index 23 (byte offset 575) using argument 24U. Cc: Roman Pen <> Fixes: 634bd210c17a ("lib/pattern: add set of functions to parse combined pattern input") Signed-off-by: Bart Van Assche <>
2020-05-23Fix spelling in a source code commentBart Van Assche
Change two occurrences of 'descritor' into 'descriptor' Cc: Roman Pen <> Fixes: 634bd210c17a ("lib/pattern: add set of functions to parse combined pattern input") Signed-off-by: Bart Van Assche <>
2020-05-19Fix 32-bit/LLP64 platform truncation issuesSitsofe Wheeler
- After 140a6888 ("rate: Convert the rate and rate_min options to FIO_OPTS_ULL") landed 32-bit/LLP64 platforms need additional changes to cope with 64 bit I/O rate values - The seed is 64 bit but was being being truncated to 32 bits in td_fill_rand_seeds_internal() on bit/LLP64 platforms Prior to this commit when running an fio compiled with CC=clang-9 ./configure \ --extra-cflags="-fsanitize=undefined,implicit-integer-truncation " "-fno-builtin" using this job ./fio --ioengine=null --bs=1M --rate=6G --rate_min=5G --name=test --size=100G warnings like the following were produced init.c:996:27: runtime error: implicit conversion from type 'uint64_t' (aka 'unsigned long long') of value 5942511153023025289 (64-bit, unsigned) to type 'unsigned int' changed the value to 2914779273 (32-bit, unsigned) [..] backend.c:212:25: runtime error: implicit conversion from type 'unsigned long long' of value 12886999040 (64-bit, unsigned) to type 'unsigned long' changed the value to 2097152 (32-bit, unsigned) inside a 32-bit Ubuntu 18.04 docker container. Fixes:
2020-03-17gauss.c: correct the stddev initializtionFeng Tang
Signed-off-by: Feng Tang <> Signed-off-by: Jens Axboe <>
2020-01-17lib/memcpy: Suppress a Coverity leak report for setup_tests()Bart Van Assche
Suppress the following Coverity memory leak complaint: CID 169324 (#1 of 1): Resource leak (RESOURCE_LEAK) 7. leaked_storage: Variable src going out of scope leaks the storage it points to. Signed-off-by: Bart Van Assche <>
2020-01-06lib/memalign: remove smalloc()/sfree() dependencyTomohiro Kusumi
fio_memalign()/fio_memfree() implementation shouldn't depend on smalloc()/sfree() which has dependency on fio code itself. e.g. This forces unittest code to prepare stab functions for smalloc()/sfree(). This smalloc()/sfree() dependency was added by 3114b675fd ("fio: enable cross-thread overlap checking with processes"). Rename fio_memalign()/fio_memfree() to __fio_memalign()/__fio_memfree() and take a function pointer instead of a boolean flag. Add fio_memalign()/fio_memfree() as an inlined wrapper for __fio_memalign()/__fio_memfree() without API change. The only real change here is lib/memalign functions got renamed. Signed-off-by: Tomohiro Kusumi <>
2019-04-17rand: fix truncated rand_seed on WindowsMing-Hung Tsai
The long data type is 32-bit on LLP64 platforms
2019-02-11Document switch fall-through casesJens Axboe
Signed-off-by: Jens Axboe <>
2019-01-08configure: add __kernel_rwf_t checkJens Axboe
Signed-off-by: Jens Axboe <>
2018-11-29rand: fix compressible data ratio per segmentBari Antebi
I've noticed a bug in fio while testing it. I expected to receive output similar to the expected output below for: "hexdump -n 4096 /dev/nvme0n1" (considering the configuration file I've used that may be found below). Expected output: 0000000 fdc6 a8a8 7190 0219 1fb8 9632 d9dd 1e64 /* random data */ 00004c0 d8a3 13fe aeec 0fb6 5b14 162e 0000 0000 00004d0 0000 0000 0000 0000 0000 0000 0000 0000 * 0001000 However, the output I've received contained data after address 4cc (which should have been filled with zeroes until 1000 as far as I understand, but as you can see 99a contains data. 0000000 fdc6 a8a8 7190 0219 1fb8 9632 d9dd 1e64 /* random data */ 00004c0 d8a3 13fe aeec 0fb6 5b14 162e 0000 0000 00004d0 0000 0000 0000 0000 0000 0000 0000 0000 * 0000990 0000 0000 0000 0000 fdc6 a8a8 7190 0219 Config file: [global] group_reporting=1 filename=/dev/nvme0n1 name=task_nvme0n1 rw=write bs=4096 numjobs=1 iodepth=32 buffer_compress_chunk=4096 buffer_compress_percentage=70 cpus_allowed=0-8 cpus_allowed_policy=split direct=1 ioengine=libaio loops=1 refill_buffers=0 [job empty] size=4096 Fio should write (100 - compress_percentage) * segemnt of random data followed by compress_percentage * segemnt of compressible data. Currently, at each itereation fio fills (100 - compress_percentage) * segemnt data, followed by (100 - compress_percentage) * segemnt of compressible data until a the segment is filled. Fixes: 811ac503a619 ("Compression fixes") Signed-off-by: Bari Antebi <> Signed-off-by: Jens Axboe <>
2018-10-26lib: fix strntol's end pointer when str has leading spacesTomohiro Kusumi
Fix unittests/lib/strntol.c test case failure. Given behavior of strtol(3), "end" pointer should be calculated based on "beg" pointer instead of "str" on success. Otherwise if "str" has leading spaces (which are explicitly ignored by strntol()), "end" won't point to "the first invalid character" for not counting those spaces. Glibc's strtol(3) says as follows. -- If endptr is not NULL, strtol() stores the address of the first invalid character in *endptr. If there were no digits at all, strtol() stores the original value of nptr in *endptr (and returns 0). In particular, if *nptr is not '\0' but **endptr is '\0' on return, the entire string is valid. Signed-off-by: Tomohiro Kusumi <> Signed-off-by: Jens Axboe <>
2018-10-19fio: enable cross-thread overlap checking with processesVincent Fu
Overlap checking with io_submit_mode=offload requires relevant jobs to access each other's io_u's and io_u_all members. This patch modifies the fio_memalign and io_u_queue helpers to include an indicator signifying whether operations should use the shared memory pool. When fio is carrying out cross-job overlap checking in offload submission mode, these variables will be allocated from shared memory so that processes can be used and threads will no longer be required. Signed-off-by: Jens Axboe <>
2018-09-30fix hung fio process with large I/O sizes and verify= optionJeff Moyer
Hi, If you use a very large I/O size (1G in this instance), and also specify the verify option, fio will hang, using up 100% cpu, for ever and ever. The problem is that __fill_random_buf_percentage does math that overflows the size of an unsigned int. Fixing that gets fio to terminate as expected. Here is the job file which showed the problem: [global] ioengine=dev-dax direct=0 filename=/dev/dax0.0 verify=crc32c bs=1G [write] rw=write runtime=5 [read] stonewall rw=read runtime=5 Reported-by: Signed-off-by: Jeff Moyer <> Signed-off-by: Jens Axboe <>
2018-09-17Random distribution 32-bit fixesJens Axboe
We calculate and use nranges as an unsigned long, but that can be 32-bit on some platforms and hence overflow. Signed-off-by: Jens Axboe <>
2018-09-16axmap: isset_fn() should use 1ULL, not 1ULJens Axboe
Signed-off-by: Jens Axboe <>
2018-09-16lfsr: use unsigned long long for 64-bit valuesJens Axboe
1UL is bit 64-bit on Windows builds. Signed-off-by: Jens Axboe <>
2018-09-16axmap: use 64-bit index for the handlersJens Axboe
Most important one here is axmap_handler_topdown(), in which the first iteration could end up truncating the index calculation. Signed-off-by: Jens Axboe <>
2018-09-16axmap: use 64-bit type for number of bitsJens Axboe
For huge devices, we can overflow on a 32-bit build of fio. Signed-off-by: Jens Axboe <>
2018-09-07num2str(): Avoid an out-of-bounds array accessBart Van Assche
This patch fixes Coverity ID 169307. Fixes: 1ec3d69b0ed8 ("Implement a better num2str()") Signed-off-by: Bart Van Assche <>
2018-09-07Micro-optimize num2str()Bart Van Assche
Instead of selecting an element from the thousand[] array several times, perform the choice between 1000 and 1024 once. This patch does not change any functionality. Signed-off-by: Bart Van Assche <>