AgeCommit message (Collapse)Author
19 hoursMerge branch 'fix_devdax' of Axboe
* 'fix_devdax' of Fix: dev-dax engine building with make
19 hoursFix: dev-dax engine building with makeHarish
With recent changes to Makefile dev-dax skips building. This patch fixes that. Signed-off-by: Harish <>
45 hoursMerge branch 'io_uring-opt' of Axboe
* 'io_uring-opt' of io_uring: Avoid needless update of completion queue head pointer
2 daysio_uring: Avoid needless update of completion queue head pointerAnton Blanchard
I'm seeing a slowdown in io_uring performance on a POWER9 box when the userspace and kernel polling threads are on two cores that share an L2 cache. fio_ioring_cqring_reap() always stores to the completion queue head pointer, even if nothing was reaped and the value hasn't changed. Changing this to only update the head pointer when it changes results in a 95% improvement in performance on this particular test. Signed-off-by: Anton Blanchard <>
7 dayst/io_uring: make bs and polled IO configurable at runtimeJens Axboe
Signed-off-by: Jens Axboe <>
10 daysMerge branch 'windows' of Axboe
* 'windows' of os/windows/posix.c: Use INVALID_SOCKET instead of < 0 os/windows/posix.c: Strip trailing whitespace
11 daysos/windows/posix.c: Use INVALID_SOCKET instead of < 0Bart Van Assche
From "Because the SOCKET type is unsigned, compiling existing source code from, for example, a UNIX environment may lead to compiler warnings about signed/unsigned data type mismatches. This means, for example, that checking for errors when the socket and accept functions return should not be done by comparing the return value with -1, or seeing if the value is negative (both common and legal approaches in UNIX). Instead, an application should use the manifest constant INVALID_SOCKET as defined in the Winsock2.h header file." This patch fixes the following build error for the latest version of the MinGW64 cross-compiler on Cygwin: os/windows/posix.c:886:17: error: comparison of unsigned expression < 0 is always false Signed-off-by: Bart Van Assche <>
11 daysos/windows/posix.c: Strip trailing whitespaceBart Van Assche
Signed-off-by: Bart Van Assche <>
11 daysMerge branch 'workqueue' of Axboe
* 'workqueue' of workqueue: Fix race conditions in the workqueue mechanism workqueue: Rework while loop locking
11 daysworkqueue: Fix race conditions in the workqueue mechanismBart Van Assche
From "The application shall ensure that these functions are called with mutex locked by the calling thread; otherwise, an error (for PTHREAD_MUTEX_ERRORCHECK and robust mutexes) or undefined behavior (for other mutexes) results. From "however, if predictable scheduling behavior is required, then that mutex shall be locked by the thread calling pthread_cond_broadcast() or pthread_cond_signal()." Hence always hold the associated mutex around pthread_cond_wait() and pthread_signal() calls. This patch fixes the hangs reported by Travis and Appveyor for test case t0013. Signed-off-by: Bart Van Assche <>
11 daysworkqueue: Rework while loop lockingBart Van Assche
Reduce the number of pthread_mutex_lock() and unlock calls by moving these outside while loops. Other than protecting the all_sw_idle() call with the wq->flush_lock, this patch does not change any functionality. Signed-off-by: Bart Van Assche <>
11 daysMerge branch 'overlap' of Axboe
* 'overlap' of Add a test for serialize_overlap=1 Enable error checking for the mutex that serializes overlapping I/O Fix a potentially infinite loop in check_overlap() t/ Increase IOPS tolerance further .appveyor.yml: Select the latest software image
11 daysAdd a test for serialize_overlap=1Bart Van Assche
This test causes fio to call check_overlap(). Signed-off-by: Bart Van Assche <>
11 daysEnable error checking for the mutex that serializes overlapping I/OBart Van Assche
Since it is nontrivial to verify that pthread_mutex_lock() and unlock calls are paired for the overlap_check mutex, enable runtime verification. The PTHREAD_ERRORCHECK_MUTEX_INITIALIZER_NP initializer causes pthread_mutex_lock() to fail with EDEADLK if the mutex was already locked and causes pthread_mutex_unlock() to fail with EPERM if the mutex is owned by another thread. Signed-off-by: Bart Van Assche <>
11 daysFix a potentially infinite loop in check_overlap()Bart Van Assche
If the following happens: * check_overlap() finds an overlap. * All other threads finish after the overlap has been found and before the next iteration of the do/while loop starts. Then the do/while loop in check_overlap() will iterate forever. Fix this by rewriting check_overlap() such that this cannot happen. This patch fixes the following Coverity complaint: CID 184174 (#2 of 2): Double lock (LOCK) Fixes: c06379a65d46 ("fio: enable overlap checking with offload submission") Signed-off-by: Bart Van Assche <>
11 dayst/ Increase IOPS tolerance furtherBart Van Assche
From an Appveyor build: DEBUG:root:Test 12: iops1: 984.676882 Signed-off-by: Bart Van Assche <>
11 days.appveyor.yml: Select the latest software imageBart Van Assche
This is necessary to make the overlap test (t0013) pass. Signed-off-by: Bart Van Assche <>
12 daysconfigure: fix bad indentationJens Axboe
Signed-off-by: Jens Axboe <>
12 daysfio: better info when engine not foundYigal Korman
Jens suggested we provide the user with the likely reasons why we failed to load the given engine. This should help better resolve these issues. Signed-off-by: Yigal Korman <> Signed-off-by: Jens Axboe <>
12 daysconfigure: new --dynamic-libengines build optionYigal Korman
When enabled, some of the more dependency-heavy internal engines are converted to "plugin" engines, i.e. they are built into separate object files and are loaded by fio on demand. This helps downstream distros package these engines separately and not force a long list of package dependencies from the base fio package. Signed-off-by: Yigal Korman <> Signed-off-by: Jens Axboe <>
12 daysconfigure/Makefile: engine LIBS consistencyYigal Korman
Some engines were adding their LIBS in the Makefile and some were declaring them in the configure script. This is inconsistent and confusing and prevents easy bulk changes to the engines. Signed-off-by: Yigal Korman <> Signed-off-by: Jens Axboe <>
12 daysfio: don't retry engine search on failureYigal Korman
ioengine_load will try to load a given engine and if it doesn't match the current one (default is always psync), it will remove the current one and try again. This makes sense when the engine is loaded successfully, but if it's not, we try to load it twice for no reason. Fix by failing if we were unable to load requested engine and not retrying. Signed-off-by: Yigal Korman <> Signed-off-by: Jens Axboe <>
12 daysMerge branch 'num2str' of Axboe
* 'num2str' of num2str(): Add the E (exa) prefix Add a num2str() unit test num2str(): Fix overflow handling num2str(): Remove the fmt[] array num2str(): Use asprintf() instead of malloc()
12 daysnum2str(): Add the E (exa) prefixBart Van Assche
This change guarantees that all 64-bit integers that should be assigned a multiplier are assigned a multiplier. Signed-off-by: Bart Van Assche <>
13 daysAdd a num2str() unit testBart Van Assche
Signed-off-by: Bart Van Assche <>
13 daysnum2str(): Fix overflow handlingBart Van Assche
This patch changes the output for the arguments (UINT64_MAX, 4, 1, 0, N2S_NONE, "18.4") from "18.4(null)" into "18.4". This patch fixes the following Coverity complaint: CID 169307 (#1 of 1): Out-of-bounds read (OVERRUN) 27. overrun-local: Overrunning array of 6 8-byte elements at element index 6 (byte offset 55) by dereferencing pointer unitprefix + post_index. Signed-off-by: Bart Van Assche <>
13 daysnum2str(): Remove the fmt[] arrayBart Van Assche
This patch does not change any functionality. Signed-off-by: Bart Van Assche <>
13 daysnum2str(): Use asprintf() instead of malloc()Bart Van Assche
This patch reduces the size of the buffer allocated by num2str(). Signed-off-by: Bart Van Assche <>
2020-06-28Merge branch 'pmemblk' of Axboe
* 'pmemblk' of Unbreak the pmemblk engine
2020-06-27Unbreak the pmemblk engineBart Van Assche
Reported-by: Yi Zhang <> Tested-by: Yi Zhang <> Fixes: e9c7be0e32e6 ("pmemblk: Fix a memory leak") Signed-off-by: Bart Van Assche <>
2020-06-24Merge branch 'compiler' of Axboe
* 'compiler' of Merge compiler/gcc4.h into compiler/compiler.h
2020-06-23Merge compiler/gcc4.h into compiler/compiler.hBart Van Assche
Follow the example of the Linux kernel and merge the file compiler/gcc4.h into compiler/gcc.h. Additionally, enable __must_check, __compiletime_warning() and __compiletime_error() for clang. Signed-off-by: Bart Van Assche <>
2020-06-23Merge branch 'master' of Axboe
* 'master' of Changed signedness of seqlock.sequence fixing comparison-warning
2020-06-23Changed signedness of seqlock.sequence fixing comparison-warningSimon A. F. Lund
Signed-off-by: Simon A. F. Lund <>
2020-06-23compiler/compiler.h: include clang 6.0 and aboveJens Axboe
Fixes: baa16486051f ("compiler/compiler.h: minimum GCC version is 4.9") Signed-off-by: Jens Axboe <>
2020-06-23oslib/linux-blkzoned: fix bogus poiter alignment warningJens Axboe
GCC 10.x reports: oslib/linux-blkzoned.c: In function ‘blkzoned_report_zones’: oslib/linux-blkzoned.c:146:18: warning: taking address of packed member of ‘struct blk_zone_report’ may result in an unaligned pointer value [-Waddress-of-packed-member] 146 | blkz = (void *) &hdr->zones[0]; which is totally fine, but we can easily avoid this warning by just setting the 'blkz' pointer to the end of 'hdr'. Signed-off-by: Jens Axboe <>
2020-06-23compiler/compiler.h: minimum GCC version is 4.9Jens Axboe
Signed-off-by: Jens Axboe <>
2020-06-21Merge branch 'atomics' of Axboe
* 'atomics' of Optimize fio_gettime_offload() Optimize the seqlock implementation Add a test that sets gtod_cpu=1 t/ Increase IOPS tolerance further fio: Use atomic_load_acquire() and atomic_store_release() where appropriate engines/io_uring: Use atomic_{load_acquire,store_release}() engines/libaio: Use atomic_store_release() instead of read_barrier() arch/arch.h: Introduce atomic_{load_acquire,store_release}() fio_sem: Remove a read_barrier() call Make __rand_0_1() compatible with clang configure: Use -Wimplicit-fallthrough=2 instead of -Wimplicit-fallthrough=3
2020-06-21Optimize fio_gettime_offload()Bart Van Assche
This patch not only an optimization but also a bug fix because it guarantees that fio_gettime_offload() never returns an inconsistent timestamp, something that could happen before. In this context inconsistent means returning the .tv_sec value from one gettimeofday() call and the .tv_usec value from another gettimeofday() call. Signed-off-by: Bart Van Assche <>
2020-06-21Optimize the seqlock implementationBart Van Assche
Use atomic_load_acquire() and atomic_store_release() instead of barriers where appropriate. Signed-off-by: Bart Van Assche <>
2020-06-21Add a test that sets gtod_cpu=1Bart Van Assche
None of the existing tests sets gtod_cpu=1. Hence add a test that sets gtod_cpu=1. Signed-off-by: Bart Van Assche <>
2020-06-21t/ Increase IOPS tolerance furtherBart Van Assche
Signed-off-by: Bart Van Assche <>
2020-06-21fio: Use atomic_load_acquire() and atomic_store_release() where appropriateBart Van Assche
The write_barrier() in io_completed() and also the read barriers in verify.c are misplaced: the write barrier should occur before the flags update instead of after and the read barriers should occur after the flags read instead of before. Fix this by using atomic_{load_acquire, store_release}() instead of read and write barriers. Signed-off-by: Bart Van Assche <>
2020-06-21engines/io_uring: Use atomic_{load_acquire,store_release}()Bart Van Assche
This patch improves performance by using acquire and release semantics instead of barriers and also brings the io_uring engine code closer to that of liburing. Signed-off-by: Bart Van Assche <>
2020-06-21engines/libaio: Use atomic_store_release() instead of read_barrier()Bart Van Assche
This is a micro-optimization that does not affect the functionality of the code. Signed-off-by: Bart Van Assche <>
2020-06-21arch/arch.h: Introduce atomic_{load_acquire,store_release}()Bart Van Assche
Implement atomic_load_acquire() and atomic_store_release() with C11 atomic operations. These two primitives will be used in later patches. This patch increases the minimum requirement for the compiler fio is built with from C90 to C11. Signed-off-by: Bart Van Assche <>
2020-06-21fio_sem: Remove a read_barrier() callBart Van Assche
Since pthread_mutex_lock() already provides load-acquire semantics, calling read_barrier() just after pthread_mutex_lock() is not necessary. See also commit 4d4e80f2b426 ("Revamp file locking"). Signed-off-by: Bart Van Assche <>
2020-06-21Make __rand_0_1() compatible with clangBart Van Assche
This patch fixes the following clang compiler error: crc/../arch/../lib/rand.h:109:25: error: implicit conversion from 'unsigned long long' to 'double' changes value from 18446744073709551615 to 18446744073709551616 [-Werror,-Wimplicit-int-float-conversion] return (val + 1.0) / (FRAND64_MAX + 1.0); ^~~~~~~~~~~ ~ crc/../arch/../lib/rand.h:9:22: note: expanded from macro 'FRAND64_MAX' #define FRAND64_MAX (-1ULL) ^~~~~ Fixes: e7b240474543 ("Fixups for poisson rate") Signed-off-by: Bart Van Assche <>
2020-06-21configure: Use -Wimplicit-fallthrough=2 instead of -Wimplicit-fallthrough=3Bart Van Assche
This makes clang accept /* fallthrough */ as a fallthrough annotation. Signed-off-by: Bart Van Assche <>
2020-06-12Priority bit log file format documentation updatePhillip Chen
Updates the description of log file formats to include and describe the priority bit entry. Signed-off-by: Phillip Chen <> Signed-off-by: Jens Axboe <>