3 months agooptions: fix buffer overrun
Sitsofe Wheeler [Sat, 16 Jan 2021 10:41:02 +0000 (10:41 +0000)]
options: fix buffer overrun

Google's OSS-fuzz turned up a buffer overrun with value of the filename
option due to an overrun in a MAX_PATH sized buffer. To reproduce
compile fio with address sanitizer options like the following

LDFLAGS="-fsanitize=address" ./configure --disable-optimizations \

The issue is demonstrated by the following job:

% COUNT=$(getconf PATH_MAX /); printf "[t]\nfilename=%${COUNT}s" \
  | sed 's/ /@/g' | fio --parse-only -
==45748==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7ffee8e35780 at pc 0x00010735a343 bp 0x7ffee8e35270 sp 0x7ffee8e34a08
WRITE of size 1025 at 0x7ffee8e35780 thread T0
    #0 0x10735a342 in wrap_vsprintf (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x22342)
    #1 0x10735a9ac in wrap_sprintf (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x229ac)
    #2 0x106e83b01 in add_file filesetup.c:1656
    #3 0x106ee8c87 in str_filename_cb options.c:1320
    #4 0x106ee1b44 in __handle_option parse.c:792
    #5 0x106ed99ad in handle_option parse.c:1014
    #6 0x106eda07d in parse_option parse.c:1184
    #7 0x106ef10ea in fio_options_parse options.c:5199
    #8 0x106e27684 in __parse_jobs_ini init.c:2076
    #9 0x106e25377 in parse_jobs_ini init.c:2127
    #10 0x106e2c971 in parse_options init.c:2989
    #11 0x106ffc884 in main fio.c:42
    #12 0x7fff702f1cc8 in start (libdyld.dylib:x86_64+0x1acc8)

Address 0x7ffee8e35780 is located in stack of thread T0 at offset 1056 in frame
    #0 0x106e836ef in add_file filesetup.c:1644

  This frame has 1 object(s):
    [32, 1056) 'file_name' (line 1646) <== Memory access at offset 1056 overflows this variable

Return an error message to the user by doing the following:

- Allow "regular" string options to have a maxlen parameter
- Set the filename option to have a maxlen of MAX_PATH

Signed-off-by: Sitsofe Wheeler <>
3 months agoMerge branch 'fix_keyword_sub' of
Jens Axboe [Sat, 16 Jan 2021 03:52:57 +0000 (20:52 -0700)]
Merge branch 'fix_keyword_sub' of

* 'fix_keyword_sub' of
  options: fix keyword substitution heap overrun

3 months agooptions: fix keyword substitution heap overrun
Sitsofe Wheeler [Fri, 15 Jan 2021 20:54:02 +0000 (20:54 +0000)]
options: fix keyword substitution heap overrun

Google's OSS-fuzz turned up a heap overrun when substituting keywords in
job files. To reproduce compile fio with address sanitizer options like
the following

LDFLAGS="-fsanitize=address" ./configure --disable-optimizations \

The issue is demonstrated by the following job:

% printf '[t]\ndescription=$ncpus_' | fio --parse-only -
opt = 'description=$ncpus'
==22547==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x603000001863 at pc 0x000107a833c9 bp 0x7ffee82ac260 sp 0x7ffee82ac258
READ of size 1 at 0x603000001863 thread T0
    #0 0x107a833c8 in fio_keyword_replace options.c:5124
    #1 0x107a7c6ab in dup_and_sub_options options.c:5158
    #2 0x107a7bb4f in fio_options_parse options.c:5203
    #3 0x1079b2214 in __parse_jobs_ini init.c:2076
    #4 0x1079aff07 in parse_jobs_ini init.c:2127
    #5 0x1079b7501 in parse_options init.c:2989
    #6 0x107b876a4 in main fio.c:42
    #7 0x7fff702f1cc8 in start (libdyld.dylib:x86_64+0x1acc8)

Fix the thinko (because opt is pointing to a later position) and
rearrange some code to make it clearer that olen is being used as an
initial offset

Signed-off-by: Sitsofe Wheeler <>
4 months agoMerge branch 'zipf-pareto-lock' of
Jens Axboe [Tue, 12 Jan 2021 17:52:54 +0000 (10:52 -0700)]
Merge branch 'zipf-pareto-lock' of

* 'zipf-pareto-lock' of
  distibutions: Extend flexibility of non-uniform random distributions

4 months agoMerge branch 'osx_fix' of
Jens Axboe [Tue, 12 Jan 2021 17:52:25 +0000 (10:52 -0700)]
Merge branch 'osx_fix' of

* 'osx_fix' of
  configure: fix compilation on recent macOS Xcode versions

4 months agoMerge branch 'fuzz' of
Jens Axboe [Tue, 12 Jan 2021 17:52:01 +0000 (10:52 -0700)]
Merge branch 'fuzz' of

* 'fuzz' of
  options: Fix buffer over read in fio_keyword_replace
  fuzz: Adds fuzz target for parse_jobs_ini

4 months agodistibutions: Extend flexibility of non-uniform random distributions
Adam Kupczyk [Tue, 29 Dec 2020 12:17:17 +0000 (13:17 +0100)]
distibutions: Extend flexibility of non-uniform random distributions

This change affects options random_distribution and file_service_type.
For pareto, zipf and gauss distribution a contept of `center` is implemented.
It allows to fix in place a value that is most probable to access.
fio --randseed=1 --ioengine=libaio --rw=randwrite --nrfiles=16 --bs=4k \
    --size=256m --allow_file_create=1 --write_iolog=log.txt \
    --file_service_type=gauss:10:0.1 --filename_format=object.\$filenum --name=x

cat log.txt |grep write |cut -f 1 -d " " |sort |uniq -c | sort -n | \
sed "s/[.]/ /" | while read a b c; do echo $c $b $a; done |sort -n
0 object 13429
1 object 17928
2 object 14724
3 object 7845
4 object 2476
5 object 468
6 object 44
7 object 3
12 object 24
13 object 318
14 object 1795
15 object 6482

Signed-off-by: Adam Kupczyk <>
4 months agooptions: Fix buffer over read in fio_keyword_replace
Philippe Antoine [Tue, 5 Jan 2021 12:00:13 +0000 (13:00 +0100)]
options: Fix buffer over read in fio_keyword_replace

By making sure allocated memory is filled with zeroes

Signed-off-by: Philippe Antoine <>
4 months agofuzz: Adds fuzz target for parse_jobs_ini
Philippe Antoine [Tue, 5 Jan 2021 11:59:46 +0000 (12:59 +0100)]
fuzz: Adds fuzz target for parse_jobs_ini

And build it

Signed-off-by: Philippe Antoine <>
4 months agoMerge branch 'num2str-patch' of
Jens Axboe [Sat, 9 Jan 2021 22:28:44 +0000 (15:28 -0700)]
Merge branch 'num2str-patch' of

* 'num2str-patch' of
  num2str: fix precision loss bug when the fractional part is close to 1

4 months agoMerge branch 'cpu-engine' of
Jens Axboe [Fri, 8 Jan 2021 03:53:00 +0000 (20:53 -0700)]
Merge branch 'cpu-engine' of

* 'cpu-engine' of
  engines/cpu: Fix td_vmsg() call

4 months agoengines/cpu: Fix td_vmsg() call
Bart Van Assche [Fri, 8 Jan 2021 03:03:45 +0000 (19:03 -0800)]
engines/cpu: Fix td_vmsg() call

Make sure that the third td_vmsg() argument is a fully expanded string and
also that the fourth argument is a string argument. This was detected by

Fixes: b213922390fe ("engines/cpu: style cleanups")
Signed-off-by: Bart Van Assche <>
4 months agoengines/cpu: style cleanups
Jens Axboe [Thu, 7 Jan 2021 23:51:18 +0000 (16:51 -0700)]
engines/cpu: style cleanups

Signed-off-by: Jens Axboe <>
4 months agoMerge branch 'evelu-qsort' of
Jens Axboe [Thu, 7 Jan 2021 23:46:32 +0000 (16:46 -0700)]
Merge branch 'evelu-qsort' of

* 'evelu-qsort' of
  engines/cpu: Adding qsort capabilities

4 months agoengines/cpu: Adding qsort capabilities
Erwan Velu [Wed, 6 Jan 2021 23:34:06 +0000 (00:34 +0100)]
engines/cpu: Adding qsort capabilities

This commit adds cpumode option into the cpuio engine.
By default, cpumode=noop to keep the current behavior.

If cpumode is set to qsort, fio will use a qsort algorithm
instead of the noop instructions to load the processor.
This mode will consume more cpu power and will be useful to
increase the pressure on the thermal and electrical components.

The expected cpu load is selected as per noop via the cpuload option.
qsort() consumes a lot of energy so the duration
of every loop will vary over time as the power management & cpu clock
changes. To ensure a proper calibration, the thinktime is adjusted after every
qsort() computation to be as precise as possible.

To give an order of magnitude, on an AMD 7502P (TDP=180W) :
cpuload=30, numjobs=64: packagewatt = 134W
cpuload=50, numjobs=64: packagewatt = 167W
cpuload=70, numjobs=64: packagewatt = 180W

The example file is updated to reflect this new capabilities.

The qsort code is coming from stress-qsort.c from stress-ng tool.
This software is also GPLv2 but author was informed and agreed with this usage.

Signed-off-by: Erwan Velu <>
4 months agoChange ARRAY_SIZE to FIO_ARRAY_SIZE
Jens Axboe [Wed, 6 Jan 2021 18:32:59 +0000 (11:32 -0700)]

Ensures that we don't clash with external symbols/macros.

Signed-off-by: Jens Axboe <>
4 months agoMerge branch 'drop_xp' of
Jens Axboe [Wed, 6 Jan 2021 14:29:48 +0000 (07:29 -0700)]
Merge branch 'drop_xp' of

* 'drop_xp' of
  windows: drop XP support

4 months agolog: only compile log_prevalist() if FIO_INC_DEBUG is set
Jens Axboe [Tue, 5 Jan 2021 20:14:28 +0000 (13:14 -0700)]
log: only compile log_prevalist() if FIO_INC_DEBUG is set

Only used for that case.

Signed-off-by: Jens Axboe <>
4 months agoMerge branch 'fix-get-next-file' of
Jens Axboe [Tue, 29 Dec 2020 23:36:32 +0000 (16:36 -0700)]
Merge branch 'fix-get-next-file' of

* 'fix-get-next-file' of
  io_u: Fix bad interaction with --openfiles and non-sequential file selection policy

4 months agoio_u: Fix bad interaction with --openfiles and non-sequential file selection policy
Adam Kupczyk [Mon, 28 Dec 2020 13:55:17 +0000 (14:55 +0100)]
io_u: Fix bad interaction with --openfiles and non-sequential file selection policy

Problem happens when --openfiles is set and file_service_type != FIO_FSERVICE_SEQ.
In function __get_next_file, we decrement file_service_left and if 0, we select
next file to operate on.
However, get_next_file_rand can return -EBUSY if too many files are already opened,
and __get_next_file exits with error.

In next invocation of __get_next_file, we decrement file_service_left again (from 0),
wrapping around to 2^32-1, effectively locking __get_next_file to always select the same.

Algorithm to observe bad behavior:
fio --randseed=1 --ioengine=libaio --rw=randwrite --nrfiles=256 --bs=4k --size=256m \
--loops=50 --allow_file_create=1 --write_iolog=log.txt --file_service_type=normal:20 \
--filename_format=object.\$filenum --name=x --openfiles=100

cat log.txt |grep write |cut -f 1 -d " " |sort |uniq -c | sort -n | sed "s/[.]/ /" \
| while read a b c; do echo $c $b $a; done |sort -n
70 object 17
71 object 19
72 object 22
73 object 65296
74 object 65255
75 object 33
76 object 27
77 object 25
78 object 65243
79 object 36
80 object 49
81 object 47

Signed-off-by: Adam Kupczyk <>
4 months agoconfigure: fix compilation on recent macOS Xcode versions
Sitsofe Wheeler [Wed, 23 Dec 2020 11:58:02 +0000 (11:58 +0000)]
configure: fix compilation on recent macOS Xcode versions

Back in ccf2d89d39b21bc8c7b497b40be5b82eadb80863 ("configure: try to
disable weak linking on OSX") disabling weak symbols was done to prevent
depending on features when building on a newer macOS but targeting an
older one. Fast-forward to Xcode 11.4 and it turns out Apple have broken
this by depending on weak symbols for critical symbols like FD_SET
leading to compilation errors like the following

ld: weak import of symbol '___darwin_check_fd_set_overflow' not supported because of option: -no_weak_imports for architecture x86_64

Other people/projects have been hit by this issue:

but Apple have kept this behaviour into XCode 12 so we should adapt.

- Introduce the concept of configure only CFLAGS
- Switch to using -Werror=partial-availability with the above
- Stop logging a message about disabling weak linking

The above should avoid finding features the target platform doesn't have
while allowing weak linking. The name CONFIGURE_CFLAGS is used to remain
similar to QEMU's configure. Fingers crossed this approach stays
supported by Apple...


Signed-off-by: Sitsofe Wheeler <>
4 months agowindows: drop XP support
Sitsofe Wheeler [Wed, 23 Dec 2020 16:01:52 +0000 (16:01 +0000)]
windows: drop XP support

Windows 2003 has been EOL since 2015 and Windows XP has been EOL since
2014. Technically fio hasn't been properly targetting XP anyway - (see
16d40a15783f9181d64082fc00a970dff485798f 'configure: be explicit about
"XP" Windows API version').

- Strip out support for XP and change the documentation to no longer
  mention it
- Stop targetting XP in the CI 32 bit build

Next stop, Windows 7...

Signed-off-by: Sitsofe Wheeler <>
4 months agoMerge branch 'terse_units' of
Jens Axboe [Thu, 24 Dec 2020 14:38:18 +0000 (07:38 -0700)]
Merge branch 'terse_units' of

* 'terse_units' of
  docs: add missing units to terse headings

4 months agodocs: add missing units to terse headings
Sitsofe Wheeler [Wed, 23 Dec 2020 07:00:02 +0000 (07:00 +0000)]
docs: add missing units to terse headings

In terse output many latency and bandwidth values have non-obvious
units. In the documentation of the format add a microsecond suffix to
latency values and a kb suffix to bandwidth values to clarify things.

Signed-off-by: Sitsofe Wheeler <>
4 months agonum2str: fix precision loss bug when the fractional part is close to 1
gloit042 [Sun, 20 Dec 2020 16:35:21 +0000 (00:35 +0800)]
num2str: fix precision loss bug when the fractional part is close to 1

The result of num2str(11999999999999, 4, 1, 0, N2S_NONE) should
be "12.0G", but current result is "11.0G".

Signed-off-by: Jiahao Li <>
4 months agoMerge branch 'github_issue' of
Jens Axboe [Thu, 17 Dec 2020 23:09:54 +0000 (16:09 -0700)]
Merge branch 'github_issue' of

* 'github_issue' of
  docs: add new section to REPORTING-BUGS and github issue templates

4 months agodocs: add new section to REPORTING-BUGS and github issue templates
Sitsofe Wheeler [Fri, 17 Feb 2017 07:09:12 +0000 (07:09 +0000)]
docs: add new section to REPORTING-BUGS and github issue templates

- Add a section to REPORTING-BUGS to clarify what is required for a good
  GitHub issue
- Add a note to REPORTING-BUGS that questions should go to the mailing
- Add a bug GitHub issue template that encourages people to read
  REPORTING-BUGS and include necessary information
- Add a GitHub issue enhancement template to try and set expectations
- Add a file because GitHub seems to link to it when a
  reporter first makes an issue

Signed-off-by: Sitsofe Wheeler <>
4 months agoMerge branch 'evelu-examples' of
Jens Axboe [Thu, 17 Dec 2020 15:51:38 +0000 (08:51 -0700)]
Merge branch 'evelu-examples' of

* 'evelu-examples' of
  examples: Clarify group_reporting usage
  examples: Clarify thread usage
  examples: Clarify time_based usage
  examples/fsx: Removing deprecated rwmixcycle options

4 months agoexamples: Clarify group_reporting usage
Erwan Velu [Wed, 16 Dec 2020 20:58:34 +0000 (21:58 +0100)]
examples: Clarify group_reporting usage

When reading examples, many jobs assign a value to group_reporting.
No value is expected, so let's have all examples using the same syntax.

Signed-off-by: Erwan Velu <>
4 months agoexamples: Clarify thread usage
Erwan Velu [Wed, 16 Dec 2020 20:48:43 +0000 (21:48 +0100)]
examples: Clarify thread usage

When reading examples, many jobs assign a value to thread.
No value is expected, so let's have all examples using the same syntax.

Signed-off-by: Erwan Velu <>
4 months agoexamples: Clarify time_based usage
Erwan Velu [Wed, 16 Dec 2020 20:42:36 +0000 (21:42 +0100)]
examples: Clarify time_based usage

When reading examples, many jobs assign a value to time_based.
It could even very misleading as some put the same value as for the runtime.
No value is expected, so let's have all examples using the same syntax.

Signed-off-by: Erwan Velu <>
4 months agoexamples/fsx: Removing deprecated rwmixcycle options
Erwan Velu [Wed, 16 Dec 2020 20:27:02 +0000 (21:27 +0100)]
examples/fsx: Removing deprecated rwmixcycle options

Let's remove deprecated option to avoid confusion if some reads it.

Signed-off-by: Erwan Velu <>
4 months agoMerge branch 'wip-rbd-engine-tweaks' of
Jens Axboe [Tue, 15 Dec 2020 16:18:28 +0000 (09:18 -0700)]
Merge branch 'wip-rbd-engine-tweaks' of

* 'wip-rbd-engine-tweaks' of
  engines/rbd: issue initial flush to enable writeback/around mode
  engines/rbd: add support for "direct=1" option

5 months agoflow: fix hang with flow control and zoned block devices
Aravind Ramesh [Mon, 7 Dec 2020 10:30:45 +0000 (10:30 +0000)]
flow: fix hang with flow control and zoned block devices

When flow control is specified between 2 or more threads executing a workload
with zonemode=zbd and flow_sleep is not specified by the user (default to zero),
then a job A can lock a zone Z with the flow ratio exceeded, resulting in the
job to not issue the IO until other jobs IOs balance the flow ratio.
Since job A will wait with the Zone Z locked, other jobs will deadlock waiting
for this zone lock if their workload resulted in this zone Z being chosen
as an IO target.

Executing io_u_quiesce() when the flow ratio is exceeded solves this potential
deadlock as job A will wait for the flow ratio to balance without holding the
zone A lock. To preserve the behavior for regular block devices and to avoid
extra overhead, io_u_quiesce() only needs to be executed when the workload
specifies zonedmode=zbd.

Signed-off-by: Aravind Ramesh <>
Signed-off-by: Shin'ichiro Kawasaki <>
Signed-off-by: Jens Axboe <>
5 months agoMerge branch 'reword-toolarge' of
Jens Axboe [Mon, 7 Dec 2020 23:47:00 +0000 (16:47 -0700)]
Merge branch 'reword-toolarge' of

* 'reword-toolarge' of
  filesetup: reword block size too large message

5 months agofilesetup: reword block size too large message
Sitsofe Wheeler [Sat, 5 Dec 2020 12:12:16 +0000 (12:12 +0000)]
filesetup: reword block size too large message


Signed-off-by: Sitsofe Wheeler <>
5 months agoMerge branch 'cufile' of
Jens Axboe [Sat, 5 Dec 2020 21:45:16 +0000 (14:45 -0700)]
Merge branch 'cufile' of

* 'cufile' of
  ioengine: Add libcufile I/O engine

5 months agoioengine: Add libcufile I/O engine
Brian T. Smith [Tue, 3 Nov 2020 23:54:55 +0000 (23:54 +0000)]
ioengine: Add libcufile I/O engine

The libcufile I/O engine uses NVIDIA GPUDirect Storage (GDS) cuFile API to perform
synchronous I/O directly against GPU buffers via nvidia-fs and a GDS-supported

'configure --enable-libcufile' enables the libcufile engine.

CFLAGS must specify the location of CUDA and cuFile headers.
e.g. CFLAGS="-I/usr/local/cuda/include -I/usr/local/cuda/lib64"

LDFLAGS must specify the location of CUDA and cuFile libraries.
e.g. LDFLAGS="-L/usr/local/cuda/lib64"

The paths used in CFLAGS and LDFLAGS depend upon the build host's
CUDA installation.

libcufile adds the following optons: gpu_dev_ids, cuda_io
Usage is documented in HOWTO, fio.1, examples/libcufile-cufile.fio
and examples/libcufile-posix.fio.

Note that enabling verify when cuda_io=cufile necessitates
cudaMemcpy() to populate the GPU buffer on a write and populate the
CPU buffer on a read. The primary goal of GDS is to not copy data
between CPU and GPU buffers.

Signed-off-by: Brian T. Smith <>
5 months agoFio 3.25 fio-3.25
Jens Axboe [Fri, 4 Dec 2020 18:47:42 +0000 (11:47 -0700)]
Fio 3.25

Signed-off-by: Jens Axboe <>
5 months agoMerge branch 'stat-int-creep3' of
Jens Axboe [Thu, 3 Dec 2020 23:09:43 +0000 (16:09 -0700)]
Merge branch 'stat-int-creep3' of

* 'stat-int-creep3' of
  stat: Prevent the BW and IOPS logging interval from creeping up

5 months agoengines/rbd: issue initial flush to enable writeback/around mode
Jason Dillaman [Wed, 2 Dec 2020 19:03:42 +0000 (14:03 -0500)]
engines/rbd: issue initial flush to enable writeback/around mode

By default librbd will use a writethrough cache until it receives
the first flush. Send a no-op flush after opening the image to
ensure writearound (default) or writeback cache modes can be
properly enabled (unlesss direct=1).

Signed-off-by: Jason Dillaman <>
5 months agoengines/rbd: add support for "direct=1" option
Jason Dillaman [Wed, 2 Dec 2020 18:54:58 +0000 (13:54 -0500)]
engines/rbd: add support for "direct=1" option

When direct IO is enabled, the librbd in-memory cache will be
automatically disabled.

Signed-off-by: Jason Dillaman <>
5 months agostat: Prevent the BW and IOPS logging interval from creeping up
Jeff Lien [Mon, 2 Nov 2020 16:35:05 +0000 (10:35 -0600)]
stat: Prevent the BW and IOPS logging interval from creeping up

With the current FIO version, we're seeing the interval for writing
the BW and IOPS logs increase by 1 msec over time.  The --log_avg_msec
option is used to set the time interval for logging BW and IOPs.

This issue will cause dip ( low ) in IOPS and BW plot vs time because
parsing/plot tool can't get correct sum total IOPS and BW from
multiple jobs when there is 1-2us increment in timestamp.

The issue was introduced in version 3.18 with the following
commit - 31eca64.  And these changes made it slightly better: 0f77d30,
and PR #973 (0e59dd6), but the problem still exists.

This commit also contains the change from PR #982.  I believe we need
both changes to totally resolve the issue of the BW and IOPs logging
interval creep.

Signed-off-by: Jeff Lien <>
Inspired-by: Alex Merenstein <>
5 months agoMerge branch 'regrow_agg_logs' of
Jens Axboe [Fri, 27 Nov 2020 15:55:12 +0000 (08:55 -0700)]
Merge branch 'regrow_agg_logs' of

* 'regrow_agg_logs' of
  stat: allow bandwidth log stats to grow to MAX_LOG_ENTRIES

5 months agostat: allow bandwidth log stats to grow to MAX_LOG_ENTRIES
Panagiotis Moustafellos [Fri, 27 Nov 2020 10:47:23 +0000 (12:47 +0200)]
stat: allow bandwidth log stats to grow to MAX_LOG_ENTRIES

This commit enables agg logs to grow to MAX_LOG_ENTRIES, like any thread logs.

Co-authored-by: Sitsofe Wheeler <>
Tested-by: Panagiotis Moustafellos <>
Signed-off-by: Panagiotis Moustafellos <>
5 months agoMerge branch 'update-fio-ioops-version' of
Jens Axboe [Wed, 25 Nov 2020 18:01:00 +0000 (11:01 -0700)]
Merge branch 'update-fio-ioops-version' of

* 'update-fio-ioops-version' of
  ioengines: increment FIO_IOOPS_VERSION

5 months agoioengines: increment FIO_IOOPS_VERSION
Ivan Andreyev [Sun, 22 Nov 2020 15:14:26 +0000 (18:14 +0300)]
ioengines: increment FIO_IOOPS_VERSION

Signed-off-by: Ivan Andreyev <>
5 months agoerror out if ENOSPC during file layout
Kushal Kumaran [Fri, 20 Nov 2020 05:00:05 +0000 (21:00 -0800)]
error out if ENOSPC during file layout


When I run fio with --create_only=1 and it runs out of space, it still
exits with status 0.  I could not figure out if this was intentional
(except for the fill_device case, where this is obviously the expected

$ cat ~/fio

$ df -h .
Filesystem      Size  Used Avail Use% Mounted on
/dev/loop6      976M  2.6M  907M   1% /home/kushal/mounts/testdisk
$ fio ~/fio --create_only=1
trivial-readwrite-1g: (g=0): rw=rw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=pos
ixaio, iodepth=1
Starting 1 thread
trivial-readwrite-1g: Laying out IO file (1 file / 2048MiB)
fio: ENOSPC on laying out file, stopping

Run status group 0 (all jobs):

Disk stats (read/write):
  loop6: ios=0/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%
$ echo $?

A trivial patch for this gives me the behavior I expect.

Signed-off-by: Jens Axboe <>
6 months agoMerge branch 'segmented-threads'
Jens Axboe [Fri, 13 Nov 2020 17:06:26 +0000 (10:06 -0700)]
Merge branch 'segmented-threads'

Pull in the segmented segments code. This avoids having to setup a
thread area segment for the max (4096) number of jobs. Instead it's
done in chunks of 8. This results in a net reduction of more than 1G
of memory used for the normal use case.

* segmented-threads:
  Kill off 'max_jobs'
  Add thread_segments as needed
  Wrap thread_data in thread_segment

6 months agoKill off 'max_jobs' segmented-threads
Jens Axboe [Fri, 13 Nov 2020 16:17:11 +0000 (09:17 -0700)]
Kill off 'max_jobs'

We don't really need this anymore - we cap the number of segments
separately, and since the chunk size is small enough, this removes the
need to have per-OS limits on the thread area size.

Signed-off-by: Jens Axboe <>
6 months agoAdd thread_segments as needed
Jens Axboe [Fri, 13 Nov 2020 16:02:35 +0000 (09:02 -0700)]
Add thread_segments as needed

Setup segments in units of 8, and add extra ones as needed. This avoids
having to setup one huge segment upfront for the maximum number of jobs.

Signed-off-by: Jens Axboe <>
6 months agoWrap thread_data in thread_segment
Jens Axboe [Fri, 13 Nov 2020 15:33:50 +0000 (08:33 -0700)]
Wrap thread_data in thread_segment

No functional changes in this patch, just in preparation for having
multiple shm segments for the thread_data structures. They are getting
very large, and since it's hard to know upfront how many we need, let's
instead refactor the code a bit to allow us to add chunks of them as
needed when parsing the job file (or command line).

Signed-off-by: Jens Axboe <>
6 months agot/ tweak terse output parse
Jens Axboe [Thu, 12 Nov 2020 18:26:58 +0000 (11:26 -0700)]
t/ tweak terse output parse

Seems like we still have some wonkiness on whether or not the fio version
string is printed. For now, let's just cut this down to '3;' for
detecting terse (v3) output, don't have time to fully investigate this
one right now.

Signed-off-by: Jens Axboe <>
6 months agoFio 3.24 fio-3.24
Jens Axboe [Thu, 12 Nov 2020 16:40:58 +0000 (09:40 -0700)]
Fio 3.24

Signed-off-by: Jens Axboe <>
6 months agoMakefile: fix fio version gen
Jens Axboe [Thu, 12 Nov 2020 16:36:45 +0000 (09:36 -0700)]
Makefile: fix fio version gen

A previous commit moved the cflags override, but neglected to move the
version generation that is being used by it.

Fixes: f4bd2c3d80bc ("fix dynamic engine build")
Signed-off-by: Jens Axboe <>
6 months agot/ correct terse parse
Jens Axboe [Mon, 9 Nov 2020 19:13:25 +0000 (12:13 -0700)]
t/ correct terse parse

For some reason the script expects '3;fio-' when it should be latency.

Signed-off-by: Jens Axboe <>
6 months agoMake sure we do libaio engine compatability names
Jens Axboe [Mon, 9 Nov 2020 18:47:50 +0000 (11:47 -0700)]
Make sure we do libaio engine compatability names

This goes for both the dynamic engines, and the builtin ones. If
'aio' or 'linuxaio' is used, we want 'libaio'.

Signed-off-by: Jens Axboe <>
6 months agoconfigure: remove libaio-uring remnant
Jens Axboe [Mon, 9 Nov 2020 17:42:03 +0000 (10:42 -0700)]
configure: remove libaio-uring remnant

This was missed in the previous patch.

Fixes: d4946e79e833 ("Remove the "libaio over io_uring" mode")
Signed-off-by: Jens Axboe <>
6 months agoMakefile: ensure that external libs are linked properly with dynamic engine
Jens Axboe [Mon, 9 Nov 2020 17:05:11 +0000 (10:05 -0700)]
Makefile: ensure that external libs are linked properly with dynamic engine

Signed-off-by: Jens Axboe <>
6 months agoRemove the "libaio over io_uring" mode
Jens Axboe [Mon, 9 Nov 2020 16:59:07 +0000 (09:59 -0700)]
Remove the "libaio over io_uring" mode

Not going to maintain this mode in liburing, so just get rid of it. It's
also caused some confusion where users think they need this enabled to
run io_uring, which isn't the case.

Signed-off-by: Jens Axboe <>
6 months agoFIO_EXT_ENG_DIR should be default path
Jens Axboe [Mon, 9 Nov 2020 16:54:10 +0000 (09:54 -0700)]
FIO_EXT_ENG_DIR should be default path

Distros will override this anyway, make sure the default is what
will work on a normal build out of git.

Signed-off-by: Jens Axboe <>
6 months agolist all available dynamic ioengines with --enghelp
Eric Sandeen [Mon, 9 Nov 2020 16:46:30 +0000 (10:46 -0600)]
list all available dynamic ioengines with --enghelp

When dynamic engines are enabled, "fio --enghelp" does not list all
available engines as the man page says it will:

If no ioengine is given, list all available ioengines.

Fix this by opening FIO_EXT_ENG_DIR and attempting dlopen_ioengine
everything that's found there.

Signed-off-by: Eric Sandeen <>
Signed-off-by: Jens Axboe <>
6 months agofix dynamic engine loading for libaio engine etc
Eric Sandeen [Mon, 9 Nov 2020 16:46:29 +0000 (10:46 -0600)]
fix dynamic engine loading for libaio engine etc

The dynamic engine loading for libaio (and some others) currently
fails because the dlopen routine is looking for ("lib%s", enginename)
which translates into "":

openat(AT_FDCWD, "/usr/lib64/fio/", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
Engine libiscsi not found; Either name is invalid, was not built, or fio-engine-libiscsi package is missing.
fio: engine libiscsi not loadable
IO engine libiscsi not found

The Makefile decide to name this engine "iscsi" instead of "libiscsi",
which leads to "" not "" hence the mismatch.

OTOH, "" seems a bit bonkers.

Try to resolve all this by:

1) make all of the engine names match the documented engine names
   in the Makefile, i.e. "iscsi" -> "libiscsi"
2) change the created library filenames to "fio-$(ENGINENAME)"
   from "lib$(ENGINENAME)" to avoid the "liblib" prefix.

So now we consistently have the libraries named "fio-$(ENGINENAME).so"


Fixes: 5a8a6a03 ("configure: new --dynamic-libengines build option")
Signed-off-by: Eric Sandeen <>
Signed-off-by: Jens Axboe <>
6 months agofix dynamic engine build
Eric Sandeen [Mon, 9 Nov 2020 16:46:28 +0000 (10:46 -0600)]
fix dynamic engine build

Builds with --dynamic-libengines are currently broken, because the rule
to build the libraries was moved out of the template that creates them:

make: *** No rule to make target 'engines/', needed by 'all'.  Stop.
make: *** Waiting for unfinished jobs....

Fix this by moving the rule back into the template, calling the template
to create rules only after FIO_CFLAGS has been incorporated into CFLAGS,
and using CFLAGS + -fPIC in that rule rather than FIO_CFLAGS.

Fixes: 8a2cf08d29ac ("Makefile: introduce FIO_CFLAGS")
Signed-off-by: Eric Sandeen <>
Signed-off-by: Jens Axboe <>
6 months agoRevert "Windows: update dobuild.cmd to run the configure/make"
Jens Axboe [Thu, 5 Nov 2020 22:33:00 +0000 (15:33 -0700)]
Revert "Windows: update dobuild.cmd to run the configure/make"

This reverts commit 38c2f9384db8dbd93f59d965d70ab0d3a53343fa.

It's causing issues for CI, revert it for now.

Signed-off-by: Jens Axboe <>
6 months agoMerge branch 'master' of
Jens Axboe [Thu, 5 Nov 2020 22:17:11 +0000 (15:17 -0700)]
Merge branch 'master' of

* 'master' of
  engines/hdfs: swap fio_hdfsio_init and fio_hdfsio_io_u_init
  Makefile: fix usage of JAVA_HOME environmental variable

6 months agoMerge branch 'patch-1' of
Jens Axboe [Thu, 5 Nov 2020 22:16:50 +0000 (15:16 -0700)]
Merge branch 'patch-1' of

* 'patch-1' of
  goptions: correct postfix

6 months agoengines/hdfs: swap fio_hdfsio_init and fio_hdfsio_io_u_init
Albert Chang [Thu, 5 Nov 2020 18:30:22 +0000 (13:30 -0500)]
engines/hdfs: swap fio_hdfsio_init and fio_hdfsio_io_u_init

Commit 08dc3bd50 initialized the io engine before the io_u buffers. The
unintended consequence for the libhdfs io engine was that the HDFS
connection was not established before attempting to open HDFS file
handles. This caused a NPE in the java layer on start-up preventing the
libhdfs io engine from being able to be used.

Signed-off-by: Albert Chang <>
6 months agoMakefile: fix usage of JAVA_HOME environmental variable
Albert Chang [Thu, 5 Nov 2020 18:29:01 +0000 (13:29 -0500)]
Makefile: fix usage of JAVA_HOME environmental variable

Signed-off-by: Albert Chang <>
6 months agoWindows: update dobuild.cmd to run the configure/make
Rebecca Cran [Tue, 3 Nov 2020 02:47:41 +0000 (02:47 +0000)]
Windows: update dobuild.cmd to run the configure/make

Add a new shell script, which runs under Cygwin. This
is invoked via dobuild.cmd to run the configure and make steps of
the build. In this way, a single command, run from a Developer Command
Prompt, can be used to go from a clean workspace to a final installer

Signed-off-by: Rebecca Cran <>
Signed-off-by: Jens Axboe <>
6 months agogoptions: correct postfix
gloit042 [Wed, 4 Nov 2020 04:46:24 +0000 (12:46 +0800)]
goptions: correct postfix

Signed-off-by: Jiahao Li <>
6 months agofio: fix dynamic engines soname definition
Yigal Korman [Mon, 5 Oct 2020 17:08:54 +0000 (20:08 +0300)]
fio: fix dynamic engines soname definition

The SONAME of the engines should not match the name of the dependent
library. Otherwise it confuses the dynamic loader into thinking the
dependency is already resolved.

Prefixing the name with fio make more sense here.

Signed-off-by: Yigal Korman <>
Signed-off-by: Jens Axboe <>
6 months agot/zbd: Fix test target size of test case #48
Shin'ichiro Kawasaki [Fri, 30 Oct 2020 06:57:40 +0000 (15:57 +0900)]
t/zbd: Fix test target size of test case #48

Option --size was not specified to the fio command of test case #48. It
resulted in write operations to all available sequential write required
zones and relaxed zone locking test condition. Specify the option to
limit test target to 16 zones so that zone locking is tested with
expected condition.

Fixes: 3bd2078bdd1c ("zbd: add test for stressing zone locking")
Signed-off-by: Shin'ichiro Kawasaki <>
Reviewed-by: Dmitry Fomichev <>
Signed-off-by: Jens Axboe <>
6 months agozbd: Avoid excessive zone resets
Shin'ichiro Kawasaki [Fri, 30 Oct 2020 06:57:39 +0000 (15:57 +0900)]
zbd: Avoid excessive zone resets

When zbd_reset_zone() is called for a zone repeatedly, reset zone command
is issued multiple times to a single zone even though its status is
empty. This is excessive and meaningless. Especially when zones are reset
at file set up with multiple jobs, zone reset is repeated for each job
and delays fio work load start.

To avoid the repeated zone resets, check the write pointer of the zone
before issuing zone reset command. If the write pointer is at the zone
start, do not reset the zone and just return.

Signed-off-by: Shin'ichiro Kawasaki <>
Reviewed-by: Dmitry Fomichev <>
Signed-off-by: Jens Axboe <>
6 months agoMerge branch 'o_dsync' of
Jens Axboe [Wed, 28 Oct 2020 22:20:43 +0000 (16:20 -0600)]
Merge branch 'o_dsync' of

* 'o_dsync' of
  extend --sync to allow {sync,dsync,0,1}, to support O_DSYNC

6 months agoextend --sync to allow {sync,dsync,0,1}, to support O_DSYNC
Andres Freund [Fri, 4 Sep 2020 00:47:10 +0000 (17:47 -0700)]
extend --sync to allow {sync,dsync,0,1}, to support O_DSYNC


7 months agoDisallow offload IO mode for engines marked with FIO_NO_OFFLOAD
Jens Axboe [Thu, 15 Oct 2020 02:11:56 +0000 (20:11 -0600)]
Disallow offload IO mode for engines marked with FIO_NO_OFFLOAD

Previous commits did this for all async engines, this can potentially
break existing job files. There are only certain cases where it fails,
for now at least mark io_uring as one of those.

Reported-by: Jeff Furlong <>
Signed-off-by: Jens Axboe <>
7 months agoMerge branch 'patch-1' of into master
Jens Axboe [Tue, 13 Oct 2020 19:07:37 +0000 (13:07 -0600)]
Merge branch 'patch-1' of into master

* 'patch-1' of
  getopt_long: avoid variable global initialization

7 months agogetopt_long: avoid variable global initialization
Cheng Li [Tue, 13 Oct 2020 17:03:12 +0000 (10:03 -0700)]
getopt_long: avoid variable global initialization

Issue #1100 shows that an address sanitizer(ASAN) complains about a few
global variables that are initialized globally in the getopt_long.c file.
We look into these variables and found they do not need to be initialized
globally. This patch fixes the issue by cleaning up the global
initialization for these variables.

Signed-off-by: Cheng Li <>
7 months agoMerge branch 'master' of into master
Jens Axboe [Sun, 11 Oct 2020 02:30:52 +0000 (20:30 -0600)]
Merge branch 'master' of into master

* 'master' of
  helper_thread: Increase timer accuracy
  helper_thread: Rework the interval timer code
  Change the return value of two functions from 'void' into 'int'
  helper_thread: Introduce the wait_for_action() function
  helper_thread: Introduce a function for blocking Unix signals
  gettime: Introduce rel_time_since()
  gettime: Simplify get_cycles_per_msec()
  gettime: Introduce fio_get_mono_time()
  configure: Remove the CLOCK_MONOTONIC_PRECISE probe

7 months agosimplify MB/s, IOPS interactive printout code
Alexey Dobriyan [Sat, 10 Oct 2020 14:35:10 +0000 (17:35 +0300)]
simplify MB/s, IOPS interactive printout code

Use loops(!) instead of hardcoded if-else pairs.

Make it easier to add DDIR_APPEND.

Signed-off-by: Alexey Dobriyan (SK hynix) <>
Signed-off-by: Jens Axboe <>
7 months agoflow: use unsigned long for the counter
Jens Axboe [Sat, 10 Oct 2020 15:16:51 +0000 (09:16 -0600)]
flow: use unsigned long for the counter

Same size on 64-bit, it'll be 32-bit on 32-bit archs. But we can't
reliably use 64-bit atomics on 32-bit archs.

Signed-off-by: Jens Axboe <>
7 months agoflow: avoid holes in struct fio_flow
Jens Axboe [Sat, 10 Oct 2020 15:06:02 +0000 (09:06 -0600)]
flow: avoid holes in struct fio_flow

Signed-off-by: Jens Axboe <>
7 months agoMerge branch 'master' of into master
Jens Axboe [Fri, 9 Oct 2020 13:28:55 +0000 (07:28 -0600)]
Merge branch 'master' of into master

* 'master' of
  td_var: avoid arithmetic on a pointer to void cast (#1096)

7 months agotd_var: avoid arithmetic on a pointer to void cast (#1096)
Cheng Li [Thu, 8 Oct 2020 18:22:34 +0000 (11:22 -0700)]
td_var: avoid arithmetic on a pointer to void cast (#1096)

Issue 1096 shows that a compiler complains an `arithmetic on a pointer to
void` error when `-Wpointer-arith` is turned on in Makefile. Although
there are many violating instances, we only fix the one in `parse.h`
because it is shared across many core source code files. This patch fixes
the `arithmetic on a pointer to void` issue by casting it to a uintptr_t
type first and perform manipulation, and then cast it back to void* type.

Signed-off-by: Cheng Li <>
7 months agohelper_thread: Increase timer accuracy
Bart Van Assche [Sat, 12 Sep 2020 14:58:00 +0000 (07:58 -0700)]
helper_thread: Increase timer accuracy

The helper_thread uses select() to wait for the next event. select() rounds
up its timeout argument to 1 / HZ. Increase timer accuracy by using a
high-res timer if possible. This patch has been tested by running the
following fio command and by verifying that the next_log and
msec_to_next_event were always above 100 ms:

./fio --debug=helperthread --bandwidth-log --write_bw_log=log --log_avg_msec=1000 --ioengine=null --name=null --size=1G --rw=randwrite --loops=999999 --runtime=60

Signed-off-by: Bart Van Assche <>
7 months agohelper_thread: Rework the interval timer code
Bart Van Assche [Wed, 9 Sep 2020 02:34:45 +0000 (19:34 -0700)]
helper_thread: Rework the interval timer code

Instead of adding the number of milliseconds that have elapsed when
a timer fires, add the timer interval. Reduce code duplication by
introducing a helper function that verifies whether or not a timer
has expired.

Signed-off-by: Bart Van Assche <>
7 months agoChange the return value of two functions from 'void' into 'int'
Bart Van Assche [Sun, 20 Sep 2020 23:27:58 +0000 (16:27 -0700)]
Change the return value of two functions from 'void' into 'int'

This patch does not change any functionality but makes the next patch in
this series easier to read.

Signed-off-by: Bart Van Assche <>
7 months agohelper_thread: Introduce the wait_for_action() function
Bart Van Assche [Wed, 9 Sep 2020 02:32:02 +0000 (19:32 -0700)]
helper_thread: Introduce the wait_for_action() function

This patch makes helper_thread_main() easier to read.

Signed-off-by: Bart Van Assche <>
7 months agohelper_thread: Introduce a function for blocking Unix signals
Bart Van Assche [Sat, 12 Sep 2020 14:43:56 +0000 (07:43 -0700)]
helper_thread: Introduce a function for blocking Unix signals

This patch reworks commit c31092b8ac06 ("Block signals for the helper
thread") but does not change any functionality.

Signed-off-by: Bart Van Assche <>
7 months agogettime: Introduce rel_time_since()
Bart Van Assche [Fri, 4 Sep 2020 23:18:53 +0000 (16:18 -0700)]
gettime: Introduce rel_time_since()

rel_time_since() subtracts two timespecs and returns the result as a
signed integer. Instead of using the implementation from mtime_since(),
use an implementation that only performs one division instead of two.

Signed-off-by: Bart Van Assche <>
7 months agogettime: Simplify get_cycles_per_msec()
Bart Van Assche [Mon, 7 Sep 2020 22:55:27 +0000 (15:55 -0700)]
gettime: Simplify get_cycles_per_msec()

Call fio_get_mono_time() directly instead of setting fio_clock_source and
calling __fio_gettime().

Signed-off-by: Bart Van Assche <>
7 months agogettime: Introduce fio_get_mono_time()
Bart Van Assche [Sun, 20 Sep 2020 18:40:26 +0000 (11:40 -0700)]
gettime: Introduce fio_get_mono_time()

Introduce a new function for querying the monotonic clock, something that
is necessary in every context where relative time is measured and where
wall clock time jumps should have no effect. Remove fill_clock_gettime()
since the only contexts where CLOCK_MONOTONIC_RAW are used are
get_cycles_per_msec() and --clocksource=clock_gettime. I think both contexts
should use CLOCK_MONOTONIC instead of its raw variant such that the values
read from the clock are frequency adjusted (a computer clock crystal can
deviate up to 500 ppm from its nominal frequency).

This patch improves accuracy of the helper_thread code on Darwin. Darwin
supports CLOCK_MONOTONIC but not pthread_condattr_setclock(). In other
words, this patch causes the helper thread code to switch from the
real-time clock to the monotonic clock on Darwin.

Signed-off-by: Bart Van Assche <>
7 months agoconfigure: Remove the CLOCK_MONOTONIC_PRECISE probe
Bart Van Assche [Mon, 7 Sep 2020 23:03:12 +0000 (16:03 -0700)]
configure: Remove the CLOCK_MONOTONIC_PRECISE probe

CLOCK_MONOTONIC_PRECISE is not used in the fio source code. Hence use the
probe for that constant. The removed test was introduced by commit
67bf982340d9 ("Add configure script").

Signed-off-by: Bart Van Assche <>
7 months agoMerge branch 'msys2' of into master
Jens Axboe [Tue, 15 Sep 2020 01:43:39 +0000 (19:43 -0600)]
Merge branch 'msys2' of into master

* 'msys2' of
  travis: cleanup build script
  configure: cleanup lex/yacc tests
  memlock: avoid type confusion in format string
  Makefile: introduce FIO_CFLAGS
  appveyor: cleanup and add separate install script
  Makefile/ci: Don't pass CFLAGS when linking
  configure/Makefile: add option to generate pdb symbols
  configure: be explicit about "XP" Windows API version
  windows: fix DWORD format string complaints
  windows: fix wrong format strings
  net: coerce the result of htonl before printing
  configure: pass non-null pointer to (v)asprintf

7 months agoMerge branch 'backend' of into master
Jens Axboe [Tue, 15 Sep 2020 01:42:23 +0000 (19:42 -0600)]
Merge branch 'backend' of into master

* 'backend' of
  backend: Use asprintf() instead of strlen() + sprintf()
  backend: Remove two superfluous casts

8 months agotravis: cleanup build script
Sitsofe Wheeler [Thu, 3 Sep 2020 22:02:31 +0000 (23:02 +0100)]
travis: cleanup build script

- Standardise on spaces for indentation
- Exit on error or using variables before they are defined
- Fix up shellcheck complaints by exporting some variables
- Drop printing of python 2 information as we don't use it
- Use type -p rather than which

Signed-off-by: Sitsofe Wheeler <>
8 months agoconfigure: cleanup lex/yacc tests
Sitsofe Wheeler [Mon, 31 Aug 2020 13:59:59 +0000 (14:59 +0100)]
configure: cleanup lex/yacc tests

- Switch to prebuilt function for finding executable
- Search for the fl library from flex too
- Test lex command output directly

This allows lexing to be more likely to be found (e.g. MSYS2 but sadly
not MSYS2 Mingw-w64).

Signed-off-by: Sitsofe Wheeler <>
8 months agomemlock: avoid type confusion in format string
Sitsofe Wheeler [Mon, 31 Aug 2020 10:44:11 +0000 (11:44 +0100)]
memlock: avoid type confusion in format string

Windows GCC has the followng grumble:

t/memlock.c: In function 'worker':
t/memlock.c:25:26: error: format '%lu' expects argument of type 'long unsigned int', but argument 3 has type 'size_t' {aka 'long long unsigned int'} [-Werror=format=]
   25 |    printf("loop%d: did %lu MiB\n", i+1, size/(1024UL*1024UL));
      |                        ~~^              ~~~~~~~~~~~~~~~~~~~~
      |                          |                  |
      |                          long unsigned int  size_t {aka long long unsigned int}
      |                        %llu

Fix it up by just using the original variable that was already in

Signed-off-by: Sitsofe Wheeler <>
8 months agoMakefile: introduce FIO_CFLAGS
Sitsofe Wheeler [Sat, 22 Aug 2020 17:01:31 +0000 (18:01 +0100)]
Makefile: introduce FIO_CFLAGS

Build on the work  done in 4c0b3d98f2d05ddd3f16262c466dcedb22158065
("configure/Makefile: don't override user CFLAGS") by builing up all the
internal flags in a temporary variable.

Signed-off-by: Sitsofe Wheeler <>