Jens Axboe [Thu, 3 Jun 2021 15:01:52 +0000 (09:01 -0600)]
Merge branch 'fix/928' of https://github.com/larsks/fio
* 'fix/928' of https://github.com/larsks/fio:
fix fio2gnuplot to work with new logging format
Jens Axboe [Thu, 3 Jun 2021 13:45:22 +0000 (07:45 -0600)]
Merge branch 'evelu-test' of https://github.com/ErwanAliasr1/fio
* 'evelu-test' of https://github.com/ErwanAliasr1/fio:
Makefile: Avoid using built-in stpcpy during clang build
ci: Reporting installed msys2 packages
ci: Installing missing toolchain
Erwan Velu [Wed, 2 Jun 2021 14:15:59 +0000 (16:15 +0200)]
Makefile: Avoid using built-in stpcpy during clang build
Since clang 12, during the clang build, noticed by the CI, the linking
fails as clang optimize some string functions to stpcpy.
LINK fio
lld-link: error: undefined symbol: stpcpy
>>> referenced by C:\projects\fio\options.c:5305
>>> options.o:(fio_options_parse)
Two possible implementations :
- Adding stpcpy in fio as the kernel did : https://lore.kernel.org/lkml/
20200815002417.
1512973-1-ndesaulniers@google.com/T/
- Disable the implicit stpcpy
To avoid adding code into fio, the latter option was used.
Signed-off-by: Erwan Velu <e.velu@criteo.com>
Erwan Velu [Wed, 2 Jun 2021 13:52:06 +0000 (15:52 +0200)]
ci: Reporting installed msys2 packages
When reproducing a build locally, it's important to be on the same
release as the CI.
So let's listi the installed packages so we can compare the two builds more easily.
Signed-off-by: Erwan Velu <e.velu@criteo.com>
Erwan Velu [Wed, 2 Jun 2021 13:05:17 +0000 (15:05 +0200)]
ci: Installing missing toolchain
When trying to rebuild a failed build on a real windows system,
the toolchain is missing.
Let's add the toolchain here so we can reuse the script locally too.
Signed-off-by: Erwan Velu <e.velu@criteo.com>
Niklas Cassel [Thu, 27 May 2021 11:12:32 +0000 (11:12 +0000)]
zbd: add a new --ignore_zone_limits option
In commit
d2f442bc0bd5 ("ioengines: add get_max_open_zones zoned block
device operation") we added a check that verifies that the specified
--max_open_zones value is lower than the max value reported by the device.
For ZNS devices there is a max open zones and a max active zones limit.
For ZAC/ZBC devices there is only a max open zones limit.
On ZAC/ZBC, there is thus no limit on the amount of zones that can be
in zone state closed.
When doing a write to an empty or closed zone, a ZAC/ZBC drive will
close an arbitrary implicit open zone in order to handle the write.
The ZNS specification has no requirement on closing a zone in order to
handle a write to an empty or closed zone. The drive is free to return
an error.
Even on ZAC/ZBC, you do not want to exceed the max open zones limit,
since it will lead to additional implicit close zone and implicit open
zone operations, which may degrade performance.
However, it seems that this is sometimes done on purpose, in order to
measure the overhead of these additional operations. Therefore, add
an option that allows the user to ignore the reported device limits.
Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Reviewed-by: Damien Le Moal <damien.lemoal@wdc.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Niklas Cassel [Thu, 27 May 2021 11:12:31 +0000 (11:12 +0000)]
zbd: add missing client/server support for option max_open_zones
Ensure that we convert the max_open_zones option for client/server.
Use __cpu_to_le32()/__le32_to_cpu() rather than
cpu_to_le32()/le32_to_cpu(), since max_open_zones is defined
as int rather than unsigned int in thread_options.h.
Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Reviewed-by: Damien Le Moal <damien.lemoal@wdc.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 27 May 2021 22:04:16 +0000 (16:04 -0600)]
Merge branch 'fix-libpmem' of https://github.com/lukaszstolarczuk/fio
* 'fix-libpmem' of https://github.com/lukaszstolarczuk/fio:
engines/libpmem: do not call drain on close
engines/libpmem: cleanup a little code, comments and example
engines/libpmem: set file open/create mode always to RW
Jens Axboe [Wed, 26 May 2021 16:10:32 +0000 (10:10 -0600)]
Fio 3.27
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Tue, 25 May 2021 22:54:13 +0000 (16:54 -0600)]
Merge branch 'master' of https://github.com/DevriesL/fio
* 'master' of https://github.com/DevriesL/fio:
android: add support for NDK sharedmem
DevriesL [Tue, 25 May 2021 15:45:11 +0000 (23:45 +0800)]
android: add support for NDK sharedmem
Android add support for NDK sharedmem since API level 26 and prohibit
the directly use of ashmem since API level 29, so we can use sharedmem
if targeting API level is higher than 26.
Signed-off-by: DevriesL <therkduan@gmail.com>
Jens Axboe [Tue, 18 May 2021 23:34:38 +0000 (17:34 -0600)]
Merge branch 'taras/nfs-upstream' of https://github.com/tarasglek/fio-1
* 'taras/nfs-upstream' of https://github.com/tarasglek/fio-1:
clean up nfs example
skip skeleton comments
single line bodies
C-style comments
NFS configure fixes
NFS engine
Jens Axboe [Fri, 14 May 2021 15:36:59 +0000 (09:36 -0600)]
Merge branch '2021-05-13/stat-fix-integer-overflow' of https://github.com/flx42/fio
* '2021-05-13/stat-fix-integer-overflow' of https://github.com/flx42/fio:
stat: fix integer overflow in convert_agg_kbytes_percent
Niklas Cassel [Fri, 14 May 2021 12:53:15 +0000 (12:53 +0000)]
engines/libzbc: add support for the get_max_open_zones io op
Add support for the new .get_max_open_zones io operation.
zbc.c will only ever call this callback for host-managed devices.
Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Reviewed-by: Damien Le Moal <damien.lemoal@wdc.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Niklas Cassel [Fri, 14 May 2021 12:53:14 +0000 (12:53 +0000)]
ioengines: add get_max_open_zones zoned block device operation
Define a new IO engine operation to get the maximum number of open zones.
Like the existing IO engine operations: .get_zoned_model, .report_zones,
and .reset_wp, this new IO engine operation is only valid for zoned block
devices.
Similarly to the other zbd IO engine operations, also provide a default
implementation inside oslib/linux-blkzoned.c that will be used if the
ioengine does not override it.
The default Linux oslib implementation is implemented similarly to
blkzoned_get_zoned_model(), i.e. it will return a successful error code
even when the sysfs attribute does not exist.
This is because the sysfs max_open_zones attribute was introduced first
in Linux v5.9.
All error handling is still there, so an ioengine that provides its own
implementation will still have its error code respected properly.
Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Niklas Cassel [Fri, 14 May 2021 12:52:51 +0000 (12:52 +0000)]
oslib/linux-blkzoned: move sysfs reading into its own function
Move the sysfs reading into its own function so that it can be reused.
This new function will be reused in a following patch.
No functional change intended.
Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Niklas Cassel [Fri, 14 May 2021 12:52:51 +0000 (12:52 +0000)]
zbd: only put an upper limit on max open zones once
There is an upper limit that is checked for each td, and for each file,
even though a file has a pointer to a zoned_block_device_info that has
already been created. Multiple files, from the same or from another td
can point to the same zoned_block_device_info.
All zoned_block_device_info:s have already been created earlier in the
call chain.
Simplify this by only checking the upper limit on max open zones when a
zoned_block_device_info is created.
This way, max_open_zones is handled from a single location, instead of
potentially being reassigned from a completely different location.
Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Felix Abecassis [Fri, 14 May 2021 00:02:40 +0000 (17:02 -0700)]
stat: fix integer overflow in convert_agg_kbytes_percent
Assuming that "int" is 32-bit, for high bandwidth values (> 21.5 GB/s)
the expression "mean * 100" will cause an integer overflow before the
conversion to "double" happens.
Signed-off-by: Felix Abecassis <fabecassis@nvidia.com>
Lars Kellogg-Stedman [Wed, 12 May 2021 22:41:43 +0000 (18:41 -0400)]
fix fio2gnuplot to work with new logging format
The logging format updates documented in
1a953d97 were never
propagated to fio2gnuplot, which since then has been failing with a
ValueError exception.
This commit explicits limits fio2gnuplot to only reading the first
4 columns in the log file.
Closes #928
Jens Axboe [Tue, 11 May 2021 13:58:03 +0000 (07:58 -0600)]
os: define EDQUOT to EIO if the OS doesn't provide it
Fixes:
418f53993b07 ("Make fill_device to stop writing on EDQUOT")
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Martin Bukatovic [Tue, 11 May 2021 07:38:55 +0000 (09:38 +0200)]
Make fill_device to stop writing on EDQUOT
Option fill_device stops writing when we run out of quota as well.
Signed-off-by: Martin Bukatovic <martin.bukatovic@gmail.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Tue, 11 May 2021 03:16:58 +0000 (21:16 -0600)]
Merge branch 'z_unit_docs' of https://github.com/ahribeng/fio
* 'z_unit_docs' of https://github.com/ahribeng/fio:
Add Documentation for z unit
Niklas Cassel [Thu, 6 May 2021 13:18:45 +0000 (13:18 +0000)]
oslib/linux-blkzoned: make sure that we always support zone capacity
A common problem is that users upgrade their kernel to support NVMe ZNS
devices, however, they still use the kernel uapi headers provided by their
distro.
This means that even if the kernel will populate the zone capacity fields
for each zone in the zone report returned by the ioctl, fio will not know
how to interpret that data.
This leads to fio writing past the zone capacity, which will lead to
I/O errors.
It is not trivial for a user to realize that the kernel uapi headers
provided by their distro is the reason for these I/O errors.
In order to make it easier for these users, provide a copy of the current
zoned block device kernel uapi structs.
If the kernel uapi headers installed on the system are too old to support
zone capacity, use the locally defined structs instead.
If the installed headers are new enough to support zone capacity, use the
installed headers.
This way, fio will always be able to handle zone capacity (if the kernel
supports it). At the same time, we will not redefine any structs from the
installed headers if they are newer than our locally defined structs.
Reviewed-by: Damien Le Moal <damien.lemoal@wdc.com>
Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Eric Sandeen [Fri, 7 May 2021 21:13:05 +0000 (16:13 -0500)]
don't access dlclose'd dynamic ioengine object after close
Alexey reported this bug when using dynamically loaded IO engines;
a segfault on the line where we set the dlhandle to NULL after
the dlclose.
I think this is because ops points to the thing we obtained from dlsym:
ops = dlsym(dlhandle, engine_lib);
and after the final dlclose, the object no longer exists and efforts
to set the handle within it will fail for obvious reasons.
I'm not sure why I hadn't seen this before.
Fixes-RH-Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=
1956963
Reported-by: Alexey Dobriyan <adobriyan@gmail.com>
Fixes:
f6931a1 ("fio: move dynamic library handle to io_ops structure")
Tested-by: Alexey Dobriyan <adobriyan@gmail.com>
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Gonzalez [Thu, 6 May 2021 18:15:41 +0000 (11:15 -0700)]
Add Documentation for z unit
Stefan Hajnoczi [Thu, 6 May 2021 16:23:31 +0000 (17:23 +0100)]
configure: fix check_min_lib_version() eval
The following shell statement:
if eval "echo \$$_feature" = "yes" ; then
executes:
echo $... = "yes"
It does not actually compare the variable named by $_feature to the
string "yes".
Add the missing "test" call so the comparison happens as intended and
wrap the eval so it doesn't include the = "yes".
Fixes:
3e48f7c9de61 ("configure: fix syntax error with NetBSD")
Cc: Dmitry Fomichev <dmitry.fomichev@wdc.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Taras Glek [Wed, 5 May 2021 16:11:06 +0000 (09:11 -0700)]
clean up nfs example
Taras Glek [Wed, 5 May 2021 16:04:56 +0000 (09:04 -0700)]
skip skeleton comments
Taras Glek [Wed, 5 May 2021 16:02:20 +0000 (09:02 -0700)]
single line bodies
Taras Glek [Wed, 5 May 2021 16:00:13 +0000 (09:00 -0700)]
C-style comments
Niklas Cassel [Tue, 27 Apr 2021 17:41:14 +0000 (17:41 +0000)]
ioengines: don't call zbd_put_io_u() for engines not implementing commit
Commit
d9ed3e63e528 ("zbd: Fix zone locking for async I/O engines") added
a call to zbd_put_io_u() in the case where td->io_ops->commit callback
is not implemented on an ioengine.
The commit in question fails to mention why this zbd_put_io_u() call was
added for ioengines not implementing the commit callback.
The code in td_io_queue() looks like this:
ret = td->io_ops->queue(td, io_u);
zbd_queue_io_u(td, io_u, ret);
if (!td->io_ops->commit) {
io_u_mark_submit(td, 1);
io_u_mark_complete(td, 1);
zbd_put_io_u(td, io_u);
}
SYNC I/O engine case (e.g. psync):
The zone will be locked by zbd_adjust_block(), td->io_ops->queue(td, io_u),
which for a sync I/O engine will return FIO_Q_COMPLETED.
This return value will be send in to zbd_queue_io_u(), which at the end
of the function, unlocks the zone if the return value from ->queue()
differs from FIO_Q_QUEUED. For a sync I/O engine, the zone will be
unlocked here, and io_u->zbd_put_io function pointer will be set to NULL.
psync does not implement the ->commit() callback, so it will call
zbd_put_io_u(), which will do nothing, because the io_u->zbd_put_io
pointer is NULL.
ASYNC I/O engine case (e.g. io_uring):
The zone will be locked by zbd_adjust_block(), td->io_ops->queue(td, io_u),
which for an async I/O engine will return FIO_Q_QUEUED.
This return value will be send in to zbd_queue_io_u(), which at the end
of the function, unlocks the zone if the return value from ->queue()
differs from FIO_Q_QUEUED. For an async I/O engine, the zone will not be
unlocked here, so the io_u->zbd_put_io function pointer will still be set.
io_uring does implement the ->commit() callback, so it will not call
zbd_put_io_u() here at all.
Instead zbd_put_io_u() will be called by do_io() -> wait_for_completions()
-> io_u_queued_complete() -> ios_completed() -> put_io_u() -> zbd_put_io_u(),
which will unlock the zone and will set the io_u->zbd_put_io function pointer
to NULL.
In conclusion, the zbd_put_io_u() should never had been added in the case
where the ->commit() callback wasn't implemented in the first place,
and removing it shouldn't affect ioengines psync or io_uring.
Commit
d9ed3e63e528 ("zbd: Fix zone locking for async I/O engines")
probably made the assumption that an async I/O engine == the ->commit()
callback is implemented, however, this is not true, there are async
I/O engines in tree (and out of tree), that does not implement the
->commit() callback. Instead, an async I/O engine is recognized by
the ->queue() callback returning FIO_Q_QUEUED.
Removing the invalid zbd_put_io_u() call will ensure that a zone is not
prematurely unlocked for async I/O engines that do not implement the
->commit() callback. Unlocking a zone prematurely leads to I/O errors.
Fixes:
d9ed3e63e528 ("zbd: Fix zone locking for async I/O engines")
Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Rebecca Cran [Thu, 22 Apr 2021 02:32:25 +0000 (20:32 -0600)]
The GPL isn't a EULA: remove it and introduce WixUI_Minimal_NoEULA
The GPL shouldn't be used as a EULA in an installer.
Remove it, and since the WixUI_Minimal dialog set requires a EULA
create a custom WixUI_Minimal_NoEULA set.
Signed-off-by: Rebecca Cran <rebecca@bsdio.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Sun, 25 Apr 2021 16:23:34 +0000 (10:23 -0600)]
Merge branch 'pthread_getaffinity_1' of https://github.com/kusumi/fio
* 'pthread_getaffinity_1' of https://github.com/kusumi/fio:
gettime: Fix compilation on non-Linux with pthread_getaffinity_np()
Tomohiro Kusumi [Sun, 25 Apr 2021 15:10:40 +0000 (00:10 +0900)]
gettime: Fix compilation on non-Linux with pthread_getaffinity_np()
874d55e50c("os/os-linux: add pthread CPU affinity helper") and a few
commits after that broke compilation on non-Linux platforms which support
pthread_getaffinity_np().
Define fio_get_thread_affinity() on non-Linux platforms, and make gettime
test FIO_HAVE_GET_THREAD_AFFINITY which may or may not depend on pthread.
FIO_HAVE_GET_THREAD_AFFINITY is currently not defined on Windows.
Signed-off-by: Tomohiro Kusumi <kusumi.tomohiro@gmail.com>
Jens Axboe [Fri, 23 Apr 2021 14:39:21 +0000 (08:39 -0600)]
Merge branch 'gpspm-add-optional-use-rpma_conn_completion_wait-function' of https://github.com/ldorau/fio
* 'gpspm-add-optional-use-rpma_conn_completion_wait-function' of https://github.com/ldorau/fio:
rpma: gpspm: introduce the busy_wait_polling toggle
Oksana Salyk [Fri, 23 Apr 2021 06:09:44 +0000 (08:09 +0200)]
rpma: gpspm: introduce the busy_wait_polling toggle
The performance of the librpma_gpspm engine depends heavily
on how much CPU power it can use to its work.
One can want either to take all available CPU power
and see what the maximum possible performance is
or configure it less aggressively and collect the results
when the CPU is not solely dedicated to doing this one task.
The librpma_gpspm engine allows toggling between one and another
by either waiting for incoming requests in the kernel
using rpma_conn_completion_wait() (busy_wait_polling=0)
or trying to collect the completion as soon as it appears
by polling all the time using rpma_conn_completion_get()
(busy_wait_polling=1).
Signed-off-by: Oksana Salyk <oksana.salyk@intel.com>
Jens Axboe [Thu, 22 Apr 2021 17:18:23 +0000 (11:18 -0600)]
Merge branch 'zbd-no-parallel-init' of https://github.com/floatious/fio
* 'zbd-no-parallel-init' of https://github.com/floatious/fio:
init: zonemode=zbd does not work with create_serialize=0
Niklas Cassel [Thu, 22 Apr 2021 09:17:58 +0000 (11:17 +0200)]
init: zonemode=zbd does not work with create_serialize=0
zbd_init_zone_info() has a comment that it only works correctly if it
called before the first fio fork() call.
However, right now, there is nothing that ensures this.
If the user specifies --create_serialize=0 and --numjobs=2, each thread
will get their own version of zbd_info.
zbd_info contains one mutex per zone, so if the threads get different
zbd_info, two threads can manage to lock the same zone at the same time,
which will lead to I/O errors.
Explicitly disallow --zonemode=zbd together with --create_serialize=0,
so that we know that all threads will use the same zbd_info, instead of
silently misbehaving.
Analysis:
setup_files() calls zbd_init_files() which calls zbd_init_zone_info().
zbd_init_zone_info() does a for_each_td(), where it checks if zbd_info
(for the same filename) has already been allocated by another thread.
This only works if create_serialize=1 (default).
If create_serialize=0, zbd_init_zone_info() will get called in parallel,
and in this case when the second thread checks if any other thread has
allocated zbd_info, the check will fail, since the first thread has not
yet been running long enough to allocate zbd_info.
Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Taras Glek [Tue, 20 Apr 2021 18:02:18 +0000 (11:02 -0700)]
NFS configure fixes
Damien Le Moal [Thu, 15 Apr 2021 02:16:54 +0000 (11:16 +0900)]
backend: fix switch_ioscheduler()
The backend.c function switch_ioscheduler() suffers from several
problems:
1) This function only considers the first file of a job. For jobs using
multiple files, the ioscheduler switch will done only for that file.
2) If the job file is a character device, a pipe or a regular file for
which the hosting block device file cannot be determined (e.g. a
remote file), thring to switch the IO scheduler causes a crash as the
file disk_util field is NULL.
Fix both problems by introducing the helper function set_ioscheduler()
and changing switch_ioscheduler() to repeatdly call this helper for all
files of the job, ignoring character device files, pipe files and files
without a hosting device information.
Also update the man page to better explain when the ioscheduler option
applies.
Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Friendy.Su@sony.com [Wed, 14 Apr 2021 03:57:09 +0000 (03:57 +0000)]
engines: add engine for file delete
This engine is to measure the performance of deleting files.
In practice, it is an important benchmark for a file system that
how quick it can release the disk space of deleted files.
Signed-off-by: friendy-su <friendy.su@sony.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Tue, 13 Apr 2021 13:51:17 +0000 (07:51 -0600)]
Merge branch 'parse-signedness-warn' of https://github.com/floatious/fio
* 'parse-signedness-warn' of https://github.com/floatious/fio:
parse: fix parse_is_percent() warning
Niklas Cassel [Tue, 13 Apr 2021 10:03:31 +0000 (12:03 +0200)]
parse: fix parse_is_percent() warning
When compiling an out of tree ioengine, such as the SPDK fio plugin,
which has -Wextra in CFLAGS, the compiler gives the following warning:
parse.h: In function ‘parse_is_percent’:
parse.h:134:13: warning: comparison of integer expressions of different signedness: ‘long long unsigned int’ and ‘int’ [-Wsign-compare]
Since this warning was introduced recently by fio
commit
b75c0fae6612 ("parse: simplify parse_is_percent()"),
and since this is the only warning seen when compiling the SPDK fio
plugin, readd the ULL prefix to parse_is_percent().
Fixes:
b75c0fae6612 ("parse: simplify parse_is_percent()")
Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Łukasz Stolarczuk [Thu, 14 Jan 2021 17:19:43 +0000 (18:19 +0100)]
engines/libpmem: do not call drain on close
no matter if direct was 1 or 0, it's not necessary.
It's either covered by non-temporal stores or it's not desired
by user (if 0 was set).
Signed-off-by: Łukasz Stolarczuk <lukasz.stolarczuk@intel.com>
Łukasz Stolarczuk [Wed, 13 Jan 2021 16:43:03 +0000 (17:43 +0100)]
engines/libpmem: cleanup a little code, comments and example
Signed-off-by: Łukasz Stolarczuk <lukasz.stolarczuk@intel.com>
Łukasz Stolarczuk [Mon, 11 Jan 2021 12:41:54 +0000 (13:41 +0100)]
engines/libpmem: set file open/create mode always to RW
previously, when created file with a 'write' job it couldn't be open
later on, when a 'read' job was ran.
Signed-off-by: Łukasz Stolarczuk <lukasz.stolarczuk@intel.com>
Taras Glek [Wed, 26 Feb 2020 17:39:52 +0000 (09:39 -0800)]
NFS engine
Shin'ichiro Kawasaki [Mon, 12 Apr 2021 07:02:25 +0000 (16:02 +0900)]
t/zbd: test repeated async write with block size unaligned to zone size
A recently fixed bug was caused by zone reset during asynchronous IOs
in-flight. The bug symptom was unaligned command error which was
observed using random write workload with libaio engine and block size
not a divisor of zone size. To confirm the bug fix and to prevent future
regression, add a test case which runs the workload.
Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Reviewed-by: Damien Le Moal <damien.lemoal@wdc.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Shin'ichiro Kawasaki [Mon, 12 Apr 2021 07:02:24 +0000 (16:02 +0900)]
zbd: avoid zone reset during asynchronous IOs in-flight
When fio repeats same workload on zoned block devices, zbd_file_reset()
is called for each repetition. This function resets target zones when
one of two conditions are met: 1) the write pointer of the zone has
offset from the device start unaligned to block size, or 2) the workload
is verify and verification is not in process. When the workload runs
with block size not a divisor of the zone size, the offsets of write
pointers from device start (not from zone start) become unaligned to
block size, then zbd_file_reset() resets target zones. This zone reset
happens even when the asynchronous IOs are in-flight and causes
unexpected IO results. Especially if write requests are in-flight, they
fail with unaligned write command error. A single thread may do both the
zone reset and the write request submit, recursive zone locks can not
prevent the zone reset during the writes.
The write pointer check for block size alignment is not working as
intended. It should have checked offset not from device start but from
zone start. Having said that, regardless of this write pointer check
correctness, the zone reset is not required since the zones are reset in
zbd_adjust_block() anyway when remainder of the zone between write
pointer and zone end is smaller than block size.
To avoid the zone reset during asynchronous IOs, do not reset zones in
zbd_file_reset() when the write pointer offset from the device start is
unaligned to block size. Modify zbd_file_reset() to call the helper
function zbd_reset_zones() only when the workload is verify and
verification is not in process. The function zbd_reset_zones() had an
argument 'all_zones' to inform that the zones should be reset when its
write pointer is unaligned to block size. This argument is not required.
Remove it and simplify the function accordingly.
The zone reset for verify workloads is still required. It does not
conflict with asynchronous IOs, since fio waits for IO completion at
verification end, then IOs are not in-flight when zbd_file_reset() is
called for repetition after verification.
Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Reviewed-by: Damien Le Moal <damien.lemoal@wdc.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Sat, 10 Apr 2021 17:46:30 +0000 (11:46 -0600)]
Merge branch 'wip-rados-dont-zerowrite' of https://github.com/aclamk/fio
* 'wip-rados-dont-zerowrite' of https://github.com/aclamk/fio:
engine/rados: Add option to skip object creation
Jens Axboe [Wed, 31 Mar 2021 02:13:16 +0000 (20:13 -0600)]
gettime: cleanup ifdef mess
Let's just set mask to all CPUs for the non-affinity case, and we
can use the same mask test case throughout.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Tue, 30 Mar 2021 23:38:53 +0000 (17:38 -0600)]
gettime: check affinity for thread, if we have it
If we have fio_get_thread_affinity(), we can support a smaller
(and sparse) CPU mask for the clock test.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Tue, 30 Mar 2021 23:38:33 +0000 (17:38 -0600)]
os/os-linux: add pthread CPU affinity helper
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Tue, 30 Mar 2021 23:38:02 +0000 (17:38 -0600)]
configure: add test case for pthread_getaffinity_np()
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 18 Mar 2021 16:19:57 +0000 (10:19 -0600)]
Merge branch 'unified-merge' of https://github.com/jeffreyalien/fio
* 'unified-merge' of https://github.com/jeffreyalien/fio:
Add functionality to the unified_rw_reporting parameter to output separate and mixed stats when set to 'both' or 2.
Brandon Paupore [Wed, 9 Dec 2020 16:47:40 +0000 (10:47 -0600)]
Add functionality to the unified_rw_reporting parameter to output
separate and mixed stats when set to 'both' or 2.
To achieve this, the stats are gathered on a per/data direction
(read/write/etc) basis and then summed into the mixed stats when
generating output.
This functionality also allows setting unified_rw_reporting to
'none' or 'mixed' instead of 0 or 1 respectively.
Signed-off-by: Brandon Paupore <brandon.paupore@wdc.com>
Co-authored-by: Jeff Lien <jeff.lien@wdc.com>
Jens Axboe [Thu, 18 Mar 2021 14:47:05 +0000 (08:47 -0600)]
Merge branch 'add-librpma-engines' of https://github.com/janekmi/fio
* 'add-librpma-engines' of https://github.com/janekmi/fio:
rpma: add librpma_apm_* and librpma_gpspm_* engines
Jens Axboe [Wed, 17 Mar 2021 15:25:46 +0000 (09:25 -0600)]
Merge branch 'free-dump-options' of https://github.com/floatious/fio
* 'free-dump-options' of https://github.com/floatious/fio:
options: free dump options list on exit
Jens Axboe [Wed, 17 Mar 2021 15:25:05 +0000 (09:25 -0600)]
Merge branch 'patch-1' of https://github.com/ihsinme/fio
* 'patch-1' of https://github.com/ihsinme/fio:
fix loop with unreachable exit condition
Jens Axboe [Wed, 17 Mar 2021 15:07:55 +0000 (09:07 -0600)]
Merge branch 'dfs_engine' of https://github.com/johannlombardi/fio
* 'dfs_engine' of https://github.com/johannlombardi/fio:
engines/dfs: add DAOS File System (dfs) engine
Disable pthread_condattr_setclock on cygwin
Johann Lombardi [Sun, 14 Feb 2021 14:23:47 +0000 (15:23 +0100)]
engines/dfs: add DAOS File System (dfs) engine
DAOS is a new scale-out open-source object store.
See https://github.com/daos-stack/daos for more information.
This patch adds a new fio engine to support the filesystem layer built
on top of DAOS called DFS (DAOS File System). It supports asynchronous
read/write operations.
Signed-off-by: Johann Lombardi <johann.lombardi@intel.com>
Co-authored-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
Co-authored-by: Vishwanath Venkatesan <vishwanath.venkatesan@intel.com>
Co-authored-by: Ioannis Galanis <ioannis.galanis@intel.com>
Johann Lombardi [Wed, 17 Mar 2021 13:39:54 +0000 (14:39 +0100)]
Disable pthread_condattr_setclock on cygwin
Signed-off-by: Johann Lombardi <johann.lombardi@intel.com>
Jan Michalski [Thu, 11 Feb 2021 11:47:06 +0000 (12:47 +0100)]
rpma: add librpma_apm_* and librpma_gpspm_* engines
The Remote Persistent Memory Access (RPMA) Library is a C library
created to simplify accessing persistent memory on remote hosts over
Remote Direct Memory Access (RDMA).
The librpma_apm_client and librpma_apm_server is a pair of engines
which allows benchmarking persistent writes achieved via
the Appliance Persistency Method (APM; natively supported by
the librpma library) and regular reads (a part of the RDMA standard).
The librpma_gpspm_client and librpma_gpspm_server is a pair of
engines which allows benchmarking persistent writes achieved via
the General Purpose Persistency Method (GPSPM; build on top of
the librpma API).
The librpma library is available here: https://github.com/pmem/rpma
along with the set of scripts using the newly introduced engines
to construct miscellaneous benchmarking scenarios:
https://github.com/pmem/rpma/tree/master/tools/perf
The full history of the development of the librpma fio engines
is available at: https://github.com/pmem/fio/tree/rpma
Co-Authored-By: Lukasz Dorau <lukasz.dorau@intel.com>
Co-Authored-By: Tomasz Gromadzki <tomasz.gromadzki@intel.com>
Co-Authored-By: Jan Michalski <jan.m.michalski@intel.com>
Co-Authored-By: Oksana Salyk <oksana.salyk@intel.com>
Jens Axboe [Thu, 11 Mar 2021 18:50:49 +0000 (11:50 -0700)]
Merge branch 'dev_luye_github' of https://github.com/louisluSCU/fio
* 'dev_luye_github' of https://github.com/louisluSCU/fio:
Fix reading multiple blktrace replay files
luye [Thu, 11 Mar 2021 03:30:51 +0000 (11:30 +0800)]
Fix reading multiple blktrace replay files
During fio replay, the input file would be checked first by is_blktrace()
if it is a blktrace output file. If so, however, this would treat the
read_iolog option as one single file and ignore multiple files seperated by ":"
This commit contains the fixation that read input file name by index
before doing the check
Signed-off-by: luye <luye.yelu@bytedance.com>
Jens Axboe [Wed, 10 Mar 2021 20:30:23 +0000 (13:30 -0700)]
Merge branch 'fallock-blkdev' of https://github.com/dmonakhov/fio
* 'fallock-blkdev' of https://github.com/dmonakhov/fio:
engines/falloc: add blockdevice as a target
Dmitry Monakhov [Wed, 10 Mar 2021 16:33:35 +0000 (19:33 +0300)]
engines/falloc: add blockdevice as a target
fallocate supoport for blockdevices was added five years ago.
It is time to stress-test it via fio.
Jens Axboe [Wed, 10 Mar 2021 15:11:54 +0000 (08:11 -0700)]
Merge branch 'master' of https://github.com/venkatrag1/fio
* 'master' of https://github.com/venkatrag1/fio:
options: allow separate values for max_latency
Venkat Ramesh [Mon, 8 Feb 2021 04:57:40 +0000 (20:57 -0800)]
options: allow separate values for max_latency
Adding the ability to specify max_latency limits separately for read, write, trim.
Uses comma-separated values similar to blocksize, rate parameters etc.
Also modifies the behavior for the one option only case for max_latency and latency_target
by excluding syncs. This makes the latency limits similar to the rate checks, in that they
only apply to reads, writes and trims.
Extends the output printed on latency exceeded event, to display io_unit information using format
similar to that iolog write.
Signed-off-by: Venkat Ramesh <venkatraghavan@fb.com>
Jens Axboe [Tue, 9 Mar 2021 14:58:39 +0000 (07:58 -0700)]
Merge branch 'master' of https://github.com/DevriesL/fio
* 'master' of https://github.com/DevriesL/fio:
engines/io_uring: fix compilation conflict with Android NDK
DevriesL [Tue, 9 Mar 2021 14:44:08 +0000 (22:44 +0800)]
engines/io_uring: fix compilation conflict with Android NDK
Android NDK already uses __errno in errno.h, rename variable __errno
to fix this conflict.
Signed-off-by: DevriesL <therkduan@gmail.com>
Jens Axboe [Tue, 9 Mar 2021 00:44:38 +0000 (17:44 -0700)]
Fio 3.26
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Mon, 8 Mar 2021 22:13:06 +0000 (15:13 -0700)]
t/io_uring: SQPOLL fixes
Use the proper primitives to load/store the ring head/tail. And as
importantly, account any submit as a full submit, even if we don't
need to call io_uring_enter().
This fixes hangs with using SQPOLL.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Sat, 6 Mar 2021 22:37:56 +0000 (15:37 -0700)]
Merge branch 'clock_monotonic_unused' of https://github.com/foxeng/fio
* 'clock_monotonic_unused' of https://github.com/foxeng/fio:
configure: remove unused CLOCK_MONOTONIC_* symbols
Fotis Xenakis [Sat, 6 Mar 2021 20:47:42 +0000 (22:47 +0200)]
configure: remove unused CLOCK_MONOTONIC_* symbols
The symbols are not used in the code and the respective probes were
removed in commits
187f39063e1b0a7baeda20a9f4f2406327ec0d41 and
69212fc41c0420f8caf272a0cc270194edbddfe7.
Signed-off-by: Fotis Xenakis <foxen@windowslive.com>
Friendy.Su@sony.com [Wed, 3 Mar 2021 06:27:27 +0000 (06:27 +0000)]
engines/filecreate: remove improper message print
'fallocate' should not be printed in filecreate engine
Signed-off-by: friendy-su <friendy.su@sony.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
ihsinme [Wed, 24 Feb 2021 21:37:14 +0000 (00:37 +0300)]
fix loop with unreachable exit condition
the essence of the error lies in a possible hit in the eternal cycle while, when exiting the higher cycle by means of break.
Signed-off-by: Sergey Valentey <ihsinme@gmail.com>
Alexey Dobriyan [Mon, 22 Feb 2021 13:27:36 +0000 (16:27 +0300)]
zbd: support 'z' suffix for zone granularity
Allow users to pass some options with zone granularity which is natural
for ZBD workloads.
This is nifty for writing quick tests and when firmware guys change
zone sizes.
Converted options are
io_size=
offset=
offset_increment=
size=
zoneskip=
Example:
rw=write
numjobs=2
offset=1z
offset_increment=10z
size=5z
io_size=6z
Thread 1 will write zones 1, 2, 3, 4, 5, 1.
Thread 2 will write zones 11, 12, 13, 14, 15, 11.
Note:
zonemode=strided doesn't create ZBD zone structure but requires
value recalculation. This is why 2 functions are split.
Signed-off-by: Alexey Dobriyan (SK hynix) <adobriyan@gmail.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Alexey Dobriyan [Mon, 22 Feb 2021 13:24:01 +0000 (16:24 +0300)]
zbd: fix check against 32-bit zone size
Zone size can be bigger than 4GB.
Signed-off-by: Alexey Dobriyan (SK hynix) <adobriyan@gmail.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Alexey Dobriyan [Mon, 22 Feb 2021 13:21:22 +0000 (16:21 +0300)]
zbd: simplify zoneskip= validness check
Simply check the remainder:
(zoneskip % zone_size) > 0
It will do the right thing for all zoneskip= values, and
zone size being positive is checked earlier.
Reviewed-by: Dmitry Fomichev <dmitry.fomichev@wdc.com>
Signed-off-by: Alexey Dobriyan (SK hynix) <adobriyan@gmail.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Alexey Dobriyan [Mon, 22 Feb 2021 13:20:03 +0000 (16:20 +0300)]
parse: simplify parse_is_percent()
Check "unsigned long long val <= -1ULL" is tautologically true
because of how value conversions work.
Signed-off-by: Alexey Dobriyan (SK hynix) <adobriyan@gmail.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Hongwei Qin [Sun, 21 Feb 2021 09:39:44 +0000 (17:39 +0800)]
Add a new file to gitignore
Add t/fuzz/fuzz_parseini to the ignore list.
Signed-off-by: Hongwei Qin <glqinhongwei@gmail.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Niklas Cassel [Thu, 4 Feb 2021 14:46:05 +0000 (14:46 +0000)]
options: free dump options list on exit
Fix the following LeakSanitizer warnings:
Indirect leak of 224 byte(s) in 7 object(s) allocated from:
#0 0x7f7377b21bc8 in malloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10dbc8)
#1 0x563951e5e09d in add_to_dump_list /home/nks/src/fio/parse.c:1135
#2 0x563951e5e09d in add_to_dump_list /home/nks/src/fio/parse.c:1127
#3 0x563951e5e09d in parse_cmd_option /home/nks/src/fio/parse.c:1162
Indirect leak of 43 byte(s) in 7 object(s) allocated from:
#0 0x7f7377aaa3dd in strdup (/lib/x86_64-linux-gnu/libasan.so.5+0x963dd)
#1 0x563951e5e0a8 in add_to_dump_list /home/nks/src/fio/parse.c:1136
#2 0x563951e5e0a8 in add_to_dump_list /home/nks/src/fio/parse.c:1127
#3 0x563951e5e0a8 in parse_cmd_option /home/nks/src/fio/parse.c:1162
Indirect leak of 36 byte(s) in 7 object(s) allocated from:
#0 0x7f7377aaa3dd in strdup (/lib/x86_64-linux-gnu/libasan.so.5+0x963dd)
#1 0x563951e5e0b9 in add_to_dump_list /home/nks/src/fio/parse.c:1138
#2 0x563951e5e0b9 in add_to_dump_list /home/nks/src/fio/parse.c:1127
#3 0x563951e5e0b9 in parse_cmd_option /home/nks/src/fio/parse.c:1162
by moving fio_dump_options_free() to options.h,
so that we can call it during exit.
Reproducer:
LD_PRELOAD=libasan.so.5 fio --name=test --filename=/dev/nullb0 --runtime=2
Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Jens Axboe [Tue, 16 Feb 2021 19:07:14 +0000 (12:07 -0700)]
engines/io_uring: add verbose error for ENOSYS
If we get ENOSYS for setting up the rings, then the kernel is too old
to support io_uring. Mention that explicitly.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Adam Kupczyk [Mon, 8 Feb 2021 11:58:05 +0000 (06:58 -0500)]
engine/rados: Add option to skip object creation
When `touch_objects` is false, connection procedure no longer attempts to create objects.
In such case ceph caches are untouched and do not affect quality of performance measurement.
But if `touch_object` is false, tests that read objects without creating them previously
will now fail.
Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
Jens Axboe [Sun, 14 Feb 2021 20:21:05 +0000 (13:21 -0700)]
Merge branch 'per-engine-pre-write-function' of https://github.com/lukaszstolarczuk/fio
* 'per-engine-pre-write-function' of https://github.com/lukaszstolarczuk/fio:
filesetup: add engine's io_ops to prepopulate file with data
Jens Axboe [Wed, 10 Feb 2021 20:22:04 +0000 (13:22 -0700)]
Merge branch 'taras/clientuid' of https://github.com/tarasglek/fio-1
* 'taras/clientuid' of https://github.com/tarasglek/fio-1:
$clientuid keyword to differentiate clients in client/server mode.
Taras Glek [Wed, 10 Feb 2021 19:08:52 +0000 (11:08 -0800)]
$clientuid keyword to differentiate clients in client/server mode.
Prior to this change getting fio to include IP as part of filename was a struggle. One had to use directory=/ to trigger IP-inclusion code and there was no way to customize that.
Signed-off-by: Taras Glek <taras@glek.net>
Łukasz Stolarczuk [Fri, 22 Jan 2021 12:12:43 +0000 (13:12 +0100)]
filesetup: add engine's io_ops to prepopulate file with data
In some cases (e.g. engine marked as diskless) files are not laid out.
If the first job is a read job, results are higher than expected
(because reading zero page). Each engine should deliver func to
prepopulate file with data to avoid this situation.
Signed-off-by: Łukasz Stolarczuk <lukasz.stolarczuk@intel.com>
Dmitry Fomichev [Sat, 30 Jan 2021 03:35:24 +0000 (12:35 +0900)]
zbd: relocate Coverity annotation
The Coverity annotation added earlier to suppress a false positive
about missing unlock in zbd_adjust_block() didn't work because it
was placed not before the return statement, but earlier in the code.
Move the annotation to the right place to avoid the warning.
Reported-by: Bart Van Assche <bvanassche@acm.org>
Fixes:
8e4b2e55512f("zbd: don't unlock zone mutex after verify replay")
Signed-off-by: Dmitry Fomichev <dmitry.fomichev@wdc.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Fri, 29 Jan 2021 19:20:48 +0000 (12:20 -0700)]
zbd: fix 32-bit compile warnings for logging
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Dmitry Fomichev [Wed, 27 Jan 2021 04:19:45 +0000 (13:19 +0900)]
t/zbd: avoid looping on invalid command line options
t/zbd/test-zbd-support loops indefinitely if an unrecognized option
is specified in the command line. Add a switch case to display usage
and exit the script.
Signed-off-by: Dmitry Fomichev <dmitry.fomichev@wdc.com>
Reviewed-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Dmitry Fomichev [Wed, 27 Jan 2021 04:19:44 +0000 (13:19 +0900)]
t/zbd: increase timeout in test #48
Test #48 runs some i/o to the test device for 30 seconds and then waits
45 seconds for fio to finish. If this wait times out, the test assumes
that fio is hung because of a zone locking issue and fails. It is
observed that 45s may not be enough for some HDDs, especially the ones
running specialized firmware.
Increase the timeout to 180 seconds to avoid any false positives.
There is no change in test duration for the most common devices.
The test will wait for the full 180 seconds only if it fails, otherwise
it will finish very soon after the 30 second i/o period ends.
Signed-off-by: Dmitry Fomichev <dmitry.fomichev@wdc.com>
Reviewed-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Dmitry Fomichev [Wed, 27 Jan 2021 04:19:43 +0000 (13:19 +0900)]
t/zbd: show elapsed time in test-zbd-support
This script may take quite a lot of time to run against large
zoned HDDs. At the end of every run, show exactly how much time
it took.
Signed-off-by: Dmitry Fomichev <dmitry.fomichev@wdc.com>
Reviewed-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Dmitry Fomichev [Wed, 27 Jan 2021 04:19:42 +0000 (13:19 +0900)]
t/zbd: add test #54 to exercise ZBD verification
Add a new test case to perform 75/25 read/write workload with varying
i/o size and verification on. It is very important to use a good random
generator for this test. Setting experimental_verify=1 is required for
this test to operate correctly.
Signed-off-by: Dmitry Fomichev <dmitry.fomichev@wdc.com>
Reviewed-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Dmitry Fomichev [Wed, 27 Jan 2021 04:19:41 +0000 (13:19 +0900)]
t/zbd: add checks for offline zone condition
Some tests, e.g. #39 an #40, try to read the first zone of the drive.
It is assumed that the first zone is readable. However, if the first
zone is offline, the read fails along with the entire test.
This commit adds two functions to perform zone report and find the
first and the last zones that are not offline. Several test cases
now call these functions to avoid test failures described above.
Fixes for two more test failures are included in this commit -
Test #14 tries to write to conventional zones if they are found at
the beginning of the LBA range of the drive, but it assumes that
these zones are online. This may not always be the case. Add "offset"
to avoid the i/o to be attempted to run against any preceding offline
zones.
Similarly, in test #17, the script tries to find the last zone.
Check for the case when the last zone is offline. The test doesn't
set the i/o file size, but it works OK in most of the cases because
typically this test operates on the last physical zone. With the
online lookup in place, this may not always be the case and if there
are any offline zones that trail the last non-offline zone,
then the i/o will try to access that zone and fail. Add the "size"
to avoid the i/o to be attempted to run against any trailing offline
zones.
Signed-off-by: Dmitry Fomichev <dmitry.fomichev@wdc.com>
Reviewed-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Dmitry Fomichev [Wed, 27 Jan 2021 04:19:40 +0000 (13:19 +0900)]
t/zbd: prevent test #31 from looping
The test 31 starts i/o to 128 zones in parallel.
There are two corner cases that are not properly handled in the
existing implementation -
1) If the total number of zones on the device is < 128, the test
will loop indefinitely because the loop increment is calculated as
zero by the script.
2) If the number of max_open_zones of the device is < 128, the
test will fail due to exceeding max_open_zones limit as the code
expects it to be >= 128.
Limit the number of open zones to the reported maximum
and skip the test if there is not enough zones on the device.
Signed-off-by: Dmitry Fomichev <dmitry.fomichev@wdc.com>
Reviewed-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Dmitry Fomichev [Wed, 27 Jan 2021 04:19:39 +0000 (13:19 +0900)]
t/zbd: add an option to bail on a failed test
Sometimes, it can be useful to inspect the state of the zones of the
test device, usually right after a test failure. Currently,
test-zbd-support script just keeps running and proper examination of
device zones can be difficult.
Add the -q option to test/zbd/support to quit immediately upon
encountering any test failure. Additionally, define the same option
in run-tests-against-nullb to propagate it to test/zbd/support.
Signed-off-by: Dmitry Fomichev <dmitry.fomichev@wdc.com>
Reviewed-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Shin'ichiro Kawasaki [Wed, 27 Jan 2021 04:19:38 +0000 (13:19 +0900)]
t/zbd: fix wrong units in test case #37
The second argument of the function total_zone_capacity is expected to
be in bytes. However, the call in test case #37 provides this argument
in sectors and this results in a wrong capacity calculation. Make sure
that the value that is passed to this function is converted to bytes.
Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Signed-off-by: Dmitry Fomichev <dmitry.fomichev@wdc.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Shin'ichiro Kawasaki [Wed, 27 Jan 2021 04:19:37 +0000 (13:19 +0900)]
t/zbd: test random I/O direction in all-conventional case
The number of 'sectors with data' is counted and used to determine the
direction of the first I/O of a random read/write ZBD workload. To
initialize the number, min_zone and max_zone fields in struct fio_file
are referred. There was a code bug that was recently fixed where
min_zone and max_zone fields were not initialized when all zones in I/O
region were conventional zones. This led to an uninitialized number of
sectors with data, and the write direction was always set for random
workloads.
Add a test case to perform random read/write workload on an I/O region
with both sequential and conventional zones. Check that both read and
write I/Os are executed.
Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Signed-off-by: Dmitry Fomichev <dmitry.fomichev@wdc.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>