fio.git
3 years agoFio 2.9 fio-2.9
Jens Axboe [Thu, 28 Apr 2016 22:23:10 +0000 (16:23 -0600)]
Fio 2.9

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agooslib/getopt_long: allow (unique) short match
Jens Axboe [Fri, 22 Apr 2016 22:14:24 +0000 (18:14 -0400)]
oslib/getopt_long: allow (unique) short match

The GNU version allows short matches, if they are unique. Our version
does not, since it's weird, but let's at least make them have the
same behavior. Otherwise users will have different behavior for
command line parsing for edge cases, on different platforms.

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoinclude sys/sysmacros.h for major/minor
Mike Frysinger [Wed, 20 Apr 2016 16:52:19 +0000 (12:52 -0400)]
include sys/sysmacros.h for major/minor

These functions have always been defined in sys/sysmacros.h under
Linux C libraries.  For some, including sys/types.h implicitly
includes that as well, but glibc wants to deprecate that, and some
others already have.  Include the header explicitly for the funcs.

The mtd change is already in upstream mtd-utils too.

Signed-off-by: Mike Frysinger <vapier@gentoo.org>
Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoDocumentation: it's 'log_max_value', not 'log_max'
Jens Axboe [Mon, 18 Apr 2016 20:54:12 +0000 (16:54 -0400)]
Documentation: it's 'log_max_value', not 'log_max'

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agot/fio-verify-state: show completions in order
Jens Axboe [Fri, 15 Apr 2016 15:03:34 +0000 (09:03 -0600)]
t/fio-verify-state: show completions in order

We currently show them in array order, but that's logged in
reverse. So show them from newest to latest.

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agofilesetup: assign fileno at setup time
Jens Axboe [Thu, 14 Apr 2016 14:57:46 +0000 (08:57 -0600)]
filesetup: assign fileno at setup time

For duplicate jobs, we don't always get fileno assigned if we have
multiple files per job. Ensure that we do.

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agot/fio-verify-state: allow multiple input files
Jens Axboe [Thu, 14 Apr 2016 14:21:18 +0000 (08:21 -0600)]
t/fio-verify-state: allow multiple input files

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoclient: bool conversion
Jens Axboe [Wed, 13 Apr 2016 21:53:06 +0000 (15:53 -0600)]
client: bool conversion

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agot/fio-verify-state: pretty up output a bit
Jens Axboe [Wed, 13 Apr 2016 21:13:55 +0000 (15:13 -0600)]
t/fio-verify-state: pretty up output a bit

Also parseable now.

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoCleanup last write logging
Jens Axboe [Wed, 13 Apr 2016 18:47:24 +0000 (12:47 -0600)]
Cleanup last write logging

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoFix verify state for multiple files
Jens Axboe [Wed, 13 Apr 2016 18:25:52 +0000 (12:25 -0600)]
Fix verify state for multiple files

Fio currently only logs the completions on a per-job basis, but
that breaks down if the job uses multiple files and we rely
on the verify state save/load to check when to stop verifying.

This patch extends the verify state format to be version 3, and
removes support for v1/v2. The new version has support for
multiple files.

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoModify RDMA engine to print strerror messages.
Logan Gunthorpe [Mon, 4 Apr 2016 16:06:03 +0000 (10:06 -0600)]
Modify RDMA engine to print strerror messages.

In order to ease debugging it would be nice to know why certain
rdma/ib functions fail. I've added %m messages to a large
number of log_err lines.

3 years agofio: register pvsync2 engine correctly
Jon Derrick [Wed, 30 Mar 2016 22:17:55 +0000 (16:17 -0600)]
fio: register pvsync2 engine correctly

I'm experimenting with the new syscalls and noticed fio's support was
incomplete. This patch fixes some of those issues.

Signed-off-by: Jon Derrick <jonathan.derrick@intel.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoHOWTO/man: clarify that the usr/sys utilization numbers are averages
Jens Axboe [Tue, 29 Mar 2016 14:34:06 +0000 (08:34 -0600)]
HOWTO/man: clarify that the usr/sys utilization numbers are averages

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoFix crash with group_reporting and one group member that never started
Jens Axboe [Tue, 29 Mar 2016 14:28:56 +0000 (08:28 -0600)]
Fix crash with group_reporting and one group member that never started

If we have a job that nevers gets started, then we crash in
setting up the group stats, if group_reporting is enabled. This
can be reproduced like this:

fio --name=iwontstart --name=null --ioengine=null --size=1g --group_reporting=1
Starting 2 processes
iwontstart: you need to specify size=
fio: pid=0, err=22/file:filesetup.c:828, func=total_file_size, error=Invalid argument
Segmentation fault (core dumped)

Check for -1 groupid, and just skip the job if so.

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agot/read-to-pipe-async: use gettimeofday() instead of clock_gettime()
Jens Axboe [Fri, 25 Mar 2016 15:40:41 +0000 (09:40 -0600)]
t/read-to-pipe-async: use gettimeofday() instead of clock_gettime()

We don't have clock_gettime() on all supported platforms, we can
just use gettimeofday() and convert to timespec.

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agot/read-to-pipe-async: synchronization fixes
Jens Axboe [Fri, 25 Mar 2016 14:30:36 +0000 (08:30 -0600)]
t/read-to-pipe-async: synchronization fixes

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agot/memlock: sample utility to use X memory from Y threads
Jens Axboe [Thu, 24 Mar 2016 23:07:05 +0000 (17:07 -0600)]
t/memlock: sample utility to use X memory from Y threads

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoMakefile: disable build of t/read-to-pipe-async by default
Jens Axboe [Thu, 24 Mar 2016 21:58:21 +0000 (15:58 -0600)]
Makefile: disable build of t/read-to-pipe-async by default

It's not part of fio, if you need it, you can just do:

$ make t/read-to-pipe-async

and it should build.

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agot/read-to-pipe-async: needs time.h on some platforms
Jens Axboe [Thu, 24 Mar 2016 21:56:09 +0000 (15:56 -0600)]
t/read-to-pipe-async: needs time.h on some platforms

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agot/read-to-pipe-async: standalone test app
Jens Axboe [Thu, 24 Mar 2016 21:52:19 +0000 (15:52 -0600)]
t/read-to-pipe-async: standalone test app

Reads from a file through reads, spawning more threads if necessary
to keep the latency target. Writes the result to a pipe, and outputs
the latencies at the end.

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agobackend: ensure that we run verification for short time based jobs
Jens Axboe [Thu, 24 Mar 2016 19:36:22 +0000 (13:36 -0600)]
backend: ensure that we run verification for short time based jobs

We need to break out of the main loop for the case where we did
exceed our bytes issued, if we have verification to run. For
time based jobs, we'll get back in this loop after verification.

Fixes c04e4661e4da ("time_based: Avoid restarting main I/O loop")
Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agotravis.yml: ensure we have libaio-dev and numa dev libs
Jens Axboe [Thu, 24 Mar 2016 03:38:59 +0000 (21:38 -0600)]
travis.yml: ensure we have libaio-dev and numa dev libs

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoMakefile: add empty 'test' target
Jens Axboe [Thu, 24 Mar 2016 03:36:09 +0000 (21:36 -0600)]
Makefile: add empty 'test' target

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoAdd .travis.yml
Jens Axboe [Thu, 24 Mar 2016 03:31:48 +0000 (21:31 -0600)]
Add .travis.yml

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agogen-rand: fix dependency on strcasestr.o
Jens Axboe [Sun, 20 Mar 2016 15:38:57 +0000 (09:38 -0600)]
gen-rand: fix dependency on strcasestr.o

Fails on systems where we have to pull that in (eg Windows).

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoinit: seed repeatable jobs differently
Jens Axboe [Fri, 18 Mar 2016 16:07:16 +0000 (09:07 -0700)]
init: seed repeatable jobs differently

Factor in the job number for the stock random seeds, so we don't
generate the same patterns for identical jobs.

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoFio 2.8 fio-2.8
Jens Axboe [Tue, 15 Mar 2016 16:10:37 +0000 (09:10 -0700)]
Fio 2.8

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoverify: clear IO verify state all_io_list before writing
Jens Axboe [Tue, 15 Mar 2016 16:09:10 +0000 (09:09 -0700)]
verify: clear IO verify state all_io_list before writing

We only fill it partially sometimes, so ensure that we clear
the whole thing before writing it out. This fixes valgrind
complaining about writing unitialized memory.

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoRevert "options: move pattern_fmt_desc where we need it"
Jens Axboe [Tue, 15 Mar 2016 15:48:25 +0000 (09:48 -0600)]
Revert "options: move pattern_fmt_desc where we need it"

This reverts commit 5f3bd0fd67254e3750681f0700a0cc162faee500.

We do need it to be globally reachable, otherwise we assign a
pointer to an on-stack structure. This causes the pattern filling
to read garbage, and segfault.

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agolib/rand: make __init_randX() static
Jens Axboe [Thu, 10 Mar 2016 19:12:09 +0000 (12:12 -0700)]
lib/rand: make __init_randX() static

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoUse 32-bit rand for parts that use rand_between()
Jens Axboe [Thu, 10 Mar 2016 19:09:14 +0000 (12:09 -0700)]
Use 32-bit rand for parts that use rand_between()

We don't need a full 32-bit space for these, and the 32-bit generator
is faster.

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agot/gen-rand: use 32-bit random generator
Jens Axboe [Thu, 10 Mar 2016 18:08:19 +0000 (11:08 -0700)]
t/gen-rand: use 32-bit random generator

This was inadvertently changed.

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoFix compile of test programs on archs that use arch_flags at runtime
Jens Axboe [Thu, 10 Mar 2016 15:09:41 +0000 (08:09 -0700)]
Fix compile of test programs on archs that use arch_flags at runtime

SuperH compile currently fails with:

gettime.o: In function fio_gettime':
/home/thomas/projets/buildroot/output/build/fio-fio-2.7/gettime.c:163: undefined reference to arch_flags'
gettime.o: In function utime_since_now':
/home/thomas/projets/buildroot/output/build/fio-fio-2.7/gettime.c:164: undefined reference to arch_flags'
gettime.o: In function mtime_since_now':
/home/thomas/projets/buildroot/output/build/fio-fio-2.7/gettime.c:164: undefined reference to arch_flags'
gettime.o: In function time_since_now':
/home/thomas/projets/buildroot/output/build/fio-fio-2.7/gettime.c:164: undefined reference to arch_flags'
mutex.o: In function fio_mutex_up':
/home/thomas/projets/buildroot/output/build/fio-fio-2.7/mutex.c:189: undefined reference to arch_flags'
collect2: error: ld returned 1 exit status
Makefile:375: recipe for target 't/stest' failed
make[2]: *** [t/stest] Error 1
make[2]: *** Waiting for unfinished jobs....

Fix that by ensuring we have a stub arch.o with the necessary arch flags
for the standalone test programs.

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agot/gen-rand: remove compile warning on 32-bit
Jens Axboe [Wed, 9 Mar 2016 21:15:11 +0000 (14:15 -0700)]
t/gen-rand: remove compile warning on 32-bit

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoFio 2.7 fio-2.7
Jens Axboe [Wed, 9 Mar 2016 16:42:56 +0000 (09:42 -0700)]
Fio 2.7

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years ago.gitignore: ignore vim undo files
Jens Axboe [Tue, 8 Mar 2016 18:19:17 +0000 (11:19 -0700)]
.gitignore: ignore vim undo files

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoAdd t/gen-rand to test random generator
Jens Axboe [Tue, 8 Mar 2016 16:58:13 +0000 (09:58 -0700)]
Add t/gen-rand to test random generator

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoUpdate documentation on log file formats
Jens Axboe [Tue, 8 Mar 2016 15:56:57 +0000 (08:56 -0700)]
Update documentation on log file formats

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agorand: use bools
Jens Axboe [Mon, 7 Mar 2016 22:38:44 +0000 (15:38 -0700)]
rand: use bools

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoAdd the sample JESD219 job file
Jens Axboe [Mon, 7 Mar 2016 21:44:24 +0000 (14:44 -0700)]
Add the sample JESD219 job file

Based on the posting from Jeff.

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoio_u: speed up __get_next_buflen()
Andrey Kuzmin [Mon, 7 Mar 2016 21:40:25 +0000 (14:40 -0700)]
io_u: speed up __get_next_buflen()

Change the divide-by-100 into a multiplier instead.

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agooptions: improvements to parse dry run
Jens Axboe [Mon, 7 Mar 2016 19:19:21 +0000 (12:19 -0700)]
options: improvements to parse dry run

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agooptions: finish merge of bssplit/rand zone code
Jens Axboe [Mon, 7 Mar 2016 19:01:40 +0000 (12:01 -0700)]
options: finish merge of bssplit/rand zone code

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agooptions: unify the bssplit/zone split code
Jens Axboe [Mon, 7 Mar 2016 16:29:13 +0000 (09:29 -0700)]
options: unify the bssplit/zone split code

Most of that parsing code was duplicated, apply a bit of abstraction
to it. Especially because it's not that pretty, so rather have one
copy of that nastiness.

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoFix double free of td zone state index
Jens Axboe [Sat, 5 Mar 2016 02:50:41 +0000 (19:50 -0700)]
Fix double free of td zone state index

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoLog to parent instead of twice to child td
Alan C [Sat, 5 Mar 2016 00:13:14 +0000 (16:13 -0800)]
Log to parent instead of twice to child td

3 years agooptions: clean number of zones if we fail parsing
Jens Axboe [Fri, 4 Mar 2016 22:34:27 +0000 (15:34 -0700)]
options: clean number of zones if we fail parsing

Leaves us in a cleaner state, since the zone generation will never
attempt to touch uninitialized data then.

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoAdd support for zones of random IO, with varying frequency of access
Jens Axboe [Fri, 4 Mar 2016 22:29:32 +0000 (15:29 -0700)]
Add support for zones of random IO, with varying frequency of access

Let's say you want to ensure that 50% of the IO falls in the first 5%
of the flie, with the remaining 50% over the last 95%. Now that's
possible with random_distribution=zoned. For this particular case,
you would do:

random_distribution=zoned:50/5:50/95

Up to 64 ranges can be specified, and multiple data directions can
be given as well. The above would apply to both reads, writes, and
trims. If you wanted to have 50% of the writes fall in the first
10%, 25% in the next 10%, and the last 25% over the remaining 80%,
you could extend the above ala:

random_distribution=zoned:50/5:50/95,50/10:25/10:25/80

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoUpdate documentation for random_distribution=gauss
Jens Axboe [Thu, 3 Mar 2016 21:00:54 +0000 (14:00 -0700)]
Update documentation for random_distribution=gauss

For some reason this was never committed, add it now.

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoAdd support for preadv2/pwritev2
Jens Axboe [Fri, 26 Feb 2016 19:02:54 +0000 (11:02 -0800)]
Add support for preadv2/pwritev2

This is coming on Linux, and supports a high priority flag that will
use polling on the completion side.

Basic support is there, we'll need to add wrappers on x86_64 linux
to use them before it shows up in the glibc devel headers.

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoioengines: account any queued IO on the engine side
Jens Axboe [Thu, 25 Feb 2016 20:24:47 +0000 (12:24 -0800)]
ioengines: account any queued IO on the engine side

Currently we don't account if it's not a read/write/trim, but that
can screw up with SYNC, for instance. Account any IO that the
IO engine has returned a queued status for.

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agobackend: ensure that fio_io_sync() commits IN on queued status
Jens Axboe [Thu, 25 Feb 2016 16:04:06 +0000 (09:04 -0700)]
backend: ensure that fio_io_sync() commits IN on queued status

If we get a FIO_Q_QUEUED back, we need to commit pending IO
before we wait for it. Otherwise we trigger this bug:

fio: ioengines.c:257: td_io_getevents: Assertion `td->io_u_in_flight >= r' failed.

Reported-by: Andrey Kuzmin <andrey.v.kuzmin@gmail.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoAllow IO engine driven allocations of IO structures
Jens Axboe [Wed, 24 Feb 2016 23:29:08 +0000 (16:29 -0700)]
Allow IO engine driven allocations of IO structures

Based on a patch from Andrey Kuzmin, but modified so that we error
out if the user has specified a specific memory backing type
through the use of mem= or iomem=.

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoAllow for the include file specification to be relative.
Andrey Kuzmin [Sat, 13 Feb 2016 19:31:00 +0000 (12:31 -0700)]
Allow for the include file specification to be relative.

Simplifies include file grouping, with same directory file included
via file name. Checked only if the lookup by the file name specified
fails.

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agofio.1: man page fixes
Jens Axboe [Fri, 12 Feb 2016 22:00:39 +0000 (15:00 -0700)]
fio.1: man page fixes

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoUpdate documentation
Jens Axboe [Fri, 12 Feb 2016 17:37:01 +0000 (10:37 -0700)]
Update documentation

Section numbering duplicated in one place in the HOWTO.
Add the missing sections to the man page.

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoserver: don't make SO_REUSEPORT errors fatal
Jens Axboe [Wed, 10 Feb 2016 15:32:13 +0000 (08:32 -0700)]
server: don't make SO_REUSEPORT errors fatal

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agodiskutil: don't print terse disk util twice for json,terse output format
Jens Axboe [Tue, 9 Feb 2016 20:45:50 +0000 (13:45 -0700)]
diskutil: don't print terse disk util twice for json,terse output format

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agodiskutil: fix segfault for both json and terse output
Jens Axboe [Tue, 9 Feb 2016 20:19:34 +0000 (13:19 -0700)]
diskutil: fix segfault for both json and terse output

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoio_ddir: io_ddir_name array should be static
Jens Axboe [Fri, 5 Feb 2016 15:42:26 +0000 (08:42 -0700)]
io_ddir: io_ddir_name array should be static

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agofio: fix copy/paste error in io_ddir.h
Andrey Kuzmin [Fri, 5 Feb 2016 15:14:52 +0000 (08:14 -0700)]
fio: fix copy/paste error in io_ddir.h

The last 'write' should be 'wait', to match with DDIR_WAIT.

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agolog: add support for logging max instead of averages
Jens Axboe [Thu, 4 Feb 2016 19:09:48 +0000 (12:09 -0700)]
log: add support for logging max instead of averages

If we log with lov_avg_msec, fio will log the average over that
period of time to the log. Add log_max_value to change this to
logging maximums over the windowed period instead.

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agostat: cleanup average logging
Jens Axboe [Thu, 4 Feb 2016 04:10:33 +0000 (21:10 -0700)]
stat: cleanup average logging

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agofio: fix ignore_error regression
Eric Whitney [Wed, 3 Feb 2016 23:58:43 +0000 (16:58 -0700)]
fio: fix ignore_error regression

Commit e9d512d8e7d2 introduced a regression affecting handling for
the ignore_error parameter.  The common code for io_u event handling
pulled from do_io() and do_verify() into io_queue_event() does not
check for and process errors that are to be ignored in the main I/O
path as I/Os are completed.  Prior to that commit, do_io() called
break_on_this_error() to handle that case.

The change causes fio to return a non-zero exit status after
encountering an error it should ignore while performing I/O, even
though the job does run to completion. This regression causes
failures for some fstests that use fio, including ext4/302, ext4/303,
and generic/300.

This patch restores the original error handling behavior.

Signed-off-by: Eric Whitney <enwlinux@gmail.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agolibmtd: compile fix for linux header changes.
Robin H. Johnson [Wed, 3 Feb 2016 16:32:19 +0000 (08:32 -0800)]
libmtd: compile fix for linux header changes.

In the Linux kernel source, mtd/mtd-user.h had a change
'mtd: mtd-user: remove stdint.h include'
(137d36af4a53858b8db7ca83c8480247118b8bdf)

This causes the uint8_t/uint64_t types to be undefined in libmtd.h
now, as they were implicitly coming from it before.

Import stdint.h explicitly into libmtd.h to resolve this.

X-URL: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=137d36af4a53858b8db7ca83c8480247118b8bdf
Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>
3 years agoMakefile: false positive on _FORTIFY_SOURCE
Robin H. Johnson [Wed, 3 Feb 2016 16:19:52 +0000 (08:19 -0800)]
Makefile: false positive on _FORTIFY_SOURCE

If the compiler is very pedantic and includes _FORTIFY_SOURCE, then you
get the following warning about redefinition:
<command-line>:0:0: warning: "_FORTIFY_SOURCE" redefined

To prevent it, undefine and redefine _FORTIFY_SOURCE.

Gentoo's gcc does this for -O2.

Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>
3 years agoAdd a note of what is the default value for random_generator
Jens Axboe [Sun, 31 Jan 2016 05:03:39 +0000 (22:03 -0700)]
Add a note of what is the default value for random_generator

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoserver: send trigger state inline
Jens Axboe [Thu, 28 Jan 2016 20:41:18 +0000 (13:41 -0700)]
server: send trigger state inline

If we don't send it inline, we can't be sure that it's transmitted
if the trigger event is a hard reboot or similar.

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoclient: fix double removal of client on job file open failure
Jens Axboe [Thu, 28 Jan 2016 19:47:11 +0000 (12:47 -0700)]
client: fix double removal of client on job file open failure

If fio is invoked as a client and the job file does not exist, it
will exit ala:

fio: job file <x.fio> open: No such file or directory
fio: client.c:240: remove_client: Assertion `client->refs' failed.

because the break path ends up doing the removal twice. Fix that
to only remove it once, so we don't trigger the assert.

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoFio 2.6 fio-2.6
Jens Axboe [Wed, 27 Jan 2016 02:02:07 +0000 (19:02 -0700)]
Fio 2.6

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoserver: split vectored commands into fragments
Jens Axboe [Tue, 26 Jan 2016 23:44:24 +0000 (16:44 -0700)]
server: split vectored commands into fragments

This fixes the fio_send_cmd_ext_pdu() path to split commands
into properly sized fragments. This also reverts the
previous commit, that bypassed the check since this path
was broken.

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoserver: disable pdu length check for now
Jens Axboe [Tue, 26 Jan 2016 18:19:16 +0000 (11:19 -0700)]
server: disable pdu length check for now

Vectored entries don't always honor it. Allow longer payloads, until
we fix the breakdown again.

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoclient: remove debug printf() for compressed iolog
Jens Axboe [Tue, 26 Jan 2016 18:02:16 +0000 (11:02 -0700)]
client: remove debug printf() for compressed iolog

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agopowerpc: fix cpus_online() to get correct max CPU number for powerpc64
Dan Qing Zhang [Mon, 25 Jan 2016 13:04:18 +0000 (13:04 +0000)]
powerpc: fix cpus_online() to get correct max CPU number for powerpc64

This patch tries to fix cpus_online() for powerpc64 when running fio
by using parameter "cpus_allowed", in order to obtain correct max CPU
number on powerpc64 architecture.

When changing SMT settings on powerpc architecture, the way of cpu
numbering is changing as well. For example, when SMT=off, CPUs are
numbered like 0,8,16,..., while when SMT=2, CPUs are numbered like
0,1,8,9,.... Therefore, sysconf(_SC_NPROCESSORS_ONLN) is not applied
to get exact max CPU number on powerpc. And the max_cpu value can be
correctly ontained by using sysconf(_SC_NPROCESSORS_CONF).

Signed-off-by: Dan Qing Zhang <dqzhang@linux.vnet.ibm.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoFix Windows build: define rand_r()
Bruce Cran [Fri, 22 Jan 2016 17:54:47 +0000 (10:54 -0700)]
Fix Windows build: define rand_r()

3 years agoMakefile: fix duplicate strcasestr for windows
Jens Axboe [Fri, 22 Jan 2016 22:10:31 +0000 (15:10 -0700)]
Makefile: fix duplicate strcasestr for windows

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoserver: use the wrapped MSG_DONTWAIT
Jens Axboe [Fri, 22 Jan 2016 18:49:57 +0000 (11:49 -0700)]
server: use the wrapped MSG_DONTWAIT

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoioengine: remove duplicate queue handler check
David Disseldorp [Thu, 21 Jan 2016 16:45:44 +0000 (17:45 +0100)]
ioengine: remove duplicate queue handler check

Signed-off-by: David Disseldorp <ddiss@suse.de>
3 years agoFio 2.5 fio-2.5
Jens Axboe [Thu, 21 Jan 2016 05:00:17 +0000 (22:00 -0700)]
Fio 2.5

2.4 was short lived, 2.5 fixes an important regression that
happened between fio-2.2.12 and fio-2.2.13.

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agomutex: fix double unlock in fio_mutex_down_timeout()
Jens Axboe [Thu, 21 Jan 2016 04:52:47 +0000 (21:52 -0700)]
mutex: fix double unlock in fio_mutex_down_timeout()

In case of error, we unlock twice.

Fixes: 09400a60042e ("mutex: fixup fio_mutex_down_timeout()")
Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoinit: fix compile for FIO_INC_DEBUG not being set
Jens Axboe [Thu, 21 Jan 2016 04:17:55 +0000 (21:17 -0700)]
init: fix compile for FIO_INC_DEBUG not being set

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoFio 2.4 fio-2.4
Jens Axboe [Thu, 21 Jan 2016 04:13:08 +0000 (21:13 -0700)]
Fio 2.4

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoinit: increment stat count if parent == default thread
Jens Axboe [Thu, 21 Jan 2016 00:45:13 +0000 (17:45 -0700)]
init: increment stat count if parent == default thread

We skip doing the stat increment if group_reporting is set,
but that means we can end up with stat_number == 0. That
isn't correct, increment for group_reporting as well, but
only if the parent is the default thread. That means we
still won't do it for sub-jobs.

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoclient: fix non-text command leak
Jens Axboe [Wed, 20 Jan 2016 19:17:54 +0000 (12:17 -0700)]
client: fix non-text command leak

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoclient: drain text output when exiting
Jens Axboe [Wed, 20 Jan 2016 18:22:22 +0000 (11:22 -0700)]
client: drain text output when exiting

If we don't, we can lose error messages from the server side. One
example would be for protocol version mismatch - the server will
notify the client, but the client doesn't check for these messages
before exiting.

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoFix issue with termination before io_size has been reached
Jens Axboe [Thu, 14 Jan 2016 21:44:23 +0000 (14:44 -0700)]
Fix issue with termination before io_size has been reached

If the size of the device isn't a multiple of the block size,
then we can exit prematurely for cases where an io_limit has
been set higher than the device size. This is because the file(s)
get marked completely done if bs is a multiple of the device size,
but doesn't for the opposite case.

We should not factor this in for deciding to keep running or not.

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoio_u: ensure that we align new start offset properly for time_based
Jens Axboe [Thu, 14 Jan 2016 17:31:38 +0000 (10:31 -0700)]
io_u: ensure that we align new start offset properly for time_based

If io_size isn't a multiple of the block size, weird things can
happen, which usually means that fio will exit with an unaligned
IO error:

fio: io_u error on file /dev/nvme0n1: Invalid argument: write offset=125830, buflen=1048576

where offset=125830 really should have been aligned to the block
size. Ensure that we do that.

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agot/verify-state: one more printf type fix
Jens Axboe [Wed, 13 Jan 2016 20:17:39 +0000 (13:17 -0700)]
t/verify-state: one more printf type fix

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agot/verify-state: fix type for printf
Jens Axboe [Wed, 13 Jan 2016 20:17:00 +0000 (13:17 -0700)]
t/verify-state: fix type for printf

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoDestroy sk_out server key when we are done
Jens Axboe [Wed, 13 Jan 2016 16:40:51 +0000 (09:40 -0700)]
Destroy sk_out server key when we are done

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoAlways create server sk_out key
Jens Axboe [Wed, 13 Jan 2016 16:34:46 +0000 (09:34 -0700)]
Always create server sk_out key

We use it from helper threads, so let's just unconditionally
set it up. This fixes a problem on fio exit with libraries or
engines that also create pthread keys.

Reported-by: Jens Rosenboom <j.rosenboom@x-ion.de>
Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agooptions: fix bs_cmp()
Jens Axboe [Tue, 12 Jan 2016 17:18:50 +0000 (10:18 -0700)]
options: fix bs_cmp()

Return the int difference between the two, not a bool.

Reported-by: Yury Gribov <>
Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agouse option smalloc_pool_size
Andreas Piesk [Fri, 8 Jan 2016 13:29:12 +0000 (14:29 +0100)]
use option smalloc_pool_size

3 years agot/verify-state: add helper to inspect verify dump state files
Jens Axboe [Thu, 7 Jan 2016 22:06:22 +0000 (15:06 -0700)]
t/verify-state: add helper to inspect verify dump state files

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoMerge branch 'master' of git://github.com/ezrapedersen/fio
Jens Axboe [Thu, 7 Jan 2016 22:01:09 +0000 (15:01 -0700)]
Merge branch 'master' of git://github.com/ezrapedersen/fio

3 years agoverify-state: forward declare io_u
Jens Axboe [Thu, 7 Jan 2016 21:26:57 +0000 (14:26 -0700)]
verify-state: forward declare io_u

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoDisable libaio for ESXi build - bug#80
root [Thu, 7 Jan 2016 21:25:10 +0000 (21:25 +0000)]
Disable libaio for ESXi build - bug#80

3 years agoverify: split out state header code
Jens Axboe [Thu, 7 Jan 2016 21:22:09 +0000 (14:22 -0700)]
verify: split out state header code

We'll need to use it in a separate helper utility, and we don't
need all the verify code for this.

Signed-off-by: Jens Axboe <axboe@fb.com>
3 years agoserver: sk_out exit error handling
Jens Axboe [Thu, 7 Jan 2016 18:19:28 +0000 (11:19 -0700)]
server: sk_out exit error handling

Signed-off-by: Jens Axboe <axboe@fb.com>