Jens Axboe [Tue, 25 Mar 2014 17:50:55 +0000 (10:50 -0700)]
server: dump where we attempted to bind in case of failure
Helps locate what the issue is, instead of having to use
strace and similar tools.
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Tue, 25 Mar 2014 17:37:56 +0000 (10:37 -0700)]
server: correct comment typo on port being after ':'
We use commas in fio, since that works equally well on IPv4
and IPv6.
Signed-off-by: Jens Axboe <axboe@fb.com>
Castor Fu [Thu, 20 Mar 2014 17:59:29 +0000 (10:59 -0700)]
Pass -Wstrict-prototypes -Wold-style-definition, whitespace
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Thu, 20 Mar 2014 15:33:13 +0000 (09:33 -0600)]
Don't re-create/unlink write file for append workload
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Thu, 20 Mar 2014 14:28:11 +0000 (08:28 -0600)]
Unlink after file close
Sébastien reports:
I'm having an issue on windows : unlink=1 is not working (temp file are
still there) and it's working fine on unix
Looking at the code in close_and_free_files function :
[...]
if (td->o.unlink && f->filetype == FIO_TYPE_FILE) {
dprint(FD_FILE, "free unlink %s\n", f->file_name);
unlink(f->file_name);
}
[...]
Unlink() fails because the file is still open : On Solaris, truss shows
the following :
[...]
write(3, "\0\0\0\0\0 ;80\0\0\0\0\0".., 32768) = 32768
unlink("/data/fio/random_rw.0.0") = 0
close(3) = 0
[...]
So unlink is called first.
I would put this unlinking phase AFTER the remove_file_hash call.
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Tue, 18 Mar 2014 19:30:32 +0000 (13:30 -0600)]
Prevent file_append=1 with IO engine that can't extend files
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Mon, 17 Mar 2014 18:55:08 +0000 (12:55 -0600)]
Update file_append documentation
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Mon, 17 Mar 2014 18:51:09 +0000 (12:51 -0600)]
Add support for file appends
Add option file_append (bool). If set, fio will append to a file
instead of operate within the size of it. This is similar to
setting offset= to the size of each file.
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Sat, 15 Mar 2014 01:41:07 +0000 (19:41 -0600)]
Update documentation on zero_buffers / scramble_buffers
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Fri, 14 Mar 2014 18:24:28 +0000 (12:24 -0600)]
Don't scramble buffers if compression is enabled
For the other buffer settings (like patterns, or whether always
to refill or not), we don't do the slight header scramble if
they are enabled. Do they same for compression settings, otherwise
we can skew the compression results.
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Fri, 14 Mar 2014 14:41:39 +0000 (08:41 -0600)]
Bump length of description field to 256 chars
Ensure that string always end up \0 terminated, too.
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Thu, 13 Mar 2014 14:49:28 +0000 (08:49 -0600)]
Disable status file updates on error
If fio doesn't have permission to unlink() the status file,
it'll continually dump status updates. If this happens, stop
doing status updates and log an error indicating why.
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Wed, 12 Mar 2014 19:40:09 +0000 (13:40 -0600)]
stat: fixup fio-dump-status file location
Don't use /tmp in the actual file name. And kill unused Windows
os_get_tmpdir().
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Wed, 12 Mar 2014 16:50:29 +0000 (10:50 -0600)]
Merge branch 'master' of ssh://git.kernel.dk/data/git/fio
Jens Axboe [Wed, 12 Mar 2014 16:49:36 +0000 (10:49 -0600)]
Update documentation for filename/directory
Sync the HOWTO and man page, and add a reference from 'directory'
to 'filename' on how to escape certain characters.
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Wed, 12 Mar 2014 02:16:38 +0000 (20:16 -0600)]
Add exceeds_number_ios() helper
Commit
26251d8d open-coded the logic in two places, collapse
that into one function.
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Tue, 11 Mar 2014 16:21:18 +0000 (10:21 -0600)]
Try a little harder to honor number_ios more accurately
Previously, fio checked this on completion, which means we could
almost spill over by the iodepth setting. Check this before
issue instead, taking the in flight IOs into account as well.
Reported-by: Robert Elliott <Elliott@hp.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
Sébastien Bouchex Bellomié [Tue, 11 Mar 2014 14:54:27 +0000 (08:54 -0600)]
Fix pointer alignment bug causing SIGBUS on Sparc64
OK, the issue was related to the alignment issue between the
thread_options structure and its "packed" version : The lexxx_to_cpu
macros are incorrectly casting the pointers and that causes an bus
error. I have fixed the issue by avoid casting the pointers and calling
directly the "swap" method.
I have also included a fixed on the thread_options structure regarding
the ddir_seq_add variable : Its length was incorrect.
Signed-off-by: Jens Axboe <axboe@fb.com>
Stefan Hajnoczi [Thu, 6 Mar 2014 17:41:59 +0000 (18:41 +0100)]
Detect Windows operating system in ./configure
We should not rely on `uname -s` if we ever want cross-compiling to
work, so detect _WIN32.
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
Stefan Hajnoczi [Thu, 6 Mar 2014 17:41:58 +0000 (18:41 +0100)]
Honor ./configure --cc= option
The cc variable is assigned before parsing command-line options. This
effectively ignores the ./configure --cc= option. Move the cc variable
assigned below option parsing.
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Thu, 6 Mar 2014 16:18:00 +0000 (09:18 -0700)]
Fio 2.1.6.1
Lets tag a minor 2.1.6 update that has the RHEL/CentOS5
compile issue fixed.
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Wed, 5 Mar 2014 22:49:54 +0000 (15:49 -0700)]
Fix compile on CentOS/RHEL5
They don't have CPU_COUNT(), so add a helper for that.
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Tue, 4 Mar 2014 16:02:10 +0000 (09:02 -0700)]
Fix typo in fio_cpu_count()
Reported-by: Paul Howarth <paul@city-fan.org>
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Mon, 3 Mar 2014 23:49:00 +0000 (15:49 -0800)]
Fio 2.1.6
Signed-off-by: Jens Axboe <axboe@fb.com>
Puthikorn Voravootivat [Mon, 3 Mar 2014 23:33:32 +0000 (15:33 -0800)]
Fix verify_only option not working properly
In commit
c4b6117, we changed the place to call log_io_piece but
the new place won't be reached when using verify_only option so
that when we run with the verify_only option, fio won't verify
anything at all.
This patch fix that by also call the log_io_piece in the do_dry_run
function for the verify_only code path.
Signed-off-by: Puthikorn Voravootivat <puthik@chromium.org>
Reviewed-by: Gwendal Grignou <gwendal@chromium.org>
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Mon, 3 Mar 2014 23:44:57 +0000 (15:44 -0800)]
Merge branch 'master' of ssh://git.kernel.dk/data/git/fio
Jens Axboe [Mon, 3 Mar 2014 20:55:44 +0000 (13:55 -0700)]
solaris: fixup new cpuset functions
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Mon, 3 Mar 2014 20:55:32 +0000 (13:55 -0700)]
Silence 'io_u might be used uninitialized' warning
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Mon, 3 Mar 2014 20:53:39 +0000 (13:53 -0700)]
server: fix warnings on some platforms on incompatible pointer type
Signed-off-by: Jens Axboe <axboe@fb.com>
Bruce Cran [Sun, 2 Mar 2014 00:05:03 +0000 (17:05 -0700)]
Windows: add install directory to the Path environment variable
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Sat, 1 Mar 2014 15:24:03 +0000 (08:24 -0700)]
Add runstate swap helpers
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Sat, 1 Mar 2014 00:43:57 +0000 (16:43 -0800)]
Update cpus_allowed_policy=split documentation
Instead of erroring out on fewer CPUs than threads/processes,
fio now does roundrobin over the set.
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Fri, 28 Feb 2014 23:08:25 +0000 (15:08 -0800)]
cpus_allowed_policy fixups
- Make 'split' roundrobin the CPUs, if we have more threads
than CPUs in the set.
- Properly use a CPU index, don't assume a sequential set of
CPUs.
Signed-off-by: Jens Axboe <axboe@fb.com>
Peter Oberparleiter [Fri, 28 Feb 2014 17:09:13 +0000 (09:09 -0800)]
fio: fix last block never being touched by random offsets
Fix the available range for random offsets which never touched the last block.
Signed-off-by: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Fri, 28 Feb 2014 16:35:40 +0000 (08:35 -0800)]
Fix off-by-one bug in CPU split mask handling
The thread number is not zero based, it starts from 1.
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Fri, 28 Feb 2014 00:24:15 +0000 (16:24 -0800)]
windows: fix fio_cpu_count() definition
Don't take a 'cpu' argument, that's a leftover when this was
(temporarily) fio_cpu_isset().
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Fri, 28 Feb 2014 00:18:30 +0000 (16:18 -0800)]
windows: add proper header for hweight64()
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Fri, 28 Feb 2014 00:10:43 +0000 (16:10 -0800)]
Update IO engine version
Commit
225ba9e3 changed the layout of struct io_u, so
update the engine version to indicate that external
engines need to be recompiled.
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Thu, 27 Feb 2014 23:52:02 +0000 (15:52 -0800)]
Add support for cpus_allowed_policy
Two policies are supported:
shared cpumask is shared between all threads. This is the original
behavior.
split cpumask is split between threads, each thread gets a unique
CPU.
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Wed, 26 Feb 2014 22:31:15 +0000 (14:31 -0800)]
Branch and cache miss speedups
Just some low hanging fruit.
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Wed, 26 Feb 2014 21:42:13 +0000 (13:42 -0800)]
blktrace: load improvements
- Set the state explicitly to SETTING_UP when loading the
traces. This could take a while, so tell the status thread
that we are setting up so it knows how to display us
correctly.
- Fix a bug with multiple devices where we didn't cache the
major/minor properly. This made blktrace spend tons of CPU
cycles looking up /sys.
- Handle multiple devices correctly when adding files.
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Tue, 25 Feb 2014 22:01:26 +0000 (14:01 -0800)]
Fixup ->open_files if not given
For engines/users that add files with add_file() after the
job has been initialized, ensure that we bump the allowed
number of open files if it wasn't explicitly given as
an option.
This unbreaks the net engine after commit
002fe73409d1e made
fio more strict wrt open files.
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Tue, 25 Feb 2014 21:43:38 +0000 (13:43 -0800)]
net: fix accidental overwrite of more than just the address
Commit
0b783341 accidentally overwrote more than just the
address, causing garbage in the connection part.
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Tue, 25 Feb 2014 21:43:17 +0000 (13:43 -0800)]
Fix crash with --debug=file and NULL file return
Don't attempt to dereference that, obviously.
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Tue, 25 Feb 2014 19:35:27 +0000 (11:35 -0800)]
Ensure that fio_get_kb_base() doesn't assume 'data' is thread_options
Nasty hack, but at least we don't corrupt option data.
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Fri, 21 Feb 2014 23:26:01 +0000 (15:26 -0800)]
Unify the time handling
Some options are in seconds, if no units are given. These include
runtime/timeout, startdelay, and ramp_time. Handle this
appropriately. Internally it should all be microseconds now, it's
just the conversion factor.
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Fri, 21 Feb 2014 21:55:31 +0000 (13:55 -0800)]
Improve latency_target runs
Reset the stats when we have found our target, and then do
another latency_window run with those settings so that the
final results reflect the probed values.
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Fri, 21 Feb 2014 19:50:09 +0000 (11:50 -0800)]
Fixup time multipliers
Commit
74454ce4 assumed default time is in msec, when it's
in reality usec. Fix that up, and also add a specific 'us'
or 'usec' multiplier (of 1).
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Thu, 20 Feb 2014 21:29:42 +0000 (13:29 -0800)]
xxhash: dos2unix'ize
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Thu, 20 Feb 2014 21:22:48 +0000 (13:22 -0800)]
Add option group/category to allrandrepeat
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Thu, 20 Feb 2014 21:21:45 +0000 (13:21 -0800)]
Add support for the Google xxhash checksumming function
It's a lot faster than the other software defined ones, so will
be a useful alternative for the cases where crc32c-intel isn't
available.
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Thu, 20 Feb 2014 17:48:17 +0000 (09:48 -0800)]
server: bump protocol version
We have new options added.
Signed-off-by: Jens Axboe <axboe@fb.com>
Peter Oberparleiter [Thu, 20 Feb 2014 13:20:07 +0000 (14:20 +0100)]
fio: allow 0 as compress percentage
Allow 0 as value for option compress_percentage which can be useful for
certain deduplication and compression based storage back ends.
Signed-off-by: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
Peter Oberparleiter [Thu, 20 Feb 2014 13:20:05 +0000 (14:20 +0100)]
fio: flush log files on test end
Ensure proper flushing of all logs at the end of tests.
Signed-off-by: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
Christian Ehrhardt [Thu, 20 Feb 2014 13:20:04 +0000 (14:20 +0100)]
fio: allow to combine terse output with any selected output type
This patch adds the option --append-terse to be able to request a
combination of any given selected output format AND terse output based
on the same data.
This will help all users that want to parse the terse data for further
use, but need to look into the logs every now and then which then should
be readable.
Signed-off-by: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
Christian Ehrhardt [Thu, 20 Feb 2014 17:13:06 +0000 (09:13 -0800)]
fio: add multi directory support
This patch adds support for ':' seperated multiple directories at the
directory config statement in order to achieve an automatic distribution
of job clones (numjob) across directories.
That way people can distribute a load across these directories (usually
mount points of disks) automatically - changing numjob will be
sufficient to get all job clones evenly (optimal if dirs % numjobs = 0,
otherwise as good as possible) distributed at all times.
To avoid confused users old config Files will behave like they always
did, old fio binaries using new config files won't abort but just use
the first specified dir. If one specifies an explcit (non generated)
filename the distribution to many directories is also deactivated.
It also fixes an issue of events seeming out of order like when running
with --debug=file seeing the "..." message meaning "I created the
clones" prior to the last clone activities. Now the clones are called
with index N-1 .. 1, zero being the base thread as before.
Signed-off-by: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
Christian Ehrhardt [Thu, 20 Feb 2014 17:07:02 +0000 (09:07 -0800)]
fio: provide an option for a startdelay range
This patch allows the specification of start delay as range. With a
range each thread will chose an individual startdelay out of the range.
That solves an issue of startdelay being the same for each numjob clone
and that way spreads all kind of activities e.g. that all clones with
mixed r/w jobs switch r/w at the same time. Also all kind of other
"thundering herd" issues can be softened by some time spread due to this
option.
Signed-off-by: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
Christian Ehrhardt [Thu, 20 Feb 2014 13:20:01 +0000 (14:20 +0100)]
fio: allow milliseconds on all time specifiers
This patch allows all time specifiers to be specified down to
milliseconds. Default will stay seconds for compatibility with old
configs.
It also adds documentation to the existing time units day, hour and
minute.
Signed-off-by: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
Christian Ehrhardt [Thu, 20 Feb 2014 17:10:17 +0000 (09:10 -0800)]
fio: allow general repeatability
This patch adds a "allrandrepeat" option similar to the existing
randrepeat. But other than the former option it does not only affect
the random I/O pattern, but also all remaining users of randomness as
well.
By that e.g. testcases using nrfiles, filesize ranges, blocksizesplit
ranges and so on will stay repeatable across (euqally parametrized) runs
as well. To maintain compatibility the default is off.
Signed-off-by: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>
Update by me to fix missing thread_options.h and cconv.c change.
Signed-off-by: Jens Axboe <axboe@fb.com>
Christian Ehrhardt [Wed, 19 Feb 2014 18:42:41 +0000 (10:42 -0800)]
fio: fix job clone mem leak
In the loop to create clones at the bottom of add_job the function
get_new_job clones the thread_data, just to occaisonally get the
allocated pointers for filename and files overwritten a few lines later.
The dup files also duplicates the name strings so the references to
these are lost by the setting to null.
This patch fixes takes care of that and frees the memory before
discarding the pointers (found via valgrind).
Signed-off-by: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
Danny Al-Gaaf [Mon, 17 Feb 2014 12:53:07 +0000 (13:53 +0100)]
add example job file for the RBD engine
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
Signed-off-by: Jens Axboe <axboe@fb.com>
Danny Al-Gaaf [Mon, 17 Feb 2014 12:53:06 +0000 (13:53 +0100)]
fio.1: add rbd options to man page
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit
6e44a82a724b7ad0a559c22eec694e8afa7d2a20
git@github.com:dalgaaf/fio.git)
Signed-off-by: Jens Axboe <axboe@fb.com>
Daniel Gollub [Mon, 17 Feb 2014 13:35:28 +0000 (14:35 +0100)]
librbd ioengine support
Initially only DDIR_WRITE/DDIR_READ and DDIR_SYNC are supported.
All other io_u->ddir get silently ignored.
A known issue requires to call the rbd engine always with --invalidate=0
Signed-off-by: Daniel Gollub <d.gollub@telekom.de>
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Fri, 14 Feb 2014 15:48:22 +0000 (08:48 -0700)]
Fix early termination of cpu id string
Commit
2d60ad8e terminated the cpu id string at 12 chars,
but the output from cpuid is 12 chars. So we need one more
character, if we want it null terminated and strcmp()
working.
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Fri, 14 Feb 2014 15:46:35 +0000 (08:46 -0700)]
Fix another typo in gtod_reduce()
Forgot one of the variables. Impressive.
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Fri, 14 Feb 2014 03:04:02 +0000 (20:04 -0700)]
Fix inverted logic in gtod_reduce()
Fixes a bug in the previous commit, where someone wasn't
thinking straight...
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Fri, 14 Feb 2014 02:59:56 +0000 (19:59 -0700)]
Don't compute iops samples for gtod_reduce=1
Signed-off-by: Jens Axboe <axboe@fb.com>
Sitsofe Wheeler [Thu, 13 Feb 2014 07:06:40 +0000 (07:06 +0000)]
io_u_qiter: Fix buffer overrun
In io_u_queue.h the io_u_qiter macro is loops around io_u_queue
structures. The problem comes with the end of loop initialisation:
i++, io_u = (q)->io_us[i]
For example, if io_us consists of one element and i is 0 then after the
first iteration is completed i++, io_u = (q)->io_us[i] will access
beyond the end of io_us.
Fix this by moving io_u initialisation to the expression part of the for
loop (yuck).
Found by Dr Memory.
Signed-off-by: Sitsofe Wheeler <sitsofe@yahoo.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
Sitsofe Wheeler [Thu, 13 Feb 2014 06:07:52 +0000 (06:07 +0000)]
cpuid: Null terminate x86 CPU string.
Signed-off-by: Sitsofe Wheeler <sitsofe@yahoo.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
Bruce Cran [Thu, 13 Feb 2014 16:06:20 +0000 (09:06 -0700)]
Fix IPv6 check on FreeBSD by including netinet/in.h
I noticed IPv6 support was being disabled on FreeBSD; it seems the
configure script needs netinet/in.h so in6_addr is known.
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Thu, 13 Feb 2014 04:25:20 +0000 (21:25 -0700)]
null: style fixup
Signed-off-by: Jens Axboe <axboe@fb.com>
Daniel Gollub [Wed, 12 Feb 2014 19:13:46 +0000 (20:13 +0100)]
engines/null: allow build as external C++ ioengine
In first place to have a simple test candidate for external
ioengines using get_ioengine() method for retriving the
io_engine struct.
Can be compiled with:
g++ -O2 -g -shared -rdynamic -fPIC -o null.so null.c -DFIO_EXTERNAL_ENGINE
Signed-off-by: Daniel Gollub <d.gollub@telekom.de>
Signed-off-by: Jens Axboe <axboe@fb.com>
Daniel Gollub [Thu, 13 Feb 2014 04:23:31 +0000 (21:23 -0700)]
Introduce get_ioengine for external engines
This makes life easier for plugins written in C++
since they do not need to deal with struct initilization issues
with the ioengine_ops symbol.
With g++ a non-static ioengine_ops in global scope like this:
struct ioengine_ops ioengine = {
.name = "null",
.version = FIO_IOOPS_VERSION,
.queue = fio_null_queue,
.commit = fio_null_commit,
};
Results in:
cpp_null2.cc: At global scope:
cpp_null2.cc:112:1: error: C99 designator ‘name’ outside aggregate initializer
cpp_null2.cc:112:1: sorry, unimplemented: non-trivial designated initializers not supported
cpp_null2.cc:112:1: sorry, unimplemented: non-trivial designated initializers not supported
cpp_null2.cc:112:1: sorry, unimplemented: non-trivial designated initializers not supported
cpp_null2.cc:112:1: sorry, unimplemented: non-trivial designated initializers not supported
$
Example get_iongine() symbol usage:
---8<---
extern "C" {
void get_ioengine(struct ioengine_ops **ioengine_ptr) {
struct ioengine_ops *ioengine;
*ioengine_ptr = (struct ioengine_ops *) malloc(sizeof(struct ioengine_ops));
ioengine = *ioengine_ptr;
strcpy(ioengine->name, "cpp_null");
ioengine->version = FIO_IOOPS_VERSION;
ioengine->queue = fio_null_queue;
ioengine->commit = fio_null_commit;
ioengine->getevents = fio_null_getevents;
ioengine->event = fio_null_event;
ioengine->init = fio_null_init;
ioengine->cleanup = fio_null_cleanup;
ioengine->open_file = fio_null_open;
ioengine->flags = FIO_DISKLESSIO;
}
}
--->8---
Signed-off-by: Daniel Gollub <d.gollub@telekom.de>
Moved get_ioengine_t typedef to ioengine.h.
Signed-off-by: Jens Axboe <axboe@fb.com>
Daniel Gollub [Wed, 12 Feb 2014 18:25:42 +0000 (19:25 +0100)]
Fix g++ warning about void* used in arithmetic
../parse.h: In function ‘void* td_var(thread_options*, fio_option*, unsigned int)’:
../parse.h:110:10: warning: pointer of type ‘void *’ used in arithmetic [-Wpointer-arith]
../parse.h:112:9: warning: pointer of type ‘void *’ used in arithmetic [-Wpointer-arith]
Signed-off-by: Daniel Gollub <d.gollub@telekom.de>
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Wed, 12 Feb 2014 16:10:10 +0000 (09:10 -0700)]
clang: fix warnings with clang build
Signed-off-by: Jens Axboe <axboe@fb.com>
Daniel Gollub [Wed, 12 Feb 2014 15:24:57 +0000 (08:24 -0700)]
Make parse.h C++ safe by avoiding "or" keyword
Fixes for g++ (4.7.2) following compiler errors when fio.h
gets included (e.g. in an external C++ ioengine):
--8<---
[...]
parse.h:31:6: error: expected unqualified-id before ‘or’ token
[...]
--->8---
Signed-off-by: Daniel Gollub <d.gollub@telekom.de>
Updated by me to apply to recent version, and fix gfio usage
of ->or as well.
Signed-off-by: Jens Axboe <axboe@fb.com>
Daniel Gollub [Wed, 12 Feb 2014 14:51:54 +0000 (15:51 +0100)]
Make file.h C++ safe by casting fio_file_flags
Fixes for g++ (4.7.2) following compiler errors when fio.h
gets included (e.g. in an external C++ ioengine):
--8<---
[...]
os/../file.h: In function ‘void fio_file_set_open(fio_file*)’:
os/../file.h:142:1: error: invalid conversion from ‘int’ to ‘fio_file_flags’ [-fpermissive]
os/../file.h: In function ‘void fio_file_clear_open(fio_file*)’:
os/../file.h:142:1: error: invalid conversion from ‘int’ to ‘fio_file_flags’ [-fpermissive]
os/../file.h: In function ‘void fio_file_set_closing(fio_file*)’:
[...]
--->8---
Signed-off-by: Daniel Gollub <d.gollub@telekom.de>
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Wed, 12 Feb 2014 15:22:01 +0000 (08:22 -0700)]
Make err.h a bit more Windows friendly
Using unsigned long to hold pointers is a linuxism, clean
it up.
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Tue, 11 Feb 2014 22:44:50 +0000 (15:44 -0700)]
parse: cleanup difference between profile and normal options
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Tue, 11 Feb 2014 21:49:43 +0000 (14:49 -0700)]
Fio 2.1.5
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Tue, 11 Feb 2014 21:19:38 +0000 (14:19 -0700)]
parse: get rid of raw option offsets
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Tue, 11 Feb 2014 17:33:06 +0000 (10:33 -0700)]
Plug a free basic memory leaks
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Tue, 11 Feb 2014 16:12:06 +0000 (09:12 -0700)]
Enfore that lockfile= must precede filename=
The way fio parses options, we need to have the file locking
defined before we start adding files. Normally we can handle
this with option priorities, but that doesn't work across
job section boundaries. So catch this case and error out,
otherwise fio will segfault when it attempts to unlock files
at the end of a job.
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Tue, 11 Feb 2014 15:31:13 +0000 (08:31 -0700)]
Fix issue with openfiles= and file sizing
Fill the given size as well as we can, given the size and nr
of files.
Fix an issue where we don't properly honor how many files to
keep open at any point in time.
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Mon, 10 Feb 2014 20:57:09 +0000 (13:57 -0700)]
Fix issue with td->mutex being used-after-free
Depending on how the OS schedules the threads, it's not necessarily
safe to immediately free a mutex after we have been woken up.
Remove the td->mutex freeing to normal cleanup after the job is done.
Reported-by: Sitsofe Wheeler <sitsofe@yahoo.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Sat, 8 Feb 2014 03:54:39 +0000 (20:54 -0700)]
crc: add option to list possible crc types
=help or =list will show them.
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Sat, 8 Feb 2014 03:52:32 +0000 (20:52 -0700)]
crc: ensure we properly match test name
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Sat, 8 Feb 2014 03:48:29 +0000 (20:48 -0700)]
README: update for crctest
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Fri, 7 Feb 2014 21:39:33 +0000 (14:39 -0700)]
Add support for testing checksumming speed
fio --crctest will test all of them, --crctest=md5,crc32c will
test md5 and crc32c, for example.
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Fri, 7 Feb 2014 17:57:13 +0000 (10:57 -0700)]
Prioritize lockfile= option
This needs to be run before adding files, or we'll potentially
make a mess of things.
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Fri, 7 Feb 2014 17:56:15 +0000 (10:56 -0700)]
Fix crash with file locking and dup'ed files
Ensure that we properly inherit the file locks when
duplicating files.
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Thu, 6 Feb 2014 19:17:37 +0000 (12:17 -0700)]
verify: always log IO in the order they are issued
We currently log verify_backlog verifies when they complete,
which means the sequence of verify and issue might be different.
Change this to log in one spot, prior to issue, and track the
completion state of the logged unit instead. This unifies the
handling of verifies.
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Wed, 5 Feb 2014 20:15:22 +0000 (13:15 -0700)]
More precise fix for verify_backlog verification failure
Commit
c2703bf3 could be a bit more precise - we only need to
add to the byte count if verify_backlog is enabled, since that
is the case where do_io() will run both the IO and verify IO.
Signed-off-by: Jens Axboe <axboe@fb.com>
Puthikorn Voravootivat [Wed, 5 Feb 2014 18:28:15 +0000 (10:28 -0800)]
Fix verify_backlog start verification before finish writing
Due to recent fix for checking rand_seed in verify phase. In write phase,
the IO transaction will be logged before completing the actual data write.
If verify_backlog is enabled, verification may start before all data has
been written. This patch disable that fix if the verify_backlog is enabled.
Signed-off-by: Puthikorn Voravootivat <puthik@chromium.org>
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Wed, 5 Feb 2014 19:36:02 +0000 (12:36 -0700)]
Fix for prematurely stopping on verify
Commit
20876c53 added a total byte check, but forgot to
account for potential verification bytes. Fix that.
Signed-off-by: Jens Axboe <axboe@fb.com>
Puthikorn Voravootivat [Wed, 5 Feb 2014 17:17:11 +0000 (10:17 -0700)]
Fix rand_seed mismatches in verify phase
In verify phase, the rand_seed generated on replay does not match
the written rand_seed.
Multiple problems are causing this:
1. In verify phase fio does not set io_u->rand_seed to compare with
hdr->rand_seed
2. In randrw scenario, fio log is stored in red-black tree in "sorted by LBA"
order. Thus, it is imposible to replay the written order, or rather
generate the seeds again in the same order.
3. In write phase, the code currently will generate rand_seed, write data
and log rand_seed. When queuedepth > 1, it's possible the writes complete
in a different order than rand_seed was generated. Thus when replaying
the log, the generated rand_seed might not match what was written.
4. verify_backlog option will start verification before all the data has been
written and it make rand_seed replay code broken with current design.
Proposed fixes:
1. Use of existing verify_state to generate verify header.
(and assumes this was the original intention of verify_state). And also
adds code to replay rand_seed in verify phase.
2. If verifysort option is not enabled, store the write log in a list instead
of the red-black tree. Otherwise, don't attempt to verify the rand_seed
in the header.
3. In write phase, generate rand_seed, log rand_seed, write data. I.e. log
IO transactions in the order generated, not completed.
4. Don't verify rand_seed when verify_backlog is enabled.
Signed-off-by: Puthikorn Voravootivat <puthik@chromium.org>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Stefan Hajnoczi [Tue, 4 Feb 2014 13:27:11 +0000 (14:27 +0100)]
init: log error on missing --output-format argument
strcmp(optarg, "foo") will crash if --output-format was given without an
argument. Log an error and exit properly instead of crashing.
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
Stefan Hajnoczi [Tue, 4 Feb 2014 13:27:10 +0000 (14:27 +0100)]
init: --minimal does not take an optional argument
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
Jens Axboe [Wed, 29 Jan 2014 23:39:04 +0000 (16:39 -0700)]
init: update email address
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Bruce Cran [Wed, 29 Jan 2014 00:34:08 +0000 (17:34 -0700)]
Enable IPv6 in the Windows build
Signed-off-by: Jens Axboe <axboe@kernel.dk>