Jens Axboe [Thu, 30 Nov 2017 05:11:09 +0000 (22:11 -0700)]
options: correct parser type for max_latency
It's a 64-bit type now, use the proper time conversion type.
Fixes:
c3a32714 ("Change latency targets to be in nsec values internally")
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 30 Nov 2017 05:00:35 +0000 (22:00 -0700)]
verify: convert hdr time to sec+nsec
Use the internal values instead of converting.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 30 Nov 2017 04:58:28 +0000 (21:58 -0700)]
verify: kill unneeded forward declaration
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 30 Nov 2017 04:52:46 +0000 (21:52 -0700)]
Change latency targets to be in nsec values internally
Since all of our timekeeping is in nsec now, it's easier to convert
these at init time and not have to do it at runtime.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 30 Nov 2017 04:47:43 +0000 (21:47 -0700)]
options: make max_latency a 64-bit variable
Also removes one pad in the thread options. This should bump
the protocol version, but we just did that, so let it slide.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 30 Nov 2017 04:30:21 +0000 (21:30 -0700)]
options: make it clear that max_latency is in usecs
The other latency options include this in their long name.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 30 Nov 2017 04:25:32 +0000 (21:25 -0700)]
io_u: do nsec -> usec converison in one spot in account_io_completion()
Should not matter for runtime, but it's cleaner.
What we should really do is convert the internal values to nsec, so
we don't have to do this conversion.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 30 Nov 2017 04:01:36 +0000 (21:01 -0700)]
io_u: don't do expensive int divide for buffer scramble
We don't need the conversion from nsec to usec to be exact, so
just shift by 10 instead.
Fixes:
8b6a404cd ("nanosecond: initial commit changing timeval to timespec")
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 30 Nov 2017 03:51:09 +0000 (20:51 -0700)]
Unify max split zone support
We have two different defines, use the one from the networking
protocol. Add a compile time check that bssplit and zones
have the same max, so we don't inadvertently add an overflow
condition if we change one of them.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 30 Nov 2017 03:45:33 +0000 (20:45 -0700)]
io_u: cleanup and simplify __get_next_rand_offset_zoned_abs()
We can drop various variables, it's easier to read this way too.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 30 Nov 2017 02:29:25 +0000 (19:29 -0700)]
examples/rand-zones.fio: add zoned_abs example
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 30 Nov 2017 02:25:59 +0000 (19:25 -0700)]
Add support for absolute random zones
We currently support random_distribution=zoned, which allows the user
to specify a percentage of access to a zoned define as a percentage
of the file/device size. This commit adds support for zoned_abs,
which works exactly like zoned, except you give the zone size in
an absolute value.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Wed, 29 Nov 2017 19:37:30 +0000 (12:37 -0700)]
options: don't quicksort zoned distribution series
If we do that, we mess up the distribution.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Tomohiro Kusumi [Wed, 29 Nov 2017 09:29:22 +0000 (11:29 +0200)]
Revert "Avoid irrelevant "offset extend ends" error message for chrdev"
This reverts commit
0f34169ac64f009b193bed1c8fb6692844bf99c2.
> Commit
0f34169a broke it, moving a check from a non-character
> device path up a level to where it now triggers on a character
> device (for which the size is left set to -1ULL).
>
> This gets it working again:
> git revert
0f34169a
> make
>
> Tomohiro, could you help prepare a better fix?
Sorry, I'll just revert it at the moment.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Tue, 28 Nov 2017 23:00:22 +0000 (16:00 -0700)]
init: did_arg cleanup
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Tue, 28 Nov 2017 21:02:11 +0000 (14:02 -0700)]
init: remove dead code
We know did_arg == 0 here, so kill the other part.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Tue, 28 Nov 2017 17:11:53 +0000 (10:11 -0700)]
init: check and fail status-interval settings that are too small
Smallest possible value is 1ms, so fail any setting that yields
a value less than that.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Stephen Bates [Thu, 23 Nov 2017 19:21:56 +0000 (12:21 -0700)]
docs: Add documention for RDMA ioengine options.
Add documentation to HOWTO and the man page for the RDMA ioengine
options as these were missing.
Signed-off-by: Stephen Bates <sbates@raithlin.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 23 Nov 2017 02:58:21 +0000 (19:58 -0700)]
Merge branch 'cleanup' of https://github.com/sitsofe/fio
Sitsofe Wheeler [Wed, 22 Nov 2017 22:58:10 +0000 (22:58 +0000)]
doc: reword buffer_compress_percentage, buffer_compress_chunk, dedupe_percentage
Make the documentation for these options better reflect how buffer
compression and dedupe tuning can be used since commit
1de80624466405bccdbc4607d71cd249320da3f1 ("Default buffer_compress_chunk
to 512").
Signed-off-by: Sitsofe Wheeler <sitsofe@yahoo.com>
Sitsofe Wheeler [Wed, 22 Nov 2017 22:38:18 +0000 (22:38 +0000)]
HOWTO: fix up broken formatting in logging options
Signed-off-by: Sitsofe Wheeler <sitsofe@yahoo.com>
Jeff Furlong [Wed, 22 Nov 2017 18:38:33 +0000 (11:38 -0700)]
add significant_figures parameter
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Stephen Bates [Wed, 22 Nov 2017 18:24:56 +0000 (11:24 -0700)]
rdma: Add bind option
In certain configurations it can be useful to bind a rdma_cm to a
particular network interface. For example in multi-path or loopback.
Add a bindname option that the local rdma_cm will try and bind too.
The bind code is based off that used in rping [1].
[1] https://github.com/linux-rdma/rdma-core/blob/ \
master/librdmacm/examples/rping.c
Signed-off-by: Stephen Bates <sbates@raithlin.com>
Reviewed-by: Logan Gunthorpe <logang@deltatee.com>>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Fri, 17 Nov 2017 16:21:19 +0000 (09:21 -0700)]
Merge branch 'libpmem'
Merge in libpmem engine, with fixes.
Jens Axboe [Fri, 17 Nov 2017 16:20:52 +0000 (09:20 -0700)]
examples/libpmem.fio: clean up example
Spelling errors, typos, etc.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Fri, 17 Nov 2017 16:19:26 +0000 (09:19 -0700)]
libpmem: move mmap alignment to init time
Also use the generally available page_size, don't need to roll our
own in there.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Fri, 17 Nov 2017 16:16:44 +0000 (09:16 -0700)]
libpmem: code cleanups
No functional changes in this patch.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Fri, 17 Nov 2017 16:03:23 +0000 (09:03 -0700)]
Merge branch 'add-libpmem-engine' of https://github.com/tishizaki/fio into libpmem
Teruaki Ishizaki [Fri, 17 Nov 2017 02:54:26 +0000 (11:54 +0900)]
fio: add libpmem engine
Adding an ioengine that access with the libpmem as memory
through a memory mmaped file on DAX filesystem.
It's very similar to the mmap engine and the dev-dax engine.
Signed-off-by: Teruaki Ishizaki <ishizaki.teruaki@lab.ntt.co.jp>
Signed-off-by: Takashi Menjo <menjo.takashi@lab.ntt.co.jp>
Jens Axboe [Fri, 17 Nov 2017 03:03:15 +0000 (20:03 -0700)]
ioengines: remove pointless list initializations
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 16 Nov 2017 21:16:41 +0000 (14:16 -0700)]
os: make fio_cpu_isset() return a bool
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Robert Elliott [Tue, 14 Nov 2017 02:22:09 +0000 (20:22 -0600)]
os-windows: fix cpumask operations
Fix cpumask manipulation on (64-bit) Windows systems.
cpus_allowed=nn values greater than 32 does not work, due to
* the compiler not promoting expressions like "1 << cpu" to
64 bits
* the clear function clearing the wrong bit (using "cpu - 1"),
* the clear function using XOR to clear (which only works if the
bit was previously set)
* the check function returning a 64-bit value through a 32-bit
return value
Example problems (from extra debug prints):
* setting CPU 32 really sets bit 0 (CPU 0)
Set mask of
0000000000000000 to add
0000000000000001 (32)
* setting CPU 63 really sets bit 31 (CPU 31)
Set mask of
0000000000000000 to add
0000000080000000 (63)
* clearing CPU 0 really clears bit 63 (CPU 63)
Clear mask of
0000000055555555 to remove
8000000000000000 (0)
* clearing CPU 2 really clears bit 2 (CPU 2)
Clear mask of
0000000055555555 to remove
0000000000000002 (2)
* checking claims CPU 32 is not in a mask of CPUs {32,34,36,38}
for
0000005500000000 check if bit 32 is set ==> FALSE
Tested with x86_64-w64-mingw32-gcc 6.4.0 from cygwin on a system
with 64 CPU cores (all fitting in one Windows processor group).
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Robert Elliott [Mon, 13 Nov 2017 21:56:59 +0000 (15:56 -0600)]
.gitignore: ignore .exe files (for Windows)
Make git ignore .exe files, which are generated by Windows
cygwin builds.
Examples:
new file: fio.exe
new file: t/axmap.exe
new file: t/fio-dedupe.exe
new file: t/fio-genzipf.exe
new file: t/fio-verify-state.exe
new file: t/gen-rand.exe
new file: t/ieee754.exe
new file: t/lfsr-test.exe
new file: t/stest.exe
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Wed, 15 Nov 2017 16:53:14 +0000 (09:53 -0700)]
man page: fix bad case for 'pre-reading file' state
Fixes: https://github.com/axboe/fio/issues/491
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Mon, 6 Nov 2017 16:11:07 +0000 (09:11 -0700)]
Merge branch 'fix-libhdfs' of https://github.com/follitude/fio
follitude [Mon, 6 Nov 2017 12:28:20 +0000 (20:28 +0800)]
Makefile: tiny fix of libhdfs
Jens Axboe [Fri, 3 Nov 2017 21:23:49 +0000 (15:23 -0600)]
Fio 3.2
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Fri, 3 Nov 2017 18:37:03 +0000 (12:37 -0600)]
Merge branch 'json_bw_bytes' of https://github.com/sitsofe/fio
Tomohiro Kusumi [Fri, 3 Nov 2017 16:46:03 +0000 (18:46 +0200)]
solaris: add get_fs_free_size() implementation
Copied from os/os-freebsd.h. POSIX statvfs works on Solaris
(actually compiled/tested on illumos) as well.
Signed-off-by: Tomohiro Kusumi <tkusumi@tuxera.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Tomohiro Kusumi [Fri, 3 Nov 2017 16:46:02 +0000 (18:46 +0200)]
solaris: add os_phys_mem() implementation
Copied from os/os-linux.h. The standard sysconf(3C) names work on
Solaris (actually compiled/tested on illumos) as well.
Signed-off-by: Tomohiro Kusumi <tkusumi@tuxera.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Tomohiro Kusumi [Fri, 3 Nov 2017 16:46:01 +0000 (18:46 +0200)]
solaris: #include <pthread.h>
os/os-solaris.h header uses pthread, but doesn't include <pthread.h>
which should be to be able to include this header alone.
Needed for unittesting of OS specific stuff.
Signed-off-by: Tomohiro Kusumi <tkusumi@tuxera.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Sitsofe Wheeler [Fri, 3 Nov 2017 07:23:55 +0000 (07:23 +0000)]
stat: add bw_bytes JSON key
Fixes: https://github.com/axboe/fio/issues/422
Signed-off-by: Sitsofe Wheeler <sitsofe@yahoo.com>
Jens Axboe [Thu, 2 Nov 2017 18:26:39 +0000 (12:26 -0600)]
io_u_queue: convert rings to bool
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 2 Nov 2017 15:09:31 +0000 (09:09 -0600)]
filesetup: allocate 'r' locally in fallocate_file()
For platforms that don't have posix or linux fallocate, we trigger
a warning on 'r' being unused.
Fixes:
c0aabe37db1b ("filesetup: don't print non-debug error on native fallocate failure")
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 2 Nov 2017 15:07:11 +0000 (09:07 -0600)]
filesetup: recurse_dir() can use bool
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 2 Nov 2017 15:06:14 +0000 (09:06 -0600)]
filesetup: create_work_dirs() can return bool
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 2 Nov 2017 15:04:56 +0000 (09:04 -0600)]
filesetup: change random file init to be bool based
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 2 Nov 2017 15:00:04 +0000 (09:00 -0600)]
filesetup: __init_rand_distribution() can be void
It can no longer fail, just make it void.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 2 Nov 2017 14:59:07 +0000 (08:59 -0600)]
filesetup: pre_read_files() can use a bool
This flips the success return from 0 to true.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 2 Nov 2017 14:55:44 +0000 (08:55 -0600)]
filesetup: don't inline native_fallocate()
The compiler will figure this out, and it isn't a performance
hot path.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 2 Nov 2017 14:53:54 +0000 (08:53 -0600)]
filesetup: don't print non-debug error on native fallocate failure
We expect this to fail in cases where it isn't available, so don't
clutter the standard/error output with it.
Also ensure that native_fallocate() returns true/false, since it's
a bool function. Just a cleanup, no functional change there.
Fixes:
2c3e17be4c7c ("filesetup: add native fallocate")
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Wed, 1 Nov 2017 20:51:03 +0000 (14:51 -0600)]
io_u: reset file to initial offset
Don't assume that initial offset is 0, we should use the set
f->file_offset when resetting.
Fixes:
17373ce2f38a ("io_u: wrap to beginning when end-of-file is reached for time_based")
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Wed, 1 Nov 2017 17:02:30 +0000 (11:02 -0600)]
io_u: wrap to beginning when end-of-file is reached for time_based
The logic around using io_size isn't correct.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Tue, 31 Oct 2017 20:01:16 +0000 (14:01 -0600)]
engines/windowsaio: style fixups
No functional changes in this patch.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Tue, 31 Oct 2017 16:50:03 +0000 (10:50 -0600)]
Default buffer_compress_chunk to 512
This makes us behave better out of the box, for the cases where
people use bigger block sizes. Without this change and if the
user set buffer_compress_percentage=50 and bs=128k, then we'd
have 64k of zeroes at the end. This is probably not what the
user expects.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Tue, 31 Oct 2017 16:18:54 +0000 (10:18 -0600)]
Update compression documentation
Make it clearer (hopefully) that buffer_compress_chunk should be
used if the user wants a finer granularity than the write block
size.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Tue, 31 Oct 2017 14:55:02 +0000 (08:55 -0600)]
init: make sure that compression enables refill buffers
Otherwise the results aren't going to be great.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Mon, 30 Oct 2017 15:05:43 +0000 (09:05 -0600)]
Merge branch 'misc' of https://github.com/sitsofe/fio
Pull misc packaging fixes from Sitsofe:
"Fixes to appease stricter packaging guidelines and some documentation
changes. The big one is updating the COPYING file in fio to use the one
from https://www.gnu.org/licenses/old-licenses/gpl-2.0.txt . It's worth
double checking that it was a like-for-like replacement given the
significance of the file..."
Sitsofe Wheeler [Sun, 29 Oct 2017 16:08:51 +0000 (16:08 +0000)]
doc: rewrite write_*_log sections
- Put the big example of how log files work in write_bw_log and make the
other sections reference that option so there's less repetition.
- Add a note that the write_hist_log depends on log_hist_msec.
- Make a link from log_hist_coarseness to write_hist_log.
Signed-off-by: Sitsofe Wheeler <sitsofe@yahoo.com>
Sitsofe Wheeler [Sun, 29 Oct 2017 09:29:38 +0000 (09:29 +0000)]
doc: minor formatting fixes
- Fix broken HOWTO formatting in serialize_overlap.
- Make a link to (and correct the name of) latency_target option within
Interpreting the output, IO latency.
Signed-off-by: Sitsofe Wheeler <sitsofe@yahoo.com>
Sitsofe Wheeler [Sun, 29 Oct 2017 09:08:14 +0000 (09:08 +0000)]
doc: IO -> I/O, sync filecreate documentation
Change IO -> I/O and copy the filecreate description from the HOWTO to
the man page.
Signed-off-by: Sitsofe Wheeler <sitsofe@yahoo.com>
Sitsofe Wheeler [Sun, 29 Oct 2017 09:04:24 +0000 (09:04 +0000)]
fio: update FSF address
Signed-off-by: Sitsofe Wheeler <sitsofe@yahoo.com>
Sitsofe Wheeler [Sun, 29 Oct 2017 08:45:48 +0000 (08:45 +0000)]
COPYING: update license file
Change COPYING to use the most recent version of the GPLv2 license file
making the following changes:
- Updates the FSF address.
- Changes some formatting.
- Changes "GNU Library General Public License" to "GNU Lesser General
Public License".
Signed-off-by: Sitsofe Wheeler <sitsofe@yahoo.com>
Sitsofe Wheeler [Sun, 29 Oct 2017 08:29:12 +0000 (08:29 +0000)]
doc: fix groff line that started with a dot
It's illegal to start a groff line with a dot if it's not part of a
macro so use the workaround described on
https://stackoverflow.com/questions/
11469341/escaping-dots-in-groff/
11566859#
11566859
to solve the problem.
Signed-off-by: Sitsofe Wheeler <sitsofe@yahoo.com>
Sitsofe Wheeler [Sun, 29 Oct 2017 16:24:24 +0000 (16:24 +0000)]
fio: fix interpreter lines
Minor changes to adhere to various distro packaging guidelines (e.g.
https://fedoraproject.org/wiki/Packaging:Guidelines#Shebang_lines ):
- doc/conf.py doesn't need an interpreter line because it's not
executable.
- Make the genfio interpreter line use an absolute path rather than env.
Signed-off-by: Sitsofe Wheeler <sitsofe@yahoo.com>
Sitsofe Wheeler [Sun, 29 Oct 2017 08:05:47 +0000 (08:05 +0000)]
tools: use /usr/bin/python2.7 as the interpreter
Distros such as Fedora
(https://fedoraproject.org/wiki/Packaging:Python#Multiple_Python_Runtimes
) are clamping down on python interpreter paths due to the desire to
switch to python3 so /usr/bin/python or /usr/bin/env python are no
longer liked. Unfortunately macOS doesn't ship a /usr/bin/python2 link
so switch all python interpreter paths to /usr/bin/python2.7 as a
compromise.
Signed-off-by: Sitsofe Wheeler <sitsofe@yahoo.com>
Sitsofe Wheeler [Sun, 29 Oct 2017 07:47:36 +0000 (07:47 +0000)]
steadystate_tests.py: fix up usage comment
Make the comment about usage match the argparse line.
Signed-off-by: Sitsofe Wheeler <sitsofe@yahoo.com>
Rebecca Cran [Thu, 26 Oct 2017 21:28:23 +0000 (15:28 -0600)]
Fix 'nice' parameter range: should be -20 to 19, not -19 to 20.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 26 Oct 2017 18:08:20 +0000 (12:08 -0600)]
io_u: re-invalidate cache when looping around without file open/close
If we're doing buffered IO and we end up wrapping around for a
time based run, then ensure that we re-invalidate the kernel
cache for the file.
Reported-by: Paolo Valente <paolo.valente@linaro.org>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jeff Furlong [Thu, 26 Oct 2017 17:47:03 +0000 (11:47 -0600)]
Add offset_align option
This allows us to align the start offsets, if percentage offsets
are used.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Wed, 25 Oct 2017 19:06:40 +0000 (13:06 -0600)]
parse: minimum options values are signed
Rebecca reports that using an invalid value for the nice
parameter yields:
min value out of range:
18446744073709551596 (
4294967277 min)
where it really should be:
min value out of range: -20 (-19 min)
Print the minimum values as signed, if we hit the out-of-range
error.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Sitsofe Wheeler [Fri, 20 Oct 2017 06:08:37 +0000 (07:08 +0100)]
doc: minor formatting fixes
- Sphinx doesn't like non-punctuation characters directly after closing
backticks unless they are escaped so add some backslashes
appropriately.
- Markup /proc/sys/kernel/shmmax in --max-jobs like other /proc/
references.
Signed-off-by: Sitsofe Wheeler <sitsofe@yahoo.com>
Jens Axboe [Tue, 17 Oct 2017 18:59:40 +0000 (12:59 -0600)]
Merge branch 'cpuclock-test'
Pulling in this fix, as Sitsofe tested it and didn't see any failures.
I wasn't able to trigger the failure here myself, either before or
after the fix...
Jens Axboe [Tue, 17 Oct 2017 18:11:44 +0000 (12:11 -0600)]
gettime: improve cpu clock test
We're missing a synchronization before reading and comparing
with the current *t->seq in the loop. Rewrite the loop to
use compare-and-exchange for the increment, and ensure that
we have the proper synchronization inserted. This should both
be faster and more reliable.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 12 Oct 2017 16:54:27 +0000 (10:54 -0600)]
flow: fix bad overflowing math
No point in multiplying with a 1/-1 integer, just assign a local
variable appropriately.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 12 Oct 2017 16:31:17 +0000 (10:31 -0600)]
Merge branch 'master' of https://github.com/Venutiwa/fio
Jens Axboe [Thu, 12 Oct 2017 16:22:55 +0000 (10:22 -0600)]
Merge branch 'overflow_fix' of https://github.com/sitsofe/fio
Sitsofe Wheeler [Thu, 12 Oct 2017 16:09:48 +0000 (17:09 +0100)]
gettime: fix cycles_per_msec overflow when using 32 bit longs
Compiling fio with clang's undefined behaviour sanitizer and unsigned
wraparound detection enabled on a 32 bit Linux build turned up the
following:
gettime.c:313:28: runtime error: unsigned integer overflow: 3600 *
2600730 cannot be represented in type 'unsigned long'
Fix this by making cycles_per_msec a long long.
Signed-off-by: Sitsofe Wheeler <sitsofe@yahoo.com>
Andrzej Jakowski [Thu, 12 Oct 2017 14:31:06 +0000 (08:31 -0600)]
Fix more overflows in percentile calculation for Windows
The previous patch caught one place where we could potentially
overflow for large number of IOs, this patches up a few more.
Signed-off-by: Andrzej Jakowski <andrzej.jakowski@intel.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Andrzej Jakowski [Wed, 11 Oct 2017 22:26:00 +0000 (16:26 -0600)]
Fix overflow in percentile calculation for Windows
This patch fixes percentile calculation for Windows where overflow
on 32 bit type happened while calculating actual percentiles.
Signed-off-by: Andrzej Jakowski <andrzej.jakowski@intel.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Wed, 11 Oct 2017 20:52:32 +0000 (14:52 -0600)]
fio: kill td_ioengine_flags()
It's unused.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Wed, 11 Oct 2017 20:25:09 +0000 (14:25 -0600)]
Windows mkdir() fix
Apparently mingw mkdir() only takes the path as an argument,
it doesn't include a permission mode...
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Josef Bacik [Wed, 11 Oct 2017 20:15:19 +0000 (16:15 -0400)]
add documentation about filename_format directory behavior
We've changed the behavior of filename_format, document that fact.
Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Josef Bacik [Wed, 11 Oct 2017 20:15:18 +0000 (16:15 -0400)]
use mkdir instead of mkdirat
Windows doesn't have a way to do mkdirat, so convert to mkdir instead.
Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Wed, 11 Oct 2017 19:07:34 +0000 (13:07 -0600)]
Fix broken path separator definition on Windows
Fixes:
53a7af851836 ("convert FIO_OS_PATH_SEPARATOR to a character")
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Josef Bacik [Wed, 11 Oct 2017 18:58:12 +0000 (14:58 -0400)]
create subdirs if specified in the filename_format
Currently there's no way for a user to specify a directory that doesn't
exist yet for a job. Fix this by looking for paths in the file name
provided by the job and creating any subdirectories that may be
required.
Signed-off-by: Josef Bacik <jbacik@fb.com>
Removed TD_ENG_FLAG_SHIFT change, not needed.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Josef Bacik [Wed, 11 Oct 2017 18:58:11 +0000 (14:58 -0400)]
convert FIO_OS_PATH_SEPARATOR to a character
Instead of a string let's use this as a character so we can search for
it in path names.
Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Wed, 11 Oct 2017 18:28:50 +0000 (12:28 -0600)]
Error if td flags overlap with engine flags
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Wed, 11 Oct 2017 18:26:37 +0000 (12:26 -0600)]
fio: rearrange TD_F_ flag logic
In preparation for checking if we need to bump TD_ENG_FLAG_SHIFT.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Wed, 11 Oct 2017 18:23:28 +0000 (12:23 -0600)]
fio: kill unused TD_F_ flag
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Wed, 11 Oct 2017 14:26:28 +0000 (08:26 -0600)]
Merge branch 'fgp_fixes' of https://github.com/sitsofe/fio
Jens Axboe [Wed, 11 Oct 2017 14:25:10 +0000 (08:25 -0600)]
engines/windowsaio: style
Don't use braces for a single line block.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Wed, 11 Oct 2017 14:24:54 +0000 (08:24 -0600)]
Merge branch 'windowaio_invalidate' of https://github.com/sitsofe/fio
Jens Axboe [Wed, 11 Oct 2017 14:23:41 +0000 (08:23 -0600)]
Merge branch 'ci_and_configure' of https://github.com/sitsofe/fio
Justin Eno [Tue, 10 Oct 2017 17:54:54 +0000 (11:54 -0600)]
backend: don't dereference ->io_ops in reap_threads()
Without some type of mutex, td->io_ps cannot be dereferenced safely in
reap_threads().
Signed-off-by: Justin Eno <jeno@micron.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Tue, 10 Oct 2017 16:46:07 +0000 (10:46 -0600)]
blktrace: use for_each_file()
Instead of open-coding its own version.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Tue, 10 Oct 2017 14:27:38 +0000 (08:27 -0600)]
engines/filecreate: set data direction for stats
When we init the engine, use whatever data direction the job has
for the latency we log. This means that we're no longer dependent
on the job file setting rw=read/randread or not setting it at all.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Tue, 10 Oct 2017 14:12:59 +0000 (08:12 -0600)]
HOWTO: include note about increasing shared memory limits
Similarly to commit
7f4811bb36d3, but for the HOWTO.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Tue, 10 Oct 2017 14:12:05 +0000 (08:12 -0600)]
Merge branch 'pr/note-for-shmmax' of https://github.com/taghos/fio
Ricardo Nabinger Sanchez [Mon, 9 Oct 2017 23:57:27 +0000 (20:57 -0300)]
Add note for increasing shmmax if necessary
When launching a large job, fio can attempt to grab more shared memory
than allowed by the kernel. Hint the user about increasing such limit
on Linux systems.
Signed-off-by: Ricardo Nabinger Sanchez <rnsanchez@taghos.com.br>