Alex Pyrgiotis [Sun, 10 Mar 2013 09:58:05 +0000 (10:58 +0100)]
lfsr: Fix spin related bug
On the previous patch on cycle detection, we would incremet spin value
when:
num_vals % cycle_length == 0
which would evaluate always to true for the first iteration since we
start with num_vals = 0. Only one corner-case is affected by this bug
and this patch should fix it.
Signed-off-by: Alex Pyrgiotis <apyrgio@grnet.gr>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Fri, 8 Mar 2013 18:12:15 +0000 (19:12 +0100)]
lfsr: indent table
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Alex Pyrgiotis [Fri, 8 Mar 2013 12:37:04 +0000 (14:37 +0200)]
Add a simple test for LFSR generator
Adds a simple test suite to check the speed of the LFSR
generator and verify its results. Just run:
make t/lfsr-test
./t/lfsr-test
to compile the test suite and print its usage
Signed-off-by: Alex Pyrgiotis <apyrgio@grnet.gr>
create mode 100644 t/lfsr-test.c
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Alex Pyrgiotis [Fri, 8 Mar 2013 12:37:03 +0000 (14:37 +0200)]
Improve LFSR implementation
Changes:
1. Use Galois LFSR instead of Fibonacci LFSR.
2. Use XNOR gates instead of XOR gates.
3. Add tap sizes for LFSRs ranging from 3-bits to 15-bits.
4. Add spin parameter.
Rationale:
1. Fibonacci LFSRs have the following drawbacks:
a. Their next state can not be computed in one cycle, since the input
bit must be propagated serially through the XOR gates. Galois
LFSRs however, can be computed instantly with XOR-masks.
b. Their state changes cannot be considered "irregular", even by I/O
standards. Actually, if the current state of an n-bit LFSR is x,
then the next will either be (x >> 1) or (2^n + (x >> 1)). Galois
LFSRs have instead their XOR gates interleaved with their bits,
which means that the inner bits are changed as well, besides of
the shifting. If the number of taps is z, this means that the
different outcomes are 2^(z + 1).
2. An LFSR with XOR gates has the all-zeroes state as illegal. Since
zero is valid for most I/O operations, it would be more intuitive to
use XNOR gates, that have as the all-ones state as illegal.
3. Allow smaller I/O benchmarks to use LFSRs.
4. The spin parameter follows the same rationale as in 1b. To make the
LFSR outcome "appear" less predictable, we can spin internally the
LFSR state and produce the i-th number. To understand the spin
parameter, consider the following state sequence of a 3-bit LFSR:
0, 2, 3, 5, 6, 1, 4
Same LFSR, spin value 2: 0, 3, 6, 4, 2, 5, 1
But what is the benefit from using spin? Well, the benefits are
two-fold:
a. For the same I/O size, we can create a different I/O sequences.
b. Following the rationale of 1b, we can have more variable outcomes.
If the spin value is "i" and the taps are "z", the number of
different outcomes become i * 2^(z + 1).
Signed-off-by: Alex Pyrgiotis <apyrgio@grnet.gr>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
SEOKYOUNG KO [Thu, 7 Mar 2013 11:34:31 +0000 (12:34 +0100)]
Fix skewed latencies for rate IOPS
- when -rate_iops is specified, FIO periodically calls usleep() to limit IOPS
- Before usleep(), FIO always wait until the completion of all pending I/O
- For all I/O completions, FIO shows erroneous behavior of logging their
latency, with that of the longest latency IO sample.
- w/ QD=8/ mixed R:W=33:66, up to 7 samples may get false latency log when
waiting all I/O completions => False report of latency distribution
Signed-off-by: Jens Axboe <axboe@kernel.dk>
SEOKYOUNG KO [Thu, 7 Mar 2013 11:31:04 +0000 (12:31 +0100)]
Fix rate iops with ramp time
When the ramp time is over, fio resets the ->io_issues[]
counters, but not the rwmix counters. This causes a problem
for mixed read/write workloads that are rate limited and
include a ramp time.
Fix it by resetting rwmix_issues with the regular issues.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 7 Mar 2013 10:31:20 +0000 (11:31 +0100)]
Add --parse-only command line option
Sometimes it's useful just being able to verify whether a job
config is valid or not, without actually starting any jobs.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 28 Feb 2013 21:13:29 +0000 (22:13 +0100)]
configure: fixup __sync_fetch_and_add()
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 28 Feb 2013 21:10:22 +0000 (22:10 +0100)]
configure: add general libs for Solaris
Assume we just need them, this makes us find inet_aton()
on Solaris too.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 28 Feb 2013 19:34:31 +0000 (20:34 +0100)]
solaris: ensure that TCP_NODELAY gets picked up
Use proper libs when linking, then we can also remove
those from the Makefile.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 28 Feb 2013 19:08:12 +0000 (20:08 +0100)]
Merge branch 'master' of ssh://brick.kernel.dk/data/git/fio
Steven Noonan [Thu, 28 Feb 2013 19:08:06 +0000 (20:08 +0100)]
powerpc: use mfspr for Time Base Register reads
The 'mftb' and 'mftbu' instructions are deprecated in Power ISA v2.06 in favor
of mfspr. Since CPUs supporting mftb/mftbu also support the TBR SPRs, there's
no known reason to keep mftb/mftbu.
Signed-off-by: Steven Noonan <steven@uplinklabs.net>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 28 Feb 2013 07:28:05 +0000 (08:28 +0100)]
Fix compile of RDMA engine for SunOS 5.x
- byteswap.h include that doesn't exist there. Kill it, we
have no use for it.
- No RLIMIT_MEMLOCK on SunOS. The use in the engine is a bit
suspect, so not a problem if we don't have it. Add configure
check for that.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 28 Feb 2013 07:24:23 +0000 (08:24 +0100)]
net: fix warning for systems without TCP_NODELAY
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Wed, 27 Feb 2013 11:32:43 +0000 (12:32 +0100)]
Revert "debug: only do getpid() if we have to"
This reverts commit
8037423f08649a7c3378dadf95327d6553105264.
Jens Axboe [Tue, 26 Feb 2013 20:59:33 +0000 (21:59 +0100)]
FIO-VERSION-GEN: use `foo` instead of $(foo)
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Tue, 26 Feb 2013 20:55:43 +0000 (21:55 +0100)]
Merge branch 'master' of ssh://brick.kernel.dk/data/git/fio
Jens Axboe [Tue, 26 Feb 2013 20:55:08 +0000 (21:55 +0100)]
configure: use `foo` instead of $(foo)
Seems to work everywhere, whereas $(foo) does not work on
some variants of FreeBSD.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Tue, 26 Feb 2013 15:11:49 +0000 (16:11 +0100)]
powerpc: harden the clock ATB probe a bit
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Tue, 26 Feb 2013 13:43:02 +0000 (14:43 +0100)]
powerpc: use ATB clock, if it's available
The ATB is more precise, but we don't know if it's available.
Check by forking off a process that attempts to use it, if
we die with SIGILL, then we know it's not there.
Based on code from Steven Noonan.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Tue, 26 Feb 2013 12:54:20 +0000 (13:54 +0100)]
gettime: print 64-bit variable with ULL
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Tue, 26 Feb 2013 07:16:43 +0000 (08:16 +0100)]
debug: make __dprint() have printf() like argument type checking
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Mon, 25 Feb 2013 13:02:42 +0000 (14:02 +0100)]
debug: only do getpid() if we have to
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Mon, 25 Feb 2013 09:23:58 +0000 (10:23 +0100)]
gettime: use 32-bit atomic sequences
Not all platforms have 64-bit wide atomic sync_and_fetch(). If
we just check for overflow, it should be OK to use a 32-bit
sequence number.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Mon, 25 Feb 2013 09:18:33 +0000 (10:18 +0100)]
Fixup wrong types for dprint()
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Sun, 24 Feb 2013 20:29:35 +0000 (21:29 +0100)]
gettime: add some sanity checks to platform clock
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Sun, 24 Feb 2013 13:24:16 +0000 (14:24 +0100)]
Merge branch 'master' of ssh://brick.kernel.dk/data/git/fio
Bruce Cran [Sun, 24 Feb 2013 11:15:26 +0000 (11:15 +0000)]
Add LICENSE to Windows installer, fix filename extensions and update URL.
Add the LICENSE file to the Windows installer, change installed filename
extensions from .rtf to .txt (assume a text editor is installed that
understands LF) and fix the "update url" to be http://bluestop.org/fio/
since that's where new Windows installers are.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Bruce Cran [Sun, 24 Feb 2013 11:09:11 +0000 (11:09 +0000)]
Add section 9 to HOWTO table of contents.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Fri, 22 Feb 2013 19:48:56 +0000 (20:48 +0100)]
Add --eta-newline option
For certain situations, it's handy to force a new line in
the ETA output. It allows you to easily track what
happened, without having to do detailed logging.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 21 Feb 2013 18:12:32 +0000 (19:12 +0100)]
Fio 2.0.14
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 21 Feb 2013 14:18:17 +0000 (15:18 +0100)]
Fix problem with terminating on unaligned sizes
Three separate little issues:
- Don't round up the number of blocks needed. That means we end
up with potential partial blocks, which we can never do IO to.
- Fix an axmap bug where we only check against the specific bit
in the mask, but we really want to check for "this bit or any
higher bit". This makes axmap_next_free() behave more like it
should, instead of failing way too often.
- For the almost-full case, we can return a next available bit
that is just outside the allowed range. Punt to first-free for
that case.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Bruce Cran [Thu, 21 Feb 2013 12:39:24 +0000 (12:39 +0000)]
Add FreeBSD cpu affinity support.
Since FreeBSD has different thread and process IDs, assume that fio_setaffinity
will always be called with a tid and fio_getaffinity with a pid.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Bruce Cran [Thu, 21 Feb 2013 13:16:17 +0000 (14:16 +0100)]
Declare 'prev' and 'this' outside the loop to avoid clang warning about 'prev' being uninitialized
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 21 Feb 2013 11:57:56 +0000 (12:57 +0100)]
Fixup ia64 32-bit register clock cycle read
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 21 Feb 2013 11:39:42 +0000 (12:39 +0100)]
Fix x86 32-bit wrong variable type for rdtsc
Bruce points out, that we are putting the result into a 32-bit
variable. That is incorrect, make it a proper ULL instead.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 21 Feb 2013 10:03:28 +0000 (11:03 +0100)]
README: kill reference to non-functional SUSE package page
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Bruce Cran [Wed, 20 Feb 2013 20:35:16 +0000 (20:35 +0000)]
Add recommendation that fio be run in a native console on Windows.
Windows console applications often don't run correctly under Cygwin shells such
as mintty because of the way they're implemented. Add a recommendation to the
README to run fio under a native console such as the Command Prompt or console2
to avoid these problems.
http://code.google.com/p/mintty/issues/detail?id=56 documents the problem.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Bruce Cran [Thu, 21 Feb 2013 10:02:41 +0000 (11:02 +0100)]
Update all Windows files except dobuild.cmd to use LF line endings
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Huadong Liu [Thu, 21 Feb 2013 09:24:20 +0000 (10:24 +0100)]
check output_format before calling show_idle_prof_stats()
Do the proper checks for output format type and whether we are
a server backend or not.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 21 Feb 2013 09:23:51 +0000 (10:23 +0100)]
t/axmap: add pre/post bit set checks
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Mon, 18 Feb 2013 19:09:28 +0000 (20:09 +0100)]
Merge branch 'master' of ssh://brick.kernel.dk/data/git/fio
Nikolaus Jeremic [Mon, 18 Feb 2013 18:44:41 +0000 (19:44 +0100)]
Fix replay of IO pattern with multiple open files
Nikolaus writes:
I've tried to replay I/Os from a textual iolog file to a couple of files
using fio 2.0.13 on Linux. However, all I/Os were performed to the last
opened file, which is '/mnt/ssd5/rand-write.1.0_1_2' in this example .
My jobfile and iolog files contain the following:
[global]
name=replay_filetraces
description=Replay file traces
ioengine=libaio
iodepth=6
direct=1
continue_on_error=1
[replay]
numjobs=1
read_iolog=file_io_patterns
unlink=0
fio version 2 iolog
/mnt/ssd0/rand-write.1.0_0_0 add
/mnt/ssd1/rand-write.1.0_0_1 add
/mnt/ssd2/rand-write.1.0_0_2 add
/mnt/ssd3/rand-write.1.0_1_0 add
/mnt/ssd4/rand-write.1.0_1_1 add
/mnt/ssd5/rand-write.1.0_1_2 add
/mnt/ssd0/rand-write.1.0_0_0 open
/mnt/ssd1/rand-write.1.0_0_1 open
/mnt/ssd2/rand-write.1.0_0_2 open
/mnt/ssd3/rand-write.1.0_1_0 open
/mnt/ssd4/rand-write.1.0_1_1 open
/mnt/ssd5/rand-write.1.0_1_2 open
/mnt/ssd0/rand-write.1.0_0_0 write 0 65536
/mnt/ssd1/rand-write.1.0_0_1 write 0 65536
/mnt/ssd2/rand-write.1.0_0_2 write 0 65536
/mnt/ssd3/rand-write.1.0_1_0 write 0 65536
/mnt/ssd4/rand-write.1.0_1_1 write 0 65536
/mnt/ssd5/rand-write.1.0_1_2 write 0 65536
/mnt/ssd0/rand-write.1.0_0_0 write 65536 65536
/mnt/ssd1/rand-write.1.0_0_1 write 65536 65536
/mnt/ssd2/rand-write.1.0_0_2 write 65536 65536
/mnt/ssd3/rand-write.1.0_1_0 write 65536 65536
/mnt/ssd4/rand-write.1.0_1_1 write 65536 65536
/mnt/ssd5/rand-write.1.0_1_2 write 65536 65536
/mnt/ssd0/rand-write.1.0_0_0 close
/mnt/ssd1/rand-write.1.0_0_1 close
/mnt/ssd2/rand-write.1.0_0_2 close
/mnt/ssd3/rand-write.1.0_1_0 close
/mnt/ssd4/rand-write.1.0_1_1 close
/mnt/ssd5/rand-write.1.0_1_2 close
I've turned on debugging and added some extra printf in read_iolog2()
within iolog.c.
file 24594 add file /mnt/ssd0/rand-write.1.0_0_0
file 24594 resize file array to 2 files
file 24594 file 0x7fbe03f53090 "/mnt/ssd0/rand-write.1.0_0_0" added at 0
file 24594 add file /mnt/ssd1/rand-write.1.0_0_1
file 24594 file 0x7fbe03f53230 "/mnt/ssd1/rand-write.1.0_0_1" added at 1
file 24594 add file /mnt/ssd2/rand-write.1.0_0_2
file 24594 resize file array to 4 files
file 24594 file 0x7fbe03f533d0 "/mnt/ssd2/rand-write.1.0_0_2" added at 2
file 24594 add file /mnt/ssd3/rand-write.1.0_1_0
file 24594 file 0x7fbe03f53570 "/mnt/ssd3/rand-write.1.0_1_0" added at 3
file 24594 add file /mnt/ssd4/rand-write.1.0_1_1
file 24594 resize file array to 6 files
file 24594 file 0x7fbe03f53710 "/mnt/ssd4/rand-write.1.0_1_1" added at 4
file 24594 add file /mnt/ssd5/rand-write.1.0_1_2
file 24594 file 0x7fbe03f538b0 "/mnt/ssd5/rand-write.1.0_1_2" added at 5
io 24594 read_iolog2: Making note of file.
offset=
7310584035475809390 bytes=
811544671 fileno=0 file_action=1
fname=/mnt/ssd0/rand-write.1.0_0_0 act=open
io 24594 read_iolog2: Making note of file.
offset=
7310584035475809390 bytes=
811544671 fileno=1 file_action=1
fname=/mnt/ssd1/rand-write.1.0_0_1 act=open
io 24594 read_iolog2: Making note of file.
offset=
7310584035475809390 bytes=
811544671 fileno=2 file_action=1
fname=/mnt/ssd2/rand-write.1.0_0_2 act=open
io 24594 read_iolog2: Making note of file.
offset=
7310584035475809390 bytes=
811544671 fileno=3 file_action=1
fname=/mnt/ssd3/rand-write.1.0_1_0 act=open
io 24594 read_iolog2: Making note of file.
offset=
7310584035475809390 bytes=
811544671 fileno=4 file_action=1
fname=/mnt/ssd4/rand-write.1.0_1_1 act=open
io 24594 read_iolog2: Making note of file.
offset=
7310584035475809390 bytes=
811544671 fileno=5 file_action=1
fname=/mnt/ssd5/rand-write.1.0_1_2 act=open
io 24594 read_iolog2: Making note of file. offset=0 bytes=65536
fileno=5 file_action=1 fname=/mnt/ssd0/rand-write.1.0_0_0 act=write
io 24594 read_iolog2: Making note of file. offset=0 bytes=65536
fileno=5 file_action=1 fname=/mnt/ssd1/rand-write.1.0_0_1 act=write
io 24594 read_iolog2: Making note of file. offset=0 bytes=65536
fileno=5 file_action=1 fname=/mnt/ssd2/rand-write.1.0_0_2 act=write
io 24594 read_iolog2: Making note of file. offset=0 bytes=65536
fileno=5 file_action=1 fname=/mnt/ssd3/rand-write.1.0_1_0 act=write
io 24594 read_iolog2: Making note of file. offset=0 bytes=65536
fileno=5 file_action=1 fname=/mnt/ssd4/rand-write.1.0_1_1 act=write
io 24594 read_iolog2: Making note of file. offset=0 bytes=65536
fileno=5 file_action=1 fname=/mnt/ssd5/rand-write.1.0_1_2 act=write
io 24594 read_iolog2: Making note of file. offset=65536
bytes=65536 fileno=5 file_action=1 fname=/mnt/ssd0/rand-write.1.0_0_0
act=write
io 24594 read_iolog2: Making note of file. offset=65536
bytes=65536 fileno=5 file_action=1 fname=/mnt/ssd1/rand-write.1.0_0_1
act=write
io 24594 read_iolog2: Making note of file. offset=65536
bytes=65536 fileno=5 file_action=1 fname=/mnt/ssd2/rand-write.1.0_0_2
act=write
io 24594 read_iolog2: Making note of file. offset=65536
bytes=65536 fileno=5 file_action=1 fname=/mnt/ssd3/rand-write.1.0_1_0
act=write
io 24594 read_iolog2: Making note of file. offset=65536
bytes=65536 fileno=5 file_action=1 fname=/mnt/ssd4/rand-write.1.0_1_1
act=write
io 24594 read_iolog2: Making note of file. offset=65536
bytes=65536 fileno=5 file_action=1 fname=/mnt/ssd5/rand-write.1.0_1_2
act=write
io 24594 read_iolog2: Making note of file. offset=65536
bytes=65536 fileno=0 file_action=2 fname=/mnt/ssd0/rand-write.1.0_0_0
act=close
io 24594 read_iolog2: Making note of file. offset=65536
bytes=65536 fileno=1 file_action=2 fname=/mnt/ssd1/rand-write.1.0_0_1
act=close
io 24594 read_iolog2: Making note of file. offset=65536
bytes=65536 fileno=2 file_action=2 fname=/mnt/ssd2/rand-write.1.0_0_2
act=close
io 24594 read_iolog2: Making note of file. offset=65536
bytes=65536 fileno=3 file_action=2 fname=/mnt/ssd3/rand-write.1.0_1_0
act=close
io 24594 read_iolog2: Making note of file. offset=65536
bytes=65536 fileno=4 file_action=2 fname=/mnt/ssd4/rand-write.1.0_1_1
act=close
io 24594 read_iolog2: Making note of file. offset=65536
bytes=65536 fileno=5 file_action=2 fname=/mnt/ssd5/rand-write.1.0_1_2
act=close
Taking a look to the debugging output shows that on write action the
'fileno' variable is always set to 5 regardless of the file that should
be written. However, for open and close we have a different fileno value
for each file. Taking a look into the source code of read_iolog2()
reveals why: In case of actions 'open' or 'close', 'fileno' is looked up
through 'get_fileno(td, fname)', but for 'write', 'read' and the other 4
argument actions 'fileno' is not looked up. When I add a 'fileno =
get_fileno(td, fname);' at the end of the '(r == 4)' branch, I/Os are
replayed to the designated files.
if (r == 4) {
/*
* Check action first
*/
if (!strcmp(act, "wait"))
rw = DDIR_WAIT;
else if (!strcmp(act, "read"))
rw = DDIR_READ;
else if (!strcmp(act, "write"))
rw = DDIR_WRITE;
else if (!strcmp(act, "sync"))
rw = DDIR_SYNC;
else if (!strcmp(act, "datasync"))
rw = DDIR_DATASYNC;
else if (!strcmp(act, "trim"))
rw = DDIR_TRIM;
else {
log_err("fio: bad iolog file action: %s\n",
act);
continue;
}
} else if (r == 2) {
rw = DDIR_INVAL;
if (!strcmp(act, "add")) {
td->o.nr_files++;
fileno = add_file(td, fname);
file_action = FIO_LOG_ADD_FILE;
continue;
} else if (!strcmp(act, "open")) {
fileno = get_fileno(td, fname);
file_action = FIO_LOG_OPEN_FILE;
} else if (!strcmp(act, "close")) {
fileno = get_fileno(td, fname);
file_action = FIO_LOG_CLOSE_FILE;
} else {
log_err("fio: bad iolog file action: %s\n",
act);
continue;
}
} else {
log_err("bad iolog2: %s", p);
continue;
}
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Aaron Carroll [Mon, 18 Feb 2013 07:52:18 +0000 (08:52 +0100)]
configure: missing Android config options
We need this for the run-time endianness check to pass. Add supported
clock monotonic while we're at it.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Aaron Carroll [Tue, 12 Feb 2013 08:58:14 +0000 (09:58 +0100)]
configure: attempt to get Android going again
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Mon, 11 Feb 2013 15:33:25 +0000 (16:33 +0100)]
Initialize global and non-static variables
Don't assume they get zeroed. The AIX linker throws a hissy fit on
these, refusing to link the program.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Mon, 11 Feb 2013 13:35:43 +0000 (14:35 +0100)]
Turn f->engine_data into a 64-bit value
Even on 32-bit platforms, we store (uint64_t) -1 in there
for some cases to make it be 'invalid'.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Mon, 11 Feb 2013 12:21:54 +0000 (13:21 +0100)]
configure: ensure that fatal errors kill config-host.h and mak files
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Mon, 11 Feb 2013 10:26:55 +0000 (11:26 +0100)]
Handle end_fsync if a file is closed
Should not happen, but better be careful since it could have
an impact on data integrity.
Additionally, add a log_err() error if we do fail it.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Sun, 10 Feb 2013 14:48:34 +0000 (15:48 +0100)]
configure: allow --cpu to be specified
And don't fail if we don't detect, it's not critical
for configure.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Fri, 8 Feb 2013 18:05:25 +0000 (19:05 +0100)]
Fix rate limiting
Commit
e0224c6b accidentally broke rate limiting, since
it bypassed the sleep logic. Rework it slightly to not
skip that.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Bruce Cran [Fri, 8 Feb 2013 01:45:17 +0000 (01:45 +0000)]
Windows: io cancellation often fails and causes crashes, so remove it.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 7 Feb 2013 18:55:24 +0000 (19:55 +0100)]
Fix bug with rate and read/write mixed workloads at 100% bias
The rate_ddir() might sometimes decide to send the other data
direction, even if we have a workload with rw=randrw and
rwmixread=0. Check for this condition.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 7 Feb 2013 14:40:48 +0000 (15:40 +0100)]
t/log: include minmax.h for min()
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 7 Feb 2013 14:40:37 +0000 (15:40 +0100)]
Makfile: use LINK for the link phase
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 7 Feb 2013 12:06:18 +0000 (13:06 +0100)]
Fix zones for numjobs=x, where x > 1
In the options fixup, we auto-set open_files to 1 if it hasn't been set.
So any duplicated jobs will clear the zone. Fixup the check to properly
check for > 1 open files.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 7 Feb 2013 11:56:09 +0000 (12:56 +0100)]
stat: fix wrong type used for 32-bit compiles
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 7 Feb 2013 11:54:10 +0000 (12:54 +0100)]
Fix crash and precision of ETA with zones
If zonesize was bigger than zoneskip, we could encounter a divide by
zero when calculating the number of bytes.
Additionally, the math was just wrong for most cases of zone settings.
Improve that.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Bruce Cran [Wed, 6 Feb 2013 23:55:46 +0000 (23:55 +0000)]
Windows: fix mlock, remove ftruncate and fix error handling.
Functions in posix.c need to use win_to_posix_err to convert errors from
Windows API calls to their equivalent POSIX error code so perror etc.
work: move it into posix.c and add the prototype to posix.h.
Fix error handling so -1 is always returned on error and errno is set.
Fix mlock to work with sizes greater than about 1MB: the working set size
needs to be increased for the new allocation.
Remove ftruncate from posix.c, since MinGW already provides it.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Bruce Cran [Wed, 6 Feb 2013 20:27:27 +0000 (20:27 +0000)]
Windows: fix sysconf(_SC_PHYS_PAGES).
sysconf(_SC_PHYS_PAGES) was returning the number of bytes of memory
instead of the number of pages. Also improve some error messages in
sysconf(). Update os_phys_mem() to use sysconf().
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Wed, 6 Feb 2013 12:52:12 +0000 (13:52 +0100)]
Get rid of fallocate on Windows
Fallocate is only useful if it is a fast operation, helping the
file system allocate and setup meta data for the given size.
So don't punt to zero filling the entire thing.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Wed, 6 Feb 2013 07:51:57 +0000 (08:51 +0100)]
Handle normal output wrapping of the percentile list
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Wed, 6 Feb 2013 07:46:55 +0000 (08:46 +0100)]
Cleanup the percentile output formatting
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Vincent Kang Fu [Wed, 6 Feb 2013 07:43:40 +0000 (08:43 +0100)]
Handle percentile lists with higher precision that 2 digits
We cap the output at %2.2f right now, that's not always enough.
Make the parser check and store the precision required to
output the list correctly.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Bruce Cran [Tue, 5 Feb 2013 17:44:19 +0000 (17:44 +0000)]
Add --build-32bit-win switch to configure --help output.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Bruce Cran [Tue, 5 Feb 2013 11:33:04 +0000 (11:33 +0000)]
windowsaio: fix file header format and improve error reporting.
Use the same file header format as other engines.
Remove the copyright line.
Add calls to log_err to improve error reporting.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Tue, 5 Feb 2013 10:03:12 +0000 (11:03 +0100)]
Makefile: fixup init.c dependency
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Huadong Liu [Tue, 5 Feb 2013 07:43:14 +0000 (08:43 +0100)]
Enable forced 32-bit build on Windows
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Bruce Cran [Tue, 5 Feb 2013 00:56:59 +0000 (00:56 +0000)]
Fix copy/paste error in windowsaio register/unregister function names.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Bruce Cran [Mon, 4 Feb 2013 12:50:51 +0000 (13:50 +0100)]
Update the Windows installer
Update the EULA copyright year to 2013, auto-generate the product ID,
remove the comment field and allow per-user installs.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Sun, 3 Feb 2013 19:54:26 +0000 (20:54 +0100)]
gettime: fixup AMD constant TSC detection
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Sun, 3 Feb 2013 13:20:44 +0000 (14:20 +0100)]
Fix failure to exit IO loop on some IO sizes
If the size of a file isn't a multiple of the block size being
used, it can cause fio to exit the IO loop, check bytes done,
and then decide to do one more loop since we didn't do quite
as much IO as we wanted to. This happens because the minimum
block size is larger than the remainder. So check for that,
and stop if we need to.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Bruce Cran [Sat, 2 Feb 2013 15:12:41 +0000 (15:12 +0000)]
Update the Windows section of the README file.
The 64-bit Windows build instructions can be simplified since Cygwin
includes all the required MinGW64 packages. Also, the pthread DLL is
available as an individual download.
Update the WiX Toolset version to 3.7 and fix the URL since it's no
longer hosted at Sourceforge.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Bruce Cran [Sat, 2 Feb 2013 20:59:31 +0000 (20:59 +0000)]
Remove duplicated Windows configure options.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Bruce Cran [Sat, 2 Feb 2013 20:54:36 +0000 (20:54 +0000)]
Default to CS_GTOD if CONFIG_CLOCK_GETTIME isn't defined.
Defaulting the clock source to CS_CGETTIME broke on OS X because it
doesn't have clock_gettime. Use CS_GTOD if CONFIG_CLOCK_GETTIME isn't
defined.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Bruce Cran [Sat, 2 Feb 2013 14:31:24 +0000 (14:31 +0000)]
values.h is obsolete: use float.h and DBL_MIN/MAX instead.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Sat, 2 Feb 2013 09:05:07 +0000 (10:05 +0100)]
clock: hardwire tsc as unreliable on Solaris for now
Need to double check the cpuid test, it probably only is reliable
on Intel. Need to check the CPU vendor and flags appropriately.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Sat, 2 Feb 2013 08:56:23 +0000 (09:56 +0100)]
parse: use MIN/MAXDOUBLE instead of some representation of NAN
Doesn't always work for C99.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Vincent Kang Fu [Sat, 2 Feb 2013 08:28:55 +0000 (09:28 +0100)]
Fix percentile_list option
Commit
802ad4a8 broke the parsing of the percentile list,
by neglecting to change the parser type fo a fio_fp64_t.
Also get rid of the def_percentile_list, just set the
default percentile list as the option default.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Fri, 1 Feb 2013 14:51:23 +0000 (15:51 +0100)]
Makefile: fixup dependency problem for files in sub dirs
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Neto, Antonio Jose Rodrigues [Fri, 1 Feb 2013 07:30:38 +0000 (08:30 +0100)]
README: how to compile fio on Windows 64 bits
Signed-off-by: Jens Axboe <axboe@kernel.dk>
David M. Lee [Thu, 31 Jan 2013 17:37:41 +0000 (18:37 +0100)]
Allow override of CFLAGS
If you attempt to specify your own CFLAGS on the command line (such as
make CFLAGS="--whatever"), this would prevent -DFIO_VERSION from being
added to CFLAGS. This patch uses the override directive to allow the
flag to be appended to CFLAGS.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 31 Jan 2013 14:24:34 +0000 (15:24 +0100)]
Only disable stdout ETA output if results are going to stdout
If they are being redirected to a file with --output, then
we can still use the normal runtime ETA output on stdout.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 31 Jan 2013 12:33:45 +0000 (13:33 +0100)]
net: 'nodelay' HOWTO/man page update
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 31 Jan 2013 09:19:51 +0000 (10:19 +0100)]
configure: add TCP_NODELAY check
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 31 Jan 2013 08:58:22 +0000 (09:58 +0100)]
windows: fix typo in <netinet/tcp.h> header
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Ken Raeburn [Wed, 30 Jan 2013 21:31:09 +0000 (22:31 +0100)]
Fix bugs in [v]snprintf usage
When calling snprintf, supply the full buffer size instead of
one byte less.
When using the returned length from vsnprintf for logging, don't write
more than the actual buffer size (minus one for the trailing \0), in
case the formatted string was truncated.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Wed, 30 Jan 2013 20:54:58 +0000 (21:54 +0100)]
Add empty <netinet/tcp.h> for Windows
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Wed, 30 Jan 2013 20:52:37 +0000 (21:52 +0100)]
net: fix compile warning on Windows (and others)
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Steven Noonan [Wed, 28 Nov 2012 22:52:36 +0000 (14:52 -0800)]
net engine: implement option "nodelay" for TCP sockets
With disks, O_DIRECT effectively bypasses all buffering/caching mechanisms and
ensures that the I/O is going directly to the disk. Since TCP is a streaming
protocol (like disk I/O), it also has a buffering mechanism. As with disks, it
is sometimes desirable to bypass buffering. To that end, we can use
TCP_NODELAY, which transmits the packet as soon as data is assembled,
regardless of whether it occupies a full frame.
Signed-off-by: Steven Noonan <snoonan@amazon.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Wed, 30 Jan 2013 13:08:00 +0000 (14:08 +0100)]
idletime: style cleanups
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Wed, 30 Jan 2013 12:58:58 +0000 (13:58 +0100)]
configure: add SCHED_IDLE check
We can't consider it always available on Linux. It fails on
Debian 6.x, for instance.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Huadong Liu [Wed, 30 Jan 2013 12:22:24 +0000 (13:22 +0100)]
Add support for idletime profiling
Idletime profiling allows a benchmark to run while filling the
idle cycles on the server, hence giving you some metric of how
much pressure the system is under. This is useful to be able
to profile and compare changes in a setup or driver, for instance.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Wed, 30 Jan 2013 11:56:23 +0000 (12:56 +0100)]
Add unified_rw_reporting option
If this is set, then fio will sum and display just a single set
of statistics for any IO type. By default, fio accounts and reports
each data direction separately.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Wed, 30 Jan 2013 11:53:55 +0000 (12:53 +0100)]
HOWTO: indentation fix
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Tue, 29 Jan 2013 21:29:09 +0000 (22:29 +0100)]
smalloc: alloc failure cleanups
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Tue, 29 Jan 2013 19:53:52 +0000 (20:53 +0100)]
client/server: fixup missing TRIM conversions
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Ken Raeburn [Tue, 29 Jan 2013 09:18:13 +0000 (10:18 +0100)]
Fix crash with absurdly but not impossibly deeply nested device stacks
Use the proper PATH_MAX for the max path for disk util, and use
snprintf() to avoid potentially overwriting it.
Modified by Jens to return NULL instead of exiting.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Tue, 29 Jan 2013 09:06:30 +0000 (10:06 +0100)]
configure: enable --extra-cflags for Windows
Add configure help as well.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Tue, 29 Jan 2013 08:26:07 +0000 (09:26 +0100)]
configure: add --extra-cflags
And ensure that -D_GNU_SOURCE is being set for configure test
programs too.
Signed-off-by: Jens Axboe <axboe@kernel.dk>