fio.git
11 years agoAdd recommendation that fio be run in a native console on Windows.
Bruce Cran [Wed, 20 Feb 2013 20:35:16 +0000 (20:35 +0000)]
Add recommendation that fio be run in a native console on Windows.

Windows console applications often don't run correctly under Cygwin shells such
as mintty because of the way they're implemented. Add a recommendation to the
README to run fio under a native console such as the Command Prompt or console2
to avoid these problems.

http://code.google.com/p/mintty/issues/detail?id=56 documents the problem.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoUpdate all Windows files except dobuild.cmd to use LF line endings
Bruce Cran [Thu, 21 Feb 2013 10:02:41 +0000 (11:02 +0100)]
Update all Windows files except dobuild.cmd to use LF line endings

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agocheck output_format before calling show_idle_prof_stats()
Huadong Liu [Thu, 21 Feb 2013 09:24:20 +0000 (10:24 +0100)]
check output_format before calling show_idle_prof_stats()

Do the proper checks for output format type and whether we are
a server backend or not.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agot/axmap: add pre/post bit set checks
Jens Axboe [Thu, 21 Feb 2013 09:23:51 +0000 (10:23 +0100)]
t/axmap: add pre/post bit set checks

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoMerge branch 'master' of ssh://brick.kernel.dk/data/git/fio
Jens Axboe [Mon, 18 Feb 2013 19:09:28 +0000 (20:09 +0100)]
Merge branch 'master' of ssh://brick.kernel.dk/data/git/fio

11 years agoFix replay of IO pattern with multiple open files
Nikolaus Jeremic [Mon, 18 Feb 2013 18:44:41 +0000 (19:44 +0100)]
Fix replay of IO pattern with multiple open files

Nikolaus writes:

I've tried to replay I/Os from a textual iolog file to a couple of files
using fio 2.0.13 on Linux. However, all I/Os were performed to the last
opened file, which is '/mnt/ssd5/rand-write.1.0_1_2' in this example .
My jobfile and iolog files contain the following:

[global]
name=replay_filetraces
description=Replay file traces
ioengine=libaio
iodepth=6
direct=1
continue_on_error=1

[replay]
numjobs=1
read_iolog=file_io_patterns
unlink=0

fio version 2 iolog
/mnt/ssd0/rand-write.1.0_0_0 add
/mnt/ssd1/rand-write.1.0_0_1 add
/mnt/ssd2/rand-write.1.0_0_2 add
/mnt/ssd3/rand-write.1.0_1_0 add
/mnt/ssd4/rand-write.1.0_1_1 add
/mnt/ssd5/rand-write.1.0_1_2 add
/mnt/ssd0/rand-write.1.0_0_0 open
/mnt/ssd1/rand-write.1.0_0_1 open
/mnt/ssd2/rand-write.1.0_0_2 open
/mnt/ssd3/rand-write.1.0_1_0 open
/mnt/ssd4/rand-write.1.0_1_1 open
/mnt/ssd5/rand-write.1.0_1_2 open
/mnt/ssd0/rand-write.1.0_0_0 write 0 65536
/mnt/ssd1/rand-write.1.0_0_1 write 0 65536
/mnt/ssd2/rand-write.1.0_0_2 write 0 65536
/mnt/ssd3/rand-write.1.0_1_0 write 0 65536
/mnt/ssd4/rand-write.1.0_1_1 write 0 65536
/mnt/ssd5/rand-write.1.0_1_2 write 0 65536
/mnt/ssd0/rand-write.1.0_0_0 write 65536 65536
/mnt/ssd1/rand-write.1.0_0_1 write 65536 65536
/mnt/ssd2/rand-write.1.0_0_2 write 65536 65536
/mnt/ssd3/rand-write.1.0_1_0 write 65536 65536
/mnt/ssd4/rand-write.1.0_1_1 write 65536 65536
/mnt/ssd5/rand-write.1.0_1_2 write 65536 65536
/mnt/ssd0/rand-write.1.0_0_0 close
/mnt/ssd1/rand-write.1.0_0_1 close
/mnt/ssd2/rand-write.1.0_0_2 close
/mnt/ssd3/rand-write.1.0_1_0 close
/mnt/ssd4/rand-write.1.0_1_1 close
/mnt/ssd5/rand-write.1.0_1_2 close

I've turned on debugging and added some extra printf in read_iolog2()
within iolog.c.

file     24594 add file /mnt/ssd0/rand-write.1.0_0_0
file     24594 resize file array to 2 files
file     24594 file 0x7fbe03f53090 "/mnt/ssd0/rand-write.1.0_0_0" added at 0
file     24594 add file /mnt/ssd1/rand-write.1.0_0_1
file     24594 file 0x7fbe03f53230 "/mnt/ssd1/rand-write.1.0_0_1" added at 1
file     24594 add file /mnt/ssd2/rand-write.1.0_0_2
file     24594 resize file array to 4 files
file     24594 file 0x7fbe03f533d0 "/mnt/ssd2/rand-write.1.0_0_2" added at 2
file     24594 add file /mnt/ssd3/rand-write.1.0_1_0
file     24594 file 0x7fbe03f53570 "/mnt/ssd3/rand-write.1.0_1_0" added at 3
file     24594 add file /mnt/ssd4/rand-write.1.0_1_1
file     24594 resize file array to 6 files
file     24594 file 0x7fbe03f53710 "/mnt/ssd4/rand-write.1.0_1_1" added at 4
file     24594 add file /mnt/ssd5/rand-write.1.0_1_2
file     24594 file 0x7fbe03f538b0 "/mnt/ssd5/rand-write.1.0_1_2" added at 5
io       24594 read_iolog2: Making note of file.
offset=7310584035475809390 bytes=811544671 fileno=0 file_action=1
fname=/mnt/ssd0/rand-write.1.0_0_0 act=open
io       24594 read_iolog2: Making note of file.
offset=7310584035475809390 bytes=811544671 fileno=1 file_action=1
fname=/mnt/ssd1/rand-write.1.0_0_1 act=open
io       24594 read_iolog2: Making note of file.
offset=7310584035475809390 bytes=811544671 fileno=2 file_action=1
fname=/mnt/ssd2/rand-write.1.0_0_2 act=open
io       24594 read_iolog2: Making note of file.
offset=7310584035475809390 bytes=811544671 fileno=3 file_action=1
fname=/mnt/ssd3/rand-write.1.0_1_0 act=open
io       24594 read_iolog2: Making note of file.
offset=7310584035475809390 bytes=811544671 fileno=4 file_action=1
fname=/mnt/ssd4/rand-write.1.0_1_1 act=open
io       24594 read_iolog2: Making note of file.
offset=7310584035475809390 bytes=811544671 fileno=5 file_action=1
fname=/mnt/ssd5/rand-write.1.0_1_2 act=open
io       24594 read_iolog2: Making note of file. offset=0 bytes=65536
fileno=5 file_action=1 fname=/mnt/ssd0/rand-write.1.0_0_0 act=write
io       24594 read_iolog2: Making note of file. offset=0 bytes=65536
fileno=5 file_action=1 fname=/mnt/ssd1/rand-write.1.0_0_1 act=write
io       24594 read_iolog2: Making note of file. offset=0 bytes=65536
fileno=5 file_action=1 fname=/mnt/ssd2/rand-write.1.0_0_2 act=write
io       24594 read_iolog2: Making note of file. offset=0 bytes=65536
fileno=5 file_action=1 fname=/mnt/ssd3/rand-write.1.0_1_0 act=write
io       24594 read_iolog2: Making note of file. offset=0 bytes=65536
fileno=5 file_action=1 fname=/mnt/ssd4/rand-write.1.0_1_1 act=write
io       24594 read_iolog2: Making note of file. offset=0 bytes=65536
fileno=5 file_action=1 fname=/mnt/ssd5/rand-write.1.0_1_2 act=write
io       24594 read_iolog2: Making note of file. offset=65536
bytes=65536 fileno=5 file_action=1 fname=/mnt/ssd0/rand-write.1.0_0_0
act=write
io       24594 read_iolog2: Making note of file. offset=65536
bytes=65536 fileno=5 file_action=1 fname=/mnt/ssd1/rand-write.1.0_0_1
act=write
io       24594 read_iolog2: Making note of file. offset=65536
bytes=65536 fileno=5 file_action=1 fname=/mnt/ssd2/rand-write.1.0_0_2
act=write
io       24594 read_iolog2: Making note of file. offset=65536
bytes=65536 fileno=5 file_action=1 fname=/mnt/ssd3/rand-write.1.0_1_0
act=write
io       24594 read_iolog2: Making note of file. offset=65536
bytes=65536 fileno=5 file_action=1 fname=/mnt/ssd4/rand-write.1.0_1_1
act=write
io       24594 read_iolog2: Making note of file. offset=65536
bytes=65536 fileno=5 file_action=1 fname=/mnt/ssd5/rand-write.1.0_1_2
act=write
io       24594 read_iolog2: Making note of file. offset=65536
bytes=65536 fileno=0 file_action=2 fname=/mnt/ssd0/rand-write.1.0_0_0
act=close
io       24594 read_iolog2: Making note of file. offset=65536
bytes=65536 fileno=1 file_action=2 fname=/mnt/ssd1/rand-write.1.0_0_1
act=close
io       24594 read_iolog2: Making note of file. offset=65536
bytes=65536 fileno=2 file_action=2 fname=/mnt/ssd2/rand-write.1.0_0_2
act=close
io       24594 read_iolog2: Making note of file. offset=65536
bytes=65536 fileno=3 file_action=2 fname=/mnt/ssd3/rand-write.1.0_1_0
act=close
io       24594 read_iolog2: Making note of file. offset=65536
bytes=65536 fileno=4 file_action=2 fname=/mnt/ssd4/rand-write.1.0_1_1
act=close
io       24594 read_iolog2: Making note of file. offset=65536
bytes=65536 fileno=5 file_action=2 fname=/mnt/ssd5/rand-write.1.0_1_2
act=close

Taking a look to the debugging output shows that on write action the
'fileno' variable is always set to 5 regardless of the file that should
be written. However, for open and close we have a different fileno value
for each file. Taking a look into the source code of read_iolog2()
reveals why: In case of actions 'open' or 'close', 'fileno' is looked up
through 'get_fileno(td, fname)', but for 'write', 'read' and the other 4
argument actions 'fileno' is not looked up. When I add a 'fileno =
get_fileno(td, fname);' at the end of the '(r == 4)' branch, I/Os are
replayed to the designated files.

                 if (r == 4) {
                         /*
                          * Check action first
                          */
                         if (!strcmp(act, "wait"))
                                 rw = DDIR_WAIT;
                         else if (!strcmp(act, "read"))
                                 rw = DDIR_READ;
                         else if (!strcmp(act, "write"))
                                 rw = DDIR_WRITE;
                         else if (!strcmp(act, "sync"))
                                 rw = DDIR_SYNC;
                         else if (!strcmp(act, "datasync"))
                                 rw = DDIR_DATASYNC;
                         else if (!strcmp(act, "trim"))
                                 rw = DDIR_TRIM;
                         else {
                                 log_err("fio: bad iolog file action: %s\n",
act);
                                 continue;
                         }
                 } else if (r == 2) {
                         rw = DDIR_INVAL;
                         if (!strcmp(act, "add")) {
                                 td->o.nr_files++;
                                 fileno = add_file(td, fname);
                                 file_action = FIO_LOG_ADD_FILE;
                                 continue;
                         } else if (!strcmp(act, "open")) {
                                 fileno = get_fileno(td, fname);
                                 file_action = FIO_LOG_OPEN_FILE;
                         } else if (!strcmp(act, "close")) {
                                 fileno = get_fileno(td, fname);
                                 file_action = FIO_LOG_CLOSE_FILE;
                         } else {
                                 log_err("fio: bad iolog file action: %s\n",
act);
                                 continue;
                         }
                 } else {
                         log_err("bad iolog2: %s", p);
                         continue;
                 }

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoconfigure: missing Android config options
Aaron Carroll [Mon, 18 Feb 2013 07:52:18 +0000 (08:52 +0100)]
configure: missing Android config options

We need this for the run-time endianness check to pass.  Add supported
clock monotonic while we're at it.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoconfigure: attempt to get Android going again
Aaron Carroll [Tue, 12 Feb 2013 08:58:14 +0000 (09:58 +0100)]
configure: attempt to get Android going again

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoInitialize global and non-static variables
Jens Axboe [Mon, 11 Feb 2013 15:33:25 +0000 (16:33 +0100)]
Initialize global and non-static variables

Don't assume they get zeroed. The AIX linker throws a hissy fit on
these, refusing to link the program.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoTurn f->engine_data into a 64-bit value
Jens Axboe [Mon, 11 Feb 2013 13:35:43 +0000 (14:35 +0100)]
Turn f->engine_data into a 64-bit value

Even on 32-bit platforms, we store (uint64_t) -1 in there
for some cases to make it be 'invalid'.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoconfigure: ensure that fatal errors kill config-host.h and mak files
Jens Axboe [Mon, 11 Feb 2013 12:21:54 +0000 (13:21 +0100)]
configure: ensure that fatal errors kill config-host.h and mak files

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoHandle end_fsync if a file is closed
Jens Axboe [Mon, 11 Feb 2013 10:26:55 +0000 (11:26 +0100)]
Handle end_fsync if a file is closed

Should not happen, but better be careful since it could have
an impact on data integrity.

Additionally, add a log_err() error if we do fail it.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoconfigure: allow --cpu to be specified
Jens Axboe [Sun, 10 Feb 2013 14:48:34 +0000 (15:48 +0100)]
configure: allow --cpu to be specified

And don't fail if we don't detect, it's not critical
for configure.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoFix rate limiting
Jens Axboe [Fri, 8 Feb 2013 18:05:25 +0000 (19:05 +0100)]
Fix rate limiting

Commit e0224c6b accidentally broke rate limiting, since
it bypassed the sleep logic. Rework it slightly to not
skip that.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoWindows: io cancellation often fails and causes crashes, so remove it.
Bruce Cran [Fri, 8 Feb 2013 01:45:17 +0000 (01:45 +0000)]
Windows: io cancellation often fails and causes crashes, so remove it.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoFix bug with rate and read/write mixed workloads at 100% bias
Jens Axboe [Thu, 7 Feb 2013 18:55:24 +0000 (19:55 +0100)]
Fix bug with rate and read/write mixed workloads at 100% bias

The rate_ddir() might sometimes decide to send the other data
direction, even if we have a workload with rw=randrw and
rwmixread=0. Check for this condition.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agot/log: include minmax.h for min()
Jens Axboe [Thu, 7 Feb 2013 14:40:48 +0000 (15:40 +0100)]
t/log: include minmax.h for min()

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoMakfile: use LINK for the link phase
Jens Axboe [Thu, 7 Feb 2013 14:40:37 +0000 (15:40 +0100)]
Makfile: use LINK for the link phase

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoFix zones for numjobs=x, where x > 1
Jens Axboe [Thu, 7 Feb 2013 12:06:18 +0000 (13:06 +0100)]
Fix zones for numjobs=x, where x > 1

In the options fixup, we auto-set open_files to 1 if it hasn't been set.
So any duplicated jobs will clear the zone. Fixup the check to properly
check for > 1 open files.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agostat: fix wrong type used for 32-bit compiles
Jens Axboe [Thu, 7 Feb 2013 11:56:09 +0000 (12:56 +0100)]
stat: fix wrong type used for 32-bit compiles

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoFix crash and precision of ETA with zones
Jens Axboe [Thu, 7 Feb 2013 11:54:10 +0000 (12:54 +0100)]
Fix crash and precision of ETA with zones

If zonesize was bigger than zoneskip, we could encounter a divide by
zero when calculating the number of bytes.

Additionally, the math was just wrong for most cases of zone settings.
Improve that.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoWindows: fix mlock, remove ftruncate and fix error handling.
Bruce Cran [Wed, 6 Feb 2013 23:55:46 +0000 (23:55 +0000)]
Windows: fix mlock, remove ftruncate and fix error handling.

Functions in posix.c need to use win_to_posix_err to convert errors from
Windows API calls to their equivalent POSIX error code so perror etc.
work: move it into posix.c and add the prototype to posix.h.

Fix error handling so -1 is always returned on error and errno is set.

Fix mlock to work with sizes greater than about 1MB: the working set size
needs to be increased for the new allocation.

Remove ftruncate from posix.c, since MinGW already provides it.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoWindows: fix sysconf(_SC_PHYS_PAGES).
Bruce Cran [Wed, 6 Feb 2013 20:27:27 +0000 (20:27 +0000)]
Windows: fix sysconf(_SC_PHYS_PAGES).

sysconf(_SC_PHYS_PAGES) was returning the number of bytes of memory
instead of the number of pages. Also improve some error messages in
sysconf(). Update os_phys_mem() to use sysconf().

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoGet rid of fallocate on Windows
Jens Axboe [Wed, 6 Feb 2013 12:52:12 +0000 (13:52 +0100)]
Get rid of fallocate on Windows

Fallocate is only useful if it is a fast operation, helping the
file system allocate and setup meta data for the given size.
So don't punt to zero filling the entire thing.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoHandle normal output wrapping of the percentile list
Jens Axboe [Wed, 6 Feb 2013 07:51:57 +0000 (08:51 +0100)]
Handle normal output wrapping of the percentile list

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoCleanup the percentile output formatting
Jens Axboe [Wed, 6 Feb 2013 07:46:55 +0000 (08:46 +0100)]
Cleanup the percentile output formatting

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoHandle percentile lists with higher precision that 2 digits
Vincent Kang Fu [Wed, 6 Feb 2013 07:43:40 +0000 (08:43 +0100)]
Handle percentile lists with higher precision that 2 digits

We cap the output at %2.2f right now, that's not always enough.
Make the parser check and store the precision required to
output the list correctly.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoAdd --build-32bit-win switch to configure --help output.
Bruce Cran [Tue, 5 Feb 2013 17:44:19 +0000 (17:44 +0000)]
Add --build-32bit-win switch to configure --help output.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agowindowsaio: fix file header format and improve error reporting.
Bruce Cran [Tue, 5 Feb 2013 11:33:04 +0000 (11:33 +0000)]
windowsaio: fix file header format and improve error reporting.

Use the same file header format as other engines.
Remove the copyright line.
Add calls to log_err to improve error reporting.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoMakefile: fixup init.c dependency
Jens Axboe [Tue, 5 Feb 2013 10:03:12 +0000 (11:03 +0100)]
Makefile: fixup init.c dependency

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoEnable forced 32-bit build on Windows
Huadong Liu [Tue, 5 Feb 2013 07:43:14 +0000 (08:43 +0100)]
Enable forced 32-bit build on Windows

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoFix copy/paste error in windowsaio register/unregister function names.
Bruce Cran [Tue, 5 Feb 2013 00:56:59 +0000 (00:56 +0000)]
Fix copy/paste error in windowsaio register/unregister function names.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoUpdate the Windows installer
Bruce Cran [Mon, 4 Feb 2013 12:50:51 +0000 (13:50 +0100)]
Update the Windows installer

Update the EULA copyright year to 2013, auto-generate the product ID,
remove the comment field and allow per-user installs.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agogettime: fixup AMD constant TSC detection
Jens Axboe [Sun, 3 Feb 2013 19:54:26 +0000 (20:54 +0100)]
gettime: fixup AMD constant TSC detection

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoFix failure to exit IO loop on some IO sizes
Jens Axboe [Sun, 3 Feb 2013 13:20:44 +0000 (14:20 +0100)]
Fix failure to exit IO loop on some IO sizes

If the size of a file isn't a multiple of the block size being
used, it can cause fio to exit the IO loop, check bytes done,
and then decide to do one more loop since we didn't do quite
as much IO as we wanted to. This happens because the minimum
block size is larger than the remainder. So check for that,
and stop if we need to.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoUpdate the Windows section of the README file.
Bruce Cran [Sat, 2 Feb 2013 15:12:41 +0000 (15:12 +0000)]
Update the Windows section of the README file.

The 64-bit Windows build instructions can be simplified since Cygwin
includes all the required MinGW64 packages. Also, the pthread DLL is
available as an individual download.

Update the WiX Toolset version to 3.7 and fix the URL since it's no
longer hosted at Sourceforge.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoRemove duplicated Windows configure options.
Bruce Cran [Sat, 2 Feb 2013 20:59:31 +0000 (20:59 +0000)]
Remove duplicated Windows configure options.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoDefault to CS_GTOD if CONFIG_CLOCK_GETTIME isn't defined.
Bruce Cran [Sat, 2 Feb 2013 20:54:36 +0000 (20:54 +0000)]
Default to CS_GTOD if CONFIG_CLOCK_GETTIME isn't defined.

Defaulting the clock source to CS_CGETTIME broke on OS X because it
doesn't have clock_gettime. Use CS_GTOD if CONFIG_CLOCK_GETTIME isn't
defined.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agovalues.h is obsolete: use float.h and DBL_MIN/MAX instead.
Bruce Cran [Sat, 2 Feb 2013 14:31:24 +0000 (14:31 +0000)]
values.h is obsolete: use float.h and DBL_MIN/MAX instead.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoclock: hardwire tsc as unreliable on Solaris for now
Jens Axboe [Sat, 2 Feb 2013 09:05:07 +0000 (10:05 +0100)]
clock: hardwire tsc as unreliable on Solaris for now

Need to double check the cpuid test, it probably only is reliable
on Intel. Need to check the CPU vendor and flags appropriately.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoparse: use MIN/MAXDOUBLE instead of some representation of NAN
Jens Axboe [Sat, 2 Feb 2013 08:56:23 +0000 (09:56 +0100)]
parse: use MIN/MAXDOUBLE instead of some representation of NAN

Doesn't always work for C99.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoFix percentile_list option
Vincent Kang Fu [Sat, 2 Feb 2013 08:28:55 +0000 (09:28 +0100)]
Fix percentile_list option

Commit 802ad4a8 broke the parsing of the percentile list,
by neglecting to change the parser type fo a fio_fp64_t.

Also get rid of the def_percentile_list, just set the
default percentile list as the option default.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoMakefile: fixup dependency problem for files in sub dirs
Jens Axboe [Fri, 1 Feb 2013 14:51:23 +0000 (15:51 +0100)]
Makefile: fixup dependency problem for files in sub dirs

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoREADME: how to compile fio on Windows 64 bits
Neto, Antonio Jose Rodrigues [Fri, 1 Feb 2013 07:30:38 +0000 (08:30 +0100)]
README: how to compile fio on Windows 64 bits

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoAllow override of CFLAGS
David M. Lee [Thu, 31 Jan 2013 17:37:41 +0000 (18:37 +0100)]
Allow override of CFLAGS

If you attempt to specify your own CFLAGS on the command line (such as
make CFLAGS="--whatever"), this would prevent -DFIO_VERSION from being
added to CFLAGS. This patch uses the override directive to allow the
flag to be appended to CFLAGS.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoOnly disable stdout ETA output if results are going to stdout
Jens Axboe [Thu, 31 Jan 2013 14:24:34 +0000 (15:24 +0100)]
Only disable stdout ETA output if results are going to stdout

If they are being redirected to a file with --output, then
we can still use the normal runtime ETA output on stdout.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agonet: 'nodelay' HOWTO/man page update
Jens Axboe [Thu, 31 Jan 2013 12:33:45 +0000 (13:33 +0100)]
net: 'nodelay' HOWTO/man page update

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoconfigure: add TCP_NODELAY check
Jens Axboe [Thu, 31 Jan 2013 09:19:51 +0000 (10:19 +0100)]
configure: add TCP_NODELAY check

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agowindows: fix typo in <netinet/tcp.h> header
Jens Axboe [Thu, 31 Jan 2013 08:58:22 +0000 (09:58 +0100)]
windows: fix typo in <netinet/tcp.h> header

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoFix bugs in [v]snprintf usage
Ken Raeburn [Wed, 30 Jan 2013 21:31:09 +0000 (22:31 +0100)]
Fix bugs in [v]snprintf usage

When calling snprintf, supply the full buffer size instead of
one byte less.

When using the returned length from vsnprintf for logging, don't write
more than the actual buffer size (minus one for the trailing \0), in
case the formatted string was truncated.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoAdd empty <netinet/tcp.h> for Windows
Jens Axboe [Wed, 30 Jan 2013 20:54:58 +0000 (21:54 +0100)]
Add empty <netinet/tcp.h> for Windows

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agonet: fix compile warning on Windows (and others)
Jens Axboe [Wed, 30 Jan 2013 20:52:37 +0000 (21:52 +0100)]
net: fix compile warning on Windows (and others)

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agonet engine: implement option "nodelay" for TCP sockets
Steven Noonan [Wed, 28 Nov 2012 22:52:36 +0000 (14:52 -0800)]
net engine: implement option "nodelay" for TCP sockets

With disks, O_DIRECT effectively bypasses all buffering/caching mechanisms and
ensures that the I/O is going directly to the disk. Since TCP is a streaming
protocol (like disk I/O), it also has a buffering mechanism. As with disks, it
is sometimes desirable to bypass buffering. To that end, we can use
TCP_NODELAY, which transmits the packet as soon as data is assembled,
regardless of whether it occupies a full frame.

Signed-off-by: Steven Noonan <snoonan@amazon.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoidletime: style cleanups
Jens Axboe [Wed, 30 Jan 2013 13:08:00 +0000 (14:08 +0100)]
idletime: style cleanups

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoconfigure: add SCHED_IDLE check
Jens Axboe [Wed, 30 Jan 2013 12:58:58 +0000 (13:58 +0100)]
configure: add SCHED_IDLE check

We can't consider it always available on Linux. It fails on
Debian 6.x, for instance.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoAdd support for idletime profiling
Huadong Liu [Wed, 30 Jan 2013 12:22:24 +0000 (13:22 +0100)]
Add support for idletime profiling

Idletime profiling allows a benchmark to run while filling the
idle cycles on the server, hence giving you some metric of how
much pressure the system is under. This is useful to be able
to profile and compare changes in a setup or driver, for instance.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoAdd unified_rw_reporting option
Jens Axboe [Wed, 30 Jan 2013 11:56:23 +0000 (12:56 +0100)]
Add unified_rw_reporting option

If this is set, then fio will sum and display just a single set
of statistics for any IO type. By default, fio accounts and reports
each data direction separately.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoHOWTO: indentation fix
Jens Axboe [Wed, 30 Jan 2013 11:53:55 +0000 (12:53 +0100)]
HOWTO: indentation fix

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agosmalloc: alloc failure cleanups
Jens Axboe [Tue, 29 Jan 2013 21:29:09 +0000 (22:29 +0100)]
smalloc: alloc failure cleanups

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoclient/server: fixup missing TRIM conversions
Jens Axboe [Tue, 29 Jan 2013 19:53:52 +0000 (20:53 +0100)]
client/server: fixup missing TRIM conversions

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoFix crash with absurdly but not impossibly deeply nested device stacks
Ken Raeburn [Tue, 29 Jan 2013 09:18:13 +0000 (10:18 +0100)]
Fix crash with absurdly but not impossibly deeply nested device stacks

Use the proper PATH_MAX for the max path for disk util, and use
snprintf() to avoid potentially overwriting it.

Modified by Jens to return NULL instead of exiting.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoconfigure: enable --extra-cflags for Windows
Jens Axboe [Tue, 29 Jan 2013 09:06:30 +0000 (10:06 +0100)]
configure: enable --extra-cflags for Windows

Add configure help as well.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoconfigure: add --extra-cflags
Jens Axboe [Tue, 29 Jan 2013 08:26:07 +0000 (09:26 +0100)]
configure: add --extra-cflags

And ensure that -D_GNU_SOURCE is being set for configure test
programs too.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoFix Windows build by defining clock sources and adding include file.
Bruce Cran [Mon, 28 Jan 2013 16:07:15 +0000 (16:07 +0000)]
Fix Windows build by defining clock sources and adding include file.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoAlways fsync if asked to
Jens Axboe [Sat, 26 Jan 2013 16:28:23 +0000 (09:28 -0700)]
Always fsync if asked to

Currently we don't fsync() is direct IO is issued. But if the
user asked for fsync(), he should get an fsync. So remove this
restriction.

Reported-by: Stefan Hajnoczi <stefanha@gmail.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoFix getopt def check
Jens Axboe [Fri, 25 Jan 2013 21:36:46 +0000 (14:36 -0700)]
Fix getopt def check

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoconfigure: HPUX warning for empty CC
Jens Axboe [Fri, 25 Jan 2013 21:35:27 +0000 (14:35 -0700)]
configure: HPUX warning for empty CC

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoMakefile: remove config-host.ld
Jens Axboe [Fri, 25 Jan 2013 15:52:43 +0000 (08:52 -0700)]
Makefile: remove config-host.ld

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoconfigure: add configure header to config-host.h
Jens Axboe [Fri, 25 Jan 2013 15:52:28 +0000 (08:52 -0700)]
configure: add configure header to config-host.h

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoconfigure: auto-generate a config-host.h
Jens Axboe [Fri, 25 Jan 2013 06:07:55 +0000 (23:07 -0700)]
configure: auto-generate a config-host.h

Don't have all these special cases to convert a .mak to a .h define,
just generate it from the configure script instead.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoDrop EXTFLAG and lib references
Jens Axboe [Thu, 24 Jan 2013 22:38:37 +0000 (15:38 -0700)]
Drop EXTFLAG and lib references

They are picked up automatically now.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoconfigure: kill windows FIO_HAVE_FDATASYNC
Jens Axboe [Thu, 24 Jan 2013 22:35:17 +0000 (15:35 -0700)]
configure: kill windows FIO_HAVE_FDATASYNC

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoconfigure: solaris and windowsaio fixups
Jens Axboe [Thu, 24 Jan 2013 22:33:33 +0000 (15:33 -0700)]
configure: solaris and windowsaio fixups

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agooptions: add help to experimental_verify
Jens Axboe [Thu, 24 Jan 2013 22:29:58 +0000 (15:29 -0700)]
options: add help to experimental_verify

Option help is required...

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoconfigure: fix more old FIO_HAVE_ checks
Jens Axboe [Thu, 24 Jan 2013 22:27:40 +0000 (15:27 -0700)]
configure: fix more old FIO_HAVE_ checks

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoNAN: use __builtin_nanf() if not available
Jens Axboe [Thu, 24 Jan 2013 22:22:20 +0000 (15:22 -0700)]
NAN: use __builtin_nanf() if not available

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agohash: make 64-bit type ULL
Jens Axboe [Thu, 24 Jan 2013 22:21:04 +0000 (15:21 -0700)]
hash: make 64-bit type ULL

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoCorrect ifdef checks for posix and linux fallocate
Jens Axboe [Thu, 24 Jan 2013 22:00:25 +0000 (15:00 -0700)]
Correct ifdef checks for posix and linux fallocate

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoconfigure: add that windows has RUSGE_THREAD
Jens Axboe [Thu, 24 Jan 2013 21:53:47 +0000 (14:53 -0700)]
configure: add that windows has RUSGE_THREAD

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoAdd lib/getrusage.c
Jens Axboe [Thu, 24 Jan 2013 21:20:09 +0000 (14:20 -0700)]
Add lib/getrusage.c

Abstract out the SELF/THREAD complication.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoUse RUSAGE_THREAD, if available
Huadong Liu [Thu, 24 Jan 2013 21:07:20 +0000 (14:07 -0700)]
Use RUSAGE_THREAD, if available

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoinit: complain on unrecognized option with builtin getopt_long_only()
Jens Axboe [Thu, 24 Jan 2013 01:11:48 +0000 (18:11 -0700)]
init: complain on unrecognized option with builtin getopt_long_only()

It returns '?' for a bad option, whereas the glibc one does
not. Catch that so we behave identically.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoFix compile warnings on Windows
Jens Axboe [Thu, 24 Jan 2013 00:21:41 +0000 (17:21 -0700)]
Fix compile warnings on Windows

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agowindows: don't provide strsep(), fio already has one
Jens Axboe [Wed, 23 Jan 2013 23:28:24 +0000 (16:28 -0700)]
windows: don't provide strsep(), fio already has one

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoconfigure: add default compiler on Windows
Jens Axboe [Wed, 23 Jan 2013 23:26:12 +0000 (16:26 -0700)]
configure: add default compiler on Windows

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoconfigure: fixup Windows
Jens Axboe [Wed, 23 Jan 2013 23:23:48 +0000 (16:23 -0700)]
configure: fixup Windows

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoconfigure: add endian check
Jens Axboe [Wed, 23 Jan 2013 22:42:16 +0000 (15:42 -0700)]
configure: add endian check

Will remove guesswork and manual hacking in the OS headers.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoKill now unneeded clock definitions
Jens Axboe [Wed, 23 Jan 2013 22:29:59 +0000 (15:29 -0700)]
Kill now unneeded clock definitions

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoOSX: Fixup warnings and clock_gettime() bug
Jens Axboe [Wed, 23 Jan 2013 21:02:23 +0000 (14:02 -0700)]
OSX: Fixup warnings and clock_gettime() bug

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoconfigure: remember to output CC for windows
Jens Axboe [Wed, 23 Jan 2013 20:51:39 +0000 (13:51 -0700)]
configure: remember to output CC for windows

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoconfigure: add --cc= option to script
Jens Axboe [Wed, 23 Jan 2013 20:47:54 +0000 (13:47 -0700)]
configure: add --cc= option to script

Get rid of uname -o usage for Windows, and specifically require
CC=bla ./configure or ./configure --cc=bla to set something else
than gcc.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoFix 32-bit compile warning in should_check_rate()
Jens Axboe [Wed, 23 Jan 2013 20:35:14 +0000 (13:35 -0700)]
Fix 32-bit compile warning in should_check_rate()

Introduced with the change from unsigned long -> uint64_T

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agot/axmap: update to catch recent bugs
Jens Axboe [Wed, 23 Jan 2013 18:02:07 +0000 (11:02 -0700)]
t/axmap: update to catch recent bugs

Also properly include axmap.h and don't declare our own prototypes.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoaxmap: ensure that we never return a value that is larger than the map size
Jens Axboe [Wed, 23 Jan 2013 18:01:09 +0000 (11:01 -0700)]
axmap: ensure that we never return a value that is larger than the map size

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoaxmap: improve axmap_find_first_free()
Jens Axboe [Wed, 23 Jan 2013 17:33:20 +0000 (10:33 -0700)]
axmap: improve axmap_find_first_free()

Add a ret variable, saves a branch on return.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoverify: stop on actual number of bytes needed to be verified
Jens Axboe [Wed, 23 Jan 2013 17:15:57 +0000 (10:15 -0700)]
verify: stop on actual number of bytes needed to be verified

If we don't use LFSR or a random map, we don't get told when
to stop by the random generator or map. So track it on the
side, using the same mechanism as do_io().

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoaxmap: fix bug in axmap_find_first_free()
Jens Axboe [Wed, 23 Jan 2013 17:15:05 +0000 (10:15 -0700)]
axmap: fix bug in axmap_find_first_free()

If the map is completely full, we could return an invalid value
for the first free bit. So check if we actually found a match,
return failure if we didn't.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoMake experimental_verify=1 handle all cases properly
Jens Axboe [Wed, 23 Jan 2013 16:27:25 +0000 (09:27 -0700)]
Make experimental_verify=1 handle all cases properly

- Don't track written bytes, just replay the workload by resetting
  all the random generators. This should work for any mixture of IO.

- Handle trims for verify.

- Ensure that rwmix is replayed properly for verify.

- Fixup logging for replay.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoEnsure all random seeds are repeatable for randrepeat=1
Jens Axboe [Wed, 23 Jan 2013 16:26:53 +0000 (09:26 -0700)]
Ensure all random seeds are repeatable for randrepeat=1

Currently we only do the offset.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
11 years agoeta: improve accuracy for rwmix workloads and verify
Jens Axboe [Wed, 23 Jan 2013 15:40:11 +0000 (08:40 -0700)]
eta: improve accuracy for rwmix workloads and verify

We assume 50/50 for mixed workloads. That's silly, use the actual
number specified instead.

Signed-off-by: Jens Axboe <axboe@kernel.dk>