fio.git
5 years agoFio 2.1.9 fio-2.1.9
Jens Axboe [Mon, 12 May 2014 23:18:13 +0000 (17:18 -0600)]
Fio 2.1.9

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoverify: fix verification of uncompleted or errored ios
Jens Axboe [Wed, 7 May 2014 01:06:51 +0000 (19:06 -0600)]
verify: fix verification of uncompleted or errored ios

Stoo Davies reports:

------

I'm doing some powerfail recovery testing on a storage array over iSCSI.
Host is RHEL 6.4 kernel 2.6.32-358.el6.x86_64.

With fio 2.1.2 -> 2.1.4 the job file below rides through the disks going
away, and continues I/O after they come back, without reporting any
errors.  With fio 2.1.5 -> 2.1.8 when the disks come back fio
immediately reports a meta verification error.

I captured a trace with an finisar analyzer, and can see that after the
disks come back and the host logs back in, a read is issued for an lba
which was never written to.  Since I don't see verification errors
outside of the powerfail testing, I suspect fio isn't correctly handling
failed writes during the time the disks are unavailable.

------

The issue is caused by a change between 2.1.4 and 2.1.5 that started
logging IO for verification at issue time, instead of at completion
time. If the IO only partially completes or completes in error, then
we need to adjust the verification unit so that we verify only what
fio actually wrote.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoAvoid buildenv conditional in thread_option struct
Daniel Gollub [Thu, 1 May 2014 15:07:49 +0000 (17:07 +0200)]
Avoid buildenv conditional in thread_option struct

Managed to run into issues with an external ioengine
which got build with CONFIG_LIBNUMA not defined. Fio
itself got build with CONFIG_LIBNUMA this resulted
in different struct members offsets in the two different
ELF objects. Causing crashes due to invalidate offsets
inside the thread_data structure (e.g. td->io_ops->data).

Ideally all structs which might be used by external
ioengines should be independent of buildenv conditionals
like CONFIG_LIBNUMA or others.

Removed the CONFIG_LIBNUMA in thread_options.h and replaced
the libnuma specific "struct bitmask" members with strings
which hold the option's input value. This should also make
the marshaling/demarshaling in cconv.c easier.
(Note: the NUMA bits are not handled in cconv.c at the
moment. And not part of the thread_options_packed struct)

Signed-off-by: Daniel Gollub <daniel.gollub@gmail.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoAvoid double declaration of leXX_to_cpu
Daniel Gollub [Thu, 1 May 2014 12:07:04 +0000 (14:07 +0200)]
Avoid double declaration of leXX_to_cpu

for external ioengines. leXX_to_cpu is a very common
declaration - e.g. Ceph is using this as well.

Signed-off-by: Daniel Gollub <daniel.gollub@gmail.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoAvoid double-declaration of ARRAY_SIZE
Daniel Gollub [Thu, 1 May 2014 12:07:03 +0000 (14:07 +0200)]
Avoid double-declaration of ARRAY_SIZE

for external ioengines. ARRAY_SIZE is a very common
declaration - e.g. Ceph is using this as well.

Signed-off-by: Daniel Gollub <daniel.gollub@gmail.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agodistinguish internal/ext. builds with FIO_INTERNAL
Daniel Gollub [Thu, 1 May 2014 12:07:02 +0000 (14:07 +0200)]
distinguish internal/ext. builds with FIO_INTERNAL

To distinguish between internal and external includes of
fio.h and other headers FIO_INTERNAL get applied only for
the fio (and gfio) internal build itself.

This helps to prevent double-declaration issues by making
ambigous declaration with common names coditional based on
the FIO_INTERNAL define.

Signed-off-by: Daniel Gollub <daniel.gollub@gmail.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoReplace CONFIG_CPU_COUNT with plain CPU_COUNT
Daniel Gollub [Thu, 1 May 2014 12:07:01 +0000 (14:07 +0200)]
Replace CONFIG_CPU_COUNT with plain CPU_COUNT

Fixes double-declartion issue of CPU_COUNT when fio.h is included by
third-party projects.

Checking for a define of CPU_COUNT seems to be a common approach
to deal with different versions of sched.h

Signed-off-by: Daniel Gollub <daniel.gollub@gmail.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agocrc/test.c: fix include of time.h
Danny Al-Gaaf [Wed, 30 Apr 2014 18:48:16 +0000 (20:48 +0200)]
crc/test.c: fix include of time.h

Replace include of time.h with renamed fio_time.h to fix build.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoMakefile: clean all *.d dependency files on make clean
Jens Axboe [Wed, 30 Apr 2014 21:01:00 +0000 (15:01 -0600)]
Makefile: clean all *.d dependency files on make clean

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoRename time.h for third-party include of fio.h
Daniel Gollub [Wed, 30 Apr 2014 09:25:04 +0000 (11:25 +0200)]
Rename time.h for third-party include of fio.h

External project might build external ioengines and need to
include there for fio.h. If a project set the include path
to the fio root source directory and the third-party source
holds an "time.h" as well things get complicated.

Signed-off-by: Daniel Gollub <d.gollub@telekom.de>
Cc: Daniel Gollub <daniel.gollub@t-online.de>
Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoFix gfio build due to disk_stat struct seperation
Daniel Gollub [Wed, 30 Apr 2014 06:27:53 +0000 (08:27 +0200)]
Fix gfio build due to disk_stat struct seperation

Signed-off-by: Daniel Gollub <d.gollub@telekom.de>
Cc: Daniel Gollub <daniel.gollub@t-online.de>
Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoWindows: always open files in binary mode
Bruce Cran [Mon, 28 Apr 2014 21:54:43 +0000 (15:54 -0600)]
Windows: always open files in binary mode

On Windows, open() defaults to text mode, causing control characters
to be interpreted. Avoid this by passing the _O_BINARY flag.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoAdd option for io_limit
Jens Axboe [Mon, 28 Apr 2014 18:08:47 +0000 (12:08 -0600)]
Add option for io_limit

'size' denotes both the size of the region for IO, as well as the
amount of IO to transfer. Sometimes it's useful to be able to set
them separately instead. With this patch, you can do:

size=100G
io_limit=10G

and have fio do 10G of IO between 0..100G of the address space.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoMerge branch 'master' of ssh://git.kernel.dk/data/git/fio
Jens Axboe [Wed, 23 Apr 2014 15:59:05 +0000 (09:59 -0600)]
Merge branch 'master' of ssh://git.kernel.dk/data/git/fio

5 years agoinit: don't set did_arg just for an engine option
Jens Axboe [Wed, 23 Apr 2014 15:58:39 +0000 (09:58 -0600)]
init: don't set did_arg just for an engine option

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoError out if the set blocksize is larger than the file size
Jens Axboe [Wed, 16 Apr 2014 21:51:27 +0000 (23:51 +0200)]
Error out if the set blocksize is larger than the file size

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agosg/binject: only restore file flags if they have been set
Jens Axboe [Tue, 15 Apr 2014 15:07:44 +0000 (09:07 -0600)]
sg/binject: only restore file flags if they have been set

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agosg: check for error in fcntl() restore of flags
Jens Axboe [Tue, 15 Apr 2014 14:30:06 +0000 (08:30 -0600)]
sg: check for error in fcntl() restore of flags

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agobinject: check for error in fcntl() restore of flags
Jens Axboe [Tue, 15 Apr 2014 14:28:51 +0000 (08:28 -0600)]
binject: check for error in fcntl() restore of flags

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agobackend: fix potential division-by-zero
Jens Axboe [Tue, 15 Apr 2014 14:26:57 +0000 (08:26 -0600)]
backend: fix potential division-by-zero

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoidletime: fix another missing unlock on error
Jens Axboe [Tue, 15 Apr 2014 14:25:16 +0000 (08:25 -0600)]
idletime: fix another missing unlock on error

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoMerge branch 'master' of ssh://git.kernel.dk/data/git/fio
Jens Axboe [Tue, 15 Apr 2014 14:18:36 +0000 (08:18 -0600)]
Merge branch 'master' of ssh://git.kernel.dk/data/git/fio

5 years agostrcpy: kill last of the suspect ones
Jens Axboe [Tue, 15 Apr 2014 14:18:08 +0000 (08:18 -0600)]
strcpy: kill last of the suspect ones

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoWindows: set sizeof(sun_path) to 260 (MAX_PATH).
Bruce Cran [Tue, 15 Apr 2014 00:33:22 +0000 (18:33 -0600)]
Windows: set sizeof(sun_path) to 260 (MAX_PATH).

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoinit: improve command line check for when to print usage/help
Jens Axboe [Mon, 14 Apr 2014 19:37:46 +0000 (13:37 -0600)]
init: improve command line check for when to print usage/help

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoinit: fix leak bug fix from commit 60c5ec73
Jens Axboe [Mon, 14 Apr 2014 19:29:52 +0000 (13:29 -0600)]
init: fix leak bug fix from commit 60c5ec73

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agodisk util: fix bug in strncpy() conversion
Jens Axboe [Mon, 14 Apr 2014 19:25:12 +0000 (13:25 -0600)]
disk util: fix bug in strncpy() conversion

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoFix potential NULL dereference in alloc_mem_mmap()
Jens Axboe [Mon, 14 Apr 2014 19:20:38 +0000 (13:20 -0600)]
Fix potential NULL dereference in alloc_mem_mmap()

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agodiskutil: make it clear that parent != NULL for json output
Jens Axboe [Mon, 14 Apr 2014 19:19:13 +0000 (13:19 -0600)]
diskutil: make it clear that parent != NULL for json output

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoparse: make it clear that 'cp' can't be NULL for o->off1
Jens Axboe [Mon, 14 Apr 2014 19:16:10 +0000 (13:16 -0600)]
parse: make it clear that 'cp' can't be NULL for o->off1

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoFix is_power_of_2() for longs
Jens Axboe [Mon, 14 Apr 2014 19:13:21 +0000 (13:13 -0600)]
Fix is_power_of_2() for longs

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoidletime: unlock ->start_lock in error cases
Jens Axboe [Mon, 14 Apr 2014 19:11:19 +0000 (13:11 -0600)]
idletime: unlock ->start_lock in error cases

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoAvoid overwriting allocated pid_file
Jens Axboe [Mon, 14 Apr 2014 19:09:04 +0000 (13:09 -0600)]
Avoid overwriting allocated pid_file

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agogettime: init 'failed' before use
Jens Axboe [Mon, 14 Apr 2014 18:20:04 +0000 (12:20 -0600)]
gettime: init 'failed' before use

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agostat: warn on failure to detach DU thread
Jens Axboe [Mon, 14 Apr 2014 18:15:35 +0000 (12:15 -0600)]
stat: warn on failure to detach DU thread

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agogettime: handle pthread_create() failure
Jens Axboe [Mon, 14 Apr 2014 18:14:17 +0000 (12:14 -0600)]
gettime: handle pthread_create() failure

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoinit: silence bogus td == NULL warning
Jens Axboe [Mon, 14 Apr 2014 18:07:36 +0000 (12:07 -0600)]
init: silence bogus td == NULL warning

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agofilesetup: silence bogus warning on close(-1)
Jens Axboe [Mon, 14 Apr 2014 18:05:22 +0000 (12:05 -0600)]
filesetup: silence bogus warning on close(-1)

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoe4defrag: pass 'error' into td_verror()
Jens Axboe [Mon, 14 Apr 2014 18:03:22 +0000 (12:03 -0600)]
e4defrag: pass 'error' into td_verror()

Don't use -1 or -Exxxx

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agotd_verror() needs errno style error
Jens Axboe [Mon, 14 Apr 2014 18:01:45 +0000 (12:01 -0600)]
td_verror() needs errno style error

put_file_log() could pass in -1, we need the actual error instead.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agonet engine: get rid of unused struct sockaddr_in6
Jens Axboe [Mon, 14 Apr 2014 17:57:05 +0000 (11:57 -0600)]
net engine: get rid of unused struct sockaddr_in6

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoserver: ensure that incoming iolog name is NULL terminated
Jens Axboe [Mon, 14 Apr 2014 17:45:34 +0000 (11:45 -0600)]
server: ensure that incoming iolog name is NULL terminated

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoserver: sanity check incoming command size (and payload)
Jens Axboe [Mon, 14 Apr 2014 17:43:47 +0000 (11:43 -0600)]
server: sanity check incoming command size (and payload)

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agobackend: better NULL termination fix for IO scheduler switching
Jens Axboe [Mon, 14 Apr 2014 17:36:01 +0000 (11:36 -0600)]
backend: better NULL termination fix for IO scheduler switching

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agonum2str: ensure we never access postfix[] out-of-bounds
Jens Axboe [Mon, 14 Apr 2014 16:21:11 +0000 (10:21 -0600)]
num2str: ensure we never access postfix[] out-of-bounds

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agomake_filename: another off-by-one buffer issue
Jens Axboe [Mon, 14 Apr 2014 16:13:46 +0000 (10:13 -0600)]
make_filename: another off-by-one buffer issue

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoError check fcntl() calls
Jens Axboe [Mon, 14 Apr 2014 16:04:21 +0000 (10:04 -0600)]
Error check fcntl() calls

Add a helper to mark an fd as non-blocking, so we can check
and complain in one place.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoinit: remove dead code
Jens Axboe [Mon, 14 Apr 2014 15:52:50 +0000 (09:52 -0600)]
init: remove dead code

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agobackend: fix potential divide-by-zero
Jens Axboe [Mon, 14 Apr 2014 15:51:10 +0000 (09:51 -0600)]
backend: fix potential divide-by-zero

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agostat: fix potential divide-by-zero
Jens Axboe [Mon, 14 Apr 2014 15:49:38 +0000 (09:49 -0600)]
stat: fix potential divide-by-zero

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoidletime: fix potential divide-by-zero
Jens Axboe [Mon, 14 Apr 2014 15:47:14 +0000 (09:47 -0600)]
idletime: fix potential divide-by-zero

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoeta: fix potential divide by zero
Jens Axboe [Mon, 14 Apr 2014 15:45:19 +0000 (09:45 -0600)]
eta: fix potential divide by zero

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoserver: fix missing comma in fio_server_ops[] strings
Jens Axboe [Mon, 14 Apr 2014 15:07:30 +0000 (09:07 -0600)]
server: fix missing comma in fio_server_ops[] strings

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agogettime: use unsigned loop counter
Jens Axboe [Mon, 14 Apr 2014 15:05:46 +0000 (09:05 -0600)]
gettime: use unsigned loop counter

We're comparing with an unsigned exit condition.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoMake td_verror() use an unsigned type
Jens Axboe [Mon, 14 Apr 2014 15:04:48 +0000 (09:04 -0600)]
Make td_verror() use an unsigned type

Get rid of warnings stating the input can't be negative.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoFix missing sign conversion in ignore_error_type()
Jens Axboe [Mon, 14 Apr 2014 14:59:17 +0000 (08:59 -0600)]
Fix missing sign conversion in ignore_error_type()

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agostat: fix wrong check for whether we have a description or not
Jens Axboe [Mon, 14 Apr 2014 14:57:35 +0000 (08:57 -0600)]
stat: fix wrong check for whether we have a description or not

It's an array, hence never NULL. Check for size instead.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoserver: fix potential buffer overrun in socket setup path
Jens Axboe [Mon, 14 Apr 2014 14:55:13 +0000 (08:55 -0600)]
server: fix potential buffer overrun in socket setup path

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoserver: fix potential buffer overrun in bind string
Jens Axboe [Mon, 14 Apr 2014 14:54:09 +0000 (08:54 -0600)]
server: fix potential buffer overrun in bind string

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoclient: fix potential buffer overrun in socket path
Jens Axboe [Mon, 14 Apr 2014 14:50:33 +0000 (08:50 -0600)]
client: fix potential buffer overrun in socket path

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agonet engine: fix potential buffer overrun in socket path
Jens Axboe [Mon, 14 Apr 2014 14:49:04 +0000 (08:49 -0600)]
net engine: fix potential buffer overrun in socket path

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agofile: fix potential buffer overrun in get_fs_free_counts()
Jens Axboe [Mon, 14 Apr 2014 14:45:13 +0000 (08:45 -0600)]
file: fix potential buffer overrun in get_fs_free_counts()

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoverify: fix potential buffer overrun in dump_buf()
Jens Axboe [Mon, 14 Apr 2014 14:43:55 +0000 (08:43 -0600)]
verify: fix potential buffer overrun in dump_buf()

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoinit: fix potential buffer overrun in make_filename()
Jens Axboe [Mon, 14 Apr 2014 14:34:43 +0000 (08:34 -0600)]
init: fix potential buffer overrun in make_filename()

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoserver: ensure that dus->name is always NULL terminated
Jens Axboe [Mon, 14 Apr 2014 14:28:08 +0000 (08:28 -0600)]
server: ensure that dus->name is always NULL terminated

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agostat: check that TEMP environment variable is half-way sane
Jens Axboe [Mon, 14 Apr 2014 14:23:23 +0000 (08:23 -0600)]
stat: check that TEMP environment variable is half-way sane

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoinit: fix more cases of leaking pid_file
Jens Axboe [Mon, 14 Apr 2014 14:17:30 +0000 (08:17 -0600)]
init: fix more cases of leaking pid_file

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agodiskutil: separate out stats from name
Jens Axboe [Fri, 11 Apr 2014 18:17:53 +0000 (12:17 -0600)]
diskutil: separate out stats from name

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoinit: remove bogus check for NULL parent
Jens Axboe [Fri, 11 Apr 2014 17:42:13 +0000 (11:42 -0600)]
init: remove bogus check for NULL parent

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoDon't leak f_out file desciptor if specified more than once
Jens Axboe [Fri, 11 Apr 2014 17:39:46 +0000 (11:39 -0600)]
Don't leak f_out file desciptor if specified more than once

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoDon't leak memory if profile option is given twice
Jens Axboe [Fri, 11 Apr 2014 17:38:38 +0000 (11:38 -0600)]
Don't leak memory if profile option is given twice

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoserver: ensure that server pid file mem is always freed
Jens Axboe [Fri, 11 Apr 2014 17:37:34 +0000 (11:37 -0600)]
server: ensure that server pid file mem is always freed

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoEnsure that fread() string is always NULL terminated
Jens Axboe [Fri, 11 Apr 2014 17:34:17 +0000 (11:34 -0600)]
Ensure that fread() string is always NULL terminated

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoFix potentially unterminated string after strncpy() call
Jens Axboe [Fri, 11 Apr 2014 17:32:49 +0000 (11:32 -0600)]
Fix potentially unterminated string after strncpy() call

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoFix leak of directory handle in find_add_disk_slaves() error case
Jens Axboe [Fri, 11 Apr 2014 17:31:17 +0000 (11:31 -0600)]
Fix leak of directory handle in find_add_disk_slaves() error case

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agonet: fix leak of fd in bind() error path
Jens Axboe [Fri, 11 Apr 2014 17:29:50 +0000 (11:29 -0600)]
net: fix leak of fd in bind() error path

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoFix leak on error path in extend_file()
Jens Axboe [Fri, 11 Apr 2014 17:28:54 +0000 (11:28 -0600)]
Fix leak on error path in extend_file()

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoFix leak of file descriptor in error path of bc_calc()
Jens Axboe [Fri, 11 Apr 2014 17:27:34 +0000 (11:27 -0600)]
Fix leak of file descriptor in error path of bc_calc()

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoFix leak of string when doing parse dryrun
Jens Axboe [Fri, 11 Apr 2014 17:25:32 +0000 (11:25 -0600)]
Fix leak of string when doing parse dryrun

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoFix directory leak in recurse_dir() on error
Jens Axboe [Fri, 11 Apr 2014 17:24:00 +0000 (11:24 -0600)]
Fix directory leak in recurse_dir() on error

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoFix leak of 'error' if we don't assign it
Jens Axboe [Fri, 11 Apr 2014 17:20:29 +0000 (11:20 -0600)]
Fix leak of 'error' if we don't assign it

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoFio 2.1.8 fio-2.1.8
Jens Axboe [Thu, 10 Apr 2014 22:50:39 +0000 (16:50 -0600)]
Fio 2.1.8

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoblktrace: fix missing line return in dprint()
Jens Axboe [Thu, 10 Apr 2014 22:08:41 +0000 (00:08 +0200)]
blktrace: fix missing line return in dprint()

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoiolog: check for thread termination in long delays
Jens Axboe [Thu, 10 Apr 2014 22:03:32 +0000 (00:03 +0200)]
iolog: check for thread termination in long delays

If we are doing a long delay now, fio won't be able to kill
the thread when you ctrl-c the job. Split the delay into .5s
intervals, and check for signal in between.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoMake cache invalidation a soft error
Jens Axboe [Thu, 10 Apr 2014 14:59:43 +0000 (08:59 -0600)]
Make cache invalidation a soft error

On some platforms, we don't have reliable means of flushing
eg block device caches. Just warn of this fact instead of
erroring out, since the user can't do anything to stop this.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agofio: fix s390 timing for older machines/binutils
Christian Ehrhardt [Thu, 10 Apr 2014 13:09:47 +0000 (15:09 +0200)]
fio: fix s390 timing for older machines/binutils

Older gcc/binutils or one with very low default march will refuse to use stckf.
On top of that machines older than a z196 don not even have the store clock
fast facility.

To fix all of those issues, check for the facilities in configure step and make
the usage of stckf and a matching march dependent to it.

Signed-off-by: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoMerge branch 'master' of ssh://git.kernel.dk/data/git/fio
Jens Axboe [Wed, 9 Apr 2014 19:58:43 +0000 (13:58 -0600)]
Merge branch 'master' of ssh://git.kernel.dk/data/git/fio

5 years agoAdd exit_on_io_done option to the CPU IO engine
Jens Axboe [Wed, 9 Apr 2014 19:57:38 +0000 (13:57 -0600)]
Add exit_on_io_done option to the CPU IO engine

The CPU IO engine is most often used to saturate the system,
while running an IO load on it. As such, it's useful to have
CPU engine threads exit automatically, when IO has completed.
Add exit_on_io_done as a CPU IO engine option for that purpose.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoiolog: skip max_bs[rw] increment for rw == DDIR_INVAL
Jens Axboe [Wed, 9 Apr 2014 03:18:37 +0000 (21:18 -0600)]
iolog: skip max_bs[rw] increment for rw == DDIR_INVAL

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agodiskutil: fix potential out-of-bounds array write
Jens Axboe [Wed, 9 Apr 2014 03:15:09 +0000 (21:15 -0600)]
diskutil: fix potential out-of-bounds array write

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoserver: fix fd leak in error case
Jens Axboe [Wed, 9 Apr 2014 03:13:42 +0000 (21:13 -0600)]
server: fix fd leak in error case

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoIncrement open file count manually
Jens Axboe [Wed, 9 Apr 2014 03:07:12 +0000 (21:07 -0600)]
Increment open file count manually

For engines/logs that manually add files, be sure to increment
the open file count.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoRevert "Fixup ->open_files if not given"
Jens Axboe [Wed, 9 Apr 2014 03:03:59 +0000 (21:03 -0600)]
Revert "Fixup ->open_files if not given"

This reverts commit bcd27f7ae1ccebd2ac1778752bf8f13fa99600e9.

The commit causes us to default to 1 file for cases where
openfiles= isn't set, where before we would default to
nr_files.

Signed-off-by: Jens Axboe <axboe@fb.com>
Conflicts:
filesetup.c

5 years agoMerge branch 'master' of ssh://git.kernel.dk/data/git/fio
Jens Axboe [Tue, 8 Apr 2014 19:44:37 +0000 (13:44 -0600)]
Merge branch 'master' of ssh://git.kernel.dk/data/git/fio

5 years agoblktrace: probe IO depth
Jens Axboe [Tue, 8 Apr 2014 19:43:19 +0000 (13:43 -0600)]
blktrace: probe IO depth

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agofio: fix s390 nop
Christian Ehrhardt [Tue, 8 Apr 2014 15:52:01 +0000 (17:52 +0200)]
fio: fix s390 nop

When trying to run rate limited fio runs we encountered that the current
definition of a nop uses a privileged instruction which is not even a nop,
but a yield call to the hipervisor.
That leads to a SIGILL as it is privileged.

To solve that issue replace it with a nop, the assembler will take care
of it (likely to become a BCR 0,0)

Signed-off-by: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agofio: fix s390 time accounting
Christian Ehrhardt [Tue, 8 Apr 2014 15:52:00 +0000 (17:52 +0200)]
fio: fix s390 time accounting

The current timer implementation could cause time warps on s390 which ends up
as time bound jobs that would never end, because they always reset themself to
the old time.

When touching this code anyway, we also change it to use the faster stckf
and avoid the calibration as we can control the result to be usecs.
This also eliminates a few calculations cycle->usec in the hot path for the
timer.

In case other architectures have similar improved timers that might not be
usec based, but nsec based or such a thing any architecture can set
ARCH_CPU_CLOCK_CYCLES_PER_USEC to an appropriate per-arch value.
This leaves the infrastructure open for others and the compiler will throw
our division by 1 away anyway.

Signed-off-by: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoSupport limited mixed command line options and job file
Jens Axboe [Sun, 6 Apr 2014 16:10:32 +0000 (10:10 -0600)]
Support limited mixed command line options and job file

With this, you can now do:

$ fio --timeout=10s --time_based=1 jobfile.fio

and have the command line options work as if they were part
of the global section of jobfile.fio

Note that more advanced uses, like:

$ fio --option_foo=1 jobfile.fio --option_bar=1 jobfile2.fio

will not work, we still clear to default options for
subsequent jobs.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoFixup --timeout command line option
Jens Axboe [Sun, 6 Apr 2014 16:09:30 +0000 (10:09 -0600)]
Fixup --timeout command line option

Ensure we properly parse the time, use check_str_time()
instead of atoi(). Propagate the timeout correctly to
new jobs.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoMerge branch 'master' of ssh://git.kernel.dk/data/git/fio
Jens Axboe [Sun, 6 Apr 2014 15:57:59 +0000 (09:57 -0600)]
Merge branch 'master' of ssh://git.kernel.dk/data/git/fio