fio.git
4 years agoFio 2.1.14 fio-2.1.14
Jens Axboe [Tue, 11 Nov 2014 18:09:59 +0000 (11:09 -0700)]
Fio 2.1.14

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agot/btrace2fio: don't dec inflight for requeue
Jens Axboe [Tue, 11 Nov 2014 16:39:11 +0000 (09:39 -0700)]
t/btrace2fio: don't dec inflight for requeue

We don't inc it again on issue, so we'll end up with an assert
trigger on inflight going negative.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agot/btrace2fio: fix divide-by-zero in rate calculation
Jens Axboe [Tue, 11 Nov 2014 15:09:04 +0000 (08:09 -0700)]
t/btrace2fio: fix divide-by-zero in rate calculation

If usec is less than 1000, it'd crash.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoMakefile: add -Wshadow
Jens Axboe [Tue, 11 Nov 2014 15:07:09 +0000 (08:07 -0700)]
Makefile: add -Wshadow

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoFixup some of the time (usec) based conversions
Stephen M. Cameron [Tue, 11 Nov 2014 03:31:26 +0000 (20:31 -0700)]
Fixup some of the time (usec) based conversions

Modified by Jens to fixup some of the mtime/utime confusion,
and add a specific ->is_time to options to get rid of future
issues in this area.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoGet rid if ddir_trim() macro
Jens Axboe [Mon, 10 Nov 2014 22:40:24 +0000 (15:40 -0700)]
Get rid if ddir_trim() macro

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoFix crash on threads being reaped before they are created
Jens Axboe [Mon, 10 Nov 2014 21:08:50 +0000 (14:08 -0700)]
Fix crash on threads being reaped before they are created

If we do that, then ->rusage_sem is never allocated.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoAdd option for statically build fio
Jens Axboe [Mon, 10 Nov 2014 18:18:06 +0000 (11:18 -0700)]
Add option for statically build fio

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agot/btrace2fio: cap depth if we don't see completion traces
Jens Axboe [Mon, 10 Nov 2014 15:52:23 +0000 (08:52 -0700)]
t/btrace2fio: cap depth if we don't see completion traces

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoFix for a race when fio prints I/O statistics periodically
Vasily Tarasov [Mon, 10 Nov 2014 03:22:24 +0000 (20:22 -0700)]
Fix for a race when fio prints I/O statistics periodically

Below is the demonstration for the latest code in git:

Job-file:

[job]
filename=/dev/sda
rw=randread
blocksize=4096
direct=1
time_based=1
runtime=300
numjobs=16

Command line:

fio --output-format=json --status-interval=10  /tmp/job.file | tee /tmp/fio.out

After parsing json and printing per-job "read->runtime" and "read->iops" in two
columns, one can see this:

runtime iops
9854 4762
19861 4724
29867 4711
39873 4705
49880 4701
59886 4698
69894 4696
79900 4694
89907 4693
99913 4692
109919 4692
119926 4691
129933 4691
139940 4691
149946 4691
159953 4690
169959 4690
179966 4690
189972 4690
199978 4690
209984 4690
219990 4690
229996 4690
240002 4690
250009 4690
260015 4690
270021 4690
280027 4690
290033 4690
600043 2344 <--- You can see a spurious jump in runtime here
300001 4690

The problem is two-fold, IMHO:

1) td->ts.runtime[DDIR_READ/DDIR_WRITE/DDIR_TRIM] are set in microseconds
   instead of milliseconds in thread_main() function. The values are later
   converted to milliseconds (after exiting from the loop in thread_main()),
   but it is too late for the out-of-band access by the helper_thread
   (the one that prints periodic statistics).

2) The helper_thread in __show_running_run_stats() temporarily adds to
   td->ts.runtime[DDIR_READ/DDIR_WRITE/DDIR_TRIM]
   the time that has passed since the beginning of the loop (td->start).
   However, the same addition happens in thread_main() by the job
   thread and td->start is not reset atomically with this addition.
   As a result, the runtime can be added twice. (That's why in the output
   we see 600043, which is approximately twice larger than the
   previous reading).

My proposal is to make the job threads updating both td->ts.runtime and
td->start atomically by using an already existing stat_mutex.
__show_running_run_stats() already takes this mutex where needed.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoclient/server: percentile_precision wasn't net converted
Jens Axboe [Sat, 8 Nov 2014 01:47:41 +0000 (18:47 -0700)]
client/server: percentile_precision wasn't net converted

We'd get 0 precision on the percentile dumps, which was confusing
and lost info.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoiolog: fix intermittent crash on exit with verify enabled
Jens Axboe [Fri, 7 Nov 2014 22:41:24 +0000 (15:41 -0700)]
iolog: fix intermittent crash on exit with verify enabled

Commit 9342d5f807795 added flist_first_entry(), but the
conversion io iolog hist list pruning was wrong. Fix it.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoAdd time_since_genesis()
Jens Axboe [Fri, 7 Nov 2014 17:36:33 +0000 (10:36 -0700)]
Add time_since_genesis()

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agosmalloc: add zeroing scalloc() variant
Jens Axboe [Thu, 6 Nov 2014 22:19:31 +0000 (15:19 -0700)]
smalloc: add zeroing scalloc() variant

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoMake fio -Wshadow clean
Jens Axboe [Thu, 6 Nov 2014 16:21:10 +0000 (09:21 -0700)]
Make fio -Wshadow clean

Found a few issues, actually.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoGet rid of __ prefix for internal frand state
Jens Axboe [Thu, 6 Nov 2014 01:39:23 +0000 (18:39 -0700)]
Get rid of __ prefix for internal frand state

It made (some) sense when we had both OS and frand in anon unions,
but now lets just kill it.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoRemove use of OS provided random functions
Jens Axboe [Thu, 6 Nov 2014 01:34:02 +0000 (18:34 -0700)]
Remove use of OS provided random functions

We added the internal random generator a long time ago, and kept
the OS variant around as an opt-in feature with using use_os_rand=1.
We defaulted to using the fio provided one, and I doubt that
anyone has used the option.

The time has come to kill it.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoAdd basic DragonFly support
Jens Axboe [Wed, 5 Nov 2014 02:53:04 +0000 (19:53 -0700)]
Add basic DragonFly support

Saw the build failing in their logs, and it hurt my feelings.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoMerge branch 'master' of ssh://git.kernel.dk/data/git/fio
Jens Axboe [Thu, 30 Oct 2014 23:16:32 +0000 (17:16 -0600)]
Merge branch 'master' of ssh://git.kernel.dk/data/git/fio

4 years agoMerge branch 'master' of ssh://git.kernel.dk/data/git/fio
Jens Axboe [Thu, 30 Oct 2014 16:13:33 +0000 (10:13 -0600)]
Merge branch 'master' of ssh://git.kernel.dk/data/git/fio

4 years agoconfigure: disable lex/yacc on Solaris
Jens Axboe [Thu, 30 Oct 2014 16:11:32 +0000 (10:11 -0600)]
configure: disable lex/yacc on Solaris

It currently breaks, disable it until we can figure it out:

   LEX lex.yy.c
"exp/expression-parser.l":line 166: Warning: Non-portable Character Class
  YACC y.tab.c
    CC lex.yy.o
exp/expression-parser.l:43:37: error: conflicting types for ‘yyunput’
lex.yy.c:31:7: note: previous declaration of ‘yyunput’ was here
exp/expression-parser.l:44:46: error: macro "input" passed 1 arguments, but takes just 0
exp/expression-parser.l: In function ‘yylex’:
exp/expression-parser.l:61:1: warning: label ‘yyfussy’ defined but not used
exp/expression-parser.l:54:22: warning: unused variable ‘yyprevious’
exp/expression-parser.l: At top level:
exp/expression-parser.l:297:1: warning: missing braces around initializer
exp/expression-parser.l:297:1: warning: (near initialization for ‘yycrank[0]’)
exp/expression-parser.l:376:1: warning: missing braces around initializer
exp/expression-parser.l:376:1: warning: (near initialization for ‘yysvec[0]’)
exp/expression-parser.l:714:6: error: conflicting types for ‘yyunput’
exp/expression-parser.l:43:37: note: previous declaration of ‘yyunput’ was here
make: *** [lex.yy.o] Error 1

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agommap: set 'fd' to -1
Jens Axboe [Thu, 30 Oct 2014 14:40:47 +0000 (08:40 -0600)]
mmap: set 'fd' to -1

This looks like a typo, using '1' makes no sense. This fixes Solaris
anon allocs.

Reported-by: Sébastien Bouchex Bellomié <sbouchex@infovista.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoengines: rename option specific pad from 'td' to 'pad'
Jens Axboe [Wed, 29 Oct 2014 14:30:07 +0000 (08:30 -0600)]
engines: rename option specific pad from 'td' to 'pad'

Except the net engine, they all just use it as padding to avoid
having a zero offset member. The latter makes the parser think
the offset isn't set, so not valid.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoengines/rbd: sort IO by start time before waiting on it
Jens Axboe [Wed, 29 Oct 2014 14:26:28 +0000 (08:26 -0600)]
engines/rbd: sort IO by start time before waiting on it

Instead of waiting on the first random IO we find, wait on the
oldest one. For higher queue depth, this should yield better
performance and lower latencies.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoMerge branch 'optimize-busy-poll' of https://github.com/ketor/fio
Jens Axboe [Wed, 29 Oct 2014 14:23:46 +0000 (08:23 -0600)]
Merge branch 'optimize-busy-poll' of https://github.com/ketor/fio

Ketor writes:

Origin code use rbd_aio_is_complete to check if the io is complete.But
this function use mutex lock and unlock to get the complete result, this
will cause the thread context switch and lock wait latency. And also
because the lock and context switch latency is in the getevents thread,
will decrease the getevents speed and increase the test latency.

This patch go back to use an varible io_complete to check if the io is
complete. This variable is updated in the aiocb callback function and
will not influence the getevents speed.

In our practice, we run fio test jobs=1 iodepth=1 bs=4k busy_poll=1.
Before this patch: IOPS is 30930 After this patch: IOPS is 37300.  And
also after this patch we get avg 25us latency, is nearly equal to the
systemtap test result of the librbd randwrite process.

So I think this patch is realy needed.

4 years agouse io_complete var to avoid the rbd_aio_is_complete lock and context switch
Ketor Meng [Wed, 29 Oct 2014 15:57:02 +0000 (15:57 +0000)]
use io_complete var to avoid the rbd_aio_is_complete lock and context switch

Signed-off-by: Ketor Meng <d.ketor@gmail.com>
4 years agofix rbd_option parse error
Ketor Meng [Wed, 29 Oct 2014 15:09:57 +0000 (15:09 +0000)]
fix rbd_option parse error

Signed-off-by: Ketor Meng <d.ketor@gmail.com>
4 years agoMerge branch 'fix-rbd-options-parse' of https://github.com/ketor/fio
Jens Axboe [Wed, 29 Oct 2014 14:22:45 +0000 (08:22 -0600)]
Merge branch 'fix-rbd-options-parse' of https://github.com/ketor/fio

Ketor writes:

There is two bug in the rbd_options parse:
1. The memeber struct thread_data *td of struct rbd_options is filled in
the cmdline parse process.So it is usable and cannot be deleted.

2.The busy_poll offset of struct rbd_options is not the "busy_poll" but
"client_name"

After fix these two bugs, the fio test then can run.

4 years agoengines/glusterfs_async: remove no-op prep function
Jens Axboe [Wed, 29 Oct 2014 02:49:56 +0000 (20:49 -0600)]
engines/glusterfs_async: remove no-op prep function

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoengines/glusterfs_async: cleanups
Jens Axboe [Wed, 29 Oct 2014 02:48:33 +0000 (20:48 -0600)]
engines/glusterfs_async: cleanups

- Style issues
- Unnecessary casts
- Unnecessary variables inits

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoengines/glfs_async: add trim support
Jens Axboe [Wed, 29 Oct 2014 02:40:21 +0000 (20:40 -0600)]
engines/glfs_async: add trim support

If available.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoengines/rbd: add option to busy poll on event completion
Jens Axboe [Tue, 28 Oct 2014 16:02:22 +0000 (10:02 -0600)]
engines/rbd: add option to busy poll on event completion

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoengines/rbd: remove unused rbd_options->td
Jens Axboe [Tue, 28 Oct 2014 15:19:55 +0000 (09:19 -0600)]
engines/rbd: remove unused rbd_options->td

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoengines/rbd: use rbd_aio_is_complete() and add TRIM support
Jens Axboe [Tue, 28 Oct 2014 15:03:11 +0000 (09:03 -0600)]
engines/rbd: use rbd_aio_is_complete() and add TRIM support

Using fri->io_complete is potentially racy, since we have no
locking around checking and waiting. Use the API provided
function to check for completion instead.

Add support for TRIM through rbd_aio_discard().

Cleanup the queue interface and completion creation.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoengines/rbd: add support for rbd_invalidate_cache()
Jens Axboe [Tue, 28 Oct 2014 15:00:06 +0000 (09:00 -0600)]
engines/rbd: add support for rbd_invalidate_cache()

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agorbd: various fixes and cleanups
Jens Axboe [Mon, 27 Oct 2014 16:53:52 +0000 (10:53 -0600)]
rbd: various fixes and cleanups

- Don't use usleep() when no completions are available, call into
  rbd to wait for one.

- Check for errors on reads/writes/syncs instead of just assuming
  that everything went fine.

- Fix leak of completions when read/write/sync fails.

- Style cleanups, and remove unnecessary casts.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agofio: Fix padding properly
Gwendal Grignou [Sat, 25 Oct 2014 23:04:32 +0000 (16:04 -0700)]
fio: Fix padding properly

Completely fix padding:
- use anonymous union for padding.
- move existing padding in thread_stat.
- add alignment checks.

Signed-off-by: Gwendal Grignou <gwendal@chromium.org>
Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoLoop Iteration Offset Bug
Brian Fulton [Fri, 24 Oct 2014 20:47:34 +0000 (14:47 -0600)]
Loop Iteration Offset Bug

Reset file descriptor offset value to the initial value when file is
closed and io state is cleared. This fixes the bug where previous offset
values persist between loop iterations.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoFix fp alignment
Jens Axboe [Fri, 24 Oct 2014 20:47:02 +0000 (14:47 -0600)]
Fix fp alignment

Commit a5bbbd8a324a fixed alignment earlier up, but then screwed
it up further down. Fix it.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agofio: fix alignement to prevent bus error on ARM
Gwendal Grignou [Fri, 24 Oct 2014 20:42:37 +0000 (13:42 -0700)]
fio: fix alignement to prevent bus error on ARM

Add a filler field to be ensure 64bit alignment.
Otherwise, we would trigger SIGBUS error in sum_stat()

Signed-off-by: Gwendal Grignou <gwendal@chromium.org>
Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agodiskutil: get rid of disk_util_start_exit()
Jens Axboe [Fri, 24 Oct 2014 05:47:01 +0000 (23:47 -0600)]
diskutil: get rid of disk_util_start_exit()

No need to wrap this in a function, just set the exit variable.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoConsolidate disk util, eta, and status check thread
Jens Axboe [Fri, 24 Oct 2014 05:16:50 +0000 (23:16 -0600)]
Consolidate disk util, eta, and status check thread

We don't need two, we can just have one do everything.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agostat: avoid hang with race in stat thread exit and rusage update
Jens Axboe [Fri, 24 Oct 2014 05:04:37 +0000 (23:04 -0600)]
stat: avoid hang with race in stat thread exit and rusage update

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoStart and stop status interval (or file) thread separately
Jens Axboe [Thu, 23 Oct 2014 15:15:20 +0000 (09:15 -0600)]
Start and stop status interval (or file) thread separately

This gets rid of the need to create threads on the fly for this,
and we can manage it and stop it before we do real stats at
the end.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agobackend: remove debug du thread exit notice
Jens Axboe [Thu, 23 Oct 2014 14:58:16 +0000 (08:58 -0600)]
backend: remove debug du thread exit notice

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoGet rid of (now unused) disk_util_mutex
Jens Axboe [Thu, 23 Oct 2014 14:55:52 +0000 (08:55 -0600)]
Get rid of (now unused) disk_util_mutex

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoFix exit being slowed by disk util stat time
Jens Axboe [Thu, 23 Oct 2014 14:53:33 +0000 (08:53 -0600)]
Fix exit being slowed by disk util stat time

Add a cv/lock to handle exit of the disk util thread. This is a
lot cleaner than the adhoc method that was used before, and it
means that we can signal the exit immediately instead of waiting
for DISKUTIL_MSEC to pass.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agot/dedupe: fix init of all thread variables
Jens Axboe [Tue, 21 Oct 2014 19:02:29 +0000 (13:02 -0600)]
t/dedupe: fix init of all thread variables

->dupes wasn't initialized, for instance. Just memset the
whole thing, so we don't run into this again.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agocompiler: improve gcc 4.x compile time check
Jens Axboe [Tue, 21 Oct 2014 14:07:33 +0000 (08:07 -0600)]
compiler: improve gcc 4.x compile time check

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoFix typo in alignment check
Shu-Yu Fu [Tue, 21 Oct 2014 13:57:55 +0000 (07:57 -0600)]
Fix typo in alignment check

It checks percentile_list twice and forgets latency_percentile
in the thread_options_pack structure. Add alignment after the
32-bit sync_file_range to make it align properly.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoAdd alignment to thread_options_pack for proper fp alignment
Jens Axboe [Mon, 20 Oct 2014 16:56:46 +0000 (10:56 -0600)]
Add alignment to thread_options_pack for proper fp alignment

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agofio: add alignment check for fp fields
Jens Axboe [Mon, 20 Oct 2014 16:52:31 +0000 (10:52 -0600)]
fio: add alignment check for fp fields

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agostat: ensure that the fp fields are 64-bit aligned
Jens Axboe [Mon, 20 Oct 2014 16:52:21 +0000 (10:52 -0600)]
stat: ensure that the fp fields are 64-bit aligned

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoAdd some build assert macros
Jens Axboe [Mon, 20 Oct 2014 16:51:24 +0000 (10:51 -0600)]
Add some build assert macros

Stolen from the linux kernel.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoBSD Yacc doesn't understand --no-lines, so use -l instead
Bruce Cran [Wed, 15 Oct 2014 20:16:35 +0000 (14:16 -0600)]
BSD Yacc doesn't understand --no-lines, so use -l instead

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoengines/net: get rid of conversion warning on clang/OSX
Jens Axboe [Wed, 15 Oct 2014 14:51:43 +0000 (08:51 -0600)]
engines/net: get rid of conversion warning on clang/OSX

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoGet rid if --header-file argument for lex
Jens Axboe [Wed, 15 Oct 2014 03:12:22 +0000 (21:12 -0600)]
Get rid if --header-file argument for lex

Since we now wrap the lexer_input() call, should be safe to just
get rid of the lexer.h define since we don't need size_t/yy_size_t.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoclient: fix potential NULL dereference
Jens Axboe [Wed, 15 Oct 2014 01:56:25 +0000 (19:56 -0600)]
client: fix potential NULL dereference

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agot/btrace2fio: fix fd leak on error
Jens Axboe [Wed, 15 Oct 2014 01:55:18 +0000 (19:55 -0600)]
t/btrace2fio: fix fd leak on error

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agot/dedupe: fix bogus items NULL compare
Jens Axboe [Wed, 15 Oct 2014 01:54:21 +0000 (19:54 -0600)]
t/dedupe: fix bogus items NULL compare

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agosha256: cast shift to uint64_t
Jens Axboe [Wed, 15 Oct 2014 01:50:31 +0000 (19:50 -0600)]
sha256: cast shift to uint64_t

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agocrc/test: fix potential div by zero
Jens Axboe [Wed, 15 Oct 2014 01:47:41 +0000 (19:47 -0600)]
crc/test: fix potential div by zero

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoengines/libaio: fix wrong setting of wait_start
Jens Axboe [Tue, 14 Oct 2014 18:17:00 +0000 (12:17 -0600)]
engines/libaio: fix wrong setting of wait_start

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoserver/client: don't use __ conversion functions if not needed
Jens Axboe [Mon, 13 Oct 2014 22:05:10 +0000 (16:05 -0600)]
server/client: don't use __ conversion functions if not needed

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoengines: don't use printf or stderr directly
Jens Axboe [Mon, 13 Oct 2014 18:18:07 +0000 (12:18 -0600)]
engines: don't use printf or stderr directly

For networked runs, we can't just print to stdout or stderr.
Use the appropriate logging functions.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agobackend: use log_info_flush(), not fflush()
Jens Axboe [Mon, 13 Oct 2014 17:57:15 +0000 (11:57 -0600)]
backend: use log_info_flush(), not fflush()

f_out may not be stdout, so use the appropriate wrapper
that flushes the appropriate FILE (or nothing, if nothing
needs to be done).

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agosmalloc: use log_err() for errors
Jens Axboe [Mon, 13 Oct 2014 17:53:55 +0000 (11:53 -0600)]
smalloc: use log_err() for errors

If this happens at runtime, stderr/stdout may have been closed
and used for server/client connections.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoclient: partial revert of 9899aff3e345
Jens Axboe [Mon, 13 Oct 2014 17:33:27 +0000 (11:33 -0600)]
client: partial revert of 9899aff3e345

We do need the client->nr_stat, otherwise things break when
group reporting is not used. We need it to keep track of
how many thread_stat's we are expecting to receive from
each client.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoclient: get rid of (now) unused client->nr_stat
Jens Axboe [Mon, 13 Oct 2014 16:15:47 +0000 (10:15 -0600)]
client: get rid of (now) unused client->nr_stat

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoclient: always output summed output for all clients
Jens Axboe [Mon, 13 Oct 2014 16:09:28 +0000 (10:09 -0600)]
client: always output summed output for all clients

At least if we have more than 1 client.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agostat: set kb/unit_base, if not already set
Jens Axboe [Mon, 13 Oct 2014 16:07:45 +0000 (10:07 -0600)]
stat: set kb/unit_base, if not already set

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoSet genesis time when running only network clients
Jens Axboe [Sun, 12 Oct 2014 19:11:35 +0000 (13:11 -0600)]
Set genesis time when running only network clients

Otherwise we get potentially no or erratic ETA reporting,
depending on the system.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoMakefile: don't install test binaries
Jens Axboe [Sat, 11 Oct 2014 16:19:42 +0000 (10:19 -0600)]
Makefile: don't install test binaries

Split the t/ binaries into internal test, and external test
tools. Prefix the external test tools with fio-, so that we
know where they are from when they are installed.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoengines/libaio: better protect against a busy loop in getevents()
Andrey Kuzmin [Fri, 10 Oct 2014 02:14:27 +0000 (20:14 -0600)]
engines/libaio: better protect against a busy loop in getevents()

If we reap nothing and we were asked to reap > 0 events, then
reuse our commit, sleep, and retry loop.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agostat: add total/short/drop ios to the json output
Jens Axboe [Fri, 10 Oct 2014 02:05:55 +0000 (20:05 -0600)]
stat: add total/short/drop ios to the json output

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoUpdate documentation on net engine port usage
Jens Axboe [Fri, 10 Oct 2014 01:58:24 +0000 (19:58 -0600)]
Update documentation on net engine port usage

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoengines/net: add subjob number to given port
Jens Axboe [Fri, 10 Oct 2014 01:55:21 +0000 (19:55 -0600)]
engines/net: add subjob number to given port

If we use numjobs=x where x > 1, then we don't want to reuse the
same port. Bump it within a group, using the subjob number.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoconfigure: only print gtk status if --enable-gfio used
Jens Axboe [Fri, 10 Oct 2014 00:15:36 +0000 (18:15 -0600)]
configure: only print gtk status if --enable-gfio used

Otherwise we always just print no, even though we did not check.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoengines/net: use link close message on TCP as well
Jens Axboe [Fri, 10 Oct 2014 00:10:11 +0000 (18:10 -0600)]
engines/net: use link close message on TCP as well

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoeta: don't count TD_SETTING_UP as a running process
Jens Axboe [Fri, 10 Oct 2014 00:06:15 +0000 (18:06 -0600)]
eta: don't count TD_SETTING_UP as a running process

We don't want to spam the console with waiting lines.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoengines/net: turn off UDP package dropping if buf size doesn't match
Jens Axboe [Thu, 9 Oct 2014 19:38:06 +0000 (13:38 -0600)]
engines/net: turn off UDP package dropping if buf size doesn't match

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agostat: add dropped ios to the standard output
Jens Axboe [Thu, 9 Oct 2014 19:27:44 +0000 (13:27 -0600)]
stat: add dropped ios to the standard output

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoengines/net: add start of tracking how many UDP packages are dropped
Jens Axboe [Thu, 9 Oct 2014 19:10:14 +0000 (13:10 -0600)]
engines/net: add start of tracking how many UDP packages are dropped

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoengines/net: add TCP_MAXSEG setting (mss)
Jens Axboe [Thu, 9 Oct 2014 18:05:44 +0000 (12:05 -0600)]
engines/net: add TCP_MAXSEG setting (mss)

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoengines/net: add socket buffer window size setting
Jens Axboe [Thu, 9 Oct 2014 17:55:16 +0000 (11:55 -0600)]
engines/net: add socket buffer window size setting

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agonet: fix error reported on job exit and full residual
Jens Axboe [Wed, 8 Oct 2014 23:04:29 +0000 (17:04 -0600)]
net: fix error reported on job exit and full residual

This is expected, when one end drops off.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agonet: use SIGTERM for terminate
Jens Axboe [Wed, 8 Oct 2014 20:14:05 +0000 (14:14 -0600)]
net: use SIGTERM for terminate

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoFix potential segfault on ENOENT on job file
Jens Axboe [Wed, 8 Oct 2014 15:55:22 +0000 (09:55 -0600)]
Fix potential segfault on ENOENT on job file

Introduced in commit 323255cc2269, if we fail opening, td may
not be set.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoclient: remember to zero pad filename
Jens Axboe [Wed, 8 Oct 2014 04:02:21 +0000 (22:02 -0600)]
client: remember to zero pad filename

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoAdd support for --remote-config
Jens Axboe [Wed, 8 Oct 2014 02:56:28 +0000 (20:56 -0600)]
Add support for --remote-config

This allows you to tell a fio server to open a local job file,
as opposed to being passed one that the fio client opens and
sends to it.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agogfio: remove warning on g_type_init() being deprecated
Jens Axboe [Wed, 8 Oct 2014 02:46:26 +0000 (20:46 -0600)]
gfio: remove warning on g_type_init() being deprecated

Compile throws this on newer glib:

gfio.c: In function ‘init_ui’:
gfio.c:1690:2: warning: ‘g_type_init’ is deprecated (declared at /usr/include/glib-2.0/gobject/gtype.h:667) [-Wdeprecated-declarations]
  g_type_init();
  ^

Fix it with a GLIB_CHECK_VERSION() check for 2.36.0.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoMakefile: fix gfio link for CONFIG_ARITHMETIC
Jens Axboe [Wed, 8 Oct 2014 02:43:04 +0000 (20:43 -0600)]
Makefile: fix gfio link for CONFIG_ARITHMETIC

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agolibaio: commit on EAGAIN
Jens Axboe [Wed, 8 Oct 2014 02:03:14 +0000 (20:03 -0600)]
libaio: commit on EAGAIN

If we have pending IOs, commit them.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agolibaio: retry on -EINTR
Slava Pestov [Wed, 8 Oct 2014 02:00:54 +0000 (20:00 -0600)]
libaio: retry on -EINTR

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoexp: small code cleanup
Jens Axboe [Tue, 7 Oct 2014 15:12:40 +0000 (09:12 -0600)]
exp: small code cleanup

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoexp: fix issues around int vs size_t
Jens Axboe [Tue, 7 Oct 2014 15:09:57 +0000 (09:09 -0600)]
exp: fix issues around int vs size_t

This seems to be the easiest way - just wrap the call in a
macro that always passes a size_t, then it doesn't matter what
yy_n_chars is typed as (int or size_t).

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoIgnore lexer.h auto-generated file
Jens Axboe [Tue, 7 Oct 2014 15:02:32 +0000 (09:02 -0600)]
Ignore lexer.h auto-generated file

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agot/dedupe: use fio abstracted OS_O_DIRECT
Jens Axboe [Tue, 7 Oct 2014 14:51:23 +0000 (08:51 -0600)]
t/dedupe: use fio abstracted OS_O_DIRECT

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agot/dedupe: remove leftover linux/fs.h include
Jens Axboe [Tue, 7 Oct 2014 14:49:58 +0000 (08:49 -0600)]
t/dedupe: remove leftover linux/fs.h include

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agot/dedupe: use generic blockdev_size() to get size
Jens Axboe [Tue, 7 Oct 2014 14:27:19 +0000 (08:27 -0600)]
t/dedupe: use generic blockdev_size() to get size

This means we can skip the Linux specific compile, so enable it
on all platforms.

Signed-off-by: Jens Axboe <axboe@fb.com>