fio.git
5 years agoclient: get rid of (now) unused client->nr_stat
Jens Axboe [Mon, 13 Oct 2014 16:15:47 +0000 (10:15 -0600)]
client: get rid of (now) unused client->nr_stat

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoclient: always output summed output for all clients
Jens Axboe [Mon, 13 Oct 2014 16:09:28 +0000 (10:09 -0600)]
client: always output summed output for all clients

At least if we have more than 1 client.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agostat: set kb/unit_base, if not already set
Jens Axboe [Mon, 13 Oct 2014 16:07:45 +0000 (10:07 -0600)]
stat: set kb/unit_base, if not already set

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoSet genesis time when running only network clients
Jens Axboe [Sun, 12 Oct 2014 19:11:35 +0000 (13:11 -0600)]
Set genesis time when running only network clients

Otherwise we get potentially no or erratic ETA reporting,
depending on the system.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoMakefile: don't install test binaries
Jens Axboe [Sat, 11 Oct 2014 16:19:42 +0000 (10:19 -0600)]
Makefile: don't install test binaries

Split the t/ binaries into internal test, and external test
tools. Prefix the external test tools with fio-, so that we
know where they are from when they are installed.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoengines/libaio: better protect against a busy loop in getevents()
Andrey Kuzmin [Fri, 10 Oct 2014 02:14:27 +0000 (20:14 -0600)]
engines/libaio: better protect against a busy loop in getevents()

If we reap nothing and we were asked to reap > 0 events, then
reuse our commit, sleep, and retry loop.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agostat: add total/short/drop ios to the json output
Jens Axboe [Fri, 10 Oct 2014 02:05:55 +0000 (20:05 -0600)]
stat: add total/short/drop ios to the json output

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoUpdate documentation on net engine port usage
Jens Axboe [Fri, 10 Oct 2014 01:58:24 +0000 (19:58 -0600)]
Update documentation on net engine port usage

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoengines/net: add subjob number to given port
Jens Axboe [Fri, 10 Oct 2014 01:55:21 +0000 (19:55 -0600)]
engines/net: add subjob number to given port

If we use numjobs=x where x > 1, then we don't want to reuse the
same port. Bump it within a group, using the subjob number.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoconfigure: only print gtk status if --enable-gfio used
Jens Axboe [Fri, 10 Oct 2014 00:15:36 +0000 (18:15 -0600)]
configure: only print gtk status if --enable-gfio used

Otherwise we always just print no, even though we did not check.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoengines/net: use link close message on TCP as well
Jens Axboe [Fri, 10 Oct 2014 00:10:11 +0000 (18:10 -0600)]
engines/net: use link close message on TCP as well

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoeta: don't count TD_SETTING_UP as a running process
Jens Axboe [Fri, 10 Oct 2014 00:06:15 +0000 (18:06 -0600)]
eta: don't count TD_SETTING_UP as a running process

We don't want to spam the console with waiting lines.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoengines/net: turn off UDP package dropping if buf size doesn't match
Jens Axboe [Thu, 9 Oct 2014 19:38:06 +0000 (13:38 -0600)]
engines/net: turn off UDP package dropping if buf size doesn't match

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agostat: add dropped ios to the standard output
Jens Axboe [Thu, 9 Oct 2014 19:27:44 +0000 (13:27 -0600)]
stat: add dropped ios to the standard output

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoengines/net: add start of tracking how many UDP packages are dropped
Jens Axboe [Thu, 9 Oct 2014 19:10:14 +0000 (13:10 -0600)]
engines/net: add start of tracking how many UDP packages are dropped

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoengines/net: add TCP_MAXSEG setting (mss)
Jens Axboe [Thu, 9 Oct 2014 18:05:44 +0000 (12:05 -0600)]
engines/net: add TCP_MAXSEG setting (mss)

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoengines/net: add socket buffer window size setting
Jens Axboe [Thu, 9 Oct 2014 17:55:16 +0000 (11:55 -0600)]
engines/net: add socket buffer window size setting

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agonet: fix error reported on job exit and full residual
Jens Axboe [Wed, 8 Oct 2014 23:04:29 +0000 (17:04 -0600)]
net: fix error reported on job exit and full residual

This is expected, when one end drops off.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agonet: use SIGTERM for terminate
Jens Axboe [Wed, 8 Oct 2014 20:14:05 +0000 (14:14 -0600)]
net: use SIGTERM for terminate

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoFix potential segfault on ENOENT on job file
Jens Axboe [Wed, 8 Oct 2014 15:55:22 +0000 (09:55 -0600)]
Fix potential segfault on ENOENT on job file

Introduced in commit 323255cc2269, if we fail opening, td may
not be set.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoclient: remember to zero pad filename
Jens Axboe [Wed, 8 Oct 2014 04:02:21 +0000 (22:02 -0600)]
client: remember to zero pad filename

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoAdd support for --remote-config
Jens Axboe [Wed, 8 Oct 2014 02:56:28 +0000 (20:56 -0600)]
Add support for --remote-config

This allows you to tell a fio server to open a local job file,
as opposed to being passed one that the fio client opens and
sends to it.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agogfio: remove warning on g_type_init() being deprecated
Jens Axboe [Wed, 8 Oct 2014 02:46:26 +0000 (20:46 -0600)]
gfio: remove warning on g_type_init() being deprecated

Compile throws this on newer glib:

gfio.c: In function ‘init_ui’:
gfio.c:1690:2: warning: ‘g_type_init’ is deprecated (declared at /usr/include/glib-2.0/gobject/gtype.h:667) [-Wdeprecated-declarations]
  g_type_init();
  ^

Fix it with a GLIB_CHECK_VERSION() check for 2.36.0.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoMakefile: fix gfio link for CONFIG_ARITHMETIC
Jens Axboe [Wed, 8 Oct 2014 02:43:04 +0000 (20:43 -0600)]
Makefile: fix gfio link for CONFIG_ARITHMETIC

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agolibaio: commit on EAGAIN
Jens Axboe [Wed, 8 Oct 2014 02:03:14 +0000 (20:03 -0600)]
libaio: commit on EAGAIN

If we have pending IOs, commit them.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agolibaio: retry on -EINTR
Slava Pestov [Wed, 8 Oct 2014 02:00:54 +0000 (20:00 -0600)]
libaio: retry on -EINTR

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoexp: small code cleanup
Jens Axboe [Tue, 7 Oct 2014 15:12:40 +0000 (09:12 -0600)]
exp: small code cleanup

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoexp: fix issues around int vs size_t
Jens Axboe [Tue, 7 Oct 2014 15:09:57 +0000 (09:09 -0600)]
exp: fix issues around int vs size_t

This seems to be the easiest way - just wrap the call in a
macro that always passes a size_t, then it doesn't matter what
yy_n_chars is typed as (int or size_t).

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoIgnore lexer.h auto-generated file
Jens Axboe [Tue, 7 Oct 2014 15:02:32 +0000 (09:02 -0600)]
Ignore lexer.h auto-generated file

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agot/dedupe: use fio abstracted OS_O_DIRECT
Jens Axboe [Tue, 7 Oct 2014 14:51:23 +0000 (08:51 -0600)]
t/dedupe: use fio abstracted OS_O_DIRECT

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agot/dedupe: remove leftover linux/fs.h include
Jens Axboe [Tue, 7 Oct 2014 14:49:58 +0000 (08:49 -0600)]
t/dedupe: remove leftover linux/fs.h include

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agot/dedupe: use generic blockdev_size() to get size
Jens Axboe [Tue, 7 Oct 2014 14:27:19 +0000 (08:27 -0600)]
t/dedupe: use generic blockdev_size() to get size

This means we can skip the Linux specific compile, so enable it
on all platforms.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agot/dedupe: avoid div-by-zero for all identical chunks
Jens Axboe [Tue, 7 Oct 2014 03:07:25 +0000 (21:07 -0600)]
t/dedupe: avoid div-by-zero for all identical chunks

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agot/dedupe: fix off-by 1.024 in bandwidth
Jens Axboe [Tue, 7 Oct 2014 03:00:10 +0000 (21:00 -0600)]
t/dedupe: fix off-by 1.024 in bandwidth

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoFix bison parsing
Stephen M. Cameron [Tue, 7 Oct 2014 02:18:57 +0000 (20:18 -0600)]
Fix bison parsing

Changes from me to switch the order in which we choose to use yacc or
bison, default to using bison if we have it.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agofix problem with yy_size_t vs int param to lexer_input
Stephen M. Cameron [Mon, 6 Oct 2014 22:58:32 +0000 (17:58 -0500)]
fix problem with yy_size_t vs int param to lexer_input

BSD yacc uses "int" for this param, linux uses size_t

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoinit: set fail error return on bad parse
Jens Axboe [Tue, 7 Oct 2014 01:06:55 +0000 (19:06 -0600)]
init: set fail error return on bad parse

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoMerge branch 'expression-parser'
Jens Axboe [Fri, 3 Oct 2014 19:20:16 +0000 (13:20 -0600)]
Merge branch 'expression-parser'

Merge in the arithmetic parser.

5 years agoFio 2.1.13 fio-2.1.13
Jens Axboe [Fri, 3 Oct 2014 19:19:04 +0000 (13:19 -0600)]
Fio 2.1.13

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoengines/libaio: modify getevents and commit error handling
Jens Axboe [Fri, 3 Oct 2014 19:10:23 +0000 (13:10 -0600)]
engines/libaio: modify getevents and commit error handling

If we get some error other than -EAGAIN in getevents(), treat
it as fatal and break.

If we get -ENOMEM on submission, reap events if we can. If we
have no pending events, treat it as a fatal error and break.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoMakefile: exp/fixup-buggy-yacc-output is no more
Jens Axboe [Fri, 3 Oct 2014 16:04:36 +0000 (10:04 -0600)]
Makefile: exp/fixup-buggy-yacc-output is no more

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agomake expression parser handle scientific notation
Stephen M. Cameron [Fri, 3 Oct 2014 01:31:11 +0000 (20:31 -0500)]
make expression parser handle scientific notation

Signed-off-by: Stephen M. Cameron <stephenmcameron@gmail.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoengines/libaio: reset 'wait_start' on non-EAGAIN
Jens Axboe [Wed, 1 Oct 2014 14:48:54 +0000 (08:48 -0600)]
engines/libaio: reset 'wait_start' on non-EAGAIN

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoBump IO engine version number
Jens Axboe [Wed, 1 Oct 2014 14:43:58 +0000 (08:43 -0600)]
Bump IO engine version number

Commit 1f440ece6127 modified the ->getevents() hook to have
a const timespec, so we should increment the engine version
number to catch that in external engines.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoengines/libaio: fix submit loop to use 'ld->queued' for exit
Jens Axboe [Wed, 1 Oct 2014 14:42:27 +0000 (08:42 -0600)]
engines/libaio: fix submit loop to use 'ld->queued' for exit

We could have head == tail for a full ring, so use the
amount queued as the gauge instead. This would be very
hard to trigger, as it would require a full ring on entry
and the first submit would have to fail to submit anything
before we loop around.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoexp: we don't need -ly
Jens Axboe [Wed, 1 Oct 2014 04:16:05 +0000 (22:16 -0600)]
exp: we don't need -ly

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoengines/libaio: don't reap on EAGAIN and no pending events
Jens Axboe [Wed, 1 Oct 2014 02:28:45 +0000 (20:28 -0600)]
engines/libaio: don't reap on EAGAIN and no pending events

Instead just loop on submit, since there are no events for us
to reap. This is usually a kernel bug, violating the principle
of forward progress guarantee.

If we can't submit anything in 30 seconds, error out.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoConstify a few more hot paths
Jens Axboe [Tue, 30 Sep 2014 22:04:12 +0000 (16:04 -0600)]
Constify a few more hot paths

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoengines/libaio: fix issue with EAGAIN
Jens Axboe [Tue, 30 Sep 2014 19:33:23 +0000 (13:33 -0600)]
engines/libaio: fix issue with EAGAIN

If we get -EAGAIN on IO submission, we should back off and reap
some events. Currently we just loop trying to resubmit, and this
has been reported to cause hangs in some virtualized environments.
There's no record of this ever being hit on raw metal.

So implement this backoff. Unfortunately it requires rewriting
the io_u tracking in libaio, since otherwise we depend on being
able to submit all the io_u's we are handed. So change the
tracking to be a ring buffer, with submission bumping the head
and completion bumping the tail. With this in place, we can't
stop issuing IO at anytime, since we'll just continue where
we left off.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoConstify 'td' in a few functions
Jens Axboe [Tue, 30 Sep 2014 19:29:57 +0000 (13:29 -0600)]
Constify 'td' in a few functions

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoremove unused bye symbol from arithmetic parser
Stephen M. Cameron [Tue, 30 Sep 2014 16:21:20 +0000 (11:21 -0500)]
remove unused bye symbol from arithmetic parser

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoenable informing arithmetic parser of implicit units
Stephen M. Cameron [Tue, 30 Sep 2014 15:32:28 +0000 (10:32 -0500)]
enable informing arithmetic parser of implicit units

This is so that "runtime=100" can be interpreted as 100 seconds
rather than 100 microseconds, but runtime=(100ms + 100s) can also
get the right answer (in microseconds).

Signed-off-by: Stephen M. Cameron <stephenmcameron@gmail.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agodo not call fprintf from yyerror in expression parser
Stephen M. Cameron [Tue, 30 Sep 2014 14:31:04 +0000 (09:31 -0500)]
do not call fprintf from yyerror in expression parser

Signed-off-by: Stephen M. Cameron <stephenmcameron@gmail.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoverify that expression parsing code gets the right answer
Stephen M. Cameron [Tue, 30 Sep 2014 13:29:33 +0000 (08:29 -0500)]
verify that expression parsing code gets the right answer

Signed-off-by: Stephen M. Cameron <stephenmcameron@gmail.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoTreat colons and commas as end of input in expression parser
Stephen M. Cameron [Tue, 30 Sep 2014 13:44:24 +0000 (08:44 -0500)]
Treat colons and commas as end of input in expression parser

Just like a comment.  This is to allow processing of lists
of numbers separated by colons or commas.  Also, fix the comment
processing to require zero rather than one character after '#'

Signed-off-by: Stephen M. Cameron <stephenmcameron@gmail.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoUse calloc() instead of malloc + memset
Jens Axboe [Tue, 30 Sep 2014 16:08:08 +0000 (10:08 -0600)]
Use calloc() instead of malloc + memset

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoHOWTO: fix typo
ordahan [Tue, 30 Sep 2014 14:18:34 +0000 (08:18 -0600)]
HOWTO: fix typo

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoMakefile: properly annotate LEX/YACC parts
Jens Axboe [Tue, 30 Sep 2014 03:39:53 +0000 (21:39 -0600)]
Makefile: properly annotate LEX/YACC parts

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoexp: update man page and HOWTO
Jens Axboe [Tue, 30 Sep 2014 03:32:43 +0000 (21:32 -0600)]
exp: update man page and HOWTO

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoMakefile: PARSER_OBJS is not used anymore
Jens Axboe [Tue, 30 Sep 2014 03:25:54 +0000 (21:25 -0600)]
Makefile: PARSER_OBJS is not used anymore

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agofilesetup: fix a bug where we overwrite the set size
Jens Axboe [Tue, 30 Sep 2014 03:23:51 +0000 (21:23 -0600)]
filesetup: fix a bug where we overwrite the set size

For the null io engine, we don't set file sizes. And we should
not overwrite a potentially legitimate o->size, if total_size
is zero.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoexp: get rid of the #line bug workaround
Jens Axboe [Tue, 30 Sep 2014 02:24:23 +0000 (20:24 -0600)]
exp: get rid of the #line bug workaround

We can use --no-lines to have bison/yacc not emit lines,
hence not hitting the missing newline bug in bision 3.0.2.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoexp: fix shift/reduce conflict complaints
Stephen M. Cameron [Mon, 29 Sep 2014 22:54:09 +0000 (16:54 -0600)]
exp: fix shift/reduce conflict complaints

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoconfigure: fix for the cases where bison provides yacc
Jens Axboe [Mon, 29 Sep 2014 22:44:31 +0000 (16:44 -0600)]
configure: fix for the cases where bison provides yacc

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoMerge branch 'master' into expression-parser
Jens Axboe [Mon, 29 Sep 2014 22:29:32 +0000 (16:29 -0600)]
Merge branch 'master' into expression-parser

5 years agoexp: fix __pow_finite being undefined
Jens Axboe [Mon, 29 Sep 2014 22:29:02 +0000 (16:29 -0600)]
exp: fix __pow_finite being undefined

Convert longs to doubles first, otherwise we fail on older
installs.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoFix compile of t/btrace2fio on RHEL5/CentOS5
Jens Axboe [Mon, 29 Sep 2014 22:07:54 +0000 (16:07 -0600)]
Fix compile of t/btrace2fio on RHEL5/CentOS5

Get these errors:

t/btrace2fio.c:395: undefined reference to `__bswap_32'
t/btrace2fio.c:396: undefined reference to `__bswap_32'
t/btrace2fio.c:397: undefined reference to `__bswap_64'
t/btrace2fio.c:398: undefined reference to `__bswap_64'
t/btrace2fio.c:399: undefined reference to `__bswap_32'
t/btrace2fio.c:400: undefined reference to `__bswap_32'
t/btrace2fio.c:401: undefined reference to `__bswap_32'
t/btrace2fio.c:402: undefined reference to `__bswap_32'
t/btrace2fio.c:403: undefined reference to `__bswap_32'
t/btrace2fio.c:404: undefined reference to `__bswap_16'
t/btrace2fio.c:405: undefined reference to `__bswap_16'

Issue is that os/os-linux.h needs to include byteswap.h.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoMakefile: silence make clean
Jens Axboe [Mon, 29 Sep 2014 22:02:35 +0000 (16:02 -0600)]
Makefile: silence make clean

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoMakefile: silence fixup
Jens Axboe [Mon, 29 Sep 2014 21:06:52 +0000 (15:06 -0600)]
Makefile: silence fixup

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoWire up exp/test-expression-parser
Jens Axboe [Mon, 29 Sep 2014 20:58:21 +0000 (14:58 -0600)]
Wire up exp/test-expression-parser

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoexp: set float output for int modulus
Jens Axboe [Mon, 29 Sep 2014 20:58:06 +0000 (14:58 -0600)]
exp: set float output for int modulus

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoUpdate .gitignore with auto-generated lex/yacc outputs
Jens Axboe [Mon, 29 Sep 2014 20:37:46 +0000 (14:37 -0600)]
Update .gitignore with auto-generated lex/yacc outputs

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agofio: support modulus for the arithmetic parser
Jens Axboe [Mon, 29 Sep 2014 19:20:40 +0000 (13:20 -0600)]
fio: support modulus for the arithmetic parser

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agofio: support exponentiation in expression parser
Stephen M. Cameron [Mon, 29 Sep 2014 18:16:54 +0000 (12:16 -0600)]
fio: support exponentiation in expression parser

Signed-off-by: Stephen M. Cameron <stephenmcameron@gmail.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agofio: support suffixes in expression parser
Stephen M. Cameron [Mon, 29 Sep 2014 18:15:35 +0000 (12:15 -0600)]
fio: support suffixes in expression parser

Note that time values in expressions by default have units of microseconds

Signed-off-by: Stephen M. Cameron <stephenmcameron@gmail.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agofio: allow arithmetic expressions to be used in job files
Stephen M. Cameron [Mon, 29 Sep 2014 18:10:49 +0000 (12:10 -0600)]
fio: allow arithmetic expressions to be used in job files

However, the arithmetic expressions must be enclosed in parentheses.
That is not a hard requirement, I just did it that way to enable the old
code to be used for the existing job files, to reduce risk of breaking
something that previously worked.

Known issues:

1) if overflow or underflow occurs during the evaluation of arithmetic
expressions, it is not detected.  Likewise, detection of floating point
divide by zero is a bit iffy.  Calculations are carried out both as long
longs and as integers.  If at any point, a double precision floating
point value is used, floating point values are used, and for the final
long long calculation the double is cast to a long long.

There may be other numeric subtleties lurking as well.  These kind of
things are why I require the parentheses to invoke the arithmetic
processing.

2) I made no effort to work this code into the autoconf system.  Have no
idea how to do that, esp. with lex and yacc involved in this.

3) Suffixes (k, M, GiB, etc.) do not work with expressions.  It is
probably not very difficult to fix this.

Signed-off-by: Stephen M. Cameron <stephenmcameron@gmail.com>
Modified by me to make configure auto-detect presence of yacc/lex and
enable the arithmetic only if those conditions are met. Also got rid
of the Makefile in exp/ and added the targets to the general Makefile,
since this makes it easier to do properly.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agot/dedupe: silence 'some_done' unused warning on some compilers
Jens Axboe [Mon, 29 Sep 2014 01:10:24 +0000 (19:10 -0600)]
t/dedupe: silence 'some_done' unused warning on some compilers

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoFix typo in bs_is_seq_ran option help
Jens Axboe [Sun, 28 Sep 2014 22:25:49 +0000 (16:25 -0600)]
Fix typo in bs_is_seq_ran option help

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoClarify what number_ios does in the HOWTO/man page/help
Jens Axboe [Sun, 28 Sep 2014 22:24:23 +0000 (16:24 -0600)]
Clarify what number_ios does in the HOWTO/man page/help

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoFix bug with zone and zone skipping and io_limit
Jens Axboe [Sun, 28 Sep 2014 22:20:58 +0000 (16:20 -0600)]
Fix bug with zone and zone skipping and io_limit

If you do:

dd if=/dev/zero of=/dev/shm/1M bs=1M count=1
fio --bs=4k --rw=write --filename=/dev/shm/1M --name=go \
--zoneskip=4k --zonesize=4k --io_limit=2M

then fio will exit after having performed 512KB of IO, instead
of the specified IO limit. This is a similar issue as was fixed
by commit ac002339c382 with rw sequencing - if we reach the
end of the file with a zone skip, wrap to the beginning.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoFix bug with rw sequence offset and io_limit
Jens Axboe [Sun, 28 Sep 2014 22:18:43 +0000 (16:18 -0600)]
Fix bug with rw sequence offset and io_limit

If you do:

dd if=/dev/zero of=/dev/shm/1M bs=1M count=1
fio --bs=4k --rw=write:4k --filename=/dev/shm/1M --name=go --io_limit=2M

then fio will exit prematurely after having done 128 4KB ios,
so only 512KB are read. This is because after 128 ios, we have
reache the end of the file. Handle this case and wrap around
properly.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agot/dedupe: fixup bloom entry calculation
Jens Axboe [Sun, 28 Sep 2014 03:29:03 +0000 (21:29 -0600)]
t/dedupe: fixup bloom entry calculation

With m/n at 8x and with 5 default hashes for blooming, we should
be around ~2% false positive rate max.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agobloom: up hashes to 5 by default
Jens Axboe [Sun, 28 Sep 2014 03:28:47 +0000 (21:28 -0600)]
bloom: up hashes to 5 by default

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agocrc/test: add fnv
Jens Axboe [Sun, 28 Sep 2014 03:27:48 +0000 (21:27 -0600)]
crc/test: add fnv

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoAdd fnv hash
Jens Axboe [Sun, 28 Sep 2014 03:26:58 +0000 (21:26 -0600)]
Add fnv hash

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agocrc/test: if the checksum needs a _final(), do it in the loop
Jens Axboe [Sat, 27 Sep 2014 20:13:58 +0000 (14:13 -0600)]
crc/test: if the checksum needs a _final(), do it in the loop

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agot/dedupe: print dedupe ratio instead of made-up factor
Jens Axboe [Sat, 27 Sep 2014 20:12:02 +0000 (14:12 -0600)]
t/dedupe: print dedupe ratio instead of made-up factor

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoMove murmur3 hash to crc/
Jens Axboe [Sat, 27 Sep 2014 15:30:56 +0000 (09:30 -0600)]
Move murmur3 hash to crc/

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agocrc/test: add jhash
Jens Axboe [Sat, 27 Sep 2014 15:29:42 +0000 (09:29 -0600)]
crc/test: add jhash

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agocrc/test: fix alignment
Jens Axboe [Sat, 27 Sep 2014 14:43:27 +0000 (08:43 -0600)]
crc/test: fix alignment

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoAdd murmurhash3
Jens Axboe [Sat, 27 Sep 2014 14:38:42 +0000 (08:38 -0600)]
Add murmurhash3

And use it in bloom, add it to the crc tester as well.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agodedupe: exit gracefully if device/file open fails
Jens Axboe [Sat, 27 Sep 2014 14:08:24 +0000 (08:08 -0600)]
dedupe: exit gracefully if device/file open fails

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoparse: fix issue with not filling leftover percentages correctly
Jens Axboe [Sat, 27 Sep 2014 14:08:00 +0000 (08:08 -0600)]
parse: fix issue with not filling leftover percentages correctly

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoImprove dedupe/compression buffer filling for mixed block sizes
Jens Axboe [Fri, 26 Sep 2014 21:04:58 +0000 (15:04 -0600)]
Improve dedupe/compression buffer filling for mixed block sizes

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agobloom: use independent hashes
Jens Axboe [Fri, 26 Sep 2014 20:57:41 +0000 (14:57 -0600)]
bloom: use independent hashes

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoClarify that include files may not contain job sections
Jens Axboe [Fri, 26 Sep 2014 19:45:56 +0000 (13:45 -0600)]
Clarify that include files may not contain job sections

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agobloom: always use a larger minimum size for bloom filter
Jens Axboe [Fri, 26 Sep 2014 19:30:00 +0000 (13:30 -0600)]
bloom: always use a larger minimum size for bloom filter

Increases accuracy at the lower end.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoAdd HOWTO section on include files
Andrey Kuzmin [Fri, 26 Sep 2014 19:29:15 +0000 (13:29 -0600)]
Add HOWTO section on include files

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoAdd ability to use an include file in a fio job file
Andrey Kuzmin [Fri, 26 Sep 2014 18:48:40 +0000 (12:48 -0600)]
Add ability to use an include file in a fio job file

With this patch, we can use the 'include' directive to tell
fio to include the contents of a file inside a job file.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agodedupe: read in larger chunks at the time
Jens Axboe [Fri, 26 Sep 2014 18:47:14 +0000 (12:47 -0600)]
dedupe: read in larger chunks at the time

Signed-off-by: Jens Axboe <axboe@fb.com>