fio.git
4 years agoembed server address, not other end's address
Ben England [Mon, 25 May 2015 12:26:24 +0000 (08:26 -0400)]
embed server address, not other end's address

4 years agocconv: add allow_mounted_write
Jens Axboe [Fri, 22 May 2015 15:09:49 +0000 (09:09 -0600)]
cconv: add allow_mounted_write

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoAdd 'allow_mounted_write' option
Jens Axboe [Fri, 22 May 2015 03:43:48 +0000 (21:43 -0600)]
Add 'allow_mounted_write' option

If this isn't set, then fio will abort if a job exists that would write
to a mounted device or partition.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoAdd start of mount check
Jens Axboe [Fri, 22 May 2015 03:19:14 +0000 (21:19 -0600)]
Add start of mount check

For raw block devices, lets add some safety check that will check for
destructive tests whether a device is mounted or not. Will plumb this
in soon, and add an override option to continue regardless.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoMerge branch 'req4' of git://github.com/kusumi/fio
Jens Axboe [Sat, 16 May 2015 02:13:58 +0000 (20:13 -0600)]
Merge branch 'req4' of git://github.com/kusumi/fio

4 years agoAdd missing fio_mutex_up() on return
Tomohiro Kusumi [Sat, 16 May 2015 01:10:57 +0000 (10:10 +0900)]
Add missing fio_mutex_up() on return

Call fio_mutex_up() before returning from this function.

4 years agoCleanup DragonFlyBSD support
Tomohiro Kusumi [Sat, 16 May 2015 01:17:14 +0000 (10:17 +0900)]
Cleanup DragonFlyBSD support

Remove these define/undef since fio compiles on DragonFlyBSD
without them.

<sys/rb.h> seems to have never existed in DragonFlyBSD history,
and it compiles without these define/undef. It seems this hack
was necessary on some older versions of NetBSD.

4 years agoChange (blank)cpu affinity macros to inline functions
Tomohiro Kusumi [Thu, 14 May 2015 20:10:05 +0000 (05:10 +0900)]
Change (blank)cpu affinity macros to inline functions

gcc warns -Wunused-value on some environments (e.g. BSD) when
the following cpu affinity macros are used in non conditional code.

Also removed #ifdef FIO_HAVE_CPU_AFFINITY in gettime-thread.c
since this variable needs to be visible when calling the function.

gettime-thread.c: In function 'gtod_thread_main':
os/os.h:82:36: warning: statement with no effect [-Wunused-value]
 #define fio_setaffinity(pid, mask) (0)
                                    ^
gettime-thread.c:48:2: note: in expansion of macro 'fio_setaffinity'
  fio_setaffinity(gettid(), fio_gtod_cpumask);

4 years agostat: add comment on why we need return
Jens Axboe [Thu, 14 May 2015 15:25:10 +0000 (11:25 -0400)]
stat: add comment on why we need return

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoFix warning from gmake on BSD
Tomohiro Kusumi [Thu, 14 May 2015 00:36:40 +0000 (09:36 +0900)]
Fix warning from gmake on BSD

stat.c: In function 'block_state_category':
stat.c:514:1: warning: control reaches end of non-void function
[-Wreturn-type]
 }

4 years agoAdd 'allow_file_create' option
Jens Axboe [Tue, 12 May 2015 15:31:32 +0000 (11:31 -0400)]
Add 'allow_file_create' option

For running certain jobs, it's convenient to tell fio that you never
want it to create files. On Linux, this prevents filling up /dev
with data for cases where the specified block device isn't available.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoAdd 'per_job_logs' option
Jens Axboe [Mon, 11 May 2015 17:39:36 +0000 (13:39 -0400)]
Add 'per_job_logs' option

If set, this generates bw/clat/iops log with per file private
filenames. If not set, jobs with identical names will share the
log filename. Default: true.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoworkqueue: style cleanup
Jens Axboe [Sun, 10 May 2015 17:16:40 +0000 (11:16 -0600)]
workqueue: style cleanup

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoMerge branch 'master' of ssh://git.kernel.dk/data/git/fio
Jens Axboe [Fri, 8 May 2015 15:21:17 +0000 (09:21 -0600)]
Merge branch 'master' of ssh://git.kernel.dk/data/git/fio

4 years agorandom: document default seed value
Jens Axboe [Fri, 8 May 2015 15:19:25 +0000 (09:19 -0600)]
random: document default seed value

Instead of hard coding this in the source, just set the default
value for the 'randseed' option'. Then it's visible when looked
up through --cmdhelp=randseed.

Also document that randrepeat is on by default in the HOWTO,
only the man page had this information.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoFio 2.2.8 fio-2.2.8
Jens Axboe [Fri, 8 May 2015 00:19:25 +0000 (18:19 -0600)]
Fio 2.2.8

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoFix Windows installer build: LICENSE is now MORAL-LICENSE
Bruce Cran [Thu, 7 May 2015 22:54:37 +0000 (16:54 -0600)]
Fix Windows installer build: LICENSE is now MORAL-LICENSE

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoUse PATH_MAX instead of _POSIX_HOST_NAME_MAX
Jens Axboe [Thu, 7 May 2015 21:33:10 +0000 (15:33 -0600)]
Use PATH_MAX instead of _POSIX_HOST_NAME_MAX

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoREADME: add git:// for github as well
Jens Axboe [Thu, 7 May 2015 21:14:10 +0000 (15:14 -0600)]
README: add git:// for github as well

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoFix compiler warning and test progs linker errors on Windows
Bruce Cran [Thu, 7 May 2015 20:56:58 +0000 (14:56 -0600)]
Fix compiler warning and test progs linker errors on Windows

Add prototype for ctime_r to os-windows.h to avoid compiler warning.
Link in os/windows/posix.o and lib/hweight.o to allow test progs to
build on Windows.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoFix keyword replacement leaks
Jens Axboe [Thu, 7 May 2015 20:23:38 +0000 (14:23 -0600)]
Fix keyword replacement leaks

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoMerge branch 'client-hostfile' of git://github.com/bengland2/fio
Jens Axboe [Thu, 7 May 2015 19:46:24 +0000 (13:46 -0600)]
Merge branch 'client-hostfile' of git://github.com/bengland2/fio

4 years agodocument changes to --client syntax and behavior
Ben England [Thu, 7 May 2015 19:33:40 +0000 (15:33 -0400)]
document changes to --client syntax and behavior

4 years agoallow --client parameter to be pathname containing client host IPs/names
Ben England [Thu, 7 May 2015 18:12:05 +0000 (14:12 -0400)]
allow --client parameter to be pathname containing client host IPs/names

4 years agoAdd Windows ctime_r implementation and add empty ioctl.h header
Bruce Cran [Wed, 6 May 2015 22:30:46 +0000 (16:30 -0600)]
Add Windows ctime_r implementation and add empty ioctl.h header

stat.c now uses ctime_r(), so add an implementation for Windows.
It's expected that ioctl.h exists on each platform, even if it's
not used: add an empty file on Windows.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoposixaio: use 'errno' for inline error
Jens Axboe [Thu, 7 May 2015 15:52:35 +0000 (09:52 -0600)]
posixaio: use 'errno' for inline error

aio_error() is only for when aio_*() didn't fail submitting.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoEnsure that set_name_idx() doesn't exceed target bounds
Jens Axboe [Thu, 7 May 2015 14:17:05 +0000 (08:17 -0600)]
Ensure that set_name_idx() doesn't exceed target bounds

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoMerge branch 'multiclnt-sharedfs-try3' of git://github.com/bengland2/fio
Jens Axboe [Thu, 7 May 2015 14:11:15 +0000 (08:11 -0600)]
Merge branch 'multiclnt-sharedfs-try3' of git://github.com/bengland2/fio

4 years agoallow --client to work with shared filesystem
Ben England [Thu, 7 May 2015 13:27:16 +0000 (09:27 -0400)]
allow --client to work with shared filesystem

4 years agoserver: bump version
Jens Axboe [Wed, 6 May 2015 21:21:19 +0000 (15:21 -0600)]
server: bump version

We added fields to the options structure, need a revision bump.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agocconv: add missing conversions of block_error_hist and skip_bad
Jens Axboe [Wed, 6 May 2015 20:20:44 +0000 (14:20 -0600)]
cconv: add missing conversions of block_error_hist and skip_bad

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoblktrace: add support for scaling and aligning replays
Jens Axboe [Wed, 6 May 2015 20:19:20 +0000 (14:19 -0600)]
blktrace: add support for scaling and aligning replays

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoAdd support for options being a power-of-2
Jens Axboe [Wed, 6 May 2015 20:15:35 +0000 (14:15 -0600)]
Add support for options being a power-of-2

Split out the is_power_of_2() from fio.h and make it independent.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoMerge branch 'master' of git://github.com/iuliur/fio
Jens Axboe [Wed, 6 May 2015 18:36:43 +0000 (12:36 -0600)]
Merge branch 'master' of git://github.com/iuliur/fio

4 years agoMake windowsaio match iometer performance. Without this, we see around 60k on local...
Julius Rus [Wed, 6 May 2015 18:30:18 +0000 (11:30 -0700)]
Make windowsaio match iometer performance. Without this, we see around 60k on local flash while iometer does 100k.

4 years agolibmtd: ->name and ->type_str can't be constant
Jens Axboe [Tue, 5 May 2015 18:11:55 +0000 (12:11 -0600)]
libmtd: ->name and ->type_str can't be constant

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoARM: Use generic assembly nop and barrier code for armv8-a
Milton Chiang [Tue, 5 May 2015 09:18:17 +0000 (17:18 +0800)]
ARM: Use generic assembly nop and barrier code for armv8-a

Signed-off-by: Milton Chiang <milton.chiang@mediatek.com>
4 years agoFix Runtime, IOPS, bandwidth recorded incorrectly
Brian Fulton [Mon, 4 May 2015 22:07:10 +0000 (16:07 -0600)]
Fix Runtime, IOPS, bandwidth recorded incorrectly

This happens with small sized timed_based runs. Change runtime
tabulation back to microseconds to handle sub millisecond loops.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agolfsr-test: print total elapsed time correctly
Steven Noonan [Fri, 1 May 2015 19:49:34 +0000 (19:49 +0000)]
lfsr-test: print total elapsed time correctly

The conversion from microseconds to seconds is 1e6 not 1e9.

Signed-off-by: Steven Noonan <steven@uplinklabs.net>
Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agogettime: add support for CLOCK_MONOTONIC_RAW
Steven Noonan [Fri, 1 May 2015 19:31:44 +0000 (12:31 -0700)]
gettime: add support for CLOCK_MONOTONIC_RAW

The clock CLOCK_MONOTONIC_RAW (introduced in Linux 2.6.28) is similar to
CLOCK_MONOTONIC except that it is not subject to NTP adjustments or incremental
adjustments performed by adjtime().

Signed-off-by: Steven Noonan <steven@uplinklabs.net>
Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoblktrace: support for non-512b sector sizes
Jens Axboe [Fri, 1 May 2015 23:02:24 +0000 (17:02 -0600)]
blktrace: support for non-512b sector sizes

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoconfigure: add support for --prefix
Jens Axboe [Wed, 29 Apr 2015 16:29:16 +0000 (10:29 -0600)]
configure: add support for --prefix

Allows other installation paths that /usr/local

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoblktrace: only set O_DIRECT if the min bs is a multiple of 4k
Jens Axboe [Tue, 28 Apr 2015 21:30:02 +0000 (15:30 -0600)]
blktrace: only set O_DIRECT if the min bs is a multiple of 4k

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoAdd recommendation to set direct=0 if first O_DIRECT fails
Jens Axboe [Tue, 28 Apr 2015 21:23:21 +0000 (15:23 -0600)]
Add recommendation to set direct=0 if first O_DIRECT fails

Fio currently tells you:

"fio: first direct IO errored. File system may not support direct IO, or
 iomem_align= is bad."

but it doesn't offer a remedy. Add a suggestion to set direct=0 if this
happens. It can happen because of a badly configured job, or from
blktrace replay because the latter adds direct=1 automatically. But that
may fail, if the trace contains IO that isn't sector aligned.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agodedupe: improve 100% dedupe case
Jens Axboe [Mon, 27 Apr 2015 15:37:11 +0000 (09:37 -0600)]
dedupe: improve 100% dedupe case

We don't need an on-stack state, just juggle the buf_state and
buf_state_prev appropriately.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agodedupe: fix dedupe_percentage=100
Jens Axboe [Mon, 27 Apr 2015 14:38:33 +0000 (08:38 -0600)]
dedupe: fix dedupe_percentage=100

The corner case of 100% dedupable was buggy, save and restore random
state appropriate for that too.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoHOWTO: clarify the 'wait' statement in iologs
Jens Axboe [Fri, 24 Apr 2015 16:45:43 +0000 (10:45 -0600)]
HOWTO: clarify the 'wait' statement in iologs

It's a relative time, not absolute wait.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoio_u: fix bug in rounding of generated buffer length
Jens Axboe [Fri, 24 Apr 2015 16:41:58 +0000 (10:41 -0600)]
io_u: fix bug in rounding of generated buffer length

If the maximum blocksize isn't a multiple of the minimum blocksize,
then fio has a bug where it will round up the block size and align
it to a size larger than the IO buffer we have. This causes random
memory corruption and crashes.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoblktrace: fix bugs in accounting
Jens Axboe [Fri, 24 Apr 2015 15:03:05 +0000 (09:03 -0600)]
blktrace: fix bugs in accounting

We need to clear the trim part of the accounting arrays, otherwise
we can use bogus values leading to attempting to allocate huge
amounts of memory. This causes fio to exit with:

fio: pid=21428, err=12/file:memory.c:242, func=iomem allocation,
error=Cannot allocate memory

Also improve the depth detection, by making it per data direction
instead of a global state.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoMerge branch 'io-threads'
Jens Axboe [Thu, 23 Apr 2015 17:13:27 +0000 (11:13 -0600)]
Merge branch 'io-threads'

4 years agoblktrace: only probe and set depth if option isn't set
Jens Axboe [Thu, 23 Apr 2015 01:54:54 +0000 (19:54 -0600)]
blktrace: only probe and set depth if option isn't set

The comment predates fio's support for checking whether a specific
option was set, we have the power to check this properly now.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoconfigure: check for more of the used MTD features
Jens Axboe [Wed, 22 Apr 2015 19:46:45 +0000 (13:46 -0600)]
configure: check for more of the used MTD features

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoFix off-by-one in cpus_allowed
Jens Axboe [Tue, 21 Apr 2015 15:39:32 +0000 (09:39 -0600)]
Fix off-by-one in cpus_allowed

Similar to commit b84113993b3a, fix it for cpus_allowed as well.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoFix off-by-one in cpu mask index handling
Jens Axboe [Tue, 21 Apr 2015 14:25:58 +0000 (08:25 -0600)]
Fix off-by-one in cpu mask index handling

This just affects how errors are printed. Kleber Sacilotto de Souza
reports:

The error message displayed by the option parser for the 'cpus_allowed'
parameter is misleading.

My system has 4 processors:

$ grep -c processor /proc/cpuinfo
4

If I provide a high number for cpus_allowed, I get the error message:

$ fio --filename=/tmp/foo.fio --cpus_allowed=5 --name=job1
fio: CPU 5 too large (max=4)
fio: failed parsing cpus_allowed=5

If it says "(max=4)", I would expect it to accept the value "4" if I
want to use the last CPU of the system (even though we know that the
CPUs are generally numbered starting with 0), but that's not what
happens:

$ fio --name=global --filename=/tmp/foo.fio --cpus_allowed=4 --name=job1
job1: (g=0): rw=read, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=1
fio-2.1.3
Starting 1 process
fio: pid=24942, err=22/file:backend.c:1192, func=cpu_set_affinity, error=Invalid argument

So for setting the affinity the CPU number really starts from 0, so 3
would be the right value in this case and the following command works as
expected:

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoDon't export state variable
Dan Ehrenberg [Mon, 20 Apr 2015 23:20:30 +0000 (16:20 -0700)]
Don't export state variable

There was no reason to export this extra variable. It was just a typo
and it is never used.

Signed-off-by: Dan Ehrenberg <dehrenberg@chromium.org>
Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoFix mtd valgrind warning
Dan Ehrenberg [Thu, 16 Apr 2015 21:50:39 +0000 (14:50 -0700)]
Fix mtd valgrind warning

strlen seems to operate in 32-bit chunks, but libmtd sometimes allocates
strings without that much padding. This patch adds something extra to
a string allocation so that the mtd ioengine is clean with respect
to valgrind memory access checks.

Signed-off-by: Dan Ehrenberg <dehrenberg@chromium.org>
Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoFix wrong index bug
Jens Axboe [Thu, 16 Apr 2015 03:47:47 +0000 (21:47 -0600)]
Fix wrong index bug

Commit 0e4dd95c548cc re-uses 'i' as an iterator, causing us to
overrun the nr_ts allocated. This subsequently causes fio to
segfault.

Fixes: 0e4dd95c548cc
Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoAdd sample job file for fixed submission rate
Jens Axboe [Wed, 15 Apr 2015 16:01:56 +0000 (10:01 -0600)]
Add sample job file for fixed submission rate

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoAdd man page and HOWTO for io_submit_mode option
Jens Axboe [Wed, 15 Apr 2015 15:58:58 +0000 (09:58 -0600)]
Add man page and HOWTO for io_submit_mode option

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agolibaio: don't call io_destroy(), let exit_aio() take care of it
Jens Axboe [Thu, 19 Mar 2015 20:52:54 +0000 (14:52 -0600)]
libaio: don't call io_destroy(), let exit_aio() take care of it

From the exit_aio() path, we can parallellize the freeing, hence
not taking an RCU grace period hit for each.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoFirst cut at supporting IO offload
Jens Axboe [Fri, 20 Mar 2015 04:50:06 +0000 (22:50 -0600)]
First cut at supporting IO offload

rate_iops=x
io_submit_mode=offload

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agobackend: split queue io_u event handling into helper
Jens Axboe [Fri, 20 Mar 2015 04:44:05 +0000 (22:44 -0600)]
backend: split queue io_u event handling into helper

io_queue_event(), can now be used from both verify and IO path.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoAdd ->bytes_done[] to struct thread_data
Jens Axboe [Fri, 20 Mar 2015 04:32:00 +0000 (22:32 -0600)]
Add ->bytes_done[] to struct thread_data

We can't keep it on the stack for async IO offload.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoChange writetrim to trimwrite
Jens Axboe [Wed, 15 Apr 2015 15:49:20 +0000 (09:49 -0600)]
Change writetrim to trimwrite

This better explains what it does. Also add HOWTO and man page entries
for trimwrite.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agomtd: example job file
Dan Ehrenberg [Tue, 14 Apr 2015 22:58:20 +0000 (15:58 -0700)]
mtd: example job file

This job file shows how one can write sequentially to an MTD device,
erase (with trim), and do a looped write/trim workload. Ignoring EIO
errors allows operations to continue as bad blocks are skipped.
The block histogram counts P/E cycles and can be used to assess
NAND flash lifetime and failure characteristics. The job file is
written to work with the nandsim fake NAND device.

Signed-off-by: Dan Ehrenberg <dehrenberg@chromium.org>
Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agomtd: ioengine
Dan Ehrenberg [Tue, 14 Apr 2015 22:58:19 +0000 (15:58 -0700)]
mtd: ioengine

This patch adds an MTD ioengine. Trims are interpreted as MTD erases.
A skip_bad option calls out to the kernel to ask if the block being
operated on is bad first; if it is bad, the operation is skipped
and -EIO is returned.

Signed-off-by: Dan Ehrenberg <dehrenberg@chromium.org>
Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agomtd: Import libmtd with modifications to make it compile
Dan Ehrenberg [Tue, 14 Apr 2015 22:58:18 +0000 (15:58 -0700)]
mtd: Import libmtd with modifications to make it compile

This patch imports files from mtdutils to access MTD devices. mtdutils
is licensed as GPLv2, like fio, so it can be included here. Files are
changed slightly in how they import headers so as to be compatible
with the fio directory structure.

Signed-off-by: Dan Ehrenberg <dehrenberg@chromium.org>
Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agomtd: Add CONFIG_MTD to ./configure script
Dan Ehrenberg [Tue, 14 Apr 2015 22:58:17 +0000 (15:58 -0700)]
mtd: Add CONFIG_MTD to ./configure script

The CONFIG_MTD option will allow the mtd ioengine to be built. It
only works when the Linux MTD header is present.

Signed-off-by: Dan Ehrenberg <dehrenberg@chromium.org>
Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoCollect a block-wise histogram of trim and write errors
Dan Ehrenberg [Tue, 14 Apr 2015 22:58:16 +0000 (15:58 -0700)]
Collect a block-wise histogram of trim and write errors

This patch collects information on how many trims or writes each trim
block sized unit can undergo before its first error. It also records
the way in which an error occurred, if one did.

Output is provided with a histogram which can be output either in
normal or json output. Terse output is omitted, since it seemed like
that would require a version rev bump.

Signed-off-by: Dan Ehrenberg <dehrenberg@chromium.org>
Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoAdd new writetrim rw= mode for trims preceding writes
Dan Ehrenberg [Tue, 14 Apr 2015 22:58:15 +0000 (15:58 -0700)]
Add new writetrim rw= mode for trims preceding writes

In this new mode, sequential trims and writes are interspersed by first
doing a trim at a particular offset, then doing writes starting from
that offset until the start of the next trim block, then another trim,
etc. This workload is designed to match the requirements of NAND flash,
if trims are implemented as erases.

Signed-off-by: Dan Ehrenberg <dehrenberg@chromium.org>
Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoAllow trim on any file type
Dan Ehrenberg [Tue, 14 Apr 2015 22:58:14 +0000 (15:58 -0700)]
Allow trim on any file type

The mtd fio engine will support trim (interpreted as erase). There is
no particular reason to tie trims to block devices, as it is only a
particular ioengine which calls the trim ioctl, not generic code.
This patch simply removes the test which checks that trim is only
run on block devices.

Signed-off-by: Dan Ehrenberg <dehrenberg@chromium.org>
Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoFio 2.2.7 fio-2.2.7
Jens Axboe [Wed, 15 Apr 2015 00:08:59 +0000 (18:08 -0600)]
Fio 2.2.7

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoconfigure: update to changes in fusion-aw packaging
Akshita Rajendra Jain [Thu, 9 Apr 2015 14:24:38 +0000 (08:24 -0600)]
configure: update to changes in fusion-aw packaging

Remove a dependency that is no longer used.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agogenzip: switch to jenkins hash
Jens Axboe [Wed, 8 Apr 2015 22:25:49 +0000 (16:25 -0600)]
genzip: switch to jenkins hash

Normal distribution causes a lot of collisions with the normal hash,
for some reason. This speeds it up a lot.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoUnify gauss and zipf/pareto input values
Jens Axboe [Wed, 8 Apr 2015 21:49:18 +0000 (15:49 -0600)]
Unify gauss and zipf/pareto input values

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agogenzip: cleanups
Jens Axboe [Wed, 8 Apr 2015 21:42:20 +0000 (15:42 -0600)]
genzip: cleanups

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agogenzip: add summed hit percentages
Jens Axboe [Wed, 8 Apr 2015 21:26:29 +0000 (15:26 -0600)]
genzip: add summed hit percentages

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agogenzipf: add support for normal distribution
Jens Axboe [Wed, 8 Apr 2015 17:25:27 +0000 (11:25 -0600)]
genzipf: add support for normal distribution

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoMake normal distribution takes deviations as a percentage
Jens Axboe [Wed, 8 Apr 2015 17:24:58 +0000 (11:24 -0600)]
Make normal distribution takes deviations as a percentage

This makes it independent of the address range generated.

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agoMerge branch 'master' of ssh://git.kernel.dk/data/git/fio
Jens Axboe [Wed, 8 Apr 2015 17:10:59 +0000 (11:10 -0600)]
Merge branch 'master' of ssh://git.kernel.dk/data/git/fio

4 years agoAdd support for normal/gaussian random distributions
Jens Axboe [Wed, 8 Apr 2015 17:02:13 +0000 (11:02 -0600)]
Add support for normal/gaussian random distributions

Signed-off-by: Jens Axboe <axboe@fb.com>
4 years agofio-genzipf results output are slightly wrong, it doesn't help to understand how...
Fabrice Bacchella [Wed, 8 Apr 2015 14:49:42 +0000 (16:49 +0200)]
fio-genzipf results output are slightly wrong, it doesn't help to understand how to use it:
 * The last row was not displayed
 * some useless variables were used
 * the calculation for the output table missed a few events, so the sum of 'No Hits' column was wrong
 * Wrong sample in genzipf.c header, used old style arguments
 * the help message for -o options was misleading.
 * useless define of DEF_NR.
 * default of default row number to a more common value of 20,
   instead of 23.

5 years agosg: don't return success if non bdev/char is used
Jens Axboe [Tue, 31 Mar 2015 19:40:25 +0000 (13:40 -0600)]
sg: don't return success if non bdev/char is used

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoAdd support for POSIX_FADV_STREAMID
Jens Axboe [Tue, 24 Mar 2015 16:20:22 +0000 (10:20 -0600)]
Add support for POSIX_FADV_STREAMID

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoAlways update rusage before grabbing stat_mutex
Jens Axboe [Mon, 23 Mar 2015 16:19:44 +0000 (10:19 -0600)]
Always update rusage before grabbing stat_mutex

zxh reports running into a rusage_sem vs stat_mutex deadlock:

"I am running into a deadlock with fio for the following lines:

thread_main

fio_mutex_down(stat_mutex) at backend.c:1529

helper_thread_main

fio_mutex_down(td->rusage_sem) at stat.c:1467

thread_main is waiting for stat_mutex, which is already locked by
helper_thread_main in function __show_running_run_stats() at
stat.c:1441. However, the helper_thread_main is waiting for
td->rusage_sem, which is supposed to be unlocked by
check_update_rusage() in do_io() at backend.c:1525 in thread_main.

The issue is not reproducible every time, and I was using a customized
ioengine derived from rbd.c.

Is there any chance that this issue is caused by the customized io
engine? Or is there a way to get around this?"

Fix this by always updating the rusage stats before grabbing
stat_mutex, so we avoid this ABBA deadlock.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoCpu clock does not wrap if there is no CPU clock
Akash Verma [Thu, 19 Mar 2015 23:19:25 +0000 (16:19 -0700)]
Cpu clock does not wrap if there is no CPU clock

This causes compiler warnings because of unused variables cycles_start
and cycles_wrap.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoio_u_queue: add debug assert check on adding too many elements
Jens Axboe [Fri, 20 Mar 2015 04:15:02 +0000 (22:15 -0600)]
io_u_queue: add debug assert check on adding too many elements

Caught a bug in the io-threads branch, lets merge it into master.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoPack io_u/fio_file
Jens Axboe [Wed, 18 Mar 2015 22:17:41 +0000 (16:17 -0600)]
Pack io_u/fio_file

Remove two unneeded holes in the struct.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoAddedd -z option to specify file size for filepaths fio files
Luca Maranzano [Wed, 18 Mar 2015 17:00:39 +0000 (18:00 +0100)]
Addedd -z option to specify file size for filepaths fio files

5 years agoRevert "Scramble IO buffers unless explicitly told not"
Jens Axboe [Wed, 18 Mar 2015 16:31:49 +0000 (10:31 -0600)]
Revert "Scramble IO buffers unless explicitly told not"

This reverts commit 7d1bb70f3d85d041b4b7dd926e98054fea980bdc.

5 years agoUpdate documentation for zero_buffers
Jens Axboe [Wed, 18 Mar 2015 16:13:33 +0000 (10:13 -0600)]
Update documentation for zero_buffers

If it is explicitly enabled, then we don't scramble the buffers as
well, unless the user has asked for that by setting
scramble_buffers=1.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoScramble IO buffers unless explicitly told not
Jens Axboe [Wed, 18 Mar 2015 00:11:07 +0000 (18:11 -0600)]
Scramble IO buffers unless explicitly told not

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoio_u: fix refill, should be random content
Jens Axboe [Tue, 17 Mar 2015 21:57:13 +0000 (15:57 -0600)]
io_u: fix refill, should be random content

Unless we asked for zeroed buffers, we fill them randomly.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoinit: fix exit status with bogus arguments
Zev Weiss [Thu, 12 Mar 2015 20:28:17 +0000 (15:28 -0500)]
init: fix exit status with bogus arguments

Had been exiting "successfully" on invalid command-line arguments.

Signed-off-by: Zev Weiss <zev@bewilderbeest.net>
Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agosmalloc: don't crash on being passed a bad pointer
Jens Axboe [Tue, 17 Mar 2015 16:45:18 +0000 (10:45 -0600)]
smalloc: don't crash on being passed a bad pointer

This can happen if we pass sfree() memory not allocated by smalloc.
Ignore it and log an error instead of triggering an assert().

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agorate: remove redundant code in rate_ddir()
Jens Axboe [Thu, 12 Mar 2015 16:05:25 +0000 (10:05 -0600)]
rate: remove redundant code in rate_ddir()

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agosmalloc: bump initial size to 8 pools
Jens Axboe [Tue, 3 Mar 2015 22:13:17 +0000 (15:13 -0700)]
smalloc: bump initial size to 8 pools

Commit 23bd40f944b7 shrank our pool size from 128 pools at 8MB max,
to one static 16MB pool. Christian reports:

"For our tests with about 250k files we found the smalloc pool being
 depleted.  Now for us values of 3-4 would be enough, but since it is a
 compile time switch I'd like to make it safe for everybody and set 8."

Bump the pool alloc to 8 pools again, retaining the 16MB size. That's
still substantially less than before, but should be enough for most
cases. Allocate the pools at init time, to avoid the issue that the
original commit fixed.

Also fix a bug where we failed to iterate some pools when restarting
the allocation scan, resulting in less-than-optimal exhaustion of all
pools before smalloc() gave up and returned NULL.

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agofio: fix smalloc strdop allocation failure
Christian Ehrhardt [Tue, 3 Mar 2015 11:44:46 +0000 (12:44 +0100)]
fio: fix smalloc strdop allocation failure

smalloc_strdup didn't check for allocation success and thereby ran into
segfaults if the single pool went out of memory.
Now with this patch applied it is still failing, but in a more consistent
way than segfaulting.

You still get a bad allocation, but it looks like this now:
  fio: smalloc OOM
  fio: filesetup.c:1495: dup_files: Assertion `0' failed.
  Aborted

In fact the upper layers expected smalloc_strdup to retrun NULL on failure.

Signed-off-by: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agoFio 2.2.6 fio-2.2.6
Jens Axboe [Fri, 27 Feb 2015 15:30:04 +0000 (08:30 -0700)]
Fio 2.2.6

Signed-off-by: Jens Axboe <axboe@fb.com>
5 years agofio: fix cycles_start build issue
Christian Ehrhardt [Fri, 27 Feb 2015 15:20:46 +0000 (08:20 -0700)]
fio: fix cycles_start build issue

There is an issue introduced with "commit 73df3e07 gettime: offset CPU cycle
counter by initial value". For architectures which define
ARCH_CPU_CLOCK_CYCLES_PER_USEC (currently only s390) this causes a build
error

gettime.c:174:11: error: â€˜cycles_start’ undeclared (first use in this
function)
    if (t < cycles_start && !cycles_wrap)

To make sure variables and code are only compiled in the same cases I
added a
clock definition called ARCH_CPU_CLOCK_WRAPS. We could merge
ARCH_CPU_CLOCK_WRAPS and the existing ARCH_CPU_CLOCK_CYCLES_PER_USEC
into one
if you prefer, so far nobody else uses it.

To avoid cluttering all architecture headers I enabled it by default in
arch-generic.h, so any arch not needing the wrap handling can undef
ARCH_CPU_CLOCK_WRAPS later in their headers.

Signed-off-by: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>
Signed-off-by: Jens Axboe <axboe@fb.com>