Shaohua Li [Fri, 14 Sep 2012 06:49:43 +0000 (08:49 +0200)]
Add support for trim as a workload type
This only works on Linux so far, and it's always sync given what
the interface to the kernel looks like. Also restricted to pure
block devices.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Bruce Cran [Thu, 6 Sep 2012 15:08:49 +0000 (09:08 -0600)]
windows: reduce block size used in posix_fallocate()
Reduce the buffer size in posix_fallocate() to avoid
compatability problems.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 6 Sep 2012 15:07:44 +0000 (09:07 -0600)]
libaio: use INT_MAX to signal that we don't want the ring buffer
We can't use 0 since the libaio library _also_ checks the
nr_events being passed in. So lets use INT_MAX instead.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Tue, 4 Sep 2012 17:47:00 +0000 (11:47 -0600)]
libaio: pass in 0 for queue depth if we can
We don't need the user ring, unless the job file specified
that we need it. So tell the kernel. Fall back to passing
in the normal queue depth, if the kernel doesn't support
disabling the user ring.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
liang xie [Fri, 31 Aug 2012 15:22:42 +0000 (08:22 -0700)]
fio:modify a deprecated parameter(change timeout to runtime)
since "timeout" had been deprecated long long ago, we should replace
to "runtime" both in code and docs
Signed-off-by: Liang Xie <xieliang@xiaomi.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
xieliang [Fri, 31 Aug 2012 15:11:26 +0000 (08:11 -0700)]
Fixup README indentation
Some tabs were missing.
Signed-off-by: xieliang <xieliang@xiaomi.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Sun, 26 Aug 2012 17:32:22 +0000 (19:32 +0200)]
README: update for newer client job file argument format
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Sun, 26 Aug 2012 15:30:45 +0000 (17:30 +0200)]
client: support per-client job files
Currently the fio network client sends all job files to all clients.
This isn't very flexible.
Add support for specifying a specific job file for each client. Example:
$ fio --client hostname1 jobfile1 --client hostname2 jobfile2
would before add jobfile1 and jobfile2 as job files for both clients.
With this change, only jobfile1 is sent to hostname1, and only
jobfile2 is sent to hostname2.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Wed, 22 Aug 2012 18:42:46 +0000 (20:42 +0200)]
example: add SSD steady state test case
Simple test case:
- Start by filling a drive with large block sequential writes
- Once that is done, start filling it with random 4k writes.
At some point the latter phase will hit steady state. Progress
is logged.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Wed, 22 Aug 2012 06:07:34 +0000 (08:07 +0200)]
Fio 2.0.9
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Bruce Cran [Tue, 21 Aug 2012 13:45:57 +0000 (15:45 +0200)]
Windows fio: implement some more POSIX functions
Implement opendir, closedir, readdir, and basic openlog, closelog and
syslog functionality on Windows.
Remove inet_aton since it wasn't being used.
Remove warning that msync isn't implemented - just return an error.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Mon, 20 Aug 2012 12:44:22 +0000 (14:44 +0200)]
eta: make CR= display more easy to understand
Currently we sum up read and write IOPS/BW rates, but that reads
a bit strange when a given workload only does reads OR writes.
The user then thinks that fio mis-parsed the setting, reading
twice as much as expected.
Only sum up for the read OR write side if one direction is given.
For a mixed workload, it's still the sum of both.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Bruce Cran [Fri, 17 Aug 2012 13:10:04 +0000 (15:10 +0200)]
Windows fio fixes
Here's the latest set of Windows fio fixes:
Windows posix.c fixes
Reset the file pointer at the end of posix_fallocate().
Use 64-bit versions of tell() and seek().
Fix basename().
Use uint64_t instead of unsigned long long.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Dan Ehrenberg [Thu, 16 Aug 2012 06:58:21 +0000 (08:58 +0200)]
Fixing wraparound behavior for time-based sequential read jobs
Previously, a buggy patch made sequential jobs wrap around when they
reached an offset of 'size'. However, the appropriate wraparound is at
size + initial offset. This patch restores the previous behavior.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Bruce Cran [Thu, 16 Aug 2012 06:49:41 +0000 (08:49 +0200)]
fix 32-bit Windows fio (overflow in getusage and gettimeofday)
Fix overflow in getrusage and gettimeofday for 32-bit versions of fio
on Windows.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Bruce Cran [Wed, 15 Aug 2012 18:24:24 +0000 (20:24 +0200)]
fio: add nanosleep() to Windows
I've attached a patch which adds an implementation of nanosleep() to
Windows, fixing rate-limiting.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Tue, 14 Aug 2012 12:34:32 +0000 (14:34 +0200)]
fio Makefile improvement - don't override $(CC)
I tried to build fio 2.0.8 today and found that the Makefile
unconditionally sets CC.
Since I've only got clang on my FreeBSD box it needs to avoid overriding
the default.
Use the default system compiler - don't override $(CC) if already set.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 2 Aug 2012 09:21:36 +0000 (11:21 +0200)]
mutex: make 0/1 FIO_MUTEX_LOCKED and FIO_MUTEX_UNLOCKED
Makes the API look cleaner.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 2 Aug 2012 06:27:41 +0000 (08:27 +0200)]
diskutil: ensure that we lock around disk_list access
And also ensure that before we free the disk util structures,
we wait for the disk util thread to exit.
This is v2 of the patch. Commit
feb41855 had a bug where it
would deadlock on the disk_util_mutex, when handling drives
that had slaves (lvm/md).
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Wed, 1 Aug 2012 18:25:53 +0000 (20:25 +0200)]
Revert "diskutil: ensure that we lock around disk_list access"
This reverts commit
feb418556a236aa041a625b083d6b99e55d23d74.
It causes hangs, needs further testing.
Jens Axboe [Wed, 1 Aug 2012 07:39:36 +0000 (09:39 +0200)]
diskutil: ensure that we lock around disk_list access
And also ensure that before we free the disk util structures,
we wait for the disk util thread to exit.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Wade Cline [Wed, 1 Aug 2012 06:45:48 +0000 (08:45 +0200)]
Fix fallocate erroneously returning ENOSYS on Linux systems
Currently, the helpers.c file provides a function defintion for Linux
fallocate that sets errno to ENOSYS and returns failure; this is useful
for a non-Linux OS. However, this definition will override the Linux
implementation of fallocate when 'fallocate=keep' is set. Adding a
preprocessor macro to check if Linux fallocate is defined fixes this
issue.
Signed-off-by: Wade Cline <wcline@us.ibm.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Tue, 26 Jun 2012 22:20:12 +0000 (18:20 -0400)]
Merge branch 'master' of ssh://brick.kernel.dk/data/git/fio
Jens Axboe [Tue, 26 Jun 2012 20:24:36 +0000 (16:24 -0400)]
Add missing arch_mips
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 14 Jun 2012 13:11:15 +0000 (15:11 +0200)]
Include end-of-run time in the output
When results are stored, it's convenient to be able to tell
when they were run.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Mon, 11 Jun 2012 11:19:13 +0000 (13:19 +0200)]
Revert "blktrace: use for_each_file() instead of open coding the loop"
There's another bug in where we don't have the number of
files in replay mode correctly stored in td->o.nr_files. So
revert this until that bug is fixed as well.
This reverts commit
adfc73488616bac66f910d53c75202d5be8afe87.
Jens Axboe [Mon, 11 Jun 2012 06:58:42 +0000 (08:58 +0200)]
blktrace: use for_each_file() instead of open coding the loop
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Shaohua Li [Mon, 11 Jun 2012 06:56:32 +0000 (08:56 +0200)]
Fix "fio: file hash not empty on exit" with blktrace replay
There are two problems:
1. blktrace data hasn't file close action
2. redirect file fileno isn't correct
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Mon, 11 Jun 2012 06:53:53 +0000 (08:53 +0200)]
Add a more verbose/immediate warning if we fail open with O_DIRECT
If we get EINVAL with O_DIRECT open of the file, then that means
that the file system does not support direct IO. Warn as such
from fio.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Bruce Cran [Fri, 8 Jun 2012 06:12:20 +0000 (08:12 +0200)]
Windows 32-bit fio fixes
Fix no-op shift in shmget by casting to 64-bit.
Generate a new Product GUID for fio 2.0.8.
Add the 32-bit MinGW-w64 compiler executable name in Makefile.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Wed, 6 Jun 2012 06:20:27 +0000 (08:20 +0200)]
Ensure that non-git basic installs return sane version information
If git wasn't available, we'd return 2.0.8 as the version info
instead of fio-2.0.8 like we otherwise would.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Tue, 29 May 2012 11:29:12 +0000 (13:29 +0200)]
Fio 2.0.8
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Fri, 11 May 2012 18:33:02 +0000 (20:33 +0200)]
eta: ETA bandwidth was off by 1.024
Same fix as
033bbb51 essentially. Since mtime is a 1000th of a second,
for the usual option of having 1024 be the KB base, we end up being
off by 1.024 if we don't multiply by 1000 before dividing by the
runtime (and then dividing by 1024 to get to next power-of-2).
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Martin Steigerwald [Mon, 7 May 2012 15:06:54 +0000 (17:06 +0200)]
Fix hyphen as minus sign errors by escaping minus signs.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Martin Steigerwald [Mon, 7 May 2012 15:06:26 +0000 (17:06 +0200)]
Fix manpage error warning: macro `hostname'' not defined.
Fix manpage error warning: macro `hostname'' not defined by using
different line wrapping.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Martin Steigerwald [Mon, 7 May 2012 15:06:13 +0000 (17:06 +0200)]
Fix spelling error in manpage.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Mon, 7 May 2012 07:46:40 +0000 (09:46 +0200)]
Fix terse bandwidth
Off by 1.024
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Wed, 2 May 2012 12:29:21 +0000 (14:29 +0200)]
Merge branch 'master' of ssh://brick.kernel.dk/data/git/fio
Jens Axboe [Wed, 2 May 2012 11:59:09 +0000 (13:59 +0200)]
Remove deprecated rwmixcycle option from the man page
It has no effect, remove it.
Suggested-by: Roger Sibert <Roger_Sibert@xyratex.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Wed, 2 May 2012 11:58:02 +0000 (13:58 +0200)]
Add create_only option
If this option is set, then fio will only run the preparation
phase of a job. This means it will create all the files it
needs, but exit after that.
Suggested-by: Georg Schönberger <gschoenberger@thomas-krenn.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 26 Apr 2012 17:39:47 +0000 (19:39 +0200)]
Fix man page type
"one" -> "done".
Reported-by: Roger Sibert <Roger_Sibert@xyratex.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Tao Ma [Thu, 26 Apr 2012 10:15:46 +0000 (12:15 +0200)]
fio: Add to cgroup if parameter "cgroup" is enabled.
Currently, we only add the tasks to cgroup if cgroup_weight is set by
the user. But if we want to test other blkcg parameters like block
throttle, the fio tasks isn't added to the corresponding cgroup and
it doesn't make sense.
So add the task to the corresponding cgroup if the cgroup name is given.
Signed-off-by: Tao Ma <boyu.mt@taobao.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Wed, 18 Apr 2012 18:39:48 +0000 (20:39 +0200)]
Fix for crash with more than ~500 jobs
We simply overwrite our status output buffer, not a terribly
good idea.
Reported-by: Roger Sibert <Roger_Sibert@xyratex.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Bruce Cran [Mon, 16 Apr 2012 17:33:03 +0000 (19:33 +0200)]
Windows build fixes
Bruce writes:
SIGUSR1 isn't defined on Windows so I've defined it.
Also, MSI packages need a version string of x.x.x.x where x is between 0
and 65535 so we can't include the git hash. It's easier just to go back
to hard-coding the version for now, and I'll update it (and the product
GUID) when I create a new package (it doesn't need checked in each
time).
Also, Cygwin doesn't like $(SHELL_PATH) but $(SHELL) seems to work -
does $(SHELL) cause a problem on other platforms?
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Mon, 16 Apr 2012 09:44:48 +0000 (11:44 +0200)]
Enlarge probe version field
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Mon, 16 Apr 2012 07:52:22 +0000 (09:52 +0200)]
Get rid of fio_version.h
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Fri, 13 Apr 2012 17:18:49 +0000 (19:18 +0200)]
Fix for non-git versions
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Fri, 13 Apr 2012 12:06:08 +0000 (14:06 +0200)]
Ensure that init depends on FIO-VERSION-FILE
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Conflicts:
Makefile
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 12 Apr 2012 07:18:38 +0000 (09:18 +0200)]
Document the use of the USR1 signal for dumping runtime statistics
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Wed, 11 Apr 2012 20:24:19 +0000 (22:24 +0200)]
Fix repeated link of fio
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Wed, 11 Apr 2012 20:07:07 +0000 (22:07 +0200)]
Make fio include the git version in the version output
Makes it easier in bug reporting, don't have to ask people what
they are running...
Here's to you, Kep.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Conflicts:
Makefile
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Fri, 30 Mar 2012 08:30:35 +0000 (10:30 +0200)]
Define SIGUSR1 to inform fio to dump run stats while continuing to run
For long running jobs, it may be interesting to see the stats without
having to terminate the job. Set up SIGUSR1 so that it does that,
gives you the stats at that point in time.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Fri, 30 Mar 2012 06:33:27 +0000 (08:33 +0200)]
Free disk util on exit, not on stats display
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Wed, 11 Apr 2012 12:49:06 +0000 (14:49 +0200)]
Fio 2.0.7
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Lucian Adrian Grijincu [Fri, 6 Apr 2012 00:18:35 +0000 (18:18 -0600)]
HOWTO: typo: write_log_log -> write_lat_log + use suffix
- I guess 'write_log_log' was a copy-pasta typo.
- You used 'suffix' everywhere except for two places where you said
'postfix'.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 5 Apr 2012 14:42:11 +0000 (08:42 -0600)]
group reporting: fix bad values of min/max
If you look at the fio output, the group reporting min and max
bandwidth don't match what the actual job reports. This is due
to an error in calculating the min and max values. The aggregate
value is correct, and matches what the job reports.
This did not affect the values that are usually used for reporting,
it's only affecting the group run status output.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Wed, 4 Apr 2012 20:11:58 +0000 (14:11 -0600)]
iolog: remove assert in io_u overlap
This can happen very rarely for time based jobs. Before we
had this as an assert since it was an impossible to hit
condition, but now it can be expected very rarely with
verify and time based jobs.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Bruce Cran [Wed, 4 Apr 2012 14:35:13 +0000 (08:35 -0600)]
Windows fixes
Only return from fio_windowsaio_getevents if the minimum number of IOs has
completed.
Add posix_fallocate implementation to avoid extending the file during the test.
Move call to ResetEvent into windowsaio_getevents where other reset code is
located.
Fix tabs vs. spaces in windowsaio.c and the installer sources.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Bruce Cran [Wed, 4 Apr 2012 00:00:00 +0000 (18:00 -0600)]
Fix address truncation on Windows
Avoid having PAGE_ALIGN truncate addresses by using a pointer-sized
variable for the page size and mask.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Lucian Adrian Grijincu [Tue, 3 Apr 2012 20:42:48 +0000 (14:42 -0600)]
HOWTO: emphasize units in --minimal mode: latency in usec, bw in KB/s
It wasn't clear what unit were used in --minimal mode.
Signed-off-by: Lucian Adrian Grijincu <lucian@fb.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Mon, 2 Apr 2012 16:47:09 +0000 (09:47 -0700)]
Add indication of whether a job got killed
We have 'X' for exited with error, add 'K' for killed by
signal as well.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Mon, 2 Apr 2012 14:03:26 +0000 (07:03 -0700)]
HOWTO: update to note base of latencies in minimal/terse output
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Fri, 30 Mar 2012 19:21:20 +0000 (21:21 +0200)]
Enhance the reaped status display
Instead of just printing a '_' for a job that has exited,
allow the user to tell if the job exited with an error or
not. A normally reaped job that ran to completion without
errors will have the '_' displayed. If the job exited with
an error, then an 'X' will be displayed instead.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Fri, 23 Mar 2012 18:22:21 +0000 (19:22 +0100)]
stat: move final \n of terse output to the end
Don't print a new line, description, then another new line.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Fri, 23 Mar 2012 12:43:19 +0000 (13:43 +0100)]
Add regression test for
ae2fafc8
Signed-off-by: Jens Axboe <axboe@kernel.dk>
webee [Fri, 23 Mar 2012 12:41:41 +0000 (13:41 +0100)]
verify: verify bytes should not add to this_io_bytes
When rw=readwrite,verify+backlog, the verify are treated as normal
read, and bytes are added to this_io_bytes, cause fio terminate early.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Fri, 23 Mar 2012 12:40:06 +0000 (13:40 +0100)]
Allow readwrite as an alias for rw in setting sequential read/write
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Fri, 23 Mar 2012 07:09:44 +0000 (08:09 +0100)]
Fio 2.0.6
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Vikram Seth [Fri, 23 Mar 2012 07:09:09 +0000 (08:09 +0100)]
Fix terse output with description set
I have seen that using 2.05 version of fio the terse output formatting
has an issue when there is a text description given in the test file.
There is no newline separating the description and the next job stats.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Steven Lang [Sat, 17 Mar 2012 12:16:28 +0000 (13:16 +0100)]
Added 99.95 and 99.99 default percentiles.
Since there are 20 slots for percentiles, but not all used by default,
I figure it doesn't hurt to add a couple extra percentiles to the
default for tail latency capture.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Sat, 17 Mar 2012 11:59:12 +0000 (12:59 +0100)]
Add regression test for
37cf9e3c
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Sat, 17 Mar 2012 11:54:30 +0000 (12:54 +0100)]
Enable read/write sequential offset to be a non-multiple of the block size
The way that fio does offset math internally, if a non-multiple
of the blocksize was given as a sequence offset, it would simply
be ignored due to the block offset division rounding it down.
In other words, if you did:
rw=read:512k
bs=1m
it did not read 1m and 0 offset, 1m at 512k offset, etc. Instead
the hole is ignored.
Fix this by leaving the generated sequential offset in bytes
instead of turning it into a 'fio block'.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Daniel Ehrenberg [Fri, 16 Mar 2012 17:54:15 +0000 (18:54 +0100)]
time_based: Avoid restarting main I/O loop
Previously, when fio had written a volume of I/O equal to the size
argument, it restarted the main do_io loop.
This patch allows time_based tests to be run for longer than one
cycle in the do_io main loop. This has a couple of advantages:
* The random number generator is not reset on each iteration
of the loop, so running longer will reach different locations.
* There is not a throughput-reducing point where all operations
must be reaped before new operations are submitted.
The implementation consists of two minor changes:
* In the do_io loop, a time_based test will not exit the loop for
reading or writing too much data.
* When reading or writing sequentially, the operations wrap around
to the beginning after reading the end within the
get_next_seq_block function.
Modified by Jens to properly restart when using a randommap
as well.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Fri, 16 Mar 2012 12:55:50 +0000 (13:55 +0100)]
Fix problem with iodepth_batch_complete=0 and accounting
If the queue is full, we always need to reap at least one event.
Otherwise we end up failing get_io_u() and restarting the file.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Fri, 16 Mar 2012 12:55:27 +0000 (13:55 +0100)]
Fix wrap bug in mtime_since()
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 15 Mar 2012 14:02:56 +0000 (15:02 +0100)]
Document compress_buffer_percentage, compress_buffer_chunk
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 15 Mar 2012 13:53:38 +0000 (14:53 +0100)]
Note offset_increment parent option
And move it up near "offset".
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 15 Mar 2012 13:50:58 +0000 (14:50 +0100)]
Fix offset_increment option and add man page entry for it
td->thread_number starts at 1, not 0. So we need to subtract
one before multiplying with the offset.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Yufei Ren [Thu, 15 Mar 2012 13:44:47 +0000 (14:44 +0100)]
rdma engine graceful teardown
For rdma ioengine's one side operation, RDMA_WRITE or RDMA_READ,
server side gets a message from client side that the task is finished.
In previous version, the server simply exit() in td_io_commit() which
causes segmentation fault in thread mode or process hanging in process
mode. This patch achieves graceful tear down by setting up `td->done`
as UDP engine did.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Dan Ehrenberg [Thu, 15 Mar 2012 13:44:26 +0000 (14:44 +0100)]
New offset_increment option
This patch adds a new option to fio job files. It is described
in the HOWTO as follows:
offset_increment=int If this is provided, then the real offset becomes
the offset + offset_increment * thread_number, where the
thread number is a counter that starts at 0 and is incremented
for each job. This option is useful if there are several jobs
which are intended to operate on a file in parallel in disjoint
segments, with even spacing between the starting points.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Tue, 13 Mar 2012 12:55:38 +0000 (13:55 +0100)]
Add regression test for
82af2a7c
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Tue, 13 Mar 2012 12:45:58 +0000 (13:45 +0100)]
Fix failure to verify in mixed read/write workload with backlog
If you run a workload like this:
fio --rw=randrw --bs=4k --direct=1 --ioengine=libaio --iodepth=32
--verify=meta --verify_backlog=1024 --verify_fatal=1 --name=ver-test
--filename=foo --size=1G --verify_pattern=0xaaa
Fio ends up never actually verifying the written blocks. This happens
because as we generate an entry to be verified, the backend checks
whether this is a read/write mixed workload. It then thinks that the
READ is just that, a normal READ, when in fact it could be coming
from our verify list.
Mark such a read as coming from our verify list, so that we know if
it's a "normal" read or one generated to verify previously written
data.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Fri, 9 Mar 2012 18:11:25 +0000 (19:11 +0100)]
Mark buffer_compress_chunk as child of buffer_compress_percentage
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Fri, 9 Mar 2012 18:06:24 +0000 (19:06 +0100)]
Add buffer_compress_chunk option
Instead of doing bs/2 of incompressible and zeroed data for a 50%
compression setting, use the smaller chunk size specified by
this option.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Fri, 9 Mar 2012 18:02:01 +0000 (19:02 +0100)]
Honor refill_buffers even for verifies
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Fri, 9 Mar 2012 18:00:31 +0000 (19:00 +0100)]
Compression buffer fill fix
We need to advance after the zero fill as well.
Reported-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Wed, 7 Mar 2012 08:32:58 +0000 (09:32 +0100)]
Fix batch mismatch for verifies
If we set verify_backlog=32, we expect 32 writes followed by 32
reads that verify the written data. But if we fail getting a
verify io_u, then we may decrement the batch counter when we should
not. The result is that we write 32, read 32, write 1, read 1, repeat.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Wed, 7 Mar 2012 08:26:13 +0000 (09:26 +0100)]
Add regression test for
f7078f7b
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Wed, 7 Mar 2012 08:24:05 +0000 (09:24 +0100)]
Fix bug that causes early termination of fio with verify_backlog
Running this job:
[global]
bs=4k
ioengine=sync
size=100m
direct=1
filename=xxx
[write]
verify=md5
verify_backlog=32
rw=write
causes fio to exit when reads + writes have exceeded 100m, where
it should only exit when writes have exceeded 100m. Fix this by
checking the appropriate counters based on what type of job it is.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Tue, 6 Mar 2012 18:34:22 +0000 (19:34 +0100)]
server: fix bad win32 indentation
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Tue, 6 Mar 2012 18:26:42 +0000 (19:26 +0100)]
Add regression test for
8a99fdf6
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Tue, 6 Mar 2012 18:24:49 +0000 (19:24 +0100)]
Fix another verify segfault
If you run this job:
[global]
ioengine=libaio
direct=1
filename=/tmp/foo
iodepth=128
size=10M
loops=1
group_reporting=1
readwrite=write
do_verify=0
verify=md5
numjobs=1
thread
verify_dump=1
[small_writes]
offset=0G
blocksize=512
verify_interval=1M
[large_writes]
stonewall
offset=1G
blocksize=1M
verify_interval=512
fio crashes because verify_interval is larger than the block size
in the small_writes job. Check for this in get_hdr_inc().
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Tue, 6 Mar 2012 18:23:54 +0000 (19:23 +0100)]
Add regression test for verify bug
It was a bug in the job files, not a zone bug after all.
The two block sizes were transposed between the pre and
post job.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Tue, 6 Mar 2012 16:46:44 +0000 (17:46 +0100)]
Avoid segfault if verify header appears longer than io_u
Running this job:
[global]
ioengine=libaio
direct=1
filename=/tmp/foo
iodepth=128
size=10M
loops=1
group_reporting=1
readwrite=write
do_verify=0
verify=md5
numjobs=1
thread
verify_dump=1
[small_writes]
offset=0G
blocksize=512
[large_writes]
offset=1G
blocksize=1M
Followed by this job:
[global]
ioengine=libaio
direct=1
filename=/tmp/foo
iodepth=128
size=1M
loops=1
group_reporting=1
readwrite=read
do_verify=1
verify=md5
verify_fatal=1
numjobs=1
thread
verify_dump=1
bs=4k
[large_reads]
offset=0G
blocksize=1M
[small_reads]
offset=1G
blocksize=512
Will cause a segfault, since the bs=512 job appears to stumble
upon a 1mb header, making fio attempt to verify a much bigger
area than what we have read.
This looks like a zone bug. For now, check in a fix that makes
fio validate the verify header a bit harder.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Tue, 6 Mar 2012 14:42:31 +0000 (15:42 +0100)]
Add client references
We have a use-after-free in the fio_handle_clients() loop.
If we receive a QUIT command, we remove the client in
fio_handle_client(). But fio_handle_clients() doesn't
have a way to detect this, so it checks client->error
after it has potentially been freed.
Add a simple reference to get rid of this problem.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Tue, 6 Mar 2012 14:37:40 +0000 (15:37 +0100)]
server: fixup bug in sending jobs_eta command
We stored nr_running in elapsed_sec, causing ETA to only
be reliable on the client when the job neared completion.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Fri, 2 Mar 2012 21:23:36 +0000 (22:23 +0100)]
Compression fixes
- memset() remainder of buffer instead of copying random segment.
Provides more reliable compression (yes yes, you told me so).
- Allow 100 as well, just means full memset. Make that just set
zero_buffers=1 instead, since we only need to do it once
instead of for every write.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Fri, 2 Mar 2012 20:02:12 +0000 (21:02 +0100)]
Add buffer_compress_percentage
The option is pending testing, so not documented yet.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Thu, 1 Mar 2012 15:07:46 +0000 (16:07 +0100)]
linux: fallocate() fixes
Enable posix_fallocate() for glibc >= 2.6 as before, but
only enable fallocate() from 2.8 and up.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Jens Axboe [Wed, 29 Feb 2012 09:01:34 +0000 (10:01 +0100)]
Rename crc functions to private namespace
Apparently we overlap with some gtk (?) crc32() functions, causing
ours to be called and subsequently crashing since the arguments
aren't the same.
Move everything to a fio_ protected namespace.
Signed-off-by: Jens Axboe <axboe@kernel.dk>