fio.git
11 years agoFio 1.20-rc1 fio-1.20-rc1
Jens Axboe [Tue, 4 Mar 2008 11:59:41 +0000 (12:59 +0100)]
Fio 1.20-rc1

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoRevamp file locking
Jens Axboe [Tue, 4 Mar 2008 09:18:56 +0000 (10:18 +0100)]
Revamp file locking

Get rid of the semaphore implementation, no need to carry both.
Add different locking modes (exclusive and readwrite) to enable
a wider range of testing. Also combine lockfile and lockfile_batch,
the latter is now a postfix option to the former.

So to enable readers-excluding-writers locking mode with a lock batch
count of 4, you would write:

lockfile=readwrite:4

instead.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agofilehash.c: remove debug hash dump
Jens Axboe [Mon, 3 Mar 2008 11:34:35 +0000 (12:34 +0100)]
filehash.c: remove debug hash dump

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agosem.c: cleanup
Jens Axboe [Mon, 3 Mar 2008 11:32:45 +0000 (12:32 +0100)]
sem.c: cleanup

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoMakefile updates
Jens Axboe [Mon, 3 Mar 2008 11:22:19 +0000 (12:22 +0100)]
Makefile updates

Last was botched.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoMakefile updates
Jens Axboe [Mon, 3 Mar 2008 11:20:26 +0000 (12:20 +0100)]
Makefile updates

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoAdd rw mutex init helper
Jens Axboe [Mon, 3 Mar 2008 10:08:19 +0000 (11:08 +0100)]
Add rw mutex init helper

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoUse read-writer locks in smalloc
Jens Axboe [Mon, 3 Mar 2008 09:36:36 +0000 (10:36 +0100)]
Use read-writer locks in smalloc

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoAdd read-write mutexes
Jens Axboe [Mon, 3 Mar 2008 09:36:27 +0000 (10:36 +0100)]
Add read-write mutexes

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoRemove debug printf() in net engine
Jens Axboe [Mon, 3 Mar 2008 09:18:09 +0000 (10:18 +0100)]
Remove debug printf() in net engine

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoAdd the file sharing bits
Jens Axboe [Sat, 1 Mar 2008 18:25:20 +0000 (19:25 +0100)]
Add the file sharing bits

When you use the same filename for several jobs now, they will share
the same file structure. Enable locking through two new options:

- lockfile. If set, a semaphore is associated with the file and it is
  held from ->prep() to ->queue() has done its work.
- lockfile_batch. This controls how many IOs the job gets to do per
  semaphore acqusition.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoHook up the file hashing
Jens Axboe [Sat, 1 Mar 2008 17:59:51 +0000 (18:59 +0100)]
Hook up the file hashing

Add on file open, remove on close.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoMissed fio.h update for filehash, oops
Jens Axboe [Sat, 1 Mar 2008 17:58:08 +0000 (18:58 +0100)]
Missed fio.h update for filehash, oops

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoAdd file hashing helpers
Jens Axboe [Sat, 1 Mar 2008 17:56:24 +0000 (18:56 +0100)]
Add file hashing helpers

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoRemember to init and exit the smalloc allocator
Jens Axboe [Sat, 1 Mar 2008 17:52:49 +0000 (18:52 +0100)]
Remember to init and exit the smalloc allocator

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoJobs must always open their own files
Jens Axboe [Sat, 1 Mar 2008 17:50:48 +0000 (18:50 +0100)]
Jobs must always open their own files

This is also related to file sharing - we cannot dup() a descriptor,
if the child hasn't inherited it. So make sure the jobs always open
the files again, even after they have been created.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoAdd list_del_init() to list.h
Jens Axboe [Sat, 1 Mar 2008 17:44:26 +0000 (18:44 +0100)]
Add list_del_init() to list.h

The file hashing will use it.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoRemember to free the file structure as well
Jens Axboe [Sat, 1 Mar 2008 17:41:50 +0000 (18:41 +0100)]
Remember to free the file structure as well

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoAllocate file structure from shared process pool
Jens Axboe [Sat, 1 Mar 2008 17:40:46 +0000 (18:40 +0100)]
Allocate file structure from shared process pool

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoAdd a semaphore implementation
Jens Axboe [Sat, 1 Mar 2008 17:38:12 +0000 (18:38 +0100)]
Add a semaphore implementation

To be used by the file sharing.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoAdd a really simple allocator, backed with mmap'ed memory
Jens Axboe [Sat, 1 Mar 2008 17:27:36 +0000 (18:27 +0100)]
Add a really simple allocator, backed with mmap'ed memory

We can use this simple allocator to get memory that can be safely
shared across processes or threads.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoMissed update of dup_files() for dynamically allocated files
Jens Axboe [Sat, 1 Mar 2008 17:22:27 +0000 (18:22 +0100)]
Missed update of dup_files() for dynamically allocated files

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoAdd file locking hooks
Jens Axboe [Sat, 1 Mar 2008 17:19:52 +0000 (18:19 +0100)]
Add file locking hooks

Does nothing so far, but adds locking calls that cover from ->prep()
to after ->queue(). That is the period where we do IO.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoRemove the file->last_completed_pos variable
Jens Axboe [Sat, 1 Mar 2008 17:09:49 +0000 (18:09 +0100)]
Remove the file->last_completed_pos variable

This tracks where we last did IO to this file, however with file
sharing that may break. So just remove this optimization, it'll
cost an extra lseek() for the sync engine but doesn't do much else
outside of that.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoMake file structures dynamically allocated
Jens Axboe [Sat, 1 Mar 2008 17:04:31 +0000 (18:04 +0100)]
Make file structures dynamically allocated

Current td->files is an array of files, make it an array of pointers
instead and allocate individual file structures on the fly. This is
a preparation patch for file sharing.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoEnable quiet build mode, enable by default
Jens Axboe [Sat, 1 Mar 2008 15:20:13 +0000 (16:20 +0100)]
Enable quiet build mode, enable by default

Add V=1 to make flags to force a verbose build.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoUpdate close file handler to return potential error
Jens Axboe [Sat, 1 Mar 2008 14:55:36 +0000 (15:55 +0100)]
Update close file handler to return potential error

Filesystems like NFS do return errors on close(), up until now we
have been ignoring them. Fix that. Adjust io_ops engine version
to 9, since this is an API change.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoAdd close_files() that closes all files
Jens Axboe [Sat, 1 Mar 2008 14:47:08 +0000 (15:47 +0100)]
Add close_files() that closes all files

Renames the current implementation to close_and_free_files(), since
that is what it actually does.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoOnly initialize random map if we are actually going to use it
Jens Axboe [Sat, 1 Mar 2008 14:34:44 +0000 (15:34 +0100)]
Only initialize random map if we are actually going to use it

We alloc the randommap even for sequential workloads right now,
if the 'norandommap' option isn't given. Fix that, check for
td_random() before continuing.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoFix potential crash in terminate_threads()
Jens Axboe [Sat, 1 Mar 2008 14:25:24 +0000 (15:25 +0100)]
Fix potential crash in terminate_threads()

td->io_ops can be NULL, if the thread is already gone. So copy the
pointer and check before dereferencing it.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoWhen retrieving a requeued IO, don't setup data pointers again
Jens Axboe [Sat, 1 Mar 2008 14:22:32 +0000 (15:22 +0100)]
When retrieving a requeued IO, don't setup data pointers again

This is a bug, if we had a residual data count we would be
retrying the full request again.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoFix bad inlining in mutex.h
Jens Axboe [Sat, 1 Mar 2008 14:12:48 +0000 (15:12 +0100)]
Fix bad inlining in mutex.h

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoFix recursive dump of options
Jens Axboe [Sat, 1 Mar 2008 14:12:14 +0000 (15:12 +0100)]
Fix recursive dump of options

If a child also had a child, we would miss it.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoConfusion between mutex and semaphore naming
Jens Axboe [Wed, 27 Feb 2008 17:58:00 +0000 (18:58 +0100)]
Confusion between mutex and semaphore naming

There's a semaphore implementation and option coming, so rename the
bits.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoAdd --debug=parse for option parsing debug
Jens Axboe [Wed, 27 Feb 2008 17:32:33 +0000 (18:32 +0100)]
Add --debug=parse for option parsing debug

Adds log.h and debug.h to split the logging and debug bits out of
fio, so that the parser can use them.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agofio.1: add section option and remove spurious command text
Aaron Carroll [Tue, 26 Feb 2008 22:10:39 +0000 (23:10 +0100)]
fio.1: add section option and remove spurious command text

Signed-off-by: Aaron Carroll <aaronc@gelato.unsw.edu.au>
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoextend_file need continue when the file exists
Zhang, Yanmin [Tue, 26 Feb 2008 14:35:52 +0000 (15:35 +0100)]
extend_file need continue when the file exists

If some testing files exist and some others don't exist, fio will
report failure on the no-exist file. Here is a patch to fix it.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoMove syslet/indirect bits into the OS specific directory
Jens Axboe [Sun, 24 Feb 2008 20:36:00 +0000 (21:36 +0100)]
Move syslet/indirect bits into the OS specific directory

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoRemove dead FIO_FILE_EXISTS flag
Jens Axboe [Thu, 21 Feb 2008 09:20:00 +0000 (10:20 +0100)]
Remove dead FIO_FILE_EXISTS flag

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoprune_io_piece_log() also needs to take list into account
Jens Axboe [Thu, 21 Feb 2008 08:52:35 +0000 (09:52 +0100)]
prune_io_piece_log() also needs to take list into account

If we didn't use the rbtree but have remaining entries to prune,
loop over the list as well and free those entries.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoRemove dead FIO_FILE_NOSORT flag
Jens Axboe [Thu, 21 Feb 2008 08:51:32 +0000 (09:51 +0100)]
Remove dead FIO_FILE_NOSORT flag

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agofix hugepage-size type mismatch
Aaron Carroll [Wed, 20 Feb 2008 08:14:13 +0000 (09:14 +0100)]
fix hugepage-size type mismatch

hugepage_size in thread_options is of type int, but listed in
options.c as type long.  This patch reconciles the two.

Signed-off-by: Aaron Carroll <aaronc@gelato.unsw.edu.au>
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoaccount for delayed jobs when reaping threads
Aaron Carroll [Wed, 20 Feb 2008 08:14:12 +0000 (09:14 +0100)]
account for delayed jobs when reaping threads

reap_threads does not account for jobs that have been delayed
(e.g. via startdelay) which may lead to all threads being
terminated.  The following job file demonstrates the problem:

[job1]
filename=job1file
size=16k
startdelay=1

Signed-off-by: Aaron Carroll <aaronc@gelato.unsw.edu.au>
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoFio 1.19 fio-1.19
Jens Axboe [Tue, 19 Feb 2008 19:24:10 +0000 (20:24 +0100)]
Fio 1.19

1.18x had some stupid bugs with random IO generation, so release
a new full version.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoDon't always print usage info
Jens Axboe [Tue, 19 Feb 2008 19:16:57 +0000 (20:16 +0100)]
Don't always print usage info

For instance, when we ask for help don't complain about missing job.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoMention new --debug=random option in README
Jens Axboe [Tue, 19 Feb 2008 19:11:41 +0000 (20:11 +0100)]
Mention new --debug=random option in README

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoMore random fixes
Jens Axboe [Tue, 19 Feb 2008 19:10:26 +0000 (20:10 +0100)]
More random fixes

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoFix TO_MAP_BLOCK() taking offset into account twice
Jens Axboe [Tue, 19 Feb 2008 17:20:14 +0000 (18:20 +0100)]
Fix TO_MAP_BLOCK() taking offset into account twice

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoAdd FIO_SIGQUIT engine option
Jens Axboe [Mon, 18 Feb 2008 20:11:24 +0000 (21:11 +0100)]
Add FIO_SIGQUIT engine option

For network engines, we need a signal to make it abort the connection
if it isn't already running. Otherwise fio would just stall on ctrl-c
before the connection was established.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoCleanup parse_cmd_line()
Jens Axboe [Mon, 18 Feb 2008 19:59:18 +0000 (20:59 +0100)]
Cleanup parse_cmd_line()

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoAdd --section command line option
Jens Axboe [Mon, 18 Feb 2008 19:53:47 +0000 (20:53 +0100)]
Add --section command line option

It enables the user to ship bigger and more complex job files, while
still allowing fio to only run a part of it.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoExit on bad command line options
Jens Axboe [Mon, 18 Feb 2008 19:26:32 +0000 (20:26 +0100)]
Exit on bad command line options

It could be dangerous, eg if you expected --read-only to do its job
when you really needed to type --readonly.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoAdd/correct missing command line options in help text
Jens Axboe [Mon, 18 Feb 2008 19:15:03 +0000 (20:15 +0100)]
Add/correct missing command line options in help text

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoFio 1.18.1 fio-1.18.1
Jens Axboe [Mon, 18 Feb 2008 19:00:08 +0000 (20:00 +0100)]
Fio 1.18.1

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoFix typo in mark_random_map()
Jens Axboe [Mon, 18 Feb 2008 13:14:53 +0000 (14:14 +0100)]
Fix typo in mark_random_map()

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoFix random location generation with offset != 0
Jens Axboe [Mon, 18 Feb 2008 13:13:08 +0000 (14:13 +0100)]
Fix random location generation with offset != 0

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agofio: HOWTO update
Gurudas Pai [Fri, 8 Feb 2008 07:50:14 +0000 (08:50 +0100)]
fio: HOWTO update

In case of  cpuio , on smp machines we need increase numjobs
to <num of cpu's> to get the desired load across cpu's.
Just updated HOWTO with this info.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoVerify doesn't really work on mixed read/write workloads, so warn
Jens Axboe [Wed, 6 Feb 2008 13:40:14 +0000 (14:40 +0100)]
Verify doesn't really work on mixed read/write workloads, so warn

Unless the file is pre-populated, this cannot work. So warn the user
of such a bad construct, since the warning he/she receives from fio
only tells you about a verification error (which, technically, is what
is happening).

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoFix misplaced exit(0) in cpus_allowed setting
Jens Axboe [Wed, 6 Feb 2008 13:17:30 +0000 (14:17 +0100)]
Fix misplaced exit(0) in cpus_allowed setting

Must have been a leftover from some debug session. It prevented any
job file with cpus_allowed set from running.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agofio: minor HOWTO fix
Gurudas Pai [Wed, 6 Feb 2008 10:16:15 +0000 (11:16 +0100)]
fio: minor HOWTO fix

writev(3) should be writev(2)

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoman page update
Jens Axboe [Tue, 5 Feb 2008 11:02:07 +0000 (12:02 +0100)]
man page update

--debug command line and bssplit job option missing. Add vsync io engine
and clarify bssrange.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoFio 1.18 fio-1.18
Jens Axboe [Tue, 5 Feb 2008 09:23:54 +0000 (10:23 +0100)]
Fio 1.18

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoFix offset retrieval problem for read and write having different bs ranges
Jens Axboe [Tue, 5 Feb 2008 09:05:50 +0000 (10:05 +0100)]
Fix offset retrieval problem for read and write having different bs ranges

"Zhang, Yanmin" <yanmin_zhang@linux.intel.com> reported that fio got
stuck with a job file that had different read and write block size
ranges. The problem is that get_next_offset() didn't take the data
direction into account, so when it generated an offset close to the
end of the file, it may not have been legal for that data direction.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoparser: always set off3/4 even if we have more
Jens Axboe [Tue, 5 Feb 2008 08:57:39 +0000 (09:57 +0100)]
parser: always set off3/4 even if we have more

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agosync engine: missing fsync check in vsync
Jens Axboe [Mon, 4 Feb 2008 14:58:24 +0000 (15:58 +0100)]
sync engine: missing fsync check in vsync

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoAdd vsync io engine
Jens Axboe [Mon, 4 Feb 2008 09:59:07 +0000 (10:59 +0100)]
Add vsync io engine

It uses readv/writev to transfer the data and coalesces adjacent
data into a single system call (emulating queueing).

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoImprove depth marking
Jens Axboe [Mon, 4 Feb 2008 09:56:26 +0000 (10:56 +0100)]
Improve depth marking

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoDecrement io_issue count when requeuing an io_u
Jens Axboe [Mon, 4 Feb 2008 09:56:07 +0000 (10:56 +0100)]
Decrement io_issue count when requeuing an io_u

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoAdd debug trace for io_u_queue_complete()
Jens Axboe [Mon, 4 Feb 2008 09:49:41 +0000 (10:49 +0100)]
Add debug trace for io_u_queue_complete()

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoDon't allow FIO_SYNCIO io engine flag to override queue depth
Jens Axboe [Mon, 4 Feb 2008 09:48:13 +0000 (10:48 +0100)]
Don't allow FIO_SYNCIO io engine flag to override queue depth

Another bad check, just kill it.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agosync engine: cleanup psync/sync seperation
Jens Axboe [Mon, 4 Feb 2008 08:35:30 +0000 (09:35 +0100)]
sync engine: cleanup psync/sync seperation

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoDon't force iodepth == 1 for SYNC io engine
Jens Axboe [Mon, 4 Feb 2008 08:35:03 +0000 (09:35 +0100)]
Don't force iodepth == 1 for SYNC io engine

Most will not do queuing, but allow it to be set in case some of
them do.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoFix bug in fio mutex initialization
Zhang, Yanmin [Mon, 4 Feb 2008 08:17:52 +0000 (09:17 +0100)]
Fix bug in fio mutex initialization

Fio sem->cond isn't initialized properly.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoMerge branch 'master' of ssh://git.kernel.dk/data/git/fio
Jens Axboe [Fri, 1 Feb 2008 20:51:29 +0000 (21:51 +0100)]
Merge branch 'master' of ssh://git.kernel.dk/data/git/fio

11 years agoA few debug debug log fixes
Jens Axboe [Fri, 1 Feb 2008 20:51:02 +0000 (21:51 +0100)]
A few debug debug log fixes

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoAdd option for not including the tracing stuff
Jens Axboe [Fri, 1 Feb 2008 19:37:09 +0000 (20:37 +0100)]
Add option for not including the tracing stuff

It eats some space.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoAdd blktrace and verify debug tracing
Jens Axboe [Fri, 1 Feb 2008 19:27:52 +0000 (20:27 +0100)]
Add blktrace and verify debug tracing

Also add "all" as a keyword for setting all values, and pretty up
the output a bit by justifying and aligning the output.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoCPU burn engine fix
Jens Axboe [Fri, 1 Feb 2008 17:33:23 +0000 (18:33 +0100)]
CPU burn engine fix

It's ok to have zero buflen, if we are not doing real IO.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoA bunch of fixes
Jens Axboe [Fri, 1 Feb 2008 12:16:38 +0000 (13:16 +0100)]
A bunch of fixes

Really should have been split up, but...

- Check this_io_bytes at the bottom of do_io() so that async engines
  have a chance to queue pending IO before deeming this job done.

- dprint() should use log_info(), may not be stdout we want.

- last block bug in get_next_free_block(), if max blocks wasn't a
  multiple of the bitmap size.

- Add more debug points.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoTypo in 'init' debug option mask
Jens Axboe [Fri, 1 Feb 2008 12:12:28 +0000 (13:12 +0100)]
Typo in 'init' debug option mask

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoAdd --debug for enabling internal dumps on various actions
Jens Axboe [Fri, 1 Feb 2008 09:30:20 +0000 (10:30 +0100)]
Add --debug for enabling internal dumps on various actions

A little weak currently, when it's fully integrated everywhere in
fio it'll help find fio problems that I can't trigger.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoStop job if it fails to do any IO for a new invocation
Jens Axboe [Fri, 1 Feb 2008 08:29:35 +0000 (09:29 +0100)]
Stop job if it fails to do any IO for a new invocation

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoFio 1.17.3 fio-1.17.3
Jens Axboe [Thu, 31 Jan 2008 13:39:16 +0000 (14:39 +0100)]
Fio 1.17.3

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoMerge branch 'master' of ssh://router/data/git/fio
Jens Axboe [Thu, 31 Jan 2008 12:28:40 +0000 (13:28 +0100)]
Merge branch 'master' of ssh://router/data/git/fio

11 years agoFix wrong accouning of zone bytes
Jens Axboe [Thu, 31 Jan 2008 12:25:42 +0000 (13:25 +0100)]
Fix wrong accouning of zone bytes

Ryan Thomas <Ryan.Thomas@nuance.com> writes:

**********

With the following job description

[global]
bs=1k
direct=1
rw=read
ioengine=libaio
iodepth=2
zonesize=1k
zoneskip=1023k
write_bw_log

[/dev/cciss/c0d1]
write_iolog=foo2

The idea here is that I wanted to line up my zones to start at 1M
boundaries across the disk by writing 1k and skipping the next 1023k.
In practice I don't get the alignment because of an extra initial I/O.
I get an iolog that looks like

fio version 2 iolog
/dev/cciss/c0d1 add
/dev/cciss/c0d1 open
/dev/cciss/c0d1 read 0 1024
/dev/cciss/c0d1 read 1024 1024
/dev/cciss/c0d1 read 1049600 1024
/dev/cciss/c0d1 read 2098176 1024

There's a read that I don't expect in that log, namely the read starting
at byte 1024.  Because that read is there, the disk zones get offset by
one block.  I expected output like

fio version 2 iolog
/dev/cciss/c0d1 add
/dev/cciss/c0d1 open
/dev/cciss/c0d1 read 0 1024
/dev/cciss/c0d1 read 1048576 1024
/dev/cciss/c0d1 read 2097152 1024
/dev/cciss/c0d1 read 3145728 1024

***********

The problem is due to the fact that fio account zone_bytes when the IO
completes, which is clearly not correct for io depth > 1. So move the
accounting to when we fill the io_u instead.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoRandom IO fixes
Jens Axboe [Fri, 25 Jan 2008 13:02:15 +0000 (14:02 +0100)]
Random IO fixes

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoMore file extend fixes
Jens Axboe [Thu, 24 Jan 2008 12:13:12 +0000 (13:13 +0100)]
More file extend fixes

Simplify the logic a bit, hope this works...

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agofile extent fix
Gurudas Pai [Thu, 24 Jan 2008 12:04:44 +0000 (13:04 +0100)]
file extent fix

I ran a test with latest fio using jobfile,

[global]
size=10m
[job1]
ioengine=sync
rw=write
filename=testfile

And second time,
[global]
size=30m --> size increased.
[job1]
ioengine=sync
rw=write
filename=testfile

fio did not extend the file to 30m.

In case of f->file_offset=0, f->io_size equals to f->real_file_size, hence no extend.
Following patch worked for me. But this patch will not handle the case where we have
to extend a file and with offset :(

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoUpdate iodepth_batch documentation
Jens Axboe [Fri, 18 Jan 2008 09:30:07 +0000 (10:30 +0100)]
Update iodepth_batch documentation

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoMake iodepth_batch=1 by default
Jens Axboe [Fri, 18 Jan 2008 09:28:11 +0000 (10:28 +0100)]
Make iodepth_batch=1 by default

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agolibio: fix bug in commit hook
Jens Axboe [Fri, 18 Jan 2008 09:26:58 +0000 (10:26 +0100)]
libio: fix bug in commit hook

Also simplifies it.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agosurface-scan: make bs 64k again
Jens Axboe [Wed, 16 Jan 2008 10:10:58 +0000 (11:10 +0100)]
surface-scan: make bs 64k again

I reduced it for testing, put it back at 64k to speed it up.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoAdd sample file for surface scan
Shawn Lewis [Wed, 16 Jan 2008 10:08:03 +0000 (11:08 +0100)]
Add sample file for surface scan

Does a write phase and then a verify phase and verifies the written
data.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoRevert "Remove verify_pattern option, replace with verify=pattern:x"
Shawn Lewis [Wed, 16 Jan 2008 10:01:33 +0000 (11:01 +0100)]
Revert "Remove verify_pattern option, replace with verify=pattern:x"

This reverts commit bfb41d98f63cb9fb72cb7c82618be10bc6d52c33.

Reverting lets us have verify=meta and specify a verify_pattern at the same
time. Sample surfacescan conf file coming up.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoDon't truncate files if we don't need to extend them
Shawn Lewis [Fri, 11 Jan 2008 08:45:11 +0000 (09:45 +0100)]
Don't truncate files if we don't need to extend them

Don't truncate files if we don't need to extend them.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoAdd fill_device option
Shawn Lewis [Fri, 11 Jan 2008 08:45:11 +0000 (09:45 +0100)]
Add fill_device option

Add fill_device option. Causes fio to write until ENOSPC occurs (assuming
rw=write).

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agofio: Fix for new file creation
Gurudas Pai [Fri, 21 Dec 2007 09:50:33 +0000 (10:50 +0100)]
fio: Fix for new file creation

fio fails to create new file and just comes out without doing anything..
This happens only when you are creating file freshly.

./fio ../jobfile/jobfile.test
job1: (g=0): rw=randrw, bs=8K-8K/8K-8K, ioengine=sync, iodepth=1
Starting 1 process

Run status group 0 (all jobs):

Disk stats (read/write):
  sda: ios=0/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%

jobfile:

[global]
 bs=8k
 iodepth_batch=1
 randrepeat=1
 size=100m
 [job1]
 ioengine=sync
 rw=randrw
 filename=newfile

Manually bisected it to commit 509eab12448823a8eefbe925804b5308ee63bf5e
Here is the patch which fixed the issue for me. Please review and apply.
In case of new file "f->io_size > f->real_file_size" becomes true, since
f->real_file_size is 0.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoShow fio help page when no arguments are given
Jens Axboe [Wed, 19 Dec 2007 12:54:38 +0000 (13:54 +0100)]
Show fio help page when no arguments are given

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoOverflow bugs in random map
Jens Axboe [Fri, 14 Dec 2007 11:42:53 +0000 (12:42 +0100)]
Overflow bugs in random map

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
11 years agoAdd Intel IOMeter file access server example job file
Jens Axboe [Fri, 14 Dec 2007 11:41:04 +0000 (12:41 +0100)]
Add Intel IOMeter file access server example job file

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>