Shawn Lewis [Fri, 27 Jul 2007 06:02:45 +0000 (08:02 +0200)]
[PATCH] Only verify_io_u on reads
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Thu, 26 Jul 2007 11:19:57 +0000 (13:19 +0200)]
Add data integrity example to netio sample job file
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Thu, 26 Jul 2007 11:13:49 +0000 (13:13 +0200)]
Missed FIO_UNIDIR io ops flags on net engine
Inadvertenly only set it on the netsplice engine, not net.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Thu, 26 Jul 2007 10:22:30 +0000 (12:22 +0200)]
IO verification improvements
- Add a FIO_UNIDIR io ops flag. This like networked IO can't be read/write,
so verify only makes sense for data summed by another writer.
- Allow a read workload to verify previously written data.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Thu, 26 Jul 2007 09:55:10 +0000 (11:55 +0200)]
Fix verify random bytes filling
Shawn Lewis <shawnlewis@google.com> reports:
---
verify.c: fill_random_bytes() doesn't work as intended.
drand48 on linux returns a double whose value is evenly distributed between
0.0 and 0.1. This is not the same as having an 8-byte value where each bit
has the same chance of being 0 as it does of being 1.
I'd fix this but it should probably be done in a cross-platformish way and
I'm not quite sure how to do it. Maybe just use smaller randoms? Have an
os_random_uint32?
---
I just got rid of os_random_double() and used os_random_long(),
truncating that to an unsigned integer.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
ljzhang,Yaxin Hu,Jianchao Tang [Thu, 26 Jul 2007 09:00:29 +0000 (11:00 +0200)]
[PATCH] empty result
1. The job file looks like this:
---------runtime------------
[global]
directory=./temp
nrfiles=1
rw=randread
size=32k
thread
[runtime]
description="Too tiny a mission using less than 1sec."
----------------------------
Most of the time,it seems like no io has been performed:
----------------------------
nonggia@nonggia-desktop:~/fio$ fio runtime
runtime: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
Starting 1 thread
runtime: (groupid=0, jobs=1): err= 0: pid=7155
Description : ["Too tiny a mission using less than 1sec."]
cpu : usr=0.00%, sys=0.00%, ctx=1
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%,
>=64=0.0%
issued r/w: total=8/0, short=0/0
lat (usec): 100=87.50%, 250=12.50%
Run status group 0 (all jobs):
Disk stats (read/write):
sda: ios=8/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%
nonggia@nonggia-desktop:~/fio$
----------------------------
2. Reason for the problem:
It is so tiny a job that it may cost less than one second to finish.And
the runtime is rounded down to 0.Just like the io never happened.
We agree that the problem is really trifle, but for testing purpose, we
think it maybe while the fixing.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
ljzhang,Yaxin Hu,Jianchao Tang [Thu, 26 Jul 2007 08:59:25 +0000 (10:59 +0200)]
[PATCH] non-stop thread
1. The job file causing the problem:
-------------------rw-------------------
[global]
directory=./temp
nrfiles=1
rw=randread:8
norandommap
size=32k
bs=2k
thread
[rw]
description="Offset over real file size."
-----------------------------------------
The job file can keep fio running with the eta increasing and the
progress pausing like this:
----------------------------------------
rw: (g=0): rw=randread, bs=2K-2K/2K-2K, ioengine=sync, iodepth=1
Starting 1 thread
Jobs: 1 (f=1): [r] [92.3% done] [ 0/ 0 kb/s] [eta 00m:01s]]
----------------------------------------
If we force it to stop, it prints like this:
--------------------------------------------
fio: terminating on signal 2
rw: (groupid=0, jobs=1): err= 0: pid=6124
Description : ["Offset over real file size."]
read : io=28KiB, bw=0KiB/s, iops=0, runt= 61900msec
clat (usec): min= 5, max= 436, avg=87.79, stdev=123.71
cpu : usr=2.01%, sys=93.48%, ctx=13644
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%,
>=64=0.0%
issued r/w: total=14/0, short=0/0
lat (usec): 10=42.86%, 100=35.71%, 250=7.14%, 500=14.29%
Run status group 0 (all jobs):
READ: io=28KiB, aggrb=0KiB/s, minb=0KiB/s, maxb=0KiB/s,
mint=61900msec, maxt=61900msec
Disk stats (read/write):
sda: ios=8/45, merge=0/40, ticks=4/368, in_queue=372, util=0.27%
--------------------------------------------
>From above we can see an uncompleted job with a io 28KiB less than what
is specified by 'size'.
2. Reason for problem:
After looking into the source codes,We guess the problem comes from the
function get_next_offset().If the io reaches the end of the file, the
function fails, and the file will be marked as DONE.We wonder if that is
the planned behavior.Isn't it very common to reach the file's end when
doing a random io?
3. Fix suggestion:
Here is the patch:
-----------------------------------------------------
diff -Nraup fio-7.25/io_u.c fio-7.25-rw/io_u.c
--- fio-7.25/io_u.c 2007-07-25 14:25:05.
000000000 +0800
+++ fio-7.25-rw/io_u.c 2007-07-26 09:55:22.
000000000 +0800
@@ -162,10 +162,12 @@ static int get_next_offset(struct thread
if (get_next_rand_offset(td, f, ddir, &b))
return 1;
} else {
- if (f->last_pos >= f->real_file_size)
- return 1;
-
- b = f->last_pos / td->o.min_bs[ddir];
+ if (f->last_pos >= f->real_file_size) {
+ if (!td_random(td) || get_next_rand_offset(td, f, ddir, &b))
+ return 1;
+ } else {
+ b = f->last_pos / td->o.min_bs[ddir];
+ }
}
io_u->offset = (b * td->o.min_bs[ddir]) + f->file_offset;
------------------------------------------------------
After applying that, we got the job file run to the end normally:
-------------------------------------------------------
nonggia@nonggia-desktop:~/fio$ ./rw_fio rw
rw: (g=0): rw=randread, bs=2K-2K/2K-2K, ioengine=sync, iodepth=1
Starting 1 thread
rw: (groupid=0, jobs=1): err= 0: pid=6519
Description : ["Offset over real file size."]
read : io=32KiB, bw=1560KiB/s, iops=761, runt= 21msec
clat (usec): min= 5, max=14251, avg=1293.50, stdev=3510.37
cpu : usr=0.00%, sys=19.05%, ctx=7
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%,
>=64=0.0%
issued r/w: total=16/0, short=0/0
lat (usec): 10=37.50%, 20=12.50%, 250=12.50%, 750=6.25%, 1000=6.25%
lat (msec): 2=18.75%, 20=6.25%
Run status group 0 (all jobs):
READ: io=32KiB, aggrb=1560KiB/s, minb=1560KiB/s, maxb=1560KiB/s,
mint=21msec, maxt=21msec
Disk stats (read/write):
sda: ios=8/0, merge=0/0, ticks=20/0, in_queue=20, util=13.89%
-------------------------------------------------------
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
ljzhang,Yaxin Hu,Jianchao Tang [Thu, 26 Jul 2007 08:51:46 +0000 (10:51 +0200)]
[PATCH] Random map block coverage
1. The job file causing the problem:
----------cover-------------------
[global]
directory=./temp
nrfiles=1
rw=randread
size=256K
bsrange=4k-16k
thread
[cover]
description="Not covering every block."
-----------------------------------------
According to HOWTO,the job file will makes fio to cover every block of
the 256k file.But running that, we sometimes got a io gross more than
256k:
----------------------------------------
nonggia@nonggia-desktop:~$ fio --version
fio 1.16.9
nonggia@nonggia-desktop:~/fio$ fio cover
cover: (g=0): rw=randread, bs=4K-16K/4K-4K, ioengine=sync, iodepth=1
Starting 1 thread
cover: (groupid=0, jobs=1): err= 0: pid=6969
Description : ["Not covering every block."]
read : io=264KiB, bw=90112KiB/s, iops=7666, runt= 3msec
clat (usec): min= 66, max= 438, avg=151.00, stdev=88.90
cpu : usr=0.00%, sys=133.33%, ctx=0
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%,
>=64=0.0%
issued r/w: total=23/0, short=0/0
lat (usec): 100=34.78%, 250=56.52%, 500=8.70%
Run status group 0 (all jobs):
READ: io=264KiB, aggrb=90112KiB/s, minb=90112KiB/s, maxb=90112KiB/s,
mint=3msec, maxt=3msec
Disk stats (read/write):
sda: ios=49/0, merge=0/0, ticks=4/0, in_queue=4, util=2.86%
--------------------------------------------
It seems fio doesn't cover every block.And using gdb we can see it in
detail:
--------------------------------------------
(gdb) where
from /lib/tls/i686/cmov/libpthread.so.0
(gdb) p td->files[0].num_maps
$1 = 2
(gdb) x/2w td->files[0].file_map
0x8073ff0: 0x00000000 0x00000000
(gdb) finish
Run till exit from #0 do_io (td=0xb7bec000) at fio.c:423
thread_main (data=0xb7bec000) at fio.c:875 0 kb/s] [eta 00m:00s]
875 clear_state = 1;
(gdb) x/2w td->files[0].file_map
0x8073ff0: 0xffffeffd 0x7ce2df47
(gdb) c
Continuing.
[Thread -
1212236896 (LWP 7099) exited]
cover: (groupid=0, jobs=1): err= 0: pid=7098
Description : ["Not covering every block."]
read : io=264KiB, bw=7KiB/s, iops=0, runt= 37522msec
clat (usec): min= 67, max= 347, avg=143.28, stdev=67.59
bw (KiB/s) : min= 0, max= 0, per=0.00%, avg= 0.00, stdev= 0.00
cpu : usr=0.00%, sys=0.01%, ctx=13
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%,
>=64=0.0%
issued r/w: total=29/0, short=0/0
lat (usec): 100=31.03%, 250=62.07%, 500=6.90%
Run status group 0 (all jobs):
READ: io=264KiB, aggrb=7KiB/s, minb=7KiB/s, maxb=7KiB/s,
mint=37522msec, maxt=37522msec
Disk stats (read/write):
sda: ios=50/23, merge=0/32, ticks=0/160, in_queue=160, util=0.13%
Program exited normally.
--------------------------------------------------
2. Reason for problem:
We think when there is no 'norandommap' in the job file, and no ':int'
appended with 'rw',just like the job file above, fio is meant to use
only the blocks not covered yet.
But it seems in the function mark_random_map(), the condition before
calling random_map_free() will never be satisfied.
3. Fix suggestion:
Here is the patch:
-----------------------------------------------------
diff -Nraup fio-7.25/io_u.c fio-7.25-cover/io_u.c
--- fio-7.25/io_u.c 2007-07-25 14:25:05.
000000000 +0800
+++ fio-7.25-cover/io_u.c 2007-07-25 20:43:58.
000000000 +0800
@@ -55,7 +55,7 @@ static void mark_random_map(struct threa
* If we have a mixed random workload, we may
* encounter blocks we already did IO to.
*/
- if (!td->o.ddir_nr && !random_map_free(td, f, block))
+ if (td->o.ddir_nr==1 && !random_map_free(td, f, block))
break;
idx = RAND_MAP_IDX(td, f, block);
------------------------------------------------------
After that, we check it with gdb:
----------------------------------------------------
(gdb) where
from /lib/tls/i686/cmov/libpthread.so.0
(gdb) p td->files[0].num_maps
$2 = 2
(gdb) x/2w td->files[0].file_map
0x8073ff0: 0x00000000 0x00000000
(gdb) finish
Run till exit from #0 do_io (td=0xb7c48000) at fio.c:423
thread_main (data=0xb7c48000) at fio.c:875 0 kb/s] [eta 00m:00s]
875 clear_state = 1;
(gdb) x/2w td->files[0].file_map
0x8073ff0: 0xffffffff 0xffffffff
(gdb) c
Continuing.
[Thread -
1212073056 (LWP 7443) exited] 5/ 0 kb/s] [eta 00m:00s]
cover: (groupid=0, jobs=1): err= 0: pid=7442
Description : ["Not covering every block."]
read : io=256KiB, bw=5KiB/s, iops=0, runt= 46612msec
clat (usec): min= 62, max= 522, avg=119.50, stdev=88.07
bw (KiB/s) : min= 0, max= 0, per=0.00%, avg= 0.00, stdev= 0.00
cpu : usr=0.00%, sys=0.01%, ctx=11
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%,
>=64=0.0%
issued r/w: total=38/0, short=0/0
lat (usec): 100=55.26%, 250=39.47%, 500=2.63%, 750=2.63%
Run status group 0 (all jobs):
READ: io=256KiB, aggrb=5KiB/s, minb=5KiB/s, maxb=5KiB/s,
mint=46612msec, maxt=46612msec
Disk stats (read/write):
sda: ios=64/45, merge=0/62, ticks=8/476, in_queue=484, util=0.19%
Program exited normally.
(gdb)
---------------------------------------------------------
Now we can see the expected '0xffffffff 0xffffffff' and the 'io=256KiB'.
By the way, we tested this both on a 32bit machine and a 64bit one.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Thu, 26 Jul 2007 08:47:51 +0000 (10:47 +0200)]
Fix problem with io_u being beyond EOF
Bug report from "ljzhang,Yaxin Hu,Jianchao Tang" <nonggia@sjtu.edu.cn> :
1. The job file causing the problem:
----------ranmap-------------------
[global]
directory=./temp
nrfiles=1
rw=randread
size=64K
bsrange=1k-32k
thread
[ranmap]
description="Over-bounded in mark_random_map()."
-----------------------------------------
Running the job file, we have a high possibility to get a core dump like
this:
----------------------------------------
nonggia@nonggia-desktop:~/fio$ fio --version
fio 1.16.9
nonggia@nonggia-desktop:~/fio$ fio ranmap
ranmap: (g=0): rw=randread, bs=1K-32K/4K-4K, ioengine=sync, iodepth=1
Starting 1 thread
file:io_u.c:64, assert idx < f->num_maps failed
Segmentation fault (core dumped)
nonggia@nonggia-desktop:~/fio$ gdb fio core.6135
... ...
Core was generated by `fio ranmap'.
Program terminated with signal 11, Segmentation fault.
io_u.c:64
64 fio_assert(td, idx < f->num_maps);
(gdb) bt
io_u.c:64
io_u.c:348
io_u.c:588
from /lib/tls/i686/cmov/libpthread.so.0
(gdb) p io_u->offset
$1 = 41984
(gdb) p io_u->buflen
$2 = 27648
(gdb) p io_u->file->real_file_size
$3 = 65536
2. Reason for the core dump:
The io_u->offset+io_u->buflen was too large to be map in the map file.
They just produced an idx out of the file_map's range.
The fix I produced made sure that get_next_buflen() doesn't go
beyond the real file size, by shrinking the buflen.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Shawn Lewis [Thu, 26 Jul 2007 05:36:14 +0000 (07:36 +0200)]
[PATCH] Don't hide pthread_mutex_setpshared in os-linux.h
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Shawn Lewis [Wed, 25 Jul 2007 05:51:58 +0000 (07:51 +0200)]
Fix headers for building with some older libs
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Mon, 23 Jul 2007 12:36:16 +0000 (14:36 +0200)]
Add ->done variable to force exit when we know we are done
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Mon, 23 Jul 2007 12:36:00 +0000 (14:36 +0200)]
Only do the root warning once per thread
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Mon, 23 Jul 2007 12:21:17 +0000 (14:21 +0200)]
Missing carriage return in latency print
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Mon, 23 Jul 2007 12:16:55 +0000 (14:16 +0200)]
Only warn once for blktrace log open failure
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Mon, 23 Jul 2007 09:26:27 +0000 (11:26 +0200)]
One more fio_ddir fixup
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Mon, 23 Jul 2007 09:25:39 +0000 (11:25 +0200)]
ddir is of type enum fio_ddir
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Mon, 23 Jul 2007 09:23:26 +0000 (11:23 +0200)]
Remove gotos from read_iolog_get()
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Mon, 23 Jul 2007 08:38:43 +0000 (10:38 +0200)]
IO log fixes
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Mon, 23 Jul 2007 06:56:43 +0000 (08:56 +0200)]
Add version 2 of the iolog format
The old format had some short comings - notably not supporting > 1
file workloads. So add a new format that defines adding/open/close
of files. Fio will still load the older format iologs, but the new
ones wont work on older fio versions.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Fri, 20 Jul 2007 14:08:28 +0000 (16:08 +0200)]
Fio 1.16.9
Lots of good bug reporting from Erwan Velu, so lets celebrate
those fixes with a new release.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Fri, 20 Jul 2007 12:25:31 +0000 (14:25 +0200)]
Set minimum value allowed for some options
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Fri, 20 Jul 2007 11:25:04 +0000 (13:25 +0200)]
Fix disk zone test case
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Fri, 20 Jul 2007 10:43:05 +0000 (12:43 +0200)]
Mention what the [ xxx / xxx kb/s ] means in the status output
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Fri, 20 Jul 2007 10:39:22 +0000 (12:39 +0200)]
Don't align memory unless using raw io
If it's not raw io, we didn't align the buffer size. So we cannot
align.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Thu, 19 Jul 2007 19:38:35 +0000 (21:38 +0200)]
Default stdev to 0.0, not -1.0
If we only have a single sample, then the deviation is by definition 0.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Thu, 19 Jul 2007 13:08:54 +0000 (15:08 +0200)]
Fio 1.16.8
1.16.7 was rather short lived, as it had a stupid bug for
the ordinary malloc() backed buffers.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Thu, 19 Jul 2007 13:06:57 +0000 (15:06 +0200)]
malloc memory still needs to be aligned, if we are doing direct IO
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Thu, 19 Jul 2007 12:53:01 +0000 (14:53 +0200)]
Treat option outside of job section as fatal and quit
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Thu, 19 Jul 2007 12:50:05 +0000 (14:50 +0200)]
Fix memory overflow bugs
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Thu, 19 Jul 2007 12:22:50 +0000 (14:22 +0200)]
Fio 1.16.7
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Thu, 19 Jul 2007 12:21:08 +0000 (14:21 +0200)]
Make a note of Mandriva package availability
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Thu, 19 Jul 2007 08:17:09 +0000 (10:17 +0200)]
Fix bad alignment in buffer size allocation
We typically end up allocating a block more than we need. Usually
not a problem for normal memory, but when we are dealing with
scarce resources such as hugepages, it can be problematic. For many
threads the memory waste could also be a problem.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Thu, 19 Jul 2007 08:09:45 +0000 (10:09 +0200)]
Add clue for ENOMEM hugepage allocation
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Thu, 19 Jul 2007 08:01:03 +0000 (10:01 +0200)]
Fix typo in shmhuge commit
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Thu, 19 Jul 2007 08:00:05 +0000 (10:00 +0200)]
Detect and print when the OS doesn't support huge pages
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Wed, 18 Jul 2007 11:51:19 +0000 (13:51 +0200)]
git:// url location change
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Fri, 6 Jul 2007 12:24:19 +0000 (14:24 +0200)]
Fio 1.16.6
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Fri, 6 Jul 2007 12:09:49 +0000 (14:09 +0200)]
Fix bug with FIO_IO_U_M/U_NR being transposed
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Sun, 24 Jun 2007 20:41:27 +0000 (22:41 +0200)]
Guard __must_check ifdef
Some systems seem to pick up the kernel definitions, so check
if __must_check is defined before (potentially) redefining it.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Sun, 24 Jun 2007 19:41:46 +0000 (21:41 +0200)]
net engine: cleanup the splice handling
The splice and vmsplice bits were largely duplicated, so abstract
the stuff out so it's clearer to see what is going on.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Sun, 24 Jun 2007 19:28:39 +0000 (21:28 +0200)]
net engine: fix listenfd/pipe fd leaks
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Fri, 22 Jun 2007 18:41:27 +0000 (20:41 +0200)]
netsplice: use zero-copy if possible
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Fri, 22 Jun 2007 18:40:10 +0000 (20:40 +0200)]
netsplice: fix a few typos
It actually works now.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Fri, 22 Jun 2007 13:42:21 +0000 (15:42 +0200)]
Support for 'netsplice' engine
This IO engine does network transfers, but uses splice() to move data
to/from a socket to a pipe and vmsplice() to move that data to/from
user space.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Tue, 19 Jun 2007 12:25:46 +0000 (14:25 +0200)]
guasi engine: remove STFU_GCC()
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Tue, 19 Jun 2007 10:49:52 +0000 (12:49 +0200)]
Fio 1.16.5
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Tue, 19 Jun 2007 10:48:41 +0000 (12:48 +0200)]
Adaptive io latency distrobution "histogram"
Log ranges all the way down to microseconds, and display the most
appropriate values for those ranges.
This makes the latency usable for sub-milisecond devices as well.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Tue, 19 Jun 2007 07:53:31 +0000 (09:53 +0200)]
HOWTO: note that clat/slat can be in micro or mili seconds
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Tue, 19 Jun 2007 07:50:28 +0000 (09:50 +0200)]
io_u_mark_depth: just set index, don't fall through
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Tue, 19 Jun 2007 07:48:44 +0000 (09:48 +0200)]
Cleanup msec <-> usec lat conversion
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Mon, 18 Jun 2007 07:48:57 +0000 (09:48 +0200)]
libaio engine: print warning for depth > 1 and buffered IO
Linux aio doesn't support queued buffered IO, it has to be
O_DIRECT. So print a warning to that effect, so that users
of fio don't get bad results by mistake.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Mon, 18 Jun 2007 07:41:23 +0000 (09:41 +0200)]
Make completion and submission latency use appropriate time base
It's hardcoded as msecs right now, switch the accounting to usecs
and print out the appropriate base determined by the value range.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Fri, 15 Jun 2007 08:33:49 +0000 (10:33 +0200)]
Add cpus_allowed option
cpumask can be cumbersome to use, since it requires you to calculate
a decimal mask to pass to fio. So add a cpus_allowed option that
takes a text list of allowed cpus, ala: cpus_allowed=1,3,8 will set
the affinity mask for CPUS 1, 3, and 8.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Thu, 14 Jun 2007 06:17:08 +0000 (08:17 +0200)]
Fio 1.16.4
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Wed, 13 Jun 2007 19:00:46 +0000 (21:00 +0200)]
HOWTO: expand the cpumask explanation a bit
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Wed, 13 Jun 2007 18:58:09 +0000 (20:58 +0200)]
Merge branch 'master' of ssh://brick.kernel.dk/data/git/fio
Gough, Corey D [Wed, 13 Jun 2007 18:58:01 +0000 (20:58 +0200)]
Fix cpu mask setting
Fio passed the mask by value, not by reference. So when
fill_cpu_mask() was done setting the valid CPUs, it was
only on the local value.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Wed, 13 Jun 2007 17:04:20 +0000 (19:04 +0200)]
Merge branch 'master' of ssh://router/data/git/fio
Jens Axboe [Wed, 13 Jun 2007 17:04:16 +0000 (19:04 +0200)]
splice engine: support for vmsplice to user space
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Fri, 8 Jun 2007 09:24:58 +0000 (11:24 +0200)]
splice flags are unsigned int, not unsigned long
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Fri, 25 May 2007 07:26:05 +0000 (09:26 +0200)]
Documentation update
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Thu, 24 May 2007 08:34:30 +0000 (10:34 +0200)]
Fio 1.16.3
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Thu, 24 May 2007 08:32:47 +0000 (10:32 +0200)]
Tweak meaning of size option
Consider it the full amount of IO to be done, even if the
files used are smaller. Then we'll just restart the workload
until we reach the desired number of bytes.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Wed, 23 May 2007 07:34:42 +0000 (09:34 +0200)]
Fix bug with time_based run
We need to clear the FIO_FILE_DONE flag and the number of
files done, to avoid do_io() exiting quickly for a re-run.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Tue, 22 May 2007 15:07:44 +0000 (17:07 +0200)]
Fio 1.16.2
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Tue, 22 May 2007 15:06:24 +0000 (17:06 +0200)]
Add comment in os/os-linux.h wrt glibc-2.3.2
It was a broken release wrt cpu affinity, they removed the
size argument from the set/get functions. If you want to
compile on such an ancient glibc, manually define GLIBC_2_3_2.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Tue, 22 May 2007 13:51:50 +0000 (15:51 +0200)]
Fix build on older systems
Avoid conflict with MINOR/MAJOR.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Tue, 22 May 2007 11:27:30 +0000 (13:27 +0200)]
Fix build/compile for non-blktrace platforms
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Tue, 22 May 2007 09:12:13 +0000 (11:12 +0200)]
IO verification needs to open and grab a reference to a file
Otherwise it can disappear in midair.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Tue, 22 May 2007 07:21:37 +0000 (09:21 +0200)]
Fix bug with random IO and network connections
It isn't a supported configuration.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Tue, 22 May 2007 07:13:31 +0000 (09:13 +0200)]
Update HOWTO on cpumask setting
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Tue, 22 May 2007 07:13:02 +0000 (09:13 +0200)]
Fix cpumask setting/getting on older kernels
Don't call into the affinity functions, unless the user has
specified a mask.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Mon, 21 May 2007 12:52:43 +0000 (14:52 +0200)]
mmap engine: fix open-for-size check
Don't call into mmap() yet, since we didn't setup ->io_size.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Mon, 21 May 2007 12:27:32 +0000 (14:27 +0200)]
Attempt to work-around possible time warp
A theory - we see gettimeofday() going backwards on some
systems, try and correct for that by just returning 0
for time passed.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Mon, 21 May 2007 11:23:30 +0000 (13:23 +0200)]
Improve random pattern without norandommap
Using the random map, random workloads have a natural tendency to
get less random at the end of the run, due to failures to find a
new unused random offset.
Improve the logic to avoid this problem.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Mon, 21 May 2007 09:42:33 +0000 (11:42 +0200)]
Rename ffz() function to fio_ffz() to avoid conflicts
Newer gcc/glibc include an ffz function. Lets just do the easy
thing and rename ours to fio_ffz(), then we can look into reusing
a provided one later if necessary.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Wed, 16 May 2007 16:28:47 +0000 (18:28 +0200)]
blktrace support: major/minor fixups
We get passed the kernel variant, so we need a bit of manual
support to untangle it (major() and minor() wont cut it).
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Wed, 16 May 2007 12:40:29 +0000 (14:40 +0200)]
blktrace support: support for looking up a device
We can get the device number out of the trace, turn that into
a major/minor pair and look it up in /dev.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Wed, 16 May 2007 10:29:06 +0000 (12:29 +0200)]
blktrace support: discard notify messages
They are not interesting to us (they contain things like timestamps
and process information).
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Wed, 16 May 2007 10:16:44 +0000 (12:16 +0200)]
blktrace support: fixups/cleanups/improvements
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Wed, 16 May 2007 09:54:43 +0000 (11:54 +0200)]
fifo updates
Jens Axboe [Wed, 16 May 2007 06:59:27 +0000 (08:59 +0200)]
Fix timeout on 64-bit BE archs
We stored the timeout in an unsigned long long pointer, but
the value was really unsigned long.
Jens Axboe [Wed, 16 May 2007 07:26:23 +0000 (09:26 +0200)]
Fix fifo leak
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Wed, 16 May 2007 07:25:09 +0000 (09:25 +0200)]
blktrace support: speedup reading of data
We used to read in data in really small chunks (48 bytes at the time,
the size of the trace). This is really slow for large traces, so
add a fifo frontend to refill the cache in much larger sizes.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Wed, 16 May 2007 06:47:46 +0000 (08:47 +0200)]
blktrace support: adjust largest io_u buffer size to match trace
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Tue, 15 May 2007 12:29:58 +0000 (14:29 +0200)]
blktrace replay: delay support
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Tue, 15 May 2007 11:23:19 +0000 (13:23 +0200)]
blktrace fixup + documentation
Add a little documentation on using blktrace data files.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Tue, 15 May 2007 11:10:41 +0000 (13:10 +0200)]
blktrace support now works
One little buglet - the blktrace offset is the sector, not the
byte offset. For now it's hardcoded at 512, should really get
the value from the device.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Tue, 15 May 2007 09:54:21 +0000 (11:54 +0200)]
blktrace support update
This should essentially work, except that it will not delay between
io but just issue the entries as fast as it can. Missing bits:
- Utilize the ipo->delay to suitably delay issuing an io
- Extract device name from blktrace file
- Document it
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Tue, 15 May 2007 09:12:19 +0000 (11:12 +0200)]
iolog bug: TD_DDIR_READ should be TD_DDIR_WRITE
It's a typo, if we don't have reads stored but writes, then the
global direction should naturally be TD_DDIR_WRITE.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Tue, 15 May 2007 08:13:04 +0000 (10:13 +0200)]
Fix typo in blktrace code
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Tue, 15 May 2007 08:12:26 +0000 (10:12 +0200)]
Some more bits of blktrace support
Basic read/write is there. Still need to extract device name from
the trace, and check time delays to wait appropriately.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Tue, 15 May 2007 06:44:04 +0000 (08:44 +0200)]
Initial blktrace support
This doesn't work yet, just committing what little bits I did as not
to lose them.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Thu, 26 Apr 2007 13:24:20 +0000 (15:24 +0200)]
Log error on io buffer allocation
Currently it just exits silently.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Thu, 26 Apr 2007 11:47:07 +0000 (13:47 +0200)]
Allow termination during io_u building
If we are setting up gigabytes of io buffers, it can take many seconds
for that operation to complete. If the user wishes to abort, fio wont
honor that request. Fix that.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Thu, 26 Apr 2007 08:22:16 +0000 (10:22 +0200)]
Fio 1.16.1
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Thu, 26 Apr 2007 08:21:15 +0000 (10:21 +0200)]
Fix bug with file references
This could cause premature exit of fio with an error at the end of
the run.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Thu, 26 Apr 2007 07:02:34 +0000 (09:02 +0200)]
Disk stat improvements
Add an option to disable the disk stats. And only dump the disk
stat header, if we actually have stats to show.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
Jens Axboe [Wed, 25 Apr 2007 12:24:12 +0000 (14:24 +0200)]
More iolog fixes
- Warn if nrfiles > 1, so that the user knows it's not supported
- Need to clear ipo on malloc, or ->file could be some random value.
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>