From: Jens Axboe Date: Thu, 27 Jul 2017 14:11:40 +0000 (-0600) Subject: Merge branch 'minor_fixes' of https://github.com/sitsofe/fio X-Git-Tag: fio-3.0~27 X-Git-Url: https://git.kernel.dk/?p=fio.git;a=commitdiff_plain;h=dad0ea54b595ad656aca4087014afb438c7d44f0;hp=e1933299ed9f1525e010e0489f0185c063d6d129 Merge branch 'minor_fixes' of https://github.com/sitsofe/fio --- diff --git a/.travis.yml b/.travis.yml index ca50e226..e84e61fa 100644 --- a/.travis.yml +++ b/.travis.yml @@ -16,7 +16,7 @@ matrix: # compiler: clang # osx_image: xcode8 # env: SDKROOT=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk MACOSX_DEPLOYMENT_TARGET=10.11 - # Build on the latest OSX version (will eventually become obsolete) + # Build on the latest OSX version (will eventually become obsolete) - os: osx compiler: clang osx_image: xcode8.2 diff --git a/HOWTO b/HOWTO index 2fa8fc2a..fe5c3cb1 100644 --- a/HOWTO +++ b/HOWTO @@ -98,7 +98,7 @@ Command line options .. option:: --parse-only - Parse options only, don\'t start any I/O. + Parse options only, don't start any I/O. .. option:: --output=filename @@ -1015,8 +1015,8 @@ I/O type ``sequential`` is only useful for random I/O, where fio would normally generate a new random offset for every I/O. If you append e.g. 8 to randread, - you would get a new random offset for every 8 I/O's. The result would be a - seek for only every 8 I/O's, instead of for every I/O. Use ``rw=randread:8`` + you would get a new random offset for every 8 I/Os. The result would be a + seek for only every 8 I/Os, instead of for every I/O. Use ``rw=randread:8`` to specify that. As sequential I/O is already sequential, setting ``sequential`` for that would not result in any differences. ``identical`` behaves in a similar fashion, except it sends the same offset 8 number of @@ -2761,7 +2761,8 @@ Measurements and reporting .. option:: log_offset=int If this is set, the iolog options will include the byte offset for the I/O - entry as well as the other data values. + entry as well as the other data values. Defaults to 0 meaning that + offsets are not present in logs. Also see `Log File Formats`_. .. option:: log_compression=int @@ -3242,7 +3243,7 @@ numbers denote: **ios** Number of I/Os performed by all groups. **merge** - Number of merges I/O the I/O scheduler. + Number of merges performed by the I/O scheduler. **ticks** Number of ticks we kept the disk busy. **in_queue** @@ -3278,7 +3279,7 @@ changed for some reason, this number will be incremented by 1 to signify that change. Split up, the format is as follows (comments in brackets denote when a -field was introduced or whether its specific to some terse version): +field was introduced or whether it's specific to some terse version): :: @@ -3531,9 +3532,10 @@ Log File Formats Fio supports a variety of log file formats, for logging latencies, bandwidth, and IOPS. The logs share a common format, which looks like this: - *time* (`msec`), *value*, *data direction*, *offset* + *time* (`msec`), *value*, *data direction*, *block size* (`bytes`), + *offset* (`bytes`) -Time for the log entry is always in milliseconds. The *value* logged depends +*Time* for the log entry is always in milliseconds. The *value* logged depends on the type of log, it will be one of the following: **Latency log** @@ -3552,16 +3554,17 @@ on the type of log, it will be one of the following: **2** I/O is a TRIM -The *offset* is the offset, in bytes, from the start of the file, for that -particular I/O. The logging of the offset can be toggled with -:option:`log_offset`. +The entry's *block size* is always in bytes. The *offset* is the offset, in bytes, +from the start of the file, for that particular I/O. The logging of the offset can be +toggled with :option:`log_offset`. Fio defaults to logging every individual I/O. When IOPS are logged for individual -I/Os the value entry will always be 1. If windowed logging is enabled through +I/Os the *value* entry will always be 1. If windowed logging is enabled through :option:`log_avg_msec`, fio logs the average values over the specified period of time. If windowed logging is enabled and :option:`log_max_value` is set, then fio logs -maximum values in that window instead of averages. Since 'data direction' and -'offset' are per-I/O values, they aren't applicable if windowed logging is enabled. +maximum values in that window instead of averages. Since *data direction*, *block +size* and *offset* are per-I/O values, if windowed logging is enabled they +aren't applicable and will be 0. Client/Server ------------- diff --git a/arch/arch-arm.h b/arch/arch-arm.h index 31671fdb..dd286d04 100644 --- a/arch/arch-arm.h +++ b/arch/arch-arm.h @@ -14,6 +14,8 @@ #define nop __asm__ __volatile__ ("nop") #define read_barrier() __sync_synchronize() #define write_barrier() __sync_synchronize() +#else +#error "unsupported ARM architecture" #endif #endif diff --git a/doc/fio_examples.rst b/doc/fio_examples.rst index ae0ef6f8..cff1f394 100644 --- a/doc/fio_examples.rst +++ b/doc/fio_examples.rst @@ -60,3 +60,13 @@ Fixed rate submission .. literalinclude:: ../examples/fixed-rate-submission.fio :language: ini + +Butterfly seek pattern +----------------------- + +.. only:: builder_html + +:download:`Download butterfly.fio <../examples/butterfly.fio>` + +.. literalinclude:: ../examples/butterfly.fio + :language: ini diff --git a/examples/butterfly.fio b/examples/butterfly.fio new file mode 100644 index 00000000..42d253d5 --- /dev/null +++ b/examples/butterfly.fio @@ -0,0 +1,19 @@ +# Perform a butterfly/funnel seek pattern. This won't always alternate ends on +# every I/O but it will get close. + +[global] +filename=/tmp/testfile +bs=4k +direct=1 + +[forward] +rw=read +flow=2 +# Uncomment the size= and offset= lines to prevent each direction going past +# the middle of the file +#size=50% + +[backward] +rw=read:-8k +flow=-2 +#offset=50% diff --git a/filesetup.c b/filesetup.c index 3b2ebd9c..362997c1 100644 --- a/filesetup.c +++ b/filesetup.c @@ -100,7 +100,6 @@ static void fallocate_file(struct thread_data *td, struct fio_file *f) log_err("fio: unknown fallocate mode: %d\n", td->o.fallocate_mode); assert(0); } - } /* diff --git a/fio.1 b/fio.1 index 768b2096..f86b7fdf 100644 --- a/fio.1 +++ b/fio.1 @@ -453,7 +453,7 @@ the same blocks will be written to. Fio defaults to read if the option is not specified. For mixed I/O, the default split is 50/50. For certain types of io the result may still be skewed a bit, since the speed may be different. It is possible to -specify a number of IO's to do before getting a new offset, this is done by +specify a number of IOs to do before getting a new offset, this is done by appending a `:\fI\fR to the end of the string given. For a random read, it would look like \fBrw=randread:8\fR for passing in an offset modifier with a value of 8. If the postfix is used with a sequential IO pattern, then the value @@ -478,8 +478,8 @@ Generate the same offset .P \fBsequential\fR is only useful for random IO, where fio would normally generate a new random offset for every IO. If you append eg 8 to randread, you -would get a new random offset for every 8 IO's. The result would be a seek for -only every 8 IO's, instead of for every IO. Use \fBrw=randread:8\fR to specify +would get a new random offset for every 8 IOs. The result would be a seek for +only every 8 IOs, instead of for every IO. Use \fBrw=randread:8\fR to specify that. As sequential IO is already sequential, setting \fBsequential\fR for that would not result in any differences. \fBidentical\fR behaves in a similar fashion, except it sends the same offset 8 number of times before generating a @@ -1794,7 +1794,8 @@ logs contain 1216 latency bins. See the \fBLOG FILE FORMATS\fR section. .TP .BI log_offset \fR=\fPbool If this is set, the iolog options will include the byte offset for the IO -entry as well as the other data values. +entry as well as the other data values. Defaults to 0 meaning that offsets are +not present in logs. See the \fBLOG FILE FORMATS\fR section. .TP .BI log_compression \fR=\fPint If this is set, fio will compress the IO logs as it goes, to keep the memory @@ -2251,7 +2252,7 @@ Finally, disk statistics are printed with reads first: Number of I/Os performed by all groups. .TP .B merge -Number of merges in the I/O scheduler. +Number of merges performed by the I/O scheduler. .TP .B ticks Number of ticks we kept the disk busy. @@ -2581,7 +2582,7 @@ the files over and load them from there. Fio supports a variety of log file formats, for logging latencies, bandwidth, and IOPS. The logs share a common format, which looks like this: -.B time (msec), value, data direction, offset +.B time (msec), value, data direction, block size (bytes), offset (bytes) Time for the log entry is always in milliseconds. The value logged depends on the type of log, it will be one of the following: @@ -2616,15 +2617,16 @@ IO is a TRIM .PD .P -The \fIoffset\fR is the offset, in bytes, from the start of the file, for that -particular IO. The logging of the offset can be toggled with \fBlog_offset\fR. +The entry's *block size* is always in bytes. The \fIoffset\fR is the offset, in +bytes, from the start of the file, for that particular IO. The logging of the +offset can be toggled with \fBlog_offset\fR. If windowed logging is enabled through \fBlog_avg_msec\fR, then fio doesn't log individual IOs. Instead of logs the average values over the specified -period of time. Since \fIdata direction\fR and \fIoffset\fR are per-IO values, -they aren't applicable if windowed logging is enabled. If windowed logging -is enabled and \fBlog_max_value\fR is set, then fio logs maximum values in -that window instead of averages. +period of time. Since \fIdata direction\fR, \fIblock size\fR and \fIoffset\fR +are per-IO values, if windowed logging is enabled they aren't applicable and +will be 0. If windowed logging is enabled and \fBlog_max_value\fR is set, then +fio logs maximum values in that window instead of averages. For histogram logging the logs look like this: diff --git a/gettime.c b/gettime.c index 9e5457e0..3dcaaf68 100644 --- a/gettime.c +++ b/gettime.c @@ -425,6 +425,7 @@ void fio_clock_init(void) fio_clock_source = CS_CPUCLOCK; } else if (fio_clock_source == CS_CPUCLOCK) log_info("fio: clocksource=cpu may not be reliable\n"); + dprint(FD_TIME, "gettime: clocksource=%d\n", (int) fio_clock_source); } uint64_t ntime_since(const struct timespec *s, const struct timespec *e) diff --git a/init.c b/init.c index 90cc0bcf..42e71071 100644 --- a/init.c +++ b/init.c @@ -781,6 +781,11 @@ static int fixup_options(struct thread_data *td) o->unit_base = 8; } +#ifndef FIO_HAVE_ANY_FALLOCATE + /* Platform doesn't support any fallocate so force it to none */ + o->fallocate_mode = FIO_FALLOCATE_NONE; +#endif + #ifndef CONFIG_FDATASYNC if (o->fdatasync_blocks) { log_info("fio: this platform does not support fdatasync()" diff --git a/options.c b/options.c index b21f09ae..5a2ab57c 100644 --- a/options.c +++ b/options.c @@ -2289,7 +2289,7 @@ struct fio_option fio_options[FIO_MAX_OPTS] = { .parent = "nrfiles", .hide = 1, }, -#if defined(CONFIG_POSIX_FALLOCATE) || defined(FIO_HAVE_NATIVE_FALLOCATE) +#ifdef FIO_HAVE_ANY_FALLOCATE { .name = "fallocate", .lname = "Fallocate", @@ -2333,14 +2333,14 @@ struct fio_option fio_options[FIO_MAX_OPTS] = { #endif }, }, -#else /* CONFIG_POSIX_FALLOCATE */ +#else /* FIO_HAVE_ANY_FALLOCATE */ { .name = "fallocate", .lname = "Fallocate", .type = FIO_OPT_UNSUPPORTED, .help = "Your platform does not support fallocate", }, -#endif /* CONFIG_POSIX_FALLOCATE || FIO_HAVE_NATIVE_FALLOCATE */ +#endif /* FIO_HAVE_ANY_FALLOCATE */ { .name = "fadvise_hint", .lname = "Fadvise hint", diff --git a/os/os-dragonfly.h b/os/os-dragonfly.h index 8a116e60..8d158335 100644 --- a/os/os-dragonfly.h +++ b/os/os-dragonfly.h @@ -5,6 +5,7 @@ #include #include +#include #include #include #include diff --git a/os/os-freebsd.h b/os/os-freebsd.h index c7863b5e..e6da286e 100644 --- a/os/os-freebsd.h +++ b/os/os-freebsd.h @@ -6,6 +6,7 @@ #include #include #include +#include #include #include #include diff --git a/os/os-netbsd.h b/os/os-netbsd.h index 7be02a78..eac76cf2 100644 --- a/os/os-netbsd.h +++ b/os/os-netbsd.h @@ -10,6 +10,7 @@ #include #include #include +#include /* XXX hack to avoid confilcts between rbtree.h and */ #define rb_node _rb_node #include diff --git a/os/os-openbsd.h b/os/os-openbsd.h index d874ee25..675bf892 100644 --- a/os/os-openbsd.h +++ b/os/os-openbsd.h @@ -9,6 +9,7 @@ #include #include #include +#include #include /* XXX hack to avoid conflicts between rbtree.h and */ #include diff --git a/os/os.h b/os/os.h index afee9f9c..2e155291 100644 --- a/os/os.h +++ b/os/os.h @@ -369,4 +369,8 @@ static inline bool fio_fallocate(struct fio_file *f, uint64_t offset, uint64_t l } #endif +#if defined(CONFIG_POSIX_FALLOCATE) || defined(FIO_HAVE_NATIVE_FALLOCATE) +# define FIO_HAVE_ANY_FALLOCATE +#endif + #endif