.. option:: --aux-path=path
- Use this `path` for fio state generated files.
+ Use the directory specified by `path` for generated state files instead
+ of the current working directory.
Any parameters following the options will be assumed to be job files, unless
they match a job file parameter. Multiple job files can be listed and each job
assigned equally distributed to job clones created by :option:`numjobs` as
long as they are using generated filenames. If specific `filename(s)` are
set fio will use the first listed directory, and thereby matching the
- `filename` semantic which generates a file each clone if not specified, but
- let all clones use the same if set.
+ `filename` semantic (which generates a file for each clone if not
+ specified, but lets all clones use the same file if set).
See the :option:`filename` option for information on how to escape "``:``" and
"``\``" characters within the directory path itself.
+ Note: To control the directory fio will use for internal state files
+ use :option:`--aux-path`.
+
.. option:: filename=str
Fio normally makes up a `filename` based on the job name, thread number, and
Unlink job files after each iteration or loop. Default: false.
-.. option:: zonesize=int
+.. option:: zonerange=int
- Divide a file into zones of the specified size. See :option:`zoneskip`.
+ Size of a single zone in which I/O occurs. See also :option:`zonesize`
+ and :option:`zoneskip`.
-.. option:: zonerange=int
+.. option:: zonesize=int
- Give size of an I/O zone. See :option:`zoneskip`.
+ Number of bytes to transfer before skipping :option:`zoneskip`
+ bytes. If this parameter is smaller than :option:`zonerange` then only
+ a fraction of each zone with :option:`zonerange` bytes will be
+ accessed. If this parameter is larger than :option:`zonerange` then
+ each zone will be accessed multiple times before skipping
.. option:: zoneskip=int
- Skip the specified number of bytes when :option:`zonesize` data has been
- read. The two zone options can be used to only do I/O on zones of a file.
+ Skip the specified number of bytes when :option:`zonesize` data have
+ been transferred. The three zone options can be used to do strided I/O
+ on a file.
I/O type
(RBD) via librbd without the need to use the kernel rbd driver. This
ioengine defines engine specific options.
+ **http**
+ I/O engine supporting GET/PUT requests over HTTP(S) with libcurl to
+ a WebDAV or S3 endpoint. This ioengine defines engine specific options.
+
+ This engine only supports direct IO of iodepth=1; you need to scale this
+ via numjobs. blocksize defines the size of the objects to be created.
+
+ TRIM is translated to object deletion.
+
**gfapi**
Using GlusterFS libgfapi sync interface to direct access to
GlusterFS volumes without having to go through FUSE. This ioengine
mounted with DAX on a persistent memory device through the PMDK
libpmem library.
+ **ime_psync**
+ Synchronous read and write using DDN's Infinite Memory Engine (IME).
+ This engine is very basic and issues calls to IME whenever an IO is
+ queued.
+
+ **ime_psyncv**
+ Synchronous read and write using DDN's Infinite Memory Engine (IME).
+ This engine uses iovecs and will try to stack as much IOs as possible
+ (if the IOs are "contiguous" and the IO depth is not exceeded)
+ before issuing a call to IME.
+
+ **ime_aio**
+ Asynchronous read and write using DDN's Infinite Memory Engine (IME).
+ This engine will try to stack as much IOs as possible by creating
+ requests for IME. FIO will then decide when to commit these requests.
+
I/O engine specific parameters
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
transferred to the device. The writefua option is ignored with this
selection.
+.. option:: http_host=str : [http]
+
+ Hostname to connect to. For S3, this could be the bucket hostname.
+ Default is **localhost**
+
+.. option:: http_user=str : [http]
+
+ Username for HTTP authentication.
+
+.. option:: http_pass=str : [http]
+
+ Password for HTTP authentication.
+
+.. option:: https=str : [http]
+
+ Enable HTTPS instead of http. *on* enables HTTPS; *insecure*
+ will enable HTTPS, but disable SSL peer verification (use with
+ caution!). Default is **off**
+
+.. option:: http_mode=str : [http]
+
+ Which HTTP access mode to use: *webdav*, *swift*, or *s3*.
+ Default is **webdav**
+
+.. option:: http_s3_region=str : [http]
+
+ The S3 region/zone string.
+ Default is **us-east-1**
+
+.. option:: http_s3_key=str : [http]
+
+ The S3 secret key.
+
+.. option:: http_s3_keyid=str : [http]
+
+ The S3 key/access id.
+
+.. option:: http_swift_auth_token=str : [http]
+
+ The Swift auth token. See the example configuration file on how
+ to retrieve this.
+
+.. option:: http_verbose=int : [http]
+
+ Enable verbose requests from libcurl. Useful for debugging. 1
+ turns on verbose logging from libcurl, 2 additionally enables
+ HTTP IO tracing. Default is **0**
+
I/O depth
~~~~~~~~~
.. option:: write_iops_log=str
Same as :option:`write_bw_log`, but writes an IOPS file (e.g.
- :file:`name_iops.x.log`) instead. See :option:`write_bw_log` for
- details about the filename format and `Log File Formats`_ for how data
- is structured within the file.
+ :file:`name_iops.x.log`) instead. Because fio defaults to individual
+ I/O logging, the value entry in the IOPS log will be 1 unless windowed
+ logging (see :option:`log_avg_msec`) has been enabled. See
+ :option:`write_bw_log` for details about the filename format and `Log
+ File Formats`_ for how data is structured within the file.
.. option:: log_avg_msec=int
**2**
I/O is a TRIM
-The entry's *block size* is always in bytes. The *offset* is the offset, in bytes,
-from the start of the file, for that particular I/O. The logging of the offset can be
+The entry's *block size* is always in bytes. The *offset* is the position in bytes
+from the start of the file for that particular I/O. The logging of the offset can be
toggled with :option:`log_offset`.
-Fio defaults to logging every individual I/O. When IOPS are logged for individual
-I/Os the *value* entry will always be 1. If windowed logging is enabled through
-:option:`log_avg_msec`, fio logs the average values over the specified period of time.
-If windowed logging is enabled and :option:`log_max_value` is set, then fio logs
-maximum values in that window instead of averages. Since *data direction*, *block
-size* and *offset* are per-I/O values, if windowed logging is enabled they
-aren't applicable and will be 0.
+Fio defaults to logging every individual I/O but when windowed logging is set
+through :option:`log_avg_msec`, either the average (by default) or the maximum
+(:option:`log_max_value` is set) *value* seen over the specified period of time
+is recorded. Each *data direction* seen within the window period will aggregate
+its values in a separate row. Further, when using windowed logging the *block
+size* and *offset* entries will always contain 0.
Client/Server
-------------
fi
TMPC="${TMPDIR1}/fio-conf-${RANDOM}-$$-${RANDOM}.c"
+TMPC2="${TMPDIR1}/fio-conf-${RANDOM}-$$-${RANDOM}-2.c"
TMPO="${TMPDIR1}/fio-conf-${RANDOM}-$$-${RANDOM}.o"
TMPE="${TMPDIR1}/fio-conf-${RANDOM}-$$-${RANDOM}.exe"
# NB: do not call "exit" in the trap handler; this is buggy with some shells;
# see <1285349658-3122-1-git-send-email-loic.minier@linaro.org>
-trap "rm -f $TMPC $TMPO $TMPE" EXIT INT QUIT TERM
+trap "rm -f $TMPC $TMPC2 $TMPO $TMPE" EXIT INT QUIT TERM
rm -rf config.log
;;
--disable-rbd) disable_rbd="yes"
;;
+ --disable-http) disable_http="yes"
+ ;;
--disable-gfapi) disable_gfapi="yes"
;;
--enable-libhdfs) libhdfs="yes"
;;
--disable-native) disable_native="yes"
;;
+ --with-ime=*) ime_path="$optarg"
+ ;;
--help)
show_help="yes"
;;
echo "--disable-optimizations Don't enable compiler optimizations"
echo "--enable-cuda Enable GPUDirect RDMA support"
echo "--disable-native Don't build for native host"
+ echo "--with-ime= Install path for DDN's Infinite Memory Engine"
exit $exit_val
fi
fi
print_config "IPv6 helpers" "$ipv6"
+##########################################
+# check for http
+if test "$http" != "yes" ; then
+ http="no"
+fi
+# check for openssl >= 1.1.0, which uses an opaque HMAC_CTX pointer
+cat > $TMPC << EOF
+#include <curl/curl.h>
+#include <openssl/hmac.h>
+
+int main(int argc, char **argv)
+{
+ CURL *curl;
+ HMAC_CTX *ctx;
+
+ curl = curl_easy_init();
+ curl_easy_cleanup(curl);
+
+ ctx = HMAC_CTX_new();
+ HMAC_CTX_reset(ctx);
+ HMAC_CTX_free(ctx);
+ return 0;
+}
+EOF
+# openssl < 1.1.0 uses the HMAC_CTX type directly
+cat > $TMPC2 << EOF
+#include <curl/curl.h>
+#include <openssl/hmac.h>
+
+int main(int argc, char **argv)
+{
+ CURL *curl;
+ HMAC_CTX ctx;
+
+ curl = curl_easy_init();
+ curl_easy_cleanup(curl);
+
+ HMAC_CTX_init(&ctx);
+ HMAC_CTX_cleanup(&ctx);
+ return 0;
+}
+EOF
+if test "$disable_http" != "yes"; then
+ HTTP_LIBS="-lcurl -lssl -lcrypto"
+ if compile_prog "" "$HTTP_LIBS" "curl-new-ssl"; then
+ output_sym "CONFIG_HAVE_OPAQUE_HMAC_CTX"
+ http="yes"
+ LIBS="$HTTP_LIBS $LIBS"
+ elif mv $TMPC2 $TMPC && compile_prog "" "$HTTP_LIBS" "curl-old-ssl"; then
+ http="yes"
+ LIBS="$HTTP_LIBS $LIBS"
+ fi
+fi
+print_config "http engine" "$http"
+
##########################################
# check for rados
if test "$rados" != "yes" ; then
# Report whether libpmem engine is enabled
print_config "PMDK libpmem engine" "$pmem"
+ ##########################################
+ # Check whether we support DDN's IME
+ if test "$libime" != "yes" ; then
+ libime="no"
+ fi
+ cat > $TMPC << EOF
+ #include <ime_native.h>
+ int main(int argc, char **argv)
+ {
+ int rc;
+ ime_native_init();
+ rc = ime_native_finalize();
+ return 0;
+ }
+ EOF
+ if compile_prog "-I${ime_path}/include" "-L${ime_path}/lib -lim_client" "libime"; then
+ libime="yes"
+ CFLAGS="-I${ime_path}/include $CFLAGS"
+ LDFLAGS="-Wl,-rpath ${ime_path}/lib -L${ime_path}/lib $LDFLAGS"
+ LIBS="-lim_client $LIBS"
+ fi
+ print_config "DDN's Infinite Memory Engine" "$libime"
+
##########################################
# Check if we have lex/yacc available
yacc="no"
if test "$ipv6" = "yes" ; then
output_sym "CONFIG_IPV6"
fi
+if test "$http" = "yes" ; then
+ output_sym "CONFIG_HTTP"
+fi
if test "$rados" = "yes" ; then
output_sym "CONFIG_RADOS"
fi
if test "$pmem" = "yes" ; then
output_sym "CONFIG_LIBPMEM"
fi
+ if test "$libime" = "yes" ; then
+ output_sym "CONFIG_IME"
+ fi
if test "$arith" = "yes" ; then
output_sym "CONFIG_ARITHMETIC"
if test "$yacc_is_bison" = "yes" ; then
return 0;
}
-static void __td_zone_gen_index(struct thread_data *td, enum fio_ddir ddir)
-{
- unsigned int i, j, sprev, aprev;
- uint64_t sprev_sz;
-
- td->zone_state_index[ddir] = malloc(sizeof(struct zone_split_index) * 100);
-
- sprev_sz = sprev = aprev = 0;
- for (i = 0; i < td->o.zone_split_nr[ddir]; i++) {
- struct zone_split *zsp = &td->o.zone_split[ddir][i];
-
- for (j = aprev; j < aprev + zsp->access_perc; j++) {
- struct zone_split_index *zsi = &td->zone_state_index[ddir][j];
-
- zsi->size_perc = sprev + zsp->size_perc;
- zsi->size_perc_prev = sprev;
-
- zsi->size = sprev_sz + zsp->size;
- zsi->size_prev = sprev_sz;
- }
-
- aprev += zsp->access_perc;
- sprev += zsp->size_perc;
- sprev_sz += zsp->size;
- }
-}
-
-/*
- * Generate state table for indexes, so we don't have to do it inline from
- * the hot IO path
- */
-static void td_zone_gen_index(struct thread_data *td)
-{
- int i;
-
- td->zone_state_index = malloc(DDIR_RWDIR_CNT *
- sizeof(struct zone_split_index *));
-
- for (i = 0; i < DDIR_RWDIR_CNT; i++)
- __td_zone_gen_index(td, i);
-}
-
static int parse_zoned_distribution(struct thread_data *td, const char *input,
bool absolute)
{
return ret;
}
- if (!ret)
- td_zone_gen_index(td);
- else {
+ if (ret) {
for (i = 0; i < DDIR_RWDIR_CNT; i++)
td->o.zone_split_nr[i] = 0;
}
},
#endif
+ #ifdef CONFIG_IME
+ { .ival = "ime_psync",
+ .help = "DDN's IME synchronous IO engine",
+ },
+ { .ival = "ime_psyncv",
+ .help = "DDN's IME synchronous IO engine using iovecs",
+ },
+ { .ival = "ime_aio",
+ .help = "DDN's IME asynchronous IO engine",
+ },
+ #endif
#ifdef CONFIG_LINUX_DEVDAX
{ .ival = "dev-dax",
.help = "DAX Device based IO engine",
{ .ival = "libpmem",
.help = "PMDK libpmem based IO engine",
},
+#endif
+#ifdef CONFIG_HTTP
+ { .ival = "http",
+ .help = "HTTP (WebDAV/S3) IO engine",
+ },
#endif
},
},
projects / fio.git / commitdiff