'#', the entire line is discarded as a comment.
So let's look at a really simple job file that defines two processes, each
-randomly reading from a 128MiB file.
+randomly reading from a 128MB file.
; -- start job file --
[global]
Here we have no global section, as we only have one job defined anyway.
We want to use async io here, with a depth of 4 for each file. We also
-increased the buffer size used to 32KiB and define numjobs to 4 to
+increased the buffer size used to 32KB and define numjobs to 4 to
fork 4 identical jobs. The result is 4 processes each randomly writing
-to their own 64MiB file. Instead of using the above job file, you could
+to their own 64MB file. Instead of using the above job file, you could
have given the parameters on the command line. For this case, you would
specify:
a string. The following types are used:
str String. This is a sequence of alpha characters.
-time Integer with possible time postfix. In seconds unless otherwise
+time Integer with possible time suffix. In seconds unless otherwise
specified, use eg 10m for 10 minutes. Accepts s/m/h for seconds,
minutes, and hours.
-int SI integer. A whole number value, which may contain a postfix
- describing the base of the number. Accepted postfixes are k/m/g,
- meaning kilo, mega, and giga. So if you want to specify 4096,
- you could either write out '4096' or just give 4k. The postfixes
- signify base 2 values, so 1024 is 1k and 1024k is 1m and so on.
- If the option accepts an upper and lower range, use a colon ':'
- or minus '-' to separate such values. May also include a prefix
- to indicate numbers base. If 0x is used, the number is assumed to
- be hexadecimal. See irange.
+int SI integer. A whole number value, which may contain a suffix
+ describing the base of the number. Accepted suffixes are k/m/g/t/p,
+ meaning kilo, mega, giga, tera, and peta. The suffix is not case
+ sensitive. So if you want to specify 4096, you could either write
+ out '4096' or just give 4k. The suffixes signify base 2 values, so
+ 1024 is 1k and 1024k is 1m and so on. If the option accepts an upper
+ and lower range, use a colon ':' or minus '-' to separate such values.
+ May also include a prefix to indicate numbers base. If 0x is used,
+ the number is assumed to be hexadecimal. See irange.
bool Boolean. Usually parsed as an integer, however only defined for
true and false (1 and 0).
-irange Integer range with postfix. Allows value range to be given, such
+irange Integer range with suffix. Allows value range to be given, such
as 1024-4096. A colon may also be used as the separator, eg
1k:4k. If the option allows two sets of ranges, they can be
specified with a ',' or '/' delimiter: 1k-4k/8k-32k. Also see
IO's, instead of for every IO. Use rw=randread:8 to specify
that.
+kb_base=int The base unit for a kilobyte. The defacto base is 2^10, 1024.
+ Storage manufacturers like to use 10^3 or 1000 as a base
+ ten unit instead, for obvious reasons. Allow values are
+ 1024 or 1000, with 1024 being the default.
+
randrepeat=bool For random IO workloads, seed the generator in a predictable
way so that results are repeatable across repetitions.
after every block.
rate=int Cap the bandwidth used by this job. The number is in bytes/sec,
- the normal postfix rules apply. You can use rate=500k to limit
+ the normal suffix rules apply. You can use rate=500k to limit
reads and writes to 500k each, or you can specify read and
writes separately. Using rate=1m,500k would limit reads to
1MB/sec and writes to 500KB/sec. Capping only reads or
that for shmhuge and mmaphuge to work, the system must have
free huge pages allocated. This can normally be checked
and set by reading/writing /proc/sys/vm/nr_hugepages on a
- Linux system. Fio assumes a huge page is 4MiB in size. So
+ Linux system. Fio assumes a huge page is 4MB in size. So
to calculate the number of huge pages you need for a given
job file, add up the io depth of all jobs (normally one unless
iodepth= is used) and multiply by the maximum bs set. Then
hugepage-size=int
Defines the size of a huge page. Must at least be equal
- to the system setting, see /proc/meminfo. Defaults to 4MiB.
+ to the system setting, see /proc/meminfo. Defaults to 4MB.
Should probably always be a multiple of megabytes, so using
hugepage-size=Xm is the preferred way to set this to avoid
setting a non-pow-2 bad value.
direction, the output looks like:
Client1 (g=0): err= 0:
- write: io= 32MiB, bw= 666KiB/s, runt= 50320msec
+ write: io= 32MB, bw= 666KB/s, runt= 50320msec
slat (msec): min= 0, max= 136, avg= 0.03, stdev= 1.92
clat (msec): min= 0, max= 631, avg=48.50, stdev=86.82
- bw (KiB/s) : min= 0, max= 1196, per=51.00%, avg=664.02, stdev=681.68
+ bw (KB/s) : min= 0, max= 1196, per=51.00%, avg=664.02, stdev=681.68
cpu : usr=1.49%, sys=0.25%, ctx=7969, majf=0, minf=17
IO depths : 1=0.1%, 2=0.3%, 4=0.5%, 8=99.0%, 16=0.0%, 32=0.0%, >32=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
will look like this:
Run status group 0 (all jobs):
- READ: io=64MiB, aggrb=22178, minb=11355, maxb=11814, mint=2840msec, maxt=2955msec
- WRITE: io=64MiB, aggrb=1302, minb=666, maxb=669, mint=50093msec, maxt=50320msec
+ READ: io=64MB, aggrb=22178, minb=11355, maxb=11814, mint=2840msec, maxt=2955msec
+ WRITE: io=64MB, aggrb=1302, minb=666, maxb=669, mint=50093msec, maxt=50320msec
For each data direction, it prints:
jobname, groupid, error
READ status:
- KiB IO, bandwidth (KiB/sec), runtime (msec)
+ KB IO, bandwidth (KB/sec), runtime (msec)
Submission latency: min, max, mean, deviation
Completion latency: min, max, mean, deviation
Bw: min, max, aggregate percentage of total, mean, deviation
WRITE status:
- KiB IO, bandwidth (KiB/sec), runtime (msec)
+ KB IO, bandwidth (KB/sec), runtime (msec)
Submission latency: min, max, mean, deviation
Completion latency: min, max, mean, deviation
Bw: min, max, aggregate percentage of total, mean, deviation
also include k/m postfix.
direct=x 1 for direct IO, 0 for buffered IO
thinktime=x "Think" x usec after each io
- rate=x Throttle rate to x KiB/sec
- ratemin=x Quit if rate of x KiB/sec can't be met
+ rate=x Throttle rate to x KB/sec
+ ratemin=x Quit if rate of x KB/sec can't be met
ratecycle=x ratemin averaged over x msecs
cpumask=x Only allow job to run on CPUs defined by mask.
cpus_allowed=x Like 'cpumask', but allow text setting of CPU affinity.
static unsigned int rate[2], iops[2];
static int linelen_last;
static int eta_good;
+ int i2p = 0;
if (temp_stall_ts || terse_output || eta_print == FIO_ETA_NEVER)
return;
eta_sec = 0;
for_each_td(td, i) {
+ if (!i2p && is_power_of_2(td->o.kb_base))
+ i2p = 1;
if (exitall_on_terminate) {
if (eta_secs[i] < eta_sec)
eta_sec = eta_secs[i];
if (m_rate || t_rate) {
char *tr, *mr;
- mr = num2str(m_rate, 4, 0, 1);
- tr = num2str(t_rate, 4, 0, 1);
- printf(", CR=%s/%s KiB/s", tr, mr);
+ mr = num2str(m_rate, 4, 0, i2p);
+ tr = num2str(t_rate, 4, 0, i2p);
+ printf(", CR=%s/%s KB/s", tr, mr);
free(tr);
free(mr);
} else if (m_iops || t_iops)
sprintf(perc_str, "%3.1f%% done", perc);
}
- rate_str[0] = num2str(rate[0], 5, 10, 1);
- rate_str[1] = num2str(rate[1], 5, 10, 1);
+ rate_str[0] = num2str(rate[0], 5, 10, i2p);
+ rate_str[1] = num2str(rate[1], 5, 10, i2p);
iops_str[0] = num2str(iops[0], 4, 1, 0);
iops_str[1] = num2str(iops[1], 4, 1, 0);
temp_stall_ts = 1;
if (!terse_output)
log_info("%s: Laying out IO file(s) (%u file(s) /"
- " %LuMiB)\n", td->o.name, need_extend,
+ " %LuMB)\n", td->o.name, need_extend,
extend_size >> 20);
for_each_file(td, f, i) {
.TP
.I int
SI integer: a whole number, possibly containing a suffix denoting the base unit
-of the value. Accepted suffixes are `k', 'M' and 'G', denoting kilo (1024),
-mega (1024*1024) and giga (1024*1024*1024) respectively. If prefixed with '0x',
-the value is assumed to be base 16 (hexadecimal).
+of the value. Accepted suffixes are `k', 'M', 'G', 'T', and 'P', denoting
+kilo (1024), mega (1024^2), giga (1024^3), tera (1024^4), and peta (1024^5)
+respectively. The suffix is not case sensitive. If prefixed with '0x', the
+value is assumed to be base 16 (hexadecimal).
.TP
.I bool
Boolean: a true or false value. `0' denotes false, `1' denotes true.
`:\fIint\fR' to the pattern type. The default is 1.
.RE
.TP
+.BI kb_base \fR=\fPint
+The base unit for a kilobyte. The defacto base is 2^10, 1024. Storage
+manufacturers like to use 10^3 or 1000 as a base ten unit instead, for obvious
+reasons. Allow values are 1024 or 1000, with 1024 being the default.
+.TP
.BI randrepeat \fR=\fPbool
Seed the random number generator in a predictable way so results are repeatable
across runs. Default: true.
.TP
.BI hugepage\-size \fR=\fPint
Defines the size of a huge page. Must be at least equal to the system setting.
-Should be a multiple of 1MiB. Default: 4MiB.
+Should be a multiple of 1MB. Default: 4MB.
.TP
.B exitall
Terminate all jobs when one finishes. Default: wait for each job to finish.
.P
Read status:
.RS
-.B KiB I/O, bandwidth \fR(KiB/s)\fP, runtime \fR(ms)\fP
+.B KB I/O, bandwidth \fR(KB/s)\fP, runtime \fR(ms)\fP
.P
Submission latency:
.RS
.P
Write status:
.RS
-.B KiB I/O, bandwidth \fR(KiB/s)\fP, runtime \fR(ms)\fP
+.B KB I/O, bandwidth \fR(KB/s)\fP, runtime \fR(ms)\fP
.P
Submission latency:
.RS
if (rate < ratemin ||
bytes < td->rate_bytes[ddir]) {
log_err("%s: min rate %u not met, got"
- " %luKiB/sec\n", td->o.name,
+ " %luKB/sec\n", td->o.name,
ratemin, rate);
return 1;
}
unsigned long long max_bw[2], min_bw[2];
unsigned long long io_kb[2];
unsigned long long agg[2];
+ unsigned int kb_base;
};
/*
unsigned continue_on_error;
unsigned long total_err_count;
int first_error;
+
+ unsigned int kb_base;
};
struct bssplit {
char *opendir;
char *ioengine;
enum td_ddir td_ddir;
+ unsigned int kb_base;
unsigned int ddir_nr;
unsigned int iodepth;
unsigned int iodepth_low;
if [ "$PLOT_LINE"x != "x" ]; then
echo Making bw logs
- echo "set title 'Bandwidth - $TITLE'; set xlabel 'time (msec)'; set ylabel 'KiB/sec'; set terminal png; set output '$TITLE-bw.png'; plot " $PLOT_LINE | $GNUPLOT -
+ echo "set title 'Bandwidth - $TITLE'; set xlabel 'time (msec)'; set ylabel 'KB/sec'; set terminal png; set output '$TITLE-bw.png'; plot " $PLOT_LINE | $GNUPLOT -
fi
PLOT_LINE=""
if (phys_mem) {
if ((mlock_size + 128 * 1024 * 1024) > phys_mem) {
mlock_size = phys_mem - 128 * 1024 * 1024;
- log_info("fio: limiting mlocked memory to %lluMiB\n",
+ log_info("fio: limiting mlocked memory to %lluMB\n",
mlock_size >> 20);
}
}
} else
perc = -1;
- if (str_to_decimal(fname, &val, 1)) {
+ if (str_to_decimal(fname, &val, 1, &td)) {
log_err("fio: bssplit conversion failed\n");
free(td->o.bssplit);
return 1;
return 0;
}
+static int kb_base_verify(struct fio_option *o, void *data)
+{
+ struct thread_data *td = data;
+
+ if (td->o.kb_base != 1024 && td->o.kb_base != 1000) {
+ log_err("fio: kb_base set to nonsensical value: %u\n",
+ td->o.kb_base);
+ return 1;
+ }
+
+ return 0;
+}
+
#define __stringify_1(x) #x
#define __stringify(x) __stringify_1(x)
.prio = -1, /* must come after "directory" */
.help = "File(s) to use for the workload",
},
+ {
+ .name = "kb_base",
+ .type = FIO_OPT_INT,
+ .off1 = td_var_offset(kb_base),
+ .verify = kb_base_verify,
+ .prio = 1,
+ .def = "1024",
+ .help = "How many bytes per KB for reporting (1000 or 1024)",
+ },
{
.name = "lockfile",
.type = FIO_OPT_STR,
__options_mem(td, 0);
#endif
}
+
+unsigned int fio_get_kb_base(void *data)
+{
+ struct thread_data *td = data;
+ unsigned int kb_base = 0;
+
+ if (td)
+ kb_base = td->o.kb_base;
+ if (!kb_base)
+ kb_base = 1024;
+
+ return kb_base;
+}
#include "debug.h"
static struct fio_option *fio_options;
+extern unsigned int fio_get_kb_base(void *);
static int vp_cmp(const void *p1, const void *p2)
{
qsort(vpmap, entries, sizeof(struct value_pair), vp_cmp);
}
-static void show_option_range(struct fio_option *o)
+static void show_option_range(struct fio_option *o, FILE *out)
{
if (!o->minval && !o->maxval)
return;
- printf("%20s: min=%d", "range", o->minval);
+ fprintf(out, "%20s: min=%d", "range", o->minval);
if (o->maxval)
- printf(", max=%d", o->maxval);
- printf("\n");
+ fprintf(out, ", max=%d", o->maxval);
+ fprintf(out, "\n");
}
static void show_option_values(struct fio_option *o)
printf("\n");
}
+static void show_option_help(struct fio_option *o, FILE *out)
+{
+ const char *typehelp[] = {
+ "string (opt=bla)",
+ "string with possible k/m/g postfix (opt=4k)",
+ "string with time postfix (opt=10s)",
+ "string (opt=bla)",
+ "string with dual range (opt=1k-4k,4k-8k)",
+ "integer value (opt=100)",
+ "boolean value (opt=1)",
+ "no argument (opt)",
+ };
+
+ if (o->alias)
+ fprintf(out, "%20s: %s\n", "alias", o->alias);
+
+ fprintf(out, "%20s: %s\n", "type", typehelp[o->type]);
+ fprintf(out, "%20s: %s\n", "default", o->def ? o->def : "no default");
+ show_option_range(o, stdout);
+ show_option_values(o);
+}
+
static unsigned long get_mult_time(char c)
{
switch (c) {
}
}
-static unsigned long get_mult_bytes(char c)
+static unsigned long long get_mult_bytes(char c, void *data)
{
+ unsigned int kb_base = fio_get_kb_base(data);
+ unsigned long long ret = 1;
+
switch (c) {
- case 'k':
- case 'K':
- return 1024;
- case 'm':
- case 'M':
- return 1024 * 1024;
+ default:
+ break;
+ case 'p':
+ case 'P':
+ ret *= (unsigned long long) kb_base;
+ case 't':
+ case 'T':
+ ret *= (unsigned long long) kb_base;
case 'g':
case 'G':
- return 1024 * 1024 * 1024;
- case 'e':
- case 'E':
- return 1024 * 1024 * 1024 * 1024UL;
- default:
- return 1;
+ ret *= (unsigned long long) kb_base;
+ case 'm':
+ case 'M':
+ ret *= (unsigned long long) kb_base;
+ case 'k':
+ case 'K':
+ ret *= (unsigned long long) kb_base;
+ break;
}
+
+ return ret;
}
/*
* convert string into decimal value, noting any size suffix
*/
-int str_to_decimal(const char *str, long long *val, int kilo)
+int str_to_decimal(const char *str, long long *val, int kilo, void *data)
{
int len, base;
return 1;
if (kilo)
- *val *= get_mult_bytes(str[len - 1]);
+ *val *= get_mult_bytes(str[len - 1], data);
else
*val *= get_mult_time(str[len - 1]);
return 0;
}
-static int check_str_bytes(const char *p, long long *val)
+static int check_str_bytes(const char *p, long long *val, void *data)
{
- return str_to_decimal(p, val, 1);
+ return str_to_decimal(p, val, 1, data);
}
static int check_str_time(const char *p, long long *val)
{
- return str_to_decimal(p, val, 0);
+ return str_to_decimal(p, val, 0, NULL);
}
void strip_blank_front(char **p)
*(s + 1) = '\0';
}
-static int check_range_bytes(const char *str, long *val)
+static int check_range_bytes(const char *str, long *val, void *data)
{
char suffix;
return 1;
if (sscanf(str, "%lu%c", val, &suffix) == 2) {
- *val *= get_mult_bytes(suffix);
+ *val *= get_mult_bytes(suffix, data);
return 0;
}
if (is_time)
ret = check_str_time(ptr, &ull);
else
- ret = check_str_bytes(ptr, &ull);
+ ret = check_str_bytes(ptr, &ull, data);
if (ret)
break;
p1 = tmp;
ret = 1;
- if (!check_range_bytes(p1, &ul1) &&
- !check_range_bytes(p2, &ul2)) {
+ if (!check_range_bytes(p1, &ul1, data) &&
+ !check_range_bytes(p2, &ul2, data)) {
ret = 0;
if (ul1 > ul2) {
unsigned long foo = ul1;
if (ret)
return ret;
- if (o->verify)
+ if (o->verify) {
ret = o->verify(o, data);
+ if (ret) {
+ fprintf(stderr,"Correct format for offending option\n");
+ fprintf(stderr, "%20s: %s\n", o->name, o->help);
+ show_option_help(o, stderr);
+ }
+ }
return ret;
}
return i;
}
-static void show_option_help(struct fio_option *o)
-{
- const char *typehelp[] = {
- "string (opt=bla)",
- "string with possible k/m/g postfix (opt=4k)",
- "string with time postfix (opt=10s)",
- "string (opt=bla)",
- "string with dual range (opt=1k-4k,4k-8k)",
- "integer value (opt=100)",
- "boolean value (opt=1)",
- "no argument (opt)",
- };
-
- if (o->alias)
- printf("%20s: %s\n", "alias", o->alias);
-
- printf("%20s: %s\n", "type", typehelp[o->type]);
- printf("%20s: %s\n", "default", o->def ? o->def : "no default");
- show_option_range(o);
- show_option_values(o);
-}
-
static struct fio_option *find_child(struct fio_option *options,
struct fio_option *o)
{
if (!match)
continue;
- show_option_help(o);
+ show_option_help(o, stdout);
}
if (found)
if (closest) {
printf(" - showing closest match\n");
printf("%20s: %s\n", closest->name, closest->help);
- show_option_help(closest);
+ show_option_help(closest, stdout);
} else
printf("\n");
extern void strip_blank_front(char **);
extern void strip_blank_end(char *);
-extern int str_to_decimal(const char *, long long *, int);
+extern int str_to_decimal(const char *, long long *, int, void *);
/*
* Handlers for the options
log_info("\nRun status group %d (all jobs):\n", id);
for (i = 0; i <= DDIR_WRITE; i++) {
+ const int i2p = is_power_of_2(rs->kb_base);
+
if (!rs->max_run[i])
continue;
- p1 = num2str(rs->io_kb[i], 6, 1024, 1);
- p2 = num2str(rs->agg[i], 6, 1024, 1);
- p3 = num2str(rs->min_bw[i], 6, 1024, 1);
- p4 = num2str(rs->max_bw[i], 6, 1024, 1);
+ p1 = num2str(rs->io_kb[i], 6, rs->kb_base, i2p);
+ p2 = num2str(rs->agg[i], 6, rs->kb_base, i2p);
+ p3 = num2str(rs->min_bw[i], 6, rs->kb_base, i2p);
+ p4 = num2str(rs->max_bw[i], 6, rs->kb_base, i2p);
- log_info("%s: io=%siB, aggrb=%siB/s, minb=%siB/s, maxb=%siB/s,"
+ log_info("%s: io=%sB, aggrb=%sB/s, minb=%sB/s, maxb=%sB/s,"
" mint=%llumsec, maxt=%llumsec\n", ddir_str[i], p1, p2,
p3, p4, rs->min_run[i],
rs->max_run[i]);
unsigned long long bw, iops;
double mean, dev;
char *io_p, *bw_p, *iops_p;
+ int i2p;
if (!ts->runtime[ddir])
return;
+ i2p = is_power_of_2(rs->kb_base);
runt = ts->runtime[ddir];
bw = (1000 * ts->io_bytes[ddir]) / runt;
- io_p = num2str(ts->io_bytes[ddir] >> 10, 6, 1024, 1);
- bw_p = num2str(bw >> 10, 6, 1024, 1);
+ io_p = num2str(ts->io_bytes[ddir], 6, 1, i2p);
+ bw_p = num2str(bw, 6, 1, i2p);
iops = (1000 * ts->total_io_u[ddir]) / runt;
iops_p = num2str(iops, 6, 1, 0);
- log_info(" %s: io=%siB, bw=%siB/s, iops=%s, runt=%6lumsec\n",
+ log_info(" %s: io=%sB, bw=%sB/s, iops=%s, runt=%6lumsec\n",
ddir_str[ddir], io_p, bw_p, iops_p,
ts->runtime[ddir]);
double p_of_agg;
p_of_agg = mean * 100 / (double) rs->agg[ddir];
- log_info(" bw (KiB/s) : min=%5lu, max=%5lu, per=%3.2f%%,"
+ log_info(" bw (KB/s) : min=%5lu, max=%5lu, per=%3.2f%%,"
" avg=%5.02f, stdev=%5.02f\n", min, max, p_of_agg,
mean, dev);
}
struct thread_data *td;
struct thread_stat *threadstats, *ts;
int i, j, k, l, nr_ts, last_ts, idx;
+ int kb_base_warned = 0;
runstats = malloc(sizeof(struct group_run_stats) * (groupid + 1));
* first pid in group, not very useful...
*/
ts->pid = td->pid;
+
+ ts->kb_base = td->o.kb_base;
+ } else if (ts->kb_base != td->o.kb_base && !kb_base_warned) {
+ log_info("fio: kb_base differs for jobs in group, using"
+ " %u as the base\n", ts->kb_base);
+ kb_base_warned = 1;
}
ts->continue_on_error = td->o.continue_on_error;
ts = &threadstats[i];
rs = &runstats[ts->groupid];
+ rs->kb_base = ts->kb_base;
for (j = 0; j <= DDIR_WRITE; j++) {
if (!ts->runtime[j])
if (ts->runtime[j]) {
unsigned long runt;
- runt = ts->runtime[j] * 1024 / 1000;
+ runt = ts->runtime[j];
bw = ts->io_bytes[j] / runt;
}
if (bw < rs->min_bw[j])
if (bw > rs->max_bw[j])
rs->max_bw[j] = bw;
- rs->io_kb[j] += ts->io_bytes[j] >> 10;
+ rs->io_kb[j] += ts->io_bytes[j] / rs->kb_base;
}
}
unsigned long max_run[2];
rs = &runstats[i];
- max_run[0] = rs->max_run[0] * 1024 / 1000;
- max_run[1] = rs->max_run[1] * 1024 / 1000;
+ max_run[0] = rs->max_run[0];
+ max_run[1] = rs->max_run[1];
if (rs->max_run[0])
- rs->agg[0] = (rs->io_kb[0]*1024) / max_run[0];
+ rs->agg[0] = (rs->io_kb[0] * 1000) / max_run[0];
if (rs->max_run[1])
- rs->agg[1] = (rs->io_kb[1]*1024) / max_run[1];
+ rs->agg[1] = (rs->io_kb[1] * 1000) / max_run[1];
}
/*
projects / fio.git / commitdiff