genfio: Splitting gen_template in 2 parts
[fio.git] / README
CommitLineData
ebac4655
JA
1fio
2---
3
79809113
JA
4fio is a tool that will spawn a number of threads or processes doing a
5particular type of io action as specified by the user. fio takes a
6number of global parameters, each inherited by the thread unless
7otherwise parameters given to them overriding that setting is given.
8The typical use of fio is to write a job file matching the io load
9one wants to simulate.
ebac4655 10
2b02b546
JA
11
12Source
13------
14
15fio resides in a git repo, the canonical place is:
16
6b3eccb1 17git://git.kernel.dk/fio.git
97f049c9 18
a9bac3f9
JA
19If you are inside a corporate firewall, git:// may not always work for
20you. In that case you can use the http protocol, path is the same:
21
22http://git.kernel.dk/fio.git
2b02b546 23
79809113
JA
24Snapshots are frequently generated and they include the git meta data as
25well. You can download them here:
2b02b546
JA
26
27http://brick.kernel.dk/snaps/
28
1053a106 29
d85b1add
SK
30Binary packages
31---------------
32
33Debian:
34Starting with Debian "Squeeze", fio packages are part of the official
35Debian repository. http://packages.debian.org/search?keywords=fio
36
37Ubuntu:
38Starting with Ubuntu 10.04 LTS (aka "Lucid Lynx"), fio packages are part
39of the Ubuntu "universe" repository.
40http://packages.ubuntu.com/search?keywords=fio
41
d85b1add 42Red Hat, CentOS & Co:
a68594cb 43Dag Wieërs has RPMs for Red Hat related distros, find them here:
a68594cb
JA
44http://dag.wieers.com/rpm/packages/fio/
45
d85b1add 46Mandriva:
244e170e
JA
47Mandriva has integrated fio into their package repository, so installing
48on that distro should be as easy as typing 'urpmi fio'.
49
d85b1add
SK
50Solaris:
51Packages for Solaris are available from OpenCSW. Install their pkgutil
52tool (http://www.opencsw.org/get-it/pkgutil/) and then install fio via
53'pkgutil -i fio'.
54
ecc314ba
BC
55Windows:
56Bruce Cran <bruce@cran.org.uk> has fio packages for Windows at
78080867 57http://www.bluestop.org/fio/ .
ecc314ba 58
2b02b546 59
726f6ff0
JA
60Mailing list
61------------
62
63There's a mailing list associated with fio. It's meant for general
2e8552b0
JA
64discussion, bug reporting, questions, and development - basically anything
65that has to do with fio. An automated mail detailing recent commits is
66automatically sent to the list at most daily. The list address is
67fio@vger.kernel.org, subscribe by sending an email to
68majordomo@vger.kernel.org with
69
70subscribe fio
71
4f5d1526
EIB
72in the body of the email. Archives can be found here:
73
74http://www.spinics.net/lists/fio/
75
76and archives for the old list can be found here:
2e8552b0
JA
77
78http://maillist.kernel.dk/fio-devel/
726f6ff0
JA
79
80
bbfd6b00
JA
81Building
82--------
83
6e1e384e 84Just type 'configure', 'make' and 'make install'.
bbfd6b00 85
d015e398
BC
86Note that GNU make is required. On BSD it's available from devel/gmake;
87on Solaris it's in the SUNWgmake package. On platforms where GNU make
88isn't the default, type 'gmake' instead of 'make'.
bbfd6b00 89
6e1e384e
JA
90Configure will print the enabled options. Note that on Linux based
91platforms, you'll need to have the libaio development packages
92installed to use the libaio engine. Depending on distro, it is
93usually called libaio-devel or libaio-dev.
6de43c1b 94
6e1e384e
JA
95For gfio, you need gtk 2.18 or newer and associated glib threads
96and cairo. gfio isn't built automatically, it needs to be enabled
97with a --enable-gfio option to configure.
6de43c1b 98
2382dca7
AC
99To build FIO with a cross-compiler:
100 $ make clean
101 $ make CROSS_COMPILE=/path/to/toolchain/prefix
102Configure will attempt to determine the target platform automatically.
103
bbfd6b00 104
53adf64f
BC
105Windows
106-------
107
f41862f7
BC
108On Windows Cygwin (http://www.cygwin.com/) is required in order to
109build fio. To create an MSI installer package install WiX 3.7 from
110http://wixtoolset.org and run dobuild.cmd from the
93bcfd20 111os/windows directory.
53adf64f 112
f41862f7
BC
113How to compile FIO on 64-bit Windows:
114
115 1. Install Cygwin (http://www.cygwin.com/setup.exe). Install 'make' and all
116 packages starting with 'mingw64-i686' and 'mingw64-x86_64'.
117 2. Download ftp://sourceware.org/pub/pthreads-win32/prebuilt-dll-2-9-1-release/dll/x64/pthreadGC2.dll
118 and copy to the fio source directory.
119 3. Open the Cygwin Terminal.
120 4. Go to the fio directory (source files).
121 5. Run 'make clean'.
122 6. Run 'make'.
444310ff 123
7409711b
HL
124To build fio on 32-bit Windows, download x86/pthreadGC2.dll instead and do
125'./configure --build-32bit-win=yes' before 'make'.
126
78080867
BC
127It's recommended that once built or installed, fio be run in a Command Prompt
128or other 'native' console such as console2, since there are known to be display
129and signal issues when running it under a Cygwin shell
130(see http://code.google.com/p/mintty/issues/detail?id=56 for details).
131
53adf64f 132
972cfd25
JA
133Command line
134------------
ebac4655
JA
135
136$ fio
1cfd036f 137 --debug Enable some debugging options (see below)
111e032d 138 --parse-only Parse options only, don't start any IO
1cfd036f 139 --output Write output to file
b2cecdc2 140 --runtime Runtime in seconds
bebe6398
JA
141 --latency-log Generate per-job latency logs
142 --bandwidth-log Generate per-job bandwidth logs
1cfd036f 143 --minimal Minimal (terse) output
f3afa57e 144 --output-format=type Output format (terse,json,normal)
3449ab8c 145 --terse-version=type Terse version output format (default 3, or 2 or 4).
f3afa57e 146 --version Print version info and exit
1cfd036f 147 --help Print this page
23893646 148 --cpuclock-test Perform test/validation of CPU clock
bebe6398 149 --cmdhelp=cmd Print command help, "all" for all of them
de890a1e
SL
150 --enghelp=engine Print ioengine help, or list available ioengines
151 --enghelp=engine,cmd Print help for an ioengine cmd
1cfd036f 152 --showcmd Turn a job file into command line options
ad0a2735 153 --readonly Turn on safety read-only checks, preventing
bebe6398 154 writes
1cfd036f 155 --eta=when When ETA estimate should be printed
bebe6398 156 May be "always", "never" or "auto"
e382e661 157 --eta-newline=time Force a new line for every 'time' period passed
06464907 158 --status-interval=t Force full status dump every 't' period passed
bebe6398
JA
159 --section=name Only run specified section in job file.
160 Multiple sections can be specified.
e7cb819b 161 --alloc-size=kb Set smalloc pool to this size in kb (def 1024)
162 --warnings-fatal Fio parser warnings are fatal
fca70358 163 --max-jobs Maximum number of threads/processes to support
bebe6398
JA
164 --server=args Start backend server. See Client/Server section.
165 --client=host Connect to specified backend.
f2a2ce0e
HL
166 --idle-prof=option Report cpu idleness on a system or percpu basis
167 (option=system,percpu) or run unit work
168 calibration only (option=calibrate).
e592a06b 169
b4692828
JA
170
171Any parameters following the options will be assumed to be job files,
172unless they match a job file parameter. You can add as many as you want,
173each job file will be regarded as a separate group and fio will stonewall
174its execution.
972cfd25 175
ecc314ba 176The --readonly switch is an extra safety guard to prevent accidentally
724e4435
JA
177turning on a write setting when that is not desired. Fio will only write
178if rw=write/randwrite/rw/randrw is given, but this extra safety net can
179be used as an extra precaution. It will also enable a write check in the
180io engine core to prevent an accidental write due to a fio bug.
181
ee56ad50
JA
182The debug switch allows adding options that trigger certain logging
183options in fio. Currently the options are:
184
185 process Dump info related to processes
186 file Dump info related to file actions
e7cb819b 187 io Dump info related to IO queuing
188 mem Dump info related to memory allocations
bd6f78b2
JA
189 blktrace Dump info related to blktrace setup
190 verify Dump info related to IO verification
e7cb819b 191 all Enable all debug options
811a0d06 192 random Dump info related to random offset generation
a3d741fa 193 parse Dump info related to option matching and parsing
cd991b9e 194 diskutil Dump info related to disk utilization updates
5e1d306e 195 job:x Dump info only related to job number x
29adda3c 196 mutex Dump info only related to mutex up/down ops
c223da83
JA
197 profile Dump info related to profile extensions
198 time Dump info related to internal time keeping
bd6f78b2 199 ? or help Show available debug options.
ee56ad50
JA
200
201You can specify as many as you want, eg --debug=file,mem will enable
bd6f78b2 202file and memory debugging.
ee56ad50 203
01f06b63
JA
204The section switch is meant to make it easier to ship a bigger job file
205instead of several smaller ones. Say you define a job file with light,
206moderate, and heavy parts. Then you can ask fio to run the given part
207only by giving it a --section=heavy command line option. The section
208option only applies to job sections, the reserved 'global' section is
209always parsed and taken into account.
210
2b386d25
JA
211Fio has an internal allocator for shared memory called smalloc. It
212allocates shared structures from this pool. The pool defaults to 1024k
931823ca 213in size, and can grow to 128 pools. If running large jobs with randommap
2b386d25 214enabled it can run out of memory, in which case the --alloc-size switch
931823ca
JA
215is handy for starting with a larger pool size. The backing store is
216files in /tmp. Fio cleans up after itself, while it is running you
217may see .fio_smalloc.* files in /tmp.
2b386d25 218
79809113
JA
219
220Job file
221--------
222
71bfa161 223See the HOWTO file for a more detailed description of parameters and what
4661f3d0
JA
224they mean. This file contains the terse version. You can describe big and
225complex setups with the command line, but generally it's a lot easier to
71bfa161 226just write a simple job file to describe the workload. The job file format
4661f3d0 227is in the ini style format, as that is easy to read and write for the user.
79809113 228
8079cb46
JA
229The HOWTO or man page has a full list of all options, along with
230descriptions, etc. The --cmdhelp option also lists all options. If
231used with an option argument, it will detail that particular option.
79809113 232
217bc04b 233
bebe6398
JA
234Client/server
235------------
236
237Normally you would run fio as a stand-alone application on the machine
238where the IO workload should be generated. However, it is also possible to
239run the frontend and backend of fio separately. This makes it possible to
240have a fio server running on the machine(s) where the IO workload should
241be running, while controlling it from another machine.
242
243To start the server, you would do:
244
245fio --server=args
246
247on that machine, where args defines what fio listens to. The arguments
811826be
JA
248are of the form 'type,hostname or IP,port'. 'type' is either 'ip' (or ip4)
249for TCP/IP v4, 'ip6' for TCP/IP v6, or 'sock' for a local unix domain socket.
250'hostname' is either a hostname or IP address, and 'port' is the port to
251listen to (only valid for TCP/IP, not a local socket). Some examples:
bebe6398
JA
252
2531) fio --server
254
255 Start a fio server, listening on all interfaces on the default port (8765).
256
811826be 2572) fio --server=ip:hostname,4444
bebe6398
JA
258
259 Start a fio server, listening on IP belonging to hostname and on port 4444.
260
811826be
JA
2613) fio --server=ip6:::1,4444
262
263 Start a fio server, listening on IPv6 localhost ::1 and on port 4444.
264
2654) fio --server=,4444
bebe6398
JA
266
267 Start a fio server, listening on all interfaces on port 4444.
268
811826be 2695) fio --server=1.2.3.4
bebe6398
JA
270
271 Start a fio server, listening on IP 1.2.3.4 on the default port.
272
811826be 2736) fio --server=sock:/tmp/fio.sock
bebe6398
JA
274
275 Start a fio server, listening on the local socket /tmp/fio.sock.
276
277When a server is running, you can connect to it from a client. The client
278is run with:
279
280fio --local-args --client=server --remote-args <job file(s)>
281
282where --local-args are arguments that are local to the client where it is
283running, 'server' is the connect string, and --remote-args and <job file(s)>
284are sent to the server. The 'server' string follows the same format as it
285does on the server side, to allow IP/hostname/socket and port strings.
286You can connect to multiple clients as well, to do that you could run:
287
a7321eed 288fio --client=server2 <job file(s)> --client=server2 <job file(s)>
bebe6398
JA
289
290
217bc04b
JA
291Platforms
292---------
293
ce600ac9
JA
294Fio works on (at least) Linux, Solaris, AIX, HP-UX, OSX, NetBSD, Windows
295and FreeBSD. Some features and/or options may only be available on some of
296the platforms, typically because those features only apply to that platform
297(like the solarisaio engine, or the splice engine on Linux).
217bc04b
JA
298
299Some features are not available on FreeBSD/Solaris even if they could be
300implemented, I'd be happy to take patches for that. An example of that is
301disk utility statistics and (I think) huge page support, support for that
302does exist in FreeBSD/Solaris.
303
304Fio uses pthread mutexes for signalling and locking and FreeBSD does not
305support process shared pthread mutexes. As a result, only threads are
306supported on FreeBSD. This could be fixed with sysv ipc locking or
307other locking alternatives.
308
309Other *BSD platforms are untested, but fio should work there almost out
310of the box. Since I don't do test runs or even compiles on those platforms,
311your mileage may vary. Sending me patches for other platforms is greatly
312appreciated. There's a lot of value in having the same test/benchmark tool
313available on all platforms.
314
bf2e821a
CC
315Note that POSIX aio is not enabled by default on AIX. If you get messages like:
316
317 Symbol resolution failed for /usr/lib/libc.a(posix_aio.o) because:
318 Symbol _posix_kaio_rdwr (number 2) is not exported from dependent module /unix.
319
320you need to enable POSIX aio. Run the following commands as root:
321
322 # lsdev -C -l posix_aio0
323 posix_aio0 Defined Posix Asynchronous I/O
324 # cfgmgr -l posix_aio0
325 # lsdev -C -l posix_aio0
326 posix_aio0 Available Posix Asynchronous I/O
327
328POSIX aio should work now. To make the change permanent:
329
330 # chdev -l posix_aio0 -P -a autoconfig='available'
331 posix_aio0 changed
217bc04b
JA
332
333
79809113
JA
334Author
335------
336
aae22ca7 337Fio was written by Jens Axboe <axboe@kernel.dk> to enable flexible testing
79809113
JA
338of the Linux IO subsystem and schedulers. He got tired of writing
339specific test applications to simulate a given workload, and found that
340the existing io benchmark/test tools out there weren't flexible enough
341to do what he wanted.
342
aae22ca7 343Jens Axboe <axboe@kernel.dk> 20060905
79809113 344