Fix a few 4.8 extra anal warnings
[fio.git] / README
CommitLineData
ebac4655
JA
1fio
2---
3
79809113
JA
4fio is a tool that will spawn a number of threads or processes doing a
5particular type of io action as specified by the user. fio takes a
6number of global parameters, each inherited by the thread unless
7otherwise parameters given to them overriding that setting is given.
8The typical use of fio is to write a job file matching the io load
9one wants to simulate.
ebac4655 10
2b02b546
JA
11
12Source
13------
14
15fio resides in a git repo, the canonical place is:
16
6b3eccb1 17git://git.kernel.dk/fio.git
97f049c9 18
a9bac3f9
JA
19If you are inside a corporate firewall, git:// may not always work for
20you. In that case you can use the http protocol, path is the same:
21
22http://git.kernel.dk/fio.git
2b02b546 23
79809113
JA
24Snapshots are frequently generated and they include the git meta data as
25well. You can download them here:
2b02b546
JA
26
27http://brick.kernel.dk/snaps/
28
1053a106 29
d85b1add
SK
30Binary packages
31---------------
32
33Debian:
34Starting with Debian "Squeeze", fio packages are part of the official
35Debian repository. http://packages.debian.org/search?keywords=fio
36
37Ubuntu:
38Starting with Ubuntu 10.04 LTS (aka "Lucid Lynx"), fio packages are part
39of the Ubuntu "universe" repository.
40http://packages.ubuntu.com/search?keywords=fio
41
d85b1add 42Red Hat, CentOS & Co:
a68594cb 43Dag Wieërs has RPMs for Red Hat related distros, find them here:
a68594cb
JA
44http://dag.wieers.com/rpm/packages/fio/
45
d85b1add 46Mandriva:
244e170e
JA
47Mandriva has integrated fio into their package repository, so installing
48on that distro should be as easy as typing 'urpmi fio'.
49
d85b1add
SK
50Solaris:
51Packages for Solaris are available from OpenCSW. Install their pkgutil
52tool (http://www.opencsw.org/get-it/pkgutil/) and then install fio via
53'pkgutil -i fio'.
54
ecc314ba
BC
55Windows:
56Bruce Cran <bruce@cran.org.uk> has fio packages for Windows at
78080867 57http://www.bluestop.org/fio/ .
ecc314ba 58
2b02b546 59
726f6ff0
JA
60Mailing list
61------------
62
63There's a mailing list associated with fio. It's meant for general
2e8552b0
JA
64discussion, bug reporting, questions, and development - basically anything
65that has to do with fio. An automated mail detailing recent commits is
66automatically sent to the list at most daily. The list address is
67fio@vger.kernel.org, subscribe by sending an email to
68majordomo@vger.kernel.org with
69
70subscribe fio
71
4f5d1526
EIB
72in the body of the email. Archives can be found here:
73
74http://www.spinics.net/lists/fio/
75
76and archives for the old list can be found here:
2e8552b0
JA
77
78http://maillist.kernel.dk/fio-devel/
726f6ff0
JA
79
80
bbfd6b00
JA
81Building
82--------
83
d015e398 84Just type 'make' and 'make install'.
bbfd6b00 85
d015e398
BC
86Note that GNU make is required. On BSD it's available from devel/gmake;
87on Solaris it's in the SUNWgmake package. On platforms where GNU make
88isn't the default, type 'gmake' instead of 'make'.
bbfd6b00 89
6de43c1b
JA
90If your compile fails with an error like this:
91
92 CC gettime.o
93In file included from fio.h:23,
94 from gettime.c:8:
95os/os.h:15:20: error: libaio.h: No such file or directory
96In file included from gettime.c:8:
97fio.h:119: error: field 'iocb' has incomplete type
98make: *** [gettime.o] Error 1
99
100Check that you have the libaio development package installed. On RPM
101based distros, it's typically called libaio-devel.
102
bbfd6b00 103
53adf64f
BC
104Windows
105-------
106
f41862f7
BC
107On Windows Cygwin (http://www.cygwin.com/) is required in order to
108build fio. To create an MSI installer package install WiX 3.7 from
109http://wixtoolset.org and run dobuild.cmd from the
93bcfd20 110os/windows directory.
53adf64f 111
f41862f7
BC
112How to compile FIO on 64-bit Windows:
113
114 1. Install Cygwin (http://www.cygwin.com/setup.exe). Install 'make' and all
115 packages starting with 'mingw64-i686' and 'mingw64-x86_64'.
116 2. Download ftp://sourceware.org/pub/pthreads-win32/prebuilt-dll-2-9-1-release/dll/x64/pthreadGC2.dll
117 and copy to the fio source directory.
118 3. Open the Cygwin Terminal.
119 4. Go to the fio directory (source files).
120 5. Run 'make clean'.
121 6. Run 'make'.
444310ff 122
7409711b
HL
123To build fio on 32-bit Windows, download x86/pthreadGC2.dll instead and do
124'./configure --build-32bit-win=yes' before 'make'.
125
78080867
BC
126It's recommended that once built or installed, fio be run in a Command Prompt
127or other 'native' console such as console2, since there are known to be display
128and signal issues when running it under a Cygwin shell
129(see http://code.google.com/p/mintty/issues/detail?id=56 for details).
130
53adf64f 131
972cfd25
JA
132Command line
133------------
ebac4655
JA
134
135$ fio
1cfd036f 136 --debug Enable some debugging options (see below)
111e032d 137 --parse-only Parse options only, don't start any IO
1cfd036f 138 --output Write output to file
b2cecdc2 139 --runtime Runtime in seconds
bebe6398
JA
140 --latency-log Generate per-job latency logs
141 --bandwidth-log Generate per-job bandwidth logs
1cfd036f 142 --minimal Minimal (terse) output
f3afa57e 143 --output-format=type Output format (terse,json,normal)
3449ab8c 144 --terse-version=type Terse version output format (default 3, or 2 or 4).
f3afa57e 145 --version Print version info and exit
1cfd036f 146 --help Print this page
23893646 147 --cpuclock-test Perform test/validation of CPU clock
bebe6398 148 --cmdhelp=cmd Print command help, "all" for all of them
de890a1e
SL
149 --enghelp=engine Print ioengine help, or list available ioengines
150 --enghelp=engine,cmd Print help for an ioengine cmd
1cfd036f 151 --showcmd Turn a job file into command line options
ad0a2735 152 --readonly Turn on safety read-only checks, preventing
bebe6398 153 writes
1cfd036f 154 --eta=when When ETA estimate should be printed
bebe6398 155 May be "always", "never" or "auto"
e382e661 156 --eta-newline=time Force a new line for every 'time' period passed
bebe6398
JA
157 --section=name Only run specified section in job file.
158 Multiple sections can be specified.
e7cb819b 159 --alloc-size=kb Set smalloc pool to this size in kb (def 1024)
160 --warnings-fatal Fio parser warnings are fatal
fca70358 161 --max-jobs Maximum number of threads/processes to support
bebe6398
JA
162 --server=args Start backend server. See Client/Server section.
163 --client=host Connect to specified backend.
f2a2ce0e
HL
164 --idle-prof=option Report cpu idleness on a system or percpu basis
165 (option=system,percpu) or run unit work
166 calibration only (option=calibrate).
e592a06b 167
b4692828
JA
168
169Any parameters following the options will be assumed to be job files,
170unless they match a job file parameter. You can add as many as you want,
171each job file will be regarded as a separate group and fio will stonewall
172its execution.
972cfd25 173
ecc314ba 174The --readonly switch is an extra safety guard to prevent accidentally
724e4435
JA
175turning on a write setting when that is not desired. Fio will only write
176if rw=write/randwrite/rw/randrw is given, but this extra safety net can
177be used as an extra precaution. It will also enable a write check in the
178io engine core to prevent an accidental write due to a fio bug.
179
ee56ad50
JA
180The debug switch allows adding options that trigger certain logging
181options in fio. Currently the options are:
182
183 process Dump info related to processes
184 file Dump info related to file actions
e7cb819b 185 io Dump info related to IO queuing
186 mem Dump info related to memory allocations
bd6f78b2
JA
187 blktrace Dump info related to blktrace setup
188 verify Dump info related to IO verification
e7cb819b 189 all Enable all debug options
811a0d06 190 random Dump info related to random offset generation
a3d741fa 191 parse Dump info related to option matching and parsing
cd991b9e 192 diskutil Dump info related to disk utilization updates
5e1d306e 193 job:x Dump info only related to job number x
29adda3c 194 mutex Dump info only related to mutex up/down ops
c223da83
JA
195 profile Dump info related to profile extensions
196 time Dump info related to internal time keeping
bd6f78b2 197 ? or help Show available debug options.
ee56ad50
JA
198
199You can specify as many as you want, eg --debug=file,mem will enable
bd6f78b2 200file and memory debugging.
ee56ad50 201
01f06b63
JA
202The section switch is meant to make it easier to ship a bigger job file
203instead of several smaller ones. Say you define a job file with light,
204moderate, and heavy parts. Then you can ask fio to run the given part
205only by giving it a --section=heavy command line option. The section
206option only applies to job sections, the reserved 'global' section is
207always parsed and taken into account.
208
2b386d25
JA
209Fio has an internal allocator for shared memory called smalloc. It
210allocates shared structures from this pool. The pool defaults to 1024k
931823ca 211in size, and can grow to 128 pools. If running large jobs with randommap
2b386d25 212enabled it can run out of memory, in which case the --alloc-size switch
931823ca
JA
213is handy for starting with a larger pool size. The backing store is
214files in /tmp. Fio cleans up after itself, while it is running you
215may see .fio_smalloc.* files in /tmp.
2b386d25 216
79809113
JA
217
218Job file
219--------
220
71bfa161 221See the HOWTO file for a more detailed description of parameters and what
4661f3d0
JA
222they mean. This file contains the terse version. You can describe big and
223complex setups with the command line, but generally it's a lot easier to
71bfa161 224just write a simple job file to describe the workload. The job file format
4661f3d0 225is in the ini style format, as that is easy to read and write for the user.
79809113 226
8079cb46
JA
227The HOWTO or man page has a full list of all options, along with
228descriptions, etc. The --cmdhelp option also lists all options. If
229used with an option argument, it will detail that particular option.
79809113 230
217bc04b 231
bebe6398
JA
232Client/server
233------------
234
235Normally you would run fio as a stand-alone application on the machine
236where the IO workload should be generated. However, it is also possible to
237run the frontend and backend of fio separately. This makes it possible to
238have a fio server running on the machine(s) where the IO workload should
239be running, while controlling it from another machine.
240
241To start the server, you would do:
242
243fio --server=args
244
245on that machine, where args defines what fio listens to. The arguments
811826be
JA
246are of the form 'type,hostname or IP,port'. 'type' is either 'ip' (or ip4)
247for TCP/IP v4, 'ip6' for TCP/IP v6, or 'sock' for a local unix domain socket.
248'hostname' is either a hostname or IP address, and 'port' is the port to
249listen to (only valid for TCP/IP, not a local socket). Some examples:
bebe6398
JA
250
2511) fio --server
252
253 Start a fio server, listening on all interfaces on the default port (8765).
254
811826be 2552) fio --server=ip:hostname,4444
bebe6398
JA
256
257 Start a fio server, listening on IP belonging to hostname and on port 4444.
258
811826be
JA
2593) fio --server=ip6:::1,4444
260
261 Start a fio server, listening on IPv6 localhost ::1 and on port 4444.
262
2634) fio --server=,4444
bebe6398
JA
264
265 Start a fio server, listening on all interfaces on port 4444.
266
811826be 2675) fio --server=1.2.3.4
bebe6398
JA
268
269 Start a fio server, listening on IP 1.2.3.4 on the default port.
270
811826be 2716) fio --server=sock:/tmp/fio.sock
bebe6398
JA
272
273 Start a fio server, listening on the local socket /tmp/fio.sock.
274
275When a server is running, you can connect to it from a client. The client
276is run with:
277
278fio --local-args --client=server --remote-args <job file(s)>
279
280where --local-args are arguments that are local to the client where it is
281running, 'server' is the connect string, and --remote-args and <job file(s)>
282are sent to the server. The 'server' string follows the same format as it
283does on the server side, to allow IP/hostname/socket and port strings.
284You can connect to multiple clients as well, to do that you could run:
285
a7321eed 286fio --client=server2 <job file(s)> --client=server2 <job file(s)>
bebe6398
JA
287
288
217bc04b
JA
289Platforms
290---------
291
ce600ac9
JA
292Fio works on (at least) Linux, Solaris, AIX, HP-UX, OSX, NetBSD, Windows
293and FreeBSD. Some features and/or options may only be available on some of
294the platforms, typically because those features only apply to that platform
295(like the solarisaio engine, or the splice engine on Linux).
217bc04b
JA
296
297Some features are not available on FreeBSD/Solaris even if they could be
298implemented, I'd be happy to take patches for that. An example of that is
299disk utility statistics and (I think) huge page support, support for that
300does exist in FreeBSD/Solaris.
301
302Fio uses pthread mutexes for signalling and locking and FreeBSD does not
303support process shared pthread mutexes. As a result, only threads are
304supported on FreeBSD. This could be fixed with sysv ipc locking or
305other locking alternatives.
306
307Other *BSD platforms are untested, but fio should work there almost out
308of the box. Since I don't do test runs or even compiles on those platforms,
309your mileage may vary. Sending me patches for other platforms is greatly
310appreciated. There's a lot of value in having the same test/benchmark tool
311available on all platforms.
312
bf2e821a
CC
313Note that POSIX aio is not enabled by default on AIX. If you get messages like:
314
315 Symbol resolution failed for /usr/lib/libc.a(posix_aio.o) because:
316 Symbol _posix_kaio_rdwr (number 2) is not exported from dependent module /unix.
317
318you need to enable POSIX aio. Run the following commands as root:
319
320 # lsdev -C -l posix_aio0
321 posix_aio0 Defined Posix Asynchronous I/O
322 # cfgmgr -l posix_aio0
323 # lsdev -C -l posix_aio0
324 posix_aio0 Available Posix Asynchronous I/O
325
326POSIX aio should work now. To make the change permanent:
327
328 # chdev -l posix_aio0 -P -a autoconfig='available'
329 posix_aio0 changed
217bc04b
JA
330
331
79809113
JA
332Author
333------
334
aae22ca7 335Fio was written by Jens Axboe <axboe@kernel.dk> to enable flexible testing
79809113
JA
336of the Linux IO subsystem and schedulers. He got tired of writing
337specific test applications to simulate a given workload, and found that
338the existing io benchmark/test tools out there weren't flexible enough
339to do what he wanted.
340
aae22ca7 341Jens Axboe <axboe@kernel.dk> 20060905
79809113 342