Fio 2.21
[fio.git] / README
CommitLineData
f80dba8d
MT
1Overview and history
2--------------------
3
4Fio was originally written to save me the hassle of writing special test case
5programs when I wanted to test a specific workload, either for performance
6reasons or to find/reproduce a bug. The process of writing such a test app can
7be tiresome, especially if you have to do it often. Hence I needed a tool that
8would be able to simulate a given I/O workload without resorting to writing a
9tailored test case again and again.
10
11A test work load is difficult to define, though. There can be any number of
12processes or threads involved, and they can each be using their own way of
13generating I/O. You could have someone dirtying large amounts of memory in an
14memory mapped file, or maybe several threads issuing reads using asynchronous
15I/O. fio needed to be flexible enough to simulate both of these cases, and many
16more.
17
18Fio spawns a number of threads or processes doing a particular type of I/O
19action as specified by the user. fio takes a number of global parameters, each
20inherited by the thread unless otherwise parameters given to them overriding
21that setting is given. The typical use of fio is to write a job file matching
22the I/O load one wants to simulate.
ebac4655 23
2b02b546
JA
24
25Source
26------
27
f80dba8d 28Fio resides in a git repo, the canonical place is:
2b02b546 29
4649b352 30 git://git.kernel.dk/fio.git
97f049c9 31
4649b352
GG
32When inside a corporate firewall, git:// URL sometimes does not work.
33If git:// does not work, use the http protocol instead:
a9bac3f9 34
4649b352 35 http://git.kernel.dk/fio.git
2b02b546 36
f80dba8d 37Snapshots are frequently generated and :file:`fio-git-*.tar.gz` include the git
88d398d0 38meta data as well. Other tarballs are archives of official fio releases.
4649b352 39Snapshots can download from:
2b02b546 40
4649b352 41 http://brick.kernel.dk/snaps/
2b02b546 42
f80dba8d
MT
43There are also two official mirrors. Both of these are automatically synced with
44the main repository, when changes are pushed. If the main repo is down for some
45reason, either one of these is safe to use as a backup:
01fa84d5
JA
46
47 git://git.kernel.org/pub/scm/linux/kernel/git/axboe/fio.git
f80dba8d 48
01fa84d5
JA
49 https://git.kernel.org/pub/scm/linux/kernel/git/axboe/fio.git
50
51or
52
3826b24a 53 git://github.com/axboe/fio.git
244e170e 54
f80dba8d 55 https://github.com/axboe/fio.git
a71844ff 56
2b02b546 57
726f6ff0
JA
58Mailing list
59------------
60
4649b352
GG
61The fio project mailing list is meant for anything related to fio including
62general discussion, bug reporting, questions, and development.
2e8552b0 63
f80dba8d
MT
64An automated mail detailing recent commits is automatically sent to the list at
65most daily. The list address is fio@vger.kernel.org, subscribe by sending an
66email to majordomo@vger.kernel.org with
4649b352
GG
67
68 subscribe fio
2e8552b0 69
4f5d1526
EIB
70in the body of the email. Archives can be found here:
71
4649b352 72 http://www.spinics.net/lists/fio/
4f5d1526
EIB
73
74and archives for the old list can be found here:
2e8552b0 75
4649b352 76 http://maillist.kernel.dk/fio-devel/
726f6ff0
JA
77
78
f80dba8d
MT
79Author
80------
bbfd6b00 81
f80dba8d
MT
82Fio was written by Jens Axboe <axboe@kernel.dk> to enable flexible testing of
83the Linux I/O subsystem and schedulers. He got tired of writing specific test
84applications to simulate a given workload, and found that the existing I/O
85benchmark/test tools out there weren't flexible enough to do what he wanted.
6de43c1b 86
f80dba8d 87Jens Axboe <axboe@kernel.dk> 20060905
2382dca7 88
c8931876 89
f80dba8d
MT
90Binary packages
91---------------
bbfd6b00 92
f80dba8d
MT
93Debian:
94 Starting with Debian "Squeeze", fio packages are part of the official
a881438b 95 Debian repository. http://packages.debian.org/search?keywords=fio .
53adf64f 96
f80dba8d
MT
97Ubuntu:
98 Starting with Ubuntu 10.04 LTS (aka "Lucid Lynx"), fio packages are part
99 of the Ubuntu "universe" repository.
a881438b 100 http://packages.ubuntu.com/search?keywords=fio .
53adf64f 101
a881438b
SW
102Red Hat, Fedora, CentOS & Co:
103 Starting with Fedora 9/Extra Packages for Enterprise Linux 4, fio
104 packages are part of the Fedora/EPEL repositories.
105 https://admin.fedoraproject.org/pkgdb/package/rpms/fio/ .
f41862f7 106
f80dba8d
MT
107Mandriva:
108 Mandriva has integrated fio into their package repository, so installing
109 on that distro should be as easy as typing ``urpmi fio``.
9aa5fe32 110
f80dba8d
MT
111Solaris:
112 Packages for Solaris are available from OpenCSW. Install their pkgutil
113 tool (http://www.opencsw.org/get-it/pkgutil/) and then install fio via
114 ``pkgutil -i fio``.
7409711b 115
f80dba8d
MT
116Windows:
117 Rebecca Cran <rebecca+fio@bluestop.org> has fio packages for Windows at
118 http://www.bluestop.org/fio/ .
78080867 119
f80dba8d
MT
120BSDs:
121 Packages for BSDs may be available from their binary package repositories.
122 Look for a package "fio" using their binary package managers.
53adf64f 123
ebac4655 124
f80dba8d 125Building
79809113
JA
126--------
127
f80dba8d 128Just type::
bebe6398 129
f80dba8d
MT
130 $ ./configure
131 $ make
132 $ make install
bebe6398 133
f80dba8d
MT
134Note that GNU make is required. On BSDs it's available from devel/gmake within
135ports directory; on Solaris it's in the SUNWgmake package. On platforms where
136GNU make isn't the default, type ``gmake`` instead of ``make``.
811826be 137
f80dba8d
MT
138Configure will print the enabled options. Note that on Linux based platforms,
139the libaio development packages must be installed to use the libaio
140engine. Depending on distro, it is usually called libaio-devel or libaio-dev.
811826be 141
f80dba8d
MT
142For gfio, gtk 2.18 (or newer), associated glib threads, and cairo are required
143to be installed. gfio isn't built automatically and can be enabled with a
144``--enable-gfio`` option to configure.
bebe6398 145
f80dba8d 146To build fio with a cross-compiler::
bebe6398 147
f80dba8d
MT
148 $ make clean
149 $ make CROSS_COMPILE=/path/to/toolchain/prefix
bebe6398 150
f80dba8d 151Configure will attempt to determine the target platform automatically.
bebe6398 152
f80dba8d
MT
153It's possible to build fio for ESX as well, use the ``--esx`` switch to
154configure.
4649b352 155
bebe6398 156
f80dba8d
MT
157Windows
158~~~~~~~
323255cc 159
f80dba8d
MT
160On Windows, Cygwin (http://www.cygwin.com/) is required in order to build
161fio. To create an MSI installer package install WiX 3.8 from
162http://wixtoolset.org and run :file:`dobuild.cmd` from the :file:`os/windows`
163directory.
323255cc 164
f80dba8d 165How to compile fio on 64-bit Windows:
323255cc 166
f80dba8d
MT
167 1. Install Cygwin (http://www.cygwin.com/). Install **make** and all
168 packages starting with **mingw64-i686** and **mingw64-x86_64**.
169 2. Open the Cygwin Terminal.
170 3. Go to the fio directory (source files).
171 4. Run ``make clean && make -j``.
39b5f61e 172
f80dba8d
MT
173To build fio on 32-bit Windows, run ``./configure --build-32bit-win`` before
174``make``.
39b5f61e 175
f80dba8d
MT
176It's recommended that once built or installed, fio be run in a Command Prompt or
177other 'native' console such as console2, since there are known to be display and
178signal issues when running it under a Cygwin shell (see
179http://code.google.com/p/mintty/issues/detail?id=56 for details).
39b5f61e 180
39b5f61e 181
f80dba8d
MT
182Documentation
183~~~~~~~~~~~~~
39b5f61e 184
f80dba8d
MT
185Fio uses Sphinx_ to generate documentation from the reStructuredText_ files.
186To build HTML formatted documentation run ``make -C doc html`` and direct your
187browser to :file:`./doc/output/html/index.html`. To build manual page run
188``make -C doc man`` and then ``man doc/output/man/fio.1``. To see what other
189output formats are supported run ``make -C doc help``.
39b5f61e 190
f80dba8d
MT
191.. _reStructuredText: http://www.sphinx-doc.org/rest.html
192.. _Sphinx: http://www.sphinx-doc.org
39b5f61e 193
bebe6398 194
217bc04b
JA
195Platforms
196---------
197
1b8c5af7 198Fio works on (at least) Linux, Solaris, AIX, HP-UX, OSX, NetBSD, OpenBSD,
2af5bd99 199Windows, FreeBSD, and DragonFly. Some features and/or options may only be
f80dba8d
MT
200available on some of the platforms, typically because those features only apply
201to that platform (like the solarisaio engine, or the splice engine on Linux).
217bc04b
JA
202
203Some features are not available on FreeBSD/Solaris even if they could be
f80dba8d
MT
204implemented, I'd be happy to take patches for that. An example of that is disk
205utility statistics and (I think) huge page support, support for that does exist
206in FreeBSD/Solaris.
217bc04b 207
06eac6b2
SW
208Fio uses pthread mutexes for signalling and locking and some platforms do not
209support process shared pthread mutexes. As a result, on such platforms only
210threads are supported. This could be fixed with sysv ipc locking or other
211locking alternatives.
217bc04b 212
f80dba8d
MT
213Other \*BSD platforms are untested, but fio should work there almost out of the
214box. Since I don't do test runs or even compiles on those platforms, your
215mileage may vary. Sending me patches for other platforms is greatly
217bc04b
JA
216appreciated. There's a lot of value in having the same test/benchmark tool
217available on all platforms.
218
f80dba8d 219Note that POSIX aio is not enabled by default on AIX. Messages like these::
bf2e821a
CC
220
221 Symbol resolution failed for /usr/lib/libc.a(posix_aio.o) because:
222 Symbol _posix_kaio_rdwr (number 2) is not exported from dependent module /unix.
223
f80dba8d 224indicate one needs to enable POSIX aio. Run the following commands as root::
bf2e821a
CC
225
226 # lsdev -C -l posix_aio0
227 posix_aio0 Defined Posix Asynchronous I/O
228 # cfgmgr -l posix_aio0
229 # lsdev -C -l posix_aio0
230 posix_aio0 Available Posix Asynchronous I/O
231
f80dba8d 232POSIX aio should work now. To make the change permanent::
bf2e821a
CC
233
234 # chdev -l posix_aio0 -P -a autoconfig='available'
235 posix_aio0 changed
217bc04b
JA
236
237
f80dba8d
MT
238Running fio
239-----------
79809113 240
f80dba8d
MT
241Running fio is normally the easiest part - you just give it the job file
242(or job files) as parameters::
79809113 243
f80dba8d
MT
244 $ fio [options] [jobfile] ...
245
246and it will start doing what the *jobfile* tells it to do. You can give more
247than one job file on the command line, fio will serialize the running of those
248files. Internally that is the same as using the :option:`stonewall` parameter
249described in the parameter section.
250
251If the job file contains only one job, you may as well just give the parameters
252on the command line. The command line parameters are identical to the job
253parameters, with a few extra that control global parameters. For example, for
254the job file parameter :option:`iodepth=2 <iodepth>`, the mirror command line
255option would be :option:`--iodepth 2 <iodepth>` or :option:`--iodepth=2
256<iodepth>`. You can also use the command line for giving more than one job
257entry. For each :option:`--name <name>` option that fio sees, it will start a
258new job with that name. Command line entries following a
259:option:`--name <name>` entry will apply to that job, until there are no more
260entries or a new :option:`--name <name>` entry is seen. This is similar to the
261job file options, where each option applies to the current job until a new []
262job entry is seen.
263
264fio does not need to run as root, except if the files or devices specified in
265the job section requires that. Some other options may also be restricted, such
266as memory locking, I/O scheduler switching, and decreasing the nice value.
79809113 267
f80dba8d
MT
268If *jobfile* is specified as ``-``, the job file will be read from standard
269input.