add .rst suffix to README and HOWTO
[fio.git] / README.rst
CommitLineData
f80dba8d
MT
1Overview and history
2--------------------
3
4Fio was originally written to save me the hassle of writing special test case
5programs when I wanted to test a specific workload, either for performance
6reasons or to find/reproduce a bug. The process of writing such a test app can
7be tiresome, especially if you have to do it often. Hence I needed a tool that
8would be able to simulate a given I/O workload without resorting to writing a
9tailored test case again and again.
10
11A test work load is difficult to define, though. There can be any number of
12processes or threads involved, and they can each be using their own way of
13generating I/O. You could have someone dirtying large amounts of memory in an
14memory mapped file, or maybe several threads issuing reads using asynchronous
15I/O. fio needed to be flexible enough to simulate both of these cases, and many
16more.
17
18Fio spawns a number of threads or processes doing a particular type of I/O
19action as specified by the user. fio takes a number of global parameters, each
20inherited by the thread unless otherwise parameters given to them overriding
21that setting is given. The typical use of fio is to write a job file matching
22the I/O load one wants to simulate.
ebac4655 23
2b02b546
JA
24
25Source
26------
27
f80dba8d 28Fio resides in a git repo, the canonical place is:
2b02b546 29
4649b352 30 git://git.kernel.dk/fio.git
97f049c9 31
4649b352
GG
32When inside a corporate firewall, git:// URL sometimes does not work.
33If git:// does not work, use the http protocol instead:
a9bac3f9 34
4649b352 35 http://git.kernel.dk/fio.git
2b02b546 36
f80dba8d 37Snapshots are frequently generated and :file:`fio-git-*.tar.gz` include the git
88d398d0 38meta data as well. Other tarballs are archives of official fio releases.
4649b352 39Snapshots can download from:
2b02b546 40
4649b352 41 http://brick.kernel.dk/snaps/
2b02b546 42
f80dba8d
MT
43There are also two official mirrors. Both of these are automatically synced with
44the main repository, when changes are pushed. If the main repo is down for some
45reason, either one of these is safe to use as a backup:
01fa84d5
JA
46
47 git://git.kernel.org/pub/scm/linux/kernel/git/axboe/fio.git
f80dba8d 48
01fa84d5
JA
49 https://git.kernel.org/pub/scm/linux/kernel/git/axboe/fio.git
50
51or
52
3826b24a 53 git://github.com/axboe/fio.git
244e170e 54
f80dba8d 55 https://github.com/axboe/fio.git
a71844ff 56
2b02b546 57
726f6ff0
JA
58Mailing list
59------------
60
4649b352
GG
61The fio project mailing list is meant for anything related to fio including
62general discussion, bug reporting, questions, and development.
2e8552b0 63
f80dba8d
MT
64An automated mail detailing recent commits is automatically sent to the list at
65most daily. The list address is fio@vger.kernel.org, subscribe by sending an
66email to majordomo@vger.kernel.org with
4649b352
GG
67
68 subscribe fio
2e8552b0 69
4f5d1526
EIB
70in the body of the email. Archives can be found here:
71
4649b352 72 http://www.spinics.net/lists/fio/
4f5d1526
EIB
73
74and archives for the old list can be found here:
2e8552b0 75
4649b352 76 http://maillist.kernel.dk/fio-devel/
726f6ff0
JA
77
78
f80dba8d
MT
79Author
80------
bbfd6b00 81
f80dba8d
MT
82Fio was written by Jens Axboe <axboe@kernel.dk> to enable flexible testing of
83the Linux I/O subsystem and schedulers. He got tired of writing specific test
84applications to simulate a given workload, and found that the existing I/O
85benchmark/test tools out there weren't flexible enough to do what he wanted.
6de43c1b 86
f80dba8d 87Jens Axboe <axboe@kernel.dk> 20060905
2382dca7 88
c8931876 89
f80dba8d
MT
90Binary packages
91---------------
bbfd6b00 92
f80dba8d
MT
93Debian:
94 Starting with Debian "Squeeze", fio packages are part of the official
95 Debian repository. http://packages.debian.org/search?keywords=fio.
53adf64f 96
f80dba8d
MT
97Ubuntu:
98 Starting with Ubuntu 10.04 LTS (aka "Lucid Lynx"), fio packages are part
99 of the Ubuntu "universe" repository.
100 http://packages.ubuntu.com/search?keywords=fio.
53adf64f 101
f80dba8d
MT
102Red Hat, CentOS & Co:
103 Dag Wieërs has RPMs for Red Hat related distros, find them here:
104 http://dag.wieers.com/rpm/packages/fio/.
f41862f7 105
f80dba8d
MT
106Mandriva:
107 Mandriva has integrated fio into their package repository, so installing
108 on that distro should be as easy as typing ``urpmi fio``.
9aa5fe32 109
f80dba8d
MT
110Solaris:
111 Packages for Solaris are available from OpenCSW. Install their pkgutil
112 tool (http://www.opencsw.org/get-it/pkgutil/) and then install fio via
113 ``pkgutil -i fio``.
7409711b 114
f80dba8d
MT
115Windows:
116 Rebecca Cran <rebecca+fio@bluestop.org> has fio packages for Windows at
117 http://www.bluestop.org/fio/ .
78080867 118
f80dba8d
MT
119BSDs:
120 Packages for BSDs may be available from their binary package repositories.
121 Look for a package "fio" using their binary package managers.
53adf64f 122
ebac4655 123
f80dba8d 124Building
79809113
JA
125--------
126
f80dba8d 127Just type::
bebe6398 128
f80dba8d
MT
129 $ ./configure
130 $ make
131 $ make install
bebe6398 132
f80dba8d
MT
133Note that GNU make is required. On BSDs it's available from devel/gmake within
134ports directory; on Solaris it's in the SUNWgmake package. On platforms where
135GNU make isn't the default, type ``gmake`` instead of ``make``.
811826be 136
f80dba8d
MT
137Configure will print the enabled options. Note that on Linux based platforms,
138the libaio development packages must be installed to use the libaio
139engine. Depending on distro, it is usually called libaio-devel or libaio-dev.
811826be 140
f80dba8d
MT
141For gfio, gtk 2.18 (or newer), associated glib threads, and cairo are required
142to be installed. gfio isn't built automatically and can be enabled with a
143``--enable-gfio`` option to configure.
bebe6398 144
f80dba8d 145To build fio with a cross-compiler::
bebe6398 146
f80dba8d
MT
147 $ make clean
148 $ make CROSS_COMPILE=/path/to/toolchain/prefix
bebe6398 149
f80dba8d 150Configure will attempt to determine the target platform automatically.
bebe6398 151
f80dba8d
MT
152It's possible to build fio for ESX as well, use the ``--esx`` switch to
153configure.
4649b352 154
bebe6398 155
f80dba8d
MT
156Windows
157~~~~~~~
323255cc 158
f80dba8d
MT
159On Windows, Cygwin (http://www.cygwin.com/) is required in order to build
160fio. To create an MSI installer package install WiX 3.8 from
161http://wixtoolset.org and run :file:`dobuild.cmd` from the :file:`os/windows`
162directory.
323255cc 163
f80dba8d 164How to compile fio on 64-bit Windows:
323255cc 165
f80dba8d
MT
166 1. Install Cygwin (http://www.cygwin.com/). Install **make** and all
167 packages starting with **mingw64-i686** and **mingw64-x86_64**.
168 2. Open the Cygwin Terminal.
169 3. Go to the fio directory (source files).
170 4. Run ``make clean && make -j``.
39b5f61e 171
f80dba8d
MT
172To build fio on 32-bit Windows, run ``./configure --build-32bit-win`` before
173``make``.
39b5f61e 174
f80dba8d
MT
175It's recommended that once built or installed, fio be run in a Command Prompt or
176other 'native' console such as console2, since there are known to be display and
177signal issues when running it under a Cygwin shell (see
178http://code.google.com/p/mintty/issues/detail?id=56 for details).
39b5f61e 179
39b5f61e 180
f80dba8d
MT
181Documentation
182~~~~~~~~~~~~~
39b5f61e 183
f80dba8d
MT
184Fio uses Sphinx_ to generate documentation from the reStructuredText_ files.
185To build HTML formatted documentation run ``make -C doc html`` and direct your
186browser to :file:`./doc/output/html/index.html`. To build manual page run
187``make -C doc man`` and then ``man doc/output/man/fio.1``. To see what other
188output formats are supported run ``make -C doc help``.
39b5f61e 189
f80dba8d
MT
190.. _reStructuredText: http://www.sphinx-doc.org/rest.html
191.. _Sphinx: http://www.sphinx-doc.org
39b5f61e 192
bebe6398 193
217bc04b
JA
194Platforms
195---------
196
1b8c5af7 197Fio works on (at least) Linux, Solaris, AIX, HP-UX, OSX, NetBSD, OpenBSD,
2af5bd99 198Windows, FreeBSD, and DragonFly. Some features and/or options may only be
f80dba8d
MT
199available on some of the platforms, typically because those features only apply
200to that platform (like the solarisaio engine, or the splice engine on Linux).
217bc04b
JA
201
202Some features are not available on FreeBSD/Solaris even if they could be
f80dba8d
MT
203implemented, I'd be happy to take patches for that. An example of that is disk
204utility statistics and (I think) huge page support, support for that does exist
205in FreeBSD/Solaris.
217bc04b
JA
206
207Fio uses pthread mutexes for signalling and locking and FreeBSD does not
208support process shared pthread mutexes. As a result, only threads are
209supported on FreeBSD. This could be fixed with sysv ipc locking or
210other locking alternatives.
211
f80dba8d
MT
212Other \*BSD platforms are untested, but fio should work there almost out of the
213box. Since I don't do test runs or even compiles on those platforms, your
214mileage may vary. Sending me patches for other platforms is greatly
217bc04b
JA
215appreciated. There's a lot of value in having the same test/benchmark tool
216available on all platforms.
217
f80dba8d 218Note that POSIX aio is not enabled by default on AIX. Messages like these::
bf2e821a
CC
219
220 Symbol resolution failed for /usr/lib/libc.a(posix_aio.o) because:
221 Symbol _posix_kaio_rdwr (number 2) is not exported from dependent module /unix.
222
f80dba8d 223indicate one needs to enable POSIX aio. Run the following commands as root::
bf2e821a
CC
224
225 # lsdev -C -l posix_aio0
226 posix_aio0 Defined Posix Asynchronous I/O
227 # cfgmgr -l posix_aio0
228 # lsdev -C -l posix_aio0
229 posix_aio0 Available Posix Asynchronous I/O
230
f80dba8d 231POSIX aio should work now. To make the change permanent::
bf2e821a
CC
232
233 # chdev -l posix_aio0 -P -a autoconfig='available'
234 posix_aio0 changed
217bc04b
JA
235
236
f80dba8d
MT
237Running fio
238-----------
79809113 239
f80dba8d
MT
240Running fio is normally the easiest part - you just give it the job file
241(or job files) as parameters::
79809113 242
f80dba8d
MT
243 $ fio [options] [jobfile] ...
244
245and it will start doing what the *jobfile* tells it to do. You can give more
246than one job file on the command line, fio will serialize the running of those
247files. Internally that is the same as using the :option:`stonewall` parameter
248described in the parameter section.
249
250If the job file contains only one job, you may as well just give the parameters
251on the command line. The command line parameters are identical to the job
252parameters, with a few extra that control global parameters. For example, for
253the job file parameter :option:`iodepth=2 <iodepth>`, the mirror command line
254option would be :option:`--iodepth 2 <iodepth>` or :option:`--iodepth=2
255<iodepth>`. You can also use the command line for giving more than one job
256entry. For each :option:`--name <name>` option that fio sees, it will start a
257new job with that name. Command line entries following a
258:option:`--name <name>` entry will apply to that job, until there are no more
259entries or a new :option:`--name <name>` entry is seen. This is similar to the
260job file options, where each option applies to the current job until a new []
261job entry is seen.
262
263fio does not need to run as root, except if the files or devices specified in
264the job section requires that. Some other options may also be restricted, such
265as memory locking, I/O scheduler switching, and decreasing the nice value.
79809113 266
f80dba8d
MT
267If *jobfile* is specified as ``-``, the job file will be read from standard
268input.