Merge branch 'wip-rbd-engine' of https://github.com/dillaman/fio
[fio.git] / README
CommitLineData
f80dba8d
MT
1Overview and history
2--------------------
3
4Fio was originally written to save me the hassle of writing special test case
5programs when I wanted to test a specific workload, either for performance
6reasons or to find/reproduce a bug. The process of writing such a test app can
7be tiresome, especially if you have to do it often. Hence I needed a tool that
8would be able to simulate a given I/O workload without resorting to writing a
9tailored test case again and again.
10
11A test work load is difficult to define, though. There can be any number of
12processes or threads involved, and they can each be using their own way of
13generating I/O. You could have someone dirtying large amounts of memory in an
14memory mapped file, or maybe several threads issuing reads using asynchronous
15I/O. fio needed to be flexible enough to simulate both of these cases, and many
16more.
17
18Fio spawns a number of threads or processes doing a particular type of I/O
19action as specified by the user. fio takes a number of global parameters, each
20inherited by the thread unless otherwise parameters given to them overriding
21that setting is given. The typical use of fio is to write a job file matching
22the I/O load one wants to simulate.
ebac4655 23
2b02b546
JA
24
25Source
26------
27
f80dba8d 28Fio resides in a git repo, the canonical place is:
2b02b546 29
4649b352 30 git://git.kernel.dk/fio.git
97f049c9 31
4649b352
GG
32When inside a corporate firewall, git:// URL sometimes does not work.
33If git:// does not work, use the http protocol instead:
a9bac3f9 34
4649b352 35 http://git.kernel.dk/fio.git
2b02b546 36
f80dba8d 37Snapshots are frequently generated and :file:`fio-git-*.tar.gz` include the git
88d398d0 38meta data as well. Other tarballs are archives of official fio releases.
4649b352 39Snapshots can download from:
2b02b546 40
4649b352 41 http://brick.kernel.dk/snaps/
2b02b546 42
f80dba8d
MT
43There are also two official mirrors. Both of these are automatically synced with
44the main repository, when changes are pushed. If the main repo is down for some
45reason, either one of these is safe to use as a backup:
01fa84d5
JA
46
47 git://git.kernel.org/pub/scm/linux/kernel/git/axboe/fio.git
f80dba8d 48
01fa84d5
JA
49 https://git.kernel.org/pub/scm/linux/kernel/git/axboe/fio.git
50
51or
52
3826b24a 53 git://github.com/axboe/fio.git
244e170e 54
f80dba8d 55 https://github.com/axboe/fio.git
a71844ff 56
2b02b546 57
726f6ff0
JA
58Mailing list
59------------
60
4649b352 61The fio project mailing list is meant for anything related to fio including
6468020d
TK
62general discussion, bug reporting, questions, and development. For bug reporting,
63see REPORTING-BUGS.
2e8552b0 64
f80dba8d
MT
65An automated mail detailing recent commits is automatically sent to the list at
66most daily. The list address is fio@vger.kernel.org, subscribe by sending an
67email to majordomo@vger.kernel.org with
4649b352
GG
68
69 subscribe fio
2e8552b0 70
4f5d1526
EIB
71in the body of the email. Archives can be found here:
72
4649b352 73 http://www.spinics.net/lists/fio/
4f5d1526
EIB
74
75and archives for the old list can be found here:
2e8552b0 76
4649b352 77 http://maillist.kernel.dk/fio-devel/
726f6ff0
JA
78
79
f80dba8d
MT
80Author
81------
bbfd6b00 82
f80dba8d
MT
83Fio was written by Jens Axboe <axboe@kernel.dk> to enable flexible testing of
84the Linux I/O subsystem and schedulers. He got tired of writing specific test
85applications to simulate a given workload, and found that the existing I/O
86benchmark/test tools out there weren't flexible enough to do what he wanted.
6de43c1b 87
f80dba8d 88Jens Axboe <axboe@kernel.dk> 20060905
2382dca7 89
c8931876 90
f80dba8d
MT
91Binary packages
92---------------
bbfd6b00 93
f80dba8d
MT
94Debian:
95 Starting with Debian "Squeeze", fio packages are part of the official
a881438b 96 Debian repository. http://packages.debian.org/search?keywords=fio .
53adf64f 97
f80dba8d
MT
98Ubuntu:
99 Starting with Ubuntu 10.04 LTS (aka "Lucid Lynx"), fio packages are part
100 of the Ubuntu "universe" repository.
a881438b 101 http://packages.ubuntu.com/search?keywords=fio .
53adf64f 102
a881438b
SW
103Red Hat, Fedora, CentOS & Co:
104 Starting with Fedora 9/Extra Packages for Enterprise Linux 4, fio
105 packages are part of the Fedora/EPEL repositories.
658f3525 106 https://apps.fedoraproject.org/packages/fio .
f41862f7 107
f80dba8d
MT
108Mandriva:
109 Mandriva has integrated fio into their package repository, so installing
110 on that distro should be as easy as typing ``urpmi fio``.
9aa5fe32 111
a6415d76
QB
112Arch Linux:
113 An Arch Linux package is provided under the Community sub-repository:
114 https://www.archlinux.org/packages/?sort=&q=fio
115
f80dba8d
MT
116Solaris:
117 Packages for Solaris are available from OpenCSW. Install their pkgutil
118 tool (http://www.opencsw.org/get-it/pkgutil/) and then install fio via
119 ``pkgutil -i fio``.
7409711b 120
f80dba8d
MT
121Windows:
122 Rebecca Cran <rebecca+fio@bluestop.org> has fio packages for Windows at
eae92fbf
JA
123 https://www.bluestop.org/fio/ . The latest builds for Windows can also
124 be grabbed from https://ci.appveyor.com/project/axboe/fio by clicking
125 the latest x86 or x64 build, then selecting the ARTIFACTS tab.
78080867 126
f80dba8d
MT
127BSDs:
128 Packages for BSDs may be available from their binary package repositories.
129 Look for a package "fio" using their binary package managers.
53adf64f 130
ebac4655 131
f80dba8d 132Building
79809113
JA
133--------
134
f80dba8d 135Just type::
bebe6398 136
f80dba8d
MT
137 $ ./configure
138 $ make
139 $ make install
bebe6398 140
f80dba8d
MT
141Note that GNU make is required. On BSDs it's available from devel/gmake within
142ports directory; on Solaris it's in the SUNWgmake package. On platforms where
143GNU make isn't the default, type ``gmake`` instead of ``make``.
811826be 144
f80dba8d
MT
145Configure will print the enabled options. Note that on Linux based platforms,
146the libaio development packages must be installed to use the libaio
147engine. Depending on distro, it is usually called libaio-devel or libaio-dev.
811826be 148
f80dba8d
MT
149For gfio, gtk 2.18 (or newer), associated glib threads, and cairo are required
150to be installed. gfio isn't built automatically and can be enabled with a
151``--enable-gfio`` option to configure.
bebe6398 152
f80dba8d 153To build fio with a cross-compiler::
bebe6398 154
f80dba8d
MT
155 $ make clean
156 $ make CROSS_COMPILE=/path/to/toolchain/prefix
bebe6398 157
f80dba8d 158Configure will attempt to determine the target platform automatically.
bebe6398 159
f80dba8d
MT
160It's possible to build fio for ESX as well, use the ``--esx`` switch to
161configure.
4649b352 162
bebe6398 163
f80dba8d
MT
164Windows
165~~~~~~~
323255cc 166
f80dba8d
MT
167On Windows, Cygwin (http://www.cygwin.com/) is required in order to build
168fio. To create an MSI installer package install WiX 3.8 from
169http://wixtoolset.org and run :file:`dobuild.cmd` from the :file:`os/windows`
170directory.
323255cc 171
f80dba8d 172How to compile fio on 64-bit Windows:
323255cc 173
f80dba8d 174 1. Install Cygwin (http://www.cygwin.com/). Install **make** and all
b86770c6
SW
175 packages starting with **mingw64-i686** and **mingw64-x86_64**. Ensure
176 **mingw64-i686-zlib** and **mingw64-x86_64-zlib** are installed if you wish
177 to enable fio's log compression functionality.
f80dba8d
MT
178 2. Open the Cygwin Terminal.
179 3. Go to the fio directory (source files).
180 4. Run ``make clean && make -j``.
39b5f61e 181
f80dba8d
MT
182To build fio on 32-bit Windows, run ``./configure --build-32bit-win`` before
183``make``.
39b5f61e 184
f80dba8d
MT
185It's recommended that once built or installed, fio be run in a Command Prompt or
186other 'native' console such as console2, since there are known to be display and
187signal issues when running it under a Cygwin shell (see
f5c5f869
SW
188https://github.com/mintty/mintty/issues/56 and
189https://github.com/mintty/mintty/wiki/Tips#inputoutput-interaction-with-alien-programs
190for details).
39b5f61e 191
39b5f61e 192
f80dba8d
MT
193Documentation
194~~~~~~~~~~~~~
39b5f61e 195
f80dba8d
MT
196Fio uses Sphinx_ to generate documentation from the reStructuredText_ files.
197To build HTML formatted documentation run ``make -C doc html`` and direct your
198browser to :file:`./doc/output/html/index.html`. To build manual page run
199``make -C doc man`` and then ``man doc/output/man/fio.1``. To see what other
200output formats are supported run ``make -C doc help``.
39b5f61e 201
f80dba8d
MT
202.. _reStructuredText: http://www.sphinx-doc.org/rest.html
203.. _Sphinx: http://www.sphinx-doc.org
39b5f61e 204
bebe6398 205
217bc04b
JA
206Platforms
207---------
208
1b8c5af7 209Fio works on (at least) Linux, Solaris, AIX, HP-UX, OSX, NetBSD, OpenBSD,
2af5bd99 210Windows, FreeBSD, and DragonFly. Some features and/or options may only be
f80dba8d
MT
211available on some of the platforms, typically because those features only apply
212to that platform (like the solarisaio engine, or the splice engine on Linux).
217bc04b
JA
213
214Some features are not available on FreeBSD/Solaris even if they could be
f80dba8d
MT
215implemented, I'd be happy to take patches for that. An example of that is disk
216utility statistics and (I think) huge page support, support for that does exist
217in FreeBSD/Solaris.
217bc04b 218
06eac6b2
SW
219Fio uses pthread mutexes for signalling and locking and some platforms do not
220support process shared pthread mutexes. As a result, on such platforms only
221threads are supported. This could be fixed with sysv ipc locking or other
222locking alternatives.
217bc04b 223
f80dba8d
MT
224Other \*BSD platforms are untested, but fio should work there almost out of the
225box. Since I don't do test runs or even compiles on those platforms, your
226mileage may vary. Sending me patches for other platforms is greatly
217bc04b
JA
227appreciated. There's a lot of value in having the same test/benchmark tool
228available on all platforms.
229
f80dba8d 230Note that POSIX aio is not enabled by default on AIX. Messages like these::
bf2e821a
CC
231
232 Symbol resolution failed for /usr/lib/libc.a(posix_aio.o) because:
233 Symbol _posix_kaio_rdwr (number 2) is not exported from dependent module /unix.
234
f80dba8d 235indicate one needs to enable POSIX aio. Run the following commands as root::
bf2e821a
CC
236
237 # lsdev -C -l posix_aio0
238 posix_aio0 Defined Posix Asynchronous I/O
239 # cfgmgr -l posix_aio0
240 # lsdev -C -l posix_aio0
241 posix_aio0 Available Posix Asynchronous I/O
242
f80dba8d 243POSIX aio should work now. To make the change permanent::
bf2e821a
CC
244
245 # chdev -l posix_aio0 -P -a autoconfig='available'
246 posix_aio0 changed
217bc04b
JA
247
248
f80dba8d
MT
249Running fio
250-----------
79809113 251
f80dba8d
MT
252Running fio is normally the easiest part - you just give it the job file
253(or job files) as parameters::
79809113 254
f80dba8d
MT
255 $ fio [options] [jobfile] ...
256
257and it will start doing what the *jobfile* tells it to do. You can give more
258than one job file on the command line, fio will serialize the running of those
259files. Internally that is the same as using the :option:`stonewall` parameter
260described in the parameter section.
261
262If the job file contains only one job, you may as well just give the parameters
263on the command line. The command line parameters are identical to the job
264parameters, with a few extra that control global parameters. For example, for
265the job file parameter :option:`iodepth=2 <iodepth>`, the mirror command line
266option would be :option:`--iodepth 2 <iodepth>` or :option:`--iodepth=2
267<iodepth>`. You can also use the command line for giving more than one job
268entry. For each :option:`--name <name>` option that fio sees, it will start a
269new job with that name. Command line entries following a
270:option:`--name <name>` entry will apply to that job, until there are no more
271entries or a new :option:`--name <name>` entry is seen. This is similar to the
272job file options, where each option applies to the current job until a new []
273job entry is seen.
274
275fio does not need to run as root, except if the files or devices specified in
276the job section requires that. Some other options may also be restricted, such
277as memory locking, I/O scheduler switching, and decreasing the nice value.
79809113 278
f80dba8d
MT
279If *jobfile* is specified as ``-``, the job file will be read from standard
280input.