fio: fix aio trim completion latencies
[fio.git] / README
CommitLineData
f80dba8d
MT
1Overview and history
2--------------------
3
4Fio was originally written to save me the hassle of writing special test case
5programs when I wanted to test a specific workload, either for performance
6reasons or to find/reproduce a bug. The process of writing such a test app can
7be tiresome, especially if you have to do it often. Hence I needed a tool that
8would be able to simulate a given I/O workload without resorting to writing a
9tailored test case again and again.
10
11A test work load is difficult to define, though. There can be any number of
12processes or threads involved, and they can each be using their own way of
13generating I/O. You could have someone dirtying large amounts of memory in an
14memory mapped file, or maybe several threads issuing reads using asynchronous
15I/O. fio needed to be flexible enough to simulate both of these cases, and many
16more.
17
18Fio spawns a number of threads or processes doing a particular type of I/O
19action as specified by the user. fio takes a number of global parameters, each
20inherited by the thread unless otherwise parameters given to them overriding
21that setting is given. The typical use of fio is to write a job file matching
22the I/O load one wants to simulate.
ebac4655 23
2b02b546
JA
24
25Source
26------
27
f80dba8d 28Fio resides in a git repo, the canonical place is:
2b02b546 29
4649b352 30 git://git.kernel.dk/fio.git
97f049c9 31
4649b352
GG
32When inside a corporate firewall, git:// URL sometimes does not work.
33If git:// does not work, use the http protocol instead:
a9bac3f9 34
4649b352 35 http://git.kernel.dk/fio.git
2b02b546 36
f80dba8d 37Snapshots are frequently generated and :file:`fio-git-*.tar.gz` include the git
88d398d0 38meta data as well. Other tarballs are archives of official fio releases.
4649b352 39Snapshots can download from:
2b02b546 40
4649b352 41 http://brick.kernel.dk/snaps/
2b02b546 42
f80dba8d
MT
43There are also two official mirrors. Both of these are automatically synced with
44the main repository, when changes are pushed. If the main repo is down for some
45reason, either one of these is safe to use as a backup:
01fa84d5
JA
46
47 git://git.kernel.org/pub/scm/linux/kernel/git/axboe/fio.git
f80dba8d 48
01fa84d5
JA
49 https://git.kernel.org/pub/scm/linux/kernel/git/axboe/fio.git
50
51or
52
3826b24a 53 git://github.com/axboe/fio.git
244e170e 54
f80dba8d 55 https://github.com/axboe/fio.git
a71844ff 56
2b02b546 57
726f6ff0
JA
58Mailing list
59------------
60
4649b352 61The fio project mailing list is meant for anything related to fio including
6468020d
TK
62general discussion, bug reporting, questions, and development. For bug reporting,
63see REPORTING-BUGS.
2e8552b0 64
f80dba8d
MT
65An automated mail detailing recent commits is automatically sent to the list at
66most daily. The list address is fio@vger.kernel.org, subscribe by sending an
67email to majordomo@vger.kernel.org with
4649b352
GG
68
69 subscribe fio
2e8552b0 70
4f5d1526
EIB
71in the body of the email. Archives can be found here:
72
4649b352 73 http://www.spinics.net/lists/fio/
4f5d1526
EIB
74
75and archives for the old list can be found here:
2e8552b0 76
4649b352 77 http://maillist.kernel.dk/fio-devel/
726f6ff0
JA
78
79
f80dba8d
MT
80Author
81------
bbfd6b00 82
f80dba8d
MT
83Fio was written by Jens Axboe <axboe@kernel.dk> to enable flexible testing of
84the Linux I/O subsystem and schedulers. He got tired of writing specific test
85applications to simulate a given workload, and found that the existing I/O
86benchmark/test tools out there weren't flexible enough to do what he wanted.
6de43c1b 87
f80dba8d 88Jens Axboe <axboe@kernel.dk> 20060905
2382dca7 89
c8931876 90
f80dba8d
MT
91Binary packages
92---------------
bbfd6b00 93
f80dba8d
MT
94Debian:
95 Starting with Debian "Squeeze", fio packages are part of the official
a881438b 96 Debian repository. http://packages.debian.org/search?keywords=fio .
53adf64f 97
f80dba8d
MT
98Ubuntu:
99 Starting with Ubuntu 10.04 LTS (aka "Lucid Lynx"), fio packages are part
100 of the Ubuntu "universe" repository.
a881438b 101 http://packages.ubuntu.com/search?keywords=fio .
53adf64f 102
a881438b
SW
103Red Hat, Fedora, CentOS & Co:
104 Starting with Fedora 9/Extra Packages for Enterprise Linux 4, fio
105 packages are part of the Fedora/EPEL repositories.
658f3525 106 https://apps.fedoraproject.org/packages/fio .
f41862f7 107
f80dba8d
MT
108Mandriva:
109 Mandriva has integrated fio into their package repository, so installing
110 on that distro should be as easy as typing ``urpmi fio``.
9aa5fe32 111
a6415d76
QB
112Arch Linux:
113 An Arch Linux package is provided under the Community sub-repository:
114 https://www.archlinux.org/packages/?sort=&q=fio
115
f80dba8d
MT
116Solaris:
117 Packages for Solaris are available from OpenCSW. Install their pkgutil
118 tool (http://www.opencsw.org/get-it/pkgutil/) and then install fio via
119 ``pkgutil -i fio``.
7409711b 120
f80dba8d
MT
121Windows:
122 Rebecca Cran <rebecca+fio@bluestop.org> has fio packages for Windows at
eae92fbf
JA
123 https://www.bluestop.org/fio/ . The latest builds for Windows can also
124 be grabbed from https://ci.appveyor.com/project/axboe/fio by clicking
125 the latest x86 or x64 build, then selecting the ARTIFACTS tab.
78080867 126
f80dba8d
MT
127BSDs:
128 Packages for BSDs may be available from their binary package repositories.
129 Look for a package "fio" using their binary package managers.
53adf64f 130
ebac4655 131
f80dba8d 132Building
79809113
JA
133--------
134
f80dba8d 135Just type::
bebe6398 136
f80dba8d
MT
137 $ ./configure
138 $ make
139 $ make install
bebe6398 140
f80dba8d
MT
141Note that GNU make is required. On BSDs it's available from devel/gmake within
142ports directory; on Solaris it's in the SUNWgmake package. On platforms where
143GNU make isn't the default, type ``gmake`` instead of ``make``.
811826be 144
f80dba8d
MT
145Configure will print the enabled options. Note that on Linux based platforms,
146the libaio development packages must be installed to use the libaio
147engine. Depending on distro, it is usually called libaio-devel or libaio-dev.
811826be 148
f80dba8d
MT
149For gfio, gtk 2.18 (or newer), associated glib threads, and cairo are required
150to be installed. gfio isn't built automatically and can be enabled with a
151``--enable-gfio`` option to configure.
bebe6398 152
f80dba8d 153To build fio with a cross-compiler::
bebe6398 154
f80dba8d
MT
155 $ make clean
156 $ make CROSS_COMPILE=/path/to/toolchain/prefix
bebe6398 157
f80dba8d 158Configure will attempt to determine the target platform automatically.
bebe6398 159
f80dba8d
MT
160It's possible to build fio for ESX as well, use the ``--esx`` switch to
161configure.
4649b352 162
bebe6398 163
f80dba8d
MT
164Windows
165~~~~~~~
323255cc 166
f80dba8d
MT
167On Windows, Cygwin (http://www.cygwin.com/) is required in order to build
168fio. To create an MSI installer package install WiX 3.8 from
169http://wixtoolset.org and run :file:`dobuild.cmd` from the :file:`os/windows`
170directory.
323255cc 171
f80dba8d 172How to compile fio on 64-bit Windows:
323255cc 173
f80dba8d 174 1. Install Cygwin (http://www.cygwin.com/). Install **make** and all
b570e037
SW
175 packages starting with **mingw64-x86_64**. Ensure
176 **mingw64-x86_64-zlib** are installed if you wish
b86770c6 177 to enable fio's log compression functionality.
f80dba8d
MT
178 2. Open the Cygwin Terminal.
179 3. Go to the fio directory (source files).
180 4. Run ``make clean && make -j``.
39b5f61e 181
b570e037
SW
182To build fio for 32-bit Windows, ensure the -i686 versions of the previously
183mentioned -x86_64 packages are installed and run ``./configure
184--build-32bit-win`` before ``make``. To build an fio that supports versions of
185Windows below Windows 7/Windows Server 2008 R2 also add ``--target-win-ver=xp``
186to the end of the configure line that you run before doing ``make``.
39b5f61e 187
f80dba8d
MT
188It's recommended that once built or installed, fio be run in a Command Prompt or
189other 'native' console such as console2, since there are known to be display and
190signal issues when running it under a Cygwin shell (see
f5c5f869
SW
191https://github.com/mintty/mintty/issues/56 and
192https://github.com/mintty/mintty/wiki/Tips#inputoutput-interaction-with-alien-programs
193for details).
39b5f61e 194
39b5f61e 195
f80dba8d
MT
196Documentation
197~~~~~~~~~~~~~
39b5f61e 198
f80dba8d
MT
199Fio uses Sphinx_ to generate documentation from the reStructuredText_ files.
200To build HTML formatted documentation run ``make -C doc html`` and direct your
201browser to :file:`./doc/output/html/index.html`. To build manual page run
202``make -C doc man`` and then ``man doc/output/man/fio.1``. To see what other
203output formats are supported run ``make -C doc help``.
39b5f61e 204
f80dba8d
MT
205.. _reStructuredText: http://www.sphinx-doc.org/rest.html
206.. _Sphinx: http://www.sphinx-doc.org
39b5f61e 207
bebe6398 208
217bc04b
JA
209Platforms
210---------
211
1b8c5af7 212Fio works on (at least) Linux, Solaris, AIX, HP-UX, OSX, NetBSD, OpenBSD,
2af5bd99 213Windows, FreeBSD, and DragonFly. Some features and/or options may only be
f80dba8d
MT
214available on some of the platforms, typically because those features only apply
215to that platform (like the solarisaio engine, or the splice engine on Linux).
217bc04b
JA
216
217Some features are not available on FreeBSD/Solaris even if they could be
f80dba8d
MT
218implemented, I'd be happy to take patches for that. An example of that is disk
219utility statistics and (I think) huge page support, support for that does exist
220in FreeBSD/Solaris.
217bc04b 221
06eac6b2
SW
222Fio uses pthread mutexes for signalling and locking and some platforms do not
223support process shared pthread mutexes. As a result, on such platforms only
224threads are supported. This could be fixed with sysv ipc locking or other
225locking alternatives.
217bc04b 226
f80dba8d
MT
227Other \*BSD platforms are untested, but fio should work there almost out of the
228box. Since I don't do test runs or even compiles on those platforms, your
229mileage may vary. Sending me patches for other platforms is greatly
217bc04b
JA
230appreciated. There's a lot of value in having the same test/benchmark tool
231available on all platforms.
232
f80dba8d 233Note that POSIX aio is not enabled by default on AIX. Messages like these::
bf2e821a
CC
234
235 Symbol resolution failed for /usr/lib/libc.a(posix_aio.o) because:
236 Symbol _posix_kaio_rdwr (number 2) is not exported from dependent module /unix.
237
f80dba8d 238indicate one needs to enable POSIX aio. Run the following commands as root::
bf2e821a
CC
239
240 # lsdev -C -l posix_aio0
241 posix_aio0 Defined Posix Asynchronous I/O
242 # cfgmgr -l posix_aio0
243 # lsdev -C -l posix_aio0
244 posix_aio0 Available Posix Asynchronous I/O
245
f80dba8d 246POSIX aio should work now. To make the change permanent::
bf2e821a
CC
247
248 # chdev -l posix_aio0 -P -a autoconfig='available'
249 posix_aio0 changed
217bc04b
JA
250
251
f80dba8d
MT
252Running fio
253-----------
79809113 254
f80dba8d
MT
255Running fio is normally the easiest part - you just give it the job file
256(or job files) as parameters::
79809113 257
f80dba8d
MT
258 $ fio [options] [jobfile] ...
259
260and it will start doing what the *jobfile* tells it to do. You can give more
261than one job file on the command line, fio will serialize the running of those
262files. Internally that is the same as using the :option:`stonewall` parameter
263described in the parameter section.
264
265If the job file contains only one job, you may as well just give the parameters
266on the command line. The command line parameters are identical to the job
267parameters, with a few extra that control global parameters. For example, for
268the job file parameter :option:`iodepth=2 <iodepth>`, the mirror command line
269option would be :option:`--iodepth 2 <iodepth>` or :option:`--iodepth=2
270<iodepth>`. You can also use the command line for giving more than one job
271entry. For each :option:`--name <name>` option that fio sees, it will start a
272new job with that name. Command line entries following a
273:option:`--name <name>` entry will apply to that job, until there are no more
274entries or a new :option:`--name <name>` entry is seen. This is similar to the
275job file options, where each option applies to the current job until a new []
276job entry is seen.
277
278fio does not need to run as root, except if the files or devices specified in
279the job section requires that. Some other options may also be restricted, such
280as memory locking, I/O scheduler switching, and decreasing the nice value.
79809113 281
f80dba8d
MT
282If *jobfile* is specified as ``-``, the job file will be read from standard
283input.