-fio
----
-
-fio is a tool that will spawn a number of thread doing a particular
-type of io action as specified by the user. fio takes a number of
-global parameters, each inherited by the thread unless otherwise
-parameters given to them overriding that setting is given.
+Overview and history
+--------------------
+
+Fio was originally written to save me the hassle of writing special test case
+programs when I wanted to test a specific workload, either for performance
+reasons or to find/reproduce a bug. The process of writing such a test app can
+be tiresome, especially if you have to do it often. Hence I needed a tool that
+would be able to simulate a given I/O workload without resorting to writing a
+tailored test case again and again.
+
+A test work load is difficult to define, though. There can be any number of
+processes or threads involved, and they can each be using their own way of
+generating I/O. You could have someone dirtying large amounts of memory in an
+memory mapped file, or maybe several threads issuing reads using asynchronous
+I/O. fio needed to be flexible enough to simulate both of these cases, and many
+more.
+
+Fio spawns a number of threads or processes doing a particular type of I/O
+action as specified by the user. fio takes a number of global parameters, each
+inherited by the thread unless otherwise parameters given to them overriding
+that setting is given. The typical use of fio is to write a job file matching
+the I/O load one wants to simulate.
Source
------
-fio resides in a git repo, the canonical place is:
+Fio resides in a git repo, the canonical place is:
+
+ git://git.kernel.dk/fio.git
+
+When inside a corporate firewall, git:// URL sometimes does not work.
+If git:// does not work, use the http protocol instead:
+
+ http://git.kernel.dk/fio.git
+
+Snapshots are frequently generated and :file:`fio-git-*.tar.gz` include the git
+meta data as well. Other tarballs are archives of official fio releases.
+Snapshots can download from:
+
+ http://brick.kernel.dk/snaps/
+
+There are also two official mirrors. Both of these are automatically synced with
+the main repository, when changes are pushed. If the main repo is down for some
+reason, either one of these is safe to use as a backup:
+
+ git://git.kernel.org/pub/scm/linux/kernel/git/axboe/fio.git
+
+ https://git.kernel.org/pub/scm/linux/kernel/git/axboe/fio.git
-git://brick.kernel.dk/data/git/fio.git
+or
-Snapshots are frequently generated as well and they include the git
-meta data as well. You can download them here:
+ git://github.com/axboe/fio.git
-http://brick.kernel.dk/snaps/
+ https://github.com/axboe/fio.git
-Pascal Bleser <guru@unixtech.be> has fio RPMs in his repository, you
-can find them here:
-http://linux01.gwdg.de/~pbleser/rpm-navigation.php?cat=System/fio
+Mailing list
+------------
+
+The fio project mailing list is meant for anything related to fio including
+general discussion, bug reporting, questions, and development. For bug reporting,
+see REPORTING-BUGS.
+
+An automated mail detailing recent commits is automatically sent to the list at
+most daily. The list address is fio@vger.kernel.org, subscribe by sending an
+email to majordomo@vger.kernel.org with
+
+ subscribe fio
+
+in the body of the email. Archives can be found here:
+
+ http://www.spinics.net/lists/fio/
+
+and archives for the old list can be found here:
+
+ http://maillist.kernel.dk/fio-devel/
+
+
+Author
+------
+
+Fio was written by Jens Axboe <axboe@kernel.dk> to enable flexible testing of
+the Linux I/O subsystem and schedulers. He got tired of writing specific test
+applications to simulate a given workload, and found that the existing I/O
+benchmark/test tools out there weren't flexible enough to do what he wanted.
+
+Jens Axboe <axboe@kernel.dk> 20060905
+
+
+Binary packages
+---------------
+
+Debian:
+ Starting with Debian "Squeeze", fio packages are part of the official
+ Debian repository. http://packages.debian.org/search?keywords=fio .
+
+Ubuntu:
+ Starting with Ubuntu 10.04 LTS (aka "Lucid Lynx"), fio packages are part
+ of the Ubuntu "universe" repository.
+ http://packages.ubuntu.com/search?keywords=fio .
+
+Red Hat, Fedora, CentOS & Co:
+ Starting with Fedora 9/Extra Packages for Enterprise Linux 4, fio
+ packages are part of the Fedora/EPEL repositories.
+ https://apps.fedoraproject.org/packages/fio .
+
+Mandriva:
+ Mandriva has integrated fio into their package repository, so installing
+ on that distro should be as easy as typing ``urpmi fio``.
+
+Arch Linux:
+ An Arch Linux package is provided under the Community sub-repository:
+ https://www.archlinux.org/packages/?sort=&q=fio
+
+Solaris:
+ Packages for Solaris are available from OpenCSW. Install their pkgutil
+ tool (http://www.opencsw.org/get-it/pkgutil/) and then install fio via
+ ``pkgutil -i fio``.
+
+Windows:
+ Rebecca Cran <rebecca+fio@bluestop.org> has fio packages for Windows at
+ https://www.bluestop.org/fio/ . The latest builds for Windows can also
+ be grabbed from https://ci.appveyor.com/project/axboe/fio by clicking
+ the latest x86 or x64 build, then selecting the ARTIFACTS tab.
+
+BSDs:
+ Packages for BSDs may be available from their binary package repositories.
+ Look for a package "fio" using their binary package managers.
Building
--------
-Just type 'make' and 'make install'. If on FreeBSD, for now you have to
-specify the FreeBSD Makefile with -f, eg:
-
-$ make -f Makefile.Freebsd && make -f Makefile.FreeBSD install
-
-This might change in the future if I opt for an autoconf type setup.
-
-
-Options
--------
-
-$ fio
- -s IO is sequential
- -b block size in KiB for each io
- -t <sec> Runtime in seconds
- -r For random io, sequence must be repeatable
- -R <on> If one thread fails to meet rate, quit all
- -o <on> Use direct IO is 1, buffered if 0
- -l Generate per-job latency logs
- -w Generate per-job bandwidth logs
- -f <file> Read <file> for job descriptions
- -h Print help info
- -v Print version information and exit
-
-The <jobs> format is as follows:
-
- name=x Use 'x' as the identifier for this job.
- directory=x Use 'x' as the top level directory for storing files
- rw=x 'x' may be: read, randread, write, randwrite,
- rw (read-write mix), randrw (read-write random mix)
- rwmixcycle=x Base cycle for switching between read and write
- in msecs.
- rwmixread=x 'x' percentage of rw mix ios will be reads. If
- rwmixwrite is also given, the last of the two will
- be used if they don't add up to 100%.
- rwmixwrite=x 'x' percentage of rw mix ios will be writes. See
- rwmixread.
- size=x Set file size to x bytes (x string can include k/m/g)
- ioengine=x 'x' may be: aio/libaio/linuxaio for Linux aio,
- posixaio for POSIX aio, sync for regular read/write io,
- mmap for mmap'ed io, splice for using splice/vmsplice,
- or sgio for direct SG_IO io. The latter only works on
- Linux on SCSI (or SCSI-like devices, such as
- usb-storage or sata/libata driven) devices.
- iodepth=x For async io, allow 'x' ios in flight
- overwrite=x If 'x', layout a write file first.
- prio=x Run io at prio X, 0-7 is the kernel allowed range
- prioclass=x Run io at prio class X
- bs=x Use 'x' for thread blocksize. May include k/m postfix.
- bsrange=x-y Mix thread block sizes randomly between x and y. May
- also include k/m postfix.
- direct=x 1 for direct IO, 0 for buffered IO
- thinktime=x "Think" x usec after each io
- rate=x Throttle rate to x KiB/sec
- ratemin=x Quit if rate of x KiB/sec can't be met
- ratecycle=x ratemin averaged over x msecs
- cpumask=x Only allow job to run on CPUs defined by mask.
- fsync=x If writing, fsync after every x blocks have been written
- startdelay=x Start this thread x seconds after startup
- timeout=x Terminate x seconds after startup
- offset=x Start io at offset x (x string can include k/m/g)
- invalidate=x Invalidate page cache for file prior to doing io
- sync=x Use sync writes if x and writing
- mem=x If x == malloc, use malloc for buffers. If x == shm,
- use shm for buffers. If x == mmap, use anon mmap.
- exitall When one thread quits, terminate the others
- bwavgtime=x Average bandwidth stats over an x msec window.
- create_serialize=x If 'x', serialize file creation.
- create_fsync=x If 'x', run fsync() after file creation.
- end_fsync=x If 'x', run fsync() after end-of-job.
- loops=x Run the job 'x' number of times.
- verify=x If 'x' == md5, use md5 for verifies. If 'x' == crc32,
- use crc32 for verifies. md5 is 'safer', but crc32 is
- a lot faster. Only makes sense for writing to a file.
- stonewall Wait for preceeding jobs to end before running.
- numjobs=x Create 'x' similar entries for this job
- thread Use pthreads instead of forked jobs
- zonesize=x
- zoneskip=y Zone options must be paired. If given, the job
- will skip y bytes for every x read/written. This
- can be used to gauge hard drive speed over the entire
- platter, without reading everything. Both x/y can
- include k/m/g suffix.
- iolog=x Open and read io pattern from file 'x'. The file must
- contain one io action per line in the following format:
- rw, offset, length
- where with rw=0/1 for read/write, and the offset
- and length entries being in bytes.
- write_iolog=x Write an iolog to file 'x' in the same format as iolog.
- The iolog options are exclusive, if both given the
- read iolog will be performed.
- lockmem=x Lock down x amount of memory on the machine, to
- simulate a machine with less memory available. x can
- include k/m/g suffix.
- nice=x Run job at given nice value.
- exec_prerun=x Run 'x' before job io is begun.
- exec_postrun=x Run 'x' after job io has finished.
- ioscheduler=x Use ioscheduler 'x' for this job.
-
-Examples using a job file
--------------------------
-
-A sample job file doing the same as above would look like this:
-
-[read_file]
-rw=0
-bs=4096
-
-[write_file]
-rw=1
-bs=16384
-
-And fio would be invoked as:
-
-$ fio -o1 -s -f file_with_above
-
-The second example would look like this:
-
-[rf1]
-rw=0
-prio=6
-
-[rf2]
-rw=0
-prio=3
-
-[rf3]
-rw=0
-prio=0
-direct=1
-
-And fio would be invoked as:
-
-$ fio -o0 -s -b4096 -f file_with_above
-
-'global' is a reserved keyword. When used as the filename, it sets the
-default options for the threads following that section. It is possible
-to have more than one global section in the file, as it only affects
-subsequent jobs.
-
-Also see the examples/ dir for sample job files.
-
-
-Interpreting the output
------------------------
-
-fio spits out a lot of output. While running, fio will display the
-status of the jobs created. An example of that would be:
-
-Threads now running: 2 : [ww] [5.73% done]
-
-The characters inside the square brackets denote the current status of
-each thread. The possible values (in typical life cycle order) are:
-
-Idle Run
----- ---
-P Thread setup, but not started.
-C Thread created and running, but not doing anything yet
- R Running, doing sequential reads.
- r Running, doing random reads.
- W Running, doing sequential writes.
- w Running, doing random writes.
-V Running, doing verification of written data.
-E Thread exited, not reaped by main thread yet.
-_ Thread reaped.
-
-The other values are fairly self explanatory - number of thread currently
-running and doing io, and the estimated completion percentage.
-
-When fio is done (or interrupted by ctrl-c), it will show the data for
-each thread, group of threads, and disks in that order. For each data
-direction, the output looks like:
-
-Client1 (g=0): err= 0:
- write: io= 32MiB, bw= 666KiB/s, runt= 50320msec
- slat (msec): min= 0, max= 136, avg= 0.03, dev= 1.92
- clat (msec): min= 0, max= 631, avg=48.50, dev=86.82
- bw (KiB/s) : min= 0, max= 1196, per=51.00%, avg=664.02, dev=681.68
- cpu : usr=1.49%, sys=0.25%, ctx=7969
-
-The client number is printed, along with the group id and error of that
-thread. Below is the io statistics, here for writes. In the order listed,
-they denote:
-
-io= Number of megabytes io performed
-bw= Average bandwidth rate
-runt= The runtime of that thread
- slat= Submission latency (avg being the average, dev being the
- standard deviation). This is the time it took to submit
- the io. For sync io, the slat is really the completion
- latency, since queue/complete is one operation there.
- clat= Completion latency. Same names as slat, this denotes the
- time from submission to completion of the io pieces. For
- sync io, clat will usually be equal (or very close) to 0,
- as the time from submit to complete is basically just
- CPU time (io has already been done, see slat explanation).
- bw= Bandwidth. Same names as the xlat stats, but also includes
- an approximate percentage of total aggregate bandwidth
- this thread received in this group. This last value is
- only really useful if the threads in this group are on the
- same disk, since they are then competing for disk access.
-cpu= CPU usage. User and system time, along with the number
- of context switches this thread went through.
-
-After each client has been listed, the group statistics are printed. They
-will look like this:
-
-Run status group 0 (all jobs):
- READ: io=64MiB, aggrb=22178, minb=11355, maxb=11814, mint=2840msec, maxt=2955msec
- WRITE: io=64MiB, aggrb=1302, minb=666, maxb=669, mint=50093msec, maxt=50320msec
-
-For each data direction, it prints:
-
-io= Number of megabytes io performed.
-aggrb= Aggregate bandwidth of threads in this group.
-minb= The minimum average bandwidth a thread saw.
-maxb= The maximum average bandwidth a thread saw.
-mint= The minimum runtime of a thread.
-maxt= The maximum runtime of a thread.
-
-And finally, the disk statistics are printed. They will look like this:
-
-Disk stats (read/write):
- sda: ios=16398/16511, merge=30/162, ticks=6853/819634, in_queue=826487, util=100.00%
-
-Each value is printed for both reads and writes, with reads first. The
-numbers denote:
-
-ios= Number of ios performed by all groups.
-merge= Number of merges io the io scheduler.
-ticks= Number of ticks we kept the disk busy.
-io_queue= Total time spent in the disk queue.
-util= The disk utilization. A value of 100% means we kept the disk
- busy constantly, 50% would be a disk idling half of the time.
+Just type::
+
+ $ ./configure
+ $ make
+ $ make install
+
+Note that GNU make is required. On BSDs it's available from devel/gmake within
+ports directory; on Solaris it's in the SUNWgmake package. On platforms where
+GNU make isn't the default, type ``gmake`` instead of ``make``.
+
+Configure will print the enabled options. Note that on Linux based platforms,
+the libaio development packages must be installed to use the libaio
+engine. Depending on distro, it is usually called libaio-devel or libaio-dev.
+
+For gfio, gtk 2.18 (or newer), associated glib threads, and cairo are required
+to be installed. gfio isn't built automatically and can be enabled with a
+``--enable-gfio`` option to configure.
+
+To build fio with a cross-compiler::
+
+ $ make clean
+ $ make CROSS_COMPILE=/path/to/toolchain/prefix
+
+Configure will attempt to determine the target platform automatically.
+
+It's possible to build fio for ESX as well, use the ``--esx`` switch to
+configure.
+
+
+Windows
+~~~~~~~
+
+On Windows, Cygwin (http://www.cygwin.com/) is required in order to build
+fio. To create an MSI installer package install WiX 3.8 from
+http://wixtoolset.org and run :file:`dobuild.cmd` from the :file:`os/windows`
+directory.
+
+How to compile fio on 64-bit Windows:
+
+ 1. Install Cygwin (http://www.cygwin.com/). Install **make** and all
+ packages starting with **mingw64-x86_64**. Ensure
+ **mingw64-x86_64-zlib** are installed if you wish
+ to enable fio's log compression functionality.
+ 2. Open the Cygwin Terminal.
+ 3. Go to the fio directory (source files).
+ 4. Run ``make clean && make -j``.
+
+To build fio for 32-bit Windows, ensure the -i686 versions of the previously
+mentioned -x86_64 packages are installed and run ``./configure
+--build-32bit-win`` before ``make``. To build an fio that supports versions of
+Windows below Windows 7/Windows Server 2008 R2 also add ``--target-win-ver=xp``
+to the end of the configure line that you run before doing ``make``.
+
+It's recommended that once built or installed, fio be run in a Command Prompt or
+other 'native' console such as console2, since there are known to be display and
+signal issues when running it under a Cygwin shell (see
+https://github.com/mintty/mintty/issues/56 and
+https://github.com/mintty/mintty/wiki/Tips#inputoutput-interaction-with-alien-programs
+for details).
+
+
+Documentation
+~~~~~~~~~~~~~
+
+Fio uses Sphinx_ to generate documentation from the reStructuredText_ files.
+To build HTML formatted documentation run ``make -C doc html`` and direct your
+browser to :file:`./doc/output/html/index.html`. To build manual page run
+``make -C doc man`` and then ``man doc/output/man/fio.1``. To see what other
+output formats are supported run ``make -C doc help``.
+
+.. _reStructuredText: http://www.sphinx-doc.org/rest.html
+.. _Sphinx: http://www.sphinx-doc.org
+
+
+Platforms
+---------
+
+Fio works on (at least) Linux, Solaris, AIX, HP-UX, OSX, NetBSD, OpenBSD,
+Windows, FreeBSD, and DragonFly. Some features and/or options may only be
+available on some of the platforms, typically because those features only apply
+to that platform (like the solarisaio engine, or the splice engine on Linux).
+
+Some features are not available on FreeBSD/Solaris even if they could be
+implemented, I'd be happy to take patches for that. An example of that is disk
+utility statistics and (I think) huge page support, support for that does exist
+in FreeBSD/Solaris.
+
+Fio uses pthread mutexes for signalling and locking and some platforms do not
+support process shared pthread mutexes. As a result, on such platforms only
+threads are supported. This could be fixed with sysv ipc locking or other
+locking alternatives.
+
+Other \*BSD platforms are untested, but fio should work there almost out of the
+box. Since I don't do test runs or even compiles on those platforms, your
+mileage may vary. Sending me patches for other platforms is greatly
+appreciated. There's a lot of value in having the same test/benchmark tool
+available on all platforms.
+
+Note that POSIX aio is not enabled by default on AIX. Messages like these::
+
+ Symbol resolution failed for /usr/lib/libc.a(posix_aio.o) because:
+ Symbol _posix_kaio_rdwr (number 2) is not exported from dependent module /unix.
+
+indicate one needs to enable POSIX aio. Run the following commands as root::
+
+ # lsdev -C -l posix_aio0
+ posix_aio0 Defined Posix Asynchronous I/O
+ # cfgmgr -l posix_aio0
+ # lsdev -C -l posix_aio0
+ posix_aio0 Available Posix Asynchronous I/O
+
+POSIX aio should work now. To make the change permanent::
+
+ # chdev -l posix_aio0 -P -a autoconfig='available'
+ posix_aio0 changed
+
+
+Running fio
+-----------
+
+Running fio is normally the easiest part - you just give it the job file
+(or job files) as parameters::
+
+ $ fio [options] [jobfile] ...
+
+and it will start doing what the *jobfile* tells it to do. You can give more
+than one job file on the command line, fio will serialize the running of those
+files. Internally that is the same as using the :option:`stonewall` parameter
+described in the parameter section.
+
+If the job file contains only one job, you may as well just give the parameters
+on the command line. The command line parameters are identical to the job
+parameters, with a few extra that control global parameters. For example, for
+the job file parameter :option:`iodepth=2 <iodepth>`, the mirror command line
+option would be :option:`--iodepth 2 <iodepth>` or :option:`--iodepth=2
+<iodepth>`. You can also use the command line for giving more than one job
+entry. For each :option:`--name <name>` option that fio sees, it will start a
+new job with that name. Command line entries following a
+:option:`--name <name>` entry will apply to that job, until there are no more
+entries or a new :option:`--name <name>` entry is seen. This is similar to the
+job file options, where each option applies to the current job until a new []
+job entry is seen.
+
+fio does not need to run as root, except if the files or devices specified in
+the job section requires that. Some other options may also be restricted, such
+as memory locking, I/O scheduler switching, and decreasing the nice value.
+
+If *jobfile* is specified as ``-``, the job file will be read from standard
+input.
projects / fio.git / blobdiff