Block IO Tracing ---------------- Written by Jens Axboe (initial version and kernel support), Alan D. Brunelle (threading and splitup into two seperate programs), Nathan Scott (bug fixes, process names, multiple devices) Requirements ------------ You need to be running a 2.6.14-rc2 kernel or newer, with the blk-trace patch included in this repository. If you forgot where you got it, the url is: rsync://rsync.kernel.org/pub/scm/linux/kernel/git/axboe/blktrace.git If you don't have git, you can get hourly snapshots from: http://www.kernel.org/pub/linux/kernel/people/axboe/blktrace/ The snapshots include the full git object database as well. kernel.org has excessively long mirror times, so if you have git installed, you can pull the master tree from: git://brick.kernel.dk/data/git/blktrace.git For browsing the repo over http and viewing history etc, you can direct your browser to: http://brick.kernel.dk/git Usage ----- $ blktrace -d [ -r relay_path ] [ -o output ] [ -k ] [ -w time ] [ -a action ] [ -A action mask ] -d Use specified device. May also be given last after options. -r Path to mounted relayfs, defaults to /relay. -o File(s) to send output to. -k Kill running trace. -w Stop after defined time, in seconds. -a Only trace specific actions (use more -a options to add actions). Available actions are: READ WRITE BARRIER SYNC QUEUE REQUEUE ISSUE COMPLETE FS PC -A Give the trace mask directly as a number. -v Print program version info. -b Sub buffer size in KiB. -n Number of sub buffers. $ blkparse -i [ -o ] [ -b rb_batch ] [ -s ] [ -t ] [ -q ] [ -w start:stop ] [ -f output format ] [ -F format spec ] -i Input file containing trace data, or '-' for stdin. -o Output file. If not given, output is stdout. -b stdin read batching. -s Show per-program io statistics. -n Hash processes by name, not pid. -t Track individual ios. Will tell you the time a request took to get queued, to get dispatched, and to get completed. -q Quiet. Don't display any stats at the end of the trace. -w Only parse data between the given time interval in seconds. If 'start' isn't given, blkparse defaults the start time to 0. -f Output format. Customize the output format. The format field identifiers are: %a - Action %c - CPU ID %C - Task command name %d - Direction (r/w) %D - Device number %e - Error number %M - Major %m - Minor %N - Number of bytes %n - Number of sectors %p - PID %P - PDU %s - Sequence number %S - Sector number %t - Time (wallclock - nanoseconds) %T - Time (wallclock - seconds) %u - Time (processing - microseconds) %U - Unplug depth -F Format specification. The individual specifiers are: B - Back merge C - Complete D - Issue F - Front merge G - Get request I - Insert M - Both front and back merge P - Plug Q - Queue R - Requeue S - Sleep requests T - Unplug timer U - Unplug IO W - Bounce X - Split -v Print program version info. $ verify_blkparse filename Verifies an output file from blkparse. All it does is check if the events in the file are correctly time ordered. If an entry is found that isn't ordered, it's dumped to stdout. If you want to do live tracing, you can pipe the data between blktrace and blkparse: % blktrace -d -o - | blkparse -i - This has a small risk of displaying some traces a little out of sync, since it will do batch sorts of input events. Similarly, you can do traces over the network with eg netcat: % blktrace -d /dev/sda -o - | netcat parsehost portno % netcat -l -p portno tracehost | blkparse -i - Which will send the traces from tracehost to parsehost over the network on the defined port number. Resources --------- vger hosts a mailing list dedicated to btrace discussion and development. The list is called linux-btrace@vger.kernel.org, subscribe by sending a mail to majordomo@vger.kernel.org with 'subscribe linux-btrace' in the mail body. 20050906, Jens Axboe