path: root/blkparse.c
AgeCommit message (Collapse)Author
2019-09-25blkparse: fix absolute timestamp when reading from fileHiroaki Mihara
This patch fixes the wrong absolute timestamps when blkparse reads data from files. The blkparse command prints out wrong timestamps if all the following conditions are met, * The blkparse command reads data from files created by blktrace. * "z" format option is set as OUTPUT DESCRIPTION. ex.) blkparse xxx.blktrace.0 -f "%z\n" * start_timestamp(=blktrace command started) != genesis_time(=first I/O traced) When blkparse reads data from pipe instead, it yields correct timestamps. The root cause of this issue comes from the fact that the time difference between start_timestamp and genesis_time is not added when blkparse reads data from files. When blkparse reads data from pipe, the time-difference is added through find_genesis() function. The following test cases show the contradictions in absolute timestams. Also the Step 4 shows that the issue is fixed with the blkparse command with the suggesting patch. * Step 1: After invoking blktrace command, test I/O traffic was generated by dd command as follows, # date +%Y%m%d_%H%M%S_%N; dd if=/dev/sda3 of=/dev/null count=1 iflag=direct 20190919_092726_077032490 1+0 records in 1+0 records out 512 bytes copied, 0.00122329 s, 419 kB/s The timestamp was recorded just before executing dd command. The test I/O would have been traced right after 09:27:26.077032490 . * Step 2: The blkparse command reads data from "pipe". $ cat test.blktrace.* | blkparse - -f "%T.%t %z %C\n" 0.000000000 09:27:22.427592 kworker/0:0 0.000002080 09:27:22.427594 kworker/0:0 . . 3.652263118 09:27:26.079855 dd 3.652265818 09:27:26.079857 dd 3.652274742 09:27:26.079866 dd 3.652277266 09:27:26.079869 dd The first I/O by dd command showed the relative timestamp as 3.652263118 and the absolute timestamp as 09:27:26.079855, which is right after the timestamp shown in the Step 1. * Step 3: The blkparse command reads from the trace "file" created in the Step 1. $ blkparse test -f "%T.%t %z %C\n" Input file test.blktrace.0 added Input file test.blktrace.1 added Input file test.blktrace.2 added Input file test.blktrace.3 added 0.000000000 09:27:21.187304 kworker/0:0 0.000002080 09:27:21.187306 kworker/0:0 . . 3.652263118 09:27:24.839567 dd 3.652265818 09:27:24.839570 dd 3.652274742 09:27:24.839578 dd 3.652277266 09:27:24.839581 dd In the previous step (Step 2), the data was passed via pipe. In this case, the blkparse command reads data from the same file, instead. The first I/O by dd command showed the relative timestamp as 3.652263118 and the absolute timestamp as 09:27:24.839567, which is a few seconds earlier than the absolute timestamp recorded in the Step 1. The order of events and the absolute timestamps contradict. * Step 4: The blkparse command with the suggesting patch (./blkparse_with_patch) reads data from the trace file created in the Step 1. $ ./blkparse_with_patch test -f "%T.%t %z %C\n" Input file test.blktrace.0 added Input file test.blktrace.1 added Input file test.blktrace.2 added Input file test.blktrace.3 added 0.000000000 09:27:22.427592 kworker/0:0 0.000002080 09:27:22.427594 kworker/0:0 . . 3.652263118 09:27:26.079855 dd 3.652265818 09:27:26.079857 dd 3.652274742 09:27:26.079866 dd 3.652277266 09:27:26.079869 dd In this case, the absolute timestamps showed the same value as shown in the Step 2(the case with pipe). The time gap between the genesis_ time and the start_timestamp was corrected even if the blkparse reads data from files. Signed-off-by: Hiroaki Mihara <> the# Signed-off-by: Jens Axboe <>
2019-09-25blkparse: split off the timestamp correction code in to a separate functionHiroaki Mihara
find_genesis() function has code to correct abs_start_time, which is later used to calculate the absolute timestamps of each traced records. Put this code in a separate function, so that it can be used later by the blkparse code. No functional change. Signed-off-by: Hiroaki Mihara <> Signed-off-by: Jens Axboe <>
2019-05-21blkparse: add support sort program by io eventWeiping Zhang
Displays each program's data sorted by program name or io event, like Queued, Read, Write and Complete. When -S is specified the -s will be ignored. The capital letters Q,R,W,C stand for KB, then q/r/w/c stand for IO. The N is used for sorting programs by name, same to -s. If you want to sort programs by how many data they queued, you can use: blkparse -i sda.blktrace. -q -S Q -o sda.parse Signed-off-by: Weiping Zhang <> Signed-off-by: Jens Axboe <>
2017-11-04Blktrace 1.2.0blktrace-1.2.0Jens Axboe
Signed-off-by: Jens Axboe <>
2014-09-24blktrace 1.1.0blktrace-1.1.0Jens Axboe
Bump it up to a full 1.1 since we now include iowatcher. Signed-off-by: Jens Axboe <>
2013-03-19More accurate calculation of the total read/write valuesIvan Dyukov
If block device has many request with size less than 1K, blkparse ignores such requests because it treats each request in Kb. Signed-off-by: Jens Axboe <>
2012-02-27blktrace 1.0.5blktrace-1.0.5originJens Axboe
Signed-off-by: Jens Axboe <>
2012-02-27Too small arrays for file namesVasily Tarasov
In our experiments blktrace/blkparse file names encode a lot of infomation about the particular experiment. We noticed that for long enough file names blkparse does not work. The reason is that per_cpu_info->fname[] is of 128 bytes. As a result, in setup_file() function only part of the file name gets to ->fname[]. Then stat() fails and we exit the function. Notice, that no error is printed in this case. In the following patch ->fname[] size is increased to POSIX defined PATH_MAX. Signed-off-by: Vasily Tarasov < Signed-off-by: Jens Axboe <>
2012-02-01avoid string overflowsEric Sandeen
Several places using strcpy would benefit from strncpy for safety. Signed-off-by: Eric Sandeen <> Signed-off-by: Jens Axboe <>
2012-02-01blkparse: initialize cpu_mapEric Sandeen
We malloc'd cpu_map, and then did: cpu_map[CPU_IDX(cpu)] |= (1UL << CPU_BIT(cpu)); ... not sure how that ever worked if cpu_map was not initialized! Signed-off-by: Eric Sandeen <> Signed-off-by: Jens Axboe <>
2012-01-31blktrace 1.0.4blktrace-1.0.4Jens Axboe
Signed-off-by: Jens Axboe <>
2011-08-11blktrace 1.0.3blktrace-1.0.3Jens Axboe
Signed-off-by: Jens Axboe <>
2011-08-03Merge branch 'master' of ssh:// Axboe
2011-08-03blkparse: fix up incorrect pc write completion countJeff Moyer
I noticed in some traces that I was seeing summaries like the following: Total (sde): Reads Queued: 76, 304KiB Writes Queued: 16,384, 1,048MiB Read Dispatches: 76, 304KiB Write Dispatches: 2,210, 1,048MiB Reads Requeued: 0 Writes Requeued: 0 Reads Completed: 76, 304KiB Writes Completed: 2,210, 1,048MiB Read Merges: 0, 0KiB Write Merges: 14,174, 907,136KiB PC Reads Queued: 0, 0KiB PC Writes Queued: 0, 0KiB PC Read Disp.: 4, 0KiB PC Write Disp.: 0, 0KiB PC Reads Req.: 0 PC Writes Req.: 0 PC Reads Compl.: 4 PC Writes Compl.: 2,210 IO unplugs: 2,124 Timer unplugs: 0 Note how there were no PC Writes dispatched, but there were 2210 completed. It turns out to be a minor typo in the code. The attached patch fixes the reporting for me. Signed-off-by: Jeff Moyer <> Signed-off-by: Jens Axboe <>
2011-05-26blkparse: Avoid segfault for wrong cpu number.Tao Ma
Currently we only check the magic number to see whether a blktrace is valid or not, but Bill Broadley did meet with a case that the cpu info is wrong with a number of 1725552676. So in resize_cpu_info, we meet with a overflow when calculating size = new_count * sizeof(struct per_cpu_info); And the program will be either segfault or has the error of out of memory. Although this is more likely a kernel problem, the blkparse shoudn't segfault for it. So this patch just check whether the cpu stored in the trace is the same as the file, if not, just warn it out and skip it. Cc: Jens Axboe <> Reported-by: Bill Broadley <> Signed-off-by: Tao Ma <> Signed-off-by: Jens Axboe <>
2011-03-16blktrace 1.0.2blktrace-1.0.2Jens Axboe
Signed-off-by: Jens Axboe <>
2011-01-11Merge branch 'master' of ssh:// Axboe
2011-01-11blkparse: Fix blktrace output pipe broken in the new kernelTao Ma
With the newest kernel(say 2.6.37, some older one should also have the similar problem), some cfq messages are added to blktrace, so it makes the old blkparse broken. See a simple example: 1. blktrace /dev/sdb -o -|blkparse -i - 2. Run the following command(/dev/sdb1 is mounted at /mnt/test_dir): dd if=/mnt/test_dir/test of=/dev/null bs=4k count=1 iflag=direct There are only 2 lines of output there: 8,16 0 1 0.000000000 13183 A R 114759 + 8 <- (8,17) 114696 8,16 0 2 0.000000491 13183 Q R 114759 + 8 [dd] And even we run a command line like: for((i=0;i<100;i++))do dd if=/mnt/ocfs2/test of=/dev/null bs=4k count=1 iflag=direct;done We are only given the same 2 lines of output. While the really one should look like: 8,16 0 1 0.000000000 13319 A R 114759 + 8 <- (8,17) 114696 8,16 0 2 0.000000376 13319 Q R 114759 + 8 [dd] 8,16 0 0 0.000005931 0 m N cfq13319 alloced 8,16 0 3 0.000006259 13319 G R 114759 + 8 [dd] 8,16 0 4 0.000007143 13319 P N [dd] 8,16 0 5 0.000007817 13319 I R 114759 + 8 [dd] 8,16 0 0 0.000008491 0 m N cfq13319 insert_request 8,16 0 0 0.000009029 0 m N cfq13319 add_to_rr ... The main reason is that in show_entries_rb, we test sequences every time, but actually with some messages like cfq, the sequence number is always 0 which makes the old sequence check refuses all the logs after it. So only check/store sequence number if it isn't a message. Signed-off-by: Tao Ma <> Signed-off-by: Jens Axboe <>
2010-10-22blktrace: blkparse documentation updateEdward Shishkin
Fixup for RH bugzilla 595615. Document blkparse options: -A, --set-mask, -a, --act-mask, -D. --input-directory Signed-off-by: Edward Shishkin <> Signed-off-by: Jens Axboe <>
2010-04-20blktrace: print correct usageEdward Shishkin
Fixup for 498898: Problem: When somebody runs blktrace without parameters, it shows the usage message. The usage message suggests that version number "x.y.z" is a required parameter, which is not true. Solution: Don't print version number when running blktrace, blkparce, btt without parameters. Signed-off-by: Edward Shishkin <> Signed-off-by: Jens Axboe <>
2010-04-19blkparse: exit with error if no tracefiles foundEric Sandeen
If no tracefiles are found, exit with non-0 status Resolves Red Hat Bugzilla #500118 Reported-by: Milos Malik <> Signed-off-by: Eric Sandeen <> Signed-off-by: Jens Axboe <>
2009-09-01blkparse: allow stdout output with -d option (using '-' as the filename)Jens Axboe
Signed-off-by: Jens Axboe <>
2009-05-11Version 1.0.1blktrace-1.0.1Jens Axboe
Signed-off-by: Jens Axboe <>
2009-01-12Added no messages option to blkparse.cAlan D. Brunelle
Added a new option (-M, --no-msgs) option to blkparse: I have found that the CFQ I/O scheduler sends a *tremendous* amount of messages, that bloat the .bin file generated when using the -d option. The file sizes can shrink by >50% when using the -M option in those case. Signed-off-by: Alan D. Brunelle <> Signed-off-by: Jens Axboe <>
2008-10-30Set release version 1.0.0blktrace-1.0.0Jens Axboe
Signed-off-by: Jens Axboe <>
2008-10-17blkparse: add hint for discarded drv_data tracesMartin Peschke
Display an informational message on blkparse exit to notify users that additional data was available which would require to be dumped to binary output. Signed-off-by: Martin Peschke <> Signed-off-by: Jens Axboe <>
2008-10-16Add driver data supportStefan Raspl
Adds a new type of action 'drv_data' for blktrace to handle binary driver-specific data. Since the data is binary, blkparse will only put it in a binary file, not in the regular human-readable output. Signed-off-by: Stefan Raspl <> Signed-off-by: Martin Peschke <> Signed-off-by: Jens Axboe <>
2008-05-27Added in handling of MESSAGE notesAlan D. Brunelle
Sample output: 8,16 1 691118 17.417000000 0 C R 2660776 + 8 [0] 8,16 1 691119 17.417000000 0 D R 2660792 + 8 [swapper] 8,16 1 691120 17.417000000 4688 U N [dd] 42 8,16 1 0 17.418000000 0 m N elv switch: noop 8,16 1 691121 17.418000000 4638 C R 2660784 + 8 [0] 8,16 1 691122 17.418000000 4638 D R 2660800 + 8 [bash] 8,16 1 691123 17.418000000 4638 C R 2660792 + 8 [0] Thanks to Carl Henrik Lunde <> for adding in sequence printing & time-stamp correction.
2008-05-18blkparse: cope with missing process notify eventJens Axboe
Signed-off-by: Jens Axboe <>
2008-05-05Fix problem with -w optionLuis Useche
This patch fixes the problem when the -w option is used in the file mode (i.e., no fifo mode). It just consists of moving the checking of the stopwatch_end after the time is updated with genesis. This also includes the checking of the stopwatch_start. Signed-off-by: Jens Axboe <>
2008-04-02blkparse: Introduce optional accounting of PC requestsChristof Schmitt
The current blkparse lists only PC requests, but does not account for number of PC requests and the size of the attached data. This patch introduces optional accounting for PC requests. If any PC event was encountered, the summary output will also include data from PC requests. For PC completion events the amount of transferred data is not available, since the kernel uses the data_len field as residual bytes count on event completion. Signed-off-by: Christof Schmitt <> Signed-off-by: Jens Axboe <>
2008-04-02blkparse: Add PC requests to depth traceChristof Schmitt
The PC requests use the same queue and also contribute to the queue length, so they should be added to the queue depth trace. Signed-off-by: Christof Schmitt <> Signed-off-by: Jens Axboe <>
2007-08-28blktrace 0.99.3blktrace-0.99.3Jens Axboe
Signed-off-by: Jens Axboe <>
2007-05-21Account size of mergesJens Axboe
Just like we do for queues/dispatches/completions etc. Signed-off-by: Jens Axboe <>
2007-05-21Fix queued vs dispatch numbersJens Axboe
A merge trace includes a queue trace. blkparse accounted both of them, resulting in the queued transfer number being a lot larger than it should be. Fix that by not accounting bytes in the merge handling. Signed-off-by: Jens Axboe <>
2007-05-16Fix crash with '-' stdin inputJens Axboe
Need to use argv[optind], not optarg. Signed-off-by: Jens Axboe <>
2007-01-02[PATCH] Fixup filename on full file inputMing Zhang
If the user by mistake specifies the full filename, strip that back down to avoid missing input files. Signed-off-by: Jens Axboe <>
2007-01-02[PATCH] Fix double free in error pathJens Axboe
Reported by zhuxudong <> If we fail reading, we will free the realloc'ed pointer and jump to the error path which will free the original pointer. Fix this by clearing the original pointer. Signed-off-by: Jens Axboe <>
2006-12-21[PATCH] check do_foo return value before show_stat()Ming Zhang
current code will show_stats() even when do_foo() return non zero. Signed-off-by: Jens Axboe <>
2006-12-21[PATCH] fix a divide by zero errorMing Zhang
When pdi->events and pdi->seq_skips are both 0, current code will do 0/0. Signed-off-by: Jens Axboe <>
2006-12-20[PATCH] Enable use of FIFO as inputJens Axboe
Just reuse the stdin stuff for handling a named pipe. Signed-off-by: Jens Axboe <>
2006-12-19PATCH: blkparse: fix probem with empty trace files stopping the check for ↵Alan D. Brunelle
other files Fix problem where empty input files cause premature parsing of files. Had a problem where CPU 0 opened a file with 0 traces, and that caused the run to end before processing other trace files. Signed-off-by: Jens Axboe <>
2006-12-01[PATCH] Bump versionblktrace-0.99.2Jens Axboe
Signed-off-by: Jens Axboe <>
2006-12-01[PATCH] Add timestamp supportOlaf Kirch
Signed-off-by: Jens Axboe <>
2006-11-13[PATCH] blkparse: A couple of fixes: correct time output & flush files.Alan D. Brunelle
Signed-off-by: Jens Axboe <>
2006-11-13[PATCH] blkparse: Converted do_file() to use a merge sort instead of RB trees.Alan D. Brunelle
Signed-off-by: Jens Axboe <>
2006-11-13[PATCH] blkparse: Make binary dumps buffered (reduces system calls).Alan D. Brunelle
Signed-off-by: Jens Axboe <>
2006-11-13[PATCH] blkparse Added --no-text-output (-O)Alan D. Brunelle
Removes textual output, useful for bin only. Signed-off-by: Jens Axboe <>
2006-09-05[PATCH] email updateJens Axboe
2006-02-27[PATCH] blkprase: binary dump capability + notify addedAlan D. Brunelle
Here is a proposed patch which would allow blkparse to dump its output to a binary file (in addition to the ascii output). It adds a new '-d <file>' parameter. The purpose of this would be to allow for the easier handling of the collected data by analysis tools. [Rather than having to parse the textual output from blkparse, or duplicate the collation efforts of blkparse itself.]