Jens Axboe [Mon, 26 Sep 2005 12:38:23 +0000 (14:38 +0200)]
[PATCH] blkparse: hash-by-name fixes
- Be more precise in dumping (name, ...) in the stats, only do it
when there really was more than one.
- The -n option needs to clear the hash-by-pid variable.
Jens Axboe [Mon, 26 Sep 2005 12:32:59 +0000 (14:32 +0200)]
[PATCH] blkparse: oops, missed removing a hash debug statement
Jens Axboe [Mon, 26 Sep 2005 12:30:12 +0000 (14:30 +0200)]
[PATCH] TODO: add entry about fixing show_entries_rb() OOO logic
Jens Axboe [Mon, 26 Sep 2005 12:26:24 +0000 (14:26 +0200)]
[PATCH] blkparse: Add option to hash process by name
For some traces, you really don't want to see thousands of cc1
processes in the stats, you want to collect the stats by name
instead. This adds the -n option to blkparse, enabling that.
Switched to Jenkins hash at the same time, as it allows for
easy hash-by-u32 or hash-by-name as we see fit.
Jens Axboe [Mon, 26 Sep 2005 11:12:33 +0000 (13:12 +0200)]
[PATCH] blkparse: more sequence fixes
Jens Axboe [Mon, 26 Sep 2005 10:26:29 +0000 (12:26 +0200)]
[PATCH] blkparse: improve skipping and genesis_time lookup
Jens Axboe [Mon, 26 Sep 2005 09:51:58 +0000 (11:51 +0200)]
[PATCH] blkparse: hardcode the default formats instead of parsing them
It takes a lot of time to manually parse each individual trace,
and there's really no point in doing that work if the user did
not specificy a custom format. We now only parse the given custom
formats.
Without any custom formats (eg the default), parsing a trace of
1.3 million entries now takes 4.3 seconds where it took 11.25 before.
Jens Axboe [Mon, 26 Sep 2005 08:50:04 +0000 (10:50 +0200)]
[PATCH] blkparse: force dump of remaining rb entries on reading done
Otherwise we may not display a significant chunk of entries.
Jens Axboe [Sun, 25 Sep 2005 12:31:24 +0000 (14:31 +0200)]
[PATCH] kernel: put back old syncronization
It's actually the best - keep the trace on the stack, so multiple
invocations from the same CPU (ie from process and irq context at
the same time) get their own 't', and keep the local irq disable
to prevent interleaving of trace and PDU with other contexts.
Jens Axboe [Sun, 25 Sep 2005 05:37:11 +0000 (07:37 +0200)]
[PATCH] blkparse: split format parsing into seperate file
Jens Axboe [Sat, 24 Sep 2005 19:54:44 +0000 (21:54 +0200)]
Revert "[PATCH] kernel: kill irq save/restore"
This reverts
68bf440bf36d239ff3cb27b34b95ea7911c7c5a6 commit.
Jens Axboe [Fri, 23 Sep 2005 15:24:04 +0000 (17:24 +0200)]
[PATCH] kernel: kill irq save/restore
It should not be needed anymore, the writes a per-CPU and the
get_cpu()/put_cpu() will prevent preemption.
Jens Axboe [Fri, 23 Sep 2005 15:21:11 +0000 (17:21 +0200)]
[PATCH] kernel: avoid trace corruption on SMP
Maintain a static trace structure per-CPU.
Jens Axboe [Fri, 23 Sep 2005 14:54:56 +0000 (16:54 +0200)]
[PATCH] blkparse: sequence fixes
Only set last sequence _after_ a potential break of the loop.
And remember to set last_allowed_time for stdin reads as well.
Jens Axboe [Fri, 23 Sep 2005 14:45:51 +0000 (16:45 +0200)]
[PATCH] blkparse: fix sorting of file events regardless of batch size
If the batch wraps, it would be possible to start displaying events
way into the future if parsing multiple devices.
Jens Axboe [Fri, 23 Sep 2005 13:09:37 +0000 (15:09 +0200)]
Revert "[PATCH] blkparse: rbtree sort primary key should be sequence, not time"
This reverts
acba447a946b022df6a30159425309f033e46ab7 commit.
Jens Axboe [Fri, 23 Sep 2005 09:46:40 +0000 (11:46 +0200)]
[PATCH] blktrace: Add option for setting sub buffer size
Use -b to give sub buffer size in KiB, -n to give number of
sub buffers wanted. This affects the relayfs trace setup.
Jens Axboe [Fri, 23 Sep 2005 09:39:48 +0000 (11:39 +0200)]
[PATCH] blkparse: rbtree sort primary key should be sequence, not time
Otherwise we may lose events if the event time is a little out
of sync.
Jens Axboe [Fri, 23 Sep 2005 09:12:48 +0000 (11:12 +0200)]
[PATCH] blkparse: can't return on-stack variables
Just make it static, access is serialized to it.
Jens Axboe [Fri, 23 Sep 2005 08:07:54 +0000 (10:07 +0200)]
[PATCH] README: add link to mailing list at vger
Jens Axboe [Fri, 23 Sep 2005 06:49:22 +0000 (08:49 +0200)]
[PATCH] blkparse: don't append .log to the -o given output file
Nathan Scott [Fri, 23 Sep 2005 06:34:23 +0000 (08:34 +0200)]
[PATCH] blkparse: make the nfiles counter live inside pdi
Gets rid of the extra malloc and is cleaner.
Nathan Scott [Fri, 23 Sep 2005 06:33:20 +0000 (08:33 +0200)]
[PATCH] blkparse: fix pointer cast warnings
Alan D. Brunelle [Thu, 22 Sep 2005 18:06:53 +0000 (20:06 +0200)]
[PATCH] blkparse: correctly account number of files per device
Jens Axboe [Thu, 22 Sep 2005 18:02:41 +0000 (20:02 +0200)]
Merge refs/heads/master from origin
Jens Axboe [Thu, 22 Sep 2005 18:02:30 +0000 (20:02 +0200)]
[PATCH] blkparse: more optimizations
The sort path unification and memory cutback introduced some
overhead, mainly due to lots of memory allocations and frees
of the same structures over and over again. Add a frontend
cache for struct trace and blk_io_trace allocations.
Also concatenate a lot of the string operations in fmt_select,
it's quite high in the profiles.
This brings us to the same performance as the memory hog version.
Jens Axboe [Thu, 22 Sep 2005 14:47:40 +0000 (16:47 +0200)]
[PATCH] kernel: export the __blk_add_trace() symbol for modules
Jens Axboe [Thu, 22 Sep 2005 14:43:52 +0000 (16:43 +0200)]
Merge refs/heads/master from origin
Jens Axboe [Thu, 22 Sep 2005 14:43:48 +0000 (16:43 +0200)]
[PATCH] blkparse: drastically reduce memory foot print
File reads needed enough memory to hold all events from all CPUs.
This quickly grew to hundreds of megabytes. Bring down this number
by interleaving reads from the files and relying on the same
mechanism as the piped reads to syncronize them. This makes file reads
use the same path as pipes and actually removes more code than it adds.
For a 900,000 entry file, memory consumption here is down from ~110MiB
from about 1MiB to process it.
Jens Axboe [Thu, 22 Sep 2005 14:40:44 +0000 (16:40 +0200)]
Revert "[PATCH] blkparse: bad ncpus check"
This reverts
08d83c704fbda722d3040d643665e2c029744aa7 commit.
Jens Axboe [Thu, 22 Sep 2005 14:15:45 +0000 (16:15 +0200)]
[PATCH] blkparse: bad ncpus check
Jens Axboe [Thu, 22 Sep 2005 12:29:50 +0000 (14:29 +0200)]
[PATCH] blkparse: shave 20% off the runtime
Various optimizations and moving code around a little.
Jens Axboe [Thu, 22 Sep 2005 11:33:55 +0000 (13:33 +0200)]
[PATCH] Add local master repo to documentation
Note that it is using the git daemon, not rsync.
Jens Axboe [Thu, 22 Sep 2005 10:30:12 +0000 (12:30 +0200)]
[PATCH] Update in-program help/usage info
Jens Axboe [Thu, 22 Sep 2005 10:11:08 +0000 (12:11 +0200)]
[PATCH] blkparse: Fix stopwatch
The option should be start:end, not start:duration (at least that is
what I intended and what is documented). Also fix the actual dumping,
we need to prune earlier entries from the rb tree or we will loop
forever.
Jens Axboe [Thu, 22 Sep 2005 08:56:05 +0000 (10:56 +0200)]
[PATCH] blkparse: more PC fixes
Add size of request as well, kill space at the end of pdu dump.
Jens Axboe [Thu, 22 Sep 2005 08:46:06 +0000 (10:46 +0200)]
[PATCH] blkparse: fix some pc command dumping
Jens Axboe [Thu, 22 Sep 2005 08:29:10 +0000 (10:29 +0200)]
[PATCH] Add support for bounce and split operations
Jens Axboe [Wed, 21 Sep 2005 16:45:29 +0000 (18:45 +0200)]
[PATCH] README: typo
Jens Axboe [Wed, 21 Sep 2005 16:38:38 +0000 (18:38 +0200)]
[PATCH] blkparse: just skip unknown events, there are not fatal
Jens Axboe [Wed, 21 Sep 2005 15:36:01 +0000 (17:36 +0200)]
Merge branch 'master' of ssh://axboe@/linux/kernel/git/axboe/blktrace
Jens Axboe [Wed, 21 Sep 2005 15:35:30 +0000 (17:35 +0200)]
[PATCH] blkparse: split queue and insert into two operations
We need this to support md/dm actions.
Jens Axboe [Wed, 21 Sep 2005 11:59:23 +0000 (13:59 +0200)]
[PATCH] Fix output naming, really
It's still confusing since the manual name wont include _out like
the automatically generated one will. So now a trace of /dev/sda
will generate sda.blktrace.cpu and blkparse -i sda will catch
those files. Similarly, a blktrace -o sda will generate the
same filename.
Jens Axboe [Wed, 21 Sep 2005 11:52:52 +0000 (13:52 +0200)]
[PATCH] blkparse: fix the alphasorting
Bad qsort size passed in and missing increment in ppis assignment.
Verified that it actually works now :-)
Jens Axboe [Wed, 21 Sep 2005 09:57:25 +0000 (11:57 +0200)]
[PATCH] blkparse: missing free of ppis
Doesn't really matter since we exit after this, but...
Jens Axboe [Wed, 21 Sep 2005 09:46:51 +0000 (11:46 +0200)]
[PATCH] blkparse: include pid in stat output
Avoids name conflict with threads.
Jens Axboe [Wed, 21 Sep 2005 09:45:39 +0000 (11:45 +0200)]
[PATCH] blkparse: sort the processes by name in stat output
Jens Axboe [Tue, 20 Sep 2005 12:41:37 +0000 (14:41 +0200)]
[PATCH] Change output/input file expectations
I think it's a little confusing right now with the naming, giving
-o foo to blktrace results in foo_dev_out.cpu files. Change this
to instead generate foo.cpu files, if the option is given.
Jens Axboe [Tue, 20 Sep 2005 07:43:49 +0000 (09:43 +0200)]
[PATCH] blktrace: fix exit
get_data() workers should return failure on is_done(), otherwise
extract() will attempt to verify a bad trace.
Alan D. Brunelle [Tue, 20 Sep 2005 07:34:45 +0000 (09:34 +0200)]
[PATCH] blkparse: Add format specifiers and format field identifiers
This allows the user to completely customize the output format.
Jens Axboe [Tue, 20 Sep 2005 07:13:34 +0000 (09:13 +0200)]
[PATCH] blktrace: prepare for mmap usage again
It isn't enabled right now, we need a proper consumed/produced
notification model to signal availability of data. poll() doesn't
work well enough, since it only works on a sub-buffer granularity.
Jens Axboe [Tue, 20 Sep 2005 07:12:34 +0000 (09:12 +0200)]
[PATCH] blkparse: account skips for file reads as well
Jens Axboe [Mon, 19 Sep 2005 11:59:30 +0000 (13:59 +0200)]
[PATCH] blkparse: allow a few more skips
Jens Axboe [Mon, 19 Sep 2005 11:46:19 +0000 (13:46 +0200)]
[PATCH] btrace: need space between options
Jens Axboe [Mon, 19 Sep 2005 11:34:56 +0000 (13:34 +0200)]
[PATCH] blkparse: more file breakage
If we have skipped entries in a file, don't wait around for more entries.
Jens Axboe [Mon, 19 Sep 2005 11:07:29 +0000 (13:07 +0200)]
[PATCH] blkparse: actually include blkparse diff
Forgot to update the cache for the file breakage patch
Jens Axboe [Mon, 19 Sep 2005 10:39:22 +0000 (12:39 +0200)]
[PATCH] blkparse: Fixup breakage in file reads
The pipe fixed broke file reads:
- Don't kill individual entries in show_entries_rb(),
since it's a big chunk on file reads
- Need to "lookup" pdi, in case it was resized.
Jens Axboe [Mon, 19 Sep 2005 10:25:14 +0000 (12:25 +0200)]
[PATCH] blkparse: reinstate __find_track() failures
It's informational and can help find bugs.
Jens Axboe [Mon, 19 Sep 2005 10:22:57 +0000 (12:22 +0200)]
[PATCH] blkparse: fix frontmerge tracking
Need to lookup by sector + size and remember that sectors are shifted
9, not 10.
Jens Axboe [Mon, 19 Sep 2005 10:14:18 +0000 (12:14 +0200)]
[PATCH] blkparse: better sorting over a pipe
It's quite easy to skip events when reading on an SMP system over
a pipe, since the events are per-CPU. Place incoming events in
a staging area and back off displaying traces if we detect skipped
events.
Jens Axboe [Sat, 17 Sep 2005 18:48:56 +0000 (20:48 +0200)]
[PATCH] Add TODO of action items before 1.0 release
Jens Axboe [Fri, 16 Sep 2005 13:25:05 +0000 (15:25 +0200)]
[PATCH] blkparse: don't print an error on failure to find request
This can happen if we lose or have out-of-order events when
passing the data through a pipe.
Jens Axboe [Fri, 16 Sep 2005 12:08:49 +0000 (14:08 +0200)]
[PATCH] blkparse: include plugging in stats
Jens Axboe [Fri, 16 Sep 2005 11:31:35 +0000 (13:31 +0200)]
[PATCH] Makefile: make sure to keep CFLAGS for link
Jens Axboe [Fri, 16 Sep 2005 08:10:50 +0000 (10:10 +0200)]
[PATCH] Improve unplug logging
Extend it to capture whether this was an io based unplug (eg someone
doing a wait_on_page() or similar), or of it was a timeout based event).
The logging code also forget to turn the unplug payload into on-wire
big endian and the parsing pdu offset was wrong.
Jens Axboe [Thu, 15 Sep 2005 16:03:41 +0000 (18:03 +0200)]
[PATCH] README: update to say it needs 2.6.14-rc1 or newer
Jens Axboe [Thu, 15 Sep 2005 13:09:49 +0000 (15:09 +0200)]
[PATCH] blkparse: add log_action()
For events where sectors and length don't really apply (eg plugging)
Jens Axboe [Thu, 15 Sep 2005 13:07:30 +0000 (15:07 +0200)]
[PATCH] blkparse: improve unplug logging
Add the queued number of requests count as the payload data.
Jens Axboe [Thu, 15 Sep 2005 08:26:20 +0000 (10:26 +0200)]
[PATCH] blkparse: Add support for the plug/unplug events
Jens Axboe [Thu, 15 Sep 2005 08:25:48 +0000 (10:25 +0200)]
[PATCH] kernel: Update to 2.6.14-rc1-git
Includes functional changes as well:
- Move the BLK_TA_QUEUE trace to where it belongs, in __elv_add_request(). This
will catch internal queueing of drivers as well.
- Add tracing of plug and unplug events.
Nathan Scott [Thu, 15 Sep 2005 07:31:12 +0000 (09:31 +0200)]
[PATCH] blktrace: better check for mounted relayfs
Don't just check for the presence of the directory, also check whether
relayfs really is mounted there.
Nathan Scott [Thu, 15 Sep 2005 07:29:24 +0000 (09:29 +0200)]
[PATCH] blkparse: add stopwatch functionality
Like the blktrace -w option, but allow for defining an interval as well.
Nathan Scott [Wed, 14 Sep 2005 17:32:27 +0000 (19:32 +0200)]
[PATCH] blktrace: add stopwatch functionality
Enables you to stop the trace after a defined period has passed.
Nathan Scott [Wed, 14 Sep 2005 17:30:49 +0000 (19:30 +0200)]
[PATCH] btrace: make the output easily parsably by default
Drop CPU/device summary unless specifically requested.
Nathan Scott [Wed, 14 Sep 2005 17:30:14 +0000 (19:30 +0200)]
[PATCH] Add btrace to Makefile
Nathan Scott [Wed, 14 Sep 2005 17:29:40 +0000 (19:29 +0200)]
[PATCH] Add btrace script
Using blktrace and blkparse in the "standard" piped setup is a little
cumbersome and not very intuitive, so add a script to do this automatically.
This is what users would expect.
Nathan Scott [Mon, 12 Sep 2005 08:06:59 +0000 (10:06 +0200)]
[PATCH] Add support for tracing multiple devices
This bumps the protocol number to 0x05, as device info was added to the
trace structure.
Nathan Scott [Mon, 12 Sep 2005 07:21:00 +0000 (09:21 +0200)]
[PATCH] blkparse: Fix mistaken double rb lookup in find_track()
Jens Axboe [Fri, 9 Sep 2005 19:10:58 +0000 (21:10 +0200)]
[PATCH] README: document the -t option
Jens Axboe [Fri, 9 Sep 2005 18:13:55 +0000 (20:13 +0200)]
[PATCH] blkparse: move magic check a little earlier in stdin read
Otherwise we don't know if ->pdu_len and other fields are truly valid.
Jens Axboe [Fri, 9 Sep 2005 15:07:17 +0000 (17:07 +0200)]
[PATCH] blkparse: fix quit on stdin reads
If the blocking read returns 0, we really are done. So safely exit.
Jens Axboe [Fri, 9 Sep 2005 14:28:14 +0000 (16:28 +0200)]
[PATCH] blkparse: fixup stats
Pretty up the output a little, split into read/write.
Jens Axboe [Fri, 9 Sep 2005 14:07:39 +0000 (16:07 +0200)]
[PATCH] blkparse: add per-process stats for longest waits
Jens Axboe [Fri, 9 Sep 2005 13:56:02 +0000 (15:56 +0200)]
[PATCH] blkparse: Add request allocation -> queue time
Jens Axboe [Fri, 9 Sep 2005 13:47:26 +0000 (15:47 +0200)]
[PATCH] blkparse: change formatting of output a little
Jens Axboe [Fri, 9 Sep 2005 11:36:42 +0000 (13:36 +0200)]
[PATCH] blkparse: optimize the read_sort_events() memory allocations
It's a little silly to malloc/free the buffer all the time, more practical
to let it settle on its own.
Jens Axboe [Fri, 9 Sep 2005 13:25:47 +0000 (15:25 +0200)]
Merge refs/heads/master from origin
Jens Axboe [Fri, 9 Sep 2005 13:24:13 +0000 (15:24 +0200)]
[PATCH] blkparse: Display io queue/completion stats
Display the time a request spent in the io scheduler and the time it took
that given request to be completed by the drive/driver.
Jens Axboe [Fri, 9 Sep 2005 10:30:33 +0000 (12:30 +0200)]
[PATCH] blkparse: fix leak in stdin read and sort
Jens Axboe [Fri, 9 Sep 2005 10:04:17 +0000 (12:04 +0200)]
[PATCH] blkparse: add framework for tracking individual ios
Jens Axboe [Fri, 9 Sep 2005 07:25:31 +0000 (09:25 +0200)]
[PATCH] blkparse: change output buffering
If logging to stdout, only use a per-line buffer instead of fully buffered.
Nathan Scott [Fri, 9 Sep 2005 07:22:52 +0000 (09:22 +0200)]
[PATCH] blktrace: Fix whitespace in log_merge()
Nathan Scott [Fri, 9 Sep 2005 07:22:13 +0000 (09:22 +0200)]
[PATCH] blktrace: remove unneeded strdup
Nathan Scott [Fri, 9 Sep 2005 07:20:26 +0000 (09:20 +0200)]
[PATCH] Fix dispatch reporting
Nathan Scott [Fri, 9 Sep 2005 07:19:26 +0000 (09:19 +0200)]
[PATCH] Fix time typo, it's in nanoseconds
Jens Axboe [Fri, 9 Sep 2005 07:18:49 +0000 (09:18 +0200)]
[PATCH] Use __WORDSIZE in hash
Originally fron Nathan, improved to remove 'bpl' as well.
Jens Axboe [Thu, 8 Sep 2005 18:15:18 +0000 (20:15 +0200)]
[PATCH] blkparse: Add per-process stats and fix output
Add the option (-s) to display per-process read/write stats as well as
the per-processor and total output. At the same time, fix the output so
it's a single file instead of per-processor (doesn't make sense here) and
put everything in the output file given with -o. If no output file is given,
stdout is used.
Jens Axboe [Thu, 8 Sep 2005 09:34:26 +0000 (11:34 +0200)]
[PATCH] Update README with info about the snapshots
Jens Axboe [Thu, 8 Sep 2005 06:37:57 +0000 (08:37 +0200)]
Merge refs/heads/master from origin
Jens Axboe [Thu, 8 Sep 2005 06:37:33 +0000 (08:37 +0200)]
[PATCH] Add patch for Linus' kernel
relayfs has been merged now, so it's easier to provide a mainline patch as well.