Jens Axboe [Wed, 5 Oct 2005 15:54:17 +0000 (17:54 +0200)]
[PATCH] blktrace.tex: add description of each possible action
Jens Axboe [Wed, 5 Oct 2005 15:28:41 +0000 (17:28 +0200)]
Merge branch 'master' of ssh://axboe@brick.kernel.dk/data/git/blktrace
Jens Axboe [Wed, 5 Oct 2005 15:28:33 +0000 (17:28 +0200)]
[PATCH] blktrace.tex: add fs/pc request explanation
Kill the 1.1 ideas/questions section, everything in there has been
covered by now.
Jens Axboe [Wed, 5 Oct 2005 15:22:51 +0000 (17:22 +0200)]
[PATCH] Typo: __BIT_ENDIAN -> __BIG_ENDIAN
Jens Axboe [Wed, 5 Oct 2005 13:16:19 +0000 (15:16 +0200)]
[PATCH] Better endianness check
Jens Axboe [Wed, 5 Oct 2005 13:16:02 +0000 (15:16 +0200)]
[PATCH] Makefile: only include .c files in deps
Jens Axboe [Wed, 5 Oct 2005 11:05:00 +0000 (13:05 +0200)]
[PATCH] blktrace: sanity check the buf_size argument, cap at 16MiB.
Jens Axboe [Wed, 5 Oct 2005 09:48:14 +0000 (11:48 +0200)]
[PATCH] Makefile: dep fixup
Jens Axboe [Wed, 5 Oct 2005 09:47:25 +0000 (11:47 +0200)]
[PATCH] Makefile: add dependencies
Jens Axboe [Wed, 5 Oct 2005 09:44:50 +0000 (11:44 +0200)]
[PATCH] Forgot to kill root blktrace.tex
Jens Axboe [Wed, 5 Oct 2005 09:38:23 +0000 (11:38 +0200)]
[PATCH] Move docs to doc/
Alan D. Brunelle [Wed, 5 Oct 2005 09:07:49 +0000 (11:07 +0200)]
[PATCH] Add blktrace users guide
Jens Axboe [Wed, 5 Oct 2005 08:23:36 +0000 (10:23 +0200)]
[PATCH] blkparse: old patch broke the sequence check again
Jens Axboe [Wed, 5 Oct 2005 07:54:55 +0000 (09:54 +0200)]
[PATCH] blkparse: we have to allow one skip to prevent sequence screwups
If one CPU file is completed in the first run, check_sequence()
would allow it to skip too early causing unordered events.
Jens Axboe [Wed, 5 Oct 2005 07:53:31 +0000 (09:53 +0200)]
[PATCH] verify_blkparse: correct sscanf return
Jens Axboe [Wed, 5 Oct 2005 07:32:53 +0000 (09:32 +0200)]
[PATCH] Makefile: kill -W again
Jens Axboe [Wed, 5 Oct 2005 07:28:20 +0000 (09:28 +0200)]
[PATCH] Fix long options
Partly from Alan.
Jens Axboe [Wed, 5 Oct 2005 07:24:49 +0000 (09:24 +0200)]
Merge refs/heads/master from origin
Jens Axboe [Wed, 5 Oct 2005 07:24:44 +0000 (09:24 +0200)]
[PATCH] Remember to terminate options structure
If not an unknown option will segfault, woops.
Jens Axboe [Wed, 5 Oct 2005 07:13:29 +0000 (09:13 +0200)]
[PATCH] blktrace: fix a few -W warnings
And leave -W on in the Makefile
Alan D. Brunelle [Wed, 5 Oct 2005 07:12:17 +0000 (09:12 +0200)]
[PATCH] blkparse: Add -a/-A options to blkparse as well
This moves the mask stuff into a seperate file, act_mask.c
Jens Axboe [Tue, 4 Oct 2005 18:05:53 +0000 (20:05 +0200)]
[PATCH] Add -D output/input directory option to blkparse and blktrace
Jens Axboe [Tue, 4 Oct 2005 17:54:09 +0000 (19:54 +0200)]
[PATCH] blktrace: note that the -b option is in KiB
Jens Axboe [Tue, 4 Oct 2005 09:13:58 +0000 (11:13 +0200)]
[PATCH] blktrace: optimize the get_event() normal case
We usually only see partial traces in very few numbers (like
a few in a million), so optimize the normal case by getting
a full trace instead of splitting it into a 4-byte and remainder
read.
Jens Axboe [Mon, 3 Oct 2005 12:27:05 +0000 (14:27 +0200)]
[PATCH] blkparse: only break out of last loop if not dumping to stdout
Otherwise sending a ctrl-c when running btrace will quit, leaving
entries never dumped.
Jens Axboe [Mon, 3 Oct 2005 12:06:33 +0000 (14:06 +0200)]
[PATCH] blkparse: stdin sequence and aliases fixes
Jens Axboe [Mon, 3 Oct 2005 12:06:07 +0000 (14:06 +0200)]
[PATCH] blktrace: allow for unaligned events
Unfortunately it looks like we have to support this for now,
as it appears relayfs will still miss events on buffer wraps
for the read cases.
Jens Axboe [Mon, 3 Oct 2005 12:05:12 +0000 (14:05 +0200)]
[PATCH verify_blkparse: dump number of sequence aliases as well
Jens Axboe [Mon, 3 Oct 2005 12:04:48 +0000 (14:04 +0200)]
[PATCH] btrace: need large rb batch count for stdin sorting
Jens Axboe [Mon, 3 Oct 2005 09:33:59 +0000 (11:33 +0200)]
[PATCH] blktrace: various fixes
- Better exit_trace() backoff
- Don't assume that write() always writes the full chunk
- kill mmap(), we will never use it for packet driven workloads
- Bug in get_data_read/read_data() for incremental reads
- Optimize tid_fd_lock() holding, drop before freeing memory
Jens Axboe [Mon, 3 Oct 2005 08:06:01 +0000 (10:06 +0200)]
[PATCH] blkparse: kill old force debug print
Jens Axboe [Mon, 3 Oct 2005 07:05:01 +0000 (09:05 +0200)]
[PATCH] kernel: update patch to 2.6.14-rc3
Jens Axboe [Sun, 2 Oct 2005 19:34:10 +0000 (21:34 +0200)]
[PATCH] blkparse: Add option to print missing entries
And default to off, it can be fairly verbose.
Jens Axboe [Sun, 2 Oct 2005 13:39:14 +0000 (15:39 +0200)]
[PATCH] blkparse: check smallest sequence read
This allows check_sequence() to know when a skip happened and
when it is ok to proceed. Should be the final fix, now we now
when a sequence is missing and we nicely limit the memory
consumed in presence of large traces and missing entries.
This change brings down memory consumption of a skippy 1.3
million trace here down from 130MiB to 15MiB.
Jens Axboe [Fri, 30 Sep 2005 10:34:19 +0000 (12:34 +0200)]
[PATCH] blkparse: invert alloc checks, more intuitive
Jens Axboe [Fri, 30 Sep 2005 09:52:24 +0000 (11:52 +0200)]
[PATCH] blkparse: reduce memory consumption
- Better skip detection
- Limit maximum number of t/bit allocation cache entries
Nathan Scott [Fri, 30 Sep 2005 05:33:30 +0000 (07:33 +0200)]
[PATCH] blkparse: fix ->sector printing warnings
Jens Axboe [Thu, 29 Sep 2005 11:08:50 +0000 (13:08 +0200)]
[PATCH] README: add reference to gitweb on brick
Jens Axboe [Thu, 29 Sep 2005 10:01:37 +0000 (12:01 +0200)]
[PATCH] Assorted compiler warnings
Jens Axboe [Wed, 28 Sep 2005 15:57:03 +0000 (17:57 +0200)]
[PATCH] verify_blkparse: print events not ordered
And fix exit code.
Jens Axboe [Wed, 28 Sep 2005 15:50:56 +0000 (17:50 +0200)]
[PATCH] Add verify_blkparse description to README
Jens Axboe [Wed, 28 Sep 2005 15:45:51 +0000 (17:45 +0200)]
[PATCH] verify_blkparse: implement in C instead
The C version checks a 92MB file in 2 seconds where the php
version takes 23 seconds.
Jens Axboe [Wed, 28 Sep 2005 14:27:52 +0000 (16:27 +0200)]
[PATCH] verify_blkparse: missing line break
Thomas Kenne [Wed, 28 Sep 2005 13:51:16 +0000 (15:51 +0200)]
[PATCH] verify_blkparse: fix parsing
Current version would break, if the device number didn't have
a leading space.
Jens Axboe [Wed, 28 Sep 2005 13:41:22 +0000 (15:41 +0200)]
[PATCH] blkparse: prune old found entries in rb_last
Jens Axboe [Wed, 28 Sep 2005 13:40:16 +0000 (15:40 +0200)]
[PATCH] verify_blkparse: make sure file is readable
Jens Axboe [Wed, 28 Sep 2005 12:52:06 +0000 (14:52 +0200)]
[PATCH] verify_blkparse: proper check of file given
Jens Axboe [Wed, 28 Sep 2005 12:17:39 +0000 (14:17 +0200)]
[PATCH] blkparse: don't check time in last rb lookup
Thomas Kenne [Wed, 28 Sep 2005 12:12:55 +0000 (14:12 +0200)]
[PATCH] Add script to verify correctness of blkparse output
Only checks the time.
Jens Axboe [Wed, 28 Sep 2005 09:34:29 +0000 (11:34 +0200)]
[PATCH] blkparse: make the rb track root per-device
Jens Axboe [Wed, 28 Sep 2005 08:56:59 +0000 (10:56 +0200)]
[PATCH] blkparse: misc assorted fixes
- Move the trace verification earlier, so that traces on
trace_list are always known valid
- Start with -1 sequence so when using the stopwatch the
skip count is correct (first entry might not be 1)
- stdin get_cpu_info() fix
Jens Axboe [Wed, 28 Sep 2005 06:55:52 +0000 (08:55 +0200)]
[PATCH] kernel: move the plug/unplug logging around a little
Jens Axboe [Tue, 27 Sep 2005 22:03:59 +0000 (00:03 +0200)]
[PATCH] blkparse: sequence fixes
This should finally fix all the remaining sequence bugs. We
keep a backlog of already processed entries so we can lookup
any out-of-sequence events there. That takes care of one
direction of sequence and time mismatches. The other direction
is handled by fixing the trace_rb_find() lookup to work well
enough without having a precise time (which is the primary
key for the rb sort).
Jens Axboe [Tue, 27 Sep 2005 16:18:04 +0000 (18:18 +0200)]
[PATCH] blkparse: improve skip detection
Jens Axboe [Tue, 27 Sep 2005 13:38:08 +0000 (15:38 +0200)]
Forgot to kill old patch
Jens Axboe [Tue, 27 Sep 2005 10:18:14 +0000 (12:18 +0200)]
[PATCH] kernel: improve serialization and unplug logging
Jens Axboe [Tue, 27 Sep 2005 09:04:43 +0000 (11:04 +0200)]
Forgot to include the change for the last stopwatch fix...
Jens Axboe [Tue, 27 Sep 2005 08:22:33 +0000 (10:22 +0200)]
[PATCH] blkparse: another stab at stopwatch_end fixing
This should work reliably, the check is moved after we
have definitely called find_genesis().
Jens Axboe [Tue, 27 Sep 2005 08:15:24 +0000 (10:15 +0200)]
Revert "[PATCH] blkparse: final stopwatch check"
This reverts
a795e5fae97e14d77c3c3c7d0f6543e6c5f8c850 commit.
Jens Axboe [Tue, 27 Sep 2005 08:15:17 +0000 (10:15 +0200)]
[PATCH] blktrace: fix sparse warning
Jens Axboe [Mon, 26 Sep 2005 17:05:40 +0000 (19:05 +0200)]
[PATCH] blkparse: add %N format for bytes
We need this for PC requests, they are not multiples of 512
like the FS requests. At the same time change the default for
PC requests to dump in bytes.
Jens Axboe [Mon, 26 Sep 2005 13:42:05 +0000 (15:42 +0200)]
[PATCH] blkparse: final stopwatch check
Delayed quit when all files have been read and there are no
further entries beyond that point, then it is safe to quit
out early there as well.
Jens Axboe [Mon, 26 Sep 2005 13:36:19 +0000 (15:36 +0200)]
[PATCH] blkparse: more stopwatch cleanups
Jens Axboe [Mon, 26 Sep 2005 13:34:37 +0000 (15:34 +0200)]
[PATCH] blkparse: improve stopwatch
Move the checks much earlier, no point in doing a lot of
work for stuff we will toss out anyways.
Jens Axboe [Mon, 26 Sep 2005 12:38:23 +0000 (14:38 +0200)]
[PATCH] blkparse: hash-by-name fixes
- Be more precise in dumping (name, ...) in the stats, only do it
when there really was more than one.
- The -n option needs to clear the hash-by-pid variable.
Jens Axboe [Mon, 26 Sep 2005 12:32:59 +0000 (14:32 +0200)]
[PATCH] blkparse: oops, missed removing a hash debug statement
Jens Axboe [Mon, 26 Sep 2005 12:30:12 +0000 (14:30 +0200)]
[PATCH] TODO: add entry about fixing show_entries_rb() OOO logic
Jens Axboe [Mon, 26 Sep 2005 12:26:24 +0000 (14:26 +0200)]
[PATCH] blkparse: Add option to hash process by name
For some traces, you really don't want to see thousands of cc1
processes in the stats, you want to collect the stats by name
instead. This adds the -n option to blkparse, enabling that.
Switched to Jenkins hash at the same time, as it allows for
easy hash-by-u32 or hash-by-name as we see fit.
Jens Axboe [Mon, 26 Sep 2005 11:12:33 +0000 (13:12 +0200)]
[PATCH] blkparse: more sequence fixes
Jens Axboe [Mon, 26 Sep 2005 10:26:29 +0000 (12:26 +0200)]
[PATCH] blkparse: improve skipping and genesis_time lookup
Jens Axboe [Mon, 26 Sep 2005 09:51:58 +0000 (11:51 +0200)]
[PATCH] blkparse: hardcode the default formats instead of parsing them
It takes a lot of time to manually parse each individual trace,
and there's really no point in doing that work if the user did
not specificy a custom format. We now only parse the given custom
formats.
Without any custom formats (eg the default), parsing a trace of
1.3 million entries now takes 4.3 seconds where it took 11.25 before.
Jens Axboe [Mon, 26 Sep 2005 08:50:04 +0000 (10:50 +0200)]
[PATCH] blkparse: force dump of remaining rb entries on reading done
Otherwise we may not display a significant chunk of entries.
Jens Axboe [Sun, 25 Sep 2005 12:31:24 +0000 (14:31 +0200)]
[PATCH] kernel: put back old syncronization
It's actually the best - keep the trace on the stack, so multiple
invocations from the same CPU (ie from process and irq context at
the same time) get their own 't', and keep the local irq disable
to prevent interleaving of trace and PDU with other contexts.
Jens Axboe [Sun, 25 Sep 2005 05:37:11 +0000 (07:37 +0200)]
[PATCH] blkparse: split format parsing into seperate file
Jens Axboe [Sat, 24 Sep 2005 19:54:44 +0000 (21:54 +0200)]
Revert "[PATCH] kernel: kill irq save/restore"
This reverts
68bf440bf36d239ff3cb27b34b95ea7911c7c5a6 commit.
Jens Axboe [Fri, 23 Sep 2005 15:24:04 +0000 (17:24 +0200)]
[PATCH] kernel: kill irq save/restore
It should not be needed anymore, the writes a per-CPU and the
get_cpu()/put_cpu() will prevent preemption.
Jens Axboe [Fri, 23 Sep 2005 15:21:11 +0000 (17:21 +0200)]
[PATCH] kernel: avoid trace corruption on SMP
Maintain a static trace structure per-CPU.
Jens Axboe [Fri, 23 Sep 2005 14:54:56 +0000 (16:54 +0200)]
[PATCH] blkparse: sequence fixes
Only set last sequence _after_ a potential break of the loop.
And remember to set last_allowed_time for stdin reads as well.
Jens Axboe [Fri, 23 Sep 2005 14:45:51 +0000 (16:45 +0200)]
[PATCH] blkparse: fix sorting of file events regardless of batch size
If the batch wraps, it would be possible to start displaying events
way into the future if parsing multiple devices.
Jens Axboe [Fri, 23 Sep 2005 13:09:37 +0000 (15:09 +0200)]
Revert "[PATCH] blkparse: rbtree sort primary key should be sequence, not time"
This reverts
acba447a946b022df6a30159425309f033e46ab7 commit.
Jens Axboe [Fri, 23 Sep 2005 09:46:40 +0000 (11:46 +0200)]
[PATCH] blktrace: Add option for setting sub buffer size
Use -b to give sub buffer size in KiB, -n to give number of
sub buffers wanted. This affects the relayfs trace setup.
Jens Axboe [Fri, 23 Sep 2005 09:39:48 +0000 (11:39 +0200)]
[PATCH] blkparse: rbtree sort primary key should be sequence, not time
Otherwise we may lose events if the event time is a little out
of sync.
Jens Axboe [Fri, 23 Sep 2005 09:12:48 +0000 (11:12 +0200)]
[PATCH] blkparse: can't return on-stack variables
Just make it static, access is serialized to it.
Jens Axboe [Fri, 23 Sep 2005 08:07:54 +0000 (10:07 +0200)]
[PATCH] README: add link to mailing list at vger
Jens Axboe [Fri, 23 Sep 2005 06:49:22 +0000 (08:49 +0200)]
[PATCH] blkparse: don't append .log to the -o given output file
Nathan Scott [Fri, 23 Sep 2005 06:34:23 +0000 (08:34 +0200)]
[PATCH] blkparse: make the nfiles counter live inside pdi
Gets rid of the extra malloc and is cleaner.
Nathan Scott [Fri, 23 Sep 2005 06:33:20 +0000 (08:33 +0200)]
[PATCH] blkparse: fix pointer cast warnings
Alan D. Brunelle [Thu, 22 Sep 2005 18:06:53 +0000 (20:06 +0200)]
[PATCH] blkparse: correctly account number of files per device
Jens Axboe [Thu, 22 Sep 2005 18:02:41 +0000 (20:02 +0200)]
Merge refs/heads/master from origin
Jens Axboe [Thu, 22 Sep 2005 18:02:30 +0000 (20:02 +0200)]
[PATCH] blkparse: more optimizations
The sort path unification and memory cutback introduced some
overhead, mainly due to lots of memory allocations and frees
of the same structures over and over again. Add a frontend
cache for struct trace and blk_io_trace allocations.
Also concatenate a lot of the string operations in fmt_select,
it's quite high in the profiles.
This brings us to the same performance as the memory hog version.
Jens Axboe [Thu, 22 Sep 2005 14:47:40 +0000 (16:47 +0200)]
[PATCH] kernel: export the __blk_add_trace() symbol for modules
Jens Axboe [Thu, 22 Sep 2005 14:43:52 +0000 (16:43 +0200)]
Merge refs/heads/master from origin
Jens Axboe [Thu, 22 Sep 2005 14:43:48 +0000 (16:43 +0200)]
[PATCH] blkparse: drastically reduce memory foot print
File reads needed enough memory to hold all events from all CPUs.
This quickly grew to hundreds of megabytes. Bring down this number
by interleaving reads from the files and relying on the same
mechanism as the piped reads to syncronize them. This makes file reads
use the same path as pipes and actually removes more code than it adds.
For a 900,000 entry file, memory consumption here is down from ~110MiB
from about 1MiB to process it.
Jens Axboe [Thu, 22 Sep 2005 14:40:44 +0000 (16:40 +0200)]
Revert "[PATCH] blkparse: bad ncpus check"
This reverts
08d83c704fbda722d3040d643665e2c029744aa7 commit.
Jens Axboe [Thu, 22 Sep 2005 14:15:45 +0000 (16:15 +0200)]
[PATCH] blkparse: bad ncpus check
Jens Axboe [Thu, 22 Sep 2005 12:29:50 +0000 (14:29 +0200)]
[PATCH] blkparse: shave 20% off the runtime
Various optimizations and moving code around a little.
Jens Axboe [Thu, 22 Sep 2005 11:33:55 +0000 (13:33 +0200)]
[PATCH] Add local master repo to documentation
Note that it is using the git daemon, not rsync.
Jens Axboe [Thu, 22 Sep 2005 10:30:12 +0000 (12:30 +0200)]
[PATCH] Update in-program help/usage info
Jens Axboe [Thu, 22 Sep 2005 10:11:08 +0000 (12:11 +0200)]
[PATCH] blkparse: Fix stopwatch
The option should be start:end, not start:duration (at least that is
what I intended and what is documented). Also fix the actual dumping,
we need to prune earlier entries from the rb tree or we will loop
forever.
Jens Axboe [Thu, 22 Sep 2005 08:56:05 +0000 (10:56 +0200)]
[PATCH] blkparse: more PC fixes
Add size of request as well, kill space at the end of pdu dump.