Optimize pattern verify

Similar to the patch last week, this optimizes the pattern verify
operation to use optimized library calls like memcmp(), and only fall
back to byte-by-byte if there is a miscompare to locate it.

This uses the same premise that the pattern is repeated as many times
as possible to do large compares in a single call.  For single byte
pattern, the setup fills the pattern space, and verify assumes it is
full.

Tested by running a script which created an 8k file with 4k bs and,
one byte at a time, tried corrupting the pattern and running a read
pass and verified it still found miscompares across the whole range of
the pattern.  This was done with a pattern length of 1 and 3 bytes.

In performance tests, this was about 8 times more efficient on
verifies than without this patch.

Signed-off-by: Jens Axboe <axboe@kernel.dk>

diff --git a/options.c b/options.c
index 8fb93ca7a0c2c6a331a75e5250c86f0d9e37e156..c3fdb56314a5007c00bab2bdee57817816c9e9ee 100644 (file)
--- a/options.c
+++ b/options.c
@@ -698,6 +698,14 @@ static int str_verify_pattern_cb(void *data, const char *input)
                memcpy(&td->o.verify_pattern[i], &td->o.verify_pattern[0], i);
                i *= 2;
        }
+       if (i == 1) {
+               /*
+                * The code in verify_io_u_pattern assumes a single byte pattern
+                * fills the whole verify pattern buffer.
+                */
+               memset(td->o.verify_pattern, td->o.verify_pattern[0],
+                      MAX_PATTERN_SIZE);
+       }
 
        td->o.verify_pattern_bytes = i;
 

diff --git a/verify.c b/verify.c
index 9ee3bc42ca215961b03951445fdb64070b3f35aa..eb8eddc78906f4b0dcaa13a9899495e5a8670dca 100644 (file)
--- a/verify.c
+++ b/verify.c
@@ -322,14 +322,27 @@ static int verify_io_u_pattern(struct verify_header *hdr, struct vcont *vc)
        struct io_u *io_u = vc->io_u;
        char *buf, *pattern;
        unsigned int header_size = __hdr_size(td->o.verify);
-       unsigned int len, mod, i;
+       unsigned int len, mod, i, size, pattern_size;
 
        pattern = td->o.verify_pattern;
+       pattern_size = td->o.verify_pattern_bytes;
+       if (pattern_size <= 1)
+               pattern_size = MAX_PATTERN_SIZE;
        buf = (void *) hdr + header_size;
        len = get_hdr_inc(td, io_u) - header_size;
-       mod = header_size % td->o.verify_pattern_bytes;
+       mod = header_size % pattern_size;
+
+       for (i = 0; i < len; i += size) {
+               size = pattern_size - mod;
+               if (size > (len - i))
+                       size = len - i;
+               if (memcmp(buf + i, pattern + mod, size))
+                       // Let the slow compare find the first mismatch byte.
+                       break;
+               mod = 0;
+       }
 
-       for (i = 0; i < len; i++) {
+       for (; i < len; i++) {
                if (buf[i] != pattern[mod]) {
                        unsigned int bits;