From 3e10fb832645e3ab3ef006f589f0459dc567cb53 Mon Sep 17 00:00:00 2001 From: Jens Axboe Date: Fri, 9 Aug 2013 14:31:06 -0600 Subject: [PATCH] verify: fix problem with hole punching on newer Linux kernels Jeff Moyer writes: Our QE team noticed fio failures on recent kernels. I simplified the job file to the following: [global] name=fio-mmap rw=write bs=4K direct=1 end_fsync=1 verify=crc32 [file3] size=100M ioengine=mmap mem=malloc direct=1 After fio completes (and returns verify errors), the file is completely full of zeroes. Here is what the verify logic is doing: static void do_verify(struct thread_data *td, uint64_t verify_bytes) { ... /* * sync io first and invalidate cache, to make sure we really * read from disk. */ for_each_file(td, f, i) { if (!fio_file_open(f)) continue; if (fio_io_sync(td, f)) break; if (file_invalidate_cache(td, f)) <-------- break; } That invalidate cache call looks like so: static int __file_invalidate_cache(struct thread_data *td, struct fio_file *f, unsigned long long off, unsigned long long len) { ... /* * FIXME: add blockdev flushing too */ if (f->mmap_ptr) { ret = posix_madvise(f->mmap_ptr, f->mmap_sz, POSIX_MADV_DONTNEED); (void) posix_madvise(f->mmap_ptr, f->mmap_sz, FIO_MADV_FREE); <------- FIO_MADV_FREE can be defined as MADV_REMOVE, which will actually punch a hole in the file (a hole the size of the entire file, btw). Of course, unallocated blocks are returned as zeroes by the file system, so that explains that! --- For now, only do the MADV_FREE on non-files to avoid punching holes. Further investigation is needed on the blockdev side, but at least this should cure the immediate problem. Reported-by: Jeff Moyer Signed-off-by: Jens Axboe --- filesetup.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/filesetup.c b/filesetup.c index 6427f3e4..7d3a0613 100644 --- a/filesetup.c +++ b/filesetup.c @@ -388,7 +388,8 @@ static int __file_invalidate_cache(struct thread_data *td, struct fio_file *f, if (f->mmap_ptr) { ret = posix_madvise(f->mmap_ptr, f->mmap_sz, POSIX_MADV_DONTNEED); #ifdef FIO_MADV_FREE - (void) posix_madvise(f->mmap_ptr, f->mmap_sz, FIO_MADV_FREE); + if (f->filetype == FIO_TYPE_BD) + (void) posix_madvise(f->mmap_ptr, f->mmap_sz, FIO_MADV_FREE); #endif } else if (f->filetype == FIO_TYPE_FILE) { ret = posix_fadvise(f->fd, off, len, POSIX_FADV_DONTNEED); -- 2.25.1