backend: fix final fsync behaviour

backend: fix final fsync behaviour

Previously, fsync_on_close's "final" fsync was done after fio's
accounting had finished thus causing the bandwidth reported to be too
high (due to the reported time being too low). An example job that
demonstrates the issue (on machines with a few gigabytes of RAM and
non-fast disks) is the following:

./fio --gtod_reduce=1 --filename=fio.tmp --size=5G --bs=2M --rw=write \
 --stonewall --name=end_fsync --end_fsync=1 --name=fsync_on_close \
 --fsync_on_close=1
end_fsync: (g=0): rw=write, bs=(R) 2048KiB-2048KiB, (W) 2048KiB-2048KiB, (T) 2048KiB-2048KiB, ioengine=psync, iodepth=1
fsync_on_close: (g=1): rw=write, bs=(R) 2048KiB-2048KiB, (W) 2048KiB-2048KiB, (T) 2048KiB-2048KiB, ioengine=psync, iodepth=1
[...]
Run status group 0 (all jobs):
  WRITE: bw=381MiB/s (400MB/s), 381MiB/s-381MiB/s (400MB/s-400MB/s), io=5120MiB (5369MB), run=13424-13424msec

Run status group 1 (all jobs):
  WRITE: bw=1726MiB/s (1810MB/s), 1726MiB/s-1726MiB/s (1810MB/s-1810MB/s), io=5120MiB (5369MB), run=2966-2966msec

This patch fixes the issue by doing an fsync for fsync_on_close at the
same point as it does for end_fsync. Technically fsync_on_close will go
and do another fsync after this point too but this should be harmless as
there is no new write data.

This patch also fixes the assert seen with the following job

./fio --size=8k --io_limit=32k --filename=fio.tmp --rw=write \
 --end_fsync=1 --name=extended_end_fsync
[...]
fio: filesetup.c:1682: void get_file(struct fio_file *): Assertion `fio_file_open(f)' failed.

by preventing auto-close on last I/O when it is being submitted as a
final fsync (due to end_fsync, fsync_on_close).

Finally it also fixes the strange IO depths seen with when using end_fsync:

./fio --size=4k --filename=fio.tmp --rw=write --end_fsync=1 \
 --name=end_fsync_test
end_fsync_test: (g=0): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
[...]
  IO depths    : 1=200.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%

by no longer including such fsyncs in accounting.

Fixes: https://github.com/axboe/fio/issues/831

Signed-off-by: Sitsofe Wheeler <sitsofe@yahoo.com>