os: Reintroduce atomic write support

os: Reintroduce atomic write support

Previously O_ATOMIC support was added in commit d01612f3ae25 ("Add support
for O_ATOMIC"). But support was removed in commit a25ba6c64fe1 ("Get rid of
O_ATOMIC"), as support was never added in the Linux kernel.

Linux kernel 6.11 will add support for RWF_ATOMIC, which can be supported
for various ioengines. See latest man pages for details.

The plumbing was left in place for thread option oatomic, so that will be
reused.

Add a flag to say whether an engine supports atomic writes, and reject
when oatomic is set for an engine which does not support atomic writes.

This is a change in behaviour, as since commit a25ba6c64fe1 ("Get rid of
O_ATOMIC"), this oatomic has been ignored. However, it is better to tell
the user that their ioengine of choice does not support atomic writes.

Today RWF_ATOMIC is only supported for direct-IO. In future it may be
supported for buffered IO. As such, do not auto-set odirect=1 when
oatomic==1.

Signed-off-by: John Garry <john.g.garry@oracle.com>
Link: https://lore.kernel.org/r/20240916165347.2226763-3-john.g.garry@oracle.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>