linux-2.6-block.git
5 months agoxfs: check parent pointer xattrs when scrubbing
Darrick J. Wong [Mon, 22 Apr 2024 16:48:05 +0000 (09:48 -0700)]
xfs: check parent pointer xattrs when scrubbing

Check parent pointer xattrs as part of scrubbing xattrs.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: walk directory parent pointers to determine backref count
Darrick J. Wong [Mon, 22 Apr 2024 16:48:05 +0000 (09:48 -0700)]
xfs: walk directory parent pointers to determine backref count

If the filesystem has parent pointers enabled, walk the parent pointers
of subdirectories to determine the true backref count.  In theory each
subdir should have a single parent reachable via dotdot, but in the case
of (corrupt) subdirs with multiple parents, we need to keep the link
counts high enough that the directory loop detector will be able to
correct the multiple parents problems.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: deferred scrub of parent pointers
Darrick J. Wong [Mon, 22 Apr 2024 16:48:04 +0000 (09:48 -0700)]
xfs: deferred scrub of parent pointers

If the trylock-based dirent check fails, retain those parent pointers
and check them at the end.  This may involve dropping the locks on the
file being scanned, so yay.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: scrub parent pointers
Darrick J. Wong [Mon, 22 Apr 2024 16:48:03 +0000 (09:48 -0700)]
xfs: scrub parent pointers

Actually check parent pointers now.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: deferred scrub of dirents
Darrick J. Wong [Mon, 22 Apr 2024 16:48:02 +0000 (09:48 -0700)]
xfs: deferred scrub of dirents

If the trylock-based parent pointer check fails, retain those dirents
and check them at the end.  This may involve dropping the locks on the
file being scanned, so yay.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: check dirents have parent pointers
Darrick J. Wong [Mon, 22 Apr 2024 16:48:01 +0000 (09:48 -0700)]
xfs: check dirents have parent pointers

If the fs has parent pointers, we need to check that each child dirent
points to a file that has a parent pointer pointing back at us.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: revert commit 44af6c7e59b12
Darrick J. Wong [Mon, 22 Apr 2024 16:48:00 +0000 (09:48 -0700)]
xfs: revert commit 44af6c7e59b12

In my haste to fix what I thought was a performance problem in the attr
scrub code, I neglected to notice that the xfs_attr_get_ilocked also had
the effect of checking that attributes can actually be looked up through
the attr dabtree.  Fix this.

Fixes: 44af6c7e59b12 ("xfs: don't load local xattr values during scrub")
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: enable parent pointers
Darrick J. Wong [Mon, 22 Apr 2024 16:48:00 +0000 (09:48 -0700)]
xfs: enable parent pointers

Add parent pointers to the list of supported features.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: drop compatibility minimum log size computations for reflink
Darrick J. Wong [Mon, 22 Apr 2024 16:47:59 +0000 (09:47 -0700)]
xfs: drop compatibility minimum log size computations for reflink

Let's also drop the oversized minimum log computations for reflink and
rmap that were the result of bugs introduced many years ago.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Allison Henderson <allison.henderson@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: fix unit conversion error in xfs_log_calc_max_attrsetm_res
Darrick J. Wong [Mon, 22 Apr 2024 16:47:58 +0000 (09:47 -0700)]
xfs: fix unit conversion error in xfs_log_calc_max_attrsetm_res

Dave and I were discussing some recent test regressions as a result of
me turning on nrext64=1 on realtime filesystems, when we noticed that
the minimum log size of a 32M filesystem jumped from 954 blocks to 4287
blocks.

Digging through xfs_log_calc_max_attrsetm_res, Dave noticed that @size
contains the maximum estimated amount of space needed for a local format
xattr, in bytes, but we feed this quantity to XFS_NEXTENTADD_SPACE_RES,
which requires units of blocks.  This has resulted in an overestimation
of the minimum log size over the years.

We should nominally correct this, but there's a backwards compatibility
problem -- if we enable it now, the minimum log size will decrease.  If
a corrected mkfs formats a filesystem with this new smaller log size, a
user will encounter mount failures on an uncorrected kernel due to the
larger minimum log size computations there.

Therefore, turn this on for parent pointers because it wasn't merged at
all upstream when this issue was discovered.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Allison Henderson <allison.henderson@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: add a incompat feature bit for parent pointers
Allison Henderson [Mon, 22 Apr 2024 16:47:57 +0000 (09:47 -0700)]
xfs: add a incompat feature bit for parent pointers

Create an incompat feature bit and a fs geometry flag so that we can
enable the feature in the ondisk superblock and advertise its existence
to userspace.

Signed-off-by: Mark Tinguely <mark.tinguely@oracle.com>
Signed-off-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Allison Henderson <allison.henderson@oracle.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
5 months agoxfs: don't remove the attr fork when parent pointers are enabled
Allison Henderson [Mon, 22 Apr 2024 16:47:56 +0000 (09:47 -0700)]
xfs: don't remove the attr fork when parent pointers are enabled

When an inode is removed, it may also cause the attribute fork to be
removed if it is the last attribute. This transaction gets flushed to
the log, but if the system goes down before we could inactivate the symlink,
the log recovery tries to inactivate this inode (since it is on the unlinked
list) but the verifier trips over the remote value and leaks it.

Hence we ended up with a file in this odd state on a "clean" mount.  The
"obvious" fix is to prohibit erasure of the attr fork to avoid tripping
over the verifiers when pptrs are enabled.

Signed-off-by: Allison Henderson <allison.henderson@oracle.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: add parent pointer ioctls
Darrick J. Wong [Mon, 22 Apr 2024 16:47:55 +0000 (09:47 -0700)]
xfs: add parent pointer ioctls

This patch adds a pair of new file ioctls to retrieve the parent pointer
of a given inode.  They both return the same results, but one operates
on the file descriptor passed to ioctl() whereas the other allows the
caller to specify a file handle for which the caller wants results.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: split out handle management helpers a bit
Darrick J. Wong [Mon, 22 Apr 2024 16:47:55 +0000 (09:47 -0700)]
xfs: split out handle management helpers a bit

Split out the functions that generate file/fs handles and map them back
into dentries in preparation for the GETPARENTS ioctl next.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: move handle ioctl code to xfs_handle.c
Darrick J. Wong [Mon, 22 Apr 2024 16:47:54 +0000 (09:47 -0700)]
xfs: move handle ioctl code to xfs_handle.c

Move the handle managemnet code (and the attrmulti code that uses it) to
xfs_handle.c.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: pass the attr value to put_listent when possible
Allison Henderson [Mon, 22 Apr 2024 16:47:53 +0000 (09:47 -0700)]
xfs: pass the attr value to put_listent when possible

Pass the attr value to put_listent when we have local xattrs or
shortform xattrs.  This will enable the GETPARENTS ioctl to use
xfs_attr_list as its backend.

Signed-off-by: Allison Henderson <allison.henderson@oracle.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: don't return XFS_ATTR_PARENT attributes via listxattr
Allison Henderson [Mon, 22 Apr 2024 16:47:52 +0000 (09:47 -0700)]
xfs: don't return XFS_ATTR_PARENT attributes via listxattr

Parent pointers are internal filesystem metadata.  They're not intended
to be directly visible to userspace, so filter them out of
xfs_xattr_put_listent so that they don't appear in listxattr.

Signed-off-by: Allison Henderson <allison.henderson@oracle.com>
Inspired-by: Andrey Albershteyn <aalbersh@redhat.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
[djwong: change this to XFS_ATTR_PRIVATE_NSP_MASK per fsverity patchset]
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: Add parent pointers to xfs_cross_rename
Allison Henderson [Mon, 22 Apr 2024 16:47:51 +0000 (09:47 -0700)]
xfs: Add parent pointers to xfs_cross_rename

Cross renames are handled separately from standard renames, and
need different handling to update the parent attributes correctly.

Signed-off-by: Allison Henderson <allison.henderson@oracle.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: Add parent pointers to rename
Allison Henderson [Mon, 22 Apr 2024 16:47:50 +0000 (09:47 -0700)]
xfs: Add parent pointers to rename

This patch removes the old parent pointer attribute during the rename
operation, and re-adds the updated parent pointer.

Signed-off-by: Allison Henderson <allison.henderson@oracle.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
[djwong: adjust to new ondisk format]
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: remove parent pointers in unlink
Allison Henderson [Mon, 22 Apr 2024 16:47:49 +0000 (09:47 -0700)]
xfs: remove parent pointers in unlink

This patch removes the parent pointer attribute during unlink

Signed-off-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Allison Henderson <allison.henderson@oracle.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
[djwong: adjust to new ondisk format, minor rebase fixes]
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: add parent attributes to symlink
Allison Henderson [Mon, 22 Apr 2024 16:47:49 +0000 (09:47 -0700)]
xfs: add parent attributes to symlink

This patch modifies xfs_symlink to add a parent pointer to the inode.

Signed-off-by: Allison Henderson <allison.henderson@oracle.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
[djwong: minor rebase fixups]
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: add parent attributes to link
Allison Henderson [Mon, 22 Apr 2024 16:47:48 +0000 (09:47 -0700)]
xfs: add parent attributes to link

This patch modifies xfs_link to add a parent pointer to the inode.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Allison Henderson <allison.henderson@oracle.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
[djwong: minor rebase fixes]
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: parent pointer attribute creation
Allison Henderson [Mon, 22 Apr 2024 16:47:47 +0000 (09:47 -0700)]
xfs: parent pointer attribute creation

Add parent pointer attribute during xfs_create, and subroutines to
initialize attributes.  Note that the xfs_attr_intent object contains a
pointer to the caller's xfs_da_args object, so the latter must persist
until transaction commit.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Allison Henderson <allison.henderson@oracle.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
[djwong: shorten names, adjust to new format, set init_xattrs for parent
pointers]
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: create a hashname function for parent pointers
Darrick J. Wong [Mon, 22 Apr 2024 16:47:46 +0000 (09:47 -0700)]
xfs: create a hashname function for parent pointers

Although directory entry and parent pointer recordsets look very similar
(name -> ino), there's one major difference between them: a file can be
hardlinked from multiple parent directories with the same filename.
This is common in shared container environments where a base directory
tree might be hardlink-copied multiple times.  IOWs the same 'ls'
program might be hardlinked to multiple /srv/*/bin/ls paths.

We don't want parent pointer operations to bog down on hash collisions
between the same dirent name, so create a special hash function that
mixes in the parent directory inode number.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: extend transaction reservations for parent attributes
Allison Henderson [Mon, 22 Apr 2024 16:47:45 +0000 (09:47 -0700)]
xfs: extend transaction reservations for parent attributes

We need to add, remove or modify parent pointer attributes during
create/link/unlink/rename operations atomically with the dirents in the
parent directories being modified. This means they need to be modified
in the same transaction as the parent directories, and so we need to add
the required space for the attribute modifications to the transaction
reservations.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Allison Henderson <allison.henderson@oracle.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
[djwong: fix indenting errors, adjust for new log format]
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: add parent pointer validator functions
Allison Henderson [Mon, 22 Apr 2024 16:47:44 +0000 (09:47 -0700)]
xfs: add parent pointer validator functions

The attr name of a parent pointer is a string, and the attr value of a
parent pointer is (more or less) a file handle.  So we need to modify
attr_namecheck to verify the parent pointer name, and add a
xfs_parent_valuecheck function to sanitize the handle.  At the same
time, we need to validate attr values during log recovery if the xattr
is really a parent pointer.

Signed-off-by: Allison Henderson <allison.henderson@oracle.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
[djwong: move functions to xfs_parent.c, adjust for new disk format]
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: Expose init_xattrs in xfs_create_tmpfile
Allison Henderson [Mon, 22 Apr 2024 16:47:43 +0000 (09:47 -0700)]
xfs: Expose init_xattrs in xfs_create_tmpfile

Tmp files are used as part of rename operations and will need attr forks
initialized for parent pointers.  Expose the init_xattrs parameter to
the calling function to initialize the fork.

Signed-off-by: Allison Henderson <allison.henderson@oracle.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: record inode generation in xattr update log intent items
Darrick J. Wong [Mon, 22 Apr 2024 16:47:43 +0000 (09:47 -0700)]
xfs: record inode generation in xattr update log intent items

For parent pointer updates, record the i_generation of the file that is
being updated so that we don't accidentally jump generations.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: create attr log item opcodes and formats for parent pointers
Darrick J. Wong [Mon, 22 Apr 2024 16:47:42 +0000 (09:47 -0700)]
xfs: create attr log item opcodes and formats for parent pointers

Make the necessary alterations to the extended attribute log intent item
ondisk format so that we can log parent pointer operations.  This
requires the creation of new opcodes specific to parent pointers, and a
new four-argument replace operation to handle renames.  At this point
this part of the patchset has changed so much from what Allison original
wrote that I no longer think her SoB applies.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: refactor xfs_is_using_logged_xattrs checks in attr item recovery
Darrick J. Wong [Mon, 22 Apr 2024 16:47:41 +0000 (09:47 -0700)]
xfs: refactor xfs_is_using_logged_xattrs checks in attr item recovery

Move this feature check down to the per-op checks so that we can ensure
that we never see parent pointer attr items on non-pptr filesystems, and
that logged xattrs are turned on for non-pptr attr items.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: allow xattr matching on name and value for parent pointers
Darrick J. Wong [Mon, 22 Apr 2024 16:47:40 +0000 (09:47 -0700)]
xfs: allow xattr matching on name and value for parent pointers

If a file is hardlinked with the same name but from multiple parents,
the parent pointers will all have the same dirent name (== attr name)
but with different parent_ino/parent_gen values.  To disambiguate, we
need to be able to match on both the attr name and the attr value.  This
is in contrast to regular xattrs, which are matchtg edit
d only on name.

Therefore, plumb in the ability to match shortform and local attrs on
name and value in the XFS_ATTR_PARENT namespace.  Parent pointer attr
values are never large enough to be stored in a remote attr, so we need
can reject these cases as corruption.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: define parent pointer ondisk extended attribute format
Allison Henderson [Mon, 22 Apr 2024 16:47:39 +0000 (09:47 -0700)]
xfs: define parent pointer ondisk extended attribute format

We need to define the parent pointer attribute format before we start
adding support for it into all the code that needs to use it. The EA
format we will use encodes the following information:

        name={dirent name}
        value={parent inumber, parent inode generation}
        hash=xfs_dir2_hashname(dirent name) ^ (parent_inumber)

The inode/gen gives all the information we need to reliably identify the
parent without requiring child->parent lock ordering, and allows
userspace to do pathname component level reconstruction without the
kernel ever needing to verify the parent itself as part of ioctl calls.

By using the name-value lookup mode in the extended attribute code to
match parent pointers using both the xattr name and value, we can
identify the exact parent pointer EA we need to modify/remove in
rename/unlink operations without searching the entire EA space.

By storing the dirent name, we have enough information to be able to
validate and reconstruct damaged directory trees.  Earlier iterations of
this patchset encoded the directory offset in the parent pointer key,
but this format required repair to keep that in sync across directory
rebuilds, which is unnecessary complexity.

Signed-off-by: Allison Henderson <allison.henderson@oracle.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: add parent pointer support to attribute code
Allison Henderson [Mon, 22 Apr 2024 16:47:38 +0000 (09:47 -0700)]
xfs: add parent pointer support to attribute code

Add the new parent attribute type. XFS_ATTR_PARENT is used only for parent pointer
entries; it uses reserved blocks like XFS_ATTR_ROOT.

Signed-off-by: Mark Tinguely <mark.tinguely@oracle.com>
Signed-off-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Allison Henderson <allison.henderson@oracle.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: create a separate hashname function for extended attributes
Darrick J. Wong [Mon, 22 Apr 2024 16:47:38 +0000 (09:47 -0700)]
xfs: create a separate hashname function for extended attributes

Create a separate function to compute name hashvalues for extended
attributes.  When we get to parent pointers we'll be altering the rules
so that metadump obfuscation doesn't turn heinous.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: move xfs_attr_defer_add to xfs_attr_item.c
Darrick J. Wong [Mon, 22 Apr 2024 16:47:37 +0000 (09:47 -0700)]
xfs: move xfs_attr_defer_add to xfs_attr_item.c

Move the code that adds the incore xfs_attr_item deferred work data to a
transaction live with the ATTRI log item code.  This means that the
upper level extended attribute code no longer has to know about the
inner workings of the ATTRI log items.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: check the flags earlier in xfs_attr_match
Christoph Hellwig [Mon, 22 Apr 2024 16:47:36 +0000 (09:47 -0700)]
xfs: check the flags earlier in xfs_attr_match

Checking the flags match is much cheaper than a memcmp, so do it early
on in xfs_attr_match, and also add a little helper to calculate the
match mask right under the comment explaining the logic for it.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
5 months agoxfs: rearrange xfs_attr_match parameters
Darrick J. Wong [Mon, 22 Apr 2024 16:47:35 +0000 (09:47 -0700)]
xfs: rearrange xfs_attr_match parameters

Rearrange the parameters to this function so that they match the order
of attr listent: attr_flags -> name -> namelen -> value -> valuelen.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: enforce one namespace per attribute
Darrick J. Wong [Mon, 22 Apr 2024 16:47:34 +0000 (09:47 -0700)]
xfs: enforce one namespace per attribute

Create a standardized helper function to enforce one namespace bit per
extended attribute, and refactor all the open-coded hweight logic.  This
function is not a static inline to avoid porting hassles in userspace.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: refactor name/value iovec validation in xlog_recover_attri_commit_pass2
Darrick J. Wong [Mon, 22 Apr 2024 16:47:33 +0000 (09:47 -0700)]
xfs: refactor name/value iovec validation in xlog_recover_attri_commit_pass2

Hoist the code that checks the attr name and value iovecs into separate
helpers so that we can add more callsites for the new parent pointer
attr intent items.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: refactor name/length checks in xfs_attri_validate
Darrick J. Wong [Mon, 22 Apr 2024 16:47:33 +0000 (09:47 -0700)]
xfs: refactor name/length checks in xfs_attri_validate

Move the name and length checks into the attr op switch statement so
that we can perform more specific checks of the value length.  Over the
next few patches we're going to add new attr op flags with different
validation requirements.

While we're at it, remove the incorrect comment.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: use local variables for name and value length in _attri_commit_pass2
Darrick J. Wong [Mon, 22 Apr 2024 16:47:32 +0000 (09:47 -0700)]
xfs: use local variables for name and value length in _attri_commit_pass2

We're about to start using tagged unions in the xattr log format, so
create a bunch of local variables in the recovery function so we only
have to decode the log item fields once.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: always set args->value in xfs_attri_item_recover
Darrick J. Wong [Mon, 22 Apr 2024 16:47:31 +0000 (09:47 -0700)]
xfs: always set args->value in xfs_attri_item_recover

Always set args->value to the recovered value buffer.  This reduces the
amount of code in the switch statement, and hence the amount of thinking
that I have to do.  We validated the recovered buffers, supposedly.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: validate recovered name buffers when recovering xattr items
Darrick J. Wong [Mon, 22 Apr 2024 16:47:30 +0000 (09:47 -0700)]
xfs: validate recovered name buffers when recovering xattr items

Strengthen the xattri log item recovery code by checking that we
actually have the required name and newname buffers for whatever
operation we're replaying.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: use helpers to extract xattr op from opflags
Darrick J. Wong [Mon, 22 Apr 2024 16:47:29 +0000 (09:47 -0700)]
xfs: use helpers to extract xattr op from opflags

Create helper functions to extract the xattr op from the ondisk xattri
log item and the incore attr intent item.  These will get more use in
the patches that follow.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: restructure xfs_attr_complete_op a bit
Darrick J. Wong [Mon, 22 Apr 2024 16:47:29 +0000 (09:47 -0700)]
xfs: restructure xfs_attr_complete_op a bit

Eliminate the local variable from this function so that we can
streamline things a bit later when we add the PPTR_REPLACE op code.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: check shortform attr entry flags specifically
Darrick J. Wong [Mon, 22 Apr 2024 16:47:28 +0000 (09:47 -0700)]
xfs: check shortform attr entry flags specifically

While reviewing flag checking in the attr scrub functions, we noticed
that the shortform attr scanner didn't catch entries that have the LOCAL
or INCOMPLETE bits set.  Neither of these flags can ever be set on a
shortform attr, so we need to check this narrower set of valid flags.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: fix missing check for invalid attr flags
Darrick J. Wong [Mon, 22 Apr 2024 16:47:27 +0000 (09:47 -0700)]
xfs: fix missing check for invalid attr flags

The xattr scrubber doesn't check for undefined flags in shortform attr
entries.  Therefore, define a mask XFS_ATTR_ONDISK_MASK that has all
possible XFS_ATTR_* flags in it, and use that to check for unknown bits
in xchk_xattr_actor.

Refactor the check in the dabtree scanner function to use the new mask
as well.  The redundant checks need to be in place because the dabtree
check examines the hash mappings and therefore needs to decode the attr
leaf entries to compute the namehash.  This happens before the walk of
the xattr entries themselves.

Fixes: ae0506eba78fd ("xfs: check used space of shortform xattr structures")
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: check opcode and iovec count match in xlog_recover_attri_commit_pass2
Darrick J. Wong [Mon, 22 Apr 2024 16:47:26 +0000 (09:47 -0700)]
xfs: check opcode and iovec count match in xlog_recover_attri_commit_pass2

Check that the number of recovered log iovecs is what is expected for
the xattri opcode is expecting.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: use an XFS_OPSTATE_ flag for detecting if logged xattrs are available
Darrick J. Wong [Mon, 22 Apr 2024 16:47:25 +0000 (09:47 -0700)]
xfs: use an XFS_OPSTATE_ flag for detecting if logged xattrs are available

Per reviewer request, use an OPSTATE flag (+ helpers) to decide if
logged xattrs are enabled, instead of querying the xfs_sb.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: require XFS_SB_FEAT_INCOMPAT_LOG_XATTRS for attr log intent item recovery
Darrick J. Wong [Mon, 22 Apr 2024 16:47:24 +0000 (09:47 -0700)]
xfs: require XFS_SB_FEAT_INCOMPAT_LOG_XATTRS for attr log intent item recovery

The XFS_SB_FEAT_INCOMPAT_LOG_XATTRS feature bit protects a filesystem
from old kernels that do not know how to recover extended attribute log
intent items.  Make this check mandatory instead of a debugging assert.

Fixes: fd920008784ea ("xfs: Set up infrastructure for log attribute replay")
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: attr fork iext must be loaded before calling xfs_attr_is_leaf
Darrick J. Wong [Mon, 22 Apr 2024 16:47:24 +0000 (09:47 -0700)]
xfs: attr fork iext must be loaded before calling xfs_attr_is_leaf

Christoph noticed that the xfs_attr_is_leaf in xfs_attr_get_ilocked can
access the incore extent tree of the attr fork, but nothing in the
xfs_attr_get path guarantees that the incore tree is actually loaded.

Most of the time it is, but seeing as xfs_attr_is_leaf ignores the
return value of xfs_iext_get_extent I guess we've been making choices
based on random stack contents and nobody's complained?

Reported-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: rearrange xfs_da_args a bit to use less space
Darrick J. Wong [Mon, 22 Apr 2024 16:47:23 +0000 (09:47 -0700)]
xfs: rearrange xfs_da_args a bit to use less space

A few notes about struct xfs_da_args:

The XFS_ATTR_* flags only go up as far as XFS_ATTR_INCOMPLETE, which
means that attr_filter could be a u8 field.

I've reduced the number of XFS_DA_OP_* flags down to the point where
op_flags would also fit into a u8.

filetype has 7 bytes of slack after it, which is wasteful.

namelen will never be greater than MAXNAMELEN, which is 256.  This field
could be reduced to a short.

Rearrange the fields in xfs_da_args to waste less space.  This reduces
the structure size from 136 bytes to 128.  Later when we add extra
fields to support parent pointer replacement, this will only bloat the
structure to 144 bytes, instead of 168.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: make attr removal an explicit operation
Darrick J. Wong [Mon, 22 Apr 2024 16:47:22 +0000 (09:47 -0700)]
xfs: make attr removal an explicit operation

Parent pointers match attrs on name+value, unlike everything else which
matches on only the name.  Therefore, we cannot keep using the heuristic
that !value means remove.  Make this an explicit operation code.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: remove xfs_da_args.attr_flags
Darrick J. Wong [Mon, 22 Apr 2024 16:47:21 +0000 (09:47 -0700)]
xfs: remove xfs_da_args.attr_flags

This field only ever contains XATTR_{CREATE,REPLACE}, and it only goes
as deep as xfs_attr_set.  Remove the field from the structure and
replace it with an enum specifying exactly what kind of change we want
to make to the xattr structure.  Upsert is the name that we'll give to
the flags==0 operation, because we're either updating an existing value
or inserting it, and the caller doesn't care.

Note: The "UPSERTR" name created here is to make userspace porting
easier.  It will be removed in the next patch.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: remove XFS_DA_OP_NOTIME
Darrick J. Wong [Mon, 22 Apr 2024 16:47:20 +0000 (09:47 -0700)]
xfs: remove XFS_DA_OP_NOTIME

The only user of this flag sets it prior to an xfs_attr_get_ilocked
call, which doesn't update anything.  Get rid of the flag.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: remove XFS_DA_OP_REMOVE
Darrick J. Wong [Mon, 22 Apr 2024 16:47:19 +0000 (09:47 -0700)]
xfs: remove XFS_DA_OP_REMOVE

Nobody checks this flag, so get rid of it.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: reinstate delalloc for RT inodes (if sb_rextsize == 1)
Christoph Hellwig [Mon, 22 Apr 2024 11:20:19 +0000 (13:20 +0200)]
xfs: reinstate delalloc for RT inodes (if sb_rextsize == 1)

Commit aff3a9edb708 ("xfs: Use preallocation for inodes with extsz
hints") disabled delayed allocation for all inodes with extent size
hints due a data exposure problem.  It turns out we fixed this data
exposure problem since by always creating unwritten extents for
delalloc conversions due to more data exposure problems, but the
writeback path doesn't actually support extent size hints when
converting delalloc these days, which probably isn't a problem given
that people using the hints know what they get.

However due to the way how xfs_get_extsz_hint is implemented, it
always claims an extent size hint for RT inodes even if the RT
extent size is a single FSB.  Due to that the above commit effectively
disabled delalloc support for RT inodes.

Switch xfs_get_extsz_hint to return 0 for this case and work around
that in a few places to reinstate delalloc support for RT inodes on
file systems with an sb_rextsize of 1.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandanbabu@kernel.org>
5 months agoxfs: stop the steal (of data blocks for RT indirect blocks)
Christoph Hellwig [Mon, 22 Apr 2024 11:20:18 +0000 (13:20 +0200)]
xfs: stop the steal (of data blocks for RT indirect blocks)

When xfs_bmap_del_extent_delay has to split an indirect block it tries
to steal blocks from the the part that gets unmapped to increase the
indirect block reservation that now needs to cover for two extents
instead of one.

This works perfectly fine on the data device, where the data and
indirect blocks come from the same pool.  It has no chance of working
when the inode sits on the RT device.  To support re-enabling delalloc
for inodes on the RT device, make this behavior conditional on not
being for rt extents.

Note that split of delalloc extents should only happen on writeback
failure, as for other kinds of hole punching we first write back all
data and thus convert the delalloc reservations covering the hole to
a real allocation.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandanbabu@kernel.org>
5 months agoxfs: rework splitting of indirect block reservations
Christoph Hellwig [Mon, 22 Apr 2024 11:20:17 +0000 (13:20 +0200)]
xfs: rework splitting of indirect block reservations

Move the check if we have enough indirect blocks and the stealing of
the deleted extent blocks out of xfs_bmap_split_indlen and into the
caller to prepare for handling delayed allocation of RT extents that
can't easily be stolen.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandanbabu@kernel.org>
5 months agoxfs: look at m_frextents in xfs_iomap_prealloc_size for RT allocations
Christoph Hellwig [Mon, 22 Apr 2024 11:20:16 +0000 (13:20 +0200)]
xfs: look at m_frextents in xfs_iomap_prealloc_size for RT allocations

Add a check for files on the RT subvolume and use m_frextents instead
of m_fdblocks to adjust the preallocation size.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandanbabu@kernel.org>
5 months agoxfs: support RT inodes in xfs_mod_delalloc
Christoph Hellwig [Mon, 22 Apr 2024 11:20:15 +0000 (13:20 +0200)]
xfs: support RT inodes in xfs_mod_delalloc

To prepare for re-enabling delalloc on RT devices, track the data blocks
(which use the RT device when the inode sits on it) and the indirect
blocks (which don't) separately to xfs_mod_delalloc, and add a new
percpu counter to also track the RT delalloc blocks.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandanbabu@kernel.org>
5 months agoxfs: cleanup fdblock/frextent accounting in xfs_bmap_del_extent_delay
Christoph Hellwig [Mon, 22 Apr 2024 11:20:14 +0000 (13:20 +0200)]
xfs: cleanup fdblock/frextent accounting in xfs_bmap_del_extent_delay

The code to account fdblocks and frextents in xfs_bmap_del_extent_delay
is a bit weird in that it accounts frextents before the iext tree
manipulations and fdblocks after it.  Given that the iext tree
manipulations cannot fail currently that's not really a problem, but
still odd.  Move the frextent manipulation to the end, and use a
fdblocks variable to account of the unconditional indirect blocks and
the data blocks only freed for !RT.  This prepares for following
updates in the area and already makes the code more readable.

Also remove the !isrt assert given that this code clearly handles
rt extents correctly, and we'll soon reinstate delalloc support for
RT inodes.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandanbabu@kernel.org>
5 months agoxfs: reinstate RT support in xfs_bmapi_reserve_delalloc
Christoph Hellwig [Mon, 22 Apr 2024 11:20:13 +0000 (13:20 +0200)]
xfs: reinstate RT support in xfs_bmapi_reserve_delalloc

Allocate data blocks for RT inodes using xfs_dec_frextents.  While at
it optimize the data device case by doing only a single xfs_dec_fdblocks
call for the extent itself and the indirect blocks.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandanbabu@kernel.org>
5 months agoxfs: split xfs_mod_freecounter
Christoph Hellwig [Mon, 22 Apr 2024 11:20:12 +0000 (13:20 +0200)]
xfs: split xfs_mod_freecounter

xfs_mod_freecounter has two entirely separate code paths for adding or
subtracting from the free counters.  Only the subtract case looks at the
rsvd flag and can return an error.

Split xfs_mod_freecounter into separate helpers for subtracting or
adding the freecounter, and remove all the impossible to reach error
handling for the addition case.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandanbabu@kernel.org>
5 months agoxfs: block deltas in xfs_trans_unreserve_and_mod_sb must be positive
Christoph Hellwig [Mon, 22 Apr 2024 11:20:11 +0000 (13:20 +0200)]
xfs: block deltas in xfs_trans_unreserve_and_mod_sb must be positive

And to make that more clear, rearrange the code a bit and add asserts
and a comment.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandanbabu@kernel.org>
5 months agoxfs: move RT inode locking out of __xfs_bunmapi
Christoph Hellwig [Mon, 22 Apr 2024 11:20:10 +0000 (13:20 +0200)]
xfs: move RT inode locking out of __xfs_bunmapi

__xfs_bunmapi is a bit of an odd place to lock the rtbitmap and rtsummary
inodes given that it is very high level code.  While this only looks ugly
right now, it will become a problem when supporting delayed allocations
for RT inodes as __xfs_bunmapi might end up deleting only delalloc extents
and thus never unlock the rt inodes.

Move the locking into xfs_bmap_del_extent_real just before the call to
xfs_rtfree_blocks instead and use a new flag in the transaction to ensure
that the locking happens only once.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandanbabu@kernel.org>
5 months agoxfs: free RT extents after updating the bmap btree
Christoph Hellwig [Mon, 22 Apr 2024 11:20:09 +0000 (13:20 +0200)]
xfs: free RT extents after updating the bmap btree

Currently xfs_bmap_del_extent_real frees RT extents before updating
the bmap btree, while it frees regular blocks after performing the bmap
btree update for convoluted historic reasons.  Switch to free the RT
blocks in the same place as the regular data blocks instead to simply
the code and fix a very theoretical bug.

A short history of this code researched by Dave Chiner below:

The truncate for data device extents was originally a two-phase
operation. First it removed the bmapbt record, but because this can
free BMBT extents, it can use up all the free space tree reservation
space. So the transaction gets rolled to commit the BMBT change and
the xfs_bmap_finish() call that frees the data extent runs with a
new transaction reservation that allows different free space btrees
to be logged without overrun.

However, on crash, this could lose the free space because there was
nothing to tell recovery about the extents removed from the BMBT,
hence EFIs were introduced. They tie the extent free operation to the
bmapbt record removal commit for recovery of the second phase of the
extent removal process.

Then RT extents came along. RT extent freeing does not require a
free space btree reservation because the free space metadata is
static and transaction size is bound. Hence we don't need to care if
the BMBT record removal modifies the per-ag free space trees and we
don't need a two-phase extent remove transaction. The only thing we
have to care about is not losing space on crash.

Hence instead of recording the extent for freeing in the bmap list
for xfs_bmap_finish() to process in a new transaction, it simply
freed the rtextent directly. So the original code (from 1994) simply
replaced the "free AG extent later" queueing with a direct free.

This code was originally at the start of xfs_dmap_del_extent(), but
the xfs_bmap_add_free() got moved to the end of the function via the
"do_fx" flag (the current code logic) in 1997 (commit c4fac74eaa58
in the historic xfs-import tree) because there was a shutdown occurring
because of a case where splitting the extent record failed because the
BMBT split and the filesystem didn't have enough space for the split to
be done. (FWIW, I'm not sure this can happen anymore.)

The commit backed out the BMBT change on ENOSPC error, and in doing
so I think this actually breaks RT free space tracking. However, it
then returns an ENOSPC error, and we have a dirty transaction in the
RT case so this will shut down the filesysetm when the transaction
is cancelled. Hence the corrupted "bmbt now points at freed rt dev
space" condition never make it to disk, but it's still the wrong way
to handle the issue.

IOWs, this proposed change fixes that "shutdown at ENOSPC on rt
devices" situation that was introduced by the above commit back in
1997.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandanbabu@kernel.org>
5 months agoxfs: refactor realtime inode locking
Christoph Hellwig [Mon, 22 Apr 2024 11:20:08 +0000 (13:20 +0200)]
xfs: refactor realtime inode locking

Create helper functions to deal with locking realtime metadata inodes.
This enables us to maintain correct locking order once we start adding
the realtime rmap and refcount btree inodes.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Chandan Babu R <chandanbabu@kernel.org>
5 months agoxfs: make XFS_TRANS_LOWMODE match the other XFS_TRANS_ definitions
Christoph Hellwig [Mon, 22 Apr 2024 11:20:07 +0000 (13:20 +0200)]
xfs: make XFS_TRANS_LOWMODE match the other XFS_TRANS_ definitions

Commit bb7b1c9c5dd3 ("xfs: tag transactions that contain intent done
items") switched the XFS_TRANS_ definitions to be bit based, and using
comments above the definitions.  As XFS_TRANS_LOWMODE was last and has
a big fat comment it was missed.  Switch it to the same style.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandanbabu@kernel.org>
5 months agoxfs: compile out v4 support if disabled
Christoph Hellwig [Mon, 22 Apr 2024 06:13:16 +0000 (08:13 +0200)]
xfs: compile out v4 support if disabled

Add a few strategic IS_ENABLED statements to let the compiler eliminate
unused code when CONFIG_XFS_SUPPORT_V4 is disabled.

This saves multiple kilobytes of .text in my .config:

$ size xfs.o.*
text    data     bss     dec     hex filename
1363633  294836     592 1659061  1950b5 xfs.o.new
1371453  294868     592 1666913  196f61 xfs.o.old

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandanbabu@kernel.org>
5 months agoxfs: remove the unused xfs_extent_busy_enomem trace event
Christoph Hellwig [Fri, 5 Apr 2024 06:07:10 +0000 (08:07 +0200)]
xfs: remove the unused xfs_extent_busy_enomem trace event

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandanbabu@kernel.org>
5 months agoxfs: unwind xfs_extent_busy_clear
Christoph Hellwig [Fri, 5 Apr 2024 06:07:09 +0000 (08:07 +0200)]
xfs: unwind xfs_extent_busy_clear

The current structure of xfs_extent_busy_clear that locks the first busy
extent in each AG and unlocks when switching to a new AG makes sparse
unhappy as the lock critical section tracking can't cope with taking the
lock conditionally and inside a loop.

Rewrite xfs_extent_busy_clear so that it has an outer loop only advancing
when moving to a new AG, and an inner loop that consumes busy extents for
the given AG to make life easier for sparse and to also make this logic
more obvious to humans.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandanbabu@kernel.org>
5 months agoxfs: move more logic into xfs_extent_busy_clear_one
Christoph Hellwig [Fri, 5 Apr 2024 06:07:08 +0000 (08:07 +0200)]
xfs: move more logic into xfs_extent_busy_clear_one

Move the handling of discarded entries into xfs_extent_busy_clear_one
to reuse the length check and tidy up the logic in the caller.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandanbabu@kernel.org>
5 months agoxfs: Remove unused function is_rt_data_fork
Jiapeng Chong [Mon, 4 Mar 2024 02:24:08 +0000 (10:24 +0800)]
xfs: Remove unused function is_rt_data_fork

The function are defined in the rmap_repair.c file, but not called
elsewhere, so delete the unused function.

fs/xfs/scrub/rmap_repair.c:436:1: warning: unused function 'is_rt_data_fork'.

Reported-by: Abaci Robot <abaci@linux.alibaba.com>
Closes: https://bugzilla.openanolis.cn/show_bug.cgi?id=8425
Signed-off-by: Jiapeng Chong <jiapeng.chong@linux.alibaba.com>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandanbabu@kernel.org>
5 months agoxfs: small cleanup in xrep_update_qflags()
Dan Carpenter [Thu, 7 Mar 2024 08:39:18 +0000 (11:39 +0300)]
xfs: small cleanup in xrep_update_qflags()

The "mp" pointer is the same as "sc->mp" so this change doesn't affect
runtime at all.  However, it's nicer to use same name for both the lock
and the unlock.

Signed-off-by: Dan Carpenter <dan.carpenter@linaro.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Chandan Babu R <chandanbabu@kernel.org>
5 months agoxfs: Fix typo in comment
Thorsten Blum [Fri, 12 Apr 2024 22:59:27 +0000 (00:59 +0200)]
xfs: Fix typo in comment

s/somethign/something/

Signed-off-by: Thorsten Blum <thorsten.blum@toblux.com>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Chandan Babu R <chandanbabu@kernel.org>
5 months agoxfs: fix sparse warnings about unused interval tree functions
Dave Chinner [Tue, 2 Apr 2024 21:28:32 +0000 (08:28 +1100)]
xfs: fix sparse warnings about unused interval tree functions

Sparse throws warnings about the interval tree functions that are
defined and then not used in the scrub bitmap code:

fs/xfs/scrub/bitmap.c:57:1: warning: unused function 'xbitmap64_tree_iter_next' [-Wunused-function]
INTERVAL_TREE_DEFINE(struct xbitmap64_node, bn_rbnode, uint64_t,
^
./include/linux/interval_tree_generic.h:151:33: note: expanded from macro 'INTERVAL_TREE_DEFINE'
ITSTATIC ITSTRUCT *                                                           \
                                                                              ^
<scratch space>:3:1: note: expanded from here
xbitmap64_tree_iter_next
^
fs/xfs/scrub/bitmap.c:331:1: warning: unused function 'xbitmap32_tree_iter_next' [-Wunused-function]
INTERVAL_TREE_DEFINE(struct xbitmap32_node, bn_rbnode, uint32_t,
^
./include/linux/interval_tree_generic.h:151:33: note: expanded from macro 'INTERVAL_TREE_DEFINE'
ITSTATIC ITSTRUCT *                                                           \
                                                                              ^
<scratch space>:59:1: note: expanded from here
xbitmap32_tree_iter_next

Fix these by marking the functions created by the interval tree
creation macro as __maybe_unused to suppress this warning.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandanbabu@kernel.org>
5 months agoxfs: silence sparse warning when checking version number
Dave Chinner [Tue, 2 Apr 2024 21:28:30 +0000 (08:28 +1100)]
xfs: silence sparse warning when checking version number

Scrub checks the superblock version number against the known good
feature bits that can be set in the version mask. It calculates
the version mask to compare like so:

vernum_mask = cpu_to_be16(~XFS_SB_VERSION_OKBITS |
                                  XFS_SB_VERSION_NUMBITS |
                                  XFS_SB_VERSION_ALIGNBIT |
                                  XFS_SB_VERSION_DALIGNBIT |
                                  XFS_SB_VERSION_SHAREDBIT |
                                  XFS_SB_VERSION_LOGV2BIT |
                                  XFS_SB_VERSION_SECTORBIT |
                                  XFS_SB_VERSION_EXTFLGBIT |
                                  XFS_SB_VERSION_DIRV2BIT);

This generates a sparse warning:

fs/xfs/scrub/agheader.c:168:23: warning: cast truncates bits from constant value (ffff3f8f becomes 3f8f)

This is because '~XFS_SB_VERSION_OKBITS' is considered a 32 bit
constant, even though it's value is always under 16 bits.

This is a kinda silly thing to do, because:

/*
 * Supported feature bit list is just all bits in the versionnum field because
 * we've used them all up and understand them all. Except, of course, for the
 * shared superblock bit, which nobody knows what it does and so is unsupported.
 */
#define XFS_SB_VERSION_OKBITS           \
        ((XFS_SB_VERSION_NUMBITS | XFS_SB_VERSION_ALLFBITS) & \
                ~XFS_SB_VERSION_SHAREDBIT)

#define XFS_SB_VERSION_NUMBITS          0x000f
#define XFS_SB_VERSION_ALLFBITS         0xfff0
#define XFS_SB_VERSION_SHAREDBIT        0x0200

XFS_SB_VERSION_OKBITS has a value of 0xfdff, and so
~XFS_SB_VERSION_OKBITS == XFS_SB_VERSION_SHAREDBIT.  The calculated
mask already sets XFS_SB_VERSION_SHAREDBIT, so starting with
~XFS_SB_VERSION_OKBITS is completely redundant....

Signed-off-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandanbabu@kernel.org>
5 months agoxfs: fix CIL sparse lock context warnings
Dave Chinner [Tue, 2 Apr 2024 21:28:28 +0000 (08:28 +1100)]
xfs: fix CIL sparse lock context warnings

Sparse reports:

fs/xfs/xfs_log_cil.c:1127:1: warning: context imbalance in 'xlog_cil_push_work' - different lock contexts for basic block
fs/xfs/xfs_log_cil.c:1380:1: warning: context imbalance in 'xlog_cil_push_background' - wrong count at exit
fs/xfs/xfs_log_cil.c:1623:9: warning: context imbalance in 'xlog_cil_commit' - unexpected unlock

xlog_cil_push_background() has a locking annotations for an rw_sem.
Sparse does not track lock contexts for rw_sems, so the
annotation generates false warnings. Remove the annotation.

xlog_wait_on_iclog() drops the log->l_ic_loglock. The function has a
sparse annotation, but the prototype in xfs_log_priv.h does not.
Hence the warning from xlog_cil_push_work() which calls
xlog_wait_on_iclog(). Add the missing annotation.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Chandan Babu R <chandanbabu@kernel.org>
5 months agoMerge tag 'retain-ilock-during-dir-ops-6.10_2024-04-15' of https://git.kernel.org...
Chandan Babu R [Tue, 16 Apr 2024 07:23:08 +0000 (12:53 +0530)]
Merge tag 'retain-ilock-during-dir-ops-6.10_2024-04-15' of https://git./linux/kernel/git/djwong/xfs-linux into xfs-6.10-mergeA

xfs: retain ILOCK during directory updates

This series changes the directory update code to retain the ILOCK on all
files involved in a rename until the end of the operation.  The upcoming
parent pointers patchset applies parent pointers in a separate chained
update from the actual directory update, which is why it is now
necessary to keep the ILOCK instead of dropping it after the first
transaction in the chain.

As a side effect, we no longer need to hold the IOLOCK during an rmapbt
scan of inodes to serialize the scan with ongoing directory updates.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandanbabu@kernel.org>
* tag 'retain-ilock-during-dir-ops-6.10_2024-04-15' of https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux:
  xfs: unlock new repair tempfiles after creation
  xfs: don't pick up IOLOCK during rmapbt repair scan
  xfs: Hold inode locks in xfs_rename
  xfs: Hold inode locks in xfs_trans_alloc_dir
  xfs: Hold inode locks in xfs_ialloc
  xfs: Increase XFS_QM_TRANS_MAXDQS to 5
  xfs: Increase XFS_DEFER_OPS_NR_INODES to 5

5 months agoMerge tag 'online-fsck-design-6.10_2024-04-15' of https://git.kernel.org/pub/scm...
Chandan Babu R [Tue, 16 Apr 2024 07:20:19 +0000 (12:50 +0530)]
Merge tag 'online-fsck-design-6.10_2024-04-15' of https://git./linux/kernel/git/djwong/xfs-linux into xfs-6.10-mergeA

xfs: design documentation for online fsck, part 2

This series updates the design documentation for online fsck to reflect
the final design of the parent pointers feature as well as the
implementation of online fsck for the new metadata.

This has been running on the djcloud for months with no problems.  Enjoy!

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandanbabu@kernel.org>
* tag 'online-fsck-design-6.10_2024-04-15' of https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux:
  docs: describe xfs directory tree online fsck
  docs: update offline parent pointer repair strategy
  docs: update online directory and parent pointer repair sections
  docs: update the parent pointers documentation to the final version

5 months agoMerge tag 'discard-relax-locks-6.10_2024-04-15' of https://git.kernel.org/pub/scm...
Chandan Babu R [Tue, 16 Apr 2024 07:16:28 +0000 (12:46 +0530)]
Merge tag 'discard-relax-locks-6.10_2024-04-15' of https://git./linux/kernel/git/djwong/xfs-linux into xfs-6.10-mergeA

xfs: less heavy locks during fstrim

Congratulations!  You have made it to the final patchset of the main
online fsck feature!  This patchset fixes some stalling behavior that I
observed when running FITRIM against large flash-based filesystems with
very heavily fragmented free space data.  In summary -- the current
fstrim implementation optimizes for trimming the largest free extents
first, and holds the AGF lock for the duration of the operation.  This
is great if fstrim is being run as a foreground process by a sysadmin.

For xfs_scrub, however, this isn't so good -- we don't really want to
block on one huge kernel call while reporting no progress information.
We don't want to hold the AGF so long that background processes stall.
These problems are easily fixable by issuing smaller FITRIM calls, but
there's still the problem of walking the entire cntbt.  To solve that
second problem, we introduce a new sub-AG FITRIM implementation.  To
solve the first problem, make it relax the AGF periodically.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandanbabu@kernel.org>
* tag 'discard-relax-locks-6.10_2024-04-15' of https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux:
  xfs: fix performance problems when fstrimming a subset of a fragmented AG

5 months agoMerge tag 'inode-repair-improvements-6.10_2024-04-15' of https://git.kernel.org/pub...
Chandan Babu R [Tue, 16 Apr 2024 07:12:17 +0000 (12:42 +0530)]
Merge tag 'inode-repair-improvements-6.10_2024-04-15' of https://git./linux/kernel/git/djwong/xfs-linux into xfs-6.10-mergeA

xfs: inode-related repair fixes

While doing QA of the online fsck code, I made a few observations:
First, nobody was checking that the di_onlink field is actually zero;
Second, that allocating a temporary file for repairs can fail (and
thus bring down the entire fs) if the inode cluster is corrupt; and
Third, that file link counts do not pin at ~0U to prevent integer
overflows.  Fourth, the x{chk,rep}_metadata_inode_fork functions
should be subclassing the main scrub context, not modifying the
parent's setup willy-nilly.

This scattered patchset fixes those three problems.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandanbabu@kernel.org>
* tag 'inode-repair-improvements-6.10_2024-04-15' of https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux:
  xfs: create subordinate scrub contexts for xchk_metadata_inode_subtype
  xfs: pin inodes that would otherwise overflow link count
  xfs: try to avoid allocating from sick inode clusters
  xfs: check unused nlink fields in the ondisk inode

5 months agoMerge tag 'repair-iunlink-6.10_2024-04-15' of https://git.kernel.org/pub/scm/linux...
Chandan Babu R [Tue, 16 Apr 2024 07:08:25 +0000 (12:38 +0530)]
Merge tag 'repair-iunlink-6.10_2024-04-15' of https://git./linux/kernel/git/djwong/xfs-linux into xfs-6.10-mergeA

xfs: online fsck of iunlink buckets

This series enhances the AGI scrub code to check the unlinked inode
bucket lists for errors, and fixes them if necessary.  Now that iunlink
pointer updates are virtual log items, we can batch updates pretty
efficiently in the logging code.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandanbabu@kernel.org>
* tag 'repair-iunlink-6.10_2024-04-15' of https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux:
  xfs: repair AGI unlinked inode bucket lists
  xfs: hoist AGI repair context to a heap object
  xfs: check AGI unlinked inode buckets

5 months agoMerge tag 'repair-symlink-6.10_2024-04-15' of https://git.kernel.org/pub/scm/linux...
Chandan Babu R [Tue, 16 Apr 2024 06:58:57 +0000 (12:28 +0530)]
Merge tag 'repair-symlink-6.10_2024-04-15' of https://git./linux/kernel/git/djwong/xfs-linux into xfs-6.10-mergeA

xfs: online repair of symbolic links

The patches in this set adds the ability to repair the target buffer of
a symbolic link, using the same salvage, rebuild, and swap strategy used
everywhere else.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandanbabu@kernel.org>
* tag 'repair-symlink-6.10_2024-04-15' of https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux:
  xfs: online repair of symbolic links
  xfs: pass the owner to xfs_symlink_write_target
  xfs: expose xfs_bmap_local_to_extents for online repair

5 months agoMerge tag 'repair-orphanage-6.10_2024-04-15' of https://git.kernel.org/pub/scm/linux...
Chandan Babu R [Tue, 16 Apr 2024 06:48:18 +0000 (12:18 +0530)]
Merge tag 'repair-orphanage-6.10_2024-04-15' of https://git./linux/kernel/git/djwong/xfs-linux into xfs-6.10-mergeA

xfs: move orphan files to lost and found

Orphaned files are defined to be files with nonzero ondisk link count
but no observable parent directory.  This series enables online repair
to reparent orphaned files into the filesystem directory tree, and wires
up this reparenting ability into the directory, file link count, and
parent pointer repair functions.  This is how we fix files with positive
link count that are not reachable through the directory tree.

This patch will also create the orphanage directory (lost+found) if it
is not present.  In contrast to xfs_repair, we follow e2fsck in creating
the lost+found without group or other-owner access to avoid accidental
disclosure of files that were previously hidden by an 0700 directory.
That's silly security, but people have been known to do it.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandanbabu@kernel.org>
* tag 'repair-orphanage-6.10_2024-04-15' of https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux:
  xfs: ensure dentry consistency when the orphanage adopts a file
  xfs: move files to orphanage instead of letting nlinks drop to zero
  xfs: move orphan files to the orphanage

5 months agoMerge tag 'repair-dirs-6.10_2024-04-15' of https://git.kernel.org/pub/scm/linux/kerne...
Chandan Babu R [Tue, 16 Apr 2024 06:31:06 +0000 (12:01 +0530)]
Merge tag 'repair-dirs-6.10_2024-04-15' of https://git./linux/kernel/git/djwong/xfs-linux into xfs-6.10-mergeA

xfs: online repair of directories

This series employs atomic extent swapping to enable safe reconstruction
of directory data.  For now, XFS does not support reverse directory
links (aka parent pointers), so we can only salvage the dirents of a
directory and construct a new structure.

Directory repair therefore consists of five main parts:

First, we walk the existing directory to salvage as many entries as we
can, by adding them as new directory entries to the repair temp dir.

Second, we validate the parent pointer found in the directory.  If one
was not found, we scan the entire filesystem looking for a potential
parent.

Third, we use atomic extent swaps to exchange the entire data fork
between the two directories.

Fourth, we reap the old directory blocks as carefully as we can.

To wrap up the directory repair code, we need to add to the regular
filesystem the ability to free all the data fork blocks in a directory.
This does not change anything with normal directories, since they must
still unlink and shrink one entry at a time.  However, this will
facilitate freeing of partially-inactivated temporary directories during
log recovery.

The second half of this patchset implements repairs for the dotdot
entries of directories.  For now there is only rudimentary support for
this, because there are no directory parent pointers, so the best we can
do is scanning the filesystem and the VFS dcache for answers.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandanbabu@kernel.org>
* tag 'repair-dirs-6.10_2024-04-15' of https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux:
  xfs: ask the dentry cache if it knows the parent of a directory
  xfs: online repair of parent pointers
  xfs: scan the filesystem to repair a directory dotdot entry
  xfs: online repair of directories
  xfs: inactivate directory data blocks

5 months agoMerge tag 'repair-unlinked-inode-state-6.10_2024-04-15' of https://git.kernel.org...
Chandan Babu R [Tue, 16 Apr 2024 06:27:10 +0000 (11:57 +0530)]
Merge tag 'repair-unlinked-inode-state-6.10_2024-04-15' of https://git./linux/kernel/git/djwong/xfs-linux into xfs-6.10-mergeA

xfs: online repair of inode unlinked state

This series adds some logic to the inode scrubbers so that they can
detect and deal with consistency errors between the link count and the
per-inode unlinked list state.  The helpers needed to do this are
presented here because they are a prequisite for rebuildng directories,
since we need to get a rebuilt non-empty directory off the unlinked
list.

Note that this patchset does not provide comprehensive reconstruction of
the AGI unlinked list; that is coming in a subsequent patchset.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandanbabu@kernel.org>
* tag 'repair-unlinked-inode-state-6.10_2024-04-15' of https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux:
  xfs: update the unlinked list when repairing link counts
  xfs: ensure unlinked list state is consistent with nlink during scrub

5 months agoMerge tag 'repair-xattrs-6.10_2024-04-15' of https://git.kernel.org/pub/scm/linux...
Chandan Babu R [Tue, 16 Apr 2024 06:23:09 +0000 (11:53 +0530)]
Merge tag 'repair-xattrs-6.10_2024-04-15' of https://git./linux/kernel/git/djwong/xfs-linux into xfs-6.10-mergeA

xfs: online repair of extended attributes

This series employs atomic extent swapping to enable safe reconstruction
of extended attribute data attached to a file.  Because xattrs do not
have any redundant information to draw off of, we can at best salvage
as much data as we can and build a new structure.

Rebuilding an extended attribute structure consists of these three
steps:

First, we walk the existing attributes to salvage as many of them as we
can, by adding them as new attributes attached to the repair tempfile.
We need to add a new xfile-based data structure to hold blobs of
arbitrary length to stage the xattr names and values.

Second, we write the salvaged attributes to a temporary file, and use
atomic extent swaps to exchange the entire attribute fork between the
two files.

Finally, we reap the old xattr blocks (which are now in the temporary
file) as carefully as we can.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandanbabu@kernel.org>
* tag 'repair-xattrs-6.10_2024-04-15' of https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux:
  xfs: create an xattr iteration function for scrub
  xfs: flag empty xattr leaf blocks for optimization
  xfs: scrub should set preen if attr leaf has holes
  xfs: repair extended attributes
  xfs: use atomic extent swapping to fix user file fork data
  xfs: create a blob array data structure
  xfs: enable discarding of folios backing an xfile

5 months agoMerge tag 'dirattr-validate-owners-6.10_2024-04-15' of https://git.kernel.org/pub...
Chandan Babu R [Tue, 16 Apr 2024 06:18:46 +0000 (11:48 +0530)]
Merge tag 'dirattr-validate-owners-6.10_2024-04-15' of https://git./linux/kernel/git/djwong/xfs-linux into xfs-6.10-mergeA

xfs: set and validate dir/attr block owners

There are a couple of significant changes that need to be made to the
directory and xattr code before we can support online repairs of those
data structures.

The first change is because online repair is designed to use libxfs to
create a replacement dir/xattr structure in a temporary file, and use
atomic extent swapping to commit the corrected structure.  To avoid the
performance hit of walking every block of the new structure to rewrite
the owner number before the swap, we instead change libxfs to allow
callers of the dir and xattr code the ability to set an explicit owner
number to be written into the header fields of any new blocks that are
created.  For regular operation this will be the directory inode number.

The second change is to update the dir/xattr code to actually *check*
the owner number in each block that is read off the disk, since we don't
currently do that.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandanbabu@kernel.org>
* tag 'dirattr-validate-owners-6.10_2024-04-15' of https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux:
  xfs: validate explicit directory free block owners
  xfs: validate explicit directory block buffer owners
  xfs: validate explicit directory data buffer owners
  xfs: validate directory leaf buffer owners
  xfs: validate dabtree node buffer owners
  xfs: validate attr remote value buffer owners
  xfs: validate attr leaf buffer owners
  xfs: reduce indenting in xfs_attr_node_list
  xfs: use the xfs_da_args owner field to set new dir/attr block owner
  xfs: add an explicit owner field to xfs_da_args

5 months agoMerge tag 'repair-rtsummary-6.10_2024-04-15' of https://git.kernel.org/pub/scm/linux...
Chandan Babu R [Tue, 16 Apr 2024 06:15:33 +0000 (11:45 +0530)]
Merge tag 'repair-rtsummary-6.10_2024-04-15' of https://git./linux/kernel/git/djwong/xfs-linux into xfs-6.10-mergeA

xfs: online repair of realtime summaries

We now have all the infrastructure we need to repair file metadata.
We'll begin with the realtime summary file, because it is the least
complex data structure.  To support this we need to add three more
pieces to the temporary file code from the previous patchset --
preallocating space in the temp file, formatting metadata into that
space and writing the blocks to disk, and swapping the fork mappings
atomically.

After that, the actual reconstruction of the realtime summary
information is pretty simple, since we can simply write the incore
copy computed by the rtsummary scrubber to the temporary file, swap the
contents, and reap the old blocks.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandanbabu@kernel.org>
* tag 'repair-rtsummary-6.10_2024-04-15' of https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux:
  xfs: online repair of realtime summaries
  xfs: teach the tempfile to set up atomic file content exchanges
  xfs: support preallocating and copying content into temporary files

5 months agoMerge tag 'repair-tempfiles-6.10_2024-04-15' of https://git.kernel.org/pub/scm/linux...
Chandan Babu R [Tue, 16 Apr 2024 06:10:13 +0000 (11:40 +0530)]
Merge tag 'repair-tempfiles-6.10_2024-04-15' of https://git./linux/kernel/git/djwong/xfs-linux into xfs-6.10-mergeA

xfs: create temporary files for online repair

As mentioned earlier, the repair strategy for file-based metadata is to
build a new copy in a temporary file and swap the file fork mappings
with the metadata inode.  We've built the atomic extent swap facility,
so now we need to build a facility for handling private temporary files.

The first step is to teach the filesystem to ignore the temporary files.
We'll mark them as PRIVATE in the VFS so that the kernel security
modules will leave it alone.  The second step is to add the online
repair code the ability to create a temporary file and reap extents from
the temporary file after the extent swap.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandanbabu@kernel.org>
* tag 'repair-tempfiles-6.10_2024-04-15' of https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux:
  xfs: add the ability to reap entire inode forks
  xfs: refactor live buffer invalidation for repairs
  xfs: create temporary files and directories for online repair
  xfs: hide private inodes from bulkstat and handle functions

5 months agoMerge tag 'atomic-file-updates-6.10_2024-04-15' of https://git.kernel.org/pub/scm...
Chandan Babu R [Tue, 16 Apr 2024 05:55:09 +0000 (11:25 +0530)]
Merge tag 'atomic-file-updates-6.10_2024-04-15' of https://git./linux/kernel/git/djwong/xfs-linux into xfs-6.10-mergeA

xfs: atomic file content exchanges

This series creates a new XFS_IOC_EXCHANGE_RANGE ioctl to exchange
ranges of bytes between two files atomically.

This new functionality enables data storage programs to stage and commit
file updates such that reader programs will see either the old contents
or the new contents in their entirety, with no chance of torn writes.  A
successful call completion guarantees that the new contents will be seen
even if the system fails.

The ability to exchange file fork mappings between files in this manner
is critical to supporting online filesystem repair, which is built upon
the strategy of constructing a clean copy of a damaged structure and
committing the new structure into the metadata file atomically.  The
ioctls exist to facilitate testing of the new functionality and to
enable future application program designs.

User programs will be able to update files atomically by opening an
O_TMPFILE, reflinking the source file to it, making whatever updates
they want to make, and exchange the relevant ranges of the temp file
with the original file.  If the updates are aligned with the file block
size, a new (since v2) flag provides for exchanging only the written
areas.  Note that application software must quiesce writes to the file
while it stages an atomic update.  This will be addressed by a
subsequent series.

This mechanism solves the clunkiness of two existing atomic file update
mechanisms: for O_TRUNC + rewrite, this eliminates the brief period
where other programs can see an empty file.  For create tempfile +
rename, the need to copy file attributes and extended attributes for
each file update is eliminated.

However, this method introduces its own awkwardness -- any program
initiating an exchange now needs to have a way to signal to other
programs that the file contents have changed.  For file access mediated
via read and write, fanotify or inotify are probably sufficient.  For
mmaped files, that may not be fast enough.

Here is the proposed manual page:

IOCTL-XFS-EXCHANGE-RANGE(2System Calls ManuIOCTL-XFS-EXCHANGE-RANGE(2)

NAME
       ioctl_xfs_exchange_range  -  exchange  the contents of parts of
       two files

SYNOPSIS
       #include <sys/ioctl.h>
       #include <xfs/xfs_fs.h>

       int ioctl(int file2_fd, XFS_IOC_EXCHANGE_RANGE, struct  xfs_ex‐
       change_range *arg);

DESCRIPTION
       Given  a  range  of bytes in a first file file1_fd and a second
       range of bytes in a second file  file2_fd,  this  ioctl(2)  ex‐
       changes the contents of the two ranges.

       Exchanges  are  atomic  with  regards to concurrent file opera‐
       tions.  Implementations must guarantee that readers see  either
       the old contents or the new contents in their entirety, even if
       the system fails.

       The system call parameters are conveyed in  structures  of  the
       following form:

           struct xfs_exchange_range {
               __s32    file1_fd;
               __u32    pad;
               __u64    file1_offset;
               __u64    file2_offset;
               __u64    length;
               __u64    flags;
           };

       The field pad must be zero.

       The  fields file1_fd, file1_offset, and length define the first
       range of bytes to be exchanged.

       The fields file2_fd, file2_offset, and length define the second
       range of bytes to be exchanged.

       Both  files must be from the same filesystem mount.  If the two
       file descriptors represent the same file, the byte ranges  must
       not  overlap.   Most  disk-based  filesystems  require that the
       starts of both ranges must be aligned to the file  block  size.
       If  this  is  the  case, the ends of the ranges must also be so
       aligned unless the XFS_EXCHANGE_RANGE_TO_EOF flag is set.

       The field flags control the behavior of the exchange operation.

           XFS_EXCHANGE_RANGE_TO_EOF
                  Ignore the length parameter.  All bytes in  file1_fd
                  from  file1_offset to EOF are moved to file2_fd, and
                  file2's size is set to  (file2_offset+(file1_length-
                  file1_offset)).   Meanwhile, all bytes in file2 from
                  file2_offset to EOF are moved to file1  and  file1's
                  size    is   set   to   (file1_offset+(file2_length-
                  file2_offset)).

           XFS_EXCHANGE_RANGE_DSYNC
                  Ensure that all modified in-core data in  both  file
                  ranges  and  all  metadata updates pertaining to the
                  exchange operation are flushed to persistent storage
                  before  the  call  returns.  Opening either file de‐
                  scriptor with O_SYNC or O_DSYNC will have  the  same
                  effect.

           XFS_EXCHANGE_RANGE_FILE1_WRITTEN
                  Only  exchange sub-ranges of file1_fd that are known
                  to contain data  written  by  application  software.
                  Each  sub-range  may  be  expanded (both upwards and
                  downwards) to align with the file  allocation  unit.
                  For files on the data device, this is one filesystem
                  block.  For files on the realtime  device,  this  is
                  the realtime extent size.  This facility can be used
                  to implement fast atomic  scatter-gather  writes  of
                  any  complexity for software-defined storage targets
                  if all writes are aligned  to  the  file  allocation
                  unit.

           XFS_EXCHANGE_RANGE_DRY_RUN
                  Check  the parameters and the feasibility of the op‐
                  eration, but do not change anything.

RETURN VALUE
       On error, -1 is returned, and errno is set to indicate the  er‐
       ror.

ERRORS
       Error  codes can be one of, but are not limited to, the follow‐
       ing:

       EBADF  file1_fd is not open for reading and writing or is  open
              for  append-only  writes;  or  file2_fd  is not open for
              reading and writing or is open for append-only writes.

       EINVAL The parameters are not correct for  these  files.   This
              error  can  also appear if either file descriptor repre‐
              sents a device, FIFO, or socket.  Disk filesystems  gen‐
              erally  require  the  offset  and length arguments to be
              aligned to the fundamental block sizes of both files.

       EIO    An I/O error occurred.

       EISDIR One of the files is a directory.

       ENOMEM The kernel was unable to allocate sufficient  memory  to
              perform the operation.

       ENOSPC There  is  not  enough  free space in the filesystem ex‐
              change the contents safely.

       EOPNOTSUPP
              The filesystem does not support exchanging bytes between
              the two files.

       EPERM  file1_fd or file2_fd are immutable.

       ETXTBSY
              One of the files is a swap file.

       EUCLEAN
              The filesystem is corrupt.

       EXDEV  file1_fd  and  file2_fd  are  not  on  the  same mounted
              filesystem.

CONFORMING TO
       This API is XFS-specific.

USE CASES
       Several use cases are imagined for this system  call.   In  all
       cases, application software must coordinate updates to the file
       because the exchange is performed unconditionally.

       The first is a data storage program that wants to  commit  non-
       contiguous  updates  to a file atomically and coordinates write
       access to that file.  This can be done by creating a  temporary
       file, calling FICLONE(2) to share the contents, and staging the
       updates into the temporary file.  The FULL_FILES flag is recom‐
       mended  for this purpose.  The temporary file can be deleted or
       punched out afterwards.

       An example program might look like this:

           int fd = open("/some/file", O_RDWR);
           int temp_fd = open("/some", O_TMPFILE | O_RDWR);

           ioctl(temp_fd, FICLONE, fd);

           /* append 1MB of records */
           lseek(temp_fd, 0, SEEK_END);
           write(temp_fd, data1, 1000000);

           /* update record index */
           pwrite(temp_fd, data1, 600, 98765);
           pwrite(temp_fd, data2, 320, 54321);
           pwrite(temp_fd, data2, 15, 0);

           /* commit the entire update */
           struct xfs_exchange_range args = {
               .file1_fd = temp_fd,
               .flags = XFS_EXCHANGE_RANGE_TO_EOF,
           };

           ioctl(fd, XFS_IOC_EXCHANGE_RANGE, &args);

       The second is a software-defined  storage  host  (e.g.  a  disk
       jukebox)  which  implements an atomic scatter-gather write com‐
       mand.  Provided the exported disk's logical block size  matches
       the file's allocation unit size, this can be done by creating a
       temporary file and writing the data at the appropriate offsets.
       It  is  recommended that the temporary file be truncated to the
       size of the regular file before any writes are  staged  to  the
       temporary  file  to avoid issues with zeroing during EOF exten‐
       sion.  Use this call with the FILE1_WRITTEN  flag  to  exchange
       only  the  file  allocation  units involved in the emulated de‐
       vice's write command.  The temporary file should  be  truncated
       or  punched out completely before being reused to stage another
       write.

       An example program might look like this:

           int fd = open("/some/file", O_RDWR);
           int temp_fd = open("/some", O_TMPFILE | O_RDWR);
           struct stat sb;
           int blksz;

           fstat(fd, &sb);
           blksz = sb.st_blksize;

           /* land scatter gather writes between 100fsb and 500fsb */
           pwrite(temp_fd, data1, blksz * 2, blksz * 100);
           pwrite(temp_fd, data2, blksz * 20, blksz * 480);
           pwrite(temp_fd, data3, blksz * 7, blksz * 257);

           /* commit the entire update */
           struct xfs_exchange_range args = {
               .file1_fd = temp_fd,
               .file1_offset = blksz * 100,
               .file2_offset = blksz * 100,
               .length       = blksz * 400,
               .flags        = XFS_EXCHANGE_RANGE_FILE1_WRITTEN |
                               XFS_EXCHANGE_RANGE_FILE1_DSYNC,
           };

           ioctl(fd, XFS_IOC_EXCHANGE_RANGE, &args);

NOTES
       Some filesystems may limit the amount of data or the number  of
       extents that can be exchanged in a single call.

SEE ALSO
       ioctl(2)

XFS                           2024-02-10   IOCTL-XFS-EXCHANGE-RANGE(2)

The reference implementation in XFS creates a new log incompat feature
and log intent items to track high level progress of swapping ranges of
two files and finish interrupted work if the system goes down.  Sample
code can be found in the corresponding changes to xfs_io to exercise the
use case mentioned above.

Note that this function is /not/ the O_DIRECT atomic untorn file writes
concept that has also been floating around for years.  It is also not
the RWF_ATOMIC patchset that has been shared.  This RFC is constructed
entirely in software, which means that there are no limitations other
than the general filesystem limits.

As a side note, the original motivation behind the kernel functionality
is online repair of file-based metadata.  The atomic file content
exchange is implemented as an atomic exchange of file fork mappings,
which means that we can implement online reconstruction of extended
attributes and directories by building a new one in another inode and
exchanging the contents.

Subsequent patchsets adapt the online filesystem repair code to use
atomic file exchanges.  This enables repair functions to construct a
clean copy of a directory, xattr information, symbolic links, realtime
bitmaps, and realtime summary information in a temporary inode.  If this
completes successfully, the new contents can be committed atomically
into the inode being repaired.  This is essential to avoid making
corruption problems worse if the system goes down in the middle of
running repair.

For userspace, this series also includes the userspace pieces needed to
test the new functionality, and a sample implementation of atomic file
updates.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandanbabu@kernel.org>
* tag 'atomic-file-updates-6.10_2024-04-15' of https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux:
  xfs: enable logged file mapping exchange feature
  docs: update swapext -> exchmaps language
  xfs: capture inode generation numbers in the ondisk exchmaps log item
  xfs: support non-power-of-two rtextsize with exchange-range
  xfs: make file range exchange support realtime files
  xfs: condense symbolic links after a mapping exchange operation
  xfs: condense directories after a mapping exchange operation
  xfs: condense extended attributes after a mapping exchange operation
  xfs: add error injection to test file mapping exchange recovery
  xfs: bind together the front and back ends of the file range exchange code
  xfs: create deferred log items for file mapping exchanges
  xfs: introduce a file mapping exchange log intent item
  xfs: create a incompat flag for atomic file mapping exchanges
  xfs: introduce new file range exchange ioctl
  vfs: export remap and write check helpers

5 months agoMerge tag 'file-exchange-refactorings-6.10_2024-04-15' of https://git.kernel.org...
Chandan Babu R [Tue, 16 Apr 2024 05:47:28 +0000 (11:17 +0530)]
Merge tag 'file-exchange-refactorings-6.10_2024-04-15' of https://git./linux/kernel/git/djwong/xfs-linux into xfs-6.10-mergeA

xfs: refactorings for atomic file content exchanges

This series applies various cleanups and refactorings to file IO
handling code ahead of the main series to implement atomic file content
exchanges.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandanbabu@kernel.org>
* tag 'file-exchange-refactorings-6.10_2024-04-15' of https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux:
  xfs: constify xfs_bmap_is_written_extent
  xfs: refactor non-power-of-two alignment checks
  xfs: hoist multi-fsb allocation unit detection to a helper
  xfs: create a new helper to return a file's allocation unit
  xfs: declare xfs_file.c symbols in xfs_file.h
  xfs: move xfs_iops.c declarations out of xfs_inode.h
  xfs: move inode lease breaking functions to xfs_inode.c

5 months agoMerge tag 'log-incompat-permissions-6.10_2024-04-15' of https://git.kernel.org/pub...
Chandan Babu R [Tue, 16 Apr 2024 05:39:58 +0000 (11:09 +0530)]
Merge tag 'log-incompat-permissions-6.10_2024-04-15' of https://git./linux/kernel/git/djwong/xfs-linux into xfs-6.10-mergeA

xfs: improve log incompat feature handling

This patchset improves the performance of log incompat feature bit
handling by making a few changes to how the filesystem handles them.
First, we now only clear the bits during a clean unmount to reduce calls
to the (expensive) upgrade function to once per bit per mount.  Second,
we now only allow incompat feature upgrades for sysadmins or if the
sysadmin explicitly allows it via mount option.  Currently the only log
incompat user is logged xattrs, which requires CONFIG_XFS_DEBUG=y, so
there should be no user visible impact to this change.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandanbabu@kernel.org>
* tag 'log-incompat-permissions-6.10_2024-04-15' of https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux:
  xfs: only clear log incompat flags at clean unmount
  xfs: fix error bailout in xrep_abt_build_new_trees
  xfs: fix potential AGI <-> ILOCK ABBA deadlock in xrep_dinode_findmode_walk_directory
  xfs: fix an AGI lock acquisition ordering problem in xrep_dinode_findmode
  xfs: pass xfs_buf lookup flags to xfs_*read_agi

5 months agoxfs: unlock new repair tempfiles after creation
Darrick J. Wong [Mon, 15 Apr 2024 21:55:16 +0000 (14:55 -0700)]
xfs: unlock new repair tempfiles after creation

After creation, drop the ILOCK on temporary files that have been created
to stage a repair.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: don't pick up IOLOCK during rmapbt repair scan
Darrick J. Wong [Mon, 15 Apr 2024 21:55:15 +0000 (14:55 -0700)]
xfs: don't pick up IOLOCK during rmapbt repair scan

Now that we've fixed the directory operations to hold the ILOCK until
they're finished with rmapbt updates for directory shape changes, we no
longer need to take this lock when scanning directories for rmapbt
records.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: Hold inode locks in xfs_rename
Allison Henderson [Mon, 15 Apr 2024 21:55:14 +0000 (14:55 -0700)]
xfs: Hold inode locks in xfs_rename

Modify xfs_rename to hold all inode locks across a rename operation
We will need this later when we add parent pointers

Signed-off-by: Allison Henderson <allison.henderson@oracle.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Catherine Hoang <catherine.hoang@oracle.com>
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: Hold inode locks in xfs_trans_alloc_dir
Allison Henderson [Mon, 15 Apr 2024 21:55:13 +0000 (14:55 -0700)]
xfs: Hold inode locks in xfs_trans_alloc_dir

Modify xfs_trans_alloc_dir to hold locks after return.  Caller will be
responsible for manual unlock.  We will need this later to hold locks
across parent pointer operations

Signed-off-by: Allison Henderson <allison.henderson@oracle.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Catherine Hoang <catherine.hoang@oracle.com>
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
5 months agoxfs: Hold inode locks in xfs_ialloc
Allison Henderson [Mon, 15 Apr 2024 21:55:12 +0000 (14:55 -0700)]
xfs: Hold inode locks in xfs_ialloc

Modify xfs_ialloc to hold locks after return.  Caller will be
responsible for manual unlock.  We will need this later to hold locks
across parent pointer operations

Signed-off-by: Allison Henderson <allison.henderson@oracle.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Catherine Hoang <catherine.hoang@oracle.com>
[djwong: hold the parent ilocked across transaction rolls too]
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>