KVM: x86/mmu: Use synthetic page fault error code to indicate private faults

Add and use a synthetic, KVM-defined page fault error code to indicate
whether a fault is to private vs. shared memory.  TDX and SNP have
different mechanisms for reporting private vs. shared, and KVM's
software-protected VMs have no mechanism at all.  Usurp an error code
flag to avoid having to plumb another parameter to kvm_mmu_page_fault()
and friends.

Alternatively, KVM could borrow AMD's PFERR_GUEST_ENC_MASK, i.e. set it
for TDX and software-protected VMs as appropriate, but that would require
*clearing* the flag for SEV and SEV-ES VMs, which support encrypted
memory at the hardware layer, but don't utilize private memory at the
KVM layer.

Opportunistically add a comment to call out that the logic for software-
protected VMs is (and was before this commit) broken for nested MMUs, i.e.
for nested TDP, as the GPA is an L2 GPA.  Punt on trying to play nice with
nested MMUs as there is a _lot_ of functionality that simply doesn't work
for software-protected VMs, e.g. all of the paths where KVM accesses guest
memory need to be updated to be aware of private vs. shared memory.

Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-Id: <20240228024147.41573-6-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 0e63dcd8ac1ca62f6077b55b69a238609321666a..816d248264dbae1cc91cf302ddb81a8cfe9b7977 100644 (file)
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -273,7 +273,12 @@ enum x86_intercept_stage;
  * when emulating instructions that triggers implicit access.
  */
 #define PFERR_IMPLICIT_ACCESS  BIT_ULL(48)
-#define PFERR_SYNTHETIC_MASK   (PFERR_IMPLICIT_ACCESS)
+/*
+ * PRIVATE_ACCESS is a KVM-defined flag us to indicate that a fault occurred
+ * when the guest was accessing private memory.
+ */
+#define PFERR_PRIVATE_ACCESS   BIT_ULL(49)
+#define PFERR_SYNTHETIC_MASK   (PFERR_IMPLICIT_ACCESS | PFERR_PRIVATE_ACCESS)
 
 #define PFERR_NESTED_GUEST_PAGE (PFERR_GUEST_PAGE_MASK |       \
                                 PFERR_WRITE_MASK |             \

diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
index 757c24de3485dfee03f070f9f4fc4fb111c2d44b..34aef57f200a190d79ceccb6af01c3eb0de8d3c8 100644 (file)
--- a/arch/x86/kvm/mmu/mmu.c
+++ b/arch/x86/kvm/mmu/mmu.c
@@ -5798,6 +5798,20 @@ int noinline kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, u64 err
        if (WARN_ON_ONCE(!VALID_PAGE(vcpu->arch.mmu->root.hpa)))
                return RET_PF_RETRY;
 
+       /*
+        * Except for reserved faults (emulated MMIO is shared-only), set the
+        * PFERR_PRIVATE_ACCESS flag for software-protected VMs based on the gfn's
+        * current attributes, which are the source of truth for such VMs.  Note,
+        * this wrong for nested MMUs as the GPA is an L2 GPA, but KVM doesn't
+        * currently supported nested virtualization (among many other things)
+        * for software-protected VMs.
+        */
+       if (IS_ENABLED(CONFIG_KVM_SW_PROTECTED_VM) &&
+           !(error_code & PFERR_RSVD_MASK) &&
+           vcpu->kvm->arch.vm_type == KVM_X86_SW_PROTECTED_VM &&
+           kvm_mem_is_private(vcpu->kvm, gpa_to_gfn(cr2_or_gpa)))
+               error_code |= PFERR_PRIVATE_ACCESS;
+
        r = RET_PF_INVALID;
        if (unlikely(error_code & PFERR_RSVD_MASK)) {
                r = handle_mmio_page_fault(vcpu, cr2_or_gpa, direct);

diff --git a/arch/x86/kvm/mmu/mmu_internal.h b/arch/x86/kvm/mmu/mmu_internal.h
index 797b80f996a773dd171bba5422160d30b845ec67..dfd9ff3836634e05e17c6dad37d75e2e85e5a491 100644 (file)
--- a/arch/x86/kvm/mmu/mmu_internal.h
+++ b/arch/x86/kvm/mmu/mmu_internal.h
@@ -306,7 +306,7 @@ static inline int kvm_mmu_do_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa,
                .max_level = KVM_MAX_HUGEPAGE_LEVEL,
                .req_level = PG_LEVEL_4K,
                .goal_level = PG_LEVEL_4K,
-               .is_private = kvm_mem_is_private(vcpu->kvm, cr2_or_gpa >> PAGE_SHIFT),
+               .is_private = err & PFERR_PRIVATE_ACCESS,
        };
        int r;