KVM: arm64: Ensure target address is granule-aligned for range TLBI

When zapping a table entry in stage2_try_break_pte(), we issue range
TLB invalidation for the region that was mapped by the table. However,
we neglect to align the base address down to the granule size and so
if we ended up reaching the table entry via a misaligned address then
we will accidentally skip invalidation for some prefix of the affected
address range.

Align 'ctx->addr' down to the granule size when performing TLB
invalidation for an unmapped table in stage2_try_break_pte().

Cc: Raghavendra Rao Ananta <rananta@google.com>
Cc: Gavin Shan <gshan@redhat.com>
Cc: Shaoqin Huang <shahuang@redhat.com>
Cc: Quentin Perret <qperret@google.com>
Fixes: defc8cc7abf0 ("KVM: arm64: Invalidate the table entries upon a range")
Signed-off-by: Will Deacon <will@kernel.org>
Reviewed-by: Shaoqin Huang <shahuang@redhat.com>
Reviewed-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20240327124853.11206-5-will@kernel.org
Signed-off-by: Oliver Upton <oliver.upton@linux.dev>

diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c
index a40dafc43bb642abde9d55f5009165d4417229da..5a59ef88b646f054b4c1f4a3994803e817efd94a 100644 (file)
--- a/arch/arm64/kvm/hyp/pgtable.c
+++ b/arch/arm64/kvm/hyp/pgtable.c
@@ -843,12 +843,15 @@ static bool stage2_try_break_pte(const struct kvm_pgtable_visit_ctx *ctx,
                 * Perform the appropriate TLB invalidation based on the
                 * evicted pte value (if any).
                 */
-               if (kvm_pte_table(ctx->old, ctx->level))
-                       kvm_tlb_flush_vmid_range(mmu, ctx->addr,
-                                               kvm_granule_size(ctx->level));
-               else if (kvm_pte_valid(ctx->old))
+               if (kvm_pte_table(ctx->old, ctx->level)) {
+                       u64 size = kvm_granule_size(ctx->level);
+                       u64 addr = ALIGN_DOWN(ctx->addr, size);
+
+                       kvm_tlb_flush_vmid_range(mmu, addr, size);
+               } else if (kvm_pte_valid(ctx->old)) {
                        kvm_call_hyp(__kvm_tlb_flush_vmid_ipa, mmu,
                                     ctx->addr, ctx->level);
+               }
        }
 
        if (stage2_pte_is_counted(ctx->old))