drm/xe/pm: fix unbalanced ref handling

In local_pci_probe() the core kernel increments the rpm for the device,
just before calling into the probe hook. If the driver/device supports
runtime pm it is then meant to put this ref during probe (like we do in
xe_pm_runtime_init()). However when removing the device we then also
need to take the reference back, otherwise the ref that is put in
pci_device_remove() will be unbalanced when for example unloading the
driver, leading to warnings like:

    [ 3808.596345] xe 0000:03:00.0: Runtime PM usage count underflow!

Fix this by incrementing the rpm ref when removing the device.

v2:
  - Improve the terminology in the commit message; s/drop/put/ etc (Lucas & Rodrigo)
  - Also call pm_runtime_forbid(dev) (Rodrigo)

Closes: https://gitlab.freedesktop.org/drm/xe/kernel/-/issues/193
Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Cc: Lucas De Marchi <lucas.demarchi@intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe/guc: Remove i915_regs.h include

i915_regs.h is not needed, particularly in a header file. What is needed
is i915_reg_defs.h for use of _MMIO() and similar macros.

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe: Remove outdated build workaround

Use the more common "call cc-disable-warning" way to disable warnings.

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe: Remove duplicate media_ver

media_verx100 supersedes the info from media_ver. Leave media_ver in the
struct xe_device_desc, used in xe_pci.c since it's easier to define
common parts of the platforms like that. However all the rest of the
driver should be using media_verx100 that is more future proof.

Closes: https://gitlab.freedesktop.org/drm/xe/kernel/-/issues/216
Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe: Add missing include xe_wait_user_fence.h

Make xe_wait_user_fence.c include xe_wait_user_fence.h so it doesn't
rely on indirect includes and also doesn't fail the build due to missing
prototype for xe_wait_user_fence_ioctl().

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe: Add missing doc for xe parameter

Fix the following warning:

../drivers/gpu/drm/xe/xe_ttm_stolen_mgr.c:55: warning: Function
parameter or member 'xe' not described in
'xe_ttm_stolen_cpu_inaccessible'

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe: Remove unused functions

xe_gt_topology_dss_group_mask and xe_gt_topology_count_dss are probably
leftover from initial implementation - they are not called from
anywhere. Remove those functions.

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe: Fix application of LRC tunings

LRC tunings were added after the gt ones and didn't add the call
in xe_gt_record_default_lrcs() to process them like is done for
workarounds. Add such a function and call it from
xe_gt_record_default_lrcs().

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe: Make local functions static

A few static functions not being declared like that break the build with
W=1, like e.g.

cc1: all warnings being treated as errors
make[2]: *** [../scripts/Makefile.build:250: drivers/gpu/drm/xe/xe_gt.o] Error 1
../drivers/gpu/drm/xe/xe_guc.c:240:6: error: no previous prototype for ‘guc_write_params’ [-Werror=missing-prototypes]
  240 | void guc_write_params(struct xe_guc *guc)
      |      ^~~~~~~~~~~~~~~~

Make them static.

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe/query: zero the region info

There are also some reserved fields in here which are not currently
cleared when handing back to userspace. Otherwise we might run into
issues if we later wish to use them.

Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Reviewed-by: Lucas De Marchi lucas.demarchi@intel.com
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe/stolen: don't map stolen on small-bar

The driver should still be functional with small-bar, just that the vram
size is clamped to the BAR size (until we add proper support for tiered
vram). For stolen vram we shouldn't iomap anything if the BAR size
doesn't also contain the stolen portion, since on discrete the stolen
portion is always at the end of normal vram. Stolen should still be
functional, just that allocating CPU visible io memory will always
return an error.

v2 (Lucas)
  - Mention in the commit message that stolen vram is always as the end
    of normal vram, which is why stolen in not mappable on small-bar
    systems.
  - Just make xe_ttm_stolen_inaccessible() return true for such cases.
    Also rename to xe_ttm_stolen_cpu_inaccessible to better describe
    that we are talking about direct CPU access. Plus add some
    kernel-doc.

Closes: https://gitlab.freedesktop.org/drm/xe/kernel/-/issues/209
Reported-by: Lucas De Marchi <lucas.demarchi@intel.com>
Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe/mmio: fix forcewake ref leak in xe_mmio_ioctl

Make sure we properly release the forcewake ref on all error paths.

v2(Lucas):
  - Make it less verbose and just fold the unimplemented options into
    the default. The exact return value doesn't seem to matter for the
    corresponding IGT.
  - Replace the user triggerable WARN() with drm_dbg().

Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe: Remove unseless xe_force_wake_prune.

(!(gt->info.engine_mask & BIT(i))) cases are already
handled in the init function. And these masks are not
modified between the init and the prune.

Suggested-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>

drm/xe: Update the list of devices to add even more TGL devices

The list of GTs got splitted a while back between GT1
and GT2 on TGL.

References: https://patchwork.freedesktop.org/patch/388414/
CC: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Carlos Santa <carlos.santa@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe: Initialize ret in mcr_lock()

ret is not initialized in mcr_lock() when running in platforms with
graphics IP version < 1270, this could cause drm_WARN_ON_ONCE()
to hit eventually(what just happened to me).

Fixes: dd08ebf6c352 ("drm/xe: Introduce a new DRM driver for Intel GPUs")
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe/rtp: Support multiple actions per entry

Just like there is support for multiple rules per entry in an rtp table,
also support multiple actions. This makes it easier to add support for
workarounds that need to change multiple registers. It also makes it
slightly more readable as now the action part resembles the rule part.

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe/rtp: Split action and entry flags

Entry flags is meant for the whole entry, including the rule
evaluation. Action flags are for flags applied to the register or
action being taken. Since there's only one action per entry, the
distinction was not important and a u8 was spared. However more and more
workarounds are needing multiple actions. This prepares for multiple
action support.

Right now there are these action flags:

 - XE_RTP_ACTION_FLAG_MASKED_REG: register in the action is a masked
   register
 - XE_RTP_ACTION_FLAG_ENGINE_BASE: the engine base should be added to
   the register in order to form the real address

And this entry flag:

 - XE_RTP_ENTRY_FLAG_FOREACH_ENGINE: the rules should be evaluated for
   each engine on the gt. It also automatically implies
   XE_RTP_ACTION_FLAG_ENGINE_BASE.

Since there are likely not that many rules, reduce n_rules to u8 so the
overall entry size doesn't increase more than needed.

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe: Rename xe_rtp_regval to xe_rtp_action

It's true that the struct records the register and the value (in form of
2 masks) to restore, but it also records more fields important to
the application of workarounds/tuning, etc. One important part is what
is the macro used to record these fields: SET/CLR/WR/FIELD_SET/etc.

Thinking of the table as a set of rules + actions is more intuitive than
rules + regval.

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe/mcr: Add SQIDI steering for DG2

Like detailed in commit 927dfdd09d8c ("drm/i915/dg2: Add SQIDI
steering"), some registers are expected to have the selector
initialized just once and never set to anything else. For xe, the
registers with SQIDI replication type (SF and MCFG) were missing,
resulting in warnings like:

[  410.685565] xe 0000:03:00.0: Did not find MCR register 0x8724 in any MCR steering table

While adding these registers, abstract the handling for
"dg2_gam_ranges", moving them together with SF/MCFG to a dedicated
table. This also avoids that range to be checked for platforms other
than DG2. For DG2, this is the new steering output:

# cat /sys/kernel/debug/dri/0/gt0/steering
...
IMPLICIT steering: group=0x0, instance=0x0
0x000b00 - 0x000bff
0x001000 - 0x001fff
0x004000 - 0x004aff
0x008700 - 0x0087ff
0x00c800 - 0x00cfff
0x00f000 - 0x00ffff

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe/mcr: Use designated init for xe_steering_types

There is already a BUILD_BUG_ON() check to make sure the size follow the
number of steering types. Also make sure the right index is being used
for each steering type.

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe: Remove TODO from workaround documentation

LRC workarounds are already implemented: remove leftover TODO.

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe: Remove TODO from rtp infra

The function pointer is already present as match_func, inside
struct xe_rtp_rule and handled as so instead of inside rtp_regval as
originally thought out when this was written.

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe: Fix xe_tuning include

xe_tuning.c should include xe_tuning.h, not xe_wa.h

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe: Fix typo in MCR documentation

Add missing "multicast" word and adapt/wrap the rest of the sentence.

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe: Add debugfs for dumping GGTT mappings

Adding a debugfs dump of GGTT was useful for some debugging I did,
and easy to add. Might be useful for others too.

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe: Drop TLB invalidation from ring operations

Now that we issue TLB invalidations on unbinds and rebind from execs we
no longer need to issue TLB invalidations from the ring operations.

Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>

drm/xe: Add TLB invalidation fence after rebinds issued from execs

If we add an TLB invalidation fence for rebinds issued from execs we
should be able to drop the TLB invalidation from the ring operations.

Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>

drm/xe: Add has_asid to device info

Rather than alias supports_usm to ASIS support, add an explicit
variable to indicate ASID support.

Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>

drm/xe: Signal invalidation fence immediately if CT send fails

This means we are in the middle of a GT reset and no need to do TLB
invalidation so just signal invalidation fence immediately.

Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>

drm/xe: Propagate VM unbind error to invalidation fence

If a VM unbind hits an error, do not issue a TLB invalidation and
propagate the error the invalidation fence.

Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>

drm/xe: Lock GGTT on when restoring kernel BOs

Make lockdep happy as we required to hold the GGTT when calling
xe_ggtt_map_bo.

Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>

drm/xe: Use GuC to do GGTT invalidations for the GuC firmware

Only the GuC should be issuing TLB invalidations if it is enabled. Part
of this patch is sanitize the device on driver unload to ensure we do
not send GuC based TLB invalidations during driver unload.

Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>

drm/xe: Propagate error from bind operations to async fence

If an bind operation fails we need to report it via the async fence.

Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>

drm/xe: Add range based TLB invalidations

If the platform supports range based TLB invalidations use them. Hide
these details in the xe_gt_tlb_invalidation layer.

Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>

drm/xe: Add has_range_tlb_invalidation device attribute

This will help implementing range based TLB invalidations.

Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>

drm/xe: Delete debugfs entry to issue TLB invalidation

Not used, let's remove this.

Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>

drm/xe: Only set VM->asid for platforms that support a ASID

This will help with TLB invalidation as the ASID in TLB invalidate
should be zero for platforms that do not support a ASID.

Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>

drm/xe: Add TDR for invalidation fence timeout cleanup

Endless fences are not good, add a TDR to cleanup any invalidation
fences which have not received an invalidation message within a timeout
period.

Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>

drm/xe: Add TLB invalidation fence ftrace

This will help debug issues with TLB invalidation fences.

Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe: Kernel doc GT TLB invalidations

Document all exported functions.

Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe: Invalidate TLB after unbind is complete

This gets tricky as we can't do the TLB invalidation until the unbind
operation is done on the hardware and we can't signal the unbind as
complete until the TLB invalidation is done. To work around this we
create an unbind fence which does a TLB invalidation after unbind is
done on the hardware, signals on TLB invalidation completion, and this
fence is installed in the BO dma-resv slot and installed in out-syncs
for the unbind operation.

Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Suggested-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com
Suggested-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Reviewed-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe: Add TLB invalidation fence

Fence will be signaled when TLB invalidation completion.

Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Suggested-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Reviewed-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe: Move TLB invalidation variable to own sub-structure in GT

TLB invalidations no longer just restricted to USM, move the variables
to own sub-structure.

Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe: Break of TLB invalidation into its own file

TLB invalidation is used by more than USM (page faults) so break this
code out into its own file.

Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe: Don't process TLB invalidation done in CT fast-path

We can't currently do this due to TLB invalidation done handler
expecting the seqno being received in-order, with the fast-path a TLB
invalidation done could pass one being processed in the slow-path in an
extreme corner case. Remove TLB invalidation done from the fast-path for
now and in a follow up reenable this once the TLB invalidation done
handler can deal with out of order seqno.

Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe/migrate: Update emit_pte to cope with a size level than 4k

emit_pte assumes the size argument is 4k aligned, this may not be true
for the PTEs emitted for CSS as seen by below call stack:

[   56.734228] xe_migrate_copy:585: size=327680, ccs_start=327680, css_size=1280,4096
[   56.734250] xe_migrate_copy:643: size=262144
[   56.734252] emit_pte:404: ptes=64
[   56.734255] emit_pte:418: chunk=64
[   56.734257] xe_migrate_copy:650: size=1024 @ CCS emit PTE
[   56.734259] emit_pte:404: ptes=1
[   56.734261] emit_pte:418: chunk=1
[   56.734339] xe_migrate_copy:643: size=65536
[   56.734342] emit_pte:404: ptes=16
[   56.734344] emit_pte:418: chunk=16
[   56.734346] xe_migrate_copy:650: size=256 # CCS emit PTE
[   56.734348] emit_pte:404: ptes=1
[   56.734350] emit_pte:418: chunk=1
[   56.734352] xe_res_next:174: size=4096, remaining=0

Update emit_pte to handle sizes less than 4k.

Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe/ggtt: fix GGTT scratch usage for DG2

Scratch page is in VRAM, and therefore requires 64K GTT layout. In GGTT
world this just means having 16 consecutive entries, with 64K GTT
alignment for the GTT address of the first entry (also matching physical
alignment). However to keep things simple just dump it into system
memory, like we already do for ppGTT.  While we are here, also give it
known default value.

Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe/ggtt: fix alignment usage for DG2

Spec says we need to use 64K VRAM pages for GGTT on platforms like DG2.
In GGTT this just means aligning the GTT address to 64K and ensuring
that we have 16 consecutive entries each pointing to the respective 4K
entry. We already ensure we have 64K pages underneath, so it's just a
case of forcing the GTT alignment.

Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe/ppgtt: fix scratch page usage on DG2

On DG2 when running the xe_vm IGT, the kernel generates loads of CAT
errors and GT resets (sometimes at least).  On small-bar systems seems
to trigger a lot more easily (maybe due to difference in allocation
strategy). Appears to be related to scratch, since we seem to use the
64K TLB hint on scratch entries, even though the scratch page is a 4K
vram page. Bumping the scratch page size and physical alignment seems
to fix it. Or at least we no longer hit:

[  148.872683] xe 0000:03:00.0: [drm] Engine memory cat error: guc_id=0
[  148.872701] xe 0000:03:00.0: [drm] Engine memory cat error: guc_id=0
[  148.875108] WARNING: CPU: 0 PID: 953 at drivers/gpu/drm/xe/xe_guc_submit.c:797

However to keep things simple, so we don't have to deal with 64K TLB
hints, just move the scratch page into system memory on platforms that
require 64K VRAM pages.

Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe/ppgtt: clear the scratch page

We need to ensure we don't leak the contents to userspace.

Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe/bo: reduce xe_bo_create_pin_map() restrictions

On DGFX this blows up if can call this with a system memory object:

XE_BUG_ON(!mem_type_is_vram(place->mem_type) && place->mem_type != XE_PL_STOLEN);

If we consider dpt it looks like we can already in theory hit this, if
we run out of vram and stolen vram. It at least seems reasonable to
allow calling this on any object which supports CPU access.

Note this also changes the behaviour with stolen VRAM and suspend, such
that we no longer attempt to migrate stolen objects into system memory.
However nothing in stolen should ever need to be restored (same on
integrated), so should be fine. Also on small-bar systems the stolen
portion is pretty much always non-CPU accessible, and currently pinned
objects use plain memcpy when being moved, which doesn't play nicely.

Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe: Map initial FB at the same place in GGTT too

I saw a flicker when booting xe, and it's very likely that the original
FB was not mapped at the same place when inheriting, fix it.

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe: Convert memory device refcount to s32

The comparison with < 0 suggests that the memory device access
should be signed to handle underflow. This makes it work more reliably.

As a result, the max refcount is now S32_MAX instead.

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe/uapi: Add XE_ENGINE_GET_PROPERTY uAPI

This is intended to get some properties that are of interest of UMDs
like the ban state.

Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe/uapi: Rename XE_ENGINE_PROPERTY_X to XE_ENGINE_SET_PROPERTY_X

Engine property get uAPI will be added, so to avoid ambiguity here
renaming XE_ENGINE_PROPERTY_X to XE_ENGINE_SET_PROPERTY_X.

No changes in behavior.

Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe: Use global macros to set PM functions

This aligns with other drivers and fixes build failure when
CONFIG_PM_SLEEP is not set, such as on RISC-V.

Signed-off-by: Francois Dugast <francois.dugast@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe: skip Kunit tests requiring real hardware when running on UML

Some tests are meant to run only on real hardware. Skip those,
if no device was found.

Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe/guc_pc: Fix Meteor Lake registers.

When adding the frequency management, Meteor Lake platform
was left behind. Handling it properly now.

Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Cc: Francois Dugast <francois.dugast@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe: Fix Meteor Lake rsa issue on guc loading

[  117.901473] xe 0000:00:02.0: [drm] GuC load failed: status = 0x400000A0
[  117.901506] xe 0000:00:02.0: [drm] GuC load failed: status: Reset = 0, BootROM = 0x50, UKernel = 0x00, MIA = 0x00, Auth = 0x01

Signed-off-by: Philippe Lecluse <philippe.lecluse@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe/gt: Enable interrupt while initializing root gt

At present the interrupts are enabled while initializing the last GT.
But this is incorrect for a Multi-GT platform, as root GT initialization
will fail with interrupt disabled. Interrupts are required for
the GuC submission triggered during initialization.
Enable the interrupt during the root GT initialization.

Signed-off-by: Balasubramani Vivekanandan <balasubramani.vivekanandan@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe: KUnit tests depend on CONFIG_DRM_FBDEV_EMULATION

ERROR:root:../drivers/gpu/drm/xe/display/intel_fbdev.c:585:5: error: redefinition of ‘intel_fbdev_init’
  585 | int intel_fbdev_init(struct drm_device *dev)
      |     ^~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/display/intel_fbdev.c:55:
../drivers/gpu/drm/xe/display/intel_fbdev.h:26:19: note: previous definition of ‘intel_fbdev_init’ with type ‘int(struct drm_device *)’
   26 | static inline int intel_fbdev_init(struct drm_device *dev)
      |                   ^~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/display/intel_fbdev.c:626:6: error: redefinition of ‘intel_fbdev_initial_config_async’
  626 | void intel_fbdev_initial_config_async(struct drm_device *dev)
      |      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/display/intel_fbdev.h:31:20: note: previous definition of ‘intel_fbdev_initial_config_async’ with type ‘void(struct drm_device *)’
   31 | static inline void intel_fbdev_initial_config_async(struct drm_device *dev)
      |                    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/display/intel_fbdev.c:646:6: error: redefinition of ‘intel_fbdev_unregister’
  646 | void intel_fbdev_unregister(struct drm_i915_private *dev_priv)
      |      ^~~~~~~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/display/intel_fbdev.h:35:20: note: previous definition of ‘intel_fbdev_unregister’ with type ‘void(struct xe_device *)’
   35 | static inline void intel_fbdev_unregister(struct drm_i915_private *dev_priv)
      |                    ^~~~~~~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/display/intel_fbdev.c:661:6: error: redefinition of ‘intel_fbdev_fini’
  661 | void intel_fbdev_fini(struct drm_i915_private *dev_priv)
      |      ^~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/display/intel_fbdev.h:39:20: note: previous definition of ‘intel_fbdev_fini’ with type ‘void(struct xe_device *)’
   39 | static inline void intel_fbdev_fini(struct drm_i915_private *dev_priv)
      |                    ^~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/display/intel_fbdev.c:692:6: error: redefinition of ‘intel_fbdev_set_suspend’
  692 | void intel_fbdev_set_suspend(struct drm_device *dev, int state, bool synchronous)
      |      ^~~~~~~~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/display/intel_fbdev.h:43:20: note: previous definition of ‘intel_fbdev_set_suspend’ with type ‘void(struct drm_device *, int,  bool)’ {aka ‘void(struct drm_device *, int,  _Bool)’}
   43 | static inline void intel_fbdev_set_suspend(struct drm_device *dev, int state, bool synchronous)
      |                    ^~~~~~~~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/display/intel_fbdev.c:751:6: error: redefinition of ‘intel_fbdev_output_poll_changed’
  751 | void intel_fbdev_output_poll_changed(struct drm_device *dev)
      |      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/display/intel_fbdev.h:47:20: note: previous definition of ‘intel_fbdev_output_poll_changed’ with type ‘void(struct drm_device *)’
   47 | static inline void intel_fbdev_output_poll_changed(struct drm_device *dev)
      |                    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/display/intel_fbdev.c:770:6: error: redefinition of ‘intel_fbdev_restore_mode’
  770 | void intel_fbdev_restore_mode(struct drm_device *dev)
      |      ^~~~~~~~~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/display/intel_fbdev.h:51:20: note: previous definition of ‘intel_fbdev_restore_mode’ with type ‘void(struct drm_device *)’
   51 | static inline void intel_fbdev_restore_mode(struct drm_device *dev)
      |                    ^~~~~~~~~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/display/intel_fbdev.c:785:27: error: redefinition of ‘intel_fbdev_framebuffer’
  785 | struct intel_framebuffer *intel_fbdev_framebuffer(struct intel_fbdev *fbdev)
      |                           ^~~~~~~~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/display/intel_fbdev.h:54:41: note: previous definition of ‘intel_fbdev_framebuffer’ with type ‘struct intel_framebuffer *(struct intel_fbdev *)’
   54 | static inline struct intel_framebuffer *intel_fbdev_framebuffer(struct intel_fbdev *fbdev)
      |                                         ^~~~~~~~~~~~~~~~~~~~~~~

Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe/Kconfig.debug: select DEBUG_FS for KUnit runs

KUnit reuquires debugfs, as otherwise, it won't build:

$ make ARCH=x86_64 O=.kunit --jobs=8
ERROR:root:../drivers/gpu/drm/xe/display/intel_display_debugfs.c:1612:6: error: redefinition of ‘intel_display_debugfs_register’
 1612 | void intel_display_debugfs_register(struct drm_i915_private *i915)
      |      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ../drivers/gpu/drm/xe/display/intel_display_debugfs.c:18:
../drivers/gpu/drm/xe/display/intel_display_debugfs.h:18:20: note: previous definition of ‘intel_display_debugfs_register’ with type ‘void(struct xe_device *)’
   18 | static inline void intel_display_debugfs_register(struct drm_i915_private *i915) {}
      |                    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/display/intel_display_debugfs.c:1935:6: error: redefinition of ‘intel_connector_debugfs_add’
 1935 | void intel_connector_debugfs_add(struct intel_connector *intel_connector)
      |      ^~~~~~~~~~~~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/display/intel_display_debugfs.h:19:20: note: previous definition of ‘intel_connector_debugfs_add’ with type ‘void(struct intel_connector *)’
   19 | static inline void intel_connector_debugfs_add(struct intel_connector *connector) {}
      |                    ^~~~~~~~~~~~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/display/intel_display_debugfs.c:1993:6: error: redefinition of ‘intel_crtc_debugfs_add’
 1993 | void intel_crtc_debugfs_add(struct drm_crtc *crtc)
      |      ^~~~~~~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/display/intel_display_debugfs.h:20:20: note: previous definition of ‘intel_crtc_debugfs_add’ with type ‘void(struct drm_crtc *)’
   20 | static inline void intel_crtc_debugfs_add(struct drm_crtc *crtc) {}
      |                    ^~~~~~~~~~~~~~~~~~~~~~

Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe: Add min config for kunit integration ARCH=um

Some of the tests may benefit from running with ARCH=um, forgoing any
additional setup on the CI build side. Add min config for that.

Tested with:

./tools/testing/kunit/kunit.py build \
--kunitconfig drivers/gpu/drm/xe/.kunitconfig \
--jobs $(nproc) \
--build_dir build_kunit

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Cc: Mauro Carvalho Chehab <mchehab@kernel.org>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Mauro Carvalho Chehab <mchehab@kernel.org>

drm/xe: Add documentation for mem_type

mem_type field was added in commit d8b52a02cb40 ("drm/xe: Implement
stolen memory.") to designate the TTM memory type for that mgr. Add
kernel-doc with its description.

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe/gt: Fix min() with u32 and u64

Fix the following error while building for 32b:

In file included from ../drivers/gpu/drm/xe/xe_gt.c:6:
../drivers/gpu/drm/xe/xe_gt.c: In function ‘gt_ttm_mgr_init’:
../include/linux/minmax.h:20:35: error: comparison of distinct pointer types lacks a cast [-Werror]
   20 |         (!!(sizeof((typeof(x) *)1 == (typeof(y) *)1)))
      |                                   ^~

Cast it to u64 so size of the second operand matches the first one when
building it for 32 bits.

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe: Fix tracepoints on 32b

Leave the types as u64, but cast the pointers to unsigned long before
assigning so the compiler doesn't throw warning about casting a pointer
to integer of different size.

Also, size_t should use %zu, not %ld.

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe/mmio: Use non-atomic writeq/readq variant for 32b

writeq() and readq() and other functions working on 64 bit variables
are not provided by 32b arch. For that it's needed to choose between
linux/io-64-nonatomic-hi-lo.h and linux/io-64-nonatomic-lo-hi.h,
spliting the read/write in 2 accesses. For xe driver, it doesn't matter
much, so just choose one and include in xe_mmio.h.

This also removes some ifdef CONFIG_64BIT we had around because of the
missing 64bit functions.

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe: Fix some log messages on 32b

Either use the proper format or cast up to 64b depending on the case.

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe/ggtt: Use BIT_ULL() for 64bit

Make sure it's 64bit value in both 32b and 64b arch.

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe: fix xe_mmio_total_vram_size

As also cause issue on PVC, moving back
to what we did before stolen was
introduced

Signed-off-by: Philippe Lecluse <philippe.lecluse@intel.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe/pcode: fix pcode error check

On DG2 we are now getting:

[  104.456607] xe 0000:03:00.0: [drm] *ERROR* PCODE timeout, retrying with preemption disabled

Looks like we just need to invert the error check for
xe_pcode_try_request(), which returns zero on success.

Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe: enforce GSMBASE for DG1 instead of BAR2

On DG1, BAR2 is not reliable for reporting Vram size, need to use GSMBASE.
Simplify xe_mmio_total_vram_size to report vram size and usable size.

Signed-off-by: Philippe Lecluse <philippe.lecluse@intel.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe: Fix hidden gotcha regression with bo create

The bo_create ioctl relied on the internal ordering of memory regions to
be the same, make sure we don't allocate stolen instead of VRAM0.

Also remove a debug warning left in.

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Reviewed-by: Philippe Lecluse <philippe.lecluse1@gmail.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe/guc: Add support GuC MMIO send / recv

SRIOV has a use case of GuC MMIO send / recv, add a function for it.

Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Philippe Lecluse <philippe.lecluse1@gmail.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe/guc: s/xe_guc_send_mmio/xe_guc_mmio_send

Now aligns with the xe_guc_ct_send naming.

Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Philippe Lecluse <philippe.lecluse1@gmail.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe/guc: Report submission version of GuC firmware

Starting in 70.6.* GuC firmware the CSS header includes the submission
version, pull this from the CSS header. Prior 70.* versions accidentally
omitted this informatio so hard code to the correct values. This
information will be used by VFs when communicating with the PF.

Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Philippe Lecluse <philippe.lecluse1@gmail.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe: Fake pulling gt->info.engine_mask from hwconfig blob

The blob doesn't fully support this yet, so fake for now to ensure our
driver load order is correct.

Once the blob supports pulling gt->info.engine_mask from the blob, this
patch can be removed.

Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe: Ensure VMA not userptr before calling xe_bo_is_stolen

Fix the below splat:

[  142.510525] [IGT] xe_exec_basic: starting subtest once-userptr
[  142.511339] BUG: kernel NULL pointer dereference, address: 0000000000000228
[  142.518311] #PF: supervisor read access in kernel mode
[  142.523458] #PF: error_code(0x0000) - not-present page
[  142.528604] PGD 0 P4D 0
[  142.531153] Oops: 0000 [#1] PREEMPT SMP NOPTI
[  142.535518] CPU: 4 PID: 1199 Comm: kworker/u16:8 Not tainted 6.1.0-rc1-xe+ #1
[  142.542656] Hardware name: Intel Corporation Tiger Lake Client Platform/TigerLake U DDR4 SODIMM RVP, BIOS TGLSFWI1.R00.3243.A01.2006102133 06/10/2020
[  142.556033] Workqueue: events_unbound async_op_work_func [xe]
[  142.561810] RIP: 0010:xe_bo_is_stolen+0x0/0x20 [xe]
[  142.566709] Code: 20 c8 75 05 83 fa 07 74 05 c3 cc cc cc cc 48 8b 87 08 02 00 00 0f b6 80 2c ff ff ff c3 cc cc cc cc 66 0f 1f 84 00 00 00 00 00 <48> 8b 87 28 02 00 00 83 78 10 07 0f 94 c0 c3 cc cc cc cc 66 66 2e
[  142.585447] RSP: 0018:ffffc900019eb888 EFLAGS: 00010246
[  142.590678] RAX: 0000000000000002 RBX: 0000000000000000 RCX: ffff88813f6a2108
[  142.597821] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
[  142.604962] RBP: ffffc900019ebbc0 R08: 0000000000000001 R09: 0000000000000000
[  142.612101] R10: 0000000000000001 R11: 0000000000000001 R12: ffff88814107d600
[  142.619242] R13: ffffc900019eba20 R14: ffff888140442000 R15: 0000000000000000
[  142.626378] FS:  0000000000000000(0000) GS:ffff88849fa00000(0000) knlGS:0000000000000000
[  142.634468] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  142.640219] CR2: 0000000000000228 CR3: 000000010a4c0006 CR4: 0000000000770ee0
[  142.647361] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  142.654505] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[  142.661639] PKRU: 55555554
[  142.664367] Call Trace:
[  142.666830]  <TASK>
[  142.668947]  __xe_pt_bind_vma+0x1a1/0xa50 [xe]
[  142.673417]  ? unwind_next_frame+0x187/0x770
[  142.677699]  ? __thaw_task+0xc0/0xc0
[  142.681293]  ? __lock_acquire+0x5e4/0x26e0
[  142.685409]  ? lockdep_hardirqs_on+0xbf/0x140
[  142.689779]  ? lock_acquire+0xd2/0x310
[  142.693548]  ? mark_held_locks+0x49/0x80
[  142.697485]  ? xe_vm_bind_vma+0xf1/0x3d0 [xe]
[  142.701866]  xe_vm_bind_vma+0xf1/0x3d0 [xe]
[  142.706082]  xe_vm_bind+0x76/0x140 [xe]
[  142.709944]  vm_bind_ioctl+0x26f/0xb40 [xe]
[  142.714161]  ? async_op_work_func+0x20c/0x450 [xe]
[  142.718974]  async_op_work_func+0x20c/0x450 [xe]
[  142.723620]  process_one_work+0x263/0x580
[  142.727645]  ? process_one_work+0x580/0x580
[  142.731839]  worker_thread+0x4d/0x3b0
[  142.735518]  ? process_one_work+0x580/0x580
[  142.739714]  kthread+0xeb/0x120
[  142.742872]  ? kthread_complete_and_exit+0x20/0x20
[  142.747671]  ret_from_fork+0x1f/0x30
[  142.751264]  </TASK>

Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe: Implement stolen memory.

This adds support for stolen memory, with the same allocator as
vram_mgr. This allows us to skip a whole lot of copy-paste,
by re-using parts of xe_ttm_vram_mgr.

The stolen memory may be bound using VM_BIND, so it performs like any
other memory region.

We should be able to map a stolen BO directly using the physical memory
location instead of through GGTT even on old platforms, but I don't know
what the effects are on coherency.

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe: Take memory ref on kernel job creation

When a job is inflight we may access memory to read the hardware seqno.
All user jobs have VM open which has a ref but kernel jobs do not
require VM so it is possible to not have memory ref. To avoid this, take
a memory ref on kernel job creation.

Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/xe/migrate: Add kerneldoc for the migrate subsystem

Add kerneldoc for structs and external functions.

Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Mauro Carvalho Chehab <mchehab@kernel.org>

drm/xe: Remove i915_utils dependency from xe_pcode.

Expand xe_mmio_wait32 to accept atomic and then use
that directly when possible, and create own routine to
wait for the pcode status.

Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>

drm/xe: Convert xe_mmio_wait32 to us so we can stop using wait_for_us.

Another clean-up towards killing the usage of i915_utils.h

Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>

drm/xe: Let's avoid i915_utils in the xe_force_wake.

We can run the bit operation locally without yet another macro.

Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>

drm/xe: Stop using i915_utils in xe_wopcm.

We don't need any macro for a simple check we can do explicitly
and clear.

Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>

drm/xe: Remove i915_utils dependency from xe_guc_pc.

To make it simpler, all of the status checks also waits and
times out.

Also, no ktime precision is needed in this case, and we
can use usleep_range because we are not in atomic paths here.

Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>

drm/xe: Wait for success on guc done.

Rather than a constant check on proto and wait not busy,
let's wait for the expected success and then check the
protocol afterwards.

With this, we can now use the regular xe_mmio_wait32
and kill this local need for the wait_for.

Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>

drm/xe: Convert guc_ready to regular xe_mmio_wait32

Possible now that the wait function returns the last read value.

So we can remove the users of i915's wait_for one by one...

Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>

drm/xe: Let's return last value read on xe_mmio_wait32.

This is already useful because it avoids some extra reads
where registers might have changed after the timeout decision.

But also, it will be important to end the kill of i915's wait_for.

Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>

drm/xe: Stop using i915's range_overflows_t macro.

Let's do it directly.

Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>

drm/xe: Implement a local xe_mmio_wait32

Then, move the i915_utils.h include to its user.

The overall goal is to kill all the usages of the i915_utils
stuff.

Yes, wait_for also depends on <linux/delay.h>, so they go
together to where it is needed. It will be likely needed
anyway directly for udelay or usleep_range.

Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>

drm/xe: Introduce a new DRM driver for Intel GPUs

Xe, is a new driver for Intel GPUs that supports both integrated and
discrete platforms starting with Tiger Lake (first Intel Xe Architecture).

The code is at a stage where it is already functional and has experimental
support for multiple platforms starting from Tiger Lake, with initial
support implemented in Mesa (for Iris and Anv, our OpenGL and Vulkan
drivers), as well as in NEO (for OpenCL and Level0).

The new Xe driver leverages a lot from i915.

As for display, the intent is to share the display code with the i915
driver so that there is maximum reuse there. But it is not added
in this patch.

This initial work is a collaboration of many people and unfortunately
the big squashed patch won't fully honor the proper credits. But let's
get some git quick stats so we can at least try to preserve some of the
credits:

Co-developed-by: Matthew Brost <matthew.brost@intel.com>
Co-developed-by: Matthew Auld <matthew.auld@intel.com>
Co-developed-by: Matt Roper <matthew.d.roper@intel.com>
Co-developed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Co-developed-by: Francois Dugast <francois.dugast@intel.com>
Co-developed-by: Lucas De Marchi <lucas.demarchi@intel.com>
Co-developed-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Co-developed-by: Philippe Lecluse <philippe.lecluse@intel.com>
Co-developed-by: Nirmoy Das <nirmoy.das@intel.com>
Co-developed-by: Jani Nikula <jani.nikula@intel.com>
Co-developed-by: José Roberto de Souza <jose.souza@intel.com>
Co-developed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Co-developed-by: Dave Airlie <airlied@redhat.com>
Co-developed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Co-developed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Co-developed-by: Mauro Carvalho Chehab <mchehab@kernel.org>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>

Merge tag 'drm-misc-next-2023-12-07' of git://anongit.freedesktop.org/drm/drm-misc into drm-next

drm-misc-next for 6.8:

UAPI Changes:
  - Remove Userspace Mode-Setting ioctls
  - v3d: New uapi to handle jobs involving the CPU

Cross-subsystem Changes:

Core Changes:
  - atomic: Add support for FB-less planes which got reverted a bit
    later for lack of IGT tests and userspace code, Dump private objects
    state in drm_state_dump.
  - dma-buf: Add fence deadline support
  - encoder: Create per-encoder debugfs directory, move the bridge chain
    file to that directory

Driver Changes:
  - Include drm_auth.h in driver that use it but don't include it, Drop
    drm_plane_helper.h from drivers that include it but don't use it
  - imagination: Plenty of small fixes
  - panfrost: Improve interrupt handling at poweroff
  - qaic: Convert to persistent DRM devices
  - tidss: Support for the AM62A7, a few probe improvements, some cleanups
  - v3d: Support for jobs involving the CPU

  - bridge:
    - Create transparent aux-bridge for DP/USB-C
    - lt8912b: Add suspend/resume support and power regulator support

  - panel:
    - himax-hx8394: Drop prepare, unprepare and shutdown logic, Support
      panel rotation
    - New panels: BOE BP101WX1-100, Powkiddy X55, Ampire AM8001280G,
      Evervision VGG644804, SDC ATNA45AF01

Signed-off-by: Dave Airlie <airlied@redhat.com>
From: Maxime Ripard <mripard@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/yu5heqaufyeo4nlowzieu4s5unwqrqyx4jixbfjmzdon677rpk@t53vceua2dao

Merge tag 'drm-intel-next-2023-12-07' of git://anongit.freedesktop.org/drm/drm-intel into drm-next

- Improve display debug msgs and other general clean-ups (Ville, Rahuul)
- PSR fixes and improvements around selective fetch (Jouni, Ville)
- Remove FBC restrictions for Xe2LPD displays (Vinod)
- Skip some timing checks on BXT/GLK DSI transcoders (Ville)
- DP MST Fixes (Ville)
- Correct the input parameter on _intel_dsb_commit (heminhong)
- Fix IP version of the display WAs (Bala)
- DGFX uses direct VBT pin mapping (Clint)
- Proper handling of bool on PIPE_CONF_CHECK macros (Jani)
- Skip state verification with TBT-ALT mod (Mika Kahona)
- General organization of display code for reusage with Xe
  (Jouni, Luca, Jani, Maarten)
- Squelch a sparse warning (Jani)
- Don't use "proxy" headers (Andy Shevchenko)
- Use devm_gpiod_get() for all GPIOs (Hans)
- Fix ADL+ tiled plane stride (Ville)
- Use octal permissions in display debugfs (Jani)

Signed-off-by: Dave Airlie <airlied@redhat.com>
From: Rodrigo Vivi <rodrigo.vivi@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/ZXIWG6bRYaUw0w6-@intel.com

drm/i915/display: Add intel_fb_bo_framebuffer_fini

Xe needs intel_fb_bo_framebuffer_fini for taking care of unpinning the fb
and taking reference.  In i915 this can be empty.

Also move intel_frontbuffer_get to be done after
intel_fb_bo_framebuffer_init to have reasonable sequences:

intel_fb_bo_framebuffer_init
intel_frontbuffer_get
...
intel_frontbuffer_put
intel_fb_bo_framebuffer_fini

v2: Empty function instead of define

Signed-off-by: Jouni Högander <jouni.hogander@intel.com>
Reviewed-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20231207083451.2184562-1-jouni.hogander@intel.com

drm/i915/display: Use i915_gem_object_get_dma_address to get dma address

Works better for xe like that. obj is no longer const.

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20231204134946.16219-1-maarten.lankhorst@linux.intel.com
Reviewed-by: Jouni Högander <jouni.hogander@intel.com>

drm/i915: use octal permissions in display debugfs

Octal permissions are preferred over the symbolics ones.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Reviewed-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20231205134143.2427661-3-jani.nikula@intel.com

drm/i915: pass struct intel_connector to connector debugfs fops

Prefer struct intel_connector over struct drm_connector, and unify the
declarations in the fops.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Reviewed-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20231205134143.2427661-2-jani.nikula@intel.com

drm/i915: use intel_connector in intel_connector_debugfs_add()

Prefer struct intel_connector over struct drm_connector.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Reviewed-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20231205134143.2427661-1-jani.nikula@intel.com

drm/i915/rpm: add rpm_to_i915() helper around container_of()

Reduce the duplication.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20231205121545.2338665-1-jani.nikula@intel.com