device to multiple heads which may be connected to one or more discrete
hosts. An example of this would be a simple memory-pool which may be
statically configured (prior to boot) to expose portions of its memory
-to Linux via the CEDT ACPI table.
+to Linux via :doc:`CEDT <../platform/acpi/cedt>`.
MHMLD
~~~~~
There can be multiple switches under an RP. There can be multiple RPs under
a CXL Host Bridge (HB). There can be multiple HBs under a CXL Fixed Memory
-Window Structure (CFMWS).
+Window Structure (CFMWS) in the :doc:`CEDT <../platform/acpi/cedt>`.
An example hierarchy::
The next step is to take the min() of the per host bridge bandwidth and the
bandwidth from the Generic Port (GP). The bandwidths for the GP are retrieved
-via ACPI tables SRAT/HMAT. The minimum bandwidth are aggregated under the same
-ACPI0017 device to form a new xarray.
+via ACPI tables (:doc:`SRAT <../platform/acpi/srat>` and
+:doc:`HMAT <../platform/acpi/hmat>`). The minimum bandwidth are aggregated
+under the same ACPI0017 device to form a new xarray.
Finally, the cxl_region_update_bandwidth() is called and the aggregated
bandwidth from all the members of the last xarray is updated for the
The Root contains links to:
-* `Host Bridge Ports` defined by ACPI CEDT CHBS.
+* `Host Bridge Ports` defined by CHBS in the :doc:`CEDT<../platform/acpi/cedt>`
* `Downstream Ports` typically connected to `Host Bridge Ports`.
-* `Root Decoders` defined by ACPI CEDT CFMWS.
+* `Root Decoders` defined by CFMWS the :doc:`CEDT<../platform/acpi/cedt>`
::
and may be one of many `logical devices` presented by a memory device. It
is still considered a type of `port` in the fabric.
-An `endpoint` contains `endpoint decoders` available for use and the
-*Coherent Device Attribute Table* (CDAT) used to describe the capabilities
-of the device. ::
+An `endpoint` contains `endpoint decoders` and the device's Coherent Device
+Attribute Table (which describes the device's capabilities). ::
# ls /sys/bus/cxl/devices/endpoint5
CDAT decoders_committed modalias uevent
Root Decoder
~~~~~~~~~~~~
A `Root Decoder` is logical construct of the physical address and interleave
-configurations present in the ACPI CEDT CFMWS. Linux presents this information
-as a decoder present in the `CXL Root`. We consider this a `Root Decoder`,
-though technically it exists on the boundary of the CXL specification and
-platform-specific CXL root implementations.
+configurations present in the CFMWS field of the :doc:`CEDT
+<../platform/acpi/cedt>`.
+Linux presents this information as a decoder present in the `CXL Root`. We
+consider this a `Root Decoder`, though technically it exists on the boundary
+of the CXL specification and platform-specific CXL root implementations.
Linux considers these logical decoders a type of `Routing Decoder`, and is the
first decoder in the CXL fabric to receive a memory access from the platform's
memory controllers.
`Root Decoders` are created during :code:`cxl_acpi_probe`. One root decoder
-is created per CFMWS entry in the ACPI CEDT.
+is created per CFMWS entry in the :doc:`CEDT <../platform/acpi/cedt>`.
The :code:`target_list` parameter is filled by the CFMWS target fields. Targets
of a root decoder are `Host Bridges`, which means interleave done at the root
Such interleaves must be configured by the platform and described in the ACPI
CEDT CFMWS, as the target CXL host bridge UIDs in the CFMWS must match the CXL
-host bridge UIDs in the ACPI CEDT CHBS and ACPI DSDT.
+host bridge UIDs in the CHBS field of the :doc:`CEDT
+<../platform/acpi/cedt>` and the UID field of CXL Host Bridges defined in
+the :doc:`DSDT <../platform/acpi/dsdt>`.
-Interleave settings in a rootdecoder describe how to interleave accesses among
+Interleave settings in a root decoder describe how to interleave accesses among
the *immediate downstream targets*, not the entire interleave set.
The memory range described in the root decoder is used to
At Root
~~~~~~~
-Root decoder interleave is defined by the ACPI CEDT CFMWS. The CEDT
-may actually define multiple CFMWS configurations to describe the same
-physical capacity - with the intent to allow users to decide at runtime
-whether to online memory as interleaved or non-interleaved. ::
+Root decoder interleave is defined by CFMWS field of the :doc:`CEDT
+<../platform/acpi/cedt>`. The CEDT may actually define multiple CFMWS
+configurations to describe the same physical capacity, with the intent to allow
+users to decide at runtime whether to online memory as interleaved or
+non-interleaved. ::
Subtable Type : 01 [CXL Fixed Memory Window Structure]
Window base address : 0000000100000000
representations of the devices.
During Linux Early Boot stage (functions in the kernel that have the __init
-decorator), the system takes the resources created by EFI/BIOS (ACPI tables)
-and turns them into resources that the kernel can consume.
+decorator), the system takes the resources created by EFI/BIOS
+(:doc:`ACPI tables <../platform/acpi>`) and turns them into resources that the
+kernel can consume.
BIOS, Build and Boot Options
NUMA Node Reservation
=====================
-Linux refers to the proximity domains (:code:`PXM`) defined in the SRAT to
-create NUMA nodes in :code:`acpi_numa_init`. Typically, there is a 1:1 relation
-between :code:`PXM` and NUMA node IDs.
+Linux refers to the proximity domains (:code:`PXM`) defined in the :doc:`SRAT
+<../platform/acpi/srat>` to create NUMA nodes in :code:`acpi_numa_init`.
+Typically, there is a 1:1 relation between :code:`PXM` and NUMA node IDs.
-SRAT is the only ACPI defined way of defining Proximity Domains. Linux chooses
-to, at most, map those 1:1 with NUMA nodes. CEDT adds a description of SPA
-ranges which Linux may wish to map to one or more NUMA nodes.
+The SRAT is the only ACPI defined way of defining Proximity Domains. Linux
+chooses to, at most, map those 1:1 with NUMA nodes.
+:doc:`CEDT <../platform/acpi/cedt>` adds a description of SPA ranges which
+Linux may map to one or more NUMA nodes.
If there are CXL ranges in the CFMWS but not in SRAT, then a fake :code:`PXM`
is created (as of v6.15). In the future, Linux may reject CFMWS not described
The relevant code exists in: :code:`linux/drivers/acpi/numa/srat.c`.
-See the Example Platform Configurations section for more information.
+See :doc:`Example Platform Configurations <../platform/example-configs>`
+for more info.
Memory Tiers Creation
=====================
/sys/devices/virtual/memory_tiering/memory_tierN/nodelist
0-1
-If nodes are grouped which have clear difference in performance, check the HMAT
-and CDAT information for the CXL nodes. All nodes default to the DRAM tier,
-unless HMAT/CDAT information is reported to the memory_tier component via
-`access_coordinates`.
+If nodes are grouped which have clear difference in performance, check the
+:doc:`HMAT <../platform/acpi/hmat>` and CDAT information for the CXL nodes. All
+nodes default to the DRAM tier, unless HMAT/CDAT information is reported to the
+memory_tier component via `access_coordinates`.
+
+For more, see :doc:`CXL access coordinates documentation
+<../linux/access-coordinates>`.
Contiguous Memory Allocation
============================
Much of what this section is concerned with is ACPI Table production and
static memory map configuration. More detail on these tables can be found
-under Platform Configuration -> ACPI Table Reference.
+at :doc:`ACPI Tables <acpi>`.
.. note::
Platform Vendors should read carefully, as this sections has recommendations
Interleave and Configuration Flexibility
----------------------------------------
-If providing cross-host-bridge interleave, a CFMWS entry in the CEDT must be
-presented with target host-bridges for the interleaved device sets (there may
-be multiple behind each host bridge).
+If providing cross-host-bridge interleave, a CFMWS entry in the :doc:`CEDT
+<acpi/cedt>` must be presented with target host-bridges for the interleaved
+device sets (there may be multiple behind each host bridge).
If providing intra-host-bridge interleaving, only 1 CFMWS entry in the CEDT is
required for that host bridge - if it covers the entire capacity of the devices
A platform may choose to add all of these, or change the mode based on a BIOS
setting. For each CFMWS entry, Linux expects descriptions of the described
-memory regions in the SRAT to determine the number of NUMA nodes it should
-reserve during early boot / init.
+memory regions in the :doc:`SRAT <acpi/srat>` to determine the number of
+NUMA nodes it should reserve during early boot / init.
As of v6.14, Linux will create a NUMA node for each CEDT CFMWS entry, even if
a matching SRAT entry does not exist; however, this is not guaranteed in the
* This SRAT describes one node for each of the above CFMWS.
* The HMAT describes performance for each node in the SRAT.
-CEDT ::
+:doc:`CEDT <../acpi/cedt>`::
Subtable Type : 00 [CXL Host Bridge Structure]
Reserved : 00
QtgId : 0001
First Target : 00000006
-SRAT ::
+:doc:`SRAT <../acpi/srat>`::
Subtable Type : 01 [Memory Affinity]
Length : 28
Hot Pluggable : 1
Non-Volatile : 0
-HMAT ::
+:doc:`HMAT <../acpi/hmat>`::
Structure Type : 0001 [SLLBI]
Data Type : 00 [Latency]
Entry : 0100
Entry : 0100
-SLIT ::
+:doc:`SLIT <../acpi/slit>`::
Signature : "SLIT" [System Locality Information Table]
Localities : 0000000000000003
Locality 6 : FF FF FF FF FF FF 0A FF
Locality 7 : FF FF FF FF FF FF FF 0A
-DSDT ::
+:doc:`DSDT <../acpi/dsdt>`::
Scope (_SB)
{
* This SRAT describes one node for both host bridges.
* The HMAT describes a single node's performance.
-CEDT ::
+:doc:`CEDT <../acpi/cedt>`::
Subtable Type : 00 [CXL Host Bridge Structure]
Reserved : 00
First Target : 00000007
Second Target : 00000006
-SRAT ::
+:doc:`SRAT <../acpi/srat>`::
Subtable Type : 01 [Memory Affinity]
Length : 28
Hot Pluggable : 1
Non-Volatile : 0
-HMAT ::
+:doc:`HMAT <../acpi/hmat>`::
Structure Type : 0001 [SLLBI]
Data Type : 00 [Latency]
Entry : 1200
Entry : 0400
-SLIT ::
+:doc:`SLIT <../acpi/slit>`::
Signature : "SLIT" [System Locality Information Table]
Localities : 0000000000000003
Locality 0 : 10 20
Locality 1 : FF 0A
-DSDT ::
+:doc:`DSDT <../acpi/dsdt>`::
Scope (_SB)
{
* This CEDT/SRAT describes one node for both devices.
* There is only one proximity domain the HMAT for both devices.
-CEDT ::
+:doc:`CEDT <../acpi/cedt>`::
Subtable Type : 00 [CXL Host Bridge Structure]
Reserved : 00
QtgId : 0001
First Target : 00000007
-SRAT ::
+:doc:`SRAT <../acpi/srat>`::
Subtable Type : 01 [Memory Affinity]
Length : 28
Hot Pluggable : 1
Non-Volatile : 0
-HMAT ::
+:doc:`HMAT <../acpi/hmat>`::
Structure Type : 0001 [SLLBI]
Data Type : 00 [Latency]
Entry : 1200
Entry : 0200
-SLIT ::
+:doc:`SLIT <../acpi/slit>`::
Signature : "SLIT" [System Locality Information Table]
Localities : 0000000000000003
Locality 0 : 10 20
Locality 1 : FF 0A
-DSDT ::
+:doc:`DSDT <../acpi/dsdt>`::
Scope (_SB)
{
* This CEDT/SRAT describes one node per device
* The expanders have the same performance and will be in the same memory tier.
-CEDT ::
+:doc:`CEDT <../acpi/cedt>`::
Subtable Type : 00 [CXL Host Bridge Structure]
Reserved : 00
QtgId : 0001
First Target : 00000006
-SRAT ::
+:doc:`SRAT <../acpi/srat>`::
Subtable Type : 01 [Memory Affinity]
Length : 28
Hot Pluggable : 1
Non-Volatile : 0
-HMAT ::
+:doc:`HMAT <../acpi/hmat>`::
Structure Type : 0001 [SLLBI]
Data Type : 00 [Latency]
Entry : 0200
Entry : 0200
-SLIT ::
+:doc:`SLIT <../acpi/slit>`::
Signature : "SLIT" [System Locality Information Table]
Localities : 0000000000000003
Locality 1 : FF 0A FF
Locality 2 : FF FF 0A
-DSDT ::
+:doc:`DSDT <../acpi/dsdt>`::
Scope (_SB)
{
projects / linux-block.git / commitdiff