From: Mark Brown Date: Mon, 23 Oct 2023 18:38:22 +0000 (+0100) Subject: ASoC: Merge up v6.6-rc7 X-Git-Tag: io_uring-6.7-2023-11-10~21^2~1^2~32 X-Git-Url: https://git.kernel.dk/?a=commitdiff_plain;h=4fc4db7a68c2c04522880c4f89317f7874a4188f;hp=61f85372d293241f4139731c14cab9ac8e9436fb;p=linux-block.git ASoC: Merge up v6.6-rc7 Get fixes needed so we can enable build of ams-delta in more configurations. --- diff --git a/.mailmap b/.mailmap index a0a6efe87186..c80903efec75 100644 --- a/.mailmap +++ b/.mailmap @@ -377,6 +377,7 @@ Matthew Wilcox Matthew Wilcox Matthew Wilcox Matthias Fuchs +Matthieu Baerts Matthieu CASTET Matti Vaittinen Matt Ranostay diff --git a/Documentation/ABI/testing/sysfs-class-firmware b/Documentation/ABI/testing/sysfs-class-firmware index 978d3d500400..fba87a55f3ca 100644 --- a/Documentation/ABI/testing/sysfs-class-firmware +++ b/Documentation/ABI/testing/sysfs-class-firmware @@ -1,7 +1,7 @@ What: /sys/class/firmware/.../data Date: July 2022 KernelVersion: 5.19 -Contact: Russ Weight +Contact: Russ Weight Description: The data sysfs file is used for firmware-fallback and for firmware uploads. Cat a firmware image to this sysfs file after you echo 1 to the loading sysfs file. When the firmware @@ -13,7 +13,7 @@ Description: The data sysfs file is used for firmware-fallback and for What: /sys/class/firmware/.../cancel Date: July 2022 KernelVersion: 5.19 -Contact: Russ Weight +Contact: Russ Weight Description: Write-only. For firmware uploads, write a "1" to this file to request that the transfer of firmware data to the lower-level device be canceled. This request will be rejected (EBUSY) if @@ -23,7 +23,7 @@ Description: Write-only. For firmware uploads, write a "1" to this file to What: /sys/class/firmware/.../error Date: July 2022 KernelVersion: 5.19 -Contact: Russ Weight +Contact: Russ Weight Description: Read-only. Returns a string describing a failed firmware upload. This string will be in the form of :, where will be one of the status strings described @@ -37,7 +37,7 @@ Description: Read-only. Returns a string describing a failed firmware What: /sys/class/firmware/.../loading Date: July 2022 KernelVersion: 5.19 -Contact: Russ Weight +Contact: Russ Weight Description: The loading sysfs file is used for both firmware-fallback and for firmware uploads. Echo 1 onto the loading file to indicate you are writing a firmware file to the data sysfs node. Echo @@ -49,7 +49,7 @@ Description: The loading sysfs file is used for both firmware-fallback and What: /sys/class/firmware/.../remaining_size Date: July 2022 KernelVersion: 5.19 -Contact: Russ Weight +Contact: Russ Weight Description: Read-only. For firmware upload, this file contains the size of the firmware data that remains to be transferred to the lower-level device driver. The size value is initialized to @@ -62,7 +62,7 @@ Description: Read-only. For firmware upload, this file contains the size What: /sys/class/firmware/.../status Date: July 2022 KernelVersion: 5.19 -Contact: Russ Weight +Contact: Russ Weight Description: Read-only. Returns a string describing the current status of a firmware upload. The string will be one of the following: idle, "receiving", "preparing", "transferring", "programming". @@ -70,7 +70,7 @@ Description: Read-only. Returns a string describing the current status of What: /sys/class/firmware/.../timeout Date: July 2022 KernelVersion: 5.19 -Contact: Russ Weight +Contact: Russ Weight Description: This file supports the timeout mechanism for firmware fallback. This file has no affect on firmware uploads. For more information on timeouts please see the documentation diff --git a/Documentation/admin-guide/cgroup-v1/memory.rst b/Documentation/admin-guide/cgroup-v1/memory.rst index 5f502bf68fbc..ff456871bf4b 100644 --- a/Documentation/admin-guide/cgroup-v1/memory.rst +++ b/Documentation/admin-guide/cgroup-v1/memory.rst @@ -92,6 +92,13 @@ Brief summary of control files. memory.oom_control set/show oom controls. memory.numa_stat show the number of memory usage per numa node + memory.kmem.limit_in_bytes Deprecated knob to set and read the kernel + memory hard limit. Kernel hard limit is not + supported since 5.16. Writing any value to + do file will not have any effect same as if + nokmem kernel parameter was specified. + Kernel memory is still charged and reported + by memory.kmem.usage_in_bytes. memory.kmem.usage_in_bytes show current kernel memory allocation memory.kmem.failcnt show the number of kernel memory usage hits limits diff --git a/Documentation/arch/arm64/cpu-feature-registers.rst b/Documentation/arch/arm64/cpu-feature-registers.rst index 4e4625f2455f..de6d8a4790e2 100644 --- a/Documentation/arch/arm64/cpu-feature-registers.rst +++ b/Documentation/arch/arm64/cpu-feature-registers.rst @@ -175,6 +175,8 @@ infrastructure: +------------------------------+---------+---------+ | Name | bits | visible | +------------------------------+---------+---------+ + | SME | [27-24] | y | + +------------------------------+---------+---------+ | MTE | [11-8] | y | +------------------------------+---------+---------+ | SSBS | [7-4] | y | @@ -288,8 +290,18 @@ infrastructure: +------------------------------+---------+---------+ | Name | bits | visible | +------------------------------+---------+---------+ + | CSSC | [55-52] | y | + +------------------------------+---------+---------+ + | RPRFM | [51-48] | y | + +------------------------------+---------+---------+ + | BC | [23-20] | y | + +------------------------------+---------+---------+ | MOPS | [19-16] | y | +------------------------------+---------+---------+ + | APA3 | [15-12] | y | + +------------------------------+---------+---------+ + | GPA3 | [11-8] | y | + +------------------------------+---------+---------+ | RPRES | [7-4] | y | +------------------------------+---------+---------+ | WFXT | [3-0] | y | diff --git a/Documentation/arch/arm64/elf_hwcaps.rst b/Documentation/arch/arm64/elf_hwcaps.rst index 8c8addb4194c..76ff9d7398fd 100644 --- a/Documentation/arch/arm64/elf_hwcaps.rst +++ b/Documentation/arch/arm64/elf_hwcaps.rst @@ -305,6 +305,9 @@ HWCAP2_SMEF16F16 HWCAP2_MOPS Functionality implied by ID_AA64ISAR2_EL1.MOPS == 0b0001. +HWCAP2_HBC + Functionality implied by ID_AA64ISAR2_EL1.BC == 0b0001. + 4. Unused AT_HWCAP bits ----------------------- diff --git a/Documentation/arch/arm64/silicon-errata.rst b/Documentation/arch/arm64/silicon-errata.rst index e96f057ea2a0..f47f63bcf67c 100644 --- a/Documentation/arch/arm64/silicon-errata.rst +++ b/Documentation/arch/arm64/silicon-errata.rst @@ -71,6 +71,8 @@ stable kernels. +----------------+-----------------+-----------------+-----------------------------+ | ARM | Cortex-A510 | #2658417 | ARM64_ERRATUM_2658417 | +----------------+-----------------+-----------------+-----------------------------+ +| ARM | Cortex-A520 | #2966298 | ARM64_ERRATUM_2966298 | ++----------------+-----------------+-----------------+-----------------------------+ | ARM | Cortex-A53 | #826319 | ARM64_ERRATUM_826319 | +----------------+-----------------+-----------------+-----------------------------+ | ARM | Cortex-A53 | #827319 | ARM64_ERRATUM_827319 | diff --git a/Documentation/arch/loongarch/introduction.rst b/Documentation/arch/loongarch/introduction.rst index 49135d451ced..8c568cfc2107 100644 --- a/Documentation/arch/loongarch/introduction.rst +++ b/Documentation/arch/loongarch/introduction.rst @@ -381,9 +381,9 @@ Documentation of LoongArch ISA: Documentation of LoongArch ELF psABI: - https://github.com/loongson/LoongArch-Documentation/releases/latest/download/LoongArch-ELF-ABI-v2.00-CN.pdf (in Chinese) + https://github.com/loongson/LoongArch-Documentation/releases/latest/download/LoongArch-ELF-ABI-v2.01-CN.pdf (in Chinese) - https://github.com/loongson/LoongArch-Documentation/releases/latest/download/LoongArch-ELF-ABI-v2.00-EN.pdf (in English) + https://github.com/loongson/LoongArch-Documentation/releases/latest/download/LoongArch-ELF-ABI-v2.01-EN.pdf (in English) Linux kernel repository of Loongson and LoongArch: diff --git a/Documentation/core-api/workqueue.rst b/Documentation/core-api/workqueue.rst index 5d7b01aed1fe..0046af06531a 100644 --- a/Documentation/core-api/workqueue.rst +++ b/Documentation/core-api/workqueue.rst @@ -244,7 +244,7 @@ unbound worker-pools and only one work item could be active at any given time thus achieving the same ordering property as ST wq. In the current implementation the above configuration only guarantees -ST behavior within a given NUMA node. Instead ``alloc_ordered_queue()`` should +ST behavior within a given NUMA node. Instead ``alloc_ordered_workqueue()`` should be used to achieve system-wide ST behavior. @@ -390,7 +390,7 @@ The default affinity scope can be changed with the module parameter scope can be changed using ``apply_workqueue_attrs()``. If ``WQ_SYSFS`` is set, the workqueue will have the following affinity scope -related interface files under its ``/sys/devices/virtual/WQ_NAME/`` +related interface files under its ``/sys/devices/virtual/workqueue/WQ_NAME/`` directory. ``affinity_scope`` diff --git a/Documentation/devicetree/bindings/ata/pata-common.yaml b/Documentation/devicetree/bindings/ata/pata-common.yaml index 337ddf1113c4..4e867dd4d402 100644 --- a/Documentation/devicetree/bindings/ata/pata-common.yaml +++ b/Documentation/devicetree/bindings/ata/pata-common.yaml @@ -38,6 +38,7 @@ patternProperties: ID number 0 and the slave drive will have ID number 1. The PATA port nodes will be named "ide-port". type: object + additionalProperties: false properties: reg: diff --git a/Documentation/devicetree/bindings/bus/fsl,imx8qxp-pixel-link-msi-bus.yaml b/Documentation/devicetree/bindings/bus/fsl,imx8qxp-pixel-link-msi-bus.yaml index b568d0ce438d..7e1ffc551046 100644 --- a/Documentation/devicetree/bindings/bus/fsl,imx8qxp-pixel-link-msi-bus.yaml +++ b/Documentation/devicetree/bindings/bus/fsl,imx8qxp-pixel-link-msi-bus.yaml @@ -73,9 +73,6 @@ patternProperties: "^.*@[0-9a-f]+$": description: Devices attached to the bus type: object - properties: - reg: - maxItems: 1 required: - reg diff --git a/Documentation/devicetree/bindings/cache/andestech,ax45mp-cache.yaml b/Documentation/devicetree/bindings/cache/andestech,ax45mp-cache.yaml index 9ab5f0c435d4..d2cbe49f4e15 100644 --- a/Documentation/devicetree/bindings/cache/andestech,ax45mp-cache.yaml +++ b/Documentation/devicetree/bindings/cache/andestech,ax45mp-cache.yaml @@ -69,7 +69,7 @@ examples: - | #include - cache-controller@2010000 { + cache-controller@13400000 { compatible = "andestech,ax45mp-cache", "cache"; reg = <0x13400000 0x100000>; interrupts = <508 IRQ_TYPE_LEVEL_HIGH>; diff --git a/Documentation/devicetree/bindings/clock/renesas,5p35023.yaml b/Documentation/devicetree/bindings/clock/renesas,5p35023.yaml index 839648e753d4..42b6f80613f3 100644 --- a/Documentation/devicetree/bindings/clock/renesas,5p35023.yaml +++ b/Documentation/devicetree/bindings/clock/renesas,5p35023.yaml @@ -37,6 +37,9 @@ properties: maxItems: 1 '#clock-cells': + description: + The index in the assigned-clocks is mapped to the output clock as below + 0 - REF, 1 - SE1, 2 - SE2, 3 - SE3, 4 - DIFF1, 5 - DIFF2. const: 1 clocks: @@ -68,7 +71,7 @@ examples: reg = <0x68>; #clock-cells = <1>; - clocks = <&x1_x2>; + clocks = <&x1>; renesas,settings = [ 80 00 11 19 4c 02 23 7f 83 19 08 a9 5f 25 24 bf @@ -79,8 +82,8 @@ examples: assigned-clocks = <&versa3 0>, <&versa3 1>, <&versa3 2>, <&versa3 3>, <&versa3 4>, <&versa3 5>; - assigned-clock-rates = <12288000>, <25000000>, - <12000000>, <11289600>, - <11289600>, <24000000>; + assigned-clock-rates = <24000000>, <11289600>, + <11289600>, <12000000>, + <25000000>, <12288000>; }; }; diff --git a/Documentation/devicetree/bindings/display/imx/fsl,imx6-hdmi.yaml b/Documentation/devicetree/bindings/display/imx/fsl,imx6-hdmi.yaml index af7fe9c4d196..7979cf07f119 100644 --- a/Documentation/devicetree/bindings/display/imx/fsl,imx6-hdmi.yaml +++ b/Documentation/devicetree/bindings/display/imx/fsl,imx6-hdmi.yaml @@ -87,7 +87,7 @@ required: - interrupts - ports -additionalProperties: false +unevaluatedProperties: false examples: - | diff --git a/Documentation/devicetree/bindings/dma/xilinx/xlnx,zynqmp-dma-1.0.yaml b/Documentation/devicetree/bindings/dma/xilinx/xlnx,zynqmp-dma-1.0.yaml index 23ada8f87526..769ce23aaac2 100644 --- a/Documentation/devicetree/bindings/dma/xilinx/xlnx,zynqmp-dma-1.0.yaml +++ b/Documentation/devicetree/bindings/dma/xilinx/xlnx,zynqmp-dma-1.0.yaml @@ -13,6 +13,8 @@ description: | maintainers: - Michael Tretter + - Harini Katakam + - Radhey Shyam Pandey allOf: - $ref: ../dma-controller.yaml# @@ -65,6 +67,7 @@ required: - interrupts - clocks - clock-names + - xlnx,bus-width additionalProperties: false diff --git a/Documentation/devicetree/bindings/i2c/i2c-mxs.yaml b/Documentation/devicetree/bindings/i2c/i2c-mxs.yaml index 21ae7bce038e..171a41407241 100644 --- a/Documentation/devicetree/bindings/i2c/i2c-mxs.yaml +++ b/Documentation/devicetree/bindings/i2c/i2c-mxs.yaml @@ -9,6 +9,9 @@ title: Freescale MXS Inter IC (I2C) Controller maintainers: - Shawn Guo +allOf: + - $ref: /schemas/i2c/i2c-controller.yaml# + properties: compatible: enum: @@ -37,7 +40,7 @@ required: - dmas - dma-names -additionalProperties: false +unevaluatedProperties: false examples: - | diff --git a/Documentation/devicetree/bindings/iio/adc/adi,ad7292.yaml b/Documentation/devicetree/bindings/iio/adc/adi,ad7292.yaml index 7cc4ddc4e9b7..2aa1f4b063eb 100644 --- a/Documentation/devicetree/bindings/iio/adc/adi,ad7292.yaml +++ b/Documentation/devicetree/bindings/iio/adc/adi,ad7292.yaml @@ -61,7 +61,7 @@ patternProperties: required: - reg - additionalProperties: true + additionalProperties: false allOf: - $ref: /schemas/spi/spi-peripheral-props.yaml# diff --git a/Documentation/devicetree/bindings/iio/light/rohm,bu27010.yaml b/Documentation/devicetree/bindings/iio/light/rohm,bu27010.yaml index 8376d64a641a..bed42d5d0d94 100644 --- a/Documentation/devicetree/bindings/iio/light/rohm,bu27010.yaml +++ b/Documentation/devicetree/bindings/iio/light/rohm,bu27010.yaml @@ -45,5 +45,6 @@ examples: light-sensor@38 { compatible = "rohm,bu27010"; reg = <0x38>; + vdd-supply = <&vdd>; }; }; diff --git a/Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.yaml b/Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.yaml index 2bc38479a41e..0f4a062c9d6f 100644 --- a/Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.yaml +++ b/Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.yaml @@ -106,6 +106,12 @@ properties: $ref: /schemas/types.yaml#/definitions/uint32 maximum: 4096 + dma-noncoherent: + description: + Present if the GIC redistributors permit programming shareability + and cacheability attributes but are connected to a non-coherent + downstream interconnect. + msi-controller: description: Only present if the Message Based Interrupt functionality is @@ -193,6 +199,12 @@ patternProperties: compatible: const: arm,gic-v3-its + dma-noncoherent: + description: + Present if the GIC ITS permits programming shareability and + cacheability attributes but is connected to a non-coherent + downstream interconnect. + msi-controller: true "#msi-cells": diff --git a/Documentation/devicetree/bindings/interrupt-controller/renesas,irqc.yaml b/Documentation/devicetree/bindings/interrupt-controller/renesas,irqc.yaml index 95033cb514fb..b417341fc8ae 100644 --- a/Documentation/devicetree/bindings/interrupt-controller/renesas,irqc.yaml +++ b/Documentation/devicetree/bindings/interrupt-controller/renesas,irqc.yaml @@ -37,6 +37,7 @@ properties: - renesas,intc-ex-r8a77990 # R-Car E3 - renesas,intc-ex-r8a77995 # R-Car D3 - renesas,intc-ex-r8a779a0 # R-Car V3U + - renesas,intc-ex-r8a779f0 # R-Car S4-8 - renesas,intc-ex-r8a779g0 # R-Car V4H - const: renesas,irqc diff --git a/Documentation/devicetree/bindings/interrupt-controller/renesas,rzg2l-irqc.yaml b/Documentation/devicetree/bindings/interrupt-controller/renesas,rzg2l-irqc.yaml index 33b90e975e33..2ef3081eaaf3 100644 --- a/Documentation/devicetree/bindings/interrupt-controller/renesas,rzg2l-irqc.yaml +++ b/Documentation/devicetree/bindings/interrupt-controller/renesas,rzg2l-irqc.yaml @@ -19,20 +19,19 @@ description: | - NMI edge select (NMI is not treated as NMI exception and supports fall edge and stand-up edge detection interrupts) -allOf: - - $ref: /schemas/interrupt-controller.yaml# - properties: compatible: items: - enum: + - renesas,r9a07g043u-irqc # RZ/G2UL - renesas,r9a07g044-irqc # RZ/G2{L,LC} - renesas,r9a07g054-irqc # RZ/V2L - const: renesas,rzg2l-irqc '#interrupt-cells': - description: The first cell should contain external interrupt number (IRQ0-7) and the - second cell is used to specify the flag. + description: The first cell should contain a macro RZG2L_{NMI,IRQX} included in the + include/dt-bindings/interrupt-controller/irqc-rzg2l.h and the second + cell is used to specify the flag. const: 2 '#address-cells': @@ -44,7 +43,96 @@ properties: maxItems: 1 interrupts: - maxItems: 41 + minItems: 41 + items: + - description: NMI interrupt + - description: IRQ0 interrupt + - description: IRQ1 interrupt + - description: IRQ2 interrupt + - description: IRQ3 interrupt + - description: IRQ4 interrupt + - description: IRQ5 interrupt + - description: IRQ6 interrupt + - description: IRQ7 interrupt + - description: GPIO interrupt, TINT0 + - description: GPIO interrupt, TINT1 + - description: GPIO interrupt, TINT2 + - description: GPIO interrupt, TINT3 + - description: GPIO interrupt, TINT4 + - description: GPIO interrupt, TINT5 + - description: GPIO interrupt, TINT6 + - description: GPIO interrupt, TINT7 + - description: GPIO interrupt, TINT8 + - description: GPIO interrupt, TINT9 + - description: GPIO interrupt, TINT10 + - description: GPIO interrupt, TINT11 + - description: GPIO interrupt, TINT12 + - description: GPIO interrupt, TINT13 + - description: GPIO interrupt, TINT14 + - description: GPIO interrupt, TINT15 + - description: GPIO interrupt, TINT16 + - description: GPIO interrupt, TINT17 + - description: GPIO interrupt, TINT18 + - description: GPIO interrupt, TINT19 + - description: GPIO interrupt, TINT20 + - description: GPIO interrupt, TINT21 + - description: GPIO interrupt, TINT22 + - description: GPIO interrupt, TINT23 + - description: GPIO interrupt, TINT24 + - description: GPIO interrupt, TINT25 + - description: GPIO interrupt, TINT26 + - description: GPIO interrupt, TINT27 + - description: GPIO interrupt, TINT28 + - description: GPIO interrupt, TINT29 + - description: GPIO interrupt, TINT30 + - description: GPIO interrupt, TINT31 + - description: Bus error interrupt + + interrupt-names: + minItems: 41 + items: + - const: nmi + - const: irq0 + - const: irq1 + - const: irq2 + - const: irq3 + - const: irq4 + - const: irq5 + - const: irq6 + - const: irq7 + - const: tint0 + - const: tint1 + - const: tint2 + - const: tint3 + - const: tint4 + - const: tint5 + - const: tint6 + - const: tint7 + - const: tint8 + - const: tint9 + - const: tint10 + - const: tint11 + - const: tint12 + - const: tint13 + - const: tint14 + - const: tint15 + - const: tint16 + - const: tint17 + - const: tint18 + - const: tint19 + - const: tint20 + - const: tint21 + - const: tint22 + - const: tint23 + - const: tint24 + - const: tint25 + - const: tint26 + - const: tint27 + - const: tint28 + - const: tint29 + - const: tint30 + - const: tint31 + - const: bus-err clocks: maxItems: 2 @@ -72,6 +160,23 @@ required: - power-domains - resets +allOf: + - $ref: /schemas/interrupt-controller.yaml# + + - if: + properties: + compatible: + contains: + const: renesas,r9a07g043u-irqc + then: + properties: + interrupts: + minItems: 42 + interrupt-names: + minItems: 42 + required: + - interrupt-names + unevaluatedProperties: false examples: @@ -80,55 +185,66 @@ examples: #include irqc: interrupt-controller@110a0000 { - compatible = "renesas,r9a07g044-irqc", "renesas,rzg2l-irqc"; - reg = <0x110a0000 0x10000>; - #interrupt-cells = <2>; - #address-cells = <0>; - interrupt-controller; - interrupts = , - , - , - , - , - , - , - , - , - , - , - , - , - , - , - , - , - , - , - , - , - , - , - , - , - , - , - , - , - , - , - , - , - , - , - , - , - , - , - , - ; - clocks = <&cpg CPG_MOD R9A07G044_IA55_CLK>, - <&cpg CPG_MOD R9A07G044_IA55_PCLK>; - clock-names = "clk", "pclk"; - power-domains = <&cpg>; - resets = <&cpg R9A07G044_IA55_RESETN>; + compatible = "renesas,r9a07g044-irqc", "renesas,rzg2l-irqc"; + reg = <0x110a0000 0x10000>; + #interrupt-cells = <2>; + #address-cells = <0>; + interrupt-controller; + interrupts = , + , + , + , + , + , + , + , + , + , + , + , + , + , + , + , + , + , + , + , + , + , + , + , + , + , + , + , + , + , + , + , + , + , + , + , + , + , + , + , + ; + interrupt-names = "nmi", + "irq0", "irq1", "irq2", "irq3", + "irq4", "irq5", "irq6", "irq7", + "tint0", "tint1", "tint2", "tint3", + "tint4", "tint5", "tint6", "tint7", + "tint8", "tint9", "tint10", "tint11", + "tint12", "tint13", "tint14", "tint15", + "tint16", "tint17", "tint18", "tint19", + "tint20", "tint21", "tint22", "tint23", + "tint24", "tint25", "tint26", "tint27", + "tint28", "tint29", "tint30", "tint31"; + clocks = <&cpg CPG_MOD R9A07G044_IA55_CLK>, + <&cpg CPG_MOD R9A07G044_IA55_PCLK>; + clock-names = "clk", "pclk"; + power-domains = <&cpg>; + resets = <&cpg R9A07G044_IA55_RESETN>; }; diff --git a/Documentation/devicetree/bindings/iommu/arm,smmu.yaml b/Documentation/devicetree/bindings/iommu/arm,smmu.yaml index cf29ab10501c..b1b2cf81b42f 100644 --- a/Documentation/devicetree/bindings/iommu/arm,smmu.yaml +++ b/Documentation/devicetree/bindings/iommu/arm,smmu.yaml @@ -270,6 +270,7 @@ allOf: contains: enum: - qcom,msm8998-smmu-v2 + - qcom,sdm630-smmu-v2 then: anyOf: - properties: @@ -311,7 +312,6 @@ allOf: compatible: contains: enum: - - qcom,sdm630-smmu-v2 - qcom,sm6375-smmu-v2 then: anyOf: diff --git a/Documentation/devicetree/bindings/media/i2c/sony,imx415.yaml b/Documentation/devicetree/bindings/media/i2c/sony,imx415.yaml index ffccf5f3c9e3..642f9b15d359 100644 --- a/Documentation/devicetree/bindings/media/i2c/sony,imx415.yaml +++ b/Documentation/devicetree/bindings/media/i2c/sony,imx415.yaml @@ -54,6 +54,7 @@ properties: port: $ref: /schemas/graph.yaml#/$defs/port-base + unevaluatedProperties: false properties: endpoint: diff --git a/Documentation/devicetree/bindings/media/i2c/toshiba,tc358746.yaml b/Documentation/devicetree/bindings/media/i2c/toshiba,tc358746.yaml index c5cab549ee8e..1c476b635b69 100644 --- a/Documentation/devicetree/bindings/media/i2c/toshiba,tc358746.yaml +++ b/Documentation/devicetree/bindings/media/i2c/toshiba,tc358746.yaml @@ -69,6 +69,7 @@ properties: properties: port@0: $ref: /schemas/graph.yaml#/$defs/port-base + unevaluatedProperties: false description: Input port properties: @@ -89,6 +90,7 @@ properties: port@1: $ref: /schemas/graph.yaml#/$defs/port-base + unevaluatedProperties: false description: Output port properties: diff --git a/Documentation/devicetree/bindings/media/nxp,imx7-csi.yaml b/Documentation/devicetree/bindings/media/nxp,imx7-csi.yaml index 358019e85d90..326284e151f6 100644 --- a/Documentation/devicetree/bindings/media/nxp,imx7-csi.yaml +++ b/Documentation/devicetree/bindings/media/nxp,imx7-csi.yaml @@ -59,7 +59,6 @@ allOf: compatible: contains: enum: - - fsl,imx8mq-csi - fsl,imx8mm-csi then: required: diff --git a/Documentation/devicetree/bindings/media/renesas,vin.yaml b/Documentation/devicetree/bindings/media/renesas,vin.yaml index 324703bfb1bd..5539d0f8e74d 100644 --- a/Documentation/devicetree/bindings/media/renesas,vin.yaml +++ b/Documentation/devicetree/bindings/media/renesas,vin.yaml @@ -95,7 +95,7 @@ properties: synchronization is selected. default: 1 - field-active-even: true + field-even-active: true bus-width: true @@ -144,7 +144,7 @@ properties: synchronization is selected. default: 1 - field-active-even: true + field-even-active: true bus-width: true diff --git a/Documentation/devicetree/bindings/media/samsung,fimc.yaml b/Documentation/devicetree/bindings/media/samsung,fimc.yaml index 79ff6d83a9fd..b3486c38a05b 100644 --- a/Documentation/devicetree/bindings/media/samsung,fimc.yaml +++ b/Documentation/devicetree/bindings/media/samsung,fimc.yaml @@ -57,6 +57,7 @@ properties: patternProperties: "^port@[01]$": $ref: /schemas/graph.yaml#/$defs/port-base + unevaluatedProperties: false description: Camera A and camera B inputs. diff --git a/Documentation/devicetree/bindings/mfd/maxim,max77693.yaml b/Documentation/devicetree/bindings/mfd/maxim,max77693.yaml index 9804d13de648..6a6f222b868f 100644 --- a/Documentation/devicetree/bindings/mfd/maxim,max77693.yaml +++ b/Documentation/devicetree/bindings/mfd/maxim,max77693.yaml @@ -31,10 +31,6 @@ properties: charger: $ref: /schemas/power/supply/maxim,max77693.yaml - connector: - $ref: /schemas/connector/usb-connector.yaml# - unevaluatedProperties: false - led: $ref: /schemas/leds/maxim,max77693.yaml diff --git a/Documentation/devicetree/bindings/mmc/sdhci-msm.yaml b/Documentation/devicetree/bindings/mmc/sdhci-msm.yaml index 80141eb7fc6b..10f34aa8ba8a 100644 --- a/Documentation/devicetree/bindings/mmc/sdhci-msm.yaml +++ b/Documentation/devicetree/bindings/mmc/sdhci-msm.yaml @@ -69,7 +69,7 @@ properties: maxItems: 4 clocks: - minItems: 3 + minItems: 2 items: - description: Main peripheral bus clock, PCLK/HCLK - AHB Bus clock - description: SDC MMC clock, MCLK diff --git a/Documentation/devicetree/bindings/pci/brcm,iproc-pcie.yaml b/Documentation/devicetree/bindings/pci/brcm,iproc-pcie.yaml index 0972868735fc..0e07ab61a48d 100644 --- a/Documentation/devicetree/bindings/pci/brcm,iproc-pcie.yaml +++ b/Documentation/devicetree/bindings/pci/brcm,iproc-pcie.yaml @@ -12,7 +12,6 @@ maintainers: allOf: - $ref: /schemas/pci/pci-bus.yaml# - - $ref: /schemas/interrupt-controller/msi-controller.yaml# properties: compatible: @@ -34,13 +33,6 @@ properties: description: > Base address and length of the PCIe controller I/O register space - interrupt-map: true - - interrupt-map-mask: true - - "#interrupt-cells": - const: 1 - ranges: minItems: 1 maxItems: 2 @@ -54,16 +46,8 @@ properties: items: - const: pcie-phy - bus-range: true - dma-coherent: true - "#address-cells": true - - "#size-cells": true - - device_type: true - brcm,pcie-ob: type: boolean description: > @@ -78,20 +62,24 @@ properties: msi: type: object + $ref: /schemas/interrupt-controller/msi-controller.yaml# + unevaluatedProperties: false + properties: compatible: items: - const: brcm,iproc-msi - msi-parent: true + interrupts: + maxItems: 4 - msi-controller: true + brcm,pcie-msi-inten: + type: boolean + description: + Needs to be present for some older iProc platforms that require the + interrupt enable registers to be set explicitly to enable MSI - brcm,pcie-msi-inten: - type: boolean - description: > - Needs to be present for some older iProc platforms that require the - interrupt enable registers to be set explicitly to enable MSI + msi-parent: true dependencies: brcm,pcie-ob-axi-offset: ["brcm,pcie-ob"] @@ -117,68 +105,69 @@ unevaluatedProperties: false examples: - | - #include - - bus { - #address-cells = <1>; - #size-cells = <1>; - pcie0: pcie@18012000 { - compatible = "brcm,iproc-pcie"; - reg = <0x18012000 0x1000>; - - #interrupt-cells = <1>; - interrupt-map-mask = <0 0 0 0>; - interrupt-map = <0 0 0 0 &gic GIC_SPI 100 IRQ_TYPE_NONE>; - - linux,pci-domain = <0>; - - bus-range = <0x00 0xff>; - - #address-cells = <3>; - #size-cells = <2>; - device_type = "pci"; - ranges = <0x81000000 0 0 0x28000000 0 0x00010000>, - <0x82000000 0 0x20000000 0x20000000 0 0x04000000>; - - phys = <&phy 0 5>; - phy-names = "pcie-phy"; - - brcm,pcie-ob; - brcm,pcie-ob-axi-offset = <0x00000000>; - - msi-parent = <&msi0>; - - /* iProc event queue based MSI */ - msi0: msi { - compatible = "brcm,iproc-msi"; - msi-controller; - interrupt-parent = <&gic>; - interrupts = , - , - , - ; - }; - }; - - pcie1: pcie@18013000 { - compatible = "brcm,iproc-pcie"; - reg = <0x18013000 0x1000>; - - #interrupt-cells = <1>; - interrupt-map-mask = <0 0 0 0>; - interrupt-map = <0 0 0 0 &gic GIC_SPI 106 IRQ_TYPE_NONE>; - - linux,pci-domain = <1>; - - bus-range = <0x00 0xff>; - - #address-cells = <3>; - #size-cells = <2>; - device_type = "pci"; - ranges = <0x81000000 0 0 0x48000000 0 0x00010000>, - <0x82000000 0 0x40000000 0x40000000 0 0x04000000>; - - phys = <&phy 1 6>; - phy-names = "pcie-phy"; - }; + #include + + gic: interrupt-controller { + interrupt-controller; + #interrupt-cells = <3>; + }; + + pcie@18012000 { + compatible = "brcm,iproc-pcie"; + reg = <0x18012000 0x1000>; + + #interrupt-cells = <1>; + interrupt-map-mask = <0 0 0 0>; + interrupt-map = <0 0 0 0 &gic GIC_SPI 100 IRQ_TYPE_NONE>; + + linux,pci-domain = <0>; + + bus-range = <0x00 0xff>; + + #address-cells = <3>; + #size-cells = <2>; + device_type = "pci"; + ranges = <0x81000000 0 0 0x28000000 0 0x00010000>, + <0x82000000 0 0x20000000 0x20000000 0 0x04000000>; + + phys = <&phy 0 5>; + phy-names = "pcie-phy"; + + brcm,pcie-ob; + brcm,pcie-ob-axi-offset = <0x00000000>; + + msi-parent = <&msi0>; + + /* iProc event queue based MSI */ + msi0: msi { + compatible = "brcm,iproc-msi"; + msi-controller; + interrupt-parent = <&gic>; + interrupts = , + , + , + ; + }; + }; + - | + pcie@18013000 { + compatible = "brcm,iproc-pcie"; + reg = <0x18013000 0x1000>; + + #interrupt-cells = <1>; + interrupt-map-mask = <0 0 0 0>; + interrupt-map = <0 0 0 0 &gic GIC_SPI 106 IRQ_TYPE_NONE>; + + linux,pci-domain = <1>; + + bus-range = <0x00 0xff>; + + #address-cells = <3>; + #size-cells = <2>; + device_type = "pci"; + ranges = <0x81000000 0 0 0x48000000 0 0x00010000>, + <0x82000000 0 0x40000000 0x40000000 0 0x04000000>; + + phys = <&phy 1 6>; + phy-names = "pcie-phy"; }; diff --git a/Documentation/devicetree/bindings/phy/qcom,ipq8074-qmp-pcie-phy.yaml b/Documentation/devicetree/bindings/phy/qcom,ipq8074-qmp-pcie-phy.yaml index 5073007267ad..634cec5d57ea 100644 --- a/Documentation/devicetree/bindings/phy/qcom,ipq8074-qmp-pcie-phy.yaml +++ b/Documentation/devicetree/bindings/phy/qcom,ipq8074-qmp-pcie-phy.yaml @@ -70,7 +70,7 @@ examples: phy@84000 { compatible = "qcom,ipq6018-qmp-pcie-phy"; - reg = <0x0 0x00084000 0x0 0x1000>; + reg = <0x00084000 0x1000>; clocks = <&gcc GCC_PCIE0_AUX_CLK>, <&gcc GCC_PCIE0_AHB_CLK>, diff --git a/Documentation/devicetree/bindings/riscv/cpus.yaml b/Documentation/devicetree/bindings/riscv/cpus.yaml index 38c0b5213736..97e8441eda1c 100644 --- a/Documentation/devicetree/bindings/riscv/cpus.yaml +++ b/Documentation/devicetree/bindings/riscv/cpus.yaml @@ -91,6 +91,7 @@ properties: interrupt-controller: type: object + additionalProperties: false description: Describes the CPU's local interrupt controller properties: diff --git a/Documentation/devicetree/bindings/soc/loongson/loongson,ls2k-pmc.yaml b/Documentation/devicetree/bindings/soc/loongson/loongson,ls2k-pmc.yaml index da2dcfeebf12..510f6cb0f084 100644 --- a/Documentation/devicetree/bindings/soc/loongson/loongson,ls2k-pmc.yaml +++ b/Documentation/devicetree/bindings/soc/loongson/loongson,ls2k-pmc.yaml @@ -11,11 +11,16 @@ maintainers: properties: compatible: - items: - - enum: - - loongson,ls2k0500-pmc - - loongson,ls2k1000-pmc - - const: syscon + oneOf: + - items: + - const: loongson,ls2k0500-pmc + - const: syscon + - items: + - enum: + - loongson,ls2k1000-pmc + - loongson,ls2k2000-pmc + - const: loongson,ls2k0500-pmc + - const: syscon reg: maxItems: 1 @@ -32,6 +37,18 @@ properties: addition, the PM need according to it to indicate that current SoC whether support Suspend To RAM. + syscon-poweroff: + $ref: /schemas/power/reset/syscon-poweroff.yaml# + type: object + description: + Node for power off method + + syscon-reboot: + $ref: /schemas/power/reset/syscon-reboot.yaml# + type: object + description: + Node for reboot method + required: - compatible - reg @@ -44,9 +61,23 @@ examples: #include power-management@1fe27000 { - compatible = "loongson,ls2k1000-pmc", "syscon"; + compatible = "loongson,ls2k1000-pmc", "loongson,ls2k0500-pmc", "syscon"; reg = <0x1fe27000 0x58>; interrupt-parent = <&liointc1>; interrupts = <11 IRQ_TYPE_LEVEL_LOW>; loongson,suspend-address = <0x0 0x1c000500>; + + syscon-reboot { + compatible = "syscon-reboot"; + offset = <0x30>; + mask = <0x1>; + }; + + syscon-poweroff { + compatible = "syscon-poweroff"; + regmap = <&pmc>; + offset = <0x14>; + mask = <0x3c00>; + value = <0x3c00>; + }; }; diff --git a/Documentation/devicetree/bindings/spi/fsl-imx-cspi.yaml b/Documentation/devicetree/bindings/spi/fsl-imx-cspi.yaml index 2f593c7225e5..14cac0e6e0a1 100644 --- a/Documentation/devicetree/bindings/spi/fsl-imx-cspi.yaml +++ b/Documentation/devicetree/bindings/spi/fsl-imx-cspi.yaml @@ -22,6 +22,13 @@ properties: - const: fsl,imx35-cspi - const: fsl,imx51-ecspi - const: fsl,imx53-ecspi + - items: + - enum: + - fsl,imx25-cspi + - fsl,imx50-cspi + - fsl,imx51-cspi + - fsl,imx53-cspi + - const: fsl,imx35-cspi - items: - const: fsl,imx8mp-ecspi - const: fsl,imx6ul-ecspi diff --git a/Documentation/devicetree/bindings/trivial-devices.yaml b/Documentation/devicetree/bindings/trivial-devices.yaml index cd58179ae337..430a814f64a5 100644 --- a/Documentation/devicetree/bindings/trivial-devices.yaml +++ b/Documentation/devicetree/bindings/trivial-devices.yaml @@ -232,7 +232,7 @@ properties: # MEMSIC magnetometer - memsic,mmc35240 # MEMSIC 3-axis accelerometer - - memsic,mx4005 + - memsic,mxc4005 # MEMSIC 2-axis 8-bit digital accelerometer - memsic,mxc6225 # MEMSIC 2-axis 8-bit digital accelerometer diff --git a/Documentation/filesystems/btrfs.rst b/Documentation/filesystems/btrfs.rst index 992eddb0e11b..a81db8f54d68 100644 --- a/Documentation/filesystems/btrfs.rst +++ b/Documentation/filesystems/btrfs.rst @@ -37,7 +37,6 @@ For more information please refer to the documentation site or wiki https://btrfs.readthedocs.io - https://btrfs.wiki.kernel.org that maintains information about administration tasks, frequently asked questions, use cases, mount options, comprehensible changelogs, features, diff --git a/Documentation/filesystems/erofs.rst b/Documentation/filesystems/erofs.rst index 4654ee57c1d5..f200d7874495 100644 --- a/Documentation/filesystems/erofs.rst +++ b/Documentation/filesystems/erofs.rst @@ -58,12 +58,14 @@ Here are the main features of EROFS: - Support extended attributes as an option; + - Support a bloom filter that speeds up negative extended attribute lookups; + - Support POSIX.1e ACLs by using extended attributes; - Support transparent data compression as an option: - LZ4 and MicroLZMA algorithms can be used on a per-file basis; In addition, - inplace decompression is also supported to avoid bounce compressed buffers - and page cache thrashing. + LZ4, MicroLZMA and DEFLATE algorithms can be used on a per-file basis; In + addition, inplace decompression is also supported to avoid bounce compressed + buffers and unnecessary page cache thrashing. - Support chunk-based data deduplication and rolling-hash compressed data deduplication; @@ -268,6 +270,38 @@ details.) By the way, chunk-based files are all uncompressed for now. +Long extended attribute name prefixes +------------------------------------- +There are use cases where extended attributes with different values can have +only a few common prefixes (such as overlayfs xattrs). The predefined prefixes +work inefficiently in both image size and runtime performance in such cases. + +The long xattr name prefixes feature is introduced to address this issue. The +overall idea is that, apart from the existing predefined prefixes, the xattr +entry could also refer to user-specified long xattr name prefixes, e.g. +"trusted.overlay.". + +When referring to a long xattr name prefix, the highest bit (bit 7) of +erofs_xattr_entry.e_name_index is set, while the lower bits (bit 0-6) as a whole +represent the index of the referred long name prefix among all long name +prefixes. Therefore, only the trailing part of the name apart from the long +xattr name prefix is stored in erofs_xattr_entry.e_name, which could be empty if +the full xattr name matches exactly as its long xattr name prefix. + +All long xattr prefixes are stored one by one in the packed inode as long as +the packed inode is valid, or in the meta inode otherwise. The +xattr_prefix_count (of the on-disk superblock) indicates the total number of +long xattr name prefixes, while (xattr_prefix_start * 4) indicates the start +offset of long name prefixes in the packed/meta inode. Note that, long extended +attribute name prefixes are disabled if xattr_prefix_count is 0. + +Each long name prefix is stored in the format: ALIGN({__le16 len, data}, 4), +where len represents the total size of the data part. The data part is actually +represented by 'struct erofs_xattr_long_prefix', where base_index represents the +index of the predefined xattr name prefix, e.g. EROFS_XATTR_INDEX_TRUSTED for +"trusted.overlay." long name prefix, while the infix string keeps the string +after stripping the short prefix, e.g. "overlay." for the example above. + Data compression ---------------- EROFS implements fixed-sized output compression which generates fixed-sized diff --git a/Documentation/filesystems/overlayfs.rst b/Documentation/filesystems/overlayfs.rst index cdefbe73d85c..5b93268e400f 100644 --- a/Documentation/filesystems/overlayfs.rst +++ b/Documentation/filesystems/overlayfs.rst @@ -339,6 +339,18 @@ The specified lower directories will be stacked beginning from the rightmost one and going left. In the above example lower1 will be the top, lower2 the middle and lower3 the bottom layer. +Note: directory names containing colons can be provided as lower layer by +escaping the colons with a single backslash. For example: + + mount -t overlay overlay -olowerdir=/a\:lower\:\:dir /merged + +Since kernel version v6.5, directory names containing colons can also +be provided as lower layer using the fsconfig syscall from new mount api: + + fsconfig(fs_fd, FSCONFIG_SET_STRING, "lowerdir", "/a:lower::dir", 0); + +In the latter case, colons in lower layer directory names will be escaped +as an octal characters (\072) when displayed in /proc/self/mountinfo. Metadata only copy up --------------------- diff --git a/Documentation/filesystems/porting.rst b/Documentation/filesystems/porting.rst index deac4e973ddc..4d05b9862451 100644 --- a/Documentation/filesystems/porting.rst +++ b/Documentation/filesystems/porting.rst @@ -949,3 +949,99 @@ mmap_lock held. All in-tree users have been audited and do not seem to depend on the mmap_lock being held, but out of tree users should verify for themselves. If they do need it, they can return VM_FAULT_RETRY to be called with the mmap_lock held. + +--- + +**mandatory** + +The order of opening block devices and matching or creating superblocks has +changed. + +The old logic opened block devices first and then tried to find a +suitable superblock to reuse based on the block device pointer. + +The new logic tries to find a suitable superblock first based on the device +number, and opening the block device afterwards. + +Since opening block devices cannot happen under s_umount because of lock +ordering requirements s_umount is now dropped while opening block devices and +reacquired before calling fill_super(). + +In the old logic concurrent mounters would find the superblock on the list of +superblocks for the filesystem type. Since the first opener of the block device +would hold s_umount they would wait until the superblock became either born or +was discarded due to initialization failure. + +Since the new logic drops s_umount concurrent mounters could grab s_umount and +would spin. Instead they are now made to wait using an explicit wait-wake +mechanism without having to hold s_umount. + +--- + +**mandatory** + +The holder of a block device is now the superblock. + +The holder of a block device used to be the file_system_type which wasn't +particularly useful. It wasn't possible to go from block device to owning +superblock without matching on the device pointer stored in the superblock. +This mechanism would only work for a single device so the block layer couldn't +find the owning superblock of any additional devices. + +In the old mechanism reusing or creating a superblock for a racing mount(2) and +umount(2) relied on the file_system_type as the holder. This was severly +underdocumented however: + +(1) Any concurrent mounter that managed to grab an active reference on an + existing superblock was made to wait until the superblock either became + ready or until the superblock was removed from the list of superblocks of + the filesystem type. If the superblock is ready the caller would simple + reuse it. + +(2) If the mounter came after deactivate_locked_super() but before + the superblock had been removed from the list of superblocks of the + filesystem type the mounter would wait until the superblock was shutdown, + reuse the block device and allocate a new superblock. + +(3) If the mounter came after deactivate_locked_super() and after + the superblock had been removed from the list of superblocks of the + filesystem type the mounter would reuse the block device and allocate a new + superblock (the bd_holder point may still be set to the filesystem type). + +Because the holder of the block device was the file_system_type any concurrent +mounter could open the block devices of any superblock of the same +file_system_type without risking seeing EBUSY because the block device was +still in use by another superblock. + +Making the superblock the owner of the block device changes this as the holder +is now a unique superblock and thus block devices associated with it cannot be +reused by concurrent mounters. So a concurrent mounter in (2) could suddenly +see EBUSY when trying to open a block device whose holder was a different +superblock. + +The new logic thus waits until the superblock and the devices are shutdown in +->kill_sb(). Removal of the superblock from the list of superblocks of the +filesystem type is now moved to a later point when the devices are closed: + +(1) Any concurrent mounter managing to grab an active reference on an existing + superblock is made to wait until the superblock is either ready or until + the superblock and all devices are shutdown in ->kill_sb(). If the + superblock is ready the caller will simply reuse it. + +(2) If the mounter comes after deactivate_locked_super() but before + the superblock has been removed from the list of superblocks of the + filesystem type the mounter is made to wait until the superblock and the + devices are shut down in ->kill_sb() and the superblock is removed from the + list of superblocks of the filesystem type. The mounter will allocate a new + superblock and grab ownership of the block device (the bd_holder pointer of + the block device will be set to the newly allocated superblock). + +(3) This case is now collapsed into (2) as the superblock is left on the list + of superblocks of the filesystem type until all devices are shutdown in + ->kill_sb(). In other words, if the superblock isn't on the list of + superblock of the filesystem type anymore then it has given up ownership of + all associated block devices (the bd_holder pointer is NULL). + +As this is a VFS level change it has no practical consequences for filesystems +other than that all of them must use one of the provided kill_litter_super(), +kill_anon_super(), or kill_block_super() helpers. diff --git a/Documentation/kbuild/kconfig-language.rst b/Documentation/kbuild/kconfig-language.rst index 858ed5d80def..0135905c0aa3 100644 --- a/Documentation/kbuild/kconfig-language.rst +++ b/Documentation/kbuild/kconfig-language.rst @@ -573,6 +573,32 @@ above, leading to: bool "Support for foo hardware" depends on ARCH_FOO_VENDOR || COMPILE_TEST +Optional dependencies +~~~~~~~~~~~~~~~~~~~~~ + +Some drivers are able to optionally use a feature from another module +or build cleanly with that module disabled, but cause a link failure +when trying to use that loadable module from a built-in driver. + +The most common way to express this optional dependency in Kconfig logic +uses the slightly counterintuitive:: + + config FOO + tristate "Support for foo hardware" + depends on BAR || !BAR + +This means that there is either a dependency on BAR that disallows +the combination of FOO=y with BAR=m, or BAR is completely disabled. +For a more formalized approach if there are multiple drivers that have +the same dependency, a helper symbol can be used, like:: + + config FOO + tristate "Support for foo hardware" + depends on BAR_OPTIONAL + + config BAR_OPTIONAL + def_tristate BAR || !BAR + Kconfig recursive dependency limitations ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/Documentation/netlink/specs/devlink.yaml b/Documentation/netlink/specs/devlink.yaml index d1ebcd927149..065661acb878 100644 --- a/Documentation/netlink/specs/devlink.yaml +++ b/Documentation/netlink/specs/devlink.yaml @@ -323,7 +323,7 @@ operations: - dev-name - sb-index reply: &sb-get-reply - value: 11 + value: 13 attributes: *sb-id-attrs dump: request: @@ -350,7 +350,7 @@ operations: - sb-index - sb-pool-index reply: &sb-pool-get-reply - value: 15 + value: 17 attributes: *sb-pool-id-attrs dump: request: @@ -378,7 +378,7 @@ operations: - sb-index - sb-pool-index reply: &sb-port-pool-get-reply - value: 19 + value: 21 attributes: *sb-port-pool-id-attrs dump: request: @@ -407,7 +407,7 @@ operations: - sb-pool-type - sb-tc-index reply: &sb-tc-pool-bind-get-reply - value: 23 + value: 25 attributes: *sb-tc-pool-bind-id-attrs dump: request: @@ -538,7 +538,7 @@ operations: - dev-name - trap-name reply: &trap-get-reply - value: 61 + value: 63 attributes: *trap-id-attrs dump: request: @@ -564,7 +564,7 @@ operations: - dev-name - trap-group-name reply: &trap-group-get-reply - value: 65 + value: 67 attributes: *trap-group-id-attrs dump: request: @@ -590,7 +590,7 @@ operations: - dev-name - trap-policer-id reply: &trap-policer-get-reply - value: 69 + value: 71 attributes: *trap-policer-id-attrs dump: request: @@ -617,7 +617,7 @@ operations: - port-index - rate-node-name reply: &rate-get-reply - value: 74 + value: 76 attributes: *rate-id-attrs dump: request: @@ -643,7 +643,7 @@ operations: - dev-name - linecard-index reply: &linecard-get-reply - value: 78 + value: 80 attributes: *linecard-id-attrs dump: request: diff --git a/Documentation/networking/ax25.rst b/Documentation/networking/ax25.rst index f060cfb1445a..605e72c6c877 100644 --- a/Documentation/networking/ax25.rst +++ b/Documentation/networking/ax25.rst @@ -7,9 +7,9 @@ AX.25 To use the amateur radio protocols within Linux you will need to get a suitable copy of the AX.25 Utilities. More detailed information about AX.25, NET/ROM and ROSE, associated programs and utilities can be -found on http://www.linux-ax25.org. +found on https://linux-ax25.in-berlin.de. -There is an active mailing list for discussing Linux amateur radio matters +There is a mailing list for discussing Linux amateur radio matters called linux-hams@vger.kernel.org. To subscribe to it, send a message to majordomo@vger.kernel.org with the words "subscribe linux-hams" in the body of the message, the subject field is ignored. You don't need to be diff --git a/Documentation/networking/representors.rst b/Documentation/networking/representors.rst index ee1f5cd54496..decb39c19b9e 100644 --- a/Documentation/networking/representors.rst +++ b/Documentation/networking/representors.rst @@ -162,9 +162,11 @@ How are representors identified? The representor netdevice should *not* directly refer to a PCIe device (e.g. through ``net_dev->dev.parent`` / ``SET_NETDEV_DEV()``), either of the representee or of the switchdev function. -Instead, it should implement the ``ndo_get_devlink_port()`` netdevice op, which -the kernel uses to provide the ``phys_switch_id`` and ``phys_port_name`` sysfs -nodes. (Some legacy drivers implement ``ndo_get_port_parent_id()`` and +Instead, the driver should use the ``SET_NETDEV_DEVLINK_PORT`` macro to +assign a devlink port instance to the netdevice before registering the +netdevice; the kernel uses the devlink port to provide the ``phys_switch_id`` +and ``phys_port_name`` sysfs nodes. +(Some legacy drivers implement ``ndo_get_port_parent_id()`` and ``ndo_get_phys_port_name()`` directly, but this is deprecated.) See :ref:`Documentation/networking/devlink/devlink-port.rst ` for the details of this API. diff --git a/Documentation/process/embargoed-hardware-issues.rst b/Documentation/process/embargoed-hardware-issues.rst index cb686238f21d..31000f075707 100644 --- a/Documentation/process/embargoed-hardware-issues.rst +++ b/Documentation/process/embargoed-hardware-issues.rst @@ -25,15 +25,15 @@ Contact The Linux kernel hardware security team is separate from the regular Linux kernel security team. -The team only handles the coordination of embargoed hardware security -issues. Reports of pure software security bugs in the Linux kernel are not +The team only handles developing fixes for embargoed hardware security +issues. Reports of pure software security bugs in the Linux kernel are not handled by this team and the reporter will be guided to contact the regular Linux kernel security team (:ref:`Documentation/admin-guide/ `) instead. The team can be contacted by email at . This -is a private list of security officers who will help you to coordinate an -issue according to our documented process. +is a private list of security officers who will help you to coordinate a +fix according to our documented process. The list is encrypted and email to the list can be sent by either PGP or S/MIME encrypted and must be signed with the reporter's PGP key or S/MIME @@ -132,11 +132,11 @@ other hardware could be affected. The hardware security team will provide an incident-specific encrypted mailing-list which will be used for initial discussion with the reporter, -further disclosure and coordination. +further disclosure, and coordination of fixes. The hardware security team will provide the disclosing party a list of developers (domain experts) who should be informed initially about the -issue after confirming with the developers that they will adhere to this +issue after confirming with the developers that they will adhere to this Memorandum of Understanding and the documented process. These developers form the initial response team and will be responsible for handling the issue after initial contact. The hardware security team is supporting the @@ -209,13 +209,18 @@ five work days this is taken as silent acknowledgement. After acknowledgement or resolution of an objection the expert is disclosed by the incident team and brought into the development process. +List participants may not communicate about the issue outside of the +private mailing list. List participants may not use any shared resources +(e.g. employer build farms, CI systems, etc) when working on patches. + Coordinated release """"""""""""""""""" The involved parties will negotiate the date and time where the embargo ends. At that point the prepared mitigations are integrated into the -relevant kernel trees and published. +relevant kernel trees and published. There is no pre-notification process: +fixes are published in public and available to everyone at the same time. While we understand that hardware security issues need coordinated embargo time, the embargo time should be constrained to the minimum time which is @@ -251,6 +256,7 @@ an involved disclosed party. The current ambassadors list: IBM Z Christian Borntraeger Intel Tony Luck Qualcomm Trilok Soni + RISC-V Palmer Dabbelt Samsung Javier González Microsoft James Morris diff --git a/Documentation/rust/general-information.rst b/Documentation/rust/general-information.rst index 49029ee82e55..081397827a7e 100644 --- a/Documentation/rust/general-information.rst +++ b/Documentation/rust/general-information.rst @@ -29,7 +29,7 @@ target with the same invocation used for compilation, e.g.:: To read the docs locally in your web browser, run e.g.:: - xdg-open rust/doc/kernel/index.html + xdg-open Documentation/output/rust/rustdoc/kernel/index.html To learn about how to write the documentation, please see coding-guidelines.rst. diff --git a/Documentation/sound/designs/midi-2.0.rst b/Documentation/sound/designs/midi-2.0.rst index 45987f256b97..086487ca7ab1 100644 --- a/Documentation/sound/designs/midi-2.0.rst +++ b/Documentation/sound/designs/midi-2.0.rst @@ -74,8 +74,8 @@ topology based on those information. When the device is older and doesn't respond to the new UMP inquiries, the driver falls back and builds the topology based on Group Terminal Block (GTB) information from the USB descriptor. Some device might be screwed up by the -unexpected UMP command; in such a case, pass `midi2_probe=0` option to -snd-usb-audio driver for skipping the UMP v1.1 inquiries. +unexpected UMP command; in such a case, pass `midi2_ump_probe=0` +option to snd-usb-audio driver for skipping the UMP v1.1 inquiries. When the MIDI 2.0 device is probed, the kernel creates a rawmidi device for each UMP Endpoint of the device. Its device name is diff --git a/Documentation/tools/rtla/rtla-timerlat-hist.rst b/Documentation/tools/rtla/rtla-timerlat-hist.rst index 057db78d4095..03b7f3deb069 100644 --- a/Documentation/tools/rtla/rtla-timerlat-hist.rst +++ b/Documentation/tools/rtla/rtla-timerlat-hist.rst @@ -36,11 +36,11 @@ EXAMPLE In the example below, **rtla timerlat hist** is set to run for *10* minutes, in the cpus *0-4*, *skipping zero* only lines. Moreover, **rtla timerlat hist** will change the priority of the *timerlat* threads to run under -*SCHED_DEADLINE* priority, with a *10us* runtime every *1ms* period. The +*SCHED_DEADLINE* priority, with a *100us* runtime every *1ms* period. The *1ms* period is also passed to the *timerlat* tracer. Auto-analysis is disabled to reduce overhead :: - [root@alien ~]# timerlat hist -d 10m -c 0-4 -P d:100us:1ms -p 1ms --no-aa + [root@alien ~]# timerlat hist -d 10m -c 0-4 -P d:100us:1ms -p 1000 --no-aa # RTLA timerlat histogram # Time unit is microseconds (us) # Duration: 0 00:10:00 diff --git a/Documentation/trace/fprobe.rst b/Documentation/trace/fprobe.rst index 7a895514b537..196f52386aaa 100644 --- a/Documentation/trace/fprobe.rst +++ b/Documentation/trace/fprobe.rst @@ -91,9 +91,9 @@ The prototype of the entry/exit callback function are as follows: .. code-block:: c - int entry_callback(struct fprobe *fp, unsigned long entry_ip, struct pt_regs *regs, void *entry_data); + int entry_callback(struct fprobe *fp, unsigned long entry_ip, unsigned long ret_ip, struct pt_regs *regs, void *entry_data); - void exit_callback(struct fprobe *fp, unsigned long entry_ip, struct pt_regs *regs, void *entry_data); + void exit_callback(struct fprobe *fp, unsigned long entry_ip, unsigned long ret_ip, struct pt_regs *regs, void *entry_data); Note that the @entry_ip is saved at function entry and passed to exit handler. If the entry callback function returns !0, the corresponding exit callback will be cancelled. @@ -108,6 +108,10 @@ If the entry callback function returns !0, the corresponding exit callback will Note that this may not be the actual entry address of the function but the address where the ftrace is instrumented. +@ret_ip + This is the return address that the traced function will return to, + somewhere in the caller. This can be used at both entry and exit. + @regs This is the `pt_regs` data structure at the entry and exit. Note that the instruction pointer of @regs may be different from the @entry_ip diff --git a/Documentation/translations/zh_CN/arch/loongarch/introduction.rst b/Documentation/translations/zh_CN/arch/loongarch/introduction.rst index cba04befc950..59d6bf33050c 100644 --- a/Documentation/translations/zh_CN/arch/loongarch/introduction.rst +++ b/Documentation/translations/zh_CN/arch/loongarch/introduction.rst @@ -344,9 +344,9 @@ LoongArch指令集架构的文档： LoongArch的ELF psABI文档： - https://github.com/loongson/LoongArch-Documentation/releases/latest/download/LoongArch-ELF-ABI-v2.00-CN.pdf （中文版） + https://github.com/loongson/LoongArch-Documentation/releases/latest/download/LoongArch-ELF-ABI-v2.01-CN.pdf （中文版） - https://github.com/loongson/LoongArch-Documentation/releases/latest/download/LoongArch-ELF-ABI-v2.00-EN.pdf （英文版） + https://github.com/loongson/LoongArch-Documentation/releases/latest/download/LoongArch-ELF-ABI-v2.01-EN.pdf （英文版） Loongson与LoongArch的Linux内核源码仓库： diff --git a/Documentation/translations/zh_CN/core-api/workqueue.rst b/Documentation/translations/zh_CN/core-api/workqueue.rst index 6c1b5ec31d75..7fac6f75d078 100644 --- a/Documentation/translations/zh_CN/core-api/workqueue.rst +++ b/Documentation/translations/zh_CN/core-api/workqueue.rst @@ -202,7 +202,7 @@ workqueue将自动创建与属性相匹配的后备工作者池。调节并发 同的排序属性。 在目前的实现中，上述配置只保证了特定NUMA节点内的ST行为。相反， -``alloc_ordered_queue()`` 应该被用来实现全系统的ST行为。 +``alloc_ordered_workqueue()`` 应该被用来实现全系统的ST行为。 执行场景示例 diff --git a/MAINTAINERS b/MAINTAINERS index cc730286ed2f..6a19a4f1e494 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -378,8 +378,9 @@ F: drivers/acpi/viot.c F: include/linux/acpi_viot.h ACPI WMI DRIVER +M: Armin Wolf L: platform-driver-x86@vger.kernel.org -S: Orphan +S: Maintained F: Documentation/driver-api/wmi.rst F: Documentation/wmi/ F: drivers/platform/x86/wmi.c @@ -470,7 +471,6 @@ F: drivers/hwmon/adm1029.c ADM8211 WIRELESS DRIVER L: linux-wireless@vger.kernel.org S: Orphan -W: https://wireless.wiki.kernel.org/ F: drivers/net/wireless/admtek/adm8211.* ADP1653 FLASH CONTROLLER DRIVER @@ -1585,6 +1585,17 @@ F: arch/arm/include/asm/arch_timer.h F: arch/arm64/include/asm/arch_timer.h F: drivers/clocksource/arm_arch_timer.c +ARM GENERIC INTERRUPT CONTROLLER DRIVERS +M: Marc Zyngier +L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) +S: Maintained +F: Documentation/devicetree/bindings/interrupt-controller/arm,gic* +F: arch/arm/include/asm/arch_gicv3.h +F: arch/arm64/include/asm/arch_gicv3.h +F: drivers/irqchip/irq-gic*.[ch] +F: include/linux/irqchip/arm-gic*.h +F: include/linux/irqchip/arm-vgic-info.h + ARM HDLCD DRM DRIVER M: Liviu Dudau S: Supported @@ -1662,7 +1673,7 @@ F: arch/arm*/include/asm/perf_event.h F: arch/arm*/kernel/hw_breakpoint.c F: arch/arm*/kernel/perf_* F: drivers/perf/ -F: include/linux/perf/arm_pmu.h +F: include/linux/perf/arm_pmu*.h ARM PORT M: Russell King @@ -1855,7 +1866,7 @@ F: Documentation/devicetree/bindings/phy/amlogic* F: arch/arm/boot/dts/amlogic/ F: arch/arm/mach-meson/ F: arch/arm64/boot/dts/amlogic/ -F: drivers/genpd/amlogic/ +F: drivers/pmdomain/amlogic/ F: drivers/mmc/host/meson* F: drivers/phy/amlogic/ F: drivers/pinctrl/meson/ @@ -1918,7 +1929,7 @@ F: drivers/bluetooth/hci_bcm4377.c F: drivers/clk/clk-apple-nco.c F: drivers/cpufreq/apple-soc-cpufreq.c F: drivers/dma/apple-admac.c -F: drivers/genpd/apple/ +F: drivers/pmdomain/apple/ F: drivers/i2c/busses/i2c-pasemi-core.c F: drivers/i2c/busses/i2c-pasemi-platform.c F: drivers/iommu/apple-dart.c @@ -1963,12 +1974,12 @@ F: drivers/irqchip/irq-aspeed-i2c-ic.c ARM/ASPEED MACHINE SUPPORT M: Joel Stanley -R: Andrew Jeffery +R: Andrew Jeffery L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) L: linux-aspeed@lists.ozlabs.org (moderated for non-subscribers) S: Supported Q: https://patchwork.ozlabs.org/project/linux-aspeed/list/ -T: git git://git.kernel.org/pub/scm/linux/kernel/git/joel/aspeed.git +T: git git://git.kernel.org/pub/scm/linux/kernel/git/joel/bmc.git F: Documentation/devicetree/bindings/arm/aspeed/ F: arch/arm/boot/dts/aspeed/ F: arch/arm/mach-aspeed/ @@ -2211,21 +2222,28 @@ F: arch/arm/boot/dts/ti/omap/omap3-igep* ARM/INTEL IXP4XX ARM ARCHITECTURE M: Linus Walleij M: Imre Kaloz -M: Krzysztof Halasa L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) S: Maintained F: Documentation/devicetree/bindings/arm/intel-ixp4xx.yaml F: Documentation/devicetree/bindings/gpio/intel,ixp4xx-gpio.txt F: Documentation/devicetree/bindings/interrupt-controller/intel,ixp4xx-interrupt.yaml F: Documentation/devicetree/bindings/memory-controllers/intel,ixp4xx-expansion* +F: Documentation/devicetree/bindings/rng/intel,ixp46x-rng.yaml F: Documentation/devicetree/bindings/timer/intel,ixp4xx-timer.yaml F: arch/arm/boot/dts/intel/ixp/ F: arch/arm/mach-ixp4xx/ F: drivers/bus/intel-ixp4xx-eb.c +F: drivers/char/hw_random/ixp4xx-rng.c F: drivers/clocksource/timer-ixp4xx.c F: drivers/crypto/intel/ixp4xx/ixp4xx_crypto.c F: drivers/gpio/gpio-ixp4xx.c F: drivers/irqchip/irq-ixp4xx.c +F: drivers/net/ethernet/xscale/ixp4xx_eth.c +F: drivers/net/wan/ixp4xx_hss.c +F: drivers/soc/ixp4xx/ixp4xx-npe.c +F: drivers/soc/ixp4xx/ixp4xx-qmgr.c +F: include/linux/soc/ixp4xx/npe.h +F: include/linux/soc/ixp4xx/qmgr.h ARM/INTEL KEEMBAY ARCHITECTURE M: Paul J. Murphy @@ -2327,7 +2345,7 @@ F: drivers/rtc/rtc-mt7622.c ARM/Mediatek SoC support M: Matthias Brugger -R: AngeloGioacchino Del Regno +M: AngeloGioacchino Del Regno L: linux-kernel@vger.kernel.org L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) L: linux-mediatek@lists.infradead.org (moderated for non-subscribers) @@ -2435,7 +2453,7 @@ F: arch/arm/mach-ux500/ F: drivers/clk/clk-nomadik.c F: drivers/clocksource/clksrc-dbx500-prcmu.c F: drivers/dma/ste_dma40* -F: drivers/genpd/st/ste-ux500-pm-domain.c +F: drivers/pmdomain/st/ste-ux500-pm-domain.c F: drivers/hwspinlock/u8500_hsem.c F: drivers/i2c/busses/i2c-nomadik.c F: drivers/iio/adc/ab8500-gpadc.c @@ -2598,7 +2616,7 @@ F: arch/arm/include/debug/renesas-scif.S F: arch/arm/mach-shmobile/ F: arch/arm64/boot/dts/renesas/ F: arch/riscv/boot/dts/renesas/ -F: drivers/genpd/renesas/ +F: drivers/pmdomain/renesas/ F: drivers/soc/renesas/ F: include/linux/soc/renesas/ K: \brenesas, @@ -3058,7 +3076,7 @@ F: Documentation/devicetree/bindings/peci/peci-aspeed.yaml F: drivers/peci/controller/peci-aspeed.c ASPEED PINCTRL DRIVERS -M: Andrew Jeffery +M: Andrew Jeffery L: linux-aspeed@lists.ozlabs.org (moderated for non-subscribers) L: openbmc@lists.ozlabs.org (moderated for non-subscribers) L: linux-gpio@vger.kernel.org @@ -3075,7 +3093,7 @@ F: drivers/irqchip/irq-aspeed-scu-ic.c F: include/dt-bindings/interrupt-controller/aspeed-scu-ic.h ASPEED SD/MMC DRIVER -M: Andrew Jeffery +M: Andrew Jeffery L: linux-aspeed@lists.ozlabs.org (moderated for non-subscribers) L: openbmc@lists.ozlabs.org (moderated for non-subscribers) L: linux-mmc@vger.kernel.org @@ -3344,7 +3362,7 @@ AX.25 NETWORK LAYER M: Ralf Baechle L: linux-hams@vger.kernel.org S: Maintained -W: http://www.linux-ax25.org/ +W: https://linux-ax25.in-berlin.de F: include/net/ax25.h F: include/uapi/linux/ax25.h F: net/ax25/ @@ -4026,7 +4044,7 @@ F: arch/mips/kernel/*bmips* F: drivers/irqchip/irq-bcm63* F: drivers/irqchip/irq-bcm7* F: drivers/irqchip/irq-brcmstb* -F: drivers/genpd/bcm/bcm63xx-power.c +F: drivers/pmdomain/bcm/bcm63xx-power.c F: include/linux/bcm963xx_nvram.h F: include/linux/bcm963xx_tag.h @@ -4082,7 +4100,7 @@ F: drivers/net/wireless/broadcom/brcm80211/ BROADCOM BRCMSTB GPIO DRIVER M: Doug Berger -M: Florian Fainelli +M: Florian Fainelli R: Broadcom internal kernel review list S: Supported F: Documentation/devicetree/bindings/gpio/brcm,brcmstb-gpio.yaml @@ -4248,7 +4266,7 @@ R: Broadcom internal kernel review list L: linux-pm@vger.kernel.org S: Maintained T: git https://github.com/broadcom/stblinux.git -F: drivers/genpd/bcm/bcm-pmb.c +F: drivers/pmdomain/bcm/bcm-pmb.c F: include/dt-bindings/soc/bcm-pmb.h BROADCOM SPECIFIC AMBA DRIVER (BCMA) @@ -4378,7 +4396,6 @@ M: David Sterba L: linux-btrfs@vger.kernel.org S: Maintained W: https://btrfs.readthedocs.io -W: https://btrfs.wiki.kernel.org/ Q: https://patchwork.kernel.org/project/linux-btrfs/list/ C: irc://irc.libera.chat/btrfs T: git git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git @@ -5986,8 +6003,8 @@ F: include/linux/devm-helpers.h DEVICE-MAPPER (LVM) M: Alasdair Kergon M: Mike Snitzer -M: dm-devel@redhat.com -L: dm-devel@redhat.com +M: dm-devel@lists.linux.dev +L: dm-devel@lists.linux.dev S: Maintained W: http://sources.redhat.com/dm Q: http://patchwork.kernel.org/project/dm-devel/list/ @@ -6646,9 +6663,9 @@ F: Documentation/devicetree/bindings/display/panel/novatek,nt36672a.yaml F: drivers/gpu/drm/panel/panel-novatek-nt36672a.c DRM DRIVER FOR NVIDIA GEFORCE/QUADRO GPUS -M: Ben Skeggs M: Karol Herbst M: Lyude Paul +M: Danilo Krummrich L: dri-devel@lists.freedesktop.org L: nouveau@lists.freedesktop.org S: Supported @@ -8094,7 +8111,7 @@ F: include/linux/arm_ffa.h FIRMWARE LOADER (request_firmware) M: Luis Chamberlain -M: Russ Weight +M: Russ Weight L: linux-kernel@vger.kernel.org S: Maintained F: Documentation/firmware_class/ @@ -8729,7 +8746,7 @@ M: Ulf Hansson L: linux-pm@vger.kernel.org S: Supported T: git git://git.kernel.org/pub/scm/linux/kernel/git/ulfh/linux-pm.git -F: drivers/genpd/ +F: drivers/pmdomain/ GENERIC RESISTIVE TOUCHSCREEN ADC DRIVER M: Eugen Hristev @@ -8875,7 +8892,7 @@ F: drivers/gpio/gpio-mockup.c F: tools/testing/selftests/gpio/ GPIO REGMAP -R: Michael Walle +M: Michael Walle S: Maintained F: drivers/gpio/gpio-regmap.c F: include/linux/gpio/regmap.h @@ -9532,10 +9549,8 @@ F: Documentation/devicetree/bindings/iio/pressure/honeywell,mprls0025pa.yaml F: drivers/iio/pressure/mprls0025pa.c HOST AP DRIVER -M: Jouni Malinen L: linux-wireless@vger.kernel.org S: Obsolete -W: http://w1.fi/hostap-driver.html F: drivers/net/wireless/intersil/hostap/ HP BIOSCFG DRIVER @@ -10623,22 +10638,6 @@ L: linux-crypto@vger.kernel.org S: Maintained F: drivers/crypto/intel/ixp4xx/ixp4xx_crypto.c -INTEL IXP4XX QMGR, NPE, ETHERNET and HSS SUPPORT -M: Krzysztof Halasa -S: Maintained -F: drivers/net/ethernet/xscale/ixp4xx_eth.c -F: drivers/net/wan/ixp4xx_hss.c -F: drivers/soc/ixp4xx/ixp4xx-npe.c -F: drivers/soc/ixp4xx/ixp4xx-qmgr.c -F: include/linux/soc/ixp4xx/npe.h -F: include/linux/soc/ixp4xx/qmgr.h - -INTEL IXP4XX RANDOM NUMBER GENERATOR SUPPORT -M: Deepak Saxena -S: Maintained -F: Documentation/devicetree/bindings/rng/intel,ixp46x-rng.yaml -F: drivers/char/hw_random/ixp4xx-rng.c - INTEL KEEM BAY DRM DRIVER M: Anitha Chrisanthus M: Edmund Dea @@ -11064,7 +11063,7 @@ F: Documentation/devicetree/bindings/sound/irondevice,* F: sound/soc/codecs/sma* IRQ DOMAINS (IRQ NUMBER MAPPING LIBRARY) -M: Marc Zyngier +M: Thomas Gleixner S: Maintained T: git git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git irq/core F: Documentation/core-api/irq/irq-domain.rst @@ -11083,7 +11082,6 @@ F: lib/group_cpus.c IRQCHIP DRIVERS M: Thomas Gleixner -M: Marc Zyngier L: linux-kernel@vger.kernel.org S: Maintained T: git git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git irq/core @@ -13617,6 +13615,7 @@ F: drivers/net/ethernet/mellanox/mlxfw/ MELLANOX HARDWARE PLATFORM SUPPORT M: Hans de Goede +M: Ilpo Järvinen M: Mark Gross M: Vadim Pasternak L: platform-driver-x86@vger.kernel.org @@ -14211,6 +14210,7 @@ F: drivers/platform/surface/surface_gpe.c MICROSOFT SURFACE HARDWARE PLATFORM SUPPORT M: Hans de Goede +M: Ilpo Järvinen M: Mark Gross M: Maximilian Luz L: platform-driver-x86@vger.kernel.org @@ -14757,7 +14757,7 @@ NETROM NETWORK LAYER M: Ralf Baechle L: linux-hams@vger.kernel.org S: Maintained -W: http://www.linux-ax25.org/ +W: https://linux-ax25.in-berlin.de F: include/net/netrom.h F: include/uapi/linux/netrom.h F: net/netrom/ @@ -14946,7 +14946,7 @@ K: macsec K: \bmdo_ NETWORKING [MPTCP] -M: Matthieu Baerts +M: Matthieu Baerts M: Mat Martineau L: netdev@vger.kernel.org L: mptcp@lists.linux.dev @@ -17601,6 +17601,7 @@ M: Kalle Valo M: Jeff Johnson L: ath12k@lists.infradead.org S: Supported +W: https://wireless.wiki.kernel.org/en/users/Drivers/ath12k T: git git://git.kernel.org/pub/scm/linux/kernel/git/kvalo/ath.git F: drivers/net/wireless/ath/ath12k/ @@ -17680,7 +17681,7 @@ L: linux-pm@vger.kernel.org L: linux-arm-msm@vger.kernel.org S: Maintained F: Documentation/devicetree/bindings/power/avs/qcom,cpr.yaml -F: drivers/genpd/qcom/cpr.c +F: drivers/pmdomain/qcom/cpr.c QUALCOMM CPUFREQ DRIVER MSM8996/APQ8096 M: Ilia Lin @@ -18131,8 +18132,6 @@ REALTEK WIRELESS DRIVER (rtlwifi family) M: Ping-Ke Shih L: linux-wireless@vger.kernel.org S: Maintained -W: https://wireless.wiki.kernel.org/ -T: git git://git.kernel.org/pub/scm/linux/kernel/git/linville/wireless-testing.git F: drivers/net/wireless/realtek/rtlwifi/ REALTEK WIRELESS DRIVER (rtw88) @@ -18608,7 +18607,7 @@ ROSE NETWORK LAYER M: Ralf Baechle L: linux-hams@vger.kernel.org S: Maintained -W: http://www.linux-ax25.org/ +W: https://linux-ax25.in-berlin.de F: include/net/rose.h F: include/uapi/linux/rose.h F: net/rose/ @@ -18660,7 +18659,6 @@ F: drivers/media/dvb-frontends/rtl2832_sdr* RTL8180 WIRELESS DRIVER L: linux-wireless@vger.kernel.org S: Orphan -W: https://wireless.wiki.kernel.org/ F: drivers/net/wireless/realtek/rtl818x/rtl8180/ RTL8187 WIRELESS DRIVER @@ -18668,14 +18666,12 @@ M: Hin-Tak Leung M: Larry Finger L: linux-wireless@vger.kernel.org S: Maintained -W: https://wireless.wiki.kernel.org/ F: drivers/net/wireless/realtek/rtl818x/rtl8187/ RTL8XXXU WIRELESS DRIVER (rtl8xxxu) M: Jes Sorensen L: linux-wireless@vger.kernel.org S: Maintained -T: git git://git.kernel.org/pub/scm/linux/kernel/git/jes/linux.git rtl8xxxu-devel F: drivers/net/wireless/realtek/rtl8xxxu/ RTRS TRANSPORT DRIVERS @@ -20500,6 +20496,7 @@ F: include/dt-bindings/clock/starfive?jh71*.h STARFIVE JH71X0 PINCTRL DRIVERS M: Emil Renner Berthing M: Jianlong Huang +M: Hal Feng L: linux-gpio@vger.kernel.org S: Maintained F: Documentation/devicetree/bindings/pinctrl/starfive,jh71*.yaml @@ -20525,7 +20522,7 @@ STARFIVE JH71XX PMU CONTROLLER DRIVER M: Walker Chen S: Supported F: Documentation/devicetree/bindings/power/starfive* -F: drivers/genpd/starfive/jh71xx-pmu.c +F: drivers/pmdomain/starfive/jh71xx-pmu.c F: include/dt-bindings/power/starfive,jh7110-pmu.h STARFIVE SOC DRIVERS @@ -21350,7 +21347,7 @@ F: drivers/irqchip/irq-ti-sci-inta.c F: drivers/irqchip/irq-ti-sci-intr.c F: drivers/reset/reset-ti-sci.c F: drivers/soc/ti/ti_sci_inta_msi.c -F: drivers/genpd/ti/ti_sci_pm_domains.c +F: drivers/pmdomain/ti/ti_sci_pm_domains.c F: include/dt-bindings/soc/ti,sci_pm_domain.h F: include/linux/soc/ti/ti_sci_inta_msi.h F: include/linux/soc/ti/ti_sci_protocol.h @@ -21592,7 +21589,7 @@ L: linux-kernel@vger.kernel.org L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) S: Maintained T: git git://git.kernel.org/pub/scm/linux/kernel/git/ti/linux.git -F: drivers/genpd/ti/omap_prm.c +F: drivers/pmdomain/ti/omap_prm.c F: drivers/soc/ti/* TI LM49xxx FAMILY ASoC CODEC DRIVERS @@ -21668,7 +21665,6 @@ L: linux-wireless@vger.kernel.org S: Orphan W: https://wireless.wiki.kernel.org/en/users/Drivers/wl12xx W: https://wireless.wiki.kernel.org/en/users/Drivers/wl1251 -T: git git://git.kernel.org/pub/scm/linux/kernel/git/luca/wl12xx.git F: drivers/net/wireless/ti/ TIMEKEEPING, CLOCKSOURCE CORE, NTP, ALARMTIMER @@ -23435,9 +23431,11 @@ F: drivers/platform/x86/x86-android-tablets/ X86 PLATFORM DRIVERS M: Hans de Goede +M: Ilpo Järvinen M: Mark Gross L: platform-driver-x86@vger.kernel.org S: Maintained +Q: https://patchwork.kernel.org/project/platform-driver-x86/list/ T: git git://git.kernel.org/pub/scm/linux/kernel/git/pdx86/platform-drivers-x86.git F: drivers/platform/olpc/ F: drivers/platform/x86/ diff --git a/Makefile b/Makefile index ceb23eed4dce..5fc735c7fed1 100644 --- a/Makefile +++ b/Makefile @@ -2,7 +2,7 @@ VERSION = 6 PATCHLEVEL = 6 SUBLEVEL = 0 -EXTRAVERSION = -rc1 +EXTRAVERSION = -rc7 NAME = Hurr durr I'ma ninja sloth # *DOCUMENTATION* @@ -1474,7 +1474,7 @@ endif # CONFIG_MODULES # Directories & files removed with 'make clean' CLEAN_FILES += vmlinux.symvers modules-only.symvers \ modules.builtin modules.builtin.modinfo modules.nsdeps \ - compile_commands.json .thinlto-cache rust/test rust/doc \ + compile_commands.json .thinlto-cache rust/test \ rust-project.json .vmlinux.objs .vmlinux.export.c # Directories & files removed with 'make mrproper' diff --git a/arch/arm/boot/dts/ti/omap/motorola-mapphone-common.dtsi b/arch/arm/boot/dts/ti/omap/motorola-mapphone-common.dtsi index 091ba310053e..d2d516d113ba 100644 --- a/arch/arm/boot/dts/ti/omap/motorola-mapphone-common.dtsi +++ b/arch/arm/boot/dts/ti/omap/motorola-mapphone-common.dtsi @@ -614,12 +614,12 @@ /* Configure pwm clock source for timers 8 & 9 */ &timer8 { assigned-clocks = <&abe_clkctrl OMAP4_TIMER8_CLKCTRL 24>; - assigned-clock-parents = <&sys_clkin_ck>; + assigned-clock-parents = <&sys_32k_ck>; }; &timer9 { assigned-clocks = <&l4_per_clkctrl OMAP4_TIMER9_CLKCTRL 24>; - assigned-clock-parents = <&sys_clkin_ck>; + assigned-clock-parents = <&sys_32k_ck>; }; /* @@ -640,6 +640,7 @@ &uart3 { interrupts-extended = <&wakeupgen GIC_SPI 74 IRQ_TYPE_LEVEL_HIGH &omap4_pmx_core 0x17c>; + overrun-throttle-ms = <500>; }; &uart4 { diff --git a/arch/arm/boot/dts/ti/omap/omap3-cpu-thermal.dtsi b/arch/arm/boot/dts/ti/omap/omap3-cpu-thermal.dtsi index 0da759f8e2c2..7dd2340bc5e4 100644 --- a/arch/arm/boot/dts/ti/omap/omap3-cpu-thermal.dtsi +++ b/arch/arm/boot/dts/ti/omap/omap3-cpu-thermal.dtsi @@ -12,8 +12,7 @@ cpu_thermal: cpu-thermal { polling-delay = <1000>; /* milliseconds */ coefficients = <0 20000>; - /* sensor ID */ - thermal-sensors = <&bandgap 0>; + thermal-sensors = <&bandgap>; cpu_trips: trips { cpu_alert0: cpu_alert { diff --git a/arch/arm/boot/dts/ti/omap/omap4-cpu-thermal.dtsi b/arch/arm/boot/dts/ti/omap/omap4-cpu-thermal.dtsi index 801b4f10350c..d484ec1e4fd8 100644 --- a/arch/arm/boot/dts/ti/omap/omap4-cpu-thermal.dtsi +++ b/arch/arm/boot/dts/ti/omap/omap4-cpu-thermal.dtsi @@ -12,7 +12,10 @@ cpu_thermal: cpu_thermal { polling-delay-passive = <250>; /* milliseconds */ polling-delay = <1000>; /* milliseconds */ - /* sensor ID */ + /* + * See 44xx files for single sensor addressing, omap5 and dra7 need + * also sensor ID for addressing. + */ thermal-sensors = <&bandgap 0>; cpu_trips: trips { diff --git a/arch/arm/boot/dts/ti/omap/omap443x.dtsi b/arch/arm/boot/dts/ti/omap/omap443x.dtsi index 238aceb799f8..2104170fe2cd 100644 --- a/arch/arm/boot/dts/ti/omap/omap443x.dtsi +++ b/arch/arm/boot/dts/ti/omap/omap443x.dtsi @@ -69,6 +69,7 @@ }; &cpu_thermal { + thermal-sensors = <&bandgap>; coefficients = <0 20000>; }; diff --git a/arch/arm/boot/dts/ti/omap/omap4460.dtsi b/arch/arm/boot/dts/ti/omap/omap4460.dtsi index 1b27a862ae81..a6764750d447 100644 --- a/arch/arm/boot/dts/ti/omap/omap4460.dtsi +++ b/arch/arm/boot/dts/ti/omap/omap4460.dtsi @@ -79,6 +79,7 @@ }; &cpu_thermal { + thermal-sensors = <&bandgap>; coefficients = <348 (-9301)>; }; diff --git a/arch/arm/include/asm/hardware/locomo.h b/arch/arm/include/asm/hardware/locomo.h index 246a3de25931..aaaedafef7cc 100644 --- a/arch/arm/include/asm/hardware/locomo.h +++ b/arch/arm/include/asm/hardware/locomo.h @@ -195,7 +195,7 @@ struct locomo_driver { #define LOCOMO_DRIVER_NAME(_ldev) ((_ldev)->dev.driver->name) -void locomo_lcd_power(struct locomo_dev *, int, unsigned int); +extern void locomolcd_power(int on); int locomo_driver_register(struct locomo_driver *); void locomo_driver_unregister(struct locomo_driver *); diff --git a/arch/arm/mach-omap2/pm44xx.c b/arch/arm/mach-omap2/pm44xx.c index f57802f3ee3a..37b168119fe4 100644 --- a/arch/arm/mach-omap2/pm44xx.c +++ b/arch/arm/mach-omap2/pm44xx.c @@ -99,7 +99,7 @@ static int omap4_pm_suspend(void) * possible causes. * http://www.spinics.net/lists/arm-kernel/msg218641.html */ - pr_warn("A possible cause could be an old bootloader - try u-boot >= v2012.07\n"); + pr_debug("A possible cause could be an old bootloader - try u-boot >= v2012.07\n"); } else { pr_info("Successfully put all powerdomains to target state\n"); } @@ -257,7 +257,7 @@ int __init omap4_pm_init(void) * http://www.spinics.net/lists/arm-kernel/msg218641.html */ if (cpu_is_omap44xx()) - pr_warn("OMAP4 PM: u-boot >= v2012.07 is required for full PM support\n"); + pr_debug("OMAP4 PM: u-boot >= v2012.07 is required for full PM support\n"); ret = pwrdm_for_each(pwrdms_setup, NULL); if (ret) { diff --git a/arch/arm/mach-sa1100/include/mach/collie.h b/arch/arm/mach-sa1100/include/mach/collie.h index b7bc23ffd3c6..c95273c9567b 100644 --- a/arch/arm/mach-sa1100/include/mach/collie.h +++ b/arch/arm/mach-sa1100/include/mach/collie.h @@ -16,8 +16,6 @@ #include "hardware.h" /* Gives GPIO_MAX */ -extern void locomolcd_power(int on); - #define COLLIE_SCOOP_GPIO_BASE (GPIO_MAX + 1) #define COLLIE_GPIO_CHARGE_ON (COLLIE_SCOOP_GPIO_BASE + 0) #define COLLIE_SCP_DIAG_BOOT1 SCOOP_GPCR_PA12 diff --git a/arch/arm/mm/cache-uniphier.c b/arch/arm/mm/cache-uniphier.c index ff2881458504..84a2f17ff32d 100644 --- a/arch/arm/mm/cache-uniphier.c +++ b/arch/arm/mm/cache-uniphier.c @@ -58,11 +58,13 @@ ((op & UNIPHIER_SSCOQM_S_MASK) == UNIPHIER_SSCOQM_S_RANGE) /** - * uniphier_cache_data - UniPhier outer cache specific data + * struct uniphier_cache_data - UniPhier outer cache specific data * * @ctrl_base: virtual base address of control registers * @rev_base: virtual base address of revision registers * @op_base: virtual base address of operation registers + * @way_ctrl_base: virtual address of the way control registers for this + * SoC revision * @way_mask: each bit specifies if the way is present * @nsets: number of associativity sets * @line_size: line size in bytes diff --git a/arch/arm/xen/enlighten.c b/arch/arm/xen/enlighten.c index 7d59765aef22..c392e18f1e43 100644 --- a/arch/arm/xen/enlighten.c +++ b/arch/arm/xen/enlighten.c @@ -207,7 +207,7 @@ static void xen_power_off(void) static irqreturn_t xen_arm_callback(int irq, void *arg) { - xen_hvm_evtchn_do_upcall(); + xen_evtchn_do_upcall(); return IRQ_HANDLED; } diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index b10515c0200b..78f20e632712 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -1037,6 +1037,19 @@ config ARM64_ERRATUM_2645198 If unsure, say Y. +config ARM64_ERRATUM_2966298 + bool "Cortex-A520: 2966298: workaround for speculatively executed unprivileged load" + default y + help + This option adds the workaround for ARM Cortex-A520 erratum 2966298. + + On an affected Cortex-A520 core, a speculatively executed unprivileged + load might leak data from a privileged level via a cache side channel. + + Work around this problem by executing a TLBI before returning to EL0. + + If unsure, say Y. + config CAVIUM_ERRATUM_22375 bool "Cavium erratum 22375, 24313" default y diff --git a/arch/arm64/boot/dts/freescale/Makefile b/arch/arm64/boot/dts/freescale/Makefile index c6872b7e9471..89aee6c92576 100644 --- a/arch/arm64/boot/dts/freescale/Makefile +++ b/arch/arm64/boot/dts/freescale/Makefile @@ -66,6 +66,7 @@ dtb-$(CONFIG_ARCH_MXC) += imx8mm-mx8menlo.dtb dtb-$(CONFIG_ARCH_MXC) += imx8mm-nitrogen-r2.dtb dtb-$(CONFIG_ARCH_MXC) += imx8mm-phg.dtb dtb-$(CONFIG_ARCH_MXC) += imx8mm-phyboard-polis-rdk.dtb +dtb-$(CONFIG_ARCH_MXC) += imx8mm-prt8mm.dtb dtb-$(CONFIG_ARCH_MXC) += imx8mm-tqma8mqml-mba8mx.dtb dtb-$(CONFIG_ARCH_MXC) += imx8mm-var-som-symphony.dtb dtb-$(CONFIG_ARCH_MXC) += imx8mm-venice-gw71xx-0x.dtb diff --git a/arch/arm64/boot/dts/freescale/imx8mm-evk.dtsi b/arch/arm64/boot/dts/freescale/imx8mm-evk.dtsi index e31ab8b4f54f..a882c86ec313 100644 --- a/arch/arm64/boot/dts/freescale/imx8mm-evk.dtsi +++ b/arch/arm64/boot/dts/freescale/imx8mm-evk.dtsi @@ -26,7 +26,7 @@ port { hdmi_connector_in: endpoint { - remote-endpoint = <&adv7533_out>; + remote-endpoint = <&adv7535_out>; }; }; }; @@ -72,6 +72,13 @@ enable-active-high; }; + reg_vddext_3v3: regulator-vddext-3v3 { + compatible = "regulator-fixed"; + regulator-name = "VDDEXT_3V3"; + regulator-min-microvolt = <3300000>; + regulator-max-microvolt = <3300000>; + }; + backlight: backlight { compatible = "pwm-backlight"; pwms = <&pwm1 0 5000000 0>; @@ -317,15 +324,16 @@ hdmi@3d { compatible = "adi,adv7535"; - reg = <0x3d>, <0x3c>, <0x3e>, <0x3f>; - reg-names = "main", "cec", "edid", "packet"; + reg = <0x3d>; + interrupt-parent = <&gpio1>; + interrupts = <9 IRQ_TYPE_EDGE_FALLING>; adi,dsi-lanes = <4>; - - adi,input-depth = <8>; - adi,input-colorspace = "rgb"; - adi,input-clock = "1x"; - adi,input-style = <1>; - adi,input-justification = "evenly"; + avdd-supply = <&buck5_reg>; + dvdd-supply = <&buck5_reg>; + pvdd-supply = <&buck5_reg>; + a2vdd-supply = <&buck5_reg>; + v3p3-supply = <®_vddext_3v3>; + v1p2-supply = <&buck5_reg>; ports { #address-cells = <1>; @@ -334,7 +342,7 @@ port@0 { reg = <0>; - adv7533_in: endpoint { + adv7535_in: endpoint { remote-endpoint = <&dsi_out>; }; }; @@ -342,7 +350,7 @@ port@1 { reg = <1>; - adv7533_out: endpoint { + adv7535_out: endpoint { remote-endpoint = <&hdmi_connector_in>; }; }; @@ -448,7 +456,7 @@ reg = <1>; dsi_out: endpoint { - remote-endpoint = <&adv7533_in>; + remote-endpoint = <&adv7535_in>; data-lanes = <1 2 3 4>; }; }; diff --git a/arch/arm64/boot/dts/freescale/imx8mp-beacon-kit.dts b/arch/arm64/boot/dts/freescale/imx8mp-beacon-kit.dts index 06e91297fb16..acd265d8b58e 100644 --- a/arch/arm64/boot/dts/freescale/imx8mp-beacon-kit.dts +++ b/arch/arm64/boot/dts/freescale/imx8mp-beacon-kit.dts @@ -381,9 +381,10 @@ &sai3 { pinctrl-names = "default"; pinctrl-0 = <&pinctrl_sai3>; - assigned-clocks = <&clk IMX8MP_CLK_SAI3>; + assigned-clocks = <&clk IMX8MP_CLK_SAI3>, + <&clk IMX8MP_AUDIO_PLL2> ; assigned-clock-parents = <&clk IMX8MP_AUDIO_PLL2_OUT>; - assigned-clock-rates = <12288000>; + assigned-clock-rates = <12288000>, <361267200>; fsl,sai-mclk-direction-output; status = "okay"; }; diff --git a/arch/arm64/boot/dts/freescale/imx8mp.dtsi b/arch/arm64/boot/dts/freescale/imx8mp.dtsi index 6f2f50e1639c..83d907294fbc 100644 --- a/arch/arm64/boot/dts/freescale/imx8mp.dtsi +++ b/arch/arm64/boot/dts/freescale/imx8mp.dtsi @@ -790,6 +790,12 @@ reg = ; clocks = <&clk IMX8MP_CLK_AUDIO_ROOT>, <&clk IMX8MP_CLK_AUDIO_AXI>; + assigned-clocks = <&clk IMX8MP_CLK_AUDIO_AHB>, + <&clk IMX8MP_CLK_AUDIO_AXI_SRC>; + assigned-clock-parents = <&clk IMX8MP_SYS_PLL1_800M>, + <&clk IMX8MP_SYS_PLL1_800M>; + assigned-clock-rates = <400000000>, + <600000000>; }; pgc_gpu2d: power-domain@6 { diff --git a/arch/arm64/boot/dts/freescale/imx93-tqma9352.dtsi b/arch/arm64/boot/dts/freescale/imx93-tqma9352.dtsi index 1c71c08becde..f6e422dc2663 100644 --- a/arch/arm64/boot/dts/freescale/imx93-tqma9352.dtsi +++ b/arch/arm64/boot/dts/freescale/imx93-tqma9352.dtsi @@ -81,7 +81,7 @@ &gpio1 { pmic-irq-hog { gpio-hog; - gpios = <2 GPIO_ACTIVE_LOW>; + gpios = <3 GPIO_ACTIVE_LOW>; input; line-name = "PMIC_IRQ#"; }; diff --git a/arch/arm64/boot/dts/freescale/imx93.dtsi b/arch/arm64/boot/dts/freescale/imx93.dtsi index 6f85a05ee7e1..dcf6e4846ac9 100644 --- a/arch/arm64/boot/dts/freescale/imx93.dtsi +++ b/arch/arm64/boot/dts/freescale/imx93.dtsi @@ -185,7 +185,7 @@ #size-cells = <1>; ranges; - anomix_ns_gpr: syscon@44210000 { + aonmix_ns_gpr: syscon@44210000 { compatible = "fsl,imx93-aonmix-ns-syscfg", "syscon"; reg = <0x44210000 0x1000>; }; @@ -319,6 +319,7 @@ assigned-clock-parents = <&clk IMX93_CLK_SYS_PLL_PFD1_DIV2>; assigned-clock-rates = <40000000>; fsl,clk-source = /bits/ 8 <0>; + fsl,stop-mode = <&aonmix_ns_gpr 0x14 0>; status = "disabled"; }; @@ -591,6 +592,7 @@ assigned-clock-parents = <&clk IMX93_CLK_SYS_PLL_PFD1_DIV2>; assigned-clock-rates = <40000000>; fsl,clk-source = /bits/ 8 <0>; + fsl,stop-mode = <&wakeupmix_gpr 0x0c 2>; status = "disabled"; }; diff --git a/arch/arm64/boot/dts/mediatek/mt7622.dtsi b/arch/arm64/boot/dts/mediatek/mt7622.dtsi index 36ef2dbe8add..3ee9266fa8e9 100644 --- a/arch/arm64/boot/dts/mediatek/mt7622.dtsi +++ b/arch/arm64/boot/dts/mediatek/mt7622.dtsi @@ -905,7 +905,7 @@ status = "disabled"; }; - sata_phy: t-phy@1a243000 { + sata_phy: t-phy { compatible = "mediatek,mt7622-tphy", "mediatek,generic-tphy-v1"; #address-cells = <2>; diff --git a/arch/arm64/boot/dts/mediatek/mt7986a.dtsi b/arch/arm64/boot/dts/mediatek/mt7986a.dtsi index 68539ea788df..24eda00e320d 100644 --- a/arch/arm64/boot/dts/mediatek/mt7986a.dtsi +++ b/arch/arm64/boot/dts/mediatek/mt7986a.dtsi @@ -434,7 +434,7 @@ }; }; - pcie_phy: t-phy@11c00000 { + pcie_phy: t-phy { compatible = "mediatek,mt7986-tphy", "mediatek,generic-tphy-v2"; #address-cells = <2>; diff --git a/arch/arm64/boot/dts/mediatek/mt8195-demo.dts b/arch/arm64/boot/dts/mediatek/mt8195-demo.dts index b2485ddfd33b..5d635085fe3f 100644 --- a/arch/arm64/boot/dts/mediatek/mt8195-demo.dts +++ b/arch/arm64/boot/dts/mediatek/mt8195-demo.dts @@ -48,7 +48,7 @@ memory@40000000 { device_type = "memory"; - reg = <0 0x40000000 0 0x80000000>; + reg = <0 0x40000000 0x2 0x00000000>; }; reserved-memory { @@ -56,13 +56,8 @@ #size-cells = <2>; ranges; - /* 2 MiB reserved for ARM Trusted Firmware (BL31) */ - bl31_secmon_reserved: secmon@54600000 { - no-map; - reg = <0 0x54600000 0x0 0x200000>; - }; - - /* 12 MiB reserved for OP-TEE (BL32) + /* + * 12 MiB reserved for OP-TEE (BL32) * +-----------------------+ 0x43e0_0000 * | SHMEM 2MiB | * +-----------------------+ 0x43c0_0000 @@ -75,6 +70,34 @@ no-map; reg = <0 0x43200000 0 0x00c00000>; }; + + scp_mem: memory@50000000 { + compatible = "shared-dma-pool"; + reg = <0 0x50000000 0 0x2900000>; + no-map; + }; + + vpu_mem: memory@53000000 { + compatible = "shared-dma-pool"; + reg = <0 0x53000000 0 0x1400000>; /* 20 MB */ + }; + + /* 2 MiB reserved for ARM Trusted Firmware (BL31) */ + bl31_secmon_mem: memory@54600000 { + no-map; + reg = <0 0x54600000 0x0 0x200000>; + }; + + snd_dma_mem: memory@60000000 { + compatible = "shared-dma-pool"; + reg = <0 0x60000000 0 0x1100000>; + no-map; + }; + + apu_mem: memory@62000000 { + compatible = "shared-dma-pool"; + reg = <0 0x62000000 0 0x1400000>; /* 20 MB */ + }; }; }; diff --git a/arch/arm64/boot/dts/mediatek/mt8195.dtsi b/arch/arm64/boot/dts/mediatek/mt8195.dtsi index 4dbbf8fdab75..54c674c45b49 100644 --- a/arch/arm64/boot/dts/mediatek/mt8195.dtsi +++ b/arch/arm64/boot/dts/mediatek/mt8195.dtsi @@ -313,6 +313,7 @@ interrupts = ; cpus = <&cpu0>, <&cpu1>, <&cpu2>, <&cpu3>, <&cpu4>, <&cpu5>, <&cpu6>, <&cpu7>; + status = "fail"; }; dmic_codec: dmic-codec { @@ -2957,7 +2958,7 @@ clock-names = "merge","merge_async"; power-domains = <&spm MT8195_POWER_DOMAIN_VDOSYS1>; mediatek,gce-client-reg = <&gce0 SUBSYS_1c10XXXX 0xc000 0x1000>; - mediatek,merge-mute = <1>; + mediatek,merge-mute; resets = <&vdosys1 MT8195_VDOSYS1_SW0_RST_B_MERGE0_DL_ASYNC>; }; @@ -2970,7 +2971,7 @@ clock-names = "merge","merge_async"; power-domains = <&spm MT8195_POWER_DOMAIN_VDOSYS1>; mediatek,gce-client-reg = <&gce0 SUBSYS_1c10XXXX 0xd000 0x1000>; - mediatek,merge-mute = <1>; + mediatek,merge-mute; resets = <&vdosys1 MT8195_VDOSYS1_SW0_RST_B_MERGE1_DL_ASYNC>; }; @@ -2983,7 +2984,7 @@ clock-names = "merge","merge_async"; power-domains = <&spm MT8195_POWER_DOMAIN_VDOSYS1>; mediatek,gce-client-reg = <&gce0 SUBSYS_1c10XXXX 0xe000 0x1000>; - mediatek,merge-mute = <1>; + mediatek,merge-mute; resets = <&vdosys1 MT8195_VDOSYS1_SW0_RST_B_MERGE2_DL_ASYNC>; }; @@ -2996,7 +2997,7 @@ clock-names = "merge","merge_async"; power-domains = <&spm MT8195_POWER_DOMAIN_VDOSYS1>; mediatek,gce-client-reg = <&gce0 SUBSYS_1c10XXXX 0xf000 0x1000>; - mediatek,merge-mute = <1>; + mediatek,merge-mute; resets = <&vdosys1 MT8195_VDOSYS1_SW0_RST_B_MERGE3_DL_ASYNC>; }; @@ -3009,7 +3010,7 @@ clock-names = "merge","merge_async"; power-domains = <&spm MT8195_POWER_DOMAIN_VDOSYS1>; mediatek,gce-client-reg = <&gce0 SUBSYS_1c11XXXX 0x0000 0x1000>; - mediatek,merge-fifo-en = <1>; + mediatek,merge-fifo-en; resets = <&vdosys1 MT8195_VDOSYS1_SW0_RST_B_MERGE4_DL_ASYNC>; }; diff --git a/arch/arm64/boot/dts/qcom/sm8150.dtsi b/arch/arm64/boot/dts/qcom/sm8150.dtsi index a7c3020a5de4..06c53000bb74 100644 --- a/arch/arm64/boot/dts/qcom/sm8150.dtsi +++ b/arch/arm64/boot/dts/qcom/sm8150.dtsi @@ -3958,7 +3958,7 @@ pdc: interrupt-controller@b220000 { compatible = "qcom,sm8150-pdc", "qcom,pdc"; - reg = <0 0x0b220000 0 0x400>; + reg = <0 0x0b220000 0 0x30000>; qcom,pdc-ranges = <0 480 94>, <94 609 31>, <125 63 1>; #interrupt-cells = <2>; diff --git a/arch/arm64/configs/defconfig b/arch/arm64/configs/defconfig index 5315789f4868..a789119e6483 100644 --- a/arch/arm64/configs/defconfig +++ b/arch/arm64/configs/defconfig @@ -636,6 +636,7 @@ CONFIG_POWER_RESET_MSM=y CONFIG_POWER_RESET_QCOM_PON=m CONFIG_POWER_RESET_XGENE=y CONFIG_POWER_RESET_SYSCON=y +CONFIG_POWER_RESET_SYSCON_POWEROFF=y CONFIG_SYSCON_REBOOT_MODE=y CONFIG_NVMEM_REBOOT_MODE=m CONFIG_BATTERY_SBS=m @@ -1175,7 +1176,6 @@ CONFIG_COMMON_CLK_S2MPS11=y CONFIG_COMMON_CLK_PWM=y CONFIG_COMMON_CLK_RS9_PCIE=y CONFIG_COMMON_CLK_VC5=y -CONFIG_COMMON_CLK_NPCM8XX=y CONFIG_COMMON_CLK_BD718XX=m CONFIG_CLK_RASPBERRYPI=m CONFIG_CLK_IMX8MM=y diff --git a/arch/arm64/include/asm/acpi.h b/arch/arm64/include/asm/acpi.h index 4d537d56eb84..6792a1f83f2a 100644 --- a/arch/arm64/include/asm/acpi.h +++ b/arch/arm64/include/asm/acpi.h @@ -9,6 +9,7 @@ #ifndef _ASM_ACPI_H #define _ASM_ACPI_H +#include #include #include #include @@ -44,6 +45,24 @@ #define ACPI_MADT_GICC_TRBE (offsetof(struct acpi_madt_generic_interrupt, \ trbe_interrupt) + sizeof(u16)) +/* + * Arm® Functional Fixed Hardware Specification Version 1.2. + * Table 2: Arm Architecture context loss flags + */ +#define CPUIDLE_CORE_CTXT BIT(0) /* Core context Lost */ + +static inline unsigned int arch_get_idle_state_flags(u32 arch_flags) +{ + if (arch_flags & CPUIDLE_CORE_CTXT) + return CPUIDLE_FLAG_TIMER_STOP; + + return 0; +} +#define arch_get_idle_state_flags arch_get_idle_state_flags + +#define CPUIDLE_TRACE_CTXT BIT(1) /* Trace context loss */ +#define CPUIDLE_GICR_CTXT BIT(2) /* GICR */ +#define CPUIDLE_GICD_CTXT BIT(3) /* GICD */ /* Basic configuration for ACPI */ #ifdef CONFIG_ACPI diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h index 96e50227f940..5bba39376055 100644 --- a/arch/arm64/include/asm/cpufeature.h +++ b/arch/arm64/include/asm/cpufeature.h @@ -663,7 +663,7 @@ static inline bool supports_clearbhb(int scope) isar2 = read_sanitised_ftr_reg(SYS_ID_AA64ISAR2_EL1); return cpuid_feature_extract_unsigned_field(isar2, - ID_AA64ISAR2_EL1_BC_SHIFT); + ID_AA64ISAR2_EL1_CLRBHB_SHIFT); } const struct cpumask *system_32bit_el0_cpumask(void); diff --git a/arch/arm64/include/asm/cputype.h b/arch/arm64/include/asm/cputype.h index 5f6f84837a49..74d00feb62f0 100644 --- a/arch/arm64/include/asm/cputype.h +++ b/arch/arm64/include/asm/cputype.h @@ -79,6 +79,7 @@ #define ARM_CPU_PART_CORTEX_A78AE 0xD42 #define ARM_CPU_PART_CORTEX_X1 0xD44 #define ARM_CPU_PART_CORTEX_A510 0xD46 +#define ARM_CPU_PART_CORTEX_A520 0xD80 #define ARM_CPU_PART_CORTEX_A710 0xD47 #define ARM_CPU_PART_CORTEX_A715 0xD4D #define ARM_CPU_PART_CORTEX_X2 0xD48 @@ -148,6 +149,7 @@ #define MIDR_CORTEX_A78AE MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A78AE) #define MIDR_CORTEX_X1 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X1) #define MIDR_CORTEX_A510 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A510) +#define MIDR_CORTEX_A520 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A520) #define MIDR_CORTEX_A710 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A710) #define MIDR_CORTEX_A715 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A715) #define MIDR_CORTEX_X2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X2) diff --git a/arch/arm64/include/asm/hugetlb.h b/arch/arm64/include/asm/hugetlb.h index f43a38ac1779..2ddc33d93b13 100644 --- a/arch/arm64/include/asm/hugetlb.h +++ b/arch/arm64/include/asm/hugetlb.h @@ -28,7 +28,7 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags); #define arch_make_huge_pte arch_make_huge_pte #define __HAVE_ARCH_HUGE_SET_HUGE_PTE_AT extern void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, pte_t pte); + pte_t *ptep, pte_t pte, unsigned long sz); #define __HAVE_ARCH_HUGE_PTEP_SET_ACCESS_FLAGS extern int huge_ptep_set_access_flags(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h index 5882b2415596..1095c6647e96 100644 --- a/arch/arm64/include/asm/kvm_arm.h +++ b/arch/arm64/include/asm/kvm_arm.h @@ -344,14 +344,14 @@ */ #define __HFGRTR_EL2_RES0 (GENMASK(63, 56) | GENMASK(53, 51)) #define __HFGRTR_EL2_MASK GENMASK(49, 0) -#define __HFGRTR_EL2_nMASK (GENMASK(55, 54) | BIT(50)) +#define __HFGRTR_EL2_nMASK (GENMASK(58, 57) | GENMASK(55, 54) | BIT(50)) #define __HFGWTR_EL2_RES0 (GENMASK(63, 56) | GENMASK(53, 51) | \ BIT(46) | BIT(42) | BIT(40) | BIT(28) | \ GENMASK(26, 25) | BIT(21) | BIT(18) | \ GENMASK(15, 14) | GENMASK(10, 9) | BIT(2)) #define __HFGWTR_EL2_MASK GENMASK(49, 0) -#define __HFGWTR_EL2_nMASK (GENMASK(55, 54) | BIT(50)) +#define __HFGWTR_EL2_nMASK (GENMASK(58, 57) | GENMASK(55, 54) | BIT(50)) #define __HFGITR_EL2_RES0 GENMASK(63, 57) #define __HFGITR_EL2_MASK GENMASK(54, 0) diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h index b7238c72a04c..66efd67ea7e8 100644 --- a/arch/arm64/include/asm/kvm_hyp.h +++ b/arch/arm64/include/asm/kvm_hyp.h @@ -118,7 +118,7 @@ void deactivate_traps_vhe_put(struct kvm_vcpu *vcpu); u64 __guest_enter(struct kvm_vcpu *vcpu); -bool kvm_host_psci_handler(struct kvm_cpu_context *host_ctxt); +bool kvm_host_psci_handler(struct kvm_cpu_context *host_ctxt, u32 func_id); #ifdef __KVM_NVHE_HYPERVISOR__ void __noreturn __hyp_do_panic(struct kvm_cpu_context *host_ctxt, u64 spsr, diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c index be66e94a21bd..5706e74c5578 100644 --- a/arch/arm64/kernel/cpu_errata.c +++ b/arch/arm64/kernel/cpu_errata.c @@ -730,6 +730,14 @@ const struct arm64_cpu_capabilities arm64_errata[] = { .cpu_enable = cpu_clear_bf16_from_user_emulation, }, #endif +#ifdef CONFIG_ARM64_ERRATUM_2966298 + { + .desc = "ARM erratum 2966298", + .capability = ARM64_WORKAROUND_2966298, + /* Cortex-A520 r0p0 - r0p1 */ + ERRATA_MIDR_REV_RANGE(MIDR_CORTEX_A520, 0, 0, 1), + }, +#endif #ifdef CONFIG_AMPERE_ERRATUM_AC03_CPU_38 { .desc = "AmpereOne erratum AC03_CPU_38", diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index b018ae12ff5f..444a73c2e638 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -222,7 +222,8 @@ static const struct arm64_ftr_bits ftr_id_aa64isar1[] = { static const struct arm64_ftr_bits ftr_id_aa64isar2[] = { ARM64_FTR_BITS(FTR_VISIBLE, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64ISAR2_EL1_CSSC_SHIFT, 4, 0), ARM64_FTR_BITS(FTR_VISIBLE, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64ISAR2_EL1_RPRFM_SHIFT, 4, 0), - ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_HIGHER_SAFE, ID_AA64ISAR2_EL1_BC_SHIFT, 4, 0), + ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR2_EL1_CLRBHB_SHIFT, 4, 0), + ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR2_EL1_BC_SHIFT, 4, 0), ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR2_EL1_MOPS_SHIFT, 4, 0), ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_PTR_AUTH), FTR_STRICT, FTR_EXACT, ID_AA64ISAR2_EL1_APA3_SHIFT, 4, 0), diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index 6ad61de03d0a..a6030913cd58 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -428,6 +428,10 @@ alternative_else_nop_endif ldp x28, x29, [sp, #16 * 14] .if \el == 0 +alternative_if ARM64_WORKAROUND_2966298 + tlbi vale1, xzr + dsb nsh +alternative_else_nop_endif alternative_if_not ARM64_UNMAP_KERNEL_AT_EL0 ldr lr, [sp, #S_LR] add sp, sp, #PT_REGS_SIZE // restore sp diff --git a/arch/arm64/kvm/arch_timer.c b/arch/arm64/kvm/arch_timer.c index 6dcdae4d38cb..a1e24228aaaa 100644 --- a/arch/arm64/kvm/arch_timer.c +++ b/arch/arm64/kvm/arch_timer.c @@ -55,11 +55,6 @@ static struct irq_ops arch_timer_irq_ops = { .get_input_level = kvm_arch_timer_get_input_level, }; -static bool has_cntpoff(void) -{ - return (has_vhe() && cpus_have_final_cap(ARM64_HAS_ECV_CNTPOFF)); -} - static int nr_timers(struct kvm_vcpu *vcpu) { if (!vcpu_has_nv(vcpu)) @@ -180,7 +175,7 @@ u64 kvm_phys_timer_read(void) return timecounter->cc->read(timecounter->cc); } -static void get_timer_map(struct kvm_vcpu *vcpu, struct timer_map *map) +void get_timer_map(struct kvm_vcpu *vcpu, struct timer_map *map) { if (vcpu_has_nv(vcpu)) { if (is_hyp_ctxt(vcpu)) { @@ -548,8 +543,7 @@ static void timer_save_state(struct arch_timer_context *ctx) timer_set_ctl(ctx, read_sysreg_el0(SYS_CNTP_CTL)); cval = read_sysreg_el0(SYS_CNTP_CVAL); - if (!has_cntpoff()) - cval -= timer_get_offset(ctx); + cval -= timer_get_offset(ctx); timer_set_cval(ctx, cval); @@ -636,8 +630,7 @@ static void timer_restore_state(struct arch_timer_context *ctx) cval = timer_get_cval(ctx); offset = timer_get_offset(ctx); set_cntpoff(offset); - if (!has_cntpoff()) - cval += offset; + cval += offset; write_sysreg_el0(cval, SYS_CNTP_CVAL); isb(); write_sysreg_el0(timer_get_ctl(ctx), SYS_CNTP_CTL); diff --git a/arch/arm64/kvm/emulate-nested.c b/arch/arm64/kvm/emulate-nested.c index 9ced1bf0c2b7..ee902ff2a50f 100644 --- a/arch/arm64/kvm/emulate-nested.c +++ b/arch/arm64/kvm/emulate-nested.c @@ -977,6 +977,8 @@ enum fg_filter_id { static const struct encoding_to_trap_config encoding_to_fgt[] __initconst = { /* HFGRTR_EL2, HFGWTR_EL2 */ + SR_FGT(SYS_PIR_EL1, HFGxTR, nPIR_EL1, 0), + SR_FGT(SYS_PIRE0_EL1, HFGxTR, nPIRE0_EL1, 0), SR_FGT(SYS_TPIDR2_EL0, HFGxTR, nTPIDR2_EL0, 0), SR_FGT(SYS_SMPRI_EL1, HFGxTR, nSMPRI_EL1, 0), SR_FGT(SYS_ACCDATA_EL1, HFGxTR, nACCDATA_EL1, 0), diff --git a/arch/arm64/kvm/hyp/include/nvhe/ffa.h b/arch/arm64/kvm/hyp/include/nvhe/ffa.h index 1becb10ecd80..d9fd5e6c7d3c 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/ffa.h +++ b/arch/arm64/kvm/hyp/include/nvhe/ffa.h @@ -12,6 +12,6 @@ #define FFA_MAX_FUNC_NUM 0x7F int hyp_ffa_init(void *pages); -bool kvm_host_ffa_handler(struct kvm_cpu_context *host_ctxt); +bool kvm_host_ffa_handler(struct kvm_cpu_context *host_ctxt, u32 func_id); #endif /* __KVM_HYP_FFA_H */ diff --git a/arch/arm64/kvm/hyp/nvhe/ffa.c b/arch/arm64/kvm/hyp/nvhe/ffa.c index ab4f5d160c58..6e4dba9eadef 100644 --- a/arch/arm64/kvm/hyp/nvhe/ffa.c +++ b/arch/arm64/kvm/hyp/nvhe/ffa.c @@ -634,9 +634,8 @@ out_handled: return true; } -bool kvm_host_ffa_handler(struct kvm_cpu_context *host_ctxt) +bool kvm_host_ffa_handler(struct kvm_cpu_context *host_ctxt, u32 func_id) { - DECLARE_REG(u64, func_id, host_ctxt, 0); struct arm_smccc_res res; /* diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-init.S b/arch/arm64/kvm/hyp/nvhe/hyp-init.S index 90fade1b032e..1cc06e6797bd 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-init.S +++ b/arch/arm64/kvm/hyp/nvhe/hyp-init.S @@ -57,6 +57,7 @@ __do_hyp_init: cmp x0, #HVC_STUB_HCALL_NR b.lo __kvm_handle_stub_hvc + bic x0, x0, #ARM_SMCCC_CALL_HINTS mov x3, #KVM_HOST_SMCCC_FUNC(__kvm_hyp_init) cmp x0, x3 b.eq 1f diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c index 857d9bc04fd4..2385fd03ed87 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -368,6 +368,7 @@ static void handle_host_hcall(struct kvm_cpu_context *host_ctxt) if (static_branch_unlikely(&kvm_protected_mode_initialized)) hcall_min = __KVM_HOST_SMCCC_FUNC___pkvm_prot_finalize; + id &= ~ARM_SMCCC_CALL_HINTS; id -= KVM_HOST_SMCCC_ID(0); if (unlikely(id < hcall_min || id >= ARRAY_SIZE(host_hcall))) @@ -392,11 +393,14 @@ static void default_host_smc_handler(struct kvm_cpu_context *host_ctxt) static void handle_host_smc(struct kvm_cpu_context *host_ctxt) { + DECLARE_REG(u64, func_id, host_ctxt, 0); bool handled; - handled = kvm_host_psci_handler(host_ctxt); + func_id &= ~ARM_SMCCC_CALL_HINTS; + + handled = kvm_host_psci_handler(host_ctxt, func_id); if (!handled) - handled = kvm_host_ffa_handler(host_ctxt); + handled = kvm_host_ffa_handler(host_ctxt, func_id); if (!handled) default_host_smc_handler(host_ctxt); diff --git a/arch/arm64/kvm/hyp/nvhe/psci-relay.c b/arch/arm64/kvm/hyp/nvhe/psci-relay.c index 24543d2a3490..d57bcb6ab94d 100644 --- a/arch/arm64/kvm/hyp/nvhe/psci-relay.c +++ b/arch/arm64/kvm/hyp/nvhe/psci-relay.c @@ -273,9 +273,8 @@ static unsigned long psci_1_0_handler(u64 func_id, struct kvm_cpu_context *host_ } } -bool kvm_host_psci_handler(struct kvm_cpu_context *host_ctxt) +bool kvm_host_psci_handler(struct kvm_cpu_context *host_ctxt, u32 func_id) { - DECLARE_REG(u64, func_id, host_ctxt, 0); unsigned long ret; switch (kvm_host_psci_config.version) { diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c index 6537f58b1a8c..448b17080d36 100644 --- a/arch/arm64/kvm/hyp/vhe/switch.c +++ b/arch/arm64/kvm/hyp/vhe/switch.c @@ -39,6 +39,26 @@ static void __activate_traps(struct kvm_vcpu *vcpu) ___activate_traps(vcpu); + if (has_cntpoff()) { + struct timer_map map; + + get_timer_map(vcpu, &map); + + /* + * We're entrering the guest. Reload the correct + * values from memory now that TGE is clear. + */ + if (map.direct_ptimer == vcpu_ptimer(vcpu)) + val = __vcpu_sys_reg(vcpu, CNTP_CVAL_EL0); + if (map.direct_ptimer == vcpu_hptimer(vcpu)) + val = __vcpu_sys_reg(vcpu, CNTHP_CVAL_EL2); + + if (map.direct_ptimer) { + write_sysreg_el0(val, SYS_CNTP_CVAL); + isb(); + } + } + val = read_sysreg(cpacr_el1); val |= CPACR_ELx_TTA; val &= ~(CPACR_EL1_ZEN_EL0EN | CPACR_EL1_ZEN_EL1EN | @@ -77,6 +97,30 @@ static void __deactivate_traps(struct kvm_vcpu *vcpu) write_sysreg(HCR_HOST_VHE_FLAGS, hcr_el2); + if (has_cntpoff()) { + struct timer_map map; + u64 val, offset; + + get_timer_map(vcpu, &map); + + /* + * We're exiting the guest. Save the latest CVAL value + * to memory and apply the offset now that TGE is set. + */ + val = read_sysreg_el0(SYS_CNTP_CVAL); + if (map.direct_ptimer == vcpu_ptimer(vcpu)) + __vcpu_sys_reg(vcpu, CNTP_CVAL_EL0) = val; + if (map.direct_ptimer == vcpu_hptimer(vcpu)) + __vcpu_sys_reg(vcpu, CNTHP_CVAL_EL2) = val; + + offset = read_sysreg_s(SYS_CNTPOFF_EL2); + + if (map.direct_ptimer && offset) { + write_sysreg_el0(val + offset, SYS_CNTP_CVAL); + isb(); + } + } + /* * ARM errata 1165522 and 1530923 require the actual execution of the * above before we can switch to the EL2/EL0 translation regime used by diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index 587a104f66c3..482280fe22d7 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -652,6 +652,9 @@ int hyp_alloc_private_va_range(size_t size, unsigned long *haddr) mutex_unlock(&kvm_hyp_pgd_mutex); + if (!ret) + *haddr = base; + return ret; } diff --git a/arch/arm64/kvm/pmu.c b/arch/arm64/kvm/pmu.c index 0eea225fd09a..a243934c5568 100644 --- a/arch/arm64/kvm/pmu.c +++ b/arch/arm64/kvm/pmu.c @@ -39,7 +39,7 @@ void kvm_set_pmu_events(u32 set, struct perf_event_attr *attr) { struct kvm_pmu_events *pmu = kvm_get_pmu_events(); - if (!kvm_arm_support_pmu_v3() || !pmu || !kvm_pmu_switch_needed(attr)) + if (!kvm_arm_support_pmu_v3() || !kvm_pmu_switch_needed(attr)) return; if (!attr->exclude_host) @@ -55,7 +55,7 @@ void kvm_clr_pmu_events(u32 clr) { struct kvm_pmu_events *pmu = kvm_get_pmu_events(); - if (!kvm_arm_support_pmu_v3() || !pmu) + if (!kvm_arm_support_pmu_v3()) return; pmu->events_host &= ~clr; diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c index e92ec810d449..0afd6136e275 100644 --- a/arch/arm64/kvm/sys_regs.c +++ b/arch/arm64/kvm/sys_regs.c @@ -2122,8 +2122,8 @@ static const struct sys_reg_desc sys_reg_descs[] = { { SYS_DESC(SYS_PMMIR_EL1), trap_raz_wi }, { SYS_DESC(SYS_MAIR_EL1), access_vm_reg, reset_unknown, MAIR_EL1 }, - { SYS_DESC(SYS_PIRE0_EL1), access_vm_reg, reset_unknown, PIRE0_EL1 }, - { SYS_DESC(SYS_PIR_EL1), access_vm_reg, reset_unknown, PIR_EL1 }, + { SYS_DESC(SYS_PIRE0_EL1), NULL, reset_unknown, PIRE0_EL1 }, + { SYS_DESC(SYS_PIR_EL1), NULL, reset_unknown, PIR_EL1 }, { SYS_DESC(SYS_AMAIR_EL1), access_vm_reg, reset_amair_el1, AMAIR_EL1 }, { SYS_DESC(SYS_LORSA_EL1), trap_loregion }, diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 9c52718ea750..13fd592228b1 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -241,15 +241,8 @@ static void clear_flush(struct mm_struct *mm, flush_tlb_range(&vma, saddr, addr); } -static inline struct folio *hugetlb_swap_entry_to_folio(swp_entry_t entry) -{ - VM_BUG_ON(!is_migration_entry(entry) && !is_hwpoison_entry(entry)); - - return page_folio(pfn_to_page(swp_offset_pfn(entry))); -} - void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, pte_t pte) + pte_t *ptep, pte_t pte, unsigned long sz) { size_t pgsize; int i; @@ -257,13 +250,10 @@ void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, unsigned long pfn, dpfn; pgprot_t hugeprot; - if (!pte_present(pte)) { - struct folio *folio; - - folio = hugetlb_swap_entry_to_folio(pte_to_swp_entry(pte)); - ncontig = num_contig_ptes(folio_size(folio), &pgsize); + ncontig = num_contig_ptes(sz, &pgsize); - for (i = 0; i < ncontig; i++, ptep++) + if (!pte_present(pte)) { + for (i = 0; i < ncontig; i++, ptep++, addr += pgsize) set_pte_at(mm, addr, ptep, pte); return; } @@ -273,7 +263,6 @@ void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, return; } - ncontig = find_num_contig(mm, addr, ptep, &pgsize); pfn = pte_pfn(pte); dpfn = pgsize >> PAGE_SHIFT; hugeprot = pte_pgprot(pte); @@ -571,5 +560,7 @@ pte_t huge_ptep_modify_prot_start(struct vm_area_struct *vma, unsigned long addr void huge_ptep_modify_prot_commit(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, pte_t old_pte, pte_t pte) { - set_huge_pte_at(vma->vm_mm, addr, ptep, pte); + unsigned long psize = huge_page_size(hstate_vma(vma)); + + set_huge_pte_at(vma->vm_mm, addr, ptep, pte, psize); } diff --git a/arch/arm64/tools/cpucaps b/arch/arm64/tools/cpucaps index c3f06fdef609..dea3dc89234b 100644 --- a/arch/arm64/tools/cpucaps +++ b/arch/arm64/tools/cpucaps @@ -84,6 +84,7 @@ WORKAROUND_2077057 WORKAROUND_2457168 WORKAROUND_2645198 WORKAROUND_2658417 +WORKAROUND_2966298 WORKAROUND_AMPERE_AC03_CPU_38 WORKAROUND_TRBE_OVERWRITE_FILL_MODE WORKAROUND_TSB_FLUSH_FAILURE diff --git a/arch/arm64/tools/sysreg b/arch/arm64/tools/sysreg index 2517ef7c21cf..76ce150e7347 100644 --- a/arch/arm64/tools/sysreg +++ b/arch/arm64/tools/sysreg @@ -1347,7 +1347,11 @@ UnsignedEnum 51:48 RPRFM 0b0000 NI 0b0001 IMP EndEnum -Res0 47:28 +Res0 47:32 +UnsignedEnum 31:28 CLRBHB + 0b0000 NI + 0b0001 IMP +EndEnum UnsignedEnum 27:24 PAC_frac 0b0000 NI 0b0001 IMP diff --git a/arch/ia64/include/asm/cpu.h b/arch/ia64/include/asm/cpu.h index db125df9e088..642d71675ddb 100644 --- a/arch/ia64/include/asm/cpu.h +++ b/arch/ia64/include/asm/cpu.h @@ -15,9 +15,4 @@ DECLARE_PER_CPU(struct ia64_cpu, cpu_devices); DECLARE_PER_CPU(int, cpu_state); -#ifdef CONFIG_HOTPLUG_CPU -extern int arch_register_cpu(int num); -extern void arch_unregister_cpu(int); -#endif - #endif /* _ASM_IA64_CPU_H_ */ diff --git a/arch/ia64/kernel/acpi.c b/arch/ia64/kernel/acpi.c index 15f6cfddcc08..41e8fe55cd98 100644 --- a/arch/ia64/kernel/acpi.c +++ b/arch/ia64/kernel/acpi.c @@ -907,3 +907,7 @@ EXPORT_SYMBOL(acpi_unregister_ioapic); * TBD when IA64 starts to support suspend... */ int acpi_suspend_lowlevel(void) { return 0; } + +void acpi_proc_quirk_mwait_check(void) +{ +} diff --git a/arch/ia64/kernel/topology.c b/arch/ia64/kernel/topology.c index 94a848b06f15..741863a187a6 100644 --- a/arch/ia64/kernel/topology.c +++ b/arch/ia64/kernel/topology.c @@ -59,7 +59,7 @@ void __ref arch_unregister_cpu(int num) } EXPORT_SYMBOL(arch_unregister_cpu); #else -static int __init arch_register_cpu(int num) +int __init arch_register_cpu(int num) { return register_cpu(&sysfs_cpus[num].cpu, num); } diff --git a/arch/loongarch/include/asm/addrspace.h b/arch/loongarch/include/asm/addrspace.h index 5c9c03bdf915..b24437e28c6e 100644 --- a/arch/loongarch/include/asm/addrspace.h +++ b/arch/loongarch/include/asm/addrspace.h @@ -19,7 +19,7 @@ */ #ifndef __ASSEMBLY__ #ifndef PHYS_OFFSET -#define PHYS_OFFSET _AC(0, UL) +#define PHYS_OFFSET _UL(0) #endif extern unsigned long vm_map_base; #endif /* __ASSEMBLY__ */ @@ -43,7 +43,7 @@ extern unsigned long vm_map_base; * Memory above this physical address will be considered highmem. */ #ifndef HIGHMEM_START -#define HIGHMEM_START (_AC(1, UL) << _AC(DMW_PABITS, UL)) +#define HIGHMEM_START (_UL(1) << _UL(DMW_PABITS)) #endif #define TO_PHYS(x) ( ((x) & TO_PHYS_MASK)) @@ -65,16 +65,16 @@ extern unsigned long vm_map_base; #define _ATYPE_ #define _ATYPE32_ #define _ATYPE64_ -#define _CONST64_(x) x #else #define _ATYPE_ __PTRDIFF_TYPE__ #define _ATYPE32_ int #define _ATYPE64_ __s64 +#endif + #ifdef CONFIG_64BIT -#define _CONST64_(x) x ## UL +#define _CONST64_(x) _UL(x) #else -#define _CONST64_(x) x ## ULL -#endif +#define _CONST64_(x) _ULL(x) #endif /* diff --git a/arch/loongarch/include/asm/elf.h b/arch/loongarch/include/asm/elf.h index 7af0cebf28d7..b9a4ab54285c 100644 --- a/arch/loongarch/include/asm/elf.h +++ b/arch/loongarch/include/asm/elf.h @@ -111,6 +111,15 @@ #define R_LARCH_TLS_GD_HI20 98 #define R_LARCH_32_PCREL 99 #define R_LARCH_RELAX 100 +#define R_LARCH_DELETE 101 +#define R_LARCH_ALIGN 102 +#define R_LARCH_PCREL20_S2 103 +#define R_LARCH_CFA 104 +#define R_LARCH_ADD6 105 +#define R_LARCH_SUB6 106 +#define R_LARCH_ADD_ULEB128 107 +#define R_LARCH_SUB_ULEB128 108 +#define R_LARCH_64_PCREL 109 #ifndef ELF_ARCH diff --git a/arch/loongarch/include/asm/exception.h b/arch/loongarch/include/asm/exception.h new file mode 100644 index 000000000000..af74a3fdcad1 --- /dev/null +++ b/arch/loongarch/include/asm/exception.h @@ -0,0 +1,45 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ + +#ifndef __ASM_EXCEPTION_H +#define __ASM_EXCEPTION_H + +#include +#include + +void show_registers(struct pt_regs *regs); + +asmlinkage void cache_parity_error(void); +asmlinkage void noinstr do_ade(struct pt_regs *regs); +asmlinkage void noinstr do_ale(struct pt_regs *regs); +asmlinkage void noinstr do_bce(struct pt_regs *regs); +asmlinkage void noinstr do_bp(struct pt_regs *regs); +asmlinkage void noinstr do_ri(struct pt_regs *regs); +asmlinkage void noinstr do_fpu(struct pt_regs *regs); +asmlinkage void noinstr do_fpe(struct pt_regs *regs, unsigned long fcsr); +asmlinkage void noinstr do_lsx(struct pt_regs *regs); +asmlinkage void noinstr do_lasx(struct pt_regs *regs); +asmlinkage void noinstr do_lbt(struct pt_regs *regs); +asmlinkage void noinstr do_watch(struct pt_regs *regs); +asmlinkage void noinstr do_syscall(struct pt_regs *regs); +asmlinkage void noinstr do_reserved(struct pt_regs *regs); +asmlinkage void noinstr do_vint(struct pt_regs *regs, unsigned long sp); +asmlinkage void __kprobes do_page_fault(struct pt_regs *regs, + unsigned long write, unsigned long address); + +asmlinkage void handle_ade(void); +asmlinkage void handle_ale(void); +asmlinkage void handle_bce(void); +asmlinkage void handle_sys(void); +asmlinkage void handle_bp(void); +asmlinkage void handle_ri(void); +asmlinkage void handle_fpu(void); +asmlinkage void handle_fpe(void); +asmlinkage void handle_lsx(void); +asmlinkage void handle_lasx(void); +asmlinkage void handle_lbt(void); +asmlinkage void handle_watch(void); +asmlinkage void handle_reserved(void); +asmlinkage void handle_vint(void); +asmlinkage void noinstr handle_loongarch_irq(struct pt_regs *regs); + +#endif /* __ASM_EXCEPTION_H */ diff --git a/arch/loongarch/include/asm/io.h b/arch/loongarch/include/asm/io.h index 0dcb36b32cb2..c486c2341b66 100644 --- a/arch/loongarch/include/asm/io.h +++ b/arch/loongarch/include/asm/io.h @@ -52,10 +52,9 @@ static inline void __iomem *ioremap_prot(phys_addr_t offset, unsigned long size, * @offset: bus address of the memory * @size: size of the resource to map */ -extern pgprot_t pgprot_wc; - #define ioremap_wc(offset, size) \ - ioremap_prot((offset), (size), pgprot_val(pgprot_wc)) + ioremap_prot((offset), (size), \ + pgprot_val(wc_enabled ? PAGE_KERNEL_WUC : PAGE_KERNEL_SUC)) #define ioremap_cache(offset, size) \ ioremap_prot((offset), (size), pgprot_val(PAGE_KERNEL)) diff --git a/arch/loongarch/include/asm/kasan.h b/arch/loongarch/include/asm/kasan.h index deeff8158f45..cd6084f4e153 100644 --- a/arch/loongarch/include/asm/kasan.h +++ b/arch/loongarch/include/asm/kasan.h @@ -10,8 +10,6 @@ #include #include -#define __HAVE_ARCH_SHADOW_MAP - #define KASAN_SHADOW_SCALE_SHIFT 3 #define KASAN_SHADOW_OFFSET _AC(CONFIG_KASAN_SHADOW_OFFSET, UL) @@ -62,61 +60,22 @@ extern bool kasan_early_stage; extern unsigned char kasan_early_shadow_page[PAGE_SIZE]; +#define kasan_mem_to_shadow kasan_mem_to_shadow +void *kasan_mem_to_shadow(const void *addr); + +#define kasan_shadow_to_mem kasan_shadow_to_mem +const void *kasan_shadow_to_mem(const void *shadow_addr); + #define kasan_arch_is_ready kasan_arch_is_ready static __always_inline bool kasan_arch_is_ready(void) { return !kasan_early_stage; } -static inline void *kasan_mem_to_shadow(const void *addr) -{ - if (!kasan_arch_is_ready()) { - return (void *)(kasan_early_shadow_page); - } else { - unsigned long maddr = (unsigned long)addr; - unsigned long xrange = (maddr >> XRANGE_SHIFT) & 0xffff; - unsigned long offset = 0; - - maddr &= XRANGE_SHADOW_MASK; - switch (xrange) { - case XKPRANGE_CC_SEG: - offset = XKPRANGE_CC_SHADOW_OFFSET; - break; - case XKPRANGE_UC_SEG: - offset = XKPRANGE_UC_SHADOW_OFFSET; - break; - case XKVRANGE_VC_SEG: - offset = XKVRANGE_VC_SHADOW_OFFSET; - break; - default: - WARN_ON(1); - return NULL; - } - - return (void *)((maddr >> KASAN_SHADOW_SCALE_SHIFT) + offset); - } -} - -static inline const void *kasan_shadow_to_mem(const void *shadow_addr) +#define addr_has_metadata addr_has_metadata +static __always_inline bool addr_has_metadata(const void *addr) { - unsigned long addr = (unsigned long)shadow_addr; - - if (unlikely(addr > KASAN_SHADOW_END) || - unlikely(addr < KASAN_SHADOW_START)) { - WARN_ON(1); - return NULL; - } - - if (addr >= XKVRANGE_VC_SHADOW_OFFSET) - return (void *)(((addr - XKVRANGE_VC_SHADOW_OFFSET) << KASAN_SHADOW_SCALE_SHIFT) + XKVRANGE_VC_START); - else if (addr >= XKPRANGE_UC_SHADOW_OFFSET) - return (void *)(((addr - XKPRANGE_UC_SHADOW_OFFSET) << KASAN_SHADOW_SCALE_SHIFT) + XKPRANGE_UC_START); - else if (addr >= XKPRANGE_CC_SHADOW_OFFSET) - return (void *)(((addr - XKPRANGE_CC_SHADOW_OFFSET) << KASAN_SHADOW_SCALE_SHIFT) + XKPRANGE_CC_START); - else { - WARN_ON(1); - return NULL; - } + return (kasan_mem_to_shadow((void *)addr) != NULL); } void kasan_init(void); diff --git a/arch/loongarch/include/asm/linkage.h b/arch/loongarch/include/asm/linkage.h index 81b0c4cfbf4f..e2eca1a25b4e 100644 --- a/arch/loongarch/include/asm/linkage.h +++ b/arch/loongarch/include/asm/linkage.h @@ -33,4 +33,12 @@ .cfi_endproc; \ SYM_END(name, SYM_T_FUNC) +#define SYM_CODE_START(name) \ + SYM_START(name, SYM_L_GLOBAL, SYM_A_ALIGN) \ + .cfi_startproc; + +#define SYM_CODE_END(name) \ + .cfi_endproc; \ + SYM_END(name, SYM_T_NONE) + #endif diff --git a/arch/loongarch/include/asm/pgtable-bits.h b/arch/loongarch/include/asm/pgtable-bits.h index 35348d4c4209..21319c1e045c 100644 --- a/arch/loongarch/include/asm/pgtable-bits.h +++ b/arch/loongarch/include/asm/pgtable-bits.h @@ -105,13 +105,15 @@ static inline pgprot_t pgprot_noncached(pgprot_t _prot) return __pgprot(prot); } +extern bool wc_enabled; + #define pgprot_writecombine pgprot_writecombine static inline pgprot_t pgprot_writecombine(pgprot_t _prot) { unsigned long prot = pgprot_val(_prot); - prot = (prot & ~_CACHE_MASK) | _CACHE_WUC; + prot = (prot & ~_CACHE_MASK) | (wc_enabled ? _CACHE_WUC : _CACHE_SUC); return __pgprot(prot); } diff --git a/arch/loongarch/include/asm/smp.h b/arch/loongarch/include/asm/smp.h index 66ecb480c894..f81e5f01d619 100644 --- a/arch/loongarch/include/asm/smp.h +++ b/arch/loongarch/include/asm/smp.h @@ -70,6 +70,7 @@ struct secondary_data { extern struct secondary_data cpuboot_data; extern asmlinkage void smpboot_entry(void); +extern asmlinkage void start_secondary(void); extern void calculate_cpu_foreign_map(void); diff --git a/arch/loongarch/kernel/Makefile b/arch/loongarch/kernel/Makefile index c56ea0b75448..4fcc168f0732 100644 --- a/arch/loongarch/kernel/Makefile +++ b/arch/loongarch/kernel/Makefile @@ -19,6 +19,10 @@ obj-$(CONFIG_CPU_HAS_LBT) += lbt.o obj-$(CONFIG_ARCH_STRICT_ALIGN) += unaligned.o +CFLAGS_module.o += $(call cc-option,-Wno-override-init,) +CFLAGS_syscall.o += $(call cc-option,-Wno-override-init,) +CFLAGS_perf_event.o += $(call cc-option,-Wno-override-init,) + ifdef CONFIG_FUNCTION_TRACER ifndef CONFIG_DYNAMIC_FTRACE obj-y += mcount.o ftrace.o diff --git a/arch/loongarch/kernel/acpi.c b/arch/loongarch/kernel/acpi.c index 9450e09073eb..8e00a754e548 100644 --- a/arch/loongarch/kernel/acpi.c +++ b/arch/loongarch/kernel/acpi.c @@ -281,7 +281,6 @@ acpi_numa_processor_affinity_init(struct acpi_srat_cpu_affinity *pa) pr_info("SRAT: PXM %u -> CPU 0x%02x -> Node %u\n", pxm, pa->apic_id, node); } -void __init acpi_numa_arch_fixup(void) {} #endif void __init arch_reserve_mem_area(acpi_physical_address addr, size_t size) diff --git a/arch/loongarch/kernel/entry.S b/arch/loongarch/kernel/entry.S index 65518bb8f472..1ec8e4c4cc2b 100644 --- a/arch/loongarch/kernel/entry.S +++ b/arch/loongarch/kernel/entry.S @@ -18,7 +18,7 @@ .text .cfi_sections .debug_frame .align 5 -SYM_FUNC_START(handle_syscall) +SYM_CODE_START(handle_syscall) csrrd t0, PERCPU_BASE_KS la.pcrel t1, kernelsp add.d t1, t1, t0 @@ -71,7 +71,7 @@ SYM_FUNC_START(handle_syscall) bl do_syscall RESTORE_ALL_AND_RET -SYM_FUNC_END(handle_syscall) +SYM_CODE_END(handle_syscall) _ASM_NOKPROBE(handle_syscall) SYM_CODE_START(ret_from_fork) diff --git a/arch/loongarch/kernel/genex.S b/arch/loongarch/kernel/genex.S index 78f066384657..2bb3aa2dcfcb 100644 --- a/arch/loongarch/kernel/genex.S +++ b/arch/loongarch/kernel/genex.S @@ -31,7 +31,7 @@ SYM_FUNC_START(__arch_cpu_idle) 1: jr ra SYM_FUNC_END(__arch_cpu_idle) -SYM_FUNC_START(handle_vint) +SYM_CODE_START(handle_vint) BACKUP_T0T1 SAVE_ALL la_abs t1, __arch_cpu_idle @@ -46,11 +46,11 @@ SYM_FUNC_START(handle_vint) la_abs t0, do_vint jirl ra, t0, 0 RESTORE_ALL_AND_RET -SYM_FUNC_END(handle_vint) +SYM_CODE_END(handle_vint) -SYM_FUNC_START(except_vec_cex) +SYM_CODE_START(except_vec_cex) b cache_parity_error -SYM_FUNC_END(except_vec_cex) +SYM_CODE_END(except_vec_cex) .macro build_prep_badv csrrd t0, LOONGARCH_CSR_BADV @@ -66,7 +66,7 @@ SYM_FUNC_END(except_vec_cex) .macro BUILD_HANDLER exception handler prep .align 5 - SYM_FUNC_START(handle_\exception) + SYM_CODE_START(handle_\exception) 666: BACKUP_T0T1 SAVE_ALL @@ -76,7 +76,7 @@ SYM_FUNC_END(except_vec_cex) jirl ra, t0, 0 668: RESTORE_ALL_AND_RET - SYM_FUNC_END(handle_\exception) + SYM_CODE_END(handle_\exception) SYM_DATA(unwind_hint_\exception, .word 668b - 666b) .endm @@ -93,7 +93,7 @@ SYM_FUNC_END(except_vec_cex) BUILD_HANDLER watch watch none BUILD_HANDLER reserved reserved none /* others */ -SYM_FUNC_START(handle_sys) +SYM_CODE_START(handle_sys) la_abs t0, handle_syscall jr t0 -SYM_FUNC_END(handle_sys) +SYM_CODE_END(handle_sys) diff --git a/arch/loongarch/kernel/mem.c b/arch/loongarch/kernel/mem.c index 4a4107a6a965..aed901c57fb4 100644 --- a/arch/loongarch/kernel/mem.c +++ b/arch/loongarch/kernel/mem.c @@ -50,7 +50,6 @@ void __init memblock_init(void) } memblock_set_current_limit(PFN_PHYS(max_low_pfn)); - memblock_set_node(0, PHYS_ADDR_MAX, &memblock.memory, 0); /* Reserve the first 2MB */ memblock_reserve(PHYS_OFFSET, 0x200000); @@ -58,4 +57,7 @@ void __init memblock_init(void) /* Reserve the kernel text/data/bss */ memblock_reserve(__pa_symbol(&_text), __pa_symbol(&_end) - __pa_symbol(&_text)); + + memblock_set_node(0, PHYS_ADDR_MAX, &memblock.memory, 0); + memblock_set_node(0, PHYS_ADDR_MAX, &memblock.reserved, 0); } diff --git a/arch/loongarch/kernel/module-sections.c b/arch/loongarch/kernel/module-sections.c index d4dbcda1c4b0..e2f30ff9afde 100644 --- a/arch/loongarch/kernel/module-sections.c +++ b/arch/loongarch/kernel/module-sections.c @@ -6,6 +6,7 @@ #include #include #include +#include #include Elf_Addr module_emit_got_entry(struct module *mod, Elf_Shdr *sechdrs, Elf_Addr val) diff --git a/arch/loongarch/kernel/module.c b/arch/loongarch/kernel/module.c index b8b86088b2dd..b13b2858fe39 100644 --- a/arch/loongarch/kernel/module.c +++ b/arch/loongarch/kernel/module.c @@ -367,6 +367,24 @@ static int apply_r_larch_got_pc(struct module *mod, return apply_r_larch_pcala(mod, location, got, rela_stack, rela_stack_top, type); } +static int apply_r_larch_32_pcrel(struct module *mod, u32 *location, Elf_Addr v, + s64 *rela_stack, size_t *rela_stack_top, unsigned int type) +{ + ptrdiff_t offset = (void *)v - (void *)location; + + *(u32 *)location = offset; + return 0; +} + +static int apply_r_larch_64_pcrel(struct module *mod, u32 *location, Elf_Addr v, + s64 *rela_stack, size_t *rela_stack_top, unsigned int type) +{ + ptrdiff_t offset = (void *)v - (void *)location; + + *(u64 *)location = offset; + return 0; +} + /* * reloc_handlers_rela() - Apply a particular relocation to a module * @mod: the module to apply the reloc to @@ -382,7 +400,7 @@ typedef int (*reloc_rela_handler)(struct module *mod, u32 *location, Elf_Addr v, /* The handlers for known reloc types */ static reloc_rela_handler reloc_rela_handlers[] = { - [R_LARCH_NONE ... R_LARCH_RELAX] = apply_r_larch_error, + [R_LARCH_NONE ... R_LARCH_64_PCREL] = apply_r_larch_error, [R_LARCH_NONE] = apply_r_larch_none, [R_LARCH_32] = apply_r_larch_32, @@ -396,6 +414,8 @@ static reloc_rela_handler reloc_rela_handlers[] = { [R_LARCH_SOP_POP_32_S_10_5 ... R_LARCH_SOP_POP_32_U] = apply_r_larch_sop_imm_field, [R_LARCH_ADD32 ... R_LARCH_SUB64] = apply_r_larch_add_sub, [R_LARCH_PCALA_HI20...R_LARCH_PCALA64_HI12] = apply_r_larch_pcala, + [R_LARCH_32_PCREL] = apply_r_larch_32_pcrel, + [R_LARCH_64_PCREL] = apply_r_larch_64_pcrel, }; int apply_relocate_add(Elf_Shdr *sechdrs, const char *strtab, diff --git a/arch/loongarch/kernel/numa.c b/arch/loongarch/kernel/numa.c index c7d33c489e04..6e65ff12d5c7 100644 --- a/arch/loongarch/kernel/numa.c +++ b/arch/loongarch/kernel/numa.c @@ -436,7 +436,7 @@ void __init paging_init(void) void __init mem_init(void) { - high_memory = (void *) __va(get_num_physpages() << PAGE_SHIFT); + high_memory = (void *) __va(max_low_pfn << PAGE_SHIFT); memblock_free_all(); } diff --git a/arch/loongarch/kernel/process.c b/arch/loongarch/kernel/process.c index 3cb082e0c992..767d94cce0de 100644 --- a/arch/loongarch/kernel/process.c +++ b/arch/loongarch/kernel/process.c @@ -37,6 +37,7 @@ #include #include #include +#include #include #include #include diff --git a/arch/loongarch/kernel/relocate_kernel.S b/arch/loongarch/kernel/relocate_kernel.S index d13252553a7c..f49f6b053763 100644 --- a/arch/loongarch/kernel/relocate_kernel.S +++ b/arch/loongarch/kernel/relocate_kernel.S @@ -72,7 +72,6 @@ copy_word: LONG_ADDI s5, s5, -1 beqz s5, process_entry b copy_word - b process_entry done: ibar 0 diff --git a/arch/loongarch/kernel/setup.c b/arch/loongarch/kernel/setup.c index 7783f0a3d742..aed65915e932 100644 --- a/arch/loongarch/kernel/setup.c +++ b/arch/loongarch/kernel/setup.c @@ -161,19 +161,19 @@ static void __init smbios_parse(void) } #ifdef CONFIG_ARCH_WRITECOMBINE -pgprot_t pgprot_wc = PAGE_KERNEL_WUC; +bool wc_enabled = true; #else -pgprot_t pgprot_wc = PAGE_KERNEL_SUC; +bool wc_enabled = false; #endif -EXPORT_SYMBOL(pgprot_wc); +EXPORT_SYMBOL(wc_enabled); static int __init setup_writecombine(char *p) { if (!strcmp(p, "on")) - pgprot_wc = PAGE_KERNEL_WUC; + wc_enabled = true; else if (!strcmp(p, "off")) - pgprot_wc = PAGE_KERNEL_SUC; + wc_enabled = false; else pr_warn("Unknown writecombine setting \"%s\".\n", p); diff --git a/arch/loongarch/kernel/signal.c b/arch/loongarch/kernel/signal.c index 504fdfe85203..4a3686d13349 100644 --- a/arch/loongarch/kernel/signal.c +++ b/arch/loongarch/kernel/signal.c @@ -13,6 +13,7 @@ #include #include #include +#include #include #include #include @@ -891,8 +892,8 @@ static unsigned long setup_extcontext(struct extctx_layout *extctx, unsigned lon return new_sp; } -void __user *get_sigframe(struct ksignal *ksig, struct pt_regs *regs, - struct extctx_layout *extctx) +static void __user *get_sigframe(struct ksignal *ksig, struct pt_regs *regs, + struct extctx_layout *extctx) { unsigned long sp; @@ -922,7 +923,7 @@ void __user *get_sigframe(struct ksignal *ksig, struct pt_regs *regs, * Atomically swap in the new signal mask, and wait for a signal. */ -asmlinkage long sys_rt_sigreturn(void) +SYSCALL_DEFINE0(rt_sigreturn) { int sig; sigset_t set; diff --git a/arch/loongarch/kernel/smp.c b/arch/loongarch/kernel/smp.c index 6667b0a90f81..ef35c871244f 100644 --- a/arch/loongarch/kernel/smp.c +++ b/arch/loongarch/kernel/smp.c @@ -13,6 +13,7 @@ #include #include #include +#include #include #include #include @@ -556,10 +557,12 @@ void smp_send_stop(void) smp_call_function(stop_this_cpu, NULL, 0); } +#ifdef CONFIG_PROFILING int setup_profiling_timer(unsigned int multiplier) { return 0; } +#endif static void flush_tlb_all_ipi(void *info) { diff --git a/arch/loongarch/kernel/syscall.c b/arch/loongarch/kernel/syscall.c index 3fc4211db989..b4c5acd7aa3b 100644 --- a/arch/loongarch/kernel/syscall.c +++ b/arch/loongarch/kernel/syscall.c @@ -13,6 +13,7 @@ #include #include +#include #include #include #include diff --git a/arch/loongarch/kernel/time.c b/arch/loongarch/kernel/time.c index c189e03cd5da..3064af94db9c 100644 --- a/arch/loongarch/kernel/time.c +++ b/arch/loongarch/kernel/time.c @@ -29,7 +29,7 @@ static void constant_event_handler(struct clock_event_device *dev) { } -irqreturn_t constant_timer_interrupt(int irq, void *data) +static irqreturn_t constant_timer_interrupt(int irq, void *data) { int cpu = smp_processor_id(); struct clock_event_device *cd; diff --git a/arch/loongarch/kernel/topology.c b/arch/loongarch/kernel/topology.c index caa7cd859078..3fd166006698 100644 --- a/arch/loongarch/kernel/topology.c +++ b/arch/loongarch/kernel/topology.c @@ -1,4 +1,5 @@ // SPDX-License-Identifier: GPL-2.0 +#include #include #include #include @@ -7,6 +8,8 @@ #include #include +#include + static DEFINE_PER_CPU(struct cpu, cpu_devices); #ifdef CONFIG_HOTPLUG_CPU diff --git a/arch/loongarch/kernel/traps.c b/arch/loongarch/kernel/traps.c index 65214774ef7c..aebfc3733a76 100644 --- a/arch/loongarch/kernel/traps.c +++ b/arch/loongarch/kernel/traps.c @@ -25,7 +25,6 @@ #include #include #include -#include #include #include #include @@ -35,6 +34,7 @@ #include #include #include +#include #include #include #include @@ -53,21 +53,6 @@ #include "access-helper.h" -extern asmlinkage void handle_ade(void); -extern asmlinkage void handle_ale(void); -extern asmlinkage void handle_bce(void); -extern asmlinkage void handle_sys(void); -extern asmlinkage void handle_bp(void); -extern asmlinkage void handle_ri(void); -extern asmlinkage void handle_fpu(void); -extern asmlinkage void handle_fpe(void); -extern asmlinkage void handle_lbt(void); -extern asmlinkage void handle_lsx(void); -extern asmlinkage void handle_lasx(void); -extern asmlinkage void handle_reserved(void); -extern asmlinkage void handle_watch(void); -extern asmlinkage void handle_vint(void); - static void show_backtrace(struct task_struct *task, const struct pt_regs *regs, const char *loglvl, bool user) { @@ -439,8 +424,8 @@ static inline void setup_vint_size(unsigned int size) * happen together with Overflow or Underflow, and `ptrace' can set * any bits. */ -void force_fcsr_sig(unsigned long fcsr, void __user *fault_addr, - struct task_struct *tsk) +static void force_fcsr_sig(unsigned long fcsr, + void __user *fault_addr, struct task_struct *tsk) { int si_code = FPE_FLTUNK; @@ -458,7 +443,7 @@ void force_fcsr_sig(unsigned long fcsr, void __user *fault_addr, force_sig_fault(SIGFPE, si_code, fault_addr); } -int process_fpemu_return(int sig, void __user *fault_addr, unsigned long fcsr) +static int process_fpemu_return(int sig, void __user *fault_addr, unsigned long fcsr) { int si_code; @@ -824,7 +809,7 @@ out: asmlinkage void noinstr do_ri(struct pt_regs *regs) { int status = SIGILL; - unsigned int opcode = 0; + unsigned int __maybe_unused opcode; unsigned int __user *era = (unsigned int __user *)exception_era(regs); irqentry_state_t state = irqentry_enter(regs); diff --git a/arch/loongarch/kernel/vmlinux.lds.S b/arch/loongarch/kernel/vmlinux.lds.S index b1686afcf876..bb2ec86f37a8 100644 --- a/arch/loongarch/kernel/vmlinux.lds.S +++ b/arch/loongarch/kernel/vmlinux.lds.S @@ -53,33 +53,6 @@ SECTIONS . = ALIGN(PECOFF_SEGMENT_ALIGN); _etext = .; - /* - * struct alt_inst entries. From the header (alternative.h): - * "Alternative instructions for different CPU types or capabilities" - * Think locking instructions on spinlocks. - */ - . = ALIGN(4); - .altinstructions : AT(ADDR(.altinstructions) - LOAD_OFFSET) { - __alt_instructions = .; - *(.altinstructions) - __alt_instructions_end = .; - } - -#ifdef CONFIG_RELOCATABLE - . = ALIGN(8); - .la_abs : AT(ADDR(.la_abs) - LOAD_OFFSET) { - __la_abs_begin = .; - *(.la_abs) - __la_abs_end = .; - } -#endif - - .got : ALIGN(16) { *(.got) } - .plt : ALIGN(16) { *(.plt) } - .got.plt : ALIGN(16) { *(.got.plt) } - - .data.rel : { *(.data.rel*) } - . = ALIGN(PECOFF_SEGMENT_ALIGN); __init_begin = .; __inittext_begin = .; @@ -94,6 +67,18 @@ SECTIONS __initdata_begin = .; + /* + * struct alt_inst entries. From the header (alternative.h): + * "Alternative instructions for different CPU types or capabilities" + * Think locking instructions on spinlocks. + */ + . = ALIGN(4); + .altinstructions : AT(ADDR(.altinstructions) - LOAD_OFFSET) { + __alt_instructions = .; + *(.altinstructions) + __alt_instructions_end = .; + } + INIT_DATA_SECTION(16) .exit.data : { EXIT_DATA @@ -113,6 +98,11 @@ SECTIONS _sdata = .; RO_DATA(4096) + + .got : ALIGN(16) { *(.got) } + .plt : ALIGN(16) { *(.plt) } + .got.plt : ALIGN(16) { *(.got.plt) } + RW_DATA(1 << CONFIG_L1_CACHE_SHIFT, PAGE_SIZE, THREAD_SIZE) .rela.dyn : ALIGN(8) { @@ -121,6 +111,17 @@ SECTIONS __rela_dyn_end = .; } + .data.rel : { *(.data.rel*) } + +#ifdef CONFIG_RELOCATABLE + . = ALIGN(8); + .la_abs : AT(ADDR(.la_abs) - LOAD_OFFSET) { + __la_abs_begin = .; + *(.la_abs) + __la_abs_end = .; + } +#endif + .sdata : { *(.sdata) } diff --git a/arch/loongarch/mm/fault.c b/arch/loongarch/mm/fault.c index e6376e3dce86..1fc2f6813ea0 100644 --- a/arch/loongarch/mm/fault.c +++ b/arch/loongarch/mm/fault.c @@ -20,12 +20,12 @@ #include #include #include -#include #include #include #include #include +#include #include #include diff --git a/arch/loongarch/mm/hugetlbpage.c b/arch/loongarch/mm/hugetlbpage.c index ba138117b124..1e76fcb83093 100644 --- a/arch/loongarch/mm/hugetlbpage.c +++ b/arch/loongarch/mm/hugetlbpage.c @@ -50,18 +50,6 @@ pte_t *huge_pte_offset(struct mm_struct *mm, unsigned long addr, return (pte_t *) pmd; } -/* - * This function checks for proper alignment of input addr and len parameters. - */ -int is_aligned_hugepage_range(unsigned long addr, unsigned long len) -{ - if (len & ~HPAGE_MASK) - return -EINVAL; - if (addr & ~HPAGE_MASK) - return -EINVAL; - return 0; -} - int pmd_huge(pmd_t pmd) { return (pmd_val(pmd) & _PAGE_HUGE) != 0; diff --git a/arch/loongarch/mm/init.c b/arch/loongarch/mm/init.c index f3fe8c06ba4d..4dd53427f657 100644 --- a/arch/loongarch/mm/init.c +++ b/arch/loongarch/mm/init.c @@ -43,11 +43,11 @@ void copy_user_highpage(struct page *to, struct page *from, { void *vfrom, *vto; - vto = kmap_atomic(to); - vfrom = kmap_atomic(from); + vfrom = kmap_local_page(from); + vto = kmap_local_page(to); copy_page(vto, vfrom); - kunmap_atomic(vfrom); - kunmap_atomic(vto); + kunmap_local(vfrom); + kunmap_local(vto); /* Make sure this page is cleared on other CPU's too before using it */ smp_wmb(); } @@ -240,6 +240,7 @@ pgd_t swapper_pg_dir[_PTRS_PER_PGD] __section(".bss..swapper_pg_dir"); pgd_t invalid_pg_dir[_PTRS_PER_PGD] __page_aligned_bss; #ifndef __PAGETABLE_PUD_FOLDED pud_t invalid_pud_table[PTRS_PER_PUD] __page_aligned_bss; +EXPORT_SYMBOL(invalid_pud_table); #endif #ifndef __PAGETABLE_PMD_FOLDED pmd_t invalid_pmd_table[PTRS_PER_PMD] __page_aligned_bss; diff --git a/arch/loongarch/mm/ioremap.c b/arch/loongarch/mm/ioremap.c index 73b0980ab6f5..70ca73019811 100644 --- a/arch/loongarch/mm/ioremap.c +++ b/arch/loongarch/mm/ioremap.c @@ -4,6 +4,7 @@ */ #include +#include void __init __iomem *early_ioremap(u64 phys_addr, unsigned long size) { diff --git a/arch/loongarch/mm/kasan_init.c b/arch/loongarch/mm/kasan_init.c index da68bc1a4643..cc3e81fe0186 100644 --- a/arch/loongarch/mm/kasan_init.c +++ b/arch/loongarch/mm/kasan_init.c @@ -35,6 +35,57 @@ static pgd_t kasan_pg_dir[PTRS_PER_PGD] __initdata __aligned(PAGE_SIZE); bool kasan_early_stage = true; +void *kasan_mem_to_shadow(const void *addr) +{ + if (!kasan_arch_is_ready()) { + return (void *)(kasan_early_shadow_page); + } else { + unsigned long maddr = (unsigned long)addr; + unsigned long xrange = (maddr >> XRANGE_SHIFT) & 0xffff; + unsigned long offset = 0; + + maddr &= XRANGE_SHADOW_MASK; + switch (xrange) { + case XKPRANGE_CC_SEG: + offset = XKPRANGE_CC_SHADOW_OFFSET; + break; + case XKPRANGE_UC_SEG: + offset = XKPRANGE_UC_SHADOW_OFFSET; + break; + case XKVRANGE_VC_SEG: + offset = XKVRANGE_VC_SHADOW_OFFSET; + break; + default: + WARN_ON(1); + return NULL; + } + + return (void *)((maddr >> KASAN_SHADOW_SCALE_SHIFT) + offset); + } +} + +const void *kasan_shadow_to_mem(const void *shadow_addr) +{ + unsigned long addr = (unsigned long)shadow_addr; + + if (unlikely(addr > KASAN_SHADOW_END) || + unlikely(addr < KASAN_SHADOW_START)) { + WARN_ON(1); + return NULL; + } + + if (addr >= XKVRANGE_VC_SHADOW_OFFSET) + return (void *)(((addr - XKVRANGE_VC_SHADOW_OFFSET) << KASAN_SHADOW_SCALE_SHIFT) + XKVRANGE_VC_START); + else if (addr >= XKPRANGE_UC_SHADOW_OFFSET) + return (void *)(((addr - XKPRANGE_UC_SHADOW_OFFSET) << KASAN_SHADOW_SCALE_SHIFT) + XKPRANGE_UC_START); + else if (addr >= XKPRANGE_CC_SHADOW_OFFSET) + return (void *)(((addr - XKPRANGE_CC_SHADOW_OFFSET) << KASAN_SHADOW_SCALE_SHIFT) + XKPRANGE_CC_START); + else { + WARN_ON(1); + return NULL; + } +} + /* * Alloc memory for shadow memory page table. */ diff --git a/arch/loongarch/mm/tlb.c b/arch/loongarch/mm/tlb.c index eb8572e201ea..2c0a411f23aa 100644 --- a/arch/loongarch/mm/tlb.c +++ b/arch/loongarch/mm/tlb.c @@ -261,7 +261,7 @@ unsigned long pcpu_handlers[NR_CPUS]; #endif extern long exception_handlers[VECSIZE * 128 / sizeof(long)]; -void setup_tlb_handler(int cpu) +static void setup_tlb_handler(int cpu) { setup_ptwalker(); local_flush_tlb_all(); diff --git a/arch/loongarch/mm/tlbex.S b/arch/loongarch/mm/tlbex.S index ca17dd3a1915..d5d682f3d29f 100644 --- a/arch/loongarch/mm/tlbex.S +++ b/arch/loongarch/mm/tlbex.S @@ -17,7 +17,7 @@ #define PTRS_PER_PTE_BITS (PAGE_SHIFT - 3) .macro tlb_do_page_fault, write - SYM_FUNC_START(tlb_do_page_fault_\write) + SYM_CODE_START(tlb_do_page_fault_\write) SAVE_ALL csrrd a2, LOONGARCH_CSR_BADV move a0, sp @@ -25,13 +25,13 @@ li.w a1, \write bl do_page_fault RESTORE_ALL_AND_RET - SYM_FUNC_END(tlb_do_page_fault_\write) + SYM_CODE_END(tlb_do_page_fault_\write) .endm tlb_do_page_fault 0 tlb_do_page_fault 1 -SYM_FUNC_START(handle_tlb_protect) +SYM_CODE_START(handle_tlb_protect) BACKUP_T0T1 SAVE_ALL move a0, sp @@ -41,9 +41,9 @@ SYM_FUNC_START(handle_tlb_protect) la_abs t0, do_page_fault jirl ra, t0, 0 RESTORE_ALL_AND_RET -SYM_FUNC_END(handle_tlb_protect) +SYM_CODE_END(handle_tlb_protect) -SYM_FUNC_START(handle_tlb_load) +SYM_CODE_START(handle_tlb_load) csrwr t0, EXCEPTION_KS0 csrwr t1, EXCEPTION_KS1 csrwr ra, EXCEPTION_KS2 @@ -187,16 +187,16 @@ nopage_tlb_load: csrrd ra, EXCEPTION_KS2 la_abs t0, tlb_do_page_fault_0 jr t0 -SYM_FUNC_END(handle_tlb_load) +SYM_CODE_END(handle_tlb_load) -SYM_FUNC_START(handle_tlb_load_ptw) +SYM_CODE_START(handle_tlb_load_ptw) csrwr t0, LOONGARCH_CSR_KS0 csrwr t1, LOONGARCH_CSR_KS1 la_abs t0, tlb_do_page_fault_0 jr t0 -SYM_FUNC_END(handle_tlb_load_ptw) +SYM_CODE_END(handle_tlb_load_ptw) -SYM_FUNC_START(handle_tlb_store) +SYM_CODE_START(handle_tlb_store) csrwr t0, EXCEPTION_KS0 csrwr t1, EXCEPTION_KS1 csrwr ra, EXCEPTION_KS2 @@ -343,16 +343,16 @@ nopage_tlb_store: csrrd ra, EXCEPTION_KS2 la_abs t0, tlb_do_page_fault_1 jr t0 -SYM_FUNC_END(handle_tlb_store) +SYM_CODE_END(handle_tlb_store) -SYM_FUNC_START(handle_tlb_store_ptw) +SYM_CODE_START(handle_tlb_store_ptw) csrwr t0, LOONGARCH_CSR_KS0 csrwr t1, LOONGARCH_CSR_KS1 la_abs t0, tlb_do_page_fault_1 jr t0 -SYM_FUNC_END(handle_tlb_store_ptw) +SYM_CODE_END(handle_tlb_store_ptw) -SYM_FUNC_START(handle_tlb_modify) +SYM_CODE_START(handle_tlb_modify) csrwr t0, EXCEPTION_KS0 csrwr t1, EXCEPTION_KS1 csrwr ra, EXCEPTION_KS2 @@ -497,16 +497,16 @@ nopage_tlb_modify: csrrd ra, EXCEPTION_KS2 la_abs t0, tlb_do_page_fault_1 jr t0 -SYM_FUNC_END(handle_tlb_modify) +SYM_CODE_END(handle_tlb_modify) -SYM_FUNC_START(handle_tlb_modify_ptw) +SYM_CODE_START(handle_tlb_modify_ptw) csrwr t0, LOONGARCH_CSR_KS0 csrwr t1, LOONGARCH_CSR_KS1 la_abs t0, tlb_do_page_fault_1 jr t0 -SYM_FUNC_END(handle_tlb_modify_ptw) +SYM_CODE_END(handle_tlb_modify_ptw) -SYM_FUNC_START(handle_tlb_refill) +SYM_CODE_START(handle_tlb_refill) csrwr t0, LOONGARCH_CSR_TLBRSAVE csrrd t0, LOONGARCH_CSR_PGD lddir t0, t0, 3 @@ -521,4 +521,4 @@ SYM_FUNC_START(handle_tlb_refill) tlbfill csrrd t0, LOONGARCH_CSR_TLBRSAVE ertn -SYM_FUNC_END(handle_tlb_refill) +SYM_CODE_END(handle_tlb_refill) diff --git a/arch/mips/alchemy/devboards/db1000.c b/arch/mips/alchemy/devboards/db1000.c index 012da042d0a4..7b9f91db227f 100644 --- a/arch/mips/alchemy/devboards/db1000.c +++ b/arch/mips/alchemy/devboards/db1000.c @@ -164,6 +164,7 @@ static struct platform_device db1x00_audio_dev = { /******************************************************************************/ +#ifdef CONFIG_MMC_AU1X static irqreturn_t db1100_mmc_cd(int irq, void *ptr) { mmc_detect_change(ptr, msecs_to_jiffies(500)); @@ -369,6 +370,7 @@ static struct platform_device db1100_mmc1_dev = { .num_resources = ARRAY_SIZE(au1100_mmc1_res), .resource = au1100_mmc1_res, }; +#endif /* CONFIG_MMC_AU1X */ /******************************************************************************/ @@ -440,8 +442,10 @@ static struct platform_device *db1x00_devs[] = { static struct platform_device *db1100_devs[] = { &au1100_lcd_device, +#ifdef CONFIG_MMC_AU1X &db1100_mmc0_dev, &db1100_mmc1_dev, +#endif }; int __init db1000_dev_setup(void) diff --git a/arch/mips/alchemy/devboards/db1200.c b/arch/mips/alchemy/devboards/db1200.c index 76080c71a2a7..f521874ebb07 100644 --- a/arch/mips/alchemy/devboards/db1200.c +++ b/arch/mips/alchemy/devboards/db1200.c @@ -326,6 +326,7 @@ static struct platform_device db1200_ide_dev = { /**********************************************************************/ +#ifdef CONFIG_MMC_AU1X /* SD carddetects: they're supposed to be edge-triggered, but ack * doesn't seem to work (CPLD Rev 2). Instead, the screaming one * is disabled and its counterpart enabled. The 200ms timeout is @@ -584,6 +585,7 @@ static struct platform_device pb1200_mmc1_dev = { .num_resources = ARRAY_SIZE(au1200_mmc1_res), .resource = au1200_mmc1_res, }; +#endif /* CONFIG_MMC_AU1X */ /**********************************************************************/ @@ -751,7 +753,9 @@ static struct platform_device db1200_audiodma_dev = { static struct platform_device *db1200_devs[] __initdata = { NULL, /* PSC0, selected by S6.8 */ &db1200_ide_dev, +#ifdef CONFIG_MMC_AU1X &db1200_mmc0_dev, +#endif &au1200_lcd_dev, &db1200_eth_dev, &db1200_nand_dev, @@ -762,7 +766,9 @@ static struct platform_device *db1200_devs[] __initdata = { }; static struct platform_device *pb1200_devs[] __initdata = { +#ifdef CONFIG_MMC_AU1X &pb1200_mmc1_dev, +#endif }; /* Some peripheral base addresses differ on the PB1200 */ diff --git a/arch/mips/alchemy/devboards/db1300.c b/arch/mips/alchemy/devboards/db1300.c index ff61901329c6..d377e043b49f 100644 --- a/arch/mips/alchemy/devboards/db1300.c +++ b/arch/mips/alchemy/devboards/db1300.c @@ -450,6 +450,7 @@ static struct platform_device db1300_ide_dev = { /**********************************************************************/ +#ifdef CONFIG_MMC_AU1X static irqreturn_t db1300_mmc_cd(int irq, void *ptr) { disable_irq_nosync(irq); @@ -632,6 +633,7 @@ static struct platform_device db1300_sd0_dev = { .resource = au1300_sd0_res, .num_resources = ARRAY_SIZE(au1300_sd0_res), }; +#endif /* CONFIG_MMC_AU1X */ /**********************************************************************/ @@ -767,8 +769,10 @@ static struct platform_device *db1300_dev[] __initdata = { &db1300_5waysw_dev, &db1300_nand_dev, &db1300_ide_dev, +#ifdef CONFIG_MMC_AU1X &db1300_sd0_dev, &db1300_sd1_dev, +#endif &db1300_lcd_dev, &db1300_ac97_dev, &db1300_i2s_dev, diff --git a/arch/mips/kvm/mmu.c b/arch/mips/kvm/mmu.c index 7b2ac1319d70..467ee6b95ae1 100644 --- a/arch/mips/kvm/mmu.c +++ b/arch/mips/kvm/mmu.c @@ -592,7 +592,7 @@ static int kvm_mips_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, gfn_t gfn = gpa >> PAGE_SHIFT; int srcu_idx, err; kvm_pfn_t pfn; - pte_t *ptep, entry, old_pte; + pte_t *ptep, entry; bool writeable; unsigned long prot_bits; unsigned long mmu_seq; @@ -664,7 +664,6 @@ retry: entry = pfn_pte(pfn, __pgprot(prot_bits)); /* Write the PTE */ - old_pte = *ptep; set_pte(ptep, entry); err = 0; diff --git a/arch/parisc/include/asm/cache.h b/arch/parisc/include/asm/cache.h index e23d06b51a20..2a60d7a72f1f 100644 --- a/arch/parisc/include/asm/cache.h +++ b/arch/parisc/include/asm/cache.h @@ -37,6 +37,7 @@ extern int split_tlb; extern int dcache_stride; extern int icache_stride; extern struct pdc_cache_info cache_info; +extern struct pdc_btlb_info btlb_info; void parisc_setup_cache_timing(void); #define pdtlb(sr, addr) asm volatile("pdtlb 0(%%sr%0,%1)" \ diff --git a/arch/parisc/include/asm/hugetlb.h b/arch/parisc/include/asm/hugetlb.h index f7f078c2872c..72daacc472a0 100644 --- a/arch/parisc/include/asm/hugetlb.h +++ b/arch/parisc/include/asm/hugetlb.h @@ -6,7 +6,7 @@ #define __HAVE_ARCH_HUGE_SET_HUGE_PTE_AT void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, pte_t pte); + pte_t *ptep, pte_t pte, unsigned long sz); #define __HAVE_ARCH_HUGE_PTEP_GET_AND_CLEAR pte_t huge_ptep_get_and_clear(struct mm_struct *mm, unsigned long addr, diff --git a/arch/parisc/include/asm/ldcw.h b/arch/parisc/include/asm/ldcw.h index 6d28b5514699..ee9e071859b2 100644 --- a/arch/parisc/include/asm/ldcw.h +++ b/arch/parisc/include/asm/ldcw.h @@ -2,39 +2,42 @@ #ifndef __PARISC_LDCW_H #define __PARISC_LDCW_H -#ifndef CONFIG_PA20 /* Because kmalloc only guarantees 8-byte alignment for kmalloc'd data, and GCC only guarantees 8-byte alignment for stack locals, we can't be assured of 16-byte alignment for atomic lock data even if we specify "__attribute ((aligned(16)))" in the type declaration. So, we use a struct containing an array of four ints for the atomic lock type and dynamically select the 16-byte aligned int from the array - for the semaphore. */ + for the semaphore. */ + +/* From: "Jim Hull" + I've attached a summary of the change, but basically, for PA 2.0, as + long as the ",CO" (coherent operation) completer is implemented, then the + 16-byte alignment requirement for ldcw and ldcd is relaxed, and instead + they only require "natural" alignment (4-byte for ldcw, 8-byte for + ldcd). + + Although the cache control hint is accepted by all PA 2.0 processors, + it is only implemented on PA8800/PA8900 CPUs. Prior PA8X00 CPUs still + require 16-byte alignment. If the address is unaligned, the operation + of the instruction is undefined. The ldcw instruction does not generate + unaligned data reference traps so misaligned accesses are not detected. + This hid the problem for years. So, restore the 16-byte alignment dropped + by Kyle McMartin in "Remove __ldcw_align for PA-RISC 2.0 processors". */ #define __PA_LDCW_ALIGNMENT 16 -#define __PA_LDCW_ALIGN_ORDER 4 #define __ldcw_align(a) ({ \ unsigned long __ret = (unsigned long) &(a)->lock[0]; \ __ret = (__ret + __PA_LDCW_ALIGNMENT - 1) \ & ~(__PA_LDCW_ALIGNMENT - 1); \ (volatile unsigned int *) __ret; \ }) -#define __LDCW "ldcw" -#else /*CONFIG_PA20*/ -/* From: "Jim Hull" - I've attached a summary of the change, but basically, for PA 2.0, as - long as the ",CO" (coherent operation) completer is specified, then the - 16-byte alignment requirement for ldcw and ldcd is relaxed, and instead - they only require "natural" alignment (4-byte for ldcw, 8-byte for - ldcd). */ - -#define __PA_LDCW_ALIGNMENT 4 -#define __PA_LDCW_ALIGN_ORDER 2 -#define __ldcw_align(a) (&(a)->slock) +#ifdef CONFIG_PA20 #define __LDCW "ldcw,co" - -#endif /*!CONFIG_PA20*/ +#else +#define __LDCW "ldcw" +#endif /* LDCW, the only atomic read-write operation PA-RISC has. *sigh*. We don't explicitly expose that "*a" may be written as reload diff --git a/arch/parisc/include/asm/mckinley.h b/arch/parisc/include/asm/mckinley.h deleted file mode 100644 index 1314390b9034..000000000000 --- a/arch/parisc/include/asm/mckinley.h +++ /dev/null @@ -1,8 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 */ -#ifndef ASM_PARISC_MCKINLEY_H -#define ASM_PARISC_MCKINLEY_H - -/* declared in arch/parisc/kernel/setup.c */ -extern struct proc_dir_entry * proc_mckinley_root; - -#endif /*ASM_PARISC_MCKINLEY_H*/ diff --git a/arch/parisc/include/asm/pdc.h b/arch/parisc/include/asm/pdc.h index 269b9a159f01..5d2d9737e579 100644 --- a/arch/parisc/include/asm/pdc.h +++ b/arch/parisc/include/asm/pdc.h @@ -44,10 +44,11 @@ int pdc_model_capabilities(unsigned long *capabilities); int pdc_model_platform_info(char *orig_prod_num, char *current_prod_num, char *serial_no); int pdc_cache_info(struct pdc_cache_info *cache); int pdc_spaceid_bits(unsigned long *space_bits); -#ifndef CONFIG_PA20 int pdc_btlb_info(struct pdc_btlb_info *btlb); +int pdc_btlb_insert(unsigned long long vpage, unsigned long physpage, unsigned long len, + unsigned long entry_info, unsigned long slot); +int pdc_btlb_purge_all(void); int pdc_mem_map_hpa(struct pdc_memory_map *r_addr, struct pdc_module_path *mod_path); -#endif /* !CONFIG_PA20 */ int pdc_pim_toc11(struct pdc_toc_pim_11 *ret); int pdc_pim_toc20(struct pdc_toc_pim_20 *ret); int pdc_lan_station_id(char *lan_addr, unsigned long net_hpa); diff --git a/arch/parisc/include/asm/processor.h b/arch/parisc/include/asm/processor.h index d77c43d32974..ff6cbdb6903b 100644 --- a/arch/parisc/include/asm/processor.h +++ b/arch/parisc/include/asm/processor.h @@ -310,6 +310,7 @@ extern void do_syscall_trace_exit(struct pt_regs *); struct seq_file; extern void early_trap_init(void); extern void collect_boot_cpu_data(void); +extern void btlb_init_per_cpu(void); extern int show_cpuinfo (struct seq_file *m, void *v); /* driver code in driver/parisc */ diff --git a/arch/parisc/include/asm/ropes.h b/arch/parisc/include/asm/ropes.h index fd96706c7234..e2d2d7e9bfde 100644 --- a/arch/parisc/include/asm/ropes.h +++ b/arch/parisc/include/asm/ropes.h @@ -29,7 +29,7 @@ struct ioc { void __iomem *ioc_hpa; /* I/O MMU base address */ char *res_map; /* resource map, bit == pdir entry */ - u64 *pdir_base; /* physical base address */ + __le64 *pdir_base; /* physical base address */ unsigned long ibase; /* pdir IOV Space base - shared w/lba_pci */ unsigned long imask; /* pdir IOV Space mask - shared w/lba_pci */ #ifdef ZX1_SUPPORT @@ -86,6 +86,9 @@ struct sba_device { struct ioc ioc[MAX_IOC]; }; +/* list of SBA's in system, see drivers/parisc/sba_iommu.c */ +extern struct sba_device *sba_list; + #define ASTRO_RUNWAY_PORT 0x582 #define IKE_MERCED_PORT 0x803 #define REO_MERCED_PORT 0x804 @@ -110,7 +113,7 @@ static inline int IS_PLUTO(struct parisc_device *d) { #define SBA_PDIR_VALID_BIT 0x8000000000000000ULL -#define SBA_AGPGART_COOKIE 0x0000badbadc0ffeeULL +#define SBA_AGPGART_COOKIE (__force __le64) 0x0000badbadc0ffeeULL #define SBA_FUNC_ID 0x0000 /* function id */ #define SBA_FCLASS 0x0008 /* function class, bist, header, rev... */ diff --git a/arch/parisc/include/asm/shmparam.h b/arch/parisc/include/asm/shmparam.h index 74f74e4d35b7..5a95b0f62b87 100644 --- a/arch/parisc/include/asm/shmparam.h +++ b/arch/parisc/include/asm/shmparam.h @@ -2,6 +2,21 @@ #ifndef _ASMPARISC_SHMPARAM_H #define _ASMPARISC_SHMPARAM_H +/* + * PA-RISC uses virtually indexed & physically tagged (VIPT) caches + * which has strict requirements when two pages to the same physical + * address are accessed through different mappings. Read the section + * "Address Aliasing" in the arch docs for more detail: + * PA-RISC 1.1 (page 3-6): + * https://parisc.wiki.kernel.org/images-parisc/6/68/Pa11_acd.pdf + * PA-RISC 2.0 (page F-5): + * https://parisc.wiki.kernel.org/images-parisc/7/73/Parisc2.0.pdf + * + * For Linux we allow kernel and userspace to map pages on page size + * granularity (SHMLBA) but have to ensure that, if two pages are + * mapped to the same physical address, the virtual and physical + * addresses modulo SHM_COLOUR are identical. + */ #define SHMLBA PAGE_SIZE /* attach addr a multiple of this */ #define SHM_COLOUR 0x00400000 /* shared mappings colouring */ diff --git a/arch/parisc/include/asm/spinlock_types.h b/arch/parisc/include/asm/spinlock_types.h index efd06a897c6a..7b986b09dba8 100644 --- a/arch/parisc/include/asm/spinlock_types.h +++ b/arch/parisc/include/asm/spinlock_types.h @@ -9,15 +9,10 @@ #ifndef __ASSEMBLY__ typedef struct { -#ifdef CONFIG_PA20 - volatile unsigned int slock; -# define __ARCH_SPIN_LOCK_UNLOCKED { __ARCH_SPIN_LOCK_UNLOCKED_VAL } -#else volatile unsigned int lock[4]; # define __ARCH_SPIN_LOCK_UNLOCKED \ { { __ARCH_SPIN_LOCK_UNLOCKED_VAL, __ARCH_SPIN_LOCK_UNLOCKED_VAL, \ __ARCH_SPIN_LOCK_UNLOCKED_VAL, __ARCH_SPIN_LOCK_UNLOCKED_VAL } } -#endif } arch_spinlock_t; diff --git a/arch/parisc/kernel/asm-offsets.c b/arch/parisc/kernel/asm-offsets.c index 94652e13c260..757816a7bd4b 100644 --- a/arch/parisc/kernel/asm-offsets.c +++ b/arch/parisc/kernel/asm-offsets.c @@ -275,6 +275,8 @@ int main(void) * and kernel data on physical huge pages */ #ifdef CONFIG_HUGETLB_PAGE DEFINE(HUGEPAGE_SIZE, 1UL << REAL_HPAGE_SHIFT); +#elif !defined(CONFIG_64BIT) + DEFINE(HUGEPAGE_SIZE, 4*1024*1024); #else DEFINE(HUGEPAGE_SIZE, PAGE_SIZE); #endif diff --git a/arch/parisc/kernel/cache.c b/arch/parisc/kernel/cache.c index 442109a48940..268d90a9325b 100644 --- a/arch/parisc/kernel/cache.c +++ b/arch/parisc/kernel/cache.c @@ -58,7 +58,7 @@ int pa_serialize_tlb_flushes __ro_after_init; struct pdc_cache_info cache_info __ro_after_init; #ifndef CONFIG_PA20 -static struct pdc_btlb_info btlb_info __ro_after_init; +struct pdc_btlb_info btlb_info __ro_after_init; #endif DEFINE_STATIC_KEY_TRUE(parisc_has_cache); @@ -264,12 +264,6 @@ parisc_cache_init(void) icache_stride = CAFL_STRIDE(cache_info.ic_conf); #undef CAFL_STRIDE -#ifndef CONFIG_PA20 - if (pdc_btlb_info(&btlb_info) < 0) { - memset(&btlb_info, 0, sizeof btlb_info); - } -#endif - if ((boot_cpu_data.pdc.capabilities & PDC_MODEL_NVA_MASK) == PDC_MODEL_NVA_UNSUPPORTED) { printk(KERN_WARNING "parisc_cache_init: Only equivalent aliasing supported!\n"); diff --git a/arch/parisc/kernel/drivers.c b/arch/parisc/kernel/drivers.c index 8f4b77648491..ed8b75948061 100644 --- a/arch/parisc/kernel/drivers.c +++ b/arch/parisc/kernel/drivers.c @@ -925,9 +925,9 @@ static __init void qemu_header(void) pr_info("#define PARISC_MODEL \"%s\"\n\n", boot_cpu_data.pdc.sys_model_name); + #define p ((unsigned long *)&boot_cpu_data.pdc.model) pr_info("#define PARISC_PDC_MODEL 0x%lx, 0x%lx, 0x%lx, " "0x%lx, 0x%lx, 0x%lx, 0x%lx, 0x%lx, 0x%lx\n\n", - #define p ((unsigned long *)&boot_cpu_data.pdc.model) p[0], p[1], p[2], p[3], p[4], p[5], p[6], p[7], p[8]); #undef p diff --git a/arch/parisc/kernel/firmware.c b/arch/parisc/kernel/firmware.c index 8f37e75f2fb9..81078abec521 100644 --- a/arch/parisc/kernel/firmware.c +++ b/arch/parisc/kernel/firmware.c @@ -687,7 +687,6 @@ int pdc_spaceid_bits(unsigned long *space_bits) return retval; } -#ifndef CONFIG_PA20 /** * pdc_btlb_info - Return block TLB information. * @btlb: The return buffer. @@ -696,18 +695,51 @@ int pdc_spaceid_bits(unsigned long *space_bits) */ int pdc_btlb_info(struct pdc_btlb_info *btlb) { - int retval; + int retval; unsigned long flags; - spin_lock_irqsave(&pdc_lock, flags); - retval = mem_pdc_call(PDC_BLOCK_TLB, PDC_BTLB_INFO, __pa(pdc_result), 0); - memcpy(btlb, pdc_result, sizeof(*btlb)); - spin_unlock_irqrestore(&pdc_lock, flags); + if (IS_ENABLED(CONFIG_PA20)) + return PDC_BAD_PROC; - if(retval < 0) { - btlb->max_size = 0; - } - return retval; + spin_lock_irqsave(&pdc_lock, flags); + retval = mem_pdc_call(PDC_BLOCK_TLB, PDC_BTLB_INFO, __pa(pdc_result), 0); + memcpy(btlb, pdc_result, sizeof(*btlb)); + spin_unlock_irqrestore(&pdc_lock, flags); + + if(retval < 0) { + btlb->max_size = 0; + } + return retval; +} + +int pdc_btlb_insert(unsigned long long vpage, unsigned long physpage, unsigned long len, + unsigned long entry_info, unsigned long slot) +{ + int retval; + unsigned long flags; + + if (IS_ENABLED(CONFIG_PA20)) + return PDC_BAD_PROC; + + spin_lock_irqsave(&pdc_lock, flags); + retval = mem_pdc_call(PDC_BLOCK_TLB, PDC_BTLB_INSERT, (unsigned long) (vpage >> 32), + (unsigned long) vpage, physpage, len, entry_info, slot); + spin_unlock_irqrestore(&pdc_lock, flags); + return retval; +} + +int pdc_btlb_purge_all(void) +{ + int retval; + unsigned long flags; + + if (IS_ENABLED(CONFIG_PA20)) + return PDC_BAD_PROC; + + spin_lock_irqsave(&pdc_lock, flags); + retval = mem_pdc_call(PDC_BLOCK_TLB, PDC_BTLB_PURGE_ALL); + spin_unlock_irqrestore(&pdc_lock, flags); + return retval; } /** @@ -728,6 +760,9 @@ int pdc_mem_map_hpa(struct pdc_memory_map *address, int retval; unsigned long flags; + if (IS_ENABLED(CONFIG_PA20)) + return PDC_BAD_PROC; + spin_lock_irqsave(&pdc_lock, flags); memcpy(pdc_result2, mod_path, sizeof(*mod_path)); retval = mem_pdc_call(PDC_MEM_MAP, PDC_MEM_MAP_HPA, __pa(pdc_result), @@ -737,7 +772,6 @@ int pdc_mem_map_hpa(struct pdc_memory_map *address, return retval; } -#endif /* !CONFIG_PA20 */ /** * pdc_lan_station_id - Get the LAN address. diff --git a/arch/parisc/kernel/head.S b/arch/parisc/kernel/head.S index fd15fd4bbb61..a171bf3c6b31 100644 --- a/arch/parisc/kernel/head.S +++ b/arch/parisc/kernel/head.S @@ -180,10 +180,10 @@ $pgt_fill_loop: std %dp,0x18(%r10) #endif -#ifdef CONFIG_64BIT - /* Get PDCE_PROC for monarch CPU. */ #define MEM_PDC_LO 0x388 #define MEM_PDC_HI 0x35C +#ifdef CONFIG_64BIT + /* Get PDCE_PROC for monarch CPU. */ ldw MEM_PDC_LO(%r0),%r3 ldw MEM_PDC_HI(%r0),%r10 depd %r10, 31, 32, %r3 /* move to upper word */ @@ -269,7 +269,17 @@ stext_pdc_ret: tovirt_r1 %r6 mtctl %r6,%cr30 /* restore task thread info */ #endif - + +#ifndef CONFIG_64BIT + /* clear all BTLBs */ + ldi PDC_BLOCK_TLB,%arg0 + load32 PA(stext_pdc_btlb_ret), %rp + ldw MEM_PDC_LO(%r0),%r3 + bv (%r3) + ldi PDC_BTLB_PURGE_ALL,%arg1 +stext_pdc_btlb_ret: +#endif + /* PARANOID: clear user scratch/user space SR's */ mtsp %r0,%sr0 mtsp %r0,%sr1 diff --git a/arch/parisc/kernel/irq.c b/arch/parisc/kernel/irq.c index 12c4d4104ade..2f81bfd4f15e 100644 --- a/arch/parisc/kernel/irq.c +++ b/arch/parisc/kernel/irq.c @@ -365,7 +365,7 @@ union irq_stack_union { volatile unsigned int lock[1]; }; -DEFINE_PER_CPU(union irq_stack_union, irq_stack_union) = { +static DEFINE_PER_CPU(union irq_stack_union, irq_stack_union) = { .slock = { 1,1,1,1 }, }; #endif diff --git a/arch/parisc/kernel/processor.c b/arch/parisc/kernel/processor.c index a0e2d37c5b3b..1fc89fa2c2d2 100644 --- a/arch/parisc/kernel/processor.c +++ b/arch/parisc/kernel/processor.c @@ -368,6 +368,8 @@ int init_per_cpu(int cpunum) /* FUTURE: Enable Performance Monitor : ccr bit 0x20 */ init_percpu_prof(cpunum); + btlb_init_per_cpu(); + return ret; } diff --git a/arch/parisc/kernel/smp.c b/arch/parisc/kernel/smp.c index 4098f9a0964b..2019c1f04bd0 100644 --- a/arch/parisc/kernel/smp.c +++ b/arch/parisc/kernel/smp.c @@ -440,7 +440,9 @@ int __cpu_up(unsigned int cpu, struct task_struct *tidle) if (cpu_online(cpu)) return 0; - if (num_online_cpus() < setup_max_cpus && smp_boot_one_cpu(cpu, tidle)) + if (num_online_cpus() < nr_cpu_ids && + num_online_cpus() < setup_max_cpus && + smp_boot_one_cpu(cpu, tidle)) return -EIO; return cpu_online(cpu) ? 0 : -EIO; diff --git a/arch/parisc/kernel/vmlinux.lds.S b/arch/parisc/kernel/vmlinux.lds.S index 1aaa2ca09800..58694d1989c2 100644 --- a/arch/parisc/kernel/vmlinux.lds.S +++ b/arch/parisc/kernel/vmlinux.lds.S @@ -154,6 +154,7 @@ SECTIONS } /* End of data section */ + . = ALIGN(PAGE_SIZE); _edata = .; /* BSS */ diff --git a/arch/parisc/mm/hugetlbpage.c b/arch/parisc/mm/hugetlbpage.c index a8a1a7c1e16e..a9f7e21f6656 100644 --- a/arch/parisc/mm/hugetlbpage.c +++ b/arch/parisc/mm/hugetlbpage.c @@ -140,7 +140,7 @@ static void __set_huge_pte_at(struct mm_struct *mm, unsigned long addr, } void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, pte_t entry) + pte_t *ptep, pte_t entry, unsigned long sz) { __set_huge_pte_at(mm, addr, ptep, entry); } diff --git a/arch/parisc/mm/init.c b/arch/parisc/mm/init.c index a088c243edea..a2a3e89f2d9a 100644 --- a/arch/parisc/mm/init.c +++ b/arch/parisc/mm/init.c @@ -32,6 +32,7 @@ #include #include #include +#include extern int data_start; extern void parisc_kernel_start(void); /* Kernel entry point in head.S */ @@ -720,6 +721,77 @@ void __init paging_init(void) parisc_bootmem_free(); } +static void alloc_btlb(unsigned long start, unsigned long end, int *slot, + unsigned long entry_info) +{ + const int slot_max = btlb_info.fixed_range_info.num_comb; + int min_num_pages = btlb_info.min_size; + unsigned long size; + + /* map at minimum 4 pages */ + if (min_num_pages < 4) + min_num_pages = 4; + + size = HUGEPAGE_SIZE; + while (start < end && *slot < slot_max && size >= PAGE_SIZE) { + /* starting address must have same alignment as size! */ + /* if correctly aligned and fits in double size, increase */ + if (((start & (2 * size - 1)) == 0) && + (end - start) >= (2 * size)) { + size <<= 1; + continue; + } + /* if current size alignment is too big, try smaller size */ + if ((start & (size - 1)) != 0) { + size >>= 1; + continue; + } + if ((end - start) >= size) { + if ((size >> PAGE_SHIFT) >= min_num_pages) + pdc_btlb_insert(start >> PAGE_SHIFT, __pa(start) >> PAGE_SHIFT, + size >> PAGE_SHIFT, entry_info, *slot); + (*slot)++; + start += size; + continue; + } + size /= 2; + continue; + } +} + +void btlb_init_per_cpu(void) +{ + unsigned long s, t, e; + int slot; + + /* BTLBs are not available on 64-bit CPUs */ + if (IS_ENABLED(CONFIG_PA20)) + return; + else if (pdc_btlb_info(&btlb_info) < 0) { + memset(&btlb_info, 0, sizeof btlb_info); + } + + /* insert BLTLBs for code and data segments */ + s = (uintptr_t) dereference_function_descriptor(&_stext); + e = (uintptr_t) dereference_function_descriptor(&_etext); + t = (uintptr_t) dereference_function_descriptor(&_sdata); + BUG_ON(t != e); + + /* code segments */ + slot = 0; + alloc_btlb(s, e, &slot, 0x13800000); + + /* sanity check */ + t = (uintptr_t) dereference_function_descriptor(&_edata); + e = (uintptr_t) dereference_function_descriptor(&__bss_start); + BUG_ON(t != e); + + /* data segments */ + s = (uintptr_t) dereference_function_descriptor(&_sdata); + e = (uintptr_t) dereference_function_descriptor(&__bss_stop); + alloc_btlb(s, e, &slot, 0x11800000); +} + #ifdef CONFIG_PA20 /* diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig index 54b9387c3691..d5d5388973ac 100644 --- a/arch/powerpc/Kconfig +++ b/arch/powerpc/Kconfig @@ -255,7 +255,7 @@ config PPC select HAVE_KPROBES select HAVE_KPROBES_ON_FTRACE select HAVE_KRETPROBES - select HAVE_LD_DEAD_CODE_DATA_ELIMINATION if HAVE_OBJTOOL_MCOUNT + select HAVE_LD_DEAD_CODE_DATA_ELIMINATION if HAVE_OBJTOOL_MCOUNT && (!ARCH_USING_PATCHABLE_FUNCTION_ENTRY || (!CC_IS_GCC || GCC_VERSION >= 110100)) select HAVE_LIVEPATCH if HAVE_DYNAMIC_FTRACE_WITH_REGS select HAVE_MOD_ARCH_SPECIFIC select HAVE_NMI if PERF_EVENTS || (PPC64 && PPC_BOOK3S) @@ -910,7 +910,7 @@ config ARCH_FORCE_MAX_ORDER default "6" if PPC32 && PPC_64K_PAGES range 4 10 if PPC32 && PPC_256K_PAGES default "4" if PPC32 && PPC_256K_PAGES - range 10 10 + range 10 12 default "10" help The kernel page allocator limits the size of maximal physically diff --git a/arch/powerpc/include/asm/nohash/32/hugetlb-8xx.h b/arch/powerpc/include/asm/nohash/32/hugetlb-8xx.h index de092b04ee1a..92df40c6cc6b 100644 --- a/arch/powerpc/include/asm/nohash/32/hugetlb-8xx.h +++ b/arch/powerpc/include/asm/nohash/32/hugetlb-8xx.h @@ -46,7 +46,8 @@ static inline int check_and_get_huge_psize(int shift) } #define __HAVE_ARCH_HUGE_SET_HUGE_PTE_AT -void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, pte_t *ptep, pte_t pte); +void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, pte_t *ptep, + pte_t pte, unsigned long sz); #define __HAVE_ARCH_HUGE_PTE_CLEAR static inline void huge_pte_clear(struct mm_struct *mm, unsigned long addr, diff --git a/arch/powerpc/include/asm/nohash/32/pte-8xx.h b/arch/powerpc/include/asm/nohash/32/pte-8xx.h index 21f681ee535a..e6fe1d5731f2 100644 --- a/arch/powerpc/include/asm/nohash/32/pte-8xx.h +++ b/arch/powerpc/include/asm/nohash/32/pte-8xx.h @@ -94,6 +94,13 @@ static inline pte_t pte_wrprotect(pte_t pte) #define pte_wrprotect pte_wrprotect +static inline int pte_read(pte_t pte) +{ + return (pte_val(pte) & _PAGE_RO) != _PAGE_NA; +} + +#define pte_read pte_read + static inline int pte_write(pte_t pte) { return !(pte_val(pte) & _PAGE_RO); diff --git a/arch/powerpc/include/asm/nohash/64/pgtable.h b/arch/powerpc/include/asm/nohash/64/pgtable.h index 5cd9acf58a7d..eb6891e34cbd 100644 --- a/arch/powerpc/include/asm/nohash/64/pgtable.h +++ b/arch/powerpc/include/asm/nohash/64/pgtable.h @@ -197,7 +197,7 @@ static inline int __ptep_test_and_clear_young(struct mm_struct *mm, { unsigned long old; - if (pte_young(*ptep)) + if (!pte_young(*ptep)) return 0; old = pte_update(mm, addr, ptep, _PAGE_ACCESSED, 0, 0); return (old & _PAGE_ACCESSED) != 0; diff --git a/arch/powerpc/include/asm/nohash/pgtable.h b/arch/powerpc/include/asm/nohash/pgtable.h index 56ea48276356..c721478c5934 100644 --- a/arch/powerpc/include/asm/nohash/pgtable.h +++ b/arch/powerpc/include/asm/nohash/pgtable.h @@ -25,7 +25,9 @@ static inline int pte_write(pte_t pte) return pte_val(pte) & _PAGE_RW; } #endif +#ifndef pte_read static inline int pte_read(pte_t pte) { return 1; } +#endif static inline int pte_dirty(pte_t pte) { return pte_val(pte) & _PAGE_DIRTY; } static inline int pte_special(pte_t pte) { return pte_val(pte) & _PAGE_SPECIAL; } static inline int pte_none(pte_t pte) { return (pte_val(pte) & ~_PTE_NONE_MASK) == 0; } diff --git a/arch/powerpc/kernel/entry_32.S b/arch/powerpc/kernel/entry_32.S index 9692acb0361f..7eda33a24bb4 100644 --- a/arch/powerpc/kernel/entry_32.S +++ b/arch/powerpc/kernel/entry_32.S @@ -137,8 +137,9 @@ ret_from_syscall: lis r4,icache_44x_need_flush@ha lwz r5,icache_44x_need_flush@l(r4) cmplwi cr0,r5,0 - bne- 2f + bne- .L44x_icache_flush #endif /* CONFIG_PPC_47x */ +.L44x_icache_flush_return: kuep_unlock lwz r4,_LINK(r1) lwz r5,_CCR(r1) @@ -172,10 +173,11 @@ syscall_exit_finish: b 1b #ifdef CONFIG_44x -2: li r7,0 +.L44x_icache_flush: + li r7,0 iccci r0,r0 stw r7,icache_44x_need_flush@l(r4) - b 1b + b .L44x_icache_flush_return #endif /* CONFIG_44x */ .globl ret_from_fork diff --git a/arch/powerpc/kernel/head_85xx.S b/arch/powerpc/kernel/head_85xx.S index 97e9ea0c7297..0f1641a31250 100644 --- a/arch/powerpc/kernel/head_85xx.S +++ b/arch/powerpc/kernel/head_85xx.S @@ -395,7 +395,7 @@ interrupt_base: #ifdef CONFIG_PPC_FPU FP_UNAVAILABLE_EXCEPTION #else - EXCEPTION(0x0800, FP_UNAVAIL, FloatingPointUnavailable, unknown_exception) + EXCEPTION(0x0800, FP_UNAVAIL, FloatingPointUnavailable, emulation_assist_interrupt) #endif /* System Call Interrupt */ diff --git a/arch/powerpc/kernel/hw_breakpoint.c b/arch/powerpc/kernel/hw_breakpoint.c index b8513dc3e53a..a1318ce18d0e 100644 --- a/arch/powerpc/kernel/hw_breakpoint.c +++ b/arch/powerpc/kernel/hw_breakpoint.c @@ -230,13 +230,15 @@ void thread_change_pc(struct task_struct *tsk, struct pt_regs *regs) struct arch_hw_breakpoint *info; int i; + preempt_disable(); + for (i = 0; i < nr_wp_slots(); i++) { struct perf_event *bp = __this_cpu_read(bp_per_reg[i]); if (unlikely(bp && counter_arch_bp(bp)->perf_single_step)) goto reset; } - return; + goto out; reset: regs_set_return_msr(regs, regs->msr & ~MSR_SE); @@ -245,6 +247,9 @@ reset: __set_breakpoint(i, info); info->perf_single_step = false; } + +out: + preempt_enable(); } static bool is_larx_stcx_instr(int type) @@ -363,6 +368,11 @@ static void handle_p10dd1_spurious_exception(struct perf_event **bp, } } +/* + * Handle a DABR or DAWR exception. + * + * Called in atomic context. + */ int hw_breakpoint_handler(struct die_args *args) { bool err = false; @@ -490,6 +500,8 @@ NOKPROBE_SYMBOL(hw_breakpoint_handler); /* * Handle single-step exceptions following a DABR hit. + * + * Called in atomic context. */ static int single_step_dabr_instruction(struct die_args *args) { @@ -541,6 +553,8 @@ NOKPROBE_SYMBOL(single_step_dabr_instruction); /* * Handle debug exception notifications. + * + * Called in atomic context. */ int hw_breakpoint_exceptions_notify( struct notifier_block *unused, unsigned long val, void *data) diff --git a/arch/powerpc/kernel/hw_breakpoint_constraints.c b/arch/powerpc/kernel/hw_breakpoint_constraints.c index a74623025f3a..9e51801c4915 100644 --- a/arch/powerpc/kernel/hw_breakpoint_constraints.c +++ b/arch/powerpc/kernel/hw_breakpoint_constraints.c @@ -131,8 +131,13 @@ void wp_get_instr_detail(struct pt_regs *regs, ppc_inst_t *instr, int *type, int *size, unsigned long *ea) { struct instruction_op op; + int err; - if (__get_user_instr(*instr, (void __user *)regs->nip)) + pagefault_disable(); + err = __get_user_instr(*instr, (void __user *)regs->nip); + pagefault_enable(); + + if (err) return; analyse_instr(&op, regs, *instr); diff --git a/arch/powerpc/kernel/stacktrace.c b/arch/powerpc/kernel/stacktrace.c index b15f15dcacb5..e6a958a5da27 100644 --- a/arch/powerpc/kernel/stacktrace.c +++ b/arch/powerpc/kernel/stacktrace.c @@ -73,29 +73,12 @@ int __no_sanitize_address arch_stack_walk_reliable(stack_trace_consume_fn consum bool firstframe; stack_end = stack_page + THREAD_SIZE; - if (!is_idle_task(task)) { - /* - * For user tasks, this is the SP value loaded on - * kernel entry, see "PACAKSAVE(r13)" in _switch() and - * system_call_common(). - * - * Likewise for non-swapper kernel threads, - * this also happens to be the top of the stack - * as setup by copy_thread(). - * - * Note that stack backlinks are not properly setup by - * copy_thread() and thus, a forked task() will have - * an unreliable stack trace until it's been - * _switch()'ed to for the first time. - */ - stack_end -= STACK_USER_INT_FRAME_SIZE; - } else { - /* - * idle tasks have a custom stack layout, - * c.f. cpu_idle_thread_init(). - */ + + // See copy_thread() for details. + if (task->flags & PF_KTHREAD) stack_end -= STACK_FRAME_MIN_SIZE; - } + else + stack_end -= STACK_USER_INT_FRAME_SIZE; if (task == current) sp = current_stack_frame(); diff --git a/arch/powerpc/kernel/traps.c b/arch/powerpc/kernel/traps.c index eeff136b83d9..64ff37721fd0 100644 --- a/arch/powerpc/kernel/traps.c +++ b/arch/powerpc/kernel/traps.c @@ -1512,23 +1512,11 @@ static void do_program_check(struct pt_regs *regs) return; } - if (cpu_has_feature(CPU_FTR_DEXCR_NPHIE) && user_mode(regs)) { - ppc_inst_t insn; - - if (get_user_instr(insn, (void __user *)regs->nip)) { - _exception(SIGSEGV, regs, SEGV_MAPERR, regs->nip); - return; - } - - if (ppc_inst_primary_opcode(insn) == 31 && - get_xop(ppc_inst_val(insn)) == OP_31_XOP_HASHCHK) { - _exception(SIGILL, regs, ILL_ILLOPN, regs->nip); - return; - } + /* User mode considers other cases after enabling IRQs */ + if (!user_mode(regs)) { + _exception(SIGTRAP, regs, TRAP_BRKPT, regs->nip); + return; } - - _exception(SIGTRAP, regs, TRAP_BRKPT, regs->nip); - return; } #ifdef CONFIG_PPC_TRANSACTIONAL_MEM if (reason & REASON_TM) { @@ -1561,16 +1549,44 @@ static void do_program_check(struct pt_regs *regs) /* * If we took the program check in the kernel skip down to sending a - * SIGILL. The subsequent cases all relate to emulating instructions - * which we should only do for userspace. We also do not want to enable - * interrupts for kernel faults because that might lead to further - * faults, and loose the context of the original exception. + * SIGILL. The subsequent cases all relate to user space, such as + * emulating instructions which we should only do for user space. We + * also do not want to enable interrupts for kernel faults because that + * might lead to further faults, and loose the context of the original + * exception. */ if (!user_mode(regs)) goto sigill; interrupt_cond_local_irq_enable(regs); + /* + * (reason & REASON_TRAP) is mostly handled before enabling IRQs, + * except get_user_instr() can sleep so we cannot reliably inspect the + * current instruction in that context. Now that we know we are + * handling a user space trap and can sleep, we can check if the trap + * was a hashchk failure. + */ + if (reason & REASON_TRAP) { + if (cpu_has_feature(CPU_FTR_DEXCR_NPHIE)) { + ppc_inst_t insn; + + if (get_user_instr(insn, (void __user *)regs->nip)) { + _exception(SIGSEGV, regs, SEGV_MAPERR, regs->nip); + return; + } + + if (ppc_inst_primary_opcode(insn) == 31 && + get_xop(ppc_inst_val(insn)) == OP_31_XOP_HASHCHK) { + _exception(SIGILL, regs, ILL_ILLOPN, regs->nip); + return; + } + } + + _exception(SIGTRAP, regs, TRAP_BRKPT, regs->nip); + return; + } + /* (reason & REASON_ILLEGAL) would be the obvious thing here, * but there seems to be a hardware bug on the 405GP (RevD) * that means ESR is sometimes set incorrectly - either to diff --git a/arch/powerpc/lib/qspinlock.c b/arch/powerpc/lib/qspinlock.c index 253620979d0c..6dd2f46bd3ef 100644 --- a/arch/powerpc/lib/qspinlock.c +++ b/arch/powerpc/lib/qspinlock.c @@ -406,6 +406,9 @@ static __always_inline bool yield_to_prev(struct qspinlock *lock, struct qnode * if ((yield_count & 1) == 0) goto yield_prev; /* owner vcpu is running */ + if (get_owner_cpu(READ_ONCE(lock->val)) != yield_cpu) + goto yield_prev; /* re-sample lock owner */ + spin_end(); preempted = true; diff --git a/arch/powerpc/mm/book3s64/hugetlbpage.c b/arch/powerpc/mm/book3s64/hugetlbpage.c index 3bc0eb21b2a0..5a2e512e96db 100644 --- a/arch/powerpc/mm/book3s64/hugetlbpage.c +++ b/arch/powerpc/mm/book3s64/hugetlbpage.c @@ -143,11 +143,14 @@ pte_t huge_ptep_modify_prot_start(struct vm_area_struct *vma, void huge_ptep_modify_prot_commit(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, pte_t old_pte, pte_t pte) { + unsigned long psize; if (radix_enabled()) return radix__huge_ptep_modify_prot_commit(vma, addr, ptep, old_pte, pte); - set_huge_pte_at(vma->vm_mm, addr, ptep, pte); + + psize = huge_page_size(hstate_vma(vma)); + set_huge_pte_at(vma->vm_mm, addr, ptep, pte, psize); } void __init hugetlbpage_init_defaultsize(void) diff --git a/arch/powerpc/mm/book3s64/radix_hugetlbpage.c b/arch/powerpc/mm/book3s64/radix_hugetlbpage.c index 17075c78d4bc..35fd2a95be24 100644 --- a/arch/powerpc/mm/book3s64/radix_hugetlbpage.c +++ b/arch/powerpc/mm/book3s64/radix_hugetlbpage.c @@ -47,6 +47,7 @@ void radix__huge_ptep_modify_prot_commit(struct vm_area_struct *vma, pte_t old_pte, pte_t pte) { struct mm_struct *mm = vma->vm_mm; + unsigned long psize = huge_page_size(hstate_vma(vma)); /* * POWER9 NMMU must flush the TLB after clearing the PTE before @@ -58,5 +59,5 @@ void radix__huge_ptep_modify_prot_commit(struct vm_area_struct *vma, atomic_read(&mm->context.copros) > 0) radix__flush_hugetlb_page(vma, addr); - set_huge_pte_at(vma->vm_mm, addr, ptep, pte); + set_huge_pte_at(vma->vm_mm, addr, ptep, pte, psize); } diff --git a/arch/powerpc/mm/book3s64/radix_tlb.c b/arch/powerpc/mm/book3s64/radix_tlb.c index 39acc2cbab4c..9e1f6558d026 100644 --- a/arch/powerpc/mm/book3s64/radix_tlb.c +++ b/arch/powerpc/mm/book3s64/radix_tlb.c @@ -1212,14 +1212,7 @@ void radix__tlb_flush(struct mmu_gather *tlb) smp_mb(); /* see radix__flush_tlb_mm */ exit_flush_lazy_tlbs(mm); - _tlbiel_pid(mm->context.id, RIC_FLUSH_ALL); - - /* - * It should not be possible to have coprocessors still - * attached here. - */ - if (WARN_ON_ONCE(atomic_read(&mm->context.copros) > 0)) - __flush_all_mm(mm, true); + __flush_all_mm(mm, true); preempt_enable(); } else { diff --git a/arch/powerpc/mm/nohash/8xx.c b/arch/powerpc/mm/nohash/8xx.c index dbbfe897455d..a642a7929892 100644 --- a/arch/powerpc/mm/nohash/8xx.c +++ b/arch/powerpc/mm/nohash/8xx.c @@ -91,7 +91,8 @@ static int __ref __early_map_kernel_hugepage(unsigned long va, phys_addr_t pa, if (new && WARN_ON(pte_present(*ptep) && pgprot_val(prot))) return -EINVAL; - set_huge_pte_at(&init_mm, va, ptep, pte_mkhuge(pfn_pte(pa >> PAGE_SHIFT, prot))); + set_huge_pte_at(&init_mm, va, ptep, + pte_mkhuge(pfn_pte(pa >> PAGE_SHIFT, prot)), psize); return 0; } diff --git a/arch/powerpc/mm/pgtable.c b/arch/powerpc/mm/pgtable.c index 3f86fd217690..3ba9fe411604 100644 --- a/arch/powerpc/mm/pgtable.c +++ b/arch/powerpc/mm/pgtable.c @@ -288,7 +288,8 @@ int huge_ptep_set_access_flags(struct vm_area_struct *vma, } #if defined(CONFIG_PPC_8xx) -void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, pte_t *ptep, pte_t pte) +void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, pte_t *ptep, + pte_t pte, unsigned long sz) { pmd_t *pmd = pmd_off(mm, addr); pte_basic_t val; diff --git a/arch/powerpc/perf/hv-24x7.c b/arch/powerpc/perf/hv-24x7.c index 317175791d23..3449be7c0d51 100644 --- a/arch/powerpc/perf/hv-24x7.c +++ b/arch/powerpc/perf/hv-24x7.c @@ -1418,7 +1418,7 @@ static int h_24x7_event_init(struct perf_event *event) } domain = event_get_domain(event); - if (domain >= HV_PERF_DOMAIN_MAX) { + if (domain == 0 || domain >= HV_PERF_DOMAIN_MAX) { pr_devel("invalid domain %d\n", domain); return -EINVAL; } diff --git a/arch/powerpc/platforms/82xx/Kconfig b/arch/powerpc/platforms/82xx/Kconfig index d9f1a2a83158..1824536cf6f2 100644 --- a/arch/powerpc/platforms/82xx/Kconfig +++ b/arch/powerpc/platforms/82xx/Kconfig @@ -2,6 +2,7 @@ menuconfig PPC_82xx bool "82xx-based boards (PQ II)" depends on PPC_BOOK3S_32 + select FSL_SOC if PPC_82xx @@ -9,7 +10,6 @@ config EP8248E bool "Embedded Planet EP8248E (a.k.a. CWH-PPC-8248N-VE)" select CPM2 select PPC_INDIRECT_PCI if PCI - select FSL_SOC select PHYLIB if NETDEVICES select MDIO_BITBANG if PHYLIB help @@ -22,7 +22,6 @@ config MGCOGE bool "Keymile MGCOGE" select CPM2 select PPC_INDIRECT_PCI if PCI - select FSL_SOC help This enables support for the Keymile MGCOGE board. diff --git a/arch/powerpc/platforms/pseries/hvCall.S b/arch/powerpc/platforms/pseries/hvCall.S index bae45b358a09..2b0cac6fb61f 100644 --- a/arch/powerpc/platforms/pseries/hvCall.S +++ b/arch/powerpc/platforms/pseries/hvCall.S @@ -184,9 +184,6 @@ _GLOBAL_TOC(plpar_hcall) plpar_hcall_trace: HCALL_INST_PRECALL(R5) - std r4,STK_PARAM(R4)(r1) - mr r0,r4 - mr r4,r5 mr r5,r6 mr r6,r7 @@ -196,7 +193,7 @@ plpar_hcall_trace: HVSC - ld r12,STK_PARAM(R4)(r1) + ld r12,STACK_FRAME_MIN_SIZE+STK_PARAM(R4)(r1) std r4,0(r12) std r5,8(r12) std r6,16(r12) @@ -296,9 +293,6 @@ _GLOBAL_TOC(plpar_hcall9) plpar_hcall9_trace: HCALL_INST_PRECALL(R5) - std r4,STK_PARAM(R4)(r1) - mr r0,r4 - mr r4,r5 mr r5,r6 mr r6,r7 diff --git a/arch/riscv/Makefile b/arch/riscv/Makefile index 1329e060c548..b43a6bb7e4dc 100644 --- a/arch/riscv/Makefile +++ b/arch/riscv/Makefile @@ -6,7 +6,6 @@ # for more details. # -OBJCOPYFLAGS := -O binary LDFLAGS_vmlinux := -z norelro ifeq ($(CONFIG_RELOCATABLE),y) LDFLAGS_vmlinux += -shared -Bsymbolic -z notext --emit-relocs diff --git a/arch/riscv/boot/dts/starfive/jh7110-starfive-visionfive-2.dtsi b/arch/riscv/boot/dts/starfive/jh7110-starfive-visionfive-2.dtsi index d79f94432b27..12ebe9792356 100644 --- a/arch/riscv/boot/dts/starfive/jh7110-starfive-visionfive-2.dtsi +++ b/arch/riscv/boot/dts/starfive/jh7110-starfive-visionfive-2.dtsi @@ -262,7 +262,7 @@ reg = <0x100000 0x400000>; }; reserved-data@600000 { - reg = <0x600000 0x1000000>; + reg = <0x600000 0xa00000>; }; }; }; @@ -440,30 +440,6 @@ }; }; - uart0_pins: uart0-0 { - tx-pins { - pinmux = ; - bias-disable; - drive-strength = <12>; - input-disable; - input-schmitt-disable; - slew-rate = <0>; - }; - - rx-pins { - pinmux = ; - bias-disable; /* external pull-up */ - drive-strength = <2>; - input-enable; - input-schmitt-enable; - slew-rate = <0>; - }; - }; - tdm_pins: tdm-0 { tx-pins { pinmux = ; + bias-disable; + drive-strength = <12>; + input-disable; + input-schmitt-disable; + slew-rate = <0>; + }; + + rx-pins { + pinmux = ; + bias-disable; /* external pull-up */ + drive-strength = <2>; + input-enable; + input-schmitt-enable; + slew-rate = <0>; + }; + }; }; &tdm { @@ -513,6 +513,7 @@ &usb0 { dr_mode = "peripheral"; + status = "okay"; }; &U74_1 { diff --git a/arch/riscv/errata/andes/Makefile b/arch/riscv/errata/andes/Makefile index 2d644e19caef..6278c389b801 100644 --- a/arch/riscv/errata/andes/Makefile +++ b/arch/riscv/errata/andes/Makefile @@ -1 +1,5 @@ +ifdef CONFIG_RISCV_ALTERNATIVE_EARLY +CFLAGS_errata.o := -mcmodel=medany +endif + obj-y += errata.o diff --git a/arch/riscv/include/asm/errata_list.h b/arch/riscv/include/asm/errata_list.h index e2ecd01bfac7..b55b434f0059 100644 --- a/arch/riscv/include/asm/errata_list.h +++ b/arch/riscv/include/asm/errata_list.h @@ -105,7 +105,7 @@ asm volatile(ALTERNATIVE( \ * | 31 - 25 | 24 - 20 | 19 - 15 | 14 - 12 | 11 - 7 | 6 - 0 | * 0000001 01001 rs1 000 00000 0001011 * dcache.cva rs1 (clean, virtual address) - * 0000001 00100 rs1 000 00000 0001011 + * 0000001 00101 rs1 000 00000 0001011 * * dcache.cipa rs1 (clean then invalidate, physical address) * | 31 - 25 | 24 - 20 | 19 - 15 | 14 - 12 | 11 - 7 | 6 - 0 | @@ -118,7 +118,7 @@ asm volatile(ALTERNATIVE( \ * 0000000 11001 00000 000 00000 0001011 */ #define THEAD_inval_A0 ".long 0x0265000b" -#define THEAD_clean_A0 ".long 0x0245000b" +#define THEAD_clean_A0 ".long 0x0255000b" #define THEAD_flush_A0 ".long 0x0275000b" #define THEAD_SYNC_S ".long 0x0190000b" diff --git a/arch/riscv/include/asm/ftrace.h b/arch/riscv/include/asm/ftrace.h index 740a979171e5..2b2f5df7ef2c 100644 --- a/arch/riscv/include/asm/ftrace.h +++ b/arch/riscv/include/asm/ftrace.h @@ -31,6 +31,27 @@ static inline unsigned long ftrace_call_adjust(unsigned long addr) return addr; } +/* + * Let's do like x86/arm64 and ignore the compat syscalls. + */ +#define ARCH_TRACE_IGNORE_COMPAT_SYSCALLS +static inline bool arch_trace_is_compat_syscall(struct pt_regs *regs) +{ + return is_compat_task(); +} + +#define ARCH_HAS_SYSCALL_MATCH_SYM_NAME +static inline bool arch_syscall_match_sym_name(const char *sym, + const char *name) +{ + /* + * Since all syscall functions have __riscv_ prefix, we must skip it. + * However, as we described above, we decided to ignore compat + * syscalls, so we don't care about __riscv_compat_ prefix here. + */ + return !strcmp(sym + 8, name); +} + struct dyn_arch_ftrace { }; #endif diff --git a/arch/riscv/include/asm/hugetlb.h b/arch/riscv/include/asm/hugetlb.h index 34e24f078cc1..4c5b0e929890 100644 --- a/arch/riscv/include/asm/hugetlb.h +++ b/arch/riscv/include/asm/hugetlb.h @@ -18,7 +18,8 @@ void huge_pte_clear(struct mm_struct *mm, unsigned long addr, #define __HAVE_ARCH_HUGE_SET_HUGE_PTE_AT void set_huge_pte_at(struct mm_struct *mm, - unsigned long addr, pte_t *ptep, pte_t pte); + unsigned long addr, pte_t *ptep, pte_t pte, + unsigned long sz); #define __HAVE_ARCH_HUGE_PTEP_GET_AND_CLEAR pte_t huge_ptep_get_and_clear(struct mm_struct *mm, diff --git a/arch/riscv/include/asm/kprobes.h b/arch/riscv/include/asm/kprobes.h index e7882ccb0fd4..78ea44f76718 100644 --- a/arch/riscv/include/asm/kprobes.h +++ b/arch/riscv/include/asm/kprobes.h @@ -40,6 +40,15 @@ void arch_remove_kprobe(struct kprobe *p); int kprobe_fault_handler(struct pt_regs *regs, unsigned int trapnr); bool kprobe_breakpoint_handler(struct pt_regs *regs); bool kprobe_single_step_handler(struct pt_regs *regs); - +#else +static inline bool kprobe_breakpoint_handler(struct pt_regs *regs) +{ + return false; +} + +static inline bool kprobe_single_step_handler(struct pt_regs *regs) +{ + return false; +} #endif /* CONFIG_KPROBES */ #endif /* _ASM_RISCV_KPROBES_H */ diff --git a/arch/riscv/include/asm/uprobes.h b/arch/riscv/include/asm/uprobes.h index f2183e00fdd2..3fc7deda9190 100644 --- a/arch/riscv/include/asm/uprobes.h +++ b/arch/riscv/include/asm/uprobes.h @@ -34,7 +34,18 @@ struct arch_uprobe { bool simulate; }; +#ifdef CONFIG_UPROBES bool uprobe_breakpoint_handler(struct pt_regs *regs); bool uprobe_single_step_handler(struct pt_regs *regs); - +#else +static inline bool uprobe_breakpoint_handler(struct pt_regs *regs) +{ + return false; +} + +static inline bool uprobe_single_step_handler(struct pt_regs *regs) +{ + return false; +} +#endif /* CONFIG_UPROBES */ #endif /* _ASM_RISCV_UPROBES_H */ diff --git a/arch/riscv/kernel/elf_kexec.c b/arch/riscv/kernel/elf_kexec.c index f4099059ed8f..e60fbd8660c4 100644 --- a/arch/riscv/kernel/elf_kexec.c +++ b/arch/riscv/kernel/elf_kexec.c @@ -98,7 +98,13 @@ static int elf_find_pbase(struct kimage *image, unsigned long kernel_len, kbuf.image = image; kbuf.buf_min = lowest_paddr; kbuf.buf_max = ULONG_MAX; - kbuf.buf_align = PAGE_SIZE; + + /* + * Current riscv boot protocol requires 2MB alignment for + * RV64 and 4MB alignment for RV32 + * + */ + kbuf.buf_align = PMD_SIZE; kbuf.mem = KEXEC_BUF_MEM_UNKNOWN; kbuf.memsz = ALIGN(kernel_len, PAGE_SIZE); kbuf.top_down = false; diff --git a/arch/riscv/kernel/irq.c b/arch/riscv/kernel/irq.c index a8efa053c4a5..9cc0a7669271 100644 --- a/arch/riscv/kernel/irq.c +++ b/arch/riscv/kernel/irq.c @@ -60,7 +60,7 @@ static void init_irq_stacks(void) } #endif /* CONFIG_VMAP_STACK */ -#ifdef CONFIG_HAVE_SOFTIRQ_ON_OWN_STACK +#ifdef CONFIG_SOFTIRQ_ON_OWN_STACK void do_softirq_own_stack(void) { #ifdef CONFIG_IRQ_STACKS @@ -92,7 +92,7 @@ void do_softirq_own_stack(void) #endif __do_softirq(); } -#endif /* CONFIG_HAVE_SOFTIRQ_ON_OWN_STACK */ +#endif /* CONFIG_SOFTIRQ_ON_OWN_STACK */ #else static void init_irq_stacks(void) {} diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c index e600aab116a4..aac853ae4eb7 100644 --- a/arch/riscv/kernel/setup.c +++ b/arch/riscv/kernel/setup.c @@ -173,19 +173,6 @@ static void __init init_resources(void) if (ret < 0) goto error; -#ifdef CONFIG_KEXEC_CORE - if (crashk_res.start != crashk_res.end) { - ret = add_resource(&iomem_resource, &crashk_res); - if (ret < 0) - goto error; - } - if (crashk_low_res.start != crashk_low_res.end) { - ret = add_resource(&iomem_resource, &crashk_low_res); - if (ret < 0) - goto error; - } -#endif - #ifdef CONFIG_CRASH_DUMP if (elfcorehdr_size > 0) { elfcorehdr_res.start = elfcorehdr_addr; diff --git a/arch/riscv/kernel/signal.c b/arch/riscv/kernel/signal.c index 180d951d3624..21a4d0e111bc 100644 --- a/arch/riscv/kernel/signal.c +++ b/arch/riscv/kernel/signal.c @@ -311,13 +311,6 @@ static inline void __user *get_sigframe(struct ksignal *ksig, /* Align the stack frame. */ sp &= ~0xfUL; - /* - * Fail if the size of the altstack is not large enough for the - * sigframe construction. - */ - if (current->sas_ss_size && sp < current->sas_ss_sp) - return (void __user __force *)-1UL; - return (void __user *)sp; } diff --git a/arch/riscv/kernel/traps.c b/arch/riscv/kernel/traps.c index 19807c4d3805..fae8f610d867 100644 --- a/arch/riscv/kernel/traps.c +++ b/arch/riscv/kernel/traps.c @@ -13,6 +13,8 @@ #include #include #include +#include +#include #include #include #include @@ -247,22 +249,28 @@ static inline unsigned long get_break_insn_length(unsigned long pc) return GET_INSN_LENGTH(insn); } +static bool probe_single_step_handler(struct pt_regs *regs) +{ + bool user = user_mode(regs); + + return user ? uprobe_single_step_handler(regs) : kprobe_single_step_handler(regs); +} + +static bool probe_breakpoint_handler(struct pt_regs *regs) +{ + bool user = user_mode(regs); + + return user ? uprobe_breakpoint_handler(regs) : kprobe_breakpoint_handler(regs); +} + void handle_break(struct pt_regs *regs) { -#ifdef CONFIG_KPROBES - if (kprobe_single_step_handler(regs)) + if (probe_single_step_handler(regs)) return; - if (kprobe_breakpoint_handler(regs)) - return; -#endif -#ifdef CONFIG_UPROBES - if (uprobe_single_step_handler(regs)) + if (probe_breakpoint_handler(regs)) return; - if (uprobe_breakpoint_handler(regs)) - return; -#endif current->thread.bad_cause = regs->cause; if (user_mode(regs)) diff --git a/arch/riscv/kvm/vcpu_onereg.c b/arch/riscv/kvm/vcpu_onereg.c index 1b7e9fa265cb..b7e0e03c69b1 100644 --- a/arch/riscv/kvm/vcpu_onereg.c +++ b/arch/riscv/kvm/vcpu_onereg.c @@ -460,8 +460,11 @@ static int riscv_vcpu_get_isa_ext_single(struct kvm_vcpu *vcpu, reg_num >= ARRAY_SIZE(kvm_isa_ext_arr)) return -ENOENT; - *reg_val = 0; host_isa_ext = kvm_isa_ext_arr[reg_num]; + if (!__riscv_isa_extension_available(NULL, host_isa_ext)) + return -ENOENT; + + *reg_val = 0; if (__riscv_isa_extension_available(vcpu->arch.isa, host_isa_ext)) *reg_val = 1; /* Mark the given extension as available */ @@ -842,7 +845,7 @@ static int copy_isa_ext_reg_indices(const struct kvm_vcpu *vcpu, u64 reg = KVM_REG_RISCV | size | KVM_REG_RISCV_ISA_EXT | i; isa_ext = kvm_isa_ext_arr[i]; - if (!__riscv_isa_extension_available(vcpu->arch.isa, isa_ext)) + if (!__riscv_isa_extension_available(NULL, isa_ext)) continue; if (uindices) { diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index 96225a8533ad..e4a2ace92dbe 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -180,7 +180,8 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, pte_t *ptep, - pte_t pte) + pte_t pte, + unsigned long sz) { int i, pte_num; diff --git a/arch/riscv/net/bpf_jit_comp64.c b/arch/riscv/net/bpf_jit_comp64.c index ecd3ae6f4116..8581693e62d3 100644 --- a/arch/riscv/net/bpf_jit_comp64.c +++ b/arch/riscv/net/bpf_jit_comp64.c @@ -245,7 +245,7 @@ static void __build_epilogue(bool is_tail_call, struct rv_jit_context *ctx) emit_addi(RV_REG_SP, RV_REG_SP, stack_adjust, ctx); /* Set return value. */ if (!is_tail_call) - emit_mv(RV_REG_A0, RV_REG_A5, ctx); + emit_addiw(RV_REG_A0, RV_REG_A5, 0, ctx); emit_jalr(RV_REG_ZERO, is_tail_call ? RV_REG_T3 : RV_REG_RA, is_tail_call ? (RV_FENTRY_NINSNS + 1) * 4 : 0, /* skip reserved nops and TCC init */ ctx); @@ -759,8 +759,10 @@ static int invoke_bpf_prog(struct bpf_tramp_link *l, int args_off, int retval_of if (ret) return ret; - if (save_ret) - emit_sd(RV_REG_FP, -retval_off, regmap[BPF_REG_0], ctx); + if (save_ret) { + emit_sd(RV_REG_FP, -retval_off, RV_REG_A0, ctx); + emit_sd(RV_REG_FP, -(retval_off - 8), regmap[BPF_REG_0], ctx); + } /* update branch with beqz */ if (ctx->insns) { @@ -853,7 +855,7 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, save_ret = flags & (BPF_TRAMP_F_CALL_ORIG | BPF_TRAMP_F_RET_FENTRY_RET); if (save_ret) { - stack_size += 8; + stack_size += 16; /* Save both A5 (BPF R0) and A0 */ retval_off = stack_size; } @@ -957,6 +959,7 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, if (ret) goto out; emit_sd(RV_REG_FP, -retval_off, RV_REG_A0, ctx); + emit_sd(RV_REG_FP, -(retval_off - 8), regmap[BPF_REG_0], ctx); im->ip_after_call = ctx->insns + ctx->ninsns; /* 2 nops reserved for auipc+jalr pair */ emit(rv_nop(), ctx); @@ -988,8 +991,10 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, if (flags & BPF_TRAMP_F_RESTORE_REGS) restore_args(nregs, args_off, ctx); - if (save_ret) + if (save_ret) { emit_ld(RV_REG_A0, -retval_off, RV_REG_FP, ctx); + emit_ld(regmap[BPF_REG_0], -(retval_off - 8), RV_REG_FP, ctx); + } emit_ld(RV_REG_S1, -sreg_off, RV_REG_FP, ctx); @@ -1515,7 +1520,8 @@ out_be: if (ret) return ret; - emit_mv(bpf_to_rv_reg(BPF_REG_0, ctx), RV_REG_A0, ctx); + if (insn->src_reg != BPF_PSEUDO_CALL) + emit_mv(bpf_to_rv_reg(BPF_REG_0, ctx), RV_REG_A0, ctx); break; } /* tail call */ diff --git a/arch/s390/boot/vmem.c b/arch/s390/boot/vmem.c index 01257ce3b89c..442a74f113cb 100644 --- a/arch/s390/boot/vmem.c +++ b/arch/s390/boot/vmem.c @@ -57,6 +57,7 @@ static void kasan_populate_shadow(void) pmd_t pmd_z = __pmd(__pa(kasan_early_shadow_pte) | _SEGMENT_ENTRY); pud_t pud_z = __pud(__pa(kasan_early_shadow_pmd) | _REGION3_ENTRY); p4d_t p4d_z = __p4d(__pa(kasan_early_shadow_pud) | _REGION2_ENTRY); + unsigned long memgap_start = 0; unsigned long untracked_end; unsigned long start, end; int i; @@ -101,8 +102,12 @@ static void kasan_populate_shadow(void) * +- shadow end ----+---------+- shadow end ---+ */ - for_each_physmem_usable_range(i, &start, &end) + for_each_physmem_usable_range(i, &start, &end) { kasan_populate(start, end, POPULATE_KASAN_MAP_SHADOW); + if (memgap_start && physmem_info.info_source == MEM_DETECT_DIAG260) + kasan_populate(memgap_start, start, POPULATE_KASAN_ZERO_SHADOW); + memgap_start = end; + } if (IS_ENABLED(CONFIG_KASAN_VMALLOC)) { untracked_end = VMALLOC_START; /* shallowly populate kasan shadow for vmalloc and modules */ diff --git a/arch/s390/configs/debug_defconfig b/arch/s390/configs/debug_defconfig index af2fbe48e16c..438cd92e6080 100644 --- a/arch/s390/configs/debug_defconfig +++ b/arch/s390/configs/debug_defconfig @@ -40,23 +40,25 @@ CONFIG_SCHED_AUTOGROUP=y CONFIG_EXPERT=y # CONFIG_SYSFS_SYSCALL is not set CONFIG_PROFILING=y +CONFIG_KEXEC_FILE=y +CONFIG_KEXEC_SIG=y +CONFIG_CRASH_DUMP=y CONFIG_LIVEPATCH=y CONFIG_MARCH_ZEC12=y CONFIG_TUNE_ZEC12=y CONFIG_NR_CPUS=512 CONFIG_NUMA=y CONFIG_HZ_100=y -CONFIG_KEXEC_FILE=y -CONFIG_KEXEC_SIG=y +CONFIG_CERT_STORE=y CONFIG_EXPOLINE=y CONFIG_EXPOLINE_AUTO=y CONFIG_CHSC_SCH=y CONFIG_VFIO_CCW=m CONFIG_VFIO_AP=m -CONFIG_CRASH_DUMP=y CONFIG_PROTECTED_VIRTUALIZATION_GUEST=y CONFIG_CMM=m CONFIG_APPLDATA_BASE=y +CONFIG_S390_HYPFS_FS=y CONFIG_KVM=m CONFIG_S390_UNWIND_SELFTEST=m CONFIG_S390_KPROBES_SANITY_TEST=m @@ -434,6 +436,7 @@ CONFIG_SCSI_DH_EMC=m CONFIG_SCSI_DH_ALUA=m CONFIG_MD=y CONFIG_BLK_DEV_MD=y +# CONFIG_MD_BITMAP_FILE is not set CONFIG_MD_LINEAR=m CONFIG_MD_MULTIPATH=m CONFIG_MD_FAULTY=m @@ -577,6 +580,7 @@ CONFIG_SOFT_WATCHDOG=m CONFIG_DIAG288_WATCHDOG=m # CONFIG_DRM_DEBUG_MODESET_LOCK is not set CONFIG_FB=y +# CONFIG_FB_DEVICE is not set CONFIG_FRAMEBUFFER_CONSOLE=y CONFIG_FRAMEBUFFER_CONSOLE_DETECT_PRIMARY=y # CONFIG_HID_SUPPORT is not set @@ -647,6 +651,7 @@ CONFIG_PROC_KCORE=y CONFIG_TMPFS=y CONFIG_TMPFS_POSIX_ACL=y CONFIG_TMPFS_INODE64=y +CONFIG_TMPFS_QUOTA=y CONFIG_HUGETLBFS=y CONFIG_ECRYPT_FS=m CONFIG_CRAMFS=m @@ -703,6 +708,7 @@ CONFIG_IMA_WRITE_POLICY=y CONFIG_IMA_APPRAISE=y CONFIG_LSM="yama,loadpin,safesetid,integrity,selinux,smack,tomoyo,apparmor" CONFIG_INIT_STACK_NONE=y +CONFIG_BUG_ON_DATA_CORRUPTION=y CONFIG_CRYPTO_USER=m # CONFIG_CRYPTO_MANAGER_DISABLE_TESTS is not set CONFIG_CRYPTO_PCRYPT=m @@ -825,9 +831,9 @@ CONFIG_LOCK_STAT=y CONFIG_DEBUG_ATOMIC_SLEEP=y CONFIG_DEBUG_LOCKING_API_SELFTESTS=y CONFIG_DEBUG_IRQFLAGS=y +CONFIG_DEBUG_LIST=y CONFIG_DEBUG_SG=y CONFIG_DEBUG_NOTIFIERS=y -CONFIG_BUG_ON_DATA_CORRUPTION=y CONFIG_DEBUG_CREDENTIALS=y CONFIG_RCU_TORTURE_TEST=m CONFIG_RCU_REF_SCALE_TEST=m diff --git a/arch/s390/configs/defconfig b/arch/s390/configs/defconfig index 3f263b767a4c..1b8150e50f6a 100644 --- a/arch/s390/configs/defconfig +++ b/arch/s390/configs/defconfig @@ -38,23 +38,25 @@ CONFIG_SCHED_AUTOGROUP=y CONFIG_EXPERT=y # CONFIG_SYSFS_SYSCALL is not set CONFIG_PROFILING=y +CONFIG_KEXEC_FILE=y +CONFIG_KEXEC_SIG=y +CONFIG_CRASH_DUMP=y CONFIG_LIVEPATCH=y CONFIG_MARCH_ZEC12=y CONFIG_TUNE_ZEC12=y CONFIG_NR_CPUS=512 CONFIG_NUMA=y CONFIG_HZ_100=y -CONFIG_KEXEC_FILE=y -CONFIG_KEXEC_SIG=y +CONFIG_CERT_STORE=y CONFIG_EXPOLINE=y CONFIG_EXPOLINE_AUTO=y CONFIG_CHSC_SCH=y CONFIG_VFIO_CCW=m CONFIG_VFIO_AP=m -CONFIG_CRASH_DUMP=y CONFIG_PROTECTED_VIRTUALIZATION_GUEST=y CONFIG_CMM=m CONFIG_APPLDATA_BASE=y +CONFIG_S390_HYPFS_FS=y CONFIG_KVM=m CONFIG_S390_UNWIND_SELFTEST=m CONFIG_S390_KPROBES_SANITY_TEST=m @@ -424,6 +426,7 @@ CONFIG_SCSI_DH_EMC=m CONFIG_SCSI_DH_ALUA=m CONFIG_MD=y CONFIG_BLK_DEV_MD=y +# CONFIG_MD_BITMAP_FILE is not set CONFIG_MD_LINEAR=m CONFIG_MD_MULTIPATH=m CONFIG_MD_FAULTY=m @@ -566,6 +569,7 @@ CONFIG_WATCHDOG_NOWAYOUT=y CONFIG_SOFT_WATCHDOG=m CONFIG_DIAG288_WATCHDOG=m CONFIG_FB=y +# CONFIG_FB_DEVICE is not set CONFIG_FRAMEBUFFER_CONSOLE=y CONFIG_FRAMEBUFFER_CONSOLE_DETECT_PRIMARY=y # CONFIG_HID_SUPPORT is not set @@ -632,6 +636,7 @@ CONFIG_PROC_KCORE=y CONFIG_TMPFS=y CONFIG_TMPFS_POSIX_ACL=y CONFIG_TMPFS_INODE64=y +CONFIG_TMPFS_QUOTA=y CONFIG_HUGETLBFS=y CONFIG_CONFIGFS_FS=m CONFIG_ECRYPT_FS=m @@ -687,6 +692,7 @@ CONFIG_IMA_WRITE_POLICY=y CONFIG_IMA_APPRAISE=y CONFIG_LSM="yama,loadpin,safesetid,integrity,selinux,smack,tomoyo,apparmor" CONFIG_INIT_STACK_NONE=y +CONFIG_BUG_ON_DATA_CORRUPTION=y CONFIG_CRYPTO_FIPS=y CONFIG_CRYPTO_USER=m # CONFIG_CRYPTO_MANAGER_DISABLE_TESTS is not set @@ -781,7 +787,6 @@ CONFIG_PTDUMP_DEBUGFS=y CONFIG_DEBUG_MEMORY_INIT=y CONFIG_PANIC_ON_OOPS=y CONFIG_TEST_LOCKUP=m -CONFIG_BUG_ON_DATA_CORRUPTION=y CONFIG_RCU_TORTURE_TEST=m CONFIG_RCU_REF_SCALE_TEST=m CONFIG_RCU_CPU_STALL_TIMEOUT=60 diff --git a/arch/s390/configs/zfcpdump_defconfig b/arch/s390/configs/zfcpdump_defconfig index e62fb2015102..b831083b4edd 100644 --- a/arch/s390/configs/zfcpdump_defconfig +++ b/arch/s390/configs/zfcpdump_defconfig @@ -8,6 +8,7 @@ CONFIG_BPF_SYSCALL=y # CONFIG_NET_NS is not set CONFIG_BLK_DEV_INITRD=y CONFIG_CC_OPTIMIZE_FOR_SIZE=y +CONFIG_CRASH_DUMP=y CONFIG_MARCH_ZEC12=y CONFIG_TUNE_ZEC12=y # CONFIG_COMPAT is not set @@ -15,9 +16,8 @@ CONFIG_NR_CPUS=2 CONFIG_HZ_100=y # CONFIG_CHSC_SCH is not set # CONFIG_SCM_BUS is not set -CONFIG_CRASH_DUMP=y # CONFIG_PFAULT is not set -# CONFIG_S390_HYPFS_FS is not set +# CONFIG_S390_HYPFS is not set # CONFIG_VIRTUALIZATION is not set # CONFIG_S390_GUEST is not set # CONFIG_SECCOMP is not set diff --git a/arch/s390/include/asm/hugetlb.h b/arch/s390/include/asm/hugetlb.h index f07267875a19..deb198a61039 100644 --- a/arch/s390/include/asm/hugetlb.h +++ b/arch/s390/include/asm/hugetlb.h @@ -16,6 +16,8 @@ #define hugepages_supported() (MACHINE_HAS_EDAT1) void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, pte_t pte, unsigned long sz); +void __set_huge_pte_at(struct mm_struct *mm, unsigned long addr, pte_t *ptep, pte_t pte); pte_t huge_ptep_get(pte_t *ptep); pte_t huge_ptep_get_and_clear(struct mm_struct *mm, @@ -65,7 +67,7 @@ static inline int huge_ptep_set_access_flags(struct vm_area_struct *vma, int changed = !pte_same(huge_ptep_get(ptep), pte); if (changed) { huge_ptep_get_and_clear(vma->vm_mm, addr, ptep); - set_huge_pte_at(vma->vm_mm, addr, ptep, pte); + __set_huge_pte_at(vma->vm_mm, addr, ptep, pte); } return changed; } @@ -74,7 +76,7 @@ static inline void huge_ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr, pte_t *ptep) { pte_t pte = huge_ptep_get_and_clear(mm, addr, ptep); - set_huge_pte_at(mm, addr, ptep, pte_wrprotect(pte)); + __set_huge_pte_at(mm, addr, ptep, pte_wrprotect(pte)); } static inline pte_t mk_huge_pte(struct page *page, pgprot_t pgprot) diff --git a/arch/s390/kernel/cert_store.c b/arch/s390/kernel/cert_store.c index 3986a044eb36..554447768bdd 100644 --- a/arch/s390/kernel/cert_store.c +++ b/arch/s390/kernel/cert_store.c @@ -432,15 +432,16 @@ static char *get_key_description(struct vcssb *vcssb, const struct vce *vce) char *desc; cs_token = vcssb->cs_token; - /* Description string contains "%64s:%04u:%08u\0". */ + /* Description string contains "%64s:%05u:%010u\0". */ name_len = sizeof(vce->vce_hdr.vc_name); - len = name_len + 1 + 4 + 1 + 8 + 1; + len = name_len + 1 + 5 + 1 + 10 + 1; desc = kmalloc(len, GFP_KERNEL); if (!desc) return NULL; memcpy(desc, vce->vce_hdr.vc_name, name_len); - sprintf(desc + name_len, ":%04u:%08u", vce->vce_hdr.vc_index, cs_token); + snprintf(desc + name_len, len - name_len, ":%05u:%010u", + vce->vce_hdr.vc_index, cs_token); return desc; } diff --git a/arch/s390/kvm/interrupt.c b/arch/s390/kvm/interrupt.c index c1b47d608a2b..efaebba5ee19 100644 --- a/arch/s390/kvm/interrupt.c +++ b/arch/s390/kvm/interrupt.c @@ -303,11 +303,6 @@ static inline u8 gisa_get_ipm_or_restore_iam(struct kvm_s390_gisa_interrupt *gi) return 0; } -static inline int gisa_in_alert_list(struct kvm_s390_gisa *gisa) -{ - return READ_ONCE(gisa->next_alert) != (u32)virt_to_phys(gisa); -} - static inline void gisa_set_ipm_gisc(struct kvm_s390_gisa *gisa, u32 gisc) { set_bit_inv(IPM_BIT_OFFSET + gisc, (unsigned long *) gisa); @@ -3216,11 +3211,12 @@ void kvm_s390_gisa_destroy(struct kvm *kvm) if (!gi->origin) return; - if (gi->alert.mask) - KVM_EVENT(3, "vm 0x%pK has unexpected iam 0x%02x", - kvm, gi->alert.mask); - while (gisa_in_alert_list(gi->origin)) - cpu_relax(); + WARN(gi->alert.mask != 0x00, + "unexpected non zero alert.mask 0x%02x", + gi->alert.mask); + gi->alert.mask = 0x00; + if (gisa_set_iam(gi->origin, gi->alert.mask)) + process_gib_alert_list(); hrtimer_cancel(&gi->timer); gi->origin = NULL; VM_EVENT(kvm, 3, "gisa 0x%pK destroyed", gisa); diff --git a/arch/s390/mm/hugetlbpage.c b/arch/s390/mm/hugetlbpage.c index c718f2a0de94..297a6d897d5a 100644 --- a/arch/s390/mm/hugetlbpage.c +++ b/arch/s390/mm/hugetlbpage.c @@ -142,7 +142,7 @@ static void clear_huge_pte_skeys(struct mm_struct *mm, unsigned long rste) __storage_key_init_range(paddr, paddr + size - 1); } -void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, +void __set_huge_pte_at(struct mm_struct *mm, unsigned long addr, pte_t *ptep, pte_t pte) { unsigned long rste; @@ -163,6 +163,12 @@ void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, set_pte(ptep, __pte(rste)); } +void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, pte_t pte, unsigned long sz) +{ + __set_huge_pte_at(mm, addr, ptep, pte); +} + pte_t huge_ptep_get(pte_t *ptep) { return __rste_to_pte(pte_val(*ptep)); diff --git a/arch/s390/net/bpf_jit_comp.c b/arch/s390/net/bpf_jit_comp.c index de2fb12120d2..e507692e51e7 100644 --- a/arch/s390/net/bpf_jit_comp.c +++ b/arch/s390/net/bpf_jit_comp.c @@ -2066,6 +2066,7 @@ struct bpf_tramp_jit { * func_addr's original caller */ int stack_size; /* Trampoline stack size */ + int backchain_off; /* Offset of backchain */ int stack_args_off; /* Offset of stack arguments for calling * func_addr, has to be at the top */ @@ -2086,9 +2087,10 @@ struct bpf_tramp_jit { * for __bpf_prog_enter() return value and * func_addr respectively */ - int r14_off; /* Offset of saved %r14 */ int run_ctx_off; /* Offset of struct bpf_tramp_run_ctx */ int tccnt_off; /* Offset of saved tailcall counter */ + int r14_off; /* Offset of saved %r14, has to be at the + * bottom */ int do_fexit; /* do_fexit: label */ }; @@ -2247,8 +2249,12 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, * Calculate the stack layout. */ - /* Reserve STACK_FRAME_OVERHEAD bytes for the callees. */ + /* + * Allocate STACK_FRAME_OVERHEAD bytes for the callees. As the s390x + * ABI requires, put our backchain at the end of the allocated memory. + */ tjit->stack_size = STACK_FRAME_OVERHEAD; + tjit->backchain_off = tjit->stack_size - sizeof(u64); tjit->stack_args_off = alloc_stack(tjit, nr_stack_args * sizeof(u64)); tjit->reg_args_off = alloc_stack(tjit, nr_reg_args * sizeof(u64)); tjit->ip_off = alloc_stack(tjit, sizeof(u64)); @@ -2256,16 +2262,25 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, tjit->bpf_args_off = alloc_stack(tjit, nr_bpf_args * sizeof(u64)); tjit->retval_off = alloc_stack(tjit, sizeof(u64)); tjit->r7_r8_off = alloc_stack(tjit, 2 * sizeof(u64)); - tjit->r14_off = alloc_stack(tjit, sizeof(u64)); tjit->run_ctx_off = alloc_stack(tjit, sizeof(struct bpf_tramp_run_ctx)); tjit->tccnt_off = alloc_stack(tjit, sizeof(u64)); - /* The caller has already reserved STACK_FRAME_OVERHEAD bytes. */ - tjit->stack_size -= STACK_FRAME_OVERHEAD; + tjit->r14_off = alloc_stack(tjit, sizeof(u64) * 2); + /* + * In accordance with the s390x ABI, the caller has allocated + * STACK_FRAME_OVERHEAD bytes for us. 8 of them contain the caller's + * backchain, and the rest we can use. + */ + tjit->stack_size -= STACK_FRAME_OVERHEAD - sizeof(u64); tjit->orig_stack_args_off = tjit->stack_size + STACK_FRAME_OVERHEAD; + /* lgr %r1,%r15 */ + EMIT4(0xb9040000, REG_1, REG_15); /* aghi %r15,-stack_size */ EMIT4_IMM(0xa70b0000, REG_15, -tjit->stack_size); + /* stg %r1,backchain_off(%r15) */ + EMIT6_DISP_LH(0xe3000000, 0x0024, REG_1, REG_0, REG_15, + tjit->backchain_off); /* mvc tccnt_off(4,%r15),stack_size+STK_OFF_TCCNT(%r15) */ _EMIT6(0xd203f000 | tjit->tccnt_off, 0xf000 | (tjit->stack_size + STK_OFF_TCCNT)); @@ -2513,7 +2528,7 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, return -E2BIG; } - return ret; + return tjit.common.prg; } bool bpf_jit_supports_subprog_tailcalls(void) diff --git a/arch/s390/pci/pci_dma.c b/arch/s390/pci/pci_dma.c index 2d9b01d7ca4c..99209085c75b 100644 --- a/arch/s390/pci/pci_dma.c +++ b/arch/s390/pci/pci_dma.c @@ -564,6 +564,17 @@ static void s390_dma_unmap_sg(struct device *dev, struct scatterlist *sg, s->dma_length = 0; } } + +static unsigned long *bitmap_vzalloc(size_t bits, gfp_t flags) +{ + size_t n = BITS_TO_LONGS(bits); + size_t bytes; + + if (unlikely(check_mul_overflow(n, sizeof(unsigned long), &bytes))) + return NULL; + + return vzalloc(bytes); +} int zpci_dma_init_device(struct zpci_dev *zdev) { @@ -604,13 +615,13 @@ int zpci_dma_init_device(struct zpci_dev *zdev) zdev->end_dma - zdev->start_dma + 1); zdev->end_dma = zdev->start_dma + zdev->iommu_size - 1; zdev->iommu_pages = zdev->iommu_size >> PAGE_SHIFT; - zdev->iommu_bitmap = vzalloc(zdev->iommu_pages / 8); + zdev->iommu_bitmap = bitmap_vzalloc(zdev->iommu_pages, GFP_KERNEL); if (!zdev->iommu_bitmap) { rc = -ENOMEM; goto free_dma_table; } if (!s390_iommu_strict) { - zdev->lazy_bitmap = vzalloc(zdev->iommu_pages / 8); + zdev->lazy_bitmap = bitmap_vzalloc(zdev->iommu_pages, GFP_KERNEL); if (!zdev->lazy_bitmap) { rc = -ENOMEM; goto free_bitmap; diff --git a/arch/sh/mm/ioremap.c b/arch/sh/mm/ioremap.c index c33b3daa4ad1..33d20f34560f 100644 --- a/arch/sh/mm/ioremap.c +++ b/arch/sh/mm/ioremap.c @@ -72,8 +72,8 @@ __ioremap_29bit(phys_addr_t offset, unsigned long size, pgprot_t prot) #define __ioremap_29bit(offset, size, prot) NULL #endif /* CONFIG_29BIT */ -void __iomem *ioremap_prot(phys_addr_t phys_addr, size_t size, - unsigned long prot) +void __iomem __ref *ioremap_prot(phys_addr_t phys_addr, size_t size, + unsigned long prot) { void __iomem *mapped; pgprot_t pgprot = __pgprot(prot); diff --git a/arch/sparc/include/asm/hugetlb.h b/arch/sparc/include/asm/hugetlb.h index 0a26cca24232..c714ca6a05aa 100644 --- a/arch/sparc/include/asm/hugetlb.h +++ b/arch/sparc/include/asm/hugetlb.h @@ -14,6 +14,8 @@ extern struct pud_huge_patch_entry __pud_huge_patch, __pud_huge_patch_end; #define __HAVE_ARCH_HUGE_SET_HUGE_PTE_AT void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, pte_t pte, unsigned long sz); +void __set_huge_pte_at(struct mm_struct *mm, unsigned long addr, pte_t *ptep, pte_t pte); #define __HAVE_ARCH_HUGE_PTEP_GET_AND_CLEAR @@ -32,7 +34,7 @@ static inline void huge_ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr, pte_t *ptep) { pte_t old_pte = *ptep; - set_huge_pte_at(mm, addr, ptep, pte_wrprotect(old_pte)); + __set_huge_pte_at(mm, addr, ptep, pte_wrprotect(old_pte)); } #define __HAVE_ARCH_HUGE_PTEP_SET_ACCESS_FLAGS @@ -42,7 +44,7 @@ static inline int huge_ptep_set_access_flags(struct vm_area_struct *vma, { int changed = !pte_same(*ptep, pte); if (changed) { - set_huge_pte_at(vma->vm_mm, addr, ptep, pte); + __set_huge_pte_at(vma->vm_mm, addr, ptep, pte); flush_tlb_page(vma, addr); } return changed; diff --git a/arch/sparc/mm/hugetlbpage.c b/arch/sparc/mm/hugetlbpage.c index d7018823206c..b432500c13a5 100644 --- a/arch/sparc/mm/hugetlbpage.c +++ b/arch/sparc/mm/hugetlbpage.c @@ -328,7 +328,7 @@ pte_t *huge_pte_offset(struct mm_struct *mm, return pte_offset_huge(pmd, addr); } -void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, +void __set_huge_pte_at(struct mm_struct *mm, unsigned long addr, pte_t *ptep, pte_t entry) { unsigned int nptes, orig_shift, shift; @@ -364,6 +364,12 @@ void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, orig_shift); } +void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, pte_t entry, unsigned long sz) +{ + __set_huge_pte_at(mm, addr, ptep, entry); +} + pte_t huge_ptep_get_and_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep) { diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 982b777eadc7..66bfabae8814 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -1945,6 +1945,7 @@ config EFI select UCS2_STRING select EFI_RUNTIME_WRAPPERS select ARCH_USE_MEMREMAP_PROT + select EFI_RUNTIME_MAP if KEXEC_CORE help This enables the kernel to use EFI runtime services that are available (such as the EFI variable services). @@ -2020,7 +2021,6 @@ config EFI_MAX_FAKE_MEM config EFI_RUNTIME_MAP bool "Export EFI runtime maps to sysfs" if EXPERT depends on EFI - default KEXEC_CORE help Export EFI runtime memory regions to /sys/firmware/efi/runtime-map. That memory map is required by the 2nd kernel to set up EFI virtual diff --git a/arch/x86/boot/compressed/ident_map_64.c b/arch/x86/boot/compressed/ident_map_64.c index bcc956c17872..08f93b0401bb 100644 --- a/arch/x86/boot/compressed/ident_map_64.c +++ b/arch/x86/boot/compressed/ident_map_64.c @@ -59,6 +59,14 @@ static void *alloc_pgt_page(void *context) return NULL; } + /* Consumed more tables than expected? */ + if (pages->pgt_buf_offset == BOOT_PGT_SIZE_WARN) { + debug_putstr("pgt_buf running low in " __FILE__ "\n"); + debug_putstr("Need to raise BOOT_PGT_SIZE?\n"); + debug_putaddr(pages->pgt_buf_offset); + debug_putaddr(pages->pgt_buf_size); + } + entry = pages->pgt_buf + pages->pgt_buf_offset; pages->pgt_buf_offset += PAGE_SIZE; diff --git a/arch/x86/boot/compressed/sev.c b/arch/x86/boot/compressed/sev.c index dc8c876fbd8f..80d76aea1f7b 100644 --- a/arch/x86/boot/compressed/sev.c +++ b/arch/x86/boot/compressed/sev.c @@ -103,6 +103,16 @@ static enum es_result vc_read_mem(struct es_em_ctxt *ctxt, return ES_OK; } +static enum es_result vc_ioio_check(struct es_em_ctxt *ctxt, u16 port, size_t size) +{ + return ES_OK; +} + +static bool fault_in_kernel_space(unsigned long address) +{ + return false; +} + #undef __init #define __init diff --git a/arch/x86/entry/common.c b/arch/x86/entry/common.c index 6c2826417b33..93c60c0c9d4a 100644 --- a/arch/x86/entry/common.c +++ b/arch/x86/entry/common.c @@ -294,7 +294,7 @@ static void __xen_pv_evtchn_do_upcall(struct pt_regs *regs) inc_irq_stat(irq_hv_callback_count); - xen_hvm_evtchn_do_upcall(); + xen_evtchn_do_upcall(); set_irq_regs(old_regs); } diff --git a/arch/x86/events/amd/core.c b/arch/x86/events/amd/core.c index abadd5f23425..e24976593a29 100644 --- a/arch/x86/events/amd/core.c +++ b/arch/x86/events/amd/core.c @@ -534,8 +534,12 @@ static void amd_pmu_cpu_reset(int cpu) /* Clear enable bits i.e. PerfCntrGlobalCtl.PerfCntrEn */ wrmsrl(MSR_AMD64_PERF_CNTR_GLOBAL_CTL, 0); - /* Clear overflow bits i.e. PerfCntrGLobalStatus.PerfCntrOvfl */ - wrmsrl(MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_CLR, amd_pmu_global_cntr_mask); + /* + * Clear freeze and overflow bits i.e. PerfCntrGLobalStatus.LbrFreeze + * and PerfCntrGLobalStatus.PerfCntrOvfl + */ + wrmsrl(MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_CLR, + GLOBAL_STATUS_LBRS_FROZEN | amd_pmu_global_cntr_mask); } static int amd_pmu_cpu_prepare(int cpu) @@ -570,6 +574,7 @@ static void amd_pmu_cpu_starting(int cpu) int i, nb_id; cpuc->perf_ctr_virt_mask = AMD64_EVENTSEL_HOSTONLY; + amd_pmu_cpu_reset(cpu); if (!x86_pmu.amd_nb_constraints) return; @@ -591,8 +596,6 @@ static void amd_pmu_cpu_starting(int cpu) cpuc->amd_nb->nb_id = nb_id; cpuc->amd_nb->refcnt++; - - amd_pmu_cpu_reset(cpu); } static void amd_pmu_cpu_dead(int cpu) @@ -601,6 +604,7 @@ static void amd_pmu_cpu_dead(int cpu) kfree(cpuhw->lbr_sel); cpuhw->lbr_sel = NULL; + amd_pmu_cpu_reset(cpu); if (!x86_pmu.amd_nb_constraints) return; @@ -613,8 +617,6 @@ static void amd_pmu_cpu_dead(int cpu) cpuhw->amd_nb = NULL; } - - amd_pmu_cpu_reset(cpu); } static inline void amd_pmu_set_global_ctl(u64 ctl) @@ -884,7 +886,7 @@ static int amd_pmu_v2_handle_irq(struct pt_regs *regs) struct hw_perf_event *hwc; struct perf_event *event; int handled = 0, idx; - u64 status, mask; + u64 reserved, status, mask; bool pmu_enabled; /* @@ -909,6 +911,14 @@ static int amd_pmu_v2_handle_irq(struct pt_regs *regs) status &= ~GLOBAL_STATUS_LBRS_FROZEN; } + reserved = status & ~amd_pmu_global_cntr_mask; + if (reserved) + pr_warn_once("Reserved PerfCntrGlobalStatus bits are set (0x%llx), please consider updating microcode\n", + reserved); + + /* Clear any reserved bits set by buggy microcode */ + status &= amd_pmu_global_cntr_mask; + for (idx = 0; idx < x86_pmu.num_counters; idx++) { if (!test_bit(idx, cpuc->active_mask)) continue; diff --git a/arch/x86/events/utils.c b/arch/x86/events/utils.c index 76b1f8bb0fd5..dab4ed199227 100644 --- a/arch/x86/events/utils.c +++ b/arch/x86/events/utils.c @@ -1,5 +1,6 @@ // SPDX-License-Identifier: GPL-2.0 #include +#include #include "perf_event.h" @@ -132,9 +133,9 @@ static int get_branch_type(unsigned long from, unsigned long to, int abort, * The LBR logs any address in the IP, even if the IP just * faulted. This means userspace can control the from address. * Ensure we don't blindly read any address by validating it is - * a known text address. + * a known text address and not a vsyscall address. */ - if (kernel_text_address(from)) { + if (kernel_text_address(from) && !in_gate_area_no_mm(from)) { addr = (void *)from; /* * Assume we can get the maximum possible size diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c index 783ed339f341..21556ad87f4b 100644 --- a/arch/x86/hyperv/hv_init.c +++ b/arch/x86/hyperv/hv_init.c @@ -7,6 +7,8 @@ * Author : K. Y. Srinivasan */ +#define pr_fmt(fmt) "Hyper-V: " fmt + #include #include #include @@ -191,7 +193,7 @@ void set_hv_tscchange_cb(void (*cb)(void)) struct hv_tsc_emulation_control emu_ctrl = {.enabled = 1}; if (!hv_reenlightenment_available()) { - pr_warn("Hyper-V: reenlightenment support is unavailable\n"); + pr_warn("reenlightenment support is unavailable\n"); return; } @@ -394,6 +396,7 @@ static void __init hv_get_partition_id(void) local_irq_restore(flags); } +#if IS_ENABLED(CONFIG_HYPERV_VTL_MODE) static u8 __init get_vtl(void) { u64 control = HV_HYPERCALL_REP_COMP_1 | HVCALL_GET_VP_REGISTERS; @@ -416,13 +419,16 @@ static u8 __init get_vtl(void) if (hv_result_success(ret)) { ret = output->as64.low & HV_X64_VTL_MASK; } else { - pr_err("Failed to get VTL(%lld) and set VTL to zero by default.\n", ret); - ret = 0; + pr_err("Failed to get VTL(error: %lld) exiting...\n", ret); + BUG(); } local_irq_restore(flags); return ret; } +#else +static inline u8 get_vtl(void) { return 0; } +#endif /* * This function is to be invoked early in the boot sequence after the @@ -564,7 +570,7 @@ skip_hypercall_pg_init: if (cpu_feature_enabled(X86_FEATURE_IBT) && *(u32 *)hv_hypercall_pg != gen_endbr()) { setup_clear_cpu_cap(X86_FEATURE_IBT); - pr_warn("Hyper-V: Disabling IBT because of Hyper-V bug\n"); + pr_warn("Disabling IBT because of Hyper-V bug\n"); } #endif @@ -604,8 +610,10 @@ skip_hypercall_pg_init: hv_query_ext_cap(0); /* Find the VTL */ - if (!ms_hyperv.paravisor_present && hv_isolation_type_snp()) - ms_hyperv.vtl = get_vtl(); + ms_hyperv.vtl = get_vtl(); + + if (ms_hyperv.vtl > 0) /* non default VTL */ + hv_vtl_early_init(); return; diff --git a/arch/x86/hyperv/hv_vtl.c b/arch/x86/hyperv/hv_vtl.c index 36a562218010..999f5ac82fe9 100644 --- a/arch/x86/hyperv/hv_vtl.c +++ b/arch/x86/hyperv/hv_vtl.c @@ -215,7 +215,7 @@ static int hv_vtl_wakeup_secondary_cpu(int apicid, unsigned long start_eip) return hv_vtl_bringup_vcpu(vp_id, start_eip); } -static int __init hv_vtl_early_init(void) +int __init hv_vtl_early_init(void) { /* * `boot_cpu_has` returns the runtime feature support, @@ -230,4 +230,3 @@ static int __init hv_vtl_early_init(void) return 0; } -early_initcall(hv_vtl_early_init); diff --git a/arch/x86/include/asm/boot.h b/arch/x86/include/asm/boot.h index 4ae14339cb8c..b3a7cfb0d99e 100644 --- a/arch/x86/include/asm/boot.h +++ b/arch/x86/include/asm/boot.h @@ -40,23 +40,40 @@ #ifdef CONFIG_X86_64 # define BOOT_STACK_SIZE 0x4000 +/* + * Used by decompressor's startup_32() to allocate page tables for identity + * mapping of the 4G of RAM in 4-level paging mode: + * - 1 level4 table; + * - 1 level3 table; + * - 4 level2 table that maps everything with 2M pages; + * + * The additional level5 table needed for 5-level paging is allocated from + * trampoline_32bit memory. + */ # define BOOT_INIT_PGT_SIZE (6*4096) -# ifdef CONFIG_RANDOMIZE_BASE + /* - * Assuming all cross the 512GB boundary: - * 1 page for level4 - * (2+2)*4 pages for kernel, param, cmd_line, and randomized kernel - * 2 pages for first 2M (video RAM: CONFIG_X86_VERBOSE_BOOTUP). - * Total is 19 pages. + * Total number of page tables kernel_add_identity_map() can allocate, + * including page tables consumed by startup_32(). + * + * Worst-case scenario: + * - 5-level paging needs 1 level5 table; + * - KASLR needs to map kernel, boot_params, cmdline and randomized kernel, + * assuming all of them cross 256T boundary: + * + 4*2 level4 table; + * + 4*2 level3 table; + * + 4*2 level2 table; + * - X86_VERBOSE_BOOTUP needs to map the first 2M (video RAM): + * + 1 level4 table; + * + 1 level3 table; + * + 1 level2 table; + * Total: 28 tables + * + * Add 4 spare table in case decompressor touches anything beyond what is + * accounted above. Warn if it happens. */ -# ifdef CONFIG_X86_VERBOSE_BOOTUP -# define BOOT_PGT_SIZE (19*4096) -# else /* !CONFIG_X86_VERBOSE_BOOTUP */ -# define BOOT_PGT_SIZE (17*4096) -# endif -# else /* !CONFIG_RANDOMIZE_BASE */ -# define BOOT_PGT_SIZE BOOT_INIT_PGT_SIZE -# endif +# define BOOT_PGT_SIZE_WARN (28*4096) +# define BOOT_PGT_SIZE (32*4096) #else /* !CONFIG_X86_64 */ # define BOOT_STACK_SIZE 0x1000 diff --git a/arch/x86/include/asm/cpu.h b/arch/x86/include/asm/cpu.h index 3a233ebff712..25050d953eee 100644 --- a/arch/x86/include/asm/cpu.h +++ b/arch/x86/include/asm/cpu.h @@ -28,8 +28,6 @@ struct x86_cpu { }; #ifdef CONFIG_HOTPLUG_CPU -extern int arch_register_cpu(int num); -extern void arch_unregister_cpu(int); extern void soft_restart_cpu(void); #endif diff --git a/arch/x86/include/asm/efi.h b/arch/x86/include/asm/efi.h index b0994ae3bc23..c4555b269a1b 100644 --- a/arch/x86/include/asm/efi.h +++ b/arch/x86/include/asm/efi.h @@ -91,19 +91,6 @@ static inline void efi_fpu_end(void) #ifdef CONFIG_X86_32 #define EFI_X86_KERNEL_ALLOC_LIMIT (SZ_512M - 1) - -#define arch_efi_call_virt_setup() \ -({ \ - efi_fpu_begin(); \ - firmware_restrict_branch_speculation_start(); \ -}) - -#define arch_efi_call_virt_teardown() \ -({ \ - firmware_restrict_branch_speculation_end(); \ - efi_fpu_end(); \ -}) - #else /* !CONFIG_X86_32 */ #define EFI_X86_KERNEL_ALLOC_LIMIT EFI_ALLOC_LIMIT @@ -116,14 +103,6 @@ extern bool efi_disable_ibt_for_runtime; __efi_call(__VA_ARGS__); \ }) -#define arch_efi_call_virt_setup() \ -({ \ - efi_sync_low_kernel_mappings(); \ - efi_fpu_begin(); \ - firmware_restrict_branch_speculation_start(); \ - efi_enter_mm(); \ -}) - #undef arch_efi_call_virt #define arch_efi_call_virt(p, f, args...) ({ \ u64 ret, ibt = ibt_save(efi_disable_ibt_for_runtime); \ @@ -132,13 +111,6 @@ extern bool efi_disable_ibt_for_runtime; ret; \ }) -#define arch_efi_call_virt_teardown() \ -({ \ - efi_leave_mm(); \ - firmware_restrict_branch_speculation_end(); \ - efi_fpu_end(); \ -}) - #ifdef CONFIG_KASAN /* * CONFIG_KASAN may redefine memset to __memset. __memset function is present @@ -168,8 +140,8 @@ extern void efi_delete_dummy_variable(void); extern void efi_crash_gracefully_on_page_fault(unsigned long phys_addr); extern void efi_free_boot_services(void); -void efi_enter_mm(void); -void efi_leave_mm(void); +void arch_efi_call_virt_setup(void); +void arch_efi_call_virt_teardown(void); /* kexec external ABI */ struct efi_setup_data { diff --git a/arch/x86/include/asm/fpu/api.h b/arch/x86/include/asm/fpu/api.h index 31089b851c4f..a2be3aefff9f 100644 --- a/arch/x86/include/asm/fpu/api.h +++ b/arch/x86/include/asm/fpu/api.h @@ -157,7 +157,8 @@ static inline void fpu_update_guest_xfd(struct fpu_guest *guest_fpu, u64 xfd) { static inline void fpu_sync_guest_vmexit_xfd_state(void) { } #endif -extern void fpu_copy_guest_fpstate_to_uabi(struct fpu_guest *gfpu, void *buf, unsigned int size, u32 pkru); +extern void fpu_copy_guest_fpstate_to_uabi(struct fpu_guest *gfpu, void *buf, + unsigned int size, u64 xfeatures, u32 pkru); extern int fpu_copy_uabi_to_guest_fpstate(struct fpu_guest *gfpu, const void *buf, u64 xcr0, u32 *vpkru); static inline void fpstate_set_confidential(struct fpu_guest *gfpu) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 1a4def36d5bb..70d139406bc8 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -528,7 +528,6 @@ struct kvm_pmu { u64 raw_event_mask; struct kvm_pmc gp_counters[KVM_INTEL_PMC_MAX_GENERIC]; struct kvm_pmc fixed_counters[KVM_PMC_MAX_FIXED]; - struct irq_work irq_work; /* * Overlay the bitmap with a 64-bit atomic so that all bits can be @@ -1419,7 +1418,6 @@ struct kvm_arch { * the thread holds the MMU lock in write mode. */ spinlock_t tdp_mmu_pages_lock; - struct workqueue_struct *tdp_mmu_zap_wq; #endif /* CONFIG_X86_64 */ /* @@ -1835,7 +1833,7 @@ void kvm_mmu_vendor_module_exit(void); void kvm_mmu_destroy(struct kvm_vcpu *vcpu); int kvm_mmu_create(struct kvm_vcpu *vcpu); -int kvm_mmu_init_vm(struct kvm *kvm); +void kvm_mmu_init_vm(struct kvm *kvm); void kvm_mmu_uninit_vm(struct kvm *kvm); void kvm_mmu_after_set_cpuid(struct kvm_vcpu *vcpu); diff --git a/arch/x86/include/asm/linkage.h b/arch/x86/include/asm/linkage.h index 97a3de7892d3..571fe4d2d232 100644 --- a/arch/x86/include/asm/linkage.h +++ b/arch/x86/include/asm/linkage.h @@ -8,6 +8,14 @@ #undef notrace #define notrace __attribute__((no_instrument_function)) +#ifdef CONFIG_64BIT +/* + * The generic version tends to create spurious ENDBR instructions under + * certain conditions. + */ +#define _THIS_IP_ ({ unsigned long __here; asm ("lea 0(%%rip), %0" : "=r" (__here)); __here; }) +#endif + #ifdef CONFIG_X86_32 #define asmlinkage CPP_ASMLINKAGE __attribute__((regparm(0))) #endif /* CONFIG_X86_32 */ @@ -97,6 +105,13 @@ CFI_POST_PADDING \ SYM_FUNC_END(__cfi_##name) +/* UML needs to be able to override memcpy() and friends for KASAN. */ +#ifdef CONFIG_UML +# define SYM_FUNC_ALIAS_MEMFUNC SYM_FUNC_ALIAS_WEAK +#else +# define SYM_FUNC_ALIAS_MEMFUNC SYM_FUNC_ALIAS +#endif + /* SYM_TYPED_FUNC_START -- use for indirectly called globals, w/ CFI type */ #define SYM_TYPED_FUNC_START(name) \ SYM_TYPED_START(name, SYM_L_GLOBAL, SYM_F_ALIGN) \ diff --git a/arch/x86/include/asm/mmu_context.h b/arch/x86/include/asm/mmu_context.h index 416901d406f8..8dac45a2c7fc 100644 --- a/arch/x86/include/asm/mmu_context.h +++ b/arch/x86/include/asm/mmu_context.h @@ -186,8 +186,7 @@ do { \ #else #define deactivate_mm(tsk, mm) \ do { \ - if (!tsk->vfork_done) \ - shstk_free(tsk); \ + shstk_free(tsk); \ load_gs_index(0); \ loadsegment(fs, 0); \ } while (0) diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyperv.h index 033b53f993c6..896445edc6a8 100644 --- a/arch/x86/include/asm/mshyperv.h +++ b/arch/x86/include/asm/mshyperv.h @@ -340,8 +340,10 @@ static inline u64 hv_get_non_nested_register(unsigned int reg) { return 0; } #ifdef CONFIG_HYPERV_VTL_MODE void __init hv_vtl_init_platform(void); +int __init hv_vtl_early_init(void); #else static inline void __init hv_vtl_init_platform(void) {} +static inline int __init hv_vtl_early_init(void) { return 0; } #endif #include diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h index 1d111350197f..b37abb55e948 100644 --- a/arch/x86/include/asm/msr-index.h +++ b/arch/x86/include/asm/msr-index.h @@ -637,12 +637,17 @@ /* AMD Last Branch Record MSRs */ #define MSR_AMD64_LBR_SELECT 0xc000010e -/* Fam 17h MSRs */ -#define MSR_F17H_IRPERF 0xc00000e9 +/* Zen4 */ +#define MSR_ZEN4_BP_CFG 0xc001102e +#define MSR_ZEN4_BP_CFG_SHARED_BTB_FIX_BIT 5 +/* Zen 2 */ #define MSR_ZEN2_SPECTRAL_CHICKEN 0xc00110e3 #define MSR_ZEN2_SPECTRAL_CHICKEN_BIT BIT_ULL(1) +/* Fam 17h MSRs */ +#define MSR_F17H_IRPERF 0xc00000e9 + /* Fam 16h MSRs */ #define MSR_F16H_L2I_PERF_CTL 0xc0010230 #define MSR_F16H_L2I_PERF_CTR 0xc0010231 diff --git a/arch/x86/include/asm/paravirt_types.h b/arch/x86/include/asm/paravirt_types.h index 4acbcddddc29..772d03487520 100644 --- a/arch/x86/include/asm/paravirt_types.h +++ b/arch/x86/include/asm/paravirt_types.h @@ -9,13 +9,6 @@ struct paravirt_patch_site { u8 type; /* type of this instruction */ u8 len; /* length of original instruction */ }; - -/* Lazy mode for batching updates / context switch */ -enum paravirt_lazy_mode { - PARAVIRT_LAZY_NONE, - PARAVIRT_LAZY_MMU, - PARAVIRT_LAZY_CPU, -}; #endif #ifdef CONFIG_PARAVIRT @@ -549,14 +542,6 @@ int paravirt_disable_iospace(void); __PVOP_VCALL(op, PVOP_CALL_ARG1(arg1), PVOP_CALL_ARG2(arg2), \ PVOP_CALL_ARG3(arg3), PVOP_CALL_ARG4(arg4)) -enum paravirt_lazy_mode paravirt_get_lazy_mode(void); -void paravirt_start_context_switch(struct task_struct *prev); -void paravirt_end_context_switch(struct task_struct *next); - -void paravirt_enter_lazy_mmu(void); -void paravirt_leave_lazy_mmu(void); -void paravirt_flush_lazy_mmu(void); - void _paravirt_nop(void); void paravirt_BUG(void); unsigned long paravirt_ret0(void); diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h index d6ad98ca1288..e02b179ec659 100644 --- a/arch/x86/include/asm/pgtable.h +++ b/arch/x86/include/asm/pgtable.h @@ -955,6 +955,14 @@ static inline int pte_same(pte_t a, pte_t b) return a.pte == b.pte; } +static inline pte_t pte_next_pfn(pte_t pte) +{ + if (__pte_needs_invert(pte_val(pte))) + return __pte(pte_val(pte) - (1UL << PFN_PTE_SHIFT)); + return __pte(pte_val(pte) + (1UL << PFN_PTE_SHIFT)); +} +#define pte_next_pfn pte_next_pfn + static inline int pte_present(pte_t a) { return pte_flags(a) & (_PAGE_PRESENT | _PAGE_PROTNONE); diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/processor.h index 0086920cda06..a3669a7774ed 100644 --- a/arch/x86/include/asm/processor.h +++ b/arch/x86/include/asm/processor.h @@ -683,13 +683,11 @@ extern u16 get_llc_id(unsigned int cpu); #ifdef CONFIG_CPU_SUP_AMD extern u32 amd_get_nodes_per_socket(void); extern u32 amd_get_highest_perf(void); -extern bool cpu_has_ibpb_brtype_microcode(void); extern void amd_clear_divider(void); extern void amd_check_microcode(void); #else static inline u32 amd_get_nodes_per_socket(void) { return 0; } static inline u32 amd_get_highest_perf(void) { return 0; } -static inline bool cpu_has_ibpb_brtype_microcode(void) { return false; } static inline void amd_clear_divider(void) { } static inline void amd_check_microcode(void) { } #endif diff --git a/arch/x86/include/asm/smp.h b/arch/x86/include/asm/smp.h index ad98dd1d9cfb..c31c633419fe 100644 --- a/arch/x86/include/asm/smp.h +++ b/arch/x86/include/asm/smp.h @@ -129,7 +129,6 @@ void native_smp_send_reschedule(int cpu); void native_send_call_func_ipi(const struct cpumask *mask); void native_send_call_func_single_ipi(int cpu); -bool smp_park_other_cpus_in_init(void); void smp_store_cpu_info(int id); asmlinkage __visible void smp_reboot_interrupt(void); diff --git a/arch/x86/include/asm/svm.h b/arch/x86/include/asm/svm.h index 19bf955b67e0..3ac0ffc4f3e2 100644 --- a/arch/x86/include/asm/svm.h +++ b/arch/x86/include/asm/svm.h @@ -268,6 +268,7 @@ enum avic_ipi_failure_cause { AVIC_IPI_FAILURE_TARGET_NOT_RUNNING, AVIC_IPI_FAILURE_INVALID_TARGET, AVIC_IPI_FAILURE_INVALID_BACKING_PAGE, + AVIC_IPI_FAILURE_INVALID_IPI_VECTOR, }; #define AVIC_PHYSICAL_MAX_INDEX_MASK GENMASK_ULL(8, 0) diff --git a/arch/x86/include/asm/xen/hypervisor.h b/arch/x86/include/asm/xen/hypervisor.h index 5fc35f889cd1..7048dfacc04b 100644 --- a/arch/x86/include/asm/xen/hypervisor.h +++ b/arch/x86/include/asm/xen/hypervisor.h @@ -36,6 +36,7 @@ extern struct shared_info *HYPERVISOR_shared_info; extern struct start_info *xen_start_info; +#include #include #define XEN_SIGNATURE "XenVMMXenVMM" @@ -63,4 +64,40 @@ void __init xen_pvh_init(struct boot_params *boot_params); void __init mem_map_via_hcall(struct boot_params *boot_params_p); #endif +/* Lazy mode for batching updates / context switch */ +enum xen_lazy_mode { + XEN_LAZY_NONE, + XEN_LAZY_MMU, + XEN_LAZY_CPU, +}; + +DECLARE_PER_CPU(enum xen_lazy_mode, xen_lazy_mode); +DECLARE_PER_CPU(unsigned int, xen_lazy_nesting); + +static inline void enter_lazy(enum xen_lazy_mode mode) +{ + enum xen_lazy_mode old_mode = this_cpu_read(xen_lazy_mode); + + if (mode == old_mode) { + this_cpu_inc(xen_lazy_nesting); + return; + } + + BUG_ON(old_mode != XEN_LAZY_NONE); + + this_cpu_write(xen_lazy_mode, mode); +} + +static inline void leave_lazy(enum xen_lazy_mode mode) +{ + BUG_ON(this_cpu_read(xen_lazy_mode) != mode); + + if (this_cpu_read(xen_lazy_nesting) == 0) + this_cpu_write(xen_lazy_mode, XEN_LAZY_NONE); + else + this_cpu_dec(xen_lazy_nesting); +} + +enum xen_lazy_mode xen_get_lazy_mode(void); + #endif /* _ASM_X86_XEN_HYPERVISOR_H */ diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c index a5ead6a6d233..73be3931e4f0 100644 --- a/arch/x86/kernel/alternative.c +++ b/arch/x86/kernel/alternative.c @@ -403,6 +403,17 @@ void __init_or_module noinline apply_alternatives(struct alt_instr *start, u8 insn_buff[MAX_PATCH_LEN]; DPRINTK(ALT, "alt table %px, -> %px", start, end); + + /* + * In the case CONFIG_X86_5LEVEL=y, KASAN_SHADOW_START is defined using + * cpu_feature_enabled(X86_FEATURE_LA57) and is therefore patched here. + * During the process, KASAN becomes confused seeing partial LA57 + * conversion and triggers a false-positive out-of-bound report. + * + * Disable KASAN until the patching is complete. + */ + kasan_disable_current(); + /* * The scan order should be from start to end. A later scanned * alternative code can overwrite previously scanned alternative code. @@ -452,6 +463,8 @@ void __init_or_module noinline apply_alternatives(struct alt_instr *start, text_poke_early(instr, insn_buff, insn_buff_sz); } + + kasan_enable_current(); } static inline bool is_jcc32(struct insn *insn) @@ -720,13 +733,8 @@ void __init_or_module noinline apply_returns(s32 *start, s32 *end) { s32 *s; - /* - * Do not patch out the default return thunks if those needed are the - * ones generated by the compiler. - */ - if (cpu_feature_enabled(X86_FEATURE_RETHUNK) && - (x86_return_thunk == __x86_return_thunk)) - return; + if (cpu_feature_enabled(X86_FEATURE_RETHUNK)) + static_call_force_reinit(); for (s = start; s < end; s++) { void *dest = NULL, *addr = (void *)s + *s; diff --git a/arch/x86/kernel/apic/x2apic_uv_x.c b/arch/x86/kernel/apic/x2apic_uv_x.c index d9f5d7492f83..205cee567629 100644 --- a/arch/x86/kernel/apic/x2apic_uv_x.c +++ b/arch/x86/kernel/apic/x2apic_uv_x.c @@ -1533,7 +1533,7 @@ static void __init build_socket_tables(void) { struct uv_gam_range_entry *gre = uv_gre_table; int nums, numn, nump; - int cpu, i, lnid; + int i, lnid, apicid; int minsock = _min_socket; int maxsock = _max_socket; int minpnode = _min_pnode; @@ -1584,15 +1584,14 @@ static void __init build_socket_tables(void) /* Set socket -> node values: */ lnid = NUMA_NO_NODE; - for_each_possible_cpu(cpu) { - int nid = cpu_to_node(cpu); - int apicid, sockid; + for (apicid = 0; apicid < ARRAY_SIZE(__apicid_to_node); apicid++) { + int nid = __apicid_to_node[apicid]; + int sockid; - if (lnid == nid) + if ((nid == NUMA_NO_NODE) || (lnid == nid)) continue; lnid = nid; - apicid = per_cpu(x86_cpu_to_apicid, cpu); sockid = apicid >> uv_cpuid.socketid_shift; if (_socket_to_node[sockid - minsock] == SOCK_EMPTY) diff --git a/arch/x86/kernel/callthunks.c b/arch/x86/kernel/callthunks.c index c06bfc086565..faa9f2299848 100644 --- a/arch/x86/kernel/callthunks.c +++ b/arch/x86/kernel/callthunks.c @@ -272,7 +272,6 @@ void __init callthunks_patch_builtin_calls(void) pr_info("Setting up call depth tracking\n"); mutex_lock(&text_mutex); callthunks_setup(&cs, &builtin_coretext); - static_call_force_reinit(); thunks_initialized = true; mutex_unlock(&text_mutex); } diff --git a/arch/x86/kernel/cpu/amd.c b/arch/x86/kernel/cpu/amd.c index dd8379d84445..ece2b5b7b0fe 100644 --- a/arch/x86/kernel/cpu/amd.c +++ b/arch/x86/kernel/cpu/amd.c @@ -80,6 +80,10 @@ static const int amd_div0[] = AMD_LEGACY_ERRATUM(AMD_MODEL_RANGE(0x17, 0x00, 0x0, 0x2f, 0xf), AMD_MODEL_RANGE(0x17, 0x50, 0x0, 0x5f, 0xf)); +static const int amd_erratum_1485[] = + AMD_LEGACY_ERRATUM(AMD_MODEL_RANGE(0x19, 0x10, 0x0, 0x1f, 0xf), + AMD_MODEL_RANGE(0x19, 0x60, 0x0, 0xaf, 0xf)); + static bool cpu_has_amd_erratum(struct cpuinfo_x86 *cpu, const int *erratum) { int osvw_id = *erratum++; @@ -766,6 +770,15 @@ static void early_init_amd(struct cpuinfo_x86 *c) if (cpu_has(c, X86_FEATURE_TOPOEXT)) smp_num_siblings = ((cpuid_ebx(0x8000001e) >> 8) & 0xff) + 1; + + if (!cpu_has(c, X86_FEATURE_HYPERVISOR) && !cpu_has(c, X86_FEATURE_IBPB_BRTYPE)) { + if (c->x86 == 0x17 && boot_cpu_has(X86_FEATURE_AMD_IBPB)) + setup_force_cpu_cap(X86_FEATURE_IBPB_BRTYPE); + else if (c->x86 >= 0x19 && !wrmsrl_safe(MSR_IA32_PRED_CMD, PRED_CMD_SBPB)) { + setup_force_cpu_cap(X86_FEATURE_IBPB_BRTYPE); + setup_force_cpu_cap(X86_FEATURE_SBPB); + } + } } static void init_amd_k8(struct cpuinfo_x86 *c) @@ -1140,6 +1153,10 @@ static void init_amd(struct cpuinfo_x86 *c) pr_notice_once("AMD Zen1 DIV0 bug detected. Disable SMT for full protection.\n"); setup_force_cpu_bug(X86_BUG_DIV0); } + + if (!cpu_has(c, X86_FEATURE_HYPERVISOR) && + cpu_has_amd_erratum(c, amd_erratum_1485)) + msr_set_bit(MSR_ZEN4_BP_CFG, MSR_ZEN4_BP_CFG_SHARED_BTB_FIX_BIT); } #ifdef CONFIG_X86_32 @@ -1301,25 +1318,6 @@ void amd_check_microcode(void) on_each_cpu(zenbleed_check_cpu, NULL, 1); } -bool cpu_has_ibpb_brtype_microcode(void) -{ - switch (boot_cpu_data.x86) { - /* Zen1/2 IBPB flushes branch type predictions too. */ - case 0x17: - return boot_cpu_has(X86_FEATURE_AMD_IBPB); - case 0x19: - /* Poke the MSR bit on Zen3/4 to check its presence. */ - if (!wrmsrl_safe(MSR_IA32_PRED_CMD, PRED_CMD_SBPB)) { - setup_force_cpu_cap(X86_FEATURE_SBPB); - return true; - } else { - return false; - } - default: - return false; - } -} - /* * Issue a DIV 0/1 insn to clear any division data from previous DIV * operations. diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c index f081d26616ac..10499bcd4e39 100644 --- a/arch/x86/kernel/cpu/bugs.c +++ b/arch/x86/kernel/cpu/bugs.c @@ -2404,26 +2404,15 @@ early_param("spec_rstack_overflow", srso_parse_cmdline); static void __init srso_select_mitigation(void) { - bool has_microcode; + bool has_microcode = boot_cpu_has(X86_FEATURE_IBPB_BRTYPE); if (!boot_cpu_has_bug(X86_BUG_SRSO) || cpu_mitigations_off()) goto pred_cmd; - /* - * The first check is for the kernel running as a guest in order - * for guests to verify whether IBPB is a viable mitigation. - */ - has_microcode = boot_cpu_has(X86_FEATURE_IBPB_BRTYPE) || cpu_has_ibpb_brtype_microcode(); if (!has_microcode) { pr_warn("IBPB-extending microcode not applied!\n"); pr_warn(SRSO_NOTICE); } else { - /* - * Enable the synthetic (even if in a real CPUID leaf) - * flags for guests. - */ - setup_force_cpu_cap(X86_FEATURE_IBPB_BRTYPE); - /* * Zen1/2 with SMT off aren't vulnerable after the right * IBPB microcode has been applied. @@ -2444,7 +2433,7 @@ static void __init srso_select_mitigation(void) switch (srso_cmd) { case SRSO_CMD_OFF: - return; + goto pred_cmd; case SRSO_CMD_MICROCODE: if (has_microcode) { @@ -2717,7 +2706,7 @@ static ssize_t srso_show_state(char *buf) return sysfs_emit(buf, "%s%s\n", srso_strings[srso_mitigation], - (cpu_has_ibpb_brtype_microcode() ? "" : ", no microcode")); + boot_cpu_has(X86_FEATURE_IBPB_BRTYPE) ? "" : ", no microcode"); } static ssize_t gds_show_state(char *buf) diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index 382d4e6b848d..4e5ffc8b0e46 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -1303,7 +1303,7 @@ static const struct x86_cpu_id cpu_vuln_blacklist[] __initconst = { VULNBL_AMD(0x15, RETBLEED), VULNBL_AMD(0x16, RETBLEED), VULNBL_AMD(0x17, RETBLEED | SMT_RSB | SRSO), - VULNBL_HYGON(0x18, RETBLEED | SMT_RSB), + VULNBL_HYGON(0x18, RETBLEED | SMT_RSB | SRSO), VULNBL_AMD(0x19, SRSO), {} }; diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c index ded1fc7cb7cb..f136ac046851 100644 --- a/arch/x86/kernel/cpu/resctrl/monitor.c +++ b/arch/x86/kernel/cpu/resctrl/monitor.c @@ -30,15 +30,15 @@ struct rmid_entry { struct list_head list; }; -/** - * @rmid_free_lru A least recently used list of free RMIDs +/* + * @rmid_free_lru - A least recently used list of free RMIDs * These RMIDs are guaranteed to have an occupancy less than the * threshold occupancy */ static LIST_HEAD(rmid_free_lru); -/** - * @rmid_limbo_count count of currently unused but (potentially) +/* + * @rmid_limbo_count - count of currently unused but (potentially) * dirty RMIDs. * This counts RMIDs that no one is currently using but that * may have a occupancy value > resctrl_rmid_realloc_threshold. User can @@ -46,7 +46,7 @@ static LIST_HEAD(rmid_free_lru); */ static unsigned int rmid_limbo_count; -/** +/* * @rmid_entry - The entry in the limbo and free lists. */ static struct rmid_entry *rmid_ptrs; diff --git a/arch/x86/kernel/cpu/sgx/encl.c b/arch/x86/kernel/cpu/sgx/encl.c index 91fa70e51004..279148e72459 100644 --- a/arch/x86/kernel/cpu/sgx/encl.c +++ b/arch/x86/kernel/cpu/sgx/encl.c @@ -235,6 +235,21 @@ static struct sgx_epc_page *sgx_encl_eldu(struct sgx_encl_page *encl_page, return epc_page; } +/* + * Ensure the SECS page is not swapped out. Must be called with encl->lock + * to protect the enclave states including SECS and ensure the SECS page is + * not swapped out again while being used. + */ +static struct sgx_epc_page *sgx_encl_load_secs(struct sgx_encl *encl) +{ + struct sgx_epc_page *epc_page = encl->secs.epc_page; + + if (!epc_page) + epc_page = sgx_encl_eldu(&encl->secs, NULL); + + return epc_page; +} + static struct sgx_encl_page *__sgx_encl_load_page(struct sgx_encl *encl, struct sgx_encl_page *entry) { @@ -248,11 +263,9 @@ static struct sgx_encl_page *__sgx_encl_load_page(struct sgx_encl *encl, return entry; } - if (!(encl->secs.epc_page)) { - epc_page = sgx_encl_eldu(&encl->secs, NULL); - if (IS_ERR(epc_page)) - return ERR_CAST(epc_page); - } + epc_page = sgx_encl_load_secs(encl); + if (IS_ERR(epc_page)) + return ERR_CAST(epc_page); epc_page = sgx_encl_eldu(entry, encl->secs.epc_page); if (IS_ERR(epc_page)) @@ -339,6 +352,13 @@ static vm_fault_t sgx_encl_eaug_page(struct vm_area_struct *vma, mutex_lock(&encl->lock); + epc_page = sgx_encl_load_secs(encl); + if (IS_ERR(epc_page)) { + if (PTR_ERR(epc_page) == -EBUSY) + vmret = VM_FAULT_NOPAGE; + goto err_out_unlock; + } + epc_page = sgx_alloc_epc_page(encl_page, false); if (IS_ERR(epc_page)) { if (PTR_ERR(epc_page) == -EBUSY) diff --git a/arch/x86/kernel/fpu/core.c b/arch/x86/kernel/fpu/core.c index a86d37052a64..a21a4d0ecc34 100644 --- a/arch/x86/kernel/fpu/core.c +++ b/arch/x86/kernel/fpu/core.c @@ -369,14 +369,15 @@ int fpu_swap_kvm_fpstate(struct fpu_guest *guest_fpu, bool enter_guest) EXPORT_SYMBOL_GPL(fpu_swap_kvm_fpstate); void fpu_copy_guest_fpstate_to_uabi(struct fpu_guest *gfpu, void *buf, - unsigned int size, u32 pkru) + unsigned int size, u64 xfeatures, u32 pkru) { struct fpstate *kstate = gfpu->fpstate; union fpregs_state *ustate = buf; struct membuf mb = { .p = buf, .left = size }; if (cpu_feature_enabled(X86_FEATURE_XSAVE)) { - __copy_xstate_to_uabi_buf(mb, kstate, pkru, XSTATE_COPY_XSAVE); + __copy_xstate_to_uabi_buf(mb, kstate, xfeatures, pkru, + XSTATE_COPY_XSAVE); } else { memcpy(&ustate->fxsave, &kstate->regs.fxsave, sizeof(ustate->fxsave)); diff --git a/arch/x86/kernel/fpu/xstate.c b/arch/x86/kernel/fpu/xstate.c index cadf68737e6b..ef6906107c54 100644 --- a/arch/x86/kernel/fpu/xstate.c +++ b/arch/x86/kernel/fpu/xstate.c @@ -1049,6 +1049,7 @@ static void copy_feature(bool from_xstate, struct membuf *to, void *xstate, * __copy_xstate_to_uabi_buf - Copy kernel saved xstate to a UABI buffer * @to: membuf descriptor * @fpstate: The fpstate buffer from which to copy + * @xfeatures: The mask of xfeatures to save (XSAVE mode only) * @pkru_val: The PKRU value to store in the PKRU component * @copy_mode: The requested copy mode * @@ -1059,7 +1060,8 @@ static void copy_feature(bool from_xstate, struct membuf *to, void *xstate, * It supports partial copy but @to.pos always starts from zero. */ void __copy_xstate_to_uabi_buf(struct membuf to, struct fpstate *fpstate, - u32 pkru_val, enum xstate_copy_mode copy_mode) + u64 xfeatures, u32 pkru_val, + enum xstate_copy_mode copy_mode) { const unsigned int off_mxcsr = offsetof(struct fxregs_state, mxcsr); struct xregs_state *xinit = &init_fpstate.regs.xsave; @@ -1083,7 +1085,7 @@ void __copy_xstate_to_uabi_buf(struct membuf to, struct fpstate *fpstate, break; case XSTATE_COPY_XSAVE: - header.xfeatures &= fpstate->user_xfeatures; + header.xfeatures &= fpstate->user_xfeatures & xfeatures; break; } @@ -1185,6 +1187,7 @@ void copy_xstate_to_uabi_buf(struct membuf to, struct task_struct *tsk, enum xstate_copy_mode copy_mode) { __copy_xstate_to_uabi_buf(to, tsk->thread.fpu.fpstate, + tsk->thread.fpu.fpstate->user_xfeatures, tsk->thread.pkru, copy_mode); } @@ -1536,10 +1539,7 @@ static int fpstate_realloc(u64 xfeatures, unsigned int ksize, fpregs_restore_userregs(); newfps->xfeatures = curfps->xfeatures | xfeatures; - - if (!guest_fpu) - newfps->user_xfeatures = curfps->user_xfeatures | xfeatures; - + newfps->user_xfeatures = curfps->user_xfeatures | xfeatures; newfps->xfd = curfps->xfd & ~xfeatures; /* Do the final updates within the locked region */ diff --git a/arch/x86/kernel/fpu/xstate.h b/arch/x86/kernel/fpu/xstate.h index a4ecb04d8d64..3518fb26d06b 100644 --- a/arch/x86/kernel/fpu/xstate.h +++ b/arch/x86/kernel/fpu/xstate.h @@ -43,7 +43,8 @@ enum xstate_copy_mode { struct membuf; extern void __copy_xstate_to_uabi_buf(struct membuf to, struct fpstate *fpstate, - u32 pkru_val, enum xstate_copy_mode copy_mode); + u64 xfeatures, u32 pkru_val, + enum xstate_copy_mode copy_mode); extern void copy_xstate_to_uabi_buf(struct membuf to, struct task_struct *tsk, enum xstate_copy_mode mode); extern int copy_uabi_from_kernel_to_xstate(struct fpstate *fpstate, const void *kbuf, u32 *pkru); diff --git a/arch/x86/kernel/kgdb.c b/arch/x86/kernel/kgdb.c index 3a43a2dee658..9c9faa1634fb 100644 --- a/arch/x86/kernel/kgdb.c +++ b/arch/x86/kernel/kgdb.c @@ -695,7 +695,6 @@ void kgdb_arch_exit(void) } /** - * * kgdb_skipexception - Bail out of KGDB when we've been triggered. * @exception: Exception vector number * @regs: Current &struct pt_regs. diff --git a/arch/x86/kernel/paravirt.c b/arch/x86/kernel/paravirt.c index 975f98d5eee5..97f1436c1a20 100644 --- a/arch/x86/kernel/paravirt.c +++ b/arch/x86/kernel/paravirt.c @@ -143,66 +143,7 @@ int paravirt_disable_iospace(void) return request_resource(&ioport_resource, &reserve_ioports); } -static DEFINE_PER_CPU(enum paravirt_lazy_mode, paravirt_lazy_mode) = PARAVIRT_LAZY_NONE; - -static inline void enter_lazy(enum paravirt_lazy_mode mode) -{ - BUG_ON(this_cpu_read(paravirt_lazy_mode) != PARAVIRT_LAZY_NONE); - - this_cpu_write(paravirt_lazy_mode, mode); -} - -static void leave_lazy(enum paravirt_lazy_mode mode) -{ - BUG_ON(this_cpu_read(paravirt_lazy_mode) != mode); - - this_cpu_write(paravirt_lazy_mode, PARAVIRT_LAZY_NONE); -} - -void paravirt_enter_lazy_mmu(void) -{ - enter_lazy(PARAVIRT_LAZY_MMU); -} - -void paravirt_leave_lazy_mmu(void) -{ - leave_lazy(PARAVIRT_LAZY_MMU); -} - -void paravirt_flush_lazy_mmu(void) -{ - preempt_disable(); - - if (paravirt_get_lazy_mode() == PARAVIRT_LAZY_MMU) { - arch_leave_lazy_mmu_mode(); - arch_enter_lazy_mmu_mode(); - } - - preempt_enable(); -} - #ifdef CONFIG_PARAVIRT_XXL -void paravirt_start_context_switch(struct task_struct *prev) -{ - BUG_ON(preemptible()); - - if (this_cpu_read(paravirt_lazy_mode) == PARAVIRT_LAZY_MMU) { - arch_leave_lazy_mmu_mode(); - set_ti_thread_flag(task_thread_info(prev), TIF_LAZY_MMU_UPDATES); - } - enter_lazy(PARAVIRT_LAZY_CPU); -} - -void paravirt_end_context_switch(struct task_struct *next) -{ - BUG_ON(preemptible()); - - leave_lazy(PARAVIRT_LAZY_CPU); - - if (test_and_clear_ti_thread_flag(task_thread_info(next), TIF_LAZY_MMU_UPDATES)) - arch_enter_lazy_mmu_mode(); -} - static noinstr void pv_native_write_cr2(unsigned long val) { native_write_cr2(val); @@ -229,14 +170,6 @@ static noinstr void pv_native_safe_halt(void) } #endif -enum paravirt_lazy_mode paravirt_get_lazy_mode(void) -{ - if (in_interrupt()) - return PARAVIRT_LAZY_NONE; - - return this_cpu_read(paravirt_lazy_mode); -} - struct pv_info pv_info = { .name = "bare hardware", #ifdef CONFIG_PARAVIRT_XXL diff --git a/arch/x86/kernel/process.c b/arch/x86/kernel/process.c index 9f0909142a0a..b6f4e8399fca 100644 --- a/arch/x86/kernel/process.c +++ b/arch/x86/kernel/process.c @@ -257,13 +257,6 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args) if (!ret && unlikely(test_tsk_thread_flag(current, TIF_IO_BITMAP))) io_bitmap_share(p); - /* - * If copy_thread() if failing, don't leak the shadow stack possibly - * allocated in shstk_alloc_thread_stack() above. - */ - if (ret) - shstk_free(p); - return ret; } diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c index b9145a63da77..b098b1fa2470 100644 --- a/arch/x86/kernel/setup.c +++ b/arch/x86/kernel/setup.c @@ -358,15 +358,11 @@ static void __init add_early_ima_buffer(u64 phys_addr) #if defined(CONFIG_HAVE_IMA_KEXEC) && !defined(CONFIG_OF_FLATTREE) int __init ima_free_kexec_buffer(void) { - int rc; - if (!ima_kexec_buffer_size) return -ENOENT; - rc = memblock_phys_free(ima_kexec_buffer_phys, - ima_kexec_buffer_size); - if (rc) - return rc; + memblock_free_late(ima_kexec_buffer_phys, + ima_kexec_buffer_size); ima_kexec_buffer_phys = 0; ima_kexec_buffer_size = 0; diff --git a/arch/x86/kernel/sev-shared.c b/arch/x86/kernel/sev-shared.c index 2eabccde94fb..ccb0915e84e1 100644 --- a/arch/x86/kernel/sev-shared.c +++ b/arch/x86/kernel/sev-shared.c @@ -256,7 +256,7 @@ static int __sev_cpuid_hv(u32 fn, int reg_idx, u32 *reg) return 0; } -static int sev_cpuid_hv(struct cpuid_leaf *leaf) +static int __sev_cpuid_hv_msr(struct cpuid_leaf *leaf) { int ret; @@ -279,6 +279,45 @@ static int sev_cpuid_hv(struct cpuid_leaf *leaf) return ret; } +static int __sev_cpuid_hv_ghcb(struct ghcb *ghcb, struct es_em_ctxt *ctxt, struct cpuid_leaf *leaf) +{ + u32 cr4 = native_read_cr4(); + int ret; + + ghcb_set_rax(ghcb, leaf->fn); + ghcb_set_rcx(ghcb, leaf->subfn); + + if (cr4 & X86_CR4_OSXSAVE) + /* Safe to read xcr0 */ + ghcb_set_xcr0(ghcb, xgetbv(XCR_XFEATURE_ENABLED_MASK)); + else + /* xgetbv will cause #UD - use reset value for xcr0 */ + ghcb_set_xcr0(ghcb, 1); + + ret = sev_es_ghcb_hv_call(ghcb, ctxt, SVM_EXIT_CPUID, 0, 0); + if (ret != ES_OK) + return ret; + + if (!(ghcb_rax_is_valid(ghcb) && + ghcb_rbx_is_valid(ghcb) && + ghcb_rcx_is_valid(ghcb) && + ghcb_rdx_is_valid(ghcb))) + return ES_VMM_ERROR; + + leaf->eax = ghcb->save.rax; + leaf->ebx = ghcb->save.rbx; + leaf->ecx = ghcb->save.rcx; + leaf->edx = ghcb->save.rdx; + + return ES_OK; +} + +static int sev_cpuid_hv(struct ghcb *ghcb, struct es_em_ctxt *ctxt, struct cpuid_leaf *leaf) +{ + return ghcb ? __sev_cpuid_hv_ghcb(ghcb, ctxt, leaf) + : __sev_cpuid_hv_msr(leaf); +} + /* * This may be called early while still running on the initial identity * mapping. Use RIP-relative addressing to obtain the correct address @@ -388,19 +427,20 @@ snp_cpuid_get_validated_func(struct cpuid_leaf *leaf) return false; } -static void snp_cpuid_hv(struct cpuid_leaf *leaf) +static void snp_cpuid_hv(struct ghcb *ghcb, struct es_em_ctxt *ctxt, struct cpuid_leaf *leaf) { - if (sev_cpuid_hv(leaf)) + if (sev_cpuid_hv(ghcb, ctxt, leaf)) sev_es_terminate(SEV_TERM_SET_LINUX, GHCB_TERM_CPUID_HV); } -static int snp_cpuid_postprocess(struct cpuid_leaf *leaf) +static int snp_cpuid_postprocess(struct ghcb *ghcb, struct es_em_ctxt *ctxt, + struct cpuid_leaf *leaf) { struct cpuid_leaf leaf_hv = *leaf; switch (leaf->fn) { case 0x1: - snp_cpuid_hv(&leaf_hv); + snp_cpuid_hv(ghcb, ctxt, &leaf_hv); /* initial APIC ID */ leaf->ebx = (leaf_hv.ebx & GENMASK(31, 24)) | (leaf->ebx & GENMASK(23, 0)); @@ -419,7 +459,7 @@ static int snp_cpuid_postprocess(struct cpuid_leaf *leaf) break; case 0xB: leaf_hv.subfn = 0; - snp_cpuid_hv(&leaf_hv); + snp_cpuid_hv(ghcb, ctxt, &leaf_hv); /* extended APIC ID */ leaf->edx = leaf_hv.edx; @@ -467,7 +507,7 @@ static int snp_cpuid_postprocess(struct cpuid_leaf *leaf) } break; case 0x8000001E: - snp_cpuid_hv(&leaf_hv); + snp_cpuid_hv(ghcb, ctxt, &leaf_hv); /* extended APIC ID */ leaf->eax = leaf_hv.eax; @@ -488,7 +528,7 @@ static int snp_cpuid_postprocess(struct cpuid_leaf *leaf) * Returns -EOPNOTSUPP if feature not enabled. Any other non-zero return value * should be treated as fatal by caller. */ -static int snp_cpuid(struct cpuid_leaf *leaf) +static int snp_cpuid(struct ghcb *ghcb, struct es_em_ctxt *ctxt, struct cpuid_leaf *leaf) { const struct snp_cpuid_table *cpuid_table = snp_cpuid_get_table(); @@ -522,7 +562,7 @@ static int snp_cpuid(struct cpuid_leaf *leaf) return 0; } - return snp_cpuid_postprocess(leaf); + return snp_cpuid_postprocess(ghcb, ctxt, leaf); } /* @@ -544,14 +584,14 @@ void __init do_vc_no_ghcb(struct pt_regs *regs, unsigned long exit_code) leaf.fn = fn; leaf.subfn = subfn; - ret = snp_cpuid(&leaf); + ret = snp_cpuid(NULL, NULL, &leaf); if (!ret) goto cpuid_done; if (ret != -EOPNOTSUPP) goto fail; - if (sev_cpuid_hv(&leaf)) + if (__sev_cpuid_hv_msr(&leaf)) goto fail; cpuid_done: @@ -592,6 +632,23 @@ fail: sev_es_terminate(SEV_TERM_SET_GEN, GHCB_SEV_ES_GEN_REQ); } +static enum es_result vc_insn_string_check(struct es_em_ctxt *ctxt, + unsigned long address, + bool write) +{ + if (user_mode(ctxt->regs) && fault_in_kernel_space(address)) { + ctxt->fi.vector = X86_TRAP_PF; + ctxt->fi.error_code = X86_PF_USER; + ctxt->fi.cr2 = address; + if (write) + ctxt->fi.error_code |= X86_PF_WRITE; + + return ES_EXCEPTION; + } + + return ES_OK; +} + static enum es_result vc_insn_string_read(struct es_em_ctxt *ctxt, void *src, char *buf, unsigned int data_size, @@ -599,7 +656,12 @@ static enum es_result vc_insn_string_read(struct es_em_ctxt *ctxt, bool backwards) { int i, b = backwards ? -1 : 1; - enum es_result ret = ES_OK; + unsigned long address = (unsigned long)src; + enum es_result ret; + + ret = vc_insn_string_check(ctxt, address, false); + if (ret != ES_OK) + return ret; for (i = 0; i < count; i++) { void *s = src + (i * data_size * b); @@ -620,7 +682,12 @@ static enum es_result vc_insn_string_write(struct es_em_ctxt *ctxt, bool backwards) { int i, s = backwards ? -1 : 1; - enum es_result ret = ES_OK; + unsigned long address = (unsigned long)dst; + enum es_result ret; + + ret = vc_insn_string_check(ctxt, address, true); + if (ret != ES_OK) + return ret; for (i = 0; i < count; i++) { void *d = dst + (i * data_size * s); @@ -656,6 +723,9 @@ static enum es_result vc_insn_string_write(struct es_em_ctxt *ctxt, static enum es_result vc_ioio_exitinfo(struct es_em_ctxt *ctxt, u64 *exitinfo) { struct insn *insn = &ctxt->insn; + size_t size; + u64 port; + *exitinfo = 0; switch (insn->opcode.bytes[0]) { @@ -664,7 +734,7 @@ static enum es_result vc_ioio_exitinfo(struct es_em_ctxt *ctxt, u64 *exitinfo) case 0x6d: *exitinfo |= IOIO_TYPE_INS; *exitinfo |= IOIO_SEG_ES; - *exitinfo |= (ctxt->regs->dx & 0xffff) << 16; + port = ctxt->regs->dx & 0xffff; break; /* OUTS opcodes */ @@ -672,41 +742,43 @@ static enum es_result vc_ioio_exitinfo(struct es_em_ctxt *ctxt, u64 *exitinfo) case 0x6f: *exitinfo |= IOIO_TYPE_OUTS; *exitinfo |= IOIO_SEG_DS; - *exitinfo |= (ctxt->regs->dx & 0xffff) << 16; + port = ctxt->regs->dx & 0xffff; break; /* IN immediate opcodes */ case 0xe4: case 0xe5: *exitinfo |= IOIO_TYPE_IN; - *exitinfo |= (u8)insn->immediate.value << 16; + port = (u8)insn->immediate.value & 0xffff; break; /* OUT immediate opcodes */ case 0xe6: case 0xe7: *exitinfo |= IOIO_TYPE_OUT; - *exitinfo |= (u8)insn->immediate.value << 16; + port = (u8)insn->immediate.value & 0xffff; break; /* IN register opcodes */ case 0xec: case 0xed: *exitinfo |= IOIO_TYPE_IN; - *exitinfo |= (ctxt->regs->dx & 0xffff) << 16; + port = ctxt->regs->dx & 0xffff; break; /* OUT register opcodes */ case 0xee: case 0xef: *exitinfo |= IOIO_TYPE_OUT; - *exitinfo |= (ctxt->regs->dx & 0xffff) << 16; + port = ctxt->regs->dx & 0xffff; break; default: return ES_DECODE_FAILED; } + *exitinfo |= port << 16; + switch (insn->opcode.bytes[0]) { case 0x6c: case 0x6e: @@ -716,12 +788,15 @@ static enum es_result vc_ioio_exitinfo(struct es_em_ctxt *ctxt, u64 *exitinfo) case 0xee: /* Single byte opcodes */ *exitinfo |= IOIO_DATA_8; + size = 1; break; default: /* Length determined by instruction parsing */ *exitinfo |= (insn->opnd_bytes == 2) ? IOIO_DATA_16 : IOIO_DATA_32; + size = (insn->opnd_bytes == 2) ? 2 : 4; } + switch (insn->addr_bytes) { case 2: *exitinfo |= IOIO_ADDR_16; @@ -737,7 +812,7 @@ static enum es_result vc_ioio_exitinfo(struct es_em_ctxt *ctxt, u64 *exitinfo) if (insn_has_rep_prefix(insn)) *exitinfo |= IOIO_REP; - return ES_OK; + return vc_ioio_check(ctxt, (u16)port, size); } static enum es_result vc_handle_ioio(struct ghcb *ghcb, struct es_em_ctxt *ctxt) @@ -848,14 +923,15 @@ static enum es_result vc_handle_ioio(struct ghcb *ghcb, struct es_em_ctxt *ctxt) return ret; } -static int vc_handle_cpuid_snp(struct pt_regs *regs) +static int vc_handle_cpuid_snp(struct ghcb *ghcb, struct es_em_ctxt *ctxt) { + struct pt_regs *regs = ctxt->regs; struct cpuid_leaf leaf; int ret; leaf.fn = regs->ax; leaf.subfn = regs->cx; - ret = snp_cpuid(&leaf); + ret = snp_cpuid(ghcb, ctxt, &leaf); if (!ret) { regs->ax = leaf.eax; regs->bx = leaf.ebx; @@ -874,7 +950,7 @@ static enum es_result vc_handle_cpuid(struct ghcb *ghcb, enum es_result ret; int snp_cpuid_ret; - snp_cpuid_ret = vc_handle_cpuid_snp(regs); + snp_cpuid_ret = vc_handle_cpuid_snp(ghcb, ctxt); if (!snp_cpuid_ret) return ES_OK; if (snp_cpuid_ret != -EOPNOTSUPP) diff --git a/arch/x86/kernel/sev.c b/arch/x86/kernel/sev.c index 2787826d9f60..6395bfd87b68 100644 --- a/arch/x86/kernel/sev.c +++ b/arch/x86/kernel/sev.c @@ -524,6 +524,33 @@ static enum es_result vc_slow_virt_to_phys(struct ghcb *ghcb, struct es_em_ctxt return ES_OK; } +static enum es_result vc_ioio_check(struct es_em_ctxt *ctxt, u16 port, size_t size) +{ + BUG_ON(size > 4); + + if (user_mode(ctxt->regs)) { + struct thread_struct *t = ¤t->thread; + struct io_bitmap *iobm = t->io_bitmap; + size_t idx; + + if (!iobm) + goto fault; + + for (idx = port; idx < port + size; ++idx) { + if (test_bit(idx, iobm->bitmap)) + goto fault; + } + } + + return ES_OK; + +fault: + ctxt->fi.vector = X86_TRAP_GP; + ctxt->fi.error_code = 0; + + return ES_EXCEPTION; +} + /* Include code shared with pre-decompression boot stage */ #include "sev-shared.c" @@ -868,8 +895,7 @@ void snp_set_memory_private(unsigned long vaddr, unsigned long npages) void snp_accept_memory(phys_addr_t start, phys_addr_t end) { - unsigned long vaddr; - unsigned int npages; + unsigned long vaddr, npages; if (!cc_platform_has(CC_ATTR_GUEST_SEV_SNP)) return; @@ -1509,6 +1535,9 @@ static enum es_result vc_handle_mmio(struct ghcb *ghcb, struct es_em_ctxt *ctxt) return ES_DECODE_FAILED; } + if (user_mode(ctxt->regs)) + return ES_UNSUPPORTED; + switch (mmio) { case INSN_MMIO_WRITE: memcpy(ghcb->shared_buffer, reg_data, bytes); diff --git a/arch/x86/kernel/shstk.c b/arch/x86/kernel/shstk.c index fd689921a1db..59e15dd8d0f8 100644 --- a/arch/x86/kernel/shstk.c +++ b/arch/x86/kernel/shstk.c @@ -205,10 +205,21 @@ unsigned long shstk_alloc_thread_stack(struct task_struct *tsk, unsigned long cl return 0; /* - * For CLONE_VM, except vfork, the child needs a separate shadow + * For CLONE_VFORK the child will share the parents shadow stack. + * Make sure to clear the internal tracking of the thread shadow + * stack so the freeing logic run for child knows to leave it alone. + */ + if (clone_flags & CLONE_VFORK) { + shstk->base = 0; + shstk->size = 0; + return 0; + } + + /* + * For !CLONE_VM the child will use a copy of the parents shadow * stack. */ - if ((clone_flags & (CLONE_VFORK | CLONE_VM)) != CLONE_VM) + if (!(clone_flags & CLONE_VM)) return 0; size = adjust_shstk_size(stack_size); @@ -408,7 +419,25 @@ void shstk_free(struct task_struct *tsk) if (!tsk->mm || tsk->mm != current->mm) return; + /* + * If shstk->base is NULL, then this task is not managing its + * own shadow stack (CLONE_VFORK). So skip freeing it. + */ + if (!shstk->base) + return; + + /* + * shstk->base is NULL for CLONE_VFORK child tasks, and so is + * normal. But size = 0 on a shstk->base is not normal and + * indicated an attempt to free the thread shadow stack twice. + * Warn about it. + */ + if (WARN_ON(!shstk->size)) + return; + unmap_shadow_stack(shstk->base, shstk->size); + + shstk->size = 0; } static int wrss_control(bool enable) diff --git a/arch/x86/kernel/smp.c b/arch/x86/kernel/smp.c index 6eb06d001bcc..96a771f9f930 100644 --- a/arch/x86/kernel/smp.c +++ b/arch/x86/kernel/smp.c @@ -131,7 +131,7 @@ static int smp_stop_nmi_callback(unsigned int val, struct pt_regs *regs) } /* - * Disable virtualization, APIC etc. and park the CPU in a HLT loop + * this function calls the 'stop' function on all other CPUs in the system. */ DEFINE_IDTENTRY_SYSVEC(sysvec_reboot) { @@ -172,17 +172,13 @@ static void native_stop_other_cpus(int wait) * 2) Wait for all other CPUs to report that they reached the * HLT loop in stop_this_cpu() * - * 3) If the system uses INIT/STARTUP for CPU bringup, then - * send all present CPUs an INIT vector, which brings them - * completely out of the way. + * 3) If #2 timed out send an NMI to the CPUs which did not + * yet report * - * 4) If #3 is not possible and #2 timed out send an NMI to the - * CPUs which did not yet report - * - * 5) Wait for all other CPUs to report that they reached the + * 4) Wait for all other CPUs to report that they reached the * HLT loop in stop_this_cpu() * - * #4 can obviously race against a CPU reaching the HLT loop late. + * #3 can obviously race against a CPU reaching the HLT loop late. * That CPU will have reported already and the "have all CPUs * reached HLT" condition will be true despite the fact that the * other CPU is still handling the NMI. Again, there is no @@ -198,7 +194,7 @@ static void native_stop_other_cpus(int wait) /* * Don't wait longer than a second for IPI completion. The * wait request is not checked here because that would - * prevent an NMI/INIT shutdown in case that not all + * prevent an NMI shutdown attempt in case that not all * CPUs reach shutdown state. */ timeout = USEC_PER_SEC; @@ -206,27 +202,7 @@ static void native_stop_other_cpus(int wait) udelay(1); } - /* - * Park all other CPUs in INIT including "offline" CPUs, if - * possible. That's a safe place where they can't resume execution - * of HLT and then execute the HLT loop from overwritten text or - * page tables. - * - * The only downside is a broadcast MCE, but up to the point where - * the kexec() kernel brought all APs online again an MCE will just - * make HLT resume and handle the MCE. The machine crashes and burns - * due to overwritten text, page tables and data. So there is a - * choice between fire and frying pan. The result is pretty much - * the same. Chose frying pan until x86 provides a sane mechanism - * to park a CPU. - */ - if (smp_park_other_cpus_in_init()) - goto done; - - /* - * If park with INIT was not possible and the REBOOT_VECTOR didn't - * take all secondary CPUs offline, try with the NMI. - */ + /* if the REBOOT_VECTOR didn't work, try with the NMI */ if (!cpumask_empty(&cpus_stop_mask)) { /* * If NMI IPI is enabled, try to register the stop handler @@ -249,7 +225,6 @@ static void native_stop_other_cpus(int wait) udelay(1); } -done: local_irq_save(flags); disable_local_APIC(); mcheck_cpu_clear(this_cpu_ptr(&cpu_info)); diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c index 4e45ff44aa07..2a187c0cbd5b 100644 --- a/arch/x86/kernel/smpboot.c +++ b/arch/x86/kernel/smpboot.c @@ -579,7 +579,6 @@ static bool match_llc(struct cpuinfo_x86 *c, struct cpuinfo_x86 *o) } -#if defined(CONFIG_SCHED_SMT) || defined(CONFIG_SCHED_CLUSTER) || defined(CONFIG_SCHED_MC) static inline int x86_sched_itmt_flags(void) { return sysctl_sched_itmt_enabled ? SD_ASYM_PACKING : 0; @@ -603,7 +602,14 @@ static int x86_cluster_flags(void) return cpu_cluster_flags() | x86_sched_itmt_flags(); } #endif -#endif + +static int x86_die_flags(void) +{ + if (cpu_feature_enabled(X86_FEATURE_HYBRID_CPU)) + return x86_sched_itmt_flags(); + + return 0; +} /* * Set if a package/die has multiple NUMA nodes inside. @@ -640,7 +646,7 @@ static void __init build_sched_topology(void) */ if (!x86_has_numa_in_package) { x86_topology[i++] = (struct sched_domain_topology_level){ - cpu_cpu_mask, SD_INIT_NAME(DIE) + cpu_cpu_mask, x86_die_flags, SD_INIT_NAME(DIE) }; } @@ -1234,33 +1240,6 @@ void arch_thaw_secondary_cpus_end(void) cache_aps_init(); } -bool smp_park_other_cpus_in_init(void) -{ - unsigned int cpu, this_cpu = smp_processor_id(); - unsigned int apicid; - - if (apic->wakeup_secondary_cpu_64 || apic->wakeup_secondary_cpu) - return false; - - /* - * If this is a crash stop which does not execute on the boot CPU, - * then this cannot use the INIT mechanism because INIT to the boot - * CPU will reset the machine. - */ - if (this_cpu) - return false; - - for_each_cpu_and(cpu, &cpus_booted_once_mask, cpu_present_mask) { - if (cpu == this_cpu) - continue; - apicid = apic->cpu_present_to_apicid(cpu); - if (apicid == BAD_APICID) - continue; - send_init_sequence(apicid); - } - return true; -} - /* * Early setup to make printk work. */ diff --git a/arch/x86/kernel/topology.c b/arch/x86/kernel/topology.c index ca004e2e4469..0bab03130033 100644 --- a/arch/x86/kernel/topology.c +++ b/arch/x86/kernel/topology.c @@ -54,7 +54,7 @@ void arch_unregister_cpu(int num) EXPORT_SYMBOL(arch_unregister_cpu); #else /* CONFIG_HOTPLUG_CPU */ -static int __init arch_register_cpu(int num) +int __init arch_register_cpu(int num) { return register_cpu(&per_cpu(cpu_devices, num).cpu, num); } diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c index 0544e30b4946..773132c3bf5a 100644 --- a/arch/x86/kvm/cpuid.c +++ b/arch/x86/kvm/cpuid.c @@ -360,14 +360,6 @@ static void kvm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) vcpu->arch.guest_supported_xcr0 = cpuid_get_supported_xcr0(vcpu->arch.cpuid_entries, vcpu->arch.cpuid_nent); - /* - * FP+SSE can always be saved/restored via KVM_{G,S}ET_XSAVE, even if - * XSAVE/XCRO are not exposed to the guest, and even if XSAVE isn't - * supported by the host. - */ - vcpu->arch.guest_fpu.fpstate->user_xfeatures = vcpu->arch.guest_supported_xcr0 | - XFEATURE_MASK_FPSSE; - kvm_update_pv_runtime(vcpu); vcpu->arch.maxphyaddr = cpuid_query_maxphyaddr(vcpu); diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c index dcd60b39e794..3e977dbbf993 100644 --- a/arch/x86/kvm/lapic.c +++ b/arch/x86/kvm/lapic.c @@ -2759,13 +2759,17 @@ int kvm_apic_local_deliver(struct kvm_lapic *apic, int lvt_type) { u32 reg = kvm_lapic_get_reg(apic, lvt_type); int vector, mode, trig_mode; + int r; if (kvm_apic_hw_enabled(apic) && !(reg & APIC_LVT_MASKED)) { vector = reg & APIC_VECTOR_MASK; mode = reg & APIC_MODE_MASK; trig_mode = reg & APIC_LVT_LEVEL_TRIGGER; - return __apic_accept_irq(apic, mode, vector, 1, trig_mode, - NULL); + + r = __apic_accept_irq(apic, mode, vector, 1, trig_mode, NULL); + if (r && lvt_type == APIC_LVTPC) + kvm_lapic_set_reg(apic, APIC_LVTPC, reg | APIC_LVT_MASKED); + return r; } return 0; } diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index e1d011c67cc6..f7901cb4d2fa 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -6167,20 +6167,15 @@ static bool kvm_has_zapped_obsolete_pages(struct kvm *kvm) return unlikely(!list_empty_careful(&kvm->arch.zapped_obsolete_pages)); } -int kvm_mmu_init_vm(struct kvm *kvm) +void kvm_mmu_init_vm(struct kvm *kvm) { - int r; - INIT_LIST_HEAD(&kvm->arch.active_mmu_pages); INIT_LIST_HEAD(&kvm->arch.zapped_obsolete_pages); INIT_LIST_HEAD(&kvm->arch.possible_nx_huge_pages); spin_lock_init(&kvm->arch.mmu_unsync_pages_lock); - if (tdp_mmu_enabled) { - r = kvm_mmu_init_tdp_mmu(kvm); - if (r < 0) - return r; - } + if (tdp_mmu_enabled) + kvm_mmu_init_tdp_mmu(kvm); kvm->arch.split_page_header_cache.kmem_cache = mmu_page_header_cache; kvm->arch.split_page_header_cache.gfp_zero = __GFP_ZERO; @@ -6189,8 +6184,6 @@ int kvm_mmu_init_vm(struct kvm *kvm) kvm->arch.split_desc_cache.kmem_cache = pte_list_desc_cache; kvm->arch.split_desc_cache.gfp_zero = __GFP_ZERO; - - return 0; } static void mmu_free_vm_memory_caches(struct kvm *kvm) @@ -6246,7 +6239,6 @@ static bool kvm_rmap_zap_gfn_range(struct kvm *kvm, gfn_t gfn_start, gfn_t gfn_e void kvm_zap_gfn_range(struct kvm *kvm, gfn_t gfn_start, gfn_t gfn_end) { bool flush; - int i; if (WARN_ON_ONCE(gfn_end <= gfn_start)) return; @@ -6257,11 +6249,8 @@ void kvm_zap_gfn_range(struct kvm *kvm, gfn_t gfn_start, gfn_t gfn_end) flush = kvm_rmap_zap_gfn_range(kvm, gfn_start, gfn_end); - if (tdp_mmu_enabled) { - for (i = 0; i < KVM_ADDRESS_SPACE_NUM; i++) - flush = kvm_tdp_mmu_zap_leafs(kvm, i, gfn_start, - gfn_end, true, flush); - } + if (tdp_mmu_enabled) + flush = kvm_tdp_mmu_zap_leafs(kvm, gfn_start, gfn_end, flush); if (flush) kvm_flush_remote_tlbs_range(kvm, gfn_start, gfn_end - gfn_start); diff --git a/arch/x86/kvm/mmu/mmu_internal.h b/arch/x86/kvm/mmu/mmu_internal.h index b102014e2c60..decc1f153669 100644 --- a/arch/x86/kvm/mmu/mmu_internal.h +++ b/arch/x86/kvm/mmu/mmu_internal.h @@ -58,7 +58,12 @@ struct kvm_mmu_page { bool tdp_mmu_page; bool unsync; - u8 mmu_valid_gen; + union { + u8 mmu_valid_gen; + + /* Only accessed under slots_lock. */ + bool tdp_mmu_scheduled_root_to_zap; + }; /* * The shadow page can't be replaced by an equivalent huge page @@ -100,13 +105,7 @@ struct kvm_mmu_page { struct kvm_rmap_head parent_ptes; /* rmap pointers to parent sptes */ tdp_ptep_t ptep; }; - union { - DECLARE_BITMAP(unsync_child_bitmap, 512); - struct { - struct work_struct tdp_mmu_async_work; - void *tdp_mmu_async_data; - }; - }; + DECLARE_BITMAP(unsync_child_bitmap, 512); /* * Tracks shadow pages that, if zapped, would allow KVM to create an NX diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index 6c63f2d1675f..6cd4dd631a2f 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -12,18 +12,10 @@ #include /* Initializes the TDP MMU for the VM, if enabled. */ -int kvm_mmu_init_tdp_mmu(struct kvm *kvm) +void kvm_mmu_init_tdp_mmu(struct kvm *kvm) { - struct workqueue_struct *wq; - - wq = alloc_workqueue("kvm", WQ_UNBOUND|WQ_MEM_RECLAIM|WQ_CPU_INTENSIVE, 0); - if (!wq) - return -ENOMEM; - INIT_LIST_HEAD(&kvm->arch.tdp_mmu_roots); spin_lock_init(&kvm->arch.tdp_mmu_pages_lock); - kvm->arch.tdp_mmu_zap_wq = wq; - return 1; } /* Arbitrarily returns true so that this may be used in if statements. */ @@ -46,20 +38,15 @@ void kvm_mmu_uninit_tdp_mmu(struct kvm *kvm) * ultimately frees all roots. */ kvm_tdp_mmu_invalidate_all_roots(kvm); - - /* - * Destroying a workqueue also first flushes the workqueue, i.e. no - * need to invoke kvm_tdp_mmu_zap_invalidated_roots(). - */ - destroy_workqueue(kvm->arch.tdp_mmu_zap_wq); + kvm_tdp_mmu_zap_invalidated_roots(kvm); WARN_ON(atomic64_read(&kvm->arch.tdp_mmu_pages)); WARN_ON(!list_empty(&kvm->arch.tdp_mmu_roots)); /* * Ensure that all the outstanding RCU callbacks to free shadow pages - * can run before the VM is torn down. Work items on tdp_mmu_zap_wq - * can call kvm_tdp_mmu_put_root and create new callbacks. + * can run before the VM is torn down. Putting the last reference to + * zapped roots will create new callbacks. */ rcu_barrier(); } @@ -86,46 +73,6 @@ static void tdp_mmu_free_sp_rcu_callback(struct rcu_head *head) tdp_mmu_free_sp(sp); } -static void tdp_mmu_zap_root(struct kvm *kvm, struct kvm_mmu_page *root, - bool shared); - -static void tdp_mmu_zap_root_work(struct work_struct *work) -{ - struct kvm_mmu_page *root = container_of(work, struct kvm_mmu_page, - tdp_mmu_async_work); - struct kvm *kvm = root->tdp_mmu_async_data; - - read_lock(&kvm->mmu_lock); - - /* - * A TLB flush is not necessary as KVM performs a local TLB flush when - * allocating a new root (see kvm_mmu_load()), and when migrating vCPU - * to a different pCPU. Note, the local TLB flush on reuse also - * invalidates any paging-structure-cache entries, i.e. TLB entries for - * intermediate paging structures, that may be zapped, as such entries - * are associated with the ASID on both VMX and SVM. - */ - tdp_mmu_zap_root(kvm, root, true); - - /* - * Drop the refcount using kvm_tdp_mmu_put_root() to test its logic for - * avoiding an infinite loop. By design, the root is reachable while - * it's being asynchronously zapped, thus a different task can put its - * last reference, i.e. flowing through kvm_tdp_mmu_put_root() for an - * asynchronously zapped root is unavoidable. - */ - kvm_tdp_mmu_put_root(kvm, root, true); - - read_unlock(&kvm->mmu_lock); -} - -static void tdp_mmu_schedule_zap_root(struct kvm *kvm, struct kvm_mmu_page *root) -{ - root->tdp_mmu_async_data = kvm; - INIT_WORK(&root->tdp_mmu_async_work, tdp_mmu_zap_root_work); - queue_work(kvm->arch.tdp_mmu_zap_wq, &root->tdp_mmu_async_work); -} - void kvm_tdp_mmu_put_root(struct kvm *kvm, struct kvm_mmu_page *root, bool shared) { @@ -211,8 +158,12 @@ static struct kvm_mmu_page *tdp_mmu_next_root(struct kvm *kvm, #define for_each_valid_tdp_mmu_root_yield_safe(_kvm, _root, _as_id, _shared) \ __for_each_tdp_mmu_root_yield_safe(_kvm, _root, _as_id, _shared, true) -#define for_each_tdp_mmu_root_yield_safe(_kvm, _root, _as_id) \ - __for_each_tdp_mmu_root_yield_safe(_kvm, _root, _as_id, false, false) +#define for_each_tdp_mmu_root_yield_safe(_kvm, _root, _shared) \ + for (_root = tdp_mmu_next_root(_kvm, NULL, _shared, false); \ + _root; \ + _root = tdp_mmu_next_root(_kvm, _root, _shared, false)) \ + if (!kvm_lockdep_assert_mmu_lock_held(_kvm, _shared)) { \ + } else /* * Iterate over all TDP MMU roots. Requires that mmu_lock be held for write, @@ -292,7 +243,7 @@ hpa_t kvm_tdp_mmu_get_vcpu_root_hpa(struct kvm_vcpu *vcpu) * by a memslot update or by the destruction of the VM. Initialize the * refcount to two; one reference for the vCPU, and one reference for * the TDP MMU itself, which is held until the root is invalidated and - * is ultimately put by tdp_mmu_zap_root_work(). + * is ultimately put by kvm_tdp_mmu_zap_invalidated_roots(). */ refcount_set(&root->tdp_mmu_root_count, 2); @@ -877,13 +828,12 @@ static bool tdp_mmu_zap_leafs(struct kvm *kvm, struct kvm_mmu_page *root, * true if a TLB flush is needed before releasing the MMU lock, i.e. if one or * more SPTEs were zapped since the MMU lock was last acquired. */ -bool kvm_tdp_mmu_zap_leafs(struct kvm *kvm, int as_id, gfn_t start, gfn_t end, - bool can_yield, bool flush) +bool kvm_tdp_mmu_zap_leafs(struct kvm *kvm, gfn_t start, gfn_t end, bool flush) { struct kvm_mmu_page *root; - for_each_tdp_mmu_root_yield_safe(kvm, root, as_id) - flush = tdp_mmu_zap_leafs(kvm, root, start, end, can_yield, flush); + for_each_tdp_mmu_root_yield_safe(kvm, root, false) + flush = tdp_mmu_zap_leafs(kvm, root, start, end, true, flush); return flush; } @@ -891,7 +841,6 @@ bool kvm_tdp_mmu_zap_leafs(struct kvm *kvm, int as_id, gfn_t start, gfn_t end, void kvm_tdp_mmu_zap_all(struct kvm *kvm) { struct kvm_mmu_page *root; - int i; /* * Zap all roots, including invalid roots, as all SPTEs must be dropped @@ -905,10 +854,8 @@ void kvm_tdp_mmu_zap_all(struct kvm *kvm) * is being destroyed or the userspace VMM has exited. In both cases, * KVM_RUN is unreachable, i.e. no vCPUs will ever service the request. */ - for (i = 0; i < KVM_ADDRESS_SPACE_NUM; i++) { - for_each_tdp_mmu_root_yield_safe(kvm, root, i) - tdp_mmu_zap_root(kvm, root, false); - } + for_each_tdp_mmu_root_yield_safe(kvm, root, false) + tdp_mmu_zap_root(kvm, root, false); } /* @@ -917,18 +864,47 @@ void kvm_tdp_mmu_zap_all(struct kvm *kvm) */ void kvm_tdp_mmu_zap_invalidated_roots(struct kvm *kvm) { - flush_workqueue(kvm->arch.tdp_mmu_zap_wq); + struct kvm_mmu_page *root; + + read_lock(&kvm->mmu_lock); + + for_each_tdp_mmu_root_yield_safe(kvm, root, true) { + if (!root->tdp_mmu_scheduled_root_to_zap) + continue; + + root->tdp_mmu_scheduled_root_to_zap = false; + KVM_BUG_ON(!root->role.invalid, kvm); + + /* + * A TLB flush is not necessary as KVM performs a local TLB + * flush when allocating a new root (see kvm_mmu_load()), and + * when migrating a vCPU to a different pCPU. Note, the local + * TLB flush on reuse also invalidates paging-structure-cache + * entries, i.e. TLB entries for intermediate paging structures, + * that may be zapped, as such entries are associated with the + * ASID on both VMX and SVM. + */ + tdp_mmu_zap_root(kvm, root, true); + + /* + * The referenced needs to be put *after* zapping the root, as + * the root must be reachable by mmu_notifiers while it's being + * zapped + */ + kvm_tdp_mmu_put_root(kvm, root, true); + } + + read_unlock(&kvm->mmu_lock); } /* * Mark each TDP MMU root as invalid to prevent vCPUs from reusing a root that * is about to be zapped, e.g. in response to a memslots update. The actual - * zapping is performed asynchronously. Using a separate workqueue makes it - * easy to ensure that the destruction is performed before the "fast zap" - * completes, without keeping a separate list of invalidated roots; the list is - * effectively the list of work items in the workqueue. + * zapping is done separately so that it happens with mmu_lock with read, + * whereas invalidating roots must be done with mmu_lock held for write (unless + * the VM is being destroyed). * - * Note, the asynchronous worker is gifted the TDP MMU's reference. + * Note, kvm_tdp_mmu_zap_invalidated_roots() is gifted the TDP MMU's reference. * See kvm_tdp_mmu_get_vcpu_root_hpa(). */ void kvm_tdp_mmu_invalidate_all_roots(struct kvm *kvm) @@ -953,19 +929,20 @@ void kvm_tdp_mmu_invalidate_all_roots(struct kvm *kvm) /* * As above, mmu_lock isn't held when destroying the VM! There can't * be other references to @kvm, i.e. nothing else can invalidate roots - * or be consuming roots, but walking the list of roots does need to be - * guarded against roots being deleted by the asynchronous zap worker. + * or get/put references to roots. */ - rcu_read_lock(); - - list_for_each_entry_rcu(root, &kvm->arch.tdp_mmu_roots, link) { + list_for_each_entry(root, &kvm->arch.tdp_mmu_roots, link) { + /* + * Note, invalid roots can outlive a memslot update! Invalid + * roots must be *zapped* before the memslot update completes, + * but a different task can acquire a reference and keep the + * root alive after its been zapped. + */ if (!root->role.invalid) { + root->tdp_mmu_scheduled_root_to_zap = true; root->role.invalid = true; - tdp_mmu_schedule_zap_root(kvm, root); } } - - rcu_read_unlock(); } /* @@ -1146,8 +1123,13 @@ retry: bool kvm_tdp_mmu_unmap_gfn_range(struct kvm *kvm, struct kvm_gfn_range *range, bool flush) { - return kvm_tdp_mmu_zap_leafs(kvm, range->slot->as_id, range->start, - range->end, range->may_block, flush); + struct kvm_mmu_page *root; + + __for_each_tdp_mmu_root_yield_safe(kvm, root, range->slot->as_id, false, false) + flush = tdp_mmu_zap_leafs(kvm, root, range->start, range->end, + range->may_block, flush); + + return flush; } typedef bool (*tdp_handler_t)(struct kvm *kvm, struct tdp_iter *iter, diff --git a/arch/x86/kvm/mmu/tdp_mmu.h b/arch/x86/kvm/mmu/tdp_mmu.h index 0a63b1afabd3..733a3aef3a96 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.h +++ b/arch/x86/kvm/mmu/tdp_mmu.h @@ -7,7 +7,7 @@ #include "spte.h" -int kvm_mmu_init_tdp_mmu(struct kvm *kvm); +void kvm_mmu_init_tdp_mmu(struct kvm *kvm); void kvm_mmu_uninit_tdp_mmu(struct kvm *kvm); hpa_t kvm_tdp_mmu_get_vcpu_root_hpa(struct kvm_vcpu *vcpu); @@ -20,8 +20,7 @@ __must_check static inline bool kvm_tdp_mmu_get_root(struct kvm_mmu_page *root) void kvm_tdp_mmu_put_root(struct kvm *kvm, struct kvm_mmu_page *root, bool shared); -bool kvm_tdp_mmu_zap_leafs(struct kvm *kvm, int as_id, gfn_t start, - gfn_t end, bool can_yield, bool flush); +bool kvm_tdp_mmu_zap_leafs(struct kvm *kvm, gfn_t start, gfn_t end, bool flush); bool kvm_tdp_mmu_zap_sp(struct kvm *kvm, struct kvm_mmu_page *sp); void kvm_tdp_mmu_zap_all(struct kvm *kvm); void kvm_tdp_mmu_invalidate_all_roots(struct kvm *kvm); diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c index edb89b51b383..9ae07db6f0f6 100644 --- a/arch/x86/kvm/pmu.c +++ b/arch/x86/kvm/pmu.c @@ -93,14 +93,6 @@ void kvm_pmu_ops_update(const struct kvm_pmu_ops *pmu_ops) #undef __KVM_X86_PMU_OP } -static void kvm_pmi_trigger_fn(struct irq_work *irq_work) -{ - struct kvm_pmu *pmu = container_of(irq_work, struct kvm_pmu, irq_work); - struct kvm_vcpu *vcpu = pmu_to_vcpu(pmu); - - kvm_pmu_deliver_pmi(vcpu); -} - static inline void __kvm_perf_overflow(struct kvm_pmc *pmc, bool in_pmi) { struct kvm_pmu *pmu = pmc_to_pmu(pmc); @@ -124,20 +116,7 @@ static inline void __kvm_perf_overflow(struct kvm_pmc *pmc, bool in_pmi) __set_bit(pmc->idx, (unsigned long *)&pmu->global_status); } - if (!pmc->intr || skip_pmi) - return; - - /* - * Inject PMI. If vcpu was in a guest mode during NMI PMI - * can be ejected on a guest mode re-entry. Otherwise we can't - * be sure that vcpu wasn't executing hlt instruction at the - * time of vmexit and is not going to re-enter guest mode until - * woken up. So we should wake it, but this is impossible from - * NMI context. Do it from irq work instead. - */ - if (in_pmi && !kvm_handling_nmi_from_guest(pmc->vcpu)) - irq_work_queue(&pmc_to_pmu(pmc)->irq_work); - else + if (pmc->intr && !skip_pmi) kvm_make_request(KVM_REQ_PMI, pmc->vcpu); } @@ -675,9 +654,6 @@ void kvm_pmu_refresh(struct kvm_vcpu *vcpu) void kvm_pmu_reset(struct kvm_vcpu *vcpu) { - struct kvm_pmu *pmu = vcpu_to_pmu(vcpu); - - irq_work_sync(&pmu->irq_work); static_call(kvm_x86_pmu_reset)(vcpu); } @@ -687,7 +663,6 @@ void kvm_pmu_init(struct kvm_vcpu *vcpu) memset(pmu, 0, sizeof(*pmu)); static_call(kvm_x86_pmu_init)(vcpu); - init_irq_work(&pmu->irq_work, kvm_pmi_trigger_fn); pmu->event_count = 0; pmu->need_cleanup = false; kvm_pmu_refresh(vcpu); diff --git a/arch/x86/kvm/pmu.h b/arch/x86/kvm/pmu.h index 7d9ba301c090..1d64113de488 100644 --- a/arch/x86/kvm/pmu.h +++ b/arch/x86/kvm/pmu.h @@ -74,6 +74,12 @@ static inline u64 pmc_read_counter(struct kvm_pmc *pmc) return counter & pmc_bitmask(pmc); } +static inline void pmc_write_counter(struct kvm_pmc *pmc, u64 val) +{ + pmc->counter += val - pmc_read_counter(pmc); + pmc->counter &= pmc_bitmask(pmc); +} + static inline void pmc_release_perf_event(struct kvm_pmc *pmc) { if (pmc->perf_event) { diff --git a/arch/x86/kvm/svm/avic.c b/arch/x86/kvm/svm/avic.c index 2092db892d7d..4b74ea91f4e6 100644 --- a/arch/x86/kvm/svm/avic.c +++ b/arch/x86/kvm/svm/avic.c @@ -529,8 +529,11 @@ int avic_incomplete_ipi_interception(struct kvm_vcpu *vcpu) case AVIC_IPI_FAILURE_INVALID_BACKING_PAGE: WARN_ONCE(1, "Invalid backing page\n"); break; + case AVIC_IPI_FAILURE_INVALID_IPI_VECTOR: + /* Invalid IPI with vector < 16 */ + break; default: - pr_err("Unknown IPI interception\n"); + vcpu_unimpl(vcpu, "Unknown avic incomplete IPI interception\n"); } return 1; diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c index dd496c9e5f91..3fea8c47679e 100644 --- a/arch/x86/kvm/svm/nested.c +++ b/arch/x86/kvm/svm/nested.c @@ -1253,6 +1253,9 @@ void svm_leave_nested(struct kvm_vcpu *vcpu) nested_svm_uninit_mmu_context(vcpu); vmcb_mark_all_dirty(svm->vmcb); + + if (kvm_apicv_activated(vcpu->kvm)) + kvm_make_request(KVM_REQ_APICV_UPDATE, vcpu); } kvm_clear_request(KVM_REQ_GET_NESTED_STATE_PAGES, vcpu); diff --git a/arch/x86/kvm/svm/pmu.c b/arch/x86/kvm/svm/pmu.c index cef5a3d0abd0..373ff6a6687b 100644 --- a/arch/x86/kvm/svm/pmu.c +++ b/arch/x86/kvm/svm/pmu.c @@ -160,7 +160,7 @@ static int amd_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) /* MSR_PERFCTRn */ pmc = get_gp_pmc_amd(pmu, msr, PMU_TYPE_COUNTER); if (pmc) { - pmc->counter += data - pmc_read_counter(pmc); + pmc_write_counter(pmc, data); pmc_update_sample_period(pmc); return 0; } diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c index b9a0a939d59f..4900c078045a 100644 --- a/arch/x86/kvm/svm/sev.c +++ b/arch/x86/kvm/svm/sev.c @@ -2962,6 +2962,32 @@ int sev_es_string_io(struct vcpu_svm *svm, int size, unsigned int port, int in) count, in); } +static void sev_es_vcpu_after_set_cpuid(struct vcpu_svm *svm) +{ + struct kvm_vcpu *vcpu = &svm->vcpu; + + if (boot_cpu_has(X86_FEATURE_V_TSC_AUX)) { + bool v_tsc_aux = guest_cpuid_has(vcpu, X86_FEATURE_RDTSCP) || + guest_cpuid_has(vcpu, X86_FEATURE_RDPID); + + set_msr_interception(vcpu, svm->msrpm, MSR_TSC_AUX, v_tsc_aux, v_tsc_aux); + } +} + +void sev_vcpu_after_set_cpuid(struct vcpu_svm *svm) +{ + struct kvm_vcpu *vcpu = &svm->vcpu; + struct kvm_cpuid_entry2 *best; + + /* For sev guests, the memory encryption bit is not reserved in CR3. */ + best = kvm_find_cpuid_entry(vcpu, 0x8000001F); + if (best) + vcpu->arch.reserved_gpa_bits &= ~(1UL << (best->ebx & 0x3f)); + + if (sev_es_guest(svm->vcpu.kvm)) + sev_es_vcpu_after_set_cpuid(svm); +} + static void sev_es_init_vmcb(struct vcpu_svm *svm) { struct vmcb *vmcb = svm->vmcb01.ptr; @@ -3024,14 +3050,6 @@ static void sev_es_init_vmcb(struct vcpu_svm *svm) set_msr_interception(vcpu, svm->msrpm, MSR_IA32_LASTBRANCHTOIP, 1, 1); set_msr_interception(vcpu, svm->msrpm, MSR_IA32_LASTINTFROMIP, 1, 1); set_msr_interception(vcpu, svm->msrpm, MSR_IA32_LASTINTTOIP, 1, 1); - - if (boot_cpu_has(X86_FEATURE_V_TSC_AUX) && - (guest_cpuid_has(&svm->vcpu, X86_FEATURE_RDTSCP) || - guest_cpuid_has(&svm->vcpu, X86_FEATURE_RDPID))) { - set_msr_interception(vcpu, svm->msrpm, MSR_TSC_AUX, 1, 1); - if (guest_cpuid_has(&svm->vcpu, X86_FEATURE_RDTSCP)) - svm_clr_intercept(svm, INTERCEPT_RDTSCP); - } } void sev_init_vmcb(struct vcpu_svm *svm) diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index f283eb47f6ac..beea99c8e8e0 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -683,6 +683,21 @@ static int svm_hardware_enable(void) amd_pmu_enable_virt(); + /* + * If TSC_AUX virtualization is supported, TSC_AUX becomes a swap type + * "B" field (see sev_es_prepare_switch_to_guest()) for SEV-ES guests. + * Since Linux does not change the value of TSC_AUX once set, prime the + * TSC_AUX field now to avoid a RDMSR on every vCPU run. + */ + if (boot_cpu_has(X86_FEATURE_V_TSC_AUX)) { + struct sev_es_save_area *hostsa; + u32 __maybe_unused msr_hi; + + hostsa = (struct sev_es_save_area *)(page_address(sd->save_area) + 0x400); + + rdmsr(MSR_TSC_AUX, hostsa->tsc_aux, msr_hi); + } + return 0; } @@ -898,8 +913,7 @@ void svm_set_x2apic_msr_interception(struct vcpu_svm *svm, bool intercept) if (intercept == svm->x2avic_msrs_intercepted) return; - if (!x2avic_enabled || - !apic_x2apic_mode(svm->vcpu.arch.apic)) + if (!x2avic_enabled) return; for (i = 0; i < MAX_DIRECT_ACCESS_MSRS; i++) { @@ -1532,7 +1546,14 @@ static void svm_prepare_switch_to_guest(struct kvm_vcpu *vcpu) if (tsc_scaling) __svm_write_tsc_multiplier(vcpu->arch.tsc_scaling_ratio); - if (likely(tsc_aux_uret_slot >= 0)) + /* + * TSC_AUX is always virtualized for SEV-ES guests when the feature is + * available. The user return MSR support is not required in this case + * because TSC_AUX is restored on #VMEXIT from the host save area + * (which has been initialized in svm_hardware_enable()). + */ + if (likely(tsc_aux_uret_slot >= 0) && + (!boot_cpu_has(X86_FEATURE_V_TSC_AUX) || !sev_es_guest(vcpu->kvm))) kvm_set_user_return_msr(tsc_aux_uret_slot, svm->tsc_aux, -1ull); svm->guest_state_loaded = true; @@ -3086,6 +3107,16 @@ static int svm_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr) svm->sysenter_esp_hi = guest_cpuid_is_intel(vcpu) ? (data >> 32) : 0; break; case MSR_TSC_AUX: + /* + * TSC_AUX is always virtualized for SEV-ES guests when the + * feature is available. The user return MSR support is not + * required in this case because TSC_AUX is restored on #VMEXIT + * from the host save area (which has been initialized in + * svm_hardware_enable()). + */ + if (boot_cpu_has(X86_FEATURE_V_TSC_AUX) && sev_es_guest(vcpu->kvm)) + break; + /* * TSC_AUX is usually changed only during boot and never read * directly. Intercept TSC_AUX instead of exposing it to the @@ -4284,7 +4315,6 @@ static bool svm_has_emulated_msr(struct kvm *kvm, u32 index) static void svm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) { struct vcpu_svm *svm = to_svm(vcpu); - struct kvm_cpuid_entry2 *best; /* * SVM doesn't provide a way to disable just XSAVES in the guest, KVM @@ -4328,12 +4358,8 @@ static void svm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) set_msr_interception(vcpu, svm->msrpm, MSR_IA32_FLUSH_CMD, 0, !!guest_cpuid_has(vcpu, X86_FEATURE_FLUSH_L1D)); - /* For sev guests, the memory encryption bit is not reserved in CR3. */ - if (sev_guest(vcpu->kvm)) { - best = kvm_find_cpuid_entry(vcpu, 0x8000001F); - if (best) - vcpu->arch.reserved_gpa_bits &= ~(1UL << (best->ebx & 0x3f)); - } + if (sev_guest(vcpu->kvm)) + sev_vcpu_after_set_cpuid(svm); init_vmcb_after_set_cpuid(vcpu); } diff --git a/arch/x86/kvm/svm/svm.h b/arch/x86/kvm/svm/svm.h index f41253958357..be67ab7fdd10 100644 --- a/arch/x86/kvm/svm/svm.h +++ b/arch/x86/kvm/svm/svm.h @@ -684,6 +684,7 @@ void __init sev_hardware_setup(void); void sev_hardware_unsetup(void); int sev_cpu_init(struct svm_cpu_data *sd); void sev_init_vmcb(struct vcpu_svm *svm); +void sev_vcpu_after_set_cpuid(struct vcpu_svm *svm); void sev_free_vcpu(struct kvm_vcpu *vcpu); int sev_handle_vmgexit(struct kvm_vcpu *vcpu); int sev_es_string_io(struct vcpu_svm *svm, int size, unsigned int port, int in); diff --git a/arch/x86/kvm/vmx/pmu_intel.c b/arch/x86/kvm/vmx/pmu_intel.c index f2efa0bf7ae8..820d3e1f6b4f 100644 --- a/arch/x86/kvm/vmx/pmu_intel.c +++ b/arch/x86/kvm/vmx/pmu_intel.c @@ -436,11 +436,11 @@ static int intel_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) if (!msr_info->host_initiated && !(msr & MSR_PMC_FULL_WIDTH_BIT)) data = (s64)(s32)data; - pmc->counter += data - pmc_read_counter(pmc); + pmc_write_counter(pmc, data); pmc_update_sample_period(pmc); break; } else if ((pmc = get_fixed_pmc(pmu, msr))) { - pmc->counter += data - pmc_read_counter(pmc); + pmc_write_counter(pmc, data); pmc_update_sample_period(pmc); break; } else if ((pmc = get_gp_pmc(pmu, msr, MSR_P6_EVNTSEL0))) { diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 6c9c81e82e65..41cce5031126 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -5382,26 +5382,37 @@ static int kvm_vcpu_ioctl_x86_set_debugregs(struct kvm_vcpu *vcpu, return 0; } -static void kvm_vcpu_ioctl_x86_get_xsave(struct kvm_vcpu *vcpu, - struct kvm_xsave *guest_xsave) -{ - if (fpstate_is_confidential(&vcpu->arch.guest_fpu)) - return; - - fpu_copy_guest_fpstate_to_uabi(&vcpu->arch.guest_fpu, - guest_xsave->region, - sizeof(guest_xsave->region), - vcpu->arch.pkru); -} static void kvm_vcpu_ioctl_x86_get_xsave2(struct kvm_vcpu *vcpu, u8 *state, unsigned int size) { + /* + * Only copy state for features that are enabled for the guest. The + * state itself isn't problematic, but setting bits in the header for + * features that are supported in *this* host but not exposed to the + * guest can result in KVM_SET_XSAVE failing when live migrating to a + * compatible host without the features that are NOT exposed to the + * guest. + * + * FP+SSE can always be saved/restored via KVM_{G,S}ET_XSAVE, even if + * XSAVE/XCRO are not exposed to the guest, and even if XSAVE isn't + * supported by the host. + */ + u64 supported_xcr0 = vcpu->arch.guest_supported_xcr0 | + XFEATURE_MASK_FPSSE; + if (fpstate_is_confidential(&vcpu->arch.guest_fpu)) return; - fpu_copy_guest_fpstate_to_uabi(&vcpu->arch.guest_fpu, - state, size, vcpu->arch.pkru); + fpu_copy_guest_fpstate_to_uabi(&vcpu->arch.guest_fpu, state, size, + supported_xcr0, vcpu->arch.pkru); +} + +static void kvm_vcpu_ioctl_x86_get_xsave(struct kvm_vcpu *vcpu, + struct kvm_xsave *guest_xsave) +{ + return kvm_vcpu_ioctl_x86_get_xsave2(vcpu, (void *)guest_xsave->region, + sizeof(guest_xsave->region)); } static int kvm_vcpu_ioctl_x86_set_xsave(struct kvm_vcpu *vcpu, @@ -12308,9 +12319,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type) if (ret) goto out; - ret = kvm_mmu_init_vm(kvm); - if (ret) - goto out_page_track; + kvm_mmu_init_vm(kvm); ret = static_call(kvm_x86_vm_init)(kvm); if (ret) @@ -12355,7 +12364,6 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type) out_uninit_mmu: kvm_mmu_uninit_vm(kvm); -out_page_track: kvm_page_track_cleanup(kvm); out: return ret; @@ -12846,6 +12854,9 @@ static inline bool kvm_vcpu_has_events(struct kvm_vcpu *vcpu) return true; #endif + if (kvm_test_request(KVM_REQ_PMI, vcpu)) + return true; + if (kvm_arch_interrupt_allowed(vcpu) && (kvm_cpu_has_interrupt(vcpu) || kvm_guest_apic_has_interrupt(vcpu))) diff --git a/arch/x86/lib/memcpy_64.S b/arch/x86/lib/memcpy_64.S index 8f95fb267caa..76697df8dfd5 100644 --- a/arch/x86/lib/memcpy_64.S +++ b/arch/x86/lib/memcpy_64.S @@ -40,7 +40,7 @@ SYM_TYPED_FUNC_START(__memcpy) SYM_FUNC_END(__memcpy) EXPORT_SYMBOL(__memcpy) -SYM_FUNC_ALIAS(memcpy, __memcpy) +SYM_FUNC_ALIAS_MEMFUNC(memcpy, __memcpy) EXPORT_SYMBOL(memcpy) SYM_FUNC_START_LOCAL(memcpy_orig) diff --git a/arch/x86/lib/memmove_64.S b/arch/x86/lib/memmove_64.S index 0559b206fb11..ccdf3a597045 100644 --- a/arch/x86/lib/memmove_64.S +++ b/arch/x86/lib/memmove_64.S @@ -212,5 +212,5 @@ SYM_FUNC_START(__memmove) SYM_FUNC_END(__memmove) EXPORT_SYMBOL(__memmove) -SYM_FUNC_ALIAS(memmove, __memmove) +SYM_FUNC_ALIAS_MEMFUNC(memmove, __memmove) EXPORT_SYMBOL(memmove) diff --git a/arch/x86/lib/memset_64.S b/arch/x86/lib/memset_64.S index 7c59a704c458..3d818b849ec6 100644 --- a/arch/x86/lib/memset_64.S +++ b/arch/x86/lib/memset_64.S @@ -40,7 +40,7 @@ SYM_FUNC_START(__memset) SYM_FUNC_END(__memset) EXPORT_SYMBOL(__memset) -SYM_FUNC_ALIAS(memset, __memset) +SYM_FUNC_ALIAS_MEMFUNC(memset, __memset) EXPORT_SYMBOL(memset) SYM_FUNC_START_LOCAL(memset_orig) diff --git a/arch/x86/lib/putuser.S b/arch/x86/lib/putuser.S index 1451e0c4ae22..235bbda6fc82 100644 --- a/arch/x86/lib/putuser.S +++ b/arch/x86/lib/putuser.S @@ -56,7 +56,6 @@ SYM_FUNC_END(__put_user_1) EXPORT_SYMBOL(__put_user_1) SYM_FUNC_START(__put_user_nocheck_1) - ENDBR ASM_STAC 2: movb %al,(%_ASM_CX) xor %ecx,%ecx @@ -76,7 +75,6 @@ SYM_FUNC_END(__put_user_2) EXPORT_SYMBOL(__put_user_2) SYM_FUNC_START(__put_user_nocheck_2) - ENDBR ASM_STAC 4: movw %ax,(%_ASM_CX) xor %ecx,%ecx @@ -96,7 +94,6 @@ SYM_FUNC_END(__put_user_4) EXPORT_SYMBOL(__put_user_4) SYM_FUNC_START(__put_user_nocheck_4) - ENDBR ASM_STAC 6: movl %eax,(%_ASM_CX) xor %ecx,%ecx @@ -119,7 +116,6 @@ SYM_FUNC_END(__put_user_8) EXPORT_SYMBOL(__put_user_8) SYM_FUNC_START(__put_user_nocheck_8) - ENDBR ASM_STAC 9: mov %_ASM_AX,(%_ASM_CX) #ifdef CONFIG_X86_32 diff --git a/arch/x86/platform/efi/efi_32.c b/arch/x86/platform/efi/efi_32.c index e06a199423c0..b2cc7b4552a1 100644 --- a/arch/x86/platform/efi/efi_32.c +++ b/arch/x86/platform/efi/efi_32.c @@ -140,3 +140,15 @@ void __init efi_runtime_update_mappings(void) } } } + +void arch_efi_call_virt_setup(void) +{ + efi_fpu_begin(); + firmware_restrict_branch_speculation_start(); +} + +void arch_efi_call_virt_teardown(void) +{ + firmware_restrict_branch_speculation_end(); + efi_fpu_end(); +} diff --git a/arch/x86/platform/efi/efi_64.c b/arch/x86/platform/efi/efi_64.c index 77f7ac3668cb..91d31ac422d6 100644 --- a/arch/x86/platform/efi/efi_64.c +++ b/arch/x86/platform/efi/efi_64.c @@ -474,19 +474,34 @@ void __init efi_dump_pagetable(void) * can not change under us. * It should be ensured that there are no concurrent calls to this function. */ -void efi_enter_mm(void) +static void efi_enter_mm(void) { efi_prev_mm = current->active_mm; current->active_mm = &efi_mm; switch_mm(efi_prev_mm, &efi_mm, NULL); } -void efi_leave_mm(void) +static void efi_leave_mm(void) { current->active_mm = efi_prev_mm; switch_mm(&efi_mm, efi_prev_mm, NULL); } +void arch_efi_call_virt_setup(void) +{ + efi_sync_low_kernel_mappings(); + efi_fpu_begin(); + firmware_restrict_branch_speculation_start(); + efi_enter_mm(); +} + +void arch_efi_call_virt_teardown(void) +{ + efi_leave_mm(); + firmware_restrict_branch_speculation_end(); + efi_fpu_end(); +} + static DEFINE_SPINLOCK(efi_runtime_lock); /* diff --git a/arch/x86/purgatory/Makefile b/arch/x86/purgatory/Makefile index c2a29be35c01..08aa0f25f12a 100644 --- a/arch/x86/purgatory/Makefile +++ b/arch/x86/purgatory/Makefile @@ -19,6 +19,10 @@ CFLAGS_sha256.o := -D__DISABLE_EXPORTS -D__NO_FORTIFY # optimization flags. KBUILD_CFLAGS := $(filter-out -fprofile-sample-use=% -fprofile-use=%,$(KBUILD_CFLAGS)) +# When LTO is enabled, llvm emits many text sections, which is not supported +# by kexec. Remove -flto=* flags. +KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_LTO),$(KBUILD_CFLAGS)) + # When linking purgatory.ro with -r unresolved symbols are not checked, # also link a purgatory.chk binary without -r to check for unresolved symbols. PURGATORY_LDFLAGS := -e purgatory_start -z nodefaultlib diff --git a/arch/x86/xen/efi.c b/arch/x86/xen/efi.c index 863d0d6b3edc..7250d0e0e1a9 100644 --- a/arch/x86/xen/efi.c +++ b/arch/x86/xen/efi.c @@ -138,7 +138,7 @@ void __init xen_efi_init(struct boot_params *boot_params) if (efi_systab_xen == NULL) return; - strncpy((char *)&boot_params->efi_info.efi_loader_signature, "Xen", + strscpy((char *)&boot_params->efi_info.efi_loader_signature, "Xen", sizeof(boot_params->efi_info.efi_loader_signature)); boot_params->efi_info.efi_systab = (__u32)__pa(efi_systab_xen); boot_params->efi_info.efi_systab_hi = (__u32)(__pa(efi_systab_xen) >> 32); diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c index b8db2148c07d..0337392a3121 100644 --- a/arch/x86/xen/enlighten.c +++ b/arch/x86/xen/enlighten.c @@ -32,7 +32,7 @@ EXPORT_SYMBOL_GPL(hypercall_page); * &HYPERVISOR_shared_info->vcpu_info[cpu]. See xen_hvm_init_shared_info * and xen_vcpu_setup for details. By default it points to share_info->vcpu_info * but during boot it is switched to point to xen_vcpu_info. - * The pointer is used in __xen_evtchn_do_upcall to acknowledge pending events. + * The pointer is used in xen_evtchn_do_upcall to acknowledge pending events. */ DEFINE_PER_CPU(struct vcpu_info *, xen_vcpu); DEFINE_PER_CPU(struct vcpu_info, xen_vcpu_info); diff --git a/arch/x86/xen/enlighten_hvm.c b/arch/x86/xen/enlighten_hvm.c index 9a192f51f1b0..3f8c34707c50 100644 --- a/arch/x86/xen/enlighten_hvm.c +++ b/arch/x86/xen/enlighten_hvm.c @@ -136,7 +136,7 @@ DEFINE_IDTENTRY_SYSVEC(sysvec_xen_hvm_callback) inc_irq_stat(irq_hv_callback_count); - xen_hvm_evtchn_do_upcall(); + xen_evtchn_do_upcall(); set_irq_regs(old_regs); } diff --git a/arch/x86/xen/enlighten_pv.c b/arch/x86/xen/enlighten_pv.c index 49352fad7d1d..bbbfdd495ebd 100644 --- a/arch/x86/xen/enlighten_pv.c +++ b/arch/x86/xen/enlighten_pv.c @@ -101,6 +101,17 @@ struct tls_descs { struct desc_struct desc[3]; }; +DEFINE_PER_CPU(enum xen_lazy_mode, xen_lazy_mode) = XEN_LAZY_NONE; +DEFINE_PER_CPU(unsigned int, xen_lazy_nesting); + +enum xen_lazy_mode xen_get_lazy_mode(void) +{ + if (in_interrupt()) + return XEN_LAZY_NONE; + + return this_cpu_read(xen_lazy_mode); +} + /* * Updating the 3 TLS descriptors in the GDT on every task switch is * surprisingly expensive so we avoid updating them if they haven't @@ -362,10 +373,25 @@ static noinstr unsigned long xen_get_debugreg(int reg) return HYPERVISOR_get_debugreg(reg); } +static void xen_start_context_switch(struct task_struct *prev) +{ + BUG_ON(preemptible()); + + if (this_cpu_read(xen_lazy_mode) == XEN_LAZY_MMU) { + arch_leave_lazy_mmu_mode(); + set_ti_thread_flag(task_thread_info(prev), TIF_LAZY_MMU_UPDATES); + } + enter_lazy(XEN_LAZY_CPU); +} + static void xen_end_context_switch(struct task_struct *next) { + BUG_ON(preemptible()); + xen_mc_flush(); - paravirt_end_context_switch(next); + leave_lazy(XEN_LAZY_CPU); + if (test_and_clear_ti_thread_flag(task_thread_info(next), TIF_LAZY_MMU_UPDATES)) + arch_enter_lazy_mmu_mode(); } static unsigned long xen_store_tr(void) @@ -472,7 +498,7 @@ static void xen_set_ldt(const void *addr, unsigned entries) MULTI_mmuext_op(mcs.mc, op, 1, NULL, DOMID_SELF); - xen_mc_issue(PARAVIRT_LAZY_CPU); + xen_mc_issue(XEN_LAZY_CPU); } static void xen_load_gdt(const struct desc_ptr *dtr) @@ -568,7 +594,7 @@ static void xen_load_tls(struct thread_struct *t, unsigned int cpu) * exception between the new %fs descriptor being loaded and * %fs being effectively cleared at __switch_to(). */ - if (paravirt_get_lazy_mode() == PARAVIRT_LAZY_CPU) + if (xen_get_lazy_mode() == XEN_LAZY_CPU) loadsegment(fs, 0); xen_mc_batch(); @@ -577,7 +603,7 @@ static void xen_load_tls(struct thread_struct *t, unsigned int cpu) load_TLS_descriptor(t, cpu, 1); load_TLS_descriptor(t, cpu, 2); - xen_mc_issue(PARAVIRT_LAZY_CPU); + xen_mc_issue(XEN_LAZY_CPU); } static void xen_load_gs_index(unsigned int idx) @@ -909,7 +935,7 @@ static void xen_load_sp0(unsigned long sp0) mcs = xen_mc_entry(0); MULTI_stack_switch(mcs.mc, __KERNEL_DS, sp0); - xen_mc_issue(PARAVIRT_LAZY_CPU); + xen_mc_issue(XEN_LAZY_CPU); this_cpu_write(cpu_tss_rw.x86_tss.sp0, sp0); } @@ -973,7 +999,7 @@ static void xen_write_cr0(unsigned long cr0) MULTI_fpu_taskswitch(mcs.mc, (cr0 & X86_CR0_TS) != 0); - xen_mc_issue(PARAVIRT_LAZY_CPU); + xen_mc_issue(XEN_LAZY_CPU); } static void xen_write_cr4(unsigned long cr4) @@ -1156,7 +1182,7 @@ static const typeof(pv_ops) xen_cpu_ops __initconst = { #endif .io_delay = xen_io_delay, - .start_context_switch = paravirt_start_context_switch, + .start_context_switch = xen_start_context_switch, .end_context_switch = xen_end_context_switch, }, }; diff --git a/arch/x86/xen/mmu_pv.c b/arch/x86/xen/mmu_pv.c index 1652c39e3dfb..b6830554ff69 100644 --- a/arch/x86/xen/mmu_pv.c +++ b/arch/x86/xen/mmu_pv.c @@ -236,7 +236,7 @@ static void xen_set_pmd_hyper(pmd_t *ptr, pmd_t val) u.val = pmd_val_ma(val); xen_extend_mmu_update(&u); - xen_mc_issue(PARAVIRT_LAZY_MMU); + xen_mc_issue(XEN_LAZY_MMU); preempt_enable(); } @@ -270,7 +270,7 @@ static bool xen_batched_set_pte(pte_t *ptep, pte_t pteval) { struct mmu_update u; - if (paravirt_get_lazy_mode() != PARAVIRT_LAZY_MMU) + if (xen_get_lazy_mode() != XEN_LAZY_MMU) return false; xen_mc_batch(); @@ -279,7 +279,7 @@ static bool xen_batched_set_pte(pte_t *ptep, pte_t pteval) u.val = pte_val_ma(pteval); xen_extend_mmu_update(&u); - xen_mc_issue(PARAVIRT_LAZY_MMU); + xen_mc_issue(XEN_LAZY_MMU); return true; } @@ -325,7 +325,7 @@ void xen_ptep_modify_prot_commit(struct vm_area_struct *vma, unsigned long addr, u.val = pte_val_ma(pte); xen_extend_mmu_update(&u); - xen_mc_issue(PARAVIRT_LAZY_MMU); + xen_mc_issue(XEN_LAZY_MMU); } /* Assume pteval_t is equivalent to all the other *val_t types. */ @@ -419,7 +419,7 @@ static void xen_set_pud_hyper(pud_t *ptr, pud_t val) u.val = pud_val_ma(val); xen_extend_mmu_update(&u); - xen_mc_issue(PARAVIRT_LAZY_MMU); + xen_mc_issue(XEN_LAZY_MMU); preempt_enable(); } @@ -499,7 +499,7 @@ static void __init xen_set_p4d_hyper(p4d_t *ptr, p4d_t val) __xen_set_p4d_hyper(ptr, val); - xen_mc_issue(PARAVIRT_LAZY_MMU); + xen_mc_issue(XEN_LAZY_MMU); preempt_enable(); } @@ -531,7 +531,7 @@ static void xen_set_p4d(p4d_t *ptr, p4d_t val) if (user_ptr) __xen_set_p4d_hyper((p4d_t *)user_ptr, val); - xen_mc_issue(PARAVIRT_LAZY_MMU); + xen_mc_issue(XEN_LAZY_MMU); } #if CONFIG_PGTABLE_LEVELS >= 5 @@ -1245,7 +1245,7 @@ static noinline void xen_flush_tlb(void) op->cmd = MMUEXT_TLB_FLUSH_LOCAL; MULTI_mmuext_op(mcs.mc, op, 1, NULL, DOMID_SELF); - xen_mc_issue(PARAVIRT_LAZY_MMU); + xen_mc_issue(XEN_LAZY_MMU); preempt_enable(); } @@ -1265,7 +1265,7 @@ static void xen_flush_tlb_one_user(unsigned long addr) op->arg1.linear_addr = addr & PAGE_MASK; MULTI_mmuext_op(mcs.mc, op, 1, NULL, DOMID_SELF); - xen_mc_issue(PARAVIRT_LAZY_MMU); + xen_mc_issue(XEN_LAZY_MMU); preempt_enable(); } @@ -1302,7 +1302,7 @@ static void xen_flush_tlb_multi(const struct cpumask *cpus, MULTI_mmuext_op(mcs.mc, &args->op, 1, NULL, DOMID_SELF); - xen_mc_issue(PARAVIRT_LAZY_MMU); + xen_mc_issue(XEN_LAZY_MMU); } static unsigned long xen_read_cr3(void) @@ -1361,7 +1361,7 @@ static void xen_write_cr3(unsigned long cr3) else __xen_write_cr3(false, 0); - xen_mc_issue(PARAVIRT_LAZY_CPU); /* interrupts restored */ + xen_mc_issue(XEN_LAZY_CPU); /* interrupts restored */ } /* @@ -1396,7 +1396,7 @@ static void __init xen_write_cr3_init(unsigned long cr3) __xen_write_cr3(true, cr3); - xen_mc_issue(PARAVIRT_LAZY_CPU); /* interrupts restored */ + xen_mc_issue(XEN_LAZY_CPU); /* interrupts restored */ } static int xen_pgd_alloc(struct mm_struct *mm) @@ -1557,7 +1557,7 @@ static inline void xen_alloc_ptpage(struct mm_struct *mm, unsigned long pfn, if (level == PT_PTE && USE_SPLIT_PTE_PTLOCKS && !pinned) __pin_pagetable_pfn(MMUEXT_PIN_L1_TABLE, pfn); - xen_mc_issue(PARAVIRT_LAZY_MMU); + xen_mc_issue(XEN_LAZY_MMU); } } @@ -1587,7 +1587,7 @@ static inline void xen_release_ptpage(unsigned long pfn, unsigned level) __set_pfn_prot(pfn, PAGE_KERNEL); - xen_mc_issue(PARAVIRT_LAZY_MMU); + xen_mc_issue(XEN_LAZY_MMU); ClearPagePinned(page); } @@ -1804,7 +1804,7 @@ void __init xen_setup_kernel_pagetable(pgd_t *pgd, unsigned long max_pfn) */ xen_mc_batch(); __xen_write_cr3(true, __pa(init_top_pgt)); - xen_mc_issue(PARAVIRT_LAZY_CPU); + xen_mc_issue(XEN_LAZY_CPU); /* We can't that easily rip out L3 and L2, as the Xen pagetables are * set out this way: [L4], [L1], [L2], [L3], [L1], [L1] ... for @@ -2083,6 +2083,23 @@ static void xen_set_fixmap(unsigned idx, phys_addr_t phys, pgprot_t prot) #endif } +static void xen_enter_lazy_mmu(void) +{ + enter_lazy(XEN_LAZY_MMU); +} + +static void xen_flush_lazy_mmu(void) +{ + preempt_disable(); + + if (xen_get_lazy_mode() == XEN_LAZY_MMU) { + arch_leave_lazy_mmu_mode(); + arch_enter_lazy_mmu_mode(); + } + + preempt_enable(); +} + static void __init xen_post_allocator_init(void) { pv_ops.mmu.set_pte = xen_set_pte; @@ -2107,7 +2124,7 @@ static void xen_leave_lazy_mmu(void) { preempt_disable(); xen_mc_flush(); - paravirt_leave_lazy_mmu(); + leave_lazy(XEN_LAZY_MMU); preempt_enable(); } @@ -2166,9 +2183,9 @@ static const typeof(pv_ops) xen_mmu_ops __initconst = { .exit_mmap = xen_exit_mmap, .lazy_mode = { - .enter = paravirt_enter_lazy_mmu, + .enter = xen_enter_lazy_mmu, .leave = xen_leave_lazy_mmu, - .flush = paravirt_flush_lazy_mmu, + .flush = xen_flush_lazy_mmu, }, .set_fixmap = xen_set_fixmap, @@ -2385,7 +2402,7 @@ static noinline void xen_flush_tlb_all(void) op->cmd = MMUEXT_TLB_FLUSH_ALL; MULTI_mmuext_op(mcs.mc, op, 1, NULL, DOMID_SELF); - xen_mc_issue(PARAVIRT_LAZY_MMU); + xen_mc_issue(XEN_LAZY_MMU); preempt_enable(); } diff --git a/arch/x86/xen/multicalls.h b/arch/x86/xen/multicalls.h index 1c51b2c87f30..c3867b585e0d 100644 --- a/arch/x86/xen/multicalls.h +++ b/arch/x86/xen/multicalls.h @@ -26,7 +26,7 @@ static inline void xen_mc_batch(void) /* need to disable interrupts until this entry is complete */ local_irq_save(flags); - trace_xen_mc_batch(paravirt_get_lazy_mode()); + trace_xen_mc_batch(xen_get_lazy_mode()); __this_cpu_write(xen_mc_irq_flags, flags); } @@ -44,7 +44,7 @@ static inline void xen_mc_issue(unsigned mode) { trace_xen_mc_issue(mode); - if ((paravirt_get_lazy_mode() & mode) == 0) + if ((xen_get_lazy_mode() & mode) == 0) xen_mc_flush(); /* restore flags saved in xen_mc_batch */ diff --git a/arch/xtensa/boot/Makefile b/arch/xtensa/boot/Makefile index a65b7a9ebff2..d8b0fadf429a 100644 --- a/arch/xtensa/boot/Makefile +++ b/arch/xtensa/boot/Makefile @@ -9,8 +9,7 @@ # KBUILD_CFLAGS used when building rest of boot (takes effect recursively) -KBUILD_CFLAGS += -fno-builtin -Iarch/$(ARCH)/boot/include -HOSTFLAGS += -Iarch/$(ARCH)/boot/include +KBUILD_CFLAGS += -fno-builtin subdir-y := lib targets += vmlinux.bin vmlinux.bin.gz diff --git a/arch/xtensa/boot/lib/zmem.c b/arch/xtensa/boot/lib/zmem.c index e3ecd743c515..b89189355122 100644 --- a/arch/xtensa/boot/lib/zmem.c +++ b/arch/xtensa/boot/lib/zmem.c @@ -4,13 +4,14 @@ /* bits taken from ppc */ extern void *avail_ram, *end_avail; +void gunzip(void *dst, int dstlen, unsigned char *src, int *lenp); -void exit (void) +static void exit(void) { for (;;); } -void *zalloc(unsigned size) +static void *zalloc(unsigned int size) { void *p = avail_ram; diff --git a/arch/xtensa/include/asm/core.h b/arch/xtensa/include/asm/core.h index 3f5ffae89b58..6f02f6f21890 100644 --- a/arch/xtensa/include/asm/core.h +++ b/arch/xtensa/include/asm/core.h @@ -6,6 +6,10 @@ #include +#ifndef XCHAL_HAVE_DIV32 +#define XCHAL_HAVE_DIV32 0 +#endif + #ifndef XCHAL_HAVE_EXCLUSIVE #define XCHAL_HAVE_EXCLUSIVE 0 #endif diff --git a/arch/xtensa/include/asm/hw_breakpoint.h b/arch/xtensa/include/asm/hw_breakpoint.h index 9f119c1ca0b5..9ec86f440a48 100644 --- a/arch/xtensa/include/asm/hw_breakpoint.h +++ b/arch/xtensa/include/asm/hw_breakpoint.h @@ -48,6 +48,7 @@ void arch_uninstall_hw_breakpoint(struct perf_event *bp); void hw_breakpoint_pmu_read(struct perf_event *bp); int check_hw_breakpoint(struct pt_regs *regs); void clear_ptrace_hw_breakpoint(struct task_struct *tsk); +void restore_dbreak(void); #else diff --git a/arch/xtensa/include/asm/processor.h b/arch/xtensa/include/asm/processor.h index a6d09fe04831..d008a153a2b9 100644 --- a/arch/xtensa/include/asm/processor.h +++ b/arch/xtensa/include/asm/processor.h @@ -14,6 +14,8 @@ #include #include + +#include #include #include #include @@ -217,6 +219,9 @@ struct mm_struct; extern unsigned long __get_wchan(struct task_struct *p); +void init_arch(bp_tag_t *bp_start); +void do_notify_resume(struct pt_regs *regs); + #define KSTK_EIP(tsk) (task_pt_regs(tsk)->pc) #define KSTK_ESP(tsk) (task_pt_regs(tsk)->areg[1]) diff --git a/arch/xtensa/include/asm/ptrace.h b/arch/xtensa/include/asm/ptrace.h index 308f209a4740..a270467556dc 100644 --- a/arch/xtensa/include/asm/ptrace.h +++ b/arch/xtensa/include/asm/ptrace.h @@ -106,6 +106,9 @@ static inline unsigned long regs_return_value(struct pt_regs *regs) return regs->areg[2]; } +int do_syscall_trace_enter(struct pt_regs *regs); +void do_syscall_trace_leave(struct pt_regs *regs); + #else /* __ASSEMBLY__ */ # include diff --git a/arch/xtensa/include/asm/smp.h b/arch/xtensa/include/asm/smp.h index 5dc5bf8cdd77..e446e6fc4557 100644 --- a/arch/xtensa/include/asm/smp.h +++ b/arch/xtensa/include/asm/smp.h @@ -23,6 +23,7 @@ struct cpumask; void arch_send_call_function_ipi_mask(const struct cpumask *mask); void arch_send_call_function_single_ipi(int cpu); +void secondary_start_kernel(void); void smp_init_cpus(void); void secondary_init_irq(void); void ipi_init(void); diff --git a/arch/xtensa/include/asm/tlb.h b/arch/xtensa/include/asm/tlb.h index 50889935138a..8c3ceb427018 100644 --- a/arch/xtensa/include/asm/tlb.h +++ b/arch/xtensa/include/asm/tlb.h @@ -18,4 +18,6 @@ #define __pte_free_tlb(tlb, pte, address) pte_free((tlb)->mm, pte) +void check_tlb_sanity(void); + #endif /* _XTENSA_TLB_H */ diff --git a/arch/xtensa/kernel/hw_breakpoint.c b/arch/xtensa/kernel/hw_breakpoint.c index 285fb2942b06..1eeecd58eb0c 100644 --- a/arch/xtensa/kernel/hw_breakpoint.c +++ b/arch/xtensa/kernel/hw_breakpoint.c @@ -13,6 +13,7 @@ #include #include #include +#include /* Breakpoint currently in use for each IBREAKA. */ static DEFINE_PER_CPU(struct perf_event *, bp_on_reg[XCHAL_NUM_IBREAK]); diff --git a/arch/xtensa/kernel/irq.c b/arch/xtensa/kernel/irq.c index 42f106004400..b1e410f6b5ab 100644 --- a/arch/xtensa/kernel/irq.c +++ b/arch/xtensa/kernel/irq.c @@ -28,6 +28,7 @@ #include #include #include +#include DECLARE_PER_CPU(unsigned long, nmi_count); diff --git a/arch/xtensa/kernel/ptrace.c b/arch/xtensa/kernel/ptrace.c index f29477162ede..9056cd1a8302 100644 --- a/arch/xtensa/kernel/ptrace.c +++ b/arch/xtensa/kernel/ptrace.c @@ -541,7 +541,6 @@ long arch_ptrace(struct task_struct *child, long request, return ret; } -void do_syscall_trace_leave(struct pt_regs *regs); int do_syscall_trace_enter(struct pt_regs *regs) { if (regs->syscall == NO_SYSCALL) diff --git a/arch/xtensa/kernel/signal.c b/arch/xtensa/kernel/signal.c index 5c01d7e70d90..81f0b106cfc1 100644 --- a/arch/xtensa/kernel/signal.c +++ b/arch/xtensa/kernel/signal.c @@ -26,6 +26,8 @@ #include #include #include +#include +#include #include extern struct task_struct *coproc_owners[]; diff --git a/arch/xtensa/kernel/smp.c b/arch/xtensa/kernel/smp.c index 07dd6baf18cf..94a23f100726 100644 --- a/arch/xtensa/kernel/smp.c +++ b/arch/xtensa/kernel/smp.c @@ -21,6 +21,7 @@ #include #include #include +#include #include #include #include diff --git a/arch/xtensa/kernel/stacktrace.c b/arch/xtensa/kernel/stacktrace.c index f643ea5e36da..831ffb648bda 100644 --- a/arch/xtensa/kernel/stacktrace.c +++ b/arch/xtensa/kernel/stacktrace.c @@ -12,6 +12,7 @@ #include #include +#include #include #include #include diff --git a/arch/xtensa/kernel/traps.c b/arch/xtensa/kernel/traps.c index 427c125a137a..38092d21acf8 100644 --- a/arch/xtensa/kernel/traps.c +++ b/arch/xtensa/kernel/traps.c @@ -23,6 +23,7 @@ * for more details. */ +#include #include #include #include diff --git a/arch/xtensa/lib/umulsidi3.S b/arch/xtensa/lib/umulsidi3.S index 8c7a94a0c5d0..5da501b57813 100644 --- a/arch/xtensa/lib/umulsidi3.S +++ b/arch/xtensa/lib/umulsidi3.S @@ -3,7 +3,9 @@ #include #include -#if !XCHAL_HAVE_MUL16 && !XCHAL_HAVE_MUL32 && !XCHAL_HAVE_MAC16 +#if XCHAL_HAVE_MUL16 || XCHAL_HAVE_MUL32 || XCHAL_HAVE_MAC16 +#define XCHAL_NO_MUL 0 +#else #define XCHAL_NO_MUL 1 #endif diff --git a/arch/xtensa/mm/fault.c b/arch/xtensa/mm/fault.c index d1eb8d6c5b82..16e11b6f6f78 100644 --- a/arch/xtensa/mm/fault.c +++ b/arch/xtensa/mm/fault.c @@ -20,6 +20,7 @@ #include #include #include +#include void bad_page_fault(struct pt_regs*, unsigned long, int); diff --git a/arch/xtensa/mm/tlb.c b/arch/xtensa/mm/tlb.c index 0a11fc5f185b..4f974b74883c 100644 --- a/arch/xtensa/mm/tlb.c +++ b/arch/xtensa/mm/tlb.c @@ -17,6 +17,7 @@ #include #include #include +#include #include #include diff --git a/arch/xtensa/platforms/iss/network.c b/arch/xtensa/platforms/iss/network.c index 85c82cd42188..e89f27f2bb18 100644 --- a/arch/xtensa/platforms/iss/network.c +++ b/arch/xtensa/platforms/iss/network.c @@ -201,7 +201,7 @@ static int tuntap_write(struct iss_net_private *lp, struct sk_buff **skb) return simc_write(lp->tp.info.tuntap.fd, (*skb)->data, (*skb)->len); } -unsigned short tuntap_protocol(struct sk_buff *skb) +static unsigned short tuntap_protocol(struct sk_buff *skb) { return eth_type_trans(skb, skb->dev); } @@ -441,7 +441,7 @@ static int iss_net_change_mtu(struct net_device *dev, int new_mtu) return -EINVAL; } -void iss_net_user_timer_expire(struct timer_list *unused) +static void iss_net_user_timer_expire(struct timer_list *unused) { } diff --git a/block/blk-mq.c b/block/blk-mq.c index ec922c6bccbe..1fafd54dce3c 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -4405,11 +4405,8 @@ static int blk_mq_realloc_tag_set_tags(struct blk_mq_tag_set *set, struct blk_mq_tags **new_tags; int i; - if (set->nr_hw_queues >= new_nr_hw_queues) { - for (i = new_nr_hw_queues; i < set->nr_hw_queues; i++) - __blk_mq_free_map_and_rqs(set, i); + if (set->nr_hw_queues >= new_nr_hw_queues) goto done; - } new_tags = kcalloc_node(new_nr_hw_queues, sizeof(struct blk_mq_tags *), GFP_KERNEL, set->numa_node); @@ -4719,7 +4716,8 @@ static void __blk_mq_update_nr_hw_queues(struct blk_mq_tag_set *set, { struct request_queue *q; LIST_HEAD(head); - int prev_nr_hw_queues; + int prev_nr_hw_queues = set->nr_hw_queues; + int i; lockdep_assert_held(&set->tag_list_lock); @@ -4746,7 +4744,6 @@ static void __blk_mq_update_nr_hw_queues(struct blk_mq_tag_set *set, blk_mq_sysfs_unregister_hctxs(q); } - prev_nr_hw_queues = set->nr_hw_queues; if (blk_mq_realloc_tag_set_tags(set, nr_hw_queues) < 0) goto reregister; @@ -4781,6 +4778,10 @@ switch_back: list_for_each_entry(q, &set->tag_list, tag_set_list) blk_mq_unfreeze_queue(q); + + /* Free the excess tags when nr_hw_queues shrink. */ + for (i = set->nr_hw_queues; i < prev_nr_hw_queues; i++) + __blk_mq_free_map_and_rqs(set, i); } void blk_mq_update_nr_hw_queues(struct blk_mq_tag_set *set, int nr_hw_queues) diff --git a/block/blk-rq-qos.c b/block/blk-rq-qos.c index 167be74df4ee..dd7310c94713 100644 --- a/block/blk-rq-qos.c +++ b/block/blk-rq-qos.c @@ -270,7 +270,7 @@ void rq_qos_wait(struct rq_wait *rqw, void *private_data, finish_wait(&rqw->wait, &data.wq); /* - * We raced with wbt_wake_function() getting a token, + * We raced with rq_qos_wake_function() getting a token, * which means we now have two. Put our local token * and wake anyone else potentially waiting for one. */ diff --git a/block/disk-events.c b/block/disk-events.c index 422db8292d09..13c3372c465a 100644 --- a/block/disk-events.c +++ b/block/disk-events.c @@ -290,7 +290,6 @@ EXPORT_SYMBOL(disk_check_media_change); /** * disk_force_media_change - force a media change event * @disk: the disk which will raise the event - * @events: the events to raise * * Should be called when the media changes for @disk. Generates a uevent * and attempts to free all dentries and inodes and invalidates all block diff --git a/block/fops.c b/block/fops.c index acff3d5d22d4..73e42742543f 100644 --- a/block/fops.c +++ b/block/fops.c @@ -772,24 +772,35 @@ static long blkdev_fallocate(struct file *file, int mode, loff_t start, filemap_invalidate_lock(inode->i_mapping); - /* Invalidate the page cache, including dirty pages. */ - error = truncate_bdev_range(bdev, file_to_blk_mode(file), start, end); - if (error) - goto fail; - + /* + * Invalidate the page cache, including dirty pages, for valid + * de-allocate mode calls to fallocate(). + */ switch (mode) { case FALLOC_FL_ZERO_RANGE: case FALLOC_FL_ZERO_RANGE | FALLOC_FL_KEEP_SIZE: + error = truncate_bdev_range(bdev, file_to_blk_mode(file), start, end); + if (error) + goto fail; + error = blkdev_issue_zeroout(bdev, start >> SECTOR_SHIFT, len >> SECTOR_SHIFT, GFP_KERNEL, BLKDEV_ZERO_NOUNMAP); break; case FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE: + error = truncate_bdev_range(bdev, file_to_blk_mode(file), start, end); + if (error) + goto fail; + error = blkdev_issue_zeroout(bdev, start >> SECTOR_SHIFT, len >> SECTOR_SHIFT, GFP_KERNEL, BLKDEV_ZERO_NOFALLBACK); break; case FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE | FALLOC_FL_NO_HIDE_STALE: + error = truncate_bdev_range(bdev, file_to_blk_mode(file), start, end); + if (error) + goto fail; + error = blkdev_issue_discard(bdev, start >> SECTOR_SHIFT, len >> SECTOR_SHIFT, GFP_KERNEL); break; diff --git a/block/sed-opal.c b/block/sed-opal.c index 6d7f25d1711b..04f38a3f5d95 100644 --- a/block/sed-opal.c +++ b/block/sed-opal.c @@ -2888,12 +2888,11 @@ static int opal_lock_unlock(struct opal_dev *dev, if (lk_unlk->session.who > OPAL_USER9) return -EINVAL; - ret = opal_get_key(dev, &lk_unlk->session.opal_key); - if (ret) - return ret; mutex_lock(&dev->dev_lock); opal_lock_check_for_saved_key(dev, lk_unlk); - ret = __opal_lock_unlock(dev, lk_unlk); + ret = opal_get_key(dev, &lk_unlk->session.opal_key); + if (!ret) + ret = __opal_lock_unlock(dev, lk_unlk); mutex_unlock(&dev->dev_lock); return ret; diff --git a/crypto/asymmetric_keys/public_key.c b/crypto/asymmetric_keys/public_key.c index abeecb8329b3..1dcab27986a6 100644 --- a/crypto/asymmetric_keys/public_key.c +++ b/crypto/asymmetric_keys/public_key.c @@ -81,14 +81,13 @@ software_key_determine_akcipher(const struct public_key *pkey, * RSA signatures usually use EMSA-PKCS1-1_5 [RFC3447 sec 8.2]. */ if (strcmp(encoding, "pkcs1") == 0) { + *sig = op == kernel_pkey_sign || + op == kernel_pkey_verify; if (!hash_algo) { - *sig = false; n = snprintf(alg_name, CRYPTO_MAX_ALG_NAME, "pkcs1pad(%s)", pkey->pkey_algo); } else { - *sig = op == kernel_pkey_sign || - op == kernel_pkey_verify; n = snprintf(alg_name, CRYPTO_MAX_ALG_NAME, "pkcs1pad(%s,%s)", pkey->pkey_algo, hash_algo); diff --git a/crypto/sm2.c b/crypto/sm2.c index 285b3cb7c0bc..5ab120d74c59 100644 --- a/crypto/sm2.c +++ b/crypto/sm2.c @@ -278,10 +278,14 @@ int sm2_compute_z_digest(struct shash_desc *desc, if (!ec) return -ENOMEM; - err = __sm2_set_pub_key(ec, key, keylen); + err = sm2_ec_ctx_init(ec); if (err) goto out_free_ec; + err = __sm2_set_pub_key(ec, key, keylen); + if (err) + goto out_deinit_ec; + bits_len = SM2_DEFAULT_USERID_LEN * 8; entl[0] = bits_len >> 8; entl[1] = bits_len & 0xff; diff --git a/drivers/Makefile b/drivers/Makefile index cb0afca2e4a0..1bec7819a837 100644 --- a/drivers/Makefile +++ b/drivers/Makefile @@ -46,7 +46,7 @@ obj-$(CONFIG_DMADEVICES) += dma/ # SOC specific infrastructure drivers. obj-y += soc/ -obj-$(CONFIG_PM_GENERIC_DOMAINS) += genpd/ +obj-$(CONFIG_PM_GENERIC_DOMAINS) += pmdomain/ obj-y += virtio/ obj-$(CONFIG_VDPA) += vdpa/ diff --git a/drivers/accel/ivpu/ivpu_drv.c b/drivers/accel/ivpu/ivpu_drv.c index ba79f397c9e8..7e9359611d69 100644 --- a/drivers/accel/ivpu/ivpu_drv.c +++ b/drivers/accel/ivpu/ivpu_drv.c @@ -327,7 +327,7 @@ static int ivpu_wait_for_ready(struct ivpu_device *vdev) } if (!ret) - ivpu_info(vdev, "VPU ready message received successfully\n"); + ivpu_dbg(vdev, PM, "VPU ready message received successfully\n"); else ivpu_hw_diagnose_failure(vdev); @@ -367,14 +367,19 @@ int ivpu_boot(struct ivpu_device *vdev) return 0; } -int ivpu_shutdown(struct ivpu_device *vdev) +void ivpu_prepare_for_reset(struct ivpu_device *vdev) { - int ret; - ivpu_hw_irq_disable(vdev); disable_irq(vdev->irq); ivpu_ipc_disable(vdev); ivpu_mmu_disable(vdev); +} + +int ivpu_shutdown(struct ivpu_device *vdev) +{ + int ret; + + ivpu_prepare_for_reset(vdev); ret = ivpu_hw_power_down(vdev); if (ret) @@ -634,6 +639,7 @@ static void ivpu_dev_fini(struct ivpu_device *vdev) static struct pci_device_id ivpu_pci_ids[] = { { PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_MTL) }, + { PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_ARL) }, { PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_LNL) }, { } }; diff --git a/drivers/accel/ivpu/ivpu_drv.h b/drivers/accel/ivpu/ivpu_drv.h index 9e8c075fe9ef..2adc349126bb 100644 --- a/drivers/accel/ivpu/ivpu_drv.h +++ b/drivers/accel/ivpu/ivpu_drv.h @@ -23,6 +23,7 @@ #define DRIVER_DATE "20230117" #define PCI_DEVICE_ID_MTL 0x7d1d +#define PCI_DEVICE_ID_ARL 0xad1d #define PCI_DEVICE_ID_LNL 0x643e #define IVPU_HW_37XX 37 @@ -150,6 +151,7 @@ void ivpu_file_priv_put(struct ivpu_file_priv **link); int ivpu_boot(struct ivpu_device *vdev); int ivpu_shutdown(struct ivpu_device *vdev); +void ivpu_prepare_for_reset(struct ivpu_device *vdev); static inline u8 ivpu_revision(struct ivpu_device *vdev) { @@ -165,6 +167,7 @@ static inline int ivpu_hw_gen(struct ivpu_device *vdev) { switch (ivpu_device_id(vdev)) { case PCI_DEVICE_ID_MTL: + case PCI_DEVICE_ID_ARL: return IVPU_HW_37XX; case PCI_DEVICE_ID_LNL: return IVPU_HW_40XX; diff --git a/drivers/accel/ivpu/ivpu_fw.c b/drivers/accel/ivpu/ivpu_fw.c index 9827ea4d7b83..a277bbae78fc 100644 --- a/drivers/accel/ivpu/ivpu_fw.c +++ b/drivers/accel/ivpu/ivpu_fw.c @@ -432,6 +432,7 @@ void ivpu_fw_boot_params_setup(struct ivpu_device *vdev, struct vpu_boot_params if (!ivpu_fw_is_cold_boot(vdev)) { boot_params->save_restore_ret_address = 0; vdev->pm->is_warmboot = true; + wmb(); /* Flush WC buffers after writing save_restore_ret_address */ return; } diff --git a/drivers/accel/ivpu/ivpu_hw.h b/drivers/accel/ivpu/ivpu_hw.h index ab341237bcf9..1079e06255ba 100644 --- a/drivers/accel/ivpu/ivpu_hw.h +++ b/drivers/accel/ivpu/ivpu_hw.h @@ -13,6 +13,7 @@ struct ivpu_hw_ops { int (*power_up)(struct ivpu_device *vdev); int (*boot_fw)(struct ivpu_device *vdev); int (*power_down)(struct ivpu_device *vdev); + int (*reset)(struct ivpu_device *vdev); bool (*is_idle)(struct ivpu_device *vdev); void (*wdt_disable)(struct ivpu_device *vdev); void (*diagnose_failure)(struct ivpu_device *vdev); @@ -91,6 +92,13 @@ static inline int ivpu_hw_power_down(struct ivpu_device *vdev) return vdev->hw->ops->power_down(vdev); }; +static inline int ivpu_hw_reset(struct ivpu_device *vdev) +{ + ivpu_dbg(vdev, PM, "HW reset\n"); + + return vdev->hw->ops->reset(vdev); +}; + static inline void ivpu_hw_wdt_disable(struct ivpu_device *vdev) { vdev->hw->ops->wdt_disable(vdev); diff --git a/drivers/accel/ivpu/ivpu_hw_37xx.c b/drivers/accel/ivpu/ivpu_hw_37xx.c index 9eae1c241bc0..976019429164 100644 --- a/drivers/accel/ivpu/ivpu_hw_37xx.c +++ b/drivers/accel/ivpu/ivpu_hw_37xx.c @@ -1029,6 +1029,7 @@ const struct ivpu_hw_ops ivpu_hw_37xx_ops = { .power_up = ivpu_hw_37xx_power_up, .is_idle = ivpu_hw_37xx_is_idle, .power_down = ivpu_hw_37xx_power_down, + .reset = ivpu_hw_37xx_reset, .boot_fw = ivpu_hw_37xx_boot_fw, .wdt_disable = ivpu_hw_37xx_wdt_disable, .diagnose_failure = ivpu_hw_37xx_diagnose_failure, diff --git a/drivers/accel/ivpu/ivpu_hw_40xx.c b/drivers/accel/ivpu/ivpu_hw_40xx.c index 34626d66fa10..85171a408363 100644 --- a/drivers/accel/ivpu/ivpu_hw_40xx.c +++ b/drivers/accel/ivpu/ivpu_hw_40xx.c @@ -57,8 +57,7 @@ #define ICB_0_1_IRQ_MASK ((((u64)ICB_1_IRQ_MASK) << 32) | ICB_0_IRQ_MASK) -#define BUTTRESS_IRQ_MASK ((REG_FLD(VPU_40XX_BUTTRESS_INTERRUPT_STAT, FREQ_CHANGE)) | \ - (REG_FLD(VPU_40XX_BUTTRESS_INTERRUPT_STAT, ATS_ERR)) | \ +#define BUTTRESS_IRQ_MASK ((REG_FLD(VPU_40XX_BUTTRESS_INTERRUPT_STAT, ATS_ERR)) | \ (REG_FLD(VPU_40XX_BUTTRESS_INTERRUPT_STAT, CFI0_ERR)) | \ (REG_FLD(VPU_40XX_BUTTRESS_INTERRUPT_STAT, CFI1_ERR)) | \ (REG_FLD(VPU_40XX_BUTTRESS_INTERRUPT_STAT, IMR0_ERR)) | \ @@ -196,6 +195,14 @@ static int ivpu_pll_wait_for_status_ready(struct ivpu_device *vdev) return REGB_POLL_FLD(VPU_40XX_BUTTRESS_VPU_STATUS, READY, 1, PLL_TIMEOUT_US); } +static int ivpu_wait_for_clock_own_resource_ack(struct ivpu_device *vdev) +{ + if (ivpu_is_simics(vdev)) + return 0; + + return REGB_POLL_FLD(VPU_40XX_BUTTRESS_VPU_STATUS, CLOCK_RESOURCE_OWN_ACK, 1, TIMEOUT_US); +} + static void ivpu_pll_init_frequency_ratios(struct ivpu_device *vdev) { struct ivpu_hw_info *hw = vdev->hw; @@ -556,6 +563,12 @@ static int ivpu_boot_pwr_domain_enable(struct ivpu_device *vdev) { int ret; + ret = ivpu_wait_for_clock_own_resource_ack(vdev); + if (ret) { + ivpu_err(vdev, "Timed out waiting for clock own resource ACK\n"); + return ret; + } + ivpu_boot_pwr_island_trickle_drive(vdev, true); ivpu_boot_pwr_island_drive(vdev, true); @@ -1046,8 +1059,6 @@ static irqreturn_t ivpu_hw_40xx_irqb_handler(struct ivpu_device *vdev, int irq) if (status == 0) return IRQ_NONE; - REGB_WR32(VPU_40XX_BUTTRESS_INTERRUPT_STAT, status); - if (REG_TEST_FLD(VPU_40XX_BUTTRESS_INTERRUPT_STAT, FREQ_CHANGE, status)) ivpu_dbg(vdev, IRQ, "FREQ_CHANGE"); @@ -1092,6 +1103,9 @@ static irqreturn_t ivpu_hw_40xx_irqb_handler(struct ivpu_device *vdev, int irq) schedule_recovery = true; } + /* This must be done after interrupts are cleared at the source. */ + REGB_WR32(VPU_40XX_BUTTRESS_INTERRUPT_STAT, status); + if (schedule_recovery) ivpu_pm_schedule_recovery(vdev); @@ -1103,9 +1117,14 @@ static irqreturn_t ivpu_hw_40xx_irq_handler(int irq, void *ptr) struct ivpu_device *vdev = ptr; irqreturn_t ret = IRQ_NONE; + REGB_WR32(VPU_40XX_BUTTRESS_GLOBAL_INT_MASK, 0x1); + ret |= ivpu_hw_40xx_irqv_handler(vdev, irq); ret |= ivpu_hw_40xx_irqb_handler(vdev, irq); + /* Re-enable global interrupts to re-trigger MSI for pending interrupts */ + REGB_WR32(VPU_40XX_BUTTRESS_GLOBAL_INT_MASK, 0x0); + if (ret & IRQ_WAKE_THREAD) return IRQ_WAKE_THREAD; @@ -1160,6 +1179,7 @@ const struct ivpu_hw_ops ivpu_hw_40xx_ops = { .power_up = ivpu_hw_40xx_power_up, .is_idle = ivpu_hw_40xx_is_idle, .power_down = ivpu_hw_40xx_power_down, + .reset = ivpu_hw_40xx_reset, .boot_fw = ivpu_hw_40xx_boot_fw, .wdt_disable = ivpu_hw_40xx_wdt_disable, .diagnose_failure = ivpu_hw_40xx_diagnose_failure, diff --git a/drivers/accel/ivpu/ivpu_hw_40xx_reg.h b/drivers/accel/ivpu/ivpu_hw_40xx_reg.h index 5139cfe88532..ff4a5d4f5821 100644 --- a/drivers/accel/ivpu/ivpu_hw_40xx_reg.h +++ b/drivers/accel/ivpu/ivpu_hw_40xx_reg.h @@ -70,6 +70,8 @@ #define VPU_40XX_BUTTRESS_VPU_STATUS_READY_MASK BIT_MASK(0) #define VPU_40XX_BUTTRESS_VPU_STATUS_IDLE_MASK BIT_MASK(1) #define VPU_40XX_BUTTRESS_VPU_STATUS_DUP_IDLE_MASK BIT_MASK(2) +#define VPU_40XX_BUTTRESS_VPU_STATUS_CLOCK_RESOURCE_OWN_ACK_MASK BIT_MASK(6) +#define VPU_40XX_BUTTRESS_VPU_STATUS_POWER_RESOURCE_OWN_ACK_MASK BIT_MASK(7) #define VPU_40XX_BUTTRESS_VPU_STATUS_PERF_CLK_MASK BIT_MASK(11) #define VPU_40XX_BUTTRESS_VPU_STATUS_DISABLE_CLK_RELINQUISH_MASK BIT_MASK(12) diff --git a/drivers/accel/ivpu/ivpu_ipc.c b/drivers/accel/ivpu/ivpu_ipc.c index fa0af59e39ab..295c0d7b5039 100644 --- a/drivers/accel/ivpu/ivpu_ipc.c +++ b/drivers/accel/ivpu/ivpu_ipc.c @@ -209,10 +209,10 @@ int ivpu_ipc_receive(struct ivpu_device *vdev, struct ivpu_ipc_consumer *cons, struct ivpu_ipc_rx_msg *rx_msg; int wait_ret, ret = 0; - wait_ret = wait_event_interruptible_timeout(cons->rx_msg_wq, - (IS_KTHREAD() && kthread_should_stop()) || - !list_empty(&cons->rx_msg_list), - msecs_to_jiffies(timeout_ms)); + wait_ret = wait_event_timeout(cons->rx_msg_wq, + (IS_KTHREAD() && kthread_should_stop()) || + !list_empty(&cons->rx_msg_list), + msecs_to_jiffies(timeout_ms)); if (IS_KTHREAD() && kthread_should_stop()) return -EINTR; @@ -220,9 +220,6 @@ int ivpu_ipc_receive(struct ivpu_device *vdev, struct ivpu_ipc_consumer *cons, if (wait_ret == 0) return -ETIMEDOUT; - if (wait_ret < 0) - return -ERESTARTSYS; - spin_lock_irq(&cons->rx_msg_lock); rx_msg = list_first_entry_or_null(&cons->rx_msg_list, struct ivpu_ipc_rx_msg, link); if (!rx_msg) { diff --git a/drivers/accel/ivpu/ivpu_mmu_context.c b/drivers/accel/ivpu/ivpu_mmu_context.c index 1d2e554e2c4a..ce94f4029127 100644 --- a/drivers/accel/ivpu/ivpu_mmu_context.c +++ b/drivers/accel/ivpu/ivpu_mmu_context.c @@ -11,6 +11,7 @@ #include "ivpu_mmu.h" #include "ivpu_mmu_context.h" +#define IVPU_MMU_VPU_ADDRESS_MASK GENMASK(47, 12) #define IVPU_MMU_PGD_INDEX_MASK GENMASK(47, 39) #define IVPU_MMU_PUD_INDEX_MASK GENMASK(38, 30) #define IVPU_MMU_PMD_INDEX_MASK GENMASK(29, 21) @@ -328,12 +329,8 @@ ivpu_mmu_context_map_sgt(struct ivpu_device *vdev, struct ivpu_mmu_context *ctx, if (!IS_ALIGNED(vpu_addr, IVPU_MMU_PAGE_SIZE)) return -EINVAL; - /* - * VPU is only 32 bit, but DMA engine is 38 bit - * Ranges < 2 GB are reserved for VPU internal registers - * Limit range to 8 GB - */ - if (vpu_addr < SZ_2G || vpu_addr > SZ_8G) + + if (vpu_addr & ~IVPU_MMU_VPU_ADDRESS_MASK) return -EINVAL; prot = IVPU_MMU_ENTRY_MAPPED; diff --git a/drivers/accel/ivpu/ivpu_pm.c b/drivers/accel/ivpu/ivpu_pm.c index e6f27daf5560..ffff2496e8e8 100644 --- a/drivers/accel/ivpu/ivpu_pm.c +++ b/drivers/accel/ivpu/ivpu_pm.c @@ -261,7 +261,8 @@ void ivpu_pm_reset_prepare_cb(struct pci_dev *pdev) ivpu_dbg(vdev, PM, "Pre-reset..\n"); atomic_inc(&vdev->pm->reset_counter); atomic_set(&vdev->pm->in_reset, 1); - ivpu_shutdown(vdev); + ivpu_prepare_for_reset(vdev); + ivpu_hw_reset(vdev); ivpu_pm_prepare_cold_boot(vdev); ivpu_jobs_abort_all(vdev); ivpu_dbg(vdev, PM, "Pre-reset done.\n"); diff --git a/drivers/acpi/acpi_processor.c b/drivers/acpi/acpi_processor.c index c711db8a9c33..0f5218e361df 100644 --- a/drivers/acpi/acpi_processor.c +++ b/drivers/acpi/acpi_processor.c @@ -12,6 +12,7 @@ #define pr_fmt(fmt) "ACPI: " fmt #include +#include #include #include #include diff --git a/drivers/acpi/acpi_video.c b/drivers/acpi/acpi_video.c index 948e31f7ce6e..b411948594ff 100644 --- a/drivers/acpi/acpi_video.c +++ b/drivers/acpi/acpi_video.c @@ -2057,7 +2057,9 @@ static int acpi_video_bus_add(struct acpi_device *device) !auto_detect) acpi_video_bus_register_backlight(video); - acpi_video_bus_add_notify_handler(video); + error = acpi_video_bus_add_notify_handler(video); + if (error) + goto err_del; error = acpi_dev_install_notify_handler(device, ACPI_DEVICE_NOTIFY, acpi_video_bus_notify); @@ -2067,10 +2069,11 @@ static int acpi_video_bus_add(struct acpi_device *device) return 0; err_remove: + acpi_video_bus_remove_notify_handler(video); +err_del: mutex_lock(&video_list_lock); list_del(&video->entry); mutex_unlock(&video_list_lock); - acpi_video_bus_remove_notify_handler(video); acpi_video_bus_unregister_backlight(video); err_put_video: acpi_video_bus_put_devices(video); diff --git a/drivers/acpi/bus.c b/drivers/acpi/bus.c index f41dda2d3493..a4aa53b7e2bb 100644 --- a/drivers/acpi/bus.c +++ b/drivers/acpi/bus.c @@ -1410,10 +1410,10 @@ static int __init acpi_init(void) acpi_init_ffh(); pci_mmcfg_late_init(); - acpi_arm_init(); acpi_viot_early_init(); acpi_hest_init(); acpi_ghes_init(); + acpi_arm_init(); acpi_scan_init(); acpi_ec_init(); acpi_debugfs_init(); diff --git a/drivers/acpi/ec.c b/drivers/acpi/ec.c index 660834a49c1f..c95d0edb0be9 100644 --- a/drivers/acpi/ec.c +++ b/drivers/acpi/ec.c @@ -1913,6 +1913,17 @@ static const struct dmi_system_id ec_dmi_table[] __initconst = { DMI_MATCH(DMI_PRODUCT_NAME, "HP 15-cx0041ur"), }, }, + { + /* + * HP Pavilion Gaming Laptop 15-dk1xxx + * https://github.com/systemd/systemd/issues/28942 + */ + .callback = ec_honor_dsdt_gpe, + .matches = { + DMI_MATCH(DMI_SYS_VENDOR, "HP"), + DMI_MATCH(DMI_PRODUCT_NAME, "HP Pavilion Gaming Laptop 15-dk1xxx"), + }, + }, { /* * Samsung hardware diff --git a/drivers/acpi/irq.c b/drivers/acpi/irq.c index c2c786eb95ab..1687483ff319 100644 --- a/drivers/acpi/irq.c +++ b/drivers/acpi/irq.c @@ -57,6 +57,7 @@ int acpi_register_gsi(struct device *dev, u32 gsi, int trigger, int polarity) { struct irq_fwspec fwspec; + unsigned int irq; fwspec.fwnode = acpi_get_gsi_domain_id(gsi); if (WARN_ON(!fwspec.fwnode)) { @@ -68,7 +69,11 @@ int acpi_register_gsi(struct device *dev, u32 gsi, int trigger, fwspec.param[1] = acpi_dev_get_irq_type(trigger, polarity); fwspec.param_count = 2; - return irq_create_fwspec_mapping(&fwspec); + irq = irq_create_fwspec_mapping(&fwspec); + if (!irq) + return -EINVAL; + + return irq; } EXPORT_SYMBOL_GPL(acpi_register_gsi); diff --git a/drivers/acpi/nfit/core.c b/drivers/acpi/nfit/core.c index f0e6738ae3c9..f96bf32cd368 100644 --- a/drivers/acpi/nfit/core.c +++ b/drivers/acpi/nfit/core.c @@ -855,7 +855,7 @@ static size_t sizeof_idt(struct acpi_nfit_interleave *idt) { if (idt->header.length < sizeof(*idt)) return 0; - return sizeof(*idt) + sizeof(u32) * (idt->line_count - 1); + return sizeof(*idt) + sizeof(u32) * idt->line_count; } static bool add_idt(struct acpi_nfit_desc *acpi_desc, diff --git a/drivers/acpi/processor_idle.c b/drivers/acpi/processor_idle.c index dc615ef6550a..3a34a8c425fe 100644 --- a/drivers/acpi/processor_idle.c +++ b/drivers/acpi/processor_idle.c @@ -1217,8 +1217,7 @@ static int acpi_processor_setup_lpi_states(struct acpi_processor *pr) strscpy(state->desc, lpi->desc, CPUIDLE_DESC_LEN); state->exit_latency = lpi->wake_latency; state->target_residency = lpi->min_residency; - if (lpi->arch_flags) - state->flags |= CPUIDLE_FLAG_TIMER_STOP; + state->flags |= arch_get_idle_state_flags(lpi->arch_flags); if (i != 0 && lpi->entry_method == ACPI_CSTATE_FFH) state->flags |= CPUIDLE_FLAG_RCU_IDLE; state->enter = acpi_idle_lpi_enter; diff --git a/drivers/acpi/processor_pdc.c b/drivers/acpi/processor_pdc.c index 1a8591e9a9bf..994091bd52de 100644 --- a/drivers/acpi/processor_pdc.c +++ b/drivers/acpi/processor_pdc.c @@ -19,6 +19,7 @@ static void acpi_set_pdc_bits(u32 *buf) { buf[0] = ACPI_PDC_REVISION_ID; buf[1] = 1; + buf[2] = 0; /* Twiddle arch-specific bits needed for _PDC */ arch_acpi_set_proc_cap_bits(&buf[2]); diff --git a/drivers/acpi/resource.c b/drivers/acpi/resource.c index 32cfa3f4efd3..297a88587031 100644 --- a/drivers/acpi/resource.c +++ b/drivers/acpi/resource.c @@ -439,6 +439,13 @@ static const struct dmi_system_id asus_laptop[] = { DMI_MATCH(DMI_BOARD_NAME, "S5602ZA"), }, }, + { + .ident = "Asus ExpertBook B1402CBA", + .matches = { + DMI_MATCH(DMI_SYS_VENDOR, "ASUSTeK COMPUTER INC."), + DMI_MATCH(DMI_BOARD_NAME, "B1402CBA"), + }, + }, { .ident = "Asus ExpertBook B1502CBA", .matches = { @@ -500,16 +507,23 @@ static const struct dmi_system_id maingear_laptop[] = { static const struct dmi_system_id pcspecialist_laptop[] = { { - .ident = "PCSpecialist Elimina Pro 16 M", - /* - * Some models have product-name "Elimina Pro 16 M", - * others "GM6BGEQ". Match on board-name to match both. - */ + /* TongFang GM6BGEQ / PCSpecialist Elimina Pro 16 M, RTX 3050 */ .matches = { - DMI_MATCH(DMI_SYS_VENDOR, "PCSpecialist"), DMI_MATCH(DMI_BOARD_NAME, "GM6BGEQ"), }, }, + { + /* TongFang GM6BG5Q, RTX 4050 */ + .matches = { + DMI_MATCH(DMI_BOARD_NAME, "GM6BG5Q"), + }, + }, + { + /* TongFang GM6BG0Q / PCSpecialist Elimina Pro 16 M, RTX 4060 */ + .matches = { + DMI_MATCH(DMI_BOARD_NAME, "GM6BG0Q"), + }, + }, { } }; diff --git a/drivers/acpi/thermal.c b/drivers/acpi/thermal.c index f14e68266ccd..312730f8272e 100644 --- a/drivers/acpi/thermal.c +++ b/drivers/acpi/thermal.c @@ -492,7 +492,7 @@ static int thermal_get_temp(struct thermal_zone_device *thermal, int *temp) } static int thermal_get_trend(struct thermal_zone_device *thermal, - struct thermal_trip *trip, + const struct thermal_trip *trip, enum thermal_trend *trend) { struct acpi_thermal *tz = thermal_zone_device_priv(thermal); diff --git a/drivers/android/binder.c b/drivers/android/binder.c index 367afac5f1bf..92128aae2d06 100644 --- a/drivers/android/binder.c +++ b/drivers/android/binder.c @@ -4812,6 +4812,8 @@ static void binder_release_work(struct binder_proc *proc, "undelivered TRANSACTION_ERROR: %u\n", e->cmd); } break; + case BINDER_WORK_TRANSACTION_PENDING: + case BINDER_WORK_TRANSACTION_ONEWAY_SPAM_SUSPECT: case BINDER_WORK_TRANSACTION_COMPLETE: { binder_debug(BINDER_DEBUG_DEAD_TRANSACTION, "undelivered TRANSACTION_COMPLETE\n"); diff --git a/drivers/ata/ahci.c b/drivers/ata/ahci.c index abb5911c9d09..08745e7db820 100644 --- a/drivers/ata/ahci.c +++ b/drivers/ata/ahci.c @@ -1883,6 +1883,15 @@ static int ahci_init_one(struct pci_dev *pdev, const struct pci_device_id *ent) else dev_info(&pdev->dev, "SSS flag set, parallel bus scan disabled\n"); + if (!(hpriv->cap & HOST_CAP_PART)) + host->flags |= ATA_HOST_NO_PART; + + if (!(hpriv->cap & HOST_CAP_SSC)) + host->flags |= ATA_HOST_NO_SSC; + + if (!(hpriv->cap2 & HOST_CAP2_SDS)) + host->flags |= ATA_HOST_NO_DEVSLP; + if (pi.flags & ATA_FLAG_EM) ahci_reset_em(host); diff --git a/drivers/ata/libahci.c b/drivers/ata/libahci.c index e2bacedf28ef..f1263364fa97 100644 --- a/drivers/ata/libahci.c +++ b/drivers/ata/libahci.c @@ -1256,6 +1256,26 @@ static ssize_t ahci_activity_show(struct ata_device *dev, char *buf) return sprintf(buf, "%d\n", emp->blink_policy); } +static void ahci_port_clear_pending_irq(struct ata_port *ap) +{ + struct ahci_host_priv *hpriv = ap->host->private_data; + void __iomem *port_mmio = ahci_port_base(ap); + u32 tmp; + + /* clear SError */ + tmp = readl(port_mmio + PORT_SCR_ERR); + dev_dbg(ap->host->dev, "PORT_SCR_ERR 0x%x\n", tmp); + writel(tmp, port_mmio + PORT_SCR_ERR); + + /* clear port IRQ */ + tmp = readl(port_mmio + PORT_IRQ_STAT); + dev_dbg(ap->host->dev, "PORT_IRQ_STAT 0x%x\n", tmp); + if (tmp) + writel(tmp, port_mmio + PORT_IRQ_STAT); + + writel(1 << ap->port_no, hpriv->mmio + HOST_IRQ_STAT); +} + static void ahci_port_init(struct device *dev, struct ata_port *ap, int port_no, void __iomem *mmio, void __iomem *port_mmio) @@ -1270,18 +1290,7 @@ static void ahci_port_init(struct device *dev, struct ata_port *ap, if (rc) dev_warn(dev, "%s (%d)\n", emsg, rc); - /* clear SError */ - tmp = readl(port_mmio + PORT_SCR_ERR); - dev_dbg(dev, "PORT_SCR_ERR 0x%x\n", tmp); - writel(tmp, port_mmio + PORT_SCR_ERR); - - /* clear port IRQ */ - tmp = readl(port_mmio + PORT_IRQ_STAT); - dev_dbg(dev, "PORT_IRQ_STAT 0x%x\n", tmp); - if (tmp) - writel(tmp, port_mmio + PORT_IRQ_STAT); - - writel(1 << port_no, mmio + HOST_IRQ_STAT); + ahci_port_clear_pending_irq(ap); /* mark esata ports */ tmp = readl(port_mmio + PORT_CMD); @@ -1603,6 +1612,8 @@ int ahci_do_hardreset(struct ata_link *link, unsigned int *class, tf.status = ATA_BUSY; ata_tf_to_fis(&tf, 0, 0, d2h_fis); + ahci_port_clear_pending_irq(ap); + rc = sata_link_hardreset(link, timing, deadline, online, ahci_check_ready); diff --git a/drivers/ata/libata-core.c b/drivers/ata/libata-core.c index 74314311295f..d8cc1e27a125 100644 --- a/drivers/ata/libata-core.c +++ b/drivers/ata/libata-core.c @@ -1972,6 +1972,96 @@ retry: return rc; } +/** + * ata_dev_power_set_standby - Set a device power mode to standby + * @dev: target device + * + * Issue a STANDBY IMMEDIATE command to set a device power mode to standby. + * For an HDD device, this spins down the disks. + * + * LOCKING: + * Kernel thread context (may sleep). + */ +void ata_dev_power_set_standby(struct ata_device *dev) +{ + unsigned long ap_flags = dev->link->ap->flags; + struct ata_taskfile tf; + unsigned int err_mask; + + /* Issue STANDBY IMMEDIATE command only if supported by the device */ + if (dev->class != ATA_DEV_ATA && dev->class != ATA_DEV_ZAC) + return; + + /* + * Some odd clown BIOSes issue spindown on power off (ACPI S4 or S5) + * causing some drives to spin up and down again. For these, do nothing + * if we are being called on shutdown. + */ + if ((ap_flags & ATA_FLAG_NO_POWEROFF_SPINDOWN) && + system_state == SYSTEM_POWER_OFF) + return; + + if ((ap_flags & ATA_FLAG_NO_HIBERNATE_SPINDOWN) && + system_entering_hibernation()) + return; + + ata_tf_init(dev, &tf); + tf.flags |= ATA_TFLAG_DEVICE | ATA_TFLAG_ISADDR; + tf.protocol = ATA_PROT_NODATA; + tf.command = ATA_CMD_STANDBYNOW1; + + ata_dev_notice(dev, "Entering standby power mode\n"); + + err_mask = ata_exec_internal(dev, &tf, NULL, DMA_NONE, NULL, 0, 0); + if (err_mask) + ata_dev_err(dev, "STANDBY IMMEDIATE failed (err_mask=0x%x)\n", + err_mask); +} + +/** + * ata_dev_power_set_active - Set a device power mode to active + * @dev: target device + * + * Issue a VERIFY command to enter to ensure that the device is in the + * active power mode. For a spun-down HDD (standby or idle power mode), + * the VERIFY command will complete after the disk spins up. + * + * LOCKING: + * Kernel thread context (may sleep). + */ +void ata_dev_power_set_active(struct ata_device *dev) +{ + struct ata_taskfile tf; + unsigned int err_mask; + + /* + * Issue READ VERIFY SECTORS command for 1 sector at lba=0 only + * if supported by the device. + */ + if (dev->class != ATA_DEV_ATA && dev->class != ATA_DEV_ZAC) + return; + + ata_tf_init(dev, &tf); + tf.flags |= ATA_TFLAG_DEVICE | ATA_TFLAG_ISADDR; + tf.protocol = ATA_PROT_NODATA; + tf.command = ATA_CMD_VERIFY; + tf.nsect = 1; + if (dev->flags & ATA_DFLAG_LBA) { + tf.flags |= ATA_TFLAG_LBA; + tf.device |= ATA_LBA; + } else { + /* CHS */ + tf.lbal = 0x1; /* sect */ + } + + ata_dev_notice(dev, "Entering active power mode\n"); + + err_mask = ata_exec_internal(dev, &tf, NULL, DMA_NONE, NULL, 0, 0); + if (err_mask) + ata_dev_err(dev, "VERIFY failed (err_mask=0x%x)\n", + err_mask); +} + /** * ata_read_log_page - read a specific log page * @dev: target device @@ -2529,7 +2619,7 @@ static int ata_dev_config_lba(struct ata_device *dev) { const u16 *id = dev->id; const char *lba_desc; - char ncq_desc[24]; + char ncq_desc[32]; int ret; dev->flags |= ATA_DFLAG_LBA; @@ -4783,11 +4873,8 @@ void ata_qc_complete(struct ata_queued_cmd *qc) * been aborted by the device due to a limit timeout using the policy * 0xD. For these commands, invoke EH to get the command sense data. */ - if (qc->result_tf.status & ATA_SENSE && - ((ata_is_ncq(qc->tf.protocol) && - dev->flags & ATA_DFLAG_CDL_ENABLED) || - (!ata_is_ncq(qc->tf.protocol) && - ata_id_sense_reporting_enabled(dev->id)))) { + if (qc->flags & ATA_QCFLAG_HAS_CDL && + qc->result_tf.status & ATA_SENSE) { /* * Tell SCSI EH to not overwrite scmd->result even if this * command is finished with result SAM_STAT_GOOD. @@ -5040,17 +5127,19 @@ static void ata_port_request_pm(struct ata_port *ap, pm_message_t mesg, struct ata_link *link; unsigned long flags; - /* Previous resume operation might still be in - * progress. Wait for PM_PENDING to clear. + spin_lock_irqsave(ap->lock, flags); + + /* + * A previous PM operation might still be in progress. Wait for + * ATA_PFLAG_PM_PENDING to clear. */ if (ap->pflags & ATA_PFLAG_PM_PENDING) { + spin_unlock_irqrestore(ap->lock, flags); ata_port_wait_eh(ap); - WARN_ON(ap->pflags & ATA_PFLAG_PM_PENDING); + spin_lock_irqsave(ap->lock, flags); } - /* request PM ops to EH */ - spin_lock_irqsave(ap->lock, flags); - + /* Request PM operation to EH */ ap->pm_mesg = mesg; ap->pflags |= ATA_PFLAG_PM_PENDING; ata_for_each_link(link, ap, HOST_FIRST) { @@ -5062,10 +5151,8 @@ static void ata_port_request_pm(struct ata_port *ap, pm_message_t mesg, spin_unlock_irqrestore(ap->lock, flags); - if (!async) { + if (!async) ata_port_wait_eh(ap); - WARN_ON(ap->pflags & ATA_PFLAG_PM_PENDING); - } } /* @@ -5081,11 +5168,27 @@ static const unsigned int ata_port_suspend_ehi = ATA_EHI_QUIET static void ata_port_suspend(struct ata_port *ap, pm_message_t mesg) { + /* + * We are about to suspend the port, so we do not care about + * scsi_rescan_device() calls scheduled by previous resume operations. + * The next resume will schedule the rescan again. So cancel any rescan + * that is not done yet. + */ + cancel_delayed_work_sync(&ap->scsi_rescan_task); + ata_port_request_pm(ap, mesg, 0, ata_port_suspend_ehi, false); } static void ata_port_suspend_async(struct ata_port *ap, pm_message_t mesg) { + /* + * We are about to suspend the port, so we do not care about + * scsi_rescan_device() calls scheduled by previous resume operations. + * The next resume will schedule the rescan again. So cancel any rescan + * that is not done yet. + */ + cancel_delayed_work_sync(&ap->scsi_rescan_task); + ata_port_request_pm(ap, mesg, 0, ata_port_suspend_ehi, true); } @@ -5232,7 +5335,7 @@ EXPORT_SYMBOL_GPL(ata_host_resume); #endif const struct device_type ata_port_type = { - .name = "ata_port", + .name = ATA_PORT_TYPE_NAME, #ifdef CONFIG_PM .pm = &ata_port_pm_ops, #endif @@ -5951,11 +6054,30 @@ static void ata_port_detach(struct ata_port *ap) struct ata_link *link; struct ata_device *dev; - /* tell EH we're leaving & flush EH */ + /* Wait for any ongoing EH */ + ata_port_wait_eh(ap); + + mutex_lock(&ap->scsi_scan_mutex); spin_lock_irqsave(ap->lock, flags); + + /* Remove scsi devices */ + ata_for_each_link(link, ap, HOST_FIRST) { + ata_for_each_dev(dev, link, ALL) { + if (dev->sdev) { + spin_unlock_irqrestore(ap->lock, flags); + scsi_remove_device(dev->sdev); + spin_lock_irqsave(ap->lock, flags); + dev->sdev = NULL; + } + } + } + + /* Tell EH to disable all devices */ ap->pflags |= ATA_PFLAG_UNLOADING; ata_port_schedule_eh(ap); + spin_unlock_irqrestore(ap->lock, flags); + mutex_unlock(&ap->scsi_scan_mutex); /* wait till EH commits suicide */ ata_port_wait_eh(ap); diff --git a/drivers/ata/libata-eh.c b/drivers/ata/libata-eh.c index 159ba6ba19eb..5686353e442c 100644 --- a/drivers/ata/libata-eh.c +++ b/drivers/ata/libata-eh.c @@ -147,6 +147,8 @@ ata_eh_cmd_timeout_table[ATA_EH_CMD_TIMEOUT_TABLE_SIZE] = { .timeouts = ata_eh_other_timeouts, }, { .commands = CMDS(ATA_CMD_FLUSH, ATA_CMD_FLUSH_EXT), .timeouts = ata_eh_flush_timeouts }, + { .commands = CMDS(ATA_CMD_VERIFY), + .timeouts = ata_eh_reset_timeouts }, }; #undef CMDS @@ -498,7 +500,19 @@ static void ata_eh_unload(struct ata_port *ap) struct ata_device *dev; unsigned long flags; - /* Restore SControl IPM and SPD for the next driver and + /* + * Unless we are restarting, transition all enabled devices to + * standby power mode. + */ + if (system_state != SYSTEM_RESTART) { + ata_for_each_link(link, ap, PMP_FIRST) { + ata_for_each_dev(dev, link, ENABLED) + ata_dev_power_set_standby(dev); + } + } + + /* + * Restore SControl IPM and SPD for the next driver and * disable attached devices. */ ata_for_each_link(link, ap, PMP_FIRST) { @@ -684,6 +698,10 @@ void ata_scsi_port_error_handler(struct Scsi_Host *host, struct ata_port *ap) ehc->saved_xfer_mode[devno] = dev->xfer_mode; if (ata_ncq_enabled(dev)) ehc->saved_ncq_enabled |= 1 << devno; + + /* If we are resuming, wake up the device */ + if (ap->pflags & ATA_PFLAG_RESUMING) + ehc->i.dev_action[devno] |= ATA_EH_SET_ACTIVE; } } @@ -743,6 +761,8 @@ void ata_scsi_port_error_handler(struct Scsi_Host *host, struct ata_port *ap) /* clean up */ spin_lock_irqsave(ap->lock, flags); + ap->pflags &= ~ATA_PFLAG_RESUMING; + if (ap->pflags & ATA_PFLAG_LOADING) ap->pflags &= ~ATA_PFLAG_LOADING; else if ((ap->pflags & ATA_PFLAG_SCSI_HOTPLUG) && @@ -1218,6 +1238,13 @@ void ata_eh_detach_dev(struct ata_device *dev) struct ata_eh_context *ehc = &link->eh_context; unsigned long flags; + /* + * If the device is still enabled, transition it to standby power mode + * (i.e. spin down HDDs). + */ + if (ata_dev_enabled(dev)) + ata_dev_power_set_standby(dev); + ata_dev_disable(dev); spin_lock_irqsave(ap->lock, flags); @@ -2305,7 +2332,7 @@ static void ata_eh_link_report(struct ata_link *link) struct ata_eh_context *ehc = &link->eh_context; struct ata_queued_cmd *qc; const char *frozen, *desc; - char tries_buf[6] = ""; + char tries_buf[16] = ""; int tag, nr_failed = 0; if (ehc->i.flags & ATA_EHI_QUIET) @@ -2796,23 +2823,13 @@ int ata_eh_reset(struct ata_link *link, int classify, } } - /* - * Some controllers can't be frozen very well and may set spurious - * error conditions during reset. Clear accumulated error - * information and re-thaw the port if frozen. As reset is the - * final recovery action and we cross check link onlineness against - * device classification later, no hotplug event is lost by this. - */ + /* clear cached SError */ spin_lock_irqsave(link->ap->lock, flags); - memset(&link->eh_info, 0, sizeof(link->eh_info)); + link->eh_info.serror = 0; if (slave) - memset(&slave->eh_info, 0, sizeof(link->eh_info)); - ap->pflags &= ~ATA_PFLAG_EH_PENDING; + slave->eh_info.serror = 0; spin_unlock_irqrestore(link->ap->lock, flags); - if (ata_port_is_frozen(ap)) - ata_eh_thaw_port(ap); - /* * Make sure onlineness and classification result correspond. * Hotplug could have happened during reset and some @@ -3026,6 +3043,15 @@ static int ata_eh_revalidate_and_attach(struct ata_link *link, if (ehc->i.flags & ATA_EHI_DID_RESET) readid_flags |= ATA_READID_POSTRESET; + /* + * When resuming, before executing any command, make sure to + * transition the device to the active power mode. + */ + if ((action & ATA_EH_SET_ACTIVE) && ata_dev_enabled(dev)) { + ata_dev_power_set_active(dev); + ata_eh_done(link, dev, ATA_EH_SET_ACTIVE); + } + if ((action & ATA_EH_REVALIDATE) && ata_dev_enabled(dev)) { WARN_ON(dev->class == ATA_DEV_PMP); @@ -3999,6 +4025,7 @@ static void ata_eh_handle_port_suspend(struct ata_port *ap) unsigned long flags; int rc = 0; struct ata_device *dev; + struct ata_link *link; /* are we suspending? */ spin_lock_irqsave(ap->lock, flags); @@ -4011,6 +4038,12 @@ static void ata_eh_handle_port_suspend(struct ata_port *ap) WARN_ON(ap->pflags & ATA_PFLAG_SUSPENDED); + /* Set all devices attached to the port in standby mode */ + ata_for_each_link(link, ap, HOST_FIRST) { + ata_for_each_dev(dev, link, ENABLED) + ata_dev_power_set_standby(dev); + } + /* * If we have a ZPODD attached, check its zero * power ready status before the port is frozen. @@ -4093,6 +4126,7 @@ static void ata_eh_handle_port_resume(struct ata_port *ap) /* update the flags */ spin_lock_irqsave(ap->lock, flags); ap->pflags &= ~(ATA_PFLAG_PM_PENDING | ATA_PFLAG_SUSPENDED); + ap->pflags |= ATA_PFLAG_RESUMING; spin_unlock_irqrestore(ap->lock, flags); } #endif /* CONFIG_PM */ diff --git a/drivers/ata/libata-sata.c b/drivers/ata/libata-sata.c index 5d31c08be013..a701e1538482 100644 --- a/drivers/ata/libata-sata.c +++ b/drivers/ata/libata-sata.c @@ -396,10 +396,23 @@ int sata_link_scr_lpm(struct ata_link *link, enum ata_lpm_policy policy, case ATA_LPM_MED_POWER_WITH_DIPM: case ATA_LPM_MIN_POWER_WITH_PARTIAL: case ATA_LPM_MIN_POWER: - if (ata_link_nr_enabled(link) > 0) - /* no restrictions on LPM transitions */ + if (ata_link_nr_enabled(link) > 0) { + /* assume no restrictions on LPM transitions */ scontrol &= ~(0x7 << 8); - else { + + /* + * If the controller does not support partial, slumber, + * or devsleep, then disallow these transitions. + */ + if (link->ap->host->flags & ATA_HOST_NO_PART) + scontrol |= (0x1 << 8); + + if (link->ap->host->flags & ATA_HOST_NO_SSC) + scontrol |= (0x2 << 8); + + if (link->ap->host->flags & ATA_HOST_NO_DEVSLP) + scontrol |= (0x4 << 8); + } else { /* empty port, power off */ scontrol &= ~0xf; scontrol |= (0x1 << 2); diff --git a/drivers/ata/libata-scsi.c b/drivers/ata/libata-scsi.c index d3f28b82c97b..a371b497035e 100644 --- a/drivers/ata/libata-scsi.c +++ b/drivers/ata/libata-scsi.c @@ -1050,14 +1050,13 @@ int ata_scsi_dev_config(struct scsi_device *sdev, struct ata_device *dev) } } else { sdev->sector_size = ata_id_logical_sector_size(dev->id); + /* - * Stop the drive on suspend but do not issue START STOP UNIT - * on resume as this is not necessary and may fail: the device - * will be woken up by ata_port_pm_resume() with a port reset - * and device revalidation. + * Ask the sd driver to issue START STOP UNIT on runtime suspend + * and resume only. For system level suspend/resume, devices + * power state is handled directly by libata EH. */ - sdev->manage_start_stop = 1; - sdev->no_start_on_resume = 1; + sdev->manage_runtime_start_stop = true; } /* @@ -1089,6 +1088,42 @@ int ata_scsi_dev_config(struct scsi_device *sdev, struct ata_device *dev) return 0; } +/** + * ata_scsi_slave_alloc - Early setup of SCSI device + * @sdev: SCSI device to examine + * + * This is called from scsi_alloc_sdev() when the scsi device + * associated with an ATA device is scanned on a port. + * + * LOCKING: + * Defined by SCSI layer. We don't really care. + */ + +int ata_scsi_slave_alloc(struct scsi_device *sdev) +{ + struct ata_port *ap = ata_shost_to_port(sdev->host); + struct device_link *link; + + ata_scsi_sdev_config(sdev); + + /* + * Create a link from the ata_port device to the scsi device to ensure + * that PM does suspend/resume in the correct order: the scsi device is + * consumer (child) and the ata port the supplier (parent). + */ + link = device_link_add(&sdev->sdev_gendev, &ap->tdev, + DL_FLAG_STATELESS | + DL_FLAG_PM_RUNTIME | DL_FLAG_RPM_ACTIVE); + if (!link) { + ata_port_err(ap, "Failed to create link to scsi device %s\n", + dev_name(&sdev->sdev_gendev)); + return -ENODEV; + } + + return 0; +} +EXPORT_SYMBOL_GPL(ata_scsi_slave_alloc); + /** * ata_scsi_slave_config - Set SCSI device attributes * @sdev: SCSI device to examine @@ -1105,14 +1140,11 @@ int ata_scsi_slave_config(struct scsi_device *sdev) { struct ata_port *ap = ata_shost_to_port(sdev->host); struct ata_device *dev = __ata_scsi_find_dev(ap, sdev); - int rc = 0; - - ata_scsi_sdev_config(sdev); if (dev) - rc = ata_scsi_dev_config(sdev, dev); + return ata_scsi_dev_config(sdev, dev); - return rc; + return 0; } EXPORT_SYMBOL_GPL(ata_scsi_slave_config); @@ -1136,6 +1168,8 @@ void ata_scsi_slave_destroy(struct scsi_device *sdev) unsigned long flags; struct ata_device *dev; + device_link_remove(&sdev->sdev_gendev, &ap->tdev); + spin_lock_irqsave(ap->lock, flags); dev = __ata_scsi_find_dev(ap, sdev); if (dev && dev->sdev) { @@ -1195,7 +1229,7 @@ static unsigned int ata_scsi_start_stop_xlat(struct ata_queued_cmd *qc) } if (cdb[4] & 0x1) { - tf->nsect = 1; /* 1 sector, lba=0 */ + tf->nsect = 1; /* 1 sector, lba=0 */ if (qc->dev->flags & ATA_DFLAG_LBA) { tf->flags |= ATA_TFLAG_LBA; @@ -1211,7 +1245,7 @@ static unsigned int ata_scsi_start_stop_xlat(struct ata_queued_cmd *qc) tf->lbah = 0x0; /* cyl high */ } - tf->command = ATA_CMD_VERIFY; /* READ VERIFY */ + tf->command = ATA_CMD_VERIFY; /* READ VERIFY */ } else { /* Some odd clown BIOSen issue spindown on power off (ACPI S4 * or S5) causing some drives to spin up and down again. @@ -1221,7 +1255,7 @@ static unsigned int ata_scsi_start_stop_xlat(struct ata_queued_cmd *qc) goto skip; if ((qc->ap->flags & ATA_FLAG_NO_HIBERNATE_SPINDOWN) && - system_entering_hibernation()) + system_entering_hibernation()) goto skip; /* Issue ATA STANDBY IMMEDIATE command */ @@ -1835,6 +1869,9 @@ static unsigned int ata_scsiop_inq_std(struct ata_scsi_args *args, u8 *rbuf) hdr[2] = 0x7; /* claim SPC-5 version compatibility */ } + if (args->dev->flags & ATA_DFLAG_CDL) + hdr[2] = 0xd; /* claim SPC-6 version compatibility */ + memcpy(rbuf, hdr, sizeof(hdr)); memcpy(&rbuf[8], "ATA ", 8); ata_id_string(args->id, &rbuf[16], ATA_ID_PROD, 16); @@ -4312,7 +4349,7 @@ void ata_scsi_simulate(struct ata_device *dev, struct scsi_cmnd *cmd) break; case MAINTENANCE_IN: - if (scsicmd[1] == MI_REPORT_SUPPORTED_OPERATION_CODES) + if ((scsicmd[1] & 0x1f) == MI_REPORT_SUPPORTED_OPERATION_CODES) ata_scsi_rbuf_fill(&args, ata_scsiop_maint_in); else ata_scsi_set_invalid_field(dev, cmd, 1, 0xff); @@ -4722,7 +4759,7 @@ void ata_scsi_dev_rescan(struct work_struct *work) struct ata_link *link; struct ata_device *dev; unsigned long flags; - bool delay_rescan = false; + int ret = 0; mutex_lock(&ap->scsi_scan_mutex); spin_lock_irqsave(ap->lock, flags); @@ -4731,37 +4768,34 @@ void ata_scsi_dev_rescan(struct work_struct *work) ata_for_each_dev(dev, link, ENABLED) { struct scsi_device *sdev = dev->sdev; + /* + * If the port was suspended before this was scheduled, + * bail out. + */ + if (ap->pflags & ATA_PFLAG_SUSPENDED) + goto unlock; + if (!sdev) continue; if (scsi_device_get(sdev)) continue; - /* - * If the rescan work was scheduled because of a resume - * event, the port is already fully resumed, but the - * SCSI device may not yet be fully resumed. In such - * case, executing scsi_rescan_device() may cause a - * deadlock with the PM code on device_lock(). Prevent - * this by giving up and retrying rescan after a short - * delay. - */ - delay_rescan = sdev->sdev_gendev.power.is_suspended; - if (delay_rescan) { - scsi_device_put(sdev); - break; - } - spin_unlock_irqrestore(ap->lock, flags); - scsi_rescan_device(sdev); + ret = scsi_rescan_device(sdev); scsi_device_put(sdev); spin_lock_irqsave(ap->lock, flags); + + if (ret) + goto unlock; } } +unlock: spin_unlock_irqrestore(ap->lock, flags); mutex_unlock(&ap->scsi_scan_mutex); - if (delay_rescan) + /* Reschedule with a delay if scsi_rescan_device() returned an error */ + if (ret) schedule_delayed_work(&ap->scsi_rescan_task, msecs_to_jiffies(5)); } diff --git a/drivers/ata/libata-transport.c b/drivers/ata/libata-transport.c index e4fb9d1b9b39..3e49a877500e 100644 --- a/drivers/ata/libata-transport.c +++ b/drivers/ata/libata-transport.c @@ -266,6 +266,10 @@ void ata_tport_delete(struct ata_port *ap) put_device(dev); } +static const struct device_type ata_port_sas_type = { + .name = ATA_PORT_TYPE_NAME, +}; + /** ata_tport_add - initialize a transport ATA port structure * * @parent: parent device @@ -283,7 +287,10 @@ int ata_tport_add(struct device *parent, struct device *dev = &ap->tdev; device_initialize(dev); - dev->type = &ata_port_type; + if (ap->flags & ATA_FLAG_SAS_HOST) + dev->type = &ata_port_sas_type; + else + dev->type = &ata_port_type; dev->parent = parent; ata_host_get(ap->host); diff --git a/drivers/ata/libata.h b/drivers/ata/libata.h index 6e7d352803bd..05ac80da8ebc 100644 --- a/drivers/ata/libata.h +++ b/drivers/ata/libata.h @@ -30,6 +30,8 @@ enum { ATA_DNXFER_QUIET = (1 << 31), }; +#define ATA_PORT_TYPE_NAME "ata_port" + extern atomic_t ata_print_id; extern int atapi_passthru16; extern int libata_fua; @@ -60,6 +62,8 @@ extern int ata_dev_reread_id(struct ata_device *dev, unsigned int readid_flags); extern int ata_dev_revalidate(struct ata_device *dev, unsigned int new_class, unsigned int readid_flags); extern int ata_dev_configure(struct ata_device *dev); +extern void ata_dev_power_set_standby(struct ata_device *dev); +extern void ata_dev_power_set_active(struct ata_device *dev); extern int sata_down_spd_limit(struct ata_link *link, u32 spd_limit); extern int ata_down_xfermask_limit(struct ata_device *dev, unsigned int sel); extern unsigned int ata_dev_set_feature(struct ata_device *dev, diff --git a/drivers/ata/pata_parport/comm.c b/drivers/ata/pata_parport/comm.c index 4839becbbd56..94b8d352102e 100644 --- a/drivers/ata/pata_parport/comm.c +++ b/drivers/ata/pata_parport/comm.c @@ -37,7 +37,7 @@ static int comm_read_regr(struct pi_adapter *pi, int cont, int regr) { int l, h, r; - r = regr + cont_map[cont]; + r = regr + cont_map[cont]; switch (pi->mode) { case 0: @@ -90,7 +90,6 @@ static void comm_connect(struct pi_adapter *pi) } static void comm_disconnect(struct pi_adapter *pi) - { w2(0); w2(0); w2(0); w2(4); w0(pi->saved_r0); @@ -172,12 +171,12 @@ static void comm_write_block(struct pi_adapter *pi, char *buf, int count) w4l(swab16(((u16 *)buf)[2 * k]) | swab16(((u16 *)buf)[2 * k + 1]) << 16); break; - } + } } static void comm_log_adapter(struct pi_adapter *pi) - -{ char *mode_string[5] = { "4-bit", "8-bit", "EPP-8", "EPP-16", "EPP-32" }; +{ + char *mode_string[5] = { "4-bit", "8-bit", "EPP-8", "EPP-16", "EPP-32" }; dev_info(&pi->dev, "DataStor Commuter at 0x%x, mode %d (%s), delay %d\n", diff --git a/drivers/ata/pata_parport/fit3.c b/drivers/ata/pata_parport/fit3.c index bad7aa920cdc..d2b81cf2e16d 100644 --- a/drivers/ata/pata_parport/fit3.c +++ b/drivers/ata/pata_parport/fit3.c @@ -9,11 +9,6 @@ * * The TD-2000 and certain older devices use a different protocol. * Try the fit2 protocol module with them. - * - * NB: The FIT adapters do not appear to support the control - * registers. So, we map ALT_STATUS to STATUS and NO-OP writes - * to the device control register - this means that IDE reset - * will not work on these devices. */ #include @@ -37,8 +32,7 @@ static void fit3_write_regr(struct pi_adapter *pi, int cont, int regr, int val) { - if (cont == 1) - return; + regr += cont << 3; switch (pi->mode) { case 0: @@ -59,11 +53,7 @@ static int fit3_read_regr(struct pi_adapter *pi, int cont, int regr) { int a, b; - if (cont) { - if (regr != 6) - return 0xff; - regr = 7; - } + regr += cont << 3; switch (pi->mode) { case 0: diff --git a/drivers/ata/pata_parport/pata_parport.c b/drivers/ata/pata_parport/pata_parport.c index 1af64d435d3c..a7adfdcb5e27 100644 --- a/drivers/ata/pata_parport/pata_parport.c +++ b/drivers/ata/pata_parport/pata_parport.c @@ -51,6 +51,13 @@ static void pata_parport_dev_select(struct ata_port *ap, unsigned int device) ata_sff_pause(ap); } +static void pata_parport_set_devctl(struct ata_port *ap, u8 ctl) +{ + struct pi_adapter *pi = ap->host->private_data; + + pi->proto->write_regr(pi, 1, 6, ctl); +} + static bool pata_parport_devchk(struct ata_port *ap, unsigned int device) { struct pi_adapter *pi = ap->host->private_data; @@ -64,7 +71,7 @@ static bool pata_parport_devchk(struct ata_port *ap, unsigned int device) pi->proto->write_regr(pi, 0, ATA_REG_NSECT, 0xaa); pi->proto->write_regr(pi, 0, ATA_REG_LBAL, 0x55); - pi->proto->write_regr(pi, 0, ATA_REG_NSECT, 055); + pi->proto->write_regr(pi, 0, ATA_REG_NSECT, 0x55); pi->proto->write_regr(pi, 0, ATA_REG_LBAL, 0xaa); nsect = pi->proto->read_regr(pi, 0, ATA_REG_NSECT); @@ -73,6 +80,72 @@ static bool pata_parport_devchk(struct ata_port *ap, unsigned int device) return (nsect == 0x55) && (lbal == 0xaa); } +static int pata_parport_wait_after_reset(struct ata_link *link, + unsigned int devmask, + unsigned long deadline) +{ + struct ata_port *ap = link->ap; + struct pi_adapter *pi = ap->host->private_data; + unsigned int dev0 = devmask & (1 << 0); + unsigned int dev1 = devmask & (1 << 1); + int rc, ret = 0; + + ata_msleep(ap, ATA_WAIT_AFTER_RESET); + + /* always check readiness of the master device */ + rc = ata_sff_wait_ready(link, deadline); + if (rc) { + /* + * some adapters return bogus values if master device is not + * present, so don't abort now if a slave device is present + */ + if (!dev1) + return rc; + ret = -ENODEV; + } + + /* + * if device 1 was found in ata_devchk, wait for register + * access briefly, then wait for BSY to clear. + */ + if (dev1) { + int i; + + pata_parport_dev_select(ap, 1); + + /* + * Wait for register access. Some ATAPI devices fail + * to set nsect/lbal after reset, so don't waste too + * much time on it. We're gonna wait for !BSY anyway. + */ + for (i = 0; i < 2; i++) { + u8 nsect, lbal; + + nsect = pi->proto->read_regr(pi, 0, ATA_REG_NSECT); + lbal = pi->proto->read_regr(pi, 0, ATA_REG_LBAL); + if (nsect == 1 && lbal == 1) + break; + /* give drive a breather */ + ata_msleep(ap, 50); + } + + rc = ata_sff_wait_ready(link, deadline); + if (rc) { + if (rc != -ENODEV) + return rc; + ret = rc; + } + } + + pata_parport_dev_select(ap, 0); + if (dev1) + pata_parport_dev_select(ap, 1); + if (dev0) + pata_parport_dev_select(ap, 0); + + return ret; +} + static int pata_parport_bus_softreset(struct ata_port *ap, unsigned int devmask, unsigned long deadline) { @@ -87,7 +160,7 @@ static int pata_parport_bus_softreset(struct ata_port *ap, unsigned int devmask, ap->last_ctl = ap->ctl; /* wait the port to become ready */ - return ata_sff_wait_after_reset(&ap->link, devmask, deadline); + return pata_parport_wait_after_reset(&ap->link, devmask, deadline); } static int pata_parport_softreset(struct ata_link *link, unsigned int *classes, @@ -252,6 +325,7 @@ static struct ata_port_operations pata_parport_port_ops = { .hardreset = NULL, .sff_dev_select = pata_parport_dev_select, + .sff_set_devctl = pata_parport_set_devctl, .sff_check_status = pata_parport_check_status, .sff_check_altstatus = pata_parport_check_altstatus, .sff_tf_load = pata_parport_tf_load, diff --git a/drivers/ata/sata_mv.c b/drivers/ata/sata_mv.c index d105db5c7d81..45e48d653c60 100644 --- a/drivers/ata/sata_mv.c +++ b/drivers/ata/sata_mv.c @@ -1255,8 +1255,8 @@ static void mv_dump_mem(struct device *dev, void __iomem *start, unsigned bytes) for (b = 0; b < bytes; ) { for (w = 0, o = 0; b < bytes && w < 4; w++) { - o += snprintf(linebuf + o, sizeof(linebuf) - o, - "%08x ", readl(start + b)); + o += scnprintf(linebuf + o, sizeof(linebuf) - o, + "%08x ", readl(start + b)); b += sizeof(u32); } dev_dbg(dev, "%s: %p: %s\n", diff --git a/drivers/base/core.c b/drivers/base/core.c index b7d7f410c256..4d8b315c48a1 100644 --- a/drivers/base/core.c +++ b/drivers/base/core.c @@ -3537,6 +3537,8 @@ int device_add(struct device *dev) /* subsystems can specify simple device enumeration */ else if (dev->bus && dev->bus->dev_name) error = dev_set_name(dev, "%s%u", dev->bus->dev_name, dev->id); + else + error = -EINVAL; if (error) goto name_error; diff --git a/drivers/base/regmap/regcache-rbtree.c b/drivers/base/regmap/regcache-rbtree.c index db716ffd083e..3db88bbcae0f 100644 --- a/drivers/base/regmap/regcache-rbtree.c +++ b/drivers/base/regmap/regcache-rbtree.c @@ -453,7 +453,8 @@ static int regcache_rbtree_write(struct regmap *map, unsigned int reg, if (!rbnode) return -ENOMEM; regcache_rbtree_set_register(map, rbnode, - reg - rbnode->base_reg, value); + (reg - rbnode->base_reg) / map->reg_stride, + value); regcache_rbtree_insert(map, &rbtree_ctx->root, rbnode); rbtree_ctx->cached_rbnode = rbnode; } diff --git a/drivers/base/regmap/regmap.c b/drivers/base/regmap/regmap.c index 884cb51c8f67..234a84ecde8b 100644 --- a/drivers/base/regmap/regmap.c +++ b/drivers/base/regmap/regmap.c @@ -1478,7 +1478,7 @@ static int dev_get_regmap_match(struct device *dev, void *res, void *data) /* If the user didn't specify a name match any */ if (data) - return !strcmp((*r)->name, data); + return (*r)->name && !strcmp((*r)->name, data); else return 1; } diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c index df1cd0f718b8..800f131222fc 100644 --- a/drivers/block/nbd.c +++ b/drivers/block/nbd.c @@ -1436,8 +1436,9 @@ static int nbd_start_device_ioctl(struct nbd_device *nbd) static void nbd_clear_sock_ioctl(struct nbd_device *nbd) { - blk_mark_disk_dead(nbd->disk); nbd_clear_sock(nbd); + disk_force_media_change(nbd->disk); + nbd_bdev_reset(nbd); if (test_and_clear_bit(NBD_RT_HAS_CONFIG_REF, &nbd->config->runtime_flags)) nbd_config_put(nbd); diff --git a/drivers/block/rbd.c b/drivers/block/rbd.c index 3de11f077144..a999b698b131 100644 --- a/drivers/block/rbd.c +++ b/drivers/block/rbd.c @@ -632,9 +632,8 @@ void rbd_warn(struct rbd_device *rbd_dev, const char *fmt, ...) static void rbd_dev_remove_parent(struct rbd_device *rbd_dev); static int rbd_dev_refresh(struct rbd_device *rbd_dev); -static int rbd_dev_v2_header_onetime(struct rbd_device *rbd_dev); -static int rbd_dev_header_info(struct rbd_device *rbd_dev); -static int rbd_dev_v2_parent_info(struct rbd_device *rbd_dev); +static int rbd_dev_v2_header_onetime(struct rbd_device *rbd_dev, + struct rbd_image_header *header); static const char *rbd_dev_v2_snap_name(struct rbd_device *rbd_dev, u64 snap_id); static int _rbd_dev_v2_snap_size(struct rbd_device *rbd_dev, u64 snap_id, @@ -995,15 +994,24 @@ static void rbd_init_layout(struct rbd_device *rbd_dev) RCU_INIT_POINTER(rbd_dev->layout.pool_ns, NULL); } +static void rbd_image_header_cleanup(struct rbd_image_header *header) +{ + kfree(header->object_prefix); + ceph_put_snap_context(header->snapc); + kfree(header->snap_sizes); + kfree(header->snap_names); + + memset(header, 0, sizeof(*header)); +} + /* * Fill an rbd image header with information from the given format 1 * on-disk header. */ -static int rbd_header_from_disk(struct rbd_device *rbd_dev, - struct rbd_image_header_ondisk *ondisk) +static int rbd_header_from_disk(struct rbd_image_header *header, + struct rbd_image_header_ondisk *ondisk, + bool first_time) { - struct rbd_image_header *header = &rbd_dev->header; - bool first_time = header->object_prefix == NULL; struct ceph_snap_context *snapc; char *object_prefix = NULL; char *snap_names = NULL; @@ -1070,11 +1078,6 @@ static int rbd_header_from_disk(struct rbd_device *rbd_dev, if (first_time) { header->object_prefix = object_prefix; header->obj_order = ondisk->options.order; - rbd_init_layout(rbd_dev); - } else { - ceph_put_snap_context(header->snapc); - kfree(header->snap_names); - kfree(header->snap_sizes); } /* The remaining fields always get updated (when we refresh) */ @@ -4859,7 +4862,9 @@ out_req: * return, the rbd_dev->header field will contain up-to-date * information about the image. */ -static int rbd_dev_v1_header_info(struct rbd_device *rbd_dev) +static int rbd_dev_v1_header_info(struct rbd_device *rbd_dev, + struct rbd_image_header *header, + bool first_time) { struct rbd_image_header_ondisk *ondisk = NULL; u32 snap_count = 0; @@ -4907,7 +4912,7 @@ static int rbd_dev_v1_header_info(struct rbd_device *rbd_dev) snap_count = le32_to_cpu(ondisk->snap_count); } while (snap_count != want_count); - ret = rbd_header_from_disk(rbd_dev, ondisk); + ret = rbd_header_from_disk(header, ondisk, first_time); out: kfree(ondisk); @@ -4931,39 +4936,6 @@ static void rbd_dev_update_size(struct rbd_device *rbd_dev) } } -static int rbd_dev_refresh(struct rbd_device *rbd_dev) -{ - u64 mapping_size; - int ret; - - down_write(&rbd_dev->header_rwsem); - mapping_size = rbd_dev->mapping.size; - - ret = rbd_dev_header_info(rbd_dev); - if (ret) - goto out; - - /* - * If there is a parent, see if it has disappeared due to the - * mapped image getting flattened. - */ - if (rbd_dev->parent) { - ret = rbd_dev_v2_parent_info(rbd_dev); - if (ret) - goto out; - } - - rbd_assert(!rbd_is_snap(rbd_dev)); - rbd_dev->mapping.size = rbd_dev->header.image_size; - -out: - up_write(&rbd_dev->header_rwsem); - if (!ret && mapping_size != rbd_dev->mapping.size) - rbd_dev_update_size(rbd_dev); - - return ret; -} - static const struct blk_mq_ops rbd_mq_ops = { .queue_rq = rbd_queue_rq, }; @@ -5503,17 +5475,12 @@ static int _rbd_dev_v2_snap_size(struct rbd_device *rbd_dev, u64 snap_id, return 0; } -static int rbd_dev_v2_image_size(struct rbd_device *rbd_dev) -{ - return _rbd_dev_v2_snap_size(rbd_dev, CEPH_NOSNAP, - &rbd_dev->header.obj_order, - &rbd_dev->header.image_size); -} - -static int rbd_dev_v2_object_prefix(struct rbd_device *rbd_dev) +static int rbd_dev_v2_object_prefix(struct rbd_device *rbd_dev, + char **pobject_prefix) { size_t size; void *reply_buf; + char *object_prefix; int ret; void *p; @@ -5531,16 +5498,16 @@ static int rbd_dev_v2_object_prefix(struct rbd_device *rbd_dev) goto out; p = reply_buf; - rbd_dev->header.object_prefix = ceph_extract_encoded_string(&p, - p + ret, NULL, GFP_NOIO); + object_prefix = ceph_extract_encoded_string(&p, p + ret, NULL, + GFP_NOIO); + if (IS_ERR(object_prefix)) { + ret = PTR_ERR(object_prefix); + goto out; + } ret = 0; - if (IS_ERR(rbd_dev->header.object_prefix)) { - ret = PTR_ERR(rbd_dev->header.object_prefix); - rbd_dev->header.object_prefix = NULL; - } else { - dout(" object_prefix = %s\n", rbd_dev->header.object_prefix); - } + *pobject_prefix = object_prefix; + dout(" object_prefix = %s\n", object_prefix); out: kfree(reply_buf); @@ -5591,13 +5558,6 @@ static int _rbd_dev_v2_snap_features(struct rbd_device *rbd_dev, u64 snap_id, return 0; } -static int rbd_dev_v2_features(struct rbd_device *rbd_dev) -{ - return _rbd_dev_v2_snap_features(rbd_dev, CEPH_NOSNAP, - rbd_is_ro(rbd_dev), - &rbd_dev->header.features); -} - /* * These are generic image flags, but since they are used only for * object map, store them in rbd_dev->object_map_flags. @@ -5634,6 +5594,14 @@ struct parent_image_info { u64 overlap; }; +static void rbd_parent_info_cleanup(struct parent_image_info *pii) +{ + kfree(pii->pool_ns); + kfree(pii->image_id); + + memset(pii, 0, sizeof(*pii)); +} + /* * The caller is responsible for @pii. */ @@ -5703,6 +5671,9 @@ static int __get_parent_info(struct rbd_device *rbd_dev, if (pii->has_overlap) ceph_decode_64_safe(&p, end, pii->overlap, e_inval); + dout("%s pool_id %llu pool_ns %s image_id %s snap_id %llu has_overlap %d overlap %llu\n", + __func__, pii->pool_id, pii->pool_ns, pii->image_id, pii->snap_id, + pii->has_overlap, pii->overlap); return 0; e_inval: @@ -5741,14 +5712,17 @@ static int __get_parent_info_legacy(struct rbd_device *rbd_dev, pii->has_overlap = true; ceph_decode_64_safe(&p, end, pii->overlap, e_inval); + dout("%s pool_id %llu pool_ns %s image_id %s snap_id %llu has_overlap %d overlap %llu\n", + __func__, pii->pool_id, pii->pool_ns, pii->image_id, pii->snap_id, + pii->has_overlap, pii->overlap); return 0; e_inval: return -EINVAL; } -static int get_parent_info(struct rbd_device *rbd_dev, - struct parent_image_info *pii) +static int rbd_dev_v2_parent_info(struct rbd_device *rbd_dev, + struct parent_image_info *pii) { struct page *req_page, *reply_page; void *p; @@ -5776,7 +5750,7 @@ static int get_parent_info(struct rbd_device *rbd_dev, return ret; } -static int rbd_dev_v2_parent_info(struct rbd_device *rbd_dev) +static int rbd_dev_setup_parent(struct rbd_device *rbd_dev) { struct rbd_spec *parent_spec; struct parent_image_info pii = { 0 }; @@ -5786,37 +5760,12 @@ static int rbd_dev_v2_parent_info(struct rbd_device *rbd_dev) if (!parent_spec) return -ENOMEM; - ret = get_parent_info(rbd_dev, &pii); + ret = rbd_dev_v2_parent_info(rbd_dev, &pii); if (ret) goto out_err; - dout("%s pool_id %llu pool_ns %s image_id %s snap_id %llu has_overlap %d overlap %llu\n", - __func__, pii.pool_id, pii.pool_ns, pii.image_id, pii.snap_id, - pii.has_overlap, pii.overlap); - - if (pii.pool_id == CEPH_NOPOOL || !pii.has_overlap) { - /* - * Either the parent never existed, or we have - * record of it but the image got flattened so it no - * longer has a parent. When the parent of a - * layered image disappears we immediately set the - * overlap to 0. The effect of this is that all new - * requests will be treated as if the image had no - * parent. - * - * If !pii.has_overlap, the parent image spec is not - * applicable. It's there to avoid duplication in each - * snapshot record. - */ - if (rbd_dev->parent_overlap) { - rbd_dev->parent_overlap = 0; - rbd_dev_parent_put(rbd_dev); - pr_info("%s: clone image has been flattened\n", - rbd_dev->disk->disk_name); - } - + if (pii.pool_id == CEPH_NOPOOL || !pii.has_overlap) goto out; /* No parent? No problem. */ - } /* The ceph file layout needs to fit pool id in 32 bits */ @@ -5828,58 +5777,46 @@ static int rbd_dev_v2_parent_info(struct rbd_device *rbd_dev) } /* - * The parent won't change (except when the clone is - * flattened, already handled that). So we only need to - * record the parent spec we have not already done so. + * The parent won't change except when the clone is flattened, + * so we only need to record the parent image spec once. */ - if (!rbd_dev->parent_spec) { - parent_spec->pool_id = pii.pool_id; - if (pii.pool_ns && *pii.pool_ns) { - parent_spec->pool_ns = pii.pool_ns; - pii.pool_ns = NULL; - } - parent_spec->image_id = pii.image_id; - pii.image_id = NULL; - parent_spec->snap_id = pii.snap_id; - - rbd_dev->parent_spec = parent_spec; - parent_spec = NULL; /* rbd_dev now owns this */ + parent_spec->pool_id = pii.pool_id; + if (pii.pool_ns && *pii.pool_ns) { + parent_spec->pool_ns = pii.pool_ns; + pii.pool_ns = NULL; } + parent_spec->image_id = pii.image_id; + pii.image_id = NULL; + parent_spec->snap_id = pii.snap_id; + + rbd_assert(!rbd_dev->parent_spec); + rbd_dev->parent_spec = parent_spec; + parent_spec = NULL; /* rbd_dev now owns this */ /* - * We always update the parent overlap. If it's zero we issue - * a warning, as we will proceed as if there was no parent. + * Record the parent overlap. If it's zero, issue a warning as + * we will proceed as if there is no parent. */ - if (!pii.overlap) { - if (parent_spec) { - /* refresh, careful to warn just once */ - if (rbd_dev->parent_overlap) - rbd_warn(rbd_dev, - "clone now standalone (overlap became 0)"); - } else { - /* initial probe */ - rbd_warn(rbd_dev, "clone is standalone (overlap 0)"); - } - } + if (!pii.overlap) + rbd_warn(rbd_dev, "clone is standalone (overlap 0)"); rbd_dev->parent_overlap = pii.overlap; out: ret = 0; out_err: - kfree(pii.pool_ns); - kfree(pii.image_id); + rbd_parent_info_cleanup(&pii); rbd_spec_put(parent_spec); return ret; } -static int rbd_dev_v2_striping_info(struct rbd_device *rbd_dev) +static int rbd_dev_v2_striping_info(struct rbd_device *rbd_dev, + u64 *stripe_unit, u64 *stripe_count) { struct { __le64 stripe_unit; __le64 stripe_count; } __attribute__ ((packed)) striping_info_buf = { 0 }; size_t size = sizeof (striping_info_buf); - void *p; int ret; ret = rbd_obj_method_sync(rbd_dev, &rbd_dev->header_oid, @@ -5891,27 +5828,33 @@ static int rbd_dev_v2_striping_info(struct rbd_device *rbd_dev) if (ret < size) return -ERANGE; - p = &striping_info_buf; - rbd_dev->header.stripe_unit = ceph_decode_64(&p); - rbd_dev->header.stripe_count = ceph_decode_64(&p); + *stripe_unit = le64_to_cpu(striping_info_buf.stripe_unit); + *stripe_count = le64_to_cpu(striping_info_buf.stripe_count); + dout(" stripe_unit = %llu stripe_count = %llu\n", *stripe_unit, + *stripe_count); + return 0; } -static int rbd_dev_v2_data_pool(struct rbd_device *rbd_dev) +static int rbd_dev_v2_data_pool(struct rbd_device *rbd_dev, s64 *data_pool_id) { - __le64 data_pool_id; + __le64 data_pool_buf; int ret; ret = rbd_obj_method_sync(rbd_dev, &rbd_dev->header_oid, &rbd_dev->header_oloc, "get_data_pool", - NULL, 0, &data_pool_id, sizeof(data_pool_id)); + NULL, 0, &data_pool_buf, + sizeof(data_pool_buf)); + dout("%s: rbd_obj_method_sync returned %d\n", __func__, ret); if (ret < 0) return ret; - if (ret < sizeof(data_pool_id)) + if (ret < sizeof(data_pool_buf)) return -EBADMSG; - rbd_dev->header.data_pool_id = le64_to_cpu(data_pool_id); - WARN_ON(rbd_dev->header.data_pool_id == CEPH_NOPOOL); + *data_pool_id = le64_to_cpu(data_pool_buf); + dout(" data_pool_id = %lld\n", *data_pool_id); + WARN_ON(*data_pool_id == CEPH_NOPOOL); + return 0; } @@ -6103,7 +6046,8 @@ out_err: return ret; } -static int rbd_dev_v2_snap_context(struct rbd_device *rbd_dev) +static int rbd_dev_v2_snap_context(struct rbd_device *rbd_dev, + struct ceph_snap_context **psnapc) { size_t size; int ret; @@ -6164,9 +6108,7 @@ static int rbd_dev_v2_snap_context(struct rbd_device *rbd_dev) for (i = 0; i < snap_count; i++) snapc->snaps[i] = ceph_decode_64(&p); - ceph_put_snap_context(rbd_dev->header.snapc); - rbd_dev->header.snapc = snapc; - + *psnapc = snapc; dout(" snap context seq = %llu, snap_count = %u\n", (unsigned long long)seq, (unsigned int)snap_count); out: @@ -6215,38 +6157,42 @@ out: return snap_name; } -static int rbd_dev_v2_header_info(struct rbd_device *rbd_dev) +static int rbd_dev_v2_header_info(struct rbd_device *rbd_dev, + struct rbd_image_header *header, + bool first_time) { - bool first_time = rbd_dev->header.object_prefix == NULL; int ret; - ret = rbd_dev_v2_image_size(rbd_dev); + ret = _rbd_dev_v2_snap_size(rbd_dev, CEPH_NOSNAP, + first_time ? &header->obj_order : NULL, + &header->image_size); if (ret) return ret; if (first_time) { - ret = rbd_dev_v2_header_onetime(rbd_dev); + ret = rbd_dev_v2_header_onetime(rbd_dev, header); if (ret) return ret; } - ret = rbd_dev_v2_snap_context(rbd_dev); - if (ret && first_time) { - kfree(rbd_dev->header.object_prefix); - rbd_dev->header.object_prefix = NULL; - } + ret = rbd_dev_v2_snap_context(rbd_dev, &header->snapc); + if (ret) + return ret; - return ret; + return 0; } -static int rbd_dev_header_info(struct rbd_device *rbd_dev) +static int rbd_dev_header_info(struct rbd_device *rbd_dev, + struct rbd_image_header *header, + bool first_time) { rbd_assert(rbd_image_format_valid(rbd_dev->image_format)); + rbd_assert(!header->object_prefix && !header->snapc); if (rbd_dev->image_format == 1) - return rbd_dev_v1_header_info(rbd_dev); + return rbd_dev_v1_header_info(rbd_dev, header, first_time); - return rbd_dev_v2_header_info(rbd_dev); + return rbd_dev_v2_header_info(rbd_dev, header, first_time); } /* @@ -6734,60 +6680,49 @@ out: */ static void rbd_dev_unprobe(struct rbd_device *rbd_dev) { - struct rbd_image_header *header; - rbd_dev_parent_put(rbd_dev); rbd_object_map_free(rbd_dev); rbd_dev_mapping_clear(rbd_dev); /* Free dynamic fields from the header, then zero it out */ - header = &rbd_dev->header; - ceph_put_snap_context(header->snapc); - kfree(header->snap_sizes); - kfree(header->snap_names); - kfree(header->object_prefix); - memset(header, 0, sizeof (*header)); + rbd_image_header_cleanup(&rbd_dev->header); } -static int rbd_dev_v2_header_onetime(struct rbd_device *rbd_dev) +static int rbd_dev_v2_header_onetime(struct rbd_device *rbd_dev, + struct rbd_image_header *header) { int ret; - ret = rbd_dev_v2_object_prefix(rbd_dev); + ret = rbd_dev_v2_object_prefix(rbd_dev, &header->object_prefix); if (ret) - goto out_err; + return ret; /* * Get the and check features for the image. Currently the * features are assumed to never change. */ - ret = rbd_dev_v2_features(rbd_dev); + ret = _rbd_dev_v2_snap_features(rbd_dev, CEPH_NOSNAP, + rbd_is_ro(rbd_dev), &header->features); if (ret) - goto out_err; + return ret; /* If the image supports fancy striping, get its parameters */ - if (rbd_dev->header.features & RBD_FEATURE_STRIPINGV2) { - ret = rbd_dev_v2_striping_info(rbd_dev); - if (ret < 0) - goto out_err; + if (header->features & RBD_FEATURE_STRIPINGV2) { + ret = rbd_dev_v2_striping_info(rbd_dev, &header->stripe_unit, + &header->stripe_count); + if (ret) + return ret; } - if (rbd_dev->header.features & RBD_FEATURE_DATA_POOL) { - ret = rbd_dev_v2_data_pool(rbd_dev); + if (header->features & RBD_FEATURE_DATA_POOL) { + ret = rbd_dev_v2_data_pool(rbd_dev, &header->data_pool_id); if (ret) - goto out_err; + return ret; } - rbd_init_layout(rbd_dev); return 0; - -out_err: - rbd_dev->header.features = 0; - kfree(rbd_dev->header.object_prefix); - rbd_dev->header.object_prefix = NULL; - return ret; } /* @@ -6982,13 +6917,15 @@ static int rbd_dev_image_probe(struct rbd_device *rbd_dev, int depth) if (!depth) down_write(&rbd_dev->header_rwsem); - ret = rbd_dev_header_info(rbd_dev); + ret = rbd_dev_header_info(rbd_dev, &rbd_dev->header, true); if (ret) { if (ret == -ENOENT && !need_watch) rbd_print_dne(rbd_dev, false); goto err_out_probe; } + rbd_init_layout(rbd_dev); + /* * If this image is the one being mapped, we have pool name and * id, image name and id, and snap name - need to fill snap id. @@ -7017,7 +6954,7 @@ static int rbd_dev_image_probe(struct rbd_device *rbd_dev, int depth) } if (rbd_dev->header.features & RBD_FEATURE_LAYERING) { - ret = rbd_dev_v2_parent_info(rbd_dev); + ret = rbd_dev_setup_parent(rbd_dev); if (ret) goto err_out_probe; } @@ -7043,6 +6980,107 @@ err_out_format: return ret; } +static void rbd_dev_update_header(struct rbd_device *rbd_dev, + struct rbd_image_header *header) +{ + rbd_assert(rbd_image_format_valid(rbd_dev->image_format)); + rbd_assert(rbd_dev->header.object_prefix); /* !first_time */ + + if (rbd_dev->header.image_size != header->image_size) { + rbd_dev->header.image_size = header->image_size; + + if (!rbd_is_snap(rbd_dev)) { + rbd_dev->mapping.size = header->image_size; + rbd_dev_update_size(rbd_dev); + } + } + + ceph_put_snap_context(rbd_dev->header.snapc); + rbd_dev->header.snapc = header->snapc; + header->snapc = NULL; + + if (rbd_dev->image_format == 1) { + kfree(rbd_dev->header.snap_names); + rbd_dev->header.snap_names = header->snap_names; + header->snap_names = NULL; + + kfree(rbd_dev->header.snap_sizes); + rbd_dev->header.snap_sizes = header->snap_sizes; + header->snap_sizes = NULL; + } +} + +static void rbd_dev_update_parent(struct rbd_device *rbd_dev, + struct parent_image_info *pii) +{ + if (pii->pool_id == CEPH_NOPOOL || !pii->has_overlap) { + /* + * Either the parent never existed, or we have + * record of it but the image got flattened so it no + * longer has a parent. When the parent of a + * layered image disappears we immediately set the + * overlap to 0. The effect of this is that all new + * requests will be treated as if the image had no + * parent. + * + * If !pii.has_overlap, the parent image spec is not + * applicable. It's there to avoid duplication in each + * snapshot record. + */ + if (rbd_dev->parent_overlap) { + rbd_dev->parent_overlap = 0; + rbd_dev_parent_put(rbd_dev); + pr_info("%s: clone has been flattened\n", + rbd_dev->disk->disk_name); + } + } else { + rbd_assert(rbd_dev->parent_spec); + + /* + * Update the parent overlap. If it became zero, issue + * a warning as we will proceed as if there is no parent. + */ + if (!pii->overlap && rbd_dev->parent_overlap) + rbd_warn(rbd_dev, + "clone has become standalone (overlap 0)"); + rbd_dev->parent_overlap = pii->overlap; + } +} + +static int rbd_dev_refresh(struct rbd_device *rbd_dev) +{ + struct rbd_image_header header = { 0 }; + struct parent_image_info pii = { 0 }; + int ret; + + dout("%s rbd_dev %p\n", __func__, rbd_dev); + + ret = rbd_dev_header_info(rbd_dev, &header, false); + if (ret) + goto out; + + /* + * If there is a parent, see if it has disappeared due to the + * mapped image getting flattened. + */ + if (rbd_dev->parent) { + ret = rbd_dev_v2_parent_info(rbd_dev, &pii); + if (ret) + goto out; + } + + down_write(&rbd_dev->header_rwsem); + rbd_dev_update_header(rbd_dev, &header); + if (rbd_dev->parent) + rbd_dev_update_parent(rbd_dev, &pii); + up_write(&rbd_dev->header_rwsem); + +out: + rbd_parent_info_cleanup(&pii); + rbd_image_header_cleanup(&header); + return ret; +} + static ssize_t do_rbd_add(const char *buf, size_t count) { struct rbd_device *rbd_dev = NULL; diff --git a/drivers/bluetooth/btrtl.c b/drivers/bluetooth/btrtl.c index 84c2c2e1122f..277d039ecbb4 100644 --- a/drivers/bluetooth/btrtl.c +++ b/drivers/bluetooth/btrtl.c @@ -962,13 +962,10 @@ static void btrtl_dmp_hdr(struct hci_dev *hdev, struct sk_buff *skb) skb_put_data(skb, buf, strlen(buf)); } -static int btrtl_register_devcoredump_support(struct hci_dev *hdev) +static void btrtl_register_devcoredump_support(struct hci_dev *hdev) { - int err; + hci_devcd_register(hdev, btrtl_coredump, btrtl_dmp_hdr, NULL); - err = hci_devcd_register(hdev, btrtl_coredump, btrtl_dmp_hdr, NULL); - - return err; } void btrtl_set_driver_name(struct hci_dev *hdev, const char *driver_name) @@ -1255,8 +1252,7 @@ int btrtl_download_firmware(struct hci_dev *hdev, } done: - if (!err) - err = btrtl_register_devcoredump_support(hdev); + btrtl_register_devcoredump_support(hdev); return err; } diff --git a/drivers/bluetooth/btusb.c b/drivers/bluetooth/btusb.c index 82597ab4f747..499f4809fcdf 100644 --- a/drivers/bluetooth/btusb.c +++ b/drivers/bluetooth/btusb.c @@ -4419,6 +4419,7 @@ static int btusb_probe(struct usb_interface *intf, if (id->driver_info & BTUSB_QCA_ROME) { data->setup_on_usb = btusb_setup_qca; + hdev->shutdown = btusb_shutdown_qca; hdev->set_bdaddr = btusb_set_bdaddr_ath3012; hdev->cmd_timeout = btusb_qca_cmd_timeout; set_bit(HCI_QUIRK_SIMULTANEOUS_DISCOVERY, &hdev->quirks); diff --git a/drivers/bluetooth/hci_vhci.c b/drivers/bluetooth/hci_vhci.c index 40e2b9fa11a2..f3892e9ce800 100644 --- a/drivers/bluetooth/hci_vhci.c +++ b/drivers/bluetooth/hci_vhci.c @@ -74,7 +74,10 @@ static int vhci_send_frame(struct hci_dev *hdev, struct sk_buff *skb) struct vhci_data *data = hci_get_drvdata(hdev); memcpy(skb_push(skb, 1), &hci_skb_pkt_type(skb), 1); + + mutex_lock(&data->open_mutex); skb_queue_tail(&data->readq, skb); + mutex_unlock(&data->open_mutex); wake_up_interruptible(&data->read_wait); return 0; diff --git a/drivers/bus/ti-sysc.c b/drivers/bus/ti-sysc.c index eb4e7bee1e20..d57bc066dce6 100644 --- a/drivers/bus/ti-sysc.c +++ b/drivers/bus/ti-sysc.c @@ -38,6 +38,7 @@ enum sysc_soc { SOC_2420, SOC_2430, SOC_3430, + SOC_AM35, SOC_3630, SOC_4430, SOC_4460, @@ -1097,6 +1098,11 @@ static int sysc_enable_module(struct device *dev) if (ddata->cfg.quirks & (SYSC_QUIRK_SWSUP_SIDLE | SYSC_QUIRK_SWSUP_SIDLE_ACT)) { best_mode = SYSC_IDLE_NO; + + /* Clear WAKEUP */ + if (regbits->enwkup_shift >= 0 && + ddata->cfg.sysc_val & BIT(regbits->enwkup_shift)) + reg &= ~BIT(regbits->enwkup_shift); } else { best_mode = fls(ddata->cfg.sidlemodes) - 1; if (best_mode > SYSC_IDLE_MASK) { @@ -1224,6 +1230,13 @@ set_sidle: } } + if (ddata->cfg.quirks & SYSC_QUIRK_SWSUP_SIDLE_ACT) { + /* Set WAKEUP */ + if (regbits->enwkup_shift >= 0 && + ddata->cfg.sysc_val & BIT(regbits->enwkup_shift)) + reg |= BIT(regbits->enwkup_shift); + } + reg &= ~(SYSC_IDLE_MASK << regbits->sidle_shift); reg |= best_mode << regbits->sidle_shift; if (regbits->autoidle_shift >= 0 && @@ -1518,16 +1531,16 @@ struct sysc_revision_quirk { static const struct sysc_revision_quirk sysc_revision_quirks[] = { /* These drivers need to be fixed to not use pm_runtime_irq_safe() */ SYSC_QUIRK("uart", 0, 0x50, 0x54, 0x58, 0x00000046, 0xffffffff, - SYSC_QUIRK_SWSUP_SIDLE | SYSC_QUIRK_LEGACY_IDLE), + SYSC_QUIRK_SWSUP_SIDLE_ACT | SYSC_QUIRK_LEGACY_IDLE), SYSC_QUIRK("uart", 0, 0x50, 0x54, 0x58, 0x00000052, 0xffffffff, - SYSC_QUIRK_SWSUP_SIDLE | SYSC_QUIRK_LEGACY_IDLE), + SYSC_QUIRK_SWSUP_SIDLE_ACT | SYSC_QUIRK_LEGACY_IDLE), /* Uarts on omap4 and later */ SYSC_QUIRK("uart", 0, 0x50, 0x54, 0x58, 0x50411e03, 0xffff00ff, - SYSC_QUIRK_SWSUP_SIDLE | SYSC_QUIRK_LEGACY_IDLE), + SYSC_QUIRK_SWSUP_SIDLE_ACT | SYSC_QUIRK_LEGACY_IDLE), SYSC_QUIRK("uart", 0, 0x50, 0x54, 0x58, 0x47422e03, 0xffffffff, - SYSC_QUIRK_SWSUP_SIDLE | SYSC_QUIRK_LEGACY_IDLE), + SYSC_QUIRK_SWSUP_SIDLE_ACT | SYSC_QUIRK_LEGACY_IDLE), SYSC_QUIRK("uart", 0, 0x50, 0x54, 0x58, 0x47424e03, 0xffffffff, - SYSC_QUIRK_SWSUP_SIDLE | SYSC_QUIRK_LEGACY_IDLE), + SYSC_QUIRK_SWSUP_SIDLE_ACT | SYSC_QUIRK_LEGACY_IDLE), /* Quirks that need to be set based on the module address */ SYSC_QUIRK("mcpdm", 0x40132000, 0, 0x10, -ENODEV, 0x50000800, 0xffffffff, @@ -1862,7 +1875,7 @@ static void sysc_pre_reset_quirk_dss(struct sysc *ddata) dev_warn(ddata->dev, "%s: timed out %08x !+ %08x\n", __func__, val, irq_mask); - if (sysc_soc->soc == SOC_3430) { + if (sysc_soc->soc == SOC_3430 || sysc_soc->soc == SOC_AM35) { /* Clear DSS_SDI_CONTROL */ sysc_write(ddata, 0x44, 0); @@ -2150,8 +2163,7 @@ static int sysc_reset(struct sysc *ddata) } if (ddata->cfg.srst_udelay) - usleep_range(ddata->cfg.srst_udelay, - ddata->cfg.srst_udelay * 2); + fsleep(ddata->cfg.srst_udelay); if (ddata->post_reset_quirk) ddata->post_reset_quirk(ddata); @@ -3025,6 +3037,7 @@ static void ti_sysc_idle(struct work_struct *work) static const struct soc_device_attribute sysc_soc_match[] = { SOC_FLAG("OMAP242*", SOC_2420), SOC_FLAG("OMAP243*", SOC_2430), + SOC_FLAG("AM35*", SOC_AM35), SOC_FLAG("OMAP3[45]*", SOC_3430), SOC_FLAG("OMAP3[67]*", SOC_3630), SOC_FLAG("OMAP443*", SOC_4430), @@ -3229,7 +3242,7 @@ static int sysc_check_active_timer(struct sysc *ddata) * can be dropped if we stop supporting old beagleboard revisions * A to B4 at some point. */ - if (sysc_soc->soc == SOC_3430) + if (sysc_soc->soc == SOC_3430 || sysc_soc->soc == SOC_AM35) error = -ENXIO; else error = -EBUSY; diff --git a/drivers/char/agp/parisc-agp.c b/drivers/char/agp/parisc-agp.c index 514f9f287a78..c6f181702b9a 100644 --- a/drivers/char/agp/parisc-agp.c +++ b/drivers/char/agp/parisc-agp.c @@ -394,8 +394,6 @@ find_quicksilver(struct device *dev, void *data) static int __init parisc_agp_init(void) { - extern struct sba_device *sba_list; - int err = -1; struct parisc_device *sba = NULL, *lba = NULL; struct lba_device *lbadev = NULL; diff --git a/drivers/char/tpm/tpm-chip.c b/drivers/char/tpm/tpm-chip.c index 23f6f2eda84c..42b1062e33cd 100644 --- a/drivers/char/tpm/tpm-chip.c +++ b/drivers/char/tpm/tpm-chip.c @@ -33,7 +33,7 @@ const struct class tpm_class = { .shutdown_pre = tpm_class_shutdown, }; const struct class tpmrm_class = { - .name = "tmprm", + .name = "tpmrm", }; dev_t tpm_devt; diff --git a/drivers/clk/clk-si521xx.c b/drivers/clk/clk-si521xx.c index 4eaf1b53f06b..ef4ba467e747 100644 --- a/drivers/clk/clk-si521xx.c +++ b/drivers/clk/clk-si521xx.c @@ -96,7 +96,7 @@ static int si521xx_regmap_i2c_write(void *context, unsigned int reg, unsigned int val) { struct i2c_client *i2c = context; - const u8 data[3] = { reg, 1, val }; + const u8 data[2] = { reg, val }; const int count = ARRAY_SIZE(data); int ret; @@ -146,7 +146,7 @@ static int si521xx_regmap_i2c_read(void *context, unsigned int reg, static const struct regmap_config si521xx_regmap_config = { .reg_bits = 8, .val_bits = 8, - .cache_type = REGCACHE_NONE, + .cache_type = REGCACHE_FLAT, .max_register = SI521XX_REG_DA, .rd_table = &si521xx_readable_table, .wr_table = &si521xx_writeable_table, @@ -281,9 +281,10 @@ static int si521xx_probe(struct i2c_client *client) { const u16 chip_info = (u16)(uintptr_t)device_get_match_data(&client->dev); const struct clk_parent_data clk_parent_data = { .index = 0 }; - struct si521xx *si; + const u8 data[3] = { SI521XX_REG_BC, 1, 1 }; unsigned char name[6] = "DIFF0"; struct clk_init_data init = {}; + struct si521xx *si; int i, ret; if (!chip_info) @@ -308,7 +309,7 @@ static int si521xx_probe(struct i2c_client *client) "Failed to allocate register map\n"); /* Always read back 1 Byte via I2C */ - ret = regmap_write(si->regmap, SI521XX_REG_BC, 1); + ret = i2c_master_send(client, data, ARRAY_SIZE(data)); if (ret < 0) return ret; diff --git a/drivers/clk/clk-versaclock3.c b/drivers/clk/clk-versaclock3.c index 7ab2447bd203..3d7de355f8f6 100644 --- a/drivers/clk/clk-versaclock3.c +++ b/drivers/clk/clk-versaclock3.c @@ -118,21 +118,21 @@ enum vc3_div { VC3_DIV5, }; -enum vc3_clk_mux { - VC3_DIFF2_MUX, - VC3_DIFF1_MUX, - VC3_SE3_MUX, - VC3_SE2_MUX, - VC3_SE1_MUX, -}; - enum vc3_clk { - VC3_DIFF2, - VC3_DIFF1, - VC3_SE3, - VC3_SE2, - VC3_SE1, VC3_REF, + VC3_SE1, + VC3_SE2, + VC3_SE3, + VC3_DIFF1, + VC3_DIFF2, +}; + +enum vc3_clk_mux { + VC3_SE1_MUX = VC3_SE1 - 1, + VC3_SE2_MUX = VC3_SE2 - 1, + VC3_SE3_MUX = VC3_SE3 - 1, + VC3_DIFF1_MUX = VC3_DIFF1 - 1, + VC3_DIFF2_MUX = VC3_DIFF2 - 1, }; struct vc3_clk_data { @@ -401,11 +401,10 @@ static long vc3_pll_round_rate(struct clk_hw *hw, unsigned long rate, /* Determine best fractional part, which is 16 bit wide */ div_frc = rate % *parent_rate; div_frc *= BIT(16) - 1; - do_div(div_frc, *parent_rate); - vc3->div_frc = (u32)div_frc; + vc3->div_frc = min_t(u64, div64_ul(div_frc, *parent_rate), U16_MAX); rate = (*parent_rate * - (vc3->div_int * VC3_2_POW_16 + div_frc) / VC3_2_POW_16); + (vc3->div_int * VC3_2_POW_16 + vc3->div_frc) / VC3_2_POW_16); } else { rate = *parent_rate * vc3->div_int; } @@ -897,33 +896,33 @@ static struct vc3_hw_data clk_div[] = { }; static struct vc3_hw_data clk_mux[] = { - [VC3_DIFF2_MUX] = { + [VC3_SE1_MUX] = { .data = &(struct vc3_clk_data) { - .offs = VC3_DIFF2_CTRL_REG, - .bitmsk = VC3_DIFF2_CTRL_REG_DIFF2_CLK_SEL + .offs = VC3_SE1_DIV4_CTRL, + .bitmsk = VC3_SE1_DIV4_CTRL_SE1_CLK_SEL }, .hw.init = &(struct clk_init_data){ - .name = "diff2_mux", + .name = "se1_mux", .ops = &vc3_clk_mux_ops, .parent_hws = (const struct clk_hw *[]) { - &clk_div[VC3_DIV1].hw, - &clk_div[VC3_DIV3].hw + &clk_div[VC3_DIV5].hw, + &clk_div[VC3_DIV4].hw }, .num_parents = 2, .flags = CLK_SET_RATE_PARENT } }, - [VC3_DIFF1_MUX] = { + [VC3_SE2_MUX] = { .data = &(struct vc3_clk_data) { - .offs = VC3_DIFF1_CTRL_REG, - .bitmsk = VC3_DIFF1_CTRL_REG_DIFF1_CLK_SEL + .offs = VC3_SE2_CTRL_REG0, + .bitmsk = VC3_SE2_CTRL_REG0_SE2_CLK_SEL }, .hw.init = &(struct clk_init_data){ - .name = "diff1_mux", + .name = "se2_mux", .ops = &vc3_clk_mux_ops, .parent_hws = (const struct clk_hw *[]) { - &clk_div[VC3_DIV1].hw, - &clk_div[VC3_DIV3].hw + &clk_div[VC3_DIV5].hw, + &clk_div[VC3_DIV4].hw }, .num_parents = 2, .flags = CLK_SET_RATE_PARENT @@ -945,33 +944,33 @@ static struct vc3_hw_data clk_mux[] = { .flags = CLK_SET_RATE_PARENT } }, - [VC3_SE2_MUX] = { + [VC3_DIFF1_MUX] = { .data = &(struct vc3_clk_data) { - .offs = VC3_SE2_CTRL_REG0, - .bitmsk = VC3_SE2_CTRL_REG0_SE2_CLK_SEL + .offs = VC3_DIFF1_CTRL_REG, + .bitmsk = VC3_DIFF1_CTRL_REG_DIFF1_CLK_SEL }, .hw.init = &(struct clk_init_data){ - .name = "se2_mux", + .name = "diff1_mux", .ops = &vc3_clk_mux_ops, .parent_hws = (const struct clk_hw *[]) { - &clk_div[VC3_DIV5].hw, - &clk_div[VC3_DIV4].hw + &clk_div[VC3_DIV1].hw, + &clk_div[VC3_DIV3].hw }, .num_parents = 2, .flags = CLK_SET_RATE_PARENT } }, - [VC3_SE1_MUX] = { + [VC3_DIFF2_MUX] = { .data = &(struct vc3_clk_data) { - .offs = VC3_SE1_DIV4_CTRL, - .bitmsk = VC3_SE1_DIV4_CTRL_SE1_CLK_SEL + .offs = VC3_DIFF2_CTRL_REG, + .bitmsk = VC3_DIFF2_CTRL_REG_DIFF2_CLK_SEL }, .hw.init = &(struct clk_init_data){ - .name = "se1_mux", + .name = "diff2_mux", .ops = &vc3_clk_mux_ops, .parent_hws = (const struct clk_hw *[]) { - &clk_div[VC3_DIV5].hw, - &clk_div[VC3_DIV4].hw + &clk_div[VC3_DIV1].hw, + &clk_div[VC3_DIV3].hw }, .num_parents = 2, .flags = CLK_SET_RATE_PARENT @@ -1110,7 +1109,7 @@ static int vc3_probe(struct i2c_client *client) name, 0, CLK_SET_RATE_PARENT, 1, 1); else clk_out[i] = devm_clk_hw_register_fixed_factor_parent_hw(dev, - name, &clk_mux[i].hw, CLK_SET_RATE_PARENT, 1, 1); + name, &clk_mux[i - 1].hw, CLK_SET_RATE_PARENT, 1, 1); if (IS_ERR(clk_out[i])) return PTR_ERR(clk_out[i]); diff --git a/drivers/clk/sprd/ums512-clk.c b/drivers/clk/sprd/ums512-clk.c index 8f4441dd572b..9384ecc6c741 100644 --- a/drivers/clk/sprd/ums512-clk.c +++ b/drivers/clk/sprd/ums512-clk.c @@ -800,7 +800,7 @@ static SPRD_MUX_CLK_DATA(uart1_clk, "uart1-clk", uart_parents, 0x250, 0, 3, UMS512_MUX_FLAG); static const struct clk_parent_data thm_parents[] = { - { .fw_name = "ext-32m" }, + { .fw_name = "ext-32k" }, { .hw = &clk_250k.hw }, }; static SPRD_MUX_CLK_DATA(thm0_clk, "thm0-clk", thm_parents, diff --git a/drivers/clk/tegra/clk-bpmp.c b/drivers/clk/tegra/clk-bpmp.c index a9f3fb448de6..7bfba0afd778 100644 --- a/drivers/clk/tegra/clk-bpmp.c +++ b/drivers/clk/tegra/clk-bpmp.c @@ -159,7 +159,7 @@ static unsigned long tegra_bpmp_clk_recalc_rate(struct clk_hw *hw, err = tegra_bpmp_clk_transfer(clk->bpmp, &msg); if (err < 0) - return err; + return 0; return response.rate; } diff --git a/drivers/comedi/Kconfig b/drivers/comedi/Kconfig index 7a8d402f05be..9af280735cba 100644 --- a/drivers/comedi/Kconfig +++ b/drivers/comedi/Kconfig @@ -67,7 +67,6 @@ config COMEDI_TEST config COMEDI_PARPORT tristate "Parallel port support" - depends on HAS_IOPORT help Enable support for the standard parallel port. A cheap and easy way to get a few more digital I/O lines. Steal @@ -80,7 +79,6 @@ config COMEDI_PARPORT config COMEDI_SSV_DNP tristate "SSV Embedded Systems DIL/Net-PC support" depends on X86_32 || COMPILE_TEST - depends on HAS_IOPORT help Enable support for SSV Embedded Systems DIL/Net-PC @@ -91,7 +89,6 @@ endif # COMEDI_MISC_DRIVERS menuconfig COMEDI_ISA_DRIVERS bool "Comedi ISA and PC/104 drivers" - depends on ISA help Enable comedi ISA and PC/104 drivers to be built @@ -103,8 +100,7 @@ if COMEDI_ISA_DRIVERS config COMEDI_PCL711 tristate "Advantech PCL-711/711b and ADlink ACL-8112 ISA card support" - depends on HAS_IOPORT - depends on COMEDI_8254 + select COMEDI_8254 help Enable support for Advantech PCL-711 and 711b, ADlink ACL-8112 @@ -165,9 +161,8 @@ config COMEDI_PCL730 config COMEDI_PCL812 tristate "Advantech PCL-812/813 and ADlink ACL-8112/8113/8113/8216" - depends on HAS_IOPORT select COMEDI_ISADMA if ISA_DMA_API - depends on COMEDI_8254 + select COMEDI_8254 help Enable support for Advantech PCL-812/PG, PCL-813/B, ADLink ACL-8112DG/HG/PG, ACL-8113, ACL-8216, ICP DAS A-821PGH/PGL/PGL-NDA, @@ -178,9 +173,8 @@ config COMEDI_PCL812 config COMEDI_PCL816 tristate "Advantech PCL-814 and PCL-816 ISA card support" - depends on HAS_IOPORT select COMEDI_ISADMA if ISA_DMA_API - depends on COMEDI_8254 + select COMEDI_8254 help Enable support for Advantech PCL-814 and PCL-816 ISA cards @@ -189,9 +183,8 @@ config COMEDI_PCL816 config COMEDI_PCL818 tristate "Advantech PCL-718 and PCL-818 ISA card support" - depends on HAS_IOPORT select COMEDI_ISADMA if ISA_DMA_API - depends on COMEDI_8254 + select COMEDI_8254 help Enable support for Advantech PCL-818 ISA cards PCL-818L, PCL-818H, PCL-818HD, PCL-818HG, PCL-818 and PCL-718 @@ -210,7 +203,7 @@ config COMEDI_PCM3724 config COMEDI_AMPLC_DIO200_ISA tristate "Amplicon PC212E/PC214E/PC215E/PC218E/PC272E" - depends on COMEDI_AMPLC_DIO200 + select COMEDI_AMPLC_DIO200 help Enable support for Amplicon PC212E, PC214E, PC215E, PC218E and PC272E ISA DIO boards @@ -262,8 +255,7 @@ config COMEDI_DAC02 config COMEDI_DAS16M1 tristate "MeasurementComputing CIO-DAS16/M1DAS-16 ISA card support" - depends on HAS_IOPORT - depends on COMEDI_8254 + select COMEDI_8254 select COMEDI_8255 help Enable support for Measurement Computing CIO-DAS16/M1 ISA cards. @@ -273,7 +265,7 @@ config COMEDI_DAS16M1 config COMEDI_DAS08_ISA tristate "DAS-08 compatible ISA and PC/104 card support" - depends on COMEDI_DAS08 + select COMEDI_DAS08 help Enable support for Keithley Metrabyte/ComputerBoards DAS08 and compatible ISA and PC/104 cards: @@ -286,9 +278,8 @@ config COMEDI_DAS08_ISA config COMEDI_DAS16 tristate "DAS-16 compatible ISA and PC/104 card support" - depends on HAS_IOPORT select COMEDI_ISADMA if ISA_DMA_API - depends on COMEDI_8254 + select COMEDI_8254 select COMEDI_8255 help Enable support for Keithley Metrabyte/ComputerBoards DAS16 @@ -305,8 +296,7 @@ config COMEDI_DAS16 config COMEDI_DAS800 tristate "DAS800 and compatible ISA card support" - depends on HAS_IOPORT - depends on COMEDI_8254 + select COMEDI_8254 help Enable support for Keithley Metrabyte DAS800 and compatible ISA cards Keithley Metrabyte DAS-800, DAS-801, DAS-802 @@ -318,9 +308,8 @@ config COMEDI_DAS800 config COMEDI_DAS1800 tristate "DAS1800 and compatible ISA card support" - depends on HAS_IOPORT select COMEDI_ISADMA if ISA_DMA_API - depends on COMEDI_8254 + select COMEDI_8254 help Enable support for DAS1800 and compatible ISA cards Keithley Metrabyte DAS-1701ST, DAS-1701ST-DA, DAS-1701/AO, @@ -334,8 +323,7 @@ config COMEDI_DAS1800 config COMEDI_DAS6402 tristate "DAS6402 and compatible ISA card support" - depends on HAS_IOPORT - depends on COMEDI_8254 + select COMEDI_8254 help Enable support for DAS6402 and compatible ISA cards Computerboards, Keithley Metrabyte DAS6402 and compatibles @@ -414,8 +402,7 @@ config COMEDI_FL512 config COMEDI_AIO_AIO12_8 tristate "I/O Products PC/104 AIO12-8 Analog I/O Board support" - depends on HAS_IOPORT - depends on COMEDI_8254 + select COMEDI_8254 select COMEDI_8255 help Enable support for I/O Products PC/104 AIO12-8 Analog I/O Board @@ -469,9 +456,8 @@ config COMEDI_ADQ12B config COMEDI_NI_AT_A2150 tristate "NI AT-A2150 ISA card support" - depends on HAS_IOPORT select COMEDI_ISADMA if ISA_DMA_API - depends on COMEDI_8254 + select COMEDI_8254 help Enable support for National Instruments AT-A2150 cards @@ -480,8 +466,7 @@ config COMEDI_NI_AT_A2150 config COMEDI_NI_AT_AO tristate "NI AT-AO-6/10 EISA card support" - depends on HAS_IOPORT - depends on COMEDI_8254 + select COMEDI_8254 help Enable support for National Instruments AT-AO-6/10 cards @@ -512,7 +497,7 @@ config COMEDI_NI_ATMIO16D config COMEDI_NI_LABPC_ISA tristate "NI Lab-PC and compatibles ISA support" - depends on COMEDI_NI_LABPC + select COMEDI_NI_LABPC help Enable support for National Instruments Lab-PC and compatibles Lab-PC-1200, Lab-PC-1200AI, Lab-PC+. @@ -576,7 +561,7 @@ endif # COMEDI_ISA_DRIVERS menuconfig COMEDI_PCI_DRIVERS tristate "Comedi PCI drivers" - depends on PCI && HAS_IOPORT + depends on PCI help Enable support for comedi PCI drivers. @@ -725,8 +710,7 @@ config COMEDI_ADL_PCI8164 config COMEDI_ADL_PCI9111 tristate "ADLink PCI-9111HR support" - depends on HAS_IOPORT - depends on COMEDI_8254 + select COMEDI_8254 help Enable support for ADlink PCI9111 cards @@ -736,7 +720,7 @@ config COMEDI_ADL_PCI9111 config COMEDI_ADL_PCI9118 tristate "ADLink PCI-9118DG, PCI-9118HG, PCI-9118HR support" depends on HAS_DMA - depends on COMEDI_8254 + select COMEDI_8254 help Enable support for ADlink PCI-9118DG, PCI-9118HG, PCI-9118HR cards @@ -745,8 +729,7 @@ config COMEDI_ADL_PCI9118 config COMEDI_ADV_PCI1710 tristate "Advantech PCI-171x and PCI-1731 support" - depends on HAS_IOPORT - depends on COMEDI_8254 + select COMEDI_8254 help Enable support for Advantech PCI-1710, PCI-1710HG, PCI-1711, PCI-1713 and PCI-1731 @@ -790,8 +773,7 @@ config COMEDI_ADV_PCI1760 config COMEDI_ADV_PCI_DIO tristate "Advantech PCI DIO card support" - depends on HAS_IOPORT - depends on COMEDI_8254 + select COMEDI_8254 select COMEDI_8255 help Enable support for Advantech PCI DIO cards @@ -804,7 +786,7 @@ config COMEDI_ADV_PCI_DIO config COMEDI_AMPLC_DIO200_PCI tristate "Amplicon PCI215/PCI272/PCIe215/PCIe236/PCIe296 DIO support" - depends on COMEDI_AMPLC_DIO200 + select COMEDI_AMPLC_DIO200 help Enable support for Amplicon PCI215, PCI272, PCIe215, PCIe236 and PCIe296 DIO boards. @@ -832,8 +814,7 @@ config COMEDI_AMPLC_PC263_PCI config COMEDI_AMPLC_PCI224 tristate "Amplicon PCI224 and PCI234 support" - depends on HAS_IOPORT - depends on COMEDI_8254 + select COMEDI_8254 help Enable support for Amplicon PCI224 and PCI234 AO boards @@ -842,8 +823,7 @@ config COMEDI_AMPLC_PCI224 config COMEDI_AMPLC_PCI230 tristate "Amplicon PCI230 and PCI260 support" - depends on HAS_IOPORT - depends on COMEDI_8254 + select COMEDI_8254 select COMEDI_8255 help Enable support for Amplicon PCI230 and PCI260 Multifunction I/O @@ -862,7 +842,7 @@ config COMEDI_CONTEC_PCI_DIO config COMEDI_DAS08_PCI tristate "DAS-08 PCI support" - depends on COMEDI_DAS08 + select COMEDI_DAS08 help Enable support for PCI DAS-08 cards. @@ -949,8 +929,7 @@ config COMEDI_CB_PCIDAS64 config COMEDI_CB_PCIDAS tristate "MeasurementComputing PCI-DAS support" - depends on HAS_IOPORT - depends on COMEDI_8254 + select COMEDI_8254 select COMEDI_8255 help Enable support for ComputerBoards/MeasurementComputing PCI-DAS with @@ -974,8 +953,7 @@ config COMEDI_CB_PCIDDA config COMEDI_CB_PCIMDAS tristate "MeasurementComputing PCIM-DAS1602/16, PCIe-DAS1602/16 support" - depends on HAS_IOPORT - depends on COMEDI_8254 + select COMEDI_8254 select COMEDI_8255 help Enable support for ComputerBoards/MeasurementComputing PCI Migration @@ -995,8 +973,7 @@ config COMEDI_CB_PCIMDDA config COMEDI_ME4000 tristate "Meilhaus ME-4000 support" - depends on HAS_IOPORT - depends on COMEDI_8254 + select COMEDI_8254 help Enable support for Meilhaus PCI data acquisition cards ME-4650, ME-4670i, ME-4680, ME-4680i and ME-4680is @@ -1054,7 +1031,7 @@ config COMEDI_NI_670X config COMEDI_NI_LABPC_PCI tristate "NI Lab-PC PCI-1200 support" - depends on COMEDI_NI_LABPC + select COMEDI_NI_LABPC help Enable support for National Instruments Lab-PC PCI-1200. @@ -1076,7 +1053,6 @@ config COMEDI_NI_PCIDIO config COMEDI_NI_PCIMIO tristate "NI PCI-MIO-E series and M series support" depends on HAS_DMA - depends on HAS_IOPORT select COMEDI_NI_TIOCMD select COMEDI_8255 help @@ -1098,8 +1074,7 @@ config COMEDI_NI_PCIMIO config COMEDI_RTD520 tristate "Real Time Devices PCI4520/DM7520 support" - depends on HAS_IOPORT - depends on COMEDI_8254 + select COMEDI_8254 help Enable support for Real Time Devices PCI4520/DM7520 @@ -1139,8 +1114,7 @@ if COMEDI_PCMCIA_DRIVERS config COMEDI_CB_DAS16_CS tristate "CB DAS16 series PCMCIA support" - depends on HAS_IOPORT - depends on COMEDI_8254 + select COMEDI_8254 help Enable support for the ComputerBoards/MeasurementComputing PCMCIA cards DAS16/16, PCM-DAS16D/12 and PCM-DAS16s/16 @@ -1150,7 +1124,7 @@ config COMEDI_CB_DAS16_CS config COMEDI_DAS08_CS tristate "CB DAS08 PCMCIA support" - depends on COMEDI_DAS08 + select COMEDI_DAS08 help Enable support for the ComputerBoards/MeasurementComputing DAS-08 PCMCIA card @@ -1160,7 +1134,6 @@ config COMEDI_DAS08_CS config COMEDI_NI_DAQ_700_CS tristate "NI DAQCard-700 PCMCIA support" - depends on HAS_IOPORT help Enable support for the National Instruments PCMCIA DAQCard-700 DIO @@ -1169,7 +1142,6 @@ config COMEDI_NI_DAQ_700_CS config COMEDI_NI_DAQ_DIO24_CS tristate "NI DAQ-Card DIO-24 PCMCIA support" - depends on HAS_IOPORT select COMEDI_8255 help Enable support for the National Instruments PCMCIA DAQ-Card DIO-24 @@ -1179,7 +1151,7 @@ config COMEDI_NI_DAQ_DIO24_CS config COMEDI_NI_LABPC_CS tristate "NI DAQCard-1200 PCMCIA support" - depends on COMEDI_NI_LABPC + select COMEDI_NI_LABPC help Enable support for the National Instruments PCMCIA DAQCard-1200 @@ -1188,7 +1160,6 @@ config COMEDI_NI_LABPC_CS config COMEDI_NI_MIO_CS tristate "NI DAQCard E series PCMCIA support" - depends on HAS_IOPORT select COMEDI_NI_TIO select COMEDI_8255 help @@ -1201,7 +1172,6 @@ config COMEDI_NI_MIO_CS config COMEDI_QUATECH_DAQP_CS tristate "Quatech DAQP PCMCIA data capture card support" - depends on HAS_IOPORT help Enable support for the Quatech DAQP PCMCIA data capture cards DAQP-208 and DAQP-308 @@ -1278,14 +1248,12 @@ endif # COMEDI_USB_DRIVERS config COMEDI_8254 tristate - depends on HAS_IOPORT config COMEDI_8255 tristate config COMEDI_8255_SA tristate "Standalone 8255 support" - depends on HAS_IOPORT select COMEDI_8255 help Enable support for 8255 digital I/O as a standalone driver. @@ -1317,7 +1285,7 @@ config COMEDI_KCOMEDILIB called kcomedilib. config COMEDI_AMPLC_DIO200 - depends on COMEDI_8254 + select COMEDI_8254 tristate config COMEDI_AMPLC_PC236 @@ -1326,7 +1294,7 @@ config COMEDI_AMPLC_PC236 config COMEDI_DAS08 tristate - depends on COMEDI_8254 + select COMEDI_8254 select COMEDI_8255 config COMEDI_ISADMA @@ -1334,8 +1302,7 @@ config COMEDI_ISADMA config COMEDI_NI_LABPC tristate - depends on HAS_IOPORT - depends on COMEDI_8254 + select COMEDI_8254 select COMEDI_8255 config COMEDI_NI_LABPC_ISADMA diff --git a/drivers/counter/counter-chrdev.c b/drivers/counter/counter-chrdev.c index 80acdf62794a..afc94d0062b1 100644 --- a/drivers/counter/counter-chrdev.c +++ b/drivers/counter/counter-chrdev.c @@ -247,8 +247,8 @@ static int counter_get_ext(const struct counter_comp *const ext, if (*id == component_id) return 0; - if (ext->type == COUNTER_COMP_ARRAY) { - element = ext->priv; + if (ext[*ext_idx].type == COUNTER_COMP_ARRAY) { + element = ext[*ext_idx].priv; if (component_id - *id < element->length) return 0; diff --git a/drivers/counter/microchip-tcb-capture.c b/drivers/counter/microchip-tcb-capture.c index 975e431d1590..b3e615cbd2ca 100644 --- a/drivers/counter/microchip-tcb-capture.c +++ b/drivers/counter/microchip-tcb-capture.c @@ -97,7 +97,7 @@ static int mchp_tc_count_function_write(struct counter_device *counter, priv->qdec_mode = 0; /* Set highest rate based on whether soc has gclk or not */ bmr &= ~(ATMEL_TC_QDEN | ATMEL_TC_POSEN); - if (priv->tc_cfg->has_gclk) + if (!priv->tc_cfg->has_gclk) cmr |= ATMEL_TC_TIMER_CLOCK2; else cmr |= ATMEL_TC_TIMER_CLOCK1; diff --git a/drivers/cxl/acpi.c b/drivers/cxl/acpi.c index d1c559879dcc..40d055560e52 100644 --- a/drivers/cxl/acpi.c +++ b/drivers/cxl/acpi.c @@ -14,7 +14,7 @@ struct cxl_cxims_data { int nr_maps; - u64 xormaps[]; + u64 xormaps[] __counted_by(nr_maps); }; /* @@ -112,9 +112,9 @@ static int cxl_parse_cxims(union acpi_subtable_headers *header, void *arg, GFP_KERNEL); if (!cximsd) return -ENOMEM; + cximsd->nr_maps = nr_maps; memcpy(cximsd->xormaps, cxims->xormap_list, nr_maps * sizeof(*cximsd->xormaps)); - cximsd->nr_maps = nr_maps; cxlrd->platform_data = cximsd; return 0; diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c index ca60bb8114f2..4df4f614f490 100644 --- a/drivers/cxl/core/mbox.c +++ b/drivers/cxl/core/mbox.c @@ -715,24 +715,25 @@ static void cxl_walk_cel(struct cxl_memdev_state *mds, size_t size, u8 *cel) for (i = 0; i < cel_entries; i++) { u16 opcode = le16_to_cpu(cel_entry[i].opcode); struct cxl_mem_command *cmd = cxl_mem_find_command(opcode); + int enabled = 0; - if (!cmd && (!cxl_is_poison_command(opcode) || - !cxl_is_security_command(opcode))) { - dev_dbg(dev, - "Opcode 0x%04x unsupported by driver\n", opcode); - continue; - } - - if (cmd) + if (cmd) { set_bit(cmd->info.id, mds->enabled_cmds); + enabled++; + } - if (cxl_is_poison_command(opcode)) + if (cxl_is_poison_command(opcode)) { cxl_set_poison_cmd_enabled(&mds->poison, opcode); + enabled++; + } - if (cxl_is_security_command(opcode)) + if (cxl_is_security_command(opcode)) { cxl_set_security_cmd_enabled(&mds->security, opcode); + enabled++; + } - dev_dbg(dev, "Opcode 0x%04x enabled\n", opcode); + dev_dbg(dev, "Opcode 0x%04x %s\n", opcode, + enabled ? "enabled" : "unsupported by driver"); } } diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index 724be8448eb4..7ca01a834e18 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -1,5 +1,6 @@ // SPDX-License-Identifier: GPL-2.0-only /* Copyright(c) 2020 Intel Corporation. All rights reserved. */ +#include #include #include #include @@ -706,16 +707,20 @@ static int cxl_setup_comp_regs(struct device *dev, struct cxl_register_map *map, return cxl_setup_regs(map); } -static inline int cxl_port_setup_regs(struct cxl_port *port, - resource_size_t component_reg_phys) +static int cxl_port_setup_regs(struct cxl_port *port, + resource_size_t component_reg_phys) { + if (dev_is_platform(port->uport_dev)) + return 0; return cxl_setup_comp_regs(&port->dev, &port->comp_map, component_reg_phys); } -static inline int cxl_dport_setup_regs(struct cxl_dport *dport, - resource_size_t component_reg_phys) +static int cxl_dport_setup_regs(struct cxl_dport *dport, + resource_size_t component_reg_phys) { + if (dev_is_platform(dport->dport_dev)) + return 0; return cxl_setup_comp_regs(dport->dport_dev, &dport->comp_map, component_reg_phys); } diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index e115ba382e04..6d63b8798c29 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -717,13 +717,35 @@ static int match_free_decoder(struct device *dev, void *data) return 0; } +static int match_auto_decoder(struct device *dev, void *data) +{ + struct cxl_region_params *p = data; + struct cxl_decoder *cxld; + struct range *r; + + if (!is_switch_decoder(dev)) + return 0; + + cxld = to_cxl_decoder(dev); + r = &cxld->hpa_range; + + if (p->res && p->res->start == r->start && p->res->end == r->end) + return 1; + + return 0; +} + static struct cxl_decoder *cxl_region_find_decoder(struct cxl_port *port, struct cxl_region *cxlr) { struct device *dev; int id = 0; - dev = device_find_child(&port->dev, &id, match_free_decoder); + if (test_bit(CXL_REGION_F_AUTO, &cxlr->flags)) + dev = device_find_child(&port->dev, &cxlr->params, + match_auto_decoder); + else + dev = device_find_child(&port->dev, &id, match_free_decoder); if (!dev) return NULL; /* @@ -1154,16 +1176,15 @@ static int cxl_port_setup_targets(struct cxl_port *port, } /* - * If @parent_port is masking address bits, pick the next unused address - * bit to route @port's targets. + * Interleave granularity is a multiple of @parent_port granularity. + * Multiplier is the parent port interleave ways. */ - if (parent_iw > 1 && cxl_rr->nr_targets > 1) { - u32 address_bit = max(peig + peiw, eiw + peig); - - eig = address_bit - eiw + 1; - } else { - eiw = peiw; - eig = peig; + rc = granularity_to_eig(parent_ig * parent_iw, &eig); + if (rc) { + dev_dbg(&cxlr->dev, + "%s: invalid granularity calculation (%d * %d)\n", + dev_name(&parent_port->dev), parent_ig, parent_iw); + return rc; } rc = eig_to_granularity(eig, &ig); diff --git a/drivers/cxl/pci.c b/drivers/cxl/pci.c index 1cb1494c28fe..44a21ab7add5 100644 --- a/drivers/cxl/pci.c +++ b/drivers/cxl/pci.c @@ -529,7 +529,6 @@ static int cxl_pci_setup_regs(struct pci_dev *pdev, enum cxl_regloc_type type, static int cxl_pci_ras_unmask(struct pci_dev *pdev) { - struct pci_host_bridge *host_bridge = pci_find_host_bridge(pdev->bus); struct cxl_dev_state *cxlds = pci_get_drvdata(pdev); void __iomem *addr; u32 orig_val, val, mask; @@ -541,9 +540,9 @@ static int cxl_pci_ras_unmask(struct pci_dev *pdev) return 0; } - /* BIOS has CXL error control */ - if (!host_bridge->native_cxl_error) - return -ENXIO; + /* BIOS has PCIe AER error control */ + if (!pcie_aer_is_native(pdev)) + return 0; rc = pcie_capability_read_word(pdev, PCI_EXP_DEVCTL, &cap); if (rc) diff --git a/drivers/dma-buf/dma-fence-unwrap.c b/drivers/dma-buf/dma-fence-unwrap.c index c625bb2b5d56..628af51c81af 100644 --- a/drivers/dma-buf/dma-fence-unwrap.c +++ b/drivers/dma-buf/dma-fence-unwrap.c @@ -76,16 +76,11 @@ struct dma_fence *__dma_fence_unwrap_merge(unsigned int num_fences, dma_fence_unwrap_for_each(tmp, &iter[i], fences[i]) { if (!dma_fence_is_signaled(tmp)) { ++count; - } else if (test_bit(DMA_FENCE_FLAG_TIMESTAMP_BIT, - &tmp->flags)) { - if (ktime_after(tmp->timestamp, timestamp)) - timestamp = tmp->timestamp; } else { - /* - * Use the current time if the fence is - * currently signaling. - */ - timestamp = ktime_get(); + ktime_t t = dma_fence_timestamp(tmp); + + if (ktime_after(t, timestamp)) + timestamp = t; } } } diff --git a/drivers/dma-buf/sync_file.c b/drivers/dma-buf/sync_file.c index af57799c86ce..2e9a316c596a 100644 --- a/drivers/dma-buf/sync_file.c +++ b/drivers/dma-buf/sync_file.c @@ -268,13 +268,10 @@ static int sync_fill_fence_info(struct dma_fence *fence, sizeof(info->driver_name)); info->status = dma_fence_get_status(fence); - while (test_bit(DMA_FENCE_FLAG_SIGNALED_BIT, &fence->flags) && - !test_bit(DMA_FENCE_FLAG_TIMESTAMP_BIT, &fence->flags)) - cpu_relax(); info->timestamp_ns = - test_bit(DMA_FENCE_FLAG_TIMESTAMP_BIT, &fence->flags) ? - ktime_to_ns(fence->timestamp) : - ktime_set(0, 0); + dma_fence_is_signaled(fence) ? + ktime_to_ns(dma_fence_timestamp(fence)) : + ktime_set(0, 0); return info->status; } diff --git a/drivers/dma/fsl-edma-common.c b/drivers/dma/fsl-edma-common.c index a0f5741abcc4..6a3abe5b1790 100644 --- a/drivers/dma/fsl-edma-common.c +++ b/drivers/dma/fsl-edma-common.c @@ -92,8 +92,14 @@ static void fsl_edma3_enable_request(struct fsl_edma_chan *fsl_chan) edma_writel_chreg(fsl_chan, val, ch_sbr); - if (flags & FSL_EDMA_DRV_HAS_CHMUX) - edma_writel_chreg(fsl_chan, fsl_chan->srcid, ch_mux); + if (flags & FSL_EDMA_DRV_HAS_CHMUX) { + /* + * ch_mux: With the exception of 0, attempts to write a value + * already in use will be forced to 0. + */ + if (!edma_readl_chreg(fsl_chan, ch_mux)) + edma_writel_chreg(fsl_chan, fsl_chan->srcid, ch_mux); + } val = edma_readl_chreg(fsl_chan, ch_csr); val |= EDMA_V3_CH_CSR_ERQ; @@ -448,12 +454,25 @@ static void fsl_edma_set_tcd_regs(struct fsl_edma_chan *fsl_chan, edma_write_tcdreg(fsl_chan, tcd->dlast_sga, dlast_sga); + csr = le16_to_cpu(tcd->csr); + if (fsl_chan->is_sw) { - csr = le16_to_cpu(tcd->csr); csr |= EDMA_TCD_CSR_START; tcd->csr = cpu_to_le16(csr); } + /* + * Must clear CHn_CSR[DONE] bit before enable TCDn_CSR[ESG] at EDMAv3 + * eDMAv4 have not such requirement. + * Change MLINK need clear CHn_CSR[DONE] for both eDMAv3 and eDMAv4. + */ + if (((fsl_edma_drvflags(fsl_chan) & FSL_EDMA_DRV_CLEAR_DONE_E_SG) && + (csr & EDMA_TCD_CSR_E_SG)) || + ((fsl_edma_drvflags(fsl_chan) & FSL_EDMA_DRV_CLEAR_DONE_E_LINK) && + (csr & EDMA_TCD_CSR_E_LINK))) + edma_writel_chreg(fsl_chan, edma_readl_chreg(fsl_chan, ch_csr), ch_csr); + + edma_write_tcdreg(fsl_chan, tcd->csr, csr); } diff --git a/drivers/dma/fsl-edma-common.h b/drivers/dma/fsl-edma-common.h index 3cc0cc8fc2d0..40d50cc3d75a 100644 --- a/drivers/dma/fsl-edma-common.h +++ b/drivers/dma/fsl-edma-common.h @@ -183,11 +183,23 @@ struct fsl_edma_desc { #define FSL_EDMA_DRV_BUS_8BYTE BIT(10) #define FSL_EDMA_DRV_DEV_TO_DEV BIT(11) #define FSL_EDMA_DRV_ALIGN_64BYTE BIT(12) +/* Need clean CHn_CSR DONE before enable TCD's ESG */ +#define FSL_EDMA_DRV_CLEAR_DONE_E_SG BIT(13) +/* Need clean CHn_CSR DONE before enable TCD's MAJORELINK */ +#define FSL_EDMA_DRV_CLEAR_DONE_E_LINK BIT(14) #define FSL_EDMA_DRV_EDMA3 (FSL_EDMA_DRV_SPLIT_REG | \ FSL_EDMA_DRV_BUS_8BYTE | \ FSL_EDMA_DRV_DEV_TO_DEV | \ - FSL_EDMA_DRV_ALIGN_64BYTE) + FSL_EDMA_DRV_ALIGN_64BYTE | \ + FSL_EDMA_DRV_CLEAR_DONE_E_SG | \ + FSL_EDMA_DRV_CLEAR_DONE_E_LINK) + +#define FSL_EDMA_DRV_EDMA4 (FSL_EDMA_DRV_SPLIT_REG | \ + FSL_EDMA_DRV_BUS_8BYTE | \ + FSL_EDMA_DRV_DEV_TO_DEV | \ + FSL_EDMA_DRV_ALIGN_64BYTE | \ + FSL_EDMA_DRV_CLEAR_DONE_E_LINK) struct fsl_edma_drvdata { u32 dmamuxs; /* only used before v3 */ diff --git a/drivers/dma/fsl-edma-main.c b/drivers/dma/fsl-edma-main.c index 63d48d046f04..8c4ed7012e23 100644 --- a/drivers/dma/fsl-edma-main.c +++ b/drivers/dma/fsl-edma-main.c @@ -154,18 +154,20 @@ static struct dma_chan *fsl_edma3_xlate(struct of_phandle_args *dma_spec, fsl_chan = to_fsl_edma_chan(chan); i = fsl_chan - fsl_edma->chans; - chan = dma_get_slave_channel(chan); - chan->device->privatecnt++; fsl_chan->priority = dma_spec->args[1]; fsl_chan->is_rxchan = dma_spec->args[2] & ARGS_RX; fsl_chan->is_remote = dma_spec->args[2] & ARGS_REMOTE; fsl_chan->is_multi_fifo = dma_spec->args[2] & ARGS_MULTI_FIFO; if (!b_chmux && i == dma_spec->args[0]) { + chan = dma_get_slave_channel(chan); + chan->device->privatecnt++; mutex_unlock(&fsl_edma->fsl_edma_mutex); return chan; } else if (b_chmux && !fsl_chan->srcid) { /* if controller support channel mux, choose a free channel */ + chan = dma_get_slave_channel(chan); + chan->device->privatecnt++; fsl_chan->srcid = dma_spec->args[0]; mutex_unlock(&fsl_edma->fsl_edma_mutex); return chan; @@ -355,7 +357,7 @@ static struct fsl_edma_drvdata imx93_data3 = { }; static struct fsl_edma_drvdata imx93_data4 = { - .flags = FSL_EDMA_DRV_HAS_CHMUX | FSL_EDMA_DRV_HAS_DMACLK | FSL_EDMA_DRV_EDMA3, + .flags = FSL_EDMA_DRV_HAS_CHMUX | FSL_EDMA_DRV_HAS_DMACLK | FSL_EDMA_DRV_EDMA4, .chreg_space_sz = 0x8000, .chreg_off = 0x10000, .setup_irq = fsl_edma3_irq_init, diff --git a/drivers/dma/idxd/device.c b/drivers/dma/idxd/device.c index 22d6f4e455b7..8f754f922217 100644 --- a/drivers/dma/idxd/device.c +++ b/drivers/dma/idxd/device.c @@ -477,6 +477,7 @@ static void idxd_cmd_exec(struct idxd_device *idxd, int cmd_code, u32 operand, union idxd_command_reg cmd; DECLARE_COMPLETION_ONSTACK(done); u32 stat; + unsigned long flags; if (idxd_device_is_halted(idxd)) { dev_warn(&idxd->pdev->dev, "Device is HALTED!\n"); @@ -490,7 +491,7 @@ static void idxd_cmd_exec(struct idxd_device *idxd, int cmd_code, u32 operand, cmd.operand = operand; cmd.int_req = 1; - spin_lock(&idxd->cmd_lock); + spin_lock_irqsave(&idxd->cmd_lock, flags); wait_event_lock_irq(idxd->cmd_waitq, !test_bit(IDXD_FLAG_CMD_RUNNING, &idxd->flags), idxd->cmd_lock); @@ -507,7 +508,7 @@ static void idxd_cmd_exec(struct idxd_device *idxd, int cmd_code, u32 operand, * After command submitted, release lock and go to sleep until * the command completes via interrupt. */ - spin_unlock(&idxd->cmd_lock); + spin_unlock_irqrestore(&idxd->cmd_lock, flags); wait_for_completion(&done); stat = ioread32(idxd->reg_base + IDXD_CMDSTS_OFFSET); spin_lock(&idxd->cmd_lock); diff --git a/drivers/dma/mediatek/mtk-uart-apdma.c b/drivers/dma/mediatek/mtk-uart-apdma.c index c51dc017b48a..06d12ac39144 100644 --- a/drivers/dma/mediatek/mtk-uart-apdma.c +++ b/drivers/dma/mediatek/mtk-uart-apdma.c @@ -450,9 +450,8 @@ static int mtk_uart_apdma_device_pause(struct dma_chan *chan) mtk_uart_apdma_write(c, VFF_EN, VFF_EN_CLR_B); mtk_uart_apdma_write(c, VFF_INT_EN, VFF_INT_EN_CLR_B); - synchronize_irq(c->irq); - spin_unlock_irqrestore(&c->vc.lock, flags); + synchronize_irq(c->irq); return 0; } diff --git a/drivers/dma/ste_dma40.c b/drivers/dma/ste_dma40.c index 89e82508c133..002833fb1fa0 100644 --- a/drivers/dma/ste_dma40.c +++ b/drivers/dma/ste_dma40.c @@ -3668,6 +3668,7 @@ static int __init d40_probe(struct platform_device *pdev) regulator_disable(base->lcpa_regulator); regulator_put(base->lcpa_regulator); } + pm_runtime_disable(base->dev); report_failure: d40_err(dev, "probe failed\n"); diff --git a/drivers/dma/stm32-dma.c b/drivers/dma/stm32-dma.c index 5c36811aa134..0b30151fb45c 100644 --- a/drivers/dma/stm32-dma.c +++ b/drivers/dma/stm32-dma.c @@ -1113,8 +1113,10 @@ static struct dma_async_tx_descriptor *stm32_dma_prep_slave_sg( chan->chan_reg.dma_scr &= ~STM32_DMA_SCR_PFCTRL; /* Activate Double Buffer Mode if DMA triggers STM32 MDMA and more than 1 sg */ - if (chan->trig_mdma && sg_len > 1) + if (chan->trig_mdma && sg_len > 1) { chan->chan_reg.dma_scr |= STM32_DMA_SCR_DBM; + chan->chan_reg.dma_scr &= ~STM32_DMA_SCR_CT; + } for_each_sg(sgl, sg, sg_len, i) { ret = stm32_dma_set_xfer_param(chan, direction, &buswidth, @@ -1387,11 +1389,12 @@ static size_t stm32_dma_desc_residue(struct stm32_dma_chan *chan, residue = stm32_dma_get_remaining_bytes(chan); - if (chan->desc->cyclic && !stm32_dma_is_current_sg(chan)) { + if ((chan->desc->cyclic || chan->trig_mdma) && !stm32_dma_is_current_sg(chan)) { n_sg++; if (n_sg == chan->desc->num_sgs) n_sg = 0; - residue = sg_req->len; + if (!chan->trig_mdma) + residue = sg_req->len; } /* @@ -1401,7 +1404,7 @@ static size_t stm32_dma_desc_residue(struct stm32_dma_chan *chan, * residue = remaining bytes from NDTR + remaining * periods/sg to be transferred */ - if (!chan->desc->cyclic || n_sg != 0) + if ((!chan->desc->cyclic && !chan->trig_mdma) || n_sg != 0) for (i = n_sg; i < desc->num_sgs; i++) residue += desc->sg_req[i].len; diff --git a/drivers/dma/stm32-mdma.c b/drivers/dma/stm32-mdma.c index 0de234022c6d..bae08b3f55c7 100644 --- a/drivers/dma/stm32-mdma.c +++ b/drivers/dma/stm32-mdma.c @@ -777,8 +777,6 @@ static int stm32_mdma_setup_xfer(struct stm32_mdma_chan *chan, /* Enable interrupts */ ccr &= ~STM32_MDMA_CCR_IRQ_MASK; ccr |= STM32_MDMA_CCR_TEIE | STM32_MDMA_CCR_CTCIE; - if (sg_len > 1) - ccr |= STM32_MDMA_CCR_BTIE; desc->ccr = ccr; return 0; @@ -1236,6 +1234,10 @@ static int stm32_mdma_resume(struct dma_chan *c) unsigned long flags; u32 status, reg; + /* Transfer can be terminated */ + if (!chan->desc || (stm32_mdma_read(dmadev, STM32_MDMA_CCR(chan->id)) & STM32_MDMA_CCR_EN)) + return -EPERM; + hwdesc = chan->desc->node[chan->curr_hwdesc].hwdesc; spin_lock_irqsave(&chan->vchan.lock, flags); @@ -1316,21 +1318,35 @@ static int stm32_mdma_slave_config(struct dma_chan *c, static size_t stm32_mdma_desc_residue(struct stm32_mdma_chan *chan, struct stm32_mdma_desc *desc, - u32 curr_hwdesc) + u32 curr_hwdesc, + struct dma_tx_state *state) { struct stm32_mdma_device *dmadev = stm32_mdma_get_dev(chan); struct stm32_mdma_hwdesc *hwdesc; - u32 cbndtr, residue, modulo, burst_size; + u32 cisr, clar, cbndtr, residue, modulo, burst_size; int i; + cisr = stm32_mdma_read(dmadev, STM32_MDMA_CISR(chan->id)); + residue = 0; - for (i = curr_hwdesc + 1; i < desc->count; i++) { + /* Get the next hw descriptor to process from current transfer */ + clar = stm32_mdma_read(dmadev, STM32_MDMA_CLAR(chan->id)); + for (i = desc->count - 1; i >= 0; i--) { hwdesc = desc->node[i].hwdesc; + + if (hwdesc->clar == clar) + break;/* Current transfer found, stop cumulating */ + + /* Cumulate residue of unprocessed hw descriptors */ residue += STM32_MDMA_CBNDTR_BNDT(hwdesc->cbndtr); } cbndtr = stm32_mdma_read(dmadev, STM32_MDMA_CBNDTR(chan->id)); residue += cbndtr & STM32_MDMA_CBNDTR_BNDT_MASK; + state->in_flight_bytes = 0; + if (chan->chan_config.m2m_hw && (cisr & STM32_MDMA_CISR_CRQA)) + state->in_flight_bytes = cbndtr & STM32_MDMA_CBNDTR_BNDT_MASK; + if (!chan->mem_burst) return residue; @@ -1360,11 +1376,10 @@ static enum dma_status stm32_mdma_tx_status(struct dma_chan *c, vdesc = vchan_find_desc(&chan->vchan, cookie); if (chan->desc && cookie == chan->desc->vdesc.tx.cookie) - residue = stm32_mdma_desc_residue(chan, chan->desc, - chan->curr_hwdesc); + residue = stm32_mdma_desc_residue(chan, chan->desc, chan->curr_hwdesc, state); else if (vdesc) - residue = stm32_mdma_desc_residue(chan, - to_stm32_mdma_desc(vdesc), 0); + residue = stm32_mdma_desc_residue(chan, to_stm32_mdma_desc(vdesc), 0, state); + dma_set_residue(state, residue); spin_unlock_irqrestore(&chan->vchan.lock, flags); diff --git a/drivers/dma/ti/k3-udma-glue.c b/drivers/dma/ti/k3-udma-glue.c index 789193ed0386..c278d5facf7d 100644 --- a/drivers/dma/ti/k3-udma-glue.c +++ b/drivers/dma/ti/k3-udma-glue.c @@ -558,6 +558,9 @@ int k3_udma_glue_tx_get_irq(struct k3_udma_glue_tx_channel *tx_chn) tx_chn->virq = k3_ringacc_get_ring_irq_num(tx_chn->ringtxcq); } + if (!tx_chn->virq) + return -ENXIO; + return tx_chn->virq; } EXPORT_SYMBOL_GPL(k3_udma_glue_tx_get_irq); diff --git a/drivers/firewire/core-device.c b/drivers/firewire/core-device.c index a3104e35412c..aa597cda0d88 100644 --- a/drivers/firewire/core-device.c +++ b/drivers/firewire/core-device.c @@ -1211,7 +1211,7 @@ void fw_node_event(struct fw_card *card, struct fw_node *node, int event) * without actually having a link. */ create: - device = kzalloc(sizeof(*device), GFP_KERNEL); + device = kzalloc(sizeof(*device), GFP_ATOMIC); if (device == NULL) break; diff --git a/drivers/firewire/core-topology.c b/drivers/firewire/core-topology.c index 88466b663482..f40c81534381 100644 --- a/drivers/firewire/core-topology.c +++ b/drivers/firewire/core-topology.c @@ -101,7 +101,7 @@ static struct fw_node *fw_node_create(u32 sid, int port_count, int color) { struct fw_node *node; - node = kzalloc(struct_size(node, ports, port_count), GFP_KERNEL); + node = kzalloc(struct_size(node, ports, port_count), GFP_ATOMIC); if (node == NULL) return NULL; diff --git a/drivers/firewire/sbp2.c b/drivers/firewire/sbp2.c index 26db5b8dfc1e..749868b9e80d 100644 --- a/drivers/firewire/sbp2.c +++ b/drivers/firewire/sbp2.c @@ -81,7 +81,8 @@ MODULE_PARM_DESC(exclusive_login, "Exclusive login to sbp2 device " * * - power condition * Set the power condition field in the START STOP UNIT commands sent by - * sd_mod on suspend, resume, and shutdown (if manage_start_stop is on). + * sd_mod on suspend, resume, and shutdown (if manage_system_start_stop or + * manage_runtime_start_stop is on). * Some disks need this to spin down or to resume properly. * * - override internal blacklist @@ -1517,8 +1518,10 @@ static int sbp2_scsi_slave_configure(struct scsi_device *sdev) sdev->use_10_for_rw = 1; - if (sbp2_param_exclusive_login) - sdev->manage_start_stop = 1; + if (sbp2_param_exclusive_login) { + sdev->manage_system_start_stop = true; + sdev->manage_runtime_start_stop = true; + } if (sdev->type == TYPE_ROM) sdev->use_10_for_ms = 1; diff --git a/drivers/firmware/arm_ffa/driver.c b/drivers/firmware/arm_ffa/driver.c index 2109cd178ff7..121f4fc903cd 100644 --- a/drivers/firmware/arm_ffa/driver.c +++ b/drivers/firmware/arm_ffa/driver.c @@ -397,6 +397,19 @@ static u32 ffa_get_num_pages_sg(struct scatterlist *sg) return num_pages; } +static u8 ffa_memory_attributes_get(u32 func_id) +{ + /* + * For the memory lend or donate operation, if the receiver is a PE or + * a proxy endpoint, the owner/sender must not specify the attributes + */ + if (func_id == FFA_FN_NATIVE(MEM_LEND) || + func_id == FFA_MEM_LEND) + return 0; + + return FFA_MEM_NORMAL | FFA_MEM_WRITE_BACK | FFA_MEM_INNER_SHAREABLE; +} + static int ffa_setup_and_transmit(u32 func_id, void *buffer, u32 max_fragsize, struct ffa_mem_ops_args *args) @@ -413,8 +426,7 @@ ffa_setup_and_transmit(u32 func_id, void *buffer, u32 max_fragsize, mem_region->tag = args->tag; mem_region->flags = args->flags; mem_region->sender_id = drv_info->vm_id; - mem_region->attributes = FFA_MEM_NORMAL | FFA_MEM_WRITE_BACK | - FFA_MEM_INNER_SHAREABLE; + mem_region->attributes = ffa_memory_attributes_get(func_id); ep_mem_access = &mem_region->ep_mem_access[0]; for (idx = 0; idx < args->nattrs; idx++, ep_mem_access++) { diff --git a/drivers/firmware/arm_scmi/perf.c b/drivers/firmware/arm_scmi/perf.c index c0cd556fbaae..30dedd6ebfde 100644 --- a/drivers/firmware/arm_scmi/perf.c +++ b/drivers/firmware/arm_scmi/perf.c @@ -1080,6 +1080,8 @@ static int scmi_perf_protocol_init(const struct scmi_protocol_handle *ph) if (!pinfo) return -ENOMEM; + pinfo->version = version; + ret = scmi_perf_attributes_get(ph, pinfo); if (ret) return ret; @@ -1104,8 +1106,6 @@ static int scmi_perf_protocol_init(const struct scmi_protocol_handle *ph) if (ret) return ret; - pinfo->version = version; - return ph->set_priv(ph, pinfo); } diff --git a/drivers/firmware/efi/efi.c b/drivers/firmware/efi/efi.c index 1599f1176842..1974f0ad32ba 100644 --- a/drivers/firmware/efi/efi.c +++ b/drivers/firmware/efi/efi.c @@ -273,9 +273,13 @@ static __init int efivar_ssdt_load(void) if (status == EFI_NOT_FOUND) { break; } else if (status == EFI_BUFFER_TOO_SMALL) { - name = krealloc(name, name_size, GFP_KERNEL); - if (!name) + efi_char16_t *name_tmp = + krealloc(name, name_size, GFP_KERNEL); + if (!name_tmp) { + kfree(name); return -ENOMEM; + } + name = name_tmp; continue; } @@ -623,6 +627,34 @@ static __init int match_config_table(const efi_guid_t *guid, return 0; } +/** + * reserve_unaccepted - Map and reserve unaccepted configuration table + * @unaccepted: Pointer to unaccepted memory table + * + * memblock_add() makes sure that the table is mapped in direct mapping. During + * normal boot it happens automatically because the table is allocated from + * usable memory. But during crashkernel boot only memory specifically reserved + * for crash scenario is mapped. memblock_add() forces the table to be mapped + * in crashkernel case. + * + * Align the range to the nearest page borders. Ranges smaller than page size + * are not going to be mapped. + * + * memblock_reserve() makes sure that future allocations will not touch the + * table. + */ + +static __init void reserve_unaccepted(struct efi_unaccepted_memory *unaccepted) +{ + phys_addr_t start, size; + + start = PAGE_ALIGN_DOWN(efi.unaccepted); + size = PAGE_ALIGN(sizeof(*unaccepted) + unaccepted->size); + + memblock_add(start, size); + memblock_reserve(start, size); +} + int __init efi_config_parse_tables(const efi_config_table_t *config_tables, int count, const efi_config_table_type_t *arch_tables) @@ -751,11 +783,9 @@ int __init efi_config_parse_tables(const efi_config_table_t *config_tables, unaccepted = early_memremap(efi.unaccepted, sizeof(*unaccepted)); if (unaccepted) { - unsigned long size; if (unaccepted->version == 1) { - size = sizeof(*unaccepted) + unaccepted->size; - memblock_reserve(efi.unaccepted, size); + reserve_unaccepted(unaccepted); } else { efi.unaccepted = EFI_INVALID_TABLE_ADDR; } diff --git a/drivers/firmware/efi/libstub/unaccepted_memory.c b/drivers/firmware/efi/libstub/unaccepted_memory.c index ca61f4733ea5..9a655f30ba47 100644 --- a/drivers/firmware/efi/libstub/unaccepted_memory.c +++ b/drivers/firmware/efi/libstub/unaccepted_memory.c @@ -62,7 +62,7 @@ efi_status_t allocate_unaccepted_bitmap(__u32 nr_desc, bitmap_size = DIV_ROUND_UP(unaccepted_end - unaccepted_start, EFI_UNACCEPTED_UNIT_SIZE * BITS_PER_BYTE); - status = efi_bs_call(allocate_pool, EFI_LOADER_DATA, + status = efi_bs_call(allocate_pool, EFI_ACPI_RECLAIM_MEMORY, sizeof(*unaccepted_table) + bitmap_size, (void **)&unaccepted_table); if (status != EFI_SUCCESS) { diff --git a/drivers/firmware/efi/libstub/x86-stub.c b/drivers/firmware/efi/libstub/x86-stub.c index 2fee52ed335d..9d5df683f882 100644 --- a/drivers/firmware/efi/libstub/x86-stub.c +++ b/drivers/firmware/efi/libstub/x86-stub.c @@ -605,11 +605,8 @@ setup_e820(struct boot_params *params, struct setup_data *e820ext, u32 e820ext_s break; case EFI_UNACCEPTED_MEMORY: - if (!IS_ENABLED(CONFIG_UNACCEPTED_MEMORY)) { - efi_warn_once( -"The system has unaccepted memory, but kernel does not support it\nConsider enabling CONFIG_UNACCEPTED_MEMORY\n"); + if (!IS_ENABLED(CONFIG_UNACCEPTED_MEMORY)) continue; - } e820_type = E820_TYPE_RAM; process_unaccepted_memory(d->phys_addr, d->phys_addr + PAGE_SIZE * d->num_pages); @@ -852,6 +849,8 @@ void __noreturn efi_stub_entry(efi_handle_t handle, unsigned long kernel_entry; efi_status_t status; + boot_params_pointer = boot_params; + efi_system_table = sys_table_arg; /* Check if we were booted by the EFI firmware */ if (efi_system_table->hdr.signature != EFI_SYSTEM_TABLE_SIGNATURE) diff --git a/drivers/firmware/efi/libstub/x86-stub.h b/drivers/firmware/efi/libstub/x86-stub.h index 37c5a36b9d8c..2748bca192df 100644 --- a/drivers/firmware/efi/libstub/x86-stub.h +++ b/drivers/firmware/efi/libstub/x86-stub.h @@ -2,6 +2,8 @@ #include +extern struct boot_params *boot_params_pointer asm("boot_params"); + extern void trampoline_32bit_src(void *, bool); extern const u16 trampoline_ljmp_imm_offset; diff --git a/drivers/firmware/efi/unaccepted_memory.c b/drivers/firmware/efi/unaccepted_memory.c index 853f7dc3c21d..135278ddaf62 100644 --- a/drivers/firmware/efi/unaccepted_memory.c +++ b/drivers/firmware/efi/unaccepted_memory.c @@ -5,9 +5,17 @@ #include #include -/* Protects unaccepted memory bitmap */ +/* Protects unaccepted memory bitmap and accepting_list */ static DEFINE_SPINLOCK(unaccepted_memory_lock); +struct accept_range { + struct list_head list; + unsigned long start; + unsigned long end; +}; + +static LIST_HEAD(accepting_list); + /* * accept_memory() -- Consult bitmap and accept the memory if needed. * @@ -24,6 +32,7 @@ void accept_memory(phys_addr_t start, phys_addr_t end) { struct efi_unaccepted_memory *unaccepted; unsigned long range_start, range_end; + struct accept_range range, *entry; unsigned long flags; u64 unit_size; @@ -78,20 +87,67 @@ void accept_memory(phys_addr_t start, phys_addr_t end) if (end > unaccepted->size * unit_size * BITS_PER_BYTE) end = unaccepted->size * unit_size * BITS_PER_BYTE; - range_start = start / unit_size; - + range.start = start / unit_size; + range.end = DIV_ROUND_UP(end, unit_size); +retry: spin_lock_irqsave(&unaccepted_memory_lock, flags); + + /* + * Check if anybody works on accepting the same range of the memory. + * + * The check is done with unit_size granularity. It is crucial to catch + * all accept requests to the same unit_size block, even if they don't + * overlap on physical address level. + */ + list_for_each_entry(entry, &accepting_list, list) { + if (entry->end < range.start) + continue; + if (entry->start >= range.end) + continue; + + /* + * Somebody else accepting the range. Or at least part of it. + * + * Drop the lock and retry until it is complete. + */ + spin_unlock_irqrestore(&unaccepted_memory_lock, flags); + goto retry; + } + + /* + * Register that the range is about to be accepted. + * Make sure nobody else will accept it. + */ + list_add(&range.list, &accepting_list); + + range_start = range.start; for_each_set_bitrange_from(range_start, range_end, unaccepted->bitmap, - DIV_ROUND_UP(end, unit_size)) { + range.end) { unsigned long phys_start, phys_end; unsigned long len = range_end - range_start; phys_start = range_start * unit_size + unaccepted->phys_base; phys_end = range_end * unit_size + unaccepted->phys_base; + /* + * Keep interrupts disabled until the accept operation is + * complete in order to prevent deadlocks. + * + * Enabling interrupts before calling arch_accept_memory() + * creates an opportunity for an interrupt handler to request + * acceptance for the same memory. The handler will continuously + * spin with interrupts disabled, preventing other task from + * making progress with the acceptance process. + */ + spin_unlock(&unaccepted_memory_lock); + arch_accept_memory(phys_start, phys_end); + + spin_lock(&unaccepted_memory_lock); bitmap_clear(unaccepted->bitmap, range_start, len); } + + list_del(&range.list); spin_unlock_irqrestore(&unaccepted_memory_lock, flags); } diff --git a/drivers/firmware/imx/imx-dsp.c b/drivers/firmware/imx/imx-dsp.c index 3dba590a2a95..508eab346fc6 100644 --- a/drivers/firmware/imx/imx-dsp.c +++ b/drivers/firmware/imx/imx-dsp.c @@ -114,6 +114,7 @@ static int imx_dsp_setup_channels(struct imx_dsp_ipc *dsp_ipc) dsp_chan->idx = i % 2; dsp_chan->ch = mbox_request_channel_byname(cl, chan_name); if (IS_ERR(dsp_chan->ch)) { + kfree(dsp_chan->name); ret = PTR_ERR(dsp_chan->ch); if (ret != -EPROBE_DEFER) dev_err(dev, "Failed to request mbox chan %s ret %d\n", diff --git a/drivers/genpd/Makefile b/drivers/genpd/Makefile deleted file mode 100644 index 666753676e5c..000000000000 --- a/drivers/genpd/Makefile +++ /dev/null @@ -1,17 +0,0 @@ -# SPDX-License-Identifier: GPL-2.0-only -obj-y += actions/ -obj-y += amlogic/ -obj-y += apple/ -obj-y += bcm/ -obj-y += imx/ -obj-y += mediatek/ -obj-y += qcom/ -obj-y += renesas/ -obj-y += rockchip/ -obj-y += samsung/ -obj-y += st/ -obj-y += starfive/ -obj-y += sunxi/ -obj-y += tegra/ -obj-y += ti/ -obj-y += xilinx/ diff --git a/drivers/genpd/actions/Makefile b/drivers/genpd/actions/Makefile deleted file mode 100644 index 7e8aa473d12d..000000000000 --- a/drivers/genpd/actions/Makefile +++ /dev/null @@ -1,3 +0,0 @@ -# SPDX-License-Identifier: GPL-2.0+ -obj-$(CONFIG_OWL_PM_DOMAINS_HELPER) += owl-sps-helper.o -obj-$(CONFIG_OWL_PM_DOMAINS) += owl-sps.o diff --git a/drivers/genpd/actions/owl-sps-helper.c b/drivers/genpd/actions/owl-sps-helper.c deleted file mode 100644 index e3f36603dd53..000000000000 --- a/drivers/genpd/actions/owl-sps-helper.c +++ /dev/null @@ -1,48 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0+ -/* - * Actions Semi Owl Smart Power System (SPS) shared helpers - * - * Copyright 2012 Actions Semi Inc. - * Author: Actions Semi, Inc. - * - * Copyright (c) 2017 Andreas Färber - */ - -#include -#include -#include - -#define OWL_SPS_PG_CTL 0x0 - -int owl_sps_set_pg(void __iomem *base, u32 pwr_mask, u32 ack_mask, bool enable) -{ - u32 val; - bool ack; - int timeout; - - val = readl(base + OWL_SPS_PG_CTL); - ack = val & ack_mask; - if (ack == enable) - return 0; - - if (enable) - val |= pwr_mask; - else - val &= ~pwr_mask; - - writel(val, base + OWL_SPS_PG_CTL); - - for (timeout = 5000; timeout > 0; timeout -= 50) { - val = readl(base + OWL_SPS_PG_CTL); - if ((val & ack_mask) == (enable ? ack_mask : 0)) - break; - udelay(50); - } - if (timeout <= 0) - return -ETIMEDOUT; - - udelay(10); - - return 0; -} -EXPORT_SYMBOL_GPL(owl_sps_set_pg); diff --git a/drivers/genpd/actions/owl-sps.c b/drivers/genpd/actions/owl-sps.c deleted file mode 100644 index 73a9e0bb7e8e..000000000000 --- a/drivers/genpd/actions/owl-sps.c +++ /dev/null @@ -1,320 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0+ -/* - * Actions Semi Owl Smart Power System (SPS) - * - * Copyright 2012 Actions Semi Inc. - * Author: Actions Semi, Inc. - * - * Copyright (c) 2017 Andreas Färber - */ - -#include -#include -#include -#include -#include -#include -#include - -struct owl_sps_domain_info { - const char *name; - int pwr_bit; - int ack_bit; - unsigned int genpd_flags; -}; - -struct owl_sps_info { - unsigned num_domains; - const struct owl_sps_domain_info *domains; -}; - -struct owl_sps { - struct device *dev; - const struct owl_sps_info *info; - void __iomem *base; - struct genpd_onecell_data genpd_data; - struct generic_pm_domain *domains[]; -}; - -#define to_owl_pd(gpd) container_of(gpd, struct owl_sps_domain, genpd) - -struct owl_sps_domain { - struct generic_pm_domain genpd; - const struct owl_sps_domain_info *info; - struct owl_sps *sps; -}; - -static int owl_sps_set_power(struct owl_sps_domain *pd, bool enable) -{ - u32 pwr_mask, ack_mask; - - ack_mask = BIT(pd->info->ack_bit); - pwr_mask = BIT(pd->info->pwr_bit); - - return owl_sps_set_pg(pd->sps->base, pwr_mask, ack_mask, enable); -} - -static int owl_sps_power_on(struct generic_pm_domain *domain) -{ - struct owl_sps_domain *pd = to_owl_pd(domain); - - dev_dbg(pd->sps->dev, "%s power on", pd->info->name); - - return owl_sps_set_power(pd, true); -} - -static int owl_sps_power_off(struct generic_pm_domain *domain) -{ - struct owl_sps_domain *pd = to_owl_pd(domain); - - dev_dbg(pd->sps->dev, "%s power off", pd->info->name); - - return owl_sps_set_power(pd, false); -} - -static int owl_sps_init_domain(struct owl_sps *sps, int index) -{ - struct owl_sps_domain *pd; - - pd = devm_kzalloc(sps->dev, sizeof(*pd), GFP_KERNEL); - if (!pd) - return -ENOMEM; - - pd->info = &sps->info->domains[index]; - pd->sps = sps; - - pd->genpd.name = pd->info->name; - pd->genpd.power_on = owl_sps_power_on; - pd->genpd.power_off = owl_sps_power_off; - pd->genpd.flags = pd->info->genpd_flags; - pm_genpd_init(&pd->genpd, NULL, false); - - sps->genpd_data.domains[index] = &pd->genpd; - - return 0; -} - -static int owl_sps_probe(struct platform_device *pdev) -{ - const struct of_device_id *match; - const struct owl_sps_info *sps_info; - struct owl_sps *sps; - int i, ret; - - if (!pdev->dev.of_node) { - dev_err(&pdev->dev, "no device node\n"); - return -ENODEV; - } - - match = of_match_device(pdev->dev.driver->of_match_table, &pdev->dev); - if (!match || !match->data) { - dev_err(&pdev->dev, "unknown compatible or missing data\n"); - return -EINVAL; - } - - sps_info = match->data; - - sps = devm_kzalloc(&pdev->dev, - struct_size(sps, domains, sps_info->num_domains), - GFP_KERNEL); - if (!sps) - return -ENOMEM; - - sps->base = of_io_request_and_map(pdev->dev.of_node, 0, "owl-sps"); - if (IS_ERR(sps->base)) { - dev_err(&pdev->dev, "failed to map sps registers\n"); - return PTR_ERR(sps->base); - } - - sps->dev = &pdev->dev; - sps->info = sps_info; - sps->genpd_data.domains = sps->domains; - sps->genpd_data.num_domains = sps_info->num_domains; - - for (i = 0; i < sps_info->num_domains; i++) { - ret = owl_sps_init_domain(sps, i); - if (ret) - return ret; - } - - ret = of_genpd_add_provider_onecell(pdev->dev.of_node, &sps->genpd_data); - if (ret) { - dev_err(&pdev->dev, "failed to add provider (%d)", ret); - return ret; - } - - return 0; -} - -static const struct owl_sps_domain_info s500_sps_domains[] = { - [S500_PD_VDE] = { - .name = "VDE", - .pwr_bit = 0, - .ack_bit = 16, - }, - [S500_PD_VCE_SI] = { - .name = "VCE_SI", - .pwr_bit = 1, - .ack_bit = 17, - }, - [S500_PD_USB2_1] = { - .name = "USB2_1", - .pwr_bit = 2, - .ack_bit = 18, - }, - [S500_PD_CPU2] = { - .name = "CPU2", - .pwr_bit = 5, - .ack_bit = 21, - .genpd_flags = GENPD_FLAG_ALWAYS_ON, - }, - [S500_PD_CPU3] = { - .name = "CPU3", - .pwr_bit = 6, - .ack_bit = 22, - .genpd_flags = GENPD_FLAG_ALWAYS_ON, - }, - [S500_PD_DMA] = { - .name = "DMA", - .pwr_bit = 8, - .ack_bit = 12, - }, - [S500_PD_DS] = { - .name = "DS", - .pwr_bit = 9, - .ack_bit = 13, - }, - [S500_PD_USB3] = { - .name = "USB3", - .pwr_bit = 10, - .ack_bit = 14, - }, - [S500_PD_USB2_0] = { - .name = "USB2_0", - .pwr_bit = 11, - .ack_bit = 15, - }, -}; - -static const struct owl_sps_info s500_sps_info = { - .num_domains = ARRAY_SIZE(s500_sps_domains), - .domains = s500_sps_domains, -}; - -static const struct owl_sps_domain_info s700_sps_domains[] = { - [S700_PD_VDE] = { - .name = "VDE", - .pwr_bit = 0, - }, - [S700_PD_VCE_SI] = { - .name = "VCE_SI", - .pwr_bit = 1, - }, - [S700_PD_USB2_1] = { - .name = "USB2_1", - .pwr_bit = 2, - }, - [S700_PD_HDE] = { - .name = "HDE", - .pwr_bit = 7, - }, - [S700_PD_DMA] = { - .name = "DMA", - .pwr_bit = 8, - }, - [S700_PD_DS] = { - .name = "DS", - .pwr_bit = 9, - }, - [S700_PD_USB3] = { - .name = "USB3", - .pwr_bit = 10, - }, - [S700_PD_USB2_0] = { - .name = "USB2_0", - .pwr_bit = 11, - }, -}; - -static const struct owl_sps_info s700_sps_info = { - .num_domains = ARRAY_SIZE(s700_sps_domains), - .domains = s700_sps_domains, -}; - -static const struct owl_sps_domain_info s900_sps_domains[] = { - [S900_PD_GPU_B] = { - .name = "GPU_B", - .pwr_bit = 3, - }, - [S900_PD_VCE] = { - .name = "VCE", - .pwr_bit = 4, - }, - [S900_PD_SENSOR] = { - .name = "SENSOR", - .pwr_bit = 5, - }, - [S900_PD_VDE] = { - .name = "VDE", - .pwr_bit = 6, - }, - [S900_PD_HDE] = { - .name = "HDE", - .pwr_bit = 7, - }, - [S900_PD_USB3] = { - .name = "USB3", - .pwr_bit = 8, - }, - [S900_PD_DDR0] = { - .name = "DDR0", - .pwr_bit = 9, - }, - [S900_PD_DDR1] = { - .name = "DDR1", - .pwr_bit = 10, - }, - [S900_PD_DE] = { - .name = "DE", - .pwr_bit = 13, - }, - [S900_PD_NAND] = { - .name = "NAND", - .pwr_bit = 14, - }, - [S900_PD_USB2_H0] = { - .name = "USB2_H0", - .pwr_bit = 15, - }, - [S900_PD_USB2_H1] = { - .name = "USB2_H1", - .pwr_bit = 16, - }, -}; - -static const struct owl_sps_info s900_sps_info = { - .num_domains = ARRAY_SIZE(s900_sps_domains), - .domains = s900_sps_domains, -}; - -static const struct of_device_id owl_sps_of_matches[] = { - { .compatible = "actions,s500-sps", .data = &s500_sps_info }, - { .compatible = "actions,s700-sps", .data = &s700_sps_info }, - { .compatible = "actions,s900-sps", .data = &s900_sps_info }, - { } -}; - -static struct platform_driver owl_sps_platform_driver = { - .probe = owl_sps_probe, - .driver = { - .name = "owl-sps", - .of_match_table = owl_sps_of_matches, - .suppress_bind_attrs = true, - }, -}; - -static int __init owl_sps_init(void) -{ - return platform_driver_register(&owl_sps_platform_driver); -} -postcore_initcall(owl_sps_init); diff --git a/drivers/genpd/amlogic/Makefile b/drivers/genpd/amlogic/Makefile deleted file mode 100644 index 3d58abd574f9..000000000000 --- a/drivers/genpd/amlogic/Makefile +++ /dev/null @@ -1,4 +0,0 @@ -# SPDX-License-Identifier: GPL-2.0-only -obj-$(CONFIG_MESON_GX_PM_DOMAINS) += meson-gx-pwrc-vpu.o -obj-$(CONFIG_MESON_EE_PM_DOMAINS) += meson-ee-pwrc.o -obj-$(CONFIG_MESON_SECURE_PM_DOMAINS) += meson-secure-pwrc.o diff --git a/drivers/genpd/amlogic/meson-ee-pwrc.c b/drivers/genpd/amlogic/meson-ee-pwrc.c deleted file mode 100644 index cfb796d40d9d..000000000000 --- a/drivers/genpd/amlogic/meson-ee-pwrc.c +++ /dev/null @@ -1,635 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0+ -/* - * Copyright (c) 2019 BayLibre, SAS - * Author: Neil Armstrong - */ - -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -/* AO Offsets */ - -#define GX_AO_RTI_GEN_PWR_SLEEP0 (0x3a << 2) -#define GX_AO_RTI_GEN_PWR_ISO0 (0x3b << 2) - -/* - * Meson8/Meson8b/Meson8m2 only expose the power management registers of the - * AO-bus as syscon. 0x3a from GX translates to 0x02, 0x3b translates to 0x03 - * and so on. - */ -#define MESON8_AO_RTI_GEN_PWR_SLEEP0 (0x02 << 2) -#define MESON8_AO_RTI_GEN_PWR_ISO0 (0x03 << 2) - -/* HHI Offsets */ - -#define HHI_MEM_PD_REG0 (0x40 << 2) -#define HHI_VPU_MEM_PD_REG0 (0x41 << 2) -#define HHI_VPU_MEM_PD_REG1 (0x42 << 2) -#define HHI_VPU_MEM_PD_REG3 (0x43 << 2) -#define HHI_VPU_MEM_PD_REG4 (0x44 << 2) -#define HHI_AUDIO_MEM_PD_REG0 (0x45 << 2) -#define HHI_NANOQ_MEM_PD_REG0 (0x46 << 2) -#define HHI_NANOQ_MEM_PD_REG1 (0x47 << 2) -#define HHI_VPU_MEM_PD_REG2 (0x4d << 2) - -#define G12A_HHI_NANOQ_MEM_PD_REG0 (0x43 << 2) -#define G12A_HHI_NANOQ_MEM_PD_REG1 (0x44 << 2) - -struct meson_ee_pwrc; -struct meson_ee_pwrc_domain; - -struct meson_ee_pwrc_mem_domain { - unsigned int reg; - unsigned int mask; -}; - -struct meson_ee_pwrc_top_domain { - unsigned int sleep_reg; - unsigned int sleep_mask; - unsigned int iso_reg; - unsigned int iso_mask; -}; - -struct meson_ee_pwrc_domain_desc { - char *name; - unsigned int reset_names_count; - unsigned int clk_names_count; - struct meson_ee_pwrc_top_domain *top_pd; - unsigned int mem_pd_count; - struct meson_ee_pwrc_mem_domain *mem_pd; - bool (*is_powered_off)(struct meson_ee_pwrc_domain *pwrc_domain); -}; - -struct meson_ee_pwrc_domain_data { - unsigned int count; - struct meson_ee_pwrc_domain_desc *domains; -}; - -/* TOP Power Domains */ - -static struct meson_ee_pwrc_top_domain gx_pwrc_vpu = { - .sleep_reg = GX_AO_RTI_GEN_PWR_SLEEP0, - .sleep_mask = BIT(8), - .iso_reg = GX_AO_RTI_GEN_PWR_SLEEP0, - .iso_mask = BIT(9), -}; - -static struct meson_ee_pwrc_top_domain meson8_pwrc_vpu = { - .sleep_reg = MESON8_AO_RTI_GEN_PWR_SLEEP0, - .sleep_mask = BIT(8), - .iso_reg = MESON8_AO_RTI_GEN_PWR_SLEEP0, - .iso_mask = BIT(9), -}; - -#define SM1_EE_PD(__bit) \ - { \ - .sleep_reg = GX_AO_RTI_GEN_PWR_SLEEP0, \ - .sleep_mask = BIT(__bit), \ - .iso_reg = GX_AO_RTI_GEN_PWR_ISO0, \ - .iso_mask = BIT(__bit), \ - } - -static struct meson_ee_pwrc_top_domain sm1_pwrc_vpu = SM1_EE_PD(8); -static struct meson_ee_pwrc_top_domain sm1_pwrc_nna = SM1_EE_PD(16); -static struct meson_ee_pwrc_top_domain sm1_pwrc_usb = SM1_EE_PD(17); -static struct meson_ee_pwrc_top_domain sm1_pwrc_pci = SM1_EE_PD(18); -static struct meson_ee_pwrc_top_domain sm1_pwrc_ge2d = SM1_EE_PD(19); - -static struct meson_ee_pwrc_top_domain g12a_pwrc_nna = { - .sleep_reg = GX_AO_RTI_GEN_PWR_SLEEP0, - .sleep_mask = BIT(16) | BIT(17), - .iso_reg = GX_AO_RTI_GEN_PWR_ISO0, - .iso_mask = BIT(16) | BIT(17), -}; - -/* Memory PD Domains */ - -#define VPU_MEMPD(__reg) \ - { __reg, GENMASK(1, 0) }, \ - { __reg, GENMASK(3, 2) }, \ - { __reg, GENMASK(5, 4) }, \ - { __reg, GENMASK(7, 6) }, \ - { __reg, GENMASK(9, 8) }, \ - { __reg, GENMASK(11, 10) }, \ - { __reg, GENMASK(13, 12) }, \ - { __reg, GENMASK(15, 14) }, \ - { __reg, GENMASK(17, 16) }, \ - { __reg, GENMASK(19, 18) }, \ - { __reg, GENMASK(21, 20) }, \ - { __reg, GENMASK(23, 22) }, \ - { __reg, GENMASK(25, 24) }, \ - { __reg, GENMASK(27, 26) }, \ - { __reg, GENMASK(29, 28) }, \ - { __reg, GENMASK(31, 30) } - -#define VPU_HHI_MEMPD(__reg) \ - { __reg, BIT(8) }, \ - { __reg, BIT(9) }, \ - { __reg, BIT(10) }, \ - { __reg, BIT(11) }, \ - { __reg, BIT(12) }, \ - { __reg, BIT(13) }, \ - { __reg, BIT(14) }, \ - { __reg, BIT(15) } - -static struct meson_ee_pwrc_mem_domain axg_pwrc_mem_vpu[] = { - VPU_MEMPD(HHI_VPU_MEM_PD_REG0), - VPU_HHI_MEMPD(HHI_MEM_PD_REG0), -}; - -static struct meson_ee_pwrc_mem_domain g12a_pwrc_mem_vpu[] = { - VPU_MEMPD(HHI_VPU_MEM_PD_REG0), - VPU_MEMPD(HHI_VPU_MEM_PD_REG1), - VPU_MEMPD(HHI_VPU_MEM_PD_REG2), - VPU_HHI_MEMPD(HHI_MEM_PD_REG0), -}; - -static struct meson_ee_pwrc_mem_domain gxbb_pwrc_mem_vpu[] = { - VPU_MEMPD(HHI_VPU_MEM_PD_REG0), - VPU_MEMPD(HHI_VPU_MEM_PD_REG1), - VPU_HHI_MEMPD(HHI_MEM_PD_REG0), -}; - -static struct meson_ee_pwrc_mem_domain meson_pwrc_mem_eth[] = { - { HHI_MEM_PD_REG0, GENMASK(3, 2) }, -}; - -static struct meson_ee_pwrc_mem_domain meson8_pwrc_audio_dsp_mem[] = { - { HHI_MEM_PD_REG0, GENMASK(1, 0) }, -}; - -static struct meson_ee_pwrc_mem_domain meson8_pwrc_mem_vpu[] = { - VPU_MEMPD(HHI_VPU_MEM_PD_REG0), - VPU_MEMPD(HHI_VPU_MEM_PD_REG1), - VPU_HHI_MEMPD(HHI_MEM_PD_REG0), -}; - -static struct meson_ee_pwrc_mem_domain sm1_pwrc_mem_vpu[] = { - VPU_MEMPD(HHI_VPU_MEM_PD_REG0), - VPU_MEMPD(HHI_VPU_MEM_PD_REG1), - VPU_MEMPD(HHI_VPU_MEM_PD_REG2), - VPU_MEMPD(HHI_VPU_MEM_PD_REG3), - { HHI_VPU_MEM_PD_REG4, GENMASK(1, 0) }, - { HHI_VPU_MEM_PD_REG4, GENMASK(3, 2) }, - { HHI_VPU_MEM_PD_REG4, GENMASK(5, 4) }, - { HHI_VPU_MEM_PD_REG4, GENMASK(7, 6) }, - VPU_HHI_MEMPD(HHI_MEM_PD_REG0), -}; - -static struct meson_ee_pwrc_mem_domain sm1_pwrc_mem_nna[] = { - { HHI_NANOQ_MEM_PD_REG0, 0xff }, - { HHI_NANOQ_MEM_PD_REG1, 0xff }, -}; - -static struct meson_ee_pwrc_mem_domain sm1_pwrc_mem_usb[] = { - { HHI_MEM_PD_REG0, GENMASK(31, 30) }, -}; - -static struct meson_ee_pwrc_mem_domain sm1_pwrc_mem_pcie[] = { - { HHI_MEM_PD_REG0, GENMASK(29, 26) }, -}; - -static struct meson_ee_pwrc_mem_domain sm1_pwrc_mem_ge2d[] = { - { HHI_MEM_PD_REG0, GENMASK(25, 18) }, -}; - -static struct meson_ee_pwrc_mem_domain axg_pwrc_mem_audio[] = { - { HHI_MEM_PD_REG0, GENMASK(5, 4) }, -}; - -static struct meson_ee_pwrc_mem_domain sm1_pwrc_mem_audio[] = { - { HHI_MEM_PD_REG0, GENMASK(5, 4) }, - { HHI_AUDIO_MEM_PD_REG0, GENMASK(1, 0) }, - { HHI_AUDIO_MEM_PD_REG0, GENMASK(3, 2) }, - { HHI_AUDIO_MEM_PD_REG0, GENMASK(5, 4) }, - { HHI_AUDIO_MEM_PD_REG0, GENMASK(7, 6) }, - { HHI_AUDIO_MEM_PD_REG0, GENMASK(13, 12) }, - { HHI_AUDIO_MEM_PD_REG0, GENMASK(15, 14) }, - { HHI_AUDIO_MEM_PD_REG0, GENMASK(17, 16) }, - { HHI_AUDIO_MEM_PD_REG0, GENMASK(19, 18) }, - { HHI_AUDIO_MEM_PD_REG0, GENMASK(21, 20) }, - { HHI_AUDIO_MEM_PD_REG0, GENMASK(23, 22) }, - { HHI_AUDIO_MEM_PD_REG0, GENMASK(25, 24) }, - { HHI_AUDIO_MEM_PD_REG0, GENMASK(27, 26) }, -}; - -static struct meson_ee_pwrc_mem_domain g12a_pwrc_mem_nna[] = { - { G12A_HHI_NANOQ_MEM_PD_REG0, GENMASK(31, 0) }, - { G12A_HHI_NANOQ_MEM_PD_REG1, GENMASK(23, 0) }, -}; - -#define VPU_PD(__name, __top_pd, __mem, __is_pwr_off, __resets, __clks) \ - { \ - .name = __name, \ - .reset_names_count = __resets, \ - .clk_names_count = __clks, \ - .top_pd = __top_pd, \ - .mem_pd_count = ARRAY_SIZE(__mem), \ - .mem_pd = __mem, \ - .is_powered_off = __is_pwr_off, \ - } - -#define TOP_PD(__name, __top_pd, __mem, __is_pwr_off) \ - { \ - .name = __name, \ - .top_pd = __top_pd, \ - .mem_pd_count = ARRAY_SIZE(__mem), \ - .mem_pd = __mem, \ - .is_powered_off = __is_pwr_off, \ - } - -#define MEM_PD(__name, __mem) \ - TOP_PD(__name, NULL, __mem, NULL) - -static bool pwrc_ee_is_powered_off(struct meson_ee_pwrc_domain *pwrc_domain); - -static struct meson_ee_pwrc_domain_desc axg_pwrc_domains[] = { - [PWRC_AXG_VPU_ID] = VPU_PD("VPU", &gx_pwrc_vpu, axg_pwrc_mem_vpu, - pwrc_ee_is_powered_off, 5, 2), - [PWRC_AXG_ETHERNET_MEM_ID] = MEM_PD("ETH", meson_pwrc_mem_eth), - [PWRC_AXG_AUDIO_ID] = MEM_PD("AUDIO", axg_pwrc_mem_audio), -}; - -static struct meson_ee_pwrc_domain_desc g12a_pwrc_domains[] = { - [PWRC_G12A_VPU_ID] = VPU_PD("VPU", &gx_pwrc_vpu, g12a_pwrc_mem_vpu, - pwrc_ee_is_powered_off, 11, 2), - [PWRC_G12A_ETH_ID] = MEM_PD("ETH", meson_pwrc_mem_eth), - [PWRC_G12A_NNA_ID] = TOP_PD("NNA", &g12a_pwrc_nna, g12a_pwrc_mem_nna, - pwrc_ee_is_powered_off), -}; - -static struct meson_ee_pwrc_domain_desc gxbb_pwrc_domains[] = { - [PWRC_GXBB_VPU_ID] = VPU_PD("VPU", &gx_pwrc_vpu, gxbb_pwrc_mem_vpu, - pwrc_ee_is_powered_off, 12, 2), - [PWRC_GXBB_ETHERNET_MEM_ID] = MEM_PD("ETH", meson_pwrc_mem_eth), -}; - -static struct meson_ee_pwrc_domain_desc meson8_pwrc_domains[] = { - [PWRC_MESON8_VPU_ID] = VPU_PD("VPU", &meson8_pwrc_vpu, - meson8_pwrc_mem_vpu, - pwrc_ee_is_powered_off, 0, 1), - [PWRC_MESON8_ETHERNET_MEM_ID] = MEM_PD("ETHERNET_MEM", - meson_pwrc_mem_eth), - [PWRC_MESON8_AUDIO_DSP_MEM_ID] = MEM_PD("AUDIO_DSP_MEM", - meson8_pwrc_audio_dsp_mem), -}; - -static struct meson_ee_pwrc_domain_desc meson8b_pwrc_domains[] = { - [PWRC_MESON8_VPU_ID] = VPU_PD("VPU", &meson8_pwrc_vpu, - meson8_pwrc_mem_vpu, - pwrc_ee_is_powered_off, 11, 1), - [PWRC_MESON8_ETHERNET_MEM_ID] = MEM_PD("ETHERNET_MEM", - meson_pwrc_mem_eth), - [PWRC_MESON8_AUDIO_DSP_MEM_ID] = MEM_PD("AUDIO_DSP_MEM", - meson8_pwrc_audio_dsp_mem), -}; - -static struct meson_ee_pwrc_domain_desc sm1_pwrc_domains[] = { - [PWRC_SM1_VPU_ID] = VPU_PD("VPU", &sm1_pwrc_vpu, sm1_pwrc_mem_vpu, - pwrc_ee_is_powered_off, 11, 2), - [PWRC_SM1_NNA_ID] = TOP_PD("NNA", &sm1_pwrc_nna, sm1_pwrc_mem_nna, - pwrc_ee_is_powered_off), - [PWRC_SM1_USB_ID] = TOP_PD("USB", &sm1_pwrc_usb, sm1_pwrc_mem_usb, - pwrc_ee_is_powered_off), - [PWRC_SM1_PCIE_ID] = TOP_PD("PCI", &sm1_pwrc_pci, sm1_pwrc_mem_pcie, - pwrc_ee_is_powered_off), - [PWRC_SM1_GE2D_ID] = TOP_PD("GE2D", &sm1_pwrc_ge2d, sm1_pwrc_mem_ge2d, - pwrc_ee_is_powered_off), - [PWRC_SM1_AUDIO_ID] = MEM_PD("AUDIO", sm1_pwrc_mem_audio), - [PWRC_SM1_ETH_ID] = MEM_PD("ETH", meson_pwrc_mem_eth), -}; - -struct meson_ee_pwrc_domain { - struct generic_pm_domain base; - bool enabled; - struct meson_ee_pwrc *pwrc; - struct meson_ee_pwrc_domain_desc desc; - struct clk_bulk_data *clks; - int num_clks; - struct reset_control *rstc; - int num_rstc; -}; - -struct meson_ee_pwrc { - struct regmap *regmap_ao; - struct regmap *regmap_hhi; - struct meson_ee_pwrc_domain *domains; - struct genpd_onecell_data xlate; -}; - -static bool pwrc_ee_is_powered_off(struct meson_ee_pwrc_domain *pwrc_domain) -{ - u32 reg; - - regmap_read(pwrc_domain->pwrc->regmap_ao, - pwrc_domain->desc.top_pd->sleep_reg, ®); - - return (reg & pwrc_domain->desc.top_pd->sleep_mask); -} - -static int meson_ee_pwrc_off(struct generic_pm_domain *domain) -{ - struct meson_ee_pwrc_domain *pwrc_domain = - container_of(domain, struct meson_ee_pwrc_domain, base); - int i; - - if (pwrc_domain->desc.top_pd) - regmap_update_bits(pwrc_domain->pwrc->regmap_ao, - pwrc_domain->desc.top_pd->sleep_reg, - pwrc_domain->desc.top_pd->sleep_mask, - pwrc_domain->desc.top_pd->sleep_mask); - udelay(20); - - for (i = 0 ; i < pwrc_domain->desc.mem_pd_count ; ++i) - regmap_update_bits(pwrc_domain->pwrc->regmap_hhi, - pwrc_domain->desc.mem_pd[i].reg, - pwrc_domain->desc.mem_pd[i].mask, - pwrc_domain->desc.mem_pd[i].mask); - - udelay(20); - - if (pwrc_domain->desc.top_pd) - regmap_update_bits(pwrc_domain->pwrc->regmap_ao, - pwrc_domain->desc.top_pd->iso_reg, - pwrc_domain->desc.top_pd->iso_mask, - pwrc_domain->desc.top_pd->iso_mask); - - if (pwrc_domain->num_clks) { - msleep(20); - clk_bulk_disable_unprepare(pwrc_domain->num_clks, - pwrc_domain->clks); - } - - return 0; -} - -static int meson_ee_pwrc_on(struct generic_pm_domain *domain) -{ - struct meson_ee_pwrc_domain *pwrc_domain = - container_of(domain, struct meson_ee_pwrc_domain, base); - int i, ret; - - if (pwrc_domain->desc.top_pd) - regmap_update_bits(pwrc_domain->pwrc->regmap_ao, - pwrc_domain->desc.top_pd->sleep_reg, - pwrc_domain->desc.top_pd->sleep_mask, 0); - udelay(20); - - for (i = 0 ; i < pwrc_domain->desc.mem_pd_count ; ++i) - regmap_update_bits(pwrc_domain->pwrc->regmap_hhi, - pwrc_domain->desc.mem_pd[i].reg, - pwrc_domain->desc.mem_pd[i].mask, 0); - - udelay(20); - - ret = reset_control_assert(pwrc_domain->rstc); - if (ret) - return ret; - - if (pwrc_domain->desc.top_pd) - regmap_update_bits(pwrc_domain->pwrc->regmap_ao, - pwrc_domain->desc.top_pd->iso_reg, - pwrc_domain->desc.top_pd->iso_mask, 0); - - ret = reset_control_deassert(pwrc_domain->rstc); - if (ret) - return ret; - - return clk_bulk_prepare_enable(pwrc_domain->num_clks, - pwrc_domain->clks); -} - -static int meson_ee_pwrc_init_domain(struct platform_device *pdev, - struct meson_ee_pwrc *pwrc, - struct meson_ee_pwrc_domain *dom) -{ - int ret; - - dom->pwrc = pwrc; - dom->num_rstc = dom->desc.reset_names_count; - dom->num_clks = dom->desc.clk_names_count; - - if (dom->num_rstc) { - int count = reset_control_get_count(&pdev->dev); - - if (count != dom->num_rstc) - dev_warn(&pdev->dev, "Invalid resets count %d for domain %s\n", - count, dom->desc.name); - - dom->rstc = devm_reset_control_array_get_exclusive(&pdev->dev); - if (IS_ERR(dom->rstc)) - return PTR_ERR(dom->rstc); - } - - if (dom->num_clks) { - int ret = devm_clk_bulk_get_all(&pdev->dev, &dom->clks); - if (ret < 0) - return ret; - - if (dom->num_clks != ret) { - dev_warn(&pdev->dev, "Invalid clocks count %d for domain %s\n", - ret, dom->desc.name); - dom->num_clks = ret; - } - } - - dom->base.name = dom->desc.name; - dom->base.power_on = meson_ee_pwrc_on; - dom->base.power_off = meson_ee_pwrc_off; - - /* - * TOFIX: This is a special case for the VPU power domain, which can - * be enabled previously by the bootloader. In this case the VPU - * pipeline may be functional but no driver maybe never attach - * to this power domain, and if the domain is disabled it could - * cause system errors. This is why the pm_domain_always_on_gov - * is used here. - * For the same reason, the clocks should be enabled in case - * we need to power the domain off, otherwise the internal clocks - * prepare/enable counters won't be in sync. - */ - if (dom->num_clks && dom->desc.is_powered_off && !dom->desc.is_powered_off(dom)) { - ret = clk_bulk_prepare_enable(dom->num_clks, dom->clks); - if (ret) - return ret; - - dom->base.flags = GENPD_FLAG_ALWAYS_ON; - ret = pm_genpd_init(&dom->base, NULL, false); - if (ret) - return ret; - } else { - ret = pm_genpd_init(&dom->base, NULL, - (dom->desc.is_powered_off ? - dom->desc.is_powered_off(dom) : true)); - if (ret) - return ret; - } - - return 0; -} - -static int meson_ee_pwrc_probe(struct platform_device *pdev) -{ - const struct meson_ee_pwrc_domain_data *match; - struct regmap *regmap_ao, *regmap_hhi; - struct device_node *parent_np; - struct meson_ee_pwrc *pwrc; - int i, ret; - - match = of_device_get_match_data(&pdev->dev); - if (!match) { - dev_err(&pdev->dev, "failed to get match data\n"); - return -ENODEV; - } - - pwrc = devm_kzalloc(&pdev->dev, sizeof(*pwrc), GFP_KERNEL); - if (!pwrc) - return -ENOMEM; - - pwrc->xlate.domains = devm_kcalloc(&pdev->dev, match->count, - sizeof(*pwrc->xlate.domains), - GFP_KERNEL); - if (!pwrc->xlate.domains) - return -ENOMEM; - - pwrc->domains = devm_kcalloc(&pdev->dev, match->count, - sizeof(*pwrc->domains), GFP_KERNEL); - if (!pwrc->domains) - return -ENOMEM; - - pwrc->xlate.num_domains = match->count; - - parent_np = of_get_parent(pdev->dev.of_node); - regmap_hhi = syscon_node_to_regmap(parent_np); - of_node_put(parent_np); - if (IS_ERR(regmap_hhi)) { - dev_err(&pdev->dev, "failed to get HHI regmap\n"); - return PTR_ERR(regmap_hhi); - } - - regmap_ao = syscon_regmap_lookup_by_phandle(pdev->dev.of_node, - "amlogic,ao-sysctrl"); - if (IS_ERR(regmap_ao)) { - dev_err(&pdev->dev, "failed to get AO regmap\n"); - return PTR_ERR(regmap_ao); - } - - pwrc->regmap_ao = regmap_ao; - pwrc->regmap_hhi = regmap_hhi; - - platform_set_drvdata(pdev, pwrc); - - for (i = 0 ; i < match->count ; ++i) { - struct meson_ee_pwrc_domain *dom = &pwrc->domains[i]; - - memcpy(&dom->desc, &match->domains[i], sizeof(dom->desc)); - - ret = meson_ee_pwrc_init_domain(pdev, pwrc, dom); - if (ret) - return ret; - - pwrc->xlate.domains[i] = &dom->base; - } - - return of_genpd_add_provider_onecell(pdev->dev.of_node, &pwrc->xlate); -} - -static void meson_ee_pwrc_shutdown(struct platform_device *pdev) -{ - struct meson_ee_pwrc *pwrc = platform_get_drvdata(pdev); - int i; - - for (i = 0 ; i < pwrc->xlate.num_domains ; ++i) { - struct meson_ee_pwrc_domain *dom = &pwrc->domains[i]; - - if (dom->desc.is_powered_off && !dom->desc.is_powered_off(dom)) - meson_ee_pwrc_off(&dom->base); - } -} - -static struct meson_ee_pwrc_domain_data meson_ee_g12a_pwrc_data = { - .count = ARRAY_SIZE(g12a_pwrc_domains), - .domains = g12a_pwrc_domains, -}; - -static struct meson_ee_pwrc_domain_data meson_ee_axg_pwrc_data = { - .count = ARRAY_SIZE(axg_pwrc_domains), - .domains = axg_pwrc_domains, -}; - -static struct meson_ee_pwrc_domain_data meson_ee_gxbb_pwrc_data = { - .count = ARRAY_SIZE(gxbb_pwrc_domains), - .domains = gxbb_pwrc_domains, -}; - -static struct meson_ee_pwrc_domain_data meson_ee_m8_pwrc_data = { - .count = ARRAY_SIZE(meson8_pwrc_domains), - .domains = meson8_pwrc_domains, -}; - -static struct meson_ee_pwrc_domain_data meson_ee_m8b_pwrc_data = { - .count = ARRAY_SIZE(meson8b_pwrc_domains), - .domains = meson8b_pwrc_domains, -}; - -static struct meson_ee_pwrc_domain_data meson_ee_sm1_pwrc_data = { - .count = ARRAY_SIZE(sm1_pwrc_domains), - .domains = sm1_pwrc_domains, -}; - -static const struct of_device_id meson_ee_pwrc_match_table[] = { - { - .compatible = "amlogic,meson8-pwrc", - .data = &meson_ee_m8_pwrc_data, - }, - { - .compatible = "amlogic,meson8b-pwrc", - .data = &meson_ee_m8b_pwrc_data, - }, - { - .compatible = "amlogic,meson8m2-pwrc", - .data = &meson_ee_m8b_pwrc_data, - }, - { - .compatible = "amlogic,meson-axg-pwrc", - .data = &meson_ee_axg_pwrc_data, - }, - { - .compatible = "amlogic,meson-gxbb-pwrc", - .data = &meson_ee_gxbb_pwrc_data, - }, - { - .compatible = "amlogic,meson-g12a-pwrc", - .data = &meson_ee_g12a_pwrc_data, - }, - { - .compatible = "amlogic,meson-sm1-pwrc", - .data = &meson_ee_sm1_pwrc_data, - }, - { /* sentinel */ } -}; -MODULE_DEVICE_TABLE(of, meson_ee_pwrc_match_table); - -static struct platform_driver meson_ee_pwrc_driver = { - .probe = meson_ee_pwrc_probe, - .shutdown = meson_ee_pwrc_shutdown, - .driver = { - .name = "meson_ee_pwrc", - .of_match_table = meson_ee_pwrc_match_table, - }, -}; -module_platform_driver(meson_ee_pwrc_driver); -MODULE_LICENSE("GPL v2"); diff --git a/drivers/genpd/amlogic/meson-gx-pwrc-vpu.c b/drivers/genpd/amlogic/meson-gx-pwrc-vpu.c deleted file mode 100644 index 33df520eab95..000000000000 --- a/drivers/genpd/amlogic/meson-gx-pwrc-vpu.c +++ /dev/null @@ -1,379 +0,0 @@ -/* - * Copyright (c) 2017 BayLibre, SAS - * Author: Neil Armstrong - * - * SPDX-License-Identifier: GPL-2.0+ - */ - -#include -#include -#include -#include -#include -#include -#include -#include -#include - -/* AO Offsets */ - -#define AO_RTI_GEN_PWR_SLEEP0 (0x3a << 2) - -#define GEN_PWR_VPU_HDMI BIT(8) -#define GEN_PWR_VPU_HDMI_ISO BIT(9) - -/* HHI Offsets */ - -#define HHI_MEM_PD_REG0 (0x40 << 2) -#define HHI_VPU_MEM_PD_REG0 (0x41 << 2) -#define HHI_VPU_MEM_PD_REG1 (0x42 << 2) -#define HHI_VPU_MEM_PD_REG2 (0x4d << 2) - -struct meson_gx_pwrc_vpu { - struct generic_pm_domain genpd; - struct regmap *regmap_ao; - struct regmap *regmap_hhi; - struct reset_control *rstc; - struct clk *vpu_clk; - struct clk *vapb_clk; -}; - -static inline -struct meson_gx_pwrc_vpu *genpd_to_pd(struct generic_pm_domain *d) -{ - return container_of(d, struct meson_gx_pwrc_vpu, genpd); -} - -static int meson_gx_pwrc_vpu_power_off(struct generic_pm_domain *genpd) -{ - struct meson_gx_pwrc_vpu *pd = genpd_to_pd(genpd); - int i; - - regmap_update_bits(pd->regmap_ao, AO_RTI_GEN_PWR_SLEEP0, - GEN_PWR_VPU_HDMI_ISO, GEN_PWR_VPU_HDMI_ISO); - udelay(20); - - /* Power Down Memories */ - for (i = 0; i < 32; i += 2) { - regmap_update_bits(pd->regmap_hhi, HHI_VPU_MEM_PD_REG0, - 0x3 << i, 0x3 << i); - udelay(5); - } - for (i = 0; i < 32; i += 2) { - regmap_update_bits(pd->regmap_hhi, HHI_VPU_MEM_PD_REG1, - 0x3 << i, 0x3 << i); - udelay(5); - } - for (i = 8; i < 16; i++) { - regmap_update_bits(pd->regmap_hhi, HHI_MEM_PD_REG0, - BIT(i), BIT(i)); - udelay(5); - } - udelay(20); - - regmap_update_bits(pd->regmap_ao, AO_RTI_GEN_PWR_SLEEP0, - GEN_PWR_VPU_HDMI, GEN_PWR_VPU_HDMI); - - msleep(20); - - clk_disable_unprepare(pd->vpu_clk); - clk_disable_unprepare(pd->vapb_clk); - - return 0; -} - -static int meson_g12a_pwrc_vpu_power_off(struct generic_pm_domain *genpd) -{ - struct meson_gx_pwrc_vpu *pd = genpd_to_pd(genpd); - int i; - - regmap_update_bits(pd->regmap_ao, AO_RTI_GEN_PWR_SLEEP0, - GEN_PWR_VPU_HDMI_ISO, GEN_PWR_VPU_HDMI_ISO); - udelay(20); - - /* Power Down Memories */ - for (i = 0; i < 32; i += 2) { - regmap_update_bits(pd->regmap_hhi, HHI_VPU_MEM_PD_REG0, - 0x3 << i, 0x3 << i); - udelay(5); - } - for (i = 0; i < 32; i += 2) { - regmap_update_bits(pd->regmap_hhi, HHI_VPU_MEM_PD_REG1, - 0x3 << i, 0x3 << i); - udelay(5); - } - for (i = 0; i < 32; i += 2) { - regmap_update_bits(pd->regmap_hhi, HHI_VPU_MEM_PD_REG2, - 0x3 << i, 0x3 << i); - udelay(5); - } - for (i = 8; i < 16; i++) { - regmap_update_bits(pd->regmap_hhi, HHI_MEM_PD_REG0, - BIT(i), BIT(i)); - udelay(5); - } - udelay(20); - - regmap_update_bits(pd->regmap_ao, AO_RTI_GEN_PWR_SLEEP0, - GEN_PWR_VPU_HDMI, GEN_PWR_VPU_HDMI); - - msleep(20); - - clk_disable_unprepare(pd->vpu_clk); - clk_disable_unprepare(pd->vapb_clk); - - return 0; -} - -static int meson_gx_pwrc_vpu_setup_clk(struct meson_gx_pwrc_vpu *pd) -{ - int ret; - - ret = clk_prepare_enable(pd->vpu_clk); - if (ret) - return ret; - - ret = clk_prepare_enable(pd->vapb_clk); - if (ret) - clk_disable_unprepare(pd->vpu_clk); - - return ret; -} - -static int meson_gx_pwrc_vpu_power_on(struct generic_pm_domain *genpd) -{ - struct meson_gx_pwrc_vpu *pd = genpd_to_pd(genpd); - int ret; - int i; - - regmap_update_bits(pd->regmap_ao, AO_RTI_GEN_PWR_SLEEP0, - GEN_PWR_VPU_HDMI, 0); - udelay(20); - - /* Power Up Memories */ - for (i = 0; i < 32; i += 2) { - regmap_update_bits(pd->regmap_hhi, HHI_VPU_MEM_PD_REG0, - 0x3 << i, 0); - udelay(5); - } - - for (i = 0; i < 32; i += 2) { - regmap_update_bits(pd->regmap_hhi, HHI_VPU_MEM_PD_REG1, - 0x3 << i, 0); - udelay(5); - } - - for (i = 8; i < 16; i++) { - regmap_update_bits(pd->regmap_hhi, HHI_MEM_PD_REG0, - BIT(i), 0); - udelay(5); - } - udelay(20); - - ret = reset_control_assert(pd->rstc); - if (ret) - return ret; - - regmap_update_bits(pd->regmap_ao, AO_RTI_GEN_PWR_SLEEP0, - GEN_PWR_VPU_HDMI_ISO, 0); - - ret = reset_control_deassert(pd->rstc); - if (ret) - return ret; - - ret = meson_gx_pwrc_vpu_setup_clk(pd); - if (ret) - return ret; - - return 0; -} - -static int meson_g12a_pwrc_vpu_power_on(struct generic_pm_domain *genpd) -{ - struct meson_gx_pwrc_vpu *pd = genpd_to_pd(genpd); - int ret; - int i; - - regmap_update_bits(pd->regmap_ao, AO_RTI_GEN_PWR_SLEEP0, - GEN_PWR_VPU_HDMI, 0); - udelay(20); - - /* Power Up Memories */ - for (i = 0; i < 32; i += 2) { - regmap_update_bits(pd->regmap_hhi, HHI_VPU_MEM_PD_REG0, - 0x3 << i, 0); - udelay(5); - } - - for (i = 0; i < 32; i += 2) { - regmap_update_bits(pd->regmap_hhi, HHI_VPU_MEM_PD_REG1, - 0x3 << i, 0); - udelay(5); - } - - for (i = 0; i < 32; i += 2) { - regmap_update_bits(pd->regmap_hhi, HHI_VPU_MEM_PD_REG2, - 0x3 << i, 0); - udelay(5); - } - - for (i = 8; i < 16; i++) { - regmap_update_bits(pd->regmap_hhi, HHI_MEM_PD_REG0, - BIT(i), 0); - udelay(5); - } - udelay(20); - - ret = reset_control_assert(pd->rstc); - if (ret) - return ret; - - regmap_update_bits(pd->regmap_ao, AO_RTI_GEN_PWR_SLEEP0, - GEN_PWR_VPU_HDMI_ISO, 0); - - ret = reset_control_deassert(pd->rstc); - if (ret) - return ret; - - ret = meson_gx_pwrc_vpu_setup_clk(pd); - if (ret) - return ret; - - return 0; -} - -static bool meson_gx_pwrc_vpu_get_power(struct meson_gx_pwrc_vpu *pd) -{ - u32 reg; - - regmap_read(pd->regmap_ao, AO_RTI_GEN_PWR_SLEEP0, ®); - - return (reg & GEN_PWR_VPU_HDMI); -} - -static struct meson_gx_pwrc_vpu vpu_hdmi_pd = { - .genpd = { - .name = "vpu_hdmi", - .power_off = meson_gx_pwrc_vpu_power_off, - .power_on = meson_gx_pwrc_vpu_power_on, - }, -}; - -static struct meson_gx_pwrc_vpu vpu_hdmi_pd_g12a = { - .genpd = { - .name = "vpu_hdmi", - .power_off = meson_g12a_pwrc_vpu_power_off, - .power_on = meson_g12a_pwrc_vpu_power_on, - }, -}; - -static int meson_gx_pwrc_vpu_probe(struct platform_device *pdev) -{ - const struct meson_gx_pwrc_vpu *vpu_pd_match; - struct regmap *regmap_ao, *regmap_hhi; - struct meson_gx_pwrc_vpu *vpu_pd; - struct device_node *parent_np; - struct reset_control *rstc; - struct clk *vpu_clk; - struct clk *vapb_clk; - bool powered_off; - int ret; - - vpu_pd_match = of_device_get_match_data(&pdev->dev); - if (!vpu_pd_match) { - dev_err(&pdev->dev, "failed to get match data\n"); - return -ENODEV; - } - - vpu_pd = devm_kzalloc(&pdev->dev, sizeof(*vpu_pd), GFP_KERNEL); - if (!vpu_pd) - return -ENOMEM; - - memcpy(vpu_pd, vpu_pd_match, sizeof(*vpu_pd)); - - parent_np = of_get_parent(pdev->dev.of_node); - regmap_ao = syscon_node_to_regmap(parent_np); - of_node_put(parent_np); - if (IS_ERR(regmap_ao)) { - dev_err(&pdev->dev, "failed to get regmap\n"); - return PTR_ERR(regmap_ao); - } - - regmap_hhi = syscon_regmap_lookup_by_phandle(pdev->dev.of_node, - "amlogic,hhi-sysctrl"); - if (IS_ERR(regmap_hhi)) { - dev_err(&pdev->dev, "failed to get HHI regmap\n"); - return PTR_ERR(regmap_hhi); - } - - rstc = devm_reset_control_array_get_exclusive(&pdev->dev); - if (IS_ERR(rstc)) - return dev_err_probe(&pdev->dev, PTR_ERR(rstc), - "failed to get reset lines\n"); - - vpu_clk = devm_clk_get(&pdev->dev, "vpu"); - if (IS_ERR(vpu_clk)) { - dev_err(&pdev->dev, "vpu clock request failed\n"); - return PTR_ERR(vpu_clk); - } - - vapb_clk = devm_clk_get(&pdev->dev, "vapb"); - if (IS_ERR(vapb_clk)) { - dev_err(&pdev->dev, "vapb clock request failed\n"); - return PTR_ERR(vapb_clk); - } - - vpu_pd->regmap_ao = regmap_ao; - vpu_pd->regmap_hhi = regmap_hhi; - vpu_pd->rstc = rstc; - vpu_pd->vpu_clk = vpu_clk; - vpu_pd->vapb_clk = vapb_clk; - - platform_set_drvdata(pdev, vpu_pd); - - powered_off = meson_gx_pwrc_vpu_get_power(vpu_pd); - - /* If already powered, sync the clock states */ - if (!powered_off) { - ret = meson_gx_pwrc_vpu_setup_clk(vpu_pd); - if (ret) - return ret; - } - - vpu_pd->genpd.flags = GENPD_FLAG_ALWAYS_ON; - pm_genpd_init(&vpu_pd->genpd, NULL, powered_off); - - return of_genpd_add_provider_simple(pdev->dev.of_node, - &vpu_pd->genpd); -} - -static void meson_gx_pwrc_vpu_shutdown(struct platform_device *pdev) -{ - struct meson_gx_pwrc_vpu *vpu_pd = platform_get_drvdata(pdev); - bool powered_off; - - powered_off = meson_gx_pwrc_vpu_get_power(vpu_pd); - if (!powered_off) - vpu_pd->genpd.power_off(&vpu_pd->genpd); -} - -static const struct of_device_id meson_gx_pwrc_vpu_match_table[] = { - { .compatible = "amlogic,meson-gx-pwrc-vpu", .data = &vpu_hdmi_pd }, - { - .compatible = "amlogic,meson-g12a-pwrc-vpu", - .data = &vpu_hdmi_pd_g12a - }, - { /* sentinel */ } -}; -MODULE_DEVICE_TABLE(of, meson_gx_pwrc_vpu_match_table); - -static struct platform_driver meson_gx_pwrc_vpu_driver = { - .probe = meson_gx_pwrc_vpu_probe, - .shutdown = meson_gx_pwrc_vpu_shutdown, - .driver = { - .name = "meson_gx_pwrc_vpu", - .of_match_table = meson_gx_pwrc_vpu_match_table, - }, -}; -module_platform_driver(meson_gx_pwrc_vpu_driver); -MODULE_LICENSE("GPL v2"); diff --git a/drivers/genpd/amlogic/meson-secure-pwrc.c b/drivers/genpd/amlogic/meson-secure-pwrc.c deleted file mode 100644 index 89c881c56cd7..000000000000 --- a/drivers/genpd/amlogic/meson-secure-pwrc.c +++ /dev/null @@ -1,257 +0,0 @@ -// SPDX-License-Identifier: (GPL-2.0+ OR MIT) -/* - * Copyright (c) 2019 Amlogic, Inc. - * Author: Jianxin Pan - */ - -#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt - -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -#define PWRC_ON 1 -#define PWRC_OFF 0 - -struct meson_secure_pwrc_domain { - struct generic_pm_domain base; - unsigned int index; - struct meson_secure_pwrc *pwrc; -}; - -struct meson_secure_pwrc { - struct meson_secure_pwrc_domain *domains; - struct genpd_onecell_data xlate; - struct meson_sm_firmware *fw; -}; - -struct meson_secure_pwrc_domain_desc { - unsigned int index; - unsigned int flags; - char *name; - bool (*is_off)(struct meson_secure_pwrc_domain *pwrc_domain); -}; - -struct meson_secure_pwrc_domain_data { - unsigned int count; - struct meson_secure_pwrc_domain_desc *domains; -}; - -static bool pwrc_secure_is_off(struct meson_secure_pwrc_domain *pwrc_domain) -{ - int is_off = 1; - - if (meson_sm_call(pwrc_domain->pwrc->fw, SM_A1_PWRC_GET, &is_off, - pwrc_domain->index, 0, 0, 0, 0) < 0) - pr_err("failed to get power domain status\n"); - - return is_off; -} - -static int meson_secure_pwrc_off(struct generic_pm_domain *domain) -{ - int ret = 0; - struct meson_secure_pwrc_domain *pwrc_domain = - container_of(domain, struct meson_secure_pwrc_domain, base); - - if (meson_sm_call(pwrc_domain->pwrc->fw, SM_A1_PWRC_SET, NULL, - pwrc_domain->index, PWRC_OFF, 0, 0, 0) < 0) { - pr_err("failed to set power domain off\n"); - ret = -EINVAL; - } - - return ret; -} - -static int meson_secure_pwrc_on(struct generic_pm_domain *domain) -{ - int ret = 0; - struct meson_secure_pwrc_domain *pwrc_domain = - container_of(domain, struct meson_secure_pwrc_domain, base); - - if (meson_sm_call(pwrc_domain->pwrc->fw, SM_A1_PWRC_SET, NULL, - pwrc_domain->index, PWRC_ON, 0, 0, 0) < 0) { - pr_err("failed to set power domain on\n"); - ret = -EINVAL; - } - - return ret; -} - -#define SEC_PD(__name, __flag) \ -[PWRC_##__name##_ID] = \ -{ \ - .name = #__name, \ - .index = PWRC_##__name##_ID, \ - .is_off = pwrc_secure_is_off, \ - .flags = __flag, \ -} - -static struct meson_secure_pwrc_domain_desc a1_pwrc_domains[] = { - SEC_PD(DSPA, 0), - SEC_PD(DSPB, 0), - /* UART should keep working in ATF after suspend and before resume */ - SEC_PD(UART, GENPD_FLAG_ALWAYS_ON), - /* DMC is for DDR PHY ana/dig and DMC, and should be always on */ - SEC_PD(DMC, GENPD_FLAG_ALWAYS_ON), - SEC_PD(I2C, 0), - SEC_PD(PSRAM, 0), - SEC_PD(ACODEC, 0), - SEC_PD(AUDIO, 0), - SEC_PD(OTP, 0), - SEC_PD(DMA, GENPD_FLAG_ALWAYS_ON | GENPD_FLAG_IRQ_SAFE), - SEC_PD(SD_EMMC, 0), - SEC_PD(RAMA, 0), - /* SRAMB is used as ATF runtime memory, and should be always on */ - SEC_PD(RAMB, GENPD_FLAG_ALWAYS_ON), - SEC_PD(IR, 0), - SEC_PD(SPICC, 0), - SEC_PD(SPIFC, 0), - SEC_PD(USB, 0), - /* NIC is for the Arm NIC-400 interconnect, and should be always on */ - SEC_PD(NIC, GENPD_FLAG_ALWAYS_ON), - SEC_PD(PDMIN, 0), - SEC_PD(RSA, 0), -}; - -static struct meson_secure_pwrc_domain_desc c3_pwrc_domains[] = { - SEC_PD(C3_NNA, 0), - SEC_PD(C3_AUDIO, GENPD_FLAG_ALWAYS_ON), - SEC_PD(C3_SDIOA, GENPD_FLAG_ALWAYS_ON), - SEC_PD(C3_EMMC, GENPD_FLAG_ALWAYS_ON), - SEC_PD(C3_USB_COMB, GENPD_FLAG_ALWAYS_ON), - SEC_PD(C3_SDCARD, GENPD_FLAG_ALWAYS_ON), - SEC_PD(C3_ETH, GENPD_FLAG_ALWAYS_ON), - SEC_PD(C3_GE2D, GENPD_FLAG_ALWAYS_ON), - SEC_PD(C3_CVE, GENPD_FLAG_ALWAYS_ON), - SEC_PD(C3_GDC_WRAP, GENPD_FLAG_ALWAYS_ON), - SEC_PD(C3_ISP_TOP, GENPD_FLAG_ALWAYS_ON), - SEC_PD(C3_MIPI_ISP_WRAP, GENPD_FLAG_ALWAYS_ON), - SEC_PD(C3_VCODEC, 0), -}; - -static struct meson_secure_pwrc_domain_desc s4_pwrc_domains[] = { - SEC_PD(S4_DOS_HEVC, 0), - SEC_PD(S4_DOS_VDEC, 0), - SEC_PD(S4_VPU_HDMI, 0), - SEC_PD(S4_USB_COMB, 0), - SEC_PD(S4_GE2D, 0), - /* ETH is for ethernet online wakeup, and should be always on */ - SEC_PD(S4_ETH, GENPD_FLAG_ALWAYS_ON), - SEC_PD(S4_DEMOD, 0), - SEC_PD(S4_AUDIO, 0), -}; - -static int meson_secure_pwrc_probe(struct platform_device *pdev) -{ - int i; - struct device_node *sm_np; - struct meson_secure_pwrc *pwrc; - const struct meson_secure_pwrc_domain_data *match; - - match = of_device_get_match_data(&pdev->dev); - if (!match) { - dev_err(&pdev->dev, "failed to get match data\n"); - return -ENODEV; - } - - sm_np = of_find_compatible_node(NULL, NULL, "amlogic,meson-gxbb-sm"); - if (!sm_np) { - dev_err(&pdev->dev, "no secure-monitor node\n"); - return -ENODEV; - } - - pwrc = devm_kzalloc(&pdev->dev, sizeof(*pwrc), GFP_KERNEL); - if (!pwrc) { - of_node_put(sm_np); - return -ENOMEM; - } - - pwrc->fw = meson_sm_get(sm_np); - of_node_put(sm_np); - if (!pwrc->fw) - return -EPROBE_DEFER; - - pwrc->xlate.domains = devm_kcalloc(&pdev->dev, match->count, - sizeof(*pwrc->xlate.domains), - GFP_KERNEL); - if (!pwrc->xlate.domains) - return -ENOMEM; - - pwrc->domains = devm_kcalloc(&pdev->dev, match->count, - sizeof(*pwrc->domains), GFP_KERNEL); - if (!pwrc->domains) - return -ENOMEM; - - pwrc->xlate.num_domains = match->count; - platform_set_drvdata(pdev, pwrc); - - for (i = 0 ; i < match->count ; ++i) { - struct meson_secure_pwrc_domain *dom = &pwrc->domains[i]; - - if (!match->domains[i].name) - continue; - - dom->pwrc = pwrc; - dom->index = match->domains[i].index; - dom->base.name = match->domains[i].name; - dom->base.flags = match->domains[i].flags; - dom->base.power_on = meson_secure_pwrc_on; - dom->base.power_off = meson_secure_pwrc_off; - - pm_genpd_init(&dom->base, NULL, match->domains[i].is_off(dom)); - - pwrc->xlate.domains[i] = &dom->base; - } - - return of_genpd_add_provider_onecell(pdev->dev.of_node, &pwrc->xlate); -} - -static struct meson_secure_pwrc_domain_data meson_secure_a1_pwrc_data = { - .domains = a1_pwrc_domains, - .count = ARRAY_SIZE(a1_pwrc_domains), -}; - -static struct meson_secure_pwrc_domain_data amlogic_secure_c3_pwrc_data = { - .domains = c3_pwrc_domains, - .count = ARRAY_SIZE(c3_pwrc_domains), -}; - -static struct meson_secure_pwrc_domain_data meson_secure_s4_pwrc_data = { - .domains = s4_pwrc_domains, - .count = ARRAY_SIZE(s4_pwrc_domains), -}; - -static const struct of_device_id meson_secure_pwrc_match_table[] = { - { - .compatible = "amlogic,meson-a1-pwrc", - .data = &meson_secure_a1_pwrc_data, - }, - { - .compatible = "amlogic,c3-pwrc", - .data = &amlogic_secure_c3_pwrc_data, - }, - { - .compatible = "amlogic,meson-s4-pwrc", - .data = &meson_secure_s4_pwrc_data, - }, - { /* sentinel */ } -}; -MODULE_DEVICE_TABLE(of, meson_secure_pwrc_match_table); - -static struct platform_driver meson_secure_pwrc_driver = { - .probe = meson_secure_pwrc_probe, - .driver = { - .name = "meson_secure_pwrc", - .of_match_table = meson_secure_pwrc_match_table, - }, -}; -module_platform_driver(meson_secure_pwrc_driver); -MODULE_LICENSE("Dual MIT/GPL"); diff --git a/drivers/genpd/apple/Makefile b/drivers/genpd/apple/Makefile deleted file mode 100644 index 53665af630be..000000000000 --- a/drivers/genpd/apple/Makefile +++ /dev/null @@ -1,2 +0,0 @@ -# SPDX-License-Identifier: GPL-2.0-only -obj-$(CONFIG_APPLE_PMGR_PWRSTATE) += pmgr-pwrstate.o diff --git a/drivers/genpd/apple/pmgr-pwrstate.c b/drivers/genpd/apple/pmgr-pwrstate.c deleted file mode 100644 index d62a776c89a1..000000000000 --- a/drivers/genpd/apple/pmgr-pwrstate.c +++ /dev/null @@ -1,326 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0-only OR MIT -/* - * Apple SoC PMGR device power state driver - * - * Copyright The Asahi Linux Contributors - */ - -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -#define APPLE_PMGR_RESET BIT(31) -#define APPLE_PMGR_AUTO_ENABLE BIT(28) -#define APPLE_PMGR_PS_AUTO GENMASK(27, 24) -#define APPLE_PMGR_PS_MIN GENMASK(19, 16) -#define APPLE_PMGR_PARENT_OFF BIT(11) -#define APPLE_PMGR_DEV_DISABLE BIT(10) -#define APPLE_PMGR_WAS_CLKGATED BIT(9) -#define APPLE_PMGR_WAS_PWRGATED BIT(8) -#define APPLE_PMGR_PS_ACTUAL GENMASK(7, 4) -#define APPLE_PMGR_PS_TARGET GENMASK(3, 0) - -#define APPLE_PMGR_FLAGS (APPLE_PMGR_WAS_CLKGATED | APPLE_PMGR_WAS_PWRGATED) - -#define APPLE_PMGR_PS_ACTIVE 0xf -#define APPLE_PMGR_PS_CLKGATE 0x4 -#define APPLE_PMGR_PS_PWRGATE 0x0 - -#define APPLE_PMGR_PS_SET_TIMEOUT 100 -#define APPLE_PMGR_RESET_TIME 1 - -struct apple_pmgr_ps { - struct device *dev; - struct generic_pm_domain genpd; - struct reset_controller_dev rcdev; - struct regmap *regmap; - u32 offset; - u32 min_state; -}; - -#define genpd_to_apple_pmgr_ps(_genpd) container_of(_genpd, struct apple_pmgr_ps, genpd) -#define rcdev_to_apple_pmgr_ps(_rcdev) container_of(_rcdev, struct apple_pmgr_ps, rcdev) - -static int apple_pmgr_ps_set(struct generic_pm_domain *genpd, u32 pstate, bool auto_enable) -{ - int ret; - struct apple_pmgr_ps *ps = genpd_to_apple_pmgr_ps(genpd); - u32 reg; - - ret = regmap_read(ps->regmap, ps->offset, ®); - if (ret < 0) - return ret; - - /* Resets are synchronous, and only work if the device is powered and clocked. */ - if (reg & APPLE_PMGR_RESET && pstate != APPLE_PMGR_PS_ACTIVE) - dev_err(ps->dev, "PS %s: powering off with RESET active\n", - genpd->name); - - reg &= ~(APPLE_PMGR_AUTO_ENABLE | APPLE_PMGR_FLAGS | APPLE_PMGR_PS_TARGET); - reg |= FIELD_PREP(APPLE_PMGR_PS_TARGET, pstate); - - dev_dbg(ps->dev, "PS %s: pwrstate = 0x%x: 0x%x\n", genpd->name, pstate, reg); - - regmap_write(ps->regmap, ps->offset, reg); - - ret = regmap_read_poll_timeout_atomic( - ps->regmap, ps->offset, reg, - (FIELD_GET(APPLE_PMGR_PS_ACTUAL, reg) == pstate), 1, - APPLE_PMGR_PS_SET_TIMEOUT); - if (ret < 0) - dev_err(ps->dev, "PS %s: Failed to reach power state 0x%x (now: 0x%x)\n", - genpd->name, pstate, reg); - - if (auto_enable) { - /* Not all devices implement this; this is a no-op where not implemented. */ - reg &= ~APPLE_PMGR_FLAGS; - reg |= APPLE_PMGR_AUTO_ENABLE; - regmap_write(ps->regmap, ps->offset, reg); - } - - return ret; -} - -static bool apple_pmgr_ps_is_active(struct apple_pmgr_ps *ps) -{ - u32 reg = 0; - - regmap_read(ps->regmap, ps->offset, ®); - /* - * We consider domains as active if they are actually on, or if they have auto-PM - * enabled and the intended target is on. - */ - return (FIELD_GET(APPLE_PMGR_PS_ACTUAL, reg) == APPLE_PMGR_PS_ACTIVE || - (FIELD_GET(APPLE_PMGR_PS_TARGET, reg) == APPLE_PMGR_PS_ACTIVE && - reg & APPLE_PMGR_AUTO_ENABLE)); -} - -static int apple_pmgr_ps_power_on(struct generic_pm_domain *genpd) -{ - return apple_pmgr_ps_set(genpd, APPLE_PMGR_PS_ACTIVE, true); -} - -static int apple_pmgr_ps_power_off(struct generic_pm_domain *genpd) -{ - return apple_pmgr_ps_set(genpd, APPLE_PMGR_PS_PWRGATE, false); -} - -static int apple_pmgr_reset_assert(struct reset_controller_dev *rcdev, unsigned long id) -{ - struct apple_pmgr_ps *ps = rcdev_to_apple_pmgr_ps(rcdev); - unsigned long flags; - - spin_lock_irqsave(&ps->genpd.slock, flags); - - if (ps->genpd.status == GENPD_STATE_OFF) - dev_err(ps->dev, "PS 0x%x: asserting RESET while powered down\n", ps->offset); - - dev_dbg(ps->dev, "PS 0x%x: assert reset\n", ps->offset); - /* Quiesce device before asserting reset */ - regmap_update_bits(ps->regmap, ps->offset, APPLE_PMGR_FLAGS | APPLE_PMGR_DEV_DISABLE, - APPLE_PMGR_DEV_DISABLE); - regmap_update_bits(ps->regmap, ps->offset, APPLE_PMGR_FLAGS | APPLE_PMGR_RESET, - APPLE_PMGR_RESET); - - spin_unlock_irqrestore(&ps->genpd.slock, flags); - - return 0; -} - -static int apple_pmgr_reset_deassert(struct reset_controller_dev *rcdev, unsigned long id) -{ - struct apple_pmgr_ps *ps = rcdev_to_apple_pmgr_ps(rcdev); - unsigned long flags; - - spin_lock_irqsave(&ps->genpd.slock, flags); - - dev_dbg(ps->dev, "PS 0x%x: deassert reset\n", ps->offset); - regmap_update_bits(ps->regmap, ps->offset, APPLE_PMGR_FLAGS | APPLE_PMGR_RESET, 0); - regmap_update_bits(ps->regmap, ps->offset, APPLE_PMGR_FLAGS | APPLE_PMGR_DEV_DISABLE, 0); - - if (ps->genpd.status == GENPD_STATE_OFF) - dev_err(ps->dev, "PS 0x%x: RESET was deasserted while powered down\n", ps->offset); - - spin_unlock_irqrestore(&ps->genpd.slock, flags); - - return 0; -} - -static int apple_pmgr_reset_reset(struct reset_controller_dev *rcdev, unsigned long id) -{ - int ret; - - ret = apple_pmgr_reset_assert(rcdev, id); - if (ret) - return ret; - - usleep_range(APPLE_PMGR_RESET_TIME, 2 * APPLE_PMGR_RESET_TIME); - - return apple_pmgr_reset_deassert(rcdev, id); -} - -static int apple_pmgr_reset_status(struct reset_controller_dev *rcdev, unsigned long id) -{ - struct apple_pmgr_ps *ps = rcdev_to_apple_pmgr_ps(rcdev); - u32 reg = 0; - - regmap_read(ps->regmap, ps->offset, ®); - - return !!(reg & APPLE_PMGR_RESET); -} - -const struct reset_control_ops apple_pmgr_reset_ops = { - .assert = apple_pmgr_reset_assert, - .deassert = apple_pmgr_reset_deassert, - .reset = apple_pmgr_reset_reset, - .status = apple_pmgr_reset_status, -}; - -static int apple_pmgr_reset_xlate(struct reset_controller_dev *rcdev, - const struct of_phandle_args *reset_spec) -{ - return 0; -} - -static int apple_pmgr_ps_probe(struct platform_device *pdev) -{ - struct device *dev = &pdev->dev; - struct device_node *node = dev->of_node; - struct apple_pmgr_ps *ps; - struct regmap *regmap; - struct of_phandle_iterator it; - int ret; - const char *name; - bool active; - - regmap = syscon_node_to_regmap(node->parent); - if (IS_ERR(regmap)) - return PTR_ERR(regmap); - - ps = devm_kzalloc(dev, sizeof(*ps), GFP_KERNEL); - if (!ps) - return -ENOMEM; - - ps->dev = dev; - ps->regmap = regmap; - - ret = of_property_read_string(node, "label", &name); - if (ret < 0) { - dev_err(dev, "missing label property\n"); - return ret; - } - - ret = of_property_read_u32(node, "reg", &ps->offset); - if (ret < 0) { - dev_err(dev, "missing reg property\n"); - return ret; - } - - ps->genpd.flags |= GENPD_FLAG_IRQ_SAFE; - ps->genpd.name = name; - ps->genpd.power_on = apple_pmgr_ps_power_on; - ps->genpd.power_off = apple_pmgr_ps_power_off; - - ret = of_property_read_u32(node, "apple,min-state", &ps->min_state); - if (ret == 0 && ps->min_state <= APPLE_PMGR_PS_ACTIVE) - regmap_update_bits(regmap, ps->offset, APPLE_PMGR_FLAGS | APPLE_PMGR_PS_MIN, - FIELD_PREP(APPLE_PMGR_PS_MIN, ps->min_state)); - - active = apple_pmgr_ps_is_active(ps); - if (of_property_read_bool(node, "apple,always-on")) { - ps->genpd.flags |= GENPD_FLAG_ALWAYS_ON; - if (!active) { - dev_warn(dev, "always-on domain %s is not on at boot\n", name); - /* Turn it on so pm_genpd_init does not fail */ - active = apple_pmgr_ps_power_on(&ps->genpd) == 0; - } - } - - /* Turn on auto-PM if the domain is already on */ - if (active) - regmap_update_bits(regmap, ps->offset, APPLE_PMGR_FLAGS | APPLE_PMGR_AUTO_ENABLE, - APPLE_PMGR_AUTO_ENABLE); - - ret = pm_genpd_init(&ps->genpd, NULL, !active); - if (ret < 0) { - dev_err(dev, "pm_genpd_init failed\n"); - return ret; - } - - ret = of_genpd_add_provider_simple(node, &ps->genpd); - if (ret < 0) { - dev_err(dev, "of_genpd_add_provider_simple failed\n"); - return ret; - } - - of_for_each_phandle(&it, ret, node, "power-domains", "#power-domain-cells", -1) { - struct of_phandle_args parent, child; - - parent.np = it.node; - parent.args_count = of_phandle_iterator_args(&it, parent.args, MAX_PHANDLE_ARGS); - child.np = node; - child.args_count = 0; - ret = of_genpd_add_subdomain(&parent, &child); - - if (ret == -EPROBE_DEFER) { - of_node_put(parent.np); - goto err_remove; - } else if (ret < 0) { - dev_err(dev, "failed to add to parent domain: %d (%s -> %s)\n", - ret, it.node->name, node->name); - of_node_put(parent.np); - goto err_remove; - } - } - - /* - * Do not participate in regular PM; parent power domains are handled via the - * genpd hierarchy. - */ - pm_genpd_remove_device(dev); - - ps->rcdev.owner = THIS_MODULE; - ps->rcdev.nr_resets = 1; - ps->rcdev.ops = &apple_pmgr_reset_ops; - ps->rcdev.of_node = dev->of_node; - ps->rcdev.of_reset_n_cells = 0; - ps->rcdev.of_xlate = apple_pmgr_reset_xlate; - - ret = devm_reset_controller_register(dev, &ps->rcdev); - if (ret < 0) - goto err_remove; - - return 0; -err_remove: - of_genpd_del_provider(node); - pm_genpd_remove(&ps->genpd); - return ret; -} - -static const struct of_device_id apple_pmgr_ps_of_match[] = { - { .compatible = "apple,pmgr-pwrstate" }, - {} -}; - -MODULE_DEVICE_TABLE(of, apple_pmgr_ps_of_match); - -static struct platform_driver apple_pmgr_ps_driver = { - .probe = apple_pmgr_ps_probe, - .driver = { - .name = "apple-pmgr-pwrstate", - .of_match_table = apple_pmgr_ps_of_match, - }, -}; - -MODULE_AUTHOR("Hector Martin "); -MODULE_DESCRIPTION("PMGR power state driver for Apple SoCs"); - -module_platform_driver(apple_pmgr_ps_driver); diff --git a/drivers/genpd/bcm/Makefile b/drivers/genpd/bcm/Makefile deleted file mode 100644 index 6bfbe4e4db13..000000000000 --- a/drivers/genpd/bcm/Makefile +++ /dev/null @@ -1,5 +0,0 @@ -# SPDX-License-Identifier: GPL-2.0-only -obj-$(CONFIG_BCM_PMB) += bcm-pmb.o -obj-$(CONFIG_BCM2835_POWER) += bcm2835-power.o -obj-$(CONFIG_BCM63XX_POWER) += bcm63xx-power.o -obj-$(CONFIG_RASPBERRYPI_POWER) += raspberrypi-power.o diff --git a/drivers/genpd/bcm/bcm-pmb.c b/drivers/genpd/bcm/bcm-pmb.c deleted file mode 100644 index a72ba26ecf9d..000000000000 --- a/drivers/genpd/bcm/bcm-pmb.c +++ /dev/null @@ -1,363 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0-or-later -/* - * Copyright (c) 2013 Broadcom - * Copyright (C) 2020 Rafał Miłecki - */ - -#include -#include -#include -#include -#include -#include -#include - -#define BPCM_ID_REG 0x00 -#define BPCM_CAPABILITIES 0x04 -#define BPCM_CAP_NUM_ZONES 0x000000ff -#define BPCM_CAP_SR_REG_BITS 0x0000ff00 -#define BPCM_CAP_PLLTYPE 0x00030000 -#define BPCM_CAP_UBUS 0x00080000 -#define BPCM_CONTROL 0x08 -#define BPCM_STATUS 0x0c -#define BPCM_ROSC_CONTROL 0x10 -#define BPCM_ROSC_THRESH_H 0x14 -#define BPCM_ROSC_THRESHOLD_BCM6838 0x14 -#define BPCM_ROSC_THRESH_S 0x18 -#define BPCM_ROSC_COUNT_BCM6838 0x18 -#define BPCM_ROSC_COUNT 0x1c -#define BPCM_PWD_CONTROL_BCM6838 0x1c -#define BPCM_PWD_CONTROL 0x20 -#define BPCM_SR_CONTROL_BCM6838 0x20 -#define BPCM_PWD_ACCUM_CONTROL 0x24 -#define BPCM_SR_CONTROL 0x28 -#define BPCM_GLOBAL_CONTROL 0x2c -#define BPCM_MISC_CONTROL 0x30 -#define BPCM_MISC_CONTROL2 0x34 -#define BPCM_SGPHY_CNTL 0x38 -#define BPCM_SGPHY_STATUS 0x3c -#define BPCM_ZONE0 0x40 -#define BPCM_ZONE_CONTROL 0x00 -#define BPCM_ZONE_CONTROL_MANUAL_CLK_EN 0x00000001 -#define BPCM_ZONE_CONTROL_MANUAL_RESET_CTL 0x00000002 -#define BPCM_ZONE_CONTROL_FREQ_SCALE_USED 0x00000004 /* R/O */ -#define BPCM_ZONE_CONTROL_DPG_CAPABLE 0x00000008 /* R/O */ -#define BPCM_ZONE_CONTROL_MANUAL_MEM_PWR 0x00000030 -#define BPCM_ZONE_CONTROL_MANUAL_ISO_CTL 0x00000040 -#define BPCM_ZONE_CONTROL_MANUAL_CTL 0x00000080 -#define BPCM_ZONE_CONTROL_DPG_CTL_EN 0x00000100 -#define BPCM_ZONE_CONTROL_PWR_DN_REQ 0x00000200 -#define BPCM_ZONE_CONTROL_PWR_UP_REQ 0x00000400 -#define BPCM_ZONE_CONTROL_MEM_PWR_CTL_EN 0x00000800 -#define BPCM_ZONE_CONTROL_BLK_RESET_ASSERT 0x00001000 -#define BPCM_ZONE_CONTROL_MEM_STBY 0x00002000 -#define BPCM_ZONE_CONTROL_RESERVED 0x0007c000 -#define BPCM_ZONE_CONTROL_PWR_CNTL_STATE 0x00f80000 -#define BPCM_ZONE_CONTROL_FREQ_SCALAR_DYN_SEL 0x01000000 /* R/O */ -#define BPCM_ZONE_CONTROL_PWR_OFF_STATE 0x02000000 /* R/O */ -#define BPCM_ZONE_CONTROL_PWR_ON_STATE 0x04000000 /* R/O */ -#define BPCM_ZONE_CONTROL_PWR_GOOD 0x08000000 /* R/O */ -#define BPCM_ZONE_CONTROL_DPG_PWR_STATE 0x10000000 /* R/O */ -#define BPCM_ZONE_CONTROL_MEM_PWR_STATE 0x20000000 /* R/O */ -#define BPCM_ZONE_CONTROL_ISO_STATE 0x40000000 /* R/O */ -#define BPCM_ZONE_CONTROL_RESET_STATE 0x80000000 /* R/O */ -#define BPCM_ZONE_CONFIG1 0x04 -#define BPCM_ZONE_CONFIG2 0x08 -#define BPCM_ZONE_FREQ_SCALAR_CONTROL 0x0c -#define BPCM_ZONE_SIZE 0x10 - -struct bcm_pmb { - struct device *dev; - void __iomem *base; - spinlock_t lock; - bool little_endian; - struct genpd_onecell_data genpd_onecell_data; -}; - -struct bcm_pmb_pd_data { - const char * const name; - int id; - u8 bus; - u8 device; -}; - -struct bcm_pmb_pm_domain { - struct bcm_pmb *pmb; - const struct bcm_pmb_pd_data *data; - struct generic_pm_domain genpd; -}; - -static int bcm_pmb_bpcm_read(struct bcm_pmb *pmb, int bus, u8 device, - int offset, u32 *val) -{ - void __iomem *base = pmb->base + bus * 0x20; - unsigned long flags; - int err; - - spin_lock_irqsave(&pmb->lock, flags); - err = bpcm_rd(base, device, offset, val); - spin_unlock_irqrestore(&pmb->lock, flags); - - if (!err) - *val = pmb->little_endian ? le32_to_cpu(*val) : be32_to_cpu(*val); - - return err; -} - -static int bcm_pmb_bpcm_write(struct bcm_pmb *pmb, int bus, u8 device, - int offset, u32 val) -{ - void __iomem *base = pmb->base + bus * 0x20; - unsigned long flags; - int err; - - val = pmb->little_endian ? cpu_to_le32(val) : cpu_to_be32(val); - - spin_lock_irqsave(&pmb->lock, flags); - err = bpcm_wr(base, device, offset, val); - spin_unlock_irqrestore(&pmb->lock, flags); - - return err; -} - -static int bcm_pmb_power_off_zone(struct bcm_pmb *pmb, int bus, u8 device, - int zone) -{ - int offset; - u32 val; - int err; - - offset = BPCM_ZONE0 + zone * BPCM_ZONE_SIZE + BPCM_ZONE_CONTROL; - - err = bcm_pmb_bpcm_read(pmb, bus, device, offset, &val); - if (err) - return err; - - val |= BPCM_ZONE_CONTROL_PWR_DN_REQ; - val &= ~BPCM_ZONE_CONTROL_PWR_UP_REQ; - - err = bcm_pmb_bpcm_write(pmb, bus, device, offset, val); - - return err; -} - -static int bcm_pmb_power_on_zone(struct bcm_pmb *pmb, int bus, u8 device, - int zone) -{ - int offset; - u32 val; - int err; - - offset = BPCM_ZONE0 + zone * BPCM_ZONE_SIZE + BPCM_ZONE_CONTROL; - - err = bcm_pmb_bpcm_read(pmb, bus, device, offset, &val); - if (err) - return err; - - if (!(val & BPCM_ZONE_CONTROL_PWR_ON_STATE)) { - val &= ~BPCM_ZONE_CONTROL_PWR_DN_REQ; - val |= BPCM_ZONE_CONTROL_DPG_CTL_EN; - val |= BPCM_ZONE_CONTROL_PWR_UP_REQ; - val |= BPCM_ZONE_CONTROL_MEM_PWR_CTL_EN; - val |= BPCM_ZONE_CONTROL_BLK_RESET_ASSERT; - - err = bcm_pmb_bpcm_write(pmb, bus, device, offset, val); - } - - return err; -} - -static int bcm_pmb_power_off_device(struct bcm_pmb *pmb, int bus, u8 device) -{ - int offset; - u32 val; - int err; - - /* Entire device can be powered off by powering off the 0th zone */ - offset = BPCM_ZONE0 + BPCM_ZONE_CONTROL; - - err = bcm_pmb_bpcm_read(pmb, bus, device, offset, &val); - if (err) - return err; - - if (!(val & BPCM_ZONE_CONTROL_PWR_OFF_STATE)) { - val = BPCM_ZONE_CONTROL_PWR_DN_REQ; - - err = bcm_pmb_bpcm_write(pmb, bus, device, offset, val); - } - - return err; -} - -static int bcm_pmb_power_on_device(struct bcm_pmb *pmb, int bus, u8 device) -{ - u32 val; - int err; - int i; - - err = bcm_pmb_bpcm_read(pmb, bus, device, BPCM_CAPABILITIES, &val); - if (err) - return err; - - for (i = 0; i < (val & BPCM_CAP_NUM_ZONES); i++) { - err = bcm_pmb_power_on_zone(pmb, bus, device, i); - if (err) - return err; - } - - return err; -} - -static int bcm_pmb_power_on_sata(struct bcm_pmb *pmb, int bus, u8 device) -{ - int err; - - err = bcm_pmb_power_on_zone(pmb, bus, device, 0); - if (err) - return err; - - /* Does not apply to the BCM963158 */ - err = bcm_pmb_bpcm_write(pmb, bus, device, BPCM_MISC_CONTROL, 0); - if (err) - return err; - - err = bcm_pmb_bpcm_write(pmb, bus, device, BPCM_SR_CONTROL, 0xffffffff); - if (err) - return err; - - err = bcm_pmb_bpcm_write(pmb, bus, device, BPCM_SR_CONTROL, 0); - - return err; -} - -static int bcm_pmb_power_on(struct generic_pm_domain *genpd) -{ - struct bcm_pmb_pm_domain *pd = container_of(genpd, struct bcm_pmb_pm_domain, genpd); - const struct bcm_pmb_pd_data *data = pd->data; - struct bcm_pmb *pmb = pd->pmb; - - switch (data->id) { - case BCM_PMB_PCIE0: - case BCM_PMB_PCIE1: - case BCM_PMB_PCIE2: - return bcm_pmb_power_on_zone(pmb, data->bus, data->device, 0); - case BCM_PMB_HOST_USB: - return bcm_pmb_power_on_device(pmb, data->bus, data->device); - case BCM_PMB_SATA: - return bcm_pmb_power_on_sata(pmb, data->bus, data->device); - default: - dev_err(pmb->dev, "unsupported device id: %d\n", data->id); - return -EINVAL; - } -} - -static int bcm_pmb_power_off(struct generic_pm_domain *genpd) -{ - struct bcm_pmb_pm_domain *pd = container_of(genpd, struct bcm_pmb_pm_domain, genpd); - const struct bcm_pmb_pd_data *data = pd->data; - struct bcm_pmb *pmb = pd->pmb; - - switch (data->id) { - case BCM_PMB_PCIE0: - case BCM_PMB_PCIE1: - case BCM_PMB_PCIE2: - return bcm_pmb_power_off_zone(pmb, data->bus, data->device, 0); - case BCM_PMB_HOST_USB: - return bcm_pmb_power_off_device(pmb, data->bus, data->device); - default: - dev_err(pmb->dev, "unsupported device id: %d\n", data->id); - return -EINVAL; - } -} - -static int bcm_pmb_probe(struct platform_device *pdev) -{ - struct device *dev = &pdev->dev; - const struct bcm_pmb_pd_data *table; - const struct bcm_pmb_pd_data *e; - struct bcm_pmb *pmb; - int max_id; - int err; - - pmb = devm_kzalloc(dev, sizeof(*pmb), GFP_KERNEL); - if (!pmb) - return -ENOMEM; - - pmb->dev = dev; - - pmb->base = devm_platform_ioremap_resource(pdev, 0); - if (IS_ERR(pmb->base)) - return PTR_ERR(pmb->base); - - spin_lock_init(&pmb->lock); - - pmb->little_endian = !of_device_is_big_endian(dev->of_node); - - table = of_device_get_match_data(dev); - if (!table) - return -EINVAL; - - max_id = 0; - for (e = table; e->name; e++) - max_id = max(max_id, e->id); - - pmb->genpd_onecell_data.num_domains = max_id + 1; - pmb->genpd_onecell_data.domains = - devm_kcalloc(dev, pmb->genpd_onecell_data.num_domains, - sizeof(struct generic_pm_domain *), GFP_KERNEL); - if (!pmb->genpd_onecell_data.domains) - return -ENOMEM; - - for (e = table; e->name; e++) { - struct bcm_pmb_pm_domain *pd = devm_kzalloc(dev, sizeof(*pd), GFP_KERNEL); - - if (!pd) - return -ENOMEM; - - pd->pmb = pmb; - pd->data = e; - pd->genpd.name = e->name; - pd->genpd.power_on = bcm_pmb_power_on; - pd->genpd.power_off = bcm_pmb_power_off; - - pm_genpd_init(&pd->genpd, NULL, true); - pmb->genpd_onecell_data.domains[e->id] = &pd->genpd; - } - - err = of_genpd_add_provider_onecell(dev->of_node, &pmb->genpd_onecell_data); - if (err) { - dev_err(dev, "failed to add genpd provider: %d\n", err); - return err; - } - - return 0; -} - -static const struct bcm_pmb_pd_data bcm_pmb_bcm4908_data[] = { - { .name = "pcie2", .id = BCM_PMB_PCIE2, .bus = 0, .device = 2, }, - { .name = "pcie0", .id = BCM_PMB_PCIE0, .bus = 1, .device = 14, }, - { .name = "pcie1", .id = BCM_PMB_PCIE1, .bus = 1, .device = 15, }, - { .name = "usb", .id = BCM_PMB_HOST_USB, .bus = 1, .device = 17, }, - { }, -}; - -static const struct bcm_pmb_pd_data bcm_pmb_bcm63138_data[] = { - { .name = "sata", .id = BCM_PMB_SATA, .bus = 0, .device = 3, }, - { }, -}; - -static const struct of_device_id bcm_pmb_of_match[] = { - { .compatible = "brcm,bcm4908-pmb", .data = &bcm_pmb_bcm4908_data, }, - { .compatible = "brcm,bcm63138-pmb", .data = &bcm_pmb_bcm63138_data, }, - { }, -}; - -static struct platform_driver bcm_pmb_driver = { - .driver = { - .name = "bcm-pmb", - .of_match_table = bcm_pmb_of_match, - }, - .probe = bcm_pmb_probe, -}; - -builtin_platform_driver(bcm_pmb_driver); diff --git a/drivers/genpd/bcm/bcm2835-power.c b/drivers/genpd/bcm/bcm2835-power.c deleted file mode 100644 index 1a179d4e011c..000000000000 --- a/drivers/genpd/bcm/bcm2835-power.c +++ /dev/null @@ -1,713 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0+ -/* - * Power domain driver for Broadcom BCM2835 - * - * Copyright (C) 2018 Broadcom - */ - -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -#define PM_GNRIC 0x00 -#define PM_AUDIO 0x04 -#define PM_STATUS 0x18 -#define PM_RSTC 0x1c -#define PM_RSTS 0x20 -#define PM_WDOG 0x24 -#define PM_PADS0 0x28 -#define PM_PADS2 0x2c -#define PM_PADS3 0x30 -#define PM_PADS4 0x34 -#define PM_PADS5 0x38 -#define PM_PADS6 0x3c -#define PM_CAM0 0x44 -#define PM_CAM0_LDOHPEN BIT(2) -#define PM_CAM0_LDOLPEN BIT(1) -#define PM_CAM0_CTRLEN BIT(0) - -#define PM_CAM1 0x48 -#define PM_CAM1_LDOHPEN BIT(2) -#define PM_CAM1_LDOLPEN BIT(1) -#define PM_CAM1_CTRLEN BIT(0) - -#define PM_CCP2TX 0x4c -#define PM_CCP2TX_LDOEN BIT(1) -#define PM_CCP2TX_CTRLEN BIT(0) - -#define PM_DSI0 0x50 -#define PM_DSI0_LDOHPEN BIT(2) -#define PM_DSI0_LDOLPEN BIT(1) -#define PM_DSI0_CTRLEN BIT(0) - -#define PM_DSI1 0x54 -#define PM_DSI1_LDOHPEN BIT(2) -#define PM_DSI1_LDOLPEN BIT(1) -#define PM_DSI1_CTRLEN BIT(0) - -#define PM_HDMI 0x58 -#define PM_HDMI_RSTDR BIT(19) -#define PM_HDMI_LDOPD BIT(1) -#define PM_HDMI_CTRLEN BIT(0) - -#define PM_USB 0x5c -/* The power gates must be enabled with this bit before enabling the LDO in the - * USB block. - */ -#define PM_USB_CTRLEN BIT(0) - -#define PM_PXLDO 0x60 -#define PM_PXBG 0x64 -#define PM_DFT 0x68 -#define PM_SMPS 0x6c -#define PM_XOSC 0x70 -#define PM_SPAREW 0x74 -#define PM_SPARER 0x78 -#define PM_AVS_RSTDR 0x7c -#define PM_AVS_STAT 0x80 -#define PM_AVS_EVENT 0x84 -#define PM_AVS_INTEN 0x88 -#define PM_DUMMY 0xfc - -#define PM_IMAGE 0x108 -#define PM_GRAFX 0x10c -#define PM_PROC 0x110 -#define PM_ENAB BIT(12) -#define PM_ISPRSTN BIT(8) -#define PM_H264RSTN BIT(7) -#define PM_PERIRSTN BIT(6) -#define PM_V3DRSTN BIT(6) -#define PM_ISFUNC BIT(5) -#define PM_MRDONE BIT(4) -#define PM_MEMREP BIT(3) -#define PM_ISPOW BIT(2) -#define PM_POWOK BIT(1) -#define PM_POWUP BIT(0) -#define PM_INRUSH_SHIFT 13 -#define PM_INRUSH_3_5_MA 0 -#define PM_INRUSH_5_MA 1 -#define PM_INRUSH_10_MA 2 -#define PM_INRUSH_20_MA 3 -#define PM_INRUSH_MASK (3 << PM_INRUSH_SHIFT) - -#define PM_PASSWORD 0x5a000000 - -#define PM_WDOG_TIME_SET 0x000fffff -#define PM_RSTC_WRCFG_CLR 0xffffffcf -#define PM_RSTS_HADWRH_SET 0x00000040 -#define PM_RSTC_WRCFG_SET 0x00000030 -#define PM_RSTC_WRCFG_FULL_RESET 0x00000020 -#define PM_RSTC_RESET 0x00000102 - -#define PM_READ(reg) readl(power->base + (reg)) -#define PM_WRITE(reg, val) writel(PM_PASSWORD | (val), power->base + (reg)) - -#define ASB_BRDG_VERSION 0x00 -#define ASB_CPR_CTRL 0x04 - -#define ASB_V3D_S_CTRL 0x08 -#define ASB_V3D_M_CTRL 0x0c -#define ASB_ISP_S_CTRL 0x10 -#define ASB_ISP_M_CTRL 0x14 -#define ASB_H264_S_CTRL 0x18 -#define ASB_H264_M_CTRL 0x1c - -#define ASB_REQ_STOP BIT(0) -#define ASB_ACK BIT(1) -#define ASB_EMPTY BIT(2) -#define ASB_FULL BIT(3) - -#define ASB_AXI_BRDG_ID 0x20 - -#define BCM2835_BRDG_ID 0x62726467 - -struct bcm2835_power_domain { - struct generic_pm_domain base; - struct bcm2835_power *power; - u32 domain; - struct clk *clk; -}; - -struct bcm2835_power { - struct device *dev; - /* PM registers. */ - void __iomem *base; - /* AXI Async bridge registers. */ - void __iomem *asb; - /* RPiVid bridge registers. */ - void __iomem *rpivid_asb; - - struct genpd_onecell_data pd_xlate; - struct bcm2835_power_domain domains[BCM2835_POWER_DOMAIN_COUNT]; - struct reset_controller_dev reset; -}; - -static int bcm2835_asb_control(struct bcm2835_power *power, u32 reg, bool enable) -{ - void __iomem *base = power->asb; - u64 start; - u32 val; - - switch (reg) { - case 0: - return 0; - case ASB_V3D_S_CTRL: - case ASB_V3D_M_CTRL: - if (power->rpivid_asb) - base = power->rpivid_asb; - break; - } - - start = ktime_get_ns(); - - /* Enable the module's async AXI bridges. */ - if (enable) { - val = readl(base + reg) & ~ASB_REQ_STOP; - } else { - val = readl(base + reg) | ASB_REQ_STOP; - } - writel(PM_PASSWORD | val, base + reg); - - while (readl(base + reg) & ASB_ACK) { - cpu_relax(); - if (ktime_get_ns() - start >= 1000) - return -ETIMEDOUT; - } - - return 0; -} - -static int bcm2835_asb_enable(struct bcm2835_power *power, u32 reg) -{ - return bcm2835_asb_control(power, reg, true); -} - -static int bcm2835_asb_disable(struct bcm2835_power *power, u32 reg) -{ - return bcm2835_asb_control(power, reg, false); -} - -static int bcm2835_power_power_off(struct bcm2835_power_domain *pd, u32 pm_reg) -{ - struct bcm2835_power *power = pd->power; - - /* We don't run this on BCM2711 */ - if (power->rpivid_asb) - return 0; - - /* Enable functional isolation */ - PM_WRITE(pm_reg, PM_READ(pm_reg) & ~PM_ISFUNC); - - /* Enable electrical isolation */ - PM_WRITE(pm_reg, PM_READ(pm_reg) & ~PM_ISPOW); - - /* Open the power switches. */ - PM_WRITE(pm_reg, PM_READ(pm_reg) & ~PM_POWUP); - - return 0; -} - -static int bcm2835_power_power_on(struct bcm2835_power_domain *pd, u32 pm_reg) -{ - struct bcm2835_power *power = pd->power; - struct device *dev = power->dev; - u64 start; - int ret; - int inrush; - bool powok; - - /* We don't run this on BCM2711 */ - if (power->rpivid_asb) - return 0; - - /* If it was already powered on by the fw, leave it that way. */ - if (PM_READ(pm_reg) & PM_POWUP) - return 0; - - /* Enable power. Allowing too much current at once may result - * in POWOK never getting set, so start low and ramp it up as - * necessary to succeed. - */ - powok = false; - for (inrush = PM_INRUSH_3_5_MA; inrush <= PM_INRUSH_20_MA; inrush++) { - PM_WRITE(pm_reg, - (PM_READ(pm_reg) & ~PM_INRUSH_MASK) | - (inrush << PM_INRUSH_SHIFT) | - PM_POWUP); - - start = ktime_get_ns(); - while (!(powok = !!(PM_READ(pm_reg) & PM_POWOK))) { - cpu_relax(); - if (ktime_get_ns() - start >= 3000) - break; - } - } - if (!powok) { - dev_err(dev, "Timeout waiting for %s power OK\n", - pd->base.name); - ret = -ETIMEDOUT; - goto err_disable_powup; - } - - /* Disable electrical isolation */ - PM_WRITE(pm_reg, PM_READ(pm_reg) | PM_ISPOW); - - /* Repair memory */ - PM_WRITE(pm_reg, PM_READ(pm_reg) | PM_MEMREP); - start = ktime_get_ns(); - while (!(PM_READ(pm_reg) & PM_MRDONE)) { - cpu_relax(); - if (ktime_get_ns() - start >= 1000) { - dev_err(dev, "Timeout waiting for %s memory repair\n", - pd->base.name); - ret = -ETIMEDOUT; - goto err_disable_ispow; - } - } - - /* Disable functional isolation */ - PM_WRITE(pm_reg, PM_READ(pm_reg) | PM_ISFUNC); - - return 0; - -err_disable_ispow: - PM_WRITE(pm_reg, PM_READ(pm_reg) & ~PM_ISPOW); -err_disable_powup: - PM_WRITE(pm_reg, PM_READ(pm_reg) & ~(PM_POWUP | PM_INRUSH_MASK)); - return ret; -} - -static int bcm2835_asb_power_on(struct bcm2835_power_domain *pd, - u32 pm_reg, - u32 asb_m_reg, - u32 asb_s_reg, - u32 reset_flags) -{ - struct bcm2835_power *power = pd->power; - int ret; - - ret = clk_prepare_enable(pd->clk); - if (ret) { - dev_err(power->dev, "Failed to enable clock for %s\n", - pd->base.name); - return ret; - } - - /* Wait 32 clocks for reset to propagate, 1 us will be enough */ - udelay(1); - - clk_disable_unprepare(pd->clk); - - /* Deassert the resets. */ - PM_WRITE(pm_reg, PM_READ(pm_reg) | reset_flags); - - ret = clk_prepare_enable(pd->clk); - if (ret) { - dev_err(power->dev, "Failed to enable clock for %s\n", - pd->base.name); - goto err_enable_resets; - } - - ret = bcm2835_asb_enable(power, asb_m_reg); - if (ret) { - dev_err(power->dev, "Failed to enable ASB master for %s\n", - pd->base.name); - goto err_disable_clk; - } - ret = bcm2835_asb_enable(power, asb_s_reg); - if (ret) { - dev_err(power->dev, "Failed to enable ASB slave for %s\n", - pd->base.name); - goto err_disable_asb_master; - } - - return 0; - -err_disable_asb_master: - bcm2835_asb_disable(power, asb_m_reg); -err_disable_clk: - clk_disable_unprepare(pd->clk); -err_enable_resets: - PM_WRITE(pm_reg, PM_READ(pm_reg) & ~reset_flags); - return ret; -} - -static int bcm2835_asb_power_off(struct bcm2835_power_domain *pd, - u32 pm_reg, - u32 asb_m_reg, - u32 asb_s_reg, - u32 reset_flags) -{ - struct bcm2835_power *power = pd->power; - int ret; - - ret = bcm2835_asb_disable(power, asb_s_reg); - if (ret) { - dev_warn(power->dev, "Failed to disable ASB slave for %s\n", - pd->base.name); - return ret; - } - ret = bcm2835_asb_disable(power, asb_m_reg); - if (ret) { - dev_warn(power->dev, "Failed to disable ASB master for %s\n", - pd->base.name); - bcm2835_asb_enable(power, asb_s_reg); - return ret; - } - - clk_disable_unprepare(pd->clk); - - /* Assert the resets. */ - PM_WRITE(pm_reg, PM_READ(pm_reg) & ~reset_flags); - - return 0; -} - -static int bcm2835_power_pd_power_on(struct generic_pm_domain *domain) -{ - struct bcm2835_power_domain *pd = - container_of(domain, struct bcm2835_power_domain, base); - struct bcm2835_power *power = pd->power; - - switch (pd->domain) { - case BCM2835_POWER_DOMAIN_GRAFX: - return bcm2835_power_power_on(pd, PM_GRAFX); - - case BCM2835_POWER_DOMAIN_GRAFX_V3D: - return bcm2835_asb_power_on(pd, PM_GRAFX, - ASB_V3D_M_CTRL, ASB_V3D_S_CTRL, - PM_V3DRSTN); - - case BCM2835_POWER_DOMAIN_IMAGE: - return bcm2835_power_power_on(pd, PM_IMAGE); - - case BCM2835_POWER_DOMAIN_IMAGE_PERI: - return bcm2835_asb_power_on(pd, PM_IMAGE, - 0, 0, - PM_PERIRSTN); - - case BCM2835_POWER_DOMAIN_IMAGE_ISP: - return bcm2835_asb_power_on(pd, PM_IMAGE, - ASB_ISP_M_CTRL, ASB_ISP_S_CTRL, - PM_ISPRSTN); - - case BCM2835_POWER_DOMAIN_IMAGE_H264: - return bcm2835_asb_power_on(pd, PM_IMAGE, - ASB_H264_M_CTRL, ASB_H264_S_CTRL, - PM_H264RSTN); - - case BCM2835_POWER_DOMAIN_USB: - PM_WRITE(PM_USB, PM_USB_CTRLEN); - return 0; - - case BCM2835_POWER_DOMAIN_DSI0: - PM_WRITE(PM_DSI0, PM_DSI0_CTRLEN); - PM_WRITE(PM_DSI0, PM_DSI0_CTRLEN | PM_DSI0_LDOHPEN); - return 0; - - case BCM2835_POWER_DOMAIN_DSI1: - PM_WRITE(PM_DSI1, PM_DSI1_CTRLEN); - PM_WRITE(PM_DSI1, PM_DSI1_CTRLEN | PM_DSI1_LDOHPEN); - return 0; - - case BCM2835_POWER_DOMAIN_CCP2TX: - PM_WRITE(PM_CCP2TX, PM_CCP2TX_CTRLEN); - PM_WRITE(PM_CCP2TX, PM_CCP2TX_CTRLEN | PM_CCP2TX_LDOEN); - return 0; - - case BCM2835_POWER_DOMAIN_HDMI: - PM_WRITE(PM_HDMI, PM_READ(PM_HDMI) | PM_HDMI_RSTDR); - PM_WRITE(PM_HDMI, PM_READ(PM_HDMI) | PM_HDMI_CTRLEN); - PM_WRITE(PM_HDMI, PM_READ(PM_HDMI) & ~PM_HDMI_LDOPD); - usleep_range(100, 200); - PM_WRITE(PM_HDMI, PM_READ(PM_HDMI) & ~PM_HDMI_RSTDR); - return 0; - - default: - dev_err(power->dev, "Invalid domain %d\n", pd->domain); - return -EINVAL; - } -} - -static int bcm2835_power_pd_power_off(struct generic_pm_domain *domain) -{ - struct bcm2835_power_domain *pd = - container_of(domain, struct bcm2835_power_domain, base); - struct bcm2835_power *power = pd->power; - - switch (pd->domain) { - case BCM2835_POWER_DOMAIN_GRAFX: - return bcm2835_power_power_off(pd, PM_GRAFX); - - case BCM2835_POWER_DOMAIN_GRAFX_V3D: - return bcm2835_asb_power_off(pd, PM_GRAFX, - ASB_V3D_M_CTRL, ASB_V3D_S_CTRL, - PM_V3DRSTN); - - case BCM2835_POWER_DOMAIN_IMAGE: - return bcm2835_power_power_off(pd, PM_IMAGE); - - case BCM2835_POWER_DOMAIN_IMAGE_PERI: - return bcm2835_asb_power_off(pd, PM_IMAGE, - 0, 0, - PM_PERIRSTN); - - case BCM2835_POWER_DOMAIN_IMAGE_ISP: - return bcm2835_asb_power_off(pd, PM_IMAGE, - ASB_ISP_M_CTRL, ASB_ISP_S_CTRL, - PM_ISPRSTN); - - case BCM2835_POWER_DOMAIN_IMAGE_H264: - return bcm2835_asb_power_off(pd, PM_IMAGE, - ASB_H264_M_CTRL, ASB_H264_S_CTRL, - PM_H264RSTN); - - case BCM2835_POWER_DOMAIN_USB: - PM_WRITE(PM_USB, 0); - return 0; - - case BCM2835_POWER_DOMAIN_DSI0: - PM_WRITE(PM_DSI0, PM_DSI0_CTRLEN); - PM_WRITE(PM_DSI0, 0); - return 0; - - case BCM2835_POWER_DOMAIN_DSI1: - PM_WRITE(PM_DSI1, PM_DSI1_CTRLEN); - PM_WRITE(PM_DSI1, 0); - return 0; - - case BCM2835_POWER_DOMAIN_CCP2TX: - PM_WRITE(PM_CCP2TX, PM_CCP2TX_CTRLEN); - PM_WRITE(PM_CCP2TX, 0); - return 0; - - case BCM2835_POWER_DOMAIN_HDMI: - PM_WRITE(PM_HDMI, PM_READ(PM_HDMI) | PM_HDMI_LDOPD); - PM_WRITE(PM_HDMI, PM_READ(PM_HDMI) & ~PM_HDMI_CTRLEN); - return 0; - - default: - dev_err(power->dev, "Invalid domain %d\n", pd->domain); - return -EINVAL; - } -} - -static int -bcm2835_init_power_domain(struct bcm2835_power *power, - int pd_xlate_index, const char *name) -{ - struct device *dev = power->dev; - struct bcm2835_power_domain *dom = &power->domains[pd_xlate_index]; - - dom->clk = devm_clk_get(dev->parent, name); - if (IS_ERR(dom->clk)) { - int ret = PTR_ERR(dom->clk); - - if (ret == -EPROBE_DEFER) - return ret; - - /* Some domains don't have a clk, so make sure that we - * don't deref an error pointer later. - */ - dom->clk = NULL; - } - - dom->base.name = name; - dom->base.power_on = bcm2835_power_pd_power_on; - dom->base.power_off = bcm2835_power_pd_power_off; - - dom->domain = pd_xlate_index; - dom->power = power; - - /* XXX: on/off at boot? */ - pm_genpd_init(&dom->base, NULL, true); - - power->pd_xlate.domains[pd_xlate_index] = &dom->base; - - return 0; -} - -/** bcm2835_reset_reset - Resets a block that has a reset line in the - * PM block. - * - * The consumer of the reset controller must have the power domain up - * -- there's no reset ability with the power domain down. To reset - * the sub-block, we just disable its access to memory through the - * ASB, reset, and re-enable. - */ -static int bcm2835_reset_reset(struct reset_controller_dev *rcdev, - unsigned long id) -{ - struct bcm2835_power *power = container_of(rcdev, struct bcm2835_power, - reset); - struct bcm2835_power_domain *pd; - int ret; - - switch (id) { - case BCM2835_RESET_V3D: - pd = &power->domains[BCM2835_POWER_DOMAIN_GRAFX_V3D]; - break; - case BCM2835_RESET_H264: - pd = &power->domains[BCM2835_POWER_DOMAIN_IMAGE_H264]; - break; - case BCM2835_RESET_ISP: - pd = &power->domains[BCM2835_POWER_DOMAIN_IMAGE_ISP]; - break; - default: - dev_err(power->dev, "Bad reset id %ld\n", id); - return -EINVAL; - } - - ret = bcm2835_power_pd_power_off(&pd->base); - if (ret) - return ret; - - return bcm2835_power_pd_power_on(&pd->base); -} - -static int bcm2835_reset_status(struct reset_controller_dev *rcdev, - unsigned long id) -{ - struct bcm2835_power *power = container_of(rcdev, struct bcm2835_power, - reset); - - switch (id) { - case BCM2835_RESET_V3D: - return !PM_READ(PM_GRAFX & PM_V3DRSTN); - case BCM2835_RESET_H264: - return !PM_READ(PM_IMAGE & PM_H264RSTN); - case BCM2835_RESET_ISP: - return !PM_READ(PM_IMAGE & PM_ISPRSTN); - default: - return -EINVAL; - } -} - -static const struct reset_control_ops bcm2835_reset_ops = { - .reset = bcm2835_reset_reset, - .status = bcm2835_reset_status, -}; - -static const char *const power_domain_names[] = { - [BCM2835_POWER_DOMAIN_GRAFX] = "grafx", - [BCM2835_POWER_DOMAIN_GRAFX_V3D] = "v3d", - - [BCM2835_POWER_DOMAIN_IMAGE] = "image", - [BCM2835_POWER_DOMAIN_IMAGE_PERI] = "peri_image", - [BCM2835_POWER_DOMAIN_IMAGE_H264] = "h264", - [BCM2835_POWER_DOMAIN_IMAGE_ISP] = "isp", - - [BCM2835_POWER_DOMAIN_USB] = "usb", - [BCM2835_POWER_DOMAIN_DSI0] = "dsi0", - [BCM2835_POWER_DOMAIN_DSI1] = "dsi1", - [BCM2835_POWER_DOMAIN_CAM0] = "cam0", - [BCM2835_POWER_DOMAIN_CAM1] = "cam1", - [BCM2835_POWER_DOMAIN_CCP2TX] = "ccp2tx", - [BCM2835_POWER_DOMAIN_HDMI] = "hdmi", -}; - -static int bcm2835_power_probe(struct platform_device *pdev) -{ - struct bcm2835_pm *pm = dev_get_drvdata(pdev->dev.parent); - struct device *dev = &pdev->dev; - struct bcm2835_power *power; - static const struct { - int parent, child; - } domain_deps[] = { - { BCM2835_POWER_DOMAIN_GRAFX, BCM2835_POWER_DOMAIN_GRAFX_V3D }, - { BCM2835_POWER_DOMAIN_IMAGE, BCM2835_POWER_DOMAIN_IMAGE_PERI }, - { BCM2835_POWER_DOMAIN_IMAGE, BCM2835_POWER_DOMAIN_IMAGE_H264 }, - { BCM2835_POWER_DOMAIN_IMAGE, BCM2835_POWER_DOMAIN_IMAGE_ISP }, - { BCM2835_POWER_DOMAIN_IMAGE_PERI, BCM2835_POWER_DOMAIN_USB }, - { BCM2835_POWER_DOMAIN_IMAGE_PERI, BCM2835_POWER_DOMAIN_CAM0 }, - { BCM2835_POWER_DOMAIN_IMAGE_PERI, BCM2835_POWER_DOMAIN_CAM1 }, - }; - int ret = 0, i; - u32 id; - - power = devm_kzalloc(dev, sizeof(*power), GFP_KERNEL); - if (!power) - return -ENOMEM; - platform_set_drvdata(pdev, power); - - power->dev = dev; - power->base = pm->base; - power->asb = pm->asb; - power->rpivid_asb = pm->rpivid_asb; - - id = readl(power->asb + ASB_AXI_BRDG_ID); - if (id != BCM2835_BRDG_ID /* "BRDG" */) { - dev_err(dev, "ASB register ID returned 0x%08x\n", id); - return -ENODEV; - } - - if (power->rpivid_asb) { - id = readl(power->rpivid_asb + ASB_AXI_BRDG_ID); - if (id != BCM2835_BRDG_ID /* "BRDG" */) { - dev_err(dev, "RPiVid ASB register ID returned 0x%08x\n", - id); - return -ENODEV; - } - } - - power->pd_xlate.domains = devm_kcalloc(dev, - ARRAY_SIZE(power_domain_names), - sizeof(*power->pd_xlate.domains), - GFP_KERNEL); - if (!power->pd_xlate.domains) - return -ENOMEM; - - power->pd_xlate.num_domains = ARRAY_SIZE(power_domain_names); - - for (i = 0; i < ARRAY_SIZE(power_domain_names); i++) { - ret = bcm2835_init_power_domain(power, i, power_domain_names[i]); - if (ret) - goto fail; - } - - for (i = 0; i < ARRAY_SIZE(domain_deps); i++) { - pm_genpd_add_subdomain(&power->domains[domain_deps[i].parent].base, - &power->domains[domain_deps[i].child].base); - } - - power->reset.owner = THIS_MODULE; - power->reset.nr_resets = BCM2835_RESET_COUNT; - power->reset.ops = &bcm2835_reset_ops; - power->reset.of_node = dev->parent->of_node; - - ret = devm_reset_controller_register(dev, &power->reset); - if (ret) - goto fail; - - of_genpd_add_provider_onecell(dev->parent->of_node, &power->pd_xlate); - - dev_info(dev, "Broadcom BCM2835 power domains driver"); - return 0; - -fail: - for (i = 0; i < ARRAY_SIZE(power_domain_names); i++) { - struct generic_pm_domain *dom = &power->domains[i].base; - - if (dom->name) - pm_genpd_remove(dom); - } - return ret; -} - -static struct platform_driver bcm2835_power_driver = { - .probe = bcm2835_power_probe, - .driver = { - .name = "bcm2835-power", - }, -}; -module_platform_driver(bcm2835_power_driver); - -MODULE_AUTHOR("Eric Anholt "); -MODULE_DESCRIPTION("Driver for Broadcom BCM2835 PM power domains and reset"); diff --git a/drivers/genpd/bcm/bcm63xx-power.c b/drivers/genpd/bcm/bcm63xx-power.c deleted file mode 100644 index 98b0c2430dbc..000000000000 --- a/drivers/genpd/bcm/bcm63xx-power.c +++ /dev/null @@ -1,375 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0-or-later -/* - * BCM63xx Power Domain Controller Driver - * - * Copyright (C) 2020 Álvaro Fernández Rojas - */ - -#include -#include -#include -#include -#include -#include -#include -#include -#include - -struct bcm63xx_power_dev { - struct generic_pm_domain genpd; - struct bcm63xx_power *power; - uint32_t mask; -}; - -struct bcm63xx_power { - void __iomem *base; - spinlock_t lock; - struct bcm63xx_power_dev *dev; - struct genpd_onecell_data genpd_data; - struct generic_pm_domain **genpd; -}; - -struct bcm63xx_power_data { - const char * const name; - uint8_t bit; - unsigned int flags; -}; - -static int bcm63xx_power_get_state(struct bcm63xx_power_dev *pmd, bool *is_on) -{ - struct bcm63xx_power *power = pmd->power; - - if (!pmd->mask) { - *is_on = false; - return -EINVAL; - } - - *is_on = !(__raw_readl(power->base) & pmd->mask); - - return 0; -} - -static int bcm63xx_power_set_state(struct bcm63xx_power_dev *pmd, bool on) -{ - struct bcm63xx_power *power = pmd->power; - unsigned long flags; - uint32_t val; - - if (!pmd->mask) - return -EINVAL; - - spin_lock_irqsave(&power->lock, flags); - val = __raw_readl(power->base); - if (on) - val &= ~pmd->mask; - else - val |= pmd->mask; - __raw_writel(val, power->base); - spin_unlock_irqrestore(&power->lock, flags); - - return 0; -} - -static int bcm63xx_power_on(struct generic_pm_domain *genpd) -{ - struct bcm63xx_power_dev *pmd = container_of(genpd, - struct bcm63xx_power_dev, genpd); - - return bcm63xx_power_set_state(pmd, true); -} - -static int bcm63xx_power_off(struct generic_pm_domain *genpd) -{ - struct bcm63xx_power_dev *pmd = container_of(genpd, - struct bcm63xx_power_dev, genpd); - - return bcm63xx_power_set_state(pmd, false); -} - -static int bcm63xx_power_probe(struct platform_device *pdev) -{ - struct device *dev = &pdev->dev; - struct device_node *np = dev->of_node; - const struct bcm63xx_power_data *entry, *table; - struct bcm63xx_power *power; - unsigned int ndom; - uint8_t max_bit = 0; - int ret; - - power = devm_kzalloc(dev, sizeof(*power), GFP_KERNEL); - if (!power) - return -ENOMEM; - - power->base = devm_platform_ioremap_resource(pdev, 0); - if (IS_ERR(power->base)) - return PTR_ERR(power->base); - - table = of_device_get_match_data(dev); - if (!table) - return -EINVAL; - - power->genpd_data.num_domains = 0; - ndom = 0; - for (entry = table; entry->name; entry++) { - max_bit = max(max_bit, entry->bit); - ndom++; - } - - if (!ndom) - return -ENODEV; - - power->genpd_data.num_domains = max_bit + 1; - - power->dev = devm_kcalloc(dev, power->genpd_data.num_domains, - sizeof(struct bcm63xx_power_dev), - GFP_KERNEL); - if (!power->dev) - return -ENOMEM; - - power->genpd = devm_kcalloc(dev, power->genpd_data.num_domains, - sizeof(struct generic_pm_domain *), - GFP_KERNEL); - if (!power->genpd) - return -ENOMEM; - - power->genpd_data.domains = power->genpd; - - ndom = 0; - for (entry = table; entry->name; entry++) { - struct bcm63xx_power_dev *pmd = &power->dev[ndom]; - bool is_on; - - pmd->power = power; - pmd->mask = BIT(entry->bit); - pmd->genpd.name = entry->name; - pmd->genpd.flags = entry->flags; - - ret = bcm63xx_power_get_state(pmd, &is_on); - if (ret) - dev_warn(dev, "unable to get current state for %s\n", - pmd->genpd.name); - - pmd->genpd.power_on = bcm63xx_power_on; - pmd->genpd.power_off = bcm63xx_power_off; - - pm_genpd_init(&pmd->genpd, NULL, !is_on); - power->genpd[entry->bit] = &pmd->genpd; - - ndom++; - } - - spin_lock_init(&power->lock); - - ret = of_genpd_add_provider_onecell(np, &power->genpd_data); - if (ret) { - dev_err(dev, "failed to register genpd driver: %d\n", ret); - return ret; - } - - dev_info(dev, "registered %u power domains\n", ndom); - - return 0; -} - -static const struct bcm63xx_power_data bcm6318_power_domains[] = { - { - .name = "pcie", - .bit = BCM6318_POWER_DOMAIN_PCIE, - }, { - .name = "usb", - .bit = BCM6318_POWER_DOMAIN_USB, - }, { - .name = "ephy0", - .bit = BCM6318_POWER_DOMAIN_EPHY0, - }, { - .name = "ephy1", - .bit = BCM6318_POWER_DOMAIN_EPHY1, - }, { - .name = "ephy2", - .bit = BCM6318_POWER_DOMAIN_EPHY2, - }, { - .name = "ephy3", - .bit = BCM6318_POWER_DOMAIN_EPHY3, - }, { - .name = "ldo2p5", - .bit = BCM6318_POWER_DOMAIN_LDO2P5, - .flags = GENPD_FLAG_ALWAYS_ON, - }, { - .name = "ldo2p9", - .bit = BCM6318_POWER_DOMAIN_LDO2P9, - .flags = GENPD_FLAG_ALWAYS_ON, - }, { - .name = "sw1p0", - .bit = BCM6318_POWER_DOMAIN_SW1P0, - .flags = GENPD_FLAG_ALWAYS_ON, - }, { - .name = "pad", - .bit = BCM6318_POWER_DOMAIN_PAD, - .flags = GENPD_FLAG_ALWAYS_ON, - }, { - /* sentinel */ - }, -}; - -static const struct bcm63xx_power_data bcm6328_power_domains[] = { - { - .name = "adsl2-mips", - .bit = BCM6328_POWER_DOMAIN_ADSL2_MIPS, - }, { - .name = "adsl2-phy", - .bit = BCM6328_POWER_DOMAIN_ADSL2_PHY, - }, { - .name = "adsl2-afe", - .bit = BCM6328_POWER_DOMAIN_ADSL2_AFE, - }, { - .name = "sar", - .bit = BCM6328_POWER_DOMAIN_SAR, - }, { - .name = "pcm", - .bit = BCM6328_POWER_DOMAIN_PCM, - }, { - .name = "usbd", - .bit = BCM6328_POWER_DOMAIN_USBD, - }, { - .name = "usbh", - .bit = BCM6328_POWER_DOMAIN_USBH, - }, { - .name = "pcie", - .bit = BCM6328_POWER_DOMAIN_PCIE, - }, { - .name = "robosw", - .bit = BCM6328_POWER_DOMAIN_ROBOSW, - }, { - .name = "ephy", - .bit = BCM6328_POWER_DOMAIN_EPHY, - }, { - /* sentinel */ - }, -}; - -static const struct bcm63xx_power_data bcm6362_power_domains[] = { - { - .name = "sar", - .bit = BCM6362_POWER_DOMAIN_SAR, - }, { - .name = "ipsec", - .bit = BCM6362_POWER_DOMAIN_IPSEC, - }, { - .name = "mips", - .bit = BCM6362_POWER_DOMAIN_MIPS, - .flags = GENPD_FLAG_ALWAYS_ON, - }, { - .name = "dect", - .bit = BCM6362_POWER_DOMAIN_DECT, - }, { - .name = "usbh", - .bit = BCM6362_POWER_DOMAIN_USBH, - }, { - .name = "usbd", - .bit = BCM6362_POWER_DOMAIN_USBD, - }, { - .name = "robosw", - .bit = BCM6362_POWER_DOMAIN_ROBOSW, - }, { - .name = "pcm", - .bit = BCM6362_POWER_DOMAIN_PCM, - }, { - .name = "periph", - .bit = BCM6362_POWER_DOMAIN_PERIPH, - .flags = GENPD_FLAG_ALWAYS_ON, - }, { - .name = "adsl-phy", - .bit = BCM6362_POWER_DOMAIN_ADSL_PHY, - }, { - .name = "gmii-pads", - .bit = BCM6362_POWER_DOMAIN_GMII_PADS, - }, { - .name = "fap", - .bit = BCM6362_POWER_DOMAIN_FAP, - }, { - .name = "pcie", - .bit = BCM6362_POWER_DOMAIN_PCIE, - }, { - .name = "wlan-pads", - .bit = BCM6362_POWER_DOMAIN_WLAN_PADS, - }, { - /* sentinel */ - }, -}; - -static const struct bcm63xx_power_data bcm63268_power_domains[] = { - { - .name = "sar", - .bit = BCM63268_POWER_DOMAIN_SAR, - }, { - .name = "ipsec", - .bit = BCM63268_POWER_DOMAIN_IPSEC, - }, { - .name = "mips", - .bit = BCM63268_POWER_DOMAIN_MIPS, - .flags = GENPD_FLAG_ALWAYS_ON, - }, { - .name = "dect", - .bit = BCM63268_POWER_DOMAIN_DECT, - }, { - .name = "usbh", - .bit = BCM63268_POWER_DOMAIN_USBH, - }, { - .name = "usbd", - .bit = BCM63268_POWER_DOMAIN_USBD, - }, { - .name = "robosw", - .bit = BCM63268_POWER_DOMAIN_ROBOSW, - }, { - .name = "pcm", - .bit = BCM63268_POWER_DOMAIN_PCM, - }, { - .name = "periph", - .bit = BCM63268_POWER_DOMAIN_PERIPH, - .flags = GENPD_FLAG_ALWAYS_ON, - }, { - .name = "vdsl-phy", - .bit = BCM63268_POWER_DOMAIN_VDSL_PHY, - }, { - .name = "vdsl-mips", - .bit = BCM63268_POWER_DOMAIN_VDSL_MIPS, - }, { - .name = "fap", - .bit = BCM63268_POWER_DOMAIN_FAP, - }, { - .name = "pcie", - .bit = BCM63268_POWER_DOMAIN_PCIE, - }, { - .name = "wlan-pads", - .bit = BCM63268_POWER_DOMAIN_WLAN_PADS, - }, { - /* sentinel */ - }, -}; - -static const struct of_device_id bcm63xx_power_of_match[] = { - { - .compatible = "brcm,bcm6318-power-controller", - .data = &bcm6318_power_domains, - }, { - .compatible = "brcm,bcm6328-power-controller", - .data = &bcm6328_power_domains, - }, { - .compatible = "brcm,bcm6362-power-controller", - .data = &bcm6362_power_domains, - }, { - .compatible = "brcm,bcm63268-power-controller", - .data = &bcm63268_power_domains, - }, { - /* sentinel */ - } -}; - -static struct platform_driver bcm63xx_power_driver = { - .driver = { - .name = "bcm63xx-power-controller", - .of_match_table = bcm63xx_power_of_match, - }, - .probe = bcm63xx_power_probe, -}; -builtin_platform_driver(bcm63xx_power_driver); diff --git a/drivers/genpd/bcm/raspberrypi-power.c b/drivers/genpd/bcm/raspberrypi-power.c deleted file mode 100644 index 06196ebfe03b..000000000000 --- a/drivers/genpd/bcm/raspberrypi-power.c +++ /dev/null @@ -1,245 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* (C) 2015 Pengutronix, Alexander Aring - * - * Authors: - * Alexander Aring - * Eric Anholt - */ - -#include -#include -#include -#include -#include -#include - -/* - * Firmware indices for the old power domains interface. Only a few - * of them were actually implemented. - */ -#define RPI_OLD_POWER_DOMAIN_USB 3 -#define RPI_OLD_POWER_DOMAIN_V3D 10 - -struct rpi_power_domain { - u32 domain; - bool enabled; - bool old_interface; - struct generic_pm_domain base; - struct rpi_firmware *fw; -}; - -struct rpi_power_domains { - bool has_new_interface; - struct genpd_onecell_data xlate; - struct rpi_firmware *fw; - struct rpi_power_domain domains[RPI_POWER_DOMAIN_COUNT]; -}; - -/* - * Packet definition used by RPI_FIRMWARE_SET_POWER_STATE and - * RPI_FIRMWARE_SET_DOMAIN_STATE - */ -struct rpi_power_domain_packet { - u32 domain; - u32 on; -}; - -/* - * Asks the firmware to enable or disable power on a specific power - * domain. - */ -static int rpi_firmware_set_power(struct rpi_power_domain *rpi_domain, bool on) -{ - struct rpi_power_domain_packet packet; - - packet.domain = rpi_domain->domain; - packet.on = on; - return rpi_firmware_property(rpi_domain->fw, - rpi_domain->old_interface ? - RPI_FIRMWARE_SET_POWER_STATE : - RPI_FIRMWARE_SET_DOMAIN_STATE, - &packet, sizeof(packet)); -} - -static int rpi_domain_off(struct generic_pm_domain *domain) -{ - struct rpi_power_domain *rpi_domain = - container_of(domain, struct rpi_power_domain, base); - - return rpi_firmware_set_power(rpi_domain, false); -} - -static int rpi_domain_on(struct generic_pm_domain *domain) -{ - struct rpi_power_domain *rpi_domain = - container_of(domain, struct rpi_power_domain, base); - - return rpi_firmware_set_power(rpi_domain, true); -} - -static void rpi_common_init_power_domain(struct rpi_power_domains *rpi_domains, - int xlate_index, const char *name) -{ - struct rpi_power_domain *dom = &rpi_domains->domains[xlate_index]; - - dom->fw = rpi_domains->fw; - - dom->base.name = name; - dom->base.power_on = rpi_domain_on; - dom->base.power_off = rpi_domain_off; - - /* - * Treat all power domains as off at boot. - * - * The firmware itself may be keeping some domains on, but - * from Linux's perspective all we control is the refcounts - * that we give to the firmware, and we can't ask the firmware - * to turn off something that we haven't ourselves turned on. - */ - pm_genpd_init(&dom->base, NULL, true); - - rpi_domains->xlate.domains[xlate_index] = &dom->base; -} - -static void rpi_init_power_domain(struct rpi_power_domains *rpi_domains, - int xlate_index, const char *name) -{ - struct rpi_power_domain *dom = &rpi_domains->domains[xlate_index]; - - if (!rpi_domains->has_new_interface) - return; - - /* The DT binding index is the firmware's domain index minus one. */ - dom->domain = xlate_index + 1; - - rpi_common_init_power_domain(rpi_domains, xlate_index, name); -} - -static void rpi_init_old_power_domain(struct rpi_power_domains *rpi_domains, - int xlate_index, int domain, - const char *name) -{ - struct rpi_power_domain *dom = &rpi_domains->domains[xlate_index]; - - dom->old_interface = true; - dom->domain = domain; - - rpi_common_init_power_domain(rpi_domains, xlate_index, name); -} - -/* - * Detects whether the firmware supports the new power domains interface. - * - * The firmware doesn't actually return an error on an unknown tag, - * and just skips over it, so we do the detection by putting an - * unexpected value in the return field and checking if it was - * unchanged. - */ -static bool -rpi_has_new_domain_support(struct rpi_power_domains *rpi_domains) -{ - struct rpi_power_domain_packet packet; - int ret; - - packet.domain = RPI_POWER_DOMAIN_ARM; - packet.on = ~0; - - ret = rpi_firmware_property(rpi_domains->fw, - RPI_FIRMWARE_GET_DOMAIN_STATE, - &packet, sizeof(packet)); - - return ret == 0 && packet.on != ~0; -} - -static int rpi_power_probe(struct platform_device *pdev) -{ - struct device_node *fw_np; - struct device *dev = &pdev->dev; - struct rpi_power_domains *rpi_domains; - - rpi_domains = devm_kzalloc(dev, sizeof(*rpi_domains), GFP_KERNEL); - if (!rpi_domains) - return -ENOMEM; - - rpi_domains->xlate.domains = - devm_kcalloc(dev, - RPI_POWER_DOMAIN_COUNT, - sizeof(*rpi_domains->xlate.domains), - GFP_KERNEL); - if (!rpi_domains->xlate.domains) - return -ENOMEM; - - rpi_domains->xlate.num_domains = RPI_POWER_DOMAIN_COUNT; - - fw_np = of_parse_phandle(pdev->dev.of_node, "firmware", 0); - if (!fw_np) { - dev_err(&pdev->dev, "no firmware node\n"); - return -ENODEV; - } - - rpi_domains->fw = devm_rpi_firmware_get(&pdev->dev, fw_np); - of_node_put(fw_np); - if (!rpi_domains->fw) - return -EPROBE_DEFER; - - rpi_domains->has_new_interface = - rpi_has_new_domain_support(rpi_domains); - - rpi_init_power_domain(rpi_domains, RPI_POWER_DOMAIN_I2C0, "I2C0"); - rpi_init_power_domain(rpi_domains, RPI_POWER_DOMAIN_I2C1, "I2C1"); - rpi_init_power_domain(rpi_domains, RPI_POWER_DOMAIN_I2C2, "I2C2"); - rpi_init_power_domain(rpi_domains, RPI_POWER_DOMAIN_VIDEO_SCALER, - "VIDEO_SCALER"); - rpi_init_power_domain(rpi_domains, RPI_POWER_DOMAIN_VPU1, "VPU1"); - rpi_init_power_domain(rpi_domains, RPI_POWER_DOMAIN_HDMI, "HDMI"); - - /* - * Use the old firmware interface for USB power, so that we - * can turn it on even if the firmware hasn't been updated. - */ - rpi_init_old_power_domain(rpi_domains, RPI_POWER_DOMAIN_USB, - RPI_OLD_POWER_DOMAIN_USB, "USB"); - - rpi_init_power_domain(rpi_domains, RPI_POWER_DOMAIN_VEC, "VEC"); - rpi_init_power_domain(rpi_domains, RPI_POWER_DOMAIN_JPEG, "JPEG"); - rpi_init_power_domain(rpi_domains, RPI_POWER_DOMAIN_H264, "H264"); - rpi_init_power_domain(rpi_domains, RPI_POWER_DOMAIN_V3D, "V3D"); - rpi_init_power_domain(rpi_domains, RPI_POWER_DOMAIN_ISP, "ISP"); - rpi_init_power_domain(rpi_domains, RPI_POWER_DOMAIN_UNICAM0, "UNICAM0"); - rpi_init_power_domain(rpi_domains, RPI_POWER_DOMAIN_UNICAM1, "UNICAM1"); - rpi_init_power_domain(rpi_domains, RPI_POWER_DOMAIN_CCP2RX, "CCP2RX"); - rpi_init_power_domain(rpi_domains, RPI_POWER_DOMAIN_CSI2, "CSI2"); - rpi_init_power_domain(rpi_domains, RPI_POWER_DOMAIN_CPI, "CPI"); - rpi_init_power_domain(rpi_domains, RPI_POWER_DOMAIN_DSI0, "DSI0"); - rpi_init_power_domain(rpi_domains, RPI_POWER_DOMAIN_DSI1, "DSI1"); - rpi_init_power_domain(rpi_domains, RPI_POWER_DOMAIN_TRANSPOSER, - "TRANSPOSER"); - rpi_init_power_domain(rpi_domains, RPI_POWER_DOMAIN_CCP2TX, "CCP2TX"); - rpi_init_power_domain(rpi_domains, RPI_POWER_DOMAIN_CDP, "CDP"); - rpi_init_power_domain(rpi_domains, RPI_POWER_DOMAIN_ARM, "ARM"); - - of_genpd_add_provider_onecell(dev->of_node, &rpi_domains->xlate); - - platform_set_drvdata(pdev, rpi_domains); - - return 0; -} - -static const struct of_device_id rpi_power_of_match[] = { - { .compatible = "raspberrypi,bcm2835-power", }, - {}, -}; -MODULE_DEVICE_TABLE(of, rpi_power_of_match); - -static struct platform_driver rpi_power_driver = { - .driver = { - .name = "raspberrypi-power", - .of_match_table = rpi_power_of_match, - }, - .probe = rpi_power_probe, -}; -builtin_platform_driver(rpi_power_driver); - -MODULE_AUTHOR("Alexander Aring "); -MODULE_AUTHOR("Eric Anholt "); -MODULE_DESCRIPTION("Raspberry Pi power domain driver"); diff --git a/drivers/genpd/imx/Makefile b/drivers/genpd/imx/Makefile deleted file mode 100644 index 52d2629014a7..000000000000 --- a/drivers/genpd/imx/Makefile +++ /dev/null @@ -1,8 +0,0 @@ -# SPDX-License-Identifier: GPL-2.0-only -obj-$(CONFIG_HAVE_IMX_GPC) += gpc.o -obj-$(CONFIG_IMX_GPCV2_PM_DOMAINS) += gpcv2.o -obj-$(CONFIG_IMX_SCU_PD) += scu-pd.o -obj-$(CONFIG_IMX8M_BLK_CTRL) += imx8m-blk-ctrl.o -obj-$(CONFIG_IMX8M_BLK_CTRL) += imx8mp-blk-ctrl.o -obj-$(CONFIG_SOC_IMX9) += imx93-pd.o -obj-$(CONFIG_IMX9_BLK_CTRL) += imx93-blk-ctrl.o diff --git a/drivers/genpd/imx/gpc.c b/drivers/genpd/imx/gpc.c deleted file mode 100644 index 90a8b2c0676f..000000000000 --- a/drivers/genpd/imx/gpc.c +++ /dev/null @@ -1,554 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0+ -/* - * Copyright 2015-2017 Pengutronix, Lucas Stach - * Copyright 2011-2013 Freescale Semiconductor, Inc. - */ - -#include -#include -#include -#include -#include -#include -#include -#include - -#define GPC_CNTR 0x000 - -#define GPC_PGC_CTRL_OFFS 0x0 -#define GPC_PGC_PUPSCR_OFFS 0x4 -#define GPC_PGC_PDNSCR_OFFS 0x8 -#define GPC_PGC_SW2ISO_SHIFT 0x8 -#define GPC_PGC_SW_SHIFT 0x0 - -#define GPC_PGC_PCI_PDN 0x200 -#define GPC_PGC_PCI_SR 0x20c - -#define GPC_PGC_GPU_PDN 0x260 -#define GPC_PGC_GPU_PUPSCR 0x264 -#define GPC_PGC_GPU_PDNSCR 0x268 -#define GPC_PGC_GPU_SR 0x26c - -#define GPC_PGC_DISP_PDN 0x240 -#define GPC_PGC_DISP_SR 0x24c - -#define GPU_VPU_PUP_REQ BIT(1) -#define GPU_VPU_PDN_REQ BIT(0) - -#define GPC_CLK_MAX 7 - -#define PGC_DOMAIN_FLAG_NO_PD BIT(0) - -struct imx_pm_domain { - struct generic_pm_domain base; - struct regmap *regmap; - struct regulator *supply; - struct clk *clk[GPC_CLK_MAX]; - int num_clks; - unsigned int reg_offs; - signed char cntr_pdn_bit; - unsigned int ipg_rate_mhz; -}; - -static inline struct imx_pm_domain * -to_imx_pm_domain(struct generic_pm_domain *genpd) -{ - return container_of(genpd, struct imx_pm_domain, base); -} - -static int imx6_pm_domain_power_off(struct generic_pm_domain *genpd) -{ - struct imx_pm_domain *pd = to_imx_pm_domain(genpd); - int iso, iso2sw; - u32 val; - - /* Read ISO and ISO2SW power down delays */ - regmap_read(pd->regmap, pd->reg_offs + GPC_PGC_PDNSCR_OFFS, &val); - iso = val & 0x3f; - iso2sw = (val >> 8) & 0x3f; - - /* Gate off domain when powered down */ - regmap_update_bits(pd->regmap, pd->reg_offs + GPC_PGC_CTRL_OFFS, - 0x1, 0x1); - - /* Request GPC to power down domain */ - val = BIT(pd->cntr_pdn_bit); - regmap_update_bits(pd->regmap, GPC_CNTR, val, val); - - /* Wait ISO + ISO2SW IPG clock cycles */ - udelay(DIV_ROUND_UP(iso + iso2sw, pd->ipg_rate_mhz)); - - if (pd->supply) - regulator_disable(pd->supply); - - return 0; -} - -static int imx6_pm_domain_power_on(struct generic_pm_domain *genpd) -{ - struct imx_pm_domain *pd = to_imx_pm_domain(genpd); - int i, ret; - u32 val, req; - - if (pd->supply) { - ret = regulator_enable(pd->supply); - if (ret) { - pr_err("%s: failed to enable regulator: %d\n", - __func__, ret); - return ret; - } - } - - /* Enable reset clocks for all devices in the domain */ - for (i = 0; i < pd->num_clks; i++) - clk_prepare_enable(pd->clk[i]); - - /* Gate off domain when powered down */ - regmap_update_bits(pd->regmap, pd->reg_offs + GPC_PGC_CTRL_OFFS, - 0x1, 0x1); - - /* Request GPC to power up domain */ - req = BIT(pd->cntr_pdn_bit + 1); - regmap_update_bits(pd->regmap, GPC_CNTR, req, req); - - /* Wait for the PGC to handle the request */ - ret = regmap_read_poll_timeout(pd->regmap, GPC_CNTR, val, !(val & req), - 1, 50); - if (ret) - pr_err("powerup request on domain %s timed out\n", genpd->name); - - /* Wait for reset to propagate through peripherals */ - usleep_range(5, 10); - - /* Disable reset clocks for all devices in the domain */ - for (i = 0; i < pd->num_clks; i++) - clk_disable_unprepare(pd->clk[i]); - - return 0; -} - -static int imx_pgc_get_clocks(struct device *dev, struct imx_pm_domain *domain) -{ - int i, ret; - - for (i = 0; ; i++) { - struct clk *clk = of_clk_get(dev->of_node, i); - if (IS_ERR(clk)) - break; - if (i >= GPC_CLK_MAX) { - dev_err(dev, "more than %d clocks\n", GPC_CLK_MAX); - ret = -EINVAL; - goto clk_err; - } - domain->clk[i] = clk; - } - domain->num_clks = i; - - return 0; - -clk_err: - while (i--) - clk_put(domain->clk[i]); - - return ret; -} - -static void imx_pgc_put_clocks(struct imx_pm_domain *domain) -{ - int i; - - for (i = domain->num_clks - 1; i >= 0; i--) - clk_put(domain->clk[i]); -} - -static int imx_pgc_parse_dt(struct device *dev, struct imx_pm_domain *domain) -{ - /* try to get the domain supply regulator */ - domain->supply = devm_regulator_get_optional(dev, "power"); - if (IS_ERR(domain->supply)) { - if (PTR_ERR(domain->supply) == -ENODEV) - domain->supply = NULL; - else - return PTR_ERR(domain->supply); - } - - /* try to get all clocks needed for reset propagation */ - return imx_pgc_get_clocks(dev, domain); -} - -static int imx_pgc_power_domain_probe(struct platform_device *pdev) -{ - struct imx_pm_domain *domain = pdev->dev.platform_data; - struct device *dev = &pdev->dev; - int ret; - - /* if this PD is associated with a DT node try to parse it */ - if (dev->of_node) { - ret = imx_pgc_parse_dt(dev, domain); - if (ret) - return ret; - } - - /* initially power on the domain */ - if (domain->base.power_on) - domain->base.power_on(&domain->base); - - if (IS_ENABLED(CONFIG_PM_GENERIC_DOMAINS)) { - pm_genpd_init(&domain->base, NULL, false); - ret = of_genpd_add_provider_simple(dev->of_node, &domain->base); - if (ret) - goto genpd_err; - } - - device_link_add(dev, dev->parent, DL_FLAG_AUTOREMOVE_CONSUMER); - - return 0; - -genpd_err: - pm_genpd_remove(&domain->base); - imx_pgc_put_clocks(domain); - - return ret; -} - -static int imx_pgc_power_domain_remove(struct platform_device *pdev) -{ - struct imx_pm_domain *domain = pdev->dev.platform_data; - - if (IS_ENABLED(CONFIG_PM_GENERIC_DOMAINS)) { - of_genpd_del_provider(pdev->dev.of_node); - pm_genpd_remove(&domain->base); - imx_pgc_put_clocks(domain); - } - - return 0; -} - -static const struct platform_device_id imx_pgc_power_domain_id[] = { - { "imx-pgc-power-domain"}, - { }, -}; - -static struct platform_driver imx_pgc_power_domain_driver = { - .driver = { - .name = "imx-pgc-pd", - }, - .probe = imx_pgc_power_domain_probe, - .remove = imx_pgc_power_domain_remove, - .id_table = imx_pgc_power_domain_id, -}; -builtin_platform_driver(imx_pgc_power_domain_driver) - -#define GPC_PGC_DOMAIN_ARM 0 -#define GPC_PGC_DOMAIN_PU 1 -#define GPC_PGC_DOMAIN_DISPLAY 2 -#define GPC_PGC_DOMAIN_PCI 3 - -static struct genpd_power_state imx6_pm_domain_pu_state = { - .power_off_latency_ns = 25000, - .power_on_latency_ns = 2000000, -}; - -static struct imx_pm_domain imx_gpc_domains[] = { - [GPC_PGC_DOMAIN_ARM] = { - .base = { - .name = "ARM", - .flags = GENPD_FLAG_ALWAYS_ON, - }, - }, - [GPC_PGC_DOMAIN_PU] = { - .base = { - .name = "PU", - .power_off = imx6_pm_domain_power_off, - .power_on = imx6_pm_domain_power_on, - .states = &imx6_pm_domain_pu_state, - .state_count = 1, - }, - .reg_offs = 0x260, - .cntr_pdn_bit = 0, - }, - [GPC_PGC_DOMAIN_DISPLAY] = { - .base = { - .name = "DISPLAY", - .power_off = imx6_pm_domain_power_off, - .power_on = imx6_pm_domain_power_on, - }, - .reg_offs = 0x240, - .cntr_pdn_bit = 4, - }, - [GPC_PGC_DOMAIN_PCI] = { - .base = { - .name = "PCI", - .power_off = imx6_pm_domain_power_off, - .power_on = imx6_pm_domain_power_on, - }, - .reg_offs = 0x200, - .cntr_pdn_bit = 6, - }, -}; - -struct imx_gpc_dt_data { - int num_domains; - bool err009619_present; - bool err006287_present; -}; - -static const struct imx_gpc_dt_data imx6q_dt_data = { - .num_domains = 2, - .err009619_present = false, - .err006287_present = false, -}; - -static const struct imx_gpc_dt_data imx6qp_dt_data = { - .num_domains = 2, - .err009619_present = true, - .err006287_present = false, -}; - -static const struct imx_gpc_dt_data imx6sl_dt_data = { - .num_domains = 3, - .err009619_present = false, - .err006287_present = true, -}; - -static const struct imx_gpc_dt_data imx6sx_dt_data = { - .num_domains = 4, - .err009619_present = false, - .err006287_present = false, -}; - -static const struct of_device_id imx_gpc_dt_ids[] = { - { .compatible = "fsl,imx6q-gpc", .data = &imx6q_dt_data }, - { .compatible = "fsl,imx6qp-gpc", .data = &imx6qp_dt_data }, - { .compatible = "fsl,imx6sl-gpc", .data = &imx6sl_dt_data }, - { .compatible = "fsl,imx6sx-gpc", .data = &imx6sx_dt_data }, - { } -}; - -static const struct regmap_range yes_ranges[] = { - regmap_reg_range(GPC_CNTR, GPC_CNTR), - regmap_reg_range(GPC_PGC_PCI_PDN, GPC_PGC_PCI_SR), - regmap_reg_range(GPC_PGC_GPU_PDN, GPC_PGC_GPU_SR), - regmap_reg_range(GPC_PGC_DISP_PDN, GPC_PGC_DISP_SR), -}; - -static const struct regmap_access_table access_table = { - .yes_ranges = yes_ranges, - .n_yes_ranges = ARRAY_SIZE(yes_ranges), -}; - -static const struct regmap_config imx_gpc_regmap_config = { - .reg_bits = 32, - .val_bits = 32, - .reg_stride = 4, - .rd_table = &access_table, - .wr_table = &access_table, - .max_register = 0x2ac, - .fast_io = true, -}; - -static struct generic_pm_domain *imx_gpc_onecell_domains[] = { - &imx_gpc_domains[GPC_PGC_DOMAIN_ARM].base, - &imx_gpc_domains[GPC_PGC_DOMAIN_PU].base, -}; - -static struct genpd_onecell_data imx_gpc_onecell_data = { - .domains = imx_gpc_onecell_domains, - .num_domains = 2, -}; - -static int imx_gpc_old_dt_init(struct device *dev, struct regmap *regmap, - unsigned int num_domains) -{ - struct imx_pm_domain *domain; - int i, ret; - - for (i = 0; i < num_domains; i++) { - domain = &imx_gpc_domains[i]; - domain->regmap = regmap; - domain->ipg_rate_mhz = 66; - - if (i == 1) { - domain->supply = devm_regulator_get(dev, "pu"); - if (IS_ERR(domain->supply)) - return PTR_ERR(domain->supply); - - ret = imx_pgc_get_clocks(dev, domain); - if (ret) - goto clk_err; - - domain->base.power_on(&domain->base); - } - } - - for (i = 0; i < num_domains; i++) - pm_genpd_init(&imx_gpc_domains[i].base, NULL, false); - - if (IS_ENABLED(CONFIG_PM_GENERIC_DOMAINS)) { - ret = of_genpd_add_provider_onecell(dev->of_node, - &imx_gpc_onecell_data); - if (ret) - goto genpd_err; - } - - return 0; - -genpd_err: - for (i = 0; i < num_domains; i++) - pm_genpd_remove(&imx_gpc_domains[i].base); - imx_pgc_put_clocks(&imx_gpc_domains[GPC_PGC_DOMAIN_PU]); -clk_err: - return ret; -} - -static int imx_gpc_probe(struct platform_device *pdev) -{ - const struct of_device_id *of_id = - of_match_device(imx_gpc_dt_ids, &pdev->dev); - const struct imx_gpc_dt_data *of_id_data = of_id->data; - struct device_node *pgc_node; - struct regmap *regmap; - void __iomem *base; - int ret; - - pgc_node = of_get_child_by_name(pdev->dev.of_node, "pgc"); - - /* bail out if DT too old and doesn't provide the necessary info */ - if (!of_property_read_bool(pdev->dev.of_node, "#power-domain-cells") && - !pgc_node) - return 0; - - base = devm_platform_ioremap_resource(pdev, 0); - if (IS_ERR(base)) - return PTR_ERR(base); - - regmap = devm_regmap_init_mmio_clk(&pdev->dev, NULL, base, - &imx_gpc_regmap_config); - if (IS_ERR(regmap)) { - ret = PTR_ERR(regmap); - dev_err(&pdev->dev, "failed to init regmap: %d\n", - ret); - return ret; - } - - /* - * Disable PU power down by runtime PM if ERR009619 is present. - * - * The PRE clock will be paused for several cycles when turning on the - * PU domain LDO from power down state. If PRE is in use at that time, - * the IPU/PRG cannot get the correct display data from the PRE. - * - * This is not a concern when the whole system enters suspend state, so - * it's safe to power down PU in this case. - */ - if (of_id_data->err009619_present) - imx_gpc_domains[GPC_PGC_DOMAIN_PU].base.flags |= - GENPD_FLAG_RPM_ALWAYS_ON; - - /* Keep DISP always on if ERR006287 is present */ - if (of_id_data->err006287_present) - imx_gpc_domains[GPC_PGC_DOMAIN_DISPLAY].base.flags |= - GENPD_FLAG_ALWAYS_ON; - - if (!pgc_node) { - ret = imx_gpc_old_dt_init(&pdev->dev, regmap, - of_id_data->num_domains); - if (ret) - return ret; - } else { - struct imx_pm_domain *domain; - struct platform_device *pd_pdev; - struct device_node *np; - struct clk *ipg_clk; - unsigned int ipg_rate_mhz; - int domain_index; - - ipg_clk = devm_clk_get(&pdev->dev, "ipg"); - if (IS_ERR(ipg_clk)) - return PTR_ERR(ipg_clk); - ipg_rate_mhz = clk_get_rate(ipg_clk) / 1000000; - - for_each_child_of_node(pgc_node, np) { - ret = of_property_read_u32(np, "reg", &domain_index); - if (ret) { - of_node_put(np); - return ret; - } - if (domain_index >= of_id_data->num_domains) - continue; - - pd_pdev = platform_device_alloc("imx-pgc-power-domain", - domain_index); - if (!pd_pdev) { - of_node_put(np); - return -ENOMEM; - } - - ret = platform_device_add_data(pd_pdev, - &imx_gpc_domains[domain_index], - sizeof(imx_gpc_domains[domain_index])); - if (ret) { - platform_device_put(pd_pdev); - of_node_put(np); - return ret; - } - domain = pd_pdev->dev.platform_data; - domain->regmap = regmap; - domain->ipg_rate_mhz = ipg_rate_mhz; - - pd_pdev->dev.parent = &pdev->dev; - pd_pdev->dev.of_node = np; - - ret = platform_device_add(pd_pdev); - if (ret) { - platform_device_put(pd_pdev); - of_node_put(np); - return ret; - } - } - } - - return 0; -} - -static int imx_gpc_remove(struct platform_device *pdev) -{ - struct device_node *pgc_node; - int ret; - - pgc_node = of_get_child_by_name(pdev->dev.of_node, "pgc"); - - /* bail out if DT too old and doesn't provide the necessary info */ - if (!of_property_read_bool(pdev->dev.of_node, "#power-domain-cells") && - !pgc_node) - return 0; - - /* - * If the old DT binding is used the toplevel driver needs to - * de-register the power domains - */ - if (!pgc_node) { - of_genpd_del_provider(pdev->dev.of_node); - - ret = pm_genpd_remove(&imx_gpc_domains[GPC_PGC_DOMAIN_PU].base); - if (ret) - return ret; - imx_pgc_put_clocks(&imx_gpc_domains[GPC_PGC_DOMAIN_PU]); - - ret = pm_genpd_remove(&imx_gpc_domains[GPC_PGC_DOMAIN_ARM].base); - if (ret) - return ret; - } - - return 0; -} - -static struct platform_driver imx_gpc_driver = { - .driver = { - .name = "imx-gpc", - .of_match_table = imx_gpc_dt_ids, - }, - .probe = imx_gpc_probe, - .remove = imx_gpc_remove, -}; -builtin_platform_driver(imx_gpc_driver) diff --git a/drivers/genpd/imx/gpcv2.c b/drivers/genpd/imx/gpcv2.c deleted file mode 100644 index fbd3d92f8cd8..000000000000 --- a/drivers/genpd/imx/gpcv2.c +++ /dev/null @@ -1,1550 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0+ -/* - * Copyright 2017 Impinj, Inc - * Author: Andrey Smirnov - * - * Based on the code of analogus driver: - * - * Copyright 2015-2017 Pengutronix, Lucas Stach - */ - -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -#define GPC_LPCR_A_CORE_BSC 0x000 - -#define GPC_PGC_CPU_MAPPING 0x0ec -#define IMX8MP_GPC_PGC_CPU_MAPPING 0x1cc - -#define IMX7_USB_HSIC_PHY_A_CORE_DOMAIN BIT(6) -#define IMX7_USB_OTG2_PHY_A_CORE_DOMAIN BIT(5) -#define IMX7_USB_OTG1_PHY_A_CORE_DOMAIN BIT(4) -#define IMX7_PCIE_PHY_A_CORE_DOMAIN BIT(3) -#define IMX7_MIPI_PHY_A_CORE_DOMAIN BIT(2) - -#define IMX8M_PCIE2_A53_DOMAIN BIT(15) -#define IMX8M_MIPI_CSI2_A53_DOMAIN BIT(14) -#define IMX8M_MIPI_CSI1_A53_DOMAIN BIT(13) -#define IMX8M_DISP_A53_DOMAIN BIT(12) -#define IMX8M_HDMI_A53_DOMAIN BIT(11) -#define IMX8M_VPU_A53_DOMAIN BIT(10) -#define IMX8M_GPU_A53_DOMAIN BIT(9) -#define IMX8M_DDR2_A53_DOMAIN BIT(8) -#define IMX8M_DDR1_A53_DOMAIN BIT(7) -#define IMX8M_OTG2_A53_DOMAIN BIT(5) -#define IMX8M_OTG1_A53_DOMAIN BIT(4) -#define IMX8M_PCIE1_A53_DOMAIN BIT(3) -#define IMX8M_MIPI_A53_DOMAIN BIT(2) - -#define IMX8MM_VPUH1_A53_DOMAIN BIT(15) -#define IMX8MM_VPUG2_A53_DOMAIN BIT(14) -#define IMX8MM_VPUG1_A53_DOMAIN BIT(13) -#define IMX8MM_DISPMIX_A53_DOMAIN BIT(12) -#define IMX8MM_VPUMIX_A53_DOMAIN BIT(10) -#define IMX8MM_GPUMIX_A53_DOMAIN BIT(9) -#define IMX8MM_GPU_A53_DOMAIN (BIT(8) | BIT(11)) -#define IMX8MM_DDR1_A53_DOMAIN BIT(7) -#define IMX8MM_OTG2_A53_DOMAIN BIT(5) -#define IMX8MM_OTG1_A53_DOMAIN BIT(4) -#define IMX8MM_PCIE_A53_DOMAIN BIT(3) -#define IMX8MM_MIPI_A53_DOMAIN BIT(2) - -#define IMX8MN_DISPMIX_A53_DOMAIN BIT(12) -#define IMX8MN_GPUMIX_A53_DOMAIN BIT(9) -#define IMX8MN_DDR1_A53_DOMAIN BIT(7) -#define IMX8MN_OTG1_A53_DOMAIN BIT(4) -#define IMX8MN_MIPI_A53_DOMAIN BIT(2) - -#define IMX8MP_MEDIA_ISPDWP_A53_DOMAIN BIT(20) -#define IMX8MP_HSIOMIX_A53_DOMAIN BIT(19) -#define IMX8MP_MIPI_PHY2_A53_DOMAIN BIT(18) -#define IMX8MP_HDMI_PHY_A53_DOMAIN BIT(17) -#define IMX8MP_HDMIMIX_A53_DOMAIN BIT(16) -#define IMX8MP_VPU_VC8000E_A53_DOMAIN BIT(15) -#define IMX8MP_VPU_G2_A53_DOMAIN BIT(14) -#define IMX8MP_VPU_G1_A53_DOMAIN BIT(13) -#define IMX8MP_MEDIAMIX_A53_DOMAIN BIT(12) -#define IMX8MP_GPU3D_A53_DOMAIN BIT(11) -#define IMX8MP_VPUMIX_A53_DOMAIN BIT(10) -#define IMX8MP_GPUMIX_A53_DOMAIN BIT(9) -#define IMX8MP_GPU2D_A53_DOMAIN BIT(8) -#define IMX8MP_AUDIOMIX_A53_DOMAIN BIT(7) -#define IMX8MP_MLMIX_A53_DOMAIN BIT(6) -#define IMX8MP_USB2_PHY_A53_DOMAIN BIT(5) -#define IMX8MP_USB1_PHY_A53_DOMAIN BIT(4) -#define IMX8MP_PCIE_PHY_A53_DOMAIN BIT(3) -#define IMX8MP_MIPI_PHY1_A53_DOMAIN BIT(2) - -#define IMX8MP_GPC_PU_PGC_SW_PUP_REQ 0x0d8 -#define IMX8MP_GPC_PU_PGC_SW_PDN_REQ 0x0e4 - -#define GPC_PU_PGC_SW_PUP_REQ 0x0f8 -#define GPC_PU_PGC_SW_PDN_REQ 0x104 - -#define IMX7_USB_HSIC_PHY_SW_Pxx_REQ BIT(4) -#define IMX7_USB_OTG2_PHY_SW_Pxx_REQ BIT(3) -#define IMX7_USB_OTG1_PHY_SW_Pxx_REQ BIT(2) -#define IMX7_PCIE_PHY_SW_Pxx_REQ BIT(1) -#define IMX7_MIPI_PHY_SW_Pxx_REQ BIT(0) - -#define IMX8M_PCIE2_SW_Pxx_REQ BIT(13) -#define IMX8M_MIPI_CSI2_SW_Pxx_REQ BIT(12) -#define IMX8M_MIPI_CSI1_SW_Pxx_REQ BIT(11) -#define IMX8M_DISP_SW_Pxx_REQ BIT(10) -#define IMX8M_HDMI_SW_Pxx_REQ BIT(9) -#define IMX8M_VPU_SW_Pxx_REQ BIT(8) -#define IMX8M_GPU_SW_Pxx_REQ BIT(7) -#define IMX8M_DDR2_SW_Pxx_REQ BIT(6) -#define IMX8M_DDR1_SW_Pxx_REQ BIT(5) -#define IMX8M_OTG2_SW_Pxx_REQ BIT(3) -#define IMX8M_OTG1_SW_Pxx_REQ BIT(2) -#define IMX8M_PCIE1_SW_Pxx_REQ BIT(1) -#define IMX8M_MIPI_SW_Pxx_REQ BIT(0) - -#define IMX8MM_VPUH1_SW_Pxx_REQ BIT(13) -#define IMX8MM_VPUG2_SW_Pxx_REQ BIT(12) -#define IMX8MM_VPUG1_SW_Pxx_REQ BIT(11) -#define IMX8MM_DISPMIX_SW_Pxx_REQ BIT(10) -#define IMX8MM_VPUMIX_SW_Pxx_REQ BIT(8) -#define IMX8MM_GPUMIX_SW_Pxx_REQ BIT(7) -#define IMX8MM_GPU_SW_Pxx_REQ (BIT(6) | BIT(9)) -#define IMX8MM_DDR1_SW_Pxx_REQ BIT(5) -#define IMX8MM_OTG2_SW_Pxx_REQ BIT(3) -#define IMX8MM_OTG1_SW_Pxx_REQ BIT(2) -#define IMX8MM_PCIE_SW_Pxx_REQ BIT(1) -#define IMX8MM_MIPI_SW_Pxx_REQ BIT(0) - -#define IMX8MN_DISPMIX_SW_Pxx_REQ BIT(10) -#define IMX8MN_GPUMIX_SW_Pxx_REQ BIT(7) -#define IMX8MN_DDR1_SW_Pxx_REQ BIT(5) -#define IMX8MN_OTG1_SW_Pxx_REQ BIT(2) -#define IMX8MN_MIPI_SW_Pxx_REQ BIT(0) - -#define IMX8MP_DDRMIX_Pxx_REQ BIT(19) -#define IMX8MP_MEDIA_ISP_DWP_Pxx_REQ BIT(18) -#define IMX8MP_HSIOMIX_Pxx_REQ BIT(17) -#define IMX8MP_MIPI_PHY2_Pxx_REQ BIT(16) -#define IMX8MP_HDMI_PHY_Pxx_REQ BIT(15) -#define IMX8MP_HDMIMIX_Pxx_REQ BIT(14) -#define IMX8MP_VPU_VC8K_Pxx_REQ BIT(13) -#define IMX8MP_VPU_G2_Pxx_REQ BIT(12) -#define IMX8MP_VPU_G1_Pxx_REQ BIT(11) -#define IMX8MP_MEDIMIX_Pxx_REQ BIT(10) -#define IMX8MP_GPU_3D_Pxx_REQ BIT(9) -#define IMX8MP_VPU_MIX_SHARE_LOGIC_Pxx_REQ BIT(8) -#define IMX8MP_GPU_SHARE_LOGIC_Pxx_REQ BIT(7) -#define IMX8MP_GPU_2D_Pxx_REQ BIT(6) -#define IMX8MP_AUDIOMIX_Pxx_REQ BIT(5) -#define IMX8MP_MLMIX_Pxx_REQ BIT(4) -#define IMX8MP_USB2_PHY_Pxx_REQ BIT(3) -#define IMX8MP_USB1_PHY_Pxx_REQ BIT(2) -#define IMX8MP_PCIE_PHY_SW_Pxx_REQ BIT(1) -#define IMX8MP_MIPI_PHY1_SW_Pxx_REQ BIT(0) - -#define GPC_M4_PU_PDN_FLG 0x1bc - -#define IMX8MP_GPC_PU_PWRHSK 0x190 -#define GPC_PU_PWRHSK 0x1fc - -#define IMX8M_GPU_HSK_PWRDNACKN BIT(26) -#define IMX8M_VPU_HSK_PWRDNACKN BIT(25) -#define IMX8M_DISP_HSK_PWRDNACKN BIT(24) -#define IMX8M_GPU_HSK_PWRDNREQN BIT(6) -#define IMX8M_VPU_HSK_PWRDNREQN BIT(5) -#define IMX8M_DISP_HSK_PWRDNREQN BIT(4) - -#define IMX8MM_GPUMIX_HSK_PWRDNACKN BIT(29) -#define IMX8MM_GPU_HSK_PWRDNACKN (BIT(27) | BIT(28)) -#define IMX8MM_VPUMIX_HSK_PWRDNACKN BIT(26) -#define IMX8MM_DISPMIX_HSK_PWRDNACKN BIT(25) -#define IMX8MM_HSIO_HSK_PWRDNACKN (BIT(23) | BIT(24)) -#define IMX8MM_GPUMIX_HSK_PWRDNREQN BIT(11) -#define IMX8MM_GPU_HSK_PWRDNREQN (BIT(9) | BIT(10)) -#define IMX8MM_VPUMIX_HSK_PWRDNREQN BIT(8) -#define IMX8MM_DISPMIX_HSK_PWRDNREQN BIT(7) -#define IMX8MM_HSIO_HSK_PWRDNREQN (BIT(5) | BIT(6)) - -#define IMX8MN_GPUMIX_HSK_PWRDNACKN (BIT(29) | BIT(27)) -#define IMX8MN_DISPMIX_HSK_PWRDNACKN BIT(25) -#define IMX8MN_HSIO_HSK_PWRDNACKN BIT(23) -#define IMX8MN_GPUMIX_HSK_PWRDNREQN (BIT(11) | BIT(9)) -#define IMX8MN_DISPMIX_HSK_PWRDNREQN BIT(7) -#define IMX8MN_HSIO_HSK_PWRDNREQN BIT(5) - -#define IMX8MP_MEDIAMIX_PWRDNACKN BIT(30) -#define IMX8MP_HDMIMIX_PWRDNACKN BIT(29) -#define IMX8MP_HSIOMIX_PWRDNACKN BIT(28) -#define IMX8MP_VPUMIX_PWRDNACKN BIT(26) -#define IMX8MP_GPUMIX_PWRDNACKN BIT(25) -#define IMX8MP_MLMIX_PWRDNACKN (BIT(23) | BIT(24)) -#define IMX8MP_AUDIOMIX_PWRDNACKN (BIT(20) | BIT(31)) -#define IMX8MP_MEDIAMIX_PWRDNREQN BIT(14) -#define IMX8MP_HDMIMIX_PWRDNREQN BIT(13) -#define IMX8MP_HSIOMIX_PWRDNREQN BIT(12) -#define IMX8MP_VPUMIX_PWRDNREQN BIT(10) -#define IMX8MP_GPUMIX_PWRDNREQN BIT(9) -#define IMX8MP_MLMIX_PWRDNREQN (BIT(7) | BIT(8)) -#define IMX8MP_AUDIOMIX_PWRDNREQN (BIT(4) | BIT(15)) - -/* - * The PGC offset values in Reference Manual - * (Rev. 1, 01/2018 and the older ones) GPC chapter's - * GPC_PGC memory map are incorrect, below offset - * values are from design RTL. - */ -#define IMX7_PGC_MIPI 16 -#define IMX7_PGC_PCIE 17 -#define IMX7_PGC_USB_HSIC 20 - -#define IMX8M_PGC_MIPI 16 -#define IMX8M_PGC_PCIE1 17 -#define IMX8M_PGC_OTG1 18 -#define IMX8M_PGC_OTG2 19 -#define IMX8M_PGC_DDR1 21 -#define IMX8M_PGC_GPU 23 -#define IMX8M_PGC_VPU 24 -#define IMX8M_PGC_DISP 26 -#define IMX8M_PGC_MIPI_CSI1 27 -#define IMX8M_PGC_MIPI_CSI2 28 -#define IMX8M_PGC_PCIE2 29 - -#define IMX8MM_PGC_MIPI 16 -#define IMX8MM_PGC_PCIE 17 -#define IMX8MM_PGC_OTG1 18 -#define IMX8MM_PGC_OTG2 19 -#define IMX8MM_PGC_DDR1 21 -#define IMX8MM_PGC_GPU2D 22 -#define IMX8MM_PGC_GPUMIX 23 -#define IMX8MM_PGC_VPUMIX 24 -#define IMX8MM_PGC_GPU3D 25 -#define IMX8MM_PGC_DISPMIX 26 -#define IMX8MM_PGC_VPUG1 27 -#define IMX8MM_PGC_VPUG2 28 -#define IMX8MM_PGC_VPUH1 29 - -#define IMX8MN_PGC_MIPI 16 -#define IMX8MN_PGC_OTG1 18 -#define IMX8MN_PGC_DDR1 21 -#define IMX8MN_PGC_GPUMIX 23 -#define IMX8MN_PGC_DISPMIX 26 - -#define IMX8MP_PGC_NOC 9 -#define IMX8MP_PGC_MIPI1 12 -#define IMX8MP_PGC_PCIE 13 -#define IMX8MP_PGC_USB1 14 -#define IMX8MP_PGC_USB2 15 -#define IMX8MP_PGC_MLMIX 16 -#define IMX8MP_PGC_AUDIOMIX 17 -#define IMX8MP_PGC_GPU2D 18 -#define IMX8MP_PGC_GPUMIX 19 -#define IMX8MP_PGC_VPUMIX 20 -#define IMX8MP_PGC_GPU3D 21 -#define IMX8MP_PGC_MEDIAMIX 22 -#define IMX8MP_PGC_VPU_G1 23 -#define IMX8MP_PGC_VPU_G2 24 -#define IMX8MP_PGC_VPU_VC8000E 25 -#define IMX8MP_PGC_HDMIMIX 26 -#define IMX8MP_PGC_HDMI 27 -#define IMX8MP_PGC_MIPI2 28 -#define IMX8MP_PGC_HSIOMIX 29 -#define IMX8MP_PGC_MEDIA_ISP_DWP 30 -#define IMX8MP_PGC_DDRMIX 31 - -#define GPC_PGC_CTRL(n) (0x800 + (n) * 0x40) -#define GPC_PGC_SR(n) (GPC_PGC_CTRL(n) + 0xc) - -#define GPC_PGC_CTRL_PCR BIT(0) - -struct imx_pgc_regs { - u16 map; - u16 pup; - u16 pdn; - u16 hsk; -}; - -struct imx_pgc_domain { - struct generic_pm_domain genpd; - struct regmap *regmap; - const struct imx_pgc_regs *regs; - struct regulator *regulator; - struct reset_control *reset; - struct clk_bulk_data *clks; - int num_clks; - - unsigned long pgc; - - const struct { - u32 pxx; - u32 map; - u32 hskreq; - u32 hskack; - } bits; - - const int voltage; - const bool keep_clocks; - struct device *dev; - - unsigned int pgc_sw_pup_reg; - unsigned int pgc_sw_pdn_reg; -}; - -struct imx_pgc_domain_data { - const struct imx_pgc_domain *domains; - size_t domains_num; - const struct regmap_access_table *reg_access_table; - const struct imx_pgc_regs *pgc_regs; -}; - -static inline struct imx_pgc_domain * -to_imx_pgc_domain(struct generic_pm_domain *genpd) -{ - return container_of(genpd, struct imx_pgc_domain, genpd); -} - -static int imx_pgc_power_up(struct generic_pm_domain *genpd) -{ - struct imx_pgc_domain *domain = to_imx_pgc_domain(genpd); - u32 reg_val, pgc; - int ret; - - ret = pm_runtime_get_sync(domain->dev); - if (ret < 0) { - pm_runtime_put_noidle(domain->dev); - return ret; - } - - if (!IS_ERR(domain->regulator)) { - ret = regulator_enable(domain->regulator); - if (ret) { - dev_err(domain->dev, - "failed to enable regulator: %pe\n", - ERR_PTR(ret)); - goto out_put_pm; - } - } - - reset_control_assert(domain->reset); - - /* Enable reset clocks for all devices in the domain */ - ret = clk_bulk_prepare_enable(domain->num_clks, domain->clks); - if (ret) { - dev_err(domain->dev, "failed to enable reset clocks\n"); - goto out_regulator_disable; - } - - /* delays for reset to propagate */ - udelay(5); - - if (domain->bits.pxx) { - /* request the domain to power up */ - regmap_update_bits(domain->regmap, domain->regs->pup, - domain->bits.pxx, domain->bits.pxx); - /* - * As per "5.5.9.4 Example Code 4" in IMX7DRM.pdf wait - * for PUP_REQ/PDN_REQ bit to be cleared - */ - ret = regmap_read_poll_timeout(domain->regmap, - domain->regs->pup, reg_val, - !(reg_val & domain->bits.pxx), - 0, USEC_PER_MSEC); - if (ret) { - dev_err(domain->dev, "failed to command PGC\n"); - goto out_clk_disable; - } - - /* disable power control */ - for_each_set_bit(pgc, &domain->pgc, 32) { - regmap_clear_bits(domain->regmap, GPC_PGC_CTRL(pgc), - GPC_PGC_CTRL_PCR); - } - } - - /* delay for reset to propagate */ - udelay(5); - - reset_control_deassert(domain->reset); - - /* request the ADB400 to power up */ - if (domain->bits.hskreq) { - regmap_update_bits(domain->regmap, domain->regs->hsk, - domain->bits.hskreq, domain->bits.hskreq); - - /* - * ret = regmap_read_poll_timeout(domain->regmap, domain->regs->hsk, reg_val, - * (reg_val & domain->bits.hskack), 0, - * USEC_PER_MSEC); - * Technically we need the commented code to wait handshake. But that needs - * the BLK-CTL module BUS clk-en bit being set. - * - * There is a separate BLK-CTL module and we will have such a driver for it, - * that driver will set the BUS clk-en bit and handshake will be triggered - * automatically there. Just add a delay and suppose the handshake finish - * after that. - */ - } - - /* Disable reset clocks for all devices in the domain */ - if (!domain->keep_clocks) - clk_bulk_disable_unprepare(domain->num_clks, domain->clks); - - return 0; - -out_clk_disable: - clk_bulk_disable_unprepare(domain->num_clks, domain->clks); -out_regulator_disable: - if (!IS_ERR(domain->regulator)) - regulator_disable(domain->regulator); -out_put_pm: - pm_runtime_put(domain->dev); - - return ret; -} - -static int imx_pgc_power_down(struct generic_pm_domain *genpd) -{ - struct imx_pgc_domain *domain = to_imx_pgc_domain(genpd); - u32 reg_val, pgc; - int ret; - - /* Enable reset clocks for all devices in the domain */ - if (!domain->keep_clocks) { - ret = clk_bulk_prepare_enable(domain->num_clks, domain->clks); - if (ret) { - dev_err(domain->dev, "failed to enable reset clocks\n"); - return ret; - } - } - - /* request the ADB400 to power down */ - if (domain->bits.hskreq) { - regmap_clear_bits(domain->regmap, domain->regs->hsk, - domain->bits.hskreq); - - ret = regmap_read_poll_timeout(domain->regmap, domain->regs->hsk, - reg_val, - !(reg_val & domain->bits.hskack), - 0, USEC_PER_MSEC); - if (ret) { - dev_err(domain->dev, "failed to power down ADB400\n"); - goto out_clk_disable; - } - } - - if (domain->bits.pxx) { - /* enable power control */ - for_each_set_bit(pgc, &domain->pgc, 32) { - regmap_update_bits(domain->regmap, GPC_PGC_CTRL(pgc), - GPC_PGC_CTRL_PCR, GPC_PGC_CTRL_PCR); - } - - /* request the domain to power down */ - regmap_update_bits(domain->regmap, domain->regs->pdn, - domain->bits.pxx, domain->bits.pxx); - /* - * As per "5.5.9.4 Example Code 4" in IMX7DRM.pdf wait - * for PUP_REQ/PDN_REQ bit to be cleared - */ - ret = regmap_read_poll_timeout(domain->regmap, - domain->regs->pdn, reg_val, - !(reg_val & domain->bits.pxx), - 0, USEC_PER_MSEC); - if (ret) { - dev_err(domain->dev, "failed to command PGC\n"); - goto out_clk_disable; - } - } - - /* Disable reset clocks for all devices in the domain */ - clk_bulk_disable_unprepare(domain->num_clks, domain->clks); - - if (!IS_ERR(domain->regulator)) { - ret = regulator_disable(domain->regulator); - if (ret) { - dev_err(domain->dev, - "failed to disable regulator: %pe\n", - ERR_PTR(ret)); - return ret; - } - } - - pm_runtime_put_sync_suspend(domain->dev); - - return 0; - -out_clk_disable: - if (!domain->keep_clocks) - clk_bulk_disable_unprepare(domain->num_clks, domain->clks); - - return ret; -} - -static const struct imx_pgc_domain imx7_pgc_domains[] = { - [IMX7_POWER_DOMAIN_MIPI_PHY] = { - .genpd = { - .name = "mipi-phy", - }, - .bits = { - .pxx = IMX7_MIPI_PHY_SW_Pxx_REQ, - .map = IMX7_MIPI_PHY_A_CORE_DOMAIN, - }, - .voltage = 1000000, - .pgc = BIT(IMX7_PGC_MIPI), - }, - - [IMX7_POWER_DOMAIN_PCIE_PHY] = { - .genpd = { - .name = "pcie-phy", - }, - .bits = { - .pxx = IMX7_PCIE_PHY_SW_Pxx_REQ, - .map = IMX7_PCIE_PHY_A_CORE_DOMAIN, - }, - .voltage = 1000000, - .pgc = BIT(IMX7_PGC_PCIE), - }, - - [IMX7_POWER_DOMAIN_USB_HSIC_PHY] = { - .genpd = { - .name = "usb-hsic-phy", - }, - .bits = { - .pxx = IMX7_USB_HSIC_PHY_SW_Pxx_REQ, - .map = IMX7_USB_HSIC_PHY_A_CORE_DOMAIN, - }, - .voltage = 1200000, - .pgc = BIT(IMX7_PGC_USB_HSIC), - }, -}; - -static const struct regmap_range imx7_yes_ranges[] = { - regmap_reg_range(GPC_LPCR_A_CORE_BSC, - GPC_M4_PU_PDN_FLG), - regmap_reg_range(GPC_PGC_CTRL(IMX7_PGC_MIPI), - GPC_PGC_SR(IMX7_PGC_MIPI)), - regmap_reg_range(GPC_PGC_CTRL(IMX7_PGC_PCIE), - GPC_PGC_SR(IMX7_PGC_PCIE)), - regmap_reg_range(GPC_PGC_CTRL(IMX7_PGC_USB_HSIC), - GPC_PGC_SR(IMX7_PGC_USB_HSIC)), -}; - -static const struct regmap_access_table imx7_access_table = { - .yes_ranges = imx7_yes_ranges, - .n_yes_ranges = ARRAY_SIZE(imx7_yes_ranges), -}; - -static const struct imx_pgc_regs imx7_pgc_regs = { - .map = GPC_PGC_CPU_MAPPING, - .pup = GPC_PU_PGC_SW_PUP_REQ, - .pdn = GPC_PU_PGC_SW_PDN_REQ, - .hsk = GPC_PU_PWRHSK, -}; - -static const struct imx_pgc_domain_data imx7_pgc_domain_data = { - .domains = imx7_pgc_domains, - .domains_num = ARRAY_SIZE(imx7_pgc_domains), - .reg_access_table = &imx7_access_table, - .pgc_regs = &imx7_pgc_regs, -}; - -static const struct imx_pgc_domain imx8m_pgc_domains[] = { - [IMX8M_POWER_DOMAIN_MIPI] = { - .genpd = { - .name = "mipi", - }, - .bits = { - .pxx = IMX8M_MIPI_SW_Pxx_REQ, - .map = IMX8M_MIPI_A53_DOMAIN, - }, - .pgc = BIT(IMX8M_PGC_MIPI), - }, - - [IMX8M_POWER_DOMAIN_PCIE1] = { - .genpd = { - .name = "pcie1", - }, - .bits = { - .pxx = IMX8M_PCIE1_SW_Pxx_REQ, - .map = IMX8M_PCIE1_A53_DOMAIN, - }, - .pgc = BIT(IMX8M_PGC_PCIE1), - }, - - [IMX8M_POWER_DOMAIN_USB_OTG1] = { - .genpd = { - .name = "usb-otg1", - }, - .bits = { - .pxx = IMX8M_OTG1_SW_Pxx_REQ, - .map = IMX8M_OTG1_A53_DOMAIN, - }, - .pgc = BIT(IMX8M_PGC_OTG1), - }, - - [IMX8M_POWER_DOMAIN_USB_OTG2] = { - .genpd = { - .name = "usb-otg2", - }, - .bits = { - .pxx = IMX8M_OTG2_SW_Pxx_REQ, - .map = IMX8M_OTG2_A53_DOMAIN, - }, - .pgc = BIT(IMX8M_PGC_OTG2), - }, - - [IMX8M_POWER_DOMAIN_DDR1] = { - .genpd = { - .name = "ddr1", - }, - .bits = { - .pxx = IMX8M_DDR1_SW_Pxx_REQ, - .map = IMX8M_DDR2_A53_DOMAIN, - }, - .pgc = BIT(IMX8M_PGC_DDR1), - }, - - [IMX8M_POWER_DOMAIN_GPU] = { - .genpd = { - .name = "gpu", - }, - .bits = { - .pxx = IMX8M_GPU_SW_Pxx_REQ, - .map = IMX8M_GPU_A53_DOMAIN, - .hskreq = IMX8M_GPU_HSK_PWRDNREQN, - .hskack = IMX8M_GPU_HSK_PWRDNACKN, - }, - .pgc = BIT(IMX8M_PGC_GPU), - }, - - [IMX8M_POWER_DOMAIN_VPU] = { - .genpd = { - .name = "vpu", - }, - .bits = { - .pxx = IMX8M_VPU_SW_Pxx_REQ, - .map = IMX8M_VPU_A53_DOMAIN, - .hskreq = IMX8M_VPU_HSK_PWRDNREQN, - .hskack = IMX8M_VPU_HSK_PWRDNACKN, - }, - .pgc = BIT(IMX8M_PGC_VPU), - .keep_clocks = true, - }, - - [IMX8M_POWER_DOMAIN_DISP] = { - .genpd = { - .name = "disp", - }, - .bits = { - .pxx = IMX8M_DISP_SW_Pxx_REQ, - .map = IMX8M_DISP_A53_DOMAIN, - .hskreq = IMX8M_DISP_HSK_PWRDNREQN, - .hskack = IMX8M_DISP_HSK_PWRDNACKN, - }, - .pgc = BIT(IMX8M_PGC_DISP), - }, - - [IMX8M_POWER_DOMAIN_MIPI_CSI1] = { - .genpd = { - .name = "mipi-csi1", - }, - .bits = { - .pxx = IMX8M_MIPI_CSI1_SW_Pxx_REQ, - .map = IMX8M_MIPI_CSI1_A53_DOMAIN, - }, - .pgc = BIT(IMX8M_PGC_MIPI_CSI1), - }, - - [IMX8M_POWER_DOMAIN_MIPI_CSI2] = { - .genpd = { - .name = "mipi-csi2", - }, - .bits = { - .pxx = IMX8M_MIPI_CSI2_SW_Pxx_REQ, - .map = IMX8M_MIPI_CSI2_A53_DOMAIN, - }, - .pgc = BIT(IMX8M_PGC_MIPI_CSI2), - }, - - [IMX8M_POWER_DOMAIN_PCIE2] = { - .genpd = { - .name = "pcie2", - }, - .bits = { - .pxx = IMX8M_PCIE2_SW_Pxx_REQ, - .map = IMX8M_PCIE2_A53_DOMAIN, - }, - .pgc = BIT(IMX8M_PGC_PCIE2), - }, -}; - -static const struct regmap_range imx8m_yes_ranges[] = { - regmap_reg_range(GPC_LPCR_A_CORE_BSC, - GPC_PU_PWRHSK), - regmap_reg_range(GPC_PGC_CTRL(IMX8M_PGC_MIPI), - GPC_PGC_SR(IMX8M_PGC_MIPI)), - regmap_reg_range(GPC_PGC_CTRL(IMX8M_PGC_PCIE1), - GPC_PGC_SR(IMX8M_PGC_PCIE1)), - regmap_reg_range(GPC_PGC_CTRL(IMX8M_PGC_OTG1), - GPC_PGC_SR(IMX8M_PGC_OTG1)), - regmap_reg_range(GPC_PGC_CTRL(IMX8M_PGC_OTG2), - GPC_PGC_SR(IMX8M_PGC_OTG2)), - regmap_reg_range(GPC_PGC_CTRL(IMX8M_PGC_DDR1), - GPC_PGC_SR(IMX8M_PGC_DDR1)), - regmap_reg_range(GPC_PGC_CTRL(IMX8M_PGC_GPU), - GPC_PGC_SR(IMX8M_PGC_GPU)), - regmap_reg_range(GPC_PGC_CTRL(IMX8M_PGC_VPU), - GPC_PGC_SR(IMX8M_PGC_VPU)), - regmap_reg_range(GPC_PGC_CTRL(IMX8M_PGC_DISP), - GPC_PGC_SR(IMX8M_PGC_DISP)), - regmap_reg_range(GPC_PGC_CTRL(IMX8M_PGC_MIPI_CSI1), - GPC_PGC_SR(IMX8M_PGC_MIPI_CSI1)), - regmap_reg_range(GPC_PGC_CTRL(IMX8M_PGC_MIPI_CSI2), - GPC_PGC_SR(IMX8M_PGC_MIPI_CSI2)), - regmap_reg_range(GPC_PGC_CTRL(IMX8M_PGC_PCIE2), - GPC_PGC_SR(IMX8M_PGC_PCIE2)), -}; - -static const struct regmap_access_table imx8m_access_table = { - .yes_ranges = imx8m_yes_ranges, - .n_yes_ranges = ARRAY_SIZE(imx8m_yes_ranges), -}; - -static const struct imx_pgc_domain_data imx8m_pgc_domain_data = { - .domains = imx8m_pgc_domains, - .domains_num = ARRAY_SIZE(imx8m_pgc_domains), - .reg_access_table = &imx8m_access_table, - .pgc_regs = &imx7_pgc_regs, -}; - -static const struct imx_pgc_domain imx8mm_pgc_domains[] = { - [IMX8MM_POWER_DOMAIN_HSIOMIX] = { - .genpd = { - .name = "hsiomix", - }, - .bits = { - .pxx = 0, /* no power sequence control */ - .map = 0, /* no power sequence control */ - .hskreq = IMX8MM_HSIO_HSK_PWRDNREQN, - .hskack = IMX8MM_HSIO_HSK_PWRDNACKN, - }, - .keep_clocks = true, - }, - - [IMX8MM_POWER_DOMAIN_PCIE] = { - .genpd = { - .name = "pcie", - }, - .bits = { - .pxx = IMX8MM_PCIE_SW_Pxx_REQ, - .map = IMX8MM_PCIE_A53_DOMAIN, - }, - .pgc = BIT(IMX8MM_PGC_PCIE), - }, - - [IMX8MM_POWER_DOMAIN_OTG1] = { - .genpd = { - .name = "usb-otg1", - .flags = GENPD_FLAG_ACTIVE_WAKEUP, - }, - .bits = { - .pxx = IMX8MM_OTG1_SW_Pxx_REQ, - .map = IMX8MM_OTG1_A53_DOMAIN, - }, - .pgc = BIT(IMX8MM_PGC_OTG1), - }, - - [IMX8MM_POWER_DOMAIN_OTG2] = { - .genpd = { - .name = "usb-otg2", - .flags = GENPD_FLAG_ACTIVE_WAKEUP, - }, - .bits = { - .pxx = IMX8MM_OTG2_SW_Pxx_REQ, - .map = IMX8MM_OTG2_A53_DOMAIN, - }, - .pgc = BIT(IMX8MM_PGC_OTG2), - }, - - [IMX8MM_POWER_DOMAIN_GPUMIX] = { - .genpd = { - .name = "gpumix", - }, - .bits = { - .pxx = IMX8MM_GPUMIX_SW_Pxx_REQ, - .map = IMX8MM_GPUMIX_A53_DOMAIN, - .hskreq = IMX8MM_GPUMIX_HSK_PWRDNREQN, - .hskack = IMX8MM_GPUMIX_HSK_PWRDNACKN, - }, - .pgc = BIT(IMX8MM_PGC_GPUMIX), - .keep_clocks = true, - }, - - [IMX8MM_POWER_DOMAIN_GPU] = { - .genpd = { - .name = "gpu", - }, - .bits = { - .pxx = IMX8MM_GPU_SW_Pxx_REQ, - .map = IMX8MM_GPU_A53_DOMAIN, - .hskreq = IMX8MM_GPU_HSK_PWRDNREQN, - .hskack = IMX8MM_GPU_HSK_PWRDNACKN, - }, - .pgc = BIT(IMX8MM_PGC_GPU2D) | BIT(IMX8MM_PGC_GPU3D), - }, - - [IMX8MM_POWER_DOMAIN_VPUMIX] = { - .genpd = { - .name = "vpumix", - }, - .bits = { - .pxx = IMX8MM_VPUMIX_SW_Pxx_REQ, - .map = IMX8MM_VPUMIX_A53_DOMAIN, - .hskreq = IMX8MM_VPUMIX_HSK_PWRDNREQN, - .hskack = IMX8MM_VPUMIX_HSK_PWRDNACKN, - }, - .pgc = BIT(IMX8MM_PGC_VPUMIX), - .keep_clocks = true, - }, - - [IMX8MM_POWER_DOMAIN_VPUG1] = { - .genpd = { - .name = "vpu-g1", - }, - .bits = { - .pxx = IMX8MM_VPUG1_SW_Pxx_REQ, - .map = IMX8MM_VPUG1_A53_DOMAIN, - }, - .pgc = BIT(IMX8MM_PGC_VPUG1), - }, - - [IMX8MM_POWER_DOMAIN_VPUG2] = { - .genpd = { - .name = "vpu-g2", - }, - .bits = { - .pxx = IMX8MM_VPUG2_SW_Pxx_REQ, - .map = IMX8MM_VPUG2_A53_DOMAIN, - }, - .pgc = BIT(IMX8MM_PGC_VPUG2), - }, - - [IMX8MM_POWER_DOMAIN_VPUH1] = { - .genpd = { - .name = "vpu-h1", - }, - .bits = { - .pxx = IMX8MM_VPUH1_SW_Pxx_REQ, - .map = IMX8MM_VPUH1_A53_DOMAIN, - }, - .pgc = BIT(IMX8MM_PGC_VPUH1), - .keep_clocks = true, - }, - - [IMX8MM_POWER_DOMAIN_DISPMIX] = { - .genpd = { - .name = "dispmix", - }, - .bits = { - .pxx = IMX8MM_DISPMIX_SW_Pxx_REQ, - .map = IMX8MM_DISPMIX_A53_DOMAIN, - .hskreq = IMX8MM_DISPMIX_HSK_PWRDNREQN, - .hskack = IMX8MM_DISPMIX_HSK_PWRDNACKN, - }, - .pgc = BIT(IMX8MM_PGC_DISPMIX), - .keep_clocks = true, - }, - - [IMX8MM_POWER_DOMAIN_MIPI] = { - .genpd = { - .name = "mipi", - }, - .bits = { - .pxx = IMX8MM_MIPI_SW_Pxx_REQ, - .map = IMX8MM_MIPI_A53_DOMAIN, - }, - .pgc = BIT(IMX8MM_PGC_MIPI), - }, -}; - -static const struct regmap_range imx8mm_yes_ranges[] = { - regmap_reg_range(GPC_LPCR_A_CORE_BSC, - GPC_PU_PWRHSK), - regmap_reg_range(GPC_PGC_CTRL(IMX8MM_PGC_MIPI), - GPC_PGC_SR(IMX8MM_PGC_MIPI)), - regmap_reg_range(GPC_PGC_CTRL(IMX8MM_PGC_PCIE), - GPC_PGC_SR(IMX8MM_PGC_PCIE)), - regmap_reg_range(GPC_PGC_CTRL(IMX8MM_PGC_OTG1), - GPC_PGC_SR(IMX8MM_PGC_OTG1)), - regmap_reg_range(GPC_PGC_CTRL(IMX8MM_PGC_OTG2), - GPC_PGC_SR(IMX8MM_PGC_OTG2)), - regmap_reg_range(GPC_PGC_CTRL(IMX8MM_PGC_DDR1), - GPC_PGC_SR(IMX8MM_PGC_DDR1)), - regmap_reg_range(GPC_PGC_CTRL(IMX8MM_PGC_GPU2D), - GPC_PGC_SR(IMX8MM_PGC_GPU2D)), - regmap_reg_range(GPC_PGC_CTRL(IMX8MM_PGC_GPUMIX), - GPC_PGC_SR(IMX8MM_PGC_GPUMIX)), - regmap_reg_range(GPC_PGC_CTRL(IMX8MM_PGC_VPUMIX), - GPC_PGC_SR(IMX8MM_PGC_VPUMIX)), - regmap_reg_range(GPC_PGC_CTRL(IMX8MM_PGC_GPU3D), - GPC_PGC_SR(IMX8MM_PGC_GPU3D)), - regmap_reg_range(GPC_PGC_CTRL(IMX8MM_PGC_DISPMIX), - GPC_PGC_SR(IMX8MM_PGC_DISPMIX)), - regmap_reg_range(GPC_PGC_CTRL(IMX8MM_PGC_VPUG1), - GPC_PGC_SR(IMX8MM_PGC_VPUG1)), - regmap_reg_range(GPC_PGC_CTRL(IMX8MM_PGC_VPUG2), - GPC_PGC_SR(IMX8MM_PGC_VPUG2)), - regmap_reg_range(GPC_PGC_CTRL(IMX8MM_PGC_VPUH1), - GPC_PGC_SR(IMX8MM_PGC_VPUH1)), -}; - -static const struct regmap_access_table imx8mm_access_table = { - .yes_ranges = imx8mm_yes_ranges, - .n_yes_ranges = ARRAY_SIZE(imx8mm_yes_ranges), -}; - -static const struct imx_pgc_domain_data imx8mm_pgc_domain_data = { - .domains = imx8mm_pgc_domains, - .domains_num = ARRAY_SIZE(imx8mm_pgc_domains), - .reg_access_table = &imx8mm_access_table, - .pgc_regs = &imx7_pgc_regs, -}; - -static const struct imx_pgc_domain imx8mp_pgc_domains[] = { - [IMX8MP_POWER_DOMAIN_MIPI_PHY1] = { - .genpd = { - .name = "mipi-phy1", - }, - .bits = { - .pxx = IMX8MP_MIPI_PHY1_SW_Pxx_REQ, - .map = IMX8MP_MIPI_PHY1_A53_DOMAIN, - }, - .pgc = BIT(IMX8MP_PGC_MIPI1), - }, - - [IMX8MP_POWER_DOMAIN_PCIE_PHY] = { - .genpd = { - .name = "pcie-phy1", - }, - .bits = { - .pxx = IMX8MP_PCIE_PHY_SW_Pxx_REQ, - .map = IMX8MP_PCIE_PHY_A53_DOMAIN, - }, - .pgc = BIT(IMX8MP_PGC_PCIE), - }, - - [IMX8MP_POWER_DOMAIN_USB1_PHY] = { - .genpd = { - .name = "usb-otg1", - }, - .bits = { - .pxx = IMX8MP_USB1_PHY_Pxx_REQ, - .map = IMX8MP_USB1_PHY_A53_DOMAIN, - }, - .pgc = BIT(IMX8MP_PGC_USB1), - }, - - [IMX8MP_POWER_DOMAIN_USB2_PHY] = { - .genpd = { - .name = "usb-otg2", - }, - .bits = { - .pxx = IMX8MP_USB2_PHY_Pxx_REQ, - .map = IMX8MP_USB2_PHY_A53_DOMAIN, - }, - .pgc = BIT(IMX8MP_PGC_USB2), - }, - - [IMX8MP_POWER_DOMAIN_MLMIX] = { - .genpd = { - .name = "mlmix", - }, - .bits = { - .pxx = IMX8MP_MLMIX_Pxx_REQ, - .map = IMX8MP_MLMIX_A53_DOMAIN, - .hskreq = IMX8MP_MLMIX_PWRDNREQN, - .hskack = IMX8MP_MLMIX_PWRDNACKN, - }, - .pgc = BIT(IMX8MP_PGC_MLMIX), - .keep_clocks = true, - }, - - [IMX8MP_POWER_DOMAIN_AUDIOMIX] = { - .genpd = { - .name = "audiomix", - }, - .bits = { - .pxx = IMX8MP_AUDIOMIX_Pxx_REQ, - .map = IMX8MP_AUDIOMIX_A53_DOMAIN, - .hskreq = IMX8MP_AUDIOMIX_PWRDNREQN, - .hskack = IMX8MP_AUDIOMIX_PWRDNACKN, - }, - .pgc = BIT(IMX8MP_PGC_AUDIOMIX), - .keep_clocks = true, - }, - - [IMX8MP_POWER_DOMAIN_GPU2D] = { - .genpd = { - .name = "gpu2d", - }, - .bits = { - .pxx = IMX8MP_GPU_2D_Pxx_REQ, - .map = IMX8MP_GPU2D_A53_DOMAIN, - }, - .pgc = BIT(IMX8MP_PGC_GPU2D), - }, - - [IMX8MP_POWER_DOMAIN_GPUMIX] = { - .genpd = { - .name = "gpumix", - }, - .bits = { - .pxx = IMX8MP_GPU_SHARE_LOGIC_Pxx_REQ, - .map = IMX8MP_GPUMIX_A53_DOMAIN, - .hskreq = IMX8MP_GPUMIX_PWRDNREQN, - .hskack = IMX8MP_GPUMIX_PWRDNACKN, - }, - .pgc = BIT(IMX8MP_PGC_GPUMIX), - .keep_clocks = true, - }, - - [IMX8MP_POWER_DOMAIN_VPUMIX] = { - .genpd = { - .name = "vpumix", - }, - .bits = { - .pxx = IMX8MP_VPU_MIX_SHARE_LOGIC_Pxx_REQ, - .map = IMX8MP_VPUMIX_A53_DOMAIN, - .hskreq = IMX8MP_VPUMIX_PWRDNREQN, - .hskack = IMX8MP_VPUMIX_PWRDNACKN, - }, - .pgc = BIT(IMX8MP_PGC_VPUMIX), - .keep_clocks = true, - }, - - [IMX8MP_POWER_DOMAIN_GPU3D] = { - .genpd = { - .name = "gpu3d", - }, - .bits = { - .pxx = IMX8MP_GPU_3D_Pxx_REQ, - .map = IMX8MP_GPU3D_A53_DOMAIN, - }, - .pgc = BIT(IMX8MP_PGC_GPU3D), - }, - - [IMX8MP_POWER_DOMAIN_MEDIAMIX] = { - .genpd = { - .name = "mediamix", - }, - .bits = { - .pxx = IMX8MP_MEDIMIX_Pxx_REQ, - .map = IMX8MP_MEDIAMIX_A53_DOMAIN, - .hskreq = IMX8MP_MEDIAMIX_PWRDNREQN, - .hskack = IMX8MP_MEDIAMIX_PWRDNACKN, - }, - .pgc = BIT(IMX8MP_PGC_MEDIAMIX), - .keep_clocks = true, - }, - - [IMX8MP_POWER_DOMAIN_VPU_G1] = { - .genpd = { - .name = "vpu-g1", - }, - .bits = { - .pxx = IMX8MP_VPU_G1_Pxx_REQ, - .map = IMX8MP_VPU_G1_A53_DOMAIN, - }, - .pgc = BIT(IMX8MP_PGC_VPU_G1), - }, - - [IMX8MP_POWER_DOMAIN_VPU_G2] = { - .genpd = { - .name = "vpu-g2", - }, - .bits = { - .pxx = IMX8MP_VPU_G2_Pxx_REQ, - .map = IMX8MP_VPU_G2_A53_DOMAIN - }, - .pgc = BIT(IMX8MP_PGC_VPU_G2), - }, - - [IMX8MP_POWER_DOMAIN_VPU_VC8000E] = { - .genpd = { - .name = "vpu-h1", - }, - .bits = { - .pxx = IMX8MP_VPU_VC8K_Pxx_REQ, - .map = IMX8MP_VPU_VC8000E_A53_DOMAIN, - }, - .pgc = BIT(IMX8MP_PGC_VPU_VC8000E), - }, - - [IMX8MP_POWER_DOMAIN_HDMIMIX] = { - .genpd = { - .name = "hdmimix", - }, - .bits = { - .pxx = IMX8MP_HDMIMIX_Pxx_REQ, - .map = IMX8MP_HDMIMIX_A53_DOMAIN, - .hskreq = IMX8MP_HDMIMIX_PWRDNREQN, - .hskack = IMX8MP_HDMIMIX_PWRDNACKN, - }, - .pgc = BIT(IMX8MP_PGC_HDMIMIX), - .keep_clocks = true, - }, - - [IMX8MP_POWER_DOMAIN_HDMI_PHY] = { - .genpd = { - .name = "hdmi-phy", - }, - .bits = { - .pxx = IMX8MP_HDMI_PHY_Pxx_REQ, - .map = IMX8MP_HDMI_PHY_A53_DOMAIN, - }, - .pgc = BIT(IMX8MP_PGC_HDMI), - }, - - [IMX8MP_POWER_DOMAIN_MIPI_PHY2] = { - .genpd = { - .name = "mipi-phy2", - }, - .bits = { - .pxx = IMX8MP_MIPI_PHY2_Pxx_REQ, - .map = IMX8MP_MIPI_PHY2_A53_DOMAIN, - }, - .pgc = BIT(IMX8MP_PGC_MIPI2), - }, - - [IMX8MP_POWER_DOMAIN_HSIOMIX] = { - .genpd = { - .name = "hsiomix", - }, - .bits = { - .pxx = IMX8MP_HSIOMIX_Pxx_REQ, - .map = IMX8MP_HSIOMIX_A53_DOMAIN, - .hskreq = IMX8MP_HSIOMIX_PWRDNREQN, - .hskack = IMX8MP_HSIOMIX_PWRDNACKN, - }, - .pgc = BIT(IMX8MP_PGC_HSIOMIX), - .keep_clocks = true, - }, - - [IMX8MP_POWER_DOMAIN_MEDIAMIX_ISPDWP] = { - .genpd = { - .name = "mediamix-isp-dwp", - }, - .bits = { - .pxx = IMX8MP_MEDIA_ISP_DWP_Pxx_REQ, - .map = IMX8MP_MEDIA_ISPDWP_A53_DOMAIN, - }, - .pgc = BIT(IMX8MP_PGC_MEDIA_ISP_DWP), - }, -}; - -static const struct regmap_range imx8mp_yes_ranges[] = { - regmap_reg_range(GPC_LPCR_A_CORE_BSC, - IMX8MP_GPC_PGC_CPU_MAPPING), - regmap_reg_range(GPC_PGC_CTRL(IMX8MP_PGC_NOC), - GPC_PGC_SR(IMX8MP_PGC_NOC)), - regmap_reg_range(GPC_PGC_CTRL(IMX8MP_PGC_MIPI1), - GPC_PGC_SR(IMX8MP_PGC_MIPI1)), - regmap_reg_range(GPC_PGC_CTRL(IMX8MP_PGC_PCIE), - GPC_PGC_SR(IMX8MP_PGC_PCIE)), - regmap_reg_range(GPC_PGC_CTRL(IMX8MP_PGC_USB1), - GPC_PGC_SR(IMX8MP_PGC_USB1)), - regmap_reg_range(GPC_PGC_CTRL(IMX8MP_PGC_USB2), - GPC_PGC_SR(IMX8MP_PGC_USB2)), - regmap_reg_range(GPC_PGC_CTRL(IMX8MP_PGC_MLMIX), - GPC_PGC_SR(IMX8MP_PGC_MLMIX)), - regmap_reg_range(GPC_PGC_CTRL(IMX8MP_PGC_AUDIOMIX), - GPC_PGC_SR(IMX8MP_PGC_AUDIOMIX)), - regmap_reg_range(GPC_PGC_CTRL(IMX8MP_PGC_GPU2D), - GPC_PGC_SR(IMX8MP_PGC_GPU2D)), - regmap_reg_range(GPC_PGC_CTRL(IMX8MP_PGC_GPUMIX), - GPC_PGC_SR(IMX8MP_PGC_GPUMIX)), - regmap_reg_range(GPC_PGC_CTRL(IMX8MP_PGC_VPUMIX), - GPC_PGC_SR(IMX8MP_PGC_VPUMIX)), - regmap_reg_range(GPC_PGC_CTRL(IMX8MP_PGC_GPU3D), - GPC_PGC_SR(IMX8MP_PGC_GPU3D)), - regmap_reg_range(GPC_PGC_CTRL(IMX8MP_PGC_MEDIAMIX), - GPC_PGC_SR(IMX8MP_PGC_MEDIAMIX)), - regmap_reg_range(GPC_PGC_CTRL(IMX8MP_PGC_VPU_G1), - GPC_PGC_SR(IMX8MP_PGC_VPU_G1)), - regmap_reg_range(GPC_PGC_CTRL(IMX8MP_PGC_VPU_G2), - GPC_PGC_SR(IMX8MP_PGC_VPU_G2)), - regmap_reg_range(GPC_PGC_CTRL(IMX8MP_PGC_VPU_VC8000E), - GPC_PGC_SR(IMX8MP_PGC_VPU_VC8000E)), - regmap_reg_range(GPC_PGC_CTRL(IMX8MP_PGC_HDMIMIX), - GPC_PGC_SR(IMX8MP_PGC_HDMIMIX)), - regmap_reg_range(GPC_PGC_CTRL(IMX8MP_PGC_HDMI), - GPC_PGC_SR(IMX8MP_PGC_HDMI)), - regmap_reg_range(GPC_PGC_CTRL(IMX8MP_PGC_MIPI2), - GPC_PGC_SR(IMX8MP_PGC_MIPI2)), - regmap_reg_range(GPC_PGC_CTRL(IMX8MP_PGC_HSIOMIX), - GPC_PGC_SR(IMX8MP_PGC_HSIOMIX)), - regmap_reg_range(GPC_PGC_CTRL(IMX8MP_PGC_MEDIA_ISP_DWP), - GPC_PGC_SR(IMX8MP_PGC_MEDIA_ISP_DWP)), - regmap_reg_range(GPC_PGC_CTRL(IMX8MP_PGC_DDRMIX), - GPC_PGC_SR(IMX8MP_PGC_DDRMIX)), -}; - -static const struct regmap_access_table imx8mp_access_table = { - .yes_ranges = imx8mp_yes_ranges, - .n_yes_ranges = ARRAY_SIZE(imx8mp_yes_ranges), -}; - -static const struct imx_pgc_regs imx8mp_pgc_regs = { - .map = IMX8MP_GPC_PGC_CPU_MAPPING, - .pup = IMX8MP_GPC_PU_PGC_SW_PUP_REQ, - .pdn = IMX8MP_GPC_PU_PGC_SW_PDN_REQ, - .hsk = IMX8MP_GPC_PU_PWRHSK, -}; -static const struct imx_pgc_domain_data imx8mp_pgc_domain_data = { - .domains = imx8mp_pgc_domains, - .domains_num = ARRAY_SIZE(imx8mp_pgc_domains), - .reg_access_table = &imx8mp_access_table, - .pgc_regs = &imx8mp_pgc_regs, -}; - -static const struct imx_pgc_domain imx8mn_pgc_domains[] = { - [IMX8MN_POWER_DOMAIN_HSIOMIX] = { - .genpd = { - .name = "hsiomix", - }, - .bits = { - .pxx = 0, /* no power sequence control */ - .map = 0, /* no power sequence control */ - .hskreq = IMX8MN_HSIO_HSK_PWRDNREQN, - .hskack = IMX8MN_HSIO_HSK_PWRDNACKN, - }, - .keep_clocks = true, - }, - - [IMX8MN_POWER_DOMAIN_OTG1] = { - .genpd = { - .name = "usb-otg1", - .flags = GENPD_FLAG_ACTIVE_WAKEUP, - }, - .bits = { - .pxx = IMX8MN_OTG1_SW_Pxx_REQ, - .map = IMX8MN_OTG1_A53_DOMAIN, - }, - .pgc = BIT(IMX8MN_PGC_OTG1), - }, - - [IMX8MN_POWER_DOMAIN_GPUMIX] = { - .genpd = { - .name = "gpumix", - }, - .bits = { - .pxx = IMX8MN_GPUMIX_SW_Pxx_REQ, - .map = IMX8MN_GPUMIX_A53_DOMAIN, - .hskreq = IMX8MN_GPUMIX_HSK_PWRDNREQN, - .hskack = IMX8MN_GPUMIX_HSK_PWRDNACKN, - }, - .pgc = BIT(IMX8MN_PGC_GPUMIX), - .keep_clocks = true, - }, - - [IMX8MN_POWER_DOMAIN_DISPMIX] = { - .genpd = { - .name = "dispmix", - }, - .bits = { - .pxx = IMX8MN_DISPMIX_SW_Pxx_REQ, - .map = IMX8MN_DISPMIX_A53_DOMAIN, - .hskreq = IMX8MN_DISPMIX_HSK_PWRDNREQN, - .hskack = IMX8MN_DISPMIX_HSK_PWRDNACKN, - }, - .pgc = BIT(IMX8MN_PGC_DISPMIX), - .keep_clocks = true, - }, - - [IMX8MN_POWER_DOMAIN_MIPI] = { - .genpd = { - .name = "mipi", - }, - .bits = { - .pxx = IMX8MN_MIPI_SW_Pxx_REQ, - .map = IMX8MN_MIPI_A53_DOMAIN, - }, - .pgc = BIT(IMX8MN_PGC_MIPI), - }, -}; - -static const struct regmap_range imx8mn_yes_ranges[] = { - regmap_reg_range(GPC_LPCR_A_CORE_BSC, - GPC_PU_PWRHSK), - regmap_reg_range(GPC_PGC_CTRL(IMX8MN_PGC_MIPI), - GPC_PGC_SR(IMX8MN_PGC_MIPI)), - regmap_reg_range(GPC_PGC_CTRL(IMX8MN_PGC_OTG1), - GPC_PGC_SR(IMX8MN_PGC_OTG1)), - regmap_reg_range(GPC_PGC_CTRL(IMX8MN_PGC_DDR1), - GPC_PGC_SR(IMX8MN_PGC_DDR1)), - regmap_reg_range(GPC_PGC_CTRL(IMX8MN_PGC_GPUMIX), - GPC_PGC_SR(IMX8MN_PGC_GPUMIX)), - regmap_reg_range(GPC_PGC_CTRL(IMX8MN_PGC_DISPMIX), - GPC_PGC_SR(IMX8MN_PGC_DISPMIX)), -}; - -static const struct regmap_access_table imx8mn_access_table = { - .yes_ranges = imx8mn_yes_ranges, - .n_yes_ranges = ARRAY_SIZE(imx8mn_yes_ranges), -}; - -static const struct imx_pgc_domain_data imx8mn_pgc_domain_data = { - .domains = imx8mn_pgc_domains, - .domains_num = ARRAY_SIZE(imx8mn_pgc_domains), - .reg_access_table = &imx8mn_access_table, - .pgc_regs = &imx7_pgc_regs, -}; - -static int imx_pgc_domain_probe(struct platform_device *pdev) -{ - struct imx_pgc_domain *domain = pdev->dev.platform_data; - int ret; - - domain->dev = &pdev->dev; - - domain->regulator = devm_regulator_get_optional(domain->dev, "power"); - if (IS_ERR(domain->regulator)) { - if (PTR_ERR(domain->regulator) != -ENODEV) - return dev_err_probe(domain->dev, PTR_ERR(domain->regulator), - "Failed to get domain's regulator\n"); - } else if (domain->voltage) { - regulator_set_voltage(domain->regulator, - domain->voltage, domain->voltage); - } - - domain->num_clks = devm_clk_bulk_get_all(domain->dev, &domain->clks); - if (domain->num_clks < 0) - return dev_err_probe(domain->dev, domain->num_clks, - "Failed to get domain's clocks\n"); - - domain->reset = devm_reset_control_array_get_optional_exclusive(domain->dev); - if (IS_ERR(domain->reset)) - return dev_err_probe(domain->dev, PTR_ERR(domain->reset), - "Failed to get domain's resets\n"); - - pm_runtime_enable(domain->dev); - - if (domain->bits.map) - regmap_update_bits(domain->regmap, domain->regs->map, - domain->bits.map, domain->bits.map); - - ret = pm_genpd_init(&domain->genpd, NULL, true); - if (ret) { - dev_err(domain->dev, "Failed to init power domain\n"); - goto out_domain_unmap; - } - - if (IS_ENABLED(CONFIG_LOCKDEP) && - of_property_read_bool(domain->dev->of_node, "power-domains")) - lockdep_set_subclass(&domain->genpd.mlock, 1); - - ret = of_genpd_add_provider_simple(domain->dev->of_node, - &domain->genpd); - if (ret) { - dev_err(domain->dev, "Failed to add genpd provider\n"); - goto out_genpd_remove; - } - - return 0; - -out_genpd_remove: - pm_genpd_remove(&domain->genpd); -out_domain_unmap: - if (domain->bits.map) - regmap_update_bits(domain->regmap, domain->regs->map, - domain->bits.map, 0); - pm_runtime_disable(domain->dev); - - return ret; -} - -static int imx_pgc_domain_remove(struct platform_device *pdev) -{ - struct imx_pgc_domain *domain = pdev->dev.platform_data; - - of_genpd_del_provider(domain->dev->of_node); - pm_genpd_remove(&domain->genpd); - - if (domain->bits.map) - regmap_update_bits(domain->regmap, domain->regs->map, - domain->bits.map, 0); - - pm_runtime_disable(domain->dev); - - return 0; -} - -#ifdef CONFIG_PM_SLEEP -static int imx_pgc_domain_suspend(struct device *dev) -{ - int ret; - - /* - * This may look strange, but is done so the generic PM_SLEEP code - * can power down our domain and more importantly power it up again - * after resume, without tripping over our usage of runtime PM to - * power up/down the nested domains. - */ - ret = pm_runtime_get_sync(dev); - if (ret < 0) { - pm_runtime_put_noidle(dev); - return ret; - } - - return 0; -} - -static int imx_pgc_domain_resume(struct device *dev) -{ - return pm_runtime_put(dev); -} -#endif - -static const struct dev_pm_ops imx_pgc_domain_pm_ops = { - SET_SYSTEM_SLEEP_PM_OPS(imx_pgc_domain_suspend, imx_pgc_domain_resume) -}; - -static const struct platform_device_id imx_pgc_domain_id[] = { - { "imx-pgc-domain", }, - { }, -}; - -static struct platform_driver imx_pgc_domain_driver = { - .driver = { - .name = "imx-pgc", - .pm = &imx_pgc_domain_pm_ops, - }, - .probe = imx_pgc_domain_probe, - .remove = imx_pgc_domain_remove, - .id_table = imx_pgc_domain_id, -}; -builtin_platform_driver(imx_pgc_domain_driver) - -static int imx_gpcv2_probe(struct platform_device *pdev) -{ - const struct imx_pgc_domain_data *domain_data = - of_device_get_match_data(&pdev->dev); - - struct regmap_config regmap_config = { - .reg_bits = 32, - .val_bits = 32, - .reg_stride = 4, - .rd_table = domain_data->reg_access_table, - .wr_table = domain_data->reg_access_table, - .max_register = SZ_4K, - }; - struct device *dev = &pdev->dev; - struct device_node *pgc_np, *np; - struct regmap *regmap; - void __iomem *base; - int ret; - - pgc_np = of_get_child_by_name(dev->of_node, "pgc"); - if (!pgc_np) { - dev_err(dev, "No power domains specified in DT\n"); - return -EINVAL; - } - - base = devm_platform_ioremap_resource(pdev, 0); - if (IS_ERR(base)) - return PTR_ERR(base); - - regmap = devm_regmap_init_mmio(dev, base, ®map_config); - if (IS_ERR(regmap)) { - ret = PTR_ERR(regmap); - dev_err(dev, "failed to init regmap (%d)\n", ret); - return ret; - } - - for_each_child_of_node(pgc_np, np) { - struct platform_device *pd_pdev; - struct imx_pgc_domain *domain; - u32 domain_index; - - if (!of_device_is_available(np)) - continue; - - ret = of_property_read_u32(np, "reg", &domain_index); - if (ret) { - dev_err(dev, "Failed to read 'reg' property\n"); - of_node_put(np); - return ret; - } - - if (domain_index >= domain_data->domains_num) { - dev_warn(dev, - "Domain index %d is out of bounds\n", - domain_index); - continue; - } - - pd_pdev = platform_device_alloc("imx-pgc-domain", - domain_index); - if (!pd_pdev) { - dev_err(dev, "Failed to allocate platform device\n"); - of_node_put(np); - return -ENOMEM; - } - - ret = platform_device_add_data(pd_pdev, - &domain_data->domains[domain_index], - sizeof(domain_data->domains[domain_index])); - if (ret) { - platform_device_put(pd_pdev); - of_node_put(np); - return ret; - } - - domain = pd_pdev->dev.platform_data; - domain->regmap = regmap; - domain->regs = domain_data->pgc_regs; - - domain->genpd.power_on = imx_pgc_power_up; - domain->genpd.power_off = imx_pgc_power_down; - - pd_pdev->dev.parent = dev; - device_set_node(&pd_pdev->dev, of_fwnode_handle(np)); - - ret = platform_device_add(pd_pdev); - if (ret) { - platform_device_put(pd_pdev); - of_node_put(np); - return ret; - } - } - - return 0; -} - -static const struct of_device_id imx_gpcv2_dt_ids[] = { - { .compatible = "fsl,imx7d-gpc", .data = &imx7_pgc_domain_data, }, - { .compatible = "fsl,imx8mm-gpc", .data = &imx8mm_pgc_domain_data, }, - { .compatible = "fsl,imx8mn-gpc", .data = &imx8mn_pgc_domain_data, }, - { .compatible = "fsl,imx8mp-gpc", .data = &imx8mp_pgc_domain_data, }, - { .compatible = "fsl,imx8mq-gpc", .data = &imx8m_pgc_domain_data, }, - { } -}; - -static struct platform_driver imx_gpc_driver = { - .driver = { - .name = "imx-gpcv2", - .of_match_table = imx_gpcv2_dt_ids, - }, - .probe = imx_gpcv2_probe, -}; -builtin_platform_driver(imx_gpc_driver) diff --git a/drivers/genpd/imx/imx8m-blk-ctrl.c b/drivers/genpd/imx/imx8m-blk-ctrl.c deleted file mode 100644 index cc5ef6e2f0a8..000000000000 --- a/drivers/genpd/imx/imx8m-blk-ctrl.c +++ /dev/null @@ -1,899 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0+ - -/* - * Copyright 2021 Pengutronix, Lucas Stach - */ - -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -#include -#include -#include -#include - -#define BLK_SFT_RSTN 0x0 -#define BLK_CLK_EN 0x4 -#define BLK_MIPI_RESET_DIV 0x8 /* Mini/Nano/Plus DISPLAY_BLK_CTRL only */ - -struct imx8m_blk_ctrl_domain; - -struct imx8m_blk_ctrl { - struct device *dev; - struct notifier_block power_nb; - struct device *bus_power_dev; - struct regmap *regmap; - struct imx8m_blk_ctrl_domain *domains; - struct genpd_onecell_data onecell_data; -}; - -struct imx8m_blk_ctrl_domain_data { - const char *name; - const char * const *clk_names; - const char * const *path_names; - const char *gpc_name; - int num_clks; - int num_paths; - u32 rst_mask; - u32 clk_mask; - - /* - * i.MX8M Mini, Nano and Plus have a third DISPLAY_BLK_CTRL register - * which is used to control the reset for the MIPI Phy. - * Since it's only present in certain circumstances, - * an if-statement should be used before setting and clearing this - * register. - */ - u32 mipi_phy_rst_mask; -}; - -#define DOMAIN_MAX_CLKS 4 -#define DOMAIN_MAX_PATHS 4 - -struct imx8m_blk_ctrl_domain { - struct generic_pm_domain genpd; - const struct imx8m_blk_ctrl_domain_data *data; - struct clk_bulk_data clks[DOMAIN_MAX_CLKS]; - struct icc_bulk_data paths[DOMAIN_MAX_PATHS]; - struct device *power_dev; - struct imx8m_blk_ctrl *bc; - int num_paths; -}; - -struct imx8m_blk_ctrl_data { - int max_reg; - notifier_fn_t power_notifier_fn; - const struct imx8m_blk_ctrl_domain_data *domains; - int num_domains; -}; - -static inline struct imx8m_blk_ctrl_domain * -to_imx8m_blk_ctrl_domain(struct generic_pm_domain *genpd) -{ - return container_of(genpd, struct imx8m_blk_ctrl_domain, genpd); -} - -static int imx8m_blk_ctrl_power_on(struct generic_pm_domain *genpd) -{ - struct imx8m_blk_ctrl_domain *domain = to_imx8m_blk_ctrl_domain(genpd); - const struct imx8m_blk_ctrl_domain_data *data = domain->data; - struct imx8m_blk_ctrl *bc = domain->bc; - int ret; - - /* make sure bus domain is awake */ - ret = pm_runtime_get_sync(bc->bus_power_dev); - if (ret < 0) { - pm_runtime_put_noidle(bc->bus_power_dev); - dev_err(bc->dev, "failed to power up bus domain\n"); - return ret; - } - - /* put devices into reset */ - regmap_clear_bits(bc->regmap, BLK_SFT_RSTN, data->rst_mask); - if (data->mipi_phy_rst_mask) - regmap_clear_bits(bc->regmap, BLK_MIPI_RESET_DIV, data->mipi_phy_rst_mask); - - /* enable upstream and blk-ctrl clocks to allow reset to propagate */ - ret = clk_bulk_prepare_enable(data->num_clks, domain->clks); - if (ret) { - dev_err(bc->dev, "failed to enable clocks\n"); - goto bus_put; - } - regmap_set_bits(bc->regmap, BLK_CLK_EN, data->clk_mask); - - /* power up upstream GPC domain */ - ret = pm_runtime_get_sync(domain->power_dev); - if (ret < 0) { - dev_err(bc->dev, "failed to power up peripheral domain\n"); - goto clk_disable; - } - - /* wait for reset to propagate */ - udelay(5); - - /* release reset */ - regmap_set_bits(bc->regmap, BLK_SFT_RSTN, data->rst_mask); - if (data->mipi_phy_rst_mask) - regmap_set_bits(bc->regmap, BLK_MIPI_RESET_DIV, data->mipi_phy_rst_mask); - - ret = icc_bulk_set_bw(domain->num_paths, domain->paths); - if (ret) - dev_err(bc->dev, "failed to set icc bw\n"); - - /* disable upstream clocks */ - clk_bulk_disable_unprepare(data->num_clks, domain->clks); - - return 0; - -clk_disable: - clk_bulk_disable_unprepare(data->num_clks, domain->clks); -bus_put: - pm_runtime_put(bc->bus_power_dev); - - return ret; -} - -static int imx8m_blk_ctrl_power_off(struct generic_pm_domain *genpd) -{ - struct imx8m_blk_ctrl_domain *domain = to_imx8m_blk_ctrl_domain(genpd); - const struct imx8m_blk_ctrl_domain_data *data = domain->data; - struct imx8m_blk_ctrl *bc = domain->bc; - - /* put devices into reset and disable clocks */ - if (data->mipi_phy_rst_mask) - regmap_clear_bits(bc->regmap, BLK_MIPI_RESET_DIV, data->mipi_phy_rst_mask); - - regmap_clear_bits(bc->regmap, BLK_SFT_RSTN, data->rst_mask); - regmap_clear_bits(bc->regmap, BLK_CLK_EN, data->clk_mask); - - /* power down upstream GPC domain */ - pm_runtime_put(domain->power_dev); - - /* allow bus domain to suspend */ - pm_runtime_put(bc->bus_power_dev); - - return 0; -} - -static struct lock_class_key blk_ctrl_genpd_lock_class; - -static int imx8m_blk_ctrl_probe(struct platform_device *pdev) -{ - const struct imx8m_blk_ctrl_data *bc_data; - struct device *dev = &pdev->dev; - struct imx8m_blk_ctrl *bc; - void __iomem *base; - int i, ret; - - struct regmap_config regmap_config = { - .reg_bits = 32, - .val_bits = 32, - .reg_stride = 4, - }; - - bc = devm_kzalloc(dev, sizeof(*bc), GFP_KERNEL); - if (!bc) - return -ENOMEM; - - bc->dev = dev; - - bc_data = of_device_get_match_data(dev); - - base = devm_platform_ioremap_resource(pdev, 0); - if (IS_ERR(base)) - return PTR_ERR(base); - - regmap_config.max_register = bc_data->max_reg; - bc->regmap = devm_regmap_init_mmio(dev, base, ®map_config); - if (IS_ERR(bc->regmap)) - return dev_err_probe(dev, PTR_ERR(bc->regmap), - "failed to init regmap\n"); - - bc->domains = devm_kcalloc(dev, bc_data->num_domains, - sizeof(struct imx8m_blk_ctrl_domain), - GFP_KERNEL); - if (!bc->domains) - return -ENOMEM; - - bc->onecell_data.num_domains = bc_data->num_domains; - bc->onecell_data.domains = - devm_kcalloc(dev, bc_data->num_domains, - sizeof(struct generic_pm_domain *), GFP_KERNEL); - if (!bc->onecell_data.domains) - return -ENOMEM; - - bc->bus_power_dev = dev_pm_domain_attach_by_name(dev, "bus"); - if (IS_ERR(bc->bus_power_dev)) { - if (PTR_ERR(bc->bus_power_dev) == -ENODEV) - return dev_err_probe(dev, -EPROBE_DEFER, - "failed to attach power domain \"bus\"\n"); - else - return dev_err_probe(dev, PTR_ERR(bc->bus_power_dev), - "failed to attach power domain \"bus\"\n"); - } - - for (i = 0; i < bc_data->num_domains; i++) { - const struct imx8m_blk_ctrl_domain_data *data = &bc_data->domains[i]; - struct imx8m_blk_ctrl_domain *domain = &bc->domains[i]; - int j; - - domain->data = data; - domain->num_paths = data->num_paths; - - for (j = 0; j < data->num_clks; j++) - domain->clks[j].id = data->clk_names[j]; - - for (j = 0; j < data->num_paths; j++) { - domain->paths[j].name = data->path_names[j]; - /* Fake value for now, just let ICC could configure NoC mode/priority */ - domain->paths[j].avg_bw = 1; - domain->paths[j].peak_bw = 1; - } - - ret = devm_of_icc_bulk_get(dev, data->num_paths, domain->paths); - if (ret) { - if (ret != -EPROBE_DEFER) { - dev_warn_once(dev, "Could not get interconnect paths, NoC will stay unconfigured!\n"); - domain->num_paths = 0; - } else { - dev_err_probe(dev, ret, "failed to get noc entries\n"); - goto cleanup_pds; - } - } - - ret = devm_clk_bulk_get(dev, data->num_clks, domain->clks); - if (ret) { - dev_err_probe(dev, ret, "failed to get clock\n"); - goto cleanup_pds; - } - - domain->power_dev = - dev_pm_domain_attach_by_name(dev, data->gpc_name); - if (IS_ERR(domain->power_dev)) { - dev_err_probe(dev, PTR_ERR(domain->power_dev), - "failed to attach power domain \"%s\"\n", - data->gpc_name); - ret = PTR_ERR(domain->power_dev); - goto cleanup_pds; - } - - domain->genpd.name = data->name; - domain->genpd.power_on = imx8m_blk_ctrl_power_on; - domain->genpd.power_off = imx8m_blk_ctrl_power_off; - domain->bc = bc; - - ret = pm_genpd_init(&domain->genpd, NULL, true); - if (ret) { - dev_err_probe(dev, ret, - "failed to init power domain \"%s\"\n", - data->gpc_name); - dev_pm_domain_detach(domain->power_dev, true); - goto cleanup_pds; - } - - /* - * We use runtime PM to trigger power on/off of the upstream GPC - * domain, as a strict hierarchical parent/child power domain - * setup doesn't allow us to meet the sequencing requirements. - * This means we have nested locking of genpd locks, without the - * nesting being visible at the genpd level, so we need a - * separate lock class to make lockdep aware of the fact that - * this are separate domain locks that can be nested without a - * self-deadlock. - */ - lockdep_set_class(&domain->genpd.mlock, - &blk_ctrl_genpd_lock_class); - - bc->onecell_data.domains[i] = &domain->genpd; - } - - ret = of_genpd_add_provider_onecell(dev->of_node, &bc->onecell_data); - if (ret) { - dev_err_probe(dev, ret, "failed to add power domain provider\n"); - goto cleanup_pds; - } - - bc->power_nb.notifier_call = bc_data->power_notifier_fn; - ret = dev_pm_genpd_add_notifier(bc->bus_power_dev, &bc->power_nb); - if (ret) { - dev_err_probe(dev, ret, "failed to add power notifier\n"); - goto cleanup_provider; - } - - dev_set_drvdata(dev, bc); - - ret = devm_of_platform_populate(dev); - if (ret) - goto cleanup_provider; - - return 0; - -cleanup_provider: - of_genpd_del_provider(dev->of_node); -cleanup_pds: - for (i--; i >= 0; i--) { - pm_genpd_remove(&bc->domains[i].genpd); - dev_pm_domain_detach(bc->domains[i].power_dev, true); - } - - dev_pm_domain_detach(bc->bus_power_dev, true); - - return ret; -} - -static int imx8m_blk_ctrl_remove(struct platform_device *pdev) -{ - struct imx8m_blk_ctrl *bc = dev_get_drvdata(&pdev->dev); - int i; - - of_genpd_del_provider(pdev->dev.of_node); - - for (i = 0; bc->onecell_data.num_domains; i++) { - struct imx8m_blk_ctrl_domain *domain = &bc->domains[i]; - - pm_genpd_remove(&domain->genpd); - dev_pm_domain_detach(domain->power_dev, true); - } - - dev_pm_genpd_remove_notifier(bc->bus_power_dev); - - dev_pm_domain_detach(bc->bus_power_dev, true); - - return 0; -} - -#ifdef CONFIG_PM_SLEEP -static int imx8m_blk_ctrl_suspend(struct device *dev) -{ - struct imx8m_blk_ctrl *bc = dev_get_drvdata(dev); - int ret, i; - - /* - * This may look strange, but is done so the generic PM_SLEEP code - * can power down our domains and more importantly power them up again - * after resume, without tripping over our usage of runtime PM to - * control the upstream GPC domains. Things happen in the right order - * in the system suspend/resume paths due to the device parent/child - * hierarchy. - */ - ret = pm_runtime_get_sync(bc->bus_power_dev); - if (ret < 0) { - pm_runtime_put_noidle(bc->bus_power_dev); - return ret; - } - - for (i = 0; i < bc->onecell_data.num_domains; i++) { - struct imx8m_blk_ctrl_domain *domain = &bc->domains[i]; - - ret = pm_runtime_get_sync(domain->power_dev); - if (ret < 0) { - pm_runtime_put_noidle(domain->power_dev); - goto out_fail; - } - } - - return 0; - -out_fail: - for (i--; i >= 0; i--) - pm_runtime_put(bc->domains[i].power_dev); - - pm_runtime_put(bc->bus_power_dev); - - return ret; -} - -static int imx8m_blk_ctrl_resume(struct device *dev) -{ - struct imx8m_blk_ctrl *bc = dev_get_drvdata(dev); - int i; - - for (i = 0; i < bc->onecell_data.num_domains; i++) - pm_runtime_put(bc->domains[i].power_dev); - - pm_runtime_put(bc->bus_power_dev); - - return 0; -} -#endif - -static const struct dev_pm_ops imx8m_blk_ctrl_pm_ops = { - SET_SYSTEM_SLEEP_PM_OPS(imx8m_blk_ctrl_suspend, imx8m_blk_ctrl_resume) -}; - -static int imx8mm_vpu_power_notifier(struct notifier_block *nb, - unsigned long action, void *data) -{ - struct imx8m_blk_ctrl *bc = container_of(nb, struct imx8m_blk_ctrl, - power_nb); - - if (action != GENPD_NOTIFY_ON && action != GENPD_NOTIFY_PRE_OFF) - return NOTIFY_OK; - - /* - * The ADB in the VPUMIX domain has no separate reset and clock - * enable bits, but is ungated together with the VPU clocks. To - * allow the handshake with the GPC to progress we put the VPUs - * in reset and ungate the clocks. - */ - regmap_clear_bits(bc->regmap, BLK_SFT_RSTN, BIT(0) | BIT(1) | BIT(2)); - regmap_set_bits(bc->regmap, BLK_CLK_EN, BIT(0) | BIT(1) | BIT(2)); - - if (action == GENPD_NOTIFY_ON) { - /* - * On power up we have no software backchannel to the GPC to - * wait for the ADB handshake to happen, so we just delay for a - * bit. On power down the GPC driver waits for the handshake. - */ - udelay(5); - - /* set "fuse" bits to enable the VPUs */ - regmap_set_bits(bc->regmap, 0x8, 0xffffffff); - regmap_set_bits(bc->regmap, 0xc, 0xffffffff); - regmap_set_bits(bc->regmap, 0x10, 0xffffffff); - regmap_set_bits(bc->regmap, 0x14, 0xffffffff); - } - - return NOTIFY_OK; -} - -static const struct imx8m_blk_ctrl_domain_data imx8mm_vpu_blk_ctl_domain_data[] = { - [IMX8MM_VPUBLK_PD_G1] = { - .name = "vpublk-g1", - .clk_names = (const char *[]){ "g1", }, - .num_clks = 1, - .gpc_name = "g1", - .rst_mask = BIT(1), - .clk_mask = BIT(1), - }, - [IMX8MM_VPUBLK_PD_G2] = { - .name = "vpublk-g2", - .clk_names = (const char *[]){ "g2", }, - .num_clks = 1, - .gpc_name = "g2", - .rst_mask = BIT(0), - .clk_mask = BIT(0), - }, - [IMX8MM_VPUBLK_PD_H1] = { - .name = "vpublk-h1", - .clk_names = (const char *[]){ "h1", }, - .num_clks = 1, - .gpc_name = "h1", - .rst_mask = BIT(2), - .clk_mask = BIT(2), - }, -}; - -static const struct imx8m_blk_ctrl_data imx8mm_vpu_blk_ctl_dev_data = { - .max_reg = 0x18, - .power_notifier_fn = imx8mm_vpu_power_notifier, - .domains = imx8mm_vpu_blk_ctl_domain_data, - .num_domains = ARRAY_SIZE(imx8mm_vpu_blk_ctl_domain_data), -}; - -static const struct imx8m_blk_ctrl_domain_data imx8mp_vpu_blk_ctl_domain_data[] = { - [IMX8MP_VPUBLK_PD_G1] = { - .name = "vpublk-g1", - .clk_names = (const char *[]){ "g1", }, - .num_clks = 1, - .gpc_name = "g1", - .rst_mask = BIT(1), - .clk_mask = BIT(1), - .path_names = (const char *[]){"g1"}, - .num_paths = 1, - }, - [IMX8MP_VPUBLK_PD_G2] = { - .name = "vpublk-g2", - .clk_names = (const char *[]){ "g2", }, - .num_clks = 1, - .gpc_name = "g2", - .rst_mask = BIT(0), - .clk_mask = BIT(0), - .path_names = (const char *[]){"g2"}, - .num_paths = 1, - }, - [IMX8MP_VPUBLK_PD_VC8000E] = { - .name = "vpublk-vc8000e", - .clk_names = (const char *[]){ "vc8000e", }, - .num_clks = 1, - .gpc_name = "vc8000e", - .rst_mask = BIT(2), - .clk_mask = BIT(2), - .path_names = (const char *[]){"vc8000e"}, - .num_paths = 1, - }, -}; - -static const struct imx8m_blk_ctrl_data imx8mp_vpu_blk_ctl_dev_data = { - .max_reg = 0x18, - .power_notifier_fn = imx8mm_vpu_power_notifier, - .domains = imx8mp_vpu_blk_ctl_domain_data, - .num_domains = ARRAY_SIZE(imx8mp_vpu_blk_ctl_domain_data), -}; - -static int imx8mm_disp_power_notifier(struct notifier_block *nb, - unsigned long action, void *data) -{ - struct imx8m_blk_ctrl *bc = container_of(nb, struct imx8m_blk_ctrl, - power_nb); - - if (action != GENPD_NOTIFY_ON && action != GENPD_NOTIFY_PRE_OFF) - return NOTIFY_OK; - - /* Enable bus clock and deassert bus reset */ - regmap_set_bits(bc->regmap, BLK_CLK_EN, BIT(12)); - regmap_set_bits(bc->regmap, BLK_SFT_RSTN, BIT(6)); - - /* - * On power up we have no software backchannel to the GPC to - * wait for the ADB handshake to happen, so we just delay for a - * bit. On power down the GPC driver waits for the handshake. - */ - if (action == GENPD_NOTIFY_ON) - udelay(5); - - - return NOTIFY_OK; -} - -static const struct imx8m_blk_ctrl_domain_data imx8mm_disp_blk_ctl_domain_data[] = { - [IMX8MM_DISPBLK_PD_CSI_BRIDGE] = { - .name = "dispblk-csi-bridge", - .clk_names = (const char *[]){ "csi-bridge-axi", "csi-bridge-apb", - "csi-bridge-core", }, - .num_clks = 3, - .gpc_name = "csi-bridge", - .rst_mask = BIT(0) | BIT(1) | BIT(2), - .clk_mask = BIT(0) | BIT(1) | BIT(2) | BIT(3) | BIT(4) | BIT(5), - }, - [IMX8MM_DISPBLK_PD_LCDIF] = { - .name = "dispblk-lcdif", - .clk_names = (const char *[]){ "lcdif-axi", "lcdif-apb", "lcdif-pix", }, - .num_clks = 3, - .gpc_name = "lcdif", - .clk_mask = BIT(6) | BIT(7), - }, - [IMX8MM_DISPBLK_PD_MIPI_DSI] = { - .name = "dispblk-mipi-dsi", - .clk_names = (const char *[]){ "dsi-pclk", "dsi-ref", }, - .num_clks = 2, - .gpc_name = "mipi-dsi", - .rst_mask = BIT(5), - .clk_mask = BIT(8) | BIT(9), - .mipi_phy_rst_mask = BIT(17), - }, - [IMX8MM_DISPBLK_PD_MIPI_CSI] = { - .name = "dispblk-mipi-csi", - .clk_names = (const char *[]){ "csi-aclk", "csi-pclk" }, - .num_clks = 2, - .gpc_name = "mipi-csi", - .rst_mask = BIT(3) | BIT(4), - .clk_mask = BIT(10) | BIT(11), - .mipi_phy_rst_mask = BIT(16), - }, -}; - -static const struct imx8m_blk_ctrl_data imx8mm_disp_blk_ctl_dev_data = { - .max_reg = 0x2c, - .power_notifier_fn = imx8mm_disp_power_notifier, - .domains = imx8mm_disp_blk_ctl_domain_data, - .num_domains = ARRAY_SIZE(imx8mm_disp_blk_ctl_domain_data), -}; - - -static int imx8mn_disp_power_notifier(struct notifier_block *nb, - unsigned long action, void *data) -{ - struct imx8m_blk_ctrl *bc = container_of(nb, struct imx8m_blk_ctrl, - power_nb); - - if (action != GENPD_NOTIFY_ON && action != GENPD_NOTIFY_PRE_OFF) - return NOTIFY_OK; - - /* Enable bus clock and deassert bus reset */ - regmap_set_bits(bc->regmap, BLK_CLK_EN, BIT(8)); - regmap_set_bits(bc->regmap, BLK_SFT_RSTN, BIT(8)); - - /* - * On power up we have no software backchannel to the GPC to - * wait for the ADB handshake to happen, so we just delay for a - * bit. On power down the GPC driver waits for the handshake. - */ - if (action == GENPD_NOTIFY_ON) - udelay(5); - - - return NOTIFY_OK; -} - -static const struct imx8m_blk_ctrl_domain_data imx8mn_disp_blk_ctl_domain_data[] = { - [IMX8MN_DISPBLK_PD_MIPI_DSI] = { - .name = "dispblk-mipi-dsi", - .clk_names = (const char *[]){ "dsi-pclk", "dsi-ref", }, - .num_clks = 2, - .gpc_name = "mipi-dsi", - .rst_mask = BIT(0) | BIT(1), - .clk_mask = BIT(0) | BIT(1), - .mipi_phy_rst_mask = BIT(17), - }, - [IMX8MN_DISPBLK_PD_MIPI_CSI] = { - .name = "dispblk-mipi-csi", - .clk_names = (const char *[]){ "csi-aclk", "csi-pclk" }, - .num_clks = 2, - .gpc_name = "mipi-csi", - .rst_mask = BIT(2) | BIT(3), - .clk_mask = BIT(2) | BIT(3), - .mipi_phy_rst_mask = BIT(16), - }, - [IMX8MN_DISPBLK_PD_LCDIF] = { - .name = "dispblk-lcdif", - .clk_names = (const char *[]){ "lcdif-axi", "lcdif-apb", "lcdif-pix", }, - .num_clks = 3, - .gpc_name = "lcdif", - .rst_mask = BIT(4) | BIT(5), - .clk_mask = BIT(4) | BIT(5), - }, - [IMX8MN_DISPBLK_PD_ISI] = { - .name = "dispblk-isi", - .clk_names = (const char *[]){ "disp_axi", "disp_apb", "disp_axi_root", - "disp_apb_root"}, - .num_clks = 4, - .gpc_name = "isi", - .rst_mask = BIT(6) | BIT(7), - .clk_mask = BIT(6) | BIT(7), - }, -}; - -static const struct imx8m_blk_ctrl_data imx8mn_disp_blk_ctl_dev_data = { - .max_reg = 0x84, - .power_notifier_fn = imx8mn_disp_power_notifier, - .domains = imx8mn_disp_blk_ctl_domain_data, - .num_domains = ARRAY_SIZE(imx8mn_disp_blk_ctl_domain_data), -}; - -#define LCDIF_ARCACHE_CTRL 0x4c -#define LCDIF_1_RD_HURRY GENMASK(15, 13) -#define LCDIF_0_RD_HURRY GENMASK(12, 10) - -static int imx8mp_media_power_notifier(struct notifier_block *nb, - unsigned long action, void *data) -{ - struct imx8m_blk_ctrl *bc = container_of(nb, struct imx8m_blk_ctrl, - power_nb); - - if (action != GENPD_NOTIFY_ON && action != GENPD_NOTIFY_PRE_OFF) - return NOTIFY_OK; - - /* Enable bus clock and deassert bus reset */ - regmap_set_bits(bc->regmap, BLK_CLK_EN, BIT(8)); - regmap_set_bits(bc->regmap, BLK_SFT_RSTN, BIT(8)); - - if (action == GENPD_NOTIFY_ON) { - /* - * On power up we have no software backchannel to the GPC to - * wait for the ADB handshake to happen, so we just delay for a - * bit. On power down the GPC driver waits for the handshake. - */ - udelay(5); - - /* - * Set panic read hurry level for both LCDIF interfaces to - * maximum priority to minimize chances of display FIFO - * underflow. - */ - regmap_set_bits(bc->regmap, LCDIF_ARCACHE_CTRL, - FIELD_PREP(LCDIF_1_RD_HURRY, 7) | - FIELD_PREP(LCDIF_0_RD_HURRY, 7)); - } - - return NOTIFY_OK; -} - -/* - * From i.MX 8M Plus Applications Processor Reference Manual, Rev. 1, - * section 13.2.2, 13.2.3 - * isp-ahb and dwe are not in Figure 13-5. Media BLK_CTRL Clocks - */ -static const struct imx8m_blk_ctrl_domain_data imx8mp_media_blk_ctl_domain_data[] = { - [IMX8MP_MEDIABLK_PD_MIPI_DSI_1] = { - .name = "mediablk-mipi-dsi-1", - .clk_names = (const char *[]){ "apb", "phy", }, - .num_clks = 2, - .gpc_name = "mipi-dsi1", - .rst_mask = BIT(0) | BIT(1), - .clk_mask = BIT(0) | BIT(1), - .mipi_phy_rst_mask = BIT(17), - }, - [IMX8MP_MEDIABLK_PD_MIPI_CSI2_1] = { - .name = "mediablk-mipi-csi2-1", - .clk_names = (const char *[]){ "apb", "cam1" }, - .num_clks = 2, - .gpc_name = "mipi-csi1", - .rst_mask = BIT(2) | BIT(3), - .clk_mask = BIT(2) | BIT(3), - .mipi_phy_rst_mask = BIT(16), - }, - [IMX8MP_MEDIABLK_PD_LCDIF_1] = { - .name = "mediablk-lcdif-1", - .clk_names = (const char *[]){ "disp1", "apb", "axi", }, - .num_clks = 3, - .gpc_name = "lcdif1", - .rst_mask = BIT(4) | BIT(5) | BIT(23), - .clk_mask = BIT(4) | BIT(5) | BIT(23), - .path_names = (const char *[]){"lcdif-rd", "lcdif-wr"}, - .num_paths = 2, - }, - [IMX8MP_MEDIABLK_PD_ISI] = { - .name = "mediablk-isi", - .clk_names = (const char *[]){ "axi", "apb" }, - .num_clks = 2, - .gpc_name = "isi", - .rst_mask = BIT(6) | BIT(7), - .clk_mask = BIT(6) | BIT(7), - .path_names = (const char *[]){"isi0", "isi1", "isi2"}, - .num_paths = 3, - }, - [IMX8MP_MEDIABLK_PD_MIPI_CSI2_2] = { - .name = "mediablk-mipi-csi2-2", - .clk_names = (const char *[]){ "apb", "cam2" }, - .num_clks = 2, - .gpc_name = "mipi-csi2", - .rst_mask = BIT(9) | BIT(10), - .clk_mask = BIT(9) | BIT(10), - .mipi_phy_rst_mask = BIT(30), - }, - [IMX8MP_MEDIABLK_PD_LCDIF_2] = { - .name = "mediablk-lcdif-2", - .clk_names = (const char *[]){ "disp2", "apb", "axi", }, - .num_clks = 3, - .gpc_name = "lcdif2", - .rst_mask = BIT(11) | BIT(12) | BIT(24), - .clk_mask = BIT(11) | BIT(12) | BIT(24), - .path_names = (const char *[]){"lcdif-rd", "lcdif-wr"}, - .num_paths = 2, - }, - [IMX8MP_MEDIABLK_PD_ISP] = { - .name = "mediablk-isp", - .clk_names = (const char *[]){ "isp", "axi", "apb" }, - .num_clks = 3, - .gpc_name = "isp", - .rst_mask = BIT(16) | BIT(17) | BIT(18), - .clk_mask = BIT(16) | BIT(17) | BIT(18), - .path_names = (const char *[]){"isp0", "isp1"}, - .num_paths = 2, - }, - [IMX8MP_MEDIABLK_PD_DWE] = { - .name = "mediablk-dwe", - .clk_names = (const char *[]){ "axi", "apb" }, - .num_clks = 2, - .gpc_name = "dwe", - .rst_mask = BIT(19) | BIT(20) | BIT(21), - .clk_mask = BIT(19) | BIT(20) | BIT(21), - .path_names = (const char *[]){"dwe"}, - .num_paths = 1, - }, - [IMX8MP_MEDIABLK_PD_MIPI_DSI_2] = { - .name = "mediablk-mipi-dsi-2", - .clk_names = (const char *[]){ "phy", }, - .num_clks = 1, - .gpc_name = "mipi-dsi2", - .rst_mask = BIT(22), - .clk_mask = BIT(22), - .mipi_phy_rst_mask = BIT(29), - }, -}; - -static const struct imx8m_blk_ctrl_data imx8mp_media_blk_ctl_dev_data = { - .max_reg = 0x138, - .power_notifier_fn = imx8mp_media_power_notifier, - .domains = imx8mp_media_blk_ctl_domain_data, - .num_domains = ARRAY_SIZE(imx8mp_media_blk_ctl_domain_data), -}; - -static int imx8mq_vpu_power_notifier(struct notifier_block *nb, - unsigned long action, void *data) -{ - struct imx8m_blk_ctrl *bc = container_of(nb, struct imx8m_blk_ctrl, - power_nb); - - if (action != GENPD_NOTIFY_ON && action != GENPD_NOTIFY_PRE_OFF) - return NOTIFY_OK; - - /* - * The ADB in the VPUMIX domain has no separate reset and clock - * enable bits, but is ungated and reset together with the VPUs. The - * reset and clock enable inputs to the ADB is a logical OR of the - * VPU bits. In order to set the G2 fuse bits, the G2 clock must - * also be enabled. - */ - regmap_set_bits(bc->regmap, BLK_SFT_RSTN, BIT(0) | BIT(1)); - regmap_set_bits(bc->regmap, BLK_CLK_EN, BIT(0) | BIT(1)); - - if (action == GENPD_NOTIFY_ON) { - /* - * On power up we have no software backchannel to the GPC to - * wait for the ADB handshake to happen, so we just delay for a - * bit. On power down the GPC driver waits for the handshake. - */ - udelay(5); - - /* set "fuse" bits to enable the VPUs */ - regmap_set_bits(bc->regmap, 0x8, 0xffffffff); - regmap_set_bits(bc->regmap, 0xc, 0xffffffff); - regmap_set_bits(bc->regmap, 0x10, 0xffffffff); - } - - return NOTIFY_OK; -} - -static const struct imx8m_blk_ctrl_domain_data imx8mq_vpu_blk_ctl_domain_data[] = { - [IMX8MQ_VPUBLK_PD_G1] = { - .name = "vpublk-g1", - .clk_names = (const char *[]){ "g1", }, - .num_clks = 1, - .gpc_name = "g1", - .rst_mask = BIT(1), - .clk_mask = BIT(1), - }, - [IMX8MQ_VPUBLK_PD_G2] = { - .name = "vpublk-g2", - .clk_names = (const char *[]){ "g2", }, - .num_clks = 1, - .gpc_name = "g2", - .rst_mask = BIT(0), - .clk_mask = BIT(0), - }, -}; - -static const struct imx8m_blk_ctrl_data imx8mq_vpu_blk_ctl_dev_data = { - .max_reg = 0x14, - .power_notifier_fn = imx8mq_vpu_power_notifier, - .domains = imx8mq_vpu_blk_ctl_domain_data, - .num_domains = ARRAY_SIZE(imx8mq_vpu_blk_ctl_domain_data), -}; - -static const struct of_device_id imx8m_blk_ctrl_of_match[] = { - { - .compatible = "fsl,imx8mm-vpu-blk-ctrl", - .data = &imx8mm_vpu_blk_ctl_dev_data - }, { - .compatible = "fsl,imx8mm-disp-blk-ctrl", - .data = &imx8mm_disp_blk_ctl_dev_data - }, { - .compatible = "fsl,imx8mn-disp-blk-ctrl", - .data = &imx8mn_disp_blk_ctl_dev_data - }, { - .compatible = "fsl,imx8mp-media-blk-ctrl", - .data = &imx8mp_media_blk_ctl_dev_data - }, { - .compatible = "fsl,imx8mq-vpu-blk-ctrl", - .data = &imx8mq_vpu_blk_ctl_dev_data - }, { - .compatible = "fsl,imx8mp-vpu-blk-ctrl", - .data = &imx8mp_vpu_blk_ctl_dev_data - }, { - /* Sentinel */ - } -}; -MODULE_DEVICE_TABLE(of, imx8m_blk_ctrl_of_match); - -static struct platform_driver imx8m_blk_ctrl_driver = { - .probe = imx8m_blk_ctrl_probe, - .remove = imx8m_blk_ctrl_remove, - .driver = { - .name = "imx8m-blk-ctrl", - .pm = &imx8m_blk_ctrl_pm_ops, - .of_match_table = imx8m_blk_ctrl_of_match, - }, -}; -module_platform_driver(imx8m_blk_ctrl_driver); -MODULE_LICENSE("GPL"); diff --git a/drivers/genpd/imx/imx8mp-blk-ctrl.c b/drivers/genpd/imx/imx8mp-blk-ctrl.c deleted file mode 100644 index c6ac32c1a8c1..000000000000 --- a/drivers/genpd/imx/imx8mp-blk-ctrl.c +++ /dev/null @@ -1,867 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0+ - -/* - * Copyright 2022 Pengutronix, Lucas Stach - */ - -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -#include - -#define GPR_REG0 0x0 -#define PCIE_CLOCK_MODULE_EN BIT(0) -#define USB_CLOCK_MODULE_EN BIT(1) -#define PCIE_PHY_APB_RST BIT(4) -#define PCIE_PHY_INIT_RST BIT(5) -#define GPR_REG1 0x4 -#define PLL_LOCK BIT(13) -#define GPR_REG2 0x8 -#define P_PLL_MASK GENMASK(5, 0) -#define M_PLL_MASK GENMASK(15, 6) -#define S_PLL_MASK GENMASK(18, 16) -#define GPR_REG3 0xc -#define PLL_CKE BIT(17) -#define PLL_RST BIT(31) - -struct imx8mp_blk_ctrl_domain; - -struct imx8mp_blk_ctrl { - struct device *dev; - struct notifier_block power_nb; - struct device *bus_power_dev; - struct regmap *regmap; - struct imx8mp_blk_ctrl_domain *domains; - struct genpd_onecell_data onecell_data; - void (*power_off) (struct imx8mp_blk_ctrl *bc, struct imx8mp_blk_ctrl_domain *domain); - void (*power_on) (struct imx8mp_blk_ctrl *bc, struct imx8mp_blk_ctrl_domain *domain); -}; - -struct imx8mp_blk_ctrl_domain_data { - const char *name; - const char * const *clk_names; - int num_clks; - const char * const *path_names; - int num_paths; - const char *gpc_name; -}; - -#define DOMAIN_MAX_CLKS 2 -#define DOMAIN_MAX_PATHS 3 - -struct imx8mp_blk_ctrl_domain { - struct generic_pm_domain genpd; - const struct imx8mp_blk_ctrl_domain_data *data; - struct clk_bulk_data clks[DOMAIN_MAX_CLKS]; - struct icc_bulk_data paths[DOMAIN_MAX_PATHS]; - struct device *power_dev; - struct imx8mp_blk_ctrl *bc; - int num_paths; - int id; -}; - -struct imx8mp_blk_ctrl_data { - int max_reg; - int (*probe) (struct imx8mp_blk_ctrl *bc); - notifier_fn_t power_notifier_fn; - void (*power_off) (struct imx8mp_blk_ctrl *bc, struct imx8mp_blk_ctrl_domain *domain); - void (*power_on) (struct imx8mp_blk_ctrl *bc, struct imx8mp_blk_ctrl_domain *domain); - const struct imx8mp_blk_ctrl_domain_data *domains; - int num_domains; -}; - -static inline struct imx8mp_blk_ctrl_domain * -to_imx8mp_blk_ctrl_domain(struct generic_pm_domain *genpd) -{ - return container_of(genpd, struct imx8mp_blk_ctrl_domain, genpd); -} - -struct clk_hsio_pll { - struct clk_hw hw; - struct regmap *regmap; -}; - -static inline struct clk_hsio_pll *to_clk_hsio_pll(struct clk_hw *hw) -{ - return container_of(hw, struct clk_hsio_pll, hw); -} - -static int clk_hsio_pll_prepare(struct clk_hw *hw) -{ - struct clk_hsio_pll *clk = to_clk_hsio_pll(hw); - u32 val; - - /* set the PLL configuration */ - regmap_update_bits(clk->regmap, GPR_REG2, - P_PLL_MASK | M_PLL_MASK | S_PLL_MASK, - FIELD_PREP(P_PLL_MASK, 12) | - FIELD_PREP(M_PLL_MASK, 800) | - FIELD_PREP(S_PLL_MASK, 4)); - - /* de-assert PLL reset */ - regmap_update_bits(clk->regmap, GPR_REG3, PLL_RST, PLL_RST); - - /* enable PLL */ - regmap_update_bits(clk->regmap, GPR_REG3, PLL_CKE, PLL_CKE); - - return regmap_read_poll_timeout(clk->regmap, GPR_REG1, val, - val & PLL_LOCK, 10, 100); -} - -static void clk_hsio_pll_unprepare(struct clk_hw *hw) -{ - struct clk_hsio_pll *clk = to_clk_hsio_pll(hw); - - regmap_update_bits(clk->regmap, GPR_REG3, PLL_RST | PLL_CKE, 0); -} - -static int clk_hsio_pll_is_prepared(struct clk_hw *hw) -{ - struct clk_hsio_pll *clk = to_clk_hsio_pll(hw); - - return regmap_test_bits(clk->regmap, GPR_REG1, PLL_LOCK); -} - -static unsigned long clk_hsio_pll_recalc_rate(struct clk_hw *hw, - unsigned long parent_rate) -{ - return 100000000; -} - -static const struct clk_ops clk_hsio_pll_ops = { - .prepare = clk_hsio_pll_prepare, - .unprepare = clk_hsio_pll_unprepare, - .is_prepared = clk_hsio_pll_is_prepared, - .recalc_rate = clk_hsio_pll_recalc_rate, -}; - -static int imx8mp_hsio_blk_ctrl_probe(struct imx8mp_blk_ctrl *bc) -{ - struct clk_hsio_pll *clk_hsio_pll; - struct clk_hw *hw; - struct clk_init_data init = {}; - int ret; - - clk_hsio_pll = devm_kzalloc(bc->dev, sizeof(*clk_hsio_pll), GFP_KERNEL); - if (!clk_hsio_pll) - return -ENOMEM; - - init.name = "hsio_pll"; - init.ops = &clk_hsio_pll_ops; - init.parent_names = (const char *[]){"osc_24m"}; - init.num_parents = 1; - - clk_hsio_pll->regmap = bc->regmap; - clk_hsio_pll->hw.init = &init; - - hw = &clk_hsio_pll->hw; - ret = devm_clk_hw_register(bc->bus_power_dev, hw); - if (ret) - return ret; - - return devm_of_clk_add_hw_provider(bc->dev, of_clk_hw_simple_get, hw); -} - -static void imx8mp_hsio_blk_ctrl_power_on(struct imx8mp_blk_ctrl *bc, - struct imx8mp_blk_ctrl_domain *domain) -{ - switch (domain->id) { - case IMX8MP_HSIOBLK_PD_USB: - regmap_set_bits(bc->regmap, GPR_REG0, USB_CLOCK_MODULE_EN); - break; - case IMX8MP_HSIOBLK_PD_PCIE: - regmap_set_bits(bc->regmap, GPR_REG0, PCIE_CLOCK_MODULE_EN); - break; - case IMX8MP_HSIOBLK_PD_PCIE_PHY: - regmap_set_bits(bc->regmap, GPR_REG0, - PCIE_PHY_APB_RST | PCIE_PHY_INIT_RST); - break; - default: - break; - } -} - -static void imx8mp_hsio_blk_ctrl_power_off(struct imx8mp_blk_ctrl *bc, - struct imx8mp_blk_ctrl_domain *domain) -{ - switch (domain->id) { - case IMX8MP_HSIOBLK_PD_USB: - regmap_clear_bits(bc->regmap, GPR_REG0, USB_CLOCK_MODULE_EN); - break; - case IMX8MP_HSIOBLK_PD_PCIE: - regmap_clear_bits(bc->regmap, GPR_REG0, PCIE_CLOCK_MODULE_EN); - break; - case IMX8MP_HSIOBLK_PD_PCIE_PHY: - regmap_clear_bits(bc->regmap, GPR_REG0, - PCIE_PHY_APB_RST | PCIE_PHY_INIT_RST); - break; - default: - break; - } -} - -static int imx8mp_hsio_power_notifier(struct notifier_block *nb, - unsigned long action, void *data) -{ - struct imx8mp_blk_ctrl *bc = container_of(nb, struct imx8mp_blk_ctrl, - power_nb); - struct clk_bulk_data *usb_clk = bc->domains[IMX8MP_HSIOBLK_PD_USB].clks; - int num_clks = bc->domains[IMX8MP_HSIOBLK_PD_USB].data->num_clks; - int ret; - - switch (action) { - case GENPD_NOTIFY_ON: - /* - * enable USB clock for a moment for the power-on ADB handshake - * to proceed - */ - ret = clk_bulk_prepare_enable(num_clks, usb_clk); - if (ret) - return NOTIFY_BAD; - regmap_set_bits(bc->regmap, GPR_REG0, USB_CLOCK_MODULE_EN); - - udelay(5); - - regmap_clear_bits(bc->regmap, GPR_REG0, USB_CLOCK_MODULE_EN); - clk_bulk_disable_unprepare(num_clks, usb_clk); - break; - case GENPD_NOTIFY_PRE_OFF: - /* enable USB clock for the power-down ADB handshake to work */ - ret = clk_bulk_prepare_enable(num_clks, usb_clk); - if (ret) - return NOTIFY_BAD; - - regmap_set_bits(bc->regmap, GPR_REG0, USB_CLOCK_MODULE_EN); - break; - case GENPD_NOTIFY_OFF: - clk_bulk_disable_unprepare(num_clks, usb_clk); - break; - default: - break; - } - - return NOTIFY_OK; -} - -static const struct imx8mp_blk_ctrl_domain_data imx8mp_hsio_domain_data[] = { - [IMX8MP_HSIOBLK_PD_USB] = { - .name = "hsioblk-usb", - .clk_names = (const char *[]){ "usb" }, - .num_clks = 1, - .gpc_name = "usb", - .path_names = (const char *[]){"usb1", "usb2"}, - .num_paths = 2, - }, - [IMX8MP_HSIOBLK_PD_USB_PHY1] = { - .name = "hsioblk-usb-phy1", - .gpc_name = "usb-phy1", - }, - [IMX8MP_HSIOBLK_PD_USB_PHY2] = { - .name = "hsioblk-usb-phy2", - .gpc_name = "usb-phy2", - }, - [IMX8MP_HSIOBLK_PD_PCIE] = { - .name = "hsioblk-pcie", - .clk_names = (const char *[]){ "pcie" }, - .num_clks = 1, - .gpc_name = "pcie", - .path_names = (const char *[]){"noc-pcie", "pcie"}, - .num_paths = 2, - }, - [IMX8MP_HSIOBLK_PD_PCIE_PHY] = { - .name = "hsioblk-pcie-phy", - .gpc_name = "pcie-phy", - }, -}; - -static const struct imx8mp_blk_ctrl_data imx8mp_hsio_blk_ctl_dev_data = { - .max_reg = 0x24, - .probe = imx8mp_hsio_blk_ctrl_probe, - .power_on = imx8mp_hsio_blk_ctrl_power_on, - .power_off = imx8mp_hsio_blk_ctrl_power_off, - .power_notifier_fn = imx8mp_hsio_power_notifier, - .domains = imx8mp_hsio_domain_data, - .num_domains = ARRAY_SIZE(imx8mp_hsio_domain_data), -}; - -#define HDMI_RTX_RESET_CTL0 0x20 -#define HDMI_RTX_CLK_CTL0 0x40 -#define HDMI_RTX_CLK_CTL1 0x50 -#define HDMI_RTX_CLK_CTL2 0x60 -#define HDMI_RTX_CLK_CTL3 0x70 -#define HDMI_RTX_CLK_CTL4 0x80 -#define HDMI_TX_CONTROL0 0x200 -#define HDMI_LCDIF_NOC_HURRY_MASK GENMASK(14, 12) - -static void imx8mp_hdmi_blk_ctrl_power_on(struct imx8mp_blk_ctrl *bc, - struct imx8mp_blk_ctrl_domain *domain) -{ - switch (domain->id) { - case IMX8MP_HDMIBLK_PD_IRQSTEER: - regmap_set_bits(bc->regmap, HDMI_RTX_CLK_CTL0, BIT(9)); - regmap_set_bits(bc->regmap, HDMI_RTX_RESET_CTL0, BIT(16)); - break; - case IMX8MP_HDMIBLK_PD_LCDIF: - regmap_set_bits(bc->regmap, HDMI_RTX_CLK_CTL0, - BIT(16) | BIT(17) | BIT(18) | - BIT(19) | BIT(20)); - regmap_set_bits(bc->regmap, HDMI_RTX_CLK_CTL1, BIT(11)); - regmap_set_bits(bc->regmap, HDMI_RTX_RESET_CTL0, - BIT(4) | BIT(5) | BIT(6)); - regmap_set_bits(bc->regmap, HDMI_TX_CONTROL0, - FIELD_PREP(HDMI_LCDIF_NOC_HURRY_MASK, 7)); - break; - case IMX8MP_HDMIBLK_PD_PAI: - regmap_set_bits(bc->regmap, HDMI_RTX_CLK_CTL1, BIT(17)); - regmap_set_bits(bc->regmap, HDMI_RTX_RESET_CTL0, BIT(18)); - break; - case IMX8MP_HDMIBLK_PD_PVI: - regmap_set_bits(bc->regmap, HDMI_RTX_CLK_CTL1, BIT(28)); - regmap_set_bits(bc->regmap, HDMI_RTX_RESET_CTL0, BIT(22)); - break; - case IMX8MP_HDMIBLK_PD_TRNG: - regmap_set_bits(bc->regmap, HDMI_RTX_CLK_CTL1, BIT(27) | BIT(30)); - regmap_set_bits(bc->regmap, HDMI_RTX_RESET_CTL0, BIT(20)); - break; - case IMX8MP_HDMIBLK_PD_HDMI_TX: - regmap_set_bits(bc->regmap, HDMI_RTX_CLK_CTL0, - BIT(2) | BIT(4) | BIT(5)); - regmap_set_bits(bc->regmap, HDMI_RTX_CLK_CTL1, - BIT(12) | BIT(13) | BIT(14) | BIT(15) | BIT(16) | - BIT(18) | BIT(19) | BIT(20) | BIT(21)); - regmap_set_bits(bc->regmap, HDMI_RTX_RESET_CTL0, - BIT(7) | BIT(10) | BIT(11)); - regmap_set_bits(bc->regmap, HDMI_TX_CONTROL0, BIT(1)); - break; - case IMX8MP_HDMIBLK_PD_HDMI_TX_PHY: - regmap_set_bits(bc->regmap, HDMI_RTX_CLK_CTL0, BIT(7)); - regmap_set_bits(bc->regmap, HDMI_RTX_CLK_CTL1, BIT(22) | BIT(24)); - regmap_set_bits(bc->regmap, HDMI_RTX_RESET_CTL0, BIT(12)); - regmap_clear_bits(bc->regmap, HDMI_TX_CONTROL0, BIT(3)); - break; - case IMX8MP_HDMIBLK_PD_HDCP: - regmap_set_bits(bc->regmap, HDMI_RTX_CLK_CTL0, BIT(11)); - break; - case IMX8MP_HDMIBLK_PD_HRV: - regmap_set_bits(bc->regmap, HDMI_RTX_CLK_CTL1, BIT(3) | BIT(4) | BIT(5)); - regmap_set_bits(bc->regmap, HDMI_RTX_RESET_CTL0, BIT(15)); - break; - default: - break; - } -} - -static void imx8mp_hdmi_blk_ctrl_power_off(struct imx8mp_blk_ctrl *bc, - struct imx8mp_blk_ctrl_domain *domain) -{ - switch (domain->id) { - case IMX8MP_HDMIBLK_PD_IRQSTEER: - regmap_clear_bits(bc->regmap, HDMI_RTX_CLK_CTL0, BIT(9)); - regmap_clear_bits(bc->regmap, HDMI_RTX_RESET_CTL0, BIT(16)); - break; - case IMX8MP_HDMIBLK_PD_LCDIF: - regmap_clear_bits(bc->regmap, HDMI_RTX_RESET_CTL0, - BIT(4) | BIT(5) | BIT(6)); - regmap_clear_bits(bc->regmap, HDMI_RTX_CLK_CTL1, BIT(11)); - regmap_clear_bits(bc->regmap, HDMI_RTX_CLK_CTL0, - BIT(16) | BIT(17) | BIT(18) | - BIT(19) | BIT(20)); - break; - case IMX8MP_HDMIBLK_PD_PAI: - regmap_clear_bits(bc->regmap, HDMI_RTX_RESET_CTL0, BIT(18)); - regmap_clear_bits(bc->regmap, HDMI_RTX_CLK_CTL1, BIT(17)); - break; - case IMX8MP_HDMIBLK_PD_PVI: - regmap_clear_bits(bc->regmap, HDMI_RTX_RESET_CTL0, BIT(22)); - regmap_clear_bits(bc->regmap, HDMI_RTX_CLK_CTL1, BIT(28)); - break; - case IMX8MP_HDMIBLK_PD_TRNG: - regmap_clear_bits(bc->regmap, HDMI_RTX_RESET_CTL0, BIT(20)); - regmap_clear_bits(bc->regmap, HDMI_RTX_CLK_CTL1, BIT(27) | BIT(30)); - break; - case IMX8MP_HDMIBLK_PD_HDMI_TX: - regmap_clear_bits(bc->regmap, HDMI_TX_CONTROL0, BIT(1)); - regmap_clear_bits(bc->regmap, HDMI_RTX_RESET_CTL0, - BIT(7) | BIT(10) | BIT(11)); - regmap_clear_bits(bc->regmap, HDMI_RTX_CLK_CTL1, - BIT(12) | BIT(13) | BIT(14) | BIT(15) | BIT(16) | - BIT(18) | BIT(19) | BIT(20) | BIT(21)); - regmap_clear_bits(bc->regmap, HDMI_RTX_CLK_CTL0, - BIT(2) | BIT(4) | BIT(5)); - break; - case IMX8MP_HDMIBLK_PD_HDMI_TX_PHY: - regmap_set_bits(bc->regmap, HDMI_TX_CONTROL0, BIT(3)); - regmap_clear_bits(bc->regmap, HDMI_RTX_RESET_CTL0, BIT(12)); - regmap_clear_bits(bc->regmap, HDMI_RTX_CLK_CTL0, BIT(7)); - regmap_clear_bits(bc->regmap, HDMI_RTX_CLK_CTL1, BIT(22) | BIT(24)); - break; - case IMX8MP_HDMIBLK_PD_HDCP: - regmap_clear_bits(bc->regmap, HDMI_RTX_CLK_CTL0, BIT(11)); - break; - case IMX8MP_HDMIBLK_PD_HRV: - regmap_clear_bits(bc->regmap, HDMI_RTX_RESET_CTL0, BIT(15)); - regmap_clear_bits(bc->regmap, HDMI_RTX_CLK_CTL1, BIT(3) | BIT(4) | BIT(5)); - break; - default: - break; - } -} - -static int imx8mp_hdmi_power_notifier(struct notifier_block *nb, - unsigned long action, void *data) -{ - struct imx8mp_blk_ctrl *bc = container_of(nb, struct imx8mp_blk_ctrl, - power_nb); - - if (action != GENPD_NOTIFY_ON) - return NOTIFY_OK; - - /* - * Contrary to other blk-ctrls the reset and clock don't clear when the - * power domain is powered down. To ensure the proper reset pulsing, - * first clear them all to asserted state, then enable the bus clocks - * and then release the ADB reset. - */ - regmap_write(bc->regmap, HDMI_RTX_RESET_CTL0, 0x0); - regmap_write(bc->regmap, HDMI_RTX_CLK_CTL0, 0x0); - regmap_write(bc->regmap, HDMI_RTX_CLK_CTL1, 0x0); - regmap_set_bits(bc->regmap, HDMI_RTX_CLK_CTL0, - BIT(0) | BIT(1) | BIT(10)); - regmap_set_bits(bc->regmap, HDMI_RTX_RESET_CTL0, BIT(0)); - - /* - * On power up we have no software backchannel to the GPC to - * wait for the ADB handshake to happen, so we just delay for a - * bit. On power down the GPC driver waits for the handshake. - */ - udelay(5); - - return NOTIFY_OK; -} - -static const struct imx8mp_blk_ctrl_domain_data imx8mp_hdmi_domain_data[] = { - [IMX8MP_HDMIBLK_PD_IRQSTEER] = { - .name = "hdmiblk-irqsteer", - .clk_names = (const char *[]){ "apb" }, - .num_clks = 1, - .gpc_name = "irqsteer", - }, - [IMX8MP_HDMIBLK_PD_LCDIF] = { - .name = "hdmiblk-lcdif", - .clk_names = (const char *[]){ "axi", "apb" }, - .num_clks = 2, - .gpc_name = "lcdif", - .path_names = (const char *[]){"lcdif-hdmi"}, - .num_paths = 1, - }, - [IMX8MP_HDMIBLK_PD_PAI] = { - .name = "hdmiblk-pai", - .clk_names = (const char *[]){ "apb" }, - .num_clks = 1, - .gpc_name = "pai", - }, - [IMX8MP_HDMIBLK_PD_PVI] = { - .name = "hdmiblk-pvi", - .clk_names = (const char *[]){ "apb" }, - .num_clks = 1, - .gpc_name = "pvi", - }, - [IMX8MP_HDMIBLK_PD_TRNG] = { - .name = "hdmiblk-trng", - .clk_names = (const char *[]){ "apb" }, - .num_clks = 1, - .gpc_name = "trng", - }, - [IMX8MP_HDMIBLK_PD_HDMI_TX] = { - .name = "hdmiblk-hdmi-tx", - .clk_names = (const char *[]){ "apb", "ref_266m" }, - .num_clks = 2, - .gpc_name = "hdmi-tx", - }, - [IMX8MP_HDMIBLK_PD_HDMI_TX_PHY] = { - .name = "hdmiblk-hdmi-tx-phy", - .clk_names = (const char *[]){ "apb", "ref_24m" }, - .num_clks = 2, - .gpc_name = "hdmi-tx-phy", - }, - [IMX8MP_HDMIBLK_PD_HRV] = { - .name = "hdmiblk-hrv", - .clk_names = (const char *[]){ "axi", "apb" }, - .num_clks = 2, - .gpc_name = "hrv", - .path_names = (const char *[]){"hrv"}, - .num_paths = 1, - }, - [IMX8MP_HDMIBLK_PD_HDCP] = { - .name = "hdmiblk-hdcp", - .clk_names = (const char *[]){ "axi", "apb" }, - .num_clks = 2, - .gpc_name = "hdcp", - .path_names = (const char *[]){"hdcp"}, - .num_paths = 1, - }, -}; - -static const struct imx8mp_blk_ctrl_data imx8mp_hdmi_blk_ctl_dev_data = { - .max_reg = 0x23c, - .power_on = imx8mp_hdmi_blk_ctrl_power_on, - .power_off = imx8mp_hdmi_blk_ctrl_power_off, - .power_notifier_fn = imx8mp_hdmi_power_notifier, - .domains = imx8mp_hdmi_domain_data, - .num_domains = ARRAY_SIZE(imx8mp_hdmi_domain_data), -}; - -static int imx8mp_blk_ctrl_power_on(struct generic_pm_domain *genpd) -{ - struct imx8mp_blk_ctrl_domain *domain = to_imx8mp_blk_ctrl_domain(genpd); - const struct imx8mp_blk_ctrl_domain_data *data = domain->data; - struct imx8mp_blk_ctrl *bc = domain->bc; - int ret; - - /* make sure bus domain is awake */ - ret = pm_runtime_resume_and_get(bc->bus_power_dev); - if (ret < 0) { - dev_err(bc->dev, "failed to power up bus domain\n"); - return ret; - } - - /* enable upstream clocks */ - ret = clk_bulk_prepare_enable(data->num_clks, domain->clks); - if (ret) { - dev_err(bc->dev, "failed to enable clocks\n"); - goto bus_put; - } - - /* domain specific blk-ctrl manipulation */ - bc->power_on(bc, domain); - - /* power up upstream GPC domain */ - ret = pm_runtime_resume_and_get(domain->power_dev); - if (ret < 0) { - dev_err(bc->dev, "failed to power up peripheral domain\n"); - goto clk_disable; - } - - ret = icc_bulk_set_bw(domain->num_paths, domain->paths); - if (ret) - dev_err(bc->dev, "failed to set icc bw\n"); - - clk_bulk_disable_unprepare(data->num_clks, domain->clks); - - return 0; - -clk_disable: - clk_bulk_disable_unprepare(data->num_clks, domain->clks); -bus_put: - pm_runtime_put(bc->bus_power_dev); - - return ret; -} - -static int imx8mp_blk_ctrl_power_off(struct generic_pm_domain *genpd) -{ - struct imx8mp_blk_ctrl_domain *domain = to_imx8mp_blk_ctrl_domain(genpd); - const struct imx8mp_blk_ctrl_domain_data *data = domain->data; - struct imx8mp_blk_ctrl *bc = domain->bc; - int ret; - - ret = clk_bulk_prepare_enable(data->num_clks, domain->clks); - if (ret) { - dev_err(bc->dev, "failed to enable clocks\n"); - return ret; - } - - /* domain specific blk-ctrl manipulation */ - bc->power_off(bc, domain); - - clk_bulk_disable_unprepare(data->num_clks, domain->clks); - - /* power down upstream GPC domain */ - pm_runtime_put(domain->power_dev); - - /* allow bus domain to suspend */ - pm_runtime_put(bc->bus_power_dev); - - return 0; -} - -static struct lock_class_key blk_ctrl_genpd_lock_class; - -static int imx8mp_blk_ctrl_probe(struct platform_device *pdev) -{ - const struct imx8mp_blk_ctrl_data *bc_data; - struct device *dev = &pdev->dev; - struct imx8mp_blk_ctrl *bc; - void __iomem *base; - int num_domains, i, ret; - - struct regmap_config regmap_config = { - .reg_bits = 32, - .val_bits = 32, - .reg_stride = 4, - }; - - bc = devm_kzalloc(dev, sizeof(*bc), GFP_KERNEL); - if (!bc) - return -ENOMEM; - - bc->dev = dev; - - bc_data = of_device_get_match_data(dev); - num_domains = bc_data->num_domains; - - base = devm_platform_ioremap_resource(pdev, 0); - if (IS_ERR(base)) - return PTR_ERR(base); - - regmap_config.max_register = bc_data->max_reg; - bc->regmap = devm_regmap_init_mmio(dev, base, ®map_config); - if (IS_ERR(bc->regmap)) - return dev_err_probe(dev, PTR_ERR(bc->regmap), - "failed to init regmap\n"); - - bc->domains = devm_kcalloc(dev, num_domains, - sizeof(struct imx8mp_blk_ctrl_domain), - GFP_KERNEL); - if (!bc->domains) - return -ENOMEM; - - bc->onecell_data.num_domains = num_domains; - bc->onecell_data.domains = - devm_kcalloc(dev, num_domains, - sizeof(struct generic_pm_domain *), GFP_KERNEL); - if (!bc->onecell_data.domains) - return -ENOMEM; - - bc->bus_power_dev = dev_pm_domain_attach_by_name(dev, "bus"); - if (IS_ERR(bc->bus_power_dev)) - return dev_err_probe(dev, PTR_ERR(bc->bus_power_dev), - "failed to attach bus power domain\n"); - - bc->power_off = bc_data->power_off; - bc->power_on = bc_data->power_on; - - for (i = 0; i < num_domains; i++) { - const struct imx8mp_blk_ctrl_domain_data *data = &bc_data->domains[i]; - struct imx8mp_blk_ctrl_domain *domain = &bc->domains[i]; - int j; - - domain->data = data; - domain->num_paths = data->num_paths; - - for (j = 0; j < data->num_clks; j++) - domain->clks[j].id = data->clk_names[j]; - - for (j = 0; j < data->num_paths; j++) { - domain->paths[j].name = data->path_names[j]; - /* Fake value for now, just let ICC could configure NoC mode/priority */ - domain->paths[j].avg_bw = 1; - domain->paths[j].peak_bw = 1; - } - - ret = devm_of_icc_bulk_get(dev, data->num_paths, domain->paths); - if (ret) { - if (ret != -EPROBE_DEFER) { - dev_warn_once(dev, "Could not get interconnect paths, NoC will stay unconfigured!\n"); - domain->num_paths = 0; - } else { - dev_err_probe(dev, ret, "failed to get noc entries\n"); - goto cleanup_pds; - } - } - - ret = devm_clk_bulk_get(dev, data->num_clks, domain->clks); - if (ret) { - dev_err_probe(dev, ret, "failed to get clock\n"); - goto cleanup_pds; - } - - domain->power_dev = - dev_pm_domain_attach_by_name(dev, data->gpc_name); - if (IS_ERR(domain->power_dev)) { - dev_err_probe(dev, PTR_ERR(domain->power_dev), - "failed to attach power domain %s\n", - data->gpc_name); - ret = PTR_ERR(domain->power_dev); - goto cleanup_pds; - } - - domain->genpd.name = data->name; - domain->genpd.power_on = imx8mp_blk_ctrl_power_on; - domain->genpd.power_off = imx8mp_blk_ctrl_power_off; - domain->bc = bc; - domain->id = i; - - ret = pm_genpd_init(&domain->genpd, NULL, true); - if (ret) { - dev_err_probe(dev, ret, "failed to init power domain\n"); - dev_pm_domain_detach(domain->power_dev, true); - goto cleanup_pds; - } - - /* - * We use runtime PM to trigger power on/off of the upstream GPC - * domain, as a strict hierarchical parent/child power domain - * setup doesn't allow us to meet the sequencing requirements. - * This means we have nested locking of genpd locks, without the - * nesting being visible at the genpd level, so we need a - * separate lock class to make lockdep aware of the fact that - * this are separate domain locks that can be nested without a - * self-deadlock. - */ - lockdep_set_class(&domain->genpd.mlock, - &blk_ctrl_genpd_lock_class); - - bc->onecell_data.domains[i] = &domain->genpd; - } - - ret = of_genpd_add_provider_onecell(dev->of_node, &bc->onecell_data); - if (ret) { - dev_err_probe(dev, ret, "failed to add power domain provider\n"); - goto cleanup_pds; - } - - bc->power_nb.notifier_call = bc_data->power_notifier_fn; - ret = dev_pm_genpd_add_notifier(bc->bus_power_dev, &bc->power_nb); - if (ret) { - dev_err_probe(dev, ret, "failed to add power notifier\n"); - goto cleanup_provider; - } - - if (bc_data->probe) { - ret = bc_data->probe(bc); - if (ret) - goto cleanup_provider; - } - - dev_set_drvdata(dev, bc); - - return 0; - -cleanup_provider: - of_genpd_del_provider(dev->of_node); -cleanup_pds: - for (i--; i >= 0; i--) { - pm_genpd_remove(&bc->domains[i].genpd); - dev_pm_domain_detach(bc->domains[i].power_dev, true); - } - - dev_pm_domain_detach(bc->bus_power_dev, true); - - return ret; -} - -static int imx8mp_blk_ctrl_remove(struct platform_device *pdev) -{ - struct imx8mp_blk_ctrl *bc = dev_get_drvdata(&pdev->dev); - int i; - - of_genpd_del_provider(pdev->dev.of_node); - - for (i = 0; bc->onecell_data.num_domains; i++) { - struct imx8mp_blk_ctrl_domain *domain = &bc->domains[i]; - - pm_genpd_remove(&domain->genpd); - dev_pm_domain_detach(domain->power_dev, true); - } - - dev_pm_genpd_remove_notifier(bc->bus_power_dev); - - dev_pm_domain_detach(bc->bus_power_dev, true); - - return 0; -} - -#ifdef CONFIG_PM_SLEEP -static int imx8mp_blk_ctrl_suspend(struct device *dev) -{ - struct imx8mp_blk_ctrl *bc = dev_get_drvdata(dev); - int ret, i; - - /* - * This may look strange, but is done so the generic PM_SLEEP code - * can power down our domains and more importantly power them up again - * after resume, without tripping over our usage of runtime PM to - * control the upstream GPC domains. Things happen in the right order - * in the system suspend/resume paths due to the device parent/child - * hierarchy. - */ - ret = pm_runtime_get_sync(bc->bus_power_dev); - if (ret < 0) { - pm_runtime_put_noidle(bc->bus_power_dev); - return ret; - } - - for (i = 0; i < bc->onecell_data.num_domains; i++) { - struct imx8mp_blk_ctrl_domain *domain = &bc->domains[i]; - - ret = pm_runtime_get_sync(domain->power_dev); - if (ret < 0) { - pm_runtime_put_noidle(domain->power_dev); - goto out_fail; - } - } - - return 0; - -out_fail: - for (i--; i >= 0; i--) - pm_runtime_put(bc->domains[i].power_dev); - - pm_runtime_put(bc->bus_power_dev); - - return ret; -} - -static int imx8mp_blk_ctrl_resume(struct device *dev) -{ - struct imx8mp_blk_ctrl *bc = dev_get_drvdata(dev); - int i; - - for (i = 0; i < bc->onecell_data.num_domains; i++) - pm_runtime_put(bc->domains[i].power_dev); - - pm_runtime_put(bc->bus_power_dev); - - return 0; -} -#endif - -static const struct dev_pm_ops imx8mp_blk_ctrl_pm_ops = { - SET_SYSTEM_SLEEP_PM_OPS(imx8mp_blk_ctrl_suspend, - imx8mp_blk_ctrl_resume) -}; - -static const struct of_device_id imx8mp_blk_ctrl_of_match[] = { - { - .compatible = "fsl,imx8mp-hsio-blk-ctrl", - .data = &imx8mp_hsio_blk_ctl_dev_data, - }, { - .compatible = "fsl,imx8mp-hdmi-blk-ctrl", - .data = &imx8mp_hdmi_blk_ctl_dev_data, - }, { - /* Sentinel */ - } -}; -MODULE_DEVICE_TABLE(of, imx8mp_blk_ctrl_of_match); - -static struct platform_driver imx8mp_blk_ctrl_driver = { - .probe = imx8mp_blk_ctrl_probe, - .remove = imx8mp_blk_ctrl_remove, - .driver = { - .name = "imx8mp-blk-ctrl", - .pm = &imx8mp_blk_ctrl_pm_ops, - .of_match_table = imx8mp_blk_ctrl_of_match, - }, -}; -module_platform_driver(imx8mp_blk_ctrl_driver); -MODULE_LICENSE("GPL"); diff --git a/drivers/genpd/imx/imx93-blk-ctrl.c b/drivers/genpd/imx/imx93-blk-ctrl.c deleted file mode 100644 index 40bd90f8b977..000000000000 --- a/drivers/genpd/imx/imx93-blk-ctrl.c +++ /dev/null @@ -1,451 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * Copyright 2022 NXP, Peng Fan - */ - -#include -#include -#include -#include -#include -#include -#include -#include -#include - -#include - -#define BLK_SFT_RSTN 0x0 -#define BLK_CLK_EN 0x4 -#define BLK_MAX_CLKS 4 - -#define DOMAIN_MAX_CLKS 4 - -#define LCDIF_QOS_REG 0xC -#define LCDIF_DEFAULT_QOS_OFF 12 -#define LCDIF_CFG_QOS_OFF 8 - -#define PXP_QOS_REG 0x10 -#define PXP_R_DEFAULT_QOS_OFF 28 -#define PXP_R_CFG_QOS_OFF 24 -#define PXP_W_DEFAULT_QOS_OFF 20 -#define PXP_W_CFG_QOS_OFF 16 - -#define ISI_CACHE_REG 0x14 - -#define ISI_QOS_REG 0x1C -#define ISI_V_DEFAULT_QOS_OFF 28 -#define ISI_V_CFG_QOS_OFF 24 -#define ISI_U_DEFAULT_QOS_OFF 20 -#define ISI_U_CFG_QOS_OFF 16 -#define ISI_Y_R_DEFAULT_QOS_OFF 12 -#define ISI_Y_R_CFG_QOS_OFF 8 -#define ISI_Y_W_DEFAULT_QOS_OFF 4 -#define ISI_Y_W_CFG_QOS_OFF 0 - -#define PRIO_MASK 0xF - -#define PRIO(X) (X) - -struct imx93_blk_ctrl_domain; - -struct imx93_blk_ctrl { - struct device *dev; - struct regmap *regmap; - int num_clks; - struct clk_bulk_data clks[BLK_MAX_CLKS]; - struct imx93_blk_ctrl_domain *domains; - struct genpd_onecell_data onecell_data; -}; - -#define DOMAIN_MAX_QOS 4 - -struct imx93_blk_ctrl_qos { - u32 reg; - u32 cfg_off; - u32 default_prio; - u32 cfg_prio; -}; - -struct imx93_blk_ctrl_domain_data { - const char *name; - const char * const *clk_names; - int num_clks; - u32 rst_mask; - u32 clk_mask; - int num_qos; - struct imx93_blk_ctrl_qos qos[DOMAIN_MAX_QOS]; -}; - -struct imx93_blk_ctrl_domain { - struct generic_pm_domain genpd; - const struct imx93_blk_ctrl_domain_data *data; - struct clk_bulk_data clks[DOMAIN_MAX_CLKS]; - struct imx93_blk_ctrl *bc; -}; - -struct imx93_blk_ctrl_data { - const struct imx93_blk_ctrl_domain_data *domains; - int num_domains; - const char * const *clk_names; - int num_clks; - const struct regmap_access_table *reg_access_table; -}; - -static inline struct imx93_blk_ctrl_domain * -to_imx93_blk_ctrl_domain(struct generic_pm_domain *genpd) -{ - return container_of(genpd, struct imx93_blk_ctrl_domain, genpd); -} - -static int imx93_blk_ctrl_set_qos(struct imx93_blk_ctrl_domain *domain) -{ - const struct imx93_blk_ctrl_domain_data *data = domain->data; - struct imx93_blk_ctrl *bc = domain->bc; - const struct imx93_blk_ctrl_qos *qos; - u32 val, mask; - int i; - - for (i = 0; i < data->num_qos; i++) { - qos = &data->qos[i]; - - mask = PRIO_MASK << qos->cfg_off; - mask |= PRIO_MASK << (qos->cfg_off + 4); - val = qos->cfg_prio << qos->cfg_off; - val |= qos->default_prio << (qos->cfg_off + 4); - - regmap_write_bits(bc->regmap, qos->reg, mask, val); - - dev_dbg(bc->dev, "data->qos[i].reg 0x%x 0x%x\n", qos->reg, val); - } - - return 0; -} - -static int imx93_blk_ctrl_power_on(struct generic_pm_domain *genpd) -{ - struct imx93_blk_ctrl_domain *domain = to_imx93_blk_ctrl_domain(genpd); - const struct imx93_blk_ctrl_domain_data *data = domain->data; - struct imx93_blk_ctrl *bc = domain->bc; - int ret; - - ret = clk_bulk_prepare_enable(bc->num_clks, bc->clks); - if (ret) { - dev_err(bc->dev, "failed to enable bus clocks\n"); - return ret; - } - - ret = clk_bulk_prepare_enable(data->num_clks, domain->clks); - if (ret) { - clk_bulk_disable_unprepare(bc->num_clks, bc->clks); - dev_err(bc->dev, "failed to enable clocks\n"); - return ret; - } - - ret = pm_runtime_get_sync(bc->dev); - if (ret < 0) { - pm_runtime_put_noidle(bc->dev); - dev_err(bc->dev, "failed to power up domain\n"); - goto disable_clk; - } - - /* ungate clk */ - regmap_clear_bits(bc->regmap, BLK_CLK_EN, data->clk_mask); - - /* release reset */ - regmap_set_bits(bc->regmap, BLK_SFT_RSTN, data->rst_mask); - - dev_dbg(bc->dev, "pd_on: name: %s\n", genpd->name); - - return imx93_blk_ctrl_set_qos(domain); - -disable_clk: - clk_bulk_disable_unprepare(data->num_clks, domain->clks); - - clk_bulk_disable_unprepare(bc->num_clks, bc->clks); - - return ret; -} - -static int imx93_blk_ctrl_power_off(struct generic_pm_domain *genpd) -{ - struct imx93_blk_ctrl_domain *domain = to_imx93_blk_ctrl_domain(genpd); - const struct imx93_blk_ctrl_domain_data *data = domain->data; - struct imx93_blk_ctrl *bc = domain->bc; - - dev_dbg(bc->dev, "pd_off: name: %s\n", genpd->name); - - regmap_clear_bits(bc->regmap, BLK_SFT_RSTN, data->rst_mask); - regmap_set_bits(bc->regmap, BLK_CLK_EN, data->clk_mask); - - pm_runtime_put(bc->dev); - - clk_bulk_disable_unprepare(data->num_clks, domain->clks); - - clk_bulk_disable_unprepare(bc->num_clks, bc->clks); - - return 0; -} - -static struct lock_class_key blk_ctrl_genpd_lock_class; - -static int imx93_blk_ctrl_probe(struct platform_device *pdev) -{ - struct device *dev = &pdev->dev; - const struct imx93_blk_ctrl_data *bc_data = of_device_get_match_data(dev); - struct imx93_blk_ctrl *bc; - void __iomem *base; - int i, ret; - - struct regmap_config regmap_config = { - .reg_bits = 32, - .val_bits = 32, - .reg_stride = 4, - .rd_table = bc_data->reg_access_table, - .wr_table = bc_data->reg_access_table, - .max_register = SZ_4K, - }; - - bc = devm_kzalloc(dev, sizeof(*bc), GFP_KERNEL); - if (!bc) - return -ENOMEM; - - bc->dev = dev; - - base = devm_platform_ioremap_resource(pdev, 0); - if (IS_ERR(base)) - return PTR_ERR(base); - - bc->regmap = devm_regmap_init_mmio(dev, base, ®map_config); - if (IS_ERR(bc->regmap)) - return dev_err_probe(dev, PTR_ERR(bc->regmap), - "failed to init regmap\n"); - - bc->domains = devm_kcalloc(dev, bc_data->num_domains, - sizeof(struct imx93_blk_ctrl_domain), - GFP_KERNEL); - if (!bc->domains) - return -ENOMEM; - - bc->onecell_data.num_domains = bc_data->num_domains; - bc->onecell_data.domains = - devm_kcalloc(dev, bc_data->num_domains, - sizeof(struct generic_pm_domain *), GFP_KERNEL); - if (!bc->onecell_data.domains) - return -ENOMEM; - - for (i = 0; i < bc_data->num_clks; i++) - bc->clks[i].id = bc_data->clk_names[i]; - bc->num_clks = bc_data->num_clks; - - ret = devm_clk_bulk_get(dev, bc->num_clks, bc->clks); - if (ret) { - dev_err_probe(dev, ret, "failed to get bus clock\n"); - return ret; - } - - for (i = 0; i < bc_data->num_domains; i++) { - const struct imx93_blk_ctrl_domain_data *data = &bc_data->domains[i]; - struct imx93_blk_ctrl_domain *domain = &bc->domains[i]; - int j; - - domain->data = data; - - for (j = 0; j < data->num_clks; j++) - domain->clks[j].id = data->clk_names[j]; - - ret = devm_clk_bulk_get(dev, data->num_clks, domain->clks); - if (ret) { - dev_err_probe(dev, ret, "failed to get clock\n"); - goto cleanup_pds; - } - - domain->genpd.name = data->name; - domain->genpd.power_on = imx93_blk_ctrl_power_on; - domain->genpd.power_off = imx93_blk_ctrl_power_off; - domain->bc = bc; - - ret = pm_genpd_init(&domain->genpd, NULL, true); - if (ret) { - dev_err_probe(dev, ret, "failed to init power domain\n"); - goto cleanup_pds; - } - - /* - * We use runtime PM to trigger power on/off of the upstream GPC - * domain, as a strict hierarchical parent/child power domain - * setup doesn't allow us to meet the sequencing requirements. - * This means we have nested locking of genpd locks, without the - * nesting being visible at the genpd level, so we need a - * separate lock class to make lockdep aware of the fact that - * this are separate domain locks that can be nested without a - * self-deadlock. - */ - lockdep_set_class(&domain->genpd.mlock, - &blk_ctrl_genpd_lock_class); - - bc->onecell_data.domains[i] = &domain->genpd; - } - - pm_runtime_enable(dev); - - ret = of_genpd_add_provider_onecell(dev->of_node, &bc->onecell_data); - if (ret) { - dev_err_probe(dev, ret, "failed to add power domain provider\n"); - goto cleanup_pds; - } - - dev_set_drvdata(dev, bc); - - return 0; - -cleanup_pds: - for (i--; i >= 0; i--) - pm_genpd_remove(&bc->domains[i].genpd); - - return ret; -} - -static int imx93_blk_ctrl_remove(struct platform_device *pdev) -{ - struct imx93_blk_ctrl *bc = dev_get_drvdata(&pdev->dev); - int i; - - of_genpd_del_provider(pdev->dev.of_node); - - for (i = 0; bc->onecell_data.num_domains; i++) { - struct imx93_blk_ctrl_domain *domain = &bc->domains[i]; - - pm_genpd_remove(&domain->genpd); - } - - return 0; -} - -static const struct imx93_blk_ctrl_domain_data imx93_media_blk_ctl_domain_data[] = { - [IMX93_MEDIABLK_PD_MIPI_DSI] = { - .name = "mediablk-mipi-dsi", - .clk_names = (const char *[]){ "dsi" }, - .num_clks = 1, - .rst_mask = BIT(11) | BIT(12), - .clk_mask = BIT(11) | BIT(12), - }, - [IMX93_MEDIABLK_PD_MIPI_CSI] = { - .name = "mediablk-mipi-csi", - .clk_names = (const char *[]){ "cam", "csi" }, - .num_clks = 2, - .rst_mask = BIT(9) | BIT(10), - .clk_mask = BIT(9) | BIT(10), - }, - [IMX93_MEDIABLK_PD_PXP] = { - .name = "mediablk-pxp", - .clk_names = (const char *[]){ "pxp" }, - .num_clks = 1, - .rst_mask = BIT(7) | BIT(8), - .clk_mask = BIT(7) | BIT(8), - .num_qos = 2, - .qos = { - { - .reg = PXP_QOS_REG, - .cfg_off = PXP_R_CFG_QOS_OFF, - .default_prio = PRIO(3), - .cfg_prio = PRIO(6), - }, { - .reg = PXP_QOS_REG, - .cfg_off = PXP_W_CFG_QOS_OFF, - .default_prio = PRIO(3), - .cfg_prio = PRIO(6), - } - } - }, - [IMX93_MEDIABLK_PD_LCDIF] = { - .name = "mediablk-lcdif", - .clk_names = (const char *[]){ "disp", "lcdif" }, - .num_clks = 2, - .rst_mask = BIT(4) | BIT(5) | BIT(6), - .clk_mask = BIT(4) | BIT(5) | BIT(6), - .num_qos = 1, - .qos = { - { - .reg = LCDIF_QOS_REG, - .cfg_off = LCDIF_CFG_QOS_OFF, - .default_prio = PRIO(3), - .cfg_prio = PRIO(7), - } - } - }, - [IMX93_MEDIABLK_PD_ISI] = { - .name = "mediablk-isi", - .clk_names = (const char *[]){ "isi" }, - .num_clks = 1, - .rst_mask = BIT(2) | BIT(3), - .clk_mask = BIT(2) | BIT(3), - .num_qos = 4, - .qos = { - { - .reg = ISI_QOS_REG, - .cfg_off = ISI_Y_W_CFG_QOS_OFF, - .default_prio = PRIO(3), - .cfg_prio = PRIO(7), - }, { - .reg = ISI_QOS_REG, - .cfg_off = ISI_Y_R_CFG_QOS_OFF, - .default_prio = PRIO(3), - .cfg_prio = PRIO(7), - }, { - .reg = ISI_QOS_REG, - .cfg_off = ISI_U_CFG_QOS_OFF, - .default_prio = PRIO(3), - .cfg_prio = PRIO(7), - }, { - .reg = ISI_QOS_REG, - .cfg_off = ISI_V_CFG_QOS_OFF, - .default_prio = PRIO(3), - .cfg_prio = PRIO(7), - } - } - }, -}; - -static const struct regmap_range imx93_media_blk_ctl_yes_ranges[] = { - regmap_reg_range(BLK_SFT_RSTN, BLK_CLK_EN), - regmap_reg_range(LCDIF_QOS_REG, ISI_CACHE_REG), - regmap_reg_range(ISI_QOS_REG, ISI_QOS_REG), -}; - -static const struct regmap_access_table imx93_media_blk_ctl_access_table = { - .yes_ranges = imx93_media_blk_ctl_yes_ranges, - .n_yes_ranges = ARRAY_SIZE(imx93_media_blk_ctl_yes_ranges), -}; - -static const struct imx93_blk_ctrl_data imx93_media_blk_ctl_dev_data = { - .domains = imx93_media_blk_ctl_domain_data, - .num_domains = ARRAY_SIZE(imx93_media_blk_ctl_domain_data), - .clk_names = (const char *[]){ "axi", "apb", "nic", }, - .num_clks = 3, - .reg_access_table = &imx93_media_blk_ctl_access_table, -}; - -static const struct of_device_id imx93_blk_ctrl_of_match[] = { - { - .compatible = "fsl,imx93-media-blk-ctrl", - .data = &imx93_media_blk_ctl_dev_data - }, { - /* Sentinel */ - } -}; -MODULE_DEVICE_TABLE(of, imx93_blk_ctrl_of_match); - -static struct platform_driver imx93_blk_ctrl_driver = { - .probe = imx93_blk_ctrl_probe, - .remove = imx93_blk_ctrl_remove, - .driver = { - .name = "imx93-blk-ctrl", - .of_match_table = imx93_blk_ctrl_of_match, - }, -}; -module_platform_driver(imx93_blk_ctrl_driver); - -MODULE_AUTHOR("Peng Fan "); -MODULE_DESCRIPTION("i.MX93 BLK CTRL driver"); -MODULE_LICENSE("GPL"); diff --git a/drivers/genpd/imx/imx93-pd.c b/drivers/genpd/imx/imx93-pd.c deleted file mode 100644 index b9e60d136875..000000000000 --- a/drivers/genpd/imx/imx93-pd.c +++ /dev/null @@ -1,176 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * Copyright 2022 NXP - */ - -#include -#include -#include -#include -#include -#include -#include - -#define MIX_SLICE_SW_CTRL_OFF 0x20 -#define SLICE_SW_CTRL_PSW_CTRL_OFF_MASK BIT(4) -#define SLICE_SW_CTRL_PDN_SOFT_MASK BIT(31) - -#define MIX_FUNC_STAT_OFF 0xB4 - -#define FUNC_STAT_PSW_STAT_MASK BIT(0) -#define FUNC_STAT_RST_STAT_MASK BIT(2) -#define FUNC_STAT_ISO_STAT_MASK BIT(4) - -struct imx93_power_domain { - struct generic_pm_domain genpd; - struct device *dev; - void __iomem *addr; - struct clk_bulk_data *clks; - int num_clks; - bool init_off; -}; - -#define to_imx93_pd(_genpd) container_of(_genpd, struct imx93_power_domain, genpd) - -static int imx93_pd_on(struct generic_pm_domain *genpd) -{ - struct imx93_power_domain *domain = to_imx93_pd(genpd); - void __iomem *addr = domain->addr; - u32 val; - int ret; - - ret = clk_bulk_prepare_enable(domain->num_clks, domain->clks); - if (ret) { - dev_err(domain->dev, "failed to enable clocks for domain: %s\n", genpd->name); - return ret; - } - - val = readl(addr + MIX_SLICE_SW_CTRL_OFF); - val &= ~SLICE_SW_CTRL_PDN_SOFT_MASK; - writel(val, addr + MIX_SLICE_SW_CTRL_OFF); - - ret = readl_poll_timeout(addr + MIX_FUNC_STAT_OFF, val, - !(val & FUNC_STAT_ISO_STAT_MASK), 1, 10000); - if (ret) { - dev_err(domain->dev, "pd_on timeout: name: %s, stat: %x\n", genpd->name, val); - return ret; - } - - return 0; -} - -static int imx93_pd_off(struct generic_pm_domain *genpd) -{ - struct imx93_power_domain *domain = to_imx93_pd(genpd); - void __iomem *addr = domain->addr; - int ret; - u32 val; - - /* Power off MIX */ - val = readl(addr + MIX_SLICE_SW_CTRL_OFF); - val |= SLICE_SW_CTRL_PDN_SOFT_MASK; - writel(val, addr + MIX_SLICE_SW_CTRL_OFF); - - ret = readl_poll_timeout(addr + MIX_FUNC_STAT_OFF, val, - val & FUNC_STAT_PSW_STAT_MASK, 1, 1000); - if (ret) { - dev_err(domain->dev, "pd_off timeout: name: %s, stat: %x\n", genpd->name, val); - return ret; - } - - clk_bulk_disable_unprepare(domain->num_clks, domain->clks); - - return 0; -}; - -static int imx93_pd_remove(struct platform_device *pdev) -{ - struct imx93_power_domain *domain = platform_get_drvdata(pdev); - struct device *dev = &pdev->dev; - struct device_node *np = dev->of_node; - - if (!domain->init_off) - clk_bulk_disable_unprepare(domain->num_clks, domain->clks); - - of_genpd_del_provider(np); - pm_genpd_remove(&domain->genpd); - - return 0; -} - -static int imx93_pd_probe(struct platform_device *pdev) -{ - struct device *dev = &pdev->dev; - struct device_node *np = dev->of_node; - struct imx93_power_domain *domain; - int ret; - - domain = devm_kzalloc(dev, sizeof(*domain), GFP_KERNEL); - if (!domain) - return -ENOMEM; - - domain->addr = devm_platform_ioremap_resource(pdev, 0); - if (IS_ERR(domain->addr)) - return PTR_ERR(domain->addr); - - domain->num_clks = devm_clk_bulk_get_all(dev, &domain->clks); - if (domain->num_clks < 0) - return dev_err_probe(dev, domain->num_clks, "Failed to get domain's clocks\n"); - - domain->genpd.name = dev_name(dev); - domain->genpd.power_off = imx93_pd_off; - domain->genpd.power_on = imx93_pd_on; - domain->dev = dev; - - domain->init_off = readl(domain->addr + MIX_FUNC_STAT_OFF) & FUNC_STAT_ISO_STAT_MASK; - /* Just to sync the status of hardware */ - if (!domain->init_off) { - ret = clk_bulk_prepare_enable(domain->num_clks, domain->clks); - if (ret) { - dev_err(domain->dev, "failed to enable clocks for domain: %s\n", - domain->genpd.name); - return ret; - } - } - - ret = pm_genpd_init(&domain->genpd, NULL, domain->init_off); - if (ret) - goto err_clk_unprepare; - - platform_set_drvdata(pdev, domain); - - ret = of_genpd_add_provider_simple(np, &domain->genpd); - if (ret) - goto err_genpd_remove; - - return 0; - -err_genpd_remove: - pm_genpd_remove(&domain->genpd); - -err_clk_unprepare: - if (!domain->init_off) - clk_bulk_disable_unprepare(domain->num_clks, domain->clks); - - return ret; -} - -static const struct of_device_id imx93_pd_ids[] = { - { .compatible = "fsl,imx93-src-slice" }, - { } -}; -MODULE_DEVICE_TABLE(of, imx93_pd_ids); - -static struct platform_driver imx93_power_domain_driver = { - .driver = { - .name = "imx93_power_domain", - .of_match_table = imx93_pd_ids, - }, - .probe = imx93_pd_probe, - .remove = imx93_pd_remove, -}; -module_platform_driver(imx93_power_domain_driver); - -MODULE_AUTHOR("Peng Fan "); -MODULE_DESCRIPTION("NXP i.MX93 power domain driver"); -MODULE_LICENSE("GPL"); diff --git a/drivers/genpd/imx/scu-pd.c b/drivers/genpd/imx/scu-pd.c deleted file mode 100644 index 2f693b67ddb4..000000000000 --- a/drivers/genpd/imx/scu-pd.c +++ /dev/null @@ -1,550 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0+ -/* - * Copyright (C) 2016 Freescale Semiconductor, Inc. - * Copyright 2017-2018 NXP - * Dong Aisheng - * - * Implementation of the SCU based Power Domains - * - * NOTE: a better implementation suggested by Ulf Hansson is using a - * single global power domain and implement the ->attach|detach_dev() - * callback for the genpd and use the regular of_genpd_add_provider_simple(). - * From within the ->attach_dev(), we could get the OF node for - * the device that is being attached and then parse the power-domain - * cell containing the "resource id" and store that in the per device - * struct generic_pm_domain_data (we have void pointer there for - * storing these kind of things). - * - * Additionally, we need to implement the ->stop() and ->start() - * callbacks of genpd, which is where you "power on/off" devices, - * rather than using the above ->power_on|off() callbacks. - * - * However, there're two known issues: - * 1. The ->attach_dev() of power domain infrastructure still does - * not support multi domains case as the struct device *dev passed - * in is a virtual PD device, it does not help for parsing the real - * device resource id from device tree, so it's unware of which - * real sub power domain of device should be attached. - * - * The framework needs some proper extension to support multi power - * domain cases. - * - * Update: Genpd assigns the ->of_node for the virtual device before it - * invokes ->attach_dev() callback, hence parsing for device resources via - * DT should work fine. - * - * 2. It also breaks most of current drivers as the driver probe sequence - * behavior changed if removing ->power_on|off() callback and use - * ->start() and ->stop() instead. genpd_dev_pm_attach will only power - * up the domain and attach device, but will not call .start() which - * relies on device runtime pm. That means the device power is still - * not up before running driver probe function. For SCU enabled - * platforms, all device drivers accessing registers/clock without power - * domain enabled will trigger a HW access error. That means we need fix - * most drivers probe sequence with proper runtime pm. - * - * Update: Runtime PM support isn't necessary. Instead, this can easily be - * fixed in drivers by adding a call to dev_pm_domain_start() during probe. - * - * In summary, the second part needs to be addressed via minor updates to the - * relevant drivers, before the "single global power domain" model can be used. - * - */ - -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -/* SCU Power Mode Protocol definition */ -struct imx_sc_msg_req_set_resource_power_mode { - struct imx_sc_rpc_msg hdr; - u16 resource; - u8 mode; -} __packed __aligned(4); - -struct req_get_resource_mode { - u16 resource; -}; - -struct resp_get_resource_mode { - u8 mode; -}; - -struct imx_sc_msg_req_get_resource_power_mode { - struct imx_sc_rpc_msg hdr; - union { - struct req_get_resource_mode req; - struct resp_get_resource_mode resp; - } data; -} __packed __aligned(4); - -#define IMX_SCU_PD_NAME_SIZE 20 -struct imx_sc_pm_domain { - struct generic_pm_domain pd; - char name[IMX_SCU_PD_NAME_SIZE]; - u32 rsrc; -}; - -struct imx_sc_pd_range { - char *name; - u32 rsrc; - u8 num; - - /* add domain index */ - bool postfix; - u8 start_from; -}; - -struct imx_sc_pd_soc { - const struct imx_sc_pd_range *pd_ranges; - u8 num_ranges; -}; - -static int imx_con_rsrc; - -/* Align with the IMX_SC_PM_PW_MODE_[OFF,STBY,LP,ON] macros */ -static const char * const imx_sc_pm_mode[] = { - "IMX_SC_PM_PW_MODE_OFF", - "IMX_SC_PM_PW_MODE_STBY", - "IMX_SC_PM_PW_MODE_LP", - "IMX_SC_PM_PW_MODE_ON" -}; - -static const struct imx_sc_pd_range imx8qxp_scu_pd_ranges[] = { - /* LSIO SS */ - { "pwm", IMX_SC_R_PWM_0, 8, true, 0 }, - { "gpio", IMX_SC_R_GPIO_0, 8, true, 0 }, - { "gpt", IMX_SC_R_GPT_0, 5, true, 0 }, - { "kpp", IMX_SC_R_KPP, 1, false, 0 }, - { "fspi", IMX_SC_R_FSPI_0, 2, true, 0 }, - { "mu_a", IMX_SC_R_MU_0A, 14, true, 0 }, - { "mu_b", IMX_SC_R_MU_5B, 9, true, 5 }, - - /* CONN SS */ - { "usb", IMX_SC_R_USB_0, 2, true, 0 }, - { "usb0phy", IMX_SC_R_USB_0_PHY, 1, false, 0 }, - { "usb1phy", IMX_SC_R_USB_1_PHY, 1, false, 0}, - { "usb2", IMX_SC_R_USB_2, 1, false, 0 }, - { "usb2phy", IMX_SC_R_USB_2_PHY, 1, false, 0 }, - { "sdhc", IMX_SC_R_SDHC_0, 3, true, 0 }, - { "enet", IMX_SC_R_ENET_0, 2, true, 0 }, - { "nand", IMX_SC_R_NAND, 1, false, 0 }, - { "mlb", IMX_SC_R_MLB_0, 1, true, 0 }, - - /* AUDIO SS */ - { "audio-pll0", IMX_SC_R_AUDIO_PLL_0, 1, false, 0 }, - { "audio-pll1", IMX_SC_R_AUDIO_PLL_1, 1, false, 0 }, - { "audio-clk-0", IMX_SC_R_AUDIO_CLK_0, 1, false, 0 }, - { "audio-clk-1", IMX_SC_R_AUDIO_CLK_1, 1, false, 0 }, - { "mclk-out-0", IMX_SC_R_MCLK_OUT_0, 1, false, 0 }, - { "mclk-out-1", IMX_SC_R_MCLK_OUT_1, 1, false, 0 }, - { "dma0-ch", IMX_SC_R_DMA_0_CH0, 32, true, 0 }, - { "dma1-ch", IMX_SC_R_DMA_1_CH0, 16, true, 0 }, - { "dma2-ch", IMX_SC_R_DMA_2_CH0, 32, true, 0 }, - { "dma3-ch", IMX_SC_R_DMA_3_CH0, 32, true, 0 }, - { "asrc0", IMX_SC_R_ASRC_0, 1, false, 0 }, - { "asrc1", IMX_SC_R_ASRC_1, 1, false, 0 }, - { "esai0", IMX_SC_R_ESAI_0, 1, false, 0 }, - { "esai1", IMX_SC_R_ESAI_1, 1, false, 0 }, - { "spdif0", IMX_SC_R_SPDIF_0, 1, false, 0 }, - { "spdif1", IMX_SC_R_SPDIF_1, 1, false, 0 }, - { "sai", IMX_SC_R_SAI_0, 3, true, 0 }, - { "sai3", IMX_SC_R_SAI_3, 1, false, 0 }, - { "sai4", IMX_SC_R_SAI_4, 1, false, 0 }, - { "sai5", IMX_SC_R_SAI_5, 1, false, 0 }, - { "sai6", IMX_SC_R_SAI_6, 1, false, 0 }, - { "sai7", IMX_SC_R_SAI_7, 1, false, 0 }, - { "amix", IMX_SC_R_AMIX, 1, false, 0 }, - { "mqs0", IMX_SC_R_MQS_0, 1, false, 0 }, - { "dsp", IMX_SC_R_DSP, 1, false, 0 }, - { "dsp-ram", IMX_SC_R_DSP_RAM, 1, false, 0 }, - - /* DMA SS */ - { "can", IMX_SC_R_CAN_0, 3, true, 0 }, - { "ftm", IMX_SC_R_FTM_0, 2, true, 0 }, - { "lpi2c", IMX_SC_R_I2C_0, 5, true, 0 }, - { "adc", IMX_SC_R_ADC_0, 2, true, 0 }, - { "lcd", IMX_SC_R_LCD_0, 1, true, 0 }, - { "lcd-pll", IMX_SC_R_ELCDIF_PLL, 1, true, 0 }, - { "lcd0-pwm", IMX_SC_R_LCD_0_PWM_0, 1, true, 0 }, - { "lpuart", IMX_SC_R_UART_0, 5, true, 0 }, - { "sim", IMX_SC_R_EMVSIM_0, 2, true, 0 }, - { "lpspi", IMX_SC_R_SPI_0, 4, true, 0 }, - { "irqstr_dsp", IMX_SC_R_IRQSTR_DSP, 1, false, 0 }, - - /* VPU SS */ - { "vpu", IMX_SC_R_VPU, 1, false, 0 }, - { "vpu-pid", IMX_SC_R_VPU_PID0, 8, true, 0 }, - { "vpu-dec0", IMX_SC_R_VPU_DEC_0, 1, false, 0 }, - { "vpu-enc0", IMX_SC_R_VPU_ENC_0, 1, false, 0 }, - { "vpu-enc1", IMX_SC_R_VPU_ENC_1, 1, false, 0 }, - { "vpu-mu0", IMX_SC_R_VPU_MU_0, 1, false, 0 }, - { "vpu-mu1", IMX_SC_R_VPU_MU_1, 1, false, 0 }, - { "vpu-mu2", IMX_SC_R_VPU_MU_2, 1, false, 0 }, - - /* GPU SS */ - { "gpu0-pid", IMX_SC_R_GPU_0_PID0, 4, true, 0 }, - { "gpu1-pid", IMX_SC_R_GPU_1_PID0, 4, true, 0 }, - - - /* HSIO SS */ - { "pcie-a", IMX_SC_R_PCIE_A, 1, false, 0 }, - { "serdes-0", IMX_SC_R_SERDES_0, 1, false, 0 }, - { "pcie-b", IMX_SC_R_PCIE_B, 1, false, 0 }, - { "serdes-1", IMX_SC_R_SERDES_1, 1, false, 0 }, - { "sata-0", IMX_SC_R_SATA_0, 1, false, 0 }, - { "hsio-gpio", IMX_SC_R_HSIO_GPIO, 1, false, 0 }, - - /* MIPI SS */ - { "mipi0", IMX_SC_R_MIPI_0, 1, false, 0 }, - { "mipi0-pwm0", IMX_SC_R_MIPI_0_PWM_0, 1, false, 0 }, - { "mipi0-i2c", IMX_SC_R_MIPI_0_I2C_0, 2, true, 0 }, - - { "mipi1", IMX_SC_R_MIPI_1, 1, false, 0 }, - { "mipi1-pwm0", IMX_SC_R_MIPI_1_PWM_0, 1, false, 0 }, - { "mipi1-i2c", IMX_SC_R_MIPI_1_I2C_0, 2, true, 0 }, - - /* LVDS SS */ - { "lvds0", IMX_SC_R_LVDS_0, 1, false, 0 }, - { "lvds0-pwm", IMX_SC_R_LVDS_0_PWM_0, 1, false, 0 }, - { "lvds0-lpi2c", IMX_SC_R_LVDS_0_I2C_0, 2, true, 0 }, - { "lvds1", IMX_SC_R_LVDS_1, 1, false, 0 }, - { "lvds1-pwm", IMX_SC_R_LVDS_1_PWM_0, 1, false, 0 }, - { "lvds1-lpi2c", IMX_SC_R_LVDS_1_I2C_0, 2, true, 0 }, - - { "mipi1", IMX_SC_R_MIPI_1, 1, 0 }, - { "mipi1-pwm0", IMX_SC_R_MIPI_1_PWM_0, 1, 0 }, - { "mipi1-i2c", IMX_SC_R_MIPI_1_I2C_0, 2, 1 }, - { "lvds1", IMX_SC_R_LVDS_1, 1, 0 }, - - /* DC SS */ - { "dc0", IMX_SC_R_DC_0, 1, false, 0 }, - { "dc0-pll", IMX_SC_R_DC_0_PLL_0, 2, true, 0 }, - { "dc0-video", IMX_SC_R_DC_0_VIDEO0, 2, true, 0 }, - - { "dc1", IMX_SC_R_DC_1, 1, false, 0 }, - { "dc1-pll", IMX_SC_R_DC_1_PLL_0, 2, true, 0 }, - { "dc1-video", IMX_SC_R_DC_1_VIDEO0, 2, true, 0 }, - - /* CM40 SS */ - { "cm40-i2c", IMX_SC_R_M4_0_I2C, 1, false, 0 }, - { "cm40-intmux", IMX_SC_R_M4_0_INTMUX, 1, false, 0 }, - { "cm40-pid", IMX_SC_R_M4_0_PID0, 5, true, 0}, - { "cm40-mu-a1", IMX_SC_R_M4_0_MU_1A, 1, false, 0}, - { "cm40-lpuart", IMX_SC_R_M4_0_UART, 1, false, 0}, - - /* CM41 SS */ - { "cm41-i2c", IMX_SC_R_M4_1_I2C, 1, false, 0 }, - { "cm41-intmux", IMX_SC_R_M4_1_INTMUX, 1, false, 0 }, - { "cm41-pid", IMX_SC_R_M4_1_PID0, 5, true, 0}, - { "cm41-mu-a1", IMX_SC_R_M4_1_MU_1A, 1, false, 0}, - { "cm41-lpuart", IMX_SC_R_M4_1_UART, 1, false, 0}, - - /* CM41 SS */ - { "cm41_i2c", IMX_SC_R_M4_1_I2C, 1, false, 0 }, - { "cm41_intmux", IMX_SC_R_M4_1_INTMUX, 1, false, 0 }, - - /* DB SS */ - { "perf", IMX_SC_R_PERF, 1, false, 0}, - - /* IMAGE SS */ - { "img-jpegdec-mp", IMX_SC_R_MJPEG_DEC_MP, 1, false, 0 }, - { "img-jpegdec-s0", IMX_SC_R_MJPEG_DEC_S0, 4, true, 0 }, - { "img-jpegenc-mp", IMX_SC_R_MJPEG_ENC_MP, 1, false, 0 }, - { "img-jpegenc-s0", IMX_SC_R_MJPEG_ENC_S0, 4, true, 0 }, - - /* SECO SS */ - { "seco_mu", IMX_SC_R_SECO_MU_2, 3, true, 2}, - - /* V2X SS */ - { "v2x_mu", IMX_SC_R_V2X_MU_0, 2, true, 0}, - { "v2x_mu", IMX_SC_R_V2X_MU_2, 1, true, 2}, - { "v2x_mu", IMX_SC_R_V2X_MU_3, 2, true, 3}, - { "img-pdma", IMX_SC_R_ISI_CH0, 8, true, 0 }, - { "img-csi0", IMX_SC_R_CSI_0, 1, false, 0 }, - { "img-csi0-i2c0", IMX_SC_R_CSI_0_I2C_0, 1, false, 0 }, - { "img-csi0-pwm0", IMX_SC_R_CSI_0_PWM_0, 1, false, 0 }, - { "img-csi1", IMX_SC_R_CSI_1, 1, false, 0 }, - { "img-csi1-i2c0", IMX_SC_R_CSI_1_I2C_0, 1, false, 0 }, - { "img-csi1-pwm0", IMX_SC_R_CSI_1_PWM_0, 1, false, 0 }, - { "img-parallel", IMX_SC_R_PI_0, 1, false, 0 }, - { "img-parallel-i2c0", IMX_SC_R_PI_0_I2C_0, 1, false, 0 }, - { "img-parallel-pwm0", IMX_SC_R_PI_0_PWM_0, 2, true, 0 }, - { "img-parallel-pll", IMX_SC_R_PI_0_PLL, 1, false, 0 }, - - /* HDMI TX SS */ - { "hdmi-tx", IMX_SC_R_HDMI, 1, false, 0}, - { "hdmi-tx-i2s", IMX_SC_R_HDMI_I2S, 1, false, 0}, - { "hdmi-tx-i2c0", IMX_SC_R_HDMI_I2C_0, 1, false, 0}, - { "hdmi-tx-pll0", IMX_SC_R_HDMI_PLL_0, 1, false, 0}, - { "hdmi-tx-pll1", IMX_SC_R_HDMI_PLL_1, 1, false, 0}, - - /* HDMI RX SS */ - { "hdmi-rx", IMX_SC_R_HDMI_RX, 1, false, 0}, - { "hdmi-rx-pwm", IMX_SC_R_HDMI_RX_PWM_0, 1, false, 0}, - { "hdmi-rx-i2c0", IMX_SC_R_HDMI_RX_I2C_0, 1, false, 0}, - { "hdmi-rx-bypass", IMX_SC_R_HDMI_RX_BYPASS, 1, false, 0}, - - /* SECURITY SS */ - { "sec-jr", IMX_SC_R_CAAM_JR2, 2, true, 2}, - - /* BOARD SS */ - { "board", IMX_SC_R_BOARD_R0, 8, true, 0}, -}; - -static const struct imx_sc_pd_soc imx8qxp_scu_pd = { - .pd_ranges = imx8qxp_scu_pd_ranges, - .num_ranges = ARRAY_SIZE(imx8qxp_scu_pd_ranges), -}; - -static struct imx_sc_ipc *pm_ipc_handle; - -static inline struct imx_sc_pm_domain * -to_imx_sc_pd(struct generic_pm_domain *genpd) -{ - return container_of(genpd, struct imx_sc_pm_domain, pd); -} - -static void imx_sc_pd_get_console_rsrc(void) -{ - struct of_phandle_args specs; - int ret; - - if (!of_stdout) - return; - - ret = of_parse_phandle_with_args(of_stdout, "power-domains", - "#power-domain-cells", - 0, &specs); - if (ret) - return; - - imx_con_rsrc = specs.args[0]; -} - -static int imx_sc_get_pd_power(struct device *dev, u32 rsrc) -{ - struct imx_sc_msg_req_get_resource_power_mode msg; - struct imx_sc_rpc_msg *hdr = &msg.hdr; - int ret; - - hdr->ver = IMX_SC_RPC_VERSION; - hdr->svc = IMX_SC_RPC_SVC_PM; - hdr->func = IMX_SC_PM_FUNC_GET_RESOURCE_POWER_MODE; - hdr->size = 2; - - msg.data.req.resource = rsrc; - - ret = imx_scu_call_rpc(pm_ipc_handle, &msg, true); - if (ret) - dev_err(dev, "failed to get power resource %d mode, ret %d\n", - rsrc, ret); - - return msg.data.resp.mode; -} - -static int imx_sc_pd_power(struct generic_pm_domain *domain, bool power_on) -{ - struct imx_sc_msg_req_set_resource_power_mode msg; - struct imx_sc_rpc_msg *hdr = &msg.hdr; - struct imx_sc_pm_domain *pd; - int ret; - - pd = to_imx_sc_pd(domain); - - hdr->ver = IMX_SC_RPC_VERSION; - hdr->svc = IMX_SC_RPC_SVC_PM; - hdr->func = IMX_SC_PM_FUNC_SET_RESOURCE_POWER_MODE; - hdr->size = 2; - - msg.resource = pd->rsrc; - msg.mode = power_on ? IMX_SC_PM_PW_MODE_ON : IMX_SC_PM_PW_MODE_LP; - - /* keep uart console power on for no_console_suspend */ - if (imx_con_rsrc == pd->rsrc && !console_suspend_enabled && !power_on) - return -EBUSY; - - ret = imx_scu_call_rpc(pm_ipc_handle, &msg, true); - if (ret) - dev_err(&domain->dev, "failed to power %s resource %d ret %d\n", - power_on ? "up" : "off", pd->rsrc, ret); - - return ret; -} - -static int imx_sc_pd_power_on(struct generic_pm_domain *domain) -{ - return imx_sc_pd_power(domain, true); -} - -static int imx_sc_pd_power_off(struct generic_pm_domain *domain) -{ - return imx_sc_pd_power(domain, false); -} - -static struct generic_pm_domain *imx_scu_pd_xlate(struct of_phandle_args *spec, - void *data) -{ - struct generic_pm_domain *domain = ERR_PTR(-ENOENT); - struct genpd_onecell_data *pd_data = data; - unsigned int i; - - for (i = 0; i < pd_data->num_domains; i++) { - struct imx_sc_pm_domain *sc_pd; - - sc_pd = to_imx_sc_pd(pd_data->domains[i]); - if (sc_pd->rsrc == spec->args[0]) { - domain = &sc_pd->pd; - break; - } - } - - return domain; -} - -static struct imx_sc_pm_domain * -imx_scu_add_pm_domain(struct device *dev, int idx, - const struct imx_sc_pd_range *pd_ranges) -{ - struct imx_sc_pm_domain *sc_pd; - bool is_off; - int mode, ret; - - if (!imx_sc_rm_is_resource_owned(pm_ipc_handle, pd_ranges->rsrc + idx)) - return NULL; - - sc_pd = devm_kzalloc(dev, sizeof(*sc_pd), GFP_KERNEL); - if (!sc_pd) - return ERR_PTR(-ENOMEM); - - sc_pd->rsrc = pd_ranges->rsrc + idx; - sc_pd->pd.power_off = imx_sc_pd_power_off; - sc_pd->pd.power_on = imx_sc_pd_power_on; - - if (pd_ranges->postfix) - snprintf(sc_pd->name, sizeof(sc_pd->name), - "%s%i", pd_ranges->name, pd_ranges->start_from + idx); - else - snprintf(sc_pd->name, sizeof(sc_pd->name), - "%s", pd_ranges->name); - - sc_pd->pd.name = sc_pd->name; - if (imx_con_rsrc == sc_pd->rsrc) - sc_pd->pd.flags = GENPD_FLAG_RPM_ALWAYS_ON; - - mode = imx_sc_get_pd_power(dev, pd_ranges->rsrc + idx); - if (mode == IMX_SC_PM_PW_MODE_ON) - is_off = false; - else - is_off = true; - - dev_dbg(dev, "%s : %s\n", sc_pd->name, imx_sc_pm_mode[mode]); - - if (sc_pd->rsrc >= IMX_SC_R_LAST) { - dev_warn(dev, "invalid pd %s rsrc id %d found", - sc_pd->name, sc_pd->rsrc); - - devm_kfree(dev, sc_pd); - return NULL; - } - - ret = pm_genpd_init(&sc_pd->pd, NULL, is_off); - if (ret) { - dev_warn(dev, "failed to init pd %s rsrc id %d", - sc_pd->name, sc_pd->rsrc); - devm_kfree(dev, sc_pd); - return NULL; - } - - return sc_pd; -} - -static int imx_scu_init_pm_domains(struct device *dev, - const struct imx_sc_pd_soc *pd_soc) -{ - const struct imx_sc_pd_range *pd_ranges = pd_soc->pd_ranges; - struct generic_pm_domain **domains; - struct genpd_onecell_data *pd_data; - struct imx_sc_pm_domain *sc_pd; - u32 count = 0; - int i, j; - - for (i = 0; i < pd_soc->num_ranges; i++) - count += pd_ranges[i].num; - - domains = devm_kcalloc(dev, count, sizeof(*domains), GFP_KERNEL); - if (!domains) - return -ENOMEM; - - pd_data = devm_kzalloc(dev, sizeof(*pd_data), GFP_KERNEL); - if (!pd_data) - return -ENOMEM; - - count = 0; - for (i = 0; i < pd_soc->num_ranges; i++) { - for (j = 0; j < pd_ranges[i].num; j++) { - sc_pd = imx_scu_add_pm_domain(dev, j, &pd_ranges[i]); - if (IS_ERR_OR_NULL(sc_pd)) - continue; - - domains[count++] = &sc_pd->pd; - dev_dbg(dev, "added power domain %s\n", sc_pd->pd.name); - } - } - - pd_data->domains = domains; - pd_data->num_domains = count; - pd_data->xlate = imx_scu_pd_xlate; - - of_genpd_add_provider_onecell(dev->of_node, pd_data); - - return 0; -} - -static int imx_sc_pd_probe(struct platform_device *pdev) -{ - const struct imx_sc_pd_soc *pd_soc; - int ret; - - ret = imx_scu_get_handle(&pm_ipc_handle); - if (ret) - return ret; - - pd_soc = of_device_get_match_data(&pdev->dev); - if (!pd_soc) - return -ENODEV; - - imx_sc_pd_get_console_rsrc(); - - return imx_scu_init_pm_domains(&pdev->dev, pd_soc); -} - -static const struct of_device_id imx_sc_pd_match[] = { - { .compatible = "fsl,imx8qxp-scu-pd", &imx8qxp_scu_pd}, - { .compatible = "fsl,scu-pd", &imx8qxp_scu_pd}, - { /* sentinel */ } -}; - -static struct platform_driver imx_sc_pd_driver = { - .driver = { - .name = "imx-scu-pd", - .of_match_table = imx_sc_pd_match, - .suppress_bind_attrs = true, - }, - .probe = imx_sc_pd_probe, -}; -builtin_platform_driver(imx_sc_pd_driver); - -MODULE_AUTHOR("Dong Aisheng "); -MODULE_DESCRIPTION("IMX SCU Power Domain driver"); -MODULE_LICENSE("GPL v2"); diff --git a/drivers/genpd/mediatek/Makefile b/drivers/genpd/mediatek/Makefile deleted file mode 100644 index 8cde09e654b3..000000000000 --- a/drivers/genpd/mediatek/Makefile +++ /dev/null @@ -1,3 +0,0 @@ -# SPDX-License-Identifier: GPL-2.0-only -obj-$(CONFIG_MTK_SCPSYS) += mtk-scpsys.o -obj-$(CONFIG_MTK_SCPSYS_PM_DOMAINS) += mtk-pm-domains.o diff --git a/drivers/genpd/mediatek/mt6795-pm-domains.h b/drivers/genpd/mediatek/mt6795-pm-domains.h deleted file mode 100644 index ef07c9dfdd9b..000000000000 --- a/drivers/genpd/mediatek/mt6795-pm-domains.h +++ /dev/null @@ -1,112 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0-only */ - -#ifndef __SOC_MEDIATEK_MT6795_PM_DOMAINS_H -#define __SOC_MEDIATEK_MT6795_PM_DOMAINS_H - -#include "mtk-pm-domains.h" -#include - -/* - * MT6795 power domain support - */ - -static const struct scpsys_domain_data scpsys_domain_data_mt6795[] = { - [MT6795_POWER_DOMAIN_VDEC] = { - .name = "vdec", - .sta_mask = PWR_STATUS_VDEC, - .ctl_offs = SPM_VDE_PWR_CON, - .pwr_sta_offs = SPM_PWR_STATUS, - .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - }, - [MT6795_POWER_DOMAIN_VENC] = { - .name = "venc", - .sta_mask = PWR_STATUS_VENC, - .ctl_offs = SPM_VEN_PWR_CON, - .pwr_sta_offs = SPM_PWR_STATUS, - .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = GENMASK(15, 12), - }, - [MT6795_POWER_DOMAIN_ISP] = { - .name = "isp", - .sta_mask = PWR_STATUS_ISP, - .ctl_offs = SPM_ISP_PWR_CON, - .pwr_sta_offs = SPM_PWR_STATUS, - .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = GENMASK(13, 12), - }, - [MT6795_POWER_DOMAIN_MM] = { - .name = "mm", - .sta_mask = PWR_STATUS_DISP, - .ctl_offs = SPM_DIS_PWR_CON, - .pwr_sta_offs = SPM_PWR_STATUS, - .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .bp_infracfg = { - BUS_PROT_UPDATE_TOPAXI(MT8173_TOP_AXI_PROT_EN_MM_M0 | - MT8173_TOP_AXI_PROT_EN_MM_M1), - }, - }, - [MT6795_POWER_DOMAIN_MJC] = { - .name = "mjc", - .sta_mask = BIT(20), - .ctl_offs = 0x298, - .pwr_sta_offs = SPM_PWR_STATUS, - .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = GENMASK(15, 12), - }, - [MT6795_POWER_DOMAIN_AUDIO] = { - .name = "audio", - .sta_mask = PWR_STATUS_AUDIO, - .ctl_offs = SPM_AUDIO_PWR_CON, - .pwr_sta_offs = SPM_PWR_STATUS, - .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = GENMASK(15, 12), - }, - [MT6795_POWER_DOMAIN_MFG_ASYNC] = { - .name = "mfg_async", - .sta_mask = PWR_STATUS_MFG_ASYNC, - .ctl_offs = SPM_MFG_ASYNC_PWR_CON, - .pwr_sta_offs = SPM_PWR_STATUS, - .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = 0, - }, - [MT6795_POWER_DOMAIN_MFG_2D] = { - .name = "mfg_2d", - .sta_mask = PWR_STATUS_MFG_2D, - .ctl_offs = SPM_MFG_2D_PWR_CON, - .pwr_sta_offs = SPM_PWR_STATUS, - .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = GENMASK(13, 12), - }, - [MT6795_POWER_DOMAIN_MFG] = { - .name = "mfg", - .sta_mask = PWR_STATUS_MFG, - .ctl_offs = SPM_MFG_PWR_CON, - .pwr_sta_offs = SPM_PWR_STATUS, - .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, - .sram_pdn_bits = GENMASK(13, 8), - .sram_pdn_ack_bits = GENMASK(21, 16), - .bp_infracfg = { - BUS_PROT_UPDATE_TOPAXI(MT8173_TOP_AXI_PROT_EN_MFG_S | - MT8173_TOP_AXI_PROT_EN_MFG_M0 | - MT8173_TOP_AXI_PROT_EN_MFG_M1 | - MT8173_TOP_AXI_PROT_EN_MFG_SNOOP_OUT), - }, - }, -}; - -static const struct scpsys_soc_data mt6795_scpsys_data = { - .domains_data = scpsys_domain_data_mt6795, - .num_domains = ARRAY_SIZE(scpsys_domain_data_mt6795), -}; - -#endif /* __SOC_MEDIATEK_MT6795_PM_DOMAINS_H */ diff --git a/drivers/genpd/mediatek/mt8167-pm-domains.h b/drivers/genpd/mediatek/mt8167-pm-domains.h deleted file mode 100644 index 4d6c32759606..000000000000 --- a/drivers/genpd/mediatek/mt8167-pm-domains.h +++ /dev/null @@ -1,105 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0-only */ - -#ifndef __SOC_MEDIATEK_MT8167_PM_DOMAINS_H -#define __SOC_MEDIATEK_MT8167_PM_DOMAINS_H - -#include "mtk-pm-domains.h" -#include - -#define MT8167_PWR_STATUS_MFG_2D BIT(24) -#define MT8167_PWR_STATUS_MFG_ASYNC BIT(25) - -/* - * MT8167 power domain support - */ - -static const struct scpsys_domain_data scpsys_domain_data_mt8167[] = { - [MT8167_POWER_DOMAIN_MM] = { - .name = "mm", - .sta_mask = PWR_STATUS_DISP, - .ctl_offs = SPM_DIS_PWR_CON, - .pwr_sta_offs = SPM_PWR_STATUS, - .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .bp_infracfg = { - BUS_PROT_UPDATE_TOPAXI(MT8167_TOP_AXI_PROT_EN_MM_EMI | - MT8167_TOP_AXI_PROT_EN_MCU_MM), - }, - .caps = MTK_SCPD_ACTIVE_WAKEUP, - }, - [MT8167_POWER_DOMAIN_VDEC] = { - .name = "vdec", - .sta_mask = PWR_STATUS_VDEC, - .ctl_offs = SPM_VDE_PWR_CON, - .pwr_sta_offs = SPM_PWR_STATUS, - .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .caps = MTK_SCPD_ACTIVE_WAKEUP, - }, - [MT8167_POWER_DOMAIN_ISP] = { - .name = "isp", - .sta_mask = PWR_STATUS_ISP, - .ctl_offs = SPM_ISP_PWR_CON, - .pwr_sta_offs = SPM_PWR_STATUS, - .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = GENMASK(13, 12), - .caps = MTK_SCPD_ACTIVE_WAKEUP, - }, - [MT8167_POWER_DOMAIN_MFG_ASYNC] = { - .name = "mfg_async", - .sta_mask = MT8167_PWR_STATUS_MFG_ASYNC, - .ctl_offs = SPM_MFG_ASYNC_PWR_CON, - .pwr_sta_offs = SPM_PWR_STATUS, - .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, - .sram_pdn_bits = 0, - .sram_pdn_ack_bits = 0, - .bp_infracfg = { - BUS_PROT_UPDATE_TOPAXI(MT8167_TOP_AXI_PROT_EN_MCU_MFG | - MT8167_TOP_AXI_PROT_EN_MFG_EMI), - }, - }, - [MT8167_POWER_DOMAIN_MFG_2D] = { - .name = "mfg_2d", - .sta_mask = MT8167_PWR_STATUS_MFG_2D, - .ctl_offs = SPM_MFG_2D_PWR_CON, - .pwr_sta_offs = SPM_PWR_STATUS, - .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = GENMASK(15, 12), - }, - [MT8167_POWER_DOMAIN_MFG] = { - .name = "mfg", - .sta_mask = PWR_STATUS_MFG, - .ctl_offs = SPM_MFG_PWR_CON, - .pwr_sta_offs = SPM_PWR_STATUS, - .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = GENMASK(15, 12), - }, - [MT8167_POWER_DOMAIN_CONN] = { - .name = "conn", - .sta_mask = PWR_STATUS_CONN, - .ctl_offs = SPM_CONN_PWR_CON, - .pwr_sta_offs = SPM_PWR_STATUS, - .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = 0, - .caps = MTK_SCPD_ACTIVE_WAKEUP, - .bp_infracfg = { - BUS_PROT_UPDATE_TOPAXI(MT8167_TOP_AXI_PROT_EN_CONN_EMI | - MT8167_TOP_AXI_PROT_EN_CONN_MCU | - MT8167_TOP_AXI_PROT_EN_MCU_CONN), - }, - }, -}; - -static const struct scpsys_soc_data mt8167_scpsys_data = { - .domains_data = scpsys_domain_data_mt8167, - .num_domains = ARRAY_SIZE(scpsys_domain_data_mt8167), -}; - -#endif /* __SOC_MEDIATEK_MT8167_PM_DOMAINS_H */ - diff --git a/drivers/genpd/mediatek/mt8173-pm-domains.h b/drivers/genpd/mediatek/mt8173-pm-domains.h deleted file mode 100644 index 1a5dc63b7357..000000000000 --- a/drivers/genpd/mediatek/mt8173-pm-domains.h +++ /dev/null @@ -1,123 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0-only */ - -#ifndef __SOC_MEDIATEK_MT8173_PM_DOMAINS_H -#define __SOC_MEDIATEK_MT8173_PM_DOMAINS_H - -#include "mtk-pm-domains.h" -#include - -/* - * MT8173 power domain support - */ - -static const struct scpsys_domain_data scpsys_domain_data_mt8173[] = { - [MT8173_POWER_DOMAIN_VDEC] = { - .name = "vdec", - .sta_mask = PWR_STATUS_VDEC, - .ctl_offs = SPM_VDE_PWR_CON, - .pwr_sta_offs = SPM_PWR_STATUS, - .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - }, - [MT8173_POWER_DOMAIN_VENC] = { - .name = "venc", - .sta_mask = PWR_STATUS_VENC, - .ctl_offs = SPM_VEN_PWR_CON, - .pwr_sta_offs = SPM_PWR_STATUS, - .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = GENMASK(15, 12), - }, - [MT8173_POWER_DOMAIN_ISP] = { - .name = "isp", - .sta_mask = PWR_STATUS_ISP, - .ctl_offs = SPM_ISP_PWR_CON, - .pwr_sta_offs = SPM_PWR_STATUS, - .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = GENMASK(13, 12), - }, - [MT8173_POWER_DOMAIN_MM] = { - .name = "mm", - .sta_mask = PWR_STATUS_DISP, - .ctl_offs = SPM_DIS_PWR_CON, - .pwr_sta_offs = SPM_PWR_STATUS, - .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .bp_infracfg = { - BUS_PROT_UPDATE_TOPAXI(MT8173_TOP_AXI_PROT_EN_MM_M0 | - MT8173_TOP_AXI_PROT_EN_MM_M1), - }, - }, - [MT8173_POWER_DOMAIN_VENC_LT] = { - .name = "venc_lt", - .sta_mask = PWR_STATUS_VENC_LT, - .ctl_offs = SPM_VEN2_PWR_CON, - .pwr_sta_offs = SPM_PWR_STATUS, - .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = GENMASK(15, 12), - }, - [MT8173_POWER_DOMAIN_AUDIO] = { - .name = "audio", - .sta_mask = PWR_STATUS_AUDIO, - .ctl_offs = SPM_AUDIO_PWR_CON, - .pwr_sta_offs = SPM_PWR_STATUS, - .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = GENMASK(15, 12), - }, - [MT8173_POWER_DOMAIN_USB] = { - .name = "usb", - .sta_mask = PWR_STATUS_USB, - .ctl_offs = SPM_USB_PWR_CON, - .pwr_sta_offs = SPM_PWR_STATUS, - .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = GENMASK(15, 12), - .caps = MTK_SCPD_ACTIVE_WAKEUP, - }, - [MT8173_POWER_DOMAIN_MFG_ASYNC] = { - .name = "mfg_async", - .sta_mask = PWR_STATUS_MFG_ASYNC, - .ctl_offs = SPM_MFG_ASYNC_PWR_CON, - .pwr_sta_offs = SPM_PWR_STATUS, - .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = 0, - .caps = MTK_SCPD_DOMAIN_SUPPLY, - }, - [MT8173_POWER_DOMAIN_MFG_2D] = { - .name = "mfg_2d", - .sta_mask = PWR_STATUS_MFG_2D, - .ctl_offs = SPM_MFG_2D_PWR_CON, - .pwr_sta_offs = SPM_PWR_STATUS, - .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = GENMASK(13, 12), - }, - [MT8173_POWER_DOMAIN_MFG] = { - .name = "mfg", - .sta_mask = PWR_STATUS_MFG, - .ctl_offs = SPM_MFG_PWR_CON, - .pwr_sta_offs = SPM_PWR_STATUS, - .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, - .sram_pdn_bits = GENMASK(13, 8), - .sram_pdn_ack_bits = GENMASK(21, 16), - .bp_infracfg = { - BUS_PROT_UPDATE_TOPAXI(MT8173_TOP_AXI_PROT_EN_MFG_S | - MT8173_TOP_AXI_PROT_EN_MFG_M0 | - MT8173_TOP_AXI_PROT_EN_MFG_M1 | - MT8173_TOP_AXI_PROT_EN_MFG_SNOOP_OUT), - }, - }, -}; - -static const struct scpsys_soc_data mt8173_scpsys_data = { - .domains_data = scpsys_domain_data_mt8173, - .num_domains = ARRAY_SIZE(scpsys_domain_data_mt8173), -}; - -#endif /* __SOC_MEDIATEK_MT8173_PM_DOMAINS_H */ diff --git a/drivers/genpd/mediatek/mt8183-pm-domains.h b/drivers/genpd/mediatek/mt8183-pm-domains.h deleted file mode 100644 index 99de67fe5de8..000000000000 --- a/drivers/genpd/mediatek/mt8183-pm-domains.h +++ /dev/null @@ -1,266 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0-only */ - -#ifndef __SOC_MEDIATEK_MT8183_PM_DOMAINS_H -#define __SOC_MEDIATEK_MT8183_PM_DOMAINS_H - -#include "mtk-pm-domains.h" -#include - -/* - * MT8183 power domain support - */ - -static const struct scpsys_domain_data scpsys_domain_data_mt8183[] = { - [MT8183_POWER_DOMAIN_AUDIO] = { - .name = "audio", - .sta_mask = PWR_STATUS_AUDIO, - .ctl_offs = 0x0314, - .pwr_sta_offs = 0x0180, - .pwr_sta2nd_offs = 0x0184, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = GENMASK(15, 12), - }, - [MT8183_POWER_DOMAIN_CONN] = { - .name = "conn", - .sta_mask = PWR_STATUS_CONN, - .ctl_offs = 0x032c, - .pwr_sta_offs = 0x0180, - .pwr_sta2nd_offs = 0x0184, - .sram_pdn_bits = 0, - .sram_pdn_ack_bits = 0, - .bp_infracfg = { - BUS_PROT_WR(MT8183_TOP_AXI_PROT_EN_CONN, MT8183_TOP_AXI_PROT_EN_SET, - MT8183_TOP_AXI_PROT_EN_CLR, MT8183_TOP_AXI_PROT_EN_STA1), - }, - }, - [MT8183_POWER_DOMAIN_MFG_ASYNC] = { - .name = "mfg_async", - .sta_mask = PWR_STATUS_MFG_ASYNC, - .ctl_offs = 0x0334, - .pwr_sta_offs = 0x0180, - .pwr_sta2nd_offs = 0x0184, - .sram_pdn_bits = 0, - .sram_pdn_ack_bits = 0, - .caps = MTK_SCPD_DOMAIN_SUPPLY, - }, - [MT8183_POWER_DOMAIN_MFG] = { - .name = "mfg", - .sta_mask = PWR_STATUS_MFG, - .ctl_offs = 0x0338, - .pwr_sta_offs = 0x0180, - .pwr_sta2nd_offs = 0x0184, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .caps = MTK_SCPD_DOMAIN_SUPPLY, - }, - [MT8183_POWER_DOMAIN_MFG_CORE0] = { - .name = "mfg_core0", - .sta_mask = BIT(7), - .ctl_offs = 0x034c, - .pwr_sta_offs = 0x0180, - .pwr_sta2nd_offs = 0x0184, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - }, - [MT8183_POWER_DOMAIN_MFG_CORE1] = { - .name = "mfg_core1", - .sta_mask = BIT(20), - .ctl_offs = 0x0310, - .pwr_sta_offs = 0x0180, - .pwr_sta2nd_offs = 0x0184, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - }, - [MT8183_POWER_DOMAIN_MFG_2D] = { - .name = "mfg_2d", - .sta_mask = PWR_STATUS_MFG_2D, - .ctl_offs = 0x0348, - .pwr_sta_offs = 0x0180, - .pwr_sta2nd_offs = 0x0184, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .bp_infracfg = { - BUS_PROT_WR(MT8183_TOP_AXI_PROT_EN_1_MFG, MT8183_TOP_AXI_PROT_EN_1_SET, - MT8183_TOP_AXI_PROT_EN_1_CLR, MT8183_TOP_AXI_PROT_EN_STA1_1), - BUS_PROT_WR(MT8183_TOP_AXI_PROT_EN_MFG, MT8183_TOP_AXI_PROT_EN_SET, - MT8183_TOP_AXI_PROT_EN_CLR, MT8183_TOP_AXI_PROT_EN_STA1), - }, - }, - [MT8183_POWER_DOMAIN_DISP] = { - .name = "disp", - .sta_mask = PWR_STATUS_DISP, - .ctl_offs = 0x030c, - .pwr_sta_offs = 0x0180, - .pwr_sta2nd_offs = 0x0184, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .bp_infracfg = { - BUS_PROT_WR(MT8183_TOP_AXI_PROT_EN_1_DISP, MT8183_TOP_AXI_PROT_EN_1_SET, - MT8183_TOP_AXI_PROT_EN_1_CLR, MT8183_TOP_AXI_PROT_EN_STA1_1), - BUS_PROT_WR(MT8183_TOP_AXI_PROT_EN_DISP, MT8183_TOP_AXI_PROT_EN_SET, - MT8183_TOP_AXI_PROT_EN_CLR, MT8183_TOP_AXI_PROT_EN_STA1), - }, - .bp_smi = { - BUS_PROT_WR(MT8183_SMI_COMMON_SMI_CLAMP_DISP, - MT8183_SMI_COMMON_CLAMP_EN_SET, - MT8183_SMI_COMMON_CLAMP_EN_CLR, - MT8183_SMI_COMMON_CLAMP_EN), - }, - }, - [MT8183_POWER_DOMAIN_CAM] = { - .name = "cam", - .sta_mask = BIT(25), - .ctl_offs = 0x0344, - .pwr_sta_offs = 0x0180, - .pwr_sta2nd_offs = 0x0184, - .sram_pdn_bits = GENMASK(9, 8), - .sram_pdn_ack_bits = GENMASK(13, 12), - .bp_infracfg = { - BUS_PROT_WR(MT8183_TOP_AXI_PROT_EN_MM_CAM, MT8183_TOP_AXI_PROT_EN_MM_SET, - MT8183_TOP_AXI_PROT_EN_MM_CLR, MT8183_TOP_AXI_PROT_EN_MM_STA1), - BUS_PROT_WR(MT8183_TOP_AXI_PROT_EN_CAM, MT8183_TOP_AXI_PROT_EN_SET, - MT8183_TOP_AXI_PROT_EN_CLR, MT8183_TOP_AXI_PROT_EN_STA1), - BUS_PROT_WR_IGN(MT8183_TOP_AXI_PROT_EN_MM_CAM_2ND, - MT8183_TOP_AXI_PROT_EN_MM_SET, - MT8183_TOP_AXI_PROT_EN_MM_CLR, - MT8183_TOP_AXI_PROT_EN_MM_STA1), - }, - .bp_smi = { - BUS_PROT_WR(MT8183_SMI_COMMON_SMI_CLAMP_CAM, - MT8183_SMI_COMMON_CLAMP_EN_SET, - MT8183_SMI_COMMON_CLAMP_EN_CLR, - MT8183_SMI_COMMON_CLAMP_EN), - }, - }, - [MT8183_POWER_DOMAIN_ISP] = { - .name = "isp", - .sta_mask = PWR_STATUS_ISP, - .ctl_offs = 0x0308, - .pwr_sta_offs = 0x0180, - .pwr_sta2nd_offs = 0x0184, - .sram_pdn_bits = GENMASK(9, 8), - .sram_pdn_ack_bits = GENMASK(13, 12), - .bp_infracfg = { - BUS_PROT_WR(MT8183_TOP_AXI_PROT_EN_MM_ISP, - MT8183_TOP_AXI_PROT_EN_MM_SET, - MT8183_TOP_AXI_PROT_EN_MM_CLR, - MT8183_TOP_AXI_PROT_EN_MM_STA1), - BUS_PROT_WR_IGN(MT8183_TOP_AXI_PROT_EN_MM_ISP_2ND, - MT8183_TOP_AXI_PROT_EN_MM_SET, - MT8183_TOP_AXI_PROT_EN_MM_CLR, - MT8183_TOP_AXI_PROT_EN_MM_STA1), - }, - .bp_smi = { - BUS_PROT_WR(MT8183_SMI_COMMON_SMI_CLAMP_ISP, - MT8183_SMI_COMMON_CLAMP_EN_SET, - MT8183_SMI_COMMON_CLAMP_EN_CLR, - MT8183_SMI_COMMON_CLAMP_EN), - }, - }, - [MT8183_POWER_DOMAIN_VDEC] = { - .name = "vdec", - .sta_mask = BIT(31), - .ctl_offs = 0x0300, - .pwr_sta_offs = 0x0180, - .pwr_sta2nd_offs = 0x0184, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .bp_smi = { - BUS_PROT_WR(MT8183_SMI_COMMON_SMI_CLAMP_VDEC, - MT8183_SMI_COMMON_CLAMP_EN_SET, - MT8183_SMI_COMMON_CLAMP_EN_CLR, - MT8183_SMI_COMMON_CLAMP_EN), - }, - }, - [MT8183_POWER_DOMAIN_VENC] = { - .name = "venc", - .sta_mask = PWR_STATUS_VENC, - .ctl_offs = 0x0304, - .pwr_sta_offs = 0x0180, - .pwr_sta2nd_offs = 0x0184, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = GENMASK(15, 12), - .bp_smi = { - BUS_PROT_WR(MT8183_SMI_COMMON_SMI_CLAMP_VENC, - MT8183_SMI_COMMON_CLAMP_EN_SET, - MT8183_SMI_COMMON_CLAMP_EN_CLR, - MT8183_SMI_COMMON_CLAMP_EN), - }, - }, - [MT8183_POWER_DOMAIN_VPU_TOP] = { - .name = "vpu_top", - .sta_mask = BIT(26), - .ctl_offs = 0x0324, - .pwr_sta_offs = 0x0180, - .pwr_sta2nd_offs = 0x0184, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .bp_infracfg = { - BUS_PROT_WR(MT8183_TOP_AXI_PROT_EN_MM_VPU_TOP, - MT8183_TOP_AXI_PROT_EN_MM_SET, - MT8183_TOP_AXI_PROT_EN_MM_CLR, - MT8183_TOP_AXI_PROT_EN_MM_STA1), - BUS_PROT_WR(MT8183_TOP_AXI_PROT_EN_VPU_TOP, - MT8183_TOP_AXI_PROT_EN_SET, - MT8183_TOP_AXI_PROT_EN_CLR, - MT8183_TOP_AXI_PROT_EN_STA1), - BUS_PROT_WR(MT8183_TOP_AXI_PROT_EN_MM_VPU_TOP_2ND, - MT8183_TOP_AXI_PROT_EN_MM_SET, - MT8183_TOP_AXI_PROT_EN_MM_CLR, - MT8183_TOP_AXI_PROT_EN_MM_STA1), - }, - .bp_smi = { - BUS_PROT_WR(MT8183_SMI_COMMON_SMI_CLAMP_VPU_TOP, - MT8183_SMI_COMMON_CLAMP_EN_SET, - MT8183_SMI_COMMON_CLAMP_EN_CLR, - MT8183_SMI_COMMON_CLAMP_EN), - }, - }, - [MT8183_POWER_DOMAIN_VPU_CORE0] = { - .name = "vpu_core0", - .sta_mask = BIT(27), - .ctl_offs = 0x33c, - .pwr_sta_offs = 0x0180, - .pwr_sta2nd_offs = 0x0184, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = GENMASK(13, 12), - .bp_infracfg = { - BUS_PROT_WR(MT8183_TOP_AXI_PROT_EN_MCU_VPU_CORE0, - MT8183_TOP_AXI_PROT_EN_MCU_SET, - MT8183_TOP_AXI_PROT_EN_MCU_CLR, - MT8183_TOP_AXI_PROT_EN_MCU_STA1), - BUS_PROT_WR(MT8183_TOP_AXI_PROT_EN_MCU_VPU_CORE0_2ND, - MT8183_TOP_AXI_PROT_EN_MCU_SET, - MT8183_TOP_AXI_PROT_EN_MCU_CLR, - MT8183_TOP_AXI_PROT_EN_MCU_STA1), - }, - .caps = MTK_SCPD_SRAM_ISO, - }, - [MT8183_POWER_DOMAIN_VPU_CORE1] = { - .name = "vpu_core1", - .sta_mask = BIT(28), - .ctl_offs = 0x0340, - .pwr_sta_offs = 0x0180, - .pwr_sta2nd_offs = 0x0184, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = GENMASK(13, 12), - .bp_infracfg = { - BUS_PROT_WR(MT8183_TOP_AXI_PROT_EN_MCU_VPU_CORE1, - MT8183_TOP_AXI_PROT_EN_MCU_SET, - MT8183_TOP_AXI_PROT_EN_MCU_CLR, - MT8183_TOP_AXI_PROT_EN_MCU_STA1), - BUS_PROT_WR(MT8183_TOP_AXI_PROT_EN_MCU_VPU_CORE1_2ND, - MT8183_TOP_AXI_PROT_EN_MCU_SET, - MT8183_TOP_AXI_PROT_EN_MCU_CLR, - MT8183_TOP_AXI_PROT_EN_MCU_STA1), - }, - .caps = MTK_SCPD_SRAM_ISO, - }, -}; - -static const struct scpsys_soc_data mt8183_scpsys_data = { - .domains_data = scpsys_domain_data_mt8183, - .num_domains = ARRAY_SIZE(scpsys_domain_data_mt8183), -}; - -#endif /* __SOC_MEDIATEK_MT8183_PM_DOMAINS_H */ diff --git a/drivers/genpd/mediatek/mt8186-pm-domains.h b/drivers/genpd/mediatek/mt8186-pm-domains.h deleted file mode 100644 index fce86f79c505..000000000000 --- a/drivers/genpd/mediatek/mt8186-pm-domains.h +++ /dev/null @@ -1,342 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0-only */ -/* - * Copyright (c) 2022 MediaTek Inc. - * Author: Chun-Jie Chen - */ - -#ifndef __SOC_MEDIATEK_MT8186_PM_DOMAINS_H -#define __SOC_MEDIATEK_MT8186_PM_DOMAINS_H - -#include "mtk-pm-domains.h" -#include - -/* - * MT8186 power domain support - */ - -static const struct scpsys_domain_data scpsys_domain_data_mt8186[] = { - [MT8186_POWER_DOMAIN_MFG0] = { - .name = "mfg0", - .sta_mask = BIT(2), - .ctl_offs = 0x308, - .pwr_sta_offs = 0x16C, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = BIT(8), - .sram_pdn_ack_bits = BIT(12), - .caps = MTK_SCPD_KEEP_DEFAULT_OFF | MTK_SCPD_DOMAIN_SUPPLY, - }, - [MT8186_POWER_DOMAIN_MFG1] = { - .name = "mfg1", - .sta_mask = BIT(3), - .ctl_offs = 0x30c, - .pwr_sta_offs = 0x16C, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = BIT(8), - .sram_pdn_ack_bits = BIT(12), - .bp_infracfg = { - BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_1_MFG1_STEP1, - MT8186_TOP_AXI_PROT_EN_1_SET, - MT8186_TOP_AXI_PROT_EN_1_CLR, - MT8186_TOP_AXI_PROT_EN_1_STA), - BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_MFG1_STEP2, - MT8186_TOP_AXI_PROT_EN_SET, - MT8186_TOP_AXI_PROT_EN_CLR, - MT8186_TOP_AXI_PROT_EN_STA), - BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_MFG1_STEP3, - MT8186_TOP_AXI_PROT_EN_SET, - MT8186_TOP_AXI_PROT_EN_CLR, - MT8186_TOP_AXI_PROT_EN_STA), - BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_1_MFG1_STEP4, - MT8186_TOP_AXI_PROT_EN_1_SET, - MT8186_TOP_AXI_PROT_EN_1_CLR, - MT8186_TOP_AXI_PROT_EN_1_STA), - }, - .caps = MTK_SCPD_KEEP_DEFAULT_OFF | MTK_SCPD_DOMAIN_SUPPLY, - }, - [MT8186_POWER_DOMAIN_MFG2] = { - .name = "mfg2", - .sta_mask = BIT(4), - .ctl_offs = 0x310, - .pwr_sta_offs = 0x16C, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = BIT(8), - .sram_pdn_ack_bits = BIT(12), - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8186_POWER_DOMAIN_MFG3] = { - .name = "mfg3", - .sta_mask = BIT(5), - .ctl_offs = 0x314, - .pwr_sta_offs = 0x16C, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = BIT(8), - .sram_pdn_ack_bits = BIT(12), - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8186_POWER_DOMAIN_SSUSB] = { - .name = "ssusb", - .sta_mask = BIT(20), - .ctl_offs = 0x9F0, - .pwr_sta_offs = 0x16C, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = BIT(8), - .sram_pdn_ack_bits = BIT(12), - .caps = MTK_SCPD_ACTIVE_WAKEUP, - }, - [MT8186_POWER_DOMAIN_SSUSB_P1] = { - .name = "ssusb_p1", - .sta_mask = BIT(19), - .ctl_offs = 0x9F4, - .pwr_sta_offs = 0x16C, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = BIT(8), - .sram_pdn_ack_bits = BIT(12), - .caps = MTK_SCPD_ACTIVE_WAKEUP, - }, - [MT8186_POWER_DOMAIN_DIS] = { - .name = "dis", - .sta_mask = BIT(21), - .ctl_offs = 0x354, - .pwr_sta_offs = 0x16C, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = BIT(8), - .sram_pdn_ack_bits = BIT(12), - .bp_infracfg = { - BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_1_DIS_STEP1, - MT8186_TOP_AXI_PROT_EN_1_SET, - MT8186_TOP_AXI_PROT_EN_1_CLR, - MT8186_TOP_AXI_PROT_EN_1_STA), - BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_DIS_STEP2, - MT8186_TOP_AXI_PROT_EN_SET, - MT8186_TOP_AXI_PROT_EN_CLR, - MT8186_TOP_AXI_PROT_EN_STA), - }, - }, - [MT8186_POWER_DOMAIN_IMG] = { - .name = "img", - .sta_mask = BIT(13), - .ctl_offs = 0x334, - .pwr_sta_offs = 0x16C, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = BIT(8), - .sram_pdn_ack_bits = BIT(12), - .bp_infracfg = { - BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_1_IMG_STEP1, - MT8186_TOP_AXI_PROT_EN_1_SET, - MT8186_TOP_AXI_PROT_EN_1_CLR, - MT8186_TOP_AXI_PROT_EN_1_STA), - BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_1_IMG_STEP2, - MT8186_TOP_AXI_PROT_EN_1_SET, - MT8186_TOP_AXI_PROT_EN_1_CLR, - MT8186_TOP_AXI_PROT_EN_1_STA), - }, - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8186_POWER_DOMAIN_IMG2] = { - .name = "img2", - .sta_mask = BIT(14), - .ctl_offs = 0x338, - .pwr_sta_offs = 0x16C, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = BIT(8), - .sram_pdn_ack_bits = BIT(12), - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8186_POWER_DOMAIN_IPE] = { - .name = "ipe", - .sta_mask = BIT(15), - .ctl_offs = 0x33C, - .pwr_sta_offs = 0x16C, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = BIT(8), - .sram_pdn_ack_bits = BIT(12), - .bp_infracfg = { - BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_1_IPE_STEP1, - MT8186_TOP_AXI_PROT_EN_1_SET, - MT8186_TOP_AXI_PROT_EN_1_CLR, - MT8186_TOP_AXI_PROT_EN_1_STA), - BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_1_IPE_STEP2, - MT8186_TOP_AXI_PROT_EN_1_SET, - MT8186_TOP_AXI_PROT_EN_1_CLR, - MT8186_TOP_AXI_PROT_EN_1_STA), - }, - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8186_POWER_DOMAIN_CAM] = { - .name = "cam", - .sta_mask = BIT(23), - .ctl_offs = 0x35C, - .pwr_sta_offs = 0x16C, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = BIT(8), - .sram_pdn_ack_bits = BIT(12), - .bp_infracfg = { - BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_1_CAM_STEP1, - MT8186_TOP_AXI_PROT_EN_1_SET, - MT8186_TOP_AXI_PROT_EN_1_CLR, - MT8186_TOP_AXI_PROT_EN_1_STA), - BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_1_CAM_STEP2, - MT8186_TOP_AXI_PROT_EN_1_SET, - MT8186_TOP_AXI_PROT_EN_1_CLR, - MT8186_TOP_AXI_PROT_EN_1_STA), - }, - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8186_POWER_DOMAIN_CAM_RAWA] = { - .name = "cam_rawa", - .sta_mask = BIT(24), - .ctl_offs = 0x360, - .pwr_sta_offs = 0x16C, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = BIT(8), - .sram_pdn_ack_bits = BIT(12), - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8186_POWER_DOMAIN_CAM_RAWB] = { - .name = "cam_rawb", - .sta_mask = BIT(25), - .ctl_offs = 0x364, - .pwr_sta_offs = 0x16C, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = BIT(8), - .sram_pdn_ack_bits = BIT(12), - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8186_POWER_DOMAIN_VENC] = { - .name = "venc", - .sta_mask = BIT(18), - .ctl_offs = 0x348, - .pwr_sta_offs = 0x16C, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = BIT(8), - .sram_pdn_ack_bits = BIT(12), - .bp_infracfg = { - BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_1_VENC_STEP1, - MT8186_TOP_AXI_PROT_EN_1_SET, - MT8186_TOP_AXI_PROT_EN_1_CLR, - MT8186_TOP_AXI_PROT_EN_1_STA), - BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_1_VENC_STEP2, - MT8186_TOP_AXI_PROT_EN_1_SET, - MT8186_TOP_AXI_PROT_EN_1_CLR, - MT8186_TOP_AXI_PROT_EN_1_STA), - }, - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8186_POWER_DOMAIN_VDEC] = { - .name = "vdec", - .sta_mask = BIT(16), - .ctl_offs = 0x340, - .pwr_sta_offs = 0x16C, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = BIT(8), - .sram_pdn_ack_bits = BIT(12), - .bp_infracfg = { - BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_1_VDEC_STEP1, - MT8186_TOP_AXI_PROT_EN_1_SET, - MT8186_TOP_AXI_PROT_EN_1_CLR, - MT8186_TOP_AXI_PROT_EN_1_STA), - BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_1_VDEC_STEP2, - MT8186_TOP_AXI_PROT_EN_1_SET, - MT8186_TOP_AXI_PROT_EN_1_CLR, - MT8186_TOP_AXI_PROT_EN_1_STA), - }, - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8186_POWER_DOMAIN_WPE] = { - .name = "wpe", - .sta_mask = BIT(0), - .ctl_offs = 0x3F8, - .pwr_sta_offs = 0x16C, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = BIT(8), - .sram_pdn_ack_bits = BIT(12), - .bp_infracfg = { - BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_2_WPE_STEP1, - MT8186_TOP_AXI_PROT_EN_2_SET, - MT8186_TOP_AXI_PROT_EN_2_CLR, - MT8186_TOP_AXI_PROT_EN_2_STA), - BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_2_WPE_STEP2, - MT8186_TOP_AXI_PROT_EN_2_SET, - MT8186_TOP_AXI_PROT_EN_2_CLR, - MT8186_TOP_AXI_PROT_EN_2_STA), - }, - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8186_POWER_DOMAIN_CONN_ON] = { - .name = "conn_on", - .sta_mask = BIT(1), - .ctl_offs = 0x304, - .pwr_sta_offs = 0x16C, - .pwr_sta2nd_offs = 0x170, - .bp_infracfg = { - BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_1_CONN_ON_STEP1, - MT8186_TOP_AXI_PROT_EN_1_SET, - MT8186_TOP_AXI_PROT_EN_1_CLR, - MT8186_TOP_AXI_PROT_EN_1_STA), - BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_CONN_ON_STEP2, - MT8186_TOP_AXI_PROT_EN_SET, - MT8186_TOP_AXI_PROT_EN_CLR, - MT8186_TOP_AXI_PROT_EN_STA), - BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_CONN_ON_STEP3, - MT8186_TOP_AXI_PROT_EN_SET, - MT8186_TOP_AXI_PROT_EN_CLR, - MT8186_TOP_AXI_PROT_EN_STA), - BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_CONN_ON_STEP4, - MT8186_TOP_AXI_PROT_EN_SET, - MT8186_TOP_AXI_PROT_EN_CLR, - MT8186_TOP_AXI_PROT_EN_STA), - }, - .caps = MTK_SCPD_KEEP_DEFAULT_OFF | MTK_SCPD_ACTIVE_WAKEUP, - }, - [MT8186_POWER_DOMAIN_CSIRX_TOP] = { - .name = "csirx_top", - .sta_mask = BIT(6), - .ctl_offs = 0x318, - .pwr_sta_offs = 0x16C, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = BIT(8), - .sram_pdn_ack_bits = BIT(12), - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8186_POWER_DOMAIN_ADSP_AO] = { - .name = "adsp_ao", - .sta_mask = BIT(17), - .ctl_offs = 0x9FC, - .pwr_sta_offs = 0x16C, - .pwr_sta2nd_offs = 0x170, - }, - [MT8186_POWER_DOMAIN_ADSP_INFRA] = { - .name = "adsp_infra", - .sta_mask = BIT(10), - .ctl_offs = 0x9F8, - .pwr_sta_offs = 0x16C, - .pwr_sta2nd_offs = 0x170, - }, - [MT8186_POWER_DOMAIN_ADSP_TOP] = { - .name = "adsp_top", - .sta_mask = BIT(31), - .ctl_offs = 0x3E4, - .pwr_sta_offs = 0x16C, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = BIT(8), - .sram_pdn_ack_bits = BIT(12), - .bp_infracfg = { - BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_3_ADSP_TOP_STEP1, - MT8186_TOP_AXI_PROT_EN_3_SET, - MT8186_TOP_AXI_PROT_EN_3_CLR, - MT8186_TOP_AXI_PROT_EN_3_STA), - BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_3_ADSP_TOP_STEP2, - MT8186_TOP_AXI_PROT_EN_3_SET, - MT8186_TOP_AXI_PROT_EN_3_CLR, - MT8186_TOP_AXI_PROT_EN_3_STA), - }, - .caps = MTK_SCPD_SRAM_ISO | MTK_SCPD_ACTIVE_WAKEUP, - }, -}; - -static const struct scpsys_soc_data mt8186_scpsys_data = { - .domains_data = scpsys_domain_data_mt8186, - .num_domains = ARRAY_SIZE(scpsys_domain_data_mt8186), -}; - -#endif /* __SOC_MEDIATEK_MT8186_PM_DOMAINS_H */ diff --git a/drivers/genpd/mediatek/mt8188-pm-domains.h b/drivers/genpd/mediatek/mt8188-pm-domains.h deleted file mode 100644 index 0692cb444ed0..000000000000 --- a/drivers/genpd/mediatek/mt8188-pm-domains.h +++ /dev/null @@ -1,623 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0-only */ -/* - * Copyright (c) 2022 MediaTek Inc. - * Author: Garmin Chang - */ - -#ifndef __SOC_MEDIATEK_MT8188_PM_DOMAINS_H -#define __SOC_MEDIATEK_MT8188_PM_DOMAINS_H - -#include "mtk-pm-domains.h" -#include - -/* - * MT8188 power domain support - */ - -static const struct scpsys_domain_data scpsys_domain_data_mt8188[] = { - [MT8188_POWER_DOMAIN_MFG0] = { - .name = "mfg0", - .sta_mask = BIT(1), - .ctl_offs = 0x300, - .pwr_sta_offs = 0x174, - .pwr_sta2nd_offs = 0x178, - .sram_pdn_bits = BIT(8), - .sram_pdn_ack_bits = BIT(12), - .caps = MTK_SCPD_KEEP_DEFAULT_OFF | MTK_SCPD_DOMAIN_SUPPLY, - }, - [MT8188_POWER_DOMAIN_MFG1] = { - .name = "mfg1", - .sta_mask = BIT(2), - .ctl_offs = 0x304, - .pwr_sta_offs = 0x174, - .pwr_sta2nd_offs = 0x178, - .sram_pdn_bits = BIT(8), - .sram_pdn_ack_bits = BIT(12), - .bp_infracfg = { - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MFG1_STEP1, - MT8188_TOP_AXI_PROT_EN_SET, - MT8188_TOP_AXI_PROT_EN_CLR, - MT8188_TOP_AXI_PROT_EN_STA), - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_2_MFG1_STEP2, - MT8188_TOP_AXI_PROT_EN_2_SET, - MT8188_TOP_AXI_PROT_EN_2_CLR, - MT8188_TOP_AXI_PROT_EN_2_STA), - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_1_MFG1_STEP3, - MT8188_TOP_AXI_PROT_EN_1_SET, - MT8188_TOP_AXI_PROT_EN_1_CLR, - MT8188_TOP_AXI_PROT_EN_1_STA), - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_2_MFG1_STEP4, - MT8188_TOP_AXI_PROT_EN_2_SET, - MT8188_TOP_AXI_PROT_EN_2_CLR, - MT8188_TOP_AXI_PROT_EN_2_STA), - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MFG1_STEP5, - MT8188_TOP_AXI_PROT_EN_SET, - MT8188_TOP_AXI_PROT_EN_CLR, - MT8188_TOP_AXI_PROT_EN_STA), - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_MFG1_STEP6, - MT8188_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_SET, - MT8188_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_CLR, - MT8188_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_STA), - }, - .caps = MTK_SCPD_KEEP_DEFAULT_OFF | MTK_SCPD_DOMAIN_SUPPLY, - }, - [MT8188_POWER_DOMAIN_MFG2] = { - .name = "mfg2", - .sta_mask = BIT(3), - .ctl_offs = 0x308, - .pwr_sta_offs = 0x174, - .pwr_sta2nd_offs = 0x178, - .sram_pdn_bits = BIT(8), - .sram_pdn_ack_bits = BIT(12), - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8188_POWER_DOMAIN_MFG3] = { - .name = "mfg3", - .sta_mask = BIT(4), - .ctl_offs = 0x30C, - .pwr_sta_offs = 0x174, - .pwr_sta2nd_offs = 0x178, - .sram_pdn_bits = BIT(8), - .sram_pdn_ack_bits = BIT(12), - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8188_POWER_DOMAIN_MFG4] = { - .name = "mfg4", - .sta_mask = BIT(5), - .ctl_offs = 0x310, - .pwr_sta_offs = 0x174, - .pwr_sta2nd_offs = 0x178, - .sram_pdn_bits = BIT(8), - .sram_pdn_ack_bits = BIT(12), - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8188_POWER_DOMAIN_PEXTP_MAC_P0] = { - .name = "pextp_mac_p0", - .sta_mask = BIT(10), - .ctl_offs = 0x324, - .pwr_sta_offs = 0x174, - .pwr_sta2nd_offs = 0x178, - .sram_pdn_bits = BIT(8), - .sram_pdn_ack_bits = BIT(12), - .bp_infracfg = { - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_PEXTP_MAC_P0_STEP1, - MT8188_TOP_AXI_PROT_EN_SET, - MT8188_TOP_AXI_PROT_EN_CLR, - MT8188_TOP_AXI_PROT_EN_STA), - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_INFRA_VDNR_PEXTP_MAC_P0_STEP2, - MT8188_TOP_AXI_PROT_EN_INFRA_VDNR_SET, - MT8188_TOP_AXI_PROT_EN_INFRA_VDNR_CLR, - MT8188_TOP_AXI_PROT_EN_INFRA_VDNR_STA), - }, - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8188_POWER_DOMAIN_PEXTP_PHY_TOP] = { - .name = "pextp_phy_top", - .sta_mask = BIT(12), - .ctl_offs = 0x328, - .pwr_sta_offs = 0x174, - .pwr_sta2nd_offs = 0x178, - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8188_POWER_DOMAIN_CSIRX_TOP] = { - .name = "pextp_csirx_top", - .sta_mask = BIT(17), - .ctl_offs = 0x3C4, - .pwr_sta_offs = 0x174, - .pwr_sta2nd_offs = 0x178, - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8188_POWER_DOMAIN_ETHER] = { - .name = "ether", - .sta_mask = BIT(1), - .ctl_offs = 0x338, - .pwr_sta_offs = 0x16C, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = BIT(8), - .sram_pdn_ack_bits = BIT(12), - .bp_infracfg = { - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_INFRA_VDNR_ETHER_STEP1, - MT8188_TOP_AXI_PROT_EN_INFRA_VDNR_SET, - MT8188_TOP_AXI_PROT_EN_INFRA_VDNR_CLR, - MT8188_TOP_AXI_PROT_EN_INFRA_VDNR_STA), - }, - .caps = MTK_SCPD_KEEP_DEFAULT_OFF | MTK_SCPD_ACTIVE_WAKEUP, - }, - [MT8188_POWER_DOMAIN_HDMI_TX] = { - .name = "hdmi_tx", - .sta_mask = BIT(18), - .ctl_offs = 0x37C, - .pwr_sta_offs = 0x16C, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = BIT(8), - .sram_pdn_ack_bits = BIT(12), - .bp_infracfg = { - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_INFRA_VDNR_HDMI_TX_STEP1, - MT8188_TOP_AXI_PROT_EN_INFRA_VDNR_SET, - MT8188_TOP_AXI_PROT_EN_INFRA_VDNR_CLR, - MT8188_TOP_AXI_PROT_EN_INFRA_VDNR_STA), - }, - .caps = MTK_SCPD_KEEP_DEFAULT_OFF | MTK_SCPD_ACTIVE_WAKEUP, - }, - [MT8188_POWER_DOMAIN_ADSP_AO] = { - .name = "adsp_ao", - .sta_mask = BIT(10), - .ctl_offs = 0x35C, - .pwr_sta_offs = 0x16C, - .pwr_sta2nd_offs = 0x170, - .bp_infracfg = { - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_2_ADSP_AO_STEP1, - MT8188_TOP_AXI_PROT_EN_2_SET, - MT8188_TOP_AXI_PROT_EN_2_CLR, - MT8188_TOP_AXI_PROT_EN_2_STA), - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_2_ADSP_AO_STEP2, - MT8188_TOP_AXI_PROT_EN_2_SET, - MT8188_TOP_AXI_PROT_EN_2_CLR, - MT8188_TOP_AXI_PROT_EN_2_STA), - }, - .caps = MTK_SCPD_ALWAYS_ON, - }, - [MT8188_POWER_DOMAIN_ADSP_INFRA] = { - .name = "adsp_infra", - .sta_mask = BIT(9), - .ctl_offs = 0x358, - .pwr_sta_offs = 0x16C, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = BIT(8), - .sram_pdn_ack_bits = BIT(12), - .bp_infracfg = { - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_2_ADSP_INFRA_STEP1, - MT8188_TOP_AXI_PROT_EN_2_SET, - MT8188_TOP_AXI_PROT_EN_2_CLR, - MT8188_TOP_AXI_PROT_EN_2_STA), - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_2_ADSP_INFRA_STEP2, - MT8188_TOP_AXI_PROT_EN_2_SET, - MT8188_TOP_AXI_PROT_EN_2_CLR, - MT8188_TOP_AXI_PROT_EN_2_STA), - }, - .caps = MTK_SCPD_SRAM_ISO | MTK_SCPD_ALWAYS_ON, - }, - [MT8188_POWER_DOMAIN_ADSP] = { - .name = "adsp", - .sta_mask = BIT(8), - .ctl_offs = 0x354, - .pwr_sta_offs = 0x16C, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = BIT(8), - .sram_pdn_ack_bits = BIT(12), - .bp_infracfg = { - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_2_ADSP_STEP1, - MT8188_TOP_AXI_PROT_EN_2_SET, - MT8188_TOP_AXI_PROT_EN_2_CLR, - MT8188_TOP_AXI_PROT_EN_2_STA), - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_2_ADSP_STEP2, - MT8188_TOP_AXI_PROT_EN_2_SET, - MT8188_TOP_AXI_PROT_EN_2_CLR, - MT8188_TOP_AXI_PROT_EN_2_STA), - }, - .caps = MTK_SCPD_KEEP_DEFAULT_OFF | MTK_SCPD_SRAM_ISO | MTK_SCPD_ACTIVE_WAKEUP, - }, - [MT8188_POWER_DOMAIN_AUDIO] = { - .name = "audio", - .sta_mask = BIT(6), - .ctl_offs = 0x34C, - .pwr_sta_offs = 0x16C, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = BIT(8), - .sram_pdn_ack_bits = BIT(12), - .bp_infracfg = { - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_2_AUDIO_STEP1, - MT8188_TOP_AXI_PROT_EN_2_SET, - MT8188_TOP_AXI_PROT_EN_2_CLR, - MT8188_TOP_AXI_PROT_EN_2_STA), - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_2_AUDIO_STEP2, - MT8188_TOP_AXI_PROT_EN_2_SET, - MT8188_TOP_AXI_PROT_EN_2_CLR, - MT8188_TOP_AXI_PROT_EN_2_STA), - }, - .caps = MTK_SCPD_KEEP_DEFAULT_OFF | MTK_SCPD_ACTIVE_WAKEUP, - }, - [MT8188_POWER_DOMAIN_AUDIO_ASRC] = { - .name = "audio_asrc", - .sta_mask = BIT(7), - .ctl_offs = 0x350, - .pwr_sta_offs = 0x16C, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = BIT(8), - .sram_pdn_ack_bits = BIT(12), - .bp_infracfg = { - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_2_AUDIO_ASRC_STEP1, - MT8188_TOP_AXI_PROT_EN_2_SET, - MT8188_TOP_AXI_PROT_EN_2_CLR, - MT8188_TOP_AXI_PROT_EN_2_STA), - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_2_AUDIO_ASRC_STEP2, - MT8188_TOP_AXI_PROT_EN_2_SET, - MT8188_TOP_AXI_PROT_EN_2_CLR, - MT8188_TOP_AXI_PROT_EN_2_STA), - }, - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8188_POWER_DOMAIN_VPPSYS0] = { - .name = "vppsys0", - .sta_mask = BIT(11), - .ctl_offs = 0x360, - .pwr_sta_offs = 0x16C, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = BIT(8), - .sram_pdn_ack_bits = BIT(12), - .bp_infracfg = { - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_VPPSYS0_STEP1, - MT8188_TOP_AXI_PROT_EN_SET, - MT8188_TOP_AXI_PROT_EN_CLR, - MT8188_TOP_AXI_PROT_EN_STA), - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_2_VPPSYS0_STEP2, - MT8188_TOP_AXI_PROT_EN_MM_2_SET, - MT8188_TOP_AXI_PROT_EN_MM_2_CLR, - MT8188_TOP_AXI_PROT_EN_MM_2_STA), - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_VPPSYS0_STEP3, - MT8188_TOP_AXI_PROT_EN_SET, - MT8188_TOP_AXI_PROT_EN_CLR, - MT8188_TOP_AXI_PROT_EN_STA), - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_2_VPPSYS0_STEP4, - MT8188_TOP_AXI_PROT_EN_MM_2_SET, - MT8188_TOP_AXI_PROT_EN_MM_2_CLR, - MT8188_TOP_AXI_PROT_EN_MM_2_STA), - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_VPPSYS0_STEP5, - MT8188_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_SET, - MT8188_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_CLR, - MT8188_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_STA), - }, - }, - [MT8188_POWER_DOMAIN_VDOSYS0] = { - .name = "vdosys0", - .sta_mask = BIT(13), - .ctl_offs = 0x368, - .pwr_sta_offs = 0x16C, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = BIT(8), - .sram_pdn_ack_bits = BIT(12), - .bp_infracfg = { - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_VDOSYS0_STEP1, - MT8188_TOP_AXI_PROT_EN_MM_SET, - MT8188_TOP_AXI_PROT_EN_MM_CLR, - MT8188_TOP_AXI_PROT_EN_MM_STA), - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_VDOSYS0_STEP2, - MT8188_TOP_AXI_PROT_EN_SET, - MT8188_TOP_AXI_PROT_EN_CLR, - MT8188_TOP_AXI_PROT_EN_STA), - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_VDOSYS0_STEP3, - MT8188_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_SET, - MT8188_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_CLR, - MT8188_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_STA), - }, - }, - [MT8188_POWER_DOMAIN_VDOSYS1] = { - .name = "vdosys1", - .sta_mask = BIT(14), - .ctl_offs = 0x36C, - .pwr_sta_offs = 0x16C, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = BIT(8), - .sram_pdn_ack_bits = BIT(12), - .bp_infracfg = { - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_VDOSYS1_STEP1, - MT8188_TOP_AXI_PROT_EN_MM_SET, - MT8188_TOP_AXI_PROT_EN_MM_CLR, - MT8188_TOP_AXI_PROT_EN_MM_STA), - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_VDOSYS1_STEP2, - MT8188_TOP_AXI_PROT_EN_MM_SET, - MT8188_TOP_AXI_PROT_EN_MM_CLR, - MT8188_TOP_AXI_PROT_EN_MM_STA), - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_2_VDOSYS1_STEP3, - MT8188_TOP_AXI_PROT_EN_MM_2_SET, - MT8188_TOP_AXI_PROT_EN_MM_2_CLR, - MT8188_TOP_AXI_PROT_EN_MM_2_STA), - }, - }, - [MT8188_POWER_DOMAIN_DP_TX] = { - .name = "dp_tx", - .sta_mask = BIT(16), - .ctl_offs = 0x374, - .pwr_sta_offs = 0x16C, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = BIT(8), - .sram_pdn_ack_bits = BIT(12), - .bp_infracfg = { - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_INFRA_VDNR_DP_TX_STEP1, - MT8188_TOP_AXI_PROT_EN_INFRA_VDNR_SET, - MT8188_TOP_AXI_PROT_EN_INFRA_VDNR_CLR, - MT8188_TOP_AXI_PROT_EN_INFRA_VDNR_STA), - }, - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8188_POWER_DOMAIN_EDP_TX] = { - .name = "edp_tx", - .sta_mask = BIT(17), - .ctl_offs = 0x378, - .pwr_sta_offs = 0x16C, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = BIT(8), - .sram_pdn_ack_bits = BIT(12), - .bp_infracfg = { - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_INFRA_VDNR_EDP_TX_STEP1, - MT8188_TOP_AXI_PROT_EN_INFRA_VDNR_SET, - MT8188_TOP_AXI_PROT_EN_INFRA_VDNR_CLR, - MT8188_TOP_AXI_PROT_EN_INFRA_VDNR_STA), - }, - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8188_POWER_DOMAIN_VPPSYS1] = { - .name = "vppsys1", - .sta_mask = BIT(12), - .ctl_offs = 0x364, - .pwr_sta_offs = 0x16C, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = BIT(8), - .sram_pdn_ack_bits = BIT(12), - .bp_infracfg = { - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_VPPSYS1_STEP1, - MT8188_TOP_AXI_PROT_EN_MM_SET, - MT8188_TOP_AXI_PROT_EN_MM_CLR, - MT8188_TOP_AXI_PROT_EN_MM_STA), - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_VPPSYS1_STEP2, - MT8188_TOP_AXI_PROT_EN_MM_SET, - MT8188_TOP_AXI_PROT_EN_MM_CLR, - MT8188_TOP_AXI_PROT_EN_MM_STA), - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_2_VPPSYS1_STEP3, - MT8188_TOP_AXI_PROT_EN_MM_2_SET, - MT8188_TOP_AXI_PROT_EN_MM_2_CLR, - MT8188_TOP_AXI_PROT_EN_MM_2_STA), - }, - }, - [MT8188_POWER_DOMAIN_WPE] = { - .name = "wpe", - .sta_mask = BIT(15), - .ctl_offs = 0x370, - .pwr_sta_offs = 0x16C, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = BIT(8), - .sram_pdn_ack_bits = BIT(12), - .bp_infracfg = { - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_2_WPE_STEP1, - MT8188_TOP_AXI_PROT_EN_MM_2_SET, - MT8188_TOP_AXI_PROT_EN_MM_2_CLR, - MT8188_TOP_AXI_PROT_EN_MM_2_STA), - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_2_WPE_STEP2, - MT8188_TOP_AXI_PROT_EN_MM_2_SET, - MT8188_TOP_AXI_PROT_EN_MM_2_CLR, - MT8188_TOP_AXI_PROT_EN_MM_2_STA), - }, - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8188_POWER_DOMAIN_VDEC0] = { - .name = "vdec0", - .sta_mask = BIT(19), - .ctl_offs = 0x380, - .pwr_sta_offs = 0x16C, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = BIT(8), - .sram_pdn_ack_bits = BIT(12), - .bp_infracfg = { - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_VDEC0_STEP1, - MT8188_TOP_AXI_PROT_EN_MM_SET, - MT8188_TOP_AXI_PROT_EN_MM_CLR, - MT8188_TOP_AXI_PROT_EN_MM_STA), - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_2_VDEC0_STEP2, - MT8188_TOP_AXI_PROT_EN_MM_2_SET, - MT8188_TOP_AXI_PROT_EN_MM_2_CLR, - MT8188_TOP_AXI_PROT_EN_MM_2_STA), - }, - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8188_POWER_DOMAIN_VDEC1] = { - .name = "vdec1", - .sta_mask = BIT(20), - .ctl_offs = 0x384, - .pwr_sta_offs = 0x16C, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = BIT(8), - .sram_pdn_ack_bits = BIT(12), - .bp_infracfg = { - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_VDEC1_STEP1, - MT8188_TOP_AXI_PROT_EN_MM_SET, - MT8188_TOP_AXI_PROT_EN_MM_CLR, - MT8188_TOP_AXI_PROT_EN_MM_STA), - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_VDEC1_STEP2, - MT8188_TOP_AXI_PROT_EN_MM_SET, - MT8188_TOP_AXI_PROT_EN_MM_CLR, - MT8188_TOP_AXI_PROT_EN_MM_STA), - }, - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8188_POWER_DOMAIN_VENC] = { - .name = "venc", - .sta_mask = BIT(22), - .ctl_offs = 0x38C, - .pwr_sta_offs = 0x16C, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = BIT(8), - .sram_pdn_ack_bits = BIT(12), - .bp_infracfg = { - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_VENC_STEP1, - MT8188_TOP_AXI_PROT_EN_MM_SET, - MT8188_TOP_AXI_PROT_EN_MM_CLR, - MT8188_TOP_AXI_PROT_EN_MM_STA), - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_VENC_STEP2, - MT8188_TOP_AXI_PROT_EN_MM_SET, - MT8188_TOP_AXI_PROT_EN_MM_CLR, - MT8188_TOP_AXI_PROT_EN_MM_STA), - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_2_VENC_STEP3, - MT8188_TOP_AXI_PROT_EN_MM_2_SET, - MT8188_TOP_AXI_PROT_EN_MM_2_CLR, - MT8188_TOP_AXI_PROT_EN_MM_2_STA), - }, - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8188_POWER_DOMAIN_IMG_VCORE] = { - .name = "vcore", - .sta_mask = BIT(28), - .ctl_offs = 0x3A4, - .pwr_sta_offs = 0x16C, - .pwr_sta2nd_offs = 0x170, - .bp_infracfg = { - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_IMG_VCORE_STEP1, - MT8188_TOP_AXI_PROT_EN_MM_SET, - MT8188_TOP_AXI_PROT_EN_MM_CLR, - MT8188_TOP_AXI_PROT_EN_MM_STA), - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_IMG_VCORE_STEP2, - MT8188_TOP_AXI_PROT_EN_MM_SET, - MT8188_TOP_AXI_PROT_EN_MM_CLR, - MT8188_TOP_AXI_PROT_EN_MM_STA), - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_2_IMG_VCORE_STEP3, - MT8188_TOP_AXI_PROT_EN_MM_2_SET, - MT8188_TOP_AXI_PROT_EN_MM_2_CLR, - MT8188_TOP_AXI_PROT_EN_MM_2_STA), - }, - .caps = MTK_SCPD_KEEP_DEFAULT_OFF | MTK_SCPD_DOMAIN_SUPPLY, - }, - [MT8188_POWER_DOMAIN_IMG_MAIN] = { - .name = "img_main", - .sta_mask = BIT(29), - .ctl_offs = 0x3A8, - .pwr_sta_offs = 0x16C, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = BIT(8), - .sram_pdn_ack_bits = BIT(12), - .bp_infracfg = { - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_2_IMG_MAIN_STEP1, - MT8188_TOP_AXI_PROT_EN_MM_2_SET, - MT8188_TOP_AXI_PROT_EN_MM_2_CLR, - MT8188_TOP_AXI_PROT_EN_MM_2_STA), - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_2_IMG_MAIN_STEP2, - MT8188_TOP_AXI_PROT_EN_MM_2_SET, - MT8188_TOP_AXI_PROT_EN_MM_2_CLR, - MT8188_TOP_AXI_PROT_EN_MM_2_STA), - }, - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8188_POWER_DOMAIN_DIP] = { - .name = "dip", - .sta_mask = BIT(30), - .ctl_offs = 0x3AC, - .pwr_sta_offs = 0x16C, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = BIT(8), - .sram_pdn_ack_bits = BIT(12), - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8188_POWER_DOMAIN_IPE] = { - .name = "ipe", - .sta_mask = BIT(31), - .ctl_offs = 0x3B0, - .pwr_sta_offs = 0x16C, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = BIT(8), - .sram_pdn_ack_bits = BIT(12), - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8188_POWER_DOMAIN_CAM_VCORE] = { - .name = "cam_vcore", - .sta_mask = BIT(27), - .ctl_offs = 0x3A0, - .pwr_sta_offs = 0x16C, - .pwr_sta2nd_offs = 0x170, - .bp_infracfg = { - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_CAM_VCORE_STEP1, - MT8188_TOP_AXI_PROT_EN_MM_SET, - MT8188_TOP_AXI_PROT_EN_MM_CLR, - MT8188_TOP_AXI_PROT_EN_MM_STA), - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_2_CAM_VCORE_STEP2, - MT8188_TOP_AXI_PROT_EN_2_SET, - MT8188_TOP_AXI_PROT_EN_2_CLR, - MT8188_TOP_AXI_PROT_EN_2_STA), - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_1_CAM_VCORE_STEP3, - MT8188_TOP_AXI_PROT_EN_1_SET, - MT8188_TOP_AXI_PROT_EN_1_CLR, - MT8188_TOP_AXI_PROT_EN_1_STA), - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_CAM_VCORE_STEP4, - MT8188_TOP_AXI_PROT_EN_MM_SET, - MT8188_TOP_AXI_PROT_EN_MM_CLR, - MT8188_TOP_AXI_PROT_EN_MM_STA), - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_2_CAM_VCORE_STEP5, - MT8188_TOP_AXI_PROT_EN_MM_2_SET, - MT8188_TOP_AXI_PROT_EN_MM_2_CLR, - MT8188_TOP_AXI_PROT_EN_MM_2_STA), - }, - .caps = MTK_SCPD_KEEP_DEFAULT_OFF | MTK_SCPD_DOMAIN_SUPPLY, - }, - [MT8188_POWER_DOMAIN_CAM_MAIN] = { - .name = "cam_main", - .sta_mask = BIT(24), - .ctl_offs = 0x394, - .pwr_sta_offs = 0x16C, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = BIT(8), - .sram_pdn_ack_bits = BIT(12), - .bp_infracfg = { - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_2_CAM_MAIN_STEP1, - MT8188_TOP_AXI_PROT_EN_MM_2_SET, - MT8188_TOP_AXI_PROT_EN_MM_2_CLR, - MT8188_TOP_AXI_PROT_EN_MM_2_STA), - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_2_CAM_MAIN_STEP2, - MT8188_TOP_AXI_PROT_EN_2_SET, - MT8188_TOP_AXI_PROT_EN_2_CLR, - MT8188_TOP_AXI_PROT_EN_2_STA), - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_2_CAM_MAIN_STEP3, - MT8188_TOP_AXI_PROT_EN_MM_2_SET, - MT8188_TOP_AXI_PROT_EN_MM_2_CLR, - MT8188_TOP_AXI_PROT_EN_MM_2_STA), - BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_2_CAM_MAIN_STEP4, - MT8188_TOP_AXI_PROT_EN_2_SET, - MT8188_TOP_AXI_PROT_EN_2_CLR, - MT8188_TOP_AXI_PROT_EN_2_STA), - }, - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8188_POWER_DOMAIN_CAM_SUBA] = { - .name = "cam_suba", - .sta_mask = BIT(25), - .ctl_offs = 0x398, - .pwr_sta_offs = 0x16C, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = BIT(8), - .sram_pdn_ack_bits = BIT(12), - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8188_POWER_DOMAIN_CAM_SUBB] = { - .name = "cam_subb", - .sta_mask = BIT(26), - .ctl_offs = 0x39C, - .pwr_sta_offs = 0x16C, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = BIT(8), - .sram_pdn_ack_bits = BIT(12), - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, -}; - -static const struct scpsys_soc_data mt8188_scpsys_data = { - .domains_data = scpsys_domain_data_mt8188, - .num_domains = ARRAY_SIZE(scpsys_domain_data_mt8188), -}; - -#endif /* __SOC_MEDIATEK_MT8188_PM_DOMAINS_H */ diff --git a/drivers/genpd/mediatek/mt8192-pm-domains.h b/drivers/genpd/mediatek/mt8192-pm-domains.h deleted file mode 100644 index b97b2051920f..000000000000 --- a/drivers/genpd/mediatek/mt8192-pm-domains.h +++ /dev/null @@ -1,355 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0-only */ - -#ifndef __SOC_MEDIATEK_MT8192_PM_DOMAINS_H -#define __SOC_MEDIATEK_MT8192_PM_DOMAINS_H - -#include "mtk-pm-domains.h" -#include - -/* - * MT8192 power domain support - */ - -static const struct scpsys_domain_data scpsys_domain_data_mt8192[] = { - [MT8192_POWER_DOMAIN_AUDIO] = { - .name = "audio", - .sta_mask = BIT(21), - .ctl_offs = 0x0354, - .pwr_sta_offs = 0x016c, - .pwr_sta2nd_offs = 0x0170, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .bp_infracfg = { - BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_2_AUDIO, - MT8192_TOP_AXI_PROT_EN_2_SET, - MT8192_TOP_AXI_PROT_EN_2_CLR, - MT8192_TOP_AXI_PROT_EN_2_STA1), - }, - }, - [MT8192_POWER_DOMAIN_CONN] = { - .name = "conn", - .sta_mask = PWR_STATUS_CONN, - .ctl_offs = 0x0304, - .pwr_sta_offs = 0x016c, - .pwr_sta2nd_offs = 0x0170, - .sram_pdn_bits = 0, - .sram_pdn_ack_bits = 0, - .bp_infracfg = { - BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_CONN, - MT8192_TOP_AXI_PROT_EN_SET, - MT8192_TOP_AXI_PROT_EN_CLR, - MT8192_TOP_AXI_PROT_EN_STA1), - BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_CONN_2ND, - MT8192_TOP_AXI_PROT_EN_SET, - MT8192_TOP_AXI_PROT_EN_CLR, - MT8192_TOP_AXI_PROT_EN_STA1), - BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_1_CONN, - MT8192_TOP_AXI_PROT_EN_1_SET, - MT8192_TOP_AXI_PROT_EN_1_CLR, - MT8192_TOP_AXI_PROT_EN_1_STA1), - }, - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8192_POWER_DOMAIN_MFG0] = { - .name = "mfg0", - .sta_mask = BIT(2), - .ctl_offs = 0x0308, - .pwr_sta_offs = 0x016c, - .pwr_sta2nd_offs = 0x0170, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .caps = MTK_SCPD_DOMAIN_SUPPLY, - }, - [MT8192_POWER_DOMAIN_MFG1] = { - .name = "mfg1", - .sta_mask = BIT(3), - .ctl_offs = 0x030c, - .pwr_sta_offs = 0x016c, - .pwr_sta2nd_offs = 0x0170, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .bp_infracfg = { - BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_1_MFG1, - MT8192_TOP_AXI_PROT_EN_1_SET, - MT8192_TOP_AXI_PROT_EN_1_CLR, - MT8192_TOP_AXI_PROT_EN_1_STA1), - BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_2_MFG1, - MT8192_TOP_AXI_PROT_EN_2_SET, - MT8192_TOP_AXI_PROT_EN_2_CLR, - MT8192_TOP_AXI_PROT_EN_2_STA1), - BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_MFG1, - MT8192_TOP_AXI_PROT_EN_SET, - MT8192_TOP_AXI_PROT_EN_CLR, - MT8192_TOP_AXI_PROT_EN_STA1), - BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_2_MFG1_2ND, - MT8192_TOP_AXI_PROT_EN_2_SET, - MT8192_TOP_AXI_PROT_EN_2_CLR, - MT8192_TOP_AXI_PROT_EN_2_STA1), - }, - .caps = MTK_SCPD_DOMAIN_SUPPLY, - }, - [MT8192_POWER_DOMAIN_MFG2] = { - .name = "mfg2", - .sta_mask = BIT(4), - .ctl_offs = 0x0310, - .pwr_sta_offs = 0x016c, - .pwr_sta2nd_offs = 0x0170, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - }, - [MT8192_POWER_DOMAIN_MFG3] = { - .name = "mfg3", - .sta_mask = BIT(5), - .ctl_offs = 0x0314, - .pwr_sta_offs = 0x016c, - .pwr_sta2nd_offs = 0x0170, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - }, - [MT8192_POWER_DOMAIN_MFG4] = { - .name = "mfg4", - .sta_mask = BIT(6), - .ctl_offs = 0x0318, - .pwr_sta_offs = 0x016c, - .pwr_sta2nd_offs = 0x0170, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - }, - [MT8192_POWER_DOMAIN_MFG5] = { - .name = "mfg5", - .sta_mask = BIT(7), - .ctl_offs = 0x031c, - .pwr_sta_offs = 0x016c, - .pwr_sta2nd_offs = 0x0170, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - }, - [MT8192_POWER_DOMAIN_MFG6] = { - .name = "mfg6", - .sta_mask = BIT(8), - .ctl_offs = 0x0320, - .pwr_sta_offs = 0x016c, - .pwr_sta2nd_offs = 0x0170, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - }, - [MT8192_POWER_DOMAIN_DISP] = { - .name = "disp", - .sta_mask = BIT(20), - .ctl_offs = 0x0350, - .pwr_sta_offs = 0x016c, - .pwr_sta2nd_offs = 0x0170, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .bp_infracfg = { - BUS_PROT_WR_IGN(MT8192_TOP_AXI_PROT_EN_MM_DISP, - MT8192_TOP_AXI_PROT_EN_MM_SET, - MT8192_TOP_AXI_PROT_EN_MM_CLR, - MT8192_TOP_AXI_PROT_EN_MM_STA1), - BUS_PROT_WR_IGN(MT8192_TOP_AXI_PROT_EN_MM_2_DISP, - MT8192_TOP_AXI_PROT_EN_MM_2_SET, - MT8192_TOP_AXI_PROT_EN_MM_2_CLR, - MT8192_TOP_AXI_PROT_EN_MM_2_STA1), - BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_DISP, - MT8192_TOP_AXI_PROT_EN_SET, - MT8192_TOP_AXI_PROT_EN_CLR, - MT8192_TOP_AXI_PROT_EN_STA1), - BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_MM_DISP_2ND, - MT8192_TOP_AXI_PROT_EN_MM_SET, - MT8192_TOP_AXI_PROT_EN_MM_CLR, - MT8192_TOP_AXI_PROT_EN_MM_STA1), - BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_MM_2_DISP_2ND, - MT8192_TOP_AXI_PROT_EN_MM_2_SET, - MT8192_TOP_AXI_PROT_EN_MM_2_CLR, - MT8192_TOP_AXI_PROT_EN_MM_2_STA1), - }, - }, - [MT8192_POWER_DOMAIN_IPE] = { - .name = "ipe", - .sta_mask = BIT(14), - .ctl_offs = 0x0338, - .pwr_sta_offs = 0x016c, - .pwr_sta2nd_offs = 0x0170, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .bp_infracfg = { - BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_MM_IPE, - MT8192_TOP_AXI_PROT_EN_MM_SET, - MT8192_TOP_AXI_PROT_EN_MM_CLR, - MT8192_TOP_AXI_PROT_EN_MM_STA1), - BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_MM_IPE_2ND, - MT8192_TOP_AXI_PROT_EN_MM_SET, - MT8192_TOP_AXI_PROT_EN_MM_CLR, - MT8192_TOP_AXI_PROT_EN_MM_STA1), - }, - }, - [MT8192_POWER_DOMAIN_ISP] = { - .name = "isp", - .sta_mask = BIT(12), - .ctl_offs = 0x0330, - .pwr_sta_offs = 0x016c, - .pwr_sta2nd_offs = 0x0170, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .bp_infracfg = { - BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_MM_2_ISP, - MT8192_TOP_AXI_PROT_EN_MM_2_SET, - MT8192_TOP_AXI_PROT_EN_MM_2_CLR, - MT8192_TOP_AXI_PROT_EN_MM_2_STA1), - BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_MM_2_ISP_2ND, - MT8192_TOP_AXI_PROT_EN_MM_2_SET, - MT8192_TOP_AXI_PROT_EN_MM_2_CLR, - MT8192_TOP_AXI_PROT_EN_MM_2_STA1), - }, - }, - [MT8192_POWER_DOMAIN_ISP2] = { - .name = "isp2", - .sta_mask = BIT(13), - .ctl_offs = 0x0334, - .pwr_sta_offs = 0x016c, - .pwr_sta2nd_offs = 0x0170, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .bp_infracfg = { - BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_MM_ISP2, - MT8192_TOP_AXI_PROT_EN_MM_SET, - MT8192_TOP_AXI_PROT_EN_MM_CLR, - MT8192_TOP_AXI_PROT_EN_MM_STA1), - BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_MM_ISP2_2ND, - MT8192_TOP_AXI_PROT_EN_MM_SET, - MT8192_TOP_AXI_PROT_EN_MM_CLR, - MT8192_TOP_AXI_PROT_EN_MM_STA1), - }, - }, - [MT8192_POWER_DOMAIN_MDP] = { - .name = "mdp", - .sta_mask = BIT(19), - .ctl_offs = 0x034c, - .pwr_sta_offs = 0x016c, - .pwr_sta2nd_offs = 0x0170, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .bp_infracfg = { - BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_MM_2_MDP, - MT8192_TOP_AXI_PROT_EN_MM_2_SET, - MT8192_TOP_AXI_PROT_EN_MM_2_CLR, - MT8192_TOP_AXI_PROT_EN_MM_2_STA1), - BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_MM_2_MDP_2ND, - MT8192_TOP_AXI_PROT_EN_MM_2_SET, - MT8192_TOP_AXI_PROT_EN_MM_2_CLR, - MT8192_TOP_AXI_PROT_EN_MM_2_STA1), - }, - }, - [MT8192_POWER_DOMAIN_VENC] = { - .name = "venc", - .sta_mask = BIT(17), - .ctl_offs = 0x0344, - .pwr_sta_offs = 0x016c, - .pwr_sta2nd_offs = 0x0170, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .bp_infracfg = { - BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_MM_VENC, - MT8192_TOP_AXI_PROT_EN_MM_SET, - MT8192_TOP_AXI_PROT_EN_MM_CLR, - MT8192_TOP_AXI_PROT_EN_MM_STA1), - BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_MM_VENC_2ND, - MT8192_TOP_AXI_PROT_EN_MM_SET, - MT8192_TOP_AXI_PROT_EN_MM_CLR, - MT8192_TOP_AXI_PROT_EN_MM_STA1), - }, - }, - [MT8192_POWER_DOMAIN_VDEC] = { - .name = "vdec", - .sta_mask = BIT(15), - .ctl_offs = 0x033c, - .pwr_sta_offs = 0x016c, - .pwr_sta2nd_offs = 0x0170, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .bp_infracfg = { - BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_MM_VDEC, - MT8192_TOP_AXI_PROT_EN_MM_SET, - MT8192_TOP_AXI_PROT_EN_MM_CLR, - MT8192_TOP_AXI_PROT_EN_MM_STA1), - BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_MM_VDEC_2ND, - MT8192_TOP_AXI_PROT_EN_MM_SET, - MT8192_TOP_AXI_PROT_EN_MM_CLR, - MT8192_TOP_AXI_PROT_EN_MM_STA1), - }, - }, - [MT8192_POWER_DOMAIN_VDEC2] = { - .name = "vdec2", - .sta_mask = BIT(16), - .ctl_offs = 0x0340, - .pwr_sta_offs = 0x016c, - .pwr_sta2nd_offs = 0x0170, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - }, - [MT8192_POWER_DOMAIN_CAM] = { - .name = "cam", - .sta_mask = BIT(23), - .ctl_offs = 0x035c, - .pwr_sta_offs = 0x016c, - .pwr_sta2nd_offs = 0x0170, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .bp_infracfg = { - BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_2_CAM, - MT8192_TOP_AXI_PROT_EN_2_SET, - MT8192_TOP_AXI_PROT_EN_2_CLR, - MT8192_TOP_AXI_PROT_EN_2_STA1), - BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_MM_CAM, - MT8192_TOP_AXI_PROT_EN_MM_SET, - MT8192_TOP_AXI_PROT_EN_MM_CLR, - MT8192_TOP_AXI_PROT_EN_MM_STA1), - BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_1_CAM, - MT8192_TOP_AXI_PROT_EN_1_SET, - MT8192_TOP_AXI_PROT_EN_1_CLR, - MT8192_TOP_AXI_PROT_EN_1_STA1), - BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_MM_CAM_2ND, - MT8192_TOP_AXI_PROT_EN_MM_SET, - MT8192_TOP_AXI_PROT_EN_MM_CLR, - MT8192_TOP_AXI_PROT_EN_MM_STA1), - BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_VDNR_CAM, - MT8192_TOP_AXI_PROT_EN_VDNR_SET, - MT8192_TOP_AXI_PROT_EN_VDNR_CLR, - MT8192_TOP_AXI_PROT_EN_VDNR_STA1), - }, - }, - [MT8192_POWER_DOMAIN_CAM_RAWA] = { - .name = "cam_rawa", - .sta_mask = BIT(24), - .ctl_offs = 0x0360, - .pwr_sta_offs = 0x016c, - .pwr_sta2nd_offs = 0x0170, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - }, - [MT8192_POWER_DOMAIN_CAM_RAWB] = { - .name = "cam_rawb", - .sta_mask = BIT(25), - .ctl_offs = 0x0364, - .pwr_sta_offs = 0x016c, - .pwr_sta2nd_offs = 0x0170, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - }, - [MT8192_POWER_DOMAIN_CAM_RAWC] = { - .name = "cam_rawc", - .sta_mask = BIT(26), - .ctl_offs = 0x0368, - .pwr_sta_offs = 0x016c, - .pwr_sta2nd_offs = 0x0170, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - }, -}; - -static const struct scpsys_soc_data mt8192_scpsys_data = { - .domains_data = scpsys_domain_data_mt8192, - .num_domains = ARRAY_SIZE(scpsys_domain_data_mt8192), -}; - -#endif /* __SOC_MEDIATEK_MT8192_PM_DOMAINS_H */ diff --git a/drivers/genpd/mediatek/mt8195-pm-domains.h b/drivers/genpd/mediatek/mt8195-pm-domains.h deleted file mode 100644 index d7387ea1b9c9..000000000000 --- a/drivers/genpd/mediatek/mt8195-pm-domains.h +++ /dev/null @@ -1,613 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0-only */ -/* - * Copyright (c) 2021 MediaTek Inc. - * Author: Chun-Jie Chen - */ - -#ifndef __SOC_MEDIATEK_MT8195_PM_DOMAINS_H -#define __SOC_MEDIATEK_MT8195_PM_DOMAINS_H - -#include "mtk-pm-domains.h" -#include - -/* - * MT8195 power domain support - */ - -static const struct scpsys_domain_data scpsys_domain_data_mt8195[] = { - [MT8195_POWER_DOMAIN_PCIE_MAC_P0] = { - .name = "pcie_mac_p0", - .sta_mask = BIT(11), - .ctl_offs = 0x328, - .pwr_sta_offs = 0x174, - .pwr_sta2nd_offs = 0x178, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .bp_infracfg = { - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_VDNR_PCIE_MAC_P0, - MT8195_TOP_AXI_PROT_EN_VDNR_SET, - MT8195_TOP_AXI_PROT_EN_VDNR_CLR, - MT8195_TOP_AXI_PROT_EN_VDNR_STA1), - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_VDNR_1_PCIE_MAC_P0, - MT8195_TOP_AXI_PROT_EN_VDNR_1_SET, - MT8195_TOP_AXI_PROT_EN_VDNR_1_CLR, - MT8195_TOP_AXI_PROT_EN_VDNR_1_STA1), - }, - }, - [MT8195_POWER_DOMAIN_PCIE_MAC_P1] = { - .name = "pcie_mac_p1", - .sta_mask = BIT(12), - .ctl_offs = 0x32C, - .pwr_sta_offs = 0x174, - .pwr_sta2nd_offs = 0x178, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .bp_infracfg = { - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_VDNR_PCIE_MAC_P1, - MT8195_TOP_AXI_PROT_EN_VDNR_SET, - MT8195_TOP_AXI_PROT_EN_VDNR_CLR, - MT8195_TOP_AXI_PROT_EN_VDNR_STA1), - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_VDNR_1_PCIE_MAC_P1, - MT8195_TOP_AXI_PROT_EN_VDNR_1_SET, - MT8195_TOP_AXI_PROT_EN_VDNR_1_CLR, - MT8195_TOP_AXI_PROT_EN_VDNR_1_STA1), - }, - }, - [MT8195_POWER_DOMAIN_PCIE_PHY] = { - .name = "pcie_phy", - .sta_mask = BIT(13), - .ctl_offs = 0x330, - .pwr_sta_offs = 0x174, - .pwr_sta2nd_offs = 0x178, - .caps = MTK_SCPD_ACTIVE_WAKEUP, - }, - [MT8195_POWER_DOMAIN_SSUSB_PCIE_PHY] = { - .name = "ssusb_pcie_phy", - .sta_mask = BIT(14), - .ctl_offs = 0x334, - .pwr_sta_offs = 0x174, - .pwr_sta2nd_offs = 0x178, - .caps = MTK_SCPD_ACTIVE_WAKEUP | MTK_SCPD_ALWAYS_ON, - }, - [MT8195_POWER_DOMAIN_CSI_RX_TOP] = { - .name = "csi_rx_top", - .sta_mask = BIT(18), - .ctl_offs = 0x3C4, - .pwr_sta_offs = 0x174, - .pwr_sta2nd_offs = 0x178, - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8195_POWER_DOMAIN_ETHER] = { - .name = "ether", - .sta_mask = BIT(3), - .ctl_offs = 0x344, - .pwr_sta_offs = 0x16c, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .caps = MTK_SCPD_ACTIVE_WAKEUP, - }, - [MT8195_POWER_DOMAIN_ADSP] = { - .name = "adsp", - .sta_mask = BIT(10), - .ctl_offs = 0x360, - .pwr_sta_offs = 0x16c, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .bp_infracfg = { - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_2_ADSP, - MT8195_TOP_AXI_PROT_EN_2_SET, - MT8195_TOP_AXI_PROT_EN_2_CLR, - MT8195_TOP_AXI_PROT_EN_2_STA1), - }, - .caps = MTK_SCPD_SRAM_ISO | MTK_SCPD_ACTIVE_WAKEUP, - }, - [MT8195_POWER_DOMAIN_AUDIO] = { - .name = "audio", - .sta_mask = BIT(8), - .ctl_offs = 0x358, - .pwr_sta_offs = 0x16c, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .bp_infracfg = { - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_2_AUDIO, - MT8195_TOP_AXI_PROT_EN_2_SET, - MT8195_TOP_AXI_PROT_EN_2_CLR, - MT8195_TOP_AXI_PROT_EN_2_STA1), - }, - }, - [MT8195_POWER_DOMAIN_MFG0] = { - .name = "mfg0", - .sta_mask = BIT(1), - .ctl_offs = 0x300, - .pwr_sta_offs = 0x174, - .pwr_sta2nd_offs = 0x178, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .caps = MTK_SCPD_KEEP_DEFAULT_OFF | MTK_SCPD_DOMAIN_SUPPLY, - }, - [MT8195_POWER_DOMAIN_MFG1] = { - .name = "mfg1", - .sta_mask = BIT(2), - .ctl_offs = 0x304, - .pwr_sta_offs = 0x174, - .pwr_sta2nd_offs = 0x178, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .bp_infracfg = { - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MFG1, - MT8195_TOP_AXI_PROT_EN_SET, - MT8195_TOP_AXI_PROT_EN_CLR, - MT8195_TOP_AXI_PROT_EN_STA1), - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_2_MFG1, - MT8195_TOP_AXI_PROT_EN_2_SET, - MT8195_TOP_AXI_PROT_EN_2_CLR, - MT8195_TOP_AXI_PROT_EN_2_STA1), - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_1_MFG1, - MT8195_TOP_AXI_PROT_EN_1_SET, - MT8195_TOP_AXI_PROT_EN_1_CLR, - MT8195_TOP_AXI_PROT_EN_1_STA1), - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_2_MFG1_2ND, - MT8195_TOP_AXI_PROT_EN_2_SET, - MT8195_TOP_AXI_PROT_EN_2_CLR, - MT8195_TOP_AXI_PROT_EN_2_STA1), - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MFG1_2ND, - MT8195_TOP_AXI_PROT_EN_SET, - MT8195_TOP_AXI_PROT_EN_CLR, - MT8195_TOP_AXI_PROT_EN_STA1), - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_MFG1, - MT8195_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_SET, - MT8195_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_CLR, - MT8195_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_STA1), - }, - .caps = MTK_SCPD_KEEP_DEFAULT_OFF | MTK_SCPD_DOMAIN_SUPPLY, - }, - [MT8195_POWER_DOMAIN_MFG2] = { - .name = "mfg2", - .sta_mask = BIT(3), - .ctl_offs = 0x308, - .pwr_sta_offs = 0x174, - .pwr_sta2nd_offs = 0x178, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8195_POWER_DOMAIN_MFG3] = { - .name = "mfg3", - .sta_mask = BIT(4), - .ctl_offs = 0x30C, - .pwr_sta_offs = 0x174, - .pwr_sta2nd_offs = 0x178, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8195_POWER_DOMAIN_MFG4] = { - .name = "mfg4", - .sta_mask = BIT(5), - .ctl_offs = 0x310, - .pwr_sta_offs = 0x174, - .pwr_sta2nd_offs = 0x178, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8195_POWER_DOMAIN_MFG5] = { - .name = "mfg5", - .sta_mask = BIT(6), - .ctl_offs = 0x314, - .pwr_sta_offs = 0x174, - .pwr_sta2nd_offs = 0x178, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8195_POWER_DOMAIN_MFG6] = { - .name = "mfg6", - .sta_mask = BIT(7), - .ctl_offs = 0x318, - .pwr_sta_offs = 0x174, - .pwr_sta2nd_offs = 0x178, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8195_POWER_DOMAIN_VPPSYS0] = { - .name = "vppsys0", - .sta_mask = BIT(11), - .ctl_offs = 0x364, - .pwr_sta_offs = 0x16c, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .bp_infracfg = { - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_VPPSYS0, - MT8195_TOP_AXI_PROT_EN_SET, - MT8195_TOP_AXI_PROT_EN_CLR, - MT8195_TOP_AXI_PROT_EN_STA1), - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_2_VPPSYS0, - MT8195_TOP_AXI_PROT_EN_MM_2_SET, - MT8195_TOP_AXI_PROT_EN_MM_2_CLR, - MT8195_TOP_AXI_PROT_EN_MM_2_STA1), - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_VPPSYS0_2ND, - MT8195_TOP_AXI_PROT_EN_SET, - MT8195_TOP_AXI_PROT_EN_CLR, - MT8195_TOP_AXI_PROT_EN_STA1), - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_2_VPPSYS0_2ND, - MT8195_TOP_AXI_PROT_EN_MM_2_SET, - MT8195_TOP_AXI_PROT_EN_MM_2_CLR, - MT8195_TOP_AXI_PROT_EN_MM_2_STA1), - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_VPPSYS0, - MT8195_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_SET, - MT8195_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_CLR, - MT8195_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_STA1), - }, - }, - [MT8195_POWER_DOMAIN_VDOSYS0] = { - .name = "vdosys0", - .sta_mask = BIT(13), - .ctl_offs = 0x36C, - .pwr_sta_offs = 0x16c, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .bp_infracfg = { - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_VDOSYS0, - MT8195_TOP_AXI_PROT_EN_MM_SET, - MT8195_TOP_AXI_PROT_EN_MM_CLR, - MT8195_TOP_AXI_PROT_EN_MM_STA1), - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_VDOSYS0, - MT8195_TOP_AXI_PROT_EN_SET, - MT8195_TOP_AXI_PROT_EN_CLR, - MT8195_TOP_AXI_PROT_EN_STA1), - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_VDOSYS0, - MT8195_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_SET, - MT8195_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_CLR, - MT8195_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_STA1), - }, - }, - [MT8195_POWER_DOMAIN_VPPSYS1] = { - .name = "vppsys1", - .sta_mask = BIT(12), - .ctl_offs = 0x368, - .pwr_sta_offs = 0x16c, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .bp_infracfg = { - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_VPPSYS1, - MT8195_TOP_AXI_PROT_EN_MM_SET, - MT8195_TOP_AXI_PROT_EN_MM_CLR, - MT8195_TOP_AXI_PROT_EN_MM_STA1), - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_VPPSYS1_2ND, - MT8195_TOP_AXI_PROT_EN_MM_SET, - MT8195_TOP_AXI_PROT_EN_MM_CLR, - MT8195_TOP_AXI_PROT_EN_MM_STA1), - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_2_VPPSYS1, - MT8195_TOP_AXI_PROT_EN_MM_2_SET, - MT8195_TOP_AXI_PROT_EN_MM_2_CLR, - MT8195_TOP_AXI_PROT_EN_MM_2_STA1), - }, - }, - [MT8195_POWER_DOMAIN_VDOSYS1] = { - .name = "vdosys1", - .sta_mask = BIT(14), - .ctl_offs = 0x370, - .pwr_sta_offs = 0x16c, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .bp_infracfg = { - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_VDOSYS1, - MT8195_TOP_AXI_PROT_EN_MM_SET, - MT8195_TOP_AXI_PROT_EN_MM_CLR, - MT8195_TOP_AXI_PROT_EN_MM_STA1), - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_VDOSYS1_2ND, - MT8195_TOP_AXI_PROT_EN_MM_SET, - MT8195_TOP_AXI_PROT_EN_MM_CLR, - MT8195_TOP_AXI_PROT_EN_MM_STA1), - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_2_VDOSYS1, - MT8195_TOP_AXI_PROT_EN_MM_2_SET, - MT8195_TOP_AXI_PROT_EN_MM_2_CLR, - MT8195_TOP_AXI_PROT_EN_MM_2_STA1), - }, - }, - [MT8195_POWER_DOMAIN_DP_TX] = { - .name = "dp_tx", - .sta_mask = BIT(16), - .ctl_offs = 0x378, - .pwr_sta_offs = 0x16c, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .bp_infracfg = { - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_VDNR_1_DP_TX, - MT8195_TOP_AXI_PROT_EN_VDNR_1_SET, - MT8195_TOP_AXI_PROT_EN_VDNR_1_CLR, - MT8195_TOP_AXI_PROT_EN_VDNR_1_STA1), - }, - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8195_POWER_DOMAIN_EPD_TX] = { - .name = "epd_tx", - .sta_mask = BIT(17), - .ctl_offs = 0x37C, - .pwr_sta_offs = 0x16c, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .bp_infracfg = { - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_VDNR_1_EPD_TX, - MT8195_TOP_AXI_PROT_EN_VDNR_1_SET, - MT8195_TOP_AXI_PROT_EN_VDNR_1_CLR, - MT8195_TOP_AXI_PROT_EN_VDNR_1_STA1), - }, - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8195_POWER_DOMAIN_HDMI_TX] = { - .name = "hdmi_tx", - .sta_mask = BIT(18), - .ctl_offs = 0x380, - .pwr_sta_offs = 0x16c, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .caps = MTK_SCPD_KEEP_DEFAULT_OFF | MTK_SCPD_ACTIVE_WAKEUP, - }, - [MT8195_POWER_DOMAIN_WPESYS] = { - .name = "wpesys", - .sta_mask = BIT(15), - .ctl_offs = 0x374, - .pwr_sta_offs = 0x16c, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .bp_infracfg = { - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_2_WPESYS, - MT8195_TOP_AXI_PROT_EN_MM_2_SET, - MT8195_TOP_AXI_PROT_EN_MM_2_CLR, - MT8195_TOP_AXI_PROT_EN_MM_2_STA1), - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_WPESYS, - MT8195_TOP_AXI_PROT_EN_MM_SET, - MT8195_TOP_AXI_PROT_EN_MM_CLR, - MT8195_TOP_AXI_PROT_EN_MM_STA1), - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_2_WPESYS_2ND, - MT8195_TOP_AXI_PROT_EN_MM_2_SET, - MT8195_TOP_AXI_PROT_EN_MM_2_CLR, - MT8195_TOP_AXI_PROT_EN_MM_2_STA1), - }, - }, - [MT8195_POWER_DOMAIN_VDEC0] = { - .name = "vdec0", - .sta_mask = BIT(20), - .ctl_offs = 0x388, - .pwr_sta_offs = 0x16c, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .bp_infracfg = { - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_VDEC0, - MT8195_TOP_AXI_PROT_EN_MM_SET, - MT8195_TOP_AXI_PROT_EN_MM_CLR, - MT8195_TOP_AXI_PROT_EN_MM_STA1), - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_2_VDEC0, - MT8195_TOP_AXI_PROT_EN_MM_2_SET, - MT8195_TOP_AXI_PROT_EN_MM_2_CLR, - MT8195_TOP_AXI_PROT_EN_MM_2_STA1), - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_VDEC0_2ND, - MT8195_TOP_AXI_PROT_EN_MM_SET, - MT8195_TOP_AXI_PROT_EN_MM_CLR, - MT8195_TOP_AXI_PROT_EN_MM_STA1), - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_2_VDEC0_2ND, - MT8195_TOP_AXI_PROT_EN_MM_2_SET, - MT8195_TOP_AXI_PROT_EN_MM_2_CLR, - MT8195_TOP_AXI_PROT_EN_MM_2_STA1), - }, - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8195_POWER_DOMAIN_VDEC1] = { - .name = "vdec1", - .sta_mask = BIT(21), - .ctl_offs = 0x38C, - .pwr_sta_offs = 0x16c, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .bp_infracfg = { - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_VDEC1, - MT8195_TOP_AXI_PROT_EN_MM_SET, - MT8195_TOP_AXI_PROT_EN_MM_CLR, - MT8195_TOP_AXI_PROT_EN_MM_STA1), - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_VDEC1_2ND, - MT8195_TOP_AXI_PROT_EN_MM_SET, - MT8195_TOP_AXI_PROT_EN_MM_CLR, - MT8195_TOP_AXI_PROT_EN_MM_STA1), - }, - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8195_POWER_DOMAIN_VDEC2] = { - .name = "vdec2", - .sta_mask = BIT(22), - .ctl_offs = 0x390, - .pwr_sta_offs = 0x16c, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .bp_infracfg = { - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_2_VDEC2, - MT8195_TOP_AXI_PROT_EN_MM_2_SET, - MT8195_TOP_AXI_PROT_EN_MM_2_CLR, - MT8195_TOP_AXI_PROT_EN_MM_2_STA1), - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_2_VDEC2_2ND, - MT8195_TOP_AXI_PROT_EN_MM_2_SET, - MT8195_TOP_AXI_PROT_EN_MM_2_CLR, - MT8195_TOP_AXI_PROT_EN_MM_2_STA1), - }, - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8195_POWER_DOMAIN_VENC] = { - .name = "venc", - .sta_mask = BIT(23), - .ctl_offs = 0x394, - .pwr_sta_offs = 0x16c, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .bp_infracfg = { - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_VENC, - MT8195_TOP_AXI_PROT_EN_MM_SET, - MT8195_TOP_AXI_PROT_EN_MM_CLR, - MT8195_TOP_AXI_PROT_EN_MM_STA1), - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_VENC_2ND, - MT8195_TOP_AXI_PROT_EN_MM_SET, - MT8195_TOP_AXI_PROT_EN_MM_CLR, - MT8195_TOP_AXI_PROT_EN_MM_STA1), - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_2_VENC, - MT8195_TOP_AXI_PROT_EN_MM_2_SET, - MT8195_TOP_AXI_PROT_EN_MM_2_CLR, - MT8195_TOP_AXI_PROT_EN_MM_2_STA1), - }, - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8195_POWER_DOMAIN_VENC_CORE1] = { - .name = "venc_core1", - .sta_mask = BIT(24), - .ctl_offs = 0x398, - .pwr_sta_offs = 0x16c, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .bp_infracfg = { - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_VENC_CORE1, - MT8195_TOP_AXI_PROT_EN_MM_SET, - MT8195_TOP_AXI_PROT_EN_MM_CLR, - MT8195_TOP_AXI_PROT_EN_MM_STA1), - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_2_VENC_CORE1, - MT8195_TOP_AXI_PROT_EN_MM_2_SET, - MT8195_TOP_AXI_PROT_EN_MM_2_CLR, - MT8195_TOP_AXI_PROT_EN_MM_2_STA1), - }, - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8195_POWER_DOMAIN_IMG] = { - .name = "img", - .sta_mask = BIT(29), - .ctl_offs = 0x3AC, - .pwr_sta_offs = 0x16c, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .bp_infracfg = { - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_IMG, - MT8195_TOP_AXI_PROT_EN_MM_SET, - MT8195_TOP_AXI_PROT_EN_MM_CLR, - MT8195_TOP_AXI_PROT_EN_MM_STA1), - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_IMG_2ND, - MT8195_TOP_AXI_PROT_EN_MM_SET, - MT8195_TOP_AXI_PROT_EN_MM_CLR, - MT8195_TOP_AXI_PROT_EN_MM_STA1), - }, - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8195_POWER_DOMAIN_DIP] = { - .name = "dip", - .sta_mask = BIT(30), - .ctl_offs = 0x3B0, - .pwr_sta_offs = 0x16c, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8195_POWER_DOMAIN_IPE] = { - .name = "ipe", - .sta_mask = BIT(31), - .ctl_offs = 0x3B4, - .pwr_sta_offs = 0x16c, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .bp_infracfg = { - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_IPE, - MT8195_TOP_AXI_PROT_EN_MM_SET, - MT8195_TOP_AXI_PROT_EN_MM_CLR, - MT8195_TOP_AXI_PROT_EN_MM_STA1), - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_2_IPE, - MT8195_TOP_AXI_PROT_EN_MM_2_SET, - MT8195_TOP_AXI_PROT_EN_MM_2_CLR, - MT8195_TOP_AXI_PROT_EN_MM_2_STA1), - }, - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8195_POWER_DOMAIN_CAM] = { - .name = "cam", - .sta_mask = BIT(25), - .ctl_offs = 0x39C, - .pwr_sta_offs = 0x16c, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .bp_infracfg = { - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_2_CAM, - MT8195_TOP_AXI_PROT_EN_2_SET, - MT8195_TOP_AXI_PROT_EN_2_CLR, - MT8195_TOP_AXI_PROT_EN_2_STA1), - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_CAM, - MT8195_TOP_AXI_PROT_EN_MM_SET, - MT8195_TOP_AXI_PROT_EN_MM_CLR, - MT8195_TOP_AXI_PROT_EN_MM_STA1), - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_1_CAM, - MT8195_TOP_AXI_PROT_EN_1_SET, - MT8195_TOP_AXI_PROT_EN_1_CLR, - MT8195_TOP_AXI_PROT_EN_1_STA1), - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_CAM_2ND, - MT8195_TOP_AXI_PROT_EN_MM_SET, - MT8195_TOP_AXI_PROT_EN_MM_CLR, - MT8195_TOP_AXI_PROT_EN_MM_STA1), - BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_2_CAM, - MT8195_TOP_AXI_PROT_EN_MM_2_SET, - MT8195_TOP_AXI_PROT_EN_MM_2_CLR, - MT8195_TOP_AXI_PROT_EN_MM_2_STA1), - }, - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8195_POWER_DOMAIN_CAM_RAWA] = { - .name = "cam_rawa", - .sta_mask = BIT(26), - .ctl_offs = 0x3A0, - .pwr_sta_offs = 0x16c, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8195_POWER_DOMAIN_CAM_RAWB] = { - .name = "cam_rawb", - .sta_mask = BIT(27), - .ctl_offs = 0x3A4, - .pwr_sta_offs = 0x16c, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, - [MT8195_POWER_DOMAIN_CAM_MRAW] = { - .name = "cam_mraw", - .sta_mask = BIT(28), - .ctl_offs = 0x3A8, - .pwr_sta_offs = 0x16c, - .pwr_sta2nd_offs = 0x170, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .caps = MTK_SCPD_KEEP_DEFAULT_OFF, - }, -}; - -static const struct scpsys_soc_data mt8195_scpsys_data = { - .domains_data = scpsys_domain_data_mt8195, - .num_domains = ARRAY_SIZE(scpsys_domain_data_mt8195), -}; - -#endif /* __SOC_MEDIATEK_MT8195_PM_DOMAINS_H */ diff --git a/drivers/genpd/mediatek/mtk-pm-domains.c b/drivers/genpd/mediatek/mtk-pm-domains.c deleted file mode 100644 index ee962804b830..000000000000 --- a/drivers/genpd/mediatek/mtk-pm-domains.c +++ /dev/null @@ -1,688 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0-only -/* - * Copyright (c) 2020 Collabora Ltd. - */ -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -#include "mt6795-pm-domains.h" -#include "mt8167-pm-domains.h" -#include "mt8173-pm-domains.h" -#include "mt8183-pm-domains.h" -#include "mt8186-pm-domains.h" -#include "mt8188-pm-domains.h" -#include "mt8192-pm-domains.h" -#include "mt8195-pm-domains.h" - -#define MTK_POLL_DELAY_US 10 -#define MTK_POLL_TIMEOUT USEC_PER_SEC - -#define PWR_RST_B_BIT BIT(0) -#define PWR_ISO_BIT BIT(1) -#define PWR_ON_BIT BIT(2) -#define PWR_ON_2ND_BIT BIT(3) -#define PWR_CLK_DIS_BIT BIT(4) -#define PWR_SRAM_CLKISO_BIT BIT(5) -#define PWR_SRAM_ISOINT_B_BIT BIT(6) - -struct scpsys_domain { - struct generic_pm_domain genpd; - const struct scpsys_domain_data *data; - struct scpsys *scpsys; - int num_clks; - struct clk_bulk_data *clks; - int num_subsys_clks; - struct clk_bulk_data *subsys_clks; - struct regmap *infracfg; - struct regmap *smi; - struct regulator *supply; -}; - -struct scpsys { - struct device *dev; - struct regmap *base; - const struct scpsys_soc_data *soc_data; - struct genpd_onecell_data pd_data; - struct generic_pm_domain *domains[]; -}; - -#define to_scpsys_domain(gpd) container_of(gpd, struct scpsys_domain, genpd) - -static bool scpsys_domain_is_on(struct scpsys_domain *pd) -{ - struct scpsys *scpsys = pd->scpsys; - u32 status, status2; - - regmap_read(scpsys->base, pd->data->pwr_sta_offs, &status); - status &= pd->data->sta_mask; - - regmap_read(scpsys->base, pd->data->pwr_sta2nd_offs, &status2); - status2 &= pd->data->sta_mask; - - /* A domain is on when both status bits are set. */ - return status && status2; -} - -static int scpsys_sram_enable(struct scpsys_domain *pd) -{ - u32 pdn_ack = pd->data->sram_pdn_ack_bits; - struct scpsys *scpsys = pd->scpsys; - unsigned int tmp; - int ret; - - regmap_clear_bits(scpsys->base, pd->data->ctl_offs, pd->data->sram_pdn_bits); - - /* Either wait until SRAM_PDN_ACK all 1 or 0 */ - ret = regmap_read_poll_timeout(scpsys->base, pd->data->ctl_offs, tmp, - (tmp & pdn_ack) == 0, MTK_POLL_DELAY_US, MTK_POLL_TIMEOUT); - if (ret < 0) - return ret; - - if (MTK_SCPD_CAPS(pd, MTK_SCPD_SRAM_ISO)) { - regmap_set_bits(scpsys->base, pd->data->ctl_offs, PWR_SRAM_ISOINT_B_BIT); - udelay(1); - regmap_clear_bits(scpsys->base, pd->data->ctl_offs, PWR_SRAM_CLKISO_BIT); - } - - return 0; -} - -static int scpsys_sram_disable(struct scpsys_domain *pd) -{ - u32 pdn_ack = pd->data->sram_pdn_ack_bits; - struct scpsys *scpsys = pd->scpsys; - unsigned int tmp; - - if (MTK_SCPD_CAPS(pd, MTK_SCPD_SRAM_ISO)) { - regmap_set_bits(scpsys->base, pd->data->ctl_offs, PWR_SRAM_CLKISO_BIT); - udelay(1); - regmap_clear_bits(scpsys->base, pd->data->ctl_offs, PWR_SRAM_ISOINT_B_BIT); - } - - regmap_set_bits(scpsys->base, pd->data->ctl_offs, pd->data->sram_pdn_bits); - - /* Either wait until SRAM_PDN_ACK all 1 or 0 */ - return regmap_read_poll_timeout(scpsys->base, pd->data->ctl_offs, tmp, - (tmp & pdn_ack) == pdn_ack, MTK_POLL_DELAY_US, - MTK_POLL_TIMEOUT); -} - -static int _scpsys_bus_protect_enable(const struct scpsys_bus_prot_data *bpd, struct regmap *regmap) -{ - int i, ret; - - for (i = 0; i < SPM_MAX_BUS_PROT_DATA; i++) { - u32 val, mask = bpd[i].bus_prot_mask; - - if (!mask) - break; - - if (bpd[i].bus_prot_reg_update) - regmap_set_bits(regmap, bpd[i].bus_prot_set, mask); - else - regmap_write(regmap, bpd[i].bus_prot_set, mask); - - ret = regmap_read_poll_timeout(regmap, bpd[i].bus_prot_sta, - val, (val & mask) == mask, - MTK_POLL_DELAY_US, MTK_POLL_TIMEOUT); - if (ret) - return ret; - } - - return 0; -} - -static int scpsys_bus_protect_enable(struct scpsys_domain *pd) -{ - int ret; - - ret = _scpsys_bus_protect_enable(pd->data->bp_infracfg, pd->infracfg); - if (ret) - return ret; - - return _scpsys_bus_protect_enable(pd->data->bp_smi, pd->smi); -} - -static int _scpsys_bus_protect_disable(const struct scpsys_bus_prot_data *bpd, - struct regmap *regmap) -{ - int i, ret; - - for (i = SPM_MAX_BUS_PROT_DATA - 1; i >= 0; i--) { - u32 val, mask = bpd[i].bus_prot_mask; - - if (!mask) - continue; - - if (bpd[i].bus_prot_reg_update) - regmap_clear_bits(regmap, bpd[i].bus_prot_clr, mask); - else - regmap_write(regmap, bpd[i].bus_prot_clr, mask); - - if (bpd[i].ignore_clr_ack) - continue; - - ret = regmap_read_poll_timeout(regmap, bpd[i].bus_prot_sta, - val, !(val & mask), - MTK_POLL_DELAY_US, MTK_POLL_TIMEOUT); - if (ret) - return ret; - } - - return 0; -} - -static int scpsys_bus_protect_disable(struct scpsys_domain *pd) -{ - int ret; - - ret = _scpsys_bus_protect_disable(pd->data->bp_smi, pd->smi); - if (ret) - return ret; - - return _scpsys_bus_protect_disable(pd->data->bp_infracfg, pd->infracfg); -} - -static int scpsys_regulator_enable(struct regulator *supply) -{ - return supply ? regulator_enable(supply) : 0; -} - -static int scpsys_regulator_disable(struct regulator *supply) -{ - return supply ? regulator_disable(supply) : 0; -} - -static int scpsys_power_on(struct generic_pm_domain *genpd) -{ - struct scpsys_domain *pd = container_of(genpd, struct scpsys_domain, genpd); - struct scpsys *scpsys = pd->scpsys; - bool tmp; - int ret; - - ret = scpsys_regulator_enable(pd->supply); - if (ret) - return ret; - - ret = clk_bulk_prepare_enable(pd->num_clks, pd->clks); - if (ret) - goto err_reg; - - if (pd->data->ext_buck_iso_offs && MTK_SCPD_CAPS(pd, MTK_SCPD_EXT_BUCK_ISO)) - regmap_clear_bits(scpsys->base, pd->data->ext_buck_iso_offs, - pd->data->ext_buck_iso_mask); - - /* subsys power on */ - regmap_set_bits(scpsys->base, pd->data->ctl_offs, PWR_ON_BIT); - regmap_set_bits(scpsys->base, pd->data->ctl_offs, PWR_ON_2ND_BIT); - - /* wait until PWR_ACK = 1 */ - ret = readx_poll_timeout(scpsys_domain_is_on, pd, tmp, tmp, MTK_POLL_DELAY_US, - MTK_POLL_TIMEOUT); - if (ret < 0) - goto err_pwr_ack; - - regmap_clear_bits(scpsys->base, pd->data->ctl_offs, PWR_CLK_DIS_BIT); - regmap_clear_bits(scpsys->base, pd->data->ctl_offs, PWR_ISO_BIT); - regmap_set_bits(scpsys->base, pd->data->ctl_offs, PWR_RST_B_BIT); - - ret = clk_bulk_prepare_enable(pd->num_subsys_clks, pd->subsys_clks); - if (ret) - goto err_pwr_ack; - - ret = scpsys_sram_enable(pd); - if (ret < 0) - goto err_disable_subsys_clks; - - ret = scpsys_bus_protect_disable(pd); - if (ret < 0) - goto err_disable_sram; - - return 0; - -err_disable_sram: - scpsys_sram_disable(pd); -err_disable_subsys_clks: - clk_bulk_disable_unprepare(pd->num_subsys_clks, pd->subsys_clks); -err_pwr_ack: - clk_bulk_disable_unprepare(pd->num_clks, pd->clks); -err_reg: - scpsys_regulator_disable(pd->supply); - return ret; -} - -static int scpsys_power_off(struct generic_pm_domain *genpd) -{ - struct scpsys_domain *pd = container_of(genpd, struct scpsys_domain, genpd); - struct scpsys *scpsys = pd->scpsys; - bool tmp; - int ret; - - ret = scpsys_bus_protect_enable(pd); - if (ret < 0) - return ret; - - ret = scpsys_sram_disable(pd); - if (ret < 0) - return ret; - - if (pd->data->ext_buck_iso_offs && MTK_SCPD_CAPS(pd, MTK_SCPD_EXT_BUCK_ISO)) - regmap_set_bits(scpsys->base, pd->data->ext_buck_iso_offs, - pd->data->ext_buck_iso_mask); - - clk_bulk_disable_unprepare(pd->num_subsys_clks, pd->subsys_clks); - - /* subsys power off */ - regmap_set_bits(scpsys->base, pd->data->ctl_offs, PWR_ISO_BIT); - regmap_set_bits(scpsys->base, pd->data->ctl_offs, PWR_CLK_DIS_BIT); - regmap_clear_bits(scpsys->base, pd->data->ctl_offs, PWR_RST_B_BIT); - regmap_clear_bits(scpsys->base, pd->data->ctl_offs, PWR_ON_2ND_BIT); - regmap_clear_bits(scpsys->base, pd->data->ctl_offs, PWR_ON_BIT); - - /* wait until PWR_ACK = 0 */ - ret = readx_poll_timeout(scpsys_domain_is_on, pd, tmp, !tmp, MTK_POLL_DELAY_US, - MTK_POLL_TIMEOUT); - if (ret < 0) - return ret; - - clk_bulk_disable_unprepare(pd->num_clks, pd->clks); - - scpsys_regulator_disable(pd->supply); - - return 0; -} - -static struct -generic_pm_domain *scpsys_add_one_domain(struct scpsys *scpsys, struct device_node *node) -{ - const struct scpsys_domain_data *domain_data; - struct scpsys_domain *pd; - struct device_node *root_node = scpsys->dev->of_node; - struct device_node *smi_node; - struct property *prop; - const char *clk_name; - int i, ret, num_clks; - struct clk *clk; - int clk_ind = 0; - u32 id; - - ret = of_property_read_u32(node, "reg", &id); - if (ret) { - dev_err(scpsys->dev, "%pOF: failed to retrieve domain id from reg: %d\n", - node, ret); - return ERR_PTR(-EINVAL); - } - - if (id >= scpsys->soc_data->num_domains) { - dev_err(scpsys->dev, "%pOF: invalid domain id %d\n", node, id); - return ERR_PTR(-EINVAL); - } - - domain_data = &scpsys->soc_data->domains_data[id]; - if (domain_data->sta_mask == 0) { - dev_err(scpsys->dev, "%pOF: undefined domain id %d\n", node, id); - return ERR_PTR(-EINVAL); - } - - pd = devm_kzalloc(scpsys->dev, sizeof(*pd), GFP_KERNEL); - if (!pd) - return ERR_PTR(-ENOMEM); - - pd->data = domain_data; - pd->scpsys = scpsys; - - if (MTK_SCPD_CAPS(pd, MTK_SCPD_DOMAIN_SUPPLY)) { - /* - * Find regulator in current power domain node. - * devm_regulator_get() finds regulator in a node and its child - * node, so set of_node to current power domain node then change - * back to original node after regulator is found for current - * power domain node. - */ - scpsys->dev->of_node = node; - pd->supply = devm_regulator_get(scpsys->dev, "domain"); - scpsys->dev->of_node = root_node; - if (IS_ERR(pd->supply)) { - dev_err_probe(scpsys->dev, PTR_ERR(pd->supply), - "%pOF: failed to get power supply.\n", - node); - return ERR_CAST(pd->supply); - } - } - - pd->infracfg = syscon_regmap_lookup_by_phandle_optional(node, "mediatek,infracfg"); - if (IS_ERR(pd->infracfg)) - return ERR_CAST(pd->infracfg); - - smi_node = of_parse_phandle(node, "mediatek,smi", 0); - if (smi_node) { - pd->smi = device_node_to_regmap(smi_node); - of_node_put(smi_node); - if (IS_ERR(pd->smi)) - return ERR_CAST(pd->smi); - } - - num_clks = of_clk_get_parent_count(node); - if (num_clks > 0) { - /* Calculate number of subsys_clks */ - of_property_for_each_string(node, "clock-names", prop, clk_name) { - char *subsys; - - subsys = strchr(clk_name, '-'); - if (subsys) - pd->num_subsys_clks++; - else - pd->num_clks++; - } - - pd->clks = devm_kcalloc(scpsys->dev, pd->num_clks, sizeof(*pd->clks), GFP_KERNEL); - if (!pd->clks) - return ERR_PTR(-ENOMEM); - - pd->subsys_clks = devm_kcalloc(scpsys->dev, pd->num_subsys_clks, - sizeof(*pd->subsys_clks), GFP_KERNEL); - if (!pd->subsys_clks) - return ERR_PTR(-ENOMEM); - - } - - for (i = 0; i < pd->num_clks; i++) { - clk = of_clk_get(node, i); - if (IS_ERR(clk)) { - ret = PTR_ERR(clk); - dev_err_probe(scpsys->dev, ret, - "%pOF: failed to get clk at index %d\n", node, i); - goto err_put_clocks; - } - - pd->clks[clk_ind++].clk = clk; - } - - for (i = 0; i < pd->num_subsys_clks; i++) { - clk = of_clk_get(node, i + clk_ind); - if (IS_ERR(clk)) { - ret = PTR_ERR(clk); - dev_err_probe(scpsys->dev, ret, - "%pOF: failed to get clk at index %d\n", node, - i + clk_ind); - goto err_put_subsys_clocks; - } - - pd->subsys_clks[i].clk = clk; - } - - /* - * Initially turn on all domains to make the domains usable - * with !CONFIG_PM and to get the hardware in sync with the - * software. The unused domains will be switched off during - * late_init time. - */ - if (MTK_SCPD_CAPS(pd, MTK_SCPD_KEEP_DEFAULT_OFF)) { - if (scpsys_domain_is_on(pd)) - dev_warn(scpsys->dev, - "%pOF: A default off power domain has been ON\n", node); - } else { - ret = scpsys_power_on(&pd->genpd); - if (ret < 0) { - dev_err(scpsys->dev, "%pOF: failed to power on domain: %d\n", node, ret); - goto err_put_subsys_clocks; - } - - if (MTK_SCPD_CAPS(pd, MTK_SCPD_ALWAYS_ON)) - pd->genpd.flags |= GENPD_FLAG_ALWAYS_ON; - } - - if (scpsys->domains[id]) { - ret = -EINVAL; - dev_err(scpsys->dev, - "power domain with id %d already exists, check your device-tree\n", id); - goto err_put_subsys_clocks; - } - - if (!pd->data->name) - pd->genpd.name = node->name; - else - pd->genpd.name = pd->data->name; - - pd->genpd.power_off = scpsys_power_off; - pd->genpd.power_on = scpsys_power_on; - - if (MTK_SCPD_CAPS(pd, MTK_SCPD_ACTIVE_WAKEUP)) - pd->genpd.flags |= GENPD_FLAG_ACTIVE_WAKEUP; - - if (MTK_SCPD_CAPS(pd, MTK_SCPD_KEEP_DEFAULT_OFF)) - pm_genpd_init(&pd->genpd, NULL, true); - else - pm_genpd_init(&pd->genpd, NULL, false); - - scpsys->domains[id] = &pd->genpd; - - return scpsys->pd_data.domains[id]; - -err_put_subsys_clocks: - clk_bulk_put(pd->num_subsys_clks, pd->subsys_clks); -err_put_clocks: - clk_bulk_put(pd->num_clks, pd->clks); - return ERR_PTR(ret); -} - -static int scpsys_add_subdomain(struct scpsys *scpsys, struct device_node *parent) -{ - struct generic_pm_domain *child_pd, *parent_pd; - struct device_node *child; - int ret; - - for_each_child_of_node(parent, child) { - u32 id; - - ret = of_property_read_u32(parent, "reg", &id); - if (ret) { - dev_err(scpsys->dev, "%pOF: failed to get parent domain id\n", child); - goto err_put_node; - } - - if (!scpsys->pd_data.domains[id]) { - ret = -EINVAL; - dev_err(scpsys->dev, "power domain with id %d does not exist\n", id); - goto err_put_node; - } - - parent_pd = scpsys->pd_data.domains[id]; - - child_pd = scpsys_add_one_domain(scpsys, child); - if (IS_ERR(child_pd)) { - ret = PTR_ERR(child_pd); - dev_err_probe(scpsys->dev, ret, "%pOF: failed to get child domain id\n", - child); - goto err_put_node; - } - - ret = pm_genpd_add_subdomain(parent_pd, child_pd); - if (ret) { - dev_err(scpsys->dev, "failed to add %s subdomain to parent %s\n", - child_pd->name, parent_pd->name); - goto err_put_node; - } else { - dev_dbg(scpsys->dev, "%s add subdomain: %s\n", parent_pd->name, - child_pd->name); - } - - /* recursive call to add all subdomains */ - ret = scpsys_add_subdomain(scpsys, child); - if (ret) - goto err_put_node; - } - - return 0; - -err_put_node: - of_node_put(child); - return ret; -} - -static void scpsys_remove_one_domain(struct scpsys_domain *pd) -{ - int ret; - - if (scpsys_domain_is_on(pd)) - scpsys_power_off(&pd->genpd); - - /* - * We're in the error cleanup already, so we only complain, - * but won't emit another error on top of the original one. - */ - ret = pm_genpd_remove(&pd->genpd); - if (ret < 0) - dev_err(pd->scpsys->dev, - "failed to remove domain '%s' : %d - state may be inconsistent\n", - pd->genpd.name, ret); - - clk_bulk_put(pd->num_clks, pd->clks); - clk_bulk_put(pd->num_subsys_clks, pd->subsys_clks); -} - -static void scpsys_domain_cleanup(struct scpsys *scpsys) -{ - struct generic_pm_domain *genpd; - struct scpsys_domain *pd; - int i; - - for (i = scpsys->pd_data.num_domains - 1; i >= 0; i--) { - genpd = scpsys->pd_data.domains[i]; - if (genpd) { - pd = to_scpsys_domain(genpd); - scpsys_remove_one_domain(pd); - } - } -} - -static const struct of_device_id scpsys_of_match[] = { - { - .compatible = "mediatek,mt6795-power-controller", - .data = &mt6795_scpsys_data, - }, - { - .compatible = "mediatek,mt8167-power-controller", - .data = &mt8167_scpsys_data, - }, - { - .compatible = "mediatek,mt8173-power-controller", - .data = &mt8173_scpsys_data, - }, - { - .compatible = "mediatek,mt8183-power-controller", - .data = &mt8183_scpsys_data, - }, - { - .compatible = "mediatek,mt8186-power-controller", - .data = &mt8186_scpsys_data, - }, - { - .compatible = "mediatek,mt8188-power-controller", - .data = &mt8188_scpsys_data, - }, - { - .compatible = "mediatek,mt8192-power-controller", - .data = &mt8192_scpsys_data, - }, - { - .compatible = "mediatek,mt8195-power-controller", - .data = &mt8195_scpsys_data, - }, - { } -}; - -static int scpsys_probe(struct platform_device *pdev) -{ - struct device *dev = &pdev->dev; - struct device_node *np = dev->of_node; - const struct scpsys_soc_data *soc; - struct device_node *node; - struct device *parent; - struct scpsys *scpsys; - int ret; - - soc = of_device_get_match_data(&pdev->dev); - if (!soc) { - dev_err(&pdev->dev, "no power controller data\n"); - return -EINVAL; - } - - scpsys = devm_kzalloc(dev, struct_size(scpsys, domains, soc->num_domains), GFP_KERNEL); - if (!scpsys) - return -ENOMEM; - - scpsys->dev = dev; - scpsys->soc_data = soc; - - scpsys->pd_data.domains = scpsys->domains; - scpsys->pd_data.num_domains = soc->num_domains; - - parent = dev->parent; - if (!parent) { - dev_err(dev, "no parent for syscon devices\n"); - return -ENODEV; - } - - scpsys->base = syscon_node_to_regmap(parent->of_node); - if (IS_ERR(scpsys->base)) { - dev_err(dev, "no regmap available\n"); - return PTR_ERR(scpsys->base); - } - - ret = -ENODEV; - for_each_available_child_of_node(np, node) { - struct generic_pm_domain *domain; - - domain = scpsys_add_one_domain(scpsys, node); - if (IS_ERR(domain)) { - ret = PTR_ERR(domain); - of_node_put(node); - goto err_cleanup_domains; - } - - ret = scpsys_add_subdomain(scpsys, node); - if (ret) { - of_node_put(node); - goto err_cleanup_domains; - } - } - - if (ret) { - dev_dbg(dev, "no power domains present\n"); - return ret; - } - - ret = of_genpd_add_provider_onecell(np, &scpsys->pd_data); - if (ret) { - dev_err(dev, "failed to add provider: %d\n", ret); - goto err_cleanup_domains; - } - - return 0; - -err_cleanup_domains: - scpsys_domain_cleanup(scpsys); - return ret; -} - -static struct platform_driver scpsys_pm_domain_driver = { - .probe = scpsys_probe, - .driver = { - .name = "mtk-power-controller", - .suppress_bind_attrs = true, - .of_match_table = scpsys_of_match, - }, -}; -builtin_platform_driver(scpsys_pm_domain_driver); diff --git a/drivers/genpd/mediatek/mtk-pm-domains.h b/drivers/genpd/mediatek/mtk-pm-domains.h deleted file mode 100644 index 5ec53ee073c4..000000000000 --- a/drivers/genpd/mediatek/mtk-pm-domains.h +++ /dev/null @@ -1,111 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0-only */ - -#ifndef __SOC_MEDIATEK_MTK_PM_DOMAINS_H -#define __SOC_MEDIATEK_MTK_PM_DOMAINS_H - -#define MTK_SCPD_ACTIVE_WAKEUP BIT(0) -#define MTK_SCPD_FWAIT_SRAM BIT(1) -#define MTK_SCPD_SRAM_ISO BIT(2) -#define MTK_SCPD_KEEP_DEFAULT_OFF BIT(3) -#define MTK_SCPD_DOMAIN_SUPPLY BIT(4) -/* can't set MTK_SCPD_KEEP_DEFAULT_OFF at the same time */ -#define MTK_SCPD_ALWAYS_ON BIT(5) -#define MTK_SCPD_EXT_BUCK_ISO BIT(6) -#define MTK_SCPD_CAPS(_scpd, _x) ((_scpd)->data->caps & (_x)) - -#define SPM_VDE_PWR_CON 0x0210 -#define SPM_MFG_PWR_CON 0x0214 -#define SPM_VEN_PWR_CON 0x0230 -#define SPM_ISP_PWR_CON 0x0238 -#define SPM_DIS_PWR_CON 0x023c -#define SPM_CONN_PWR_CON 0x0280 -#define SPM_VEN2_PWR_CON 0x0298 -#define SPM_AUDIO_PWR_CON 0x029c -#define SPM_MFG_2D_PWR_CON 0x02c0 -#define SPM_MFG_ASYNC_PWR_CON 0x02c4 -#define SPM_USB_PWR_CON 0x02cc - -#define SPM_PWR_STATUS 0x060c -#define SPM_PWR_STATUS_2ND 0x0610 - -#define PWR_STATUS_CONN BIT(1) -#define PWR_STATUS_DISP BIT(3) -#define PWR_STATUS_MFG BIT(4) -#define PWR_STATUS_ISP BIT(5) -#define PWR_STATUS_VDEC BIT(7) -#define PWR_STATUS_VENC_LT BIT(20) -#define PWR_STATUS_VENC BIT(21) -#define PWR_STATUS_MFG_2D BIT(22) -#define PWR_STATUS_MFG_ASYNC BIT(23) -#define PWR_STATUS_AUDIO BIT(24) -#define PWR_STATUS_USB BIT(25) - -#define SPM_MAX_BUS_PROT_DATA 6 - -#define _BUS_PROT(_mask, _set, _clr, _sta, _update, _ignore) { \ - .bus_prot_mask = (_mask), \ - .bus_prot_set = _set, \ - .bus_prot_clr = _clr, \ - .bus_prot_sta = _sta, \ - .bus_prot_reg_update = _update, \ - .ignore_clr_ack = _ignore, \ - } - -#define BUS_PROT_WR(_mask, _set, _clr, _sta) \ - _BUS_PROT(_mask, _set, _clr, _sta, false, false) - -#define BUS_PROT_WR_IGN(_mask, _set, _clr, _sta) \ - _BUS_PROT(_mask, _set, _clr, _sta, false, true) - -#define BUS_PROT_UPDATE(_mask, _set, _clr, _sta) \ - _BUS_PROT(_mask, _set, _clr, _sta, true, false) - -#define BUS_PROT_UPDATE_TOPAXI(_mask) \ - BUS_PROT_UPDATE(_mask, \ - INFRA_TOPAXI_PROTECTEN, \ - INFRA_TOPAXI_PROTECTEN, \ - INFRA_TOPAXI_PROTECTSTA1) - -struct scpsys_bus_prot_data { - u32 bus_prot_mask; - u32 bus_prot_set; - u32 bus_prot_clr; - u32 bus_prot_sta; - bool bus_prot_reg_update; - bool ignore_clr_ack; -}; - -/** - * struct scpsys_domain_data - scp domain data for power on/off flow - * @name: The name of the power domain. - * @sta_mask: The mask for power on/off status bit. - * @ctl_offs: The offset for main power control register. - * @sram_pdn_bits: The mask for sram power control bits. - * @sram_pdn_ack_bits: The mask for sram power control acked bits. - * @ext_buck_iso_offs: The offset for external buck isolation - * @ext_buck_iso_mask: The mask for external buck isolation - * @caps: The flag for active wake-up action. - * @bp_infracfg: bus protection for infracfg subsystem - * @bp_smi: bus protection for smi subsystem - */ -struct scpsys_domain_data { - const char *name; - u32 sta_mask; - int ctl_offs; - u32 sram_pdn_bits; - u32 sram_pdn_ack_bits; - int ext_buck_iso_offs; - u32 ext_buck_iso_mask; - u8 caps; - const struct scpsys_bus_prot_data bp_infracfg[SPM_MAX_BUS_PROT_DATA]; - const struct scpsys_bus_prot_data bp_smi[SPM_MAX_BUS_PROT_DATA]; - int pwr_sta_offs; - int pwr_sta2nd_offs; -}; - -struct scpsys_soc_data { - const struct scpsys_domain_data *domains_data; - int num_domains; -}; - -#endif /* __SOC_MEDIATEK_MTK_PM_DOMAINS_H */ diff --git a/drivers/genpd/mediatek/mtk-scpsys.c b/drivers/genpd/mediatek/mtk-scpsys.c deleted file mode 100644 index b374d01fdac7..000000000000 --- a/drivers/genpd/mediatek/mtk-scpsys.c +++ /dev/null @@ -1,1147 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0-only -/* - * Copyright (c) 2015 Pengutronix, Sascha Hauer - */ -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -#include -#include -#include -#include -#include -#include - -#define MTK_POLL_DELAY_US 10 -#define MTK_POLL_TIMEOUT USEC_PER_SEC - -#define MTK_SCPD_ACTIVE_WAKEUP BIT(0) -#define MTK_SCPD_FWAIT_SRAM BIT(1) -#define MTK_SCPD_CAPS(_scpd, _x) ((_scpd)->data->caps & (_x)) - -#define SPM_VDE_PWR_CON 0x0210 -#define SPM_MFG_PWR_CON 0x0214 -#define SPM_VEN_PWR_CON 0x0230 -#define SPM_ISP_PWR_CON 0x0238 -#define SPM_DIS_PWR_CON 0x023c -#define SPM_CONN_PWR_CON 0x0280 -#define SPM_VEN2_PWR_CON 0x0298 -#define SPM_AUDIO_PWR_CON 0x029c /* MT8173, MT2712 */ -#define SPM_BDP_PWR_CON 0x029c /* MT2701 */ -#define SPM_ETH_PWR_CON 0x02a0 -#define SPM_HIF_PWR_CON 0x02a4 -#define SPM_IFR_MSC_PWR_CON 0x02a8 -#define SPM_MFG_2D_PWR_CON 0x02c0 -#define SPM_MFG_ASYNC_PWR_CON 0x02c4 -#define SPM_USB_PWR_CON 0x02cc -#define SPM_USB2_PWR_CON 0x02d4 /* MT2712 */ -#define SPM_ETHSYS_PWR_CON 0x02e0 /* MT7622 */ -#define SPM_HIF0_PWR_CON 0x02e4 /* MT7622 */ -#define SPM_HIF1_PWR_CON 0x02e8 /* MT7622 */ -#define SPM_WB_PWR_CON 0x02ec /* MT7622 */ - -#define SPM_PWR_STATUS 0x060c -#define SPM_PWR_STATUS_2ND 0x0610 - -#define PWR_RST_B_BIT BIT(0) -#define PWR_ISO_BIT BIT(1) -#define PWR_ON_BIT BIT(2) -#define PWR_ON_2ND_BIT BIT(3) -#define PWR_CLK_DIS_BIT BIT(4) - -#define PWR_STATUS_CONN BIT(1) -#define PWR_STATUS_DISP BIT(3) -#define PWR_STATUS_MFG BIT(4) -#define PWR_STATUS_ISP BIT(5) -#define PWR_STATUS_VDEC BIT(7) -#define PWR_STATUS_BDP BIT(14) -#define PWR_STATUS_ETH BIT(15) -#define PWR_STATUS_HIF BIT(16) -#define PWR_STATUS_IFR_MSC BIT(17) -#define PWR_STATUS_USB2 BIT(19) /* MT2712 */ -#define PWR_STATUS_VENC_LT BIT(20) -#define PWR_STATUS_VENC BIT(21) -#define PWR_STATUS_MFG_2D BIT(22) /* MT8173 */ -#define PWR_STATUS_MFG_ASYNC BIT(23) /* MT8173 */ -#define PWR_STATUS_AUDIO BIT(24) /* MT8173, MT2712 */ -#define PWR_STATUS_USB BIT(25) /* MT8173, MT2712 */ -#define PWR_STATUS_ETHSYS BIT(24) /* MT7622 */ -#define PWR_STATUS_HIF0 BIT(25) /* MT7622 */ -#define PWR_STATUS_HIF1 BIT(26) /* MT7622 */ -#define PWR_STATUS_WB BIT(27) /* MT7622 */ - -enum clk_id { - CLK_NONE, - CLK_MM, - CLK_MFG, - CLK_VENC, - CLK_VENC_LT, - CLK_ETHIF, - CLK_VDEC, - CLK_HIFSEL, - CLK_JPGDEC, - CLK_AUDIO, - CLK_MAX, -}; - -static const char * const clk_names[] = { - NULL, - "mm", - "mfg", - "venc", - "venc_lt", - "ethif", - "vdec", - "hif_sel", - "jpgdec", - "audio", - NULL, -}; - -#define MAX_CLKS 3 - -/** - * struct scp_domain_data - scp domain data for power on/off flow - * @name: The domain name. - * @sta_mask: The mask for power on/off status bit. - * @ctl_offs: The offset for main power control register. - * @sram_pdn_bits: The mask for sram power control bits. - * @sram_pdn_ack_bits: The mask for sram power control acked bits. - * @bus_prot_mask: The mask for single step bus protection. - * @clk_id: The basic clocks required by this power domain. - * @caps: The flag for active wake-up action. - */ -struct scp_domain_data { - const char *name; - u32 sta_mask; - int ctl_offs; - u32 sram_pdn_bits; - u32 sram_pdn_ack_bits; - u32 bus_prot_mask; - enum clk_id clk_id[MAX_CLKS]; - u8 caps; -}; - -struct scp; - -struct scp_domain { - struct generic_pm_domain genpd; - struct scp *scp; - struct clk *clk[MAX_CLKS]; - const struct scp_domain_data *data; - struct regulator *supply; -}; - -struct scp_ctrl_reg { - int pwr_sta_offs; - int pwr_sta2nd_offs; -}; - -struct scp { - struct scp_domain *domains; - struct genpd_onecell_data pd_data; - struct device *dev; - void __iomem *base; - struct regmap *infracfg; - struct scp_ctrl_reg ctrl_reg; - bool bus_prot_reg_update; -}; - -struct scp_subdomain { - int origin; - int subdomain; -}; - -struct scp_soc_data { - const struct scp_domain_data *domains; - int num_domains; - const struct scp_subdomain *subdomains; - int num_subdomains; - const struct scp_ctrl_reg regs; - bool bus_prot_reg_update; -}; - -static int scpsys_domain_is_on(struct scp_domain *scpd) -{ - struct scp *scp = scpd->scp; - - u32 status = readl(scp->base + scp->ctrl_reg.pwr_sta_offs) & - scpd->data->sta_mask; - u32 status2 = readl(scp->base + scp->ctrl_reg.pwr_sta2nd_offs) & - scpd->data->sta_mask; - - /* - * A domain is on when both status bits are set. If only one is set - * return an error. This happens while powering up a domain - */ - - if (status && status2) - return true; - if (!status && !status2) - return false; - - return -EINVAL; -} - -static int scpsys_regulator_enable(struct scp_domain *scpd) -{ - if (!scpd->supply) - return 0; - - return regulator_enable(scpd->supply); -} - -static int scpsys_regulator_disable(struct scp_domain *scpd) -{ - if (!scpd->supply) - return 0; - - return regulator_disable(scpd->supply); -} - -static void scpsys_clk_disable(struct clk *clk[], int max_num) -{ - int i; - - for (i = max_num - 1; i >= 0; i--) - clk_disable_unprepare(clk[i]); -} - -static int scpsys_clk_enable(struct clk *clk[], int max_num) -{ - int i, ret = 0; - - for (i = 0; i < max_num && clk[i]; i++) { - ret = clk_prepare_enable(clk[i]); - if (ret) { - scpsys_clk_disable(clk, i); - break; - } - } - - return ret; -} - -static int scpsys_sram_enable(struct scp_domain *scpd, void __iomem *ctl_addr) -{ - u32 val; - u32 pdn_ack = scpd->data->sram_pdn_ack_bits; - int tmp; - - val = readl(ctl_addr); - val &= ~scpd->data->sram_pdn_bits; - writel(val, ctl_addr); - - /* Either wait until SRAM_PDN_ACK all 0 or have a force wait */ - if (MTK_SCPD_CAPS(scpd, MTK_SCPD_FWAIT_SRAM)) { - /* - * Currently, MTK_SCPD_FWAIT_SRAM is necessary only for - * MT7622_POWER_DOMAIN_WB and thus just a trivial setup - * is applied here. - */ - usleep_range(12000, 12100); - } else { - /* Either wait until SRAM_PDN_ACK all 1 or 0 */ - int ret = readl_poll_timeout(ctl_addr, tmp, - (tmp & pdn_ack) == 0, - MTK_POLL_DELAY_US, MTK_POLL_TIMEOUT); - if (ret < 0) - return ret; - } - - return 0; -} - -static int scpsys_sram_disable(struct scp_domain *scpd, void __iomem *ctl_addr) -{ - u32 val; - u32 pdn_ack = scpd->data->sram_pdn_ack_bits; - int tmp; - - val = readl(ctl_addr); - val |= scpd->data->sram_pdn_bits; - writel(val, ctl_addr); - - /* Either wait until SRAM_PDN_ACK all 1 or 0 */ - return readl_poll_timeout(ctl_addr, tmp, - (tmp & pdn_ack) == pdn_ack, - MTK_POLL_DELAY_US, MTK_POLL_TIMEOUT); -} - -static int scpsys_bus_protect_enable(struct scp_domain *scpd) -{ - struct scp *scp = scpd->scp; - - if (!scpd->data->bus_prot_mask) - return 0; - - return mtk_infracfg_set_bus_protection(scp->infracfg, - scpd->data->bus_prot_mask, - scp->bus_prot_reg_update); -} - -static int scpsys_bus_protect_disable(struct scp_domain *scpd) -{ - struct scp *scp = scpd->scp; - - if (!scpd->data->bus_prot_mask) - return 0; - - return mtk_infracfg_clear_bus_protection(scp->infracfg, - scpd->data->bus_prot_mask, - scp->bus_prot_reg_update); -} - -static int scpsys_power_on(struct generic_pm_domain *genpd) -{ - struct scp_domain *scpd = container_of(genpd, struct scp_domain, genpd); - struct scp *scp = scpd->scp; - void __iomem *ctl_addr = scp->base + scpd->data->ctl_offs; - u32 val; - int ret, tmp; - - ret = scpsys_regulator_enable(scpd); - if (ret < 0) - return ret; - - ret = scpsys_clk_enable(scpd->clk, MAX_CLKS); - if (ret) - goto err_clk; - - /* subsys power on */ - val = readl(ctl_addr); - val |= PWR_ON_BIT; - writel(val, ctl_addr); - val |= PWR_ON_2ND_BIT; - writel(val, ctl_addr); - - /* wait until PWR_ACK = 1 */ - ret = readx_poll_timeout(scpsys_domain_is_on, scpd, tmp, tmp > 0, - MTK_POLL_DELAY_US, MTK_POLL_TIMEOUT); - if (ret < 0) - goto err_pwr_ack; - - val &= ~PWR_CLK_DIS_BIT; - writel(val, ctl_addr); - - val &= ~PWR_ISO_BIT; - writel(val, ctl_addr); - - val |= PWR_RST_B_BIT; - writel(val, ctl_addr); - - ret = scpsys_sram_enable(scpd, ctl_addr); - if (ret < 0) - goto err_pwr_ack; - - ret = scpsys_bus_protect_disable(scpd); - if (ret < 0) - goto err_pwr_ack; - - return 0; - -err_pwr_ack: - scpsys_clk_disable(scpd->clk, MAX_CLKS); -err_clk: - scpsys_regulator_disable(scpd); - - dev_err(scp->dev, "Failed to power on domain %s\n", genpd->name); - - return ret; -} - -static int scpsys_power_off(struct generic_pm_domain *genpd) -{ - struct scp_domain *scpd = container_of(genpd, struct scp_domain, genpd); - struct scp *scp = scpd->scp; - void __iomem *ctl_addr = scp->base + scpd->data->ctl_offs; - u32 val; - int ret, tmp; - - ret = scpsys_bus_protect_enable(scpd); - if (ret < 0) - goto out; - - ret = scpsys_sram_disable(scpd, ctl_addr); - if (ret < 0) - goto out; - - /* subsys power off */ - val = readl(ctl_addr); - val |= PWR_ISO_BIT; - writel(val, ctl_addr); - - val &= ~PWR_RST_B_BIT; - writel(val, ctl_addr); - - val |= PWR_CLK_DIS_BIT; - writel(val, ctl_addr); - - val &= ~PWR_ON_BIT; - writel(val, ctl_addr); - - val &= ~PWR_ON_2ND_BIT; - writel(val, ctl_addr); - - /* wait until PWR_ACK = 0 */ - ret = readx_poll_timeout(scpsys_domain_is_on, scpd, tmp, tmp == 0, - MTK_POLL_DELAY_US, MTK_POLL_TIMEOUT); - if (ret < 0) - goto out; - - scpsys_clk_disable(scpd->clk, MAX_CLKS); - - ret = scpsys_regulator_disable(scpd); - if (ret < 0) - goto out; - - return 0; - -out: - dev_err(scp->dev, "Failed to power off domain %s\n", genpd->name); - - return ret; -} - -static void init_clks(struct platform_device *pdev, struct clk **clk) -{ - int i; - - for (i = CLK_NONE + 1; i < CLK_MAX; i++) - clk[i] = devm_clk_get(&pdev->dev, clk_names[i]); -} - -static struct scp *init_scp(struct platform_device *pdev, - const struct scp_domain_data *scp_domain_data, int num, - const struct scp_ctrl_reg *scp_ctrl_reg, - bool bus_prot_reg_update) -{ - struct genpd_onecell_data *pd_data; - struct resource *res; - int i, j; - struct scp *scp; - struct clk *clk[CLK_MAX]; - - scp = devm_kzalloc(&pdev->dev, sizeof(*scp), GFP_KERNEL); - if (!scp) - return ERR_PTR(-ENOMEM); - - scp->ctrl_reg.pwr_sta_offs = scp_ctrl_reg->pwr_sta_offs; - scp->ctrl_reg.pwr_sta2nd_offs = scp_ctrl_reg->pwr_sta2nd_offs; - - scp->bus_prot_reg_update = bus_prot_reg_update; - - scp->dev = &pdev->dev; - - res = platform_get_resource(pdev, IORESOURCE_MEM, 0); - scp->base = devm_ioremap_resource(&pdev->dev, res); - if (IS_ERR(scp->base)) - return ERR_CAST(scp->base); - - scp->domains = devm_kcalloc(&pdev->dev, - num, sizeof(*scp->domains), GFP_KERNEL); - if (!scp->domains) - return ERR_PTR(-ENOMEM); - - pd_data = &scp->pd_data; - - pd_data->domains = devm_kcalloc(&pdev->dev, - num, sizeof(*pd_data->domains), GFP_KERNEL); - if (!pd_data->domains) - return ERR_PTR(-ENOMEM); - - scp->infracfg = syscon_regmap_lookup_by_phandle(pdev->dev.of_node, - "infracfg"); - if (IS_ERR(scp->infracfg)) { - dev_err(&pdev->dev, "Cannot find infracfg controller: %ld\n", - PTR_ERR(scp->infracfg)); - return ERR_CAST(scp->infracfg); - } - - for (i = 0; i < num; i++) { - struct scp_domain *scpd = &scp->domains[i]; - const struct scp_domain_data *data = &scp_domain_data[i]; - - scpd->supply = devm_regulator_get_optional(&pdev->dev, data->name); - if (IS_ERR(scpd->supply)) { - if (PTR_ERR(scpd->supply) == -ENODEV) - scpd->supply = NULL; - else - return ERR_CAST(scpd->supply); - } - } - - pd_data->num_domains = num; - - init_clks(pdev, clk); - - for (i = 0; i < num; i++) { - struct scp_domain *scpd = &scp->domains[i]; - struct generic_pm_domain *genpd = &scpd->genpd; - const struct scp_domain_data *data = &scp_domain_data[i]; - - pd_data->domains[i] = genpd; - scpd->scp = scp; - - scpd->data = data; - - for (j = 0; j < MAX_CLKS && data->clk_id[j]; j++) { - struct clk *c = clk[data->clk_id[j]]; - - if (IS_ERR(c)) { - dev_err(&pdev->dev, "%s: clk unavailable\n", - data->name); - return ERR_CAST(c); - } - - scpd->clk[j] = c; - } - - genpd->name = data->name; - genpd->power_off = scpsys_power_off; - genpd->power_on = scpsys_power_on; - if (MTK_SCPD_CAPS(scpd, MTK_SCPD_ACTIVE_WAKEUP)) - genpd->flags |= GENPD_FLAG_ACTIVE_WAKEUP; - } - - return scp; -} - -static void mtk_register_power_domains(struct platform_device *pdev, - struct scp *scp, int num) -{ - struct genpd_onecell_data *pd_data; - int i, ret; - - for (i = 0; i < num; i++) { - struct scp_domain *scpd = &scp->domains[i]; - struct generic_pm_domain *genpd = &scpd->genpd; - bool on; - - /* - * Initially turn on all domains to make the domains usable - * with !CONFIG_PM and to get the hardware in sync with the - * software. The unused domains will be switched off during - * late_init time. - */ - on = !WARN_ON(genpd->power_on(genpd) < 0); - - pm_genpd_init(genpd, NULL, !on); - } - - /* - * We are not allowed to fail here since there is no way to unregister - * a power domain. Once registered above we have to keep the domains - * valid. - */ - - pd_data = &scp->pd_data; - - ret = of_genpd_add_provider_onecell(pdev->dev.of_node, pd_data); - if (ret) - dev_err(&pdev->dev, "Failed to add OF provider: %d\n", ret); -} - -/* - * MT2701 power domain support - */ - -static const struct scp_domain_data scp_domain_data_mt2701[] = { - [MT2701_POWER_DOMAIN_CONN] = { - .name = "conn", - .sta_mask = PWR_STATUS_CONN, - .ctl_offs = SPM_CONN_PWR_CON, - .bus_prot_mask = MT2701_TOP_AXI_PROT_EN_CONN_M | - MT2701_TOP_AXI_PROT_EN_CONN_S, - .clk_id = {CLK_NONE}, - .caps = MTK_SCPD_ACTIVE_WAKEUP, - }, - [MT2701_POWER_DOMAIN_DISP] = { - .name = "disp", - .sta_mask = PWR_STATUS_DISP, - .ctl_offs = SPM_DIS_PWR_CON, - .sram_pdn_bits = GENMASK(11, 8), - .clk_id = {CLK_MM}, - .bus_prot_mask = MT2701_TOP_AXI_PROT_EN_MM_M0, - .caps = MTK_SCPD_ACTIVE_WAKEUP, - }, - [MT2701_POWER_DOMAIN_MFG] = { - .name = "mfg", - .sta_mask = PWR_STATUS_MFG, - .ctl_offs = SPM_MFG_PWR_CON, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .clk_id = {CLK_MFG}, - .caps = MTK_SCPD_ACTIVE_WAKEUP, - }, - [MT2701_POWER_DOMAIN_VDEC] = { - .name = "vdec", - .sta_mask = PWR_STATUS_VDEC, - .ctl_offs = SPM_VDE_PWR_CON, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .clk_id = {CLK_MM}, - .caps = MTK_SCPD_ACTIVE_WAKEUP, - }, - [MT2701_POWER_DOMAIN_ISP] = { - .name = "isp", - .sta_mask = PWR_STATUS_ISP, - .ctl_offs = SPM_ISP_PWR_CON, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = GENMASK(13, 12), - .clk_id = {CLK_MM}, - .caps = MTK_SCPD_ACTIVE_WAKEUP, - }, - [MT2701_POWER_DOMAIN_BDP] = { - .name = "bdp", - .sta_mask = PWR_STATUS_BDP, - .ctl_offs = SPM_BDP_PWR_CON, - .sram_pdn_bits = GENMASK(11, 8), - .clk_id = {CLK_NONE}, - .caps = MTK_SCPD_ACTIVE_WAKEUP, - }, - [MT2701_POWER_DOMAIN_ETH] = { - .name = "eth", - .sta_mask = PWR_STATUS_ETH, - .ctl_offs = SPM_ETH_PWR_CON, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = GENMASK(15, 12), - .clk_id = {CLK_ETHIF}, - .caps = MTK_SCPD_ACTIVE_WAKEUP, - }, - [MT2701_POWER_DOMAIN_HIF] = { - .name = "hif", - .sta_mask = PWR_STATUS_HIF, - .ctl_offs = SPM_HIF_PWR_CON, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = GENMASK(15, 12), - .clk_id = {CLK_ETHIF}, - .caps = MTK_SCPD_ACTIVE_WAKEUP, - }, - [MT2701_POWER_DOMAIN_IFR_MSC] = { - .name = "ifr_msc", - .sta_mask = PWR_STATUS_IFR_MSC, - .ctl_offs = SPM_IFR_MSC_PWR_CON, - .clk_id = {CLK_NONE}, - .caps = MTK_SCPD_ACTIVE_WAKEUP, - }, -}; - -/* - * MT2712 power domain support - */ -static const struct scp_domain_data scp_domain_data_mt2712[] = { - [MT2712_POWER_DOMAIN_MM] = { - .name = "mm", - .sta_mask = PWR_STATUS_DISP, - .ctl_offs = SPM_DIS_PWR_CON, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .clk_id = {CLK_MM}, - .caps = MTK_SCPD_ACTIVE_WAKEUP, - }, - [MT2712_POWER_DOMAIN_VDEC] = { - .name = "vdec", - .sta_mask = PWR_STATUS_VDEC, - .ctl_offs = SPM_VDE_PWR_CON, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .clk_id = {CLK_MM, CLK_VDEC}, - .caps = MTK_SCPD_ACTIVE_WAKEUP, - }, - [MT2712_POWER_DOMAIN_VENC] = { - .name = "venc", - .sta_mask = PWR_STATUS_VENC, - .ctl_offs = SPM_VEN_PWR_CON, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = GENMASK(15, 12), - .clk_id = {CLK_MM, CLK_VENC, CLK_JPGDEC}, - .caps = MTK_SCPD_ACTIVE_WAKEUP, - }, - [MT2712_POWER_DOMAIN_ISP] = { - .name = "isp", - .sta_mask = PWR_STATUS_ISP, - .ctl_offs = SPM_ISP_PWR_CON, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = GENMASK(13, 12), - .clk_id = {CLK_MM}, - .caps = MTK_SCPD_ACTIVE_WAKEUP, - }, - [MT2712_POWER_DOMAIN_AUDIO] = { - .name = "audio", - .sta_mask = PWR_STATUS_AUDIO, - .ctl_offs = SPM_AUDIO_PWR_CON, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = GENMASK(15, 12), - .clk_id = {CLK_AUDIO}, - .caps = MTK_SCPD_ACTIVE_WAKEUP, - }, - [MT2712_POWER_DOMAIN_USB] = { - .name = "usb", - .sta_mask = PWR_STATUS_USB, - .ctl_offs = SPM_USB_PWR_CON, - .sram_pdn_bits = GENMASK(10, 8), - .sram_pdn_ack_bits = GENMASK(14, 12), - .clk_id = {CLK_NONE}, - .caps = MTK_SCPD_ACTIVE_WAKEUP, - }, - [MT2712_POWER_DOMAIN_USB2] = { - .name = "usb2", - .sta_mask = PWR_STATUS_USB2, - .ctl_offs = SPM_USB2_PWR_CON, - .sram_pdn_bits = GENMASK(10, 8), - .sram_pdn_ack_bits = GENMASK(14, 12), - .clk_id = {CLK_NONE}, - .caps = MTK_SCPD_ACTIVE_WAKEUP, - }, - [MT2712_POWER_DOMAIN_MFG] = { - .name = "mfg", - .sta_mask = PWR_STATUS_MFG, - .ctl_offs = SPM_MFG_PWR_CON, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(16, 16), - .clk_id = {CLK_MFG}, - .bus_prot_mask = BIT(14) | BIT(21) | BIT(23), - .caps = MTK_SCPD_ACTIVE_WAKEUP, - }, - [MT2712_POWER_DOMAIN_MFG_SC1] = { - .name = "mfg_sc1", - .sta_mask = BIT(22), - .ctl_offs = 0x02c0, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(16, 16), - .clk_id = {CLK_NONE}, - .caps = MTK_SCPD_ACTIVE_WAKEUP, - }, - [MT2712_POWER_DOMAIN_MFG_SC2] = { - .name = "mfg_sc2", - .sta_mask = BIT(23), - .ctl_offs = 0x02c4, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(16, 16), - .clk_id = {CLK_NONE}, - .caps = MTK_SCPD_ACTIVE_WAKEUP, - }, - [MT2712_POWER_DOMAIN_MFG_SC3] = { - .name = "mfg_sc3", - .sta_mask = BIT(30), - .ctl_offs = 0x01f8, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(16, 16), - .clk_id = {CLK_NONE}, - .caps = MTK_SCPD_ACTIVE_WAKEUP, - }, -}; - -static const struct scp_subdomain scp_subdomain_mt2712[] = { - {MT2712_POWER_DOMAIN_MM, MT2712_POWER_DOMAIN_VDEC}, - {MT2712_POWER_DOMAIN_MM, MT2712_POWER_DOMAIN_VENC}, - {MT2712_POWER_DOMAIN_MM, MT2712_POWER_DOMAIN_ISP}, - {MT2712_POWER_DOMAIN_MFG, MT2712_POWER_DOMAIN_MFG_SC1}, - {MT2712_POWER_DOMAIN_MFG_SC1, MT2712_POWER_DOMAIN_MFG_SC2}, - {MT2712_POWER_DOMAIN_MFG_SC2, MT2712_POWER_DOMAIN_MFG_SC3}, -}; - -/* - * MT6797 power domain support - */ - -static const struct scp_domain_data scp_domain_data_mt6797[] = { - [MT6797_POWER_DOMAIN_VDEC] = { - .name = "vdec", - .sta_mask = BIT(7), - .ctl_offs = 0x300, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .clk_id = {CLK_VDEC}, - }, - [MT6797_POWER_DOMAIN_VENC] = { - .name = "venc", - .sta_mask = BIT(21), - .ctl_offs = 0x304, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = GENMASK(15, 12), - .clk_id = {CLK_NONE}, - }, - [MT6797_POWER_DOMAIN_ISP] = { - .name = "isp", - .sta_mask = BIT(5), - .ctl_offs = 0x308, - .sram_pdn_bits = GENMASK(9, 8), - .sram_pdn_ack_bits = GENMASK(13, 12), - .clk_id = {CLK_NONE}, - }, - [MT6797_POWER_DOMAIN_MM] = { - .name = "mm", - .sta_mask = BIT(3), - .ctl_offs = 0x30C, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .clk_id = {CLK_MM}, - .bus_prot_mask = (BIT(1) | BIT(2)), - }, - [MT6797_POWER_DOMAIN_AUDIO] = { - .name = "audio", - .sta_mask = BIT(24), - .ctl_offs = 0x314, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = GENMASK(15, 12), - .clk_id = {CLK_NONE}, - }, - [MT6797_POWER_DOMAIN_MFG_ASYNC] = { - .name = "mfg_async", - .sta_mask = BIT(13), - .ctl_offs = 0x334, - .sram_pdn_bits = 0, - .sram_pdn_ack_bits = 0, - .clk_id = {CLK_MFG}, - }, - [MT6797_POWER_DOMAIN_MJC] = { - .name = "mjc", - .sta_mask = BIT(20), - .ctl_offs = 0x310, - .sram_pdn_bits = GENMASK(8, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .clk_id = {CLK_NONE}, - }, -}; - -#define SPM_PWR_STATUS_MT6797 0x0180 -#define SPM_PWR_STATUS_2ND_MT6797 0x0184 - -static const struct scp_subdomain scp_subdomain_mt6797[] = { - {MT6797_POWER_DOMAIN_MM, MT6797_POWER_DOMAIN_VDEC}, - {MT6797_POWER_DOMAIN_MM, MT6797_POWER_DOMAIN_ISP}, - {MT6797_POWER_DOMAIN_MM, MT6797_POWER_DOMAIN_VENC}, - {MT6797_POWER_DOMAIN_MM, MT6797_POWER_DOMAIN_MJC}, -}; - -/* - * MT7622 power domain support - */ - -static const struct scp_domain_data scp_domain_data_mt7622[] = { - [MT7622_POWER_DOMAIN_ETHSYS] = { - .name = "ethsys", - .sta_mask = PWR_STATUS_ETHSYS, - .ctl_offs = SPM_ETHSYS_PWR_CON, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = GENMASK(15, 12), - .clk_id = {CLK_NONE}, - .bus_prot_mask = MT7622_TOP_AXI_PROT_EN_ETHSYS, - .caps = MTK_SCPD_ACTIVE_WAKEUP, - }, - [MT7622_POWER_DOMAIN_HIF0] = { - .name = "hif0", - .sta_mask = PWR_STATUS_HIF0, - .ctl_offs = SPM_HIF0_PWR_CON, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = GENMASK(15, 12), - .clk_id = {CLK_HIFSEL}, - .bus_prot_mask = MT7622_TOP_AXI_PROT_EN_HIF0, - .caps = MTK_SCPD_ACTIVE_WAKEUP, - }, - [MT7622_POWER_DOMAIN_HIF1] = { - .name = "hif1", - .sta_mask = PWR_STATUS_HIF1, - .ctl_offs = SPM_HIF1_PWR_CON, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = GENMASK(15, 12), - .clk_id = {CLK_HIFSEL}, - .bus_prot_mask = MT7622_TOP_AXI_PROT_EN_HIF1, - .caps = MTK_SCPD_ACTIVE_WAKEUP, - }, - [MT7622_POWER_DOMAIN_WB] = { - .name = "wb", - .sta_mask = PWR_STATUS_WB, - .ctl_offs = SPM_WB_PWR_CON, - .sram_pdn_bits = 0, - .sram_pdn_ack_bits = 0, - .clk_id = {CLK_NONE}, - .bus_prot_mask = MT7622_TOP_AXI_PROT_EN_WB, - .caps = MTK_SCPD_ACTIVE_WAKEUP | MTK_SCPD_FWAIT_SRAM, - }, -}; - -/* - * MT7623A power domain support - */ - -static const struct scp_domain_data scp_domain_data_mt7623a[] = { - [MT7623A_POWER_DOMAIN_CONN] = { - .name = "conn", - .sta_mask = PWR_STATUS_CONN, - .ctl_offs = SPM_CONN_PWR_CON, - .bus_prot_mask = MT2701_TOP_AXI_PROT_EN_CONN_M | - MT2701_TOP_AXI_PROT_EN_CONN_S, - .clk_id = {CLK_NONE}, - .caps = MTK_SCPD_ACTIVE_WAKEUP, - }, - [MT7623A_POWER_DOMAIN_ETH] = { - .name = "eth", - .sta_mask = PWR_STATUS_ETH, - .ctl_offs = SPM_ETH_PWR_CON, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = GENMASK(15, 12), - .clk_id = {CLK_ETHIF}, - .caps = MTK_SCPD_ACTIVE_WAKEUP, - }, - [MT7623A_POWER_DOMAIN_HIF] = { - .name = "hif", - .sta_mask = PWR_STATUS_HIF, - .ctl_offs = SPM_HIF_PWR_CON, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = GENMASK(15, 12), - .clk_id = {CLK_ETHIF}, - .caps = MTK_SCPD_ACTIVE_WAKEUP, - }, - [MT7623A_POWER_DOMAIN_IFR_MSC] = { - .name = "ifr_msc", - .sta_mask = PWR_STATUS_IFR_MSC, - .ctl_offs = SPM_IFR_MSC_PWR_CON, - .clk_id = {CLK_NONE}, - .caps = MTK_SCPD_ACTIVE_WAKEUP, - }, -}; - -/* - * MT8173 power domain support - */ - -static const struct scp_domain_data scp_domain_data_mt8173[] = { - [MT8173_POWER_DOMAIN_VDEC] = { - .name = "vdec", - .sta_mask = PWR_STATUS_VDEC, - .ctl_offs = SPM_VDE_PWR_CON, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .clk_id = {CLK_MM}, - }, - [MT8173_POWER_DOMAIN_VENC] = { - .name = "venc", - .sta_mask = PWR_STATUS_VENC, - .ctl_offs = SPM_VEN_PWR_CON, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = GENMASK(15, 12), - .clk_id = {CLK_MM, CLK_VENC}, - }, - [MT8173_POWER_DOMAIN_ISP] = { - .name = "isp", - .sta_mask = PWR_STATUS_ISP, - .ctl_offs = SPM_ISP_PWR_CON, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = GENMASK(13, 12), - .clk_id = {CLK_MM}, - }, - [MT8173_POWER_DOMAIN_MM] = { - .name = "mm", - .sta_mask = PWR_STATUS_DISP, - .ctl_offs = SPM_DIS_PWR_CON, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = GENMASK(12, 12), - .clk_id = {CLK_MM}, - .bus_prot_mask = MT8173_TOP_AXI_PROT_EN_MM_M0 | - MT8173_TOP_AXI_PROT_EN_MM_M1, - }, - [MT8173_POWER_DOMAIN_VENC_LT] = { - .name = "venc_lt", - .sta_mask = PWR_STATUS_VENC_LT, - .ctl_offs = SPM_VEN2_PWR_CON, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = GENMASK(15, 12), - .clk_id = {CLK_MM, CLK_VENC_LT}, - }, - [MT8173_POWER_DOMAIN_AUDIO] = { - .name = "audio", - .sta_mask = PWR_STATUS_AUDIO, - .ctl_offs = SPM_AUDIO_PWR_CON, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = GENMASK(15, 12), - .clk_id = {CLK_NONE}, - }, - [MT8173_POWER_DOMAIN_USB] = { - .name = "usb", - .sta_mask = PWR_STATUS_USB, - .ctl_offs = SPM_USB_PWR_CON, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = GENMASK(15, 12), - .clk_id = {CLK_NONE}, - .caps = MTK_SCPD_ACTIVE_WAKEUP, - }, - [MT8173_POWER_DOMAIN_MFG_ASYNC] = { - .name = "mfg_async", - .sta_mask = PWR_STATUS_MFG_ASYNC, - .ctl_offs = SPM_MFG_ASYNC_PWR_CON, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = 0, - .clk_id = {CLK_MFG}, - }, - [MT8173_POWER_DOMAIN_MFG_2D] = { - .name = "mfg_2d", - .sta_mask = PWR_STATUS_MFG_2D, - .ctl_offs = SPM_MFG_2D_PWR_CON, - .sram_pdn_bits = GENMASK(11, 8), - .sram_pdn_ack_bits = GENMASK(13, 12), - .clk_id = {CLK_NONE}, - }, - [MT8173_POWER_DOMAIN_MFG] = { - .name = "mfg", - .sta_mask = PWR_STATUS_MFG, - .ctl_offs = SPM_MFG_PWR_CON, - .sram_pdn_bits = GENMASK(13, 8), - .sram_pdn_ack_bits = GENMASK(21, 16), - .clk_id = {CLK_NONE}, - .bus_prot_mask = MT8173_TOP_AXI_PROT_EN_MFG_S | - MT8173_TOP_AXI_PROT_EN_MFG_M0 | - MT8173_TOP_AXI_PROT_EN_MFG_M1 | - MT8173_TOP_AXI_PROT_EN_MFG_SNOOP_OUT, - }, -}; - -static const struct scp_subdomain scp_subdomain_mt8173[] = { - {MT8173_POWER_DOMAIN_MFG_ASYNC, MT8173_POWER_DOMAIN_MFG_2D}, - {MT8173_POWER_DOMAIN_MFG_2D, MT8173_POWER_DOMAIN_MFG}, -}; - -static const struct scp_soc_data mt2701_data = { - .domains = scp_domain_data_mt2701, - .num_domains = ARRAY_SIZE(scp_domain_data_mt2701), - .regs = { - .pwr_sta_offs = SPM_PWR_STATUS, - .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND - }, - .bus_prot_reg_update = true, -}; - -static const struct scp_soc_data mt2712_data = { - .domains = scp_domain_data_mt2712, - .num_domains = ARRAY_SIZE(scp_domain_data_mt2712), - .subdomains = scp_subdomain_mt2712, - .num_subdomains = ARRAY_SIZE(scp_subdomain_mt2712), - .regs = { - .pwr_sta_offs = SPM_PWR_STATUS, - .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND - }, - .bus_prot_reg_update = false, -}; - -static const struct scp_soc_data mt6797_data = { - .domains = scp_domain_data_mt6797, - .num_domains = ARRAY_SIZE(scp_domain_data_mt6797), - .subdomains = scp_subdomain_mt6797, - .num_subdomains = ARRAY_SIZE(scp_subdomain_mt6797), - .regs = { - .pwr_sta_offs = SPM_PWR_STATUS_MT6797, - .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND_MT6797 - }, - .bus_prot_reg_update = true, -}; - -static const struct scp_soc_data mt7622_data = { - .domains = scp_domain_data_mt7622, - .num_domains = ARRAY_SIZE(scp_domain_data_mt7622), - .regs = { - .pwr_sta_offs = SPM_PWR_STATUS, - .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND - }, - .bus_prot_reg_update = true, -}; - -static const struct scp_soc_data mt7623a_data = { - .domains = scp_domain_data_mt7623a, - .num_domains = ARRAY_SIZE(scp_domain_data_mt7623a), - .regs = { - .pwr_sta_offs = SPM_PWR_STATUS, - .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND - }, - .bus_prot_reg_update = true, -}; - -static const struct scp_soc_data mt8173_data = { - .domains = scp_domain_data_mt8173, - .num_domains = ARRAY_SIZE(scp_domain_data_mt8173), - .subdomains = scp_subdomain_mt8173, - .num_subdomains = ARRAY_SIZE(scp_subdomain_mt8173), - .regs = { - .pwr_sta_offs = SPM_PWR_STATUS, - .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND - }, - .bus_prot_reg_update = true, -}; - -/* - * scpsys driver init - */ - -static const struct of_device_id of_scpsys_match_tbl[] = { - { - .compatible = "mediatek,mt2701-scpsys", - .data = &mt2701_data, - }, { - .compatible = "mediatek,mt2712-scpsys", - .data = &mt2712_data, - }, { - .compatible = "mediatek,mt6797-scpsys", - .data = &mt6797_data, - }, { - .compatible = "mediatek,mt7622-scpsys", - .data = &mt7622_data, - }, { - .compatible = "mediatek,mt7623a-scpsys", - .data = &mt7623a_data, - }, { - .compatible = "mediatek,mt8173-scpsys", - .data = &mt8173_data, - }, { - /* sentinel */ - } -}; - -static int scpsys_probe(struct platform_device *pdev) -{ - const struct scp_subdomain *sd; - const struct scp_soc_data *soc; - struct scp *scp; - struct genpd_onecell_data *pd_data; - int i, ret; - - soc = of_device_get_match_data(&pdev->dev); - - scp = init_scp(pdev, soc->domains, soc->num_domains, &soc->regs, - soc->bus_prot_reg_update); - if (IS_ERR(scp)) - return PTR_ERR(scp); - - mtk_register_power_domains(pdev, scp, soc->num_domains); - - pd_data = &scp->pd_data; - - for (i = 0, sd = soc->subdomains; i < soc->num_subdomains; i++, sd++) { - ret = pm_genpd_add_subdomain(pd_data->domains[sd->origin], - pd_data->domains[sd->subdomain]); - if (ret && IS_ENABLED(CONFIG_PM)) - dev_err(&pdev->dev, "Failed to add subdomain: %d\n", - ret); - } - - return 0; -} - -static struct platform_driver scpsys_drv = { - .probe = scpsys_probe, - .driver = { - .name = "mtk-scpsys", - .suppress_bind_attrs = true, - .owner = THIS_MODULE, - .of_match_table = of_scpsys_match_tbl, - }, -}; -builtin_platform_driver(scpsys_drv); diff --git a/drivers/genpd/qcom/Makefile b/drivers/genpd/qcom/Makefile deleted file mode 100644 index 403dfc5af095..000000000000 --- a/drivers/genpd/qcom/Makefile +++ /dev/null @@ -1,4 +0,0 @@ -# SPDX-License-Identifier: GPL-2.0 -obj-$(CONFIG_QCOM_CPR) += cpr.o -obj-$(CONFIG_QCOM_RPMPD) += rpmpd.o -obj-$(CONFIG_QCOM_RPMHPD) += rpmhpd.o diff --git a/drivers/genpd/qcom/cpr.c b/drivers/genpd/qcom/cpr.c deleted file mode 100644 index 94a3f0977212..000000000000 --- a/drivers/genpd/qcom/cpr.c +++ /dev/null @@ -1,1756 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * Copyright (c) 2013-2015, The Linux Foundation. All rights reserved. - * Copyright (c) 2019, Linaro Limited - */ - -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -/* Register Offsets for RB-CPR and Bit Definitions */ - -/* RBCPR Version Register */ -#define REG_RBCPR_VERSION 0 -#define RBCPR_VER_2 0x02 -#define FLAGS_IGNORE_1ST_IRQ_STATUS BIT(0) - -/* RBCPR Gate Count and Target Registers */ -#define REG_RBCPR_GCNT_TARGET(n) (0x60 + 4 * (n)) - -#define RBCPR_GCNT_TARGET_TARGET_SHIFT 0 -#define RBCPR_GCNT_TARGET_TARGET_MASK GENMASK(11, 0) -#define RBCPR_GCNT_TARGET_GCNT_SHIFT 12 -#define RBCPR_GCNT_TARGET_GCNT_MASK GENMASK(9, 0) - -/* RBCPR Timer Control */ -#define REG_RBCPR_TIMER_INTERVAL 0x44 -#define REG_RBIF_TIMER_ADJUST 0x4c - -#define RBIF_TIMER_ADJ_CONS_UP_MASK GENMASK(3, 0) -#define RBIF_TIMER_ADJ_CONS_UP_SHIFT 0 -#define RBIF_TIMER_ADJ_CONS_DOWN_MASK GENMASK(3, 0) -#define RBIF_TIMER_ADJ_CONS_DOWN_SHIFT 4 -#define RBIF_TIMER_ADJ_CLAMP_INT_MASK GENMASK(7, 0) -#define RBIF_TIMER_ADJ_CLAMP_INT_SHIFT 8 - -/* RBCPR Config Register */ -#define REG_RBIF_LIMIT 0x48 -#define RBIF_LIMIT_CEILING_MASK GENMASK(5, 0) -#define RBIF_LIMIT_CEILING_SHIFT 6 -#define RBIF_LIMIT_FLOOR_BITS 6 -#define RBIF_LIMIT_FLOOR_MASK GENMASK(5, 0) - -#define RBIF_LIMIT_CEILING_DEFAULT RBIF_LIMIT_CEILING_MASK -#define RBIF_LIMIT_FLOOR_DEFAULT 0 - -#define REG_RBIF_SW_VLEVEL 0x94 -#define RBIF_SW_VLEVEL_DEFAULT 0x20 - -#define REG_RBCPR_STEP_QUOT 0x80 -#define RBCPR_STEP_QUOT_STEPQUOT_MASK GENMASK(7, 0) -#define RBCPR_STEP_QUOT_IDLE_CLK_MASK GENMASK(3, 0) -#define RBCPR_STEP_QUOT_IDLE_CLK_SHIFT 8 - -/* RBCPR Control Register */ -#define REG_RBCPR_CTL 0x90 - -#define RBCPR_CTL_LOOP_EN BIT(0) -#define RBCPR_CTL_TIMER_EN BIT(3) -#define RBCPR_CTL_SW_AUTO_CONT_ACK_EN BIT(5) -#define RBCPR_CTL_SW_AUTO_CONT_NACK_DN_EN BIT(6) -#define RBCPR_CTL_COUNT_MODE BIT(10) -#define RBCPR_CTL_UP_THRESHOLD_MASK GENMASK(3, 0) -#define RBCPR_CTL_UP_THRESHOLD_SHIFT 24 -#define RBCPR_CTL_DN_THRESHOLD_MASK GENMASK(3, 0) -#define RBCPR_CTL_DN_THRESHOLD_SHIFT 28 - -/* RBCPR Ack/Nack Response */ -#define REG_RBIF_CONT_ACK_CMD 0x98 -#define REG_RBIF_CONT_NACK_CMD 0x9c - -/* RBCPR Result status Register */ -#define REG_RBCPR_RESULT_0 0xa0 - -#define RBCPR_RESULT0_BUSY_SHIFT 19 -#define RBCPR_RESULT0_BUSY_MASK BIT(RBCPR_RESULT0_BUSY_SHIFT) -#define RBCPR_RESULT0_ERROR_LT0_SHIFT 18 -#define RBCPR_RESULT0_ERROR_SHIFT 6 -#define RBCPR_RESULT0_ERROR_MASK GENMASK(11, 0) -#define RBCPR_RESULT0_ERROR_STEPS_SHIFT 2 -#define RBCPR_RESULT0_ERROR_STEPS_MASK GENMASK(3, 0) -#define RBCPR_RESULT0_STEP_UP_SHIFT 1 - -/* RBCPR Interrupt Control Register */ -#define REG_RBIF_IRQ_EN(n) (0x100 + 4 * (n)) -#define REG_RBIF_IRQ_CLEAR 0x110 -#define REG_RBIF_IRQ_STATUS 0x114 - -#define CPR_INT_DONE BIT(0) -#define CPR_INT_MIN BIT(1) -#define CPR_INT_DOWN BIT(2) -#define CPR_INT_MID BIT(3) -#define CPR_INT_UP BIT(4) -#define CPR_INT_MAX BIT(5) -#define CPR_INT_CLAMP BIT(6) -#define CPR_INT_ALL (CPR_INT_DONE | CPR_INT_MIN | CPR_INT_DOWN | \ - CPR_INT_MID | CPR_INT_UP | CPR_INT_MAX | CPR_INT_CLAMP) -#define CPR_INT_DEFAULT (CPR_INT_UP | CPR_INT_DOWN) - -#define CPR_NUM_RING_OSC 8 - -/* CPR eFuse parameters */ -#define CPR_FUSE_TARGET_QUOT_BITS_MASK GENMASK(11, 0) - -#define CPR_FUSE_MIN_QUOT_DIFF 50 - -#define FUSE_REVISION_UNKNOWN (-1) - -enum voltage_change_dir { - NO_CHANGE, - DOWN, - UP, -}; - -struct cpr_fuse { - char *ring_osc; - char *init_voltage; - char *quotient; - char *quotient_offset; -}; - -struct fuse_corner_data { - int ref_uV; - int max_uV; - int min_uV; - int max_volt_scale; - int max_quot_scale; - /* fuse quot */ - int quot_offset; - int quot_scale; - int quot_adjust; - /* fuse quot_offset */ - int quot_offset_scale; - int quot_offset_adjust; -}; - -struct cpr_fuses { - int init_voltage_step; - int init_voltage_width; - struct fuse_corner_data *fuse_corner_data; -}; - -struct corner_data { - unsigned int fuse_corner; - unsigned long freq; -}; - -struct cpr_desc { - unsigned int num_fuse_corners; - int min_diff_quot; - int *step_quot; - - unsigned int timer_delay_us; - unsigned int timer_cons_up; - unsigned int timer_cons_down; - unsigned int up_threshold; - unsigned int down_threshold; - unsigned int idle_clocks; - unsigned int gcnt_us; - unsigned int vdd_apc_step_up_limit; - unsigned int vdd_apc_step_down_limit; - unsigned int clamp_timer_interval; - - struct cpr_fuses cpr_fuses; - bool reduce_to_fuse_uV; - bool reduce_to_corner_uV; -}; - -struct acc_desc { - unsigned int enable_reg; - u32 enable_mask; - - struct reg_sequence *config; - struct reg_sequence *settings; - int num_regs_per_fuse; -}; - -struct cpr_acc_desc { - const struct cpr_desc *cpr_desc; - const struct acc_desc *acc_desc; -}; - -struct fuse_corner { - int min_uV; - int max_uV; - int uV; - int quot; - int step_quot; - const struct reg_sequence *accs; - int num_accs; - unsigned long max_freq; - u8 ring_osc_idx; -}; - -struct corner { - int min_uV; - int max_uV; - int uV; - int last_uV; - int quot_adjust; - u32 save_ctl; - u32 save_irq; - unsigned long freq; - struct fuse_corner *fuse_corner; -}; - -struct cpr_drv { - unsigned int num_corners; - unsigned int ref_clk_khz; - - struct generic_pm_domain pd; - struct device *dev; - struct device *attached_cpu_dev; - struct mutex lock; - void __iomem *base; - struct corner *corner; - struct regulator *vdd_apc; - struct clk *cpu_clk; - struct regmap *tcsr; - bool loop_disabled; - u32 gcnt; - unsigned long flags; - - struct fuse_corner *fuse_corners; - struct corner *corners; - - const struct cpr_desc *desc; - const struct acc_desc *acc_desc; - const struct cpr_fuse *cpr_fuses; - - struct dentry *debugfs; -}; - -static bool cpr_is_allowed(struct cpr_drv *drv) -{ - return !drv->loop_disabled; -} - -static void cpr_write(struct cpr_drv *drv, u32 offset, u32 value) -{ - writel_relaxed(value, drv->base + offset); -} - -static u32 cpr_read(struct cpr_drv *drv, u32 offset) -{ - return readl_relaxed(drv->base + offset); -} - -static void -cpr_masked_write(struct cpr_drv *drv, u32 offset, u32 mask, u32 value) -{ - u32 val; - - val = readl_relaxed(drv->base + offset); - val &= ~mask; - val |= value & mask; - writel_relaxed(val, drv->base + offset); -} - -static void cpr_irq_clr(struct cpr_drv *drv) -{ - cpr_write(drv, REG_RBIF_IRQ_CLEAR, CPR_INT_ALL); -} - -static void cpr_irq_clr_nack(struct cpr_drv *drv) -{ - cpr_irq_clr(drv); - cpr_write(drv, REG_RBIF_CONT_NACK_CMD, 1); -} - -static void cpr_irq_clr_ack(struct cpr_drv *drv) -{ - cpr_irq_clr(drv); - cpr_write(drv, REG_RBIF_CONT_ACK_CMD, 1); -} - -static void cpr_irq_set(struct cpr_drv *drv, u32 int_bits) -{ - cpr_write(drv, REG_RBIF_IRQ_EN(0), int_bits); -} - -static void cpr_ctl_modify(struct cpr_drv *drv, u32 mask, u32 value) -{ - cpr_masked_write(drv, REG_RBCPR_CTL, mask, value); -} - -static void cpr_ctl_enable(struct cpr_drv *drv, struct corner *corner) -{ - u32 val, mask; - const struct cpr_desc *desc = drv->desc; - - /* Program Consecutive Up & Down */ - val = desc->timer_cons_down << RBIF_TIMER_ADJ_CONS_DOWN_SHIFT; - val |= desc->timer_cons_up << RBIF_TIMER_ADJ_CONS_UP_SHIFT; - mask = RBIF_TIMER_ADJ_CONS_UP_MASK | RBIF_TIMER_ADJ_CONS_DOWN_MASK; - cpr_masked_write(drv, REG_RBIF_TIMER_ADJUST, mask, val); - cpr_masked_write(drv, REG_RBCPR_CTL, - RBCPR_CTL_SW_AUTO_CONT_NACK_DN_EN | - RBCPR_CTL_SW_AUTO_CONT_ACK_EN, - corner->save_ctl); - cpr_irq_set(drv, corner->save_irq); - - if (cpr_is_allowed(drv) && corner->max_uV > corner->min_uV) - val = RBCPR_CTL_LOOP_EN; - else - val = 0; - cpr_ctl_modify(drv, RBCPR_CTL_LOOP_EN, val); -} - -static void cpr_ctl_disable(struct cpr_drv *drv) -{ - cpr_irq_set(drv, 0); - cpr_ctl_modify(drv, RBCPR_CTL_SW_AUTO_CONT_NACK_DN_EN | - RBCPR_CTL_SW_AUTO_CONT_ACK_EN, 0); - cpr_masked_write(drv, REG_RBIF_TIMER_ADJUST, - RBIF_TIMER_ADJ_CONS_UP_MASK | - RBIF_TIMER_ADJ_CONS_DOWN_MASK, 0); - cpr_irq_clr(drv); - cpr_write(drv, REG_RBIF_CONT_ACK_CMD, 1); - cpr_write(drv, REG_RBIF_CONT_NACK_CMD, 1); - cpr_ctl_modify(drv, RBCPR_CTL_LOOP_EN, 0); -} - -static bool cpr_ctl_is_enabled(struct cpr_drv *drv) -{ - u32 reg_val; - - reg_val = cpr_read(drv, REG_RBCPR_CTL); - return reg_val & RBCPR_CTL_LOOP_EN; -} - -static bool cpr_ctl_is_busy(struct cpr_drv *drv) -{ - u32 reg_val; - - reg_val = cpr_read(drv, REG_RBCPR_RESULT_0); - return reg_val & RBCPR_RESULT0_BUSY_MASK; -} - -static void cpr_corner_save(struct cpr_drv *drv, struct corner *corner) -{ - corner->save_ctl = cpr_read(drv, REG_RBCPR_CTL); - corner->save_irq = cpr_read(drv, REG_RBIF_IRQ_EN(0)); -} - -static void cpr_corner_restore(struct cpr_drv *drv, struct corner *corner) -{ - u32 gcnt, ctl, irq, ro_sel, step_quot; - struct fuse_corner *fuse = corner->fuse_corner; - const struct cpr_desc *desc = drv->desc; - int i; - - ro_sel = fuse->ring_osc_idx; - gcnt = drv->gcnt; - gcnt |= fuse->quot - corner->quot_adjust; - - /* Program the step quotient and idle clocks */ - step_quot = desc->idle_clocks << RBCPR_STEP_QUOT_IDLE_CLK_SHIFT; - step_quot |= fuse->step_quot & RBCPR_STEP_QUOT_STEPQUOT_MASK; - cpr_write(drv, REG_RBCPR_STEP_QUOT, step_quot); - - /* Clear the target quotient value and gate count of all ROs */ - for (i = 0; i < CPR_NUM_RING_OSC; i++) - cpr_write(drv, REG_RBCPR_GCNT_TARGET(i), 0); - - cpr_write(drv, REG_RBCPR_GCNT_TARGET(ro_sel), gcnt); - ctl = corner->save_ctl; - cpr_write(drv, REG_RBCPR_CTL, ctl); - irq = corner->save_irq; - cpr_irq_set(drv, irq); - dev_dbg(drv->dev, "gcnt = %#08x, ctl = %#08x, irq = %#08x\n", gcnt, - ctl, irq); -} - -static void cpr_set_acc(struct regmap *tcsr, struct fuse_corner *f, - struct fuse_corner *end) -{ - if (f == end) - return; - - if (f < end) { - for (f += 1; f <= end; f++) - regmap_multi_reg_write(tcsr, f->accs, f->num_accs); - } else { - for (f -= 1; f >= end; f--) - regmap_multi_reg_write(tcsr, f->accs, f->num_accs); - } -} - -static int cpr_pre_voltage(struct cpr_drv *drv, - struct fuse_corner *fuse_corner, - enum voltage_change_dir dir) -{ - struct fuse_corner *prev_fuse_corner = drv->corner->fuse_corner; - - if (drv->tcsr && dir == DOWN) - cpr_set_acc(drv->tcsr, prev_fuse_corner, fuse_corner); - - return 0; -} - -static int cpr_post_voltage(struct cpr_drv *drv, - struct fuse_corner *fuse_corner, - enum voltage_change_dir dir) -{ - struct fuse_corner *prev_fuse_corner = drv->corner->fuse_corner; - - if (drv->tcsr && dir == UP) - cpr_set_acc(drv->tcsr, prev_fuse_corner, fuse_corner); - - return 0; -} - -static int cpr_scale_voltage(struct cpr_drv *drv, struct corner *corner, - int new_uV, enum voltage_change_dir dir) -{ - int ret; - struct fuse_corner *fuse_corner = corner->fuse_corner; - - ret = cpr_pre_voltage(drv, fuse_corner, dir); - if (ret) - return ret; - - ret = regulator_set_voltage(drv->vdd_apc, new_uV, new_uV); - if (ret) { - dev_err_ratelimited(drv->dev, "failed to set apc voltage %d\n", - new_uV); - return ret; - } - - ret = cpr_post_voltage(drv, fuse_corner, dir); - if (ret) - return ret; - - return 0; -} - -static unsigned int cpr_get_cur_perf_state(struct cpr_drv *drv) -{ - return drv->corner ? drv->corner - drv->corners + 1 : 0; -} - -static int cpr_scale(struct cpr_drv *drv, enum voltage_change_dir dir) -{ - u32 val, error_steps, reg_mask; - int last_uV, new_uV, step_uV, ret; - struct corner *corner; - const struct cpr_desc *desc = drv->desc; - - if (dir != UP && dir != DOWN) - return 0; - - step_uV = regulator_get_linear_step(drv->vdd_apc); - if (!step_uV) - return -EINVAL; - - corner = drv->corner; - - val = cpr_read(drv, REG_RBCPR_RESULT_0); - - error_steps = val >> RBCPR_RESULT0_ERROR_STEPS_SHIFT; - error_steps &= RBCPR_RESULT0_ERROR_STEPS_MASK; - last_uV = corner->last_uV; - - if (dir == UP) { - if (desc->clamp_timer_interval && - error_steps < desc->up_threshold) { - /* - * Handle the case where another measurement started - * after the interrupt was triggered due to a core - * exiting from power collapse. - */ - error_steps = max(desc->up_threshold, - desc->vdd_apc_step_up_limit); - } - - if (last_uV >= corner->max_uV) { - cpr_irq_clr_nack(drv); - - /* Maximize the UP threshold */ - reg_mask = RBCPR_CTL_UP_THRESHOLD_MASK; - reg_mask <<= RBCPR_CTL_UP_THRESHOLD_SHIFT; - val = reg_mask; - cpr_ctl_modify(drv, reg_mask, val); - - /* Disable UP interrupt */ - cpr_irq_set(drv, CPR_INT_DEFAULT & ~CPR_INT_UP); - - return 0; - } - - if (error_steps > desc->vdd_apc_step_up_limit) - error_steps = desc->vdd_apc_step_up_limit; - - /* Calculate new voltage */ - new_uV = last_uV + error_steps * step_uV; - new_uV = min(new_uV, corner->max_uV); - - dev_dbg(drv->dev, - "UP: -> new_uV: %d last_uV: %d perf state: %u\n", - new_uV, last_uV, cpr_get_cur_perf_state(drv)); - } else { - if (desc->clamp_timer_interval && - error_steps < desc->down_threshold) { - /* - * Handle the case where another measurement started - * after the interrupt was triggered due to a core - * exiting from power collapse. - */ - error_steps = max(desc->down_threshold, - desc->vdd_apc_step_down_limit); - } - - if (last_uV <= corner->min_uV) { - cpr_irq_clr_nack(drv); - - /* Enable auto nack down */ - reg_mask = RBCPR_CTL_SW_AUTO_CONT_NACK_DN_EN; - val = RBCPR_CTL_SW_AUTO_CONT_NACK_DN_EN; - - cpr_ctl_modify(drv, reg_mask, val); - - /* Disable DOWN interrupt */ - cpr_irq_set(drv, CPR_INT_DEFAULT & ~CPR_INT_DOWN); - - return 0; - } - - if (error_steps > desc->vdd_apc_step_down_limit) - error_steps = desc->vdd_apc_step_down_limit; - - /* Calculate new voltage */ - new_uV = last_uV - error_steps * step_uV; - new_uV = max(new_uV, corner->min_uV); - - dev_dbg(drv->dev, - "DOWN: -> new_uV: %d last_uV: %d perf state: %u\n", - new_uV, last_uV, cpr_get_cur_perf_state(drv)); - } - - ret = cpr_scale_voltage(drv, corner, new_uV, dir); - if (ret) { - cpr_irq_clr_nack(drv); - return ret; - } - drv->corner->last_uV = new_uV; - - if (dir == UP) { - /* Disable auto nack down */ - reg_mask = RBCPR_CTL_SW_AUTO_CONT_NACK_DN_EN; - val = 0; - } else { - /* Restore default threshold for UP */ - reg_mask = RBCPR_CTL_UP_THRESHOLD_MASK; - reg_mask <<= RBCPR_CTL_UP_THRESHOLD_SHIFT; - val = desc->up_threshold; - val <<= RBCPR_CTL_UP_THRESHOLD_SHIFT; - } - - cpr_ctl_modify(drv, reg_mask, val); - - /* Re-enable default interrupts */ - cpr_irq_set(drv, CPR_INT_DEFAULT); - - /* Ack */ - cpr_irq_clr_ack(drv); - - return 0; -} - -static irqreturn_t cpr_irq_handler(int irq, void *dev) -{ - struct cpr_drv *drv = dev; - const struct cpr_desc *desc = drv->desc; - irqreturn_t ret = IRQ_HANDLED; - u32 val; - - mutex_lock(&drv->lock); - - val = cpr_read(drv, REG_RBIF_IRQ_STATUS); - if (drv->flags & FLAGS_IGNORE_1ST_IRQ_STATUS) - val = cpr_read(drv, REG_RBIF_IRQ_STATUS); - - dev_dbg(drv->dev, "IRQ_STATUS = %#02x\n", val); - - if (!cpr_ctl_is_enabled(drv)) { - dev_dbg(drv->dev, "CPR is disabled\n"); - ret = IRQ_NONE; - } else if (cpr_ctl_is_busy(drv) && !desc->clamp_timer_interval) { - dev_dbg(drv->dev, "CPR measurement is not ready\n"); - } else if (!cpr_is_allowed(drv)) { - val = cpr_read(drv, REG_RBCPR_CTL); - dev_err_ratelimited(drv->dev, - "Interrupt broken? RBCPR_CTL = %#02x\n", - val); - ret = IRQ_NONE; - } else { - /* - * Following sequence of handling is as per each IRQ's - * priority - */ - if (val & CPR_INT_UP) { - cpr_scale(drv, UP); - } else if (val & CPR_INT_DOWN) { - cpr_scale(drv, DOWN); - } else if (val & CPR_INT_MIN) { - cpr_irq_clr_nack(drv); - } else if (val & CPR_INT_MAX) { - cpr_irq_clr_nack(drv); - } else if (val & CPR_INT_MID) { - /* RBCPR_CTL_SW_AUTO_CONT_ACK_EN is enabled */ - dev_dbg(drv->dev, "IRQ occurred for Mid Flag\n"); - } else { - dev_dbg(drv->dev, - "IRQ occurred for unknown flag (%#08x)\n", val); - } - - /* Save register values for the corner */ - cpr_corner_save(drv, drv->corner); - } - - mutex_unlock(&drv->lock); - - return ret; -} - -static int cpr_enable(struct cpr_drv *drv) -{ - int ret; - - ret = regulator_enable(drv->vdd_apc); - if (ret) - return ret; - - mutex_lock(&drv->lock); - - if (cpr_is_allowed(drv) && drv->corner) { - cpr_irq_clr(drv); - cpr_corner_restore(drv, drv->corner); - cpr_ctl_enable(drv, drv->corner); - } - - mutex_unlock(&drv->lock); - - return 0; -} - -static int cpr_disable(struct cpr_drv *drv) -{ - mutex_lock(&drv->lock); - - if (cpr_is_allowed(drv)) { - cpr_ctl_disable(drv); - cpr_irq_clr(drv); - } - - mutex_unlock(&drv->lock); - - return regulator_disable(drv->vdd_apc); -} - -static int cpr_config(struct cpr_drv *drv) -{ - int i; - u32 val, gcnt; - struct corner *corner; - const struct cpr_desc *desc = drv->desc; - - /* Disable interrupt and CPR */ - cpr_write(drv, REG_RBIF_IRQ_EN(0), 0); - cpr_write(drv, REG_RBCPR_CTL, 0); - - /* Program the default HW ceiling, floor and vlevel */ - val = (RBIF_LIMIT_CEILING_DEFAULT & RBIF_LIMIT_CEILING_MASK) - << RBIF_LIMIT_CEILING_SHIFT; - val |= RBIF_LIMIT_FLOOR_DEFAULT & RBIF_LIMIT_FLOOR_MASK; - cpr_write(drv, REG_RBIF_LIMIT, val); - cpr_write(drv, REG_RBIF_SW_VLEVEL, RBIF_SW_VLEVEL_DEFAULT); - - /* - * Clear the target quotient value and gate count of all - * ring oscillators - */ - for (i = 0; i < CPR_NUM_RING_OSC; i++) - cpr_write(drv, REG_RBCPR_GCNT_TARGET(i), 0); - - /* Init and save gcnt */ - gcnt = (drv->ref_clk_khz * desc->gcnt_us) / 1000; - gcnt = gcnt & RBCPR_GCNT_TARGET_GCNT_MASK; - gcnt <<= RBCPR_GCNT_TARGET_GCNT_SHIFT; - drv->gcnt = gcnt; - - /* Program the delay count for the timer */ - val = (drv->ref_clk_khz * desc->timer_delay_us) / 1000; - cpr_write(drv, REG_RBCPR_TIMER_INTERVAL, val); - dev_dbg(drv->dev, "Timer count: %#0x (for %d us)\n", val, - desc->timer_delay_us); - - /* Program Consecutive Up & Down */ - val = desc->timer_cons_down << RBIF_TIMER_ADJ_CONS_DOWN_SHIFT; - val |= desc->timer_cons_up << RBIF_TIMER_ADJ_CONS_UP_SHIFT; - val |= desc->clamp_timer_interval << RBIF_TIMER_ADJ_CLAMP_INT_SHIFT; - cpr_write(drv, REG_RBIF_TIMER_ADJUST, val); - - /* Program the control register */ - val = desc->up_threshold << RBCPR_CTL_UP_THRESHOLD_SHIFT; - val |= desc->down_threshold << RBCPR_CTL_DN_THRESHOLD_SHIFT; - val |= RBCPR_CTL_TIMER_EN | RBCPR_CTL_COUNT_MODE; - val |= RBCPR_CTL_SW_AUTO_CONT_ACK_EN; - cpr_write(drv, REG_RBCPR_CTL, val); - - for (i = 0; i < drv->num_corners; i++) { - corner = &drv->corners[i]; - corner->save_ctl = val; - corner->save_irq = CPR_INT_DEFAULT; - } - - cpr_irq_set(drv, CPR_INT_DEFAULT); - - val = cpr_read(drv, REG_RBCPR_VERSION); - if (val <= RBCPR_VER_2) - drv->flags |= FLAGS_IGNORE_1ST_IRQ_STATUS; - - return 0; -} - -static int cpr_set_performance_state(struct generic_pm_domain *domain, - unsigned int state) -{ - struct cpr_drv *drv = container_of(domain, struct cpr_drv, pd); - struct corner *corner, *end; - enum voltage_change_dir dir; - int ret = 0, new_uV; - - mutex_lock(&drv->lock); - - dev_dbg(drv->dev, "%s: setting perf state: %u (prev state: %u)\n", - __func__, state, cpr_get_cur_perf_state(drv)); - - /* - * Determine new corner we're going to. - * Remove one since lowest performance state is 1. - */ - corner = drv->corners + state - 1; - end = &drv->corners[drv->num_corners - 1]; - if (corner > end || corner < drv->corners) { - ret = -EINVAL; - goto unlock; - } - - /* Determine direction */ - if (drv->corner > corner) - dir = DOWN; - else if (drv->corner < corner) - dir = UP; - else - dir = NO_CHANGE; - - if (cpr_is_allowed(drv)) - new_uV = corner->last_uV; - else - new_uV = corner->uV; - - if (cpr_is_allowed(drv)) - cpr_ctl_disable(drv); - - ret = cpr_scale_voltage(drv, corner, new_uV, dir); - if (ret) - goto unlock; - - if (cpr_is_allowed(drv)) { - cpr_irq_clr(drv); - if (drv->corner != corner) - cpr_corner_restore(drv, corner); - cpr_ctl_enable(drv, corner); - } - - drv->corner = corner; - -unlock: - mutex_unlock(&drv->lock); - - return ret; -} - -static int -cpr_populate_ring_osc_idx(struct cpr_drv *drv) -{ - struct fuse_corner *fuse = drv->fuse_corners; - struct fuse_corner *end = fuse + drv->desc->num_fuse_corners; - const struct cpr_fuse *fuses = drv->cpr_fuses; - u32 data; - int ret; - - for (; fuse < end; fuse++, fuses++) { - ret = nvmem_cell_read_variable_le_u32(drv->dev, fuses->ring_osc, &data); - if (ret) - return ret; - fuse->ring_osc_idx = data; - } - - return 0; -} - -static int cpr_read_fuse_uV(const struct cpr_desc *desc, - const struct fuse_corner_data *fdata, - const char *init_v_efuse, - int step_volt, - struct cpr_drv *drv) -{ - int step_size_uV, steps, uV; - u32 bits = 0; - int ret; - - ret = nvmem_cell_read_variable_le_u32(drv->dev, init_v_efuse, &bits); - if (ret) - return ret; - - steps = bits & ~BIT(desc->cpr_fuses.init_voltage_width - 1); - /* Not two's complement.. instead highest bit is sign bit */ - if (bits & BIT(desc->cpr_fuses.init_voltage_width - 1)) - steps = -steps; - - step_size_uV = desc->cpr_fuses.init_voltage_step; - - uV = fdata->ref_uV + steps * step_size_uV; - return DIV_ROUND_UP(uV, step_volt) * step_volt; -} - -static int cpr_fuse_corner_init(struct cpr_drv *drv) -{ - const struct cpr_desc *desc = drv->desc; - const struct cpr_fuse *fuses = drv->cpr_fuses; - const struct acc_desc *acc_desc = drv->acc_desc; - int i; - unsigned int step_volt; - struct fuse_corner_data *fdata; - struct fuse_corner *fuse, *end; - int uV; - const struct reg_sequence *accs; - int ret; - - accs = acc_desc->settings; - - step_volt = regulator_get_linear_step(drv->vdd_apc); - if (!step_volt) - return -EINVAL; - - /* Populate fuse_corner members */ - fuse = drv->fuse_corners; - end = &fuse[desc->num_fuse_corners - 1]; - fdata = desc->cpr_fuses.fuse_corner_data; - - for (i = 0; fuse <= end; fuse++, fuses++, i++, fdata++) { - /* - * Update SoC voltages: platforms might choose a different - * regulators than the one used to characterize the algorithms - * (ie, init_voltage_step). - */ - fdata->min_uV = roundup(fdata->min_uV, step_volt); - fdata->max_uV = roundup(fdata->max_uV, step_volt); - - /* Populate uV */ - uV = cpr_read_fuse_uV(desc, fdata, fuses->init_voltage, - step_volt, drv); - if (uV < 0) - return uV; - - fuse->min_uV = fdata->min_uV; - fuse->max_uV = fdata->max_uV; - fuse->uV = clamp(uV, fuse->min_uV, fuse->max_uV); - - if (fuse == end) { - /* - * Allow the highest fuse corner's PVS voltage to - * define the ceiling voltage for that corner in order - * to support SoC's in which variable ceiling values - * are required. - */ - end->max_uV = max(end->max_uV, end->uV); - } - - /* Populate target quotient by scaling */ - ret = nvmem_cell_read_variable_le_u32(drv->dev, fuses->quotient, &fuse->quot); - if (ret) - return ret; - - fuse->quot *= fdata->quot_scale; - fuse->quot += fdata->quot_offset; - fuse->quot += fdata->quot_adjust; - fuse->step_quot = desc->step_quot[fuse->ring_osc_idx]; - - /* Populate acc settings */ - fuse->accs = accs; - fuse->num_accs = acc_desc->num_regs_per_fuse; - accs += acc_desc->num_regs_per_fuse; - } - - /* - * Restrict all fuse corner PVS voltages based upon per corner - * ceiling and floor voltages. - */ - for (fuse = drv->fuse_corners, i = 0; fuse <= end; fuse++, i++) { - if (fuse->uV > fuse->max_uV) - fuse->uV = fuse->max_uV; - else if (fuse->uV < fuse->min_uV) - fuse->uV = fuse->min_uV; - - ret = regulator_is_supported_voltage(drv->vdd_apc, - fuse->min_uV, - fuse->min_uV); - if (!ret) { - dev_err(drv->dev, - "min uV: %d (fuse corner: %d) not supported by regulator\n", - fuse->min_uV, i); - return -EINVAL; - } - - ret = regulator_is_supported_voltage(drv->vdd_apc, - fuse->max_uV, - fuse->max_uV); - if (!ret) { - dev_err(drv->dev, - "max uV: %d (fuse corner: %d) not supported by regulator\n", - fuse->max_uV, i); - return -EINVAL; - } - - dev_dbg(drv->dev, - "fuse corner %d: [%d %d %d] RO%hhu quot %d squot %d\n", - i, fuse->min_uV, fuse->uV, fuse->max_uV, - fuse->ring_osc_idx, fuse->quot, fuse->step_quot); - } - - return 0; -} - -static int cpr_calculate_scaling(const char *quot_offset, - struct cpr_drv *drv, - const struct fuse_corner_data *fdata, - const struct corner *corner) -{ - u32 quot_diff = 0; - unsigned long freq_diff; - int scaling; - const struct fuse_corner *fuse, *prev_fuse; - int ret; - - fuse = corner->fuse_corner; - prev_fuse = fuse - 1; - - if (quot_offset) { - ret = nvmem_cell_read_variable_le_u32(drv->dev, quot_offset, "_diff); - if (ret) - return ret; - - quot_diff *= fdata->quot_offset_scale; - quot_diff += fdata->quot_offset_adjust; - } else { - quot_diff = fuse->quot - prev_fuse->quot; - } - - freq_diff = fuse->max_freq - prev_fuse->max_freq; - freq_diff /= 1000000; /* Convert to MHz */ - scaling = 1000 * quot_diff / freq_diff; - return min(scaling, fdata->max_quot_scale); -} - -static int cpr_interpolate(const struct corner *corner, int step_volt, - const struct fuse_corner_data *fdata) -{ - unsigned long f_high, f_low, f_diff; - int uV_high, uV_low, uV; - u64 temp, temp_limit; - const struct fuse_corner *fuse, *prev_fuse; - - fuse = corner->fuse_corner; - prev_fuse = fuse - 1; - - f_high = fuse->max_freq; - f_low = prev_fuse->max_freq; - uV_high = fuse->uV; - uV_low = prev_fuse->uV; - f_diff = fuse->max_freq - corner->freq; - - /* - * Don't interpolate in the wrong direction. This could happen - * if the adjusted fuse voltage overlaps with the previous fuse's - * adjusted voltage. - */ - if (f_high <= f_low || uV_high <= uV_low || f_high <= corner->freq) - return corner->uV; - - temp = f_diff * (uV_high - uV_low); - temp = div64_ul(temp, f_high - f_low); - - /* - * max_volt_scale has units of uV/MHz while freq values - * have units of Hz. Divide by 1000000 to convert to. - */ - temp_limit = f_diff * fdata->max_volt_scale; - do_div(temp_limit, 1000000); - - uV = uV_high - min(temp, temp_limit); - return roundup(uV, step_volt); -} - -static unsigned int cpr_get_fuse_corner(struct dev_pm_opp *opp) -{ - struct device_node *np; - unsigned int fuse_corner = 0; - - np = dev_pm_opp_get_of_node(opp); - if (of_property_read_u32(np, "qcom,opp-fuse-level", &fuse_corner)) - pr_err("%s: missing 'qcom,opp-fuse-level' property\n", - __func__); - - of_node_put(np); - - return fuse_corner; -} - -static unsigned long cpr_get_opp_hz_for_req(struct dev_pm_opp *ref, - struct device *cpu_dev) -{ - u64 rate = 0; - struct device_node *ref_np; - struct device_node *desc_np; - struct device_node *child_np = NULL; - struct device_node *child_req_np = NULL; - - desc_np = dev_pm_opp_of_get_opp_desc_node(cpu_dev); - if (!desc_np) - return 0; - - ref_np = dev_pm_opp_get_of_node(ref); - if (!ref_np) - goto out_ref; - - do { - of_node_put(child_req_np); - child_np = of_get_next_available_child(desc_np, child_np); - child_req_np = of_parse_phandle(child_np, "required-opps", 0); - } while (child_np && child_req_np != ref_np); - - if (child_np && child_req_np == ref_np) - of_property_read_u64(child_np, "opp-hz", &rate); - - of_node_put(child_req_np); - of_node_put(child_np); - of_node_put(ref_np); -out_ref: - of_node_put(desc_np); - - return (unsigned long) rate; -} - -static int cpr_corner_init(struct cpr_drv *drv) -{ - const struct cpr_desc *desc = drv->desc; - const struct cpr_fuse *fuses = drv->cpr_fuses; - int i, level, scaling = 0; - unsigned int fnum, fc; - const char *quot_offset; - struct fuse_corner *fuse, *prev_fuse; - struct corner *corner, *end; - struct corner_data *cdata; - const struct fuse_corner_data *fdata; - bool apply_scaling; - unsigned long freq_diff, freq_diff_mhz; - unsigned long freq; - int step_volt = regulator_get_linear_step(drv->vdd_apc); - struct dev_pm_opp *opp; - - if (!step_volt) - return -EINVAL; - - corner = drv->corners; - end = &corner[drv->num_corners - 1]; - - cdata = devm_kcalloc(drv->dev, drv->num_corners, - sizeof(struct corner_data), - GFP_KERNEL); - if (!cdata) - return -ENOMEM; - - /* - * Store maximum frequency for each fuse corner based on the frequency - * plan - */ - for (level = 1; level <= drv->num_corners; level++) { - opp = dev_pm_opp_find_level_exact(&drv->pd.dev, level); - if (IS_ERR(opp)) - return -EINVAL; - fc = cpr_get_fuse_corner(opp); - if (!fc) { - dev_pm_opp_put(opp); - return -EINVAL; - } - fnum = fc - 1; - freq = cpr_get_opp_hz_for_req(opp, drv->attached_cpu_dev); - if (!freq) { - dev_pm_opp_put(opp); - return -EINVAL; - } - cdata[level - 1].fuse_corner = fnum; - cdata[level - 1].freq = freq; - - fuse = &drv->fuse_corners[fnum]; - dev_dbg(drv->dev, "freq: %lu level: %u fuse level: %u\n", - freq, dev_pm_opp_get_level(opp) - 1, fnum); - if (freq > fuse->max_freq) - fuse->max_freq = freq; - dev_pm_opp_put(opp); - } - - /* - * Get the quotient adjustment scaling factor, according to: - * - * scaling = min(1000 * (QUOT(corner_N) - QUOT(corner_N-1)) - * / (freq(corner_N) - freq(corner_N-1)), max_factor) - * - * QUOT(corner_N): quotient read from fuse for fuse corner N - * QUOT(corner_N-1): quotient read from fuse for fuse corner (N - 1) - * freq(corner_N): max frequency in MHz supported by fuse corner N - * freq(corner_N-1): max frequency in MHz supported by fuse corner - * (N - 1) - * - * Then walk through the corners mapped to each fuse corner - * and calculate the quotient adjustment for each one using the - * following formula: - * - * quot_adjust = (freq_max - freq_corner) * scaling / 1000 - * - * freq_max: max frequency in MHz supported by the fuse corner - * freq_corner: frequency in MHz corresponding to the corner - * scaling: calculated from above equation - * - * - * + + - * | v | - * q | f c o | f c - * u | c l | c - * o | f t | f - * t | c a | c - * | c f g | c f - * | e | - * +--------------- +---------------- - * 0 1 2 3 4 5 6 0 1 2 3 4 5 6 - * corner corner - * - * c = corner - * f = fuse corner - * - */ - for (apply_scaling = false, i = 0; corner <= end; corner++, i++) { - fnum = cdata[i].fuse_corner; - fdata = &desc->cpr_fuses.fuse_corner_data[fnum]; - quot_offset = fuses[fnum].quotient_offset; - fuse = &drv->fuse_corners[fnum]; - if (fnum) - prev_fuse = &drv->fuse_corners[fnum - 1]; - else - prev_fuse = NULL; - - corner->fuse_corner = fuse; - corner->freq = cdata[i].freq; - corner->uV = fuse->uV; - - if (prev_fuse && cdata[i - 1].freq == prev_fuse->max_freq) { - scaling = cpr_calculate_scaling(quot_offset, drv, - fdata, corner); - if (scaling < 0) - return scaling; - - apply_scaling = true; - } else if (corner->freq == fuse->max_freq) { - /* This is a fuse corner; don't scale anything */ - apply_scaling = false; - } - - if (apply_scaling) { - freq_diff = fuse->max_freq - corner->freq; - freq_diff_mhz = freq_diff / 1000000; - corner->quot_adjust = scaling * freq_diff_mhz / 1000; - - corner->uV = cpr_interpolate(corner, step_volt, fdata); - } - - corner->max_uV = fuse->max_uV; - corner->min_uV = fuse->min_uV; - corner->uV = clamp(corner->uV, corner->min_uV, corner->max_uV); - corner->last_uV = corner->uV; - - /* Reduce the ceiling voltage if needed */ - if (desc->reduce_to_corner_uV && corner->uV < corner->max_uV) - corner->max_uV = corner->uV; - else if (desc->reduce_to_fuse_uV && fuse->uV < corner->max_uV) - corner->max_uV = max(corner->min_uV, fuse->uV); - - dev_dbg(drv->dev, "corner %d: [%d %d %d] quot %d\n", i, - corner->min_uV, corner->uV, corner->max_uV, - fuse->quot - corner->quot_adjust); - } - - return 0; -} - -static const struct cpr_fuse *cpr_get_fuses(struct cpr_drv *drv) -{ - const struct cpr_desc *desc = drv->desc; - struct cpr_fuse *fuses; - int i; - - fuses = devm_kcalloc(drv->dev, desc->num_fuse_corners, - sizeof(struct cpr_fuse), - GFP_KERNEL); - if (!fuses) - return ERR_PTR(-ENOMEM); - - for (i = 0; i < desc->num_fuse_corners; i++) { - char tbuf[32]; - - snprintf(tbuf, 32, "cpr_ring_osc%d", i + 1); - fuses[i].ring_osc = devm_kstrdup(drv->dev, tbuf, GFP_KERNEL); - if (!fuses[i].ring_osc) - return ERR_PTR(-ENOMEM); - - snprintf(tbuf, 32, "cpr_init_voltage%d", i + 1); - fuses[i].init_voltage = devm_kstrdup(drv->dev, tbuf, - GFP_KERNEL); - if (!fuses[i].init_voltage) - return ERR_PTR(-ENOMEM); - - snprintf(tbuf, 32, "cpr_quotient%d", i + 1); - fuses[i].quotient = devm_kstrdup(drv->dev, tbuf, GFP_KERNEL); - if (!fuses[i].quotient) - return ERR_PTR(-ENOMEM); - - snprintf(tbuf, 32, "cpr_quotient_offset%d", i + 1); - fuses[i].quotient_offset = devm_kstrdup(drv->dev, tbuf, - GFP_KERNEL); - if (!fuses[i].quotient_offset) - return ERR_PTR(-ENOMEM); - } - - return fuses; -} - -static void cpr_set_loop_allowed(struct cpr_drv *drv) -{ - drv->loop_disabled = false; -} - -static int cpr_init_parameters(struct cpr_drv *drv) -{ - const struct cpr_desc *desc = drv->desc; - struct clk *clk; - - clk = clk_get(drv->dev, "ref"); - if (IS_ERR(clk)) - return PTR_ERR(clk); - - drv->ref_clk_khz = clk_get_rate(clk) / 1000; - clk_put(clk); - - if (desc->timer_cons_up > RBIF_TIMER_ADJ_CONS_UP_MASK || - desc->timer_cons_down > RBIF_TIMER_ADJ_CONS_DOWN_MASK || - desc->up_threshold > RBCPR_CTL_UP_THRESHOLD_MASK || - desc->down_threshold > RBCPR_CTL_DN_THRESHOLD_MASK || - desc->idle_clocks > RBCPR_STEP_QUOT_IDLE_CLK_MASK || - desc->clamp_timer_interval > RBIF_TIMER_ADJ_CLAMP_INT_MASK) - return -EINVAL; - - dev_dbg(drv->dev, "up threshold = %u, down threshold = %u\n", - desc->up_threshold, desc->down_threshold); - - return 0; -} - -static int cpr_find_initial_corner(struct cpr_drv *drv) -{ - unsigned long rate; - const struct corner *end; - struct corner *iter; - unsigned int i = 0; - - if (!drv->cpu_clk) { - dev_err(drv->dev, "cannot get rate from NULL clk\n"); - return -EINVAL; - } - - end = &drv->corners[drv->num_corners - 1]; - rate = clk_get_rate(drv->cpu_clk); - - /* - * Some bootloaders set a CPU clock frequency that is not defined - * in the OPP table. When running at an unlisted frequency, - * cpufreq_online() will change to the OPP which has the lowest - * frequency, at or above the unlisted frequency. - * Since cpufreq_online() always "rounds up" in the case of an - * unlisted frequency, this function always "rounds down" in case - * of an unlisted frequency. That way, when cpufreq_online() - * triggers the first ever call to cpr_set_performance_state(), - * it will correctly determine the direction as UP. - */ - for (iter = drv->corners; iter <= end; iter++) { - if (iter->freq > rate) - break; - i++; - if (iter->freq == rate) { - drv->corner = iter; - break; - } - if (iter->freq < rate) - drv->corner = iter; - } - - if (!drv->corner) { - dev_err(drv->dev, "boot up corner not found\n"); - return -EINVAL; - } - - dev_dbg(drv->dev, "boot up perf state: %u\n", i); - - return 0; -} - -static const struct cpr_desc qcs404_cpr_desc = { - .num_fuse_corners = 3, - .min_diff_quot = CPR_FUSE_MIN_QUOT_DIFF, - .step_quot = (int []){ 25, 25, 25, }, - .timer_delay_us = 5000, - .timer_cons_up = 0, - .timer_cons_down = 2, - .up_threshold = 1, - .down_threshold = 3, - .idle_clocks = 15, - .gcnt_us = 1, - .vdd_apc_step_up_limit = 1, - .vdd_apc_step_down_limit = 1, - .cpr_fuses = { - .init_voltage_step = 8000, - .init_voltage_width = 6, - .fuse_corner_data = (struct fuse_corner_data[]){ - /* fuse corner 0 */ - { - .ref_uV = 1224000, - .max_uV = 1224000, - .min_uV = 1048000, - .max_volt_scale = 0, - .max_quot_scale = 0, - .quot_offset = 0, - .quot_scale = 1, - .quot_adjust = 0, - .quot_offset_scale = 5, - .quot_offset_adjust = 0, - }, - /* fuse corner 1 */ - { - .ref_uV = 1288000, - .max_uV = 1288000, - .min_uV = 1048000, - .max_volt_scale = 2000, - .max_quot_scale = 1400, - .quot_offset = 0, - .quot_scale = 1, - .quot_adjust = -20, - .quot_offset_scale = 5, - .quot_offset_adjust = 0, - }, - /* fuse corner 2 */ - { - .ref_uV = 1352000, - .max_uV = 1384000, - .min_uV = 1088000, - .max_volt_scale = 2000, - .max_quot_scale = 1400, - .quot_offset = 0, - .quot_scale = 1, - .quot_adjust = 0, - .quot_offset_scale = 5, - .quot_offset_adjust = 0, - }, - }, - }, -}; - -static const struct acc_desc qcs404_acc_desc = { - .settings = (struct reg_sequence[]){ - { 0xb120, 0x1041040 }, - { 0xb124, 0x41 }, - { 0xb120, 0x0 }, - { 0xb124, 0x0 }, - { 0xb120, 0x0 }, - { 0xb124, 0x0 }, - }, - .config = (struct reg_sequence[]){ - { 0xb138, 0xff }, - { 0xb130, 0x5555 }, - }, - .num_regs_per_fuse = 2, -}; - -static const struct cpr_acc_desc qcs404_cpr_acc_desc = { - .cpr_desc = &qcs404_cpr_desc, - .acc_desc = &qcs404_acc_desc, -}; - -static unsigned int cpr_get_performance_state(struct generic_pm_domain *genpd, - struct dev_pm_opp *opp) -{ - return dev_pm_opp_get_level(opp); -} - -static int cpr_power_off(struct generic_pm_domain *domain) -{ - struct cpr_drv *drv = container_of(domain, struct cpr_drv, pd); - - return cpr_disable(drv); -} - -static int cpr_power_on(struct generic_pm_domain *domain) -{ - struct cpr_drv *drv = container_of(domain, struct cpr_drv, pd); - - return cpr_enable(drv); -} - -static int cpr_pd_attach_dev(struct generic_pm_domain *domain, - struct device *dev) -{ - struct cpr_drv *drv = container_of(domain, struct cpr_drv, pd); - const struct acc_desc *acc_desc = drv->acc_desc; - int ret = 0; - - mutex_lock(&drv->lock); - - dev_dbg(drv->dev, "attach callback for: %s\n", dev_name(dev)); - - /* - * This driver only supports scaling voltage for a CPU cluster - * where all CPUs in the cluster share a single regulator. - * Therefore, save the struct device pointer only for the first - * CPU device that gets attached. There is no need to do any - * additional initialization when further CPUs get attached. - */ - if (drv->attached_cpu_dev) - goto unlock; - - /* - * cpr_scale_voltage() requires the direction (if we are changing - * to a higher or lower OPP). The first time - * cpr_set_performance_state() is called, there is no previous - * performance state defined. Therefore, we call - * cpr_find_initial_corner() that gets the CPU clock frequency - * set by the bootloader, so that we can determine the direction - * the first time cpr_set_performance_state() is called. - */ - drv->cpu_clk = devm_clk_get(dev, NULL); - if (IS_ERR(drv->cpu_clk)) { - ret = PTR_ERR(drv->cpu_clk); - if (ret != -EPROBE_DEFER) - dev_err(drv->dev, "could not get cpu clk: %d\n", ret); - goto unlock; - } - drv->attached_cpu_dev = dev; - - dev_dbg(drv->dev, "using cpu clk from: %s\n", - dev_name(drv->attached_cpu_dev)); - - /* - * Everything related to (virtual) corners has to be initialized - * here, when attaching to the power domain, since we need to know - * the maximum frequency for each fuse corner, and this is only - * available after the cpufreq driver has attached to us. - * The reason for this is that we need to know the highest - * frequency associated with each fuse corner. - */ - ret = dev_pm_opp_get_opp_count(&drv->pd.dev); - if (ret < 0) { - dev_err(drv->dev, "could not get OPP count\n"); - goto unlock; - } - drv->num_corners = ret; - - if (drv->num_corners < 2) { - dev_err(drv->dev, "need at least 2 OPPs to use CPR\n"); - ret = -EINVAL; - goto unlock; - } - - drv->corners = devm_kcalloc(drv->dev, drv->num_corners, - sizeof(*drv->corners), - GFP_KERNEL); - if (!drv->corners) { - ret = -ENOMEM; - goto unlock; - } - - ret = cpr_corner_init(drv); - if (ret) - goto unlock; - - cpr_set_loop_allowed(drv); - - ret = cpr_init_parameters(drv); - if (ret) - goto unlock; - - /* Configure CPR HW but keep it disabled */ - ret = cpr_config(drv); - if (ret) - goto unlock; - - ret = cpr_find_initial_corner(drv); - if (ret) - goto unlock; - - if (acc_desc->config) - regmap_multi_reg_write(drv->tcsr, acc_desc->config, - acc_desc->num_regs_per_fuse); - - /* Enable ACC if required */ - if (acc_desc->enable_mask) - regmap_update_bits(drv->tcsr, acc_desc->enable_reg, - acc_desc->enable_mask, - acc_desc->enable_mask); - - dev_info(drv->dev, "driver initialized with %u OPPs\n", - drv->num_corners); - -unlock: - mutex_unlock(&drv->lock); - - return ret; -} - -static int cpr_debug_info_show(struct seq_file *s, void *unused) -{ - u32 gcnt, ro_sel, ctl, irq_status, reg, error_steps; - u32 step_dn, step_up, error, error_lt0, busy; - struct cpr_drv *drv = s->private; - struct fuse_corner *fuse_corner; - struct corner *corner; - - corner = drv->corner; - fuse_corner = corner->fuse_corner; - - seq_printf(s, "corner, current_volt = %d uV\n", - corner->last_uV); - - ro_sel = fuse_corner->ring_osc_idx; - gcnt = cpr_read(drv, REG_RBCPR_GCNT_TARGET(ro_sel)); - seq_printf(s, "rbcpr_gcnt_target (%u) = %#02X\n", ro_sel, gcnt); - - ctl = cpr_read(drv, REG_RBCPR_CTL); - seq_printf(s, "rbcpr_ctl = %#02X\n", ctl); - - irq_status = cpr_read(drv, REG_RBIF_IRQ_STATUS); - seq_printf(s, "rbcpr_irq_status = %#02X\n", irq_status); - - reg = cpr_read(drv, REG_RBCPR_RESULT_0); - seq_printf(s, "rbcpr_result_0 = %#02X\n", reg); - - step_dn = reg & 0x01; - step_up = (reg >> RBCPR_RESULT0_STEP_UP_SHIFT) & 0x01; - seq_printf(s, " [step_dn = %u", step_dn); - - seq_printf(s, ", step_up = %u", step_up); - - error_steps = (reg >> RBCPR_RESULT0_ERROR_STEPS_SHIFT) - & RBCPR_RESULT0_ERROR_STEPS_MASK; - seq_printf(s, ", error_steps = %u", error_steps); - - error = (reg >> RBCPR_RESULT0_ERROR_SHIFT) & RBCPR_RESULT0_ERROR_MASK; - seq_printf(s, ", error = %u", error); - - error_lt0 = (reg >> RBCPR_RESULT0_ERROR_LT0_SHIFT) & 0x01; - seq_printf(s, ", error_lt_0 = %u", error_lt0); - - busy = (reg >> RBCPR_RESULT0_BUSY_SHIFT) & 0x01; - seq_printf(s, ", busy = %u]\n", busy); - - return 0; -} -DEFINE_SHOW_ATTRIBUTE(cpr_debug_info); - -static void cpr_debugfs_init(struct cpr_drv *drv) -{ - drv->debugfs = debugfs_create_dir("qcom_cpr", NULL); - - debugfs_create_file("debug_info", 0444, drv->debugfs, - drv, &cpr_debug_info_fops); -} - -static int cpr_probe(struct platform_device *pdev) -{ - struct device *dev = &pdev->dev; - struct cpr_drv *drv; - int irq, ret; - const struct cpr_acc_desc *data; - struct device_node *np; - u32 cpr_rev = FUSE_REVISION_UNKNOWN; - - data = of_device_get_match_data(dev); - if (!data || !data->cpr_desc || !data->acc_desc) - return -EINVAL; - - drv = devm_kzalloc(dev, sizeof(*drv), GFP_KERNEL); - if (!drv) - return -ENOMEM; - drv->dev = dev; - drv->desc = data->cpr_desc; - drv->acc_desc = data->acc_desc; - - drv->fuse_corners = devm_kcalloc(dev, drv->desc->num_fuse_corners, - sizeof(*drv->fuse_corners), - GFP_KERNEL); - if (!drv->fuse_corners) - return -ENOMEM; - - np = of_parse_phandle(dev->of_node, "acc-syscon", 0); - if (!np) - return -ENODEV; - - drv->tcsr = syscon_node_to_regmap(np); - of_node_put(np); - if (IS_ERR(drv->tcsr)) - return PTR_ERR(drv->tcsr); - - drv->base = devm_platform_ioremap_resource(pdev, 0); - if (IS_ERR(drv->base)) - return PTR_ERR(drv->base); - - irq = platform_get_irq(pdev, 0); - if (irq < 0) - return -EINVAL; - - drv->vdd_apc = devm_regulator_get(dev, "vdd-apc"); - if (IS_ERR(drv->vdd_apc)) - return PTR_ERR(drv->vdd_apc); - - /* - * Initialize fuse corners, since it simply depends - * on data in efuses. - * Everything related to (virtual) corners has to be - * initialized after attaching to the power domain, - * since it depends on the CPU's OPP table. - */ - ret = nvmem_cell_read_variable_le_u32(dev, "cpr_fuse_revision", &cpr_rev); - if (ret) - return ret; - - drv->cpr_fuses = cpr_get_fuses(drv); - if (IS_ERR(drv->cpr_fuses)) - return PTR_ERR(drv->cpr_fuses); - - ret = cpr_populate_ring_osc_idx(drv); - if (ret) - return ret; - - ret = cpr_fuse_corner_init(drv); - if (ret) - return ret; - - mutex_init(&drv->lock); - - ret = devm_request_threaded_irq(dev, irq, NULL, - cpr_irq_handler, - IRQF_ONESHOT | IRQF_TRIGGER_RISING, - "cpr", drv); - if (ret) - return ret; - - drv->pd.name = devm_kstrdup_const(dev, dev->of_node->full_name, - GFP_KERNEL); - if (!drv->pd.name) - return -EINVAL; - - drv->pd.power_off = cpr_power_off; - drv->pd.power_on = cpr_power_on; - drv->pd.set_performance_state = cpr_set_performance_state; - drv->pd.opp_to_performance_state = cpr_get_performance_state; - drv->pd.attach_dev = cpr_pd_attach_dev; - - ret = pm_genpd_init(&drv->pd, NULL, true); - if (ret) - return ret; - - ret = of_genpd_add_provider_simple(dev->of_node, &drv->pd); - if (ret) - goto err_remove_genpd; - - platform_set_drvdata(pdev, drv); - cpr_debugfs_init(drv); - - return 0; - -err_remove_genpd: - pm_genpd_remove(&drv->pd); - return ret; -} - -static int cpr_remove(struct platform_device *pdev) -{ - struct cpr_drv *drv = platform_get_drvdata(pdev); - - if (cpr_is_allowed(drv)) { - cpr_ctl_disable(drv); - cpr_irq_set(drv, 0); - } - - of_genpd_del_provider(pdev->dev.of_node); - pm_genpd_remove(&drv->pd); - - debugfs_remove_recursive(drv->debugfs); - - return 0; -} - -static const struct of_device_id cpr_match_table[] = { - { .compatible = "qcom,qcs404-cpr", .data = &qcs404_cpr_acc_desc }, - { } -}; -MODULE_DEVICE_TABLE(of, cpr_match_table); - -static struct platform_driver cpr_driver = { - .probe = cpr_probe, - .remove = cpr_remove, - .driver = { - .name = "qcom-cpr", - .of_match_table = cpr_match_table, - }, -}; -module_platform_driver(cpr_driver); - -MODULE_DESCRIPTION("Core Power Reduction (CPR) driver"); -MODULE_LICENSE("GPL v2"); diff --git a/drivers/genpd/qcom/rpmhpd.c b/drivers/genpd/qcom/rpmhpd.c deleted file mode 100644 index a87e336d5e33..000000000000 --- a/drivers/genpd/qcom/rpmhpd.c +++ /dev/null @@ -1,886 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* Copyright (c) 2018, The Linux Foundation. All rights reserved.*/ - -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -#define domain_to_rpmhpd(domain) container_of(domain, struct rpmhpd, pd) - -#define RPMH_ARC_MAX_LEVELS 16 - -/** - * struct rpmhpd - top level RPMh power domain resource data structure - * @dev: rpmh power domain controller device - * @pd: generic_pm_domain corresponding to the power domain - * @parent: generic_pm_domain corresponding to the parent's power domain - * @peer: A peer power domain in case Active only Voting is - * supported - * @active_only: True if it represents an Active only peer - * @corner: current corner - * @active_corner: current active corner - * @enable_corner: lowest non-zero corner - * @level: An array of level (vlvl) to corner (hlvl) mappings - * derived from cmd-db - * @level_count: Number of levels supported by the power domain. max - * being 16 (0 - 15) - * @enabled: true if the power domain is enabled - * @res_name: Resource name used for cmd-db lookup - * @addr: Resource address as looped up using resource name from - * cmd-db - * @state_synced: Indicator that sync_state has been invoked for the rpmhpd resource - */ -struct rpmhpd { - struct device *dev; - struct generic_pm_domain pd; - struct generic_pm_domain *parent; - struct rpmhpd *peer; - const bool active_only; - unsigned int corner; - unsigned int active_corner; - unsigned int enable_corner; - u32 level[RPMH_ARC_MAX_LEVELS]; - size_t level_count; - bool enabled; - const char *res_name; - u32 addr; - bool state_synced; -}; - -struct rpmhpd_desc { - struct rpmhpd **rpmhpds; - size_t num_pds; -}; - -static DEFINE_MUTEX(rpmhpd_lock); - -/* RPMH powerdomains */ - -static struct rpmhpd cx_ao; -static struct rpmhpd mx; -static struct rpmhpd mx_ao; -static struct rpmhpd cx = { - .pd = { .name = "cx", }, - .peer = &cx_ao, - .res_name = "cx.lvl", -}; - -static struct rpmhpd cx_ao = { - .pd = { .name = "cx_ao", }, - .active_only = true, - .peer = &cx, - .res_name = "cx.lvl", -}; - -static struct rpmhpd cx_ao_w_mx_parent; -static struct rpmhpd cx_w_mx_parent = { - .pd = { .name = "cx", }, - .peer = &cx_ao_w_mx_parent, - .parent = &mx.pd, - .res_name = "cx.lvl", -}; - -static struct rpmhpd cx_ao_w_mx_parent = { - .pd = { .name = "cx_ao", }, - .active_only = true, - .peer = &cx_w_mx_parent, - .parent = &mx_ao.pd, - .res_name = "cx.lvl", -}; - -static struct rpmhpd ebi = { - .pd = { .name = "ebi", }, - .res_name = "ebi.lvl", -}; - -static struct rpmhpd gfx = { - .pd = { .name = "gfx", }, - .res_name = "gfx.lvl", -}; - -static struct rpmhpd lcx = { - .pd = { .name = "lcx", }, - .res_name = "lcx.lvl", -}; - -static struct rpmhpd lmx = { - .pd = { .name = "lmx", }, - .res_name = "lmx.lvl", -}; - -static struct rpmhpd mmcx_ao; -static struct rpmhpd mmcx = { - .pd = { .name = "mmcx", }, - .peer = &mmcx_ao, - .res_name = "mmcx.lvl", -}; - -static struct rpmhpd mmcx_ao = { - .pd = { .name = "mmcx_ao", }, - .active_only = true, - .peer = &mmcx, - .res_name = "mmcx.lvl", -}; - -static struct rpmhpd mmcx_ao_w_cx_parent; -static struct rpmhpd mmcx_w_cx_parent = { - .pd = { .name = "mmcx", }, - .peer = &mmcx_ao_w_cx_parent, - .parent = &cx.pd, - .res_name = "mmcx.lvl", -}; - -static struct rpmhpd mmcx_ao_w_cx_parent = { - .pd = { .name = "mmcx_ao", }, - .active_only = true, - .peer = &mmcx_w_cx_parent, - .parent = &cx_ao.pd, - .res_name = "mmcx.lvl", -}; - -static struct rpmhpd mss = { - .pd = { .name = "mss", }, - .res_name = "mss.lvl", -}; - -static struct rpmhpd mx_ao; -static struct rpmhpd mx = { - .pd = { .name = "mx", }, - .peer = &mx_ao, - .res_name = "mx.lvl", -}; - -static struct rpmhpd mx_ao = { - .pd = { .name = "mx_ao", }, - .active_only = true, - .peer = &mx, - .res_name = "mx.lvl", -}; - -static struct rpmhpd mxc_ao; -static struct rpmhpd mxc = { - .pd = { .name = "mxc", }, - .peer = &mxc_ao, - .res_name = "mxc.lvl", -}; - -static struct rpmhpd mxc_ao = { - .pd = { .name = "mxc_ao", }, - .active_only = true, - .peer = &mxc, - .res_name = "mxc.lvl", -}; - -static struct rpmhpd nsp = { - .pd = { .name = "nsp", }, - .res_name = "nsp.lvl", -}; - -static struct rpmhpd nsp0 = { - .pd = { .name = "nsp0", }, - .res_name = "nsp0.lvl", -}; - -static struct rpmhpd nsp1 = { - .pd = { .name = "nsp1", }, - .res_name = "nsp1.lvl", -}; - -static struct rpmhpd qphy = { - .pd = { .name = "qphy", }, - .res_name = "qphy.lvl", -}; - -/* SA8540P RPMH powerdomains */ -static struct rpmhpd *sa8540p_rpmhpds[] = { - [SC8280XP_CX] = &cx, - [SC8280XP_CX_AO] = &cx_ao, - [SC8280XP_EBI] = &ebi, - [SC8280XP_GFX] = &gfx, - [SC8280XP_LCX] = &lcx, - [SC8280XP_LMX] = &lmx, - [SC8280XP_MMCX] = &mmcx, - [SC8280XP_MMCX_AO] = &mmcx_ao, - [SC8280XP_MX] = &mx, - [SC8280XP_MX_AO] = &mx_ao, - [SC8280XP_NSP] = &nsp, -}; - -static const struct rpmhpd_desc sa8540p_desc = { - .rpmhpds = sa8540p_rpmhpds, - .num_pds = ARRAY_SIZE(sa8540p_rpmhpds), -}; - -/* SA8775P RPMH power domains */ -static struct rpmhpd *sa8775p_rpmhpds[] = { - [SA8775P_CX] = &cx, - [SA8775P_CX_AO] = &cx_ao, - [SA8775P_EBI] = &ebi, - [SA8775P_GFX] = &gfx, - [SA8775P_LCX] = &lcx, - [SA8775P_LMX] = &lmx, - [SA8775P_MMCX] = &mmcx, - [SA8775P_MMCX_AO] = &mmcx_ao, - [SA8775P_MXC] = &mxc, - [SA8775P_MXC_AO] = &mxc_ao, - [SA8775P_MX] = &mx, - [SA8775P_MX_AO] = &mx_ao, - [SA8775P_NSP0] = &nsp0, - [SA8775P_NSP1] = &nsp1, -}; - -static const struct rpmhpd_desc sa8775p_desc = { - .rpmhpds = sa8775p_rpmhpds, - .num_pds = ARRAY_SIZE(sa8775p_rpmhpds), -}; - -/* SDM670 RPMH powerdomains */ -static struct rpmhpd *sdm670_rpmhpds[] = { - [SDM670_CX] = &cx_w_mx_parent, - [SDM670_CX_AO] = &cx_ao_w_mx_parent, - [SDM670_GFX] = &gfx, - [SDM670_LCX] = &lcx, - [SDM670_LMX] = &lmx, - [SDM670_MSS] = &mss, - [SDM670_MX] = &mx, - [SDM670_MX_AO] = &mx_ao, -}; - -static const struct rpmhpd_desc sdm670_desc = { - .rpmhpds = sdm670_rpmhpds, - .num_pds = ARRAY_SIZE(sdm670_rpmhpds), -}; - -/* SDM845 RPMH powerdomains */ -static struct rpmhpd *sdm845_rpmhpds[] = { - [SDM845_CX] = &cx_w_mx_parent, - [SDM845_CX_AO] = &cx_ao_w_mx_parent, - [SDM845_EBI] = &ebi, - [SDM845_GFX] = &gfx, - [SDM845_LCX] = &lcx, - [SDM845_LMX] = &lmx, - [SDM845_MSS] = &mss, - [SDM845_MX] = &mx, - [SDM845_MX_AO] = &mx_ao, -}; - -static const struct rpmhpd_desc sdm845_desc = { - .rpmhpds = sdm845_rpmhpds, - .num_pds = ARRAY_SIZE(sdm845_rpmhpds), -}; - -/* SDX55 RPMH powerdomains */ -static struct rpmhpd *sdx55_rpmhpds[] = { - [SDX55_CX] = &cx_w_mx_parent, - [SDX55_MSS] = &mss, - [SDX55_MX] = &mx, -}; - -static const struct rpmhpd_desc sdx55_desc = { - .rpmhpds = sdx55_rpmhpds, - .num_pds = ARRAY_SIZE(sdx55_rpmhpds), -}; - -/* SDX65 RPMH powerdomains */ -static struct rpmhpd *sdx65_rpmhpds[] = { - [SDX65_CX] = &cx_w_mx_parent, - [SDX65_CX_AO] = &cx_ao_w_mx_parent, - [SDX65_MSS] = &mss, - [SDX65_MX] = &mx, - [SDX65_MX_AO] = &mx_ao, - [SDX65_MXC] = &mxc, -}; - -static const struct rpmhpd_desc sdx65_desc = { - .rpmhpds = sdx65_rpmhpds, - .num_pds = ARRAY_SIZE(sdx65_rpmhpds), -}; - -/* SDX75 RPMH powerdomains */ -static struct rpmhpd *sdx75_rpmhpds[] = { - [RPMHPD_CX] = &cx, - [RPMHPD_CX_AO] = &cx_ao, - [RPMHPD_MSS] = &mss, - [RPMHPD_MX] = &mx, - [RPMHPD_MX_AO] = &mx_ao, - [RPMHPD_MXC] = &mxc, -}; - -static const struct rpmhpd_desc sdx75_desc = { - .rpmhpds = sdx75_rpmhpds, - .num_pds = ARRAY_SIZE(sdx75_rpmhpds), -}; - -/* SM6350 RPMH powerdomains */ -static struct rpmhpd *sm6350_rpmhpds[] = { - [SM6350_CX] = &cx_w_mx_parent, - [SM6350_GFX] = &gfx, - [SM6350_LCX] = &lcx, - [SM6350_LMX] = &lmx, - [SM6350_MSS] = &mss, - [SM6350_MX] = &mx, -}; - -static const struct rpmhpd_desc sm6350_desc = { - .rpmhpds = sm6350_rpmhpds, - .num_pds = ARRAY_SIZE(sm6350_rpmhpds), -}; - -/* SM8150 RPMH powerdomains */ -static struct rpmhpd *sm8150_rpmhpds[] = { - [SM8150_CX] = &cx_w_mx_parent, - [SM8150_CX_AO] = &cx_ao_w_mx_parent, - [SM8150_EBI] = &ebi, - [SM8150_GFX] = &gfx, - [SM8150_LCX] = &lcx, - [SM8150_LMX] = &lmx, - [SM8150_MMCX] = &mmcx, - [SM8150_MMCX_AO] = &mmcx_ao, - [SM8150_MSS] = &mss, - [SM8150_MX] = &mx, - [SM8150_MX_AO] = &mx_ao, -}; - -static const struct rpmhpd_desc sm8150_desc = { - .rpmhpds = sm8150_rpmhpds, - .num_pds = ARRAY_SIZE(sm8150_rpmhpds), -}; - -static struct rpmhpd *sa8155p_rpmhpds[] = { - [SA8155P_CX] = &cx_w_mx_parent, - [SA8155P_CX_AO] = &cx_ao_w_mx_parent, - [SA8155P_EBI] = &ebi, - [SA8155P_GFX] = &gfx, - [SA8155P_MSS] = &mss, - [SA8155P_MX] = &mx, - [SA8155P_MX_AO] = &mx_ao, -}; - -static const struct rpmhpd_desc sa8155p_desc = { - .rpmhpds = sa8155p_rpmhpds, - .num_pds = ARRAY_SIZE(sa8155p_rpmhpds), -}; - -/* SM8250 RPMH powerdomains */ -static struct rpmhpd *sm8250_rpmhpds[] = { - [RPMHPD_CX] = &cx_w_mx_parent, - [RPMHPD_CX_AO] = &cx_ao_w_mx_parent, - [RPMHPD_EBI] = &ebi, - [RPMHPD_GFX] = &gfx, - [RPMHPD_LCX] = &lcx, - [RPMHPD_LMX] = &lmx, - [RPMHPD_MMCX] = &mmcx, - [RPMHPD_MMCX_AO] = &mmcx_ao, - [RPMHPD_MX] = &mx, - [RPMHPD_MX_AO] = &mx_ao, -}; - -static const struct rpmhpd_desc sm8250_desc = { - .rpmhpds = sm8250_rpmhpds, - .num_pds = ARRAY_SIZE(sm8250_rpmhpds), -}; - -/* SM8350 Power domains */ -static struct rpmhpd *sm8350_rpmhpds[] = { - [RPMHPD_CX] = &cx_w_mx_parent, - [RPMHPD_CX_AO] = &cx_ao_w_mx_parent, - [RPMHPD_EBI] = &ebi, - [RPMHPD_GFX] = &gfx, - [RPMHPD_LCX] = &lcx, - [RPMHPD_LMX] = &lmx, - [RPMHPD_MMCX] = &mmcx, - [RPMHPD_MMCX_AO] = &mmcx_ao, - [RPMHPD_MSS] = &mss, - [RPMHPD_MX] = &mx, - [RPMHPD_MX_AO] = &mx_ao, - [RPMHPD_MXC] = &mxc, - [RPMHPD_MXC_AO] = &mxc_ao, -}; - -static const struct rpmhpd_desc sm8350_desc = { - .rpmhpds = sm8350_rpmhpds, - .num_pds = ARRAY_SIZE(sm8350_rpmhpds), -}; - -/* SM8450 RPMH powerdomains */ -static struct rpmhpd *sm8450_rpmhpds[] = { - [RPMHPD_CX] = &cx, - [RPMHPD_CX_AO] = &cx_ao, - [RPMHPD_EBI] = &ebi, - [RPMHPD_GFX] = &gfx, - [RPMHPD_LCX] = &lcx, - [RPMHPD_LMX] = &lmx, - [RPMHPD_MMCX] = &mmcx_w_cx_parent, - [RPMHPD_MMCX_AO] = &mmcx_ao_w_cx_parent, - [RPMHPD_MSS] = &mss, - [RPMHPD_MX] = &mx, - [RPMHPD_MX_AO] = &mx_ao, - [RPMHPD_MXC] = &mxc, - [RPMHPD_MXC_AO] = &mxc_ao, -}; - -static const struct rpmhpd_desc sm8450_desc = { - .rpmhpds = sm8450_rpmhpds, - .num_pds = ARRAY_SIZE(sm8450_rpmhpds), -}; - -/* SM8550 RPMH powerdomains */ -static struct rpmhpd *sm8550_rpmhpds[] = { - [RPMHPD_CX] = &cx, - [RPMHPD_CX_AO] = &cx_ao, - [RPMHPD_EBI] = &ebi, - [RPMHPD_GFX] = &gfx, - [RPMHPD_LCX] = &lcx, - [RPMHPD_LMX] = &lmx, - [RPMHPD_MMCX] = &mmcx_w_cx_parent, - [RPMHPD_MMCX_AO] = &mmcx_ao_w_cx_parent, - [RPMHPD_MSS] = &mss, - [RPMHPD_MX] = &mx, - [RPMHPD_MX_AO] = &mx_ao, - [RPMHPD_MXC] = &mxc, - [RPMHPD_MXC_AO] = &mxc_ao, - [RPMHPD_NSP] = &nsp, -}; - -static const struct rpmhpd_desc sm8550_desc = { - .rpmhpds = sm8550_rpmhpds, - .num_pds = ARRAY_SIZE(sm8550_rpmhpds), -}; - -/* QDU1000/QRU1000 RPMH powerdomains */ -static struct rpmhpd *qdu1000_rpmhpds[] = { - [QDU1000_CX] = &cx, - [QDU1000_EBI] = &ebi, - [QDU1000_MSS] = &mss, - [QDU1000_MX] = &mx, -}; - -static const struct rpmhpd_desc qdu1000_desc = { - .rpmhpds = qdu1000_rpmhpds, - .num_pds = ARRAY_SIZE(qdu1000_rpmhpds), -}; - -/* SC7180 RPMH powerdomains */ -static struct rpmhpd *sc7180_rpmhpds[] = { - [SC7180_CX] = &cx_w_mx_parent, - [SC7180_CX_AO] = &cx_ao_w_mx_parent, - [SC7180_GFX] = &gfx, - [SC7180_LCX] = &lcx, - [SC7180_LMX] = &lmx, - [SC7180_MSS] = &mss, - [SC7180_MX] = &mx, - [SC7180_MX_AO] = &mx_ao, -}; - -static const struct rpmhpd_desc sc7180_desc = { - .rpmhpds = sc7180_rpmhpds, - .num_pds = ARRAY_SIZE(sc7180_rpmhpds), -}; - -/* SC7280 RPMH powerdomains */ -static struct rpmhpd *sc7280_rpmhpds[] = { - [SC7280_CX] = &cx, - [SC7280_CX_AO] = &cx_ao, - [SC7280_EBI] = &ebi, - [SC7280_GFX] = &gfx, - [SC7280_LCX] = &lcx, - [SC7280_LMX] = &lmx, - [SC7280_MSS] = &mss, - [SC7280_MX] = &mx, - [SC7280_MX_AO] = &mx_ao, -}; - -static const struct rpmhpd_desc sc7280_desc = { - .rpmhpds = sc7280_rpmhpds, - .num_pds = ARRAY_SIZE(sc7280_rpmhpds), -}; - -/* SC8180x RPMH powerdomains */ -static struct rpmhpd *sc8180x_rpmhpds[] = { - [SC8180X_CX] = &cx_w_mx_parent, - [SC8180X_CX_AO] = &cx_ao_w_mx_parent, - [SC8180X_EBI] = &ebi, - [SC8180X_GFX] = &gfx, - [SC8180X_LCX] = &lcx, - [SC8180X_LMX] = &lmx, - [SC8180X_MMCX] = &mmcx, - [SC8180X_MMCX_AO] = &mmcx_ao, - [SC8180X_MSS] = &mss, - [SC8180X_MX] = &mx, - [SC8180X_MX_AO] = &mx_ao, -}; - -static const struct rpmhpd_desc sc8180x_desc = { - .rpmhpds = sc8180x_rpmhpds, - .num_pds = ARRAY_SIZE(sc8180x_rpmhpds), -}; - -/* SC8280xp RPMH powerdomains */ -static struct rpmhpd *sc8280xp_rpmhpds[] = { - [SC8280XP_CX] = &cx, - [SC8280XP_CX_AO] = &cx_ao, - [SC8280XP_EBI] = &ebi, - [SC8280XP_GFX] = &gfx, - [SC8280XP_LCX] = &lcx, - [SC8280XP_LMX] = &lmx, - [SC8280XP_MMCX] = &mmcx, - [SC8280XP_MMCX_AO] = &mmcx_ao, - [SC8280XP_MX] = &mx, - [SC8280XP_MX_AO] = &mx_ao, - [SC8280XP_NSP] = &nsp, - [SC8280XP_QPHY] = &qphy, -}; - -static const struct rpmhpd_desc sc8280xp_desc = { - .rpmhpds = sc8280xp_rpmhpds, - .num_pds = ARRAY_SIZE(sc8280xp_rpmhpds), -}; - -static const struct of_device_id rpmhpd_match_table[] = { - { .compatible = "qcom,qdu1000-rpmhpd", .data = &qdu1000_desc }, - { .compatible = "qcom,sa8155p-rpmhpd", .data = &sa8155p_desc }, - { .compatible = "qcom,sa8540p-rpmhpd", .data = &sa8540p_desc }, - { .compatible = "qcom,sa8775p-rpmhpd", .data = &sa8775p_desc }, - { .compatible = "qcom,sc7180-rpmhpd", .data = &sc7180_desc }, - { .compatible = "qcom,sc7280-rpmhpd", .data = &sc7280_desc }, - { .compatible = "qcom,sc8180x-rpmhpd", .data = &sc8180x_desc }, - { .compatible = "qcom,sc8280xp-rpmhpd", .data = &sc8280xp_desc }, - { .compatible = "qcom,sdm670-rpmhpd", .data = &sdm670_desc }, - { .compatible = "qcom,sdm845-rpmhpd", .data = &sdm845_desc }, - { .compatible = "qcom,sdx55-rpmhpd", .data = &sdx55_desc}, - { .compatible = "qcom,sdx65-rpmhpd", .data = &sdx65_desc}, - { .compatible = "qcom,sdx75-rpmhpd", .data = &sdx75_desc}, - { .compatible = "qcom,sm6350-rpmhpd", .data = &sm6350_desc }, - { .compatible = "qcom,sm8150-rpmhpd", .data = &sm8150_desc }, - { .compatible = "qcom,sm8250-rpmhpd", .data = &sm8250_desc }, - { .compatible = "qcom,sm8350-rpmhpd", .data = &sm8350_desc }, - { .compatible = "qcom,sm8450-rpmhpd", .data = &sm8450_desc }, - { .compatible = "qcom,sm8550-rpmhpd", .data = &sm8550_desc }, - { } -}; -MODULE_DEVICE_TABLE(of, rpmhpd_match_table); - -static int rpmhpd_send_corner(struct rpmhpd *pd, int state, - unsigned int corner, bool sync) -{ - struct tcs_cmd cmd = { - .addr = pd->addr, - .data = corner, - }; - - /* - * Wait for an ack only when we are increasing the - * perf state of the power domain - */ - if (sync) - return rpmh_write(pd->dev, state, &cmd, 1); - else - return rpmh_write_async(pd->dev, state, &cmd, 1); -} - -static void to_active_sleep(struct rpmhpd *pd, unsigned int corner, - unsigned int *active, unsigned int *sleep) -{ - *active = corner; - - if (pd->active_only) - *sleep = 0; - else - *sleep = *active; -} - -/* - * This function is used to aggregate the votes across the active only - * resources and its peers. The aggregated votes are sent to RPMh as - * ACTIVE_ONLY votes (which take effect immediately), as WAKE_ONLY votes - * (applied by RPMh on system wakeup) and as SLEEP votes (applied by RPMh - * on system sleep). - * We send ACTIVE_ONLY votes for resources without any peers. For others, - * which have an active only peer, all 3 votes are sent. - */ -static int rpmhpd_aggregate_corner(struct rpmhpd *pd, unsigned int corner) -{ - int ret; - struct rpmhpd *peer = pd->peer; - unsigned int active_corner, sleep_corner; - unsigned int this_active_corner = 0, this_sleep_corner = 0; - unsigned int peer_active_corner = 0, peer_sleep_corner = 0; - - if (pd->state_synced) { - to_active_sleep(pd, corner, &this_active_corner, &this_sleep_corner); - } else { - /* Clamp to highest corner if sync_state hasn't happened */ - this_active_corner = pd->level_count - 1; - this_sleep_corner = pd->level_count - 1; - } - - if (peer && peer->enabled) - to_active_sleep(peer, peer->corner, &peer_active_corner, - &peer_sleep_corner); - - active_corner = max(this_active_corner, peer_active_corner); - - ret = rpmhpd_send_corner(pd, RPMH_ACTIVE_ONLY_STATE, active_corner, - active_corner > pd->active_corner); - if (ret) - return ret; - - pd->active_corner = active_corner; - - if (peer) { - peer->active_corner = active_corner; - - ret = rpmhpd_send_corner(pd, RPMH_WAKE_ONLY_STATE, - active_corner, false); - if (ret) - return ret; - - sleep_corner = max(this_sleep_corner, peer_sleep_corner); - - return rpmhpd_send_corner(pd, RPMH_SLEEP_STATE, sleep_corner, - false); - } - - return ret; -} - -static int rpmhpd_power_on(struct generic_pm_domain *domain) -{ - struct rpmhpd *pd = domain_to_rpmhpd(domain); - unsigned int corner; - int ret; - - mutex_lock(&rpmhpd_lock); - - corner = max(pd->corner, pd->enable_corner); - ret = rpmhpd_aggregate_corner(pd, corner); - if (!ret) - pd->enabled = true; - - mutex_unlock(&rpmhpd_lock); - - return ret; -} - -static int rpmhpd_power_off(struct generic_pm_domain *domain) -{ - struct rpmhpd *pd = domain_to_rpmhpd(domain); - int ret; - - mutex_lock(&rpmhpd_lock); - - ret = rpmhpd_aggregate_corner(pd, 0); - if (!ret) - pd->enabled = false; - - mutex_unlock(&rpmhpd_lock); - - return ret; -} - -static int rpmhpd_set_performance_state(struct generic_pm_domain *domain, - unsigned int level) -{ - struct rpmhpd *pd = domain_to_rpmhpd(domain); - int ret = 0, i; - - mutex_lock(&rpmhpd_lock); - - for (i = 0; i < pd->level_count; i++) - if (level <= pd->level[i]) - break; - - /* - * If the level requested is more than that supported by the - * max corner, just set it to max anyway. - */ - if (i == pd->level_count) - i--; - - if (pd->enabled) { - /* Ensure that the domain isn't turn off */ - if (i < pd->enable_corner) - i = pd->enable_corner; - - ret = rpmhpd_aggregate_corner(pd, i); - if (ret) - goto out; - } - - pd->corner = i; -out: - mutex_unlock(&rpmhpd_lock); - - return ret; -} - -static unsigned int rpmhpd_get_performance_state(struct generic_pm_domain *genpd, - struct dev_pm_opp *opp) -{ - return dev_pm_opp_get_level(opp); -} - -static int rpmhpd_update_level_mapping(struct rpmhpd *rpmhpd) -{ - int i; - const u16 *buf; - - buf = cmd_db_read_aux_data(rpmhpd->res_name, &rpmhpd->level_count); - if (IS_ERR(buf)) - return PTR_ERR(buf); - - /* 2 bytes used for each command DB aux data entry */ - rpmhpd->level_count >>= 1; - - if (rpmhpd->level_count > RPMH_ARC_MAX_LEVELS) - return -EINVAL; - - for (i = 0; i < rpmhpd->level_count; i++) { - rpmhpd->level[i] = buf[i]; - - /* Remember the first corner with non-zero level */ - if (!rpmhpd->level[rpmhpd->enable_corner] && rpmhpd->level[i]) - rpmhpd->enable_corner = i; - - /* - * The AUX data may be zero padded. These 0 valued entries at - * the end of the map must be ignored. - */ - if (i > 0 && rpmhpd->level[i] == 0) { - rpmhpd->level_count = i; - break; - } - pr_debug("%s: ARC hlvl=%2d --> vlvl=%4u\n", rpmhpd->res_name, i, - rpmhpd->level[i]); - } - - return 0; -} - -static int rpmhpd_probe(struct platform_device *pdev) -{ - int i, ret; - size_t num_pds; - struct device *dev = &pdev->dev; - struct genpd_onecell_data *data; - struct rpmhpd **rpmhpds; - const struct rpmhpd_desc *desc; - - desc = of_device_get_match_data(dev); - if (!desc) - return -EINVAL; - - rpmhpds = desc->rpmhpds; - num_pds = desc->num_pds; - - data = devm_kzalloc(dev, sizeof(*data), GFP_KERNEL); - if (!data) - return -ENOMEM; - - data->domains = devm_kcalloc(dev, num_pds, sizeof(*data->domains), - GFP_KERNEL); - if (!data->domains) - return -ENOMEM; - - data->num_domains = num_pds; - - for (i = 0; i < num_pds; i++) { - if (!rpmhpds[i]) - continue; - - rpmhpds[i]->dev = dev; - rpmhpds[i]->addr = cmd_db_read_addr(rpmhpds[i]->res_name); - if (!rpmhpds[i]->addr) { - dev_err(dev, "Could not find RPMh address for resource %s\n", - rpmhpds[i]->res_name); - return -ENODEV; - } - - ret = cmd_db_read_slave_id(rpmhpds[i]->res_name); - if (ret != CMD_DB_HW_ARC) { - dev_err(dev, "RPMh slave ID mismatch\n"); - return -EINVAL; - } - - ret = rpmhpd_update_level_mapping(rpmhpds[i]); - if (ret) - return ret; - - rpmhpds[i]->pd.power_off = rpmhpd_power_off; - rpmhpds[i]->pd.power_on = rpmhpd_power_on; - rpmhpds[i]->pd.set_performance_state = rpmhpd_set_performance_state; - rpmhpds[i]->pd.opp_to_performance_state = rpmhpd_get_performance_state; - pm_genpd_init(&rpmhpds[i]->pd, NULL, true); - - data->domains[i] = &rpmhpds[i]->pd; - } - - /* Add subdomains */ - for (i = 0; i < num_pds; i++) { - if (!rpmhpds[i]) - continue; - if (rpmhpds[i]->parent) - pm_genpd_add_subdomain(rpmhpds[i]->parent, - &rpmhpds[i]->pd); - } - - return of_genpd_add_provider_onecell(pdev->dev.of_node, data); -} - -static void rpmhpd_sync_state(struct device *dev) -{ - const struct rpmhpd_desc *desc = of_device_get_match_data(dev); - struct rpmhpd **rpmhpds = desc->rpmhpds; - unsigned int corner; - struct rpmhpd *pd; - unsigned int i; - int ret; - - mutex_lock(&rpmhpd_lock); - for (i = 0; i < desc->num_pds; i++) { - pd = rpmhpds[i]; - if (!pd) - continue; - - pd->state_synced = true; - if (pd->enabled) - corner = max(pd->corner, pd->enable_corner); - else - corner = 0; - - ret = rpmhpd_aggregate_corner(pd, corner); - if (ret) - dev_err(dev, "failed to sync %s\n", pd->res_name); - } - mutex_unlock(&rpmhpd_lock); -} - -static struct platform_driver rpmhpd_driver = { - .driver = { - .name = "qcom-rpmhpd", - .of_match_table = rpmhpd_match_table, - .suppress_bind_attrs = true, - .sync_state = rpmhpd_sync_state, - }, - .probe = rpmhpd_probe, -}; - -static int __init rpmhpd_init(void) -{ - return platform_driver_register(&rpmhpd_driver); -} -core_initcall(rpmhpd_init); - -MODULE_DESCRIPTION("Qualcomm Technologies, Inc. RPMh Power Domain Driver"); -MODULE_LICENSE("GPL v2"); diff --git a/drivers/genpd/qcom/rpmpd.c b/drivers/genpd/qcom/rpmpd.c deleted file mode 100644 index 3135dd1dafe0..000000000000 --- a/drivers/genpd/qcom/rpmpd.c +++ /dev/null @@ -1,1023 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* Copyright (c) 2017-2018, The Linux Foundation. All rights reserved. */ - -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -#include - -#define domain_to_rpmpd(domain) container_of(domain, struct rpmpd, pd) - -/* Resource types: - * RPMPD_X is X encoded as a little-endian, lower-case, ASCII string */ -#define RPMPD_SMPA 0x61706d73 -#define RPMPD_LDOA 0x616f646c -#define RPMPD_SMPB 0x62706d73 -#define RPMPD_LDOB 0x626f646c -#define RPMPD_RWCX 0x78637772 -#define RPMPD_RWMX 0x786d7772 -#define RPMPD_RWLC 0x636c7772 -#define RPMPD_RWLM 0x6d6c7772 -#define RPMPD_RWSC 0x63737772 -#define RPMPD_RWSM 0x6d737772 -#define RPMPD_RWGX 0x78677772 - -/* Operation Keys */ -#define KEY_CORNER 0x6e726f63 /* corn */ -#define KEY_ENABLE 0x6e657773 /* swen */ -#define KEY_FLOOR_CORNER 0x636676 /* vfc */ -#define KEY_FLOOR_LEVEL 0x6c6676 /* vfl */ -#define KEY_LEVEL 0x6c766c76 /* vlvl */ - -#define MAX_CORNER_RPMPD_STATE 6 - -struct rpmpd_req { - __le32 key; - __le32 nbytes; - __le32 value; -}; - -struct rpmpd { - struct generic_pm_domain pd; - struct generic_pm_domain *parent; - struct rpmpd *peer; - const bool active_only; - unsigned int corner; - bool enabled; - const int res_type; - const int res_id; - struct qcom_smd_rpm *rpm; - unsigned int max_state; - __le32 key; - bool state_synced; -}; - -struct rpmpd_desc { - struct rpmpd **rpmpds; - size_t num_pds; - unsigned int max_state; -}; - -static DEFINE_MUTEX(rpmpd_lock); - -/* CX */ -static struct rpmpd cx_rwcx0_lvl_ao; -static struct rpmpd cx_rwcx0_lvl = { - .pd = { .name = "cx", }, - .peer = &cx_rwcx0_lvl_ao, - .res_type = RPMPD_RWCX, - .res_id = 0, - .key = KEY_LEVEL, -}; - -static struct rpmpd cx_rwcx0_lvl_ao = { - .pd = { .name = "cx_ao", }, - .peer = &cx_rwcx0_lvl, - .active_only = true, - .res_type = RPMPD_RWCX, - .res_id = 0, - .key = KEY_LEVEL, -}; - -static struct rpmpd cx_s1a_corner_ao; -static struct rpmpd cx_s1a_corner = { - .pd = { .name = "cx", }, - .peer = &cx_s1a_corner_ao, - .res_type = RPMPD_SMPA, - .res_id = 1, - .key = KEY_CORNER, -}; - -static struct rpmpd cx_s1a_corner_ao = { - .pd = { .name = "cx_ao", }, - .peer = &cx_s1a_corner, - .active_only = true, - .res_type = RPMPD_SMPA, - .res_id = 1, - .key = KEY_CORNER, -}; - -static struct rpmpd cx_s2a_corner_ao; -static struct rpmpd cx_s2a_corner = { - .pd = { .name = "cx", }, - .peer = &cx_s2a_corner_ao, - .res_type = RPMPD_SMPA, - .res_id = 2, - .key = KEY_CORNER, -}; - -static struct rpmpd cx_s2a_corner_ao = { - .pd = { .name = "cx_ao", }, - .peer = &cx_s2a_corner, - .active_only = true, - .res_type = RPMPD_SMPA, - .res_id = 2, - .key = KEY_CORNER, -}; - -static struct rpmpd cx_s2a_lvl_ao; -static struct rpmpd cx_s2a_lvl = { - .pd = { .name = "cx", }, - .peer = &cx_s2a_lvl_ao, - .res_type = RPMPD_SMPA, - .res_id = 2, - .key = KEY_LEVEL, -}; - -static struct rpmpd cx_s2a_lvl_ao = { - .pd = { .name = "cx_ao", }, - .peer = &cx_s2a_lvl, - .active_only = true, - .res_type = RPMPD_SMPA, - .res_id = 2, - .key = KEY_LEVEL, -}; - -static struct rpmpd cx_s3a_lvl_ao; -static struct rpmpd cx_s3a_lvl = { - .pd = { .name = "cx", }, - .peer = &cx_s3a_lvl_ao, - .res_type = RPMPD_SMPA, - .res_id = 3, - .key = KEY_LEVEL, -}; - -static struct rpmpd cx_s3a_lvl_ao = { - .pd = { .name = "cx_ao", }, - .peer = &cx_s3a_lvl, - .active_only = true, - .res_type = RPMPD_SMPA, - .res_id = 3, - .key = KEY_LEVEL, -}; - -static struct rpmpd cx_rwcx0_vfl = { - .pd = { .name = "cx_vfl", }, - .res_type = RPMPD_RWCX, - .res_id = 0, - .key = KEY_FLOOR_LEVEL, -}; - -static struct rpmpd cx_rwsc2_vfl = { - .pd = { .name = "cx_vfl", }, - .res_type = RPMPD_RWSC, - .res_id = 2, - .key = KEY_FLOOR_LEVEL, -}; - -static struct rpmpd cx_s1a_vfc = { - .pd = { .name = "cx_vfc", }, - .res_type = RPMPD_SMPA, - .res_id = 1, - .key = KEY_FLOOR_CORNER, -}; - -static struct rpmpd cx_s2a_vfc = { - .pd = { .name = "cx_vfc", }, - .res_type = RPMPD_SMPA, - .res_id = 2, - .key = KEY_FLOOR_CORNER, -}; - -static struct rpmpd cx_s2a_vfl = { - .pd = { .name = "cx_vfl", }, - .res_type = RPMPD_SMPA, - .res_id = 2, - .key = KEY_FLOOR_LEVEL, -}; - -static struct rpmpd cx_s3a_vfl = { - .pd = { .name = "cx_vfl", }, - .res_type = RPMPD_SMPA, - .res_id = 3, - .key = KEY_FLOOR_LEVEL, -}; - -/* G(F)X */ -static struct rpmpd gfx_s2b_corner = { - .pd = { .name = "gfx", }, - .res_type = RPMPD_SMPB, - .res_id = 2, - .key = KEY_CORNER, -}; - -static struct rpmpd gfx_s2b_vfc = { - .pd = { .name = "gfx_vfc", }, - .res_type = RPMPD_SMPB, - .res_id = 2, - .key = KEY_FLOOR_CORNER, -}; - -static struct rpmpd mx_rwmx0_lvl; -static struct rpmpd gx_rwgx0_lvl_ao; -static struct rpmpd gx_rwgx0_lvl = { - .pd = { .name = "gx", }, - .peer = &gx_rwgx0_lvl_ao, - .res_type = RPMPD_RWGX, - .parent = &mx_rwmx0_lvl.pd, - .res_id = 0, - .key = KEY_LEVEL, -}; - -static struct rpmpd mx_rwmx0_lvl_ao; -static struct rpmpd gx_rwgx0_lvl_ao = { - .pd = { .name = "gx_ao", }, - .peer = &gx_rwgx0_lvl, - .parent = &mx_rwmx0_lvl_ao.pd, - .active_only = true, - .res_type = RPMPD_RWGX, - .res_id = 0, - .key = KEY_LEVEL, -}; - -/* MX */ -static struct rpmpd mx_l3a_corner_ao; -static struct rpmpd mx_l3a_corner = { - .pd = { .name = "mx", }, - .peer = &mx_l3a_corner_ao, - .res_type = RPMPD_LDOA, - .res_id = 3, - .key = KEY_CORNER, -}; - -static struct rpmpd mx_l3a_corner_ao = { - .pd = { .name = "mx_ao", }, - .peer = &mx_l3a_corner, - .active_only = true, - .res_type = RPMPD_LDOA, - .res_id = 3, - .key = KEY_CORNER, -}; - -static struct rpmpd mx_l12a_lvl_ao; -static struct rpmpd mx_l12a_lvl = { - .pd = { .name = "mx", }, - .peer = &mx_l12a_lvl_ao, - .res_type = RPMPD_LDOA, - .res_id = 12, - .key = KEY_LEVEL, -}; - -static struct rpmpd mx_l12a_lvl_ao = { - .pd = { .name = "mx_ao", }, - .peer = &mx_l12a_lvl, - .active_only = true, - .res_type = RPMPD_LDOA, - .res_id = 12, - .key = KEY_LEVEL, -}; - -static struct rpmpd mx_s2a_corner_ao; -static struct rpmpd mx_s2a_corner = { - .pd = { .name = "mx", }, - .peer = &mx_s2a_corner_ao, - .res_type = RPMPD_SMPA, - .res_id = 2, - .key = KEY_CORNER, -}; - -static struct rpmpd mx_s2a_corner_ao = { - .pd = { .name = "mx_ao", }, - .peer = &mx_s2a_corner, - .active_only = true, - .res_type = RPMPD_SMPA, - .res_id = 2, - .key = KEY_CORNER, -}; - -static struct rpmpd mx_rwmx0_lvl_ao; -static struct rpmpd mx_rwmx0_lvl = { - .pd = { .name = "mx", }, - .peer = &mx_rwmx0_lvl_ao, - .res_type = RPMPD_RWMX, - .res_id = 0, - .key = KEY_LEVEL, -}; - -static struct rpmpd mx_rwmx0_lvl_ao = { - .pd = { .name = "mx_ao", }, - .peer = &mx_rwmx0_lvl, - .active_only = true, - .res_type = RPMPD_RWMX, - .res_id = 0, - .key = KEY_LEVEL, -}; - -static struct rpmpd mx_s6a_lvl_ao; -static struct rpmpd mx_s6a_lvl = { - .pd = { .name = "mx", }, - .peer = &mx_s6a_lvl_ao, - .res_type = RPMPD_SMPA, - .res_id = 6, - .key = KEY_LEVEL, -}; - -static struct rpmpd mx_s6a_lvl_ao = { - .pd = { .name = "mx_ao", }, - .peer = &mx_s6a_lvl, - .active_only = true, - .res_type = RPMPD_SMPA, - .res_id = 6, - .key = KEY_LEVEL, -}; - -static struct rpmpd mx_s7a_lvl_ao; -static struct rpmpd mx_s7a_lvl = { - .pd = { .name = "mx", }, - .peer = &mx_s7a_lvl_ao, - .res_type = RPMPD_SMPA, - .res_id = 7, - .key = KEY_LEVEL, -}; - -static struct rpmpd mx_s7a_lvl_ao = { - .pd = { .name = "mx_ao", }, - .peer = &mx_s7a_lvl, - .active_only = true, - .res_type = RPMPD_SMPA, - .res_id = 7, - .key = KEY_LEVEL, -}; - -static struct rpmpd mx_l12a_vfl = { - .pd = { .name = "mx_vfl", }, - .res_type = RPMPD_LDOA, - .res_id = 12, - .key = KEY_FLOOR_LEVEL, -}; - -static struct rpmpd mx_rwmx0_vfl = { - .pd = { .name = "mx_vfl", }, - .res_type = RPMPD_RWMX, - .res_id = 0, - .key = KEY_FLOOR_LEVEL, -}; - -static struct rpmpd mx_rwsm6_vfl = { - .pd = { .name = "mx_vfl", }, - .res_type = RPMPD_RWSM, - .res_id = 6, - .key = KEY_FLOOR_LEVEL, -}; - -/* MD */ -static struct rpmpd md_s1a_corner_ao; -static struct rpmpd md_s1a_corner = { - .pd = { .name = "md", }, - .peer = &md_s1a_corner_ao, - .res_type = RPMPD_SMPA, - .res_id = 1, - .key = KEY_CORNER, -}; - -static struct rpmpd md_s1a_corner_ao = { - .pd = { .name = "md_ao", }, - .peer = &md_s1a_corner, - .active_only = true, - .res_type = RPMPD_SMPA, - .res_id = 1, - .key = KEY_CORNER, -}; - -static struct rpmpd md_s1a_lvl_ao; -static struct rpmpd md_s1a_lvl = { - .pd = { .name = "md", }, - .peer = &md_s1a_lvl_ao, - .res_type = RPMPD_SMPA, - .res_id = 1, - .key = KEY_LEVEL, -}; - -static struct rpmpd md_s1a_lvl_ao = { - .pd = { .name = "md_ao", }, - .peer = &md_s1a_lvl, - .active_only = true, - .res_type = RPMPD_SMPA, - .res_id = 1, - .key = KEY_LEVEL, -}; - -static struct rpmpd md_s1a_vfc = { - .pd = { .name = "md_vfc", }, - .res_type = RPMPD_SMPA, - .res_id = 1, - .key = KEY_FLOOR_CORNER, -}; - -/* LPI_CX */ -static struct rpmpd lpi_cx_rwlc0_lvl = { - .pd = { .name = "lpi_cx", }, - .res_type = RPMPD_RWLC, - .res_id = 0, - .key = KEY_LEVEL, -}; - -static struct rpmpd lpi_cx_rwlc0_vfl = { - .pd = { .name = "lpi_cx_vfl", }, - .res_type = RPMPD_RWLC, - .res_id = 0, - .key = KEY_FLOOR_LEVEL, -}; - -/* LPI_MX */ -static struct rpmpd lpi_mx_rwlm0_lvl = { - .pd = { .name = "lpi_mx", }, - .res_type = RPMPD_RWLM, - .res_id = 0, - .key = KEY_LEVEL, -}; - -static struct rpmpd lpi_mx_rwlm0_vfl = { - .pd = { .name = "lpi_mx_vfl", }, - .res_type = RPMPD_RWLM, - .res_id = 0, - .key = KEY_FLOOR_LEVEL, -}; - -/* SSC_CX */ -static struct rpmpd ssc_cx_l26a_corner = { - .pd = { .name = "ssc_cx", }, - .res_type = RPMPD_LDOA, - .res_id = 26, - .key = KEY_CORNER, -}; - -static struct rpmpd ssc_cx_rwlc0_lvl = { - .pd = { .name = "ssc_cx", }, - .res_type = RPMPD_RWLC, - .res_id = 0, - .key = KEY_LEVEL, -}; - -static struct rpmpd ssc_cx_rwsc0_lvl = { - .pd = { .name = "ssc_cx", }, - .res_type = RPMPD_RWSC, - .res_id = 0, - .key = KEY_LEVEL, -}; - -static struct rpmpd ssc_cx_l26a_vfc = { - .pd = { .name = "ssc_cx_vfc", }, - .res_type = RPMPD_LDOA, - .res_id = 26, - .key = KEY_FLOOR_CORNER, -}; - -static struct rpmpd ssc_cx_rwlc0_vfl = { - .pd = { .name = "ssc_cx_vfl", }, - .res_type = RPMPD_RWLC, - .res_id = 0, - .key = KEY_FLOOR_LEVEL, -}; - -static struct rpmpd ssc_cx_rwsc0_vfl = { - .pd = { .name = "ssc_cx_vfl", }, - .res_type = RPMPD_RWSC, - .res_id = 0, - .key = KEY_FLOOR_LEVEL, -}; - -/* SSC_MX */ -static struct rpmpd ssc_mx_rwlm0_lvl = { - .pd = { .name = "ssc_mx", }, - .res_type = RPMPD_RWLM, - .res_id = 0, - .key = KEY_LEVEL, -}; - -static struct rpmpd ssc_mx_rwsm0_lvl = { - .pd = { .name = "ssc_mx", }, - .res_type = RPMPD_RWSM, - .res_id = 0, - .key = KEY_LEVEL, -}; - -static struct rpmpd ssc_mx_rwlm0_vfl = { - .pd = { .name = "ssc_mx_vfl", }, - .res_type = RPMPD_RWLM, - .res_id = 0, - .key = KEY_FLOOR_LEVEL, -}; - -static struct rpmpd ssc_mx_rwsm0_vfl = { - .pd = { .name = "ssc_mx_vfl", }, - .res_type = RPMPD_RWSM, - .res_id = 0, - .key = KEY_FLOOR_LEVEL, -}; - -static struct rpmpd *mdm9607_rpmpds[] = { - [MDM9607_VDDCX] = &cx_s3a_lvl, - [MDM9607_VDDCX_AO] = &cx_s3a_lvl_ao, - [MDM9607_VDDCX_VFL] = &cx_s3a_vfl, - [MDM9607_VDDMX] = &mx_l12a_lvl, - [MDM9607_VDDMX_AO] = &mx_l12a_lvl_ao, - [MDM9607_VDDMX_VFL] = &mx_l12a_vfl, -}; - -static const struct rpmpd_desc mdm9607_desc = { - .rpmpds = mdm9607_rpmpds, - .num_pds = ARRAY_SIZE(mdm9607_rpmpds), - .max_state = RPM_SMD_LEVEL_TURBO, -}; - -static struct rpmpd *msm8226_rpmpds[] = { - [MSM8226_VDDCX] = &cx_s1a_corner, - [MSM8226_VDDCX_AO] = &cx_s1a_corner_ao, - [MSM8226_VDDCX_VFC] = &cx_s1a_vfc, -}; - -static const struct rpmpd_desc msm8226_desc = { - .rpmpds = msm8226_rpmpds, - .num_pds = ARRAY_SIZE(msm8226_rpmpds), - .max_state = MAX_CORNER_RPMPD_STATE, -}; - -static struct rpmpd *msm8939_rpmpds[] = { - [MSM8939_VDDMDCX] = &md_s1a_corner, - [MSM8939_VDDMDCX_AO] = &md_s1a_corner_ao, - [MSM8939_VDDMDCX_VFC] = &md_s1a_vfc, - [MSM8939_VDDCX] = &cx_s2a_corner, - [MSM8939_VDDCX_AO] = &cx_s2a_corner_ao, - [MSM8939_VDDCX_VFC] = &cx_s2a_vfc, - [MSM8939_VDDMX] = &mx_l3a_corner, - [MSM8939_VDDMX_AO] = &mx_l3a_corner_ao, -}; - -static const struct rpmpd_desc msm8939_desc = { - .rpmpds = msm8939_rpmpds, - .num_pds = ARRAY_SIZE(msm8939_rpmpds), - .max_state = MAX_CORNER_RPMPD_STATE, -}; - -static struct rpmpd *msm8916_rpmpds[] = { - [MSM8916_VDDCX] = &cx_s1a_corner, - [MSM8916_VDDCX_AO] = &cx_s1a_corner_ao, - [MSM8916_VDDCX_VFC] = &cx_s1a_vfc, - [MSM8916_VDDMX] = &mx_l3a_corner, - [MSM8916_VDDMX_AO] = &mx_l3a_corner_ao, -}; - -static const struct rpmpd_desc msm8916_desc = { - .rpmpds = msm8916_rpmpds, - .num_pds = ARRAY_SIZE(msm8916_rpmpds), - .max_state = MAX_CORNER_RPMPD_STATE, -}; - -static struct rpmpd *msm8953_rpmpds[] = { - [MSM8953_VDDMD] = &md_s1a_lvl, - [MSM8953_VDDMD_AO] = &md_s1a_lvl_ao, - [MSM8953_VDDCX] = &cx_s2a_lvl, - [MSM8953_VDDCX_AO] = &cx_s2a_lvl_ao, - [MSM8953_VDDCX_VFL] = &cx_s2a_vfl, - [MSM8953_VDDMX] = &mx_s7a_lvl, - [MSM8953_VDDMX_AO] = &mx_s7a_lvl_ao, -}; - -static const struct rpmpd_desc msm8953_desc = { - .rpmpds = msm8953_rpmpds, - .num_pds = ARRAY_SIZE(msm8953_rpmpds), - .max_state = RPM_SMD_LEVEL_TURBO, -}; - -static struct rpmpd *msm8976_rpmpds[] = { - [MSM8976_VDDCX] = &cx_s2a_lvl, - [MSM8976_VDDCX_AO] = &cx_s2a_lvl_ao, - [MSM8976_VDDCX_VFL] = &cx_rwsc2_vfl, - [MSM8976_VDDMX] = &mx_s6a_lvl, - [MSM8976_VDDMX_AO] = &mx_s6a_lvl_ao, - [MSM8976_VDDMX_VFL] = &mx_rwsm6_vfl, -}; - -static const struct rpmpd_desc msm8976_desc = { - .rpmpds = msm8976_rpmpds, - .num_pds = ARRAY_SIZE(msm8976_rpmpds), - .max_state = RPM_SMD_LEVEL_TURBO_HIGH, -}; - -static struct rpmpd *msm8994_rpmpds[] = { - [MSM8994_VDDCX] = &cx_s1a_corner, - [MSM8994_VDDCX_AO] = &cx_s1a_corner_ao, - [MSM8994_VDDCX_VFC] = &cx_s1a_vfc, - [MSM8994_VDDMX] = &mx_s2a_corner, - [MSM8994_VDDMX_AO] = &mx_s2a_corner_ao, - - /* Attention! *Some* 8994 boards with pm8004 may use SMPC here! */ - [MSM8994_VDDGFX] = &gfx_s2b_corner, - [MSM8994_VDDGFX_VFC] = &gfx_s2b_vfc, -}; - -static const struct rpmpd_desc msm8994_desc = { - .rpmpds = msm8994_rpmpds, - .num_pds = ARRAY_SIZE(msm8994_rpmpds), - .max_state = MAX_CORNER_RPMPD_STATE, -}; - -static struct rpmpd *msm8996_rpmpds[] = { - [MSM8996_VDDCX] = &cx_s1a_corner, - [MSM8996_VDDCX_AO] = &cx_s1a_corner_ao, - [MSM8996_VDDCX_VFC] = &cx_s1a_vfc, - [MSM8996_VDDMX] = &mx_s2a_corner, - [MSM8996_VDDMX_AO] = &mx_s2a_corner_ao, - [MSM8996_VDDSSCX] = &ssc_cx_l26a_corner, - [MSM8996_VDDSSCX_VFC] = &ssc_cx_l26a_vfc, -}; - -static const struct rpmpd_desc msm8996_desc = { - .rpmpds = msm8996_rpmpds, - .num_pds = ARRAY_SIZE(msm8996_rpmpds), - .max_state = MAX_CORNER_RPMPD_STATE, -}; - -static struct rpmpd *msm8998_rpmpds[] = { - [MSM8998_VDDCX] = &cx_rwcx0_lvl, - [MSM8998_VDDCX_AO] = &cx_rwcx0_lvl_ao, - [MSM8998_VDDCX_VFL] = &cx_rwcx0_vfl, - [MSM8998_VDDMX] = &mx_rwmx0_lvl, - [MSM8998_VDDMX_AO] = &mx_rwmx0_lvl_ao, - [MSM8998_VDDMX_VFL] = &mx_rwmx0_vfl, - [MSM8998_SSCCX] = &ssc_cx_rwsc0_lvl, - [MSM8998_SSCCX_VFL] = &ssc_cx_rwsc0_vfl, - [MSM8998_SSCMX] = &ssc_mx_rwsm0_lvl, - [MSM8998_SSCMX_VFL] = &ssc_mx_rwsm0_vfl, -}; - -static const struct rpmpd_desc msm8998_desc = { - .rpmpds = msm8998_rpmpds, - .num_pds = ARRAY_SIZE(msm8998_rpmpds), - .max_state = RPM_SMD_LEVEL_BINNING, -}; - -static struct rpmpd *qcs404_rpmpds[] = { - [QCS404_VDDMX] = &mx_rwmx0_lvl, - [QCS404_VDDMX_AO] = &mx_rwmx0_lvl_ao, - [QCS404_VDDMX_VFL] = &mx_rwmx0_vfl, - [QCS404_LPICX] = &lpi_cx_rwlc0_lvl, - [QCS404_LPICX_VFL] = &lpi_cx_rwlc0_vfl, - [QCS404_LPIMX] = &lpi_mx_rwlm0_lvl, - [QCS404_LPIMX_VFL] = &lpi_mx_rwlm0_vfl, -}; - -static const struct rpmpd_desc qcs404_desc = { - .rpmpds = qcs404_rpmpds, - .num_pds = ARRAY_SIZE(qcs404_rpmpds), - .max_state = RPM_SMD_LEVEL_BINNING, -}; - -static struct rpmpd *sdm660_rpmpds[] = { - [SDM660_VDDCX] = &cx_rwcx0_lvl, - [SDM660_VDDCX_AO] = &cx_rwcx0_lvl_ao, - [SDM660_VDDCX_VFL] = &cx_rwcx0_vfl, - [SDM660_VDDMX] = &mx_rwmx0_lvl, - [SDM660_VDDMX_AO] = &mx_rwmx0_lvl_ao, - [SDM660_VDDMX_VFL] = &mx_rwmx0_vfl, - [SDM660_SSCCX] = &ssc_cx_rwlc0_lvl, - [SDM660_SSCCX_VFL] = &ssc_cx_rwlc0_vfl, - [SDM660_SSCMX] = &ssc_mx_rwlm0_lvl, - [SDM660_SSCMX_VFL] = &ssc_mx_rwlm0_vfl, -}; - -static const struct rpmpd_desc sdm660_desc = { - .rpmpds = sdm660_rpmpds, - .num_pds = ARRAY_SIZE(sdm660_rpmpds), - .max_state = RPM_SMD_LEVEL_TURBO, -}; - -static struct rpmpd *sm6115_rpmpds[] = { - [SM6115_VDDCX] = &cx_rwcx0_lvl, - [SM6115_VDDCX_AO] = &cx_rwcx0_lvl_ao, - [SM6115_VDDCX_VFL] = &cx_rwcx0_vfl, - [SM6115_VDDMX] = &mx_rwmx0_lvl, - [SM6115_VDDMX_AO] = &mx_rwmx0_lvl_ao, - [SM6115_VDDMX_VFL] = &mx_rwmx0_vfl, - [SM6115_VDD_LPI_CX] = &lpi_cx_rwlc0_lvl, - [SM6115_VDD_LPI_MX] = &lpi_mx_rwlm0_lvl, -}; - -static const struct rpmpd_desc sm6115_desc = { - .rpmpds = sm6115_rpmpds, - .num_pds = ARRAY_SIZE(sm6115_rpmpds), - .max_state = RPM_SMD_LEVEL_TURBO_NO_CPR, -}; - -static struct rpmpd *sm6125_rpmpds[] = { - [SM6125_VDDCX] = &cx_rwcx0_lvl, - [SM6125_VDDCX_AO] = &cx_rwcx0_lvl_ao, - [SM6125_VDDCX_VFL] = &cx_rwcx0_vfl, - [SM6125_VDDMX] = &mx_rwmx0_lvl, - [SM6125_VDDMX_AO] = &mx_rwmx0_lvl_ao, - [SM6125_VDDMX_VFL] = &mx_rwmx0_vfl, -}; - -static const struct rpmpd_desc sm6125_desc = { - .rpmpds = sm6125_rpmpds, - .num_pds = ARRAY_SIZE(sm6125_rpmpds), - .max_state = RPM_SMD_LEVEL_BINNING, -}; - -static struct rpmpd *sm6375_rpmpds[] = { - [SM6375_VDDCX] = &cx_rwcx0_lvl, - [SM6375_VDDCX_AO] = &cx_rwcx0_lvl_ao, - [SM6375_VDDCX_VFL] = &cx_rwcx0_vfl, - [SM6375_VDDMX] = &mx_rwmx0_lvl, - [SM6375_VDDMX_AO] = &mx_rwmx0_lvl_ao, - [SM6375_VDDMX_VFL] = &mx_rwmx0_vfl, - [SM6375_VDDGX] = &gx_rwgx0_lvl, - [SM6375_VDDGX_AO] = &gx_rwgx0_lvl_ao, - [SM6375_VDD_LPI_CX] = &lpi_cx_rwlc0_lvl, - [SM6375_VDD_LPI_MX] = &lpi_mx_rwlm0_lvl, -}; - -static const struct rpmpd_desc sm6375_desc = { - .rpmpds = sm6375_rpmpds, - .num_pds = ARRAY_SIZE(sm6375_rpmpds), - .max_state = RPM_SMD_LEVEL_TURBO_NO_CPR, -}; - -static struct rpmpd *qcm2290_rpmpds[] = { - [QCM2290_VDDCX] = &cx_rwcx0_lvl, - [QCM2290_VDDCX_AO] = &cx_rwcx0_lvl_ao, - [QCM2290_VDDCX_VFL] = &cx_rwcx0_vfl, - [QCM2290_VDDMX] = &mx_rwmx0_lvl, - [QCM2290_VDDMX_AO] = &mx_rwmx0_lvl_ao, - [QCM2290_VDDMX_VFL] = &mx_rwmx0_vfl, - [QCM2290_VDD_LPI_CX] = &lpi_cx_rwlc0_lvl, - [QCM2290_VDD_LPI_MX] = &lpi_mx_rwlm0_lvl, -}; - -static const struct rpmpd_desc qcm2290_desc = { - .rpmpds = qcm2290_rpmpds, - .num_pds = ARRAY_SIZE(qcm2290_rpmpds), - .max_state = RPM_SMD_LEVEL_TURBO_NO_CPR, -}; - -static const struct of_device_id rpmpd_match_table[] = { - { .compatible = "qcom,mdm9607-rpmpd", .data = &mdm9607_desc }, - { .compatible = "qcom,msm8226-rpmpd", .data = &msm8226_desc }, - { .compatible = "qcom,msm8909-rpmpd", .data = &msm8916_desc }, - { .compatible = "qcom,msm8916-rpmpd", .data = &msm8916_desc }, - { .compatible = "qcom,msm8939-rpmpd", .data = &msm8939_desc }, - { .compatible = "qcom,msm8953-rpmpd", .data = &msm8953_desc }, - { .compatible = "qcom,msm8976-rpmpd", .data = &msm8976_desc }, - { .compatible = "qcom,msm8994-rpmpd", .data = &msm8994_desc }, - { .compatible = "qcom,msm8996-rpmpd", .data = &msm8996_desc }, - { .compatible = "qcom,msm8998-rpmpd", .data = &msm8998_desc }, - { .compatible = "qcom,qcm2290-rpmpd", .data = &qcm2290_desc }, - { .compatible = "qcom,qcs404-rpmpd", .data = &qcs404_desc }, - { .compatible = "qcom,sdm660-rpmpd", .data = &sdm660_desc }, - { .compatible = "qcom,sm6115-rpmpd", .data = &sm6115_desc }, - { .compatible = "qcom,sm6125-rpmpd", .data = &sm6125_desc }, - { .compatible = "qcom,sm6375-rpmpd", .data = &sm6375_desc }, - { } -}; -MODULE_DEVICE_TABLE(of, rpmpd_match_table); - -static int rpmpd_send_enable(struct rpmpd *pd, bool enable) -{ - struct rpmpd_req req = { - .key = KEY_ENABLE, - .nbytes = cpu_to_le32(sizeof(u32)), - .value = cpu_to_le32(enable), - }; - - return qcom_rpm_smd_write(pd->rpm, QCOM_SMD_RPM_ACTIVE_STATE, - pd->res_type, pd->res_id, &req, sizeof(req)); -} - -static int rpmpd_send_corner(struct rpmpd *pd, int state, unsigned int corner) -{ - struct rpmpd_req req = { - .key = pd->key, - .nbytes = cpu_to_le32(sizeof(u32)), - .value = cpu_to_le32(corner), - }; - - return qcom_rpm_smd_write(pd->rpm, state, pd->res_type, pd->res_id, - &req, sizeof(req)); -}; - -static void to_active_sleep(struct rpmpd *pd, unsigned int corner, - unsigned int *active, unsigned int *sleep) -{ - *active = corner; - - if (pd->active_only) - *sleep = 0; - else - *sleep = *active; -} - -static int rpmpd_aggregate_corner(struct rpmpd *pd) -{ - int ret; - struct rpmpd *peer = pd->peer; - unsigned int active_corner, sleep_corner; - unsigned int this_active_corner = 0, this_sleep_corner = 0; - unsigned int peer_active_corner = 0, peer_sleep_corner = 0; - - /* Clamp to the highest corner/level if sync_state isn't done yet */ - if (!pd->state_synced) - this_active_corner = this_sleep_corner = pd->max_state - 1; - else - to_active_sleep(pd, pd->corner, &this_active_corner, &this_sleep_corner); - - if (peer && peer->enabled) - to_active_sleep(peer, peer->corner, &peer_active_corner, - &peer_sleep_corner); - - active_corner = max(this_active_corner, peer_active_corner); - - ret = rpmpd_send_corner(pd, QCOM_SMD_RPM_ACTIVE_STATE, active_corner); - if (ret) - return ret; - - sleep_corner = max(this_sleep_corner, peer_sleep_corner); - - return rpmpd_send_corner(pd, QCOM_SMD_RPM_SLEEP_STATE, sleep_corner); -} - -static int rpmpd_power_on(struct generic_pm_domain *domain) -{ - int ret; - struct rpmpd *pd = domain_to_rpmpd(domain); - - mutex_lock(&rpmpd_lock); - - ret = rpmpd_send_enable(pd, true); - if (ret) - goto out; - - pd->enabled = true; - - if (pd->corner) - ret = rpmpd_aggregate_corner(pd); - -out: - mutex_unlock(&rpmpd_lock); - - return ret; -} - -static int rpmpd_power_off(struct generic_pm_domain *domain) -{ - int ret; - struct rpmpd *pd = domain_to_rpmpd(domain); - - mutex_lock(&rpmpd_lock); - - ret = rpmpd_send_enable(pd, false); - if (!ret) - pd->enabled = false; - - mutex_unlock(&rpmpd_lock); - - return ret; -} - -static int rpmpd_set_performance(struct generic_pm_domain *domain, - unsigned int state) -{ - int ret = 0; - struct rpmpd *pd = domain_to_rpmpd(domain); - - if (state > pd->max_state) - state = pd->max_state; - - mutex_lock(&rpmpd_lock); - - pd->corner = state; - - /* Always send updates for vfc and vfl */ - if (!pd->enabled && pd->key != cpu_to_le32(KEY_FLOOR_CORNER) && - pd->key != cpu_to_le32(KEY_FLOOR_LEVEL)) - goto out; - - ret = rpmpd_aggregate_corner(pd); - -out: - mutex_unlock(&rpmpd_lock); - - return ret; -} - -static unsigned int rpmpd_get_performance(struct generic_pm_domain *genpd, - struct dev_pm_opp *opp) -{ - return dev_pm_opp_get_level(opp); -} - -static int rpmpd_probe(struct platform_device *pdev) -{ - int i; - size_t num; - struct genpd_onecell_data *data; - struct qcom_smd_rpm *rpm; - struct rpmpd **rpmpds; - const struct rpmpd_desc *desc; - - rpm = dev_get_drvdata(pdev->dev.parent); - if (!rpm) { - dev_err(&pdev->dev, "Unable to retrieve handle to RPM\n"); - return -ENODEV; - } - - desc = of_device_get_match_data(&pdev->dev); - if (!desc) - return -EINVAL; - - rpmpds = desc->rpmpds; - num = desc->num_pds; - - data = devm_kzalloc(&pdev->dev, sizeof(*data), GFP_KERNEL); - if (!data) - return -ENOMEM; - - data->domains = devm_kcalloc(&pdev->dev, num, sizeof(*data->domains), - GFP_KERNEL); - if (!data->domains) - return -ENOMEM; - - data->num_domains = num; - - for (i = 0; i < num; i++) { - if (!rpmpds[i]) { - dev_warn(&pdev->dev, "rpmpds[] with empty entry at index=%d\n", - i); - continue; - } - - rpmpds[i]->rpm = rpm; - rpmpds[i]->max_state = desc->max_state; - rpmpds[i]->pd.power_off = rpmpd_power_off; - rpmpds[i]->pd.power_on = rpmpd_power_on; - rpmpds[i]->pd.set_performance_state = rpmpd_set_performance; - rpmpds[i]->pd.opp_to_performance_state = rpmpd_get_performance; - pm_genpd_init(&rpmpds[i]->pd, NULL, true); - - data->domains[i] = &rpmpds[i]->pd; - } - - /* Add subdomains */ - for (i = 0; i < num; i++) { - if (!rpmpds[i]) - continue; - - if (rpmpds[i]->parent) - pm_genpd_add_subdomain(rpmpds[i]->parent, &rpmpds[i]->pd); - } - - return of_genpd_add_provider_onecell(pdev->dev.of_node, data); -} - -static void rpmpd_sync_state(struct device *dev) -{ - const struct rpmpd_desc *desc = of_device_get_match_data(dev); - struct rpmpd **rpmpds = desc->rpmpds; - struct rpmpd *pd; - unsigned int i; - int ret; - - mutex_lock(&rpmpd_lock); - for (i = 0; i < desc->num_pds; i++) { - pd = rpmpds[i]; - if (!pd) - continue; - - pd->state_synced = true; - - if (!pd->enabled) - pd->corner = 0; - - ret = rpmpd_aggregate_corner(pd); - if (ret) - dev_err(dev, "failed to sync %s: %d\n", pd->pd.name, ret); - } - mutex_unlock(&rpmpd_lock); -} - -static struct platform_driver rpmpd_driver = { - .driver = { - .name = "qcom-rpmpd", - .of_match_table = rpmpd_match_table, - .suppress_bind_attrs = true, - .sync_state = rpmpd_sync_state, - }, - .probe = rpmpd_probe, -}; - -static int __init rpmpd_init(void) -{ - return platform_driver_register(&rpmpd_driver); -} -core_initcall(rpmpd_init); - -MODULE_DESCRIPTION("Qualcomm Technologies, Inc. RPM Power Domain Driver"); -MODULE_LICENSE("GPL v2"); diff --git a/drivers/genpd/renesas/Makefile b/drivers/genpd/renesas/Makefile deleted file mode 100644 index e306e396fc8c..000000000000 --- a/drivers/genpd/renesas/Makefile +++ /dev/null @@ -1,30 +0,0 @@ -# SPDX-License-Identifier: GPL-2.0 -# SoC -obj-$(CONFIG_SYSC_R8A7742) += r8a7742-sysc.o -obj-$(CONFIG_SYSC_R8A7743) += r8a7743-sysc.o -obj-$(CONFIG_SYSC_R8A7745) += r8a7745-sysc.o -obj-$(CONFIG_SYSC_R8A77470) += r8a77470-sysc.o -obj-$(CONFIG_SYSC_R8A774A1) += r8a774a1-sysc.o -obj-$(CONFIG_SYSC_R8A774B1) += r8a774b1-sysc.o -obj-$(CONFIG_SYSC_R8A774C0) += r8a774c0-sysc.o -obj-$(CONFIG_SYSC_R8A774E1) += r8a774e1-sysc.o -obj-$(CONFIG_SYSC_R8A7779) += r8a7779-sysc.o -obj-$(CONFIG_SYSC_R8A7790) += r8a7790-sysc.o -obj-$(CONFIG_SYSC_R8A7791) += r8a7791-sysc.o -obj-$(CONFIG_SYSC_R8A7792) += r8a7792-sysc.o -obj-$(CONFIG_SYSC_R8A7794) += r8a7794-sysc.o -obj-$(CONFIG_SYSC_R8A7795) += r8a7795-sysc.o -obj-$(CONFIG_SYSC_R8A77960) += r8a7796-sysc.o -obj-$(CONFIG_SYSC_R8A77961) += r8a7796-sysc.o -obj-$(CONFIG_SYSC_R8A77965) += r8a77965-sysc.o -obj-$(CONFIG_SYSC_R8A77970) += r8a77970-sysc.o -obj-$(CONFIG_SYSC_R8A77980) += r8a77980-sysc.o -obj-$(CONFIG_SYSC_R8A77990) += r8a77990-sysc.o -obj-$(CONFIG_SYSC_R8A77995) += r8a77995-sysc.o -obj-$(CONFIG_SYSC_R8A779A0) += r8a779a0-sysc.o -obj-$(CONFIG_SYSC_R8A779F0) += r8a779f0-sysc.o -obj-$(CONFIG_SYSC_R8A779G0) += r8a779g0-sysc.o -# Family -obj-$(CONFIG_SYSC_RCAR) += rcar-sysc.o -obj-$(CONFIG_SYSC_RCAR_GEN4) += rcar-gen4-sysc.o -obj-$(CONFIG_SYSC_RMOBILE) += rmobile-sysc.o diff --git a/drivers/genpd/renesas/r8a7742-sysc.c b/drivers/genpd/renesas/r8a7742-sysc.c deleted file mode 100644 index 219a675f83f4..000000000000 --- a/drivers/genpd/renesas/r8a7742-sysc.c +++ /dev/null @@ -1,42 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * Renesas RZ/G1H System Controller - * - * Copyright (C) 2020 Renesas Electronics Corp. - */ - -#include - -#include - -#include "rcar-sysc.h" - -static const struct rcar_sysc_area r8a7742_areas[] __initconst = { - { "always-on", 0, 0, R8A7742_PD_ALWAYS_ON, -1, PD_ALWAYS_ON }, - { "ca15-scu", 0x180, 0, R8A7742_PD_CA15_SCU, R8A7742_PD_ALWAYS_ON, - PD_SCU }, - { "ca15-cpu0", 0x40, 0, R8A7742_PD_CA15_CPU0, R8A7742_PD_CA15_SCU, - PD_CPU_NOCR }, - { "ca15-cpu1", 0x40, 1, R8A7742_PD_CA15_CPU1, R8A7742_PD_CA15_SCU, - PD_CPU_NOCR }, - { "ca15-cpu2", 0x40, 2, R8A7742_PD_CA15_CPU2, R8A7742_PD_CA15_SCU, - PD_CPU_NOCR }, - { "ca15-cpu3", 0x40, 3, R8A7742_PD_CA15_CPU3, R8A7742_PD_CA15_SCU, - PD_CPU_NOCR }, - { "ca7-scu", 0x100, 0, R8A7742_PD_CA7_SCU, R8A7742_PD_ALWAYS_ON, - PD_SCU }, - { "ca7-cpu0", 0x1c0, 0, R8A7742_PD_CA7_CPU0, R8A7742_PD_CA7_SCU, - PD_CPU_NOCR }, - { "ca7-cpu1", 0x1c0, 1, R8A7742_PD_CA7_CPU1, R8A7742_PD_CA7_SCU, - PD_CPU_NOCR }, - { "ca7-cpu2", 0x1c0, 2, R8A7742_PD_CA7_CPU2, R8A7742_PD_CA7_SCU, - PD_CPU_NOCR }, - { "ca7-cpu3", 0x1c0, 3, R8A7742_PD_CA7_CPU3, R8A7742_PD_CA7_SCU, - PD_CPU_NOCR }, - { "rgx", 0xc0, 0, R8A7742_PD_RGX, R8A7742_PD_ALWAYS_ON }, -}; - -const struct rcar_sysc_info r8a7742_sysc_info __initconst = { - .areas = r8a7742_areas, - .num_areas = ARRAY_SIZE(r8a7742_areas), -}; diff --git a/drivers/genpd/renesas/r8a7743-sysc.c b/drivers/genpd/renesas/r8a7743-sysc.c deleted file mode 100644 index 4e2c0ab951b3..000000000000 --- a/drivers/genpd/renesas/r8a7743-sysc.c +++ /dev/null @@ -1,28 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * Renesas RZ/G1M System Controller - * - * Copyright (C) 2016 Cogent Embedded Inc. - */ - -#include - -#include - -#include "rcar-sysc.h" - -static const struct rcar_sysc_area r8a7743_areas[] __initconst = { - { "always-on", 0, 0, R8A7743_PD_ALWAYS_ON, -1, PD_ALWAYS_ON }, - { "ca15-scu", 0x180, 0, R8A7743_PD_CA15_SCU, R8A7743_PD_ALWAYS_ON, - PD_SCU }, - { "ca15-cpu0", 0x40, 0, R8A7743_PD_CA15_CPU0, R8A7743_PD_CA15_SCU, - PD_CPU_NOCR }, - { "ca15-cpu1", 0x40, 1, R8A7743_PD_CA15_CPU1, R8A7743_PD_CA15_SCU, - PD_CPU_NOCR }, - { "sgx", 0xc0, 0, R8A7743_PD_SGX, R8A7743_PD_ALWAYS_ON }, -}; - -const struct rcar_sysc_info r8a7743_sysc_info __initconst = { - .areas = r8a7743_areas, - .num_areas = ARRAY_SIZE(r8a7743_areas), -}; diff --git a/drivers/genpd/renesas/r8a7745-sysc.c b/drivers/genpd/renesas/r8a7745-sysc.c deleted file mode 100644 index 865821a2f0c6..000000000000 --- a/drivers/genpd/renesas/r8a7745-sysc.c +++ /dev/null @@ -1,28 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * Renesas RZ/G1E System Controller - * - * Copyright (C) 2016 Cogent Embedded Inc. - */ - -#include - -#include - -#include "rcar-sysc.h" - -static const struct rcar_sysc_area r8a7745_areas[] __initconst = { - { "always-on", 0, 0, R8A7745_PD_ALWAYS_ON, -1, PD_ALWAYS_ON }, - { "ca7-scu", 0x100, 0, R8A7745_PD_CA7_SCU, R8A7745_PD_ALWAYS_ON, - PD_SCU }, - { "ca7-cpu0", 0x1c0, 0, R8A7745_PD_CA7_CPU0, R8A7745_PD_CA7_SCU, - PD_CPU_NOCR }, - { "ca7-cpu1", 0x1c0, 1, R8A7745_PD_CA7_CPU1, R8A7745_PD_CA7_SCU, - PD_CPU_NOCR }, - { "sgx", 0xc0, 0, R8A7745_PD_SGX, R8A7745_PD_ALWAYS_ON }, -}; - -const struct rcar_sysc_info r8a7745_sysc_info __initconst = { - .areas = r8a7745_areas, - .num_areas = ARRAY_SIZE(r8a7745_areas), -}; diff --git a/drivers/genpd/renesas/r8a77470-sysc.c b/drivers/genpd/renesas/r8a77470-sysc.c deleted file mode 100644 index 1eeb8018df50..000000000000 --- a/drivers/genpd/renesas/r8a77470-sysc.c +++ /dev/null @@ -1,28 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * Renesas RZ/G1C System Controller - * - * Copyright (C) 2018 Renesas Electronics Corp. - */ - -#include - -#include - -#include "rcar-sysc.h" - -static const struct rcar_sysc_area r8a77470_areas[] __initconst = { - { "always-on", 0, 0, R8A77470_PD_ALWAYS_ON, -1, PD_ALWAYS_ON }, - { "ca7-scu", 0x100, 0, R8A77470_PD_CA7_SCU, R8A77470_PD_ALWAYS_ON, - PD_SCU }, - { "ca7-cpu0", 0x1c0, 0, R8A77470_PD_CA7_CPU0, R8A77470_PD_CA7_SCU, - PD_CPU_NOCR }, - { "ca7-cpu1", 0x1c0, 1, R8A77470_PD_CA7_CPU1, R8A77470_PD_CA7_SCU, - PD_CPU_NOCR }, - { "sgx", 0xc0, 0, R8A77470_PD_SGX, R8A77470_PD_ALWAYS_ON }, -}; - -const struct rcar_sysc_info r8a77470_sysc_info __initconst = { - .areas = r8a77470_areas, - .num_areas = ARRAY_SIZE(r8a77470_areas), -}; diff --git a/drivers/genpd/renesas/r8a774a1-sysc.c b/drivers/genpd/renesas/r8a774a1-sysc.c deleted file mode 100644 index 38ac2c689ff0..000000000000 --- a/drivers/genpd/renesas/r8a774a1-sysc.c +++ /dev/null @@ -1,44 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * Renesas RZ/G2M System Controller - * Copyright (C) 2018 Renesas Electronics Corp. - * - * Based on Renesas R-Car M3-W System Controller - * Copyright (C) 2016 Glider bvba - */ - -#include - -#include - -#include "rcar-sysc.h" - -static const struct rcar_sysc_area r8a774a1_areas[] __initconst = { - { "always-on", 0, 0, R8A774A1_PD_ALWAYS_ON, -1, PD_ALWAYS_ON }, - { "ca57-scu", 0x1c0, 0, R8A774A1_PD_CA57_SCU, R8A774A1_PD_ALWAYS_ON, - PD_SCU }, - { "ca57-cpu0", 0x80, 0, R8A774A1_PD_CA57_CPU0, R8A774A1_PD_CA57_SCU, - PD_CPU_NOCR }, - { "ca57-cpu1", 0x80, 1, R8A774A1_PD_CA57_CPU1, R8A774A1_PD_CA57_SCU, - PD_CPU_NOCR }, - { "ca53-scu", 0x140, 0, R8A774A1_PD_CA53_SCU, R8A774A1_PD_ALWAYS_ON, - PD_SCU }, - { "ca53-cpu0", 0x200, 0, R8A774A1_PD_CA53_CPU0, R8A774A1_PD_CA53_SCU, - PD_CPU_NOCR }, - { "ca53-cpu1", 0x200, 1, R8A774A1_PD_CA53_CPU1, R8A774A1_PD_CA53_SCU, - PD_CPU_NOCR }, - { "ca53-cpu2", 0x200, 2, R8A774A1_PD_CA53_CPU2, R8A774A1_PD_CA53_SCU, - PD_CPU_NOCR }, - { "ca53-cpu3", 0x200, 3, R8A774A1_PD_CA53_CPU3, R8A774A1_PD_CA53_SCU, - PD_CPU_NOCR }, - { "a3vc", 0x380, 0, R8A774A1_PD_A3VC, R8A774A1_PD_ALWAYS_ON }, - { "a2vc0", 0x3c0, 0, R8A774A1_PD_A2VC0, R8A774A1_PD_A3VC }, - { "a2vc1", 0x3c0, 1, R8A774A1_PD_A2VC1, R8A774A1_PD_A3VC }, - { "3dg-a", 0x100, 0, R8A774A1_PD_3DG_A, R8A774A1_PD_ALWAYS_ON }, - { "3dg-b", 0x100, 1, R8A774A1_PD_3DG_B, R8A774A1_PD_3DG_A }, -}; - -const struct rcar_sysc_info r8a774a1_sysc_info __initconst = { - .areas = r8a774a1_areas, - .num_areas = ARRAY_SIZE(r8a774a1_areas), -}; diff --git a/drivers/genpd/renesas/r8a774b1-sysc.c b/drivers/genpd/renesas/r8a774b1-sysc.c deleted file mode 100644 index 5f97ff26f3f8..000000000000 --- a/drivers/genpd/renesas/r8a774b1-sysc.c +++ /dev/null @@ -1,37 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * Renesas RZ/G2N System Controller - * Copyright (C) 2019 Renesas Electronics Corp. - * - * Based on Renesas R-Car M3-W System Controller - * Copyright (C) 2016 Glider bvba - */ - -#include -#include - -#include - -#include "rcar-sysc.h" - -static const struct rcar_sysc_area r8a774b1_areas[] __initconst = { - { "always-on", 0, 0, R8A774B1_PD_ALWAYS_ON, -1, PD_ALWAYS_ON }, - { "ca57-scu", 0x1c0, 0, R8A774B1_PD_CA57_SCU, R8A774B1_PD_ALWAYS_ON, - PD_SCU }, - { "ca57-cpu0", 0x80, 0, R8A774B1_PD_CA57_CPU0, R8A774B1_PD_CA57_SCU, - PD_CPU_NOCR }, - { "ca57-cpu1", 0x80, 1, R8A774B1_PD_CA57_CPU1, R8A774B1_PD_CA57_SCU, - PD_CPU_NOCR }, - { "a3vc", 0x380, 0, R8A774B1_PD_A3VC, R8A774B1_PD_ALWAYS_ON }, - { "a3vp", 0x340, 0, R8A774B1_PD_A3VP, R8A774B1_PD_ALWAYS_ON }, - { "a2vc1", 0x3c0, 1, R8A774B1_PD_A2VC1, R8A774B1_PD_A3VC }, - { "3dg-a", 0x100, 0, R8A774B1_PD_3DG_A, R8A774B1_PD_ALWAYS_ON }, - { "3dg-b", 0x100, 1, R8A774B1_PD_3DG_B, R8A774B1_PD_3DG_A }, -}; - -const struct rcar_sysc_info r8a774b1_sysc_info __initconst = { - .areas = r8a774b1_areas, - .num_areas = ARRAY_SIZE(r8a774b1_areas), - .extmask_offs = 0x2f8, - .extmask_val = BIT(0), -}; diff --git a/drivers/genpd/renesas/r8a774c0-sysc.c b/drivers/genpd/renesas/r8a774c0-sysc.c deleted file mode 100644 index c1c216f7d073..000000000000 --- a/drivers/genpd/renesas/r8a774c0-sysc.c +++ /dev/null @@ -1,55 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * Renesas RZ/G2E System Controller - * Copyright (C) 2018 Renesas Electronics Corp. - * - * Based on Renesas R-Car E3 System Controller - */ - -#include -#include -#include - -#include - -#include "rcar-sysc.h" - -static struct rcar_sysc_area r8a774c0_areas[] __initdata = { - { "always-on", 0, 0, R8A774C0_PD_ALWAYS_ON, -1, PD_ALWAYS_ON }, - { "ca53-scu", 0x140, 0, R8A774C0_PD_CA53_SCU, R8A774C0_PD_ALWAYS_ON, - PD_SCU }, - { "ca53-cpu0", 0x200, 0, R8A774C0_PD_CA53_CPU0, R8A774C0_PD_CA53_SCU, - PD_CPU_NOCR }, - { "ca53-cpu1", 0x200, 1, R8A774C0_PD_CA53_CPU1, R8A774C0_PD_CA53_SCU, - PD_CPU_NOCR }, - { "a3vc", 0x380, 0, R8A774C0_PD_A3VC, R8A774C0_PD_ALWAYS_ON }, - { "a2vc1", 0x3c0, 1, R8A774C0_PD_A2VC1, R8A774C0_PD_A3VC }, - { "3dg-a", 0x100, 0, R8A774C0_PD_3DG_A, R8A774C0_PD_ALWAYS_ON }, - { "3dg-b", 0x100, 1, R8A774C0_PD_3DG_B, R8A774C0_PD_3DG_A }, -}; - -/* Fixups for RZ/G2E ES1.0 revision */ -static const struct soc_device_attribute r8a774c0[] __initconst = { - { .soc_id = "r8a774c0", .revision = "ES1.0" }, - { /* sentinel */ } -}; - -static int __init r8a774c0_sysc_init(void) -{ - if (soc_device_match(r8a774c0)) { - /* Fix incorrect 3DG hierarchy */ - swap(r8a774c0_areas[6], r8a774c0_areas[7]); - r8a774c0_areas[6].parent = R8A774C0_PD_ALWAYS_ON; - r8a774c0_areas[7].parent = R8A774C0_PD_3DG_B; - } - - return 0; -} - -const struct rcar_sysc_info r8a774c0_sysc_info __initconst = { - .init = r8a774c0_sysc_init, - .areas = r8a774c0_areas, - .num_areas = ARRAY_SIZE(r8a774c0_areas), - .extmask_offs = 0x2f8, - .extmask_val = BIT(0), -}; diff --git a/drivers/genpd/renesas/r8a774e1-sysc.c b/drivers/genpd/renesas/r8a774e1-sysc.c deleted file mode 100644 index 18449f746455..000000000000 --- a/drivers/genpd/renesas/r8a774e1-sysc.c +++ /dev/null @@ -1,43 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * Renesas RZ/G2H System Controller - * Copyright (C) 2020 Renesas Electronics Corp. - * - * Based on Renesas R-Car H3 System Controller - * Copyright (C) 2016-2017 Glider bvba - */ - -#include - -#include - -#include "rcar-sysc.h" - -static const struct rcar_sysc_area r8a774e1_areas[] __initconst = { - { "always-on", 0, 0, R8A774E1_PD_ALWAYS_ON, -1, PD_ALWAYS_ON }, - { "ca57-scu", 0x1c0, 0, R8A774E1_PD_CA57_SCU, R8A774E1_PD_ALWAYS_ON, PD_SCU }, - { "ca57-cpu0", 0x80, 0, R8A774E1_PD_CA57_CPU0, R8A774E1_PD_CA57_SCU, PD_CPU_NOCR }, - { "ca57-cpu1", 0x80, 1, R8A774E1_PD_CA57_CPU1, R8A774E1_PD_CA57_SCU, PD_CPU_NOCR }, - { "ca57-cpu2", 0x80, 2, R8A774E1_PD_CA57_CPU2, R8A774E1_PD_CA57_SCU, PD_CPU_NOCR }, - { "ca57-cpu3", 0x80, 3, R8A774E1_PD_CA57_CPU3, R8A774E1_PD_CA57_SCU, PD_CPU_NOCR }, - { "ca53-scu", 0x140, 0, R8A774E1_PD_CA53_SCU, R8A774E1_PD_ALWAYS_ON, PD_SCU }, - { "ca53-cpu0", 0x200, 0, R8A774E1_PD_CA53_CPU0, R8A774E1_PD_CA53_SCU, PD_CPU_NOCR }, - { "ca53-cpu1", 0x200, 1, R8A774E1_PD_CA53_CPU1, R8A774E1_PD_CA53_SCU, PD_CPU_NOCR }, - { "ca53-cpu2", 0x200, 2, R8A774E1_PD_CA53_CPU2, R8A774E1_PD_CA53_SCU, PD_CPU_NOCR }, - { "ca53-cpu3", 0x200, 3, R8A774E1_PD_CA53_CPU3, R8A774E1_PD_CA53_SCU, PD_CPU_NOCR }, - { "a3vp", 0x340, 0, R8A774E1_PD_A3VP, R8A774E1_PD_ALWAYS_ON }, - { "a3vc", 0x380, 0, R8A774E1_PD_A3VC, R8A774E1_PD_ALWAYS_ON }, - { "a2vc1", 0x3c0, 1, R8A774E1_PD_A2VC1, R8A774E1_PD_A3VC }, - { "3dg-a", 0x100, 0, R8A774E1_PD_3DG_A, R8A774E1_PD_ALWAYS_ON }, - { "3dg-b", 0x100, 1, R8A774E1_PD_3DG_B, R8A774E1_PD_3DG_A }, - { "3dg-c", 0x100, 2, R8A774E1_PD_3DG_C, R8A774E1_PD_3DG_B }, - { "3dg-d", 0x100, 3, R8A774E1_PD_3DG_D, R8A774E1_PD_3DG_C }, - { "3dg-e", 0x100, 4, R8A774E1_PD_3DG_E, R8A774E1_PD_3DG_D }, -}; - -const struct rcar_sysc_info r8a774e1_sysc_info __initconst = { - .areas = r8a774e1_areas, - .num_areas = ARRAY_SIZE(r8a774e1_areas), - .extmask_offs = 0x2f8, - .extmask_val = BIT(0), -}; diff --git a/drivers/genpd/renesas/r8a7779-sysc.c b/drivers/genpd/renesas/r8a7779-sysc.c deleted file mode 100644 index e24a7151d55f..000000000000 --- a/drivers/genpd/renesas/r8a7779-sysc.c +++ /dev/null @@ -1,30 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * Renesas R-Car H1 System Controller - * - * Copyright (C) 2016 Glider bvba - */ - -#include - -#include - -#include "rcar-sysc.h" - -static const struct rcar_sysc_area r8a7779_areas[] __initconst = { - { "always-on", 0, 0, R8A7779_PD_ALWAYS_ON, -1, PD_ALWAYS_ON }, - { "arm1", 0x40, 1, R8A7779_PD_ARM1, R8A7779_PD_ALWAYS_ON, - PD_CPU_CR }, - { "arm2", 0x40, 2, R8A7779_PD_ARM2, R8A7779_PD_ALWAYS_ON, - PD_CPU_CR }, - { "arm3", 0x40, 3, R8A7779_PD_ARM3, R8A7779_PD_ALWAYS_ON, - PD_CPU_CR }, - { "sgx", 0xc0, 0, R8A7779_PD_SGX, R8A7779_PD_ALWAYS_ON }, - { "vdp", 0x100, 0, R8A7779_PD_VDP, R8A7779_PD_ALWAYS_ON }, - { "imp", 0x140, 0, R8A7779_PD_IMP, R8A7779_PD_ALWAYS_ON }, -}; - -const struct rcar_sysc_info r8a7779_sysc_info __initconst = { - .areas = r8a7779_areas, - .num_areas = ARRAY_SIZE(r8a7779_areas), -}; diff --git a/drivers/genpd/renesas/r8a7790-sysc.c b/drivers/genpd/renesas/r8a7790-sysc.c deleted file mode 100644 index b9afe7f6245b..000000000000 --- a/drivers/genpd/renesas/r8a7790-sysc.c +++ /dev/null @@ -1,44 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * Renesas R-Car H2 System Controller - * - * Copyright (C) 2016 Glider bvba - */ - -#include - -#include - -#include "rcar-sysc.h" - -static const struct rcar_sysc_area r8a7790_areas[] __initconst = { - { "always-on", 0, 0, R8A7790_PD_ALWAYS_ON, -1, PD_ALWAYS_ON }, - { "ca15-scu", 0x180, 0, R8A7790_PD_CA15_SCU, R8A7790_PD_ALWAYS_ON, - PD_SCU }, - { "ca15-cpu0", 0x40, 0, R8A7790_PD_CA15_CPU0, R8A7790_PD_CA15_SCU, - PD_CPU_NOCR }, - { "ca15-cpu1", 0x40, 1, R8A7790_PD_CA15_CPU1, R8A7790_PD_CA15_SCU, - PD_CPU_NOCR }, - { "ca15-cpu2", 0x40, 2, R8A7790_PD_CA15_CPU2, R8A7790_PD_CA15_SCU, - PD_CPU_NOCR }, - { "ca15-cpu3", 0x40, 3, R8A7790_PD_CA15_CPU3, R8A7790_PD_CA15_SCU, - PD_CPU_NOCR }, - { "ca7-scu", 0x100, 0, R8A7790_PD_CA7_SCU, R8A7790_PD_ALWAYS_ON, - PD_SCU }, - { "ca7-cpu0", 0x1c0, 0, R8A7790_PD_CA7_CPU0, R8A7790_PD_CA7_SCU, - PD_CPU_NOCR }, - { "ca7-cpu1", 0x1c0, 1, R8A7790_PD_CA7_CPU1, R8A7790_PD_CA7_SCU, - PD_CPU_NOCR }, - { "ca7-cpu2", 0x1c0, 2, R8A7790_PD_CA7_CPU2, R8A7790_PD_CA7_SCU, - PD_CPU_NOCR }, - { "ca7-cpu3", 0x1c0, 3, R8A7790_PD_CA7_CPU3, R8A7790_PD_CA7_SCU, - PD_CPU_NOCR }, - { "sh-4a", 0x80, 0, R8A7790_PD_SH_4A, R8A7790_PD_ALWAYS_ON }, - { "rgx", 0xc0, 0, R8A7790_PD_RGX, R8A7790_PD_ALWAYS_ON }, - { "imp", 0x140, 0, R8A7790_PD_IMP, R8A7790_PD_ALWAYS_ON }, -}; - -const struct rcar_sysc_info r8a7790_sysc_info __initconst = { - .areas = r8a7790_areas, - .num_areas = ARRAY_SIZE(r8a7790_areas), -}; diff --git a/drivers/genpd/renesas/r8a7791-sysc.c b/drivers/genpd/renesas/r8a7791-sysc.c deleted file mode 100644 index f00fa24522a3..000000000000 --- a/drivers/genpd/renesas/r8a7791-sysc.c +++ /dev/null @@ -1,29 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * Renesas R-Car M2-W/N System Controller - * - * Copyright (C) 2016 Glider bvba - */ - -#include - -#include - -#include "rcar-sysc.h" - -static const struct rcar_sysc_area r8a7791_areas[] __initconst = { - { "always-on", 0, 0, R8A7791_PD_ALWAYS_ON, -1, PD_ALWAYS_ON }, - { "ca15-scu", 0x180, 0, R8A7791_PD_CA15_SCU, R8A7791_PD_ALWAYS_ON, - PD_SCU }, - { "ca15-cpu0", 0x40, 0, R8A7791_PD_CA15_CPU0, R8A7791_PD_CA15_SCU, - PD_CPU_NOCR }, - { "ca15-cpu1", 0x40, 1, R8A7791_PD_CA15_CPU1, R8A7791_PD_CA15_SCU, - PD_CPU_NOCR }, - { "sh-4a", 0x80, 0, R8A7791_PD_SH_4A, R8A7791_PD_ALWAYS_ON }, - { "sgx", 0xc0, 0, R8A7791_PD_SGX, R8A7791_PD_ALWAYS_ON }, -}; - -const struct rcar_sysc_info r8a7791_sysc_info __initconst = { - .areas = r8a7791_areas, - .num_areas = ARRAY_SIZE(r8a7791_areas), -}; diff --git a/drivers/genpd/renesas/r8a7792-sysc.c b/drivers/genpd/renesas/r8a7792-sysc.c deleted file mode 100644 index 60aae242c43f..000000000000 --- a/drivers/genpd/renesas/r8a7792-sysc.c +++ /dev/null @@ -1,30 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * Renesas R-Car V2H (R8A7792) System Controller - * - * Copyright (C) 2016 Cogent Embedded Inc. - */ - -#include -#include - -#include - -#include "rcar-sysc.h" - -static const struct rcar_sysc_area r8a7792_areas[] __initconst = { - { "always-on", 0, 0, R8A7792_PD_ALWAYS_ON, -1, PD_ALWAYS_ON }, - { "ca15-scu", 0x180, 0, R8A7792_PD_CA15_SCU, R8A7792_PD_ALWAYS_ON, - PD_SCU }, - { "ca15-cpu0", 0x40, 0, R8A7792_PD_CA15_CPU0, R8A7792_PD_CA15_SCU, - PD_CPU_NOCR }, - { "ca15-cpu1", 0x40, 1, R8A7792_PD_CA15_CPU1, R8A7792_PD_CA15_SCU, - PD_CPU_NOCR }, - { "sgx", 0xc0, 0, R8A7792_PD_SGX, R8A7792_PD_ALWAYS_ON }, - { "imp", 0x140, 0, R8A7792_PD_IMP, R8A7792_PD_ALWAYS_ON }, -}; - -const struct rcar_sysc_info r8a7792_sysc_info __initconst = { - .areas = r8a7792_areas, - .num_areas = ARRAY_SIZE(r8a7792_areas), -}; diff --git a/drivers/genpd/renesas/r8a7794-sysc.c b/drivers/genpd/renesas/r8a7794-sysc.c deleted file mode 100644 index 72ef4e85458f..000000000000 --- a/drivers/genpd/renesas/r8a7794-sysc.c +++ /dev/null @@ -1,29 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * Renesas R-Car E2 System Controller - * - * Copyright (C) 2016 Glider bvba - */ - -#include - -#include - -#include "rcar-sysc.h" - -static const struct rcar_sysc_area r8a7794_areas[] __initconst = { - { "always-on", 0, 0, R8A7794_PD_ALWAYS_ON, -1, PD_ALWAYS_ON }, - { "ca7-scu", 0x100, 0, R8A7794_PD_CA7_SCU, R8A7794_PD_ALWAYS_ON, - PD_SCU }, - { "ca7-cpu0", 0x1c0, 0, R8A7794_PD_CA7_CPU0, R8A7794_PD_CA7_SCU, - PD_CPU_NOCR }, - { "ca7-cpu1", 0x1c0, 1, R8A7794_PD_CA7_CPU1, R8A7794_PD_CA7_SCU, - PD_CPU_NOCR }, - { "sh-4a", 0x80, 0, R8A7794_PD_SH_4A, R8A7794_PD_ALWAYS_ON }, - { "sgx", 0xc0, 0, R8A7794_PD_SGX, R8A7794_PD_ALWAYS_ON }, -}; - -const struct rcar_sysc_info r8a7794_sysc_info __initconst = { - .areas = r8a7794_areas, - .num_areas = ARRAY_SIZE(r8a7794_areas), -}; diff --git a/drivers/genpd/renesas/r8a7795-sysc.c b/drivers/genpd/renesas/r8a7795-sysc.c deleted file mode 100644 index cbe1ff0fc583..000000000000 --- a/drivers/genpd/renesas/r8a7795-sysc.c +++ /dev/null @@ -1,86 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * Renesas R-Car H3 System Controller - * - * Copyright (C) 2016-2017 Glider bvba - */ - -#include -#include -#include - -#include - -#include "rcar-sysc.h" - -static struct rcar_sysc_area r8a7795_areas[] __initdata = { - { "always-on", 0, 0, R8A7795_PD_ALWAYS_ON, -1, PD_ALWAYS_ON }, - { "ca57-scu", 0x1c0, 0, R8A7795_PD_CA57_SCU, R8A7795_PD_ALWAYS_ON, - PD_SCU }, - { "ca57-cpu0", 0x80, 0, R8A7795_PD_CA57_CPU0, R8A7795_PD_CA57_SCU, - PD_CPU_NOCR }, - { "ca57-cpu1", 0x80, 1, R8A7795_PD_CA57_CPU1, R8A7795_PD_CA57_SCU, - PD_CPU_NOCR }, - { "ca57-cpu2", 0x80, 2, R8A7795_PD_CA57_CPU2, R8A7795_PD_CA57_SCU, - PD_CPU_NOCR }, - { "ca57-cpu3", 0x80, 3, R8A7795_PD_CA57_CPU3, R8A7795_PD_CA57_SCU, - PD_CPU_NOCR }, - { "ca53-scu", 0x140, 0, R8A7795_PD_CA53_SCU, R8A7795_PD_ALWAYS_ON, - PD_SCU }, - { "ca53-cpu0", 0x200, 0, R8A7795_PD_CA53_CPU0, R8A7795_PD_CA53_SCU, - PD_CPU_NOCR }, - { "ca53-cpu1", 0x200, 1, R8A7795_PD_CA53_CPU1, R8A7795_PD_CA53_SCU, - PD_CPU_NOCR }, - { "ca53-cpu2", 0x200, 2, R8A7795_PD_CA53_CPU2, R8A7795_PD_CA53_SCU, - PD_CPU_NOCR }, - { "ca53-cpu3", 0x200, 3, R8A7795_PD_CA53_CPU3, R8A7795_PD_CA53_SCU, - PD_CPU_NOCR }, - { "a3vp", 0x340, 0, R8A7795_PD_A3VP, R8A7795_PD_ALWAYS_ON }, - { "cr7", 0x240, 0, R8A7795_PD_CR7, R8A7795_PD_ALWAYS_ON }, - { "a3vc", 0x380, 0, R8A7795_PD_A3VC, R8A7795_PD_ALWAYS_ON }, - { "a2vc1", 0x3c0, 1, R8A7795_PD_A2VC1, R8A7795_PD_A3VC }, - { "3dg-a", 0x100, 0, R8A7795_PD_3DG_A, R8A7795_PD_ALWAYS_ON }, - { "3dg-b", 0x100, 1, R8A7795_PD_3DG_B, R8A7795_PD_3DG_A }, - { "3dg-c", 0x100, 2, R8A7795_PD_3DG_C, R8A7795_PD_3DG_B }, - { "3dg-d", 0x100, 3, R8A7795_PD_3DG_D, R8A7795_PD_3DG_C }, - { "3dg-e", 0x100, 4, R8A7795_PD_3DG_E, R8A7795_PD_3DG_D }, - { "a3ir", 0x180, 0, R8A7795_PD_A3IR, R8A7795_PD_ALWAYS_ON }, -}; - - - /* - * Fixups for R-Car H3 revisions - */ - -#define NO_EXTMASK BIT(1) /* Missing SYSCEXTMASK register */ - -static const struct soc_device_attribute r8a7795_quirks_match[] __initconst = { - { - .soc_id = "r8a7795", .revision = "ES2.*", - .data = (void *)(NO_EXTMASK), - }, - { /* sentinel */ } -}; - -static int __init r8a7795_sysc_init(void) -{ - const struct soc_device_attribute *attr; - u32 quirks = 0; - - attr = soc_device_match(r8a7795_quirks_match); - if (attr) - quirks = (uintptr_t)attr->data; - - if (quirks & NO_EXTMASK) - r8a7795_sysc_info.extmask_val = 0; - - return 0; -} - -struct rcar_sysc_info r8a7795_sysc_info __initdata = { - .init = r8a7795_sysc_init, - .areas = r8a7795_areas, - .num_areas = ARRAY_SIZE(r8a7795_areas), - .extmask_offs = 0x2f8, - .extmask_val = BIT(0), -}; diff --git a/drivers/genpd/renesas/r8a7796-sysc.c b/drivers/genpd/renesas/r8a7796-sysc.c deleted file mode 100644 index 471bd5b3b6ad..000000000000 --- a/drivers/genpd/renesas/r8a7796-sysc.c +++ /dev/null @@ -1,67 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * Renesas R-Car M3-W/W+ System Controller - * - * Copyright (C) 2016 Glider bvba - * Copyright (C) 2018-2019 Renesas Electronics Corporation - */ - -#include -#include - -#include - -#include "rcar-sysc.h" - -static struct rcar_sysc_area r8a7796_areas[] __initdata = { - { "always-on", 0, 0, R8A7796_PD_ALWAYS_ON, -1, PD_ALWAYS_ON }, - { "ca57-scu", 0x1c0, 0, R8A7796_PD_CA57_SCU, R8A7796_PD_ALWAYS_ON, - PD_SCU }, - { "ca57-cpu0", 0x80, 0, R8A7796_PD_CA57_CPU0, R8A7796_PD_CA57_SCU, - PD_CPU_NOCR }, - { "ca57-cpu1", 0x80, 1, R8A7796_PD_CA57_CPU1, R8A7796_PD_CA57_SCU, - PD_CPU_NOCR }, - { "ca53-scu", 0x140, 0, R8A7796_PD_CA53_SCU, R8A7796_PD_ALWAYS_ON, - PD_SCU }, - { "ca53-cpu0", 0x200, 0, R8A7796_PD_CA53_CPU0, R8A7796_PD_CA53_SCU, - PD_CPU_NOCR }, - { "ca53-cpu1", 0x200, 1, R8A7796_PD_CA53_CPU1, R8A7796_PD_CA53_SCU, - PD_CPU_NOCR }, - { "ca53-cpu2", 0x200, 2, R8A7796_PD_CA53_CPU2, R8A7796_PD_CA53_SCU, - PD_CPU_NOCR }, - { "ca53-cpu3", 0x200, 3, R8A7796_PD_CA53_CPU3, R8A7796_PD_CA53_SCU, - PD_CPU_NOCR }, - { "cr7", 0x240, 0, R8A7796_PD_CR7, R8A7796_PD_ALWAYS_ON }, - { "a3vc", 0x380, 0, R8A7796_PD_A3VC, R8A7796_PD_ALWAYS_ON }, - { "a2vc0", 0x3c0, 0, R8A7796_PD_A2VC0, R8A7796_PD_A3VC }, - { "a2vc1", 0x3c0, 1, R8A7796_PD_A2VC1, R8A7796_PD_A3VC }, - { "3dg-a", 0x100, 0, R8A7796_PD_3DG_A, R8A7796_PD_ALWAYS_ON }, - { "3dg-b", 0x100, 1, R8A7796_PD_3DG_B, R8A7796_PD_3DG_A }, - { "a3ir", 0x180, 0, R8A7796_PD_A3IR, R8A7796_PD_ALWAYS_ON }, -}; - - -#ifdef CONFIG_SYSC_R8A77960 -const struct rcar_sysc_info r8a77960_sysc_info __initconst = { - .areas = r8a7796_areas, - .num_areas = ARRAY_SIZE(r8a7796_areas), -}; -#endif /* CONFIG_SYSC_R8A77960 */ - -#ifdef CONFIG_SYSC_R8A77961 -static int __init r8a77961_sysc_init(void) -{ - rcar_sysc_nullify(r8a7796_areas, ARRAY_SIZE(r8a7796_areas), - R8A7796_PD_A2VC0); - - return 0; -} - -const struct rcar_sysc_info r8a77961_sysc_info __initconst = { - .init = r8a77961_sysc_init, - .areas = r8a7796_areas, - .num_areas = ARRAY_SIZE(r8a7796_areas), - .extmask_offs = 0x2f8, - .extmask_val = BIT(0), -}; -#endif /* CONFIG_SYSC_R8A77961 */ diff --git a/drivers/genpd/renesas/r8a77965-sysc.c b/drivers/genpd/renesas/r8a77965-sysc.c deleted file mode 100644 index ff0b0d116992..000000000000 --- a/drivers/genpd/renesas/r8a77965-sysc.c +++ /dev/null @@ -1,38 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * Renesas R-Car M3-N System Controller - * Copyright (C) 2018 Jacopo Mondi - * - * Based on Renesas R-Car M3-W System Controller - * Copyright (C) 2016 Glider bvba - */ - -#include -#include - -#include - -#include "rcar-sysc.h" - -static const struct rcar_sysc_area r8a77965_areas[] __initconst = { - { "always-on", 0, 0, R8A77965_PD_ALWAYS_ON, -1, PD_ALWAYS_ON }, - { "ca57-scu", 0x1c0, 0, R8A77965_PD_CA57_SCU, R8A77965_PD_ALWAYS_ON, - PD_SCU }, - { "ca57-cpu0", 0x80, 0, R8A77965_PD_CA57_CPU0, R8A77965_PD_CA57_SCU, - PD_CPU_NOCR }, - { "ca57-cpu1", 0x80, 1, R8A77965_PD_CA57_CPU1, R8A77965_PD_CA57_SCU, - PD_CPU_NOCR }, - { "cr7", 0x240, 0, R8A77965_PD_CR7, R8A77965_PD_ALWAYS_ON }, - { "a3vc", 0x380, 0, R8A77965_PD_A3VC, R8A77965_PD_ALWAYS_ON }, - { "a3vp", 0x340, 0, R8A77965_PD_A3VP, R8A77965_PD_ALWAYS_ON }, - { "a2vc1", 0x3c0, 1, R8A77965_PD_A2VC1, R8A77965_PD_A3VC }, - { "3dg-a", 0x100, 0, R8A77965_PD_3DG_A, R8A77965_PD_ALWAYS_ON }, - { "3dg-b", 0x100, 1, R8A77965_PD_3DG_B, R8A77965_PD_3DG_A }, -}; - -const struct rcar_sysc_info r8a77965_sysc_info __initconst = { - .areas = r8a77965_areas, - .num_areas = ARRAY_SIZE(r8a77965_areas), - .extmask_offs = 0x2f8, - .extmask_val = BIT(0), -}; diff --git a/drivers/genpd/renesas/r8a77970-sysc.c b/drivers/genpd/renesas/r8a77970-sysc.c deleted file mode 100644 index 706258250600..000000000000 --- a/drivers/genpd/renesas/r8a77970-sysc.c +++ /dev/null @@ -1,37 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * Renesas R-Car V3M System Controller - * - * Copyright (C) 2017 Cogent Embedded Inc. - */ - -#include -#include - -#include - -#include "rcar-sysc.h" - -static const struct rcar_sysc_area r8a77970_areas[] __initconst = { - { "always-on", 0, 0, R8A77970_PD_ALWAYS_ON, -1, PD_ALWAYS_ON }, - { "ca53-scu", 0x140, 0, R8A77970_PD_CA53_SCU, R8A77970_PD_ALWAYS_ON, - PD_SCU }, - { "ca53-cpu0", 0x200, 0, R8A77970_PD_CA53_CPU0, R8A77970_PD_CA53_SCU, - PD_CPU_NOCR }, - { "ca53-cpu1", 0x200, 1, R8A77970_PD_CA53_CPU1, R8A77970_PD_CA53_SCU, - PD_CPU_NOCR }, - { "a3ir", 0x180, 0, R8A77970_PD_A3IR, R8A77970_PD_ALWAYS_ON }, - { "a2ir0", 0x400, 0, R8A77970_PD_A2IR0, R8A77970_PD_A3IR }, - { "a2ir1", 0x400, 1, R8A77970_PD_A2IR1, R8A77970_PD_A3IR }, - { "a2dp", 0x400, 2, R8A77970_PD_A2DP, R8A77970_PD_A3IR }, - { "a2cn", 0x400, 3, R8A77970_PD_A2CN, R8A77970_PD_A3IR }, - { "a2sc0", 0x400, 4, R8A77970_PD_A2SC0, R8A77970_PD_A3IR }, - { "a2sc1", 0x400, 5, R8A77970_PD_A2SC1, R8A77970_PD_A3IR }, -}; - -const struct rcar_sysc_info r8a77970_sysc_info __initconst = { - .areas = r8a77970_areas, - .num_areas = ARRAY_SIZE(r8a77970_areas), - .extmask_offs = 0x1b0, - .extmask_val = BIT(0), -}; diff --git a/drivers/genpd/renesas/r8a77980-sysc.c b/drivers/genpd/renesas/r8a77980-sysc.c deleted file mode 100644 index 39ca84a67daa..000000000000 --- a/drivers/genpd/renesas/r8a77980-sysc.c +++ /dev/null @@ -1,54 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * Renesas R-Car V3H System Controller - * - * Copyright (C) 2018 Renesas Electronics Corp. - * Copyright (C) 2018 Cogent Embedded, Inc. - */ - -#include -#include - -#include - -#include "rcar-sysc.h" - -static const struct rcar_sysc_area r8a77980_areas[] __initconst = { - { "always-on", 0, 0, R8A77980_PD_ALWAYS_ON, -1, PD_ALWAYS_ON }, - { "ca53-scu", 0x140, 0, R8A77980_PD_CA53_SCU, R8A77980_PD_ALWAYS_ON, - PD_SCU }, - { "ca53-cpu0", 0x200, 0, R8A77980_PD_CA53_CPU0, R8A77980_PD_CA53_SCU, - PD_CPU_NOCR }, - { "ca53-cpu1", 0x200, 1, R8A77980_PD_CA53_CPU1, R8A77980_PD_CA53_SCU, - PD_CPU_NOCR }, - { "ca53-cpu2", 0x200, 2, R8A77980_PD_CA53_CPU2, R8A77980_PD_CA53_SCU, - PD_CPU_NOCR }, - { "ca53-cpu3", 0x200, 3, R8A77980_PD_CA53_CPU3, R8A77980_PD_CA53_SCU, - PD_CPU_NOCR }, - { "cr7", 0x240, 0, R8A77980_PD_CR7, R8A77980_PD_ALWAYS_ON }, - { "a3ir", 0x180, 0, R8A77980_PD_A3IR, R8A77980_PD_ALWAYS_ON }, - { "a2ir0", 0x400, 0, R8A77980_PD_A2IR0, R8A77980_PD_A3IR }, - { "a2ir1", 0x400, 1, R8A77980_PD_A2IR1, R8A77980_PD_A3IR }, - { "a2ir2", 0x400, 2, R8A77980_PD_A2IR2, R8A77980_PD_A3IR }, - { "a2ir3", 0x400, 3, R8A77980_PD_A2IR3, R8A77980_PD_A3IR }, - { "a2ir4", 0x400, 4, R8A77980_PD_A2IR4, R8A77980_PD_A3IR }, - { "a2ir5", 0x400, 5, R8A77980_PD_A2IR5, R8A77980_PD_A3IR }, - { "a2sc0", 0x400, 6, R8A77980_PD_A2SC0, R8A77980_PD_A3IR }, - { "a2sc1", 0x400, 7, R8A77980_PD_A2SC1, R8A77980_PD_A3IR }, - { "a2sc2", 0x400, 8, R8A77980_PD_A2SC2, R8A77980_PD_A3IR }, - { "a2sc3", 0x400, 9, R8A77980_PD_A2SC3, R8A77980_PD_A3IR }, - { "a2sc4", 0x400, 10, R8A77980_PD_A2SC4, R8A77980_PD_A3IR }, - { "a2dp0", 0x400, 11, R8A77980_PD_A2DP0, R8A77980_PD_A3IR }, - { "a2dp1", 0x400, 12, R8A77980_PD_A2DP1, R8A77980_PD_A3IR }, - { "a2cn", 0x400, 13, R8A77980_PD_A2CN, R8A77980_PD_A3IR }, - { "a3vip0", 0x2c0, 0, R8A77980_PD_A3VIP0, R8A77980_PD_ALWAYS_ON }, - { "a3vip1", 0x300, 0, R8A77980_PD_A3VIP1, R8A77980_PD_ALWAYS_ON }, - { "a3vip2", 0x280, 0, R8A77980_PD_A3VIP2, R8A77980_PD_ALWAYS_ON }, -}; - -const struct rcar_sysc_info r8a77980_sysc_info __initconst = { - .areas = r8a77980_areas, - .num_areas = ARRAY_SIZE(r8a77980_areas), - .extmask_offs = 0x138, - .extmask_val = BIT(0), -}; diff --git a/drivers/genpd/renesas/r8a77990-sysc.c b/drivers/genpd/renesas/r8a77990-sysc.c deleted file mode 100644 index 9f92737dc352..000000000000 --- a/drivers/genpd/renesas/r8a77990-sysc.c +++ /dev/null @@ -1,55 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * Renesas R-Car E3 System Controller - * - * Copyright (C) 2018 Renesas Electronics Corp. - */ - -#include -#include -#include - -#include - -#include "rcar-sysc.h" - -static struct rcar_sysc_area r8a77990_areas[] __initdata = { - { "always-on", 0, 0, R8A77990_PD_ALWAYS_ON, -1, PD_ALWAYS_ON }, - { "ca53-scu", 0x140, 0, R8A77990_PD_CA53_SCU, R8A77990_PD_ALWAYS_ON, - PD_SCU }, - { "ca53-cpu0", 0x200, 0, R8A77990_PD_CA53_CPU0, R8A77990_PD_CA53_SCU, - PD_CPU_NOCR }, - { "ca53-cpu1", 0x200, 1, R8A77990_PD_CA53_CPU1, R8A77990_PD_CA53_SCU, - PD_CPU_NOCR }, - { "cr7", 0x240, 0, R8A77990_PD_CR7, R8A77990_PD_ALWAYS_ON }, - { "a3vc", 0x380, 0, R8A77990_PD_A3VC, R8A77990_PD_ALWAYS_ON }, - { "a2vc1", 0x3c0, 1, R8A77990_PD_A2VC1, R8A77990_PD_A3VC }, - { "3dg-a", 0x100, 0, R8A77990_PD_3DG_A, R8A77990_PD_ALWAYS_ON }, - { "3dg-b", 0x100, 1, R8A77990_PD_3DG_B, R8A77990_PD_3DG_A }, -}; - -/* Fixups for R-Car E3 ES1.0 revision */ -static const struct soc_device_attribute r8a77990[] __initconst = { - { .soc_id = "r8a77990", .revision = "ES1.0" }, - { /* sentinel */ } -}; - -static int __init r8a77990_sysc_init(void) -{ - if (soc_device_match(r8a77990)) { - /* Fix incorrect 3DG hierarchy */ - swap(r8a77990_areas[7], r8a77990_areas[8]); - r8a77990_areas[7].parent = R8A77990_PD_ALWAYS_ON; - r8a77990_areas[8].parent = R8A77990_PD_3DG_B; - } - - return 0; -} - -const struct rcar_sysc_info r8a77990_sysc_info __initconst = { - .init = r8a77990_sysc_init, - .areas = r8a77990_areas, - .num_areas = ARRAY_SIZE(r8a77990_areas), - .extmask_offs = 0x2f8, - .extmask_val = BIT(0), -}; diff --git a/drivers/genpd/renesas/r8a77995-sysc.c b/drivers/genpd/renesas/r8a77995-sysc.c deleted file mode 100644 index efcc67e3d76d..000000000000 --- a/drivers/genpd/renesas/r8a77995-sysc.c +++ /dev/null @@ -1,26 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * Renesas R-Car D3 System Controller - * - * Copyright (C) 2017 Glider bvba - */ - -#include - -#include - -#include "rcar-sysc.h" - -static const struct rcar_sysc_area r8a77995_areas[] __initconst = { - { "always-on", 0, 0, R8A77995_PD_ALWAYS_ON, -1, PD_ALWAYS_ON }, - { "ca53-scu", 0x140, 0, R8A77995_PD_CA53_SCU, R8A77995_PD_ALWAYS_ON, - PD_SCU }, - { "ca53-cpu0", 0x200, 0, R8A77995_PD_CA53_CPU0, R8A77995_PD_CA53_SCU, - PD_CPU_NOCR }, -}; - - -const struct rcar_sysc_info r8a77995_sysc_info __initconst = { - .areas = r8a77995_areas, - .num_areas = ARRAY_SIZE(r8a77995_areas), -}; diff --git a/drivers/genpd/renesas/r8a779a0-sysc.c b/drivers/genpd/renesas/r8a779a0-sysc.c deleted file mode 100644 index 04f1bc322ae7..000000000000 --- a/drivers/genpd/renesas/r8a779a0-sysc.c +++ /dev/null @@ -1,76 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * Renesas R-Car V3U System Controller - * - * Copyright (C) 2020 Renesas Electronics Corp. - */ - -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -#include - -#include "rcar-gen4-sysc.h" - -static struct rcar_gen4_sysc_area r8a779a0_areas[] __initdata = { - { "always-on", R8A779A0_PD_ALWAYS_ON, -1, PD_ALWAYS_ON }, - { "a3e0", R8A779A0_PD_A3E0, R8A779A0_PD_ALWAYS_ON, PD_SCU }, - { "a3e1", R8A779A0_PD_A3E1, R8A779A0_PD_ALWAYS_ON, PD_SCU }, - { "a2e0d0", R8A779A0_PD_A2E0D0, R8A779A0_PD_A3E0, PD_SCU }, - { "a2e0d1", R8A779A0_PD_A2E0D1, R8A779A0_PD_A3E0, PD_SCU }, - { "a2e1d0", R8A779A0_PD_A2E1D0, R8A779A0_PD_A3E1, PD_SCU }, - { "a2e1d1", R8A779A0_PD_A2E1D1, R8A779A0_PD_A3E1, PD_SCU }, - { "a1e0d0c0", R8A779A0_PD_A1E0D0C0, R8A779A0_PD_A2E0D0, PD_CPU_NOCR }, - { "a1e0d0c1", R8A779A0_PD_A1E0D0C1, R8A779A0_PD_A2E0D0, PD_CPU_NOCR }, - { "a1e0d1c0", R8A779A0_PD_A1E0D1C0, R8A779A0_PD_A2E0D1, PD_CPU_NOCR }, - { "a1e0d1c1", R8A779A0_PD_A1E0D1C1, R8A779A0_PD_A2E0D1, PD_CPU_NOCR }, - { "a1e1d0c0", R8A779A0_PD_A1E1D0C0, R8A779A0_PD_A2E1D0, PD_CPU_NOCR }, - { "a1e1d0c1", R8A779A0_PD_A1E1D0C1, R8A779A0_PD_A2E1D0, PD_CPU_NOCR }, - { "a1e1d1c0", R8A779A0_PD_A1E1D1C0, R8A779A0_PD_A2E1D1, PD_CPU_NOCR }, - { "a1e1d1c1", R8A779A0_PD_A1E1D1C1, R8A779A0_PD_A2E1D1, PD_CPU_NOCR }, - { "3dg-a", R8A779A0_PD_3DG_A, R8A779A0_PD_ALWAYS_ON }, - { "3dg-b", R8A779A0_PD_3DG_B, R8A779A0_PD_3DG_A }, - { "a3vip0", R8A779A0_PD_A3VIP0, R8A779A0_PD_ALWAYS_ON }, - { "a3vip1", R8A779A0_PD_A3VIP1, R8A779A0_PD_ALWAYS_ON }, - { "a3vip3", R8A779A0_PD_A3VIP3, R8A779A0_PD_ALWAYS_ON }, - { "a3vip2", R8A779A0_PD_A3VIP2, R8A779A0_PD_ALWAYS_ON }, - { "a3isp01", R8A779A0_PD_A3ISP01, R8A779A0_PD_ALWAYS_ON }, - { "a3isp23", R8A779A0_PD_A3ISP23, R8A779A0_PD_ALWAYS_ON }, - { "a3ir", R8A779A0_PD_A3IR, R8A779A0_PD_ALWAYS_ON }, - { "a2cn0", R8A779A0_PD_A2CN0, R8A779A0_PD_A3IR }, - { "a2imp01", R8A779A0_PD_A2IMP01, R8A779A0_PD_A3IR }, - { "a2dp0", R8A779A0_PD_A2DP0, R8A779A0_PD_A3IR }, - { "a2cv0", R8A779A0_PD_A2CV0, R8A779A0_PD_A3IR }, - { "a2cv1", R8A779A0_PD_A2CV1, R8A779A0_PD_A3IR }, - { "a2cv4", R8A779A0_PD_A2CV4, R8A779A0_PD_A3IR }, - { "a2cv6", R8A779A0_PD_A2CV6, R8A779A0_PD_A3IR }, - { "a2cn2", R8A779A0_PD_A2CN2, R8A779A0_PD_A3IR }, - { "a2imp23", R8A779A0_PD_A2IMP23, R8A779A0_PD_A3IR }, - { "a2dp1", R8A779A0_PD_A2DP1, R8A779A0_PD_A3IR }, - { "a2cv2", R8A779A0_PD_A2CV2, R8A779A0_PD_A3IR }, - { "a2cv3", R8A779A0_PD_A2CV3, R8A779A0_PD_A3IR }, - { "a2cv5", R8A779A0_PD_A2CV5, R8A779A0_PD_A3IR }, - { "a2cv7", R8A779A0_PD_A2CV7, R8A779A0_PD_A3IR }, - { "a2cn1", R8A779A0_PD_A2CN1, R8A779A0_PD_A3IR }, - { "a1cnn0", R8A779A0_PD_A1CNN0, R8A779A0_PD_A2CN0 }, - { "a1cnn2", R8A779A0_PD_A1CNN2, R8A779A0_PD_A2CN2 }, - { "a1dsp0", R8A779A0_PD_A1DSP0, R8A779A0_PD_A2CN2 }, - { "a1cnn1", R8A779A0_PD_A1CNN1, R8A779A0_PD_A2CN1 }, - { "a1dsp1", R8A779A0_PD_A1DSP1, R8A779A0_PD_A2CN1 }, -}; - -const struct rcar_gen4_sysc_info r8a779a0_sysc_info __initconst = { - .areas = r8a779a0_areas, - .num_areas = ARRAY_SIZE(r8a779a0_areas), -}; diff --git a/drivers/genpd/renesas/r8a779f0-sysc.c b/drivers/genpd/renesas/r8a779f0-sysc.c deleted file mode 100644 index 5602aa6bd7ed..000000000000 --- a/drivers/genpd/renesas/r8a779f0-sysc.c +++ /dev/null @@ -1,47 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * Renesas R-Car S4-8 System Controller - * - * Copyright (C) 2021 Renesas Electronics Corp. - */ - -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -#include - -#include "rcar-gen4-sysc.h" - -static struct rcar_gen4_sysc_area r8a779f0_areas[] __initdata = { - { "always-on", R8A779F0_PD_ALWAYS_ON, -1, PD_ALWAYS_ON }, - { "a3e0", R8A779F0_PD_A3E0, R8A779F0_PD_ALWAYS_ON, PD_SCU }, - { "a3e1", R8A779F0_PD_A3E1, R8A779F0_PD_ALWAYS_ON, PD_SCU }, - { "a2e0d0", R8A779F0_PD_A2E0D0, R8A779F0_PD_A3E0, PD_SCU }, - { "a2e0d1", R8A779F0_PD_A2E0D1, R8A779F0_PD_A3E0, PD_SCU }, - { "a2e1d0", R8A779F0_PD_A2E1D0, R8A779F0_PD_A3E1, PD_SCU }, - { "a2e1d1", R8A779F0_PD_A2E1D1, R8A779F0_PD_A3E1, PD_SCU }, - { "a1e0d0c0", R8A779F0_PD_A1E0D0C0, R8A779F0_PD_A2E0D0, PD_CPU_NOCR }, - { "a1e0d0c1", R8A779F0_PD_A1E0D0C1, R8A779F0_PD_A2E0D0, PD_CPU_NOCR }, - { "a1e0d1c0", R8A779F0_PD_A1E0D1C0, R8A779F0_PD_A2E0D1, PD_CPU_NOCR }, - { "a1e0d1c1", R8A779F0_PD_A1E0D1C1, R8A779F0_PD_A2E0D1, PD_CPU_NOCR }, - { "a1e1d0c0", R8A779F0_PD_A1E1D0C0, R8A779F0_PD_A2E1D0, PD_CPU_NOCR }, - { "a1e1d0c1", R8A779F0_PD_A1E1D0C1, R8A779F0_PD_A2E1D0, PD_CPU_NOCR }, - { "a1e1d1c0", R8A779F0_PD_A1E1D1C0, R8A779F0_PD_A2E1D1, PD_CPU_NOCR }, - { "a1e1d1c1", R8A779F0_PD_A1E1D1C1, R8A779F0_PD_A2E1D1, PD_CPU_NOCR }, -}; - -const struct rcar_gen4_sysc_info r8a779f0_sysc_info __initconst = { - .areas = r8a779f0_areas, - .num_areas = ARRAY_SIZE(r8a779f0_areas), -}; diff --git a/drivers/genpd/renesas/r8a779g0-sysc.c b/drivers/genpd/renesas/r8a779g0-sysc.c deleted file mode 100644 index b932eba1b804..000000000000 --- a/drivers/genpd/renesas/r8a779g0-sysc.c +++ /dev/null @@ -1,63 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * Renesas R-Car V4H System Controller - * - * Copyright (C) 2022 Renesas Electronics Corp. - */ - -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -#include - -#include "rcar-gen4-sysc.h" - -static struct rcar_gen4_sysc_area r8a779g0_areas[] __initdata = { - { "always-on", R8A779G0_PD_ALWAYS_ON, -1, PD_ALWAYS_ON }, - { "a3e0", R8A779G0_PD_A3E0, R8A779G0_PD_ALWAYS_ON, PD_SCU }, - { "a2e0d0", R8A779G0_PD_A2E0D0, R8A779G0_PD_A3E0, PD_SCU }, - { "a2e0d1", R8A779G0_PD_A2E0D1, R8A779G0_PD_A3E0, PD_SCU }, - { "a1e0d0c0", R8A779G0_PD_A1E0D0C0, R8A779G0_PD_A2E0D0, PD_CPU_NOCR }, - { "a1e0d0c1", R8A779G0_PD_A1E0D0C1, R8A779G0_PD_A2E0D0, PD_CPU_NOCR }, - { "a1e0d1c0", R8A779G0_PD_A1E0D1C0, R8A779G0_PD_A2E0D1, PD_CPU_NOCR }, - { "a1e0d1c1", R8A779G0_PD_A1E0D1C1, R8A779G0_PD_A2E0D1, PD_CPU_NOCR }, - { "a33dga", R8A779G0_PD_A33DGA, R8A779G0_PD_ALWAYS_ON }, - { "a23dgb", R8A779G0_PD_A23DGB, R8A779G0_PD_A33DGA }, - { "a3vip0", R8A779G0_PD_A3VIP0, R8A779G0_PD_ALWAYS_ON }, - { "a3vip1", R8A779G0_PD_A3VIP1, R8A779G0_PD_ALWAYS_ON }, - { "a3vip2", R8A779G0_PD_A3VIP2, R8A779G0_PD_ALWAYS_ON }, - { "a3dul", R8A779G0_PD_A3DUL, R8A779G0_PD_ALWAYS_ON }, - { "a3isp0", R8A779G0_PD_A3ISP0, R8A779G0_PD_ALWAYS_ON }, - { "a3isp1", R8A779G0_PD_A3ISP1, R8A779G0_PD_ALWAYS_ON }, - { "a3ir", R8A779G0_PD_A3IR, R8A779G0_PD_ALWAYS_ON }, - { "a2cn0", R8A779G0_PD_A2CN0, R8A779G0_PD_A3IR }, - { "a1cnn0", R8A779G0_PD_A1CNN0, R8A779G0_PD_A2CN0 }, - { "a1dsp0", R8A779G0_PD_A1DSP0, R8A779G0_PD_A2CN0 }, - { "a1dsp1", R8A779G0_PD_A1DSP1, R8A779G0_PD_A2CN0 }, - { "a1dsp2", R8A779G0_PD_A1DSP2, R8A779G0_PD_A2CN0 }, - { "a1dsp3", R8A779G0_PD_A1DSP3, R8A779G0_PD_A2CN0 }, - { "a2imp01", R8A779G0_PD_A2IMP01, R8A779G0_PD_A3IR }, - { "a2imp23", R8A779G0_PD_A2IMP23, R8A779G0_PD_A3IR }, - { "a2psc", R8A779G0_PD_A2PSC, R8A779G0_PD_A3IR }, - { "a2dma", R8A779G0_PD_A2DMA, R8A779G0_PD_A3IR }, - { "a2cv0", R8A779G0_PD_A2CV0, R8A779G0_PD_A3IR }, - { "a2cv1", R8A779G0_PD_A2CV1, R8A779G0_PD_A3IR }, - { "a2cv2", R8A779G0_PD_A2CV2, R8A779G0_PD_A3IR }, - { "a2cv3", R8A779G0_PD_A2CV3, R8A779G0_PD_A3IR }, -}; - -const struct rcar_gen4_sysc_info r8a779g0_sysc_info __initconst = { - .areas = r8a779g0_areas, - .num_areas = ARRAY_SIZE(r8a779g0_areas), -}; diff --git a/drivers/genpd/renesas/rcar-gen4-sysc.c b/drivers/genpd/renesas/rcar-gen4-sysc.c deleted file mode 100644 index 9e5e6e077abc..000000000000 --- a/drivers/genpd/renesas/rcar-gen4-sysc.c +++ /dev/null @@ -1,379 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * R-Car Gen4 SYSC Power management support - * - * Copyright (C) 2021 Renesas Electronics Corp. - */ - -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -#include "rcar-gen4-sysc.h" - -/* SYSC Common */ -#define SYSCSR 0x000 /* SYSC Status Register */ -#define SYSCPONSR(x) (0x800 + ((x) * 0x4)) /* Power-ON Status Register 0 */ -#define SYSCPOFFSR(x) (0x808 + ((x) * 0x4)) /* Power-OFF Status Register */ -#define SYSCISCR(x) (0x810 + ((x) * 0x4)) /* Interrupt Status/Clear Register */ -#define SYSCIER(x) (0x820 + ((x) * 0x4)) /* Interrupt Enable Register */ -#define SYSCIMR(x) (0x830 + ((x) * 0x4)) /* Interrupt Mask Register */ - -/* Power Domain Registers */ -#define PDRSR(n) (0x1000 + ((n) * 0x40)) -#define PDRONCR(n) (0x1004 + ((n) * 0x40)) -#define PDROFFCR(n) (0x1008 + ((n) * 0x40)) -#define PDRESR(n) (0x100C + ((n) * 0x40)) - -/* PWRON/PWROFF */ -#define PWRON_PWROFF BIT(0) /* Power-ON/OFF request */ - -/* PDRESR */ -#define PDRESR_ERR BIT(0) - -/* PDRSR */ -#define PDRSR_OFF BIT(0) /* Power-OFF state */ -#define PDRSR_ON BIT(4) /* Power-ON state */ -#define PDRSR_OFF_STATE BIT(8) /* Processing Power-OFF sequence */ -#define PDRSR_ON_STATE BIT(12) /* Processing Power-ON sequence */ - -#define SYSCSR_BUSY GENMASK(1, 0) /* All bit sets is not busy */ - -#define SYSCSR_TIMEOUT 10000 -#define SYSCSR_DELAY_US 10 - -#define PDRESR_RETRIES 1000 -#define PDRESR_DELAY_US 10 - -#define SYSCISR_TIMEOUT 10000 -#define SYSCISR_DELAY_US 10 - -#define RCAR_GEN4_PD_ALWAYS_ON 64 -#define NUM_DOMAINS_EACH_REG BITS_PER_TYPE(u32) - -static void __iomem *rcar_gen4_sysc_base; -static DEFINE_SPINLOCK(rcar_gen4_sysc_lock); /* SMP CPUs + I/O devices */ - -static int rcar_gen4_sysc_pwr_on_off(u8 pdr, bool on) -{ - unsigned int reg_offs; - u32 val; - int ret; - - if (on) - reg_offs = PDRONCR(pdr); - else - reg_offs = PDROFFCR(pdr); - - /* Wait until SYSC is ready to accept a power request */ - ret = readl_poll_timeout_atomic(rcar_gen4_sysc_base + SYSCSR, val, - (val & SYSCSR_BUSY) == SYSCSR_BUSY, - SYSCSR_DELAY_US, SYSCSR_TIMEOUT); - if (ret < 0) - return -EAGAIN; - - /* Submit power shutoff or power resume request */ - iowrite32(PWRON_PWROFF, rcar_gen4_sysc_base + reg_offs); - - return 0; -} - -static int clear_irq_flags(unsigned int reg_idx, unsigned int isr_mask) -{ - u32 val; - int ret; - - iowrite32(isr_mask, rcar_gen4_sysc_base + SYSCISCR(reg_idx)); - - ret = readl_poll_timeout_atomic(rcar_gen4_sysc_base + SYSCISCR(reg_idx), - val, !(val & isr_mask), - SYSCISR_DELAY_US, SYSCISR_TIMEOUT); - if (ret < 0) { - pr_err("\n %s : Can not clear IRQ flags in SYSCISCR", __func__); - return -EIO; - } - - return 0; -} - -static int rcar_gen4_sysc_power(u8 pdr, bool on) -{ - unsigned int isr_mask; - unsigned int reg_idx, bit_idx; - unsigned int status; - unsigned long flags; - int ret = 0; - u32 val; - int k; - - spin_lock_irqsave(&rcar_gen4_sysc_lock, flags); - - reg_idx = pdr / NUM_DOMAINS_EACH_REG; - bit_idx = pdr % NUM_DOMAINS_EACH_REG; - - isr_mask = BIT(bit_idx); - - /* - * The interrupt source needs to be enabled, but masked, to prevent the - * CPU from receiving it. - */ - iowrite32(ioread32(rcar_gen4_sysc_base + SYSCIER(reg_idx)) | isr_mask, - rcar_gen4_sysc_base + SYSCIER(reg_idx)); - iowrite32(ioread32(rcar_gen4_sysc_base + SYSCIMR(reg_idx)) | isr_mask, - rcar_gen4_sysc_base + SYSCIMR(reg_idx)); - - ret = clear_irq_flags(reg_idx, isr_mask); - if (ret) - goto out; - - /* Submit power shutoff or resume request until it was accepted */ - for (k = 0; k < PDRESR_RETRIES; k++) { - ret = rcar_gen4_sysc_pwr_on_off(pdr, on); - if (ret) - goto out; - - status = ioread32(rcar_gen4_sysc_base + PDRESR(pdr)); - if (!(status & PDRESR_ERR)) - break; - - udelay(PDRESR_DELAY_US); - } - - if (k == PDRESR_RETRIES) { - ret = -EIO; - goto out; - } - - /* Wait until the power shutoff or resume request has completed * */ - ret = readl_poll_timeout_atomic(rcar_gen4_sysc_base + SYSCISCR(reg_idx), - val, (val & isr_mask), - SYSCISR_DELAY_US, SYSCISR_TIMEOUT); - if (ret < 0) { - ret = -EIO; - goto out; - } - - /* Clear interrupt flags */ - ret = clear_irq_flags(reg_idx, isr_mask); - if (ret) - goto out; - - out: - spin_unlock_irqrestore(&rcar_gen4_sysc_lock, flags); - - pr_debug("sysc power %s domain %d: %08x -> %d\n", on ? "on" : "off", - pdr, ioread32(rcar_gen4_sysc_base + SYSCISCR(reg_idx)), ret); - return ret; -} - -static bool rcar_gen4_sysc_power_is_off(u8 pdr) -{ - unsigned int st; - - st = ioread32(rcar_gen4_sysc_base + PDRSR(pdr)); - - if (st & PDRSR_OFF) - return true; - - return false; -} - -struct rcar_gen4_sysc_pd { - struct generic_pm_domain genpd; - u8 pdr; - unsigned int flags; - char name[]; -}; - -static inline struct rcar_gen4_sysc_pd *to_rcar_gen4_pd(struct generic_pm_domain *d) -{ - return container_of(d, struct rcar_gen4_sysc_pd, genpd); -} - -static int rcar_gen4_sysc_pd_power_off(struct generic_pm_domain *genpd) -{ - struct rcar_gen4_sysc_pd *pd = to_rcar_gen4_pd(genpd); - - pr_debug("%s: %s\n", __func__, genpd->name); - return rcar_gen4_sysc_power(pd->pdr, false); -} - -static int rcar_gen4_sysc_pd_power_on(struct generic_pm_domain *genpd) -{ - struct rcar_gen4_sysc_pd *pd = to_rcar_gen4_pd(genpd); - - pr_debug("%s: %s\n", __func__, genpd->name); - return rcar_gen4_sysc_power(pd->pdr, true); -} - -static int __init rcar_gen4_sysc_pd_setup(struct rcar_gen4_sysc_pd *pd) -{ - struct generic_pm_domain *genpd = &pd->genpd; - const char *name = pd->genpd.name; - int error; - - if (pd->flags & PD_CPU) { - /* - * This domain contains a CPU core and therefore it should - * only be turned off if the CPU is not in use. - */ - pr_debug("PM domain %s contains %s\n", name, "CPU"); - genpd->flags |= GENPD_FLAG_ALWAYS_ON; - } else if (pd->flags & PD_SCU) { - /* - * This domain contains an SCU and cache-controller, and - * therefore it should only be turned off if the CPU cores are - * not in use. - */ - pr_debug("PM domain %s contains %s\n", name, "SCU"); - genpd->flags |= GENPD_FLAG_ALWAYS_ON; - } else if (pd->flags & PD_NO_CR) { - /* - * This domain cannot be turned off. - */ - genpd->flags |= GENPD_FLAG_ALWAYS_ON; - } - - if (!(pd->flags & (PD_CPU | PD_SCU))) { - /* Enable Clock Domain for I/O devices */ - genpd->flags |= GENPD_FLAG_PM_CLK | GENPD_FLAG_ACTIVE_WAKEUP; - genpd->attach_dev = cpg_mssr_attach_dev; - genpd->detach_dev = cpg_mssr_detach_dev; - } - - genpd->power_off = rcar_gen4_sysc_pd_power_off; - genpd->power_on = rcar_gen4_sysc_pd_power_on; - - if (pd->flags & (PD_CPU | PD_NO_CR)) { - /* Skip CPUs (handled by SMP code) and areas without control */ - pr_debug("%s: Not touching %s\n", __func__, genpd->name); - goto finalize; - } - - if (!rcar_gen4_sysc_power_is_off(pd->pdr)) { - pr_debug("%s: %s is already powered\n", __func__, genpd->name); - goto finalize; - } - - rcar_gen4_sysc_power(pd->pdr, true); - -finalize: - error = pm_genpd_init(genpd, &simple_qos_governor, false); - if (error) - pr_err("Failed to init PM domain %s: %d\n", name, error); - - return error; -} - -static const struct of_device_id rcar_gen4_sysc_matches[] __initconst = { -#ifdef CONFIG_SYSC_R8A779A0 - { .compatible = "renesas,r8a779a0-sysc", .data = &r8a779a0_sysc_info }, -#endif -#ifdef CONFIG_SYSC_R8A779F0 - { .compatible = "renesas,r8a779f0-sysc", .data = &r8a779f0_sysc_info }, -#endif -#ifdef CONFIG_SYSC_R8A779G0 - { .compatible = "renesas,r8a779g0-sysc", .data = &r8a779g0_sysc_info }, -#endif - { /* sentinel */ } -}; - -struct rcar_gen4_pm_domains { - struct genpd_onecell_data onecell_data; - struct generic_pm_domain *domains[RCAR_GEN4_PD_ALWAYS_ON + 1]; -}; - -static struct genpd_onecell_data *rcar_gen4_sysc_onecell_data; - -static int __init rcar_gen4_sysc_pd_init(void) -{ - const struct rcar_gen4_sysc_info *info; - const struct of_device_id *match; - struct rcar_gen4_pm_domains *domains; - struct device_node *np; - void __iomem *base; - unsigned int i; - int error; - - np = of_find_matching_node_and_match(NULL, rcar_gen4_sysc_matches, &match); - if (!np) - return -ENODEV; - - info = match->data; - - base = of_iomap(np, 0); - if (!base) { - pr_warn("%pOF: Cannot map regs\n", np); - error = -ENOMEM; - goto out_put; - } - - rcar_gen4_sysc_base = base; - - domains = kzalloc(sizeof(*domains), GFP_KERNEL); - if (!domains) { - error = -ENOMEM; - goto out_put; - } - - domains->onecell_data.domains = domains->domains; - domains->onecell_data.num_domains = ARRAY_SIZE(domains->domains); - rcar_gen4_sysc_onecell_data = &domains->onecell_data; - - for (i = 0; i < info->num_areas; i++) { - const struct rcar_gen4_sysc_area *area = &info->areas[i]; - struct rcar_gen4_sysc_pd *pd; - size_t n; - - if (!area->name) { - /* Skip NULLified area */ - continue; - } - - n = strlen(area->name) + 1; - pd = kzalloc(sizeof(*pd) + n, GFP_KERNEL); - if (!pd) { - error = -ENOMEM; - goto out_put; - } - - memcpy(pd->name, area->name, n); - pd->genpd.name = pd->name; - pd->pdr = area->pdr; - pd->flags = area->flags; - - error = rcar_gen4_sysc_pd_setup(pd); - if (error) - goto out_put; - - domains->domains[area->pdr] = &pd->genpd; - - if (area->parent < 0) - continue; - - error = pm_genpd_add_subdomain(domains->domains[area->parent], - &pd->genpd); - if (error) { - pr_warn("Failed to add PM subdomain %s to parent %u\n", - area->name, area->parent); - goto out_put; - } - } - - error = of_genpd_add_provider_onecell(np, &domains->onecell_data); - -out_put: - of_node_put(np); - return error; -} -early_initcall(rcar_gen4_sysc_pd_init); diff --git a/drivers/genpd/renesas/rcar-gen4-sysc.h b/drivers/genpd/renesas/rcar-gen4-sysc.h deleted file mode 100644 index 388cfa8f8f9f..000000000000 --- a/drivers/genpd/renesas/rcar-gen4-sysc.h +++ /dev/null @@ -1,44 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 */ -/* - * R-Car Gen4 System Controller - * - * Copyright (C) 2021 Renesas Electronics Corp. - */ -#ifndef __SOC_RENESAS_RCAR_GEN4_SYSC_H__ -#define __SOC_RENESAS_RCAR_GEN4_SYSC_H__ - -#include - -/* - * Power Domain flags - */ -#define PD_CPU BIT(0) /* Area contains main CPU core */ -#define PD_SCU BIT(1) /* Area contains SCU and L2 cache */ -#define PD_NO_CR BIT(2) /* Area lacks PWR{ON,OFF}CR registers */ - -#define PD_CPU_NOCR (PD_CPU | PD_NO_CR) /* CPU area lacks CR */ -#define PD_ALWAYS_ON PD_NO_CR /* Always-on area */ - -/* - * Description of a Power Area - */ -struct rcar_gen4_sysc_area { - const char *name; - u8 pdr; /* PDRn */ - s8 parent; /* -1 if none */ - u8 flags; /* See PD_* */ -}; - -/* - * SoC-specific Power Area Description - */ -struct rcar_gen4_sysc_info { - const struct rcar_gen4_sysc_area *areas; - unsigned int num_areas; -}; - -extern const struct rcar_gen4_sysc_info r8a779a0_sysc_info; -extern const struct rcar_gen4_sysc_info r8a779f0_sysc_info; -extern const struct rcar_gen4_sysc_info r8a779g0_sysc_info; - -#endif /* __SOC_RENESAS_RCAR_GEN4_SYSC_H__ */ diff --git a/drivers/genpd/renesas/rcar-sysc.c b/drivers/genpd/renesas/rcar-sysc.c deleted file mode 100644 index eed47696e825..000000000000 --- a/drivers/genpd/renesas/rcar-sysc.c +++ /dev/null @@ -1,494 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * R-Car SYSC Power management support - * - * Copyright (C) 2014 Magnus Damm - * Copyright (C) 2015-2017 Glider bvba - */ - -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -#include "rcar-sysc.h" - -/* SYSC Common */ -#define SYSCSR 0x00 /* SYSC Status Register */ -#define SYSCISR 0x04 /* Interrupt Status Register */ -#define SYSCISCR 0x08 /* Interrupt Status Clear Register */ -#define SYSCIER 0x0c /* Interrupt Enable Register */ -#define SYSCIMR 0x10 /* Interrupt Mask Register */ - -/* SYSC Status Register */ -#define SYSCSR_PONENB 1 /* Ready for power resume requests */ -#define SYSCSR_POFFENB 0 /* Ready for power shutoff requests */ - -/* - * Power Control Register Offsets inside the register block for each domain - * Note: The "CR" registers for ARM cores exist on H1 only - * Use WFI to power off, CPG/APMU to resume ARM cores on R-Car Gen2 - * Use PSCI on R-Car Gen3 - */ -#define PWRSR_OFFS 0x00 /* Power Status Register */ -#define PWROFFCR_OFFS 0x04 /* Power Shutoff Control Register */ -#define PWROFFSR_OFFS 0x08 /* Power Shutoff Status Register */ -#define PWRONCR_OFFS 0x0c /* Power Resume Control Register */ -#define PWRONSR_OFFS 0x10 /* Power Resume Status Register */ -#define PWRER_OFFS 0x14 /* Power Shutoff/Resume Error */ - - -#define SYSCSR_TIMEOUT 100 -#define SYSCSR_DELAY_US 1 - -#define PWRER_RETRIES 100 -#define PWRER_DELAY_US 1 - -#define SYSCISR_TIMEOUT 1000 -#define SYSCISR_DELAY_US 1 - -#define RCAR_PD_ALWAYS_ON 32 /* Always-on power area */ - -struct rcar_sysc_ch { - u16 chan_offs; - u8 chan_bit; - u8 isr_bit; -}; - -static void __iomem *rcar_sysc_base; -static DEFINE_SPINLOCK(rcar_sysc_lock); /* SMP CPUs + I/O devices */ -static u32 rcar_sysc_extmask_offs, rcar_sysc_extmask_val; - -static int rcar_sysc_pwr_on_off(const struct rcar_sysc_ch *sysc_ch, bool on) -{ - unsigned int sr_bit, reg_offs; - u32 val; - int ret; - - if (on) { - sr_bit = SYSCSR_PONENB; - reg_offs = PWRONCR_OFFS; - } else { - sr_bit = SYSCSR_POFFENB; - reg_offs = PWROFFCR_OFFS; - } - - /* Wait until SYSC is ready to accept a power request */ - ret = readl_poll_timeout_atomic(rcar_sysc_base + SYSCSR, val, - val & BIT(sr_bit), SYSCSR_DELAY_US, - SYSCSR_TIMEOUT); - if (ret) - return -EAGAIN; - - /* Submit power shutoff or power resume request */ - iowrite32(BIT(sysc_ch->chan_bit), - rcar_sysc_base + sysc_ch->chan_offs + reg_offs); - - return 0; -} - -static int rcar_sysc_power(const struct rcar_sysc_ch *sysc_ch, bool on) -{ - unsigned int isr_mask = BIT(sysc_ch->isr_bit); - unsigned int chan_mask = BIT(sysc_ch->chan_bit); - unsigned int status, k; - unsigned long flags; - int ret; - - spin_lock_irqsave(&rcar_sysc_lock, flags); - - /* - * Mask external power requests for CPU or 3DG domains - */ - if (rcar_sysc_extmask_val) { - iowrite32(rcar_sysc_extmask_val, - rcar_sysc_base + rcar_sysc_extmask_offs); - } - - /* - * The interrupt source needs to be enabled, but masked, to prevent the - * CPU from receiving it. - */ - iowrite32(ioread32(rcar_sysc_base + SYSCIMR) | isr_mask, - rcar_sysc_base + SYSCIMR); - iowrite32(ioread32(rcar_sysc_base + SYSCIER) | isr_mask, - rcar_sysc_base + SYSCIER); - - iowrite32(isr_mask, rcar_sysc_base + SYSCISCR); - - /* Submit power shutoff or resume request until it was accepted */ - for (k = 0; k < PWRER_RETRIES; k++) { - ret = rcar_sysc_pwr_on_off(sysc_ch, on); - if (ret) - goto out; - - status = ioread32(rcar_sysc_base + - sysc_ch->chan_offs + PWRER_OFFS); - if (!(status & chan_mask)) - break; - - udelay(PWRER_DELAY_US); - } - - if (k == PWRER_RETRIES) { - ret = -EIO; - goto out; - } - - /* Wait until the power shutoff or resume request has completed * */ - ret = readl_poll_timeout_atomic(rcar_sysc_base + SYSCISR, status, - status & isr_mask, SYSCISR_DELAY_US, - SYSCISR_TIMEOUT); - if (ret) - ret = -EIO; - - iowrite32(isr_mask, rcar_sysc_base + SYSCISCR); - - out: - if (rcar_sysc_extmask_val) - iowrite32(0, rcar_sysc_base + rcar_sysc_extmask_offs); - - spin_unlock_irqrestore(&rcar_sysc_lock, flags); - - pr_debug("sysc power %s domain %d: %08x -> %d\n", on ? "on" : "off", - sysc_ch->isr_bit, ioread32(rcar_sysc_base + SYSCISR), ret); - return ret; -} - -static bool rcar_sysc_power_is_off(const struct rcar_sysc_ch *sysc_ch) -{ - unsigned int st; - - st = ioread32(rcar_sysc_base + sysc_ch->chan_offs + PWRSR_OFFS); - if (st & BIT(sysc_ch->chan_bit)) - return true; - - return false; -} - -struct rcar_sysc_pd { - struct generic_pm_domain genpd; - struct rcar_sysc_ch ch; - unsigned int flags; - char name[]; -}; - -static inline struct rcar_sysc_pd *to_rcar_pd(struct generic_pm_domain *d) -{ - return container_of(d, struct rcar_sysc_pd, genpd); -} - -static int rcar_sysc_pd_power_off(struct generic_pm_domain *genpd) -{ - struct rcar_sysc_pd *pd = to_rcar_pd(genpd); - - pr_debug("%s: %s\n", __func__, genpd->name); - return rcar_sysc_power(&pd->ch, false); -} - -static int rcar_sysc_pd_power_on(struct generic_pm_domain *genpd) -{ - struct rcar_sysc_pd *pd = to_rcar_pd(genpd); - - pr_debug("%s: %s\n", __func__, genpd->name); - return rcar_sysc_power(&pd->ch, true); -} - -static bool has_cpg_mstp; - -static int __init rcar_sysc_pd_setup(struct rcar_sysc_pd *pd) -{ - struct generic_pm_domain *genpd = &pd->genpd; - const char *name = pd->genpd.name; - int error; - - if (pd->flags & PD_CPU) { - /* - * This domain contains a CPU core and therefore it should - * only be turned off if the CPU is not in use. - */ - pr_debug("PM domain %s contains %s\n", name, "CPU"); - genpd->flags |= GENPD_FLAG_ALWAYS_ON; - } else if (pd->flags & PD_SCU) { - /* - * This domain contains an SCU and cache-controller, and - * therefore it should only be turned off if the CPU cores are - * not in use. - */ - pr_debug("PM domain %s contains %s\n", name, "SCU"); - genpd->flags |= GENPD_FLAG_ALWAYS_ON; - } else if (pd->flags & PD_NO_CR) { - /* - * This domain cannot be turned off. - */ - genpd->flags |= GENPD_FLAG_ALWAYS_ON; - } - - if (!(pd->flags & (PD_CPU | PD_SCU))) { - /* Enable Clock Domain for I/O devices */ - genpd->flags |= GENPD_FLAG_PM_CLK | GENPD_FLAG_ACTIVE_WAKEUP; - if (has_cpg_mstp) { - genpd->attach_dev = cpg_mstp_attach_dev; - genpd->detach_dev = cpg_mstp_detach_dev; - } else { - genpd->attach_dev = cpg_mssr_attach_dev; - genpd->detach_dev = cpg_mssr_detach_dev; - } - } - - genpd->power_off = rcar_sysc_pd_power_off; - genpd->power_on = rcar_sysc_pd_power_on; - - if (pd->flags & (PD_CPU | PD_NO_CR)) { - /* Skip CPUs (handled by SMP code) and areas without control */ - pr_debug("%s: Not touching %s\n", __func__, genpd->name); - goto finalize; - } - - if (!rcar_sysc_power_is_off(&pd->ch)) { - pr_debug("%s: %s is already powered\n", __func__, genpd->name); - goto finalize; - } - - rcar_sysc_power(&pd->ch, true); - -finalize: - error = pm_genpd_init(genpd, &simple_qos_governor, false); - if (error) - pr_err("Failed to init PM domain %s: %d\n", name, error); - - return error; -} - -static const struct of_device_id rcar_sysc_matches[] __initconst = { -#ifdef CONFIG_SYSC_R8A7742 - { .compatible = "renesas,r8a7742-sysc", .data = &r8a7742_sysc_info }, -#endif -#ifdef CONFIG_SYSC_R8A7743 - { .compatible = "renesas,r8a7743-sysc", .data = &r8a7743_sysc_info }, - /* RZ/G1N is identical to RZ/G2M w.r.t. power domains. */ - { .compatible = "renesas,r8a7744-sysc", .data = &r8a7743_sysc_info }, -#endif -#ifdef CONFIG_SYSC_R8A7745 - { .compatible = "renesas,r8a7745-sysc", .data = &r8a7745_sysc_info }, -#endif -#ifdef CONFIG_SYSC_R8A77470 - { .compatible = "renesas,r8a77470-sysc", .data = &r8a77470_sysc_info }, -#endif -#ifdef CONFIG_SYSC_R8A774A1 - { .compatible = "renesas,r8a774a1-sysc", .data = &r8a774a1_sysc_info }, -#endif -#ifdef CONFIG_SYSC_R8A774B1 - { .compatible = "renesas,r8a774b1-sysc", .data = &r8a774b1_sysc_info }, -#endif -#ifdef CONFIG_SYSC_R8A774C0 - { .compatible = "renesas,r8a774c0-sysc", .data = &r8a774c0_sysc_info }, -#endif -#ifdef CONFIG_SYSC_R8A774E1 - { .compatible = "renesas,r8a774e1-sysc", .data = &r8a774e1_sysc_info }, -#endif -#ifdef CONFIG_SYSC_R8A7779 - { .compatible = "renesas,r8a7779-sysc", .data = &r8a7779_sysc_info }, -#endif -#ifdef CONFIG_SYSC_R8A7790 - { .compatible = "renesas,r8a7790-sysc", .data = &r8a7790_sysc_info }, -#endif -#ifdef CONFIG_SYSC_R8A7791 - { .compatible = "renesas,r8a7791-sysc", .data = &r8a7791_sysc_info }, - /* R-Car M2-N is identical to R-Car M2-W w.r.t. power domains. */ - { .compatible = "renesas,r8a7793-sysc", .data = &r8a7791_sysc_info }, -#endif -#ifdef CONFIG_SYSC_R8A7792 - { .compatible = "renesas,r8a7792-sysc", .data = &r8a7792_sysc_info }, -#endif -#ifdef CONFIG_SYSC_R8A7794 - { .compatible = "renesas,r8a7794-sysc", .data = &r8a7794_sysc_info }, -#endif -#ifdef CONFIG_SYSC_R8A7795 - { .compatible = "renesas,r8a7795-sysc", .data = &r8a7795_sysc_info }, -#endif -#ifdef CONFIG_SYSC_R8A77960 - { .compatible = "renesas,r8a7796-sysc", .data = &r8a77960_sysc_info }, -#endif -#ifdef CONFIG_SYSC_R8A77961 - { .compatible = "renesas,r8a77961-sysc", .data = &r8a77961_sysc_info }, -#endif -#ifdef CONFIG_SYSC_R8A77965 - { .compatible = "renesas,r8a77965-sysc", .data = &r8a77965_sysc_info }, -#endif -#ifdef CONFIG_SYSC_R8A77970 - { .compatible = "renesas,r8a77970-sysc", .data = &r8a77970_sysc_info }, -#endif -#ifdef CONFIG_SYSC_R8A77980 - { .compatible = "renesas,r8a77980-sysc", .data = &r8a77980_sysc_info }, -#endif -#ifdef CONFIG_SYSC_R8A77990 - { .compatible = "renesas,r8a77990-sysc", .data = &r8a77990_sysc_info }, -#endif -#ifdef CONFIG_SYSC_R8A77995 - { .compatible = "renesas,r8a77995-sysc", .data = &r8a77995_sysc_info }, -#endif - { /* sentinel */ } -}; - -struct rcar_pm_domains { - struct genpd_onecell_data onecell_data; - struct generic_pm_domain *domains[RCAR_PD_ALWAYS_ON + 1]; -}; - -static struct genpd_onecell_data *rcar_sysc_onecell_data; - -static int __init rcar_sysc_pd_init(void) -{ - const struct rcar_sysc_info *info; - const struct of_device_id *match; - struct rcar_pm_domains *domains; - struct device_node *np; - void __iomem *base; - unsigned int i; - int error; - - np = of_find_matching_node_and_match(NULL, rcar_sysc_matches, &match); - if (!np) - return -ENODEV; - - info = match->data; - - if (info->init) { - error = info->init(); - if (error) - goto out_put; - } - - has_cpg_mstp = of_find_compatible_node(NULL, NULL, - "renesas,cpg-mstp-clocks"); - - base = of_iomap(np, 0); - if (!base) { - pr_warn("%pOF: Cannot map regs\n", np); - error = -ENOMEM; - goto out_put; - } - - rcar_sysc_base = base; - - /* Optional External Request Mask Register */ - rcar_sysc_extmask_offs = info->extmask_offs; - rcar_sysc_extmask_val = info->extmask_val; - - domains = kzalloc(sizeof(*domains), GFP_KERNEL); - if (!domains) { - error = -ENOMEM; - goto out_put; - } - - domains->onecell_data.domains = domains->domains; - domains->onecell_data.num_domains = ARRAY_SIZE(domains->domains); - rcar_sysc_onecell_data = &domains->onecell_data; - - for (i = 0; i < info->num_areas; i++) { - const struct rcar_sysc_area *area = &info->areas[i]; - struct rcar_sysc_pd *pd; - size_t n; - - if (!area->name) { - /* Skip NULLified area */ - continue; - } - - n = strlen(area->name) + 1; - pd = kzalloc(sizeof(*pd) + n, GFP_KERNEL); - if (!pd) { - error = -ENOMEM; - goto out_put; - } - - memcpy(pd->name, area->name, n); - pd->genpd.name = pd->name; - pd->ch.chan_offs = area->chan_offs; - pd->ch.chan_bit = area->chan_bit; - pd->ch.isr_bit = area->isr_bit; - pd->flags = area->flags; - - error = rcar_sysc_pd_setup(pd); - if (error) - goto out_put; - - domains->domains[area->isr_bit] = &pd->genpd; - - if (area->parent < 0) - continue; - - error = pm_genpd_add_subdomain(domains->domains[area->parent], - &pd->genpd); - if (error) { - pr_warn("Failed to add PM subdomain %s to parent %u\n", - area->name, area->parent); - goto out_put; - } - } - - error = of_genpd_add_provider_onecell(np, &domains->onecell_data); - if (!error) - fwnode_dev_initialized(of_fwnode_handle(np), true); - -out_put: - of_node_put(np); - return error; -} -early_initcall(rcar_sysc_pd_init); - -void __init rcar_sysc_nullify(struct rcar_sysc_area *areas, - unsigned int num_areas, u8 id) -{ - unsigned int i; - - for (i = 0; i < num_areas; i++) - if (areas[i].isr_bit == id) { - areas[i].name = NULL; - return; - } -} - -#ifdef CONFIG_ARCH_R8A7779 -static int rcar_sysc_power_cpu(unsigned int idx, bool on) -{ - struct generic_pm_domain *genpd; - struct rcar_sysc_pd *pd; - unsigned int i; - - if (!rcar_sysc_onecell_data) - return -ENODEV; - - for (i = 0; i < rcar_sysc_onecell_data->num_domains; i++) { - genpd = rcar_sysc_onecell_data->domains[i]; - if (!genpd) - continue; - - pd = to_rcar_pd(genpd); - if (!(pd->flags & PD_CPU) || pd->ch.chan_bit != idx) - continue; - - return rcar_sysc_power(&pd->ch, on); - } - - return -ENOENT; -} - -int rcar_sysc_power_down_cpu(unsigned int cpu) -{ - return rcar_sysc_power_cpu(cpu, false); -} - -int rcar_sysc_power_up_cpu(unsigned int cpu) -{ - return rcar_sysc_power_cpu(cpu, true); -} -#endif /* CONFIG_ARCH_R8A7779 */ diff --git a/drivers/genpd/renesas/rcar-sysc.h b/drivers/genpd/renesas/rcar-sysc.h deleted file mode 100644 index 266c599a0a9b..000000000000 --- a/drivers/genpd/renesas/rcar-sysc.h +++ /dev/null @@ -1,82 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 */ -/* - * Renesas R-Car System Controller - * - * Copyright (C) 2016 Glider bvba - */ -#ifndef __SOC_RENESAS_RCAR_SYSC_H__ -#define __SOC_RENESAS_RCAR_SYSC_H__ - -#include - - -/* - * Power Domain flags - */ -#define PD_CPU BIT(0) /* Area contains main CPU core */ -#define PD_SCU BIT(1) /* Area contains SCU and L2 cache */ -#define PD_NO_CR BIT(2) /* Area lacks PWR{ON,OFF}CR registers */ - -#define PD_CPU_CR PD_CPU /* CPU area has CR (R-Car H1) */ -#define PD_CPU_NOCR PD_CPU | PD_NO_CR /* CPU area lacks CR (R-Car Gen2/3) */ -#define PD_ALWAYS_ON PD_NO_CR /* Always-on area */ - - -/* - * Description of a Power Area - */ - -struct rcar_sysc_area { - const char *name; - u16 chan_offs; /* Offset of PWRSR register for this area */ - u8 chan_bit; /* Bit in PWR* (except for PWRUP in PWRSR) */ - u8 isr_bit; /* Bit in SYSCI*R */ - s8 parent; /* -1 if none */ - u8 flags; /* See PD_* */ -}; - - -/* - * SoC-specific Power Area Description - */ - -struct rcar_sysc_info { - int (*init)(void); /* Optional */ - const struct rcar_sysc_area *areas; - unsigned int num_areas; - /* Optional External Request Mask Register */ - u32 extmask_offs; /* SYSCEXTMASK register offset */ - u32 extmask_val; /* SYSCEXTMASK register mask value */ -}; - -extern const struct rcar_sysc_info r8a7742_sysc_info; -extern const struct rcar_sysc_info r8a7743_sysc_info; -extern const struct rcar_sysc_info r8a7745_sysc_info; -extern const struct rcar_sysc_info r8a77470_sysc_info; -extern const struct rcar_sysc_info r8a774a1_sysc_info; -extern const struct rcar_sysc_info r8a774b1_sysc_info; -extern const struct rcar_sysc_info r8a774c0_sysc_info; -extern const struct rcar_sysc_info r8a774e1_sysc_info; -extern const struct rcar_sysc_info r8a7779_sysc_info; -extern const struct rcar_sysc_info r8a7790_sysc_info; -extern const struct rcar_sysc_info r8a7791_sysc_info; -extern const struct rcar_sysc_info r8a7792_sysc_info; -extern const struct rcar_sysc_info r8a7794_sysc_info; -extern struct rcar_sysc_info r8a7795_sysc_info; -extern const struct rcar_sysc_info r8a77960_sysc_info; -extern const struct rcar_sysc_info r8a77961_sysc_info; -extern const struct rcar_sysc_info r8a77965_sysc_info; -extern const struct rcar_sysc_info r8a77970_sysc_info; -extern const struct rcar_sysc_info r8a77980_sysc_info; -extern const struct rcar_sysc_info r8a77990_sysc_info; -extern const struct rcar_sysc_info r8a77995_sysc_info; - - - /* - * Helpers for fixing up power area tables depending on SoC revision - */ - -extern void rcar_sysc_nullify(struct rcar_sysc_area *areas, - unsigned int num_areas, u8 id); - -#endif /* __SOC_RENESAS_RCAR_SYSC_H__ */ diff --git a/drivers/genpd/renesas/rmobile-sysc.c b/drivers/genpd/renesas/rmobile-sysc.c deleted file mode 100644 index 912daadaa10d..000000000000 --- a/drivers/genpd/renesas/rmobile-sysc.c +++ /dev/null @@ -1,343 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * rmobile power management support - * - * Copyright (C) 2012 Renesas Solutions Corp. - * Copyright (C) 2012 Kuninori Morimoto - * Copyright (C) 2014 Glider bvba - * - * based on pm-sh7372.c - * Copyright (C) 2011 Magnus Damm - */ -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -/* SYSC */ -#define SPDCR 0x08 /* SYS Power Down Control Register */ -#define SWUCR 0x14 /* SYS Wakeup Control Register */ -#define PSTR 0x80 /* Power Status Register */ - -#define PSTR_RETRIES 100 -#define PSTR_DELAY_US 10 - -struct rmobile_pm_domain { - struct generic_pm_domain genpd; - struct dev_power_governor *gov; - int (*suspend)(void); - void __iomem *base; - unsigned int bit_shift; -}; - -static inline -struct rmobile_pm_domain *to_rmobile_pd(struct generic_pm_domain *d) -{ - return container_of(d, struct rmobile_pm_domain, genpd); -} - -static int rmobile_pd_power_down(struct generic_pm_domain *genpd) -{ - struct rmobile_pm_domain *rmobile_pd = to_rmobile_pd(genpd); - unsigned int mask = BIT(rmobile_pd->bit_shift); - u32 val; - - if (rmobile_pd->suspend) { - int ret = rmobile_pd->suspend(); - - if (ret) - return ret; - } - - if (readl(rmobile_pd->base + PSTR) & mask) { - writel(mask, rmobile_pd->base + SPDCR); - - readl_poll_timeout_atomic(rmobile_pd->base + SPDCR, val, - !(val & mask), 0, PSTR_RETRIES); - } - - pr_debug("%s: Power off, 0x%08x -> PSTR = 0x%08x\n", genpd->name, mask, - readl(rmobile_pd->base + PSTR)); - - return 0; -} - -static int __rmobile_pd_power_up(struct rmobile_pm_domain *rmobile_pd) -{ - unsigned int val, mask = BIT(rmobile_pd->bit_shift); - int ret = 0; - - if (readl(rmobile_pd->base + PSTR) & mask) - return ret; - - writel(mask, rmobile_pd->base + SWUCR); - - ret = readl_poll_timeout_atomic(rmobile_pd->base + SWUCR, val, - (val & mask), PSTR_DELAY_US, - PSTR_RETRIES * PSTR_DELAY_US); - - pr_debug("%s: Power on, 0x%08x -> PSTR = 0x%08x\n", - rmobile_pd->genpd.name, mask, - readl(rmobile_pd->base + PSTR)); - - return ret; -} - -static int rmobile_pd_power_up(struct generic_pm_domain *genpd) -{ - return __rmobile_pd_power_up(to_rmobile_pd(genpd)); -} - -static void rmobile_init_pm_domain(struct rmobile_pm_domain *rmobile_pd) -{ - struct generic_pm_domain *genpd = &rmobile_pd->genpd; - struct dev_power_governor *gov = rmobile_pd->gov; - - genpd->flags |= GENPD_FLAG_PM_CLK | GENPD_FLAG_ACTIVE_WAKEUP; - genpd->attach_dev = cpg_mstp_attach_dev; - genpd->detach_dev = cpg_mstp_detach_dev; - - if (!(genpd->flags & GENPD_FLAG_ALWAYS_ON)) { - genpd->power_off = rmobile_pd_power_down; - genpd->power_on = rmobile_pd_power_up; - __rmobile_pd_power_up(rmobile_pd); - } - - pm_genpd_init(genpd, gov ? : &simple_qos_governor, false); -} - -static int rmobile_pd_suspend_console(void) -{ - /* - * Serial consoles make use of SCIF hardware located in this domain, - * hence keep the power domain on if "no_console_suspend" is set. - */ - return console_suspend_enabled ? 0 : -EBUSY; -} - -enum pd_types { - PD_NORMAL, - PD_CPU, - PD_CONSOLE, - PD_DEBUG, - PD_MEMCTL, -}; - -#define MAX_NUM_SPECIAL_PDS 16 - -static struct special_pd { - struct device_node *pd; - enum pd_types type; -} special_pds[MAX_NUM_SPECIAL_PDS] __initdata; - -static unsigned int num_special_pds __initdata; - -static const struct of_device_id special_ids[] __initconst = { - { .compatible = "arm,coresight-etm3x", .data = (void *)PD_DEBUG }, - { .compatible = "renesas,dbsc-r8a73a4", .data = (void *)PD_MEMCTL, }, - { .compatible = "renesas,dbsc3-r8a7740", .data = (void *)PD_MEMCTL, }, - { .compatible = "renesas,sbsc-sh73a0", .data = (void *)PD_MEMCTL, }, - { /* sentinel */ }, -}; - -static void __init add_special_pd(struct device_node *np, enum pd_types type) -{ - unsigned int i; - struct device_node *pd; - - pd = of_parse_phandle(np, "power-domains", 0); - if (!pd) - return; - - for (i = 0; i < num_special_pds; i++) - if (pd == special_pds[i].pd && type == special_pds[i].type) { - of_node_put(pd); - return; - } - - if (num_special_pds == ARRAY_SIZE(special_pds)) { - pr_warn("Too many special PM domains\n"); - of_node_put(pd); - return; - } - - pr_debug("Special PM domain %pOFn type %d for %pOF\n", pd, type, np); - - special_pds[num_special_pds].pd = pd; - special_pds[num_special_pds].type = type; - num_special_pds++; -} - -static void __init get_special_pds(void) -{ - struct device_node *np; - const struct of_device_id *id; - - /* PM domains containing CPUs */ - for_each_of_cpu_node(np) - add_special_pd(np, PD_CPU); - - /* PM domain containing console */ - if (of_stdout) - add_special_pd(of_stdout, PD_CONSOLE); - - /* PM domains containing other special devices */ - for_each_matching_node_and_match(np, special_ids, &id) - add_special_pd(np, (enum pd_types)id->data); -} - -static void __init put_special_pds(void) -{ - unsigned int i; - - for (i = 0; i < num_special_pds; i++) - of_node_put(special_pds[i].pd); -} - -static enum pd_types __init pd_type(const struct device_node *pd) -{ - unsigned int i; - - for (i = 0; i < num_special_pds; i++) - if (pd == special_pds[i].pd) - return special_pds[i].type; - - return PD_NORMAL; -} - -static void __init rmobile_setup_pm_domain(struct device_node *np, - struct rmobile_pm_domain *pd) -{ - const char *name = pd->genpd.name; - - switch (pd_type(np)) { - case PD_CPU: - /* - * This domain contains the CPU core and therefore it should - * only be turned off if the CPU is not in use. - */ - pr_debug("PM domain %s contains CPU\n", name); - pd->genpd.flags |= GENPD_FLAG_ALWAYS_ON; - break; - - case PD_CONSOLE: - pr_debug("PM domain %s contains serial console\n", name); - pd->gov = &pm_domain_always_on_gov; - pd->suspend = rmobile_pd_suspend_console; - break; - - case PD_DEBUG: - /* - * This domain contains the Coresight-ETM hardware block and - * therefore it should only be turned off if the debug module - * is not in use. - */ - pr_debug("PM domain %s contains Coresight-ETM\n", name); - pd->genpd.flags |= GENPD_FLAG_ALWAYS_ON; - break; - - case PD_MEMCTL: - /* - * This domain contains a memory-controller and therefore it - * should only be turned off if memory is not in use. - */ - pr_debug("PM domain %s contains MEMCTL\n", name); - pd->genpd.flags |= GENPD_FLAG_ALWAYS_ON; - break; - - case PD_NORMAL: - if (pd->bit_shift == ~0) { - /* Top-level always-on domain */ - pr_debug("PM domain %s is always-on domain\n", name); - pd->genpd.flags |= GENPD_FLAG_ALWAYS_ON; - } - break; - } - - rmobile_init_pm_domain(pd); -} - -static int __init rmobile_add_pm_domains(void __iomem *base, - struct device_node *parent, - struct generic_pm_domain *genpd_parent) -{ - struct device_node *np; - - for_each_child_of_node(parent, np) { - struct rmobile_pm_domain *pd; - u32 idx = ~0; - - if (of_property_read_u32(np, "reg", &idx)) { - /* always-on domain */ - } - - pd = kzalloc(sizeof(*pd), GFP_KERNEL); - if (!pd) { - of_node_put(np); - return -ENOMEM; - } - - pd->genpd.name = np->name; - pd->base = base; - pd->bit_shift = idx; - - rmobile_setup_pm_domain(np, pd); - if (genpd_parent) - pm_genpd_add_subdomain(genpd_parent, &pd->genpd); - of_genpd_add_provider_simple(np, &pd->genpd); - - rmobile_add_pm_domains(base, np, &pd->genpd); - } - return 0; -} - -static int __init rmobile_init_pm_domains(void) -{ - struct device_node *np, *pmd; - bool scanned = false; - void __iomem *base; - int ret = 0; - - for_each_compatible_node(np, NULL, "renesas,sysc-rmobile") { - base = of_iomap(np, 0); - if (!base) { - pr_warn("%pOF cannot map reg 0\n", np); - continue; - } - - pmd = of_get_child_by_name(np, "pm-domains"); - if (!pmd) { - iounmap(base); - pr_warn("%pOF lacks pm-domains node\n", np); - continue; - } - - if (!scanned) { - /* Find PM domains containing special blocks */ - get_special_pds(); - scanned = true; - } - - ret = rmobile_add_pm_domains(base, pmd, NULL); - of_node_put(pmd); - if (ret) { - of_node_put(np); - break; - } - - fwnode_dev_initialized(of_fwnode_handle(np), true); - } - - put_special_pds(); - - return ret; -} - -core_initcall(rmobile_init_pm_domains); diff --git a/drivers/genpd/rockchip/Makefile b/drivers/genpd/rockchip/Makefile deleted file mode 100644 index 8fb9d88a3492..000000000000 --- a/drivers/genpd/rockchip/Makefile +++ /dev/null @@ -1,2 +0,0 @@ -# SPDX-License-Identifier: GPL-2.0-only -obj-$(CONFIG_ROCKCHIP_PM_DOMAINS) += pm-domains.o diff --git a/drivers/genpd/rockchip/pm-domains.c b/drivers/genpd/rockchip/pm-domains.c deleted file mode 100644 index d5d3ecb38283..000000000000 --- a/drivers/genpd/rockchip/pm-domains.c +++ /dev/null @@ -1,1396 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0-only -/* - * Rockchip Generic power domain support. - * - * Copyright (c) 2015 ROCKCHIP, Co. Ltd. - */ - -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -struct rockchip_domain_info { - const char *name; - int pwr_mask; - int status_mask; - int req_mask; - int idle_mask; - int ack_mask; - bool active_wakeup; - int pwr_w_mask; - int req_w_mask; - int mem_status_mask; - int repair_status_mask; - u32 pwr_offset; - u32 mem_offset; - u32 req_offset; -}; - -struct rockchip_pmu_info { - u32 pwr_offset; - u32 status_offset; - u32 req_offset; - u32 idle_offset; - u32 ack_offset; - u32 mem_pwr_offset; - u32 chain_status_offset; - u32 mem_status_offset; - u32 repair_status_offset; - - u32 core_pwrcnt_offset; - u32 gpu_pwrcnt_offset; - - unsigned int core_power_transition_time; - unsigned int gpu_power_transition_time; - - int num_domains; - const struct rockchip_domain_info *domain_info; -}; - -#define MAX_QOS_REGS_NUM 5 -#define QOS_PRIORITY 0x08 -#define QOS_MODE 0x0c -#define QOS_BANDWIDTH 0x10 -#define QOS_SATURATION 0x14 -#define QOS_EXTCONTROL 0x18 - -struct rockchip_pm_domain { - struct generic_pm_domain genpd; - const struct rockchip_domain_info *info; - struct rockchip_pmu *pmu; - int num_qos; - struct regmap **qos_regmap; - u32 *qos_save_regs[MAX_QOS_REGS_NUM]; - int num_clks; - struct clk_bulk_data *clks; -}; - -struct rockchip_pmu { - struct device *dev; - struct regmap *regmap; - const struct rockchip_pmu_info *info; - struct mutex mutex; /* mutex lock for pmu */ - struct genpd_onecell_data genpd_data; - struct generic_pm_domain *domains[]; -}; - -#define to_rockchip_pd(gpd) container_of(gpd, struct rockchip_pm_domain, genpd) - -#define DOMAIN(_name, pwr, status, req, idle, ack, wakeup) \ -{ \ - .name = _name, \ - .pwr_mask = (pwr), \ - .status_mask = (status), \ - .req_mask = (req), \ - .idle_mask = (idle), \ - .ack_mask = (ack), \ - .active_wakeup = (wakeup), \ -} - -#define DOMAIN_M(_name, pwr, status, req, idle, ack, wakeup) \ -{ \ - .name = _name, \ - .pwr_w_mask = (pwr) << 16, \ - .pwr_mask = (pwr), \ - .status_mask = (status), \ - .req_w_mask = (req) << 16, \ - .req_mask = (req), \ - .idle_mask = (idle), \ - .ack_mask = (ack), \ - .active_wakeup = wakeup, \ -} - -#define DOMAIN_M_O_R(_name, p_offset, pwr, status, m_offset, m_status, r_status, r_offset, req, idle, ack, wakeup) \ -{ \ - .name = _name, \ - .pwr_offset = p_offset, \ - .pwr_w_mask = (pwr) << 16, \ - .pwr_mask = (pwr), \ - .status_mask = (status), \ - .mem_offset = m_offset, \ - .mem_status_mask = (m_status), \ - .repair_status_mask = (r_status), \ - .req_offset = r_offset, \ - .req_w_mask = (req) << 16, \ - .req_mask = (req), \ - .idle_mask = (idle), \ - .ack_mask = (ack), \ - .active_wakeup = wakeup, \ -} - -#define DOMAIN_RK3036(_name, req, ack, idle, wakeup) \ -{ \ - .name = _name, \ - .req_mask = (req), \ - .req_w_mask = (req) << 16, \ - .ack_mask = (ack), \ - .idle_mask = (idle), \ - .active_wakeup = wakeup, \ -} - -#define DOMAIN_PX30(name, pwr, status, req, wakeup) \ - DOMAIN_M(name, pwr, status, req, (req) << 16, req, wakeup) - -#define DOMAIN_RV1126(name, pwr, req, idle, wakeup) \ - DOMAIN_M(name, pwr, pwr, req, idle, idle, wakeup) - -#define DOMAIN_RK3288(name, pwr, status, req, wakeup) \ - DOMAIN(name, pwr, status, req, req, (req) << 16, wakeup) - -#define DOMAIN_RK3328(name, pwr, status, req, wakeup) \ - DOMAIN_M(name, pwr, pwr, req, (req) << 10, req, wakeup) - -#define DOMAIN_RK3368(name, pwr, status, req, wakeup) \ - DOMAIN(name, pwr, status, req, (req) << 16, req, wakeup) - -#define DOMAIN_RK3399(name, pwr, status, req, wakeup) \ - DOMAIN(name, pwr, status, req, req, req, wakeup) - -#define DOMAIN_RK3568(name, pwr, req, wakeup) \ - DOMAIN_M(name, pwr, pwr, req, req, req, wakeup) - -/* - * Dynamic Memory Controller may need to coordinate with us -- see - * rockchip_pmu_block(). - * - * dmc_pmu_mutex protects registration-time races, so DMC driver doesn't try to - * block() while we're initializing the PMU. - */ -static DEFINE_MUTEX(dmc_pmu_mutex); -static struct rockchip_pmu *dmc_pmu; - -/* - * Block PMU transitions and make sure they don't interfere with ARM Trusted - * Firmware operations. There are two conflicts, noted in the comments below. - * - * Caller must unblock PMU transitions via rockchip_pmu_unblock(). - */ -int rockchip_pmu_block(void) -{ - struct rockchip_pmu *pmu; - struct generic_pm_domain *genpd; - struct rockchip_pm_domain *pd; - int i, ret; - - mutex_lock(&dmc_pmu_mutex); - - /* No PMU (yet)? Then we just block rockchip_pmu_probe(). */ - if (!dmc_pmu) - return 0; - pmu = dmc_pmu; - - /* - * mutex blocks all idle transitions: we can't touch the - * PMU_BUS_IDLE_REQ (our ".idle_offset") register while ARM Trusted - * Firmware might be using it. - */ - mutex_lock(&pmu->mutex); - - /* - * Power domain clocks: Per Rockchip, we *must* keep certain clocks - * enabled for the duration of power-domain transitions. Most - * transitions are handled by this driver, but some cases (in - * particular, DRAM DVFS / memory-controller idle) must be handled by - * firmware. Firmware can handle most clock management via a special - * "ungate" register (PMU_CRU_GATEDIS_CON0), but unfortunately, this - * doesn't handle PLLs. We can assist this transition by doing the - * clock management on behalf of firmware. - */ - for (i = 0; i < pmu->genpd_data.num_domains; i++) { - genpd = pmu->genpd_data.domains[i]; - if (genpd) { - pd = to_rockchip_pd(genpd); - ret = clk_bulk_enable(pd->num_clks, pd->clks); - if (ret < 0) { - dev_err(pmu->dev, - "failed to enable clks for domain '%s': %d\n", - genpd->name, ret); - goto err; - } - } - } - - return 0; - -err: - for (i = i - 1; i >= 0; i--) { - genpd = pmu->genpd_data.domains[i]; - if (genpd) { - pd = to_rockchip_pd(genpd); - clk_bulk_disable(pd->num_clks, pd->clks); - } - } - mutex_unlock(&pmu->mutex); - mutex_unlock(&dmc_pmu_mutex); - - return ret; -} -EXPORT_SYMBOL_GPL(rockchip_pmu_block); - -/* Unblock PMU transitions. */ -void rockchip_pmu_unblock(void) -{ - struct rockchip_pmu *pmu; - struct generic_pm_domain *genpd; - struct rockchip_pm_domain *pd; - int i; - - if (dmc_pmu) { - pmu = dmc_pmu; - for (i = 0; i < pmu->genpd_data.num_domains; i++) { - genpd = pmu->genpd_data.domains[i]; - if (genpd) { - pd = to_rockchip_pd(genpd); - clk_bulk_disable(pd->num_clks, pd->clks); - } - } - - mutex_unlock(&pmu->mutex); - } - - mutex_unlock(&dmc_pmu_mutex); -} -EXPORT_SYMBOL_GPL(rockchip_pmu_unblock); - -#define DOMAIN_RK3588(name, p_offset, pwr, status, m_offset, m_status, r_status, r_offset, req, idle, wakeup) \ - DOMAIN_M_O_R(name, p_offset, pwr, status, m_offset, m_status, r_status, r_offset, req, idle, idle, wakeup) - -static bool rockchip_pmu_domain_is_idle(struct rockchip_pm_domain *pd) -{ - struct rockchip_pmu *pmu = pd->pmu; - const struct rockchip_domain_info *pd_info = pd->info; - unsigned int val; - - regmap_read(pmu->regmap, pmu->info->idle_offset, &val); - return (val & pd_info->idle_mask) == pd_info->idle_mask; -} - -static unsigned int rockchip_pmu_read_ack(struct rockchip_pmu *pmu) -{ - unsigned int val; - - regmap_read(pmu->regmap, pmu->info->ack_offset, &val); - return val; -} - -static int rockchip_pmu_set_idle_request(struct rockchip_pm_domain *pd, - bool idle) -{ - const struct rockchip_domain_info *pd_info = pd->info; - struct generic_pm_domain *genpd = &pd->genpd; - struct rockchip_pmu *pmu = pd->pmu; - u32 pd_req_offset = pd_info->req_offset; - unsigned int target_ack; - unsigned int val; - bool is_idle; - int ret; - - if (pd_info->req_mask == 0) - return 0; - else if (pd_info->req_w_mask) - regmap_write(pmu->regmap, pmu->info->req_offset + pd_req_offset, - idle ? (pd_info->req_mask | pd_info->req_w_mask) : - pd_info->req_w_mask); - else - regmap_update_bits(pmu->regmap, pmu->info->req_offset + pd_req_offset, - pd_info->req_mask, idle ? -1U : 0); - - wmb(); - - /* Wait util idle_ack = 1 */ - target_ack = idle ? pd_info->ack_mask : 0; - ret = readx_poll_timeout_atomic(rockchip_pmu_read_ack, pmu, val, - (val & pd_info->ack_mask) == target_ack, - 0, 10000); - if (ret) { - dev_err(pmu->dev, - "failed to get ack on domain '%s', val=0x%x\n", - genpd->name, val); - return ret; - } - - ret = readx_poll_timeout_atomic(rockchip_pmu_domain_is_idle, pd, - is_idle, is_idle == idle, 0, 10000); - if (ret) { - dev_err(pmu->dev, - "failed to set idle on domain '%s', val=%d\n", - genpd->name, is_idle); - return ret; - } - - return 0; -} - -static int rockchip_pmu_save_qos(struct rockchip_pm_domain *pd) -{ - int i; - - for (i = 0; i < pd->num_qos; i++) { - regmap_read(pd->qos_regmap[i], - QOS_PRIORITY, - &pd->qos_save_regs[0][i]); - regmap_read(pd->qos_regmap[i], - QOS_MODE, - &pd->qos_save_regs[1][i]); - regmap_read(pd->qos_regmap[i], - QOS_BANDWIDTH, - &pd->qos_save_regs[2][i]); - regmap_read(pd->qos_regmap[i], - QOS_SATURATION, - &pd->qos_save_regs[3][i]); - regmap_read(pd->qos_regmap[i], - QOS_EXTCONTROL, - &pd->qos_save_regs[4][i]); - } - return 0; -} - -static int rockchip_pmu_restore_qos(struct rockchip_pm_domain *pd) -{ - int i; - - for (i = 0; i < pd->num_qos; i++) { - regmap_write(pd->qos_regmap[i], - QOS_PRIORITY, - pd->qos_save_regs[0][i]); - regmap_write(pd->qos_regmap[i], - QOS_MODE, - pd->qos_save_regs[1][i]); - regmap_write(pd->qos_regmap[i], - QOS_BANDWIDTH, - pd->qos_save_regs[2][i]); - regmap_write(pd->qos_regmap[i], - QOS_SATURATION, - pd->qos_save_regs[3][i]); - regmap_write(pd->qos_regmap[i], - QOS_EXTCONTROL, - pd->qos_save_regs[4][i]); - } - - return 0; -} - -static bool rockchip_pmu_domain_is_on(struct rockchip_pm_domain *pd) -{ - struct rockchip_pmu *pmu = pd->pmu; - unsigned int val; - - if (pd->info->repair_status_mask) { - regmap_read(pmu->regmap, pmu->info->repair_status_offset, &val); - /* 1'b1: power on, 1'b0: power off */ - return val & pd->info->repair_status_mask; - } - - /* check idle status for idle-only domains */ - if (pd->info->status_mask == 0) - return !rockchip_pmu_domain_is_idle(pd); - - regmap_read(pmu->regmap, pmu->info->status_offset, &val); - - /* 1'b0: power on, 1'b1: power off */ - return !(val & pd->info->status_mask); -} - -static bool rockchip_pmu_domain_is_mem_on(struct rockchip_pm_domain *pd) -{ - struct rockchip_pmu *pmu = pd->pmu; - unsigned int val; - - regmap_read(pmu->regmap, - pmu->info->mem_status_offset + pd->info->mem_offset, &val); - - /* 1'b0: power on, 1'b1: power off */ - return !(val & pd->info->mem_status_mask); -} - -static bool rockchip_pmu_domain_is_chain_on(struct rockchip_pm_domain *pd) -{ - struct rockchip_pmu *pmu = pd->pmu; - unsigned int val; - - regmap_read(pmu->regmap, - pmu->info->chain_status_offset + pd->info->mem_offset, &val); - - /* 1'b1: power on, 1'b0: power off */ - return val & pd->info->mem_status_mask; -} - -static int rockchip_pmu_domain_mem_reset(struct rockchip_pm_domain *pd) -{ - struct rockchip_pmu *pmu = pd->pmu; - struct generic_pm_domain *genpd = &pd->genpd; - bool is_on; - int ret = 0; - - ret = readx_poll_timeout_atomic(rockchip_pmu_domain_is_chain_on, pd, is_on, - is_on == true, 0, 10000); - if (ret) { - dev_err(pmu->dev, - "failed to get chain status '%s', target_on=1, val=%d\n", - genpd->name, is_on); - goto error; - } - - udelay(20); - - regmap_write(pmu->regmap, pmu->info->mem_pwr_offset + pd->info->pwr_offset, - (pd->info->pwr_mask | pd->info->pwr_w_mask)); - wmb(); - - ret = readx_poll_timeout_atomic(rockchip_pmu_domain_is_mem_on, pd, is_on, - is_on == false, 0, 10000); - if (ret) { - dev_err(pmu->dev, - "failed to get mem status '%s', target_on=0, val=%d\n", - genpd->name, is_on); - goto error; - } - - regmap_write(pmu->regmap, pmu->info->mem_pwr_offset + pd->info->pwr_offset, - pd->info->pwr_w_mask); - wmb(); - - ret = readx_poll_timeout_atomic(rockchip_pmu_domain_is_mem_on, pd, is_on, - is_on == true, 0, 10000); - if (ret) { - dev_err(pmu->dev, - "failed to get mem status '%s', target_on=1, val=%d\n", - genpd->name, is_on); - } - -error: - return ret; -} - -static void rockchip_do_pmu_set_power_domain(struct rockchip_pm_domain *pd, - bool on) -{ - struct rockchip_pmu *pmu = pd->pmu; - struct generic_pm_domain *genpd = &pd->genpd; - u32 pd_pwr_offset = pd->info->pwr_offset; - bool is_on, is_mem_on = false; - - if (pd->info->pwr_mask == 0) - return; - - if (on && pd->info->mem_status_mask) - is_mem_on = rockchip_pmu_domain_is_mem_on(pd); - - if (pd->info->pwr_w_mask) - regmap_write(pmu->regmap, pmu->info->pwr_offset + pd_pwr_offset, - on ? pd->info->pwr_w_mask : - (pd->info->pwr_mask | pd->info->pwr_w_mask)); - else - regmap_update_bits(pmu->regmap, pmu->info->pwr_offset + pd_pwr_offset, - pd->info->pwr_mask, on ? 0 : -1U); - - wmb(); - - if (is_mem_on && rockchip_pmu_domain_mem_reset(pd)) - return; - - if (readx_poll_timeout_atomic(rockchip_pmu_domain_is_on, pd, is_on, - is_on == on, 0, 10000)) { - dev_err(pmu->dev, - "failed to set domain '%s', val=%d\n", - genpd->name, is_on); - return; - } -} - -static int rockchip_pd_power(struct rockchip_pm_domain *pd, bool power_on) -{ - struct rockchip_pmu *pmu = pd->pmu; - int ret; - - mutex_lock(&pmu->mutex); - - if (rockchip_pmu_domain_is_on(pd) != power_on) { - ret = clk_bulk_enable(pd->num_clks, pd->clks); - if (ret < 0) { - dev_err(pmu->dev, "failed to enable clocks\n"); - mutex_unlock(&pmu->mutex); - return ret; - } - - if (!power_on) { - rockchip_pmu_save_qos(pd); - - /* if powering down, idle request to NIU first */ - rockchip_pmu_set_idle_request(pd, true); - } - - rockchip_do_pmu_set_power_domain(pd, power_on); - - if (power_on) { - /* if powering up, leave idle mode */ - rockchip_pmu_set_idle_request(pd, false); - - rockchip_pmu_restore_qos(pd); - } - - clk_bulk_disable(pd->num_clks, pd->clks); - } - - mutex_unlock(&pmu->mutex); - return 0; -} - -static int rockchip_pd_power_on(struct generic_pm_domain *domain) -{ - struct rockchip_pm_domain *pd = to_rockchip_pd(domain); - - return rockchip_pd_power(pd, true); -} - -static int rockchip_pd_power_off(struct generic_pm_domain *domain) -{ - struct rockchip_pm_domain *pd = to_rockchip_pd(domain); - - return rockchip_pd_power(pd, false); -} - -static int rockchip_pd_attach_dev(struct generic_pm_domain *genpd, - struct device *dev) -{ - struct clk *clk; - int i; - int error; - - dev_dbg(dev, "attaching to power domain '%s'\n", genpd->name); - - error = pm_clk_create(dev); - if (error) { - dev_err(dev, "pm_clk_create failed %d\n", error); - return error; - } - - i = 0; - while ((clk = of_clk_get(dev->of_node, i++)) && !IS_ERR(clk)) { - dev_dbg(dev, "adding clock '%pC' to list of PM clocks\n", clk); - error = pm_clk_add_clk(dev, clk); - if (error) { - dev_err(dev, "pm_clk_add_clk failed %d\n", error); - clk_put(clk); - pm_clk_destroy(dev); - return error; - } - } - - return 0; -} - -static void rockchip_pd_detach_dev(struct generic_pm_domain *genpd, - struct device *dev) -{ - dev_dbg(dev, "detaching from power domain '%s'\n", genpd->name); - - pm_clk_destroy(dev); -} - -static int rockchip_pm_add_one_domain(struct rockchip_pmu *pmu, - struct device_node *node) -{ - const struct rockchip_domain_info *pd_info; - struct rockchip_pm_domain *pd; - struct device_node *qos_node; - int i, j; - u32 id; - int error; - - error = of_property_read_u32(node, "reg", &id); - if (error) { - dev_err(pmu->dev, - "%pOFn: failed to retrieve domain id (reg): %d\n", - node, error); - return -EINVAL; - } - - if (id >= pmu->info->num_domains) { - dev_err(pmu->dev, "%pOFn: invalid domain id %d\n", - node, id); - return -EINVAL; - } - /* RK3588 has domains with two parents (RKVDEC0/RKVDEC1) */ - if (pmu->genpd_data.domains[id]) - return 0; - - pd_info = &pmu->info->domain_info[id]; - if (!pd_info) { - dev_err(pmu->dev, "%pOFn: undefined domain id %d\n", - node, id); - return -EINVAL; - } - - pd = devm_kzalloc(pmu->dev, sizeof(*pd), GFP_KERNEL); - if (!pd) - return -ENOMEM; - - pd->info = pd_info; - pd->pmu = pmu; - - pd->num_clks = of_clk_get_parent_count(node); - if (pd->num_clks > 0) { - pd->clks = devm_kcalloc(pmu->dev, pd->num_clks, - sizeof(*pd->clks), GFP_KERNEL); - if (!pd->clks) - return -ENOMEM; - } else { - dev_dbg(pmu->dev, "%pOFn: doesn't have clocks: %d\n", - node, pd->num_clks); - pd->num_clks = 0; - } - - for (i = 0; i < pd->num_clks; i++) { - pd->clks[i].clk = of_clk_get(node, i); - if (IS_ERR(pd->clks[i].clk)) { - error = PTR_ERR(pd->clks[i].clk); - dev_err(pmu->dev, - "%pOFn: failed to get clk at index %d: %d\n", - node, i, error); - return error; - } - } - - error = clk_bulk_prepare(pd->num_clks, pd->clks); - if (error) - goto err_put_clocks; - - pd->num_qos = of_count_phandle_with_args(node, "pm_qos", - NULL); - - if (pd->num_qos > 0) { - pd->qos_regmap = devm_kcalloc(pmu->dev, pd->num_qos, - sizeof(*pd->qos_regmap), - GFP_KERNEL); - if (!pd->qos_regmap) { - error = -ENOMEM; - goto err_unprepare_clocks; - } - - for (j = 0; j < MAX_QOS_REGS_NUM; j++) { - pd->qos_save_regs[j] = devm_kcalloc(pmu->dev, - pd->num_qos, - sizeof(u32), - GFP_KERNEL); - if (!pd->qos_save_regs[j]) { - error = -ENOMEM; - goto err_unprepare_clocks; - } - } - - for (j = 0; j < pd->num_qos; j++) { - qos_node = of_parse_phandle(node, "pm_qos", j); - if (!qos_node) { - error = -ENODEV; - goto err_unprepare_clocks; - } - pd->qos_regmap[j] = syscon_node_to_regmap(qos_node); - if (IS_ERR(pd->qos_regmap[j])) { - error = -ENODEV; - of_node_put(qos_node); - goto err_unprepare_clocks; - } - of_node_put(qos_node); - } - } - - if (pd->info->name) - pd->genpd.name = pd->info->name; - else - pd->genpd.name = kbasename(node->full_name); - pd->genpd.power_off = rockchip_pd_power_off; - pd->genpd.power_on = rockchip_pd_power_on; - pd->genpd.attach_dev = rockchip_pd_attach_dev; - pd->genpd.detach_dev = rockchip_pd_detach_dev; - pd->genpd.flags = GENPD_FLAG_PM_CLK; - if (pd_info->active_wakeup) - pd->genpd.flags |= GENPD_FLAG_ACTIVE_WAKEUP; - pm_genpd_init(&pd->genpd, NULL, - !rockchip_pmu_domain_is_on(pd) || - (pd->info->mem_status_mask && !rockchip_pmu_domain_is_mem_on(pd))); - - pmu->genpd_data.domains[id] = &pd->genpd; - return 0; - -err_unprepare_clocks: - clk_bulk_unprepare(pd->num_clks, pd->clks); -err_put_clocks: - clk_bulk_put(pd->num_clks, pd->clks); - return error; -} - -static void rockchip_pm_remove_one_domain(struct rockchip_pm_domain *pd) -{ - int ret; - - /* - * We're in the error cleanup already, so we only complain, - * but won't emit another error on top of the original one. - */ - ret = pm_genpd_remove(&pd->genpd); - if (ret < 0) - dev_err(pd->pmu->dev, "failed to remove domain '%s' : %d - state may be inconsistent\n", - pd->genpd.name, ret); - - clk_bulk_unprepare(pd->num_clks, pd->clks); - clk_bulk_put(pd->num_clks, pd->clks); - - /* protect the zeroing of pm->num_clks */ - mutex_lock(&pd->pmu->mutex); - pd->num_clks = 0; - mutex_unlock(&pd->pmu->mutex); - - /* devm will free our memory */ -} - -static void rockchip_pm_domain_cleanup(struct rockchip_pmu *pmu) -{ - struct generic_pm_domain *genpd; - struct rockchip_pm_domain *pd; - int i; - - for (i = 0; i < pmu->genpd_data.num_domains; i++) { - genpd = pmu->genpd_data.domains[i]; - if (genpd) { - pd = to_rockchip_pd(genpd); - rockchip_pm_remove_one_domain(pd); - } - } - - /* devm will free our memory */ -} - -static void rockchip_configure_pd_cnt(struct rockchip_pmu *pmu, - u32 domain_reg_offset, - unsigned int count) -{ - /* First configure domain power down transition count ... */ - regmap_write(pmu->regmap, domain_reg_offset, count); - /* ... and then power up count. */ - regmap_write(pmu->regmap, domain_reg_offset + 4, count); -} - -static int rockchip_pm_add_subdomain(struct rockchip_pmu *pmu, - struct device_node *parent) -{ - struct device_node *np; - struct generic_pm_domain *child_domain, *parent_domain; - int error; - - for_each_child_of_node(parent, np) { - u32 idx; - - error = of_property_read_u32(parent, "reg", &idx); - if (error) { - dev_err(pmu->dev, - "%pOFn: failed to retrieve domain id (reg): %d\n", - parent, error); - goto err_out; - } - parent_domain = pmu->genpd_data.domains[idx]; - - error = rockchip_pm_add_one_domain(pmu, np); - if (error) { - dev_err(pmu->dev, "failed to handle node %pOFn: %d\n", - np, error); - goto err_out; - } - - error = of_property_read_u32(np, "reg", &idx); - if (error) { - dev_err(pmu->dev, - "%pOFn: failed to retrieve domain id (reg): %d\n", - np, error); - goto err_out; - } - child_domain = pmu->genpd_data.domains[idx]; - - error = pm_genpd_add_subdomain(parent_domain, child_domain); - if (error) { - dev_err(pmu->dev, "%s failed to add subdomain %s: %d\n", - parent_domain->name, child_domain->name, error); - goto err_out; - } else { - dev_dbg(pmu->dev, "%s add subdomain: %s\n", - parent_domain->name, child_domain->name); - } - - rockchip_pm_add_subdomain(pmu, np); - } - - return 0; - -err_out: - of_node_put(np); - return error; -} - -static int rockchip_pm_domain_probe(struct platform_device *pdev) -{ - struct device *dev = &pdev->dev; - struct device_node *np = dev->of_node; - struct device_node *node; - struct device *parent; - struct rockchip_pmu *pmu; - const struct of_device_id *match; - const struct rockchip_pmu_info *pmu_info; - int error; - - if (!np) { - dev_err(dev, "device tree node not found\n"); - return -ENODEV; - } - - match = of_match_device(dev->driver->of_match_table, dev); - if (!match || !match->data) { - dev_err(dev, "missing pmu data\n"); - return -EINVAL; - } - - pmu_info = match->data; - - pmu = devm_kzalloc(dev, - struct_size(pmu, domains, pmu_info->num_domains), - GFP_KERNEL); - if (!pmu) - return -ENOMEM; - - pmu->dev = &pdev->dev; - mutex_init(&pmu->mutex); - - pmu->info = pmu_info; - - pmu->genpd_data.domains = pmu->domains; - pmu->genpd_data.num_domains = pmu_info->num_domains; - - parent = dev->parent; - if (!parent) { - dev_err(dev, "no parent for syscon devices\n"); - return -ENODEV; - } - - pmu->regmap = syscon_node_to_regmap(parent->of_node); - if (IS_ERR(pmu->regmap)) { - dev_err(dev, "no regmap available\n"); - return PTR_ERR(pmu->regmap); - } - - /* - * Configure power up and down transition delays for CORE - * and GPU domains. - */ - if (pmu_info->core_power_transition_time) - rockchip_configure_pd_cnt(pmu, pmu_info->core_pwrcnt_offset, - pmu_info->core_power_transition_time); - if (pmu_info->gpu_pwrcnt_offset) - rockchip_configure_pd_cnt(pmu, pmu_info->gpu_pwrcnt_offset, - pmu_info->gpu_power_transition_time); - - error = -ENODEV; - - /* - * Prevent any rockchip_pmu_block() from racing with the remainder of - * setup (clocks, register initialization). - */ - mutex_lock(&dmc_pmu_mutex); - - for_each_available_child_of_node(np, node) { - error = rockchip_pm_add_one_domain(pmu, node); - if (error) { - dev_err(dev, "failed to handle node %pOFn: %d\n", - node, error); - of_node_put(node); - goto err_out; - } - - error = rockchip_pm_add_subdomain(pmu, node); - if (error < 0) { - dev_err(dev, "failed to handle subdomain node %pOFn: %d\n", - node, error); - of_node_put(node); - goto err_out; - } - } - - if (error) { - dev_dbg(dev, "no power domains defined\n"); - goto err_out; - } - - error = of_genpd_add_provider_onecell(np, &pmu->genpd_data); - if (error) { - dev_err(dev, "failed to add provider: %d\n", error); - goto err_out; - } - - /* We only expect one PMU. */ - if (!WARN_ON_ONCE(dmc_pmu)) - dmc_pmu = pmu; - - mutex_unlock(&dmc_pmu_mutex); - - return 0; - -err_out: - rockchip_pm_domain_cleanup(pmu); - mutex_unlock(&dmc_pmu_mutex); - return error; -} - -static const struct rockchip_domain_info px30_pm_domains[] = { - [PX30_PD_USB] = DOMAIN_PX30("usb", BIT(5), BIT(5), BIT(10), false), - [PX30_PD_SDCARD] = DOMAIN_PX30("sdcard", BIT(8), BIT(8), BIT(9), false), - [PX30_PD_GMAC] = DOMAIN_PX30("gmac", BIT(10), BIT(10), BIT(6), false), - [PX30_PD_MMC_NAND] = DOMAIN_PX30("mmc_nand", BIT(11), BIT(11), BIT(5), false), - [PX30_PD_VPU] = DOMAIN_PX30("vpu", BIT(12), BIT(12), BIT(14), false), - [PX30_PD_VO] = DOMAIN_PX30("vo", BIT(13), BIT(13), BIT(7), false), - [PX30_PD_VI] = DOMAIN_PX30("vi", BIT(14), BIT(14), BIT(8), false), - [PX30_PD_GPU] = DOMAIN_PX30("gpu", BIT(15), BIT(15), BIT(2), false), -}; - -static const struct rockchip_domain_info rv1126_pm_domains[] = { - [RV1126_PD_VEPU] = DOMAIN_RV1126("vepu", BIT(2), BIT(9), BIT(9), false), - [RV1126_PD_VI] = DOMAIN_RV1126("vi", BIT(4), BIT(6), BIT(6), false), - [RV1126_PD_VO] = DOMAIN_RV1126("vo", BIT(5), BIT(7), BIT(7), false), - [RV1126_PD_ISPP] = DOMAIN_RV1126("ispp", BIT(1), BIT(8), BIT(8), false), - [RV1126_PD_VDPU] = DOMAIN_RV1126("vdpu", BIT(3), BIT(10), BIT(10), false), - [RV1126_PD_NVM] = DOMAIN_RV1126("nvm", BIT(7), BIT(11), BIT(11), false), - [RV1126_PD_SDIO] = DOMAIN_RV1126("sdio", BIT(8), BIT(13), BIT(13), false), - [RV1126_PD_USB] = DOMAIN_RV1126("usb", BIT(9), BIT(15), BIT(15), false), -}; - -static const struct rockchip_domain_info rk3036_pm_domains[] = { - [RK3036_PD_MSCH] = DOMAIN_RK3036("msch", BIT(14), BIT(23), BIT(30), true), - [RK3036_PD_CORE] = DOMAIN_RK3036("core", BIT(13), BIT(17), BIT(24), false), - [RK3036_PD_PERI] = DOMAIN_RK3036("peri", BIT(12), BIT(18), BIT(25), false), - [RK3036_PD_VIO] = DOMAIN_RK3036("vio", BIT(11), BIT(19), BIT(26), false), - [RK3036_PD_VPU] = DOMAIN_RK3036("vpu", BIT(10), BIT(20), BIT(27), false), - [RK3036_PD_GPU] = DOMAIN_RK3036("gpu", BIT(9), BIT(21), BIT(28), false), - [RK3036_PD_SYS] = DOMAIN_RK3036("sys", BIT(8), BIT(22), BIT(29), false), -}; - -static const struct rockchip_domain_info rk3066_pm_domains[] = { - [RK3066_PD_GPU] = DOMAIN("gpu", BIT(9), BIT(9), BIT(3), BIT(24), BIT(29), false), - [RK3066_PD_VIDEO] = DOMAIN("video", BIT(8), BIT(8), BIT(4), BIT(23), BIT(28), false), - [RK3066_PD_VIO] = DOMAIN("vio", BIT(7), BIT(7), BIT(5), BIT(22), BIT(27), false), - [RK3066_PD_PERI] = DOMAIN("peri", BIT(6), BIT(6), BIT(2), BIT(25), BIT(30), false), - [RK3066_PD_CPU] = DOMAIN("cpu", 0, BIT(5), BIT(1), BIT(26), BIT(31), false), -}; - -static const struct rockchip_domain_info rk3128_pm_domains[] = { - [RK3128_PD_CORE] = DOMAIN_RK3288("core", BIT(0), BIT(0), BIT(4), false), - [RK3128_PD_MSCH] = DOMAIN_RK3288("msch", 0, 0, BIT(6), true), - [RK3128_PD_VIO] = DOMAIN_RK3288("vio", BIT(3), BIT(3), BIT(2), false), - [RK3128_PD_VIDEO] = DOMAIN_RK3288("video", BIT(2), BIT(2), BIT(1), false), - [RK3128_PD_GPU] = DOMAIN_RK3288("gpu", BIT(1), BIT(1), BIT(3), false), -}; - -static const struct rockchip_domain_info rk3188_pm_domains[] = { - [RK3188_PD_GPU] = DOMAIN("gpu", BIT(9), BIT(9), BIT(3), BIT(24), BIT(29), false), - [RK3188_PD_VIDEO] = DOMAIN("video", BIT(8), BIT(8), BIT(4), BIT(23), BIT(28), false), - [RK3188_PD_VIO] = DOMAIN("vio", BIT(7), BIT(7), BIT(5), BIT(22), BIT(27), false), - [RK3188_PD_PERI] = DOMAIN("peri", BIT(6), BIT(6), BIT(2), BIT(25), BIT(30), false), - [RK3188_PD_CPU] = DOMAIN("cpu", BIT(5), BIT(5), BIT(1), BIT(26), BIT(31), false), -}; - -static const struct rockchip_domain_info rk3228_pm_domains[] = { - [RK3228_PD_CORE] = DOMAIN_RK3036("core", BIT(0), BIT(0), BIT(16), true), - [RK3228_PD_MSCH] = DOMAIN_RK3036("msch", BIT(1), BIT(1), BIT(17), true), - [RK3228_PD_BUS] = DOMAIN_RK3036("bus", BIT(2), BIT(2), BIT(18), true), - [RK3228_PD_SYS] = DOMAIN_RK3036("sys", BIT(3), BIT(3), BIT(19), true), - [RK3228_PD_VIO] = DOMAIN_RK3036("vio", BIT(4), BIT(4), BIT(20), false), - [RK3228_PD_VOP] = DOMAIN_RK3036("vop", BIT(5), BIT(5), BIT(21), false), - [RK3228_PD_VPU] = DOMAIN_RK3036("vpu", BIT(6), BIT(6), BIT(22), false), - [RK3228_PD_RKVDEC] = DOMAIN_RK3036("vdec", BIT(7), BIT(7), BIT(23), false), - [RK3228_PD_GPU] = DOMAIN_RK3036("gpu", BIT(8), BIT(8), BIT(24), false), - [RK3228_PD_PERI] = DOMAIN_RK3036("peri", BIT(9), BIT(9), BIT(25), true), - [RK3228_PD_GMAC] = DOMAIN_RK3036("gmac", BIT(10), BIT(10), BIT(26), false), -}; - -static const struct rockchip_domain_info rk3288_pm_domains[] = { - [RK3288_PD_VIO] = DOMAIN_RK3288("vio", BIT(7), BIT(7), BIT(4), false), - [RK3288_PD_HEVC] = DOMAIN_RK3288("hevc", BIT(14), BIT(10), BIT(9), false), - [RK3288_PD_VIDEO] = DOMAIN_RK3288("video", BIT(8), BIT(8), BIT(3), false), - [RK3288_PD_GPU] = DOMAIN_RK3288("gpu", BIT(9), BIT(9), BIT(2), false), -}; - -static const struct rockchip_domain_info rk3328_pm_domains[] = { - [RK3328_PD_CORE] = DOMAIN_RK3328("core", 0, BIT(0), BIT(0), false), - [RK3328_PD_GPU] = DOMAIN_RK3328("gpu", 0, BIT(1), BIT(1), false), - [RK3328_PD_BUS] = DOMAIN_RK3328("bus", 0, BIT(2), BIT(2), true), - [RK3328_PD_MSCH] = DOMAIN_RK3328("msch", 0, BIT(3), BIT(3), true), - [RK3328_PD_PERI] = DOMAIN_RK3328("peri", 0, BIT(4), BIT(4), true), - [RK3328_PD_VIDEO] = DOMAIN_RK3328("video", 0, BIT(5), BIT(5), false), - [RK3328_PD_HEVC] = DOMAIN_RK3328("hevc", 0, BIT(6), BIT(6), false), - [RK3328_PD_VIO] = DOMAIN_RK3328("vio", 0, BIT(8), BIT(8), false), - [RK3328_PD_VPU] = DOMAIN_RK3328("vpu", 0, BIT(9), BIT(9), false), -}; - -static const struct rockchip_domain_info rk3366_pm_domains[] = { - [RK3366_PD_PERI] = DOMAIN_RK3368("peri", BIT(10), BIT(10), BIT(6), true), - [RK3366_PD_VIO] = DOMAIN_RK3368("vio", BIT(14), BIT(14), BIT(8), false), - [RK3366_PD_VIDEO] = DOMAIN_RK3368("video", BIT(13), BIT(13), BIT(7), false), - [RK3366_PD_RKVDEC] = DOMAIN_RK3368("vdec", BIT(11), BIT(11), BIT(7), false), - [RK3366_PD_WIFIBT] = DOMAIN_RK3368("wifibt", BIT(8), BIT(8), BIT(9), false), - [RK3366_PD_VPU] = DOMAIN_RK3368("vpu", BIT(12), BIT(12), BIT(7), false), - [RK3366_PD_GPU] = DOMAIN_RK3368("gpu", BIT(15), BIT(15), BIT(2), false), -}; - -static const struct rockchip_domain_info rk3368_pm_domains[] = { - [RK3368_PD_PERI] = DOMAIN_RK3368("peri", BIT(13), BIT(12), BIT(6), true), - [RK3368_PD_VIO] = DOMAIN_RK3368("vio", BIT(15), BIT(14), BIT(8), false), - [RK3368_PD_VIDEO] = DOMAIN_RK3368("video", BIT(14), BIT(13), BIT(7), false), - [RK3368_PD_GPU_0] = DOMAIN_RK3368("gpu_0", BIT(16), BIT(15), BIT(2), false), - [RK3368_PD_GPU_1] = DOMAIN_RK3368("gpu_1", BIT(17), BIT(16), BIT(2), false), -}; - -static const struct rockchip_domain_info rk3399_pm_domains[] = { - [RK3399_PD_TCPD0] = DOMAIN_RK3399("tcpd0", BIT(8), BIT(8), 0, false), - [RK3399_PD_TCPD1] = DOMAIN_RK3399("tcpd1", BIT(9), BIT(9), 0, false), - [RK3399_PD_CCI] = DOMAIN_RK3399("cci", BIT(10), BIT(10), 0, true), - [RK3399_PD_CCI0] = DOMAIN_RK3399("cci0", 0, 0, BIT(15), true), - [RK3399_PD_CCI1] = DOMAIN_RK3399("cci1", 0, 0, BIT(16), true), - [RK3399_PD_PERILP] = DOMAIN_RK3399("perilp", BIT(11), BIT(11), BIT(1), true), - [RK3399_PD_PERIHP] = DOMAIN_RK3399("perihp", BIT(12), BIT(12), BIT(2), true), - [RK3399_PD_CENTER] = DOMAIN_RK3399("center", BIT(13), BIT(13), BIT(14), true), - [RK3399_PD_VIO] = DOMAIN_RK3399("vio", BIT(14), BIT(14), BIT(17), false), - [RK3399_PD_GPU] = DOMAIN_RK3399("gpu", BIT(15), BIT(15), BIT(0), false), - [RK3399_PD_VCODEC] = DOMAIN_RK3399("vcodec", BIT(16), BIT(16), BIT(3), false), - [RK3399_PD_VDU] = DOMAIN_RK3399("vdu", BIT(17), BIT(17), BIT(4), false), - [RK3399_PD_RGA] = DOMAIN_RK3399("rga", BIT(18), BIT(18), BIT(5), false), - [RK3399_PD_IEP] = DOMAIN_RK3399("iep", BIT(19), BIT(19), BIT(6), false), - [RK3399_PD_VO] = DOMAIN_RK3399("vo", BIT(20), BIT(20), 0, false), - [RK3399_PD_VOPB] = DOMAIN_RK3399("vopb", 0, 0, BIT(7), false), - [RK3399_PD_VOPL] = DOMAIN_RK3399("vopl", 0, 0, BIT(8), false), - [RK3399_PD_ISP0] = DOMAIN_RK3399("isp0", BIT(22), BIT(22), BIT(9), false), - [RK3399_PD_ISP1] = DOMAIN_RK3399("isp1", BIT(23), BIT(23), BIT(10), false), - [RK3399_PD_HDCP] = DOMAIN_RK3399("hdcp", BIT(24), BIT(24), BIT(11), false), - [RK3399_PD_GMAC] = DOMAIN_RK3399("gmac", BIT(25), BIT(25), BIT(23), true), - [RK3399_PD_EMMC] = DOMAIN_RK3399("emmc", BIT(26), BIT(26), BIT(24), true), - [RK3399_PD_USB3] = DOMAIN_RK3399("usb3", BIT(27), BIT(27), BIT(12), true), - [RK3399_PD_EDP] = DOMAIN_RK3399("edp", BIT(28), BIT(28), BIT(22), false), - [RK3399_PD_GIC] = DOMAIN_RK3399("gic", BIT(29), BIT(29), BIT(27), true), - [RK3399_PD_SD] = DOMAIN_RK3399("sd", BIT(30), BIT(30), BIT(28), true), - [RK3399_PD_SDIOAUDIO] = DOMAIN_RK3399("sdioaudio", BIT(31), BIT(31), BIT(29), true), -}; - -static const struct rockchip_domain_info rk3568_pm_domains[] = { - [RK3568_PD_NPU] = DOMAIN_RK3568("npu", BIT(1), BIT(2), false), - [RK3568_PD_GPU] = DOMAIN_RK3568("gpu", BIT(0), BIT(1), false), - [RK3568_PD_VI] = DOMAIN_RK3568("vi", BIT(6), BIT(3), false), - [RK3568_PD_VO] = DOMAIN_RK3568("vo", BIT(7), BIT(4), false), - [RK3568_PD_RGA] = DOMAIN_RK3568("rga", BIT(5), BIT(5), false), - [RK3568_PD_VPU] = DOMAIN_RK3568("vpu", BIT(2), BIT(6), false), - [RK3568_PD_RKVDEC] = DOMAIN_RK3568("vdec", BIT(4), BIT(8), false), - [RK3568_PD_RKVENC] = DOMAIN_RK3568("venc", BIT(3), BIT(7), false), - [RK3568_PD_PIPE] = DOMAIN_RK3568("pipe", BIT(8), BIT(11), false), -}; - -static const struct rockchip_domain_info rk3588_pm_domains[] = { - [RK3588_PD_GPU] = DOMAIN_RK3588("gpu", 0x0, BIT(0), 0, 0x0, 0, BIT(1), 0x0, BIT(0), BIT(0), false), - [RK3588_PD_NPU] = DOMAIN_RK3588("npu", 0x0, BIT(1), BIT(1), 0x0, 0, 0, 0x0, 0, 0, false), - [RK3588_PD_VCODEC] = DOMAIN_RK3588("vcodec", 0x0, BIT(2), BIT(2), 0x0, 0, 0, 0x0, 0, 0, false), - [RK3588_PD_NPUTOP] = DOMAIN_RK3588("nputop", 0x0, BIT(3), 0, 0x0, BIT(11), BIT(2), 0x0, BIT(1), BIT(1), false), - [RK3588_PD_NPU1] = DOMAIN_RK3588("npu1", 0x0, BIT(4), 0, 0x0, BIT(12), BIT(3), 0x0, BIT(2), BIT(2), false), - [RK3588_PD_NPU2] = DOMAIN_RK3588("npu2", 0x0, BIT(5), 0, 0x0, BIT(13), BIT(4), 0x0, BIT(3), BIT(3), false), - [RK3588_PD_VENC0] = DOMAIN_RK3588("venc0", 0x0, BIT(6), 0, 0x0, BIT(14), BIT(5), 0x0, BIT(4), BIT(4), false), - [RK3588_PD_VENC1] = DOMAIN_RK3588("venc1", 0x0, BIT(7), 0, 0x0, BIT(15), BIT(6), 0x0, BIT(5), BIT(5), false), - [RK3588_PD_RKVDEC0] = DOMAIN_RK3588("rkvdec0", 0x0, BIT(8), 0, 0x0, BIT(16), BIT(7), 0x0, BIT(6), BIT(6), false), - [RK3588_PD_RKVDEC1] = DOMAIN_RK3588("rkvdec1", 0x0, BIT(9), 0, 0x0, BIT(17), BIT(8), 0x0, BIT(7), BIT(7), false), - [RK3588_PD_VDPU] = DOMAIN_RK3588("vdpu", 0x0, BIT(10), 0, 0x0, BIT(18), BIT(9), 0x0, BIT(8), BIT(8), false), - [RK3588_PD_RGA30] = DOMAIN_RK3588("rga30", 0x0, BIT(11), 0, 0x0, BIT(19), BIT(10), 0x0, 0, 0, false), - [RK3588_PD_AV1] = DOMAIN_RK3588("av1", 0x0, BIT(12), 0, 0x0, BIT(20), BIT(11), 0x0, BIT(9), BIT(9), false), - [RK3588_PD_VI] = DOMAIN_RK3588("vi", 0x0, BIT(13), 0, 0x0, BIT(21), BIT(12), 0x0, BIT(10), BIT(10), false), - [RK3588_PD_FEC] = DOMAIN_RK3588("fec", 0x0, BIT(14), 0, 0x0, BIT(22), BIT(13), 0x0, 0, 0, false), - [RK3588_PD_ISP1] = DOMAIN_RK3588("isp1", 0x0, BIT(15), 0, 0x0, BIT(23), BIT(14), 0x0, BIT(11), BIT(11), false), - [RK3588_PD_RGA31] = DOMAIN_RK3588("rga31", 0x4, BIT(0), 0, 0x0, BIT(24), BIT(15), 0x0, BIT(12), BIT(12), false), - [RK3588_PD_VOP] = DOMAIN_RK3588("vop", 0x4, BIT(1), 0, 0x0, BIT(25), BIT(16), 0x0, BIT(13) | BIT(14), BIT(13) | BIT(14), false), - [RK3588_PD_VO0] = DOMAIN_RK3588("vo0", 0x4, BIT(2), 0, 0x0, BIT(26), BIT(17), 0x0, BIT(15), BIT(15), false), - [RK3588_PD_VO1] = DOMAIN_RK3588("vo1", 0x4, BIT(3), 0, 0x0, BIT(27), BIT(18), 0x4, BIT(0), BIT(16), false), - [RK3588_PD_AUDIO] = DOMAIN_RK3588("audio", 0x4, BIT(4), 0, 0x0, BIT(28), BIT(19), 0x4, BIT(1), BIT(17), false), - [RK3588_PD_PHP] = DOMAIN_RK3588("php", 0x4, BIT(5), 0, 0x0, BIT(29), BIT(20), 0x4, BIT(5), BIT(21), false), - [RK3588_PD_GMAC] = DOMAIN_RK3588("gmac", 0x4, BIT(6), 0, 0x0, BIT(30), BIT(21), 0x0, 0, 0, false), - [RK3588_PD_PCIE] = DOMAIN_RK3588("pcie", 0x4, BIT(7), 0, 0x0, BIT(31), BIT(22), 0x0, 0, 0, true), - [RK3588_PD_NVM] = DOMAIN_RK3588("nvm", 0x4, BIT(8), BIT(24), 0x4, 0, 0, 0x4, BIT(2), BIT(18), false), - [RK3588_PD_NVM0] = DOMAIN_RK3588("nvm0", 0x4, BIT(9), 0, 0x4, BIT(1), BIT(23), 0x0, 0, 0, false), - [RK3588_PD_SDIO] = DOMAIN_RK3588("sdio", 0x4, BIT(10), 0, 0x4, BIT(2), BIT(24), 0x4, BIT(3), BIT(19), false), - [RK3588_PD_USB] = DOMAIN_RK3588("usb", 0x4, BIT(11), 0, 0x4, BIT(3), BIT(25), 0x4, BIT(4), BIT(20), true), - [RK3588_PD_SDMMC] = DOMAIN_RK3588("sdmmc", 0x4, BIT(13), 0, 0x4, BIT(5), BIT(26), 0x0, 0, 0, false), -}; - -static const struct rockchip_pmu_info px30_pmu = { - .pwr_offset = 0x18, - .status_offset = 0x20, - .req_offset = 0x64, - .idle_offset = 0x6c, - .ack_offset = 0x6c, - - .num_domains = ARRAY_SIZE(px30_pm_domains), - .domain_info = px30_pm_domains, -}; - -static const struct rockchip_pmu_info rk3036_pmu = { - .req_offset = 0x148, - .idle_offset = 0x14c, - .ack_offset = 0x14c, - - .num_domains = ARRAY_SIZE(rk3036_pm_domains), - .domain_info = rk3036_pm_domains, -}; - -static const struct rockchip_pmu_info rk3066_pmu = { - .pwr_offset = 0x08, - .status_offset = 0x0c, - .req_offset = 0x38, /* PMU_MISC_CON1 */ - .idle_offset = 0x0c, - .ack_offset = 0x0c, - - .num_domains = ARRAY_SIZE(rk3066_pm_domains), - .domain_info = rk3066_pm_domains, -}; - -static const struct rockchip_pmu_info rk3128_pmu = { - .pwr_offset = 0x04, - .status_offset = 0x08, - .req_offset = 0x0c, - .idle_offset = 0x10, - .ack_offset = 0x10, - - .num_domains = ARRAY_SIZE(rk3128_pm_domains), - .domain_info = rk3128_pm_domains, -}; - -static const struct rockchip_pmu_info rk3188_pmu = { - .pwr_offset = 0x08, - .status_offset = 0x0c, - .req_offset = 0x38, /* PMU_MISC_CON1 */ - .idle_offset = 0x0c, - .ack_offset = 0x0c, - - .num_domains = ARRAY_SIZE(rk3188_pm_domains), - .domain_info = rk3188_pm_domains, -}; - -static const struct rockchip_pmu_info rk3228_pmu = { - .req_offset = 0x40c, - .idle_offset = 0x488, - .ack_offset = 0x488, - - .num_domains = ARRAY_SIZE(rk3228_pm_domains), - .domain_info = rk3228_pm_domains, -}; - -static const struct rockchip_pmu_info rk3288_pmu = { - .pwr_offset = 0x08, - .status_offset = 0x0c, - .req_offset = 0x10, - .idle_offset = 0x14, - .ack_offset = 0x14, - - .core_pwrcnt_offset = 0x34, - .gpu_pwrcnt_offset = 0x3c, - - .core_power_transition_time = 24, /* 1us */ - .gpu_power_transition_time = 24, /* 1us */ - - .num_domains = ARRAY_SIZE(rk3288_pm_domains), - .domain_info = rk3288_pm_domains, -}; - -static const struct rockchip_pmu_info rk3328_pmu = { - .req_offset = 0x414, - .idle_offset = 0x484, - .ack_offset = 0x484, - - .num_domains = ARRAY_SIZE(rk3328_pm_domains), - .domain_info = rk3328_pm_domains, -}; - -static const struct rockchip_pmu_info rk3366_pmu = { - .pwr_offset = 0x0c, - .status_offset = 0x10, - .req_offset = 0x3c, - .idle_offset = 0x40, - .ack_offset = 0x40, - - .core_pwrcnt_offset = 0x48, - .gpu_pwrcnt_offset = 0x50, - - .core_power_transition_time = 24, - .gpu_power_transition_time = 24, - - .num_domains = ARRAY_SIZE(rk3366_pm_domains), - .domain_info = rk3366_pm_domains, -}; - -static const struct rockchip_pmu_info rk3368_pmu = { - .pwr_offset = 0x0c, - .status_offset = 0x10, - .req_offset = 0x3c, - .idle_offset = 0x40, - .ack_offset = 0x40, - - .core_pwrcnt_offset = 0x48, - .gpu_pwrcnt_offset = 0x50, - - .core_power_transition_time = 24, - .gpu_power_transition_time = 24, - - .num_domains = ARRAY_SIZE(rk3368_pm_domains), - .domain_info = rk3368_pm_domains, -}; - -static const struct rockchip_pmu_info rk3399_pmu = { - .pwr_offset = 0x14, - .status_offset = 0x18, - .req_offset = 0x60, - .idle_offset = 0x64, - .ack_offset = 0x68, - - /* ARM Trusted Firmware manages power transition times */ - - .num_domains = ARRAY_SIZE(rk3399_pm_domains), - .domain_info = rk3399_pm_domains, -}; - -static const struct rockchip_pmu_info rk3568_pmu = { - .pwr_offset = 0xa0, - .status_offset = 0x98, - .req_offset = 0x50, - .idle_offset = 0x68, - .ack_offset = 0x60, - - .num_domains = ARRAY_SIZE(rk3568_pm_domains), - .domain_info = rk3568_pm_domains, -}; - -static const struct rockchip_pmu_info rk3588_pmu = { - .pwr_offset = 0x14c, - .status_offset = 0x180, - .req_offset = 0x10c, - .idle_offset = 0x120, - .ack_offset = 0x118, - .mem_pwr_offset = 0x1a0, - .chain_status_offset = 0x1f0, - .mem_status_offset = 0x1f8, - .repair_status_offset = 0x290, - - .num_domains = ARRAY_SIZE(rk3588_pm_domains), - .domain_info = rk3588_pm_domains, -}; - -static const struct rockchip_pmu_info rv1126_pmu = { - .pwr_offset = 0x110, - .status_offset = 0x108, - .req_offset = 0xc0, - .idle_offset = 0xd8, - .ack_offset = 0xd0, - - .num_domains = ARRAY_SIZE(rv1126_pm_domains), - .domain_info = rv1126_pm_domains, -}; - -static const struct of_device_id rockchip_pm_domain_dt_match[] = { - { - .compatible = "rockchip,px30-power-controller", - .data = (void *)&px30_pmu, - }, - { - .compatible = "rockchip,rk3036-power-controller", - .data = (void *)&rk3036_pmu, - }, - { - .compatible = "rockchip,rk3066-power-controller", - .data = (void *)&rk3066_pmu, - }, - { - .compatible = "rockchip,rk3128-power-controller", - .data = (void *)&rk3128_pmu, - }, - { - .compatible = "rockchip,rk3188-power-controller", - .data = (void *)&rk3188_pmu, - }, - { - .compatible = "rockchip,rk3228-power-controller", - .data = (void *)&rk3228_pmu, - }, - { - .compatible = "rockchip,rk3288-power-controller", - .data = (void *)&rk3288_pmu, - }, - { - .compatible = "rockchip,rk3328-power-controller", - .data = (void *)&rk3328_pmu, - }, - { - .compatible = "rockchip,rk3366-power-controller", - .data = (void *)&rk3366_pmu, - }, - { - .compatible = "rockchip,rk3368-power-controller", - .data = (void *)&rk3368_pmu, - }, - { - .compatible = "rockchip,rk3399-power-controller", - .data = (void *)&rk3399_pmu, - }, - { - .compatible = "rockchip,rk3568-power-controller", - .data = (void *)&rk3568_pmu, - }, - { - .compatible = "rockchip,rk3588-power-controller", - .data = (void *)&rk3588_pmu, - }, - { - .compatible = "rockchip,rv1126-power-controller", - .data = (void *)&rv1126_pmu, - }, - { /* sentinel */ }, -}; - -static struct platform_driver rockchip_pm_domain_driver = { - .probe = rockchip_pm_domain_probe, - .driver = { - .name = "rockchip-pm-domain", - .of_match_table = rockchip_pm_domain_dt_match, - /* - * We can't forcibly eject devices from the power - * domain, so we can't really remove power domains - * once they were added. - */ - .suppress_bind_attrs = true, - }, -}; - -static int __init rockchip_pm_domain_drv_register(void) -{ - return platform_driver_register(&rockchip_pm_domain_driver); -} -postcore_initcall(rockchip_pm_domain_drv_register); diff --git a/drivers/genpd/samsung/Makefile b/drivers/genpd/samsung/Makefile deleted file mode 100644 index 397aa5908c1d..000000000000 --- a/drivers/genpd/samsung/Makefile +++ /dev/null @@ -1,2 +0,0 @@ -# SPDX-License-Identifier: GPL-2.0-only -obj-$(CONFIG_EXYNOS_PM_DOMAINS) += exynos-pm-domains.o diff --git a/drivers/genpd/samsung/exynos-pm-domains.c b/drivers/genpd/samsung/exynos-pm-domains.c deleted file mode 100644 index 9b502e8751d1..000000000000 --- a/drivers/genpd/samsung/exynos-pm-domains.c +++ /dev/null @@ -1,167 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -// -// Exynos Generic power domain support. -// -// Copyright (c) 2012 Samsung Electronics Co., Ltd. -// http://www.samsung.com -// -// Implementation of Exynos specific power domain control which is used in -// conjunction with runtime-pm. Support for both device-tree and non-device-tree -// based power domain support is included. - -#include -#include -#include -#include -#include -#include -#include -#include -#include - -struct exynos_pm_domain_config { - /* Value for LOCAL_PWR_CFG and STATUS fields for each domain */ - u32 local_pwr_cfg; -}; - -/* - * Exynos specific wrapper around the generic power domain - */ -struct exynos_pm_domain { - void __iomem *base; - struct generic_pm_domain pd; - u32 local_pwr_cfg; -}; - -static int exynos_pd_power(struct generic_pm_domain *domain, bool power_on) -{ - struct exynos_pm_domain *pd; - void __iomem *base; - u32 timeout, pwr; - char *op; - - pd = container_of(domain, struct exynos_pm_domain, pd); - base = pd->base; - - pwr = power_on ? pd->local_pwr_cfg : 0; - writel_relaxed(pwr, base); - - /* Wait max 1ms */ - timeout = 10; - - while ((readl_relaxed(base + 0x4) & pd->local_pwr_cfg) != pwr) { - if (!timeout) { - op = (power_on) ? "enable" : "disable"; - pr_err("Power domain %s %s failed\n", domain->name, op); - return -ETIMEDOUT; - } - timeout--; - cpu_relax(); - usleep_range(80, 100); - } - - return 0; -} - -static int exynos_pd_power_on(struct generic_pm_domain *domain) -{ - return exynos_pd_power(domain, true); -} - -static int exynos_pd_power_off(struct generic_pm_domain *domain) -{ - return exynos_pd_power(domain, false); -} - -static const struct exynos_pm_domain_config exynos4210_cfg = { - .local_pwr_cfg = 0x7, -}; - -static const struct exynos_pm_domain_config exynos5433_cfg = { - .local_pwr_cfg = 0xf, -}; - -static const struct of_device_id exynos_pm_domain_of_match[] = { - { - .compatible = "samsung,exynos4210-pd", - .data = &exynos4210_cfg, - }, { - .compatible = "samsung,exynos5433-pd", - .data = &exynos5433_cfg, - }, - { }, -}; - -static const char *exynos_get_domain_name(struct device_node *node) -{ - const char *name; - - if (of_property_read_string(node, "label", &name) < 0) - name = kbasename(node->full_name); - return kstrdup_const(name, GFP_KERNEL); -} - -static int exynos_pd_probe(struct platform_device *pdev) -{ - const struct exynos_pm_domain_config *pm_domain_cfg; - struct device *dev = &pdev->dev; - struct device_node *np = dev->of_node; - struct of_phandle_args child, parent; - struct exynos_pm_domain *pd; - int on, ret; - - pm_domain_cfg = of_device_get_match_data(dev); - pd = devm_kzalloc(dev, sizeof(*pd), GFP_KERNEL); - if (!pd) - return -ENOMEM; - - pd->pd.name = exynos_get_domain_name(np); - if (!pd->pd.name) - return -ENOMEM; - - pd->base = of_iomap(np, 0); - if (!pd->base) { - kfree_const(pd->pd.name); - return -ENODEV; - } - - pd->pd.power_off = exynos_pd_power_off; - pd->pd.power_on = exynos_pd_power_on; - pd->local_pwr_cfg = pm_domain_cfg->local_pwr_cfg; - - on = readl_relaxed(pd->base + 0x4) & pd->local_pwr_cfg; - - pm_genpd_init(&pd->pd, NULL, !on); - ret = of_genpd_add_provider_simple(np, &pd->pd); - - if (ret == 0 && of_parse_phandle_with_args(np, "power-domains", - "#power-domain-cells", 0, &parent) == 0) { - child.np = np; - child.args_count = 0; - - if (of_genpd_add_subdomain(&parent, &child)) - pr_warn("%pOF failed to add subdomain: %pOF\n", - parent.np, child.np); - else - pr_info("%pOF has as child subdomain: %pOF.\n", - parent.np, child.np); - } - - pm_runtime_enable(dev); - return ret; -} - -static struct platform_driver exynos_pd_driver = { - .probe = exynos_pd_probe, - .driver = { - .name = "exynos-pd", - .of_match_table = exynos_pm_domain_of_match, - .suppress_bind_attrs = true, - } -}; - -static __init int exynos4_pm_init_power_domain(void) -{ - return platform_driver_register(&exynos_pd_driver); -} -core_initcall(exynos4_pm_init_power_domain); diff --git a/drivers/genpd/st/Makefile b/drivers/genpd/st/Makefile deleted file mode 100644 index 8fa5f9855460..000000000000 --- a/drivers/genpd/st/Makefile +++ /dev/null @@ -1,2 +0,0 @@ -# SPDX-License-Identifier: GPL-2.0-only -obj-$(CONFIG_ARCH_U8500) += ste-ux500-pm-domain.o diff --git a/drivers/genpd/st/ste-ux500-pm-domain.c b/drivers/genpd/st/ste-ux500-pm-domain.c deleted file mode 100644 index 3d4f111ed156..000000000000 --- a/drivers/genpd/st/ste-ux500-pm-domain.c +++ /dev/null @@ -1,94 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0-only -/* - * Copyright (C) 2014 Linaro Ltd. - * - * Author: Ulf Hansson - * - * Implements PM domains using the generic PM domain for ux500. - */ -#include -#include -#include -#include -#include -#include -#include -#include - -#include - -static int pd_power_off(struct generic_pm_domain *domain) -{ - /* - * Handle the gating of the PM domain regulator here. - * - * Drivers/subsystems handling devices in the PM domain needs to perform - * register context save/restore from their respective runtime PM - * callbacks, to be able to enable PM domain gating/ungating. - */ - return 0; -} - -static int pd_power_on(struct generic_pm_domain *domain) -{ - /* - * Handle the ungating of the PM domain regulator here. - * - * Drivers/subsystems handling devices in the PM domain needs to perform - * register context save/restore from their respective runtime PM - * callbacks, to be able to enable PM domain gating/ungating. - */ - return 0; -} - -static struct generic_pm_domain ux500_pm_domain_vape = { - .name = "VAPE", - .power_off = pd_power_off, - .power_on = pd_power_on, -}; - -static struct generic_pm_domain *ux500_pm_domains[NR_DOMAINS] = { - [DOMAIN_VAPE] = &ux500_pm_domain_vape, -}; - -static const struct of_device_id ux500_pm_domain_matches[] = { - { .compatible = "stericsson,ux500-pm-domains", }, - { }, -}; - -static int ux500_pm_domains_probe(struct platform_device *pdev) -{ - struct device_node *np = pdev->dev.of_node; - struct genpd_onecell_data *genpd_data; - int i; - - if (!np) - return -ENODEV; - - genpd_data = kzalloc(sizeof(*genpd_data), GFP_KERNEL); - if (!genpd_data) - return -ENOMEM; - - genpd_data->domains = ux500_pm_domains; - genpd_data->num_domains = ARRAY_SIZE(ux500_pm_domains); - - for (i = 0; i < ARRAY_SIZE(ux500_pm_domains); ++i) - pm_genpd_init(ux500_pm_domains[i], NULL, false); - - of_genpd_add_provider_onecell(np, genpd_data); - return 0; -} - -static struct platform_driver ux500_pm_domains_driver = { - .probe = ux500_pm_domains_probe, - .driver = { - .name = "ux500_pm_domains", - .of_match_table = ux500_pm_domain_matches, - }, -}; - -static int __init ux500_pm_domains_init(void) -{ - return platform_driver_register(&ux500_pm_domains_driver); -} -arch_initcall(ux500_pm_domains_init); diff --git a/drivers/genpd/starfive/Makefile b/drivers/genpd/starfive/Makefile deleted file mode 100644 index 975bba2a29a9..000000000000 --- a/drivers/genpd/starfive/Makefile +++ /dev/null @@ -1,2 +0,0 @@ -# SPDX-License-Identifier: GPL-2.0 -obj-$(CONFIG_JH71XX_PMU) += jh71xx-pmu.o diff --git a/drivers/genpd/starfive/jh71xx-pmu.c b/drivers/genpd/starfive/jh71xx-pmu.c deleted file mode 100644 index 7d5f50d71c0d..000000000000 --- a/drivers/genpd/starfive/jh71xx-pmu.c +++ /dev/null @@ -1,383 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0-or-later -/* - * StarFive JH71XX PMU (Power Management Unit) Controller Driver - * - * Copyright (C) 2022 StarFive Technology Co., Ltd. - */ - -#include -#include -#include -#include -#include -#include -#include -#include -#include - -/* register offset */ -#define JH71XX_PMU_SW_TURN_ON_POWER 0x0C -#define JH71XX_PMU_SW_TURN_OFF_POWER 0x10 -#define JH71XX_PMU_SW_ENCOURAGE 0x44 -#define JH71XX_PMU_TIMER_INT_MASK 0x48 -#define JH71XX_PMU_CURR_POWER_MODE 0x80 -#define JH71XX_PMU_EVENT_STATUS 0x88 -#define JH71XX_PMU_INT_STATUS 0x8C - -/* sw encourage cfg */ -#define JH71XX_PMU_SW_ENCOURAGE_EN_LO 0x05 -#define JH71XX_PMU_SW_ENCOURAGE_EN_HI 0x50 -#define JH71XX_PMU_SW_ENCOURAGE_DIS_LO 0x0A -#define JH71XX_PMU_SW_ENCOURAGE_DIS_HI 0xA0 -#define JH71XX_PMU_SW_ENCOURAGE_ON 0xFF - -/* pmu int status */ -#define JH71XX_PMU_INT_SEQ_DONE BIT(0) -#define JH71XX_PMU_INT_HW_REQ BIT(1) -#define JH71XX_PMU_INT_SW_FAIL GENMASK(3, 2) -#define JH71XX_PMU_INT_HW_FAIL GENMASK(5, 4) -#define JH71XX_PMU_INT_PCH_FAIL GENMASK(8, 6) -#define JH71XX_PMU_INT_ALL_MASK GENMASK(8, 0) - -/* - * The time required for switching power status is based on the time - * to turn on the largest domain's power, which is at microsecond level - */ -#define JH71XX_PMU_TIMEOUT_US 100 - -struct jh71xx_domain_info { - const char * const name; - unsigned int flags; - u8 bit; -}; - -struct jh71xx_pmu_match_data { - const struct jh71xx_domain_info *domain_info; - int num_domains; -}; - -struct jh71xx_pmu { - struct device *dev; - const struct jh71xx_pmu_match_data *match_data; - void __iomem *base; - struct generic_pm_domain **genpd; - struct genpd_onecell_data genpd_data; - int irq; - spinlock_t lock; /* protects pmu reg */ -}; - -struct jh71xx_pmu_dev { - const struct jh71xx_domain_info *domain_info; - struct jh71xx_pmu *pmu; - struct generic_pm_domain genpd; -}; - -static int jh71xx_pmu_get_state(struct jh71xx_pmu_dev *pmd, u32 mask, bool *is_on) -{ - struct jh71xx_pmu *pmu = pmd->pmu; - - if (!mask) - return -EINVAL; - - *is_on = readl(pmu->base + JH71XX_PMU_CURR_POWER_MODE) & mask; - - return 0; -} - -static int jh71xx_pmu_set_state(struct jh71xx_pmu_dev *pmd, u32 mask, bool on) -{ - struct jh71xx_pmu *pmu = pmd->pmu; - unsigned long flags; - u32 val; - u32 mode; - u32 encourage_lo; - u32 encourage_hi; - bool is_on; - int ret; - - ret = jh71xx_pmu_get_state(pmd, mask, &is_on); - if (ret) { - dev_dbg(pmu->dev, "unable to get current state for %s\n", - pmd->genpd.name); - return ret; - } - - if (is_on == on) { - dev_dbg(pmu->dev, "pm domain [%s] is already %sable status.\n", - pmd->genpd.name, on ? "en" : "dis"); - return 0; - } - - spin_lock_irqsave(&pmu->lock, flags); - - /* - * The PMU accepts software encourage to switch power mode in the following 2 steps: - * - * 1.Configure the register SW_TURN_ON_POWER (offset 0x0c) by writing 1 to - * the bit corresponding to the power domain that will be turned on - * and writing 0 to the others. - * Likewise, configure the register SW_TURN_OFF_POWER (offset 0x10) by - * writing 1 to the bit corresponding to the power domain that will be - * turned off and writing 0 to the others. - */ - if (on) { - mode = JH71XX_PMU_SW_TURN_ON_POWER; - encourage_lo = JH71XX_PMU_SW_ENCOURAGE_EN_LO; - encourage_hi = JH71XX_PMU_SW_ENCOURAGE_EN_HI; - } else { - mode = JH71XX_PMU_SW_TURN_OFF_POWER; - encourage_lo = JH71XX_PMU_SW_ENCOURAGE_DIS_LO; - encourage_hi = JH71XX_PMU_SW_ENCOURAGE_DIS_HI; - } - - writel(mask, pmu->base + mode); - - /* - * 2.Write SW encourage command sequence to the Software Encourage Reg (offset 0x44) - * First write SW_MODE_ENCOURAGE_ON to JH71XX_PMU_SW_ENCOURAGE. This will reset - * the state machine which parses the command sequence. This register must be - * written every time software wants to power on/off a domain. - * Then write the lower bits of the command sequence, followed by the upper - * bits. The sequence differs between powering on & off a domain. - */ - writel(JH71XX_PMU_SW_ENCOURAGE_ON, pmu->base + JH71XX_PMU_SW_ENCOURAGE); - writel(encourage_lo, pmu->base + JH71XX_PMU_SW_ENCOURAGE); - writel(encourage_hi, pmu->base + JH71XX_PMU_SW_ENCOURAGE); - - spin_unlock_irqrestore(&pmu->lock, flags); - - /* Wait for the power domain bit to be enabled / disabled */ - if (on) { - ret = readl_poll_timeout_atomic(pmu->base + JH71XX_PMU_CURR_POWER_MODE, - val, val & mask, - 1, JH71XX_PMU_TIMEOUT_US); - } else { - ret = readl_poll_timeout_atomic(pmu->base + JH71XX_PMU_CURR_POWER_MODE, - val, !(val & mask), - 1, JH71XX_PMU_TIMEOUT_US); - } - - if (ret) { - dev_err(pmu->dev, "%s: failed to power %s\n", - pmd->genpd.name, on ? "on" : "off"); - return -ETIMEDOUT; - } - - return 0; -} - -static int jh71xx_pmu_on(struct generic_pm_domain *genpd) -{ - struct jh71xx_pmu_dev *pmd = container_of(genpd, - struct jh71xx_pmu_dev, genpd); - u32 pwr_mask = BIT(pmd->domain_info->bit); - - return jh71xx_pmu_set_state(pmd, pwr_mask, true); -} - -static int jh71xx_pmu_off(struct generic_pm_domain *genpd) -{ - struct jh71xx_pmu_dev *pmd = container_of(genpd, - struct jh71xx_pmu_dev, genpd); - u32 pwr_mask = BIT(pmd->domain_info->bit); - - return jh71xx_pmu_set_state(pmd, pwr_mask, false); -} - -static void jh71xx_pmu_int_enable(struct jh71xx_pmu *pmu, u32 mask, bool enable) -{ - u32 val; - unsigned long flags; - - spin_lock_irqsave(&pmu->lock, flags); - val = readl(pmu->base + JH71XX_PMU_TIMER_INT_MASK); - - if (enable) - val &= ~mask; - else - val |= mask; - - writel(val, pmu->base + JH71XX_PMU_TIMER_INT_MASK); - spin_unlock_irqrestore(&pmu->lock, flags); -} - -static irqreturn_t jh71xx_pmu_interrupt(int irq, void *data) -{ - struct jh71xx_pmu *pmu = data; - u32 val; - - val = readl(pmu->base + JH71XX_PMU_INT_STATUS); - - if (val & JH71XX_PMU_INT_SEQ_DONE) - dev_dbg(pmu->dev, "sequence done.\n"); - if (val & JH71XX_PMU_INT_HW_REQ) - dev_dbg(pmu->dev, "hardware encourage requestion.\n"); - if (val & JH71XX_PMU_INT_SW_FAIL) - dev_err(pmu->dev, "software encourage fail.\n"); - if (val & JH71XX_PMU_INT_HW_FAIL) - dev_err(pmu->dev, "hardware encourage fail.\n"); - if (val & JH71XX_PMU_INT_PCH_FAIL) - dev_err(pmu->dev, "p-channel fail event.\n"); - - /* clear interrupts */ - writel(val, pmu->base + JH71XX_PMU_INT_STATUS); - writel(val, pmu->base + JH71XX_PMU_EVENT_STATUS); - - return IRQ_HANDLED; -} - -static int jh71xx_pmu_init_domain(struct jh71xx_pmu *pmu, int index) -{ - struct jh71xx_pmu_dev *pmd; - u32 pwr_mask; - int ret; - bool is_on = false; - - pmd = devm_kzalloc(pmu->dev, sizeof(*pmd), GFP_KERNEL); - if (!pmd) - return -ENOMEM; - - pmd->domain_info = &pmu->match_data->domain_info[index]; - pmd->pmu = pmu; - pwr_mask = BIT(pmd->domain_info->bit); - - pmd->genpd.name = pmd->domain_info->name; - pmd->genpd.flags = pmd->domain_info->flags; - - ret = jh71xx_pmu_get_state(pmd, pwr_mask, &is_on); - if (ret) - dev_warn(pmu->dev, "unable to get current state for %s\n", - pmd->genpd.name); - - pmd->genpd.power_on = jh71xx_pmu_on; - pmd->genpd.power_off = jh71xx_pmu_off; - pm_genpd_init(&pmd->genpd, NULL, !is_on); - - pmu->genpd_data.domains[index] = &pmd->genpd; - - return 0; -} - -static int jh71xx_pmu_probe(struct platform_device *pdev) -{ - struct device *dev = &pdev->dev; - struct device_node *np = dev->of_node; - const struct jh71xx_pmu_match_data *match_data; - struct jh71xx_pmu *pmu; - unsigned int i; - int ret; - - pmu = devm_kzalloc(dev, sizeof(*pmu), GFP_KERNEL); - if (!pmu) - return -ENOMEM; - - pmu->base = devm_platform_ioremap_resource(pdev, 0); - if (IS_ERR(pmu->base)) - return PTR_ERR(pmu->base); - - pmu->irq = platform_get_irq(pdev, 0); - if (pmu->irq < 0) - return pmu->irq; - - ret = devm_request_irq(dev, pmu->irq, jh71xx_pmu_interrupt, - 0, pdev->name, pmu); - if (ret) - dev_err(dev, "failed to request irq\n"); - - match_data = of_device_get_match_data(dev); - if (!match_data) - return -EINVAL; - - pmu->genpd = devm_kcalloc(dev, match_data->num_domains, - sizeof(struct generic_pm_domain *), - GFP_KERNEL); - if (!pmu->genpd) - return -ENOMEM; - - pmu->dev = dev; - pmu->match_data = match_data; - pmu->genpd_data.domains = pmu->genpd; - pmu->genpd_data.num_domains = match_data->num_domains; - - for (i = 0; i < match_data->num_domains; i++) { - ret = jh71xx_pmu_init_domain(pmu, i); - if (ret) { - dev_err(dev, "failed to initialize power domain\n"); - return ret; - } - } - - spin_lock_init(&pmu->lock); - jh71xx_pmu_int_enable(pmu, JH71XX_PMU_INT_ALL_MASK & ~JH71XX_PMU_INT_PCH_FAIL, true); - - ret = of_genpd_add_provider_onecell(np, &pmu->genpd_data); - if (ret) { - dev_err(dev, "failed to register genpd driver: %d\n", ret); - return ret; - } - - dev_dbg(dev, "registered %u power domains\n", i); - - return 0; -} - -static const struct jh71xx_domain_info jh7110_power_domains[] = { - [JH7110_PD_SYSTOP] = { - .name = "SYSTOP", - .bit = 0, - .flags = GENPD_FLAG_ALWAYS_ON, - }, - [JH7110_PD_CPU] = { - .name = "CPU", - .bit = 1, - .flags = GENPD_FLAG_ALWAYS_ON, - }, - [JH7110_PD_GPUA] = { - .name = "GPUA", - .bit = 2, - }, - [JH7110_PD_VDEC] = { - .name = "VDEC", - .bit = 3, - }, - [JH7110_PD_VOUT] = { - .name = "VOUT", - .bit = 4, - }, - [JH7110_PD_ISP] = { - .name = "ISP", - .bit = 5, - }, - [JH7110_PD_VENC] = { - .name = "VENC", - .bit = 6, - }, -}; - -static const struct jh71xx_pmu_match_data jh7110_pmu = { - .num_domains = ARRAY_SIZE(jh7110_power_domains), - .domain_info = jh7110_power_domains, -}; - -static const struct of_device_id jh71xx_pmu_of_match[] = { - { - .compatible = "starfive,jh7110-pmu", - .data = (void *)&jh7110_pmu, - }, { - /* sentinel */ - } -}; - -static struct platform_driver jh71xx_pmu_driver = { - .probe = jh71xx_pmu_probe, - .driver = { - .name = "jh71xx-pmu", - .of_match_table = jh71xx_pmu_of_match, - .suppress_bind_attrs = true, - }, -}; -builtin_platform_driver(jh71xx_pmu_driver); - -MODULE_AUTHOR("Walker Chen "); -MODULE_DESCRIPTION("StarFive JH71XX PMU Driver"); -MODULE_LICENSE("GPL"); diff --git a/drivers/genpd/sunxi/Makefile b/drivers/genpd/sunxi/Makefile deleted file mode 100644 index ec1d7a2fb21d..000000000000 --- a/drivers/genpd/sunxi/Makefile +++ /dev/null @@ -1,2 +0,0 @@ -# SPDX-License-Identifier: GPL-2.0-only -obj-$(CONFIG_SUN20I_PPU) += sun20i-ppu.o diff --git a/drivers/genpd/sunxi/sun20i-ppu.c b/drivers/genpd/sunxi/sun20i-ppu.c deleted file mode 100644 index 8700f9dd5f75..000000000000 --- a/drivers/genpd/sunxi/sun20i-ppu.c +++ /dev/null @@ -1,207 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0-only - -#include -#include -#include -#include -#include -#include -#include -#include -#include - -#define PD_STATE_ON 1 -#define PD_STATE_OFF 2 - -#define PD_RSTN_REG 0x00 -#define PD_CLK_GATE_REG 0x04 -#define PD_PWROFF_GATE_REG 0x08 -#define PD_PSW_ON_REG 0x0c -#define PD_PSW_OFF_REG 0x10 -#define PD_PSW_DELAY_REG 0x14 -#define PD_OFF_DELAY_REG 0x18 -#define PD_ON_DELAY_REG 0x1c -#define PD_COMMAND_REG 0x20 -#define PD_STATUS_REG 0x24 -#define PD_STATUS_COMPLETE BIT(1) -#define PD_STATUS_BUSY BIT(3) -#define PD_STATUS_STATE GENMASK(17, 16) -#define PD_ACTIVE_CTRL_REG 0x2c -#define PD_GATE_STATUS_REG 0x30 -#define PD_RSTN_STATUS BIT(0) -#define PD_CLK_GATE_STATUS BIT(1) -#define PD_PWROFF_GATE_STATUS BIT(2) -#define PD_PSW_STATUS_REG 0x34 - -#define PD_REGS_SIZE 0x80 - -struct sun20i_ppu_desc { - const char *const *names; - unsigned int num_domains; -}; - -struct sun20i_ppu_pd { - struct generic_pm_domain genpd; - void __iomem *base; -}; - -#define to_sun20i_ppu_pd(_genpd) \ - container_of(_genpd, struct sun20i_ppu_pd, genpd) - -static bool sun20i_ppu_pd_is_on(const struct sun20i_ppu_pd *pd) -{ - u32 status = readl(pd->base + PD_STATUS_REG); - - return FIELD_GET(PD_STATUS_STATE, status) == PD_STATE_ON; -} - -static int sun20i_ppu_pd_set_power(const struct sun20i_ppu_pd *pd, bool power_on) -{ - u32 state, status; - int ret; - - if (sun20i_ppu_pd_is_on(pd) == power_on) - return 0; - - /* Wait for the power controller to be idle. */ - ret = readl_poll_timeout(pd->base + PD_STATUS_REG, status, - !(status & PD_STATUS_BUSY), 100, 1000); - if (ret) - return ret; - - state = power_on ? PD_STATE_ON : PD_STATE_OFF; - writel(state, pd->base + PD_COMMAND_REG); - - /* Wait for the state transition to complete. */ - ret = readl_poll_timeout(pd->base + PD_STATUS_REG, status, - FIELD_GET(PD_STATUS_STATE, status) == state && - (status & PD_STATUS_COMPLETE), 100, 1000); - if (ret) - return ret; - - /* Clear the completion flag. */ - writel(status, pd->base + PD_STATUS_REG); - - return 0; -} - -static int sun20i_ppu_pd_power_on(struct generic_pm_domain *genpd) -{ - const struct sun20i_ppu_pd *pd = to_sun20i_ppu_pd(genpd); - - return sun20i_ppu_pd_set_power(pd, true); -} - -static int sun20i_ppu_pd_power_off(struct generic_pm_domain *genpd) -{ - const struct sun20i_ppu_pd *pd = to_sun20i_ppu_pd(genpd); - - return sun20i_ppu_pd_set_power(pd, false); -} - -static int sun20i_ppu_probe(struct platform_device *pdev) -{ - const struct sun20i_ppu_desc *desc; - struct device *dev = &pdev->dev; - struct genpd_onecell_data *ppu; - struct sun20i_ppu_pd *pds; - struct reset_control *rst; - void __iomem *base; - struct clk *clk; - int ret; - - desc = of_device_get_match_data(dev); - if (!desc) - return -EINVAL; - - pds = devm_kcalloc(dev, desc->num_domains, sizeof(*pds), GFP_KERNEL); - if (!pds) - return -ENOMEM; - - ppu = devm_kzalloc(dev, sizeof(*ppu), GFP_KERNEL); - if (!ppu) - return -ENOMEM; - - ppu->domains = devm_kcalloc(dev, desc->num_domains, - sizeof(*ppu->domains), GFP_KERNEL); - if (!ppu->domains) - return -ENOMEM; - - ppu->num_domains = desc->num_domains; - platform_set_drvdata(pdev, ppu); - - base = devm_platform_ioremap_resource(pdev, 0); - if (IS_ERR(base)) - return PTR_ERR(base); - - clk = devm_clk_get_enabled(dev, NULL); - if (IS_ERR(clk)) - return PTR_ERR(clk); - - rst = devm_reset_control_get_exclusive(dev, NULL); - if (IS_ERR(rst)) - return PTR_ERR(rst); - - ret = reset_control_deassert(rst); - if (ret) - return ret; - - for (unsigned int i = 0; i < ppu->num_domains; ++i) { - struct sun20i_ppu_pd *pd = &pds[i]; - - pd->genpd.name = desc->names[i]; - pd->genpd.power_off = sun20i_ppu_pd_power_off; - pd->genpd.power_on = sun20i_ppu_pd_power_on; - pd->base = base + PD_REGS_SIZE * i; - - ret = pm_genpd_init(&pd->genpd, NULL, sun20i_ppu_pd_is_on(pd)); - if (ret) { - dev_warn(dev, "Failed to add '%s' domain: %d\n", - pd->genpd.name, ret); - continue; - } - - ppu->domains[i] = &pd->genpd; - } - - ret = of_genpd_add_provider_onecell(dev->of_node, ppu); - if (ret) - dev_warn(dev, "Failed to add provider: %d\n", ret); - - return 0; -} - -static const char *const sun20i_d1_ppu_pd_names[] = { - "CPU", - "VE", - "DSP", -}; - -static const struct sun20i_ppu_desc sun20i_d1_ppu_desc = { - .names = sun20i_d1_ppu_pd_names, - .num_domains = ARRAY_SIZE(sun20i_d1_ppu_pd_names), -}; - -static const struct of_device_id sun20i_ppu_of_match[] = { - { - .compatible = "allwinner,sun20i-d1-ppu", - .data = &sun20i_d1_ppu_desc, - }, - { } -}; -MODULE_DEVICE_TABLE(of, sun20i_ppu_of_match); - -static struct platform_driver sun20i_ppu_driver = { - .probe = sun20i_ppu_probe, - .driver = { - .name = "sun20i-ppu", - .of_match_table = sun20i_ppu_of_match, - /* Power domains cannot be removed while they are in use. */ - .suppress_bind_attrs = true, - }, -}; -module_platform_driver(sun20i_ppu_driver); - -MODULE_AUTHOR("Samuel Holland "); -MODULE_DESCRIPTION("Allwinner D1 PPU power domain driver"); -MODULE_LICENSE("GPL"); diff --git a/drivers/genpd/tegra/Makefile b/drivers/genpd/tegra/Makefile deleted file mode 100644 index ec8acfd2c77c..000000000000 --- a/drivers/genpd/tegra/Makefile +++ /dev/null @@ -1,2 +0,0 @@ -# SPDX-License-Identifier: GPL-2.0 -obj-$(CONFIG_SOC_TEGRA_POWERGATE_BPMP) += powergate-bpmp.o diff --git a/drivers/genpd/tegra/powergate-bpmp.c b/drivers/genpd/tegra/powergate-bpmp.c deleted file mode 100644 index 179ed895c279..000000000000 --- a/drivers/genpd/tegra/powergate-bpmp.c +++ /dev/null @@ -1,361 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0-only -/* - * Copyright (c) 2016-2017, NVIDIA CORPORATION. All rights reserved - */ - -#include -#include -#include -#include - -#include -#include - -struct tegra_powergate_info { - unsigned int id; - char *name; -}; - -struct tegra_powergate { - struct generic_pm_domain genpd; - struct tegra_bpmp *bpmp; - unsigned int id; -}; - -static inline struct tegra_powergate * -to_tegra_powergate(struct generic_pm_domain *genpd) -{ - return container_of(genpd, struct tegra_powergate, genpd); -} - -static int tegra_bpmp_powergate_set_state(struct tegra_bpmp *bpmp, - unsigned int id, u32 state) -{ - struct mrq_pg_request request; - struct tegra_bpmp_message msg; - int err; - - memset(&request, 0, sizeof(request)); - request.cmd = CMD_PG_SET_STATE; - request.id = id; - request.set_state.state = state; - - memset(&msg, 0, sizeof(msg)); - msg.mrq = MRQ_PG; - msg.tx.data = &request; - msg.tx.size = sizeof(request); - - err = tegra_bpmp_transfer(bpmp, &msg); - if (err < 0) - return err; - else if (msg.rx.ret < 0) - return -EINVAL; - - return 0; -} - -static int tegra_bpmp_powergate_get_state(struct tegra_bpmp *bpmp, - unsigned int id) -{ - struct mrq_pg_response response; - struct mrq_pg_request request; - struct tegra_bpmp_message msg; - int err; - - memset(&request, 0, sizeof(request)); - request.cmd = CMD_PG_GET_STATE; - request.id = id; - - memset(&response, 0, sizeof(response)); - - memset(&msg, 0, sizeof(msg)); - msg.mrq = MRQ_PG; - msg.tx.data = &request; - msg.tx.size = sizeof(request); - msg.rx.data = &response; - msg.rx.size = sizeof(response); - - err = tegra_bpmp_transfer(bpmp, &msg); - if (err < 0) - return PG_STATE_OFF; - else if (msg.rx.ret < 0) - return -EINVAL; - - return response.get_state.state; -} - -static int tegra_bpmp_powergate_get_max_id(struct tegra_bpmp *bpmp) -{ - struct mrq_pg_response response; - struct mrq_pg_request request; - struct tegra_bpmp_message msg; - int err; - - memset(&request, 0, sizeof(request)); - request.cmd = CMD_PG_GET_MAX_ID; - - memset(&response, 0, sizeof(response)); - - memset(&msg, 0, sizeof(msg)); - msg.mrq = MRQ_PG; - msg.tx.data = &request; - msg.tx.size = sizeof(request); - msg.rx.data = &response; - msg.rx.size = sizeof(response); - - err = tegra_bpmp_transfer(bpmp, &msg); - if (err < 0) - return err; - else if (msg.rx.ret < 0) - return -EINVAL; - - return response.get_max_id.max_id; -} - -static char *tegra_bpmp_powergate_get_name(struct tegra_bpmp *bpmp, - unsigned int id) -{ - struct mrq_pg_response response; - struct mrq_pg_request request; - struct tegra_bpmp_message msg; - int err; - - memset(&request, 0, sizeof(request)); - request.cmd = CMD_PG_GET_NAME; - request.id = id; - - memset(&response, 0, sizeof(response)); - - memset(&msg, 0, sizeof(msg)); - msg.mrq = MRQ_PG; - msg.tx.data = &request; - msg.tx.size = sizeof(request); - msg.rx.data = &response; - msg.rx.size = sizeof(response); - - err = tegra_bpmp_transfer(bpmp, &msg); - if (err < 0 || msg.rx.ret < 0) - return NULL; - - return kstrdup(response.get_name.name, GFP_KERNEL); -} - -static inline bool tegra_bpmp_powergate_is_powered(struct tegra_bpmp *bpmp, - unsigned int id) -{ - return tegra_bpmp_powergate_get_state(bpmp, id) != PG_STATE_OFF; -} - -static int tegra_powergate_power_on(struct generic_pm_domain *domain) -{ - struct tegra_powergate *powergate = to_tegra_powergate(domain); - struct tegra_bpmp *bpmp = powergate->bpmp; - - return tegra_bpmp_powergate_set_state(bpmp, powergate->id, - PG_STATE_ON); -} - -static int tegra_powergate_power_off(struct generic_pm_domain *domain) -{ - struct tegra_powergate *powergate = to_tegra_powergate(domain); - struct tegra_bpmp *bpmp = powergate->bpmp; - - return tegra_bpmp_powergate_set_state(bpmp, powergate->id, - PG_STATE_OFF); -} - -static struct tegra_powergate * -tegra_powergate_add(struct tegra_bpmp *bpmp, - const struct tegra_powergate_info *info) -{ - struct tegra_powergate *powergate; - bool off; - int err; - - off = !tegra_bpmp_powergate_is_powered(bpmp, info->id); - - powergate = devm_kzalloc(bpmp->dev, sizeof(*powergate), GFP_KERNEL); - if (!powergate) - return ERR_PTR(-ENOMEM); - - powergate->id = info->id; - powergate->bpmp = bpmp; - - powergate->genpd.name = kstrdup(info->name, GFP_KERNEL); - powergate->genpd.power_on = tegra_powergate_power_on; - powergate->genpd.power_off = tegra_powergate_power_off; - - err = pm_genpd_init(&powergate->genpd, NULL, off); - if (err < 0) { - kfree(powergate->genpd.name); - return ERR_PTR(err); - } - - return powergate; -} - -static void tegra_powergate_remove(struct tegra_powergate *powergate) -{ - struct generic_pm_domain *genpd = &powergate->genpd; - struct tegra_bpmp *bpmp = powergate->bpmp; - int err; - - err = pm_genpd_remove(genpd); - if (err < 0) - dev_err(bpmp->dev, "failed to remove power domain %s: %d\n", - genpd->name, err); - - kfree(genpd->name); -} - -static int -tegra_bpmp_probe_powergates(struct tegra_bpmp *bpmp, - struct tegra_powergate_info **powergatesp) -{ - struct tegra_powergate_info *powergates; - unsigned int max_id, id, count = 0; - unsigned int num_holes = 0; - int err; - - err = tegra_bpmp_powergate_get_max_id(bpmp); - if (err < 0) - return err; - - max_id = err; - - dev_dbg(bpmp->dev, "maximum powergate ID: %u\n", max_id); - - powergates = kcalloc(max_id + 1, sizeof(*powergates), GFP_KERNEL); - if (!powergates) - return -ENOMEM; - - for (id = 0; id <= max_id; id++) { - struct tegra_powergate_info *info = &powergates[count]; - - info->name = tegra_bpmp_powergate_get_name(bpmp, id); - if (!info->name || info->name[0] == '\0') { - num_holes++; - continue; - } - - info->id = id; - count++; - } - - dev_dbg(bpmp->dev, "holes: %u\n", num_holes); - - *powergatesp = powergates; - - return count; -} - -static int tegra_bpmp_add_powergates(struct tegra_bpmp *bpmp, - struct tegra_powergate_info *powergates, - unsigned int count) -{ - struct genpd_onecell_data *genpd = &bpmp->genpd; - struct generic_pm_domain **domains; - struct tegra_powergate *powergate; - unsigned int i; - int err; - - domains = kcalloc(count, sizeof(*domains), GFP_KERNEL); - if (!domains) - return -ENOMEM; - - for (i = 0; i < count; i++) { - powergate = tegra_powergate_add(bpmp, &powergates[i]); - if (IS_ERR(powergate)) { - err = PTR_ERR(powergate); - goto remove; - } - - dev_dbg(bpmp->dev, "added power domain %s\n", - powergate->genpd.name); - domains[i] = &powergate->genpd; - } - - genpd->num_domains = count; - genpd->domains = domains; - - return 0; - -remove: - while (i--) { - powergate = to_tegra_powergate(domains[i]); - tegra_powergate_remove(powergate); - } - - kfree(domains); - return err; -} - -static void tegra_bpmp_remove_powergates(struct tegra_bpmp *bpmp) -{ - struct genpd_onecell_data *genpd = &bpmp->genpd; - unsigned int i = genpd->num_domains; - struct tegra_powergate *powergate; - - while (i--) { - dev_dbg(bpmp->dev, "removing power domain %s\n", - genpd->domains[i]->name); - powergate = to_tegra_powergate(genpd->domains[i]); - tegra_powergate_remove(powergate); - } -} - -static struct generic_pm_domain * -tegra_powergate_xlate(struct of_phandle_args *spec, void *data) -{ - struct generic_pm_domain *domain = ERR_PTR(-ENOENT); - struct genpd_onecell_data *genpd = data; - unsigned int i; - - for (i = 0; i < genpd->num_domains; i++) { - struct tegra_powergate *powergate; - - powergate = to_tegra_powergate(genpd->domains[i]); - if (powergate->id == spec->args[0]) { - domain = &powergate->genpd; - break; - } - } - - return domain; -} - -int tegra_bpmp_init_powergates(struct tegra_bpmp *bpmp) -{ - struct device_node *np = bpmp->dev->of_node; - struct tegra_powergate_info *powergates; - struct device *dev = bpmp->dev; - unsigned int count, i; - int err; - - err = tegra_bpmp_probe_powergates(bpmp, &powergates); - if (err < 0) - return err; - - count = err; - - dev_dbg(dev, "%u power domains probed\n", count); - - err = tegra_bpmp_add_powergates(bpmp, powergates, count); - if (err < 0) - goto free; - - bpmp->genpd.xlate = tegra_powergate_xlate; - - err = of_genpd_add_provider_onecell(np, &bpmp->genpd); - if (err < 0) { - dev_err(dev, "failed to add power domain provider: %d\n", err); - tegra_bpmp_remove_powergates(bpmp); - } - -free: - for (i = 0; i < count; i++) - kfree(powergates[i].name); - - kfree(powergates); - return err; -} diff --git a/drivers/genpd/ti/Makefile b/drivers/genpd/ti/Makefile deleted file mode 100644 index 69580afbb436..000000000000 --- a/drivers/genpd/ti/Makefile +++ /dev/null @@ -1,3 +0,0 @@ -# SPDX-License-Identifier: GPL-2.0 -obj-$(CONFIG_ARCH_OMAP2PLUS) += omap_prm.o -obj-$(CONFIG_TI_SCI_PM_DOMAINS) += ti_sci_pm_domains.o diff --git a/drivers/genpd/ti/omap_prm.c b/drivers/genpd/ti/omap_prm.c deleted file mode 100644 index c2feae3a634c..000000000000 --- a/drivers/genpd/ti/omap_prm.c +++ /dev/null @@ -1,989 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * OMAP2+ PRM driver - * - * Copyright (C) 2019 Texas Instruments Incorporated - http://www.ti.com/ - * Tero Kristo - */ - -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -#include - -enum omap_prm_domain_mode { - OMAP_PRMD_OFF, - OMAP_PRMD_RETENTION, - OMAP_PRMD_ON_INACTIVE, - OMAP_PRMD_ON_ACTIVE, -}; - -struct omap_prm_domain_map { - unsigned int usable_modes; /* Mask of hardware supported modes */ - unsigned long statechange:1; /* Optional low-power state change */ - unsigned long logicretstate:1; /* Optional logic off mode */ -}; - -struct omap_prm_domain { - struct device *dev; - struct omap_prm *prm; - struct generic_pm_domain pd; - u16 pwrstctrl; - u16 pwrstst; - const struct omap_prm_domain_map *cap; - u32 pwrstctrl_saved; - unsigned int uses_pm_clk:1; -}; - -struct omap_rst_map { - s8 rst; - s8 st; -}; - -struct omap_prm_data { - u32 base; - const char *name; - const char *clkdm_name; - u16 pwrstctrl; - u16 pwrstst; - const struct omap_prm_domain_map *dmap; - u16 rstctrl; - u16 rstst; - const struct omap_rst_map *rstmap; - u8 flags; -}; - -struct omap_prm { - const struct omap_prm_data *data; - void __iomem *base; - struct omap_prm_domain *prmd; -}; - -struct omap_reset_data { - struct reset_controller_dev rcdev; - struct omap_prm *prm; - u32 mask; - spinlock_t lock; - struct clockdomain *clkdm; - struct device *dev; -}; - -#define genpd_to_prm_domain(gpd) container_of(gpd, struct omap_prm_domain, pd) -#define to_omap_reset_data(p) container_of((p), struct omap_reset_data, rcdev) - -#define OMAP_MAX_RESETS 8 -#define OMAP_RESET_MAX_WAIT 10000 - -#define OMAP_PRM_HAS_RSTCTRL BIT(0) -#define OMAP_PRM_HAS_RSTST BIT(1) -#define OMAP_PRM_HAS_NO_CLKDM BIT(2) -#define OMAP_PRM_RET_WHEN_IDLE BIT(3) - -#define OMAP_PRM_HAS_RESETS (OMAP_PRM_HAS_RSTCTRL | OMAP_PRM_HAS_RSTST) - -#define PRM_STATE_MAX_WAIT 10000 -#define PRM_LOGICRETSTATE BIT(2) -#define PRM_LOWPOWERSTATECHANGE BIT(4) -#define PRM_POWERSTATE_MASK OMAP_PRMD_ON_ACTIVE - -#define PRM_ST_INTRANSITION BIT(20) - -static const struct omap_prm_domain_map omap_prm_all = { - .usable_modes = BIT(OMAP_PRMD_ON_ACTIVE) | BIT(OMAP_PRMD_ON_INACTIVE) | - BIT(OMAP_PRMD_RETENTION) | BIT(OMAP_PRMD_OFF), - .statechange = 1, - .logicretstate = 1, -}; - -static const struct omap_prm_domain_map omap_prm_noinact = { - .usable_modes = BIT(OMAP_PRMD_ON_ACTIVE) | BIT(OMAP_PRMD_RETENTION) | - BIT(OMAP_PRMD_OFF), - .statechange = 1, - .logicretstate = 1, -}; - -static const struct omap_prm_domain_map omap_prm_nooff = { - .usable_modes = BIT(OMAP_PRMD_ON_ACTIVE) | BIT(OMAP_PRMD_ON_INACTIVE) | - BIT(OMAP_PRMD_RETENTION), - .statechange = 1, - .logicretstate = 1, -}; - -static const struct omap_prm_domain_map omap_prm_onoff_noauto = { - .usable_modes = BIT(OMAP_PRMD_ON_ACTIVE) | BIT(OMAP_PRMD_OFF), - .statechange = 1, -}; - -static const struct omap_prm_domain_map omap_prm_alwon = { - .usable_modes = BIT(OMAP_PRMD_ON_ACTIVE), -}; - -static const struct omap_prm_domain_map omap_prm_reton = { - .usable_modes = BIT(OMAP_PRMD_ON_ACTIVE) | BIT(OMAP_PRMD_RETENTION), - .statechange = 1, - .logicretstate = 1, -}; - -static const struct omap_rst_map rst_map_0[] = { - { .rst = 0, .st = 0 }, - { .rst = -1 }, -}; - -static const struct omap_rst_map rst_map_01[] = { - { .rst = 0, .st = 0 }, - { .rst = 1, .st = 1 }, - { .rst = -1 }, -}; - -static const struct omap_rst_map rst_map_012[] = { - { .rst = 0, .st = 0 }, - { .rst = 1, .st = 1 }, - { .rst = 2, .st = 2 }, - { .rst = -1 }, -}; - -static const struct omap_prm_data omap4_prm_data[] = { - { - .name = "mpu", .base = 0x4a306300, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_reton, - }, - { - .name = "tesla", .base = 0x4a306400, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_noinact, - .rstctrl = 0x10, .rstst = 0x14, .rstmap = rst_map_01 - }, - { - .name = "abe", .base = 0x4a306500, - .pwrstctrl = 0, .pwrstst = 0x4, .dmap = &omap_prm_all, - }, - { - .name = "always_on_core", .base = 0x4a306600, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_alwon, - }, - { - .name = "core", .base = 0x4a306700, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_reton, - .rstctrl = 0x210, .rstst = 0x214, .clkdm_name = "ducati", - .rstmap = rst_map_012, - .flags = OMAP_PRM_RET_WHEN_IDLE, - }, - { - .name = "ivahd", .base = 0x4a306f00, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_noinact, - .rstctrl = 0x10, .rstst = 0x14, .rstmap = rst_map_012 - }, - { - .name = "cam", .base = 0x4a307000, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto, - }, - { - .name = "dss", .base = 0x4a307100, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_noinact - }, - { - .name = "gfx", .base = 0x4a307200, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto - }, - { - .name = "l3init", .base = 0x4a307300, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_reton - }, - { - .name = "l4per", .base = 0x4a307400, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_reton, - .flags = OMAP_PRM_RET_WHEN_IDLE, - }, - { - .name = "cefuse", .base = 0x4a307600, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto - }, - { - .name = "wkup", .base = 0x4a307700, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_alwon - }, - { - .name = "emu", .base = 0x4a307900, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto - }, - { - .name = "device", .base = 0x4a307b00, - .rstctrl = 0x0, .rstst = 0x4, .rstmap = rst_map_01, - .flags = OMAP_PRM_HAS_RSTCTRL | OMAP_PRM_HAS_NO_CLKDM - }, - { }, -}; - -static const struct omap_prm_data omap5_prm_data[] = { - { - .name = "mpu", .base = 0x4ae06300, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_reton, - }, - { - .name = "dsp", .base = 0x4ae06400, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_noinact, - .rstctrl = 0x10, .rstst = 0x14, .rstmap = rst_map_01 - }, - { - .name = "abe", .base = 0x4ae06500, - .pwrstctrl = 0, .pwrstst = 0x4, .dmap = &omap_prm_nooff, - }, - { - .name = "coreaon", .base = 0x4ae06600, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_alwon - }, - { - .name = "core", .base = 0x4ae06700, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_reton, - .rstctrl = 0x210, .rstst = 0x214, .clkdm_name = "ipu", - .rstmap = rst_map_012 - }, - { - .name = "iva", .base = 0x4ae07200, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_noinact, - .rstctrl = 0x10, .rstst = 0x14, .rstmap = rst_map_012 - }, - { - .name = "cam", .base = 0x4ae07300, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto - }, - { - .name = "dss", .base = 0x4ae07400, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_noinact - }, - { - .name = "gpu", .base = 0x4ae07500, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto - }, - { - .name = "l3init", .base = 0x4ae07600, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_reton - }, - { - .name = "custefuse", .base = 0x4ae07700, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto - }, - { - .name = "wkupaon", .base = 0x4ae07800, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_alwon - }, - { - .name = "emu", .base = 0x4ae07a00, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto - }, - { - .name = "device", .base = 0x4ae07c00, - .rstctrl = 0x0, .rstst = 0x4, .rstmap = rst_map_01, - .flags = OMAP_PRM_HAS_RSTCTRL | OMAP_PRM_HAS_NO_CLKDM - }, - { }, -}; - -static const struct omap_prm_data dra7_prm_data[] = { - { - .name = "mpu", .base = 0x4ae06300, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_reton, - }, - { - .name = "dsp1", .base = 0x4ae06400, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto, - .rstctrl = 0x10, .rstst = 0x14, .rstmap = rst_map_01, - }, - { - .name = "ipu", .base = 0x4ae06500, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto, - .rstctrl = 0x10, .rstst = 0x14, .rstmap = rst_map_012, - .clkdm_name = "ipu1" - }, - { - .name = "coreaon", .base = 0x4ae06628, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_alwon, - }, - { - .name = "core", .base = 0x4ae06700, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_alwon, - .rstctrl = 0x210, .rstst = 0x214, .rstmap = rst_map_012, - .clkdm_name = "ipu2" - }, - { - .name = "iva", .base = 0x4ae06f00, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto, - .rstctrl = 0x10, .rstst = 0x14, .rstmap = rst_map_012, - }, - { - .name = "cam", .base = 0x4ae07000, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto, - }, - { - .name = "dss", .base = 0x4ae07100, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto, - }, - { - .name = "gpu", .base = 0x4ae07200, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto, - }, - { - .name = "l3init", .base = 0x4ae07300, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_alwon, - .rstctrl = 0x10, .rstst = 0x14, .rstmap = rst_map_01, - .clkdm_name = "pcie" - }, - { - .name = "l4per", .base = 0x4ae07400, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_alwon, - }, - { - .name = "custefuse", .base = 0x4ae07600, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto, - }, - { - .name = "wkupaon", .base = 0x4ae07724, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_alwon, - }, - { - .name = "emu", .base = 0x4ae07900, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto, - }, - { - .name = "dsp2", .base = 0x4ae07b00, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto, - .rstctrl = 0x10, .rstst = 0x14, .rstmap = rst_map_01 - }, - { - .name = "eve1", .base = 0x4ae07b40, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto, - .rstctrl = 0x10, .rstst = 0x14, .rstmap = rst_map_01 - }, - { - .name = "eve2", .base = 0x4ae07b80, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto, - .rstctrl = 0x10, .rstst = 0x14, .rstmap = rst_map_01 - }, - { - .name = "eve3", .base = 0x4ae07bc0, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto, - .rstctrl = 0x10, .rstst = 0x14, .rstmap = rst_map_01 - }, - { - .name = "eve4", .base = 0x4ae07c00, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto, - .rstctrl = 0x10, .rstst = 0x14, .rstmap = rst_map_01 - }, - { - .name = "rtc", .base = 0x4ae07c60, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_alwon, - }, - { - .name = "vpe", .base = 0x4ae07c80, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto, - }, - { }, -}; - -static const struct omap_rst_map am3_per_rst_map[] = { - { .rst = 1 }, - { .rst = -1 }, -}; - -static const struct omap_rst_map am3_wkup_rst_map[] = { - { .rst = 3, .st = 5 }, - { .rst = -1 }, -}; - -static const struct omap_prm_data am3_prm_data[] = { - { - .name = "per", .base = 0x44e00c00, - .pwrstctrl = 0xc, .pwrstst = 0x8, .dmap = &omap_prm_noinact, - .rstctrl = 0x0, .rstmap = am3_per_rst_map, - .flags = OMAP_PRM_HAS_RSTCTRL, .clkdm_name = "pruss_ocp" - }, - { - .name = "wkup", .base = 0x44e00d00, - .pwrstctrl = 0x4, .pwrstst = 0x4, .dmap = &omap_prm_alwon, - .rstctrl = 0x0, .rstst = 0xc, .rstmap = am3_wkup_rst_map, - .flags = OMAP_PRM_HAS_RSTCTRL | OMAP_PRM_HAS_NO_CLKDM - }, - { - .name = "mpu", .base = 0x44e00e00, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_noinact, - }, - { - .name = "device", .base = 0x44e00f00, - .rstctrl = 0x0, .rstst = 0x8, .rstmap = rst_map_01, - .flags = OMAP_PRM_HAS_RSTCTRL | OMAP_PRM_HAS_NO_CLKDM - }, - { - .name = "rtc", .base = 0x44e01000, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_alwon, - }, - { - .name = "gfx", .base = 0x44e01100, - .pwrstctrl = 0, .pwrstst = 0x10, .dmap = &omap_prm_noinact, - .rstctrl = 0x4, .rstst = 0x14, .rstmap = rst_map_0, .clkdm_name = "gfx_l3", - }, - { - .name = "cefuse", .base = 0x44e01200, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto, - }, - { }, -}; - -static const struct omap_rst_map am4_per_rst_map[] = { - { .rst = 1, .st = 0 }, - { .rst = -1 }, -}; - -static const struct omap_rst_map am4_device_rst_map[] = { - { .rst = 0, .st = 1 }, - { .rst = 1, .st = 0 }, - { .rst = -1 }, -}; - -static const struct omap_prm_data am4_prm_data[] = { - { - .name = "mpu", .base = 0x44df0300, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_noinact, - }, - { - .name = "gfx", .base = 0x44df0400, - .pwrstctrl = 0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto, - .rstctrl = 0x10, .rstst = 0x14, .rstmap = rst_map_0, .clkdm_name = "gfx_l3", - }, - { - .name = "rtc", .base = 0x44df0500, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_alwon, - }, - { - .name = "tamper", .base = 0x44df0600, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_alwon, - }, - { - .name = "cefuse", .base = 0x44df0700, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto, - }, - { - .name = "per", .base = 0x44df0800, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_noinact, - .rstctrl = 0x10, .rstst = 0x14, .rstmap = am4_per_rst_map, - .clkdm_name = "pruss_ocp" - }, - { - .name = "wkup", .base = 0x44df2000, - .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_alwon, - .rstctrl = 0x10, .rstst = 0x14, .rstmap = am3_wkup_rst_map, - .flags = OMAP_PRM_HAS_NO_CLKDM - }, - { - .name = "device", .base = 0x44df4000, - .rstctrl = 0x0, .rstst = 0x4, .rstmap = am4_device_rst_map, - .flags = OMAP_PRM_HAS_RSTCTRL | OMAP_PRM_HAS_NO_CLKDM - }, - { }, -}; - -static const struct of_device_id omap_prm_id_table[] = { - { .compatible = "ti,omap4-prm-inst", .data = omap4_prm_data }, - { .compatible = "ti,omap5-prm-inst", .data = omap5_prm_data }, - { .compatible = "ti,dra7-prm-inst", .data = dra7_prm_data }, - { .compatible = "ti,am3-prm-inst", .data = am3_prm_data }, - { .compatible = "ti,am4-prm-inst", .data = am4_prm_data }, - { }, -}; - -#ifdef DEBUG -static void omap_prm_domain_show_state(struct omap_prm_domain *prmd, - const char *desc) -{ - dev_dbg(prmd->dev, "%s %s: %08x/%08x\n", - prmd->pd.name, desc, - readl_relaxed(prmd->prm->base + prmd->pwrstctrl), - readl_relaxed(prmd->prm->base + prmd->pwrstst)); -} -#else -static inline void omap_prm_domain_show_state(struct omap_prm_domain *prmd, - const char *desc) -{ -} -#endif - -static int omap_prm_domain_power_on(struct generic_pm_domain *domain) -{ - struct omap_prm_domain *prmd; - int ret; - u32 v, mode; - - prmd = genpd_to_prm_domain(domain); - if (!prmd->cap) - return 0; - - omap_prm_domain_show_state(prmd, "on: previous state"); - - if (prmd->pwrstctrl_saved) - v = prmd->pwrstctrl_saved; - else - v = readl_relaxed(prmd->prm->base + prmd->pwrstctrl); - - if (prmd->prm->data->flags & OMAP_PRM_RET_WHEN_IDLE) - mode = OMAP_PRMD_RETENTION; - else - mode = OMAP_PRMD_ON_ACTIVE; - - writel_relaxed((v & ~PRM_POWERSTATE_MASK) | mode, - prmd->prm->base + prmd->pwrstctrl); - - /* wait for the transition bit to get cleared */ - ret = readl_relaxed_poll_timeout(prmd->prm->base + prmd->pwrstst, - v, !(v & PRM_ST_INTRANSITION), 1, - PRM_STATE_MAX_WAIT); - if (ret) - dev_err(prmd->dev, "%s: %s timed out\n", - prmd->pd.name, __func__); - - omap_prm_domain_show_state(prmd, "on: new state"); - - return ret; -} - -/* No need to check for holes in the mask for the lowest mode */ -static int omap_prm_domain_find_lowest(struct omap_prm_domain *prmd) -{ - return __ffs(prmd->cap->usable_modes); -} - -static int omap_prm_domain_power_off(struct generic_pm_domain *domain) -{ - struct omap_prm_domain *prmd; - int ret; - u32 v; - - prmd = genpd_to_prm_domain(domain); - if (!prmd->cap) - return 0; - - omap_prm_domain_show_state(prmd, "off: previous state"); - - v = readl_relaxed(prmd->prm->base + prmd->pwrstctrl); - prmd->pwrstctrl_saved = v; - - v &= ~PRM_POWERSTATE_MASK; - v |= omap_prm_domain_find_lowest(prmd); - - if (prmd->cap->statechange) - v |= PRM_LOWPOWERSTATECHANGE; - if (prmd->cap->logicretstate) - v &= ~PRM_LOGICRETSTATE; - else - v |= PRM_LOGICRETSTATE; - - writel_relaxed(v, prmd->prm->base + prmd->pwrstctrl); - - /* wait for the transition bit to get cleared */ - ret = readl_relaxed_poll_timeout(prmd->prm->base + prmd->pwrstst, - v, !(v & PRM_ST_INTRANSITION), 1, - PRM_STATE_MAX_WAIT); - if (ret) - dev_warn(prmd->dev, "%s: %s timed out\n", - __func__, prmd->pd.name); - - omap_prm_domain_show_state(prmd, "off: new state"); - - return 0; -} - -/* - * Note that ti-sysc already manages the module clocks separately so - * no need to manage those. Interconnect instances need clocks managed - * for simple-pm-bus. - */ -static int omap_prm_domain_attach_clock(struct device *dev, - struct omap_prm_domain *prmd) -{ - struct device_node *np = dev->of_node; - int error; - - if (!of_device_is_compatible(np, "simple-pm-bus")) - return 0; - - if (!of_property_read_bool(np, "clocks")) - return 0; - - error = pm_clk_create(dev); - if (error) - return error; - - error = of_pm_clk_add_clks(dev); - if (error < 0) { - pm_clk_destroy(dev); - return error; - } - - prmd->uses_pm_clk = 1; - - return 0; -} - -static int omap_prm_domain_attach_dev(struct generic_pm_domain *domain, - struct device *dev) -{ - struct generic_pm_domain_data *genpd_data; - struct of_phandle_args pd_args; - struct omap_prm_domain *prmd; - struct device_node *np; - int ret; - - prmd = genpd_to_prm_domain(domain); - np = dev->of_node; - - ret = of_parse_phandle_with_args(np, "power-domains", - "#power-domain-cells", 0, &pd_args); - if (ret < 0) - return ret; - - if (pd_args.args_count != 0) - dev_warn(dev, "%s: unusupported #power-domain-cells: %i\n", - prmd->pd.name, pd_args.args_count); - - genpd_data = dev_gpd_data(dev); - genpd_data->data = NULL; - - ret = omap_prm_domain_attach_clock(dev, prmd); - if (ret) - return ret; - - return 0; -} - -static void omap_prm_domain_detach_dev(struct generic_pm_domain *domain, - struct device *dev) -{ - struct generic_pm_domain_data *genpd_data; - struct omap_prm_domain *prmd; - - prmd = genpd_to_prm_domain(domain); - if (prmd->uses_pm_clk) - pm_clk_destroy(dev); - genpd_data = dev_gpd_data(dev); - genpd_data->data = NULL; -} - -static int omap_prm_domain_init(struct device *dev, struct omap_prm *prm) -{ - struct omap_prm_domain *prmd; - struct device_node *np = dev->of_node; - const struct omap_prm_data *data; - const char *name; - int error; - - if (!of_property_present(dev->of_node, "#power-domain-cells")) - return 0; - - of_node_put(dev->of_node); - - prmd = devm_kzalloc(dev, sizeof(*prmd), GFP_KERNEL); - if (!prmd) - return -ENOMEM; - - data = prm->data; - name = devm_kasprintf(dev, GFP_KERNEL, "prm_%s", - data->name); - - prmd->dev = dev; - prmd->prm = prm; - prmd->cap = prmd->prm->data->dmap; - prmd->pwrstctrl = prmd->prm->data->pwrstctrl; - prmd->pwrstst = prmd->prm->data->pwrstst; - - prmd->pd.name = name; - prmd->pd.power_on = omap_prm_domain_power_on; - prmd->pd.power_off = omap_prm_domain_power_off; - prmd->pd.attach_dev = omap_prm_domain_attach_dev; - prmd->pd.detach_dev = omap_prm_domain_detach_dev; - prmd->pd.flags = GENPD_FLAG_PM_CLK; - - pm_genpd_init(&prmd->pd, NULL, true); - error = of_genpd_add_provider_simple(np, &prmd->pd); - if (error) - pm_genpd_remove(&prmd->pd); - else - prm->prmd = prmd; - - return error; -} - -static bool _is_valid_reset(struct omap_reset_data *reset, unsigned long id) -{ - if (reset->mask & BIT(id)) - return true; - - return false; -} - -static int omap_reset_get_st_bit(struct omap_reset_data *reset, - unsigned long id) -{ - const struct omap_rst_map *map = reset->prm->data->rstmap; - - while (map->rst >= 0) { - if (map->rst == id) - return map->st; - - map++; - } - - return id; -} - -static int omap_reset_status(struct reset_controller_dev *rcdev, - unsigned long id) -{ - struct omap_reset_data *reset = to_omap_reset_data(rcdev); - u32 v; - int st_bit = omap_reset_get_st_bit(reset, id); - bool has_rstst = reset->prm->data->rstst || - (reset->prm->data->flags & OMAP_PRM_HAS_RSTST); - - /* Check if we have rstst */ - if (!has_rstst) - return -ENOTSUPP; - - /* Check if hw reset line is asserted */ - v = readl_relaxed(reset->prm->base + reset->prm->data->rstctrl); - if (v & BIT(id)) - return 1; - - /* - * Check reset status, high value means reset sequence has been - * completed successfully so we can return 0 here (reset deasserted) - */ - v = readl_relaxed(reset->prm->base + reset->prm->data->rstst); - v >>= st_bit; - v &= 1; - - return !v; -} - -static int omap_reset_assert(struct reset_controller_dev *rcdev, - unsigned long id) -{ - struct omap_reset_data *reset = to_omap_reset_data(rcdev); - u32 v; - unsigned long flags; - - /* assert the reset control line */ - spin_lock_irqsave(&reset->lock, flags); - v = readl_relaxed(reset->prm->base + reset->prm->data->rstctrl); - v |= 1 << id; - writel_relaxed(v, reset->prm->base + reset->prm->data->rstctrl); - spin_unlock_irqrestore(&reset->lock, flags); - - return 0; -} - -static int omap_reset_deassert(struct reset_controller_dev *rcdev, - unsigned long id) -{ - struct omap_reset_data *reset = to_omap_reset_data(rcdev); - u32 v; - int st_bit; - bool has_rstst; - unsigned long flags; - struct ti_prm_platform_data *pdata = dev_get_platdata(reset->dev); - int ret = 0; - - /* Nothing to do if the reset is already deasserted */ - if (!omap_reset_status(rcdev, id)) - return 0; - - has_rstst = reset->prm->data->rstst || - (reset->prm->data->flags & OMAP_PRM_HAS_RSTST); - - if (has_rstst) { - st_bit = omap_reset_get_st_bit(reset, id); - - /* Clear the reset status by writing 1 to the status bit */ - v = 1 << st_bit; - writel_relaxed(v, reset->prm->base + reset->prm->data->rstst); - } - - if (reset->clkdm) - pdata->clkdm_deny_idle(reset->clkdm); - - /* de-assert the reset control line */ - spin_lock_irqsave(&reset->lock, flags); - v = readl_relaxed(reset->prm->base + reset->prm->data->rstctrl); - v &= ~(1 << id); - writel_relaxed(v, reset->prm->base + reset->prm->data->rstctrl); - spin_unlock_irqrestore(&reset->lock, flags); - - /* wait for the reset bit to clear */ - ret = readl_relaxed_poll_timeout_atomic(reset->prm->base + - reset->prm->data->rstctrl, - v, !(v & BIT(id)), 1, - OMAP_RESET_MAX_WAIT); - if (ret) - pr_err("%s: timedout waiting for %s:%lu\n", __func__, - reset->prm->data->name, id); - - /* wait for the status to be set */ - if (has_rstst) { - ret = readl_relaxed_poll_timeout_atomic(reset->prm->base + - reset->prm->data->rstst, - v, v & BIT(st_bit), 1, - OMAP_RESET_MAX_WAIT); - if (ret) - pr_err("%s: timedout waiting for %s:%lu\n", __func__, - reset->prm->data->name, id); - } - - if (reset->clkdm) - pdata->clkdm_allow_idle(reset->clkdm); - - return ret; -} - -static const struct reset_control_ops omap_reset_ops = { - .assert = omap_reset_assert, - .deassert = omap_reset_deassert, - .status = omap_reset_status, -}; - -static int omap_prm_reset_xlate(struct reset_controller_dev *rcdev, - const struct of_phandle_args *reset_spec) -{ - struct omap_reset_data *reset = to_omap_reset_data(rcdev); - - if (!_is_valid_reset(reset, reset_spec->args[0])) - return -EINVAL; - - return reset_spec->args[0]; -} - -static int omap_prm_reset_init(struct platform_device *pdev, - struct omap_prm *prm) -{ - struct omap_reset_data *reset; - const struct omap_rst_map *map; - struct ti_prm_platform_data *pdata = dev_get_platdata(&pdev->dev); - char buf[32]; - u32 v; - - /* - * Check if we have controllable resets. If either rstctrl is non-zero - * or OMAP_PRM_HAS_RSTCTRL flag is set, we have reset control register - * for the domain. - */ - if (!prm->data->rstctrl && !(prm->data->flags & OMAP_PRM_HAS_RSTCTRL)) - return 0; - - /* Check if we have the pdata callbacks in place */ - if (!pdata || !pdata->clkdm_lookup || !pdata->clkdm_deny_idle || - !pdata->clkdm_allow_idle) - return -EINVAL; - - map = prm->data->rstmap; - if (!map) - return -EINVAL; - - reset = devm_kzalloc(&pdev->dev, sizeof(*reset), GFP_KERNEL); - if (!reset) - return -ENOMEM; - - reset->rcdev.owner = THIS_MODULE; - reset->rcdev.ops = &omap_reset_ops; - reset->rcdev.of_node = pdev->dev.of_node; - reset->rcdev.nr_resets = OMAP_MAX_RESETS; - reset->rcdev.of_xlate = omap_prm_reset_xlate; - reset->rcdev.of_reset_n_cells = 1; - reset->dev = &pdev->dev; - spin_lock_init(&reset->lock); - - reset->prm = prm; - - sprintf(buf, "%s_clkdm", prm->data->clkdm_name ? prm->data->clkdm_name : - prm->data->name); - - if (!(prm->data->flags & OMAP_PRM_HAS_NO_CLKDM)) { - reset->clkdm = pdata->clkdm_lookup(buf); - if (!reset->clkdm) - return -EINVAL; - } - - while (map->rst >= 0) { - reset->mask |= BIT(map->rst); - map++; - } - - /* Quirk handling to assert rst_map_012 bits on reset and avoid errors */ - if (prm->data->rstmap == rst_map_012) { - v = readl_relaxed(reset->prm->base + reset->prm->data->rstctrl); - if ((v & reset->mask) != reset->mask) { - dev_dbg(&pdev->dev, "Asserting all resets: %08x\n", v); - writel_relaxed(reset->mask, reset->prm->base + - reset->prm->data->rstctrl); - } - } - - return devm_reset_controller_register(&pdev->dev, &reset->rcdev); -} - -static int omap_prm_probe(struct platform_device *pdev) -{ - struct resource *res; - const struct omap_prm_data *data; - struct omap_prm *prm; - int ret; - - data = of_device_get_match_data(&pdev->dev); - if (!data) - return -ENOTSUPP; - - prm = devm_kzalloc(&pdev->dev, sizeof(*prm), GFP_KERNEL); - if (!prm) - return -ENOMEM; - - prm->base = devm_platform_get_and_ioremap_resource(pdev, 0, &res); - if (IS_ERR(prm->base)) - return PTR_ERR(prm->base); - - while (data->base != res->start) { - if (!data->base) - return -EINVAL; - data++; - } - - prm->data = data; - - ret = omap_prm_domain_init(&pdev->dev, prm); - if (ret) - return ret; - - ret = omap_prm_reset_init(pdev, prm); - if (ret) - goto err_domain; - - return 0; - -err_domain: - of_genpd_del_provider(pdev->dev.of_node); - pm_genpd_remove(&prm->prmd->pd); - - return ret; -} - -static struct platform_driver omap_prm_driver = { - .probe = omap_prm_probe, - .driver = { - .name = KBUILD_MODNAME, - .of_match_table = omap_prm_id_table, - }, -}; -builtin_platform_driver(omap_prm_driver); diff --git a/drivers/genpd/ti/ti_sci_pm_domains.c b/drivers/genpd/ti/ti_sci_pm_domains.c deleted file mode 100644 index 34645104fe45..000000000000 --- a/drivers/genpd/ti/ti_sci_pm_domains.c +++ /dev/null @@ -1,204 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0-only -/* - * TI SCI Generic Power Domain Driver - * - * Copyright (C) 2015-2017 Texas Instruments Incorporated - http://www.ti.com/ - * J Keerthy - * Dave Gerlach - */ - -#include -#include -#include -#include -#include -#include -#include -#include - -/** - * struct ti_sci_genpd_provider: holds common TI SCI genpd provider data - * @ti_sci: handle to TI SCI protocol driver that provides ops to - * communicate with system control processor. - * @dev: pointer to dev for the driver for devm allocs - * @pd_list: list of all the power domains on the device - * @data: onecell data for genpd core - */ -struct ti_sci_genpd_provider { - const struct ti_sci_handle *ti_sci; - struct device *dev; - struct list_head pd_list; - struct genpd_onecell_data data; -}; - -/** - * struct ti_sci_pm_domain: TI specific data needed for power domain - * @idx: index of the device that identifies it with the system - * control processor. - * @exclusive: Permissions for exclusive request or shared request of the - * device. - * @pd: generic_pm_domain for use with the genpd framework - * @node: link for the genpd list - * @parent: link to the parent TI SCI genpd provider - */ -struct ti_sci_pm_domain { - int idx; - u8 exclusive; - struct generic_pm_domain pd; - struct list_head node; - struct ti_sci_genpd_provider *parent; -}; - -#define genpd_to_ti_sci_pd(gpd) container_of(gpd, struct ti_sci_pm_domain, pd) - -/* - * ti_sci_pd_power_off(): genpd power down hook - * @domain: pointer to the powerdomain to power off - */ -static int ti_sci_pd_power_off(struct generic_pm_domain *domain) -{ - struct ti_sci_pm_domain *pd = genpd_to_ti_sci_pd(domain); - const struct ti_sci_handle *ti_sci = pd->parent->ti_sci; - - return ti_sci->ops.dev_ops.put_device(ti_sci, pd->idx); -} - -/* - * ti_sci_pd_power_on(): genpd power up hook - * @domain: pointer to the powerdomain to power on - */ -static int ti_sci_pd_power_on(struct generic_pm_domain *domain) -{ - struct ti_sci_pm_domain *pd = genpd_to_ti_sci_pd(domain); - const struct ti_sci_handle *ti_sci = pd->parent->ti_sci; - - if (pd->exclusive) - return ti_sci->ops.dev_ops.get_device_exclusive(ti_sci, - pd->idx); - else - return ti_sci->ops.dev_ops.get_device(ti_sci, pd->idx); -} - -/* - * ti_sci_pd_xlate(): translation service for TI SCI genpds - * @genpdspec: DT identification data for the genpd - * @data: genpd core data for all the powerdomains on the device - */ -static struct generic_pm_domain *ti_sci_pd_xlate( - struct of_phandle_args *genpdspec, - void *data) -{ - struct genpd_onecell_data *genpd_data = data; - unsigned int idx = genpdspec->args[0]; - - if (genpdspec->args_count != 1 && genpdspec->args_count != 2) - return ERR_PTR(-EINVAL); - - if (idx >= genpd_data->num_domains) { - pr_err("%s: invalid domain index %u\n", __func__, idx); - return ERR_PTR(-EINVAL); - } - - if (!genpd_data->domains[idx]) - return ERR_PTR(-ENOENT); - - genpd_to_ti_sci_pd(genpd_data->domains[idx])->exclusive = - genpdspec->args[1]; - - return genpd_data->domains[idx]; -} - -static const struct of_device_id ti_sci_pm_domain_matches[] = { - { .compatible = "ti,sci-pm-domain", }, - { }, -}; -MODULE_DEVICE_TABLE(of, ti_sci_pm_domain_matches); - -static int ti_sci_pm_domain_probe(struct platform_device *pdev) -{ - struct device *dev = &pdev->dev; - struct ti_sci_genpd_provider *pd_provider; - struct ti_sci_pm_domain *pd; - struct device_node *np; - struct of_phandle_args args; - int ret; - u32 max_id = 0; - int index; - - pd_provider = devm_kzalloc(dev, sizeof(*pd_provider), GFP_KERNEL); - if (!pd_provider) - return -ENOMEM; - - pd_provider->ti_sci = devm_ti_sci_get_handle(dev); - if (IS_ERR(pd_provider->ti_sci)) - return PTR_ERR(pd_provider->ti_sci); - - pd_provider->dev = dev; - - INIT_LIST_HEAD(&pd_provider->pd_list); - - /* Find highest device ID used for power domains */ - for_each_node_with_property(np, "power-domains") { - index = 0; - - while (1) { - ret = of_parse_phandle_with_args(np, "power-domains", - "#power-domain-cells", - index, &args); - if (ret) - break; - - if (args.args_count >= 1 && args.np == dev->of_node) { - if (args.args[0] > max_id) - max_id = args.args[0]; - - pd = devm_kzalloc(dev, sizeof(*pd), GFP_KERNEL); - if (!pd) - return -ENOMEM; - - pd->pd.name = devm_kasprintf(dev, GFP_KERNEL, - "pd:%d", - args.args[0]); - if (!pd->pd.name) - return -ENOMEM; - - pd->pd.power_off = ti_sci_pd_power_off; - pd->pd.power_on = ti_sci_pd_power_on; - pd->idx = args.args[0]; - pd->parent = pd_provider; - - pm_genpd_init(&pd->pd, NULL, true); - - list_add(&pd->node, &pd_provider->pd_list); - } - index++; - } - } - - pd_provider->data.domains = - devm_kcalloc(dev, max_id + 1, - sizeof(*pd_provider->data.domains), - GFP_KERNEL); - if (!pd_provider->data.domains) - return -ENOMEM; - - pd_provider->data.num_domains = max_id + 1; - pd_provider->data.xlate = ti_sci_pd_xlate; - - list_for_each_entry(pd, &pd_provider->pd_list, node) - pd_provider->data.domains[pd->idx] = &pd->pd; - - return of_genpd_add_provider_onecell(dev->of_node, &pd_provider->data); -} - -static struct platform_driver ti_sci_pm_domains_driver = { - .probe = ti_sci_pm_domain_probe, - .driver = { - .name = "ti_sci_pm_domains", - .of_match_table = ti_sci_pm_domain_matches, - }, -}; -module_platform_driver(ti_sci_pm_domains_driver); -MODULE_LICENSE("GPL v2"); -MODULE_DESCRIPTION("TI System Control Interface (SCI) Power Domain driver"); -MODULE_AUTHOR("Dave Gerlach"); diff --git a/drivers/genpd/xilinx/Makefile b/drivers/genpd/xilinx/Makefile deleted file mode 100644 index a706ab699cfa..000000000000 --- a/drivers/genpd/xilinx/Makefile +++ /dev/null @@ -1,2 +0,0 @@ -# SPDX-License-Identifier: GPL-2.0 -obj-$(CONFIG_ZYNQMP_PM_DOMAINS) += zynqmp-pm-domains.o diff --git a/drivers/genpd/xilinx/zynqmp-pm-domains.c b/drivers/genpd/xilinx/zynqmp-pm-domains.c deleted file mode 100644 index 69d03ad4cf1e..000000000000 --- a/drivers/genpd/xilinx/zynqmp-pm-domains.c +++ /dev/null @@ -1,322 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * ZynqMP Generic PM domain support - * - * Copyright (C) 2015-2019 Xilinx, Inc. - * - * Davorin Mista - * Jolly Shah - * Rajan Vaja - */ - -#include -#include -#include -#include -#include -#include -#include - -#include - -#define ZYNQMP_NUM_DOMAINS (100) - -static int min_capability; - -/** - * struct zynqmp_pm_domain - Wrapper around struct generic_pm_domain - * @gpd: Generic power domain - * @node_id: PM node ID corresponding to device inside PM domain - * @requested: The PM node mapped to the PM domain has been requested - */ -struct zynqmp_pm_domain { - struct generic_pm_domain gpd; - u32 node_id; - bool requested; -}; - -#define to_zynqmp_pm_domain(pm_domain) \ - container_of(pm_domain, struct zynqmp_pm_domain, gpd) - -/** - * zynqmp_gpd_is_active_wakeup_path() - Check if device is in wakeup source - * path - * @dev: Device to check for wakeup source path - * @not_used: Data member (not required) - * - * This function is checks device's child hierarchy and checks if any device is - * set as wakeup source. - * - * Return: 1 if device is in wakeup source path else 0 - */ -static int zynqmp_gpd_is_active_wakeup_path(struct device *dev, void *not_used) -{ - int may_wakeup; - - may_wakeup = device_may_wakeup(dev); - if (may_wakeup) - return may_wakeup; - - return device_for_each_child(dev, NULL, - zynqmp_gpd_is_active_wakeup_path); -} - -/** - * zynqmp_gpd_power_on() - Power on PM domain - * @domain: Generic PM domain - * - * This function is called before devices inside a PM domain are resumed, to - * power on PM domain. - * - * Return: 0 on success, error code otherwise - */ -static int zynqmp_gpd_power_on(struct generic_pm_domain *domain) -{ - struct zynqmp_pm_domain *pd = to_zynqmp_pm_domain(domain); - int ret; - - ret = zynqmp_pm_set_requirement(pd->node_id, - ZYNQMP_PM_CAPABILITY_ACCESS, - ZYNQMP_PM_MAX_QOS, - ZYNQMP_PM_REQUEST_ACK_BLOCKING); - if (ret) { - dev_err(&domain->dev, - "failed to set requirement to 0x%x for PM node id %d: %d\n", - ZYNQMP_PM_CAPABILITY_ACCESS, pd->node_id, ret); - return ret; - } - - dev_dbg(&domain->dev, "set requirement to 0x%x for PM node id %d\n", - ZYNQMP_PM_CAPABILITY_ACCESS, pd->node_id); - - return 0; -} - -/** - * zynqmp_gpd_power_off() - Power off PM domain - * @domain: Generic PM domain - * - * This function is called after devices inside a PM domain are suspended, to - * power off PM domain. - * - * Return: 0 on success, error code otherwise - */ -static int zynqmp_gpd_power_off(struct generic_pm_domain *domain) -{ - struct zynqmp_pm_domain *pd = to_zynqmp_pm_domain(domain); - int ret; - struct pm_domain_data *pdd, *tmp; - u32 capabilities = min_capability; - bool may_wakeup; - - /* If domain is already released there is nothing to be done */ - if (!pd->requested) { - dev_dbg(&domain->dev, "PM node id %d is already released\n", - pd->node_id); - return 0; - } - - list_for_each_entry_safe(pdd, tmp, &domain->dev_list, list_node) { - /* If device is in wakeup path, set capability to WAKEUP */ - may_wakeup = zynqmp_gpd_is_active_wakeup_path(pdd->dev, NULL); - if (may_wakeup) { - dev_dbg(pdd->dev, "device is in wakeup path in %s\n", - domain->name); - capabilities = ZYNQMP_PM_CAPABILITY_WAKEUP; - break; - } - } - - ret = zynqmp_pm_set_requirement(pd->node_id, capabilities, 0, - ZYNQMP_PM_REQUEST_ACK_NO); - if (ret) { - dev_err(&domain->dev, - "failed to set requirement to 0x%x for PM node id %d: %d\n", - capabilities, pd->node_id, ret); - return ret; - } - - dev_dbg(&domain->dev, "set requirement to 0x%x for PM node id %d\n", - capabilities, pd->node_id); - - return 0; -} - -/** - * zynqmp_gpd_attach_dev() - Attach device to the PM domain - * @domain: Generic PM domain - * @dev: Device to attach - * - * Return: 0 on success, error code otherwise - */ -static int zynqmp_gpd_attach_dev(struct generic_pm_domain *domain, - struct device *dev) -{ - struct zynqmp_pm_domain *pd = to_zynqmp_pm_domain(domain); - struct device_link *link; - int ret; - - link = device_link_add(dev, &domain->dev, DL_FLAG_SYNC_STATE_ONLY); - if (!link) - dev_dbg(&domain->dev, "failed to create device link for %s\n", - dev_name(dev)); - - /* If this is not the first device to attach there is nothing to do */ - if (domain->device_count) - return 0; - - ret = zynqmp_pm_request_node(pd->node_id, 0, 0, - ZYNQMP_PM_REQUEST_ACK_BLOCKING); - if (ret) { - dev_err(&domain->dev, "%s request failed for node %d: %d\n", - domain->name, pd->node_id, ret); - return ret; - } - - pd->requested = true; - - dev_dbg(&domain->dev, "%s requested PM node id %d\n", - dev_name(dev), pd->node_id); - - return 0; -} - -/** - * zynqmp_gpd_detach_dev() - Detach device from the PM domain - * @domain: Generic PM domain - * @dev: Device to detach - */ -static void zynqmp_gpd_detach_dev(struct generic_pm_domain *domain, - struct device *dev) -{ - struct zynqmp_pm_domain *pd = to_zynqmp_pm_domain(domain); - int ret; - - /* If this is not the last device to detach there is nothing to do */ - if (domain->device_count) - return; - - ret = zynqmp_pm_release_node(pd->node_id); - if (ret) { - dev_err(&domain->dev, "failed to release PM node id %d: %d\n", - pd->node_id, ret); - return; - } - - pd->requested = false; - - dev_dbg(&domain->dev, "%s released PM node id %d\n", - dev_name(dev), pd->node_id); -} - -static struct generic_pm_domain *zynqmp_gpd_xlate - (struct of_phandle_args *genpdspec, void *data) -{ - struct genpd_onecell_data *genpd_data = data; - unsigned int i, idx = genpdspec->args[0]; - struct zynqmp_pm_domain *pd; - - pd = to_zynqmp_pm_domain(genpd_data->domains[0]); - - if (genpdspec->args_count != 1) - return ERR_PTR(-EINVAL); - - /* Check for existing pm domains */ - for (i = 0; i < ZYNQMP_NUM_DOMAINS; i++) { - if (pd[i].node_id == idx) - goto done; - } - - /* - * Add index in empty node_id of power domain list as no existing - * power domain found for current index. - */ - for (i = 0; i < ZYNQMP_NUM_DOMAINS; i++) { - if (pd[i].node_id == 0) { - pd[i].node_id = idx; - break; - } - } - -done: - if (!genpd_data->domains[i] || i == ZYNQMP_NUM_DOMAINS) - return ERR_PTR(-ENOENT); - - return genpd_data->domains[i]; -} - -static int zynqmp_gpd_probe(struct platform_device *pdev) -{ - int i; - struct genpd_onecell_data *zynqmp_pd_data; - struct generic_pm_domain **domains; - struct zynqmp_pm_domain *pd; - struct device *dev = &pdev->dev; - - pd = devm_kcalloc(dev, ZYNQMP_NUM_DOMAINS, sizeof(*pd), GFP_KERNEL); - if (!pd) - return -ENOMEM; - - zynqmp_pd_data = devm_kzalloc(dev, sizeof(*zynqmp_pd_data), GFP_KERNEL); - if (!zynqmp_pd_data) - return -ENOMEM; - - zynqmp_pd_data->xlate = zynqmp_gpd_xlate; - - domains = devm_kcalloc(dev, ZYNQMP_NUM_DOMAINS, sizeof(*domains), - GFP_KERNEL); - if (!domains) - return -ENOMEM; - - if (!of_device_is_compatible(dev->parent->of_node, - "xlnx,zynqmp-firmware")) - min_capability = ZYNQMP_PM_CAPABILITY_UNUSABLE; - - for (i = 0; i < ZYNQMP_NUM_DOMAINS; i++, pd++) { - pd->node_id = 0; - pd->gpd.name = kasprintf(GFP_KERNEL, "domain%d", i); - pd->gpd.power_off = zynqmp_gpd_power_off; - pd->gpd.power_on = zynqmp_gpd_power_on; - pd->gpd.attach_dev = zynqmp_gpd_attach_dev; - pd->gpd.detach_dev = zynqmp_gpd_detach_dev; - - domains[i] = &pd->gpd; - - /* Mark all PM domains as initially powered off */ - pm_genpd_init(&pd->gpd, NULL, true); - } - - zynqmp_pd_data->domains = domains; - zynqmp_pd_data->num_domains = ZYNQMP_NUM_DOMAINS; - of_genpd_add_provider_onecell(dev->parent->of_node, zynqmp_pd_data); - - return 0; -} - -static int zynqmp_gpd_remove(struct platform_device *pdev) -{ - of_genpd_del_provider(pdev->dev.parent->of_node); - - return 0; -} - -static void zynqmp_gpd_sync_state(struct device *dev) -{ - int ret; - - ret = zynqmp_pm_init_finalize(); - if (ret) - dev_warn(dev, "failed to release power management to firmware\n"); -} - -static struct platform_driver zynqmp_power_domain_driver = { - .driver = { - .name = "zynqmp_power_controller", - .sync_state = zynqmp_gpd_sync_state, - }, - .probe = zynqmp_gpd_probe, - .remove = zynqmp_gpd_remove, -}; -module_platform_driver(zynqmp_power_domain_driver); - -MODULE_ALIAS("platform:zynqmp_power_controller"); diff --git a/drivers/gpio/gpio-aspeed.c b/drivers/gpio/gpio-aspeed.c index da33bbbdacb9..58f107194fda 100644 --- a/drivers/gpio/gpio-aspeed.c +++ b/drivers/gpio/gpio-aspeed.c @@ -973,7 +973,7 @@ static int aspeed_gpio_set_config(struct gpio_chip *chip, unsigned int offset, else if (param == PIN_CONFIG_BIAS_DISABLE || param == PIN_CONFIG_BIAS_PULL_DOWN || param == PIN_CONFIG_DRIVE_STRENGTH) - return pinctrl_gpio_set_config(offset, config); + return pinctrl_gpio_set_config(chip->base + offset, config); else if (param == PIN_CONFIG_DRIVE_OPEN_DRAIN || param == PIN_CONFIG_DRIVE_OPEN_SOURCE) /* Return -ENOTSUPP to trigger emulation, as per datasheet */ diff --git a/drivers/gpio/gpio-pmic-eic-sprd.c b/drivers/gpio/gpio-pmic-eic-sprd.c index 2b9b7be9b8fd..01c0fd0a9d8c 100644 --- a/drivers/gpio/gpio-pmic-eic-sprd.c +++ b/drivers/gpio/gpio-pmic-eic-sprd.c @@ -352,6 +352,7 @@ static int sprd_pmic_eic_probe(struct platform_device *pdev) pmic_eic->chip.set_config = sprd_pmic_eic_set_config; pmic_eic->chip.set = sprd_pmic_eic_set; pmic_eic->chip.get = sprd_pmic_eic_get; + pmic_eic->chip.can_sleep = true; irq = &pmic_eic->chip.irq; gpio_irq_chip_set_chip(irq, &pmic_eic_irq_chip); diff --git a/drivers/gpio/gpio-pxa.c b/drivers/gpio/gpio-pxa.c index 7e9f7a32d3ee..cae9661862fe 100644 --- a/drivers/gpio/gpio-pxa.c +++ b/drivers/gpio/gpio-pxa.c @@ -237,6 +237,7 @@ static bool pxa_gpio_has_pinctrl(void) switch (gpio_type) { case PXA3XX_GPIO: case MMP2_GPIO: + case MMP_GPIO: return false; default: diff --git a/drivers/gpio/gpio-sim.c b/drivers/gpio/gpio-sim.c index 271db3639a78..44bf1709a648 100644 --- a/drivers/gpio/gpio-sim.c +++ b/drivers/gpio/gpio-sim.c @@ -19,6 +19,7 @@ #include #include #include +#include #include #include #include @@ -685,52 +686,32 @@ gpio_sim_device_config_live_show(struct config_item *item, char *page) return sprintf(page, "%c\n", live ? '1' : '0'); } -static char **gpio_sim_make_line_names(struct gpio_sim_bank *bank, - unsigned int *line_names_size) +static unsigned int gpio_sim_get_line_names_size(struct gpio_sim_bank *bank) { - unsigned int max_offset = 0; - bool has_line_names = false; struct gpio_sim_line *line; - char **line_names; + unsigned int size = 0; list_for_each_entry(line, &bank->line_list, siblings) { - if (line->offset >= bank->num_lines) + if (!line->name || (line->offset >= bank->num_lines)) continue; - if (line->name) { - if (line->offset > max_offset) - max_offset = line->offset; - - /* - * max_offset can stay at 0 so it's not an indicator - * of whether line names were configured at all. - */ - has_line_names = true; - } + size = max(size, line->offset + 1); } - if (!has_line_names) - /* - * This is not an error - NULL means, there are no line - * names configured. - */ - return NULL; - - *line_names_size = max_offset + 1; + return size; +} - line_names = kcalloc(*line_names_size, sizeof(*line_names), GFP_KERNEL); - if (!line_names) - return ERR_PTR(-ENOMEM); +static void +gpio_sim_set_line_names(struct gpio_sim_bank *bank, char **line_names) +{ + struct gpio_sim_line *line; list_for_each_entry(line, &bank->line_list, siblings) { - if (line->offset >= bank->num_lines) + if (!line->name || (line->offset >= bank->num_lines)) continue; - if (line->name && (line->offset <= max_offset)) - line_names[line->offset] = line->name; + line_names[line->offset] = line->name; } - - return line_names; } static void gpio_sim_remove_hogs(struct gpio_sim_device *dev) @@ -834,7 +815,7 @@ gpio_sim_make_bank_swnode(struct gpio_sim_bank *bank, struct fwnode_handle *parent) { struct property_entry properties[GPIO_SIM_PROP_MAX]; - unsigned int prop_idx = 0, line_names_size = 0; + unsigned int prop_idx = 0, line_names_size; char **line_names __free(kfree) = NULL; memset(properties, 0, sizeof(properties)); @@ -845,14 +826,19 @@ gpio_sim_make_bank_swnode(struct gpio_sim_bank *bank, properties[prop_idx++] = PROPERTY_ENTRY_STRING("gpio-sim,label", bank->label); - line_names = gpio_sim_make_line_names(bank, &line_names_size); - if (IS_ERR(line_names)) - return ERR_CAST(line_names); + line_names_size = gpio_sim_get_line_names_size(bank); + if (line_names_size) { + line_names = kcalloc(line_names_size, sizeof(*line_names), + GFP_KERNEL); + if (!line_names) + return ERR_PTR(-ENOMEM); + + gpio_sim_set_line_names(bank, line_names); - if (line_names) properties[prop_idx++] = PROPERTY_ENTRY_STRING_ARRAY_LEN( "gpio-line-names", line_names, line_names_size); + } return fwnode_create_software_node(properties, parent); } diff --git a/drivers/gpio/gpio-tb10x.c b/drivers/gpio/gpio-tb10x.c index 78f8790168ae..f96d260a4a19 100644 --- a/drivers/gpio/gpio-tb10x.c +++ b/drivers/gpio/gpio-tb10x.c @@ -195,7 +195,7 @@ static int tb10x_gpio_probe(struct platform_device *pdev) handle_edge_irq, IRQ_NOREQUEST, IRQ_NOPROBE, IRQ_GC_INIT_MASK_CACHE); if (ret) - return ret; + goto err_remove_domain; gc = tb10x_gpio->domain->gc->gc[0]; gc->reg_base = tb10x_gpio->base; @@ -209,6 +209,10 @@ static int tb10x_gpio_probe(struct platform_device *pdev) } return 0; + +err_remove_domain: + irq_domain_remove(tb10x_gpio->domain); + return ret; } static int tb10x_gpio_remove(struct platform_device *pdev) diff --git a/drivers/gpio/gpio-timberdale.c b/drivers/gpio/gpio-timberdale.c index bbd9e9191199..fad979797486 100644 --- a/drivers/gpio/gpio-timberdale.c +++ b/drivers/gpio/gpio-timberdale.c @@ -43,9 +43,10 @@ static int timbgpio_update_bit(struct gpio_chip *gpio, unsigned index, unsigned offset, bool enabled) { struct timbgpio *tgpio = gpiochip_get_data(gpio); + unsigned long flags; u32 reg; - spin_lock(&tgpio->lock); + spin_lock_irqsave(&tgpio->lock, flags); reg = ioread32(tgpio->membase + offset); if (enabled) @@ -54,7 +55,7 @@ static int timbgpio_update_bit(struct gpio_chip *gpio, unsigned index, reg &= ~(1 << index); iowrite32(reg, tgpio->membase + offset); - spin_unlock(&tgpio->lock); + spin_unlock_irqrestore(&tgpio->lock, flags); return 0; } diff --git a/drivers/gpio/gpio-vf610.c b/drivers/gpio/gpio-vf610.c index dbc7ba0ee72c..656d6b1dddb5 100644 --- a/drivers/gpio/gpio-vf610.c +++ b/drivers/gpio/gpio-vf610.c @@ -126,14 +126,14 @@ static int vf610_gpio_direction_output(struct gpio_chip *chip, unsigned gpio, unsigned long mask = BIT(gpio); u32 val; + vf610_gpio_set(chip, gpio, value); + if (port->sdata && port->sdata->have_paddr) { val = vf610_gpio_readl(port->gpio_base + GPIO_PDDR); val |= mask; vf610_gpio_writel(val, port->gpio_base + GPIO_PDDR); } - vf610_gpio_set(chip, gpio, value); - return pinctrl_gpio_direction_output(chip->base + gpio); } @@ -246,7 +246,8 @@ static const struct irq_chip vf610_irqchip = { .irq_unmask = vf610_gpio_irq_unmask, .irq_set_type = vf610_gpio_irq_set_type, .irq_set_wake = vf610_gpio_irq_set_wake, - .flags = IRQCHIP_IMMUTABLE, + .flags = IRQCHIP_IMMUTABLE | IRQCHIP_MASK_ON_SUSPEND + | IRQCHIP_ENABLE_WAKEUP_ON_SUSPEND, GPIOCHIP_IRQ_RESOURCE_HELPERS, }; diff --git a/drivers/gpio/gpiolib-acpi.c b/drivers/gpio/gpiolib-acpi.c index fbda452fb4d6..51e41676de0b 100644 --- a/drivers/gpio/gpiolib-acpi.c +++ b/drivers/gpio/gpiolib-acpi.c @@ -951,6 +951,7 @@ static struct gpio_desc *acpi_get_gpiod_from_data(struct fwnode_handle *fwnode, if (!propname) return ERR_PTR(-EINVAL); + memset(&lookup, 0, sizeof(lookup)); lookup.index = index; ret = acpi_gpio_property_lookup(fwnode, propname, index, &lookup); diff --git a/drivers/gpu/drm/Kconfig b/drivers/gpu/drm/Kconfig index ab9ef1c20349..3caa020391c7 100644 --- a/drivers/gpu/drm/Kconfig +++ b/drivers/gpu/drm/Kconfig @@ -136,7 +136,7 @@ config DRM_FBDEV_EMULATION bool "Enable legacy fbdev support for your modesetting driver" depends on DRM select FRAMEBUFFER_CONSOLE_DETECT_PRIMARY if FRAMEBUFFER_CONSOLE - default y + default FB help Choose this option if you have a need for the legacy fbdev support. Note that this support also provides the linux console diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h index dc2d53081e80..a79d53bdbe13 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h @@ -1293,7 +1293,6 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev, void amdgpu_device_pci_config_reset(struct amdgpu_device *adev); int amdgpu_device_pci_reset(struct amdgpu_device *adev); bool amdgpu_device_need_post(struct amdgpu_device *adev); -bool amdgpu_sg_display_supported(struct amdgpu_device *adev); bool amdgpu_device_pcie_dynamic_switching_supported(void); bool amdgpu_device_should_use_aspm(struct amdgpu_device *adev); bool amdgpu_device_aspm_support_quirk(void); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c index cdf6087706aa..25d5fda5b243 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c @@ -478,7 +478,7 @@ void amdgpu_amdkfd_get_cu_info(struct amdgpu_device *adev, struct kfd_cu_info *c cu_info->cu_active_number = acu_info.number; cu_info->cu_ao_mask = acu_info.ao_cu_mask; memcpy(&cu_info->cu_bitmap[0], &acu_info.bitmap[0], - sizeof(acu_info.bitmap)); + sizeof(cu_info->cu_bitmap)); cu_info->num_shader_engines = adev->gfx.config.max_shader_engines; cu_info->num_shader_arrays_per_engine = adev->gfx.config.max_sh_per_se; cu_info->num_cu_per_sh = adev->gfx.config.max_cu_per_sh; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v10.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v10.c index f1f2c24de081..69810b3f1c63 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v10.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v10.c @@ -980,8 +980,7 @@ void kgd_gfx_v10_build_grace_period_packet_info(struct amdgpu_device *adev, uint32_t wait_times, uint32_t grace_period, uint32_t *reg_offset, - uint32_t *reg_data, - uint32_t inst) + uint32_t *reg_data) { *reg_data = wait_times; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v10.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v10.h index ecaead24e8c9..67bcaa3d4226 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v10.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v10.h @@ -55,5 +55,4 @@ void kgd_gfx_v10_build_grace_period_packet_info(struct amdgpu_device *adev, uint32_t wait_times, uint32_t grace_period, uint32_t *reg_offset, - uint32_t *reg_data, - uint32_t inst); + uint32_t *reg_data); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v9.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v9.c index fa5ee96f8845..3c45a188b701 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v9.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v9.c @@ -1103,8 +1103,7 @@ void kgd_gfx_v9_build_grace_period_packet_info(struct amdgpu_device *adev, uint32_t wait_times, uint32_t grace_period, uint32_t *reg_offset, - uint32_t *reg_data, - uint32_t inst) + uint32_t *reg_data) { *reg_data = wait_times; @@ -1120,8 +1119,7 @@ void kgd_gfx_v9_build_grace_period_packet_info(struct amdgpu_device *adev, SCH_WAVE, grace_period); - *reg_offset = SOC15_REG_OFFSET(GC, GET_INST(GC, inst), - mmCP_IQ_WAIT_TIME2); + *reg_offset = SOC15_REG_OFFSET(GC, 0, mmCP_IQ_WAIT_TIME2); } void kgd_gfx_v9_program_trap_handler_settings(struct amdgpu_device *adev, diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v9.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v9.h index 936e501908ce..ce424615f59b 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v9.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v9.h @@ -100,5 +100,4 @@ void kgd_gfx_v9_build_grace_period_packet_info(struct amdgpu_device *adev, uint32_t wait_times, uint32_t grace_period, uint32_t *reg_offset, - uint32_t *reg_data, - uint32_t inst); + uint32_t *reg_data); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c index 73ee14f7a9a4..dce9e7d5e4ec 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c @@ -1776,7 +1776,7 @@ static ssize_t amdgpu_atombios_get_vbios_version(struct device *dev, struct amdgpu_device *adev = drm_to_adev(ddev); struct atom_context *ctx = adev->mode_info.atom_context; - return sysfs_emit(buf, "%s\n", ctx->vbios_ver_str); + return sysfs_emit(buf, "%s\n", ctx->vbios_pn); } static DEVICE_ATTR(vbios_version, 0444, amdgpu_atombios_get_vbios_version, diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c index 0dc9c655c4fb..aac52d9754e6 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c @@ -47,7 +47,6 @@ const unsigned int amdgpu_ctx_num_entities[AMDGPU_HW_IP_NUM] = { bool amdgpu_ctx_priority_is_valid(int32_t ctx_prio) { switch (ctx_prio) { - case AMDGPU_CTX_PRIORITY_UNSET: case AMDGPU_CTX_PRIORITY_VERY_LOW: case AMDGPU_CTX_PRIORITY_LOW: case AMDGPU_CTX_PRIORITY_NORMAL: @@ -55,6 +54,7 @@ bool amdgpu_ctx_priority_is_valid(int32_t ctx_prio) case AMDGPU_CTX_PRIORITY_VERY_HIGH: return true; default: + case AMDGPU_CTX_PRIORITY_UNSET: return false; } } @@ -64,7 +64,8 @@ amdgpu_ctx_to_drm_sched_prio(int32_t ctx_prio) { switch (ctx_prio) { case AMDGPU_CTX_PRIORITY_UNSET: - return DRM_SCHED_PRIORITY_UNSET; + pr_warn_once("AMD-->DRM context priority value UNSET-->NORMAL"); + return DRM_SCHED_PRIORITY_NORMAL; case AMDGPU_CTX_PRIORITY_VERY_LOW: return DRM_SCHED_PRIORITY_MIN; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 3f001a50b34a..2b8356699f23 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -1244,32 +1244,6 @@ bool amdgpu_device_need_post(struct amdgpu_device *adev) return true; } -/* - * On APUs with >= 64GB white flickering has been observed w/ SG enabled. - * Disable S/G on such systems until we have a proper fix. - * https://gitlab.freedesktop.org/drm/amd/-/issues/2354 - * https://gitlab.freedesktop.org/drm/amd/-/issues/2735 - */ -bool amdgpu_sg_display_supported(struct amdgpu_device *adev) -{ - switch (amdgpu_sg_display) { - case -1: - break; - case 0: - return false; - case 1: - return true; - default: - return false; - } - if ((totalram_pages() << (PAGE_SHIFT - 10)) + - (adev->gmc.real_vram_size / 1024) >= 64000000) { - DRM_WARN("Disabling S/G due to >=64GB RAM\n"); - return false; - } - return true; -} - /* * Intel hosts such as Raptor Lake and Sapphire Rapids don't support dynamic * speed switching. Until we have confirmation from Intel that a specific host @@ -2119,7 +2093,7 @@ static int amdgpu_device_ip_early_init(struct amdgpu_device *adev) adev->flags |= AMD_IS_PX; if (!(adev->flags & AMD_IS_APU)) { - parent = pci_upstream_bridge(adev->pdev); + parent = pcie_find_root_port(adev->pdev); adev->has_pr3 = parent ? pci_pr3_present(parent) : false; } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c index 12210598e5b8..ba3a87cb88cc 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c @@ -403,7 +403,10 @@ amdgpu_dma_buf_move_notify(struct dma_buf_attachment *attach) continue; } - r = amdgpu_vm_clear_freed(adev, vm, NULL); + /* Reserve fences for two SDMA page table updates */ + r = dma_resv_reserve_fences(resv, 2); + if (!r) + r = amdgpu_vm_clear_freed(adev, vm, NULL); if (!r) r = amdgpu_vm_handle_moved(adev, vm); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c index da4be0bbb446..8eee5d783a92 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c @@ -142,6 +142,10 @@ int amdgpu_doorbell_create_kernel_doorbells(struct amdgpu_device *adev) int r; int size; + /* SI HW does not have doorbells, skip allocation */ + if (adev->doorbell.num_kernel_doorbells == 0) + return 0; + /* Reserve first num_kernel_doorbells (page-aligned) for kernel ops */ size = ALIGN(adev->doorbell.num_kernel_doorbells * sizeof(u32), PAGE_SIZE); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_fru_eeprom.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_fru_eeprom.c index 9c66d98af6d8..7cd0dfaeee20 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_fru_eeprom.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_fru_eeprom.c @@ -170,6 +170,7 @@ int amdgpu_fru_get_product_info(struct amdgpu_device *adev) csum += pia[size - 1]; if (csum) { DRM_ERROR("Bad Product Info Area checksum: 0x%02x", csum); + kfree(pia); return -EIO; } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h index 395c1768b9fc..0ca95c4d4bfb 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h @@ -43,6 +43,7 @@ #define AMDGPU_GFX_LBPW_DISABLED_MODE 0x00000008L #define AMDGPU_MAX_GC_INSTANCES 8 +#define KGD_MAX_QUEUES 128 #define AMDGPU_MAX_GFX_QUEUES KGD_MAX_QUEUES #define AMDGPU_MAX_COMPUTE_QUEUES KGD_MAX_QUEUES @@ -257,7 +258,7 @@ struct amdgpu_cu_info { uint32_t number; uint32_t ao_cu_mask; uint32_t ao_cu_bitmap[4][4]; - uint32_t bitmap[4][4]; + uint32_t bitmap[AMDGPU_MAX_GC_INSTANCES][4][4]; }; struct amdgpu_gfx_ras { diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c index 99f4df133ed3..d30dc0b718c7 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c @@ -839,7 +839,7 @@ int amdgpu_info_ioctl(struct drm_device *dev, void *data, struct drm_file *filp) memcpy(&dev_info->cu_ao_bitmap[0], &adev->gfx.cu_info.ao_cu_bitmap[0], sizeof(adev->gfx.cu_info.ao_cu_bitmap)); memcpy(&dev_info->cu_bitmap[0], &adev->gfx.cu_info.bitmap[0], - sizeof(adev->gfx.cu_info.bitmap)); + sizeof(dev_info->cu_bitmap)); dev_info->vram_type = adev->gmc.vram_type; dev_info->vram_bit_width = adev->gmc.vram_width; dev_info->vce_harvest_config = adev->vce.harvest_config; @@ -940,12 +940,17 @@ int amdgpu_info_ioctl(struct drm_device *dev, void *data, struct drm_file *filp) struct atom_context *atom_context; atom_context = adev->mode_info.atom_context; - memcpy(vbios_info.name, atom_context->name, sizeof(atom_context->name)); - memcpy(vbios_info.vbios_pn, atom_context->vbios_pn, sizeof(atom_context->vbios_pn)); - vbios_info.version = atom_context->version; - memcpy(vbios_info.vbios_ver_str, atom_context->vbios_ver_str, - sizeof(atom_context->vbios_ver_str)); - memcpy(vbios_info.date, atom_context->date, sizeof(atom_context->date)); + if (atom_context) { + memcpy(vbios_info.name, atom_context->name, + sizeof(atom_context->name)); + memcpy(vbios_info.vbios_pn, atom_context->vbios_pn, + sizeof(atom_context->vbios_pn)); + vbios_info.version = atom_context->version; + memcpy(vbios_info.vbios_ver_str, atom_context->vbios_ver_str, + sizeof(atom_context->vbios_ver_str)); + memcpy(vbios_info.date, atom_context->date, + sizeof(atom_context->date)); + } return copy_to_user(out, &vbios_info, min((size_t)size, sizeof(vbios_info))) ? -EFAULT : 0; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h index f3ee83cdf97e..d28e21baef16 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h @@ -252,7 +252,7 @@ static inline bool amdgpu_bo_in_cpu_visible_vram(struct amdgpu_bo *bo) struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev); struct amdgpu_res_cursor cursor; - if (bo->tbo.resource->mem_type != TTM_PL_VRAM) + if (!bo->tbo.resource || bo->tbo.resource->mem_type != TTM_PL_VRAM) return false; amdgpu_res_first(bo->tbo.resource, 0, amdgpu_bo_size(bo), &cursor); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c index 3c4600e15b86..163445baa4fc 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c @@ -801,6 +801,7 @@ int amdgpu_ras_feature_enable(struct amdgpu_device *adev, enable ? "enable":"disable", get_ras_block_str(head), amdgpu_ras_is_poison_mode_supported(adev), ret); + kfree(info); return ret; } @@ -1052,7 +1053,8 @@ int amdgpu_ras_query_error_status(struct amdgpu_device *adev, info->ce_count = obj->err_data.ce_count; if (err_data.ce_count) { - if (adev->smuio.funcs && + if (!adev->aid_mask && + adev->smuio.funcs && adev->smuio.funcs->get_socket_id && adev->smuio.funcs->get_die_id) { dev_info(adev->dev, "socket: %d, die: %d " @@ -1072,7 +1074,8 @@ int amdgpu_ras_query_error_status(struct amdgpu_device *adev, } } if (err_data.ue_count) { - if (adev->smuio.funcs && + if (!adev->aid_mask && + adev->smuio.funcs && adev->smuio.funcs->get_socket_id && adev->smuio.funcs->get_die_id) { dev_info(adev->dev, "socket: %d, die: %d " diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_sa.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_sa.c index c6b4337eb20c..10df731998b2 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_sa.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_sa.c @@ -81,7 +81,7 @@ int amdgpu_sa_bo_new(struct amdgpu_sa_manager *sa_manager, unsigned int size) { struct drm_suballoc *sa = drm_suballoc_new(&sa_manager->base, size, - GFP_KERNEL, true, 0); + GFP_KERNEL, false, 0); if (IS_ERR(sa)) { *sa_bo = NULL; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c index f5daadcec865..82f25996ff5e 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c @@ -1090,7 +1090,8 @@ int amdgpu_vm_bo_update(struct amdgpu_device *adev, struct amdgpu_bo_va *bo_va, struct drm_gem_object *gobj = dma_buf->priv; struct amdgpu_bo *abo = gem_to_amdgpu_bo(gobj); - if (abo->tbo.resource->mem_type == TTM_PL_VRAM) + if (abo->tbo.resource && + abo->tbo.resource->mem_type == TTM_PL_VRAM) bo = gem_to_amdgpu_bo(gobj); } mem = bo->tbo.resource; diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v10_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v10_0.c index 0aee9c8288a2..9032d7a24d7c 100644 --- a/drivers/gpu/drm/amd/amdgpu/gfx_v10_0.c +++ b/drivers/gpu/drm/amd/amdgpu/gfx_v10_0.c @@ -9449,7 +9449,7 @@ static int gfx_v10_0_get_cu_info(struct amdgpu_device *adev, gfx_v10_0_set_user_wgp_inactive_bitmap_per_sh( adev, disable_masks[i * 2 + j]); bitmap = gfx_v10_0_get_cu_active_bitmap_per_sh(adev); - cu_info->bitmap[i][j] = bitmap; + cu_info->bitmap[0][i][j] = bitmap; for (k = 0; k < adev->gfx.config.max_cu_per_sh; k++) { if (bitmap & mask) { diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c index 5c3db694afa8..762d7a19f1be 100644 --- a/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c +++ b/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c @@ -6368,7 +6368,7 @@ static int gfx_v11_0_get_cu_info(struct amdgpu_device *adev, * SE6: {SH0,SH1} --> {bitmap[2][2], bitmap[2][3]} * SE7: {SH0,SH1} --> {bitmap[3][2], bitmap[3][3]} */ - cu_info->bitmap[i % 4][j + (i / 4) * 2] = bitmap; + cu_info->bitmap[0][i % 4][j + (i / 4) * 2] = bitmap; for (k = 0; k < adev->gfx.config.max_cu_per_sh; k++) { if (bitmap & mask) diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v6_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v6_0.c index da6caff78c22..34f9211b2679 100644 --- a/drivers/gpu/drm/amd/amdgpu/gfx_v6_0.c +++ b/drivers/gpu/drm/amd/amdgpu/gfx_v6_0.c @@ -3577,7 +3577,7 @@ static void gfx_v6_0_get_cu_info(struct amdgpu_device *adev) gfx_v6_0_set_user_cu_inactive_bitmap( adev, disable_masks[i * 2 + j]); bitmap = gfx_v6_0_get_cu_enabled(adev); - cu_info->bitmap[i][j] = bitmap; + cu_info->bitmap[0][i][j] = bitmap; for (k = 0; k < adev->gfx.config.max_cu_per_sh; k++) { if (bitmap & mask) { diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v7_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v7_0.c index 90b034b173c1..c2faf6b4c2fc 100644 --- a/drivers/gpu/drm/amd/amdgpu/gfx_v7_0.c +++ b/drivers/gpu/drm/amd/amdgpu/gfx_v7_0.c @@ -5119,7 +5119,7 @@ static void gfx_v7_0_get_cu_info(struct amdgpu_device *adev) gfx_v7_0_set_user_cu_inactive_bitmap( adev, disable_masks[i * 2 + j]); bitmap = gfx_v7_0_get_cu_active_bitmap(adev); - cu_info->bitmap[i][j] = bitmap; + cu_info->bitmap[0][i][j] = bitmap; for (k = 0; k < adev->gfx.config.max_cu_per_sh; k++) { if (bitmap & mask) { diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c index 51c1745c8369..885ebd703260 100644 --- a/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c +++ b/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c @@ -7121,7 +7121,7 @@ static void gfx_v8_0_get_cu_info(struct amdgpu_device *adev) gfx_v8_0_set_user_cu_inactive_bitmap( adev, disable_masks[i * 2 + j]); bitmap = gfx_v8_0_get_cu_active_bitmap(adev); - cu_info->bitmap[i][j] = bitmap; + cu_info->bitmap[0][i][j] = bitmap; for (k = 0; k < adev->gfx.config.max_cu_per_sh; k ++) { if (bitmap & mask) { diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c index 458faf657042..fd61574a737c 100644 --- a/drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c +++ b/drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c @@ -1499,7 +1499,7 @@ static void gfx_v9_0_init_always_on_cu_mask(struct amdgpu_device *adev) amdgpu_gfx_select_se_sh(adev, i, j, 0xffffffff, 0); for (k = 0; k < adev->gfx.config.max_cu_per_sh; k ++) { - if (cu_info->bitmap[i][j] & mask) { + if (cu_info->bitmap[0][i][j] & mask) { if (counter == pg_always_on_cu_num) WREG32_SOC15(GC, 0, mmRLC_PG_ALWAYS_ON_CU_MASK, cu_bitmap); if (counter < always_on_cu_num) @@ -7233,7 +7233,7 @@ static int gfx_v9_0_get_cu_info(struct amdgpu_device *adev, * SE6,SH0 --> bitmap[2][1] * SE7,SH0 --> bitmap[3][1] */ - cu_info->bitmap[i % 4][j + i / 4] = bitmap; + cu_info->bitmap[0][i % 4][j + i / 4] = bitmap; for (k = 0; k < adev->gfx.config.max_cu_per_sh; k ++) { if (bitmap & mask) { diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c b/drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c index 0a26a00074a6..18ce5fe45f6f 100644 --- a/drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c +++ b/drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c @@ -4259,7 +4259,7 @@ static void gfx_v9_4_3_set_gds_init(struct amdgpu_device *adev) } static void gfx_v9_4_3_set_user_cu_inactive_bitmap(struct amdgpu_device *adev, - u32 bitmap) + u32 bitmap, int xcc_id) { u32 data; @@ -4269,15 +4269,15 @@ static void gfx_v9_4_3_set_user_cu_inactive_bitmap(struct amdgpu_device *adev, data = bitmap << GC_USER_SHADER_ARRAY_CONFIG__INACTIVE_CUS__SHIFT; data &= GC_USER_SHADER_ARRAY_CONFIG__INACTIVE_CUS_MASK; - WREG32_SOC15(GC, GET_INST(GC, 0), regGC_USER_SHADER_ARRAY_CONFIG, data); + WREG32_SOC15(GC, GET_INST(GC, xcc_id), regGC_USER_SHADER_ARRAY_CONFIG, data); } -static u32 gfx_v9_4_3_get_cu_active_bitmap(struct amdgpu_device *adev) +static u32 gfx_v9_4_3_get_cu_active_bitmap(struct amdgpu_device *adev, int xcc_id) { u32 data, mask; - data = RREG32_SOC15(GC, GET_INST(GC, 0), regCC_GC_SHADER_ARRAY_CONFIG); - data |= RREG32_SOC15(GC, GET_INST(GC, 0), regGC_USER_SHADER_ARRAY_CONFIG); + data = RREG32_SOC15(GC, GET_INST(GC, xcc_id), regCC_GC_SHADER_ARRAY_CONFIG); + data |= RREG32_SOC15(GC, GET_INST(GC, xcc_id), regGC_USER_SHADER_ARRAY_CONFIG); data &= CC_GC_SHADER_ARRAY_CONFIG__INACTIVE_CUS_MASK; data >>= CC_GC_SHADER_ARRAY_CONFIG__INACTIVE_CUS__SHIFT; @@ -4290,7 +4290,7 @@ static u32 gfx_v9_4_3_get_cu_active_bitmap(struct amdgpu_device *adev) static int gfx_v9_4_3_get_cu_info(struct amdgpu_device *adev, struct amdgpu_cu_info *cu_info) { - int i, j, k, counter, active_cu_number = 0; + int i, j, k, counter, xcc_id, active_cu_number = 0; u32 mask, bitmap, ao_bitmap, ao_cu_mask = 0; unsigned disable_masks[4 * 4]; @@ -4309,46 +4309,38 @@ static int gfx_v9_4_3_get_cu_info(struct amdgpu_device *adev, adev->gfx.config.max_sh_per_se); mutex_lock(&adev->grbm_idx_mutex); - for (i = 0; i < adev->gfx.config.max_shader_engines; i++) { - for (j = 0; j < adev->gfx.config.max_sh_per_se; j++) { - mask = 1; - ao_bitmap = 0; - counter = 0; - gfx_v9_4_3_xcc_select_se_sh(adev, i, j, 0xffffffff, 0); - gfx_v9_4_3_set_user_cu_inactive_bitmap( - adev, disable_masks[i * adev->gfx.config.max_sh_per_se + j]); - bitmap = gfx_v9_4_3_get_cu_active_bitmap(adev); - - /* - * The bitmap(and ao_cu_bitmap) in cu_info structure is - * 4x4 size array, and it's usually suitable for Vega - * ASICs which has 4*2 SE/SH layout. - * But for Arcturus, SE/SH layout is changed to 8*1. - * To mostly reduce the impact, we make it compatible - * with current bitmap array as below: - * SE4,SH0 --> bitmap[0][1] - * SE5,SH0 --> bitmap[1][1] - * SE6,SH0 --> bitmap[2][1] - * SE7,SH0 --> bitmap[3][1] - */ - cu_info->bitmap[i % 4][j + i / 4] = bitmap; - - for (k = 0; k < adev->gfx.config.max_cu_per_sh; k++) { - if (bitmap & mask) { - if (counter < adev->gfx.config.max_cu_per_sh) - ao_bitmap |= mask; - counter++; + for (xcc_id = 0; xcc_id < NUM_XCC(adev->gfx.xcc_mask); xcc_id++) { + for (i = 0; i < adev->gfx.config.max_shader_engines; i++) { + for (j = 0; j < adev->gfx.config.max_sh_per_se; j++) { + mask = 1; + ao_bitmap = 0; + counter = 0; + gfx_v9_4_3_xcc_select_se_sh(adev, i, j, 0xffffffff, xcc_id); + gfx_v9_4_3_set_user_cu_inactive_bitmap( + adev, + disable_masks[i * adev->gfx.config.max_sh_per_se + j], + xcc_id); + bitmap = gfx_v9_4_3_get_cu_active_bitmap(adev, xcc_id); + + cu_info->bitmap[xcc_id][i][j] = bitmap; + + for (k = 0; k < adev->gfx.config.max_cu_per_sh; k++) { + if (bitmap & mask) { + if (counter < adev->gfx.config.max_cu_per_sh) + ao_bitmap |= mask; + counter++; + } + mask <<= 1; } - mask <<= 1; + active_cu_number += counter; + if (i < 2 && j < 2) + ao_cu_mask |= (ao_bitmap << (i * 16 + j * 8)); + cu_info->ao_cu_bitmap[i][j] = ao_bitmap; } - active_cu_number += counter; - if (i < 2 && j < 2) - ao_cu_mask |= (ao_bitmap << (i * 16 + j * 8)); - cu_info->ao_cu_bitmap[i % 4][j + i / 4] = ao_bitmap; } + gfx_v9_4_3_xcc_select_se_sh(adev, 0xffffffff, 0xffffffff, 0xffffffff, + xcc_id); } - gfx_v9_4_3_xcc_select_se_sh(adev, 0xffffffff, 0xffffffff, 0xffffffff, - 0); mutex_unlock(&adev->grbm_idx_mutex); cu_info->number = active_cu_number; diff --git a/drivers/gpu/drm/amd/amdgpu/nbio_v4_3.c b/drivers/gpu/drm/amd/amdgpu/nbio_v4_3.c index d5ed9e0e1a5f..e5b5b0f4940f 100644 --- a/drivers/gpu/drm/amd/amdgpu/nbio_v4_3.c +++ b/drivers/gpu/drm/amd/amdgpu/nbio_v4_3.c @@ -345,6 +345,9 @@ static void nbio_v4_3_init_registers(struct amdgpu_device *adev) data &= ~RCC_DEV0_EPF2_STRAP2__STRAP_NO_SOFT_RESET_DEV0_F2_MASK; WREG32_SOC15(NBIO, 0, regRCC_DEV0_EPF2_STRAP2, data); } + if (amdgpu_sriov_vf(adev)) + adev->rmmio_remap.reg_offset = SOC15_REG_OFFSET(NBIO, 0, + regBIF_BX_DEV0_EPF0_VF0_HDP_MEM_COHERENCY_FLUSH_CNTL) << 2; } static u32 nbio_v4_3_get_rom_offset(struct amdgpu_device *adev) diff --git a/drivers/gpu/drm/amd/amdgpu/soc21.c b/drivers/gpu/drm/amd/amdgpu/soc21.c index 40d23738ee4e..8b2ff2b281b0 100644 --- a/drivers/gpu/drm/amd/amdgpu/soc21.c +++ b/drivers/gpu/drm/amd/amdgpu/soc21.c @@ -766,7 +766,7 @@ static int soc21_common_hw_init(void *handle) * for the purpose of expose those registers * to process space */ - if (adev->nbio.funcs->remap_hdp_registers) + if (adev->nbio.funcs->remap_hdp_registers && !amdgpu_sriov_vf(adev)) adev->nbio.funcs->remap_hdp_registers(adev); /* enable the doorbell aperture */ adev->nbio.funcs->enable_doorbell_aperture(adev, true); diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_crat.c b/drivers/gpu/drm/amd/amdkfd/kfd_crat.c index 86fb7ac7982a..f76b7aee5c0a 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_crat.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_crat.c @@ -2087,7 +2087,8 @@ static int kfd_create_vcrat_image_gpu(void *pcrat_image, amdgpu_amdkfd_get_cu_info(kdev->adev, &cu_info); cu->num_simd_per_cu = cu_info.simd_per_cu; - cu->num_simd_cores = cu_info.simd_per_cu * cu_info.cu_active_number; + cu->num_simd_cores = cu_info.simd_per_cu * + (cu_info.cu_active_number / kdev->kfd->num_nodes); cu->max_waves_simd = cu_info.max_waves_per_simd; cu->wave_front_size = cu_info.wave_front_size; diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_crat.h b/drivers/gpu/drm/amd/amdkfd/kfd_crat.h index 387a8ef49385..74c2d7a0d628 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_crat.h +++ b/drivers/gpu/drm/amd/amdkfd/kfd_crat.h @@ -79,6 +79,10 @@ struct crat_header { #define CRAT_SUBTYPE_IOLINK_AFFINITY 5 #define CRAT_SUBTYPE_MAX 6 +/* + * Do not change the value of CRAT_SIBLINGMAP_SIZE from 32 + * as it breaks the ABI. + */ #define CRAT_SIBLINGMAP_SIZE 32 /* diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c index b166f30f083e..0d3d538b64eb 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c @@ -216,7 +216,7 @@ static int add_queue_mes(struct device_queue_manager *dqm, struct queue *q, if (q->wptr_bo) { wptr_addr_off = (uint64_t)q->properties.write_ptr & (PAGE_SIZE - 1); - queue_input.wptr_mc_addr = ((uint64_t)q->wptr_bo->tbo.resource->start << PAGE_SHIFT) + wptr_addr_off; + queue_input.wptr_mc_addr = amdgpu_bo_gpu_offset(q->wptr_bo) + wptr_addr_off; } queue_input.is_kfd_process = 1; @@ -1677,8 +1677,7 @@ static int start_cpsch(struct device_queue_manager *dqm) dqm->dev->kfd2kgd->build_grace_period_packet_info( dqm->dev->adev, dqm->wait_times, grace_period, ®_offset, - &dqm->wait_times, - ffs(dqm->dev->xcc_mask) - 1); + &dqm->wait_times); } dqm_unlock(dqm); diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_doorbell.c b/drivers/gpu/drm/amd/amdkfd/kfd_doorbell.c index c2e0b79dcc6d..7b38537c7c99 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_doorbell.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_doorbell.c @@ -162,6 +162,7 @@ void __iomem *kfd_get_kernel_doorbell(struct kfd_dev *kfd, return NULL; *doorbell_off = amdgpu_doorbell_index_on_bar(kfd->adev, kfd->doorbells, inx); + inx *= 2; pr_debug("Get kernel queue doorbell\n" " doorbell offset == 0x%08X\n" @@ -176,6 +177,7 @@ void kfd_release_kernel_doorbell(struct kfd_dev *kfd, u32 __iomem *db_addr) unsigned int inx; inx = (unsigned int)(db_addr - kfd->doorbell_kernel_ptr); + inx /= 2; mutex_lock(&kfd->doorbell_mutex); __clear_bit(inx, kfd->doorbell_bitmap); diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.c index d01bb57733b3..447829c22295 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.c @@ -97,18 +97,22 @@ void free_mqd_hiq_sdma(struct mqd_manager *mm, void *mqd, void mqd_symmetrically_map_cu_mask(struct mqd_manager *mm, const uint32_t *cu_mask, uint32_t cu_mask_count, - uint32_t *se_mask) + uint32_t *se_mask, uint32_t inst) { struct kfd_cu_info cu_info; uint32_t cu_per_sh[KFD_MAX_NUM_SE][KFD_MAX_NUM_SH_PER_SE] = {0}; bool wgp_mode_req = KFD_GC_VERSION(mm->dev) >= IP_VERSION(10, 0, 0); uint32_t en_mask = wgp_mode_req ? 0x3 : 0x1; - int i, se, sh, cu, cu_bitmap_sh_mul, inc = wgp_mode_req ? 2 : 1; + int i, se, sh, cu, cu_bitmap_sh_mul, cu_inc = wgp_mode_req ? 2 : 1; + uint32_t cu_active_per_node; + int inc = cu_inc * NUM_XCC(mm->dev->xcc_mask); + int xcc_inst = inst + ffs(mm->dev->xcc_mask) - 1; amdgpu_amdkfd_get_cu_info(mm->dev->adev, &cu_info); - if (cu_mask_count > cu_info.cu_active_number) - cu_mask_count = cu_info.cu_active_number; + cu_active_per_node = cu_info.cu_active_number / mm->dev->kfd->num_nodes; + if (cu_mask_count > cu_active_per_node) + cu_mask_count = cu_active_per_node; /* Exceeding these bounds corrupts the stack and indicates a coding error. * Returning with no CU's enabled will hang the queue, which should be @@ -141,7 +145,8 @@ void mqd_symmetrically_map_cu_mask(struct mqd_manager *mm, for (se = 0; se < cu_info.num_shader_engines; se++) for (sh = 0; sh < cu_info.num_shader_arrays_per_engine; sh++) cu_per_sh[se][sh] = hweight32( - cu_info.cu_bitmap[se % 4][sh + (se / 4) * cu_bitmap_sh_mul]); + cu_info.cu_bitmap[xcc_inst][se % 4][sh + (se / 4) * + cu_bitmap_sh_mul]); /* Symmetrically map cu_mask to all SEs & SHs: * se_mask programs up to 2 SH in the upper and lower 16 bits. @@ -164,20 +169,33 @@ void mqd_symmetrically_map_cu_mask(struct mqd_manager *mm, * cu_mask[0] bit8 -> se_mask[0] bit1 (SE0,SH0,CU1) * ... * + * For GFX 9.4.3, the following code only looks at a + * subset of the cu_mask corresponding to the inst parameter. + * If we have n XCCs under one GPU node + * cu_mask[0] bit0 -> XCC0 se_mask[0] bit0 (XCC0,SE0,SH0,CU0) + * cu_mask[0] bit1 -> XCC1 se_mask[0] bit0 (XCC1,SE0,SH0,CU0) + * .. + * cu_mask[0] bitn -> XCCn se_mask[0] bit0 (XCCn,SE0,SH0,CU0) + * cu_mask[0] bit n+1 -> XCC0 se_mask[1] bit0 (XCC0,SE1,SH0,CU0) + * + * For example, if there are 6 XCCs under 1 KFD node, this code + * running for each inst, will look at the bits as: + * inst, inst + 6, inst + 12... + * * First ensure all CUs are disabled, then enable user specified CUs. */ for (i = 0; i < cu_info.num_shader_engines; i++) se_mask[i] = 0; - i = 0; - for (cu = 0; cu < 16; cu += inc) { + i = inst; + for (cu = 0; cu < 16; cu += cu_inc) { for (sh = 0; sh < cu_info.num_shader_arrays_per_engine; sh++) { for (se = 0; se < cu_info.num_shader_engines; se++) { if (cu_per_sh[se][sh] > cu) { if (cu_mask[i / 32] & (en_mask << (i % 32))) se_mask[se] |= en_mask << (cu + sh * 16); i += inc; - if (i == cu_mask_count) + if (i >= cu_mask_count) return; } } diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.h b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.h index 23158db7da03..57bf5e513f4d 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.h +++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.h @@ -138,7 +138,7 @@ void free_mqd_hiq_sdma(struct mqd_manager *mm, void *mqd, void mqd_symmetrically_map_cu_mask(struct mqd_manager *mm, const uint32_t *cu_mask, uint32_t cu_mask_count, - uint32_t *se_mask); + uint32_t *se_mask, uint32_t inst); int kfd_hiq_load_mqd_kiq(struct mqd_manager *mm, void *mqd, uint32_t pipe_id, uint32_t queue_id, diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_cik.c b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_cik.c index ee1d32d957f2..1a4a69943c71 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_cik.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_cik.c @@ -52,7 +52,7 @@ static void update_cu_mask(struct mqd_manager *mm, void *mqd, return; mqd_symmetrically_map_cu_mask(mm, - minfo->cu_mask.ptr, minfo->cu_mask.count, se_mask); + minfo->cu_mask.ptr, minfo->cu_mask.count, se_mask, 0); m = get_mqd(mqd); m->compute_static_thread_mgmt_se0 = se_mask[0]; diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v10.c b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v10.c index 83699392c808..8b7fed913526 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v10.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v10.c @@ -52,7 +52,7 @@ static void update_cu_mask(struct mqd_manager *mm, void *mqd, return; mqd_symmetrically_map_cu_mask(mm, - minfo->cu_mask.ptr, minfo->cu_mask.count, se_mask); + minfo->cu_mask.ptr, minfo->cu_mask.count, se_mask, 0); m = get_mqd(mqd); m->compute_static_thread_mgmt_se0 = se_mask[0]; diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v11.c b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v11.c index 0bbf0edbabd4..15277f1d5cf0 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v11.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v11.c @@ -71,7 +71,7 @@ static void update_cu_mask(struct mqd_manager *mm, void *mqd, } mqd_symmetrically_map_cu_mask(mm, - minfo->cu_mask.ptr, minfo->cu_mask.count, se_mask); + minfo->cu_mask.ptr, minfo->cu_mask.count, se_mask, 0); m->compute_static_thread_mgmt_se0 = se_mask[0]; m->compute_static_thread_mgmt_se1 = se_mask[1]; @@ -321,6 +321,43 @@ static int get_wave_state(struct mqd_manager *mm, void *mqd, return 0; } +static void checkpoint_mqd(struct mqd_manager *mm, void *mqd, void *mqd_dst, void *ctl_stack_dst) +{ + struct v11_compute_mqd *m; + + m = get_mqd(mqd); + + memcpy(mqd_dst, m, sizeof(struct v11_compute_mqd)); +} + +static void restore_mqd(struct mqd_manager *mm, void **mqd, + struct kfd_mem_obj *mqd_mem_obj, uint64_t *gart_addr, + struct queue_properties *qp, + const void *mqd_src, + const void *ctl_stack_src, const u32 ctl_stack_size) +{ + uint64_t addr; + struct v11_compute_mqd *m; + + m = (struct v11_compute_mqd *) mqd_mem_obj->cpu_ptr; + addr = mqd_mem_obj->gpu_addr; + + memcpy(m, mqd_src, sizeof(*m)); + + *mqd = m; + if (gart_addr) + *gart_addr = addr; + + m->cp_hqd_pq_doorbell_control = + qp->doorbell_off << + CP_HQD_PQ_DOORBELL_CONTROL__DOORBELL_OFFSET__SHIFT; + pr_debug("cp_hqd_pq_doorbell_control 0x%x\n", + m->cp_hqd_pq_doorbell_control); + + qp->is_active = 0; +} + + static void init_mqd_hiq(struct mqd_manager *mm, void **mqd, struct kfd_mem_obj *mqd_mem_obj, uint64_t *gart_addr, struct queue_properties *q) @@ -458,6 +495,8 @@ struct mqd_manager *mqd_manager_init_v11(enum KFD_MQD_TYPE type, mqd->mqd_size = sizeof(struct v11_compute_mqd); mqd->get_wave_state = get_wave_state; mqd->mqd_stride = kfd_mqd_stride; + mqd->checkpoint_mqd = checkpoint_mqd; + mqd->restore_mqd = restore_mqd; #if defined(CONFIG_DEBUG_FS) mqd->debugfs_show_mqd = debugfs_show_mqd; #endif @@ -502,6 +541,8 @@ struct mqd_manager *mqd_manager_init_v11(enum KFD_MQD_TYPE type, mqd->update_mqd = update_mqd_sdma; mqd->destroy_mqd = kfd_destroy_mqd_sdma; mqd->is_occupied = kfd_is_occupied_sdma; + mqd->checkpoint_mqd = checkpoint_mqd; + mqd->restore_mqd = restore_mqd; mqd->mqd_size = sizeof(struct v11_sdma_mqd); mqd->mqd_stride = kfd_mqd_stride; #if defined(CONFIG_DEBUG_FS) diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c index e23d32f35607..42d881809dc7 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c @@ -60,7 +60,7 @@ static inline struct v9_sdma_mqd *get_sdma_mqd(void *mqd) } static void update_cu_mask(struct mqd_manager *mm, void *mqd, - struct mqd_update_info *minfo) + struct mqd_update_info *minfo, uint32_t inst) { struct v9_mqd *m; uint32_t se_mask[KFD_MAX_NUM_SE] = {0}; @@ -69,27 +69,36 @@ static void update_cu_mask(struct mqd_manager *mm, void *mqd, return; mqd_symmetrically_map_cu_mask(mm, - minfo->cu_mask.ptr, minfo->cu_mask.count, se_mask); + minfo->cu_mask.ptr, minfo->cu_mask.count, se_mask, inst); m = get_mqd(mqd); + m->compute_static_thread_mgmt_se0 = se_mask[0]; m->compute_static_thread_mgmt_se1 = se_mask[1]; m->compute_static_thread_mgmt_se2 = se_mask[2]; m->compute_static_thread_mgmt_se3 = se_mask[3]; - m->compute_static_thread_mgmt_se4 = se_mask[4]; - m->compute_static_thread_mgmt_se5 = se_mask[5]; - m->compute_static_thread_mgmt_se6 = se_mask[6]; - m->compute_static_thread_mgmt_se7 = se_mask[7]; - - pr_debug("update cu mask to %#x %#x %#x %#x %#x %#x %#x %#x\n", - m->compute_static_thread_mgmt_se0, - m->compute_static_thread_mgmt_se1, - m->compute_static_thread_mgmt_se2, - m->compute_static_thread_mgmt_se3, - m->compute_static_thread_mgmt_se4, - m->compute_static_thread_mgmt_se5, - m->compute_static_thread_mgmt_se6, - m->compute_static_thread_mgmt_se7); + if (KFD_GC_VERSION(mm->dev) != IP_VERSION(9, 4, 3)) { + m->compute_static_thread_mgmt_se4 = se_mask[4]; + m->compute_static_thread_mgmt_se5 = se_mask[5]; + m->compute_static_thread_mgmt_se6 = se_mask[6]; + m->compute_static_thread_mgmt_se7 = se_mask[7]; + + pr_debug("update cu mask to %#x %#x %#x %#x %#x %#x %#x %#x\n", + m->compute_static_thread_mgmt_se0, + m->compute_static_thread_mgmt_se1, + m->compute_static_thread_mgmt_se2, + m->compute_static_thread_mgmt_se3, + m->compute_static_thread_mgmt_se4, + m->compute_static_thread_mgmt_se5, + m->compute_static_thread_mgmt_se6, + m->compute_static_thread_mgmt_se7); + } else { + pr_debug("inst: %u, update cu mask to %#x %#x %#x %#x\n", + inst, m->compute_static_thread_mgmt_se0, + m->compute_static_thread_mgmt_se1, + m->compute_static_thread_mgmt_se2, + m->compute_static_thread_mgmt_se3); + } } static void set_priority(struct v9_mqd *m, struct queue_properties *q) @@ -290,7 +299,8 @@ static void update_mqd(struct mqd_manager *mm, void *mqd, if (mm->dev->kfd->cwsr_enabled && q->ctx_save_restore_area_address) m->cp_hqd_ctx_save_control = 0; - update_cu_mask(mm, mqd, minfo); + if (KFD_GC_VERSION(mm->dev) != IP_VERSION(9, 4, 3)) + update_cu_mask(mm, mqd, minfo, 0); set_priority(m, q); q->is_active = QUEUE_IS_ACTIVE(*q); @@ -676,6 +686,8 @@ static void update_mqd_v9_4_3(struct mqd_manager *mm, void *mqd, m = get_mqd(mqd + size * xcc); update_mqd(mm, m, q, minfo); + update_cu_mask(mm, mqd, minfo, xcc); + if (q->format == KFD_QUEUE_FORMAT_AQL) { switch (xcc) { case 0: diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c index 657c37822980..3e1a574d4ea6 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c @@ -55,7 +55,7 @@ static void update_cu_mask(struct mqd_manager *mm, void *mqd, return; mqd_symmetrically_map_cu_mask(mm, - minfo->cu_mask.ptr, minfo->cu_mask.count, se_mask); + minfo->cu_mask.ptr, minfo->cu_mask.count, se_mask, 0); m = get_mqd(mqd); m->compute_static_thread_mgmt_se0 = se_mask[0]; diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_v9.c b/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_v9.c index 8ce6f5200905..1a03173e2313 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_v9.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_v9.c @@ -299,8 +299,7 @@ static int pm_set_grace_period_v9(struct packet_manager *pm, pm->dqm->wait_times, grace_period, ®_offset, - ®_data, - 0); + ®_data); if (grace_period == USE_DEFAULT_GRACE_PERIOD) reg_data = pm->dqm->wait_times; diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h index 3d9ce44d88da..fa24e1852493 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h +++ b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h @@ -1466,8 +1466,7 @@ void kfd_flush_tlb(struct kfd_process_device *pdd, enum TLB_FLUSH_TYPE type); static inline bool kfd_flush_tlb_after_unmap(struct kfd_dev *dev) { - return KFD_GC_VERSION(dev) == IP_VERSION(9, 4, 3) || - KFD_GC_VERSION(dev) == IP_VERSION(9, 4, 2) || + return KFD_GC_VERSION(dev) > IP_VERSION(9, 4, 2) || (KFD_GC_VERSION(dev) == IP_VERSION(9, 4, 1) && dev->sdma_fw_version >= 18) || KFD_GC_VERSION(dev) == IP_VERSION(9, 4, 0); } diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_topology.c b/drivers/gpu/drm/amd/amdkfd/kfd_topology.c index ff98fded9534..c8c75ff7cea8 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_topology.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_topology.c @@ -450,8 +450,7 @@ static ssize_t node_show(struct kobject *kobj, struct attribute *attr, sysfs_show_32bit_prop(buffer, offs, "cpu_cores_count", dev->node_props.cpu_cores_count); sysfs_show_32bit_prop(buffer, offs, "simd_count", - dev->gpu ? (dev->node_props.simd_count * - NUM_XCC(dev->gpu->xcc_mask)) : 0); + dev->gpu ? dev->node_props.simd_count : 0); sysfs_show_32bit_prop(buffer, offs, "mem_banks_count", dev->node_props.mem_banks_count); sysfs_show_32bit_prop(buffer, offs, "caches_count", @@ -1597,14 +1596,17 @@ static int fill_in_l1_pcache(struct kfd_cache_properties **props_ext, static int fill_in_l2_l3_pcache(struct kfd_cache_properties **props_ext, struct kfd_gpu_cache_info *pcache_info, struct kfd_cu_info *cu_info, - int cache_type, unsigned int cu_processor_id) + int cache_type, unsigned int cu_processor_id, + struct kfd_node *knode) { unsigned int cu_sibling_map_mask; int first_active_cu; - int i, j, k; + int i, j, k, xcc, start, end; struct kfd_cache_properties *pcache = NULL; - cu_sibling_map_mask = cu_info->cu_bitmap[0][0]; + start = ffs(knode->xcc_mask) - 1; + end = start + NUM_XCC(knode->xcc_mask); + cu_sibling_map_mask = cu_info->cu_bitmap[start][0][0]; cu_sibling_map_mask &= ((1 << pcache_info[cache_type].num_cu_shared) - 1); first_active_cu = ffs(cu_sibling_map_mask); @@ -1639,16 +1641,18 @@ static int fill_in_l2_l3_pcache(struct kfd_cache_properties **props_ext, cu_sibling_map_mask = cu_sibling_map_mask >> (first_active_cu - 1); k = 0; - for (i = 0; i < cu_info->num_shader_engines; i++) { - for (j = 0; j < cu_info->num_shader_arrays_per_engine; j++) { - pcache->sibling_map[k] = (uint8_t)(cu_sibling_map_mask & 0xFF); - pcache->sibling_map[k+1] = (uint8_t)((cu_sibling_map_mask >> 8) & 0xFF); - pcache->sibling_map[k+2] = (uint8_t)((cu_sibling_map_mask >> 16) & 0xFF); - pcache->sibling_map[k+3] = (uint8_t)((cu_sibling_map_mask >> 24) & 0xFF); - k += 4; - - cu_sibling_map_mask = cu_info->cu_bitmap[i % 4][j + i / 4]; - cu_sibling_map_mask &= ((1 << pcache_info[cache_type].num_cu_shared) - 1); + for (xcc = start; xcc < end; xcc++) { + for (i = 0; i < cu_info->num_shader_engines; i++) { + for (j = 0; j < cu_info->num_shader_arrays_per_engine; j++) { + pcache->sibling_map[k] = (uint8_t)(cu_sibling_map_mask & 0xFF); + pcache->sibling_map[k+1] = (uint8_t)((cu_sibling_map_mask >> 8) & 0xFF); + pcache->sibling_map[k+2] = (uint8_t)((cu_sibling_map_mask >> 16) & 0xFF); + pcache->sibling_map[k+3] = (uint8_t)((cu_sibling_map_mask >> 24) & 0xFF); + k += 4; + + cu_sibling_map_mask = cu_info->cu_bitmap[xcc][i % 4][j + i / 4]; + cu_sibling_map_mask &= ((1 << pcache_info[cache_type].num_cu_shared) - 1); + } } } pcache->sibling_map_size = k; @@ -1666,7 +1670,7 @@ static int fill_in_l2_l3_pcache(struct kfd_cache_properties **props_ext, static void kfd_fill_cache_non_crat_info(struct kfd_topology_device *dev, struct kfd_node *kdev) { struct kfd_gpu_cache_info *pcache_info = NULL; - int i, j, k; + int i, j, k, xcc, start, end; int ct = 0; unsigned int cu_processor_id; int ret; @@ -1700,37 +1704,42 @@ static void kfd_fill_cache_non_crat_info(struct kfd_topology_device *dev, struct * then it will consider only one CU from * the shared unit */ + start = ffs(kdev->xcc_mask) - 1; + end = start + NUM_XCC(kdev->xcc_mask); + for (ct = 0; ct < num_of_cache_types; ct++) { cu_processor_id = gpu_processor_id; if (pcache_info[ct].cache_level == 1) { - for (i = 0; i < pcu_info->num_shader_engines; i++) { - for (j = 0; j < pcu_info->num_shader_arrays_per_engine; j++) { - for (k = 0; k < pcu_info->num_cu_per_sh; k += pcache_info[ct].num_cu_shared) { + for (xcc = start; xcc < end; xcc++) { + for (i = 0; i < pcu_info->num_shader_engines; i++) { + for (j = 0; j < pcu_info->num_shader_arrays_per_engine; j++) { + for (k = 0; k < pcu_info->num_cu_per_sh; k += pcache_info[ct].num_cu_shared) { - ret = fill_in_l1_pcache(&props_ext, pcache_info, pcu_info, - pcu_info->cu_bitmap[i % 4][j + i / 4], ct, + ret = fill_in_l1_pcache(&props_ext, pcache_info, pcu_info, + pcu_info->cu_bitmap[xcc][i % 4][j + i / 4], ct, cu_processor_id, k); - if (ret < 0) - break; + if (ret < 0) + break; - if (!ret) { - num_of_entries++; - list_add_tail(&props_ext->list, &dev->cache_props); - } + if (!ret) { + num_of_entries++; + list_add_tail(&props_ext->list, &dev->cache_props); + } - /* Move to next CU block */ - num_cu_shared = ((k + pcache_info[ct].num_cu_shared) <= - pcu_info->num_cu_per_sh) ? - pcache_info[ct].num_cu_shared : - (pcu_info->num_cu_per_sh - k); - cu_processor_id += num_cu_shared; + /* Move to next CU block */ + num_cu_shared = ((k + pcache_info[ct].num_cu_shared) <= + pcu_info->num_cu_per_sh) ? + pcache_info[ct].num_cu_shared : + (pcu_info->num_cu_per_sh - k); + cu_processor_id += num_cu_shared; + } } } } } else { ret = fill_in_l2_l3_pcache(&props_ext, pcache_info, - pcu_info, ct, cu_processor_id); + pcu_info, ct, cu_processor_id, kdev); if (ret < 0) break; diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_topology.h b/drivers/gpu/drm/amd/amdkfd/kfd_topology.h index dea32a9e5506..27386ce9a021 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_topology.h +++ b/drivers/gpu/drm/amd/amdkfd/kfd_topology.h @@ -89,7 +89,7 @@ struct kfd_mem_properties { struct attribute attr; }; -#define CACHE_SIBLINGMAP_SIZE 64 +#define CACHE_SIBLINGMAP_SIZE 128 struct kfd_cache_properties { struct list_head list; diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c index 88ba8b66de1f..868946dd7ef1 100644 --- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c +++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c @@ -1274,11 +1274,15 @@ static void mmhub_read_system_context(struct amdgpu_device *adev, struct dc_phy_ pt_base = amdgpu_gmc_pd_addr(adev->gart.bo); - page_table_start.high_part = (u32)(adev->gmc.gart_start >> 44) & 0xF; - page_table_start.low_part = (u32)(adev->gmc.gart_start >> 12); - page_table_end.high_part = (u32)(adev->gmc.gart_end >> 44) & 0xF; - page_table_end.low_part = (u32)(adev->gmc.gart_end >> 12); - page_table_base.high_part = upper_32_bits(pt_base) & 0xF; + page_table_start.high_part = upper_32_bits(adev->gmc.gart_start >> + AMDGPU_GPU_PAGE_SHIFT); + page_table_start.low_part = lower_32_bits(adev->gmc.gart_start >> + AMDGPU_GPU_PAGE_SHIFT); + page_table_end.high_part = upper_32_bits(adev->gmc.gart_end >> + AMDGPU_GPU_PAGE_SHIFT); + page_table_end.low_part = lower_32_bits(adev->gmc.gart_end >> + AMDGPU_GPU_PAGE_SHIFT); + page_table_base.high_part = upper_32_bits(pt_base); page_table_base.low_part = lower_32_bits(pt_base); pa_config->system_aperture.start_addr = (uint64_t)logical_addr_low << 18; @@ -1640,8 +1644,9 @@ static int amdgpu_dm_init(struct amdgpu_device *adev) } break; } - if (init_data.flags.gpu_vm_support) - init_data.flags.gpu_vm_support = amdgpu_sg_display_supported(adev); + if (init_data.flags.gpu_vm_support && + (amdgpu_sg_display == 0)) + init_data.flags.gpu_vm_support = false; if (init_data.flags.gpu_vm_support) adev->mode_info.gpu_vm_support = true; @@ -2335,14 +2340,62 @@ static int dm_late_init(void *handle) return detect_mst_link_for_all_connectors(adev_to_drm(adev)); } +static void resume_mst_branch_status(struct drm_dp_mst_topology_mgr *mgr) +{ + int ret; + u8 guid[16]; + u64 tmp64; + + mutex_lock(&mgr->lock); + if (!mgr->mst_primary) + goto out_fail; + + if (drm_dp_read_dpcd_caps(mgr->aux, mgr->dpcd) < 0) { + drm_dbg_kms(mgr->dev, "dpcd read failed - undocked during suspend?\n"); + goto out_fail; + } + + ret = drm_dp_dpcd_writeb(mgr->aux, DP_MSTM_CTRL, + DP_MST_EN | + DP_UP_REQ_EN | + DP_UPSTREAM_IS_SRC); + if (ret < 0) { + drm_dbg_kms(mgr->dev, "mst write failed - undocked during suspend?\n"); + goto out_fail; + } + + /* Some hubs forget their guids after they resume */ + ret = drm_dp_dpcd_read(mgr->aux, DP_GUID, guid, 16); + if (ret != 16) { + drm_dbg_kms(mgr->dev, "dpcd read failed - undocked during suspend?\n"); + goto out_fail; + } + + if (memchr_inv(guid, 0, 16) == NULL) { + tmp64 = get_jiffies_64(); + memcpy(&guid[0], &tmp64, sizeof(u64)); + memcpy(&guid[8], &tmp64, sizeof(u64)); + + ret = drm_dp_dpcd_write(mgr->aux, DP_GUID, guid, 16); + + if (ret != 16) { + drm_dbg_kms(mgr->dev, "check mstb guid failed - undocked during suspend?\n"); + goto out_fail; + } + } + + memcpy(mgr->mst_primary->guid, guid, 16); + +out_fail: + mutex_unlock(&mgr->lock); +} + static void s3_handle_mst(struct drm_device *dev, bool suspend) { struct amdgpu_dm_connector *aconnector; struct drm_connector *connector; struct drm_connector_list_iter iter; struct drm_dp_mst_topology_mgr *mgr; - int ret; - bool need_hotplug = false; drm_connector_list_iter_begin(dev, &iter); drm_for_each_connector_iter(connector, &iter) { @@ -2364,18 +2417,15 @@ static void s3_handle_mst(struct drm_device *dev, bool suspend) if (!dp_is_lttpr_present(aconnector->dc_link)) try_to_configure_aux_timeout(aconnector->dc_link->ddc, LINK_AUX_DEFAULT_TIMEOUT_PERIOD); - ret = drm_dp_mst_topology_mgr_resume(mgr, true); - if (ret < 0) { - dm_helpers_dp_mst_stop_top_mgr(aconnector->dc_link->ctx, - aconnector->dc_link); - need_hotplug = true; - } + /* TODO: move resume_mst_branch_status() into drm mst resume again + * once topology probing work is pulled out from mst resume into mst + * resume 2nd step. mst resume 2nd step should be called after old + * state getting restored (i.e. drm_atomic_helper_resume()). + */ + resume_mst_branch_status(mgr); } } drm_connector_list_iter_end(&iter); - - if (need_hotplug) - drm_kms_helper_hotplug_event(dev); } static int amdgpu_dm_smu_write_watermarks_table(struct amdgpu_device *adev) @@ -2769,7 +2819,8 @@ static int dm_resume(void *handle) struct dm_atomic_state *dm_state = to_dm_atomic_state(dm->atomic_obj.state); enum dc_connection_type new_connection_type = dc_connection_none; struct dc_state *dc_state; - int i, r, j; + int i, r, j, ret; + bool need_hotplug = false; if (amdgpu_in_reset(adev)) { dc_state = dm->cached_dc_state; @@ -2867,7 +2918,7 @@ static int dm_resume(void *handle) continue; /* - * this is the case when traversing through already created + * this is the case when traversing through already created end sink * MST connectors, should be skipped */ if (aconnector && aconnector->mst_root) @@ -2927,6 +2978,27 @@ static int dm_resume(void *handle) dm->cached_state = NULL; + /* Do mst topology probing after resuming cached state*/ + drm_connector_list_iter_begin(ddev, &iter); + drm_for_each_connector_iter(connector, &iter) { + aconnector = to_amdgpu_dm_connector(connector); + if (aconnector->dc_link->type != dc_connection_mst_branch || + aconnector->mst_root) + continue; + + ret = drm_dp_mst_topology_mgr_resume(&aconnector->mst_mgr, true); + + if (ret < 0) { + dm_helpers_dp_mst_stop_top_mgr(aconnector->dc_link->ctx, + aconnector->dc_link); + need_hotplug = true; + } + } + drm_connector_list_iter_end(&iter); + + if (need_hotplug) + drm_kms_helper_hotplug_event(ddev); + amdgpu_dm_irq_resume_late(adev); amdgpu_dm_smu_write_watermarks_table(adev); @@ -6026,8 +6098,6 @@ create_stream_for_sink(struct amdgpu_dm_connector *aconnector, if (recalculate_timing) drm_mode_set_crtcinfo(&saved_mode, 0); - else if (!old_stream) - drm_mode_set_crtcinfo(&mode, 0); /* * If scaling is enabled and refresh rate didn't change @@ -6589,6 +6659,8 @@ enum drm_mode_status amdgpu_dm_connector_mode_valid(struct drm_connector *connec goto fail; } + drm_mode_set_crtcinfo(mode, 0); + stream = create_validate_stream_for_sink(aconnector, mode, to_dm_connector_state(connector->state), NULL); @@ -8073,7 +8145,8 @@ static void amdgpu_dm_commit_planes(struct drm_atomic_state *state, bundle->surface_updates[planes_count].plane_info = &bundle->plane_infos[planes_count]; - if (acrtc_state->stream->link->psr_settings.psr_feature_enabled) { + if (acrtc_state->stream->link->psr_settings.psr_feature_enabled || + acrtc_state->stream->link->replay_settings.replay_feature_enabled) { fill_dc_dirty_rects(plane, old_plane_state, new_plane_state, new_crtc_state, &bundle->flip_addrs[planes_count], diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.h b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.h index a2d34be82613..9e4cc5eeda76 100644 --- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.h +++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.h @@ -620,7 +620,7 @@ struct amdgpu_hdmi_vsdb_info { unsigned int max_refresh_rate_hz; /** - * @replay mode: Replay supported + * @replay_mode: Replay supported */ bool replay_mode; }; diff --git a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn20/dcn20_clk_mgr.c b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn20/dcn20_clk_mgr.c index c435f7632e8e..5ee87965a078 100644 --- a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn20/dcn20_clk_mgr.c +++ b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn20/dcn20_clk_mgr.c @@ -157,7 +157,7 @@ void dcn20_update_clocks_update_dentist(struct clk_mgr_internal *clk_mgr, struct int32_t N; int32_t j; - if (!pipe_ctx->stream) + if (!resource_is_pipe_type(pipe_ctx, OTG_MASTER)) continue; /* Virtual encoders don't have this function */ if (!stream_enc->funcs->get_fifo_cal_average_level) @@ -188,7 +188,7 @@ void dcn20_update_clocks_update_dentist(struct clk_mgr_internal *clk_mgr, struct int32_t N; int32_t j; - if (!pipe_ctx->stream) + if (!resource_is_pipe_type(pipe_ctx, OTG_MASTER)) continue; /* Virtual encoders don't have this function */ if (!stream_enc->funcs->get_fifo_cal_average_level) diff --git a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn32/dcn32_clk_mgr.c b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn32/dcn32_clk_mgr.c index 984b52923534..e9345f6554db 100644 --- a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn32/dcn32_clk_mgr.c +++ b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn32/dcn32_clk_mgr.c @@ -355,7 +355,7 @@ static void dcn32_update_clocks_update_dentist( int32_t N; int32_t j; - if (!pipe_ctx->stream) + if (!resource_is_pipe_type(pipe_ctx, OTG_MASTER)) continue; /* Virtual encoders don't have this function */ if (!stream_enc->funcs->get_fifo_cal_average_level) @@ -401,7 +401,7 @@ static void dcn32_update_clocks_update_dentist( int32_t N; int32_t j; - if (!pipe_ctx->stream) + if (!resource_is_pipe_type(pipe_ctx, OTG_MASTER)) continue; /* Virtual encoders don't have this function */ if (!stream_enc->funcs->get_fifo_cal_average_level) diff --git a/drivers/gpu/drm/amd/display/dc/core/dc.c b/drivers/gpu/drm/amd/display/dc/core/dc.c index 3a9077b60029..d08e60dff46d 100644 --- a/drivers/gpu/drm/amd/display/dc/core/dc.c +++ b/drivers/gpu/drm/amd/display/dc/core/dc.c @@ -1262,6 +1262,9 @@ static void disable_vbios_mode_if_required( if (stream == NULL) continue; + if (stream->apply_seamless_boot_optimization) + continue; + // only looking for first odm pipe if (pipe->prev_odm_pipe) continue; diff --git a/drivers/gpu/drm/amd/display/dc/core/dc_link_enc_cfg.c b/drivers/gpu/drm/amd/display/dc/core/dc_link_enc_cfg.c index 30c0644d4418..be5a6d008b29 100644 --- a/drivers/gpu/drm/amd/display/dc/core/dc_link_enc_cfg.c +++ b/drivers/gpu/drm/amd/display/dc/core/dc_link_enc_cfg.c @@ -169,11 +169,23 @@ static void add_link_enc_assignment( /* Return first available DIG link encoder. */ static enum engine_id find_first_avail_link_enc( const struct dc_context *ctx, - const struct dc_state *state) + const struct dc_state *state, + enum engine_id eng_id_requested) { enum engine_id eng_id = ENGINE_ID_UNKNOWN; int i; + if (eng_id_requested != ENGINE_ID_UNKNOWN) { + + for (i = 0; i < ctx->dc->res_pool->res_cap->num_dig_link_enc; i++) { + eng_id = state->res_ctx.link_enc_cfg_ctx.link_enc_avail[i]; + if (eng_id == eng_id_requested) + return eng_id; + } + } + + eng_id = ENGINE_ID_UNKNOWN; + for (i = 0; i < ctx->dc->res_pool->res_cap->num_dig_link_enc; i++) { eng_id = state->res_ctx.link_enc_cfg_ctx.link_enc_avail[i]; if (eng_id != ENGINE_ID_UNKNOWN) @@ -287,7 +299,7 @@ void link_enc_cfg_link_encs_assign( struct dc_stream_state *streams[], uint8_t stream_count) { - enum engine_id eng_id = ENGINE_ID_UNKNOWN; + enum engine_id eng_id = ENGINE_ID_UNKNOWN, eng_id_req = ENGINE_ID_UNKNOWN; int i; int j; @@ -377,8 +389,14 @@ void link_enc_cfg_link_encs_assign( * assigned to that endpoint. */ link_enc = get_link_enc_used_by_link(state, stream->link); - if (link_enc == NULL) - eng_id = find_first_avail_link_enc(stream->ctx, state); + if (link_enc == NULL) { + + if (stream->link->ep_type == DISPLAY_ENDPOINT_USB4_DPIA && + stream->link->dpia_preferred_eng_id != ENGINE_ID_UNKNOWN) + eng_id_req = stream->link->dpia_preferred_eng_id; + + eng_id = find_first_avail_link_enc(stream->ctx, state, eng_id_req); + } else eng_id = link_enc->preferred_engine; @@ -402,7 +420,9 @@ void link_enc_cfg_link_encs_assign( DC_LOG_DEBUG("%s: CUR %s(%d) - enc_id(%d)\n", __func__, assignment.ep_id.ep_type == DISPLAY_ENDPOINT_PHY ? "PHY" : "DPIA", - assignment.ep_id.link_id.enum_id - 1, + assignment.ep_id.ep_type == DISPLAY_ENDPOINT_PHY ? + assignment.ep_id.link_id.enum_id : + assignment.ep_id.link_id.enum_id - 1, assignment.eng_id); } for (i = 0; i < MAX_PIPES; i++) { @@ -413,7 +433,9 @@ void link_enc_cfg_link_encs_assign( DC_LOG_DEBUG("%s: NEW %s(%d) - enc_id(%d)\n", __func__, assignment.ep_id.ep_type == DISPLAY_ENDPOINT_PHY ? "PHY" : "DPIA", - assignment.ep_id.link_id.enum_id - 1, + assignment.ep_id.ep_type == DISPLAY_ENDPOINT_PHY ? + assignment.ep_id.link_id.enum_id : + assignment.ep_id.link_id.enum_id - 1, assignment.eng_id); } @@ -478,7 +500,6 @@ struct dc_link *link_enc_cfg_get_link_using_link_enc( if (stream) link = stream->link; - // dm_output_to_console("%s: No link using DIG(%d).\n", __func__, eng_id); return link; } diff --git a/drivers/gpu/drm/amd/display/dc/dc.h b/drivers/gpu/drm/amd/display/dc/dc.h index 0d0bef8eb331..31e3183497a7 100644 --- a/drivers/gpu/drm/amd/display/dc/dc.h +++ b/drivers/gpu/drm/amd/display/dc/dc.h @@ -1496,6 +1496,7 @@ struct dc_link { * object creation. */ enum engine_id eng_id; + enum engine_id dpia_preferred_eng_id; bool test_pattern_enabled; enum dp_test_pattern current_test_pattern; diff --git a/drivers/gpu/drm/amd/display/dc/dce110/dce110_hw_sequencer.c b/drivers/gpu/drm/amd/display/dc/dce110/dce110_hw_sequencer.c index ad967b58d7be..2a6157555fd1 100644 --- a/drivers/gpu/drm/amd/display/dc/dce110/dce110_hw_sequencer.c +++ b/drivers/gpu/drm/amd/display/dc/dce110/dce110_hw_sequencer.c @@ -964,7 +964,9 @@ void dce110_edp_backlight_control( return; } - if (link->panel_cntl) { + if (link->panel_cntl && !(link->dpcd_sink_ext_caps.bits.oled || + link->dpcd_sink_ext_caps.bits.hdr_aux_backlight_control == 1 || + link->dpcd_sink_ext_caps.bits.sdr_aux_backlight_control == 1)) { bool is_backlight_on = link->panel_cntl->funcs->is_panel_backlight_on(link->panel_cntl); if ((enable && is_backlight_on) || (!enable && !is_backlight_on)) { @@ -1176,12 +1178,15 @@ void dce110_disable_stream(struct pipe_ctx *pipe_ctx) dto_params.otg_inst = tg->inst; dto_params.timing = &pipe_ctx->stream->timing; dp_hpo_inst = pipe_ctx->stream_res.hpo_dp_stream_enc->inst; - dccg->funcs->set_dtbclk_dto(dccg, &dto_params); - dccg->funcs->disable_symclk32_se(dccg, dp_hpo_inst); - dccg->funcs->set_dpstreamclk(dccg, REFCLK, tg->inst, dp_hpo_inst); - } else if (pipe_ctx->stream->signal == SIGNAL_TYPE_DISPLAY_PORT_MST && dccg->funcs->disable_symclk_se) + if (dccg) { + dccg->funcs->set_dtbclk_dto(dccg, &dto_params); + dccg->funcs->disable_symclk32_se(dccg, dp_hpo_inst); + dccg->funcs->set_dpstreamclk(dccg, REFCLK, tg->inst, dp_hpo_inst); + } + } else if (dccg && dccg->funcs->disable_symclk_se) { dccg->funcs->disable_symclk_se(dccg, stream_enc->stream_enc_inst, link_enc->transmitter - TRANSMITTER_UNIPHY_A); + } if (dc->link_srv->dp_is_128b_132b_signal(pipe_ctx)) { /* TODO: This looks like a bug to me as we are disabling HPO IO when @@ -2656,11 +2661,11 @@ void dce110_prepare_bandwidth( struct clk_mgr *dccg = dc->clk_mgr; dce110_set_safe_displaymarks(&context->res_ctx, dc->res_pool); - - dccg->funcs->update_clocks( - dccg, - context, - false); + if (dccg) + dccg->funcs->update_clocks( + dccg, + context, + false); } void dce110_optimize_bandwidth( @@ -2671,10 +2676,11 @@ void dce110_optimize_bandwidth( dce110_set_displaymarks(dc, context); - dccg->funcs->update_clocks( - dccg, - context, - true); + if (dccg) + dccg->funcs->update_clocks( + dccg, + context, + true); } static void dce110_program_front_end_for_pipe( diff --git a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_hwseq.c b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_hwseq.c index e72f15ac0048..aeadc587433f 100644 --- a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_hwseq.c +++ b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_hwseq.c @@ -2692,8 +2692,6 @@ void dcn20_enable_stream(struct pipe_ctx *pipe_ctx) struct dce_hwseq *hws = dc->hwseq; unsigned int k1_div = PIXEL_RATE_DIV_NA; unsigned int k2_div = PIXEL_RATE_DIV_NA; - struct link_encoder *link_enc = link_enc_cfg_get_link_enc(pipe_ctx->stream->link); - struct stream_encoder *stream_enc = pipe_ctx->stream_res.stream_enc; if (dc->link_srv->dp_is_128b_132b_signal(pipe_ctx)) { if (dc->hwseq->funcs.setup_hpo_hw_control) @@ -2713,10 +2711,8 @@ void dcn20_enable_stream(struct pipe_ctx *pipe_ctx) dto_params.timing = &pipe_ctx->stream->timing; dto_params.ref_dtbclk_khz = dc->clk_mgr->funcs->get_dtb_ref_clk_frequency(dc->clk_mgr); dccg->funcs->set_dtbclk_dto(dccg, &dto_params); - } else if (pipe_ctx->stream->signal == SIGNAL_TYPE_DISPLAY_PORT_MST && dccg->funcs->enable_symclk_se) - dccg->funcs->enable_symclk_se(dccg, - stream_enc->stream_enc_inst, link_enc->transmitter - TRANSMITTER_UNIPHY_A); - + } else { + } if (hws->funcs.calculate_dccg_k1_k2_values && dc->res_pool->dccg->funcs->set_pixel_rate_div) { hws->funcs.calculate_dccg_k1_k2_values(pipe_ctx, &k1_div, &k2_div); diff --git a/drivers/gpu/drm/amd/display/dc/dcn314/dcn314_resource.c b/drivers/gpu/drm/amd/display/dc/dcn314/dcn314_resource.c index 1c1fb2fa0822..004beed9bd44 100644 --- a/drivers/gpu/drm/amd/display/dc/dcn314/dcn314_resource.c +++ b/drivers/gpu/drm/amd/display/dc/dcn314/dcn314_resource.c @@ -1032,6 +1032,28 @@ static const struct dce_i2c_mask i2c_masks = { I2C_COMMON_MASK_SH_LIST_DCN30(_MASK) }; +/* ========================================================== */ + +/* + * DPIA index | Preferred Encoder | Host Router + * 0 | C | 0 + * 1 | First Available | 0 + * 2 | D | 1 + * 3 | First Available | 1 + */ +/* ========================================================== */ +static const enum engine_id dpia_to_preferred_enc_id_table[] = { + ENGINE_ID_DIGC, + ENGINE_ID_DIGC, + ENGINE_ID_DIGD, + ENGINE_ID_DIGD +}; + +static enum engine_id dcn314_get_preferred_eng_id_dpia(unsigned int dpia_index) +{ + return dpia_to_preferred_enc_id_table[dpia_index]; +} + static struct dce_i2c_hw *dcn31_i2c_hw_create( struct dc_context *ctx, uint32_t inst) @@ -1785,6 +1807,7 @@ static struct resource_funcs dcn314_res_pool_funcs = { .update_bw_bounding_box = dcn314_update_bw_bounding_box, .patch_unknown_plane_state = dcn20_patch_unknown_plane_state, .get_panel_config_defaults = dcn314_get_panel_config_defaults, + .get_preferred_eng_id_dpia = dcn314_get_preferred_eng_id_dpia, }; static struct clock_source *dcn30_clock_source_create( diff --git a/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_mpc.c b/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_mpc.c index 3082da04a63d..1d052f08aff5 100644 --- a/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_mpc.c +++ b/drivers/gpu/drm/amd/display/dc/dcn32/dcn32_mpc.c @@ -75,7 +75,7 @@ void mpc32_power_on_blnd_lut( if (power_on) { REG_UPDATE(MPCC_MCM_MEM_PWR_CTRL[mpcc_id], MPCC_MCM_1DLUT_MEM_PWR_FORCE, 0); REG_WAIT(MPCC_MCM_MEM_PWR_CTRL[mpcc_id], MPCC_MCM_1DLUT_MEM_PWR_STATE, 0, 1, 5); - } else { + } else if (!mpc->ctx->dc->debug.disable_mem_low_power) { ASSERT(false); /* TODO: change to mpc * dpp_base->ctx->dc->optimized_required = true; diff --git a/drivers/gpu/drm/amd/display/dc/inc/core_types.h b/drivers/gpu/drm/amd/display/dc/inc/core_types.h index 027aec70c070..eaad1260bfd1 100644 --- a/drivers/gpu/drm/amd/display/dc/inc/core_types.h +++ b/drivers/gpu/drm/amd/display/dc/inc/core_types.h @@ -65,6 +65,7 @@ struct resource_context; struct clk_bw_params; struct resource_funcs { + enum engine_id (*get_preferred_eng_id_dpia)(unsigned int dpia_index); void (*destroy)(struct resource_pool **pool); void (*link_init)(struct dc_link *link); struct panel_cntl*(*panel_cntl_create)( diff --git a/drivers/gpu/drm/amd/display/dc/link/link_factory.c b/drivers/gpu/drm/amd/display/dc/link/link_factory.c index 195ca9e52eda..0895742a3102 100644 --- a/drivers/gpu/drm/amd/display/dc/link/link_factory.c +++ b/drivers/gpu/drm/amd/display/dc/link/link_factory.c @@ -791,6 +791,10 @@ static bool construct_dpia(struct dc_link *link, /* Set dpia port index : 0 to number of dpia ports */ link->ddc_hw_inst = init_params->connector_index; + // Assign Dpia preferred eng_id + if (link->dc->res_pool->funcs->get_preferred_eng_id_dpia) + link->dpia_preferred_eng_id = link->dc->res_pool->funcs->get_preferred_eng_id_dpia(link->ddc_hw_inst); + /* TODO: Create link encoder */ link->psr_settings.psr_version = DC_PSR_VERSION_UNSUPPORTED; diff --git a/drivers/gpu/drm/amd/include/kgd_kfd_interface.h b/drivers/gpu/drm/amd/include/kgd_kfd_interface.h index 8433f99f6667..3b5a56585c4b 100644 --- a/drivers/gpu/drm/amd/include/kgd_kfd_interface.h +++ b/drivers/gpu/drm/amd/include/kgd_kfd_interface.h @@ -31,12 +31,12 @@ #include #include #include +#include "amdgpu_irq.h" +#include "amdgpu_gfx.h" struct pci_dev; struct amdgpu_device; -#define KGD_MAX_QUEUES 128 - struct kfd_dev; struct kgd_mem; @@ -68,7 +68,7 @@ struct kfd_cu_info { uint32_t wave_front_size; uint32_t max_scratch_slots_per_cu; uint32_t lds_size; - uint32_t cu_bitmap[4][4]; + uint32_t cu_bitmap[AMDGPU_MAX_GC_INSTANCES][4][4]; }; /* For getting GPU local memory information from KGD */ @@ -326,8 +326,7 @@ struct kfd2kgd_calls { uint32_t wait_times, uint32_t grace_period, uint32_t *reg_offset, - uint32_t *reg_data, - uint32_t inst); + uint32_t *reg_data); void (*get_cu_occupancy)(struct amdgpu_device *adev, int pasid, int *wave_cnt, int *max_waves_per_cu, uint32_t inst); void (*program_trap_handler_settings)(struct amdgpu_device *adev, diff --git a/drivers/gpu/drm/amd/pm/amdgpu_pm.c b/drivers/gpu/drm/amd/pm/amdgpu_pm.c index 41147da54458..8bb2da13826f 100644 --- a/drivers/gpu/drm/amd/pm/amdgpu_pm.c +++ b/drivers/gpu/drm/amd/pm/amdgpu_pm.c @@ -2040,6 +2040,7 @@ static int default_attr_update(struct amdgpu_device *adev, struct amdgpu_device_ case IP_VERSION(11, 0, 0): case IP_VERSION(11, 0, 1): case IP_VERSION(11, 0, 2): + case IP_VERSION(11, 0, 3): *states = ATTR_STATE_SUPPORTED; break; default: diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu11/sienna_cichlid_ppt.c b/drivers/gpu/drm/amd/pm/swsmu/smu11/sienna_cichlid_ppt.c index 4bb289f9b4b8..da2860da6018 100644 --- a/drivers/gpu/drm/amd/pm/swsmu/smu11/sienna_cichlid_ppt.c +++ b/drivers/gpu/drm/amd/pm/swsmu/smu11/sienna_cichlid_ppt.c @@ -2082,36 +2082,41 @@ static int sienna_cichlid_display_disable_memory_clock_switch(struct smu_context return ret; } +#define MAX(a, b) ((a) > (b) ? (a) : (b)) + static int sienna_cichlid_update_pcie_parameters(struct smu_context *smu, uint32_t pcie_gen_cap, uint32_t pcie_width_cap) { struct smu_11_0_dpm_context *dpm_context = smu->smu_dpm.dpm_context; struct smu_11_0_pcie_table *pcie_table = &dpm_context->dpm_tables.pcie_table; - u32 smu_pcie_arg; + uint8_t *table_member1, *table_member2; + uint32_t min_gen_speed, max_gen_speed; + uint32_t min_lane_width, max_lane_width; + uint32_t smu_pcie_arg; int ret, i; - /* PCIE gen speed and lane width override */ - if (!amdgpu_device_pcie_dynamic_switching_supported()) { - if (pcie_table->pcie_gen[NUM_LINK_LEVELS - 1] < pcie_gen_cap) - pcie_gen_cap = pcie_table->pcie_gen[NUM_LINK_LEVELS - 1]; + GET_PPTABLE_MEMBER(PcieGenSpeed, &table_member1); + GET_PPTABLE_MEMBER(PcieLaneCount, &table_member2); - if (pcie_table->pcie_lane[NUM_LINK_LEVELS - 1] < pcie_width_cap) - pcie_width_cap = pcie_table->pcie_lane[NUM_LINK_LEVELS - 1]; + min_gen_speed = MAX(0, table_member1[0]); + max_gen_speed = MIN(pcie_gen_cap, table_member1[1]); + min_gen_speed = min_gen_speed > max_gen_speed ? + max_gen_speed : min_gen_speed; + min_lane_width = MAX(1, table_member2[0]); + max_lane_width = MIN(pcie_width_cap, table_member2[1]); + min_lane_width = min_lane_width > max_lane_width ? + max_lane_width : min_lane_width; - /* Force all levels to use the same settings */ - for (i = 0; i < NUM_LINK_LEVELS; i++) { - pcie_table->pcie_gen[i] = pcie_gen_cap; - pcie_table->pcie_lane[i] = pcie_width_cap; - } + if (!amdgpu_device_pcie_dynamic_switching_supported()) { + pcie_table->pcie_gen[0] = max_gen_speed; + pcie_table->pcie_lane[0] = max_lane_width; } else { - for (i = 0; i < NUM_LINK_LEVELS; i++) { - if (pcie_table->pcie_gen[i] > pcie_gen_cap) - pcie_table->pcie_gen[i] = pcie_gen_cap; - if (pcie_table->pcie_lane[i] > pcie_width_cap) - pcie_table->pcie_lane[i] = pcie_width_cap; - } + pcie_table->pcie_gen[0] = min_gen_speed; + pcie_table->pcie_lane[0] = min_lane_width; } + pcie_table->pcie_gen[1] = max_gen_speed; + pcie_table->pcie_lane[1] = max_lane_width; for (i = 0; i < NUM_LINK_LEVELS; i++) { smu_pcie_arg = (i << 16 | diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_6_ppt.c b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_6_ppt.c index 199a673b8120..de80e191a92c 100644 --- a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_6_ppt.c +++ b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_6_ppt.c @@ -336,7 +336,7 @@ static int smu_v13_0_6_setup_driver_pptable(struct smu_context *smu) /* Store one-time values in driver PPTable */ if (!pptable->Init) { - while (retry--) { + while (--retry) { ret = smu_v13_0_6_get_metrics_table(smu, NULL, true); if (ret) return ret; diff --git a/drivers/gpu/drm/bridge/ti-sn65dsi86.c b/drivers/gpu/drm/bridge/ti-sn65dsi86.c index f448b903e190..84148a79414b 100644 --- a/drivers/gpu/drm/bridge/ti-sn65dsi86.c +++ b/drivers/gpu/drm/bridge/ti-sn65dsi86.c @@ -692,7 +692,7 @@ static struct ti_sn65dsi86 *bridge_to_ti_sn65dsi86(struct drm_bridge *bridge) return container_of(bridge, struct ti_sn65dsi86, bridge); } -static int ti_sn_attach_host(struct ti_sn65dsi86 *pdata) +static int ti_sn_attach_host(struct auxiliary_device *adev, struct ti_sn65dsi86 *pdata) { int val; struct mipi_dsi_host *host; @@ -707,7 +707,7 @@ static int ti_sn_attach_host(struct ti_sn65dsi86 *pdata) if (!host) return -EPROBE_DEFER; - dsi = devm_mipi_dsi_device_register_full(dev, host, &info); + dsi = devm_mipi_dsi_device_register_full(&adev->dev, host, &info); if (IS_ERR(dsi)) return PTR_ERR(dsi); @@ -725,7 +725,7 @@ static int ti_sn_attach_host(struct ti_sn65dsi86 *pdata) pdata->dsi = dsi; - return devm_mipi_dsi_attach(dev, dsi); + return devm_mipi_dsi_attach(&adev->dev, dsi); } static int ti_sn_bridge_attach(struct drm_bridge *bridge, @@ -1298,9 +1298,9 @@ static int ti_sn_bridge_probe(struct auxiliary_device *adev, struct device_node *np = pdata->dev->of_node; int ret; - pdata->next_bridge = devm_drm_of_get_bridge(pdata->dev, np, 1, 0); + pdata->next_bridge = devm_drm_of_get_bridge(&adev->dev, np, 1, 0); if (IS_ERR(pdata->next_bridge)) - return dev_err_probe(pdata->dev, PTR_ERR(pdata->next_bridge), + return dev_err_probe(&adev->dev, PTR_ERR(pdata->next_bridge), "failed to create panel bridge\n"); ti_sn_bridge_parse_lanes(pdata, np); @@ -1319,9 +1319,9 @@ static int ti_sn_bridge_probe(struct auxiliary_device *adev, drm_bridge_add(&pdata->bridge); - ret = ti_sn_attach_host(pdata); + ret = ti_sn_attach_host(adev, pdata); if (ret) { - dev_err_probe(pdata->dev, ret, "failed to attach dsi host\n"); + dev_err_probe(&adev->dev, ret, "failed to attach dsi host\n"); goto err_remove_bridge; } diff --git a/drivers/gpu/drm/drm_atomic_helper.c b/drivers/gpu/drm/drm_atomic_helper.c index 292e38eb6218..60794fcde1d5 100644 --- a/drivers/gpu/drm/drm_atomic_helper.c +++ b/drivers/gpu/drm/drm_atomic_helper.c @@ -290,7 +290,8 @@ static int update_connector_routing(struct drm_atomic_state *state, struct drm_connector *connector, struct drm_connector_state *old_connector_state, - struct drm_connector_state *new_connector_state) + struct drm_connector_state *new_connector_state, + bool added_by_user) { const struct drm_connector_helper_funcs *funcs; struct drm_encoder *new_encoder; @@ -339,9 +340,13 @@ update_connector_routing(struct drm_atomic_state *state, * there's a chance the connector may have been destroyed during the * process, but it's better to ignore that then cause * drm_atomic_helper_resume() to fail. + * + * Last, we want to ignore connector registration when the connector + * was not pulled in the atomic state by user-space (ie, was pulled + * in by the driver, e.g. when updating a DP-MST stream). */ if (!state->duplicated && drm_connector_is_unregistered(connector) && - crtc_state->active) { + added_by_user && crtc_state->active) { drm_dbg_atomic(connector->dev, "[CONNECTOR:%d:%s] is not registered\n", connector->base.id, connector->name); @@ -620,7 +625,10 @@ drm_atomic_helper_check_modeset(struct drm_device *dev, struct drm_connector *connector; struct drm_connector_state *old_connector_state, *new_connector_state; int i, ret; - unsigned int connectors_mask = 0; + unsigned int connectors_mask = 0, user_connectors_mask = 0; + + for_each_oldnew_connector_in_state(state, connector, old_connector_state, new_connector_state, i) + user_connectors_mask |= BIT(i); for_each_oldnew_crtc_in_state(state, crtc, old_crtc_state, new_crtc_state, i) { bool has_connectors = @@ -685,7 +693,8 @@ drm_atomic_helper_check_modeset(struct drm_device *dev, */ ret = update_connector_routing(state, connector, old_connector_state, - new_connector_state); + new_connector_state, + BIT(i) & user_connectors_mask); if (ret) return ret; if (old_connector_state->crtc) { diff --git a/drivers/gpu/drm/drm_connector.c b/drivers/gpu/drm/drm_connector.c index bf8371dc2a61..c44d5bcf1284 100644 --- a/drivers/gpu/drm/drm_connector.c +++ b/drivers/gpu/drm/drm_connector.c @@ -2203,6 +2203,7 @@ static int drm_mode_create_colorspace_property(struct drm_connector *connector, /** * drm_mode_create_hdmi_colorspace_property - create hdmi colorspace property * @connector: connector to create the Colorspace property on. + * @supported_colorspaces: bitmap of supported color spaces * * Called by a driver the first time it's needed, must be attached to desired * HDMI connectors. @@ -2227,6 +2228,7 @@ EXPORT_SYMBOL(drm_mode_create_hdmi_colorspace_property); /** * drm_mode_create_dp_colorspace_property - create dp colorspace property * @connector: connector to create the Colorspace property on. + * @supported_colorspaces: bitmap of supported color spaces * * Called by a driver the first time it's needed, must be attached to desired * DP connectors. diff --git a/drivers/gpu/drm/drm_edid.c b/drivers/gpu/drm/drm_edid.c index 340da8257b51..4b71040ae5be 100644 --- a/drivers/gpu/drm/drm_edid.c +++ b/drivers/gpu/drm/drm_edid.c @@ -123,6 +123,9 @@ static const struct edid_quirk { /* AEO model 0 reports 8 bpc, but is a 6 bpc panel */ EDID_QUIRK('A', 'E', 'O', 0, EDID_QUIRK_FORCE_6BPC), + /* BenQ GW2765 */ + EDID_QUIRK('B', 'N', 'Q', 0x78d6, EDID_QUIRK_FORCE_8BPC), + /* BOE model on HP Pavilion 15-n233sl reports 8 bpc, but is a 6 bpc panel */ EDID_QUIRK('B', 'O', 'E', 0x78b, EDID_QUIRK_FORCE_6BPC), diff --git a/drivers/gpu/drm/drm_exec.c b/drivers/gpu/drm/drm_exec.c index ff69cf0fb42a..5d2809de4517 100644 --- a/drivers/gpu/drm/drm_exec.c +++ b/drivers/gpu/drm/drm_exec.c @@ -56,7 +56,7 @@ static void drm_exec_unlock_all(struct drm_exec *exec) struct drm_gem_object *obj; unsigned long index; - drm_exec_for_each_locked_object(exec, index, obj) { + drm_exec_for_each_locked_object_reverse(exec, index, obj) { dma_resv_unlock(obj->resv); drm_gem_object_put(obj); } diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c index 6129b89bb366..44a948b80ee1 100644 --- a/drivers/gpu/drm/drm_gem.c +++ b/drivers/gpu/drm/drm_gem.c @@ -540,7 +540,7 @@ struct page **drm_gem_get_pages(struct drm_gem_object *obj) struct page **pages; struct folio *folio; struct folio_batch fbatch; - int i, j, npages; + long i, j, npages; if (WARN_ON(!obj->filp)) return ERR_PTR(-EINVAL); @@ -564,11 +564,13 @@ struct page **drm_gem_get_pages(struct drm_gem_object *obj) i = 0; while (i < npages) { + long nr; folio = shmem_read_folio_gfp(mapping, i, mapping_gfp_mask(mapping)); if (IS_ERR(folio)) goto fail; - for (j = 0; j < folio_nr_pages(folio); j++, i++) + nr = min(npages - i, folio_nr_pages(folio)); + for (j = 0; j < nr; j++, i++) pages[i] = folio_file_page(folio, i); /* Make sure shmem keeps __GFP_DMA32 allocated pages in the diff --git a/drivers/gpu/drm/drm_panel_orientation_quirks.c b/drivers/gpu/drm/drm_panel_orientation_quirks.c index 0cb646cb04ee..d5c15292ae93 100644 --- a/drivers/gpu/drm/drm_panel_orientation_quirks.c +++ b/drivers/gpu/drm/drm_panel_orientation_quirks.c @@ -38,6 +38,14 @@ static const struct drm_dmi_panel_orientation_data gpd_micropc = { .orientation = DRM_MODE_PANEL_ORIENTATION_RIGHT_UP, }; +static const struct drm_dmi_panel_orientation_data gpd_onemix2s = { + .width = 1200, + .height = 1920, + .bios_dates = (const char * const []){ "05/21/2018", "10/26/2018", + "03/04/2019", NULL }, + .orientation = DRM_MODE_PANEL_ORIENTATION_RIGHT_UP, +}; + static const struct drm_dmi_panel_orientation_data gpd_pocket = { .width = 1200, .height = 1920, @@ -401,6 +409,14 @@ static const struct dmi_system_id orientation_data[] = { DMI_EXACT_MATCH(DMI_PRODUCT_NAME, "LTH17"), }, .driver_data = (void *)&lcd800x1280_rightside_up, + }, { /* One Mix 2S (generic strings, also match on bios date) */ + .matches = { + DMI_EXACT_MATCH(DMI_SYS_VENDOR, "Default string"), + DMI_EXACT_MATCH(DMI_PRODUCT_NAME, "Default string"), + DMI_EXACT_MATCH(DMI_BOARD_VENDOR, "Default string"), + DMI_EXACT_MATCH(DMI_BOARD_NAME, "Default string"), + }, + .driver_data = (void *)&gpd_onemix2s, }, {} }; diff --git a/drivers/gpu/drm/i915/display/intel_bios.c b/drivers/gpu/drm/i915/display/intel_bios.c index 858c959f7bab..f735b035436c 100644 --- a/drivers/gpu/drm/i915/display/intel_bios.c +++ b/drivers/gpu/drm/i915/display/intel_bios.c @@ -3540,6 +3540,27 @@ enum aux_ch intel_bios_dp_aux_ch(const struct intel_bios_encoder_data *devdata) return map_aux_ch(devdata->i915, devdata->child.aux_channel); } +bool intel_bios_dp_has_shared_aux_ch(const struct intel_bios_encoder_data *devdata) +{ + struct drm_i915_private *i915; + u8 aux_channel; + int count = 0; + + if (!devdata || !devdata->child.aux_channel) + return false; + + i915 = devdata->i915; + aux_channel = devdata->child.aux_channel; + + list_for_each_entry(devdata, &i915->display.vbt.display_devices, node) { + if (intel_bios_encoder_supports_dp(devdata) && + aux_channel == devdata->child.aux_channel) + count++; + } + + return count > 1; +} + int intel_bios_dp_boost_level(const struct intel_bios_encoder_data *devdata) { if (!devdata || devdata->i915->display.vbt.version < 196 || !devdata->child.iboost) diff --git a/drivers/gpu/drm/i915/display/intel_bios.h b/drivers/gpu/drm/i915/display/intel_bios.h index 9680e3e92bb5..49e24b7cf675 100644 --- a/drivers/gpu/drm/i915/display/intel_bios.h +++ b/drivers/gpu/drm/i915/display/intel_bios.h @@ -273,6 +273,7 @@ enum aux_ch intel_bios_dp_aux_ch(const struct intel_bios_encoder_data *devdata); int intel_bios_dp_boost_level(const struct intel_bios_encoder_data *devdata); int intel_bios_dp_max_lane_count(const struct intel_bios_encoder_data *devdata); int intel_bios_dp_max_link_rate(const struct intel_bios_encoder_data *devdata); +bool intel_bios_dp_has_shared_aux_ch(const struct intel_bios_encoder_data *devdata); int intel_bios_hdmi_boost_level(const struct intel_bios_encoder_data *devdata); int intel_bios_hdmi_ddc_pin(const struct intel_bios_encoder_data *devdata); int intel_bios_hdmi_level_shift(const struct intel_bios_encoder_data *devdata); diff --git a/drivers/gpu/drm/i915/display/intel_cx0_phy.c b/drivers/gpu/drm/i915/display/intel_cx0_phy.c index 1b00ef2c6185..80e4ec6ee403 100644 --- a/drivers/gpu/drm/i915/display/intel_cx0_phy.c +++ b/drivers/gpu/drm/i915/display/intel_cx0_phy.c @@ -2553,8 +2553,7 @@ static void intel_cx0_phy_lane_reset(struct drm_i915_private *i915, drm_warn(&i915->drm, "PHY %c failed to bring out of SOC reset after %dus.\n", phy_name(phy), XELPDP_PORT_BUF_SOC_READY_TIMEOUT_US); - intel_de_rmw(i915, XELPDP_PORT_BUF_CTL2(port), - XELPDP_LANE_PIPE_RESET(0) | XELPDP_LANE_PIPE_RESET(1), + intel_de_rmw(i915, XELPDP_PORT_BUF_CTL2(port), lane_pipe_reset, lane_pipe_reset); if (__intel_de_wait_for_register(i915, XELPDP_PORT_BUF_CTL2(port), diff --git a/drivers/gpu/drm/i915/display/intel_dp.c b/drivers/gpu/drm/i915/display/intel_dp.c index 12bd2f322e62..e0e4cb529284 100644 --- a/drivers/gpu/drm/i915/display/intel_dp.c +++ b/drivers/gpu/drm/i915/display/intel_dp.c @@ -5512,8 +5512,13 @@ static bool intel_edp_init_connector(struct intel_dp *intel_dp, /* * VBT and straps are liars. Also check HPD as that seems * to be the most reliable piece of information available. + * + * ... expect on devices that forgot to hook HPD up for eDP + * (eg. Acer Chromebook C710), so we'll check it only if multiple + * ports are attempting to use the same AUX CH, according to VBT. */ - if (!intel_digital_port_connected(encoder)) { + if (intel_bios_dp_has_shared_aux_ch(encoder->devdata) && + !intel_digital_port_connected(encoder)) { /* * If this fails, presume the DPCD answer came * from some other port using the same AUX CH. diff --git a/drivers/gpu/drm/i915/gem/i915_gem_mman.c b/drivers/gpu/drm/i915/gem/i915_gem_mman.c index aa4d842d4c5a..310654542b42 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_mman.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_mman.c @@ -235,6 +235,7 @@ static vm_fault_t i915_error_to_vmf_fault(int err) case 0: case -EAGAIN: case -ENOSPC: /* transient failure to evict? */ + case -ENOBUFS: /* temporarily out of fences? */ case -ERESTARTSYS: case -EINTR: case -EBUSY: diff --git a/drivers/gpu/drm/i915/gem/i915_gem_pages.c b/drivers/gpu/drm/i915/gem/i915_gem_pages.c index 6b6d22c19411..0ba955611dfb 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_pages.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_pages.c @@ -198,7 +198,7 @@ static void flush_tlb_invalidate(struct drm_i915_gem_object *obj) for_each_gt(gt, i915, id) { if (!obj->mm.tlb[id]) - return; + continue; intel_gt_invalidate_tlb_full(gt, obj->mm.tlb[id]); obj->mm.tlb[id] = 0; diff --git a/drivers/gpu/drm/i915/gem/i915_gem_shmem.c b/drivers/gpu/drm/i915/gem/i915_gem_shmem.c index 8f1633c3fb93..73a4a4eb29e0 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_shmem.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_shmem.c @@ -100,6 +100,7 @@ int shmem_sg_alloc_table(struct drm_i915_private *i915, struct sg_table *st, st->nents = 0; for (i = 0; i < page_count; i++) { struct folio *folio; + unsigned long nr_pages; const unsigned int shrink[] = { I915_SHRINK_BOUND | I915_SHRINK_UNBOUND, 0, @@ -150,6 +151,8 @@ int shmem_sg_alloc_table(struct drm_i915_private *i915, struct sg_table *st, } } while (1); + nr_pages = min_t(unsigned long, + folio_nr_pages(folio), page_count - i); if (!i || sg->length >= max_segment || folio_pfn(folio) != next_pfn) { @@ -157,13 +160,13 @@ int shmem_sg_alloc_table(struct drm_i915_private *i915, struct sg_table *st, sg = sg_next(sg); st->nents++; - sg_set_folio(sg, folio, folio_size(folio), 0); + sg_set_folio(sg, folio, nr_pages * PAGE_SIZE, 0); } else { /* XXX: could overflow? */ - sg->length += folio_size(folio); + sg->length += nr_pages * PAGE_SIZE; } - next_pfn = folio_pfn(folio) + folio_nr_pages(folio); - i += folio_nr_pages(folio) - 1; + next_pfn = folio_pfn(folio) + nr_pages; + i += nr_pages - 1; /* Check that the i965g/gm workaround works. */ GEM_BUG_ON(gfp & __GFP_DMA32 && next_pfn >= 0x00100000UL); diff --git a/drivers/gpu/drm/i915/gt/gen8_engine_cs.c b/drivers/gpu/drm/i915/gt/gen8_engine_cs.c index a4ff55aa5e55..7ad36198aab2 100644 --- a/drivers/gpu/drm/i915/gt/gen8_engine_cs.c +++ b/drivers/gpu/drm/i915/gt/gen8_engine_cs.c @@ -271,8 +271,17 @@ int gen12_emit_flush_rcs(struct i915_request *rq, u32 mode) if (GRAPHICS_VER_FULL(rq->i915) >= IP_VER(12, 70)) bit_group_0 |= PIPE_CONTROL_CCS_FLUSH; + /* + * L3 fabric flush is needed for AUX CCS invalidation + * which happens as part of pipe-control so we can + * ignore PIPE_CONTROL_FLUSH_L3. Also PIPE_CONTROL_FLUSH_L3 + * deals with Protected Memory which is not needed for + * AUX CCS invalidation and lead to unwanted side effects. + */ + if (mode & EMIT_FLUSH) + bit_group_1 |= PIPE_CONTROL_FLUSH_L3; + bit_group_1 |= PIPE_CONTROL_TILE_CACHE_FLUSH; - bit_group_1 |= PIPE_CONTROL_FLUSH_L3; bit_group_1 |= PIPE_CONTROL_RENDER_TARGET_CACHE_FLUSH; bit_group_1 |= PIPE_CONTROL_DEPTH_CACHE_FLUSH; /* Wa_1409600907:tgl,adl-p */ diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c index ee15486fed0d..e85d70a62123 100644 --- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c +++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c @@ -558,7 +558,6 @@ static int intel_engine_setup(struct intel_gt *gt, enum intel_engine_id id, DRIVER_CAPS(i915)->has_logical_contexts = true; ewma__engine_latency_init(&engine->latency); - seqcount_init(&engine->stats.execlists.lock); ATOMIC_INIT_NOTIFIER_HEAD(&engine->context_status_notifier); diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c index 8a641bcf777c..3292524469d5 100644 --- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c +++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c @@ -3550,6 +3550,8 @@ int intel_execlists_submission_setup(struct intel_engine_cs *engine) logical_ring_default_vfuncs(engine); logical_ring_default_irqs(engine); + seqcount_init(&engine->stats.execlists.lock); + if (engine->flags & I915_ENGINE_HAS_RCS_REG_STATE) rcs_submission_override(engine); diff --git a/drivers/gpu/drm/i915/gt/intel_ggtt.c b/drivers/gpu/drm/i915/gt/intel_ggtt.c index dd0ed941441a..da21f2786b5d 100644 --- a/drivers/gpu/drm/i915/gt/intel_ggtt.c +++ b/drivers/gpu/drm/i915/gt/intel_ggtt.c @@ -511,20 +511,31 @@ void intel_ggtt_unbind_vma(struct i915_address_space *vm, vm->clear_range(vm, vma_res->start, vma_res->vma_size); } +/* + * Reserve the top of the GuC address space for firmware images. Addresses + * beyond GUC_GGTT_TOP in the GuC address space are inaccessible by GuC, + * which makes for a suitable range to hold GuC/HuC firmware images if the + * size of the GGTT is 4G. However, on a 32-bit platform the size of the GGTT + * is limited to 2G, which is less than GUC_GGTT_TOP, but we reserve a chunk + * of the same size anyway, which is far more than needed, to keep the logic + * in uc_fw_ggtt_offset() simple. + */ +#define GUC_TOP_RESERVE_SIZE (SZ_4G - GUC_GGTT_TOP) + static int ggtt_reserve_guc_top(struct i915_ggtt *ggtt) { - u64 size; + u64 offset; int ret; if (!intel_uc_uses_guc(&ggtt->vm.gt->uc)) return 0; - GEM_BUG_ON(ggtt->vm.total <= GUC_GGTT_TOP); - size = ggtt->vm.total - GUC_GGTT_TOP; + GEM_BUG_ON(ggtt->vm.total <= GUC_TOP_RESERVE_SIZE); + offset = ggtt->vm.total - GUC_TOP_RESERVE_SIZE; - ret = i915_gem_gtt_reserve(&ggtt->vm, NULL, &ggtt->uc_fw, size, - GUC_GGTT_TOP, I915_COLOR_UNEVICTABLE, - PIN_NOEVICT); + ret = i915_gem_gtt_reserve(&ggtt->vm, NULL, &ggtt->uc_fw, + GUC_TOP_RESERVE_SIZE, offset, + I915_COLOR_UNEVICTABLE, PIN_NOEVICT); if (ret) drm_dbg(&ggtt->vm.i915->drm, "Failed to reserve top of GGTT for GuC\n"); diff --git a/drivers/gpu/drm/i915/gt/intel_lrc.c b/drivers/gpu/drm/i915/gt/intel_lrc.c index 957d0aeb0c02..c378cc7c953c 100644 --- a/drivers/gpu/drm/i915/gt/intel_lrc.c +++ b/drivers/gpu/drm/i915/gt/intel_lrc.c @@ -1094,6 +1094,9 @@ __lrc_alloc_state(struct intel_context *ce, struct intel_engine_cs *engine) I915_BO_ALLOC_PM_VOLATILE); if (IS_ERR(obj)) { obj = i915_gem_object_create_shmem(engine->i915, context_size); + if (IS_ERR(obj)) + return ERR_CAST(obj); + /* * Wa_22016122933: For Media version 13.0, all Media GT shared * memory needs to be mapped as WC on CPU side and UC (PAT @@ -1102,8 +1105,6 @@ __lrc_alloc_state(struct intel_context *ce, struct intel_engine_cs *engine) if (intel_gt_needs_wa_22016122933(engine->gt)) i915_gem_object_set_cache_coherency(obj, I915_CACHE_NONE); } - if (IS_ERR(obj)) - return ERR_CAST(obj); vma = i915_vma_instance(obj, &engine->gt->ggtt->vm, NULL); if (IS_ERR(vma)) { diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c index b5b7f2fe8c78..dc7b40e06e38 100644 --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c @@ -1432,6 +1432,36 @@ static void guc_timestamp_ping(struct work_struct *wrk) unsigned long index; int srcu, ret; + /* + * Ideally the busyness worker should take a gt pm wakeref because the + * worker only needs to be active while gt is awake. However, the + * gt_park path cancels the worker synchronously and this complicates + * the flow if the worker is also running at the same time. The cancel + * waits for the worker and when the worker releases the wakeref, that + * would call gt_park and would lead to a deadlock. + * + * The resolution is to take the global pm wakeref if runtime pm is + * already active. If not, we don't need to update the busyness stats as + * the stats would already be updated when the gt was parked. + * + * Note: + * - We do not requeue the worker if we cannot take a reference to runtime + * pm since intel_guc_busyness_unpark would requeue the worker in the + * resume path. + * + * - If the gt was parked longer than time taken for GT timestamp to roll + * over, we ignore those rollovers since we don't care about tracking + * the exact GT time. We only care about roll overs when the gt is + * active and running workloads. + * + * - There is a window of time between gt_park and runtime suspend, + * where the worker may run. This is acceptable since the worker will + * not find any new data to update busyness. + */ + wakeref = intel_runtime_pm_get_if_active(>->i915->runtime_pm); + if (!wakeref) + return; + /* * Synchronize with gt reset to make sure the worker does not * corrupt the engine/guc stats. NB: can't actually block waiting @@ -1440,10 +1470,9 @@ static void guc_timestamp_ping(struct work_struct *wrk) */ ret = intel_gt_reset_trylock(gt, &srcu); if (ret) - return; + goto err_trylock; - with_intel_runtime_pm(>->i915->runtime_pm, wakeref) - __update_guc_busyness_stats(guc); + __update_guc_busyness_stats(guc); /* adjust context stats for overflow */ xa_for_each(&guc->context_lookup, index, ce) @@ -1452,6 +1481,9 @@ static void guc_timestamp_ping(struct work_struct *wrk) intel_gt_reset_unlock(gt, srcu); guc_enable_busyness_worker(guc); + +err_trylock: + intel_runtime_pm_put(>->i915->runtime_pm, wakeref); } static int guc_action_enable_usage_stats(struct intel_guc *guc) diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c index 1f65bb33dd21..a8551ce322de 100644 --- a/drivers/gpu/drm/i915/i915_gem.c +++ b/drivers/gpu/drm/i915/i915_gem.c @@ -1199,6 +1199,13 @@ int i915_gem_init(struct drm_i915_private *dev_priv) goto err_unlock; } + /* + * Register engines early to ensure the engine list is in its final + * rb-tree form, lowering the amount of code that has to deal with + * the intermediate llist state. + */ + intel_engines_driver_register(dev_priv); + return 0; /* @@ -1246,8 +1253,6 @@ err_unlock: void i915_gem_driver_register(struct drm_i915_private *i915) { i915_gem_driver_register__shrinker(i915); - - intel_engines_driver_register(i915); } void i915_gem_driver_unregister(struct drm_i915_private *i915) diff --git a/drivers/gpu/drm/mediatek/mtk_drm_gem.c b/drivers/gpu/drm/mediatek/mtk_drm_gem.c index 9f364df52478..0e0a41b2f57f 100644 --- a/drivers/gpu/drm/mediatek/mtk_drm_gem.c +++ b/drivers/gpu/drm/mediatek/mtk_drm_gem.c @@ -239,6 +239,7 @@ int mtk_drm_gem_prime_vmap(struct drm_gem_object *obj, struct iosys_map *map) npages = obj->size >> PAGE_SHIFT; mtk_gem->pages = kcalloc(npages, sizeof(*mtk_gem->pages), GFP_KERNEL); if (!mtk_gem->pages) { + sg_free_table(sgt); kfree(sgt); return -ENOMEM; } @@ -248,12 +249,15 @@ int mtk_drm_gem_prime_vmap(struct drm_gem_object *obj, struct iosys_map *map) mtk_gem->kvaddr = vmap(mtk_gem->pages, npages, VM_MAP, pgprot_writecombine(PAGE_KERNEL)); if (!mtk_gem->kvaddr) { + sg_free_table(sgt); kfree(sgt); kfree(mtk_gem->pages); return -ENOMEM; } -out: + sg_free_table(sgt); kfree(sgt); + +out: iosys_map_set_vaddr(map, mtk_gem->kvaddr); return 0; diff --git a/drivers/gpu/drm/meson/meson_encoder_hdmi.c b/drivers/gpu/drm/meson/meson_encoder_hdmi.c index 9913971fa5d2..25ea76558690 100644 --- a/drivers/gpu/drm/meson/meson_encoder_hdmi.c +++ b/drivers/gpu/drm/meson/meson_encoder_hdmi.c @@ -334,6 +334,8 @@ static void meson_encoder_hdmi_hpd_notify(struct drm_bridge *bridge, return; cec_notifier_set_phys_addr_from_edid(encoder_hdmi->cec_notifier, edid); + + kfree(edid); } else cec_notifier_phys_addr_invalidate(encoder_hdmi->cec_notifier); } diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_plane.c b/drivers/gpu/drm/msm/disp/dpu1/dpu_plane.c index c2aaaded07ed..0be195f9149c 100644 --- a/drivers/gpu/drm/msm/disp/dpu1/dpu_plane.c +++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_plane.c @@ -119,6 +119,7 @@ static u64 _dpu_plane_calc_bw(const struct dpu_mdss_cfg *catalog, struct dpu_sw_pipe_cfg *pipe_cfg) { int src_width, src_height, dst_height, fps; + u64 plane_pixel_rate, plane_bit_rate; u64 plane_prefill_bw; u64 plane_bw; u32 hw_latency_lines; @@ -136,13 +137,12 @@ static u64 _dpu_plane_calc_bw(const struct dpu_mdss_cfg *catalog, scale_factor = src_height > dst_height ? mult_frac(src_height, 1, dst_height) : 1; - plane_bw = - src_width * mode->vtotal * fps * fmt->bpp * - scale_factor; + plane_pixel_rate = src_width * mode->vtotal * fps; + plane_bit_rate = plane_pixel_rate * fmt->bpp; - plane_prefill_bw = - src_width * hw_latency_lines * fps * fmt->bpp * - scale_factor * mode->vtotal; + plane_bw = plane_bit_rate * scale_factor; + + plane_prefill_bw = plane_bw * hw_latency_lines; if ((vbp+vpw) > hw_latency_lines) do_div(plane_prefill_bw, (vbp+vpw)); @@ -733,9 +733,11 @@ static int dpu_plane_check_inline_rotation(struct dpu_plane *pdpu, static int dpu_plane_atomic_check_pipe(struct dpu_plane *pdpu, struct dpu_sw_pipe *pipe, struct dpu_sw_pipe_cfg *pipe_cfg, - const struct dpu_format *fmt) + const struct dpu_format *fmt, + const struct drm_display_mode *mode) { uint32_t min_src_size; + struct dpu_kms *kms = _dpu_plane_get_kms(&pdpu->base); min_src_size = DPU_FORMAT_IS_YUV(fmt) ? 2 : 1; @@ -774,6 +776,12 @@ static int dpu_plane_atomic_check_pipe(struct dpu_plane *pdpu, return -EINVAL; } + /* max clk check */ + if (_dpu_plane_calc_clk(mode, pipe_cfg) > kms->perf.max_core_clk_rate) { + DPU_DEBUG_PLANE(pdpu, "plane exceeds max mdp core clk limits\n"); + return -E2BIG; + } + return 0; } @@ -899,12 +907,13 @@ static int dpu_plane_atomic_check(struct drm_plane *plane, r_pipe_cfg->dst_rect.x1 = pipe_cfg->dst_rect.x2; } - ret = dpu_plane_atomic_check_pipe(pdpu, pipe, pipe_cfg, fmt); + ret = dpu_plane_atomic_check_pipe(pdpu, pipe, pipe_cfg, fmt, &crtc_state->adjusted_mode); if (ret) return ret; if (r_pipe->sspp) { - ret = dpu_plane_atomic_check_pipe(pdpu, r_pipe, r_pipe_cfg, fmt); + ret = dpu_plane_atomic_check_pipe(pdpu, r_pipe, r_pipe_cfg, fmt, + &crtc_state->adjusted_mode); if (ret) return ret; } diff --git a/drivers/gpu/drm/msm/dp/dp_ctrl.c b/drivers/gpu/drm/msm/dp/dp_ctrl.c index a7a5c7e0ab92..77a8d9366ed7 100644 --- a/drivers/gpu/drm/msm/dp/dp_ctrl.c +++ b/drivers/gpu/drm/msm/dp/dp_ctrl.c @@ -1774,13 +1774,6 @@ int dp_ctrl_on_link(struct dp_ctrl *dp_ctrl) return rc; while (--link_train_max_retries) { - rc = dp_ctrl_reinitialize_mainlink(ctrl); - if (rc) { - DRM_ERROR("Failed to reinitialize mainlink. rc=%d\n", - rc); - break; - } - training_step = DP_TRAINING_NONE; rc = dp_ctrl_setup_main_link(ctrl, &training_step); if (rc == 0) { @@ -1832,6 +1825,12 @@ int dp_ctrl_on_link(struct dp_ctrl *dp_ctrl) /* stop link training before start re training */ dp_ctrl_clear_training_pattern(ctrl); } + + rc = dp_ctrl_reinitialize_mainlink(ctrl); + if (rc) { + DRM_ERROR("Failed to reinitialize mainlink. rc=%d\n", rc); + break; + } } if (ctrl->link->sink_request & DP_TEST_LINK_PHY_TEST_PATTERN) diff --git a/drivers/gpu/drm/msm/dp/dp_link.c b/drivers/gpu/drm/msm/dp/dp_link.c index 42427129acea..6375daaeb98e 100644 --- a/drivers/gpu/drm/msm/dp/dp_link.c +++ b/drivers/gpu/drm/msm/dp/dp_link.c @@ -1090,7 +1090,7 @@ int dp_link_process_request(struct dp_link *dp_link) } else if (dp_link_read_psr_error_status(link)) { DRM_ERROR("PSR IRQ_HPD received\n"); } else if (dp_link_psr_capability_changed(link)) { - drm_dbg_dp(link->drm_dev, "PSR Capability changed"); + drm_dbg_dp(link->drm_dev, "PSR Capability changed\n"); } else { ret = dp_link_process_link_status_update(link); if (!ret) { @@ -1107,7 +1107,7 @@ int dp_link_process_request(struct dp_link *dp_link) } } - drm_dbg_dp(link->drm_dev, "sink request=%#x", + drm_dbg_dp(link->drm_dev, "sink request=%#x\n", dp_link->sink_request); return ret; } diff --git a/drivers/gpu/drm/msm/dsi/dsi_host.c b/drivers/gpu/drm/msm/dsi/dsi_host.c index 5d9ec27c89d3..3d6fb708dc22 100644 --- a/drivers/gpu/drm/msm/dsi/dsi_host.c +++ b/drivers/gpu/drm/msm/dsi/dsi_host.c @@ -1082,9 +1082,21 @@ static void dsi_wait4video_done(struct msm_dsi_host *msm_host) static void dsi_wait4video_eng_busy(struct msm_dsi_host *msm_host) { + u32 data; + if (!(msm_host->mode_flags & MIPI_DSI_MODE_VIDEO)) return; + data = dsi_read(msm_host, REG_DSI_STATUS0); + + /* if video mode engine is not busy, its because + * either timing engine was not turned on or the + * DSI controller has finished transmitting the video + * data already, so no need to wait in those cases + */ + if (!(data & DSI_STATUS0_VIDEO_MODE_ENGINE_BUSY)) + return; + if (msm_host->power_on && msm_host->enabled) { dsi_wait4video_done(msm_host); /* delay 4 ms to skip BLLP */ @@ -1894,10 +1906,9 @@ int msm_dsi_host_init(struct msm_dsi *msm_dsi) } msm_host->irq = irq_of_parse_and_map(pdev->dev.of_node, 0); - if (msm_host->irq < 0) { - ret = msm_host->irq; - dev_err(&pdev->dev, "failed to get irq: %d\n", ret); - return ret; + if (!msm_host->irq) { + dev_err(&pdev->dev, "failed to get irq\n"); + return -EINVAL; } /* do not autoenable, will be enabled later */ diff --git a/drivers/gpu/drm/msm/msm_mdss.c b/drivers/gpu/drm/msm/msm_mdss.c index 2e87dd6cb17b..348c66b14683 100644 --- a/drivers/gpu/drm/msm/msm_mdss.c +++ b/drivers/gpu/drm/msm/msm_mdss.c @@ -511,7 +511,7 @@ static int mdss_remove(struct platform_device *pdev) static const struct msm_mdss_data msm8998_data = { .ubwc_enc_version = UBWC_1_0, .ubwc_dec_version = UBWC_1_0, - .highest_bank_bit = 1, + .highest_bank_bit = 2, }; static const struct msm_mdss_data qcm2290_data = { diff --git a/drivers/gpu/drm/nouveau/nouveau_abi16.c b/drivers/gpu/drm/nouveau/nouveau_abi16.c index 30afbec9e3b1..2edd7bb13fae 100644 --- a/drivers/gpu/drm/nouveau/nouveau_abi16.c +++ b/drivers/gpu/drm/nouveau/nouveau_abi16.c @@ -31,6 +31,7 @@ #include "nouveau_drv.h" #include "nouveau_dma.h" +#include "nouveau_exec.h" #include "nouveau_gem.h" #include "nouveau_chan.h" #include "nouveau_abi16.h" @@ -183,6 +184,20 @@ nouveau_abi16_fini(struct nouveau_abi16 *abi16) cli->abi16 = NULL; } +static inline int +getparam_dma_ib_max(struct nvif_device *device) +{ + const struct nvif_mclass dmas[] = { + { NV03_CHANNEL_DMA, 0 }, + { NV10_CHANNEL_DMA, 0 }, + { NV17_CHANNEL_DMA, 0 }, + { NV40_CHANNEL_DMA, 0 }, + {} + }; + + return nvif_mclass(&device->object, dmas) < 0 ? NV50_DMA_IB_MAX : 0; +} + int nouveau_abi16_ioctl_getparam(ABI16_IOCTL_ARGS) { @@ -247,6 +262,12 @@ nouveau_abi16_ioctl_getparam(ABI16_IOCTL_ARGS) case NOUVEAU_GETPARAM_GRAPH_UNITS: getparam->value = nvkm_gr_units(gr); break; + case NOUVEAU_GETPARAM_EXEC_PUSH_MAX: { + int ib_max = getparam_dma_ib_max(device); + + getparam->value = nouveau_exec_push_max_from_ib_max(ib_max); + break; + } default: NV_PRINTK(dbg, cli, "unknown parameter %lld\n", getparam->param); return -EINVAL; diff --git a/drivers/gpu/drm/nouveau/nouveau_chan.c b/drivers/gpu/drm/nouveau/nouveau_chan.c index bb3d6e5c122f..7c97b2886807 100644 --- a/drivers/gpu/drm/nouveau/nouveau_chan.c +++ b/drivers/gpu/drm/nouveau/nouveau_chan.c @@ -257,10 +257,7 @@ static int nouveau_channel_ctor(struct nouveau_drm *drm, struct nvif_device *device, bool priv, u64 runm, struct nouveau_channel **pchan) { - static const struct { - s32 oclass; - int version; - } hosts[] = { + const struct nvif_mclass hosts[] = { { AMPERE_CHANNEL_GPFIFO_B, 0 }, { AMPERE_CHANNEL_GPFIFO_A, 0 }, { TURING_CHANNEL_GPFIFO_A, 0 }, @@ -443,9 +440,11 @@ nouveau_channel_init(struct nouveau_channel *chan, u32 vram, u32 gart) } /* initialise dma tracking parameters */ - switch (chan->user.oclass & 0x00ff) { - case 0x006b: - case 0x006e: + switch (chan->user.oclass) { + case NV03_CHANNEL_DMA: + case NV10_CHANNEL_DMA: + case NV17_CHANNEL_DMA: + case NV40_CHANNEL_DMA: chan->user_put = 0x40; chan->user_get = 0x44; chan->dma.max = (0x10000 / 4) - 2; @@ -455,7 +454,7 @@ nouveau_channel_init(struct nouveau_channel *chan, u32 vram, u32 gart) chan->user_get = 0x44; chan->user_get_hi = 0x60; chan->dma.ib_base = 0x10000 / 4; - chan->dma.ib_max = (0x02000 / 8) - 1; + chan->dma.ib_max = NV50_DMA_IB_MAX; chan->dma.ib_put = 0; chan->dma.ib_free = chan->dma.ib_max - chan->dma.ib_put; chan->dma.max = chan->dma.ib_base; diff --git a/drivers/gpu/drm/nouveau/nouveau_dma.h b/drivers/gpu/drm/nouveau/nouveau_dma.h index 1744d95b233e..c52cda82353e 100644 --- a/drivers/gpu/drm/nouveau/nouveau_dma.h +++ b/drivers/gpu/drm/nouveau/nouveau_dma.h @@ -49,6 +49,9 @@ void nv50_dma_push(struct nouveau_channel *, u64 addr, u32 length, /* Maximum push buffer size. */ #define NV50_DMA_PUSH_MAX_LENGTH 0x7fffff +/* Maximum IBs per ring. */ +#define NV50_DMA_IB_MAX ((0x02000 / 8) - 1) + /* Object handles - for stuff that's doesn't use handle == oclass. */ enum { NvDmaFB = 0x80000002, diff --git a/drivers/gpu/drm/nouveau/nouveau_drv.h b/drivers/gpu/drm/nouveau/nouveau_drv.h index 1fe17ff95f5e..e73a233c6572 100644 --- a/drivers/gpu/drm/nouveau/nouveau_drv.h +++ b/drivers/gpu/drm/nouveau/nouveau_drv.h @@ -189,21 +189,12 @@ u_free(void *addr) static inline void * u_memcpya(uint64_t user, unsigned int nmemb, unsigned int size) { - void *mem; - void __user *userptr = (void __force __user *)(uintptr_t)user; + void __user *userptr = u64_to_user_ptr(user); + size_t bytes; - size *= nmemb; - - mem = kvmalloc(size, GFP_KERNEL); - if (!mem) - return ERR_PTR(-ENOMEM); - - if (copy_from_user(mem, userptr, size)) { - u_free(mem); - return ERR_PTR(-EFAULT); - } - - return mem; + if (unlikely(check_mul_overflow(nmemb, size, &bytes))) + return ERR_PTR(-EOVERFLOW); + return vmemdup_user(userptr, bytes); } #include diff --git a/drivers/gpu/drm/nouveau/nouveau_exec.c b/drivers/gpu/drm/nouveau/nouveau_exec.c index 19024ce21fbb..c1837ba95fb5 100644 --- a/drivers/gpu/drm/nouveau/nouveau_exec.c +++ b/drivers/gpu/drm/nouveau/nouveau_exec.c @@ -213,7 +213,7 @@ nouveau_exec_job_timeout(struct nouveau_job *job) nouveau_sched_entity_fini(job->entity); - return DRM_GPU_SCHED_STAT_ENODEV; + return DRM_GPU_SCHED_STAT_NOMINAL; } static struct nouveau_job_ops nouveau_exec_job_ops = { @@ -379,7 +379,7 @@ nouveau_exec_ioctl_exec(struct drm_device *dev, struct nouveau_channel *chan = NULL; struct nouveau_exec_job_args args = {}; struct drm_nouveau_exec *req = data; - int ret = 0; + int push_max, ret = 0; if (unlikely(!abi16)) return -ENOMEM; @@ -404,9 +404,10 @@ nouveau_exec_ioctl_exec(struct drm_device *dev, if (!chan->dma.ib_max) return nouveau_abi16_put(abi16, -ENOSYS); - if (unlikely(req->push_count > NOUVEAU_GEM_MAX_PUSH)) { + push_max = nouveau_exec_push_max_from_ib_max(chan->dma.ib_max); + if (unlikely(req->push_count > push_max)) { NV_PRINTK(err, cli, "pushbuf push count exceeds limit: %d max %d\n", - req->push_count, NOUVEAU_GEM_MAX_PUSH); + req->push_count, push_max); return nouveau_abi16_put(abi16, -EINVAL); } diff --git a/drivers/gpu/drm/nouveau/nouveau_exec.h b/drivers/gpu/drm/nouveau/nouveau_exec.h index 778cacd90f65..5488d337bcc0 100644 --- a/drivers/gpu/drm/nouveau/nouveau_exec.h +++ b/drivers/gpu/drm/nouveau/nouveau_exec.h @@ -51,4 +51,14 @@ int nouveau_exec_job_init(struct nouveau_exec_job **job, int nouveau_exec_ioctl_exec(struct drm_device *dev, void *data, struct drm_file *file_priv); +static inline unsigned int +nouveau_exec_push_max_from_ib_max(int ib_max) +{ + /* Limit the number of IBs per job to half the size of the ring in order + * to avoid the ring running dry between submissions and preserve one + * more slot for the job's HW fence. + */ + return ib_max > 1 ? ib_max / 2 - 1 : 0; +} + #endif diff --git a/drivers/gpu/drm/nouveau/nouveau_fence.c b/drivers/gpu/drm/nouveau/nouveau_fence.c index 61d9e70da9fd..ca762ea55413 100644 --- a/drivers/gpu/drm/nouveau/nouveau_fence.c +++ b/drivers/gpu/drm/nouveau/nouveau_fence.c @@ -207,7 +207,7 @@ nouveau_fence_context_new(struct nouveau_channel *chan, struct nouveau_fence_cha int nouveau_fence_emit(struct nouveau_fence *fence) { - struct nouveau_channel *chan = fence->channel; + struct nouveau_channel *chan = unrcu_pointer(fence->channel); struct nouveau_fence_chan *fctx = chan->fence; struct nouveau_fence_priv *priv = (void*)chan->drm->fence; int ret; diff --git a/drivers/gpu/drm/nouveau/nouveau_sched.c b/drivers/gpu/drm/nouveau/nouveau_sched.c index 88217185e0f3..3b7ea5221226 100644 --- a/drivers/gpu/drm/nouveau/nouveau_sched.c +++ b/drivers/gpu/drm/nouveau/nouveau_sched.c @@ -375,14 +375,20 @@ nouveau_sched_run_job(struct drm_sched_job *sched_job) static enum drm_gpu_sched_stat nouveau_sched_timedout_job(struct drm_sched_job *sched_job) { + struct drm_gpu_scheduler *sched = sched_job->sched; struct nouveau_job *job = to_nouveau_job(sched_job); + enum drm_gpu_sched_stat stat = DRM_GPU_SCHED_STAT_NOMINAL; - NV_PRINTK(warn, job->cli, "Job timed out.\n"); + drm_sched_stop(sched, sched_job); if (job->ops->timeout) - return job->ops->timeout(job); + stat = job->ops->timeout(job); + else + NV_PRINTK(warn, job->cli, "Generic job timeout.\n"); + + drm_sched_start(sched, true); - return DRM_GPU_SCHED_STAT_ENODEV; + return stat; } static void diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/disp/uconn.c b/drivers/gpu/drm/nouveau/nvkm/engine/disp/uconn.c index 46b057fe1412..3249e5c1c893 100644 --- a/drivers/gpu/drm/nouveau/nvkm/engine/disp/uconn.c +++ b/drivers/gpu/drm/nouveau/nvkm/engine/disp/uconn.c @@ -62,6 +62,18 @@ nvkm_uconn_uevent_gpio(struct nvkm_object *object, u64 token, u32 bits) return object->client->event(token, &args, sizeof(args.v0)); } +static bool +nvkm_connector_is_dp_dms(u8 type) +{ + switch (type) { + case DCB_CONNECTOR_DMS59_DP0: + case DCB_CONNECTOR_DMS59_DP1: + return true; + default: + return false; + } +} + static int nvkm_uconn_uevent(struct nvkm_object *object, void *argv, u32 argc, struct nvkm_uevent *uevent) { @@ -101,7 +113,7 @@ nvkm_uconn_uevent(struct nvkm_object *object, void *argv, u32 argc, struct nvkm_ if (args->v0.types & NVIF_CONN_EVENT_V0_UNPLUG) bits |= NVKM_GPIO_LO; if (args->v0.types & NVIF_CONN_EVENT_V0_IRQ) { /* TODO: support DP IRQ on ANX9805 and remove this hack. */ - if (!outp->info.location) + if (!outp->info.location && !nvkm_connector_is_dp_dms(conn->info.type)) return -EINVAL; } diff --git a/drivers/gpu/drm/panel/panel-boe-tv101wum-nl6.c b/drivers/gpu/drm/panel/panel-boe-tv101wum-nl6.c index 5ac926281d2c..c9087f474cbc 100644 --- a/drivers/gpu/drm/panel/panel-boe-tv101wum-nl6.c +++ b/drivers/gpu/drm/panel/panel-boe-tv101wum-nl6.c @@ -1342,9 +1342,7 @@ static const struct panel_init_cmd starry_himax83102_j02_init_cmd[] = { _INIT_DCS_CMD(0xB1, 0x01, 0xBF, 0x11), _INIT_DCS_CMD(0xCB, 0x86), _INIT_DCS_CMD(0xD2, 0x3C, 0xFA), - _INIT_DCS_CMD(0xE9, 0xC5), - _INIT_DCS_CMD(0xD3, 0x00, 0x00, 0x00, 0x00, 0x80, 0x0C, 0x01), - _INIT_DCS_CMD(0xE9, 0x3F), + _INIT_DCS_CMD(0xD3, 0x00, 0x00, 0x44, 0x00, 0x00, 0x00, 0x00, 0x00, 0x80, 0x0C, 0x01), _INIT_DCS_CMD(0xE7, 0x02, 0x00, 0x28, 0x01, 0x7E, 0x0F, 0x7E, 0x10, 0xA0, 0x00, 0x00, 0x20, 0x40, 0x50, 0x40), _INIT_DCS_CMD(0xBD, 0x02), _INIT_DCS_CMD(0xD8, 0xFF, 0xFF, 0xBF, 0xFE, 0xAA, 0xA0, 0xFF, 0xFF, 0xBF, 0xFE, 0xAA, 0xA0), diff --git a/drivers/gpu/drm/panel/panel-edp.c b/drivers/gpu/drm/panel/panel-edp.c index feb665df35a1..95c8472d878a 100644 --- a/drivers/gpu/drm/panel/panel-edp.c +++ b/drivers/gpu/drm/panel/panel-edp.c @@ -976,32 +976,6 @@ static const struct panel_desc auo_b116xak01 = { }, }; -static const struct drm_display_mode auo_b116xw03_mode = { - .clock = 70589, - .hdisplay = 1366, - .hsync_start = 1366 + 40, - .hsync_end = 1366 + 40 + 40, - .htotal = 1366 + 40 + 40 + 32, - .vdisplay = 768, - .vsync_start = 768 + 10, - .vsync_end = 768 + 10 + 12, - .vtotal = 768 + 10 + 12 + 6, - .flags = DRM_MODE_FLAG_NVSYNC | DRM_MODE_FLAG_NHSYNC, -}; - -static const struct panel_desc auo_b116xw03 = { - .modes = &auo_b116xw03_mode, - .num_modes = 1, - .bpc = 6, - .size = { - .width = 256, - .height = 144, - }, - .delay = { - .enable = 400, - }, -}; - static const struct drm_display_mode auo_b133han05_mode = { .clock = 142600, .hdisplay = 1920, @@ -1725,9 +1699,6 @@ static const struct of_device_id platform_of_match[] = { }, { .compatible = "auo,b116xa01", .data = &auo_b116xak01, - }, { - .compatible = "auo,b116xw03", - .data = &auo_b116xw03, }, { .compatible = "auo,b133han05", .data = &auo_b133han05, diff --git a/drivers/gpu/drm/panel/panel-simple.c b/drivers/gpu/drm/panel/panel-simple.c index 95959dcc6e0e..dd7928d9570f 100644 --- a/drivers/gpu/drm/panel/panel-simple.c +++ b/drivers/gpu/drm/panel/panel-simple.c @@ -919,6 +919,38 @@ static const struct panel_desc auo_b101xtn01 = { }, }; +static const struct drm_display_mode auo_b116xw03_mode = { + .clock = 70589, + .hdisplay = 1366, + .hsync_start = 1366 + 40, + .hsync_end = 1366 + 40 + 40, + .htotal = 1366 + 40 + 40 + 32, + .vdisplay = 768, + .vsync_start = 768 + 10, + .vsync_end = 768 + 10 + 12, + .vtotal = 768 + 10 + 12 + 6, + .flags = DRM_MODE_FLAG_NVSYNC | DRM_MODE_FLAG_NHSYNC, +}; + +static const struct panel_desc auo_b116xw03 = { + .modes = &auo_b116xw03_mode, + .num_modes = 1, + .bpc = 6, + .size = { + .width = 256, + .height = 144, + }, + .delay = { + .prepare = 1, + .enable = 200, + .disable = 200, + .unprepare = 500, + }, + .bus_format = MEDIA_BUS_FMT_RGB666_1X7X3_SPWG, + .bus_flags = DRM_BUS_FLAG_DE_HIGH, + .connector_type = DRM_MODE_CONNECTOR_LVDS, +}; + static const struct display_timing auo_g070vvn01_timings = { .pixelclock = { 33300000, 34209000, 45000000 }, .hactive = { 800, 800, 800 }, @@ -4102,6 +4134,9 @@ static const struct of_device_id platform_of_match[] = { }, { .compatible = "auo,b101xtn01", .data = &auo_b101xtn01, + }, { + .compatible = "auo,b116xw03", + .data = &auo_b116xw03, }, { .compatible = "auo,g070vvn01", .data = &auo_g070vvn01, diff --git a/drivers/gpu/drm/radeon/radeon_sa.c b/drivers/gpu/drm/radeon/radeon_sa.c index c87a57c9c592..22dd8b445685 100644 --- a/drivers/gpu/drm/radeon/radeon_sa.c +++ b/drivers/gpu/drm/radeon/radeon_sa.c @@ -123,7 +123,7 @@ int radeon_sa_bo_new(struct radeon_sa_manager *sa_manager, unsigned int size, unsigned int align) { struct drm_suballoc *sa = drm_suballoc_new(&sa_manager->base, size, - GFP_KERNEL, true, align); + GFP_KERNEL, false, align); if (IS_ERR(sa)) { *sa_bo = NULL; diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c index 506371c42745..5a3a622fc672 100644 --- a/drivers/gpu/drm/scheduler/sched_main.c +++ b/drivers/gpu/drm/scheduler/sched_main.c @@ -929,7 +929,7 @@ drm_sched_get_cleanup_job(struct drm_gpu_scheduler *sched) if (next) { next->s_fence->scheduled.timestamp = - job->s_fence->finished.timestamp; + dma_fence_timestamp(&job->s_fence->finished); /* start TO timer for next job */ drm_sched_start_timeout(sched); } diff --git a/drivers/gpu/drm/tests/drm_kunit_helpers.c b/drivers/gpu/drm/tests/drm_kunit_helpers.c index c1dfbfcaa000..bccb33b900f3 100644 --- a/drivers/gpu/drm/tests/drm_kunit_helpers.c +++ b/drivers/gpu/drm/tests/drm_kunit_helpers.c @@ -118,7 +118,7 @@ void drm_kunit_helper_free_device(struct kunit *test, struct device *dev) kunit_release_action(test, kunit_action_platform_driver_unregister, - pdev); + &fake_platform_driver); } EXPORT_SYMBOL_GPL(drm_kunit_helper_free_device); diff --git a/drivers/gpu/drm/tests/drm_mm_test.c b/drivers/gpu/drm/tests/drm_mm_test.c index 186b28dc7038..05d5e7af6d25 100644 --- a/drivers/gpu/drm/tests/drm_mm_test.c +++ b/drivers/gpu/drm/tests/drm_mm_test.c @@ -939,7 +939,7 @@ static void drm_test_mm_insert_range(struct kunit *test) KUNIT_ASSERT_FALSE(test, __drm_test_mm_insert_range(test, count, size, 0, max - 1)); KUNIT_ASSERT_FALSE(test, __drm_test_mm_insert_range(test, count, size, 0, max / 2)); KUNIT_ASSERT_FALSE(test, __drm_test_mm_insert_range(test, count, size, - max / 2, max / 2)); + max / 2, max)); KUNIT_ASSERT_FALSE(test, __drm_test_mm_insert_range(test, count, size, max / 4 + 1, 3 * max / 4 - 1)); diff --git a/drivers/gpu/drm/tiny/gm12u320.c b/drivers/gpu/drm/tiny/gm12u320.c index c5bb683e440c..0187539ff5ea 100644 --- a/drivers/gpu/drm/tiny/gm12u320.c +++ b/drivers/gpu/drm/tiny/gm12u320.c @@ -70,10 +70,10 @@ MODULE_PARM_DESC(eco_mode, "Turn on Eco mode (less bright, more silent)"); #define READ_STATUS_SIZE 13 #define MISC_VALUE_SIZE 4 -#define CMD_TIMEOUT msecs_to_jiffies(200) -#define DATA_TIMEOUT msecs_to_jiffies(1000) -#define IDLE_TIMEOUT msecs_to_jiffies(2000) -#define FIRST_FRAME_TIMEOUT msecs_to_jiffies(2000) +#define CMD_TIMEOUT 200 +#define DATA_TIMEOUT 1000 +#define IDLE_TIMEOUT 2000 +#define FIRST_FRAME_TIMEOUT 2000 #define MISC_REQ_GET_SET_ECO_A 0xff #define MISC_REQ_GET_SET_ECO_B 0x35 @@ -389,7 +389,7 @@ static void gm12u320_fb_update_work(struct work_struct *work) * switches back to showing its logo. */ queue_delayed_work(system_long_wq, &gm12u320->fb_update.work, - IDLE_TIMEOUT); + msecs_to_jiffies(IDLE_TIMEOUT)); return; err: diff --git a/drivers/gpu/drm/tiny/simpledrm.c b/drivers/gpu/drm/tiny/simpledrm.c index ff86ba1ae1b8..8ea120eb8674 100644 --- a/drivers/gpu/drm/tiny/simpledrm.c +++ b/drivers/gpu/drm/tiny/simpledrm.c @@ -745,7 +745,7 @@ static struct simpledrm_device *simpledrm_device_create(struct drm_driver *drv, ret = devm_aperture_acquire_from_firmware(dev, res->start, resource_size(res)); if (ret) { - drm_err(dev, "could not acquire memory range %pr: %d\n", &res, ret); + drm_err(dev, "could not acquire memory range %pr: %d\n", res, ret); return ERR_PTR(ret); } diff --git a/drivers/gpu/drm/ttm/ttm_device.c b/drivers/gpu/drm/ttm/ttm_device.c index 7726a72befc5..d48b39132b32 100644 --- a/drivers/gpu/drm/ttm/ttm_device.c +++ b/drivers/gpu/drm/ttm/ttm_device.c @@ -232,10 +232,6 @@ void ttm_device_fini(struct ttm_device *bdev) struct ttm_resource_manager *man; unsigned i; - man = ttm_manager_type(bdev, TTM_PL_SYSTEM); - ttm_resource_manager_set_used(man, false); - ttm_set_driver_manager(bdev, TTM_PL_SYSTEM, NULL); - mutex_lock(&ttm_global_mutex); list_del(&bdev->device_list); mutex_unlock(&ttm_global_mutex); @@ -243,6 +239,10 @@ void ttm_device_fini(struct ttm_device *bdev) drain_workqueue(bdev->wq); destroy_workqueue(bdev->wq); + man = ttm_manager_type(bdev, TTM_PL_SYSTEM); + ttm_resource_manager_set_used(man, false); + ttm_set_driver_manager(bdev, TTM_PL_SYSTEM, NULL); + spin_lock(&bdev->lru_lock); for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i) if (list_empty(&man->lru[0])) diff --git a/drivers/gpu/drm/virtio/virtgpu_submit.c b/drivers/gpu/drm/virtio/virtgpu_submit.c index 3c00135ead45..5c514946bbad 100644 --- a/drivers/gpu/drm/virtio/virtgpu_submit.c +++ b/drivers/gpu/drm/virtio/virtgpu_submit.c @@ -361,7 +361,6 @@ static void virtio_gpu_complete_submit(struct virtio_gpu_submit *submit) submit->buf = NULL; submit->buflist = NULL; submit->sync_file = NULL; - submit->out_fence = NULL; submit->out_fence_fd = -1; } diff --git a/drivers/gpu/drm/vkms/vkms_composer.c b/drivers/gpu/drm/vkms/vkms_composer.c index d5d4f642d367..3c99fb8b54e2 100644 --- a/drivers/gpu/drm/vkms/vkms_composer.c +++ b/drivers/gpu/drm/vkms/vkms_composer.c @@ -408,15 +408,10 @@ void vkms_set_composer(struct vkms_output *out, bool enabled) if (enabled) drm_crtc_vblank_get(&out->crtc); - mutex_lock(&out->enabled_lock); + spin_lock_irq(&out->lock); old_enabled = out->composer_enabled; out->composer_enabled = enabled; - - /* the composition wasn't enabled, so unlock the lock to make sure the lock - * will be balanced even if we have a failed commit - */ - if (!out->composer_enabled) - mutex_unlock(&out->enabled_lock); + spin_unlock_irq(&out->lock); if (old_enabled) drm_crtc_vblank_put(&out->crtc); diff --git a/drivers/gpu/drm/vkms/vkms_crtc.c b/drivers/gpu/drm/vkms/vkms_crtc.c index 3c5ebf106b66..61e500b8c9da 100644 --- a/drivers/gpu/drm/vkms/vkms_crtc.c +++ b/drivers/gpu/drm/vkms/vkms_crtc.c @@ -16,7 +16,7 @@ static enum hrtimer_restart vkms_vblank_simulate(struct hrtimer *timer) struct drm_crtc *crtc = &output->crtc; struct vkms_crtc_state *state; u64 ret_overrun; - bool ret, fence_cookie, composer_enabled; + bool ret, fence_cookie; fence_cookie = dma_fence_begin_signalling(); @@ -25,15 +25,15 @@ static enum hrtimer_restart vkms_vblank_simulate(struct hrtimer *timer) if (ret_overrun != 1) pr_warn("%s: vblank timer overrun\n", __func__); + spin_lock(&output->lock); ret = drm_crtc_handle_vblank(crtc); if (!ret) DRM_ERROR("vkms failure on handling vblank"); state = output->composer_state; - composer_enabled = output->composer_enabled; - mutex_unlock(&output->enabled_lock); + spin_unlock(&output->lock); - if (state && composer_enabled) { + if (state && output->composer_enabled) { u64 frame = drm_crtc_accurate_vblank_count(crtc); /* update frame_start only if a queued vkms_composer_worker() @@ -295,7 +295,6 @@ int vkms_crtc_init(struct drm_device *dev, struct drm_crtc *crtc, spin_lock_init(&vkms_out->lock); spin_lock_init(&vkms_out->composer_lock); - mutex_init(&vkms_out->enabled_lock); vkms_out->composer_workq = alloc_ordered_workqueue("vkms_composer", 0); if (!vkms_out->composer_workq) diff --git a/drivers/gpu/drm/vkms/vkms_drv.h b/drivers/gpu/drm/vkms/vkms_drv.h index c7ae6c2ba1df..8f5710debb1e 100644 --- a/drivers/gpu/drm/vkms/vkms_drv.h +++ b/drivers/gpu/drm/vkms/vkms_drv.h @@ -108,10 +108,8 @@ struct vkms_output { struct workqueue_struct *composer_workq; /* protects concurrent access to composer */ spinlock_t lock; - /* guarantees that if the composer is enabled, a job will be queued */ - struct mutex enabled_lock; - /* protected by @enabled_lock */ + /* protected by @lock */ bool composer_enabled; struct vkms_crtc_state *composer_state; diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_bo.c b/drivers/gpu/drm/vmwgfx/vmwgfx_bo.c index c43853597776..2bfac3aad7b7 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_bo.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_bo.c @@ -34,6 +34,8 @@ static void vmw_bo_release(struct vmw_bo *vbo) { + WARN_ON(vbo->tbo.base.funcs && + kref_read(&vbo->tbo.base.refcount) != 0); vmw_bo_unmap(vbo); drm_gem_object_release(&vbo->tbo.base); } @@ -497,7 +499,7 @@ static int vmw_user_bo_synccpu_release(struct drm_file *filp, if (!(flags & drm_vmw_synccpu_allow_cs)) { atomic_dec(&vmw_bo->cpu_writers); } - vmw_user_bo_unref(vmw_bo); + vmw_user_bo_unref(&vmw_bo); } return ret; @@ -539,7 +541,7 @@ int vmw_user_bo_synccpu_ioctl(struct drm_device *dev, void *data, return ret; ret = vmw_user_bo_synccpu_grab(vbo, arg->flags); - vmw_user_bo_unref(vbo); + vmw_user_bo_unref(&vbo); if (unlikely(ret != 0)) { if (ret == -ERESTARTSYS || ret == -EBUSY) return -EBUSY; @@ -612,7 +614,6 @@ int vmw_user_bo_lookup(struct drm_file *filp, } *out = to_vmw_bo(gobj); - ttm_bo_get(&(*out)->tbo); return 0; } diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_bo.h b/drivers/gpu/drm/vmwgfx/vmwgfx_bo.h index 1d433fceed3d..0d496dc9c6af 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_bo.h +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_bo.h @@ -195,12 +195,19 @@ static inline struct vmw_bo *vmw_bo_reference(struct vmw_bo *buf) return buf; } -static inline void vmw_user_bo_unref(struct vmw_bo *vbo) +static inline struct vmw_bo *vmw_user_bo_ref(struct vmw_bo *vbo) { - if (vbo) { - ttm_bo_put(&vbo->tbo); - drm_gem_object_put(&vbo->tbo.base); - } + drm_gem_object_get(&vbo->tbo.base); + return vbo; +} + +static inline void vmw_user_bo_unref(struct vmw_bo **buf) +{ + struct vmw_bo *tmp_buf = *buf; + + *buf = NULL; + if (tmp_buf) + drm_gem_object_put(&tmp_buf->tbo.base); } static inline struct vmw_bo *to_vmw_bo(struct drm_gem_object *gobj) diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_cotable.c b/drivers/gpu/drm/vmwgfx/vmwgfx_cotable.c index c0b24d1cacbf..a7c07692262b 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_cotable.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_cotable.c @@ -432,7 +432,7 @@ static int vmw_cotable_resize(struct vmw_resource *res, size_t new_size) * for the new COTable. Initially pin the buffer object to make sure * we can use tryreserve without failure. */ - ret = vmw_bo_create(dev_priv, &bo_params, &buf); + ret = vmw_gem_object_create(dev_priv, &bo_params, &buf); if (ret) { DRM_ERROR("Failed initializing new cotable MOB.\n"); goto out_done; @@ -502,7 +502,7 @@ static int vmw_cotable_resize(struct vmw_resource *res, size_t new_size) vmw_resource_mob_attach(res); /* Let go of the old mob. */ - vmw_bo_unreference(&old_buf); + vmw_user_bo_unref(&old_buf); res->id = vcotbl->type; ret = dma_resv_reserve_fences(bo->base.resv, 1); @@ -521,7 +521,7 @@ out_map_new: out_wait: ttm_bo_unpin(bo); ttm_bo_unreserve(bo); - vmw_bo_unreference(&buf); + vmw_user_bo_unref(&buf); out_done: MKS_STAT_TIME_POP(MKSSTAT_KERN_COTABLE_RESIZE); diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_drv.h b/drivers/gpu/drm/vmwgfx/vmwgfx_drv.h index 58bfdf203eca..3cd5090dedfc 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_drv.h +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_drv.h @@ -853,6 +853,10 @@ static inline bool vmw_resource_mob_attached(const struct vmw_resource *res) /** * GEM related functionality - vmwgfx_gem.c */ +struct vmw_bo_params; +int vmw_gem_object_create(struct vmw_private *vmw, + struct vmw_bo_params *params, + struct vmw_bo **p_vbo); extern int vmw_gem_object_create_with_handle(struct vmw_private *dev_priv, struct drm_file *filp, uint32_t size, diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_execbuf.c b/drivers/gpu/drm/vmwgfx/vmwgfx_execbuf.c index 98e0723ca6f5..36987ef3fc30 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_execbuf.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_execbuf.c @@ -1151,7 +1151,7 @@ static int vmw_translate_mob_ptr(struct vmw_private *dev_priv, SVGAMobId *id, struct vmw_bo **vmw_bo_p) { - struct vmw_bo *vmw_bo; + struct vmw_bo *vmw_bo, *tmp_bo; uint32_t handle = *id; struct vmw_relocation *reloc; int ret; @@ -1164,7 +1164,8 @@ static int vmw_translate_mob_ptr(struct vmw_private *dev_priv, } vmw_bo_placement_set(vmw_bo, VMW_BO_DOMAIN_MOB, VMW_BO_DOMAIN_MOB); ret = vmw_validation_add_bo(sw_context->ctx, vmw_bo); - vmw_user_bo_unref(vmw_bo); + tmp_bo = vmw_bo; + vmw_user_bo_unref(&tmp_bo); if (unlikely(ret != 0)) return ret; @@ -1206,7 +1207,7 @@ static int vmw_translate_guest_ptr(struct vmw_private *dev_priv, SVGAGuestPtr *ptr, struct vmw_bo **vmw_bo_p) { - struct vmw_bo *vmw_bo; + struct vmw_bo *vmw_bo, *tmp_bo; uint32_t handle = ptr->gmrId; struct vmw_relocation *reloc; int ret; @@ -1220,7 +1221,8 @@ static int vmw_translate_guest_ptr(struct vmw_private *dev_priv, vmw_bo_placement_set(vmw_bo, VMW_BO_DOMAIN_GMR | VMW_BO_DOMAIN_VRAM, VMW_BO_DOMAIN_GMR | VMW_BO_DOMAIN_VRAM); ret = vmw_validation_add_bo(sw_context->ctx, vmw_bo); - vmw_user_bo_unref(vmw_bo); + tmp_bo = vmw_bo; + vmw_user_bo_unref(&tmp_bo); if (unlikely(ret != 0)) return ret; @@ -1619,7 +1621,7 @@ static int vmw_cmd_tex_state(struct vmw_private *dev_priv, { VMW_DECLARE_CMD_VAR(*cmd, SVGA3dCmdSetTextureState); SVGA3dTextureState *last_state = (SVGA3dTextureState *) - ((unsigned long) header + header->size + sizeof(header)); + ((unsigned long) header + header->size + sizeof(*header)); SVGA3dTextureState *cur_state = (SVGA3dTextureState *) ((unsigned long) header + sizeof(*cmd)); struct vmw_resource *ctx; diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_gem.c b/drivers/gpu/drm/vmwgfx/vmwgfx_gem.c index c0da89e16e6f..8b1eb0061610 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_gem.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_gem.c @@ -111,6 +111,20 @@ static const struct drm_gem_object_funcs vmw_gem_object_funcs = { .vm_ops = &vmw_vm_ops, }; +int vmw_gem_object_create(struct vmw_private *vmw, + struct vmw_bo_params *params, + struct vmw_bo **p_vbo) +{ + int ret = vmw_bo_create(vmw, params, p_vbo); + + if (ret != 0) + goto out_no_bo; + + (*p_vbo)->tbo.base.funcs = &vmw_gem_object_funcs; +out_no_bo: + return ret; +} + int vmw_gem_object_create_with_handle(struct vmw_private *dev_priv, struct drm_file *filp, uint32_t size, @@ -126,12 +140,10 @@ int vmw_gem_object_create_with_handle(struct vmw_private *dev_priv, .pin = false }; - ret = vmw_bo_create(dev_priv, ¶ms, p_vbo); + ret = vmw_gem_object_create(dev_priv, ¶ms, p_vbo); if (ret != 0) goto out_no_bo; - (*p_vbo)->tbo.base.funcs = &vmw_gem_object_funcs; - ret = drm_gem_handle_create(filp, &(*p_vbo)->tbo.base, handle); out_no_bo: return ret; diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_kms.c b/drivers/gpu/drm/vmwgfx/vmwgfx_kms.c index 1489ad73c103..818b7f109f53 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_kms.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_kms.c @@ -1471,8 +1471,8 @@ static int vmw_create_bo_proxy(struct drm_device *dev, /* Reserve and switch the backing mob. */ mutex_lock(&res->dev_priv->cmdbuf_mutex); (void) vmw_resource_reserve(res, false, true); - vmw_bo_unreference(&res->guest_memory_bo); - res->guest_memory_bo = vmw_bo_reference(bo_mob); + vmw_user_bo_unref(&res->guest_memory_bo); + res->guest_memory_bo = vmw_user_bo_ref(bo_mob); res->guest_memory_offset = 0; vmw_resource_unreserve(res, false, false, false, NULL, 0); mutex_unlock(&res->dev_priv->cmdbuf_mutex); @@ -1666,7 +1666,7 @@ static struct drm_framebuffer *vmw_kms_fb_create(struct drm_device *dev, err_out: /* vmw_user_lookup_handle takes one ref so does new_fb */ if (bo) - vmw_user_bo_unref(bo); + vmw_user_bo_unref(&bo); if (surface) vmw_surface_unreference(&surface); diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_overlay.c b/drivers/gpu/drm/vmwgfx/vmwgfx_overlay.c index fb85f244c3d0..c45b4724e414 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_overlay.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_overlay.c @@ -451,7 +451,7 @@ int vmw_overlay_ioctl(struct drm_device *dev, void *data, ret = vmw_overlay_update_stream(dev_priv, buf, arg, true); - vmw_user_bo_unref(buf); + vmw_user_bo_unref(&buf); out_unlock: mutex_unlock(&overlay->mutex); diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_resource.c b/drivers/gpu/drm/vmwgfx/vmwgfx_resource.c index 71eeabf001c8..ca300c7427d2 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_resource.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_resource.c @@ -141,7 +141,7 @@ static void vmw_resource_release(struct kref *kref) if (res->coherent) vmw_bo_dirty_release(res->guest_memory_bo); ttm_bo_unreserve(bo); - vmw_bo_unreference(&res->guest_memory_bo); + vmw_user_bo_unref(&res->guest_memory_bo); } if (likely(res->hw_destroy != NULL)) { @@ -338,7 +338,7 @@ static int vmw_resource_buf_alloc(struct vmw_resource *res, return 0; } - ret = vmw_bo_create(res->dev_priv, &bo_params, &gbo); + ret = vmw_gem_object_create(res->dev_priv, &bo_params, &gbo); if (unlikely(ret != 0)) goto out_no_bo; @@ -457,11 +457,11 @@ void vmw_resource_unreserve(struct vmw_resource *res, vmw_resource_mob_detach(res); if (res->coherent) vmw_bo_dirty_release(res->guest_memory_bo); - vmw_bo_unreference(&res->guest_memory_bo); + vmw_user_bo_unref(&res->guest_memory_bo); } if (new_guest_memory_bo) { - res->guest_memory_bo = vmw_bo_reference(new_guest_memory_bo); + res->guest_memory_bo = vmw_user_bo_ref(new_guest_memory_bo); /* * The validation code should already have added a @@ -551,7 +551,7 @@ out_no_reserve: ttm_bo_put(val_buf->bo); val_buf->bo = NULL; if (guest_memory_dirty) - vmw_bo_unreference(&res->guest_memory_bo); + vmw_user_bo_unref(&res->guest_memory_bo); return ret; } @@ -727,7 +727,7 @@ int vmw_resource_validate(struct vmw_resource *res, bool intr, goto out_no_validate; else if (!res->func->needs_guest_memory && res->guest_memory_bo) { WARN_ON_ONCE(vmw_resource_mob_attached(res)); - vmw_bo_unreference(&res->guest_memory_bo); + vmw_user_bo_unref(&res->guest_memory_bo); } return 0; diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_shader.c b/drivers/gpu/drm/vmwgfx/vmwgfx_shader.c index 1e81ff2422cf..a01ca3226d0a 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_shader.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_shader.c @@ -180,7 +180,7 @@ static int vmw_gb_shader_init(struct vmw_private *dev_priv, res->guest_memory_size = size; if (byte_code) { - res->guest_memory_bo = vmw_bo_reference(byte_code); + res->guest_memory_bo = vmw_user_bo_ref(byte_code); res->guest_memory_offset = offset; } shader->size = size; @@ -809,7 +809,7 @@ static int vmw_shader_define(struct drm_device *dev, struct drm_file *file_priv, shader_type, num_input_sig, num_output_sig, tfile, shader_handle); out_bad_arg: - vmw_user_bo_unref(buffer); + vmw_user_bo_unref(&buffer); return ret; } diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_surface.c b/drivers/gpu/drm/vmwgfx/vmwgfx_surface.c index 5db403ee8261..3829be282ff0 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_surface.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_surface.c @@ -686,9 +686,6 @@ static void vmw_user_surface_base_release(struct ttm_base_object **p_base) container_of(base, struct vmw_user_surface, prime.base); struct vmw_resource *res = &user_srf->srf.res; - if (res->guest_memory_bo) - drm_gem_object_put(&res->guest_memory_bo->tbo.base); - *p_base = NULL; vmw_resource_unreference(&res); } @@ -855,23 +852,21 @@ int vmw_surface_define_ioctl(struct drm_device *dev, void *data, * expect a backup buffer to be present. */ if (dev_priv->has_mob && req->shareable) { - uint32_t backup_handle; - - ret = vmw_gem_object_create_with_handle(dev_priv, - file_priv, - res->guest_memory_size, - &backup_handle, - &res->guest_memory_bo); + struct vmw_bo_params params = { + .domain = VMW_BO_DOMAIN_SYS, + .busy_domain = VMW_BO_DOMAIN_SYS, + .bo_type = ttm_bo_type_device, + .size = res->guest_memory_size, + .pin = false + }; + + ret = vmw_gem_object_create(dev_priv, + ¶ms, + &res->guest_memory_bo); if (unlikely(ret != 0)) { vmw_resource_unreference(&res); goto out_unlock; } - vmw_bo_reference(res->guest_memory_bo); - /* - * We don't expose the handle to the userspace and surface - * already holds a gem reference - */ - drm_gem_handle_delete(file_priv, backup_handle); } tmp = vmw_resource_reference(&srf->res); @@ -1512,7 +1507,7 @@ vmw_gb_surface_define_internal(struct drm_device *dev, if (ret == 0) { if (res->guest_memory_bo->tbo.base.size < res->guest_memory_size) { VMW_DEBUG_USER("Surface backup buffer too small.\n"); - vmw_bo_unreference(&res->guest_memory_bo); + vmw_user_bo_unref(&res->guest_memory_bo); ret = -EINVAL; goto out_unlock; } else { @@ -1526,8 +1521,6 @@ vmw_gb_surface_define_internal(struct drm_device *dev, res->guest_memory_size, &backup_handle, &res->guest_memory_bo); - if (ret == 0) - vmw_bo_reference(res->guest_memory_bo); } if (unlikely(ret != 0)) { diff --git a/drivers/hid/Kconfig b/drivers/hid/Kconfig index 0cea301cc9a9..790aa908e2a7 100644 --- a/drivers/hid/Kconfig +++ b/drivers/hid/Kconfig @@ -799,6 +799,8 @@ config HID_NVIDIA_SHIELD tristate "NVIDIA SHIELD devices" depends on USB_HID depends on BT_HIDP + depends on LEDS_CLASS + select POWER_SUPPLY help Support for NVIDIA SHIELD accessories. diff --git a/drivers/hid/hid-holtek-kbd.c b/drivers/hid/hid-holtek-kbd.c index 403506b9697e..b346d68a06f5 100644 --- a/drivers/hid/hid-holtek-kbd.c +++ b/drivers/hid/hid-holtek-kbd.c @@ -130,6 +130,10 @@ static int holtek_kbd_input_event(struct input_dev *dev, unsigned int type, return -ENODEV; boot_hid = usb_get_intfdata(boot_interface); + if (list_empty(&boot_hid->inputs)) { + hid_err(hid, "no inputs found\n"); + return -ENODEV; + } boot_hid_input = list_first_entry(&boot_hid->inputs, struct hid_input, list); diff --git a/drivers/hid/hid-ids.h b/drivers/hid/hid-ids.h index 7e499992a793..e4d2dfd5d253 100644 --- a/drivers/hid/hid-ids.h +++ b/drivers/hid/hid-ids.h @@ -425,6 +425,7 @@ #define I2C_DEVICE_ID_HP_SPECTRE_X360_13T_AW100 0x29F5 #define I2C_DEVICE_ID_HP_SPECTRE_X360_14T_EA100_V1 0x2BED #define I2C_DEVICE_ID_HP_SPECTRE_X360_14T_EA100_V2 0x2BEE +#define I2C_DEVICE_ID_HP_ENVY_X360_15_EU0556NG 0x2D02 #define USB_VENDOR_ID_ELECOM 0x056e #define USB_DEVICE_ID_ELECOM_BM084 0x0061 diff --git a/drivers/hid/hid-input.c b/drivers/hid/hid-input.c index 0235cc1690a1..c8b20d44b147 100644 --- a/drivers/hid/hid-input.c +++ b/drivers/hid/hid-input.c @@ -409,6 +409,8 @@ static const struct hid_device_id hid_battery_quirks[] = { HID_BATTERY_QUIRK_IGNORE }, { HID_I2C_DEVICE(USB_VENDOR_ID_ELAN, I2C_DEVICE_ID_HP_SPECTRE_X360_14T_EA100_V2), HID_BATTERY_QUIRK_IGNORE }, + { HID_I2C_DEVICE(USB_VENDOR_ID_ELAN, I2C_DEVICE_ID_HP_ENVY_X360_15_EU0556NG), + HID_BATTERY_QUIRK_IGNORE }, {} }; diff --git a/drivers/hid/hid-logitech-hidpp.c b/drivers/hid/hid-logitech-hidpp.c index 05f5b5f588a2..a209d51bd247 100644 --- a/drivers/hid/hid-logitech-hidpp.c +++ b/drivers/hid/hid-logitech-hidpp.c @@ -4515,7 +4515,8 @@ static int hidpp_probe(struct hid_device *hdev, const struct hid_device_id *id) goto hid_hw_init_fail; } - hidpp_connect_event(hidpp); + schedule_work(&hidpp->work); + flush_work(&hidpp->work); if (will_restart) { /* Reset the HID node state */ @@ -4677,6 +4678,8 @@ static const struct hid_device_id hidpp_devices[] = { HID_BLUETOOTH_DEVICE(USB_VENDOR_ID_LOGITECH, 0xb008) }, { /* MX Master mouse over Bluetooth */ HID_BLUETOOTH_DEVICE(USB_VENDOR_ID_LOGITECH, 0xb012) }, + { /* M720 Triathlon mouse over Bluetooth */ + HID_BLUETOOTH_DEVICE(USB_VENDOR_ID_LOGITECH, 0xb015) }, { /* MX Ergo trackball over Bluetooth */ HID_BLUETOOTH_DEVICE(USB_VENDOR_ID_LOGITECH, 0xb01d) }, { HID_BLUETOOTH_DEVICE(USB_VENDOR_ID_LOGITECH, 0xb01e) }, diff --git a/drivers/hid/hid-multitouch.c b/drivers/hid/hid-multitouch.c index 521b2ffb4244..8db4ae05febc 100644 --- a/drivers/hid/hid-multitouch.c +++ b/drivers/hid/hid-multitouch.c @@ -2144,6 +2144,10 @@ static const struct hid_device_id mt_devices[] = { USB_DEVICE_ID_MTP_STM)}, /* Synaptics devices */ + { .driver_data = MT_CLS_WIN_8_FORCE_MULTI_INPUT, + HID_DEVICE(BUS_I2C, HID_GROUP_MULTITOUCH_WIN_8, + USB_VENDOR_ID_SYNAPTICS, 0xcd7e) }, + { .driver_data = MT_CLS_WIN_8_FORCE_MULTI_INPUT, HID_DEVICE(BUS_I2C, HID_GROUP_MULTITOUCH_WIN_8, USB_VENDOR_ID_SYNAPTICS, 0xce08) }, diff --git a/drivers/hid/hid-nintendo.c b/drivers/hid/hid-nintendo.c index 250f5d2f888a..10468f727e5b 100644 --- a/drivers/hid/hid-nintendo.c +++ b/drivers/hid/hid-nintendo.c @@ -2088,7 +2088,9 @@ static int joycon_read_info(struct joycon_ctlr *ctlr) struct joycon_input_report *report; req.subcmd_id = JC_SUBCMD_REQ_DEV_INFO; + mutex_lock(&ctlr->output_mutex); ret = joycon_send_subcmd(ctlr, &req, 0, HZ); + mutex_unlock(&ctlr->output_mutex); if (ret) { hid_err(ctlr->hdev, "Failed to get joycon info; ret=%d\n", ret); return ret; @@ -2117,6 +2119,85 @@ static int joycon_read_info(struct joycon_ctlr *ctlr) return 0; } +static int joycon_init(struct hid_device *hdev) +{ + struct joycon_ctlr *ctlr = hid_get_drvdata(hdev); + int ret = 0; + + mutex_lock(&ctlr->output_mutex); + /* if handshake command fails, assume ble pro controller */ + if ((jc_type_is_procon(ctlr) || jc_type_is_chrggrip(ctlr)) && + !joycon_send_usb(ctlr, JC_USB_CMD_HANDSHAKE, HZ)) { + hid_dbg(hdev, "detected USB controller\n"); + /* set baudrate for improved latency */ + ret = joycon_send_usb(ctlr, JC_USB_CMD_BAUDRATE_3M, HZ); + if (ret) { + hid_err(hdev, "Failed to set baudrate; ret=%d\n", ret); + goto out_unlock; + } + /* handshake */ + ret = joycon_send_usb(ctlr, JC_USB_CMD_HANDSHAKE, HZ); + if (ret) { + hid_err(hdev, "Failed handshake; ret=%d\n", ret); + goto out_unlock; + } + /* + * Set no timeout (to keep controller in USB mode). + * This doesn't send a response, so ignore the timeout. + */ + joycon_send_usb(ctlr, JC_USB_CMD_NO_TIMEOUT, HZ/10); + } else if (jc_type_is_chrggrip(ctlr)) { + hid_err(hdev, "Failed charging grip handshake\n"); + ret = -ETIMEDOUT; + goto out_unlock; + } + + /* get controller calibration data, and parse it */ + ret = joycon_request_calibration(ctlr); + if (ret) { + /* + * We can function with default calibration, but it may be + * inaccurate. Provide a warning, and continue on. + */ + hid_warn(hdev, "Analog stick positions may be inaccurate\n"); + } + + /* get IMU calibration data, and parse it */ + ret = joycon_request_imu_calibration(ctlr); + if (ret) { + /* + * We can function with default calibration, but it may be + * inaccurate. Provide a warning, and continue on. + */ + hid_warn(hdev, "Unable to read IMU calibration data\n"); + } + + /* Set the reporting mode to 0x30, which is the full report mode */ + ret = joycon_set_report_mode(ctlr); + if (ret) { + hid_err(hdev, "Failed to set report mode; ret=%d\n", ret); + goto out_unlock; + } + + /* Enable rumble */ + ret = joycon_enable_rumble(ctlr); + if (ret) { + hid_err(hdev, "Failed to enable rumble; ret=%d\n", ret); + goto out_unlock; + } + + /* Enable the IMU */ + ret = joycon_enable_imu(ctlr); + if (ret) { + hid_err(hdev, "Failed to enable the IMU; ret=%d\n", ret); + goto out_unlock; + } + +out_unlock: + mutex_unlock(&ctlr->output_mutex); + return ret; +} + /* Common handler for parsing inputs */ static int joycon_ctlr_read_handler(struct joycon_ctlr *ctlr, u8 *data, int size) @@ -2248,85 +2329,19 @@ static int nintendo_hid_probe(struct hid_device *hdev, hid_device_io_start(hdev); - /* Initialize the controller */ - mutex_lock(&ctlr->output_mutex); - /* if handshake command fails, assume ble pro controller */ - if ((jc_type_is_procon(ctlr) || jc_type_is_chrggrip(ctlr)) && - !joycon_send_usb(ctlr, JC_USB_CMD_HANDSHAKE, HZ)) { - hid_dbg(hdev, "detected USB controller\n"); - /* set baudrate for improved latency */ - ret = joycon_send_usb(ctlr, JC_USB_CMD_BAUDRATE_3M, HZ); - if (ret) { - hid_err(hdev, "Failed to set baudrate; ret=%d\n", ret); - goto err_mutex; - } - /* handshake */ - ret = joycon_send_usb(ctlr, JC_USB_CMD_HANDSHAKE, HZ); - if (ret) { - hid_err(hdev, "Failed handshake; ret=%d\n", ret); - goto err_mutex; - } - /* - * Set no timeout (to keep controller in USB mode). - * This doesn't send a response, so ignore the timeout. - */ - joycon_send_usb(ctlr, JC_USB_CMD_NO_TIMEOUT, HZ/10); - } else if (jc_type_is_chrggrip(ctlr)) { - hid_err(hdev, "Failed charging grip handshake\n"); - ret = -ETIMEDOUT; - goto err_mutex; - } - - /* get controller calibration data, and parse it */ - ret = joycon_request_calibration(ctlr); + ret = joycon_init(hdev); if (ret) { - /* - * We can function with default calibration, but it may be - * inaccurate. Provide a warning, and continue on. - */ - hid_warn(hdev, "Analog stick positions may be inaccurate\n"); - } - - /* get IMU calibration data, and parse it */ - ret = joycon_request_imu_calibration(ctlr); - if (ret) { - /* - * We can function with default calibration, but it may be - * inaccurate. Provide a warning, and continue on. - */ - hid_warn(hdev, "Unable to read IMU calibration data\n"); - } - - /* Set the reporting mode to 0x30, which is the full report mode */ - ret = joycon_set_report_mode(ctlr); - if (ret) { - hid_err(hdev, "Failed to set report mode; ret=%d\n", ret); - goto err_mutex; - } - - /* Enable rumble */ - ret = joycon_enable_rumble(ctlr); - if (ret) { - hid_err(hdev, "Failed to enable rumble; ret=%d\n", ret); - goto err_mutex; - } - - /* Enable the IMU */ - ret = joycon_enable_imu(ctlr); - if (ret) { - hid_err(hdev, "Failed to enable the IMU; ret=%d\n", ret); - goto err_mutex; + hid_err(hdev, "Failed to initialize controller; ret=%d\n", ret); + goto err_close; } ret = joycon_read_info(ctlr); if (ret) { hid_err(hdev, "Failed to retrieve controller info; ret=%d\n", ret); - goto err_mutex; + goto err_close; } - mutex_unlock(&ctlr->output_mutex); - /* Initialize the leds */ ret = joycon_leds_create(ctlr); if (ret) { @@ -2352,8 +2367,6 @@ static int nintendo_hid_probe(struct hid_device *hdev, hid_dbg(hdev, "probe - success\n"); return 0; -err_mutex: - mutex_unlock(&ctlr->output_mutex); err_close: hid_hw_close(hdev); err_stop: @@ -2383,6 +2396,20 @@ static void nintendo_hid_remove(struct hid_device *hdev) hid_hw_stop(hdev); } +#ifdef CONFIG_PM + +static int nintendo_hid_resume(struct hid_device *hdev) +{ + int ret = joycon_init(hdev); + + if (ret) + hid_err(hdev, "Failed to restore controller after resume"); + + return ret; +} + +#endif + static const struct hid_device_id nintendo_hid_devices[] = { { HID_USB_DEVICE(USB_VENDOR_ID_NINTENDO, USB_DEVICE_ID_NINTENDO_PROCON) }, @@ -2404,6 +2431,10 @@ static struct hid_driver nintendo_hid_driver = { .probe = nintendo_hid_probe, .remove = nintendo_hid_remove, .raw_event = nintendo_hid_event, + +#ifdef CONFIG_PM + .resume = nintendo_hid_resume, +#endif }; module_hid_driver(nintendo_hid_driver); diff --git a/drivers/hid/hid-nvidia-shield.c b/drivers/hid/hid-nvidia-shield.c index 9a3576dbf421..c463e54decbc 100644 --- a/drivers/hid/hid-nvidia-shield.c +++ b/drivers/hid/hid-nvidia-shield.c @@ -801,7 +801,7 @@ static inline int thunderstrike_led_create(struct thunderstrike *ts) led->name = devm_kasprintf(&ts->base.hdev->dev, GFP_KERNEL, "thunderstrike%d:blue:led", ts->id); led->max_brightness = 1; - led->flags = LED_CORE_SUSPENDRESUME; + led->flags = LED_CORE_SUSPENDRESUME | LED_RETAIN_AT_SHUTDOWN; led->brightness_get = &thunderstrike_led_get_brightness; led->brightness_set = &thunderstrike_led_set_brightness; @@ -1058,7 +1058,7 @@ static int shield_probe(struct hid_device *hdev, const struct hid_device_id *id) ret = hid_hw_start(hdev, HID_CONNECT_HIDINPUT); if (ret) { hid_err(hdev, "Failed to start HID device\n"); - goto err_haptics; + goto err_ts_create; } ret = hid_hw_open(hdev); @@ -1073,9 +1073,12 @@ static int shield_probe(struct hid_device *hdev, const struct hid_device_id *id) err_stop: hid_hw_stop(hdev); -err_haptics: +err_ts_create: + power_supply_unregister(ts->base.battery_dev.psy); if (ts->haptics_dev) input_unregister_device(ts->haptics_dev); + led_classdev_unregister(&ts->led_dev); + ida_free(&thunderstrike_ida, ts->id); return ret; } diff --git a/drivers/hid/hid-sony.c b/drivers/hid/hid-sony.c index dd942061fd77..ebc0aa4e4345 100644 --- a/drivers/hid/hid-sony.c +++ b/drivers/hid/hid-sony.c @@ -2155,6 +2155,8 @@ static int sony_probe(struct hid_device *hdev, const struct hid_device_id *id) return ret; err: + usb_free_urb(sc->ghl_urb); + hid_hw_stop(hdev); return ret; } diff --git a/drivers/hid/hid-steelseries.c b/drivers/hid/hid-steelseries.c index 43d2cf7153d7..b3edadf42d6d 100644 --- a/drivers/hid/hid-steelseries.c +++ b/drivers/hid/hid-steelseries.c @@ -390,7 +390,7 @@ static int steelseries_headset_arctis_1_fetch_battery(struct hid_device *hdev) ret = hid_hw_raw_request(hdev, arctis_1_battery_request[0], write_buf, sizeof(arctis_1_battery_request), HID_OUTPUT_REPORT, HID_REQ_SET_REPORT); - if (ret < sizeof(arctis_1_battery_request)) { + if (ret < (int)sizeof(arctis_1_battery_request)) { hid_err(hdev, "hid_hw_raw_request() failed with %d\n", ret); ret = -ENODATA; } diff --git a/drivers/hid/i2c-hid/i2c-hid-core.c b/drivers/hid/i2c-hid/i2c-hid-core.c index 9601c0605fd9..2735cd585af0 100644 --- a/drivers/hid/i2c-hid/i2c-hid-core.c +++ b/drivers/hid/i2c-hid/i2c-hid-core.c @@ -998,45 +998,29 @@ static int i2c_hid_core_resume(struct i2c_hid *ihid) return hid_driver_reset_resume(hid); } -/** - * __do_i2c_hid_core_initial_power_up() - First time power up of the i2c-hid device. - * @ihid: The ihid object created during probe. - * - * This function is called at probe time. - * - * The initial power on is where we do some basic validation that the device - * exists, where we fetch the HID descriptor, and where we create the actual - * HID devices. - * - * Return: 0 or error code. +/* + * Check that the device exists and parse the HID descriptor. */ -static int __do_i2c_hid_core_initial_power_up(struct i2c_hid *ihid) +static int __i2c_hid_core_probe(struct i2c_hid *ihid) { struct i2c_client *client = ihid->client; struct hid_device *hid = ihid->hid; int ret; - ret = i2c_hid_core_power_up(ihid); - if (ret) - return ret; - /* Make sure there is something at this address */ ret = i2c_smbus_read_byte(client); if (ret < 0) { i2c_hid_dbg(ihid, "nothing at this address: %d\n", ret); - ret = -ENXIO; - goto err; + return -ENXIO; } ret = i2c_hid_fetch_hid_descriptor(ihid); if (ret < 0) { dev_err(&client->dev, "Failed to fetch the HID Descriptor\n"); - goto err; + return ret; } - enable_irq(client->irq); - hid->version = le16_to_cpu(ihid->hdesc.bcdVersion); hid->vendor = le16_to_cpu(ihid->hdesc.wVendorID); hid->product = le16_to_cpu(ihid->hdesc.wProductID); @@ -1050,17 +1034,49 @@ static int __do_i2c_hid_core_initial_power_up(struct i2c_hid *ihid) ihid->quirks = i2c_hid_lookup_quirk(hid->vendor, hid->product); + return 0; +} + +static int i2c_hid_core_register_hid(struct i2c_hid *ihid) +{ + struct i2c_client *client = ihid->client; + struct hid_device *hid = ihid->hid; + int ret; + + enable_irq(client->irq); + ret = hid_add_device(hid); if (ret) { if (ret != -ENODEV) hid_err(client, "can't add hid device: %d\n", ret); - goto err; + disable_irq(client->irq); + return ret; } return 0; +} + +static int i2c_hid_core_probe_panel_follower(struct i2c_hid *ihid) +{ + int ret; + + ret = i2c_hid_core_power_up(ihid); + if (ret) + return ret; -err: + ret = __i2c_hid_core_probe(ihid); + if (ret) + goto err_power_down; + + ret = i2c_hid_core_register_hid(ihid); + if (ret) + goto err_power_down; + + return 0; + +err_power_down: i2c_hid_core_power_down(ihid); + return ret; } @@ -1077,7 +1093,7 @@ static void ihid_core_panel_prepare_work(struct work_struct *work) * steps. */ if (!hid->version) - ret = __do_i2c_hid_core_initial_power_up(ihid); + ret = i2c_hid_core_probe_panel_follower(ihid); else ret = i2c_hid_core_resume(ihid); @@ -1136,7 +1152,6 @@ static int i2c_hid_core_register_panel_follower(struct i2c_hid *ihid) struct device *dev = &ihid->client->dev; int ret; - ihid->is_panel_follower = true; ihid->panel_follower.funcs = &i2c_hid_core_panel_follower_funcs; /* @@ -1156,30 +1171,6 @@ static int i2c_hid_core_register_panel_follower(struct i2c_hid *ihid) return 0; } -static int i2c_hid_core_initial_power_up(struct i2c_hid *ihid) -{ - /* - * If we're a panel follower, we'll register and do our initial power - * up when the panel turns on; otherwise we do it right away. - */ - if (drm_is_panel_follower(&ihid->client->dev)) - return i2c_hid_core_register_panel_follower(ihid); - else - return __do_i2c_hid_core_initial_power_up(ihid); -} - -static void i2c_hid_core_final_power_down(struct i2c_hid *ihid) -{ - /* - * If we're a follower, the act of unfollowing will cause us to be - * powered down. Otherwise we need to manually do it. - */ - if (ihid->is_panel_follower) - drm_panel_remove_follower(&ihid->panel_follower); - else - i2c_hid_core_suspend(ihid, true); -} - int i2c_hid_core_probe(struct i2c_client *client, struct i2chid_ops *ops, u16 hid_descriptor_address, u32 quirks) { @@ -1211,6 +1202,7 @@ int i2c_hid_core_probe(struct i2c_client *client, struct i2chid_ops *ops, ihid->ops = ops; ihid->client = client; ihid->wHIDDescRegister = cpu_to_le16(hid_descriptor_address); + ihid->is_panel_follower = drm_is_panel_follower(&client->dev); init_waitqueue_head(&ihid->wait); mutex_init(&ihid->reset_lock); @@ -1224,14 +1216,10 @@ int i2c_hid_core_probe(struct i2c_client *client, struct i2chid_ops *ops, return ret; device_enable_async_suspend(&client->dev); - ret = i2c_hid_init_irq(client); - if (ret < 0) - goto err_buffers_allocated; - hid = hid_allocate_device(); if (IS_ERR(hid)) { ret = PTR_ERR(hid); - goto err_irq; + goto err_free_buffers; } ihid->hid = hid; @@ -1242,19 +1230,42 @@ int i2c_hid_core_probe(struct i2c_client *client, struct i2chid_ops *ops, hid->bus = BUS_I2C; hid->initial_quirks = quirks; - ret = i2c_hid_core_initial_power_up(ihid); + /* Power on and probe unless device is a panel follower. */ + if (!ihid->is_panel_follower) { + ret = i2c_hid_core_power_up(ihid); + if (ret < 0) + goto err_destroy_device; + + ret = __i2c_hid_core_probe(ihid); + if (ret < 0) + goto err_power_down; + } + + ret = i2c_hid_init_irq(client); + if (ret < 0) + goto err_power_down; + + /* + * If we're a panel follower, we'll register when the panel turns on; + * otherwise we do it right away. + */ + if (ihid->is_panel_follower) + ret = i2c_hid_core_register_panel_follower(ihid); + else + ret = i2c_hid_core_register_hid(ihid); if (ret) - goto err_mem_free; + goto err_free_irq; return 0; -err_mem_free: - hid_destroy_device(hid); - -err_irq: +err_free_irq: free_irq(client->irq, ihid); - -err_buffers_allocated: +err_power_down: + if (!ihid->is_panel_follower) + i2c_hid_core_power_down(ihid); +err_destroy_device: + hid_destroy_device(hid); +err_free_buffers: i2c_hid_free_buffers(ihid); return ret; @@ -1266,7 +1277,14 @@ void i2c_hid_core_remove(struct i2c_client *client) struct i2c_hid *ihid = i2c_get_clientdata(client); struct hid_device *hid; - i2c_hid_core_final_power_down(ihid); + /* + * If we're a follower, the act of unfollowing will cause us to be + * powered down. Otherwise we need to manually do it. + */ + if (ihid->is_panel_follower) + drm_panel_remove_follower(&ihid->panel_follower); + else + i2c_hid_core_suspend(ihid, true); hid = ihid->hid; hid_destroy_device(hid); diff --git a/drivers/hid/intel-ish-hid/ipc/pci-ish.c b/drivers/hid/intel-ish-hid/ipc/pci-ish.c index 55cb25038e63..710fda5f19e1 100644 --- a/drivers/hid/intel-ish-hid/ipc/pci-ish.c +++ b/drivers/hid/intel-ish-hid/ipc/pci-ish.c @@ -133,6 +133,14 @@ static int enable_gpe(struct device *dev) } wakeup = &adev->wakeup; + /* + * Call acpi_disable_gpe(), so that reference count + * gpe_event_info->runtime_count doesn't overflow. + * When gpe_event_info->runtime_count = 0, the call + * to acpi_disable_gpe() simply return. + */ + acpi_disable_gpe(wakeup->gpe_device, wakeup->gpe_number); + acpi_sts = acpi_enable_gpe(wakeup->gpe_device, wakeup->gpe_number); if (ACPI_FAILURE(acpi_sts)) { dev_err(dev, "enable ose_gpe failed\n"); diff --git a/drivers/hwmon/nct6775-core.c b/drivers/hwmon/nct6775-core.c index 02a71244fc3b..b5b81bd83bb1 100644 --- a/drivers/hwmon/nct6775-core.c +++ b/drivers/hwmon/nct6775-core.c @@ -1910,6 +1910,10 @@ static umode_t nct6775_in_is_visible(struct kobject *kobj, struct device *dev = kobj_to_dev(kobj); struct nct6775_data *data = dev_get_drvdata(dev); int in = index / 5; /* voltage index */ + int nr = index % 5; /* attribute index */ + + if (nr == 1 && data->ALARM_BITS[in] == -1) + return 0; if (!(data->have_in & BIT(in))) return 0; diff --git a/drivers/hwtracing/coresight/coresight-tmc-etr.c b/drivers/hwtracing/coresight/coresight-tmc-etr.c index 66dc5f97a009..8311e1028ddb 100644 --- a/drivers/hwtracing/coresight/coresight-tmc-etr.c +++ b/drivers/hwtracing/coresight/coresight-tmc-etr.c @@ -610,7 +610,8 @@ static int tmc_etr_alloc_flat_buf(struct tmc_drvdata *drvdata, flat_buf->vaddr = dma_alloc_noncoherent(real_dev, etr_buf->size, &flat_buf->daddr, - DMA_FROM_DEVICE, GFP_KERNEL); + DMA_FROM_DEVICE, + GFP_KERNEL | __GFP_NOWARN); if (!flat_buf->vaddr) { kfree(flat_buf); return -ENOMEM; @@ -1173,16 +1174,6 @@ static struct etr_buf *tmc_etr_get_sysfs_buffer(struct coresight_device *csdev) goto out; } - /* - * In sysFS mode we can have multiple writers per sink. Since this - * sink is already enabled no memory is needed and the HW need not be - * touched, even if the buffer size has changed. - */ - if (drvdata->mode == CS_MODE_SYSFS) { - atomic_inc(&csdev->refcnt); - goto out; - } - /* * If we don't have a buffer or it doesn't match the requested size, * use the buffer allocated above. Otherwise reuse the existing buffer. @@ -1204,7 +1195,7 @@ out: static int tmc_enable_etr_sink_sysfs(struct coresight_device *csdev) { - int ret; + int ret = 0; unsigned long flags; struct tmc_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent); struct etr_buf *sysfs_buf = tmc_etr_get_sysfs_buffer(csdev); @@ -1213,12 +1204,24 @@ static int tmc_enable_etr_sink_sysfs(struct coresight_device *csdev) return PTR_ERR(sysfs_buf); spin_lock_irqsave(&drvdata->spinlock, flags); + + /* + * In sysFS mode we can have multiple writers per sink. Since this + * sink is already enabled no memory is needed and the HW need not be + * touched, even if the buffer size has changed. + */ + if (drvdata->mode == CS_MODE_SYSFS) { + atomic_inc(&csdev->refcnt); + goto out; + } + ret = tmc_etr_enable_hw(drvdata, sysfs_buf); if (!ret) { drvdata->mode = CS_MODE_SYSFS; atomic_inc(&csdev->refcnt); } +out: spin_unlock_irqrestore(&drvdata->spinlock, flags); if (!ret) diff --git a/drivers/i2c/Kconfig b/drivers/i2c/Kconfig index c6d1a345ea6d..9388823bb0bb 100644 --- a/drivers/i2c/Kconfig +++ b/drivers/i2c/Kconfig @@ -72,7 +72,7 @@ config I2C_MUX source "drivers/i2c/muxes/Kconfig" config I2C_ATR - tristate "I2C Address Translator (ATR) support" + tristate "I2C Address Translator (ATR) support" if COMPILE_TEST help Enable support for I2C Address Translator (ATR) chips. diff --git a/drivers/i2c/busses/Kconfig b/drivers/i2c/busses/Kconfig index 169607e80331..6644eebedaf3 100644 --- a/drivers/i2c/busses/Kconfig +++ b/drivers/i2c/busses/Kconfig @@ -1384,10 +1384,10 @@ config I2C_ICY config I2C_MLXCPLD tristate "Mellanox I2C driver" - depends on X86_64 || ARM64 || COMPILE_TEST + depends on X86_64 || (ARM64 && ACPI) || COMPILE_TEST help This exposes the Mellanox platform I2C busses to the linux I2C layer - for X86 based systems. + for X86 and ARM64/ACPI based systems. Controller is implemented as CPLD logic. This driver can also be built as a module. If so, the module will be diff --git a/drivers/i2c/busses/i2c-aspeed.c b/drivers/i2c/busses/i2c-aspeed.c index 2e5acfeb76c8..5a416b39b818 100644 --- a/drivers/i2c/busses/i2c-aspeed.c +++ b/drivers/i2c/busses/i2c-aspeed.c @@ -698,13 +698,16 @@ static int aspeed_i2c_master_xfer(struct i2c_adapter *adap, if (time_left == 0) { /* - * If timed out and bus is still busy in a multi master - * environment, attempt recovery at here. + * In a multi-master setup, if a timeout occurs, attempt + * recovery. But if the bus is idle, we still need to reset the + * i2c controller to clear the remaining interrupts. */ if (bus->multi_master && (readl(bus->base + ASPEED_I2C_CMD_REG) & ASPEED_I2CD_BUS_BUSY_STS)) aspeed_i2c_recover_bus(bus); + else + aspeed_i2c_reset(bus); /* * If timed out and the state is still pending, drop the pending diff --git a/drivers/i2c/busses/i2c-cadence.c b/drivers/i2c/busses/i2c-cadence.c index 9849f4502570..de3f58b60dce 100644 --- a/drivers/i2c/busses/i2c-cadence.c +++ b/drivers/i2c/busses/i2c-cadence.c @@ -182,6 +182,7 @@ enum cdns_i2c_slave_state { * @reset: Reset control for the device * @quirks: flag for broken hold bit usage in r1p10 * @ctrl_reg: Cached value of the control register. + * @rinfo: I2C GPIO recovery information * @ctrl_reg_diva_divb: value of fields DIV_A and DIV_B from CR register * @slave: Registered slave instance. * @dev_mode: I2C operating role(master/slave). diff --git a/drivers/i2c/busses/i2c-designware-common.c b/drivers/i2c/busses/i2c-designware-common.c index cdd8c67d9129..affcfb243f0f 100644 --- a/drivers/i2c/busses/i2c-designware-common.c +++ b/drivers/i2c/busses/i2c-designware-common.c @@ -441,8 +441,25 @@ err_release_lock: void __i2c_dw_disable(struct dw_i2c_dev *dev) { + unsigned int raw_intr_stats; + unsigned int enable; int timeout = 100; + bool abort_needed; unsigned int status; + int ret; + + regmap_read(dev->map, DW_IC_RAW_INTR_STAT, &raw_intr_stats); + regmap_read(dev->map, DW_IC_ENABLE, &enable); + + abort_needed = raw_intr_stats & DW_IC_INTR_MST_ON_HOLD; + if (abort_needed) { + regmap_write(dev->map, DW_IC_ENABLE, enable | DW_IC_ENABLE_ABORT); + ret = regmap_read_poll_timeout(dev->map, DW_IC_ENABLE, enable, + !(enable & DW_IC_ENABLE_ABORT), 10, + 100); + if (ret) + dev_err(dev->dev, "timeout while trying to abort current transfer\n"); + } do { __i2c_dw_disable_nowait(dev); diff --git a/drivers/i2c/busses/i2c-designware-core.h b/drivers/i2c/busses/i2c-designware-core.h index cf4f684f5356..a7f6f3eafad7 100644 --- a/drivers/i2c/busses/i2c-designware-core.h +++ b/drivers/i2c/busses/i2c-designware-core.h @@ -98,6 +98,7 @@ #define DW_IC_INTR_START_DET BIT(10) #define DW_IC_INTR_GEN_CALL BIT(11) #define DW_IC_INTR_RESTART_DET BIT(12) +#define DW_IC_INTR_MST_ON_HOLD BIT(13) #define DW_IC_INTR_DEFAULT_MASK (DW_IC_INTR_RX_FULL | \ DW_IC_INTR_TX_ABRT | \ @@ -108,6 +109,8 @@ DW_IC_INTR_RX_UNDER | \ DW_IC_INTR_RD_REQ) +#define DW_IC_ENABLE_ABORT BIT(1) + #define DW_IC_STATUS_ACTIVITY BIT(0) #define DW_IC_STATUS_TFE BIT(2) #define DW_IC_STATUS_RFNE BIT(3) diff --git a/drivers/i2c/busses/i2c-i801.c b/drivers/i2c/busses/i2c-i801.c index 73ae06432133..1d855258a45d 100644 --- a/drivers/i2c/busses/i2c-i801.c +++ b/drivers/i2c/busses/i2c-i801.c @@ -1754,6 +1754,7 @@ static int i801_probe(struct pci_dev *dev, const struct pci_device_id *id) "SMBus I801 adapter at %04lx", priv->smba); err = i2c_add_adapter(&priv->adapter); if (err) { + platform_device_unregister(priv->tco_pdev); i801_acpi_remove(priv); return err; } diff --git a/drivers/i2c/busses/i2c-npcm7xx.c b/drivers/i2c/busses/i2c-npcm7xx.c index 495a8b5f6a2b..ae4bae63ad4f 100644 --- a/drivers/i2c/busses/i2c-npcm7xx.c +++ b/drivers/i2c/busses/i2c-npcm7xx.c @@ -694,6 +694,7 @@ static void npcm_i2c_callback(struct npcm_i2c *bus, { struct i2c_msg *msgs; int msgs_num; + bool do_complete = false; msgs = bus->msgs; msgs_num = bus->msgs_num; @@ -722,23 +723,17 @@ static void npcm_i2c_callback(struct npcm_i2c *bus, msgs[1].flags & I2C_M_RD) msgs[1].len = info; } - if (completion_done(&bus->cmd_complete) == false) - complete(&bus->cmd_complete); - break; - + do_complete = true; + break; case I2C_NACK_IND: /* MASTER transmit got a NACK before tx all bytes */ bus->cmd_err = -ENXIO; - if (bus->master_or_slave == I2C_MASTER) - complete(&bus->cmd_complete); - + do_complete = true; break; case I2C_BUS_ERR_IND: /* Bus error */ bus->cmd_err = -EAGAIN; - if (bus->master_or_slave == I2C_MASTER) - complete(&bus->cmd_complete); - + do_complete = true; break; case I2C_WAKE_UP_IND: /* I2C wake up */ @@ -752,6 +747,8 @@ static void npcm_i2c_callback(struct npcm_i2c *bus, if (bus->slave) bus->master_or_slave = I2C_SLAVE; #endif + if (do_complete) + complete(&bus->cmd_complete); } static u8 npcm_i2c_fifo_usage(struct npcm_i2c *bus) diff --git a/drivers/i2c/busses/i2c-xiic.c b/drivers/i2c/busses/i2c-xiic.c index b3bb97762c85..71391b590ada 100644 --- a/drivers/i2c/busses/i2c-xiic.c +++ b/drivers/i2c/busses/i2c-xiic.c @@ -710,7 +710,7 @@ static irqreturn_t xiic_process(int irq, void *dev_id) * reset the IP instead of just flush fifos */ ret = xiic_reinit(i2c); - if (!ret) + if (ret < 0) dev_dbg(i2c->adap.dev.parent, "reinit failed\n"); if (i2c->rx_msg) { diff --git a/drivers/i2c/i2c-mux.c b/drivers/i2c/i2c-mux.c index 313904be5f3b..57ff09f18c37 100644 --- a/drivers/i2c/i2c-mux.c +++ b/drivers/i2c/i2c-mux.c @@ -341,7 +341,7 @@ int i2c_mux_add_adapter(struct i2c_mux_core *muxc, priv->adap.lock_ops = &i2c_parent_lock_ops; /* Sanity check on class */ - if (i2c_mux_parent_classes(parent) & class) + if (i2c_mux_parent_classes(parent) & class & ~I2C_CLASS_DEPRECATED) dev_err(&parent->dev, "Segment %d behind mux can't share classes with ancestors\n", chan_id); diff --git a/drivers/i2c/muxes/i2c-demux-pinctrl.c b/drivers/i2c/muxes/i2c-demux-pinctrl.c index a3a122fae71e..22f2280eab7f 100644 --- a/drivers/i2c/muxes/i2c-demux-pinctrl.c +++ b/drivers/i2c/muxes/i2c-demux-pinctrl.c @@ -243,6 +243,10 @@ static int i2c_demux_pinctrl_probe(struct platform_device *pdev) props[i].name = devm_kstrdup(&pdev->dev, "status", GFP_KERNEL); props[i].value = devm_kstrdup(&pdev->dev, "ok", GFP_KERNEL); + if (!props[i].name || !props[i].value) { + err = -ENOMEM; + goto err_rollback; + } props[i].length = 3; of_changeset_init(&priv->chan[i].chgset); diff --git a/drivers/i2c/muxes/i2c-mux-gpio.c b/drivers/i2c/muxes/i2c-mux-gpio.c index 5d5cbe0130cd..5ca03bd34c8d 100644 --- a/drivers/i2c/muxes/i2c-mux-gpio.c +++ b/drivers/i2c/muxes/i2c-mux-gpio.c @@ -105,8 +105,10 @@ static int i2c_mux_gpio_probe_fw(struct gpiomux *mux, } else if (is_acpi_node(child)) { rc = acpi_get_local_address(ACPI_HANDLE_FWNODE(child), values + i); - if (rc) + if (rc) { + fwnode_handle_put(child); return dev_err_probe(dev, rc, "Cannot get address\n"); + } } i++; diff --git a/drivers/iio/adc/ad7192.c b/drivers/iio/adc/ad7192.c index 69d1103b9508..b64fd365f83f 100644 --- a/drivers/iio/adc/ad7192.c +++ b/drivers/iio/adc/ad7192.c @@ -177,6 +177,7 @@ struct ad7192_chip_info { struct ad7192_state { const struct ad7192_chip_info *chip_info; struct regulator *avdd; + struct regulator *vref; struct clk *mclk; u16 int_vref_mv; u32 fclk; @@ -1008,10 +1009,30 @@ static int ad7192_probe(struct spi_device *spi) if (ret) return dev_err_probe(&spi->dev, ret, "Failed to enable specified DVdd supply\n"); - ret = regulator_get_voltage(st->avdd); - if (ret < 0) { - dev_err(&spi->dev, "Device tree error, reference voltage undefined\n"); - return ret; + st->vref = devm_regulator_get_optional(&spi->dev, "vref"); + if (IS_ERR(st->vref)) { + if (PTR_ERR(st->vref) != -ENODEV) + return PTR_ERR(st->vref); + + ret = regulator_get_voltage(st->avdd); + if (ret < 0) + return dev_err_probe(&spi->dev, ret, + "Device tree error, AVdd voltage undefined\n"); + } else { + ret = regulator_enable(st->vref); + if (ret) { + dev_err(&spi->dev, "Failed to enable specified Vref supply\n"); + return ret; + } + + ret = devm_add_action_or_reset(&spi->dev, ad7192_reg_disable, st->vref); + if (ret) + return ret; + + ret = regulator_get_voltage(st->vref); + if (ret < 0) + return dev_err_probe(&spi->dev, ret, + "Device tree error, Vref voltage undefined\n"); } st->int_vref_mv = ret / 1000; diff --git a/drivers/iio/adc/imx8qxp-adc.c b/drivers/iio/adc/imx8qxp-adc.c index f5a0fc9e64c5..fff6e5a2d956 100644 --- a/drivers/iio/adc/imx8qxp-adc.c +++ b/drivers/iio/adc/imx8qxp-adc.c @@ -38,8 +38,8 @@ #define IMX8QXP_ADR_ADC_FCTRL 0x30 #define IMX8QXP_ADR_ADC_SWTRIG 0x34 #define IMX8QXP_ADR_ADC_TCTRL(tid) (0xc0 + (tid) * 4) -#define IMX8QXP_ADR_ADC_CMDH(cid) (0x100 + (cid) * 8) -#define IMX8QXP_ADR_ADC_CMDL(cid) (0x104 + (cid) * 8) +#define IMX8QXP_ADR_ADC_CMDL(cid) (0x100 + (cid) * 8) +#define IMX8QXP_ADR_ADC_CMDH(cid) (0x104 + (cid) * 8) #define IMX8QXP_ADR_ADC_RESFIFO 0x300 #define IMX8QXP_ADR_ADC_TST 0xffc diff --git a/drivers/iio/addac/Kconfig b/drivers/iio/addac/Kconfig index 877f9124803c..397544f23b85 100644 --- a/drivers/iio/addac/Kconfig +++ b/drivers/iio/addac/Kconfig @@ -24,6 +24,8 @@ config AD74413R depends on GPIOLIB && SPI select REGMAP_SPI select CRC8 + select IIO_BUFFER + select IIO_TRIGGERED_BUFFER help Say yes here to build support for Analog Devices AD74412R/AD74413R quad-channel software configurable input/output solution. diff --git a/drivers/iio/common/cros_ec_sensors/cros_ec_sensors_core.c b/drivers/iio/common/cros_ec_sensors/cros_ec_sensors_core.c index b72d39fc2434..6bfe5d6847e7 100644 --- a/drivers/iio/common/cros_ec_sensors/cros_ec_sensors_core.c +++ b/drivers/iio/common/cros_ec_sensors/cros_ec_sensors_core.c @@ -190,8 +190,11 @@ int cros_ec_sensors_push_data(struct iio_dev *indio_dev, /* * Ignore samples if the buffer is not set: it is needed if the ODR is * set but the buffer is not enabled yet. + * + * Note: iio_device_claim_buffer_mode() returns -EBUSY if the buffer + * is not enabled. */ - if (!iio_buffer_enabled(indio_dev)) + if (iio_device_claim_buffer_mode(indio_dev) < 0) return 0; out = (s16 *)st->samples; @@ -210,6 +213,7 @@ int cros_ec_sensors_push_data(struct iio_dev *indio_dev, iio_push_to_buffers_with_timestamp(indio_dev, st->samples, timestamp + delta); + iio_device_release_buffer_mode(indio_dev); return 0; } EXPORT_SYMBOL_GPL(cros_ec_sensors_push_data); diff --git a/drivers/iio/dac/ad3552r.c b/drivers/iio/dac/ad3552r.c index d5ea1a1be122..a492e8f2fc0f 100644 --- a/drivers/iio/dac/ad3552r.c +++ b/drivers/iio/dac/ad3552r.c @@ -140,8 +140,8 @@ enum ad3552r_ch_vref_select { }; enum ad3542r_id { - AD3542R_ID = 0x4008, - AD3552R_ID = 0x4009, + AD3542R_ID = 0x4009, + AD3552R_ID = 0x4008, }; enum ad3552r_ch_output_range { diff --git a/drivers/iio/frequency/admv1013.c b/drivers/iio/frequency/admv1013.c index 6355c1f28423..92923074f930 100644 --- a/drivers/iio/frequency/admv1013.c +++ b/drivers/iio/frequency/admv1013.c @@ -351,9 +351,9 @@ static int admv1013_update_mixer_vgate(struct admv1013_state *st) if (vcm < 0) return vcm; - if (vcm < 1800000) + if (vcm <= 1800000) mixer_vgate = (2389 * vcm / 1000000 + 8100) / 100; - else if (vcm > 1800000 && vcm < 2600000) + else if (vcm > 1800000 && vcm <= 2600000) mixer_vgate = (2375 * vcm / 1000000 + 125) / 100; else return -EINVAL; diff --git a/drivers/iio/imu/bno055/Kconfig b/drivers/iio/imu/bno055/Kconfig index fa79b1ac4f85..83e53acfbe88 100644 --- a/drivers/iio/imu/bno055/Kconfig +++ b/drivers/iio/imu/bno055/Kconfig @@ -2,6 +2,8 @@ config BOSCH_BNO055 tristate + select IIO_BUFFER + select IIO_TRIGGERED_BUFFER config BOSCH_BNO055_SERIAL tristate "Bosch BNO055 attached via UART" diff --git a/drivers/iio/light/vcnl4000.c b/drivers/iio/light/vcnl4000.c index 3a52b09c2823..fdf763a04b0b 100644 --- a/drivers/iio/light/vcnl4000.c +++ b/drivers/iio/light/vcnl4000.c @@ -1513,7 +1513,6 @@ static int vcnl4040_write_event_config(struct iio_dev *indio_dev, out: mutex_unlock(&data->vcnl4000_lock); - data->chip_spec->set_power_state(data, data->ps_int || data->als_int); return ret; } diff --git a/drivers/iio/pressure/bmp280-core.c b/drivers/iio/pressure/bmp280-core.c index 6089f3f9d8f4..a2ef1373a274 100644 --- a/drivers/iio/pressure/bmp280-core.c +++ b/drivers/iio/pressure/bmp280-core.c @@ -2179,7 +2179,7 @@ int bmp280_common_probe(struct device *dev, * however as it happens, the BMP085 shares the chip ID of BMP180 * so we look for an IRQ if we have that. */ - if (irq > 0 || (chip_id == BMP180_CHIP_ID)) { + if (irq > 0 && (chip_id == BMP180_CHIP_ID)) { ret = bmp085_fetch_eoc_irq(dev, name, irq, data); if (ret) return ret; diff --git a/drivers/iio/pressure/dps310.c b/drivers/iio/pressure/dps310.c index b10dbf5cf494..1ff091b2f764 100644 --- a/drivers/iio/pressure/dps310.c +++ b/drivers/iio/pressure/dps310.c @@ -57,8 +57,8 @@ #define DPS310_RESET_MAGIC 0x09 #define DPS310_COEF_BASE 0x10 -/* Make sure sleep time is <= 20ms for usleep_range */ -#define DPS310_POLL_SLEEP_US(t) min(20000, (t) / 8) +/* Make sure sleep time is <= 30ms for usleep_range */ +#define DPS310_POLL_SLEEP_US(t) min(30000, (t) / 8) /* Silently handle error in rate value here */ #define DPS310_POLL_TIMEOUT_US(rc) ((rc) <= 0 ? 1000000 : 1000000 / (rc)) @@ -402,8 +402,8 @@ static int dps310_reset_wait(struct dps310_data *data) if (rc) return rc; - /* Wait for device chip access: 2.5ms in specification */ - usleep_range(2500, 12000); + /* Wait for device chip access: 15ms in specification */ + usleep_range(15000, 55000); return 0; } diff --git a/drivers/iio/pressure/ms5611_core.c b/drivers/iio/pressure/ms5611_core.c index 627497e61a63..2fc706f9d8ae 100644 --- a/drivers/iio/pressure/ms5611_core.c +++ b/drivers/iio/pressure/ms5611_core.c @@ -76,7 +76,7 @@ static bool ms5611_prom_is_valid(u16 *prom, size_t len) crc = (crc >> 12) & 0x000F; - return crc_orig != 0x0000 && crc == crc_orig; + return crc == crc_orig; } static int ms5611_read_prom(struct iio_dev *indio_dev) diff --git a/drivers/iio/proximity/irsd200.c b/drivers/iio/proximity/irsd200.c index 5bd791b46d98..bdff91f6b1a3 100644 --- a/drivers/iio/proximity/irsd200.c +++ b/drivers/iio/proximity/irsd200.c @@ -759,14 +759,14 @@ static irqreturn_t irsd200_trigger_handler(int irq, void *pollf) { struct iio_dev *indio_dev = ((struct iio_poll_func *)pollf)->indio_dev; struct irsd200_data *data = iio_priv(indio_dev); - s16 buf = 0; + s64 buf[2] = {}; int ret; - ret = irsd200_read_data(data, &buf); + ret = irsd200_read_data(data, (s16 *)buf); if (ret) goto end; - iio_push_to_buffers_with_timestamp(indio_dev, &buf, + iio_push_to_buffers_with_timestamp(indio_dev, buf, iio_get_time_ns(indio_dev)); end: diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c index c343edf2f664..1e2cd7c8716e 100644 --- a/drivers/infiniband/core/cma.c +++ b/drivers/infiniband/core/cma.c @@ -4968,7 +4968,7 @@ static int cma_iboe_join_multicast(struct rdma_id_private *id_priv, int err = 0; struct sockaddr *addr = (struct sockaddr *)&mc->addr; struct net_device *ndev = NULL; - struct ib_sa_multicast ib; + struct ib_sa_multicast ib = {}; enum ib_gid_type gid_type; bool send_only; diff --git a/drivers/infiniband/core/cma_configfs.c b/drivers/infiniband/core/cma_configfs.c index 7b68b3ea979f..f2fb2d8a6597 100644 --- a/drivers/infiniband/core/cma_configfs.c +++ b/drivers/infiniband/core/cma_configfs.c @@ -217,7 +217,7 @@ static int make_cma_ports(struct cma_dev_group *cma_dev_group, return -ENOMEM; for (i = 0; i < ports_num; i++) { - char port_str[10]; + char port_str[11]; ports[i].port_num = i + 1; snprintf(port_str, sizeof(port_str), "%u", i + 1); diff --git a/drivers/infiniband/core/nldev.c b/drivers/infiniband/core/nldev.c index d5d3e4f0de77..6d1dbc978759 100644 --- a/drivers/infiniband/core/nldev.c +++ b/drivers/infiniband/core/nldev.c @@ -2529,6 +2529,7 @@ static const struct rdma_nl_cbs nldev_cb_table[RDMA_NLDEV_NUM_OPS] = { }, [RDMA_NLDEV_CMD_SYS_SET] = { .doit = nldev_set_sys_set_doit, + .flags = RDMA_NL_ADMIN_PERM, }, [RDMA_NLDEV_CMD_STAT_SET] = { .doit = nldev_stat_set_doit, diff --git a/drivers/infiniband/core/uverbs_main.c b/drivers/infiniband/core/uverbs_main.c index bf800f8cb3e4..495d5a5d0373 100644 --- a/drivers/infiniband/core/uverbs_main.c +++ b/drivers/infiniband/core/uverbs_main.c @@ -546,7 +546,7 @@ static ssize_t verify_hdr(struct ib_uverbs_cmd_hdr *hdr, if (hdr->in_words * 4 != count) return -EINVAL; - if (count < method_elm->req_size + sizeof(hdr)) { + if (count < method_elm->req_size + sizeof(*hdr)) { /* * rdma-core v18 and v19 have a bug where they send DESTROY_CQ * with a 16 byte write instead of 24. Old kernels didn't diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.c b/drivers/infiniband/hw/bnxt_re/ib_verbs.c index 0848c2c2ffcf..faa88d12ee86 100644 --- a/drivers/infiniband/hw/bnxt_re/ib_verbs.c +++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.c @@ -910,6 +910,10 @@ int bnxt_re_destroy_qp(struct ib_qp *ib_qp, struct ib_udata *udata) list_del(&qp->list); mutex_unlock(&rdev->qp_lock); atomic_dec(&rdev->stats.res.qp_count); + if (qp->qplib_qp.type == CMDQ_CREATE_QP_TYPE_RC) + atomic_dec(&rdev->stats.res.rc_qp_count); + else if (qp->qplib_qp.type == CMDQ_CREATE_QP_TYPE_UD) + atomic_dec(&rdev->stats.res.ud_qp_count); ib_umem_release(qp->rumem); ib_umem_release(qp->sumem); diff --git a/drivers/infiniband/hw/bnxt_re/qplib_rcfw.c b/drivers/infiniband/hw/bnxt_re/qplib_rcfw.c index c8c4017fe405..e47b4ca64d33 100644 --- a/drivers/infiniband/hw/bnxt_re/qplib_rcfw.c +++ b/drivers/infiniband/hw/bnxt_re/qplib_rcfw.c @@ -665,7 +665,6 @@ static int bnxt_qplib_process_qp_event(struct bnxt_qplib_rcfw *rcfw, blocked = cookie & RCFW_CMD_IS_BLOCKING; cookie &= RCFW_MAX_COOKIE_VALUE; crsqe = &rcfw->crsqe_tbl[cookie]; - crsqe->is_in_used = false; if (WARN_ONCE(test_bit(FIRMWARE_STALL_DETECTED, &rcfw->cmdq.flags), @@ -681,8 +680,14 @@ static int bnxt_qplib_process_qp_event(struct bnxt_qplib_rcfw *rcfw, atomic_dec(&rcfw->timeout_send); if (crsqe->is_waiter_alive) { - if (crsqe->resp) + if (crsqe->resp) { memcpy(crsqe->resp, qp_event, sizeof(*qp_event)); + /* Insert write memory barrier to ensure that + * response data is copied before clearing the + * flags + */ + smp_wmb(); + } if (!blocked) wait_cmds++; } @@ -694,6 +699,8 @@ static int bnxt_qplib_process_qp_event(struct bnxt_qplib_rcfw *rcfw, if (!is_waiter_alive) crsqe->resp = NULL; + crsqe->is_in_used = false; + hwq->cons += req_size; /* This is a case to handle below scenario - diff --git a/drivers/infiniband/hw/cxgb4/cm.c b/drivers/infiniband/hw/cxgb4/cm.c index ced615b5ea09..040ba2224f9f 100644 --- a/drivers/infiniband/hw/cxgb4/cm.c +++ b/drivers/infiniband/hw/cxgb4/cm.c @@ -1965,6 +1965,9 @@ static int send_fw_act_open_req(struct c4iw_ep *ep, unsigned int atid) int win; skb = get_skb(NULL, sizeof(*req), GFP_KERNEL); + if (!skb) + return -ENOMEM; + req = __skb_put_zero(skb, sizeof(*req)); req->op_compl = htonl(WR_OP_V(FW_OFLD_CONNECTION_WR)); req->len16_pkd = htonl(FW_WR_LEN16_V(DIV_ROUND_UP(sizeof(*req), 16))); diff --git a/drivers/infiniband/hw/erdma/erdma_verbs.c b/drivers/infiniband/hw/erdma/erdma_verbs.c index dcccb6015232..c317947563fb 100644 --- a/drivers/infiniband/hw/erdma/erdma_verbs.c +++ b/drivers/infiniband/hw/erdma/erdma_verbs.c @@ -133,8 +133,8 @@ static int create_qp_cmd(struct erdma_ucontext *uctx, struct erdma_qp *qp) static int regmr_cmd(struct erdma_dev *dev, struct erdma_mr *mr) { struct erdma_pd *pd = to_epd(mr->ibmr.pd); + u32 mtt_level = ERDMA_MR_MTT_0LEVEL; struct erdma_cmdq_reg_mr_req req; - u32 mtt_level; erdma_cmdq_build_reqhdr(&req.hdr, CMDQ_SUBMOD_RDMA, CMDQ_OPCODE_REG_MR); @@ -147,10 +147,9 @@ static int regmr_cmd(struct erdma_dev *dev, struct erdma_mr *mr) req.phy_addr[0] = sg_dma_address(mr->mem.mtt->sglist); mtt_level = mr->mem.mtt->level; } - } else { + } else if (mr->type != ERDMA_MR_TYPE_DMA) { memcpy(req.phy_addr, mr->mem.mtt->buf, MTT_SIZE(mr->mem.page_cnt)); - mtt_level = ERDMA_MR_MTT_0LEVEL; } req.cfg0 = FIELD_PREP(ERDMA_CMD_MR_VALID_MASK, mr->valid) | @@ -655,7 +654,7 @@ static struct erdma_mtt *erdma_create_scatter_mtt(struct erdma_dev *dev, mtt = kzalloc(sizeof(*mtt), GFP_KERNEL); if (!mtt) - return NULL; + return ERR_PTR(-ENOMEM); mtt->size = ALIGN(size, PAGE_SIZE); mtt->buf = vzalloc(mtt->size); diff --git a/drivers/infiniband/hw/mlx4/sysfs.c b/drivers/infiniband/hw/mlx4/sysfs.c index 24ee79aa2122..88f534cf690e 100644 --- a/drivers/infiniband/hw/mlx4/sysfs.c +++ b/drivers/infiniband/hw/mlx4/sysfs.c @@ -223,7 +223,7 @@ void del_sysfs_port_mcg_attr(struct mlx4_ib_dev *device, int port_num, static int add_port_entries(struct mlx4_ib_dev *device, int port_num) { int i; - char buff[11]; + char buff[12]; struct mlx4_ib_iov_port *port = NULL; int ret = 0 ; struct ib_port_attr attr; diff --git a/drivers/infiniband/hw/mlx5/fs.c b/drivers/infiniband/hw/mlx5/fs.c index 1e419e080b53..520034acf73a 100644 --- a/drivers/infiniband/hw/mlx5/fs.c +++ b/drivers/infiniband/hw/mlx5/fs.c @@ -2470,8 +2470,8 @@ destroy_res: mlx5_steering_anchor_destroy_res(ft_prio); put_flow_table: put_flow_table(dev, ft_prio, true); - mutex_unlock(&dev->flow_db->lock); free_obj: + mutex_unlock(&dev->flow_db->lock); kfree(obj); return err; diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c index aed5cdea50e6..555629b798b9 100644 --- a/drivers/infiniband/hw/mlx5/main.c +++ b/drivers/infiniband/hw/mlx5/main.c @@ -2084,7 +2084,7 @@ static inline char *mmap_cmd2str(enum mlx5_ib_mmap_cmd cmd) case MLX5_IB_MMAP_DEVICE_MEM: return "Device Memory"; default: - return NULL; + return "Unknown"; } } diff --git a/drivers/infiniband/hw/mlx5/mr.c b/drivers/infiniband/hw/mlx5/mr.c index 3e345ef380f1..8a3762d9ff58 100644 --- a/drivers/infiniband/hw/mlx5/mr.c +++ b/drivers/infiniband/hw/mlx5/mr.c @@ -301,7 +301,8 @@ static int get_mkc_octo_size(unsigned int access_mode, unsigned int ndescs) static void set_cache_mkc(struct mlx5_cache_ent *ent, void *mkc) { - set_mkc_access_pd_addr_fields(mkc, 0, 0, ent->dev->umrc.pd); + set_mkc_access_pd_addr_fields(mkc, ent->rb_key.access_flags, 0, + ent->dev->umrc.pd); MLX5_SET(mkc, mkc, free, 1); MLX5_SET(mkc, mkc, umr_en, 1); MLX5_SET(mkc, mkc, access_mode_1_0, ent->rb_key.access_mode & 0x3); @@ -1024,19 +1025,26 @@ void mlx5_mkey_cache_cleanup(struct mlx5_ib_dev *dev) if (!dev->cache.wq) return; - cancel_delayed_work_sync(&dev->cache.remove_ent_dwork); mutex_lock(&dev->cache.rb_lock); for (node = rb_first(root); node; node = rb_next(node)) { ent = rb_entry(node, struct mlx5_cache_ent, node); xa_lock_irq(&ent->mkeys); ent->disabled = true; xa_unlock_irq(&ent->mkeys); - cancel_delayed_work_sync(&ent->dwork); } + mutex_unlock(&dev->cache.rb_lock); + + /* + * After all entries are disabled and will not reschedule on WQ, + * flush it and all async commands. + */ + flush_workqueue(dev->cache.wq); mlx5_mkey_cache_debugfs_cleanup(dev); mlx5_cmd_cleanup_async_ctx(&dev->async_ctx); + /* At this point all entries are disabled and have no concurrent work. */ + mutex_lock(&dev->cache.rb_lock); node = rb_first(root); while (node) { ent = rb_entry(node, struct mlx5_cache_ent, node); diff --git a/drivers/infiniband/sw/siw/siw_cm.c b/drivers/infiniband/sw/siw/siw_cm.c index a2605178f4ed..43e776073f49 100644 --- a/drivers/infiniband/sw/siw/siw_cm.c +++ b/drivers/infiniband/sw/siw/siw_cm.c @@ -976,6 +976,7 @@ static void siw_accept_newconn(struct siw_cep *cep) siw_cep_put(cep); new_cep->listen_cep = NULL; if (rv) { + siw_cancel_mpatimer(new_cep); siw_cep_set_free(new_cep); goto error; } @@ -1100,9 +1101,12 @@ static void siw_cm_work_handler(struct work_struct *w) /* * Socket close before MPA request received. */ - siw_dbg_cep(cep, "no mpareq: drop listener\n"); - siw_cep_put(cep->listen_cep); - cep->listen_cep = NULL; + if (cep->listen_cep) { + siw_dbg_cep(cep, + "no mpareq: drop listener\n"); + siw_cep_put(cep->listen_cep); + cep->listen_cep = NULL; + } } } release_cep = 1; @@ -1227,7 +1231,11 @@ static void siw_cm_llp_data_ready(struct sock *sk) if (!cep) goto out; - siw_dbg_cep(cep, "state: %d\n", cep->state); + siw_dbg_cep(cep, "cep state: %d, socket state %d\n", + cep->state, sk->sk_state); + + if (sk->sk_state != TCP_ESTABLISHED) + goto out; switch (cep->state) { case SIW_EPSTATE_RDMA_MODE: diff --git a/drivers/infiniband/ulp/srp/ib_srp.c b/drivers/infiniband/ulp/srp/ib_srp.c index 1574218764e0..2916e77f589b 100644 --- a/drivers/infiniband/ulp/srp/ib_srp.c +++ b/drivers/infiniband/ulp/srp/ib_srp.c @@ -2784,7 +2784,6 @@ static int srp_abort(struct scsi_cmnd *scmnd) u32 tag; u16 ch_idx; struct srp_rdma_ch *ch; - int ret; shost_printk(KERN_ERR, target->scsi_host, "SRP abort called\n"); @@ -2798,19 +2797,14 @@ static int srp_abort(struct scsi_cmnd *scmnd) shost_printk(KERN_ERR, target->scsi_host, "Sending SRP abort for tag %#x\n", tag); if (srp_send_tsk_mgmt(ch, tag, scmnd->device->lun, - SRP_TSK_ABORT_TASK, NULL) == 0) - ret = SUCCESS; - else if (target->rport->state == SRP_RPORT_LOST) - ret = FAST_IO_FAIL; - else - ret = FAILED; - if (ret == SUCCESS) { + SRP_TSK_ABORT_TASK, NULL) == 0) { srp_free_req(ch, req, scmnd, 0); - scmnd->result = DID_ABORT << 16; - scsi_done(scmnd); + return SUCCESS; } + if (target->rport->state == SRP_RPORT_LOST) + return FAST_IO_FAIL; - return ret; + return FAILED; } static int srp_reset_device(struct scsi_cmnd *scmnd) diff --git a/drivers/input/joystick/xpad.c b/drivers/input/joystick/xpad.c index ede380551e55..f5c21565bb3c 100644 --- a/drivers/input/joystick/xpad.c +++ b/drivers/input/joystick/xpad.c @@ -130,6 +130,7 @@ static const struct xpad_device { { 0x0079, 0x18d4, "GPD Win 2 X-Box Controller", 0, XTYPE_XBOX360 }, { 0x03eb, 0xff01, "Wooting One (Legacy)", 0, XTYPE_XBOX360 }, { 0x03eb, 0xff02, "Wooting Two (Legacy)", 0, XTYPE_XBOX360 }, + { 0x03f0, 0x0495, "HyperX Clutch Gladiate", 0, XTYPE_XBOXONE }, { 0x044f, 0x0f00, "Thrustmaster Wheel", 0, XTYPE_XBOX }, { 0x044f, 0x0f03, "Thrustmaster Wheel", 0, XTYPE_XBOX }, { 0x044f, 0x0f07, "Thrustmaster, Inc. Controller", 0, XTYPE_XBOX }, @@ -272,6 +273,7 @@ static const struct xpad_device { { 0x1038, 0x1430, "SteelSeries Stratus Duo", 0, XTYPE_XBOX360 }, { 0x1038, 0x1431, "SteelSeries Stratus Duo", 0, XTYPE_XBOX360 }, { 0x11c9, 0x55f0, "Nacon GC-100XF", 0, XTYPE_XBOX360 }, + { 0x11ff, 0x0511, "PXN V900", 0, XTYPE_XBOX360 }, { 0x1209, 0x2882, "Ardwiino Controller", 0, XTYPE_XBOX360 }, { 0x12ab, 0x0004, "Honey Bee Xbox360 dancepad", MAP_DPAD_TO_BUTTONS, XTYPE_XBOX360 }, { 0x12ab, 0x0301, "PDP AFTERGLOW AX.1", 0, XTYPE_XBOX360 }, @@ -459,6 +461,7 @@ static const struct usb_device_id xpad_table[] = { { USB_INTERFACE_INFO('X', 'B', 0) }, /* Xbox USB-IF not-approved class */ XPAD_XBOX360_VENDOR(0x0079), /* GPD Win 2 controller */ XPAD_XBOX360_VENDOR(0x03eb), /* Wooting Keyboards (Legacy) */ + XPAD_XBOXONE_VENDOR(0x03f0), /* HP HyperX Xbox One controllers */ XPAD_XBOX360_VENDOR(0x044f), /* Thrustmaster Xbox 360 controllers */ XPAD_XBOX360_VENDOR(0x045e), /* Microsoft Xbox 360 controllers */ XPAD_XBOXONE_VENDOR(0x045e), /* Microsoft Xbox One controllers */ @@ -477,6 +480,7 @@ static const struct usb_device_id xpad_table[] = { XPAD_XBOX360_VENDOR(0x1038), /* SteelSeries controllers */ XPAD_XBOXONE_VENDOR(0x10f5), /* Turtle Beach Controllers */ XPAD_XBOX360_VENDOR(0x11c9), /* Nacon GC100XF */ + XPAD_XBOX360_VENDOR(0x11ff), /* PXN V900 */ XPAD_XBOX360_VENDOR(0x1209), /* Ardwiino Controllers */ XPAD_XBOX360_VENDOR(0x12ab), /* Xbox 360 dance pads */ XPAD_XBOX360_VENDOR(0x1430), /* RedOctane Xbox 360 controllers */ diff --git a/drivers/input/misc/powermate.c b/drivers/input/misc/powermate.c index c1c733a9cb89..db2ba89adaef 100644 --- a/drivers/input/misc/powermate.c +++ b/drivers/input/misc/powermate.c @@ -425,6 +425,7 @@ static void powermate_disconnect(struct usb_interface *intf) pm->requires_update = 0; usb_kill_urb(pm->irq); input_unregister_device(pm->input); + usb_kill_urb(pm->config); usb_free_urb(pm->irq); usb_free_urb(pm->config); powermate_free_buffers(interface_to_usbdev(intf), pm); diff --git a/drivers/input/mouse/elantech.c b/drivers/input/mouse/elantech.c index 2118b2075f43..4e38229404b4 100644 --- a/drivers/input/mouse/elantech.c +++ b/drivers/input/mouse/elantech.c @@ -2114,6 +2114,7 @@ static int elantech_setup_ps2(struct psmouse *psmouse, psmouse->protocol_handler = elantech_process_byte; psmouse->disconnect = elantech_disconnect; psmouse->reconnect = elantech_reconnect; + psmouse->fast_reconnect = NULL; psmouse->pktsize = info->hw_version > 1 ? 6 : 4; return 0; diff --git a/drivers/input/mouse/psmouse-smbus.c b/drivers/input/mouse/psmouse-smbus.c index 7b13de979908..2a2459b1b4f2 100644 --- a/drivers/input/mouse/psmouse-smbus.c +++ b/drivers/input/mouse/psmouse-smbus.c @@ -5,7 +5,6 @@ #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt -#include #include #include #include @@ -119,18 +118,13 @@ static psmouse_ret_t psmouse_smbus_process_byte(struct psmouse *psmouse) return PSMOUSE_FULL_PACKET; } -static void psmouse_activate_smbus_mode(struct psmouse_smbus_dev *smbdev) -{ - if (smbdev->need_deactivate) { - psmouse_deactivate(smbdev->psmouse); - /* Give the device time to switch into SMBus mode */ - msleep(30); - } -} - static int psmouse_smbus_reconnect(struct psmouse *psmouse) { - psmouse_activate_smbus_mode(psmouse->private); + struct psmouse_smbus_dev *smbdev = psmouse->private; + + if (smbdev->need_deactivate) + psmouse_deactivate(psmouse); + return 0; } @@ -263,7 +257,8 @@ int psmouse_smbus_init(struct psmouse *psmouse, } } - psmouse_activate_smbus_mode(smbdev); + if (need_deactivate) + psmouse_deactivate(psmouse); psmouse->private = smbdev; psmouse->protocol_handler = psmouse_smbus_process_byte; diff --git a/drivers/input/mouse/synaptics.c b/drivers/input/mouse/synaptics.c index ada299ec5bba..22d16d80efb9 100644 --- a/drivers/input/mouse/synaptics.c +++ b/drivers/input/mouse/synaptics.c @@ -1623,6 +1623,7 @@ static int synaptics_init_ps2(struct psmouse *psmouse, psmouse->set_rate = synaptics_set_rate; psmouse->disconnect = synaptics_disconnect; psmouse->reconnect = synaptics_reconnect; + psmouse->fast_reconnect = NULL; psmouse->cleanup = synaptics_reset; /* Synaptics can usually stay in sync without extra help */ psmouse->resync_time = 0; @@ -1752,6 +1753,7 @@ static int synaptics_create_intertouch(struct psmouse *psmouse, psmouse_matches_pnp_id(psmouse, topbuttonpad_pnp_ids) && !SYN_CAP_EXT_BUTTONS_STICK(info->ext_cap_10); const struct rmi_device_platform_data pdata = { + .reset_delay_ms = 30, .sensor_pdata = { .sensor_type = rmi_sensor_touchpad, .axis_align.flip_y = true, diff --git a/drivers/input/rmi4/rmi_smbus.c b/drivers/input/rmi4/rmi_smbus.c index 7059a2762aeb..b0b099b5528a 100644 --- a/drivers/input/rmi4/rmi_smbus.c +++ b/drivers/input/rmi4/rmi_smbus.c @@ -235,12 +235,29 @@ static void rmi_smb_clear_state(struct rmi_smb_xport *rmi_smb) static int rmi_smb_enable_smbus_mode(struct rmi_smb_xport *rmi_smb) { - int retval; + struct i2c_client *client = rmi_smb->client; + int smbus_version; + + /* + * psmouse driver resets the controller, we only need to wait + * to give the firmware chance to fully reinitialize. + */ + if (rmi_smb->xport.pdata.reset_delay_ms) + msleep(rmi_smb->xport.pdata.reset_delay_ms); /* we need to get the smbus version to activate the touchpad */ - retval = rmi_smb_get_version(rmi_smb); - if (retval < 0) - return retval; + smbus_version = rmi_smb_get_version(rmi_smb); + if (smbus_version < 0) + return smbus_version; + + rmi_dbg(RMI_DEBUG_XPORT, &client->dev, "Smbus version is %d", + smbus_version); + + if (smbus_version != 2 && smbus_version != 3) { + dev_err(&client->dev, "Unrecognized SMB version %d\n", + smbus_version); + return -ENODEV; + } return 0; } @@ -253,11 +270,10 @@ static int rmi_smb_reset(struct rmi_transport_dev *xport, u16 reset_addr) rmi_smb_clear_state(rmi_smb); /* - * we do not call the actual reset command, it has to be handled in - * PS/2 or there will be races between PS/2 and SMBus. - * PS/2 should ensure that a psmouse_reset is called before - * intializing the device and after it has been removed to be in a known - * state. + * We do not call the actual reset command, it has to be handled in + * PS/2 or there will be races between PS/2 and SMBus. PS/2 should + * ensure that a psmouse_reset is called before initializing the + * device and after it has been removed to be in a known state. */ return rmi_smb_enable_smbus_mode(rmi_smb); } @@ -272,7 +288,6 @@ static int rmi_smb_probe(struct i2c_client *client) { struct rmi_device_platform_data *pdata = dev_get_platdata(&client->dev); struct rmi_smb_xport *rmi_smb; - int smbus_version; int error; if (!pdata) { @@ -311,18 +326,9 @@ static int rmi_smb_probe(struct i2c_client *client) rmi_smb->xport.proto_name = "smb"; rmi_smb->xport.ops = &rmi_smb_ops; - smbus_version = rmi_smb_get_version(rmi_smb); - if (smbus_version < 0) - return smbus_version; - - rmi_dbg(RMI_DEBUG_XPORT, &client->dev, "Smbus version is %d", - smbus_version); - - if (smbus_version != 2 && smbus_version != 3) { - dev_err(&client->dev, "Unrecognized SMB version %d\n", - smbus_version); - return -ENODEV; - } + error = rmi_smb_enable_smbus_mode(rmi_smb); + if (error) + return error; i2c_set_clientdata(client, rmi_smb); diff --git a/drivers/input/serio/i8042-acpipnpio.h b/drivers/input/serio/i8042-acpipnpio.h index 1724d6cb8649..9c39553d30fa 100644 --- a/drivers/input/serio/i8042-acpipnpio.h +++ b/drivers/input/serio/i8042-acpipnpio.h @@ -618,6 +618,14 @@ static const struct dmi_system_id i8042_dmi_quirk_table[] __initconst = { }, .driver_data = (void *)(SERIO_QUIRK_NOMUX) }, + { + /* Fujitsu Lifebook E5411 */ + .matches = { + DMI_MATCH(DMI_SYS_VENDOR, "FUJITSU CLIENT COMPUTING LIMITED"), + DMI_MATCH(DMI_PRODUCT_NAME, "LIFEBOOK E5411"), + }, + .driver_data = (void *)(SERIO_QUIRK_NOAUX) + }, { /* Gigabyte M912 */ .matches = { diff --git a/drivers/input/touchscreen/goodix.c b/drivers/input/touchscreen/goodix.c index da9954d6df44..af32fbe57b63 100644 --- a/drivers/input/touchscreen/goodix.c +++ b/drivers/input/touchscreen/goodix.c @@ -900,6 +900,25 @@ static int goodix_add_acpi_gpio_mappings(struct goodix_ts_data *ts) dev_info(dev, "No ACPI GpioInt resource, assuming that the GPIO order is reset, int\n"); ts->irq_pin_access_method = IRQ_PIN_ACCESS_ACPI_GPIO; gpio_mapping = acpi_goodix_int_last_gpios; + } else if (ts->gpio_count == 1 && ts->gpio_int_idx == 0) { + /* + * On newer devices there is only 1 GpioInt resource and _PS0 + * does the whole reset sequence for us. + */ + acpi_device_fix_up_power(ACPI_COMPANION(dev)); + + /* + * Before the _PS0 call the int GPIO may have been in output + * mode and the call should have put the int GPIO in input mode, + * but the GPIO subsys cached state may still think it is + * in output mode, causing gpiochip_lock_as_irq() failure. + * + * Add a mapping for the int GPIO to make the + * gpiod_int = gpiod_get(..., GPIOD_IN) call succeed, + * which will explicitly set the direction to input. + */ + ts->irq_pin_access_method = IRQ_PIN_ACCESS_NONE; + gpio_mapping = acpi_goodix_int_first_gpios; } else { dev_warn(dev, "Unexpected ACPI resources: gpio_count %d, gpio_int_idx %d\n", ts->gpio_count, ts->gpio_int_idx); diff --git a/drivers/iommu/apple-dart.c b/drivers/iommu/apple-dart.c index 2082081402d3..0b8927508427 100644 --- a/drivers/iommu/apple-dart.c +++ b/drivers/iommu/apple-dart.c @@ -671,8 +671,7 @@ static int apple_dart_attach_dev(struct iommu_domain *domain, return ret; switch (domain->type) { - case IOMMU_DOMAIN_DMA: - case IOMMU_DOMAIN_UNMANAGED: + default: ret = apple_dart_domain_add_streams(dart_domain, cfg); if (ret) return ret; diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 4d83edc2be99..8a16cd3ef487 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -186,6 +186,15 @@ static void arm_smmu_free_shared_cd(struct arm_smmu_ctx_desc *cd) } } +/* + * Cloned from the MAX_TLBI_OPS in arch/arm64/include/asm/tlbflush.h, this + * is used as a threshold to replace per-page TLBI commands to issue in the + * command queue with an address-space TLBI command, when SMMU w/o a range + * invalidation feature handles too many per-page TLBI commands, which will + * otherwise result in a soft lockup. + */ +#define CMDQ_MAX_TLBI_OPS (1 << (PAGE_SHIFT - 3)) + static void arm_smmu_mm_arch_invalidate_secondary_tlbs(struct mmu_notifier *mn, struct mm_struct *mm, unsigned long start, @@ -201,8 +210,13 @@ static void arm_smmu_mm_arch_invalidate_secondary_tlbs(struct mmu_notifier *mn, * range. So do a simple translation here by calculating size correctly. */ size = end - start; - if (size == ULONG_MAX) - size = 0; + if (!(smmu_domain->smmu->features & ARM_SMMU_FEAT_RANGE_INV)) { + if (size >= CMDQ_MAX_TLBI_OPS * PAGE_SIZE) + size = 0; + } else { + if (size == ULONG_MAX) + size = 0; + } if (!(smmu_domain->smmu->features & ARM_SMMU_FEAT_BTM)) { if (!size) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index e82bf1c449a3..bd0a596f9863 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1895,18 +1895,23 @@ static void __arm_smmu_tlb_inv_range(struct arm_smmu_cmdq_ent *cmd, /* Get the leaf page size */ tg = __ffs(smmu_domain->domain.pgsize_bitmap); + num_pages = size >> tg; + /* Convert page size of 12,14,16 (log2) to 1,2,3 */ cmd->tlbi.tg = (tg - 10) / 2; /* - * Determine what level the granule is at. For non-leaf, io-pgtable - * assumes .tlb_flush_walk can invalidate multiple levels at once, - * so ignore the nominal last-level granule and leave TTL=0. + * Determine what level the granule is at. For non-leaf, both + * io-pgtable and SVA pass a nominal last-level granule because + * they don't know what level(s) actually apply, so ignore that + * and leave TTL=0. However for various errata reasons we still + * want to use a range command, so avoid the SVA corner case + * where both scale and num could be 0 as well. */ if (cmd->tlbi.leaf) cmd->tlbi.ttl = 4 - ((ilog2(granule) - 3) / (tg - 3)); - - num_pages = size >> tg; + else if ((num_pages & CMDQ_TLBI_RANGE_NUM_MAX) == 1) + num_pages++; } cmds.num = 0; diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 5db283c17e0d..3685ba90ec88 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -2998,13 +2998,6 @@ static int iommu_suspend(void) struct intel_iommu *iommu = NULL; unsigned long flag; - for_each_active_iommu(iommu, drhd) { - iommu->iommu_state = kcalloc(MAX_SR_DMAR_REGS, sizeof(u32), - GFP_KERNEL); - if (!iommu->iommu_state) - goto nomem; - } - iommu_flush_all(); for_each_active_iommu(iommu, drhd) { @@ -3024,12 +3017,6 @@ static int iommu_suspend(void) raw_spin_unlock_irqrestore(&iommu->register_lock, flag); } return 0; - -nomem: - for_each_active_iommu(iommu, drhd) - kfree(iommu->iommu_state); - - return -ENOMEM; } static void iommu_resume(void) @@ -3061,9 +3048,6 @@ static void iommu_resume(void) raw_spin_unlock_irqrestore(&iommu->register_lock, flag); } - - for_each_active_iommu(iommu, drhd) - kfree(iommu->iommu_state); } static struct syscore_ops iommu_syscore_ops = { diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h index c18fb699c87a..7dac94f62b4e 100644 --- a/drivers/iommu/intel/iommu.h +++ b/drivers/iommu/intel/iommu.h @@ -681,7 +681,7 @@ struct intel_iommu { struct iopf_queue *iopf_queue; unsigned char iopfq_name[16]; struct q_inval *qi; /* Queued invalidation info */ - u32 *iommu_state; /* Store iommu states between suspend and resume.*/ + u32 iommu_state[MAX_SR_DMAR_REGS]; /* Store iommu states between suspend and resume.*/ #ifdef CONFIG_IRQ_REMAP struct ir_table *ir_table; /* Interrupt remapping info */ diff --git a/drivers/iommu/mtk_iommu.c b/drivers/iommu/mtk_iommu.c index 640275873a27..fab6c347ce57 100644 --- a/drivers/iommu/mtk_iommu.c +++ b/drivers/iommu/mtk_iommu.c @@ -262,7 +262,7 @@ struct mtk_iommu_data { struct device *smicomm_dev; struct mtk_iommu_bank_data *bank; - struct mtk_iommu_domain *share_dom; /* For 2 HWs share pgtable */ + struct mtk_iommu_domain *share_dom; struct regmap *pericfg; struct mutex mutex; /* Protect m4u_group/m4u_dom above */ @@ -643,8 +643,8 @@ static int mtk_iommu_domain_finalise(struct mtk_iommu_domain *dom, struct mtk_iommu_domain *share_dom = data->share_dom; const struct mtk_iommu_iova_region *region; - /* Always use share domain in sharing pgtable case */ - if (MTK_IOMMU_HAS_FLAG(data->plat_data, SHARE_PGTABLE) && share_dom) { + /* Share pgtable when 2 MM IOMMU share the pgtable or one IOMMU use multiple iova ranges */ + if (share_dom) { dom->iop = share_dom->iop; dom->cfg = share_dom->cfg; dom->domain.pgsize_bitmap = share_dom->cfg.pgsize_bitmap; @@ -677,8 +677,7 @@ static int mtk_iommu_domain_finalise(struct mtk_iommu_domain *dom, /* Update our support page sizes bitmap */ dom->domain.pgsize_bitmap = dom->cfg.pgsize_bitmap; - if (MTK_IOMMU_HAS_FLAG(data->plat_data, SHARE_PGTABLE)) - data->share_dom = dom; + data->share_dom = dom; update_iova_region: /* Update the iova region for this domain */ diff --git a/drivers/irqchip/irq-gic-common.h b/drivers/irqchip/irq-gic-common.h index 3db4592cda1c..f407cce9ecaa 100644 --- a/drivers/irqchip/irq-gic-common.h +++ b/drivers/irqchip/irq-gic-common.h @@ -29,4 +29,8 @@ void gic_enable_quirks(u32 iidr, const struct gic_quirk *quirks, void gic_enable_of_quirks(const struct device_node *np, const struct gic_quirk *quirks, void *data); +#define RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING (1 << 0) +#define RDIST_FLAGS_RD_TABLES_PREALLOCATED (1 << 1) +#define RDIST_FLAGS_FORCE_NON_SHAREABLE (1 << 2) + #endif /* _IRQ_GIC_COMMON_H */ diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c index e0c2b10d154d..75a2dd550625 100644 --- a/drivers/irqchip/irq-gic-v3-its.c +++ b/drivers/irqchip/irq-gic-v3-its.c @@ -44,10 +44,6 @@ #define ITS_FLAGS_WORKAROUND_CAVIUM_23144 (1ULL << 2) #define ITS_FLAGS_FORCE_NON_SHAREABLE (1ULL << 3) -#define RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING (1 << 0) -#define RDIST_FLAGS_RD_TABLES_PREALLOCATED (1 << 1) -#define RDIST_FLAGS_FORCE_NON_SHAREABLE (1 << 2) - #define RD_LOCAL_LPI_ENABLED BIT(0) #define RD_LOCAL_PENDTABLE_PREALLOCATED BIT(1) #define RD_LOCAL_MEMRESERVE_DONE BIT(2) @@ -4754,6 +4750,14 @@ static bool __maybe_unused its_enable_rk3588001(void *data) return true; } +static bool its_set_non_coherent(void *data) +{ + struct its_node *its = data; + + its->flags |= ITS_FLAGS_FORCE_NON_SHAREABLE; + return true; +} + static const struct gic_quirk its_quirks[] = { #ifdef CONFIG_CAVIUM_ERRATUM_22375 { @@ -4808,6 +4812,11 @@ static const struct gic_quirk its_quirks[] = { .init = its_enable_rk3588001, }, #endif + { + .desc = "ITS: non-coherent attribute", + .property = "dma-noncoherent", + .init = its_set_non_coherent, + }, { } }; @@ -4817,6 +4826,10 @@ static void its_enable_quirks(struct its_node *its) u32 iidr = readl_relaxed(its->base + GITS_IIDR); gic_enable_quirks(iidr, its_quirks, its); + + if (is_of_node(its->fwnode_handle)) + gic_enable_of_quirks(to_of_node(its->fwnode_handle), + its_quirks, its); } static int its_save_disable(void) @@ -4952,7 +4965,7 @@ out_unmap: return NULL; } -static int its_init_domain(struct fwnode_handle *handle, struct its_node *its) +static int its_init_domain(struct its_node *its) { struct irq_domain *inner_domain; struct msi_domain_info *info; @@ -4966,7 +4979,7 @@ static int its_init_domain(struct fwnode_handle *handle, struct its_node *its) inner_domain = irq_domain_create_hierarchy(its_parent, its->msi_domain_flags, 0, - handle, &its_domain_ops, + its->fwnode_handle, &its_domain_ops, info); if (!inner_domain) { kfree(info); @@ -5017,8 +5030,7 @@ static int its_init_vpe_domain(void) return 0; } -static int __init its_compute_its_list_map(struct resource *res, - void __iomem *its_base) +static int __init its_compute_its_list_map(struct its_node *its) { int its_number; u32 ctlr; @@ -5032,15 +5044,15 @@ static int __init its_compute_its_list_map(struct resource *res, its_number = find_first_zero_bit(&its_list_map, GICv4_ITS_LIST_MAX); if (its_number >= GICv4_ITS_LIST_MAX) { pr_err("ITS@%pa: No ITSList entry available!\n", - &res->start); + &its->phys_base); return -EINVAL; } - ctlr = readl_relaxed(its_base + GITS_CTLR); + ctlr = readl_relaxed(its->base + GITS_CTLR); ctlr &= ~GITS_CTLR_ITS_NUMBER; ctlr |= its_number << GITS_CTLR_ITS_NUMBER_SHIFT; - writel_relaxed(ctlr, its_base + GITS_CTLR); - ctlr = readl_relaxed(its_base + GITS_CTLR); + writel_relaxed(ctlr, its->base + GITS_CTLR); + ctlr = readl_relaxed(its->base + GITS_CTLR); if ((ctlr & GITS_CTLR_ITS_NUMBER) != (its_number << GITS_CTLR_ITS_NUMBER_SHIFT)) { its_number = ctlr & GITS_CTLR_ITS_NUMBER; its_number >>= GITS_CTLR_ITS_NUMBER_SHIFT; @@ -5048,75 +5060,50 @@ static int __init its_compute_its_list_map(struct resource *res, if (test_and_set_bit(its_number, &its_list_map)) { pr_err("ITS@%pa: Duplicate ITSList entry %d\n", - &res->start, its_number); + &its->phys_base, its_number); return -EINVAL; } return its_number; } -static int __init its_probe_one(struct resource *res, - struct fwnode_handle *handle, int numa_node) +static int __init its_probe_one(struct its_node *its) { - struct its_node *its; - void __iomem *its_base; - u64 baser, tmp, typer; + u64 baser, tmp; struct page *page; u32 ctlr; int err; - its_base = its_map_one(res, &err); - if (!its_base) - return err; - - pr_info("ITS %pR\n", res); - - its = kzalloc(sizeof(*its), GFP_KERNEL); - if (!its) { - err = -ENOMEM; - goto out_unmap; - } - - raw_spin_lock_init(&its->lock); - mutex_init(&its->dev_alloc_lock); - INIT_LIST_HEAD(&its->entry); - INIT_LIST_HEAD(&its->its_device_list); - typer = gic_read_typer(its_base + GITS_TYPER); - its->typer = typer; - its->base = its_base; - its->phys_base = res->start; if (is_v4(its)) { - if (!(typer & GITS_TYPER_VMOVP)) { - err = its_compute_its_list_map(res, its_base); + if (!(its->typer & GITS_TYPER_VMOVP)) { + err = its_compute_its_list_map(its); if (err < 0) - goto out_free_its; + goto out; its->list_nr = err; pr_info("ITS@%pa: Using ITS number %d\n", - &res->start, err); + &its->phys_base, err); } else { - pr_info("ITS@%pa: Single VMOVP capable\n", &res->start); + pr_info("ITS@%pa: Single VMOVP capable\n", &its->phys_base); } if (is_v4_1(its)) { - u32 svpet = FIELD_GET(GITS_TYPER_SVPET, typer); + u32 svpet = FIELD_GET(GITS_TYPER_SVPET, its->typer); - its->sgir_base = ioremap(res->start + SZ_128K, SZ_64K); + its->sgir_base = ioremap(its->phys_base + SZ_128K, SZ_64K); if (!its->sgir_base) { err = -ENOMEM; - goto out_free_its; + goto out; } - its->mpidr = readl_relaxed(its_base + GITS_MPIDR); + its->mpidr = readl_relaxed(its->base + GITS_MPIDR); pr_info("ITS@%pa: Using GICv4.1 mode %08x %08x\n", - &res->start, its->mpidr, svpet); + &its->phys_base, its->mpidr, svpet); } } - its->numa_node = numa_node; - page = alloc_pages_node(its->numa_node, GFP_KERNEL | __GFP_ZERO, get_order(ITS_CMD_QUEUE_SZ)); if (!page) { @@ -5125,12 +5112,9 @@ static int __init its_probe_one(struct resource *res, } its->cmd_base = (void *)page_address(page); its->cmd_write = its->cmd_base; - its->fwnode_handle = handle; its->get_msi_base = its_irq_get_msi_base; its->msi_domain_flags = IRQ_DOMAIN_FLAG_ISOLATED_MSI; - its_enable_quirks(its); - err = its_alloc_tables(its); if (err) goto out_free_cmd; @@ -5174,7 +5158,7 @@ static int __init its_probe_one(struct resource *res, ctlr |= GITS_CTLR_ImDe; writel_relaxed(ctlr, its->base + GITS_CTLR); - err = its_init_domain(handle, its); + err = its_init_domain(its); if (err) goto out_free_tables; @@ -5191,11 +5175,8 @@ out_free_cmd: out_unmap_sgir: if (its->sgir_base) iounmap(its->sgir_base); -out_free_its: - kfree(its); -out_unmap: - iounmap(its_base); - pr_err("ITS@%pa: failed probing (%d)\n", &res->start, err); +out: + pr_err("ITS@%pa: failed probing (%d)\n", &its->phys_base, err); return err; } @@ -5356,10 +5337,53 @@ static const struct of_device_id its_device_id[] = { {}, }; +static struct its_node __init *its_node_init(struct resource *res, + struct fwnode_handle *handle, int numa_node) +{ + void __iomem *its_base; + struct its_node *its; + int err; + + its_base = its_map_one(res, &err); + if (!its_base) + return NULL; + + pr_info("ITS %pR\n", res); + + its = kzalloc(sizeof(*its), GFP_KERNEL); + if (!its) + goto out_unmap; + + raw_spin_lock_init(&its->lock); + mutex_init(&its->dev_alloc_lock); + INIT_LIST_HEAD(&its->entry); + INIT_LIST_HEAD(&its->its_device_list); + + its->typer = gic_read_typer(its_base + GITS_TYPER); + its->base = its_base; + its->phys_base = res->start; + + its->numa_node = numa_node; + its->fwnode_handle = handle; + + return its; + +out_unmap: + iounmap(its_base); + return NULL; +} + +static void its_node_destroy(struct its_node *its) +{ + iounmap(its->base); + kfree(its); +} + static int __init its_of_probe(struct device_node *node) { struct device_node *np; struct resource res; + int err; /* * Make sure *all* the ITS are reset before we probe any, as @@ -5369,8 +5393,6 @@ static int __init its_of_probe(struct device_node *node) */ for (np = of_find_matching_node(node, its_device_id); np; np = of_find_matching_node(np, its_device_id)) { - int err; - if (!of_device_is_available(np) || !of_property_read_bool(np, "msi-controller") || of_address_to_resource(np, 0, &res)) @@ -5383,6 +5405,8 @@ static int __init its_of_probe(struct device_node *node) for (np = of_find_matching_node(node, its_device_id); np; np = of_find_matching_node(np, its_device_id)) { + struct its_node *its; + if (!of_device_is_available(np)) continue; if (!of_property_read_bool(np, "msi-controller")) { @@ -5396,7 +5420,17 @@ static int __init its_of_probe(struct device_node *node) continue; } - its_probe_one(&res, &np->fwnode, of_node_to_nid(np)); + + its = its_node_init(&res, &np->fwnode, of_node_to_nid(np)); + if (!its) + return -ENOMEM; + + its_enable_quirks(its); + err = its_probe_one(its); + if (err) { + its_node_destroy(its); + return err; + } } return 0; } @@ -5508,6 +5542,7 @@ static int __init gic_acpi_parse_madt_its(union acpi_subtable_headers *header, { struct acpi_madt_generic_translator *its_entry; struct fwnode_handle *dom_handle; + struct its_node *its; struct resource res; int err; @@ -5532,11 +5567,18 @@ static int __init gic_acpi_parse_madt_its(union acpi_subtable_headers *header, goto dom_err; } - err = its_probe_one(&res, dom_handle, - acpi_get_its_numa_node(its_entry->translation_id)); + its = its_node_init(&res, dom_handle, + acpi_get_its_numa_node(its_entry->translation_id)); + if (!its) { + err = -ENOMEM; + goto node_err; + } + + err = its_probe_one(its); if (!err) return 0; +node_err: iort_deregister_domain_token(its_entry->translation_id); dom_err: irq_domain_free_fwnode(dom_handle); diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c index eedfa8e9f077..f59ac9586b7b 100644 --- a/drivers/irqchip/irq-gic-v3.c +++ b/drivers/irqchip/irq-gic-v3.c @@ -1857,6 +1857,14 @@ static bool gic_enable_quirk_arm64_2941627(void *data) return true; } +static bool rd_set_non_coherent(void *data) +{ + struct gic_chip_data *d = data; + + d->rdists.flags |= RDIST_FLAGS_FORCE_NON_SHAREABLE; + return true; +} + static const struct gic_quirk gic_quirks[] = { { .desc = "GICv3: Qualcomm MSM8996 broken firmware", @@ -1923,6 +1931,11 @@ static const struct gic_quirk gic_quirks[] = { .mask = 0xff0f0fff, .init = gic_enable_quirk_arm64_2941627, }, + { + .desc = "GICv3: non-coherent attribute", + .property = "dma-noncoherent", + .init = rd_set_non_coherent, + }, { } }; diff --git a/drivers/irqchip/irq-renesas-rzg2l.c b/drivers/irqchip/irq-renesas-rzg2l.c index 4bbfa2b0a4df..96f4e322ed6b 100644 --- a/drivers/irqchip/irq-renesas-rzg2l.c +++ b/drivers/irqchip/irq-renesas-rzg2l.c @@ -118,7 +118,7 @@ static void rzg2l_irqc_irq_disable(struct irq_data *d) raw_spin_lock(&priv->lock); reg = readl_relaxed(priv->base + TSSR(tssr_index)); - reg &= ~(TSSEL_MASK << tssr_offset); + reg &= ~(TSSEL_MASK << TSSEL_SHIFT(tssr_offset)); writel_relaxed(reg, priv->base + TSSR(tssr_index)); raw_spin_unlock(&priv->lock); } @@ -130,8 +130,8 @@ static void rzg2l_irqc_irq_enable(struct irq_data *d) unsigned int hw_irq = irqd_to_hwirq(d); if (hw_irq >= IRQC_TINT_START && hw_irq < IRQC_NUM_IRQ) { + unsigned long tint = (uintptr_t)irq_data_get_irq_chip_data(d); struct rzg2l_irqc_priv *priv = irq_data_to_priv(d); - unsigned long tint = (uintptr_t)d->chip_data; u32 offset = hw_irq - IRQC_TINT_START; u32 tssr_offset = TSSR_OFFSET(offset); u8 tssr_index = TSSR_INDEX(offset); diff --git a/drivers/irqchip/irq-riscv-intc.c b/drivers/irqchip/irq-riscv-intc.c index 4adeee1bc391..e8d01b14ccdd 100644 --- a/drivers/irqchip/irq-riscv-intc.c +++ b/drivers/irqchip/irq-riscv-intc.c @@ -155,8 +155,16 @@ static int __init riscv_intc_init(struct device_node *node, * for each INTC DT node. We only need to do INTC initialization * for the INTC DT node belonging to boot CPU (or boot HART). */ - if (riscv_hartid_to_cpuid(hartid) != smp_processor_id()) + if (riscv_hartid_to_cpuid(hartid) != smp_processor_id()) { + /* + * The INTC nodes of each CPU are suppliers for downstream + * interrupt controllers (such as PLIC, IMSIC and APLIC + * direct-mode) so we should mark an INTC node as initialized + * if we are not creating IRQ domain for it. + */ + fwnode_dev_initialized(of_fwnode_handle(node), true); return 0; + } return riscv_intc_init_common(of_node_to_fwnode(node)); } diff --git a/drivers/irqchip/irq-stm32-exti.c b/drivers/irqchip/irq-stm32-exti.c index d8ba5fba7450..971240e2e31b 100644 --- a/drivers/irqchip/irq-stm32-exti.c +++ b/drivers/irqchip/irq-stm32-exti.c @@ -460,6 +460,7 @@ static const struct irq_domain_ops irq_exti_domain_ops = { .map = irq_map_generic_chip, .alloc = stm32_exti_alloc, .free = stm32_exti_free, + .xlate = irq_domain_xlate_twocell, }; static void stm32_irq_ack(struct irq_data *d) diff --git a/drivers/irqchip/irq-xtensa-mx.c b/drivers/irqchip/irq-xtensa-mx.c index 8c581c985aa7..7f314e58f3ce 100644 --- a/drivers/irqchip/irq-xtensa-mx.c +++ b/drivers/irqchip/irq-xtensa-mx.c @@ -12,6 +12,7 @@ #include #include #include +#include #include #include diff --git a/drivers/irqchip/qcom-pdc.c b/drivers/irqchip/qcom-pdc.c index a32c0d28d038..74b2f124116e 100644 --- a/drivers/irqchip/qcom-pdc.c +++ b/drivers/irqchip/qcom-pdc.c @@ -22,9 +22,20 @@ #define PDC_MAX_GPIO_IRQS 256 +/* Valid only on HW version < 3.2 */ #define IRQ_ENABLE_BANK 0x10 #define IRQ_i_CFG 0x110 +/* Valid only on HW version >= 3.2 */ +#define IRQ_i_CFG_IRQ_ENABLE 3 + +#define IRQ_i_CFG_TYPE_MASK GENMASK(2, 0) + +#define PDC_VERSION_REG 0x1000 + +/* Notable PDC versions */ +#define PDC_VERSION_3_2 0x30200 + struct pdc_pin_region { u32 pin_base; u32 parent_base; @@ -37,6 +48,7 @@ static DEFINE_RAW_SPINLOCK(pdc_lock); static void __iomem *pdc_base; static struct pdc_pin_region *pdc_region; static int pdc_region_cnt; +static unsigned int pdc_version; static void pdc_reg_write(int reg, u32 i, u32 val) { @@ -48,20 +60,32 @@ static u32 pdc_reg_read(int reg, u32 i) return readl_relaxed(pdc_base + reg + i * sizeof(u32)); } -static void pdc_enable_intr(struct irq_data *d, bool on) +static void __pdc_enable_intr(int pin_out, bool on) { - int pin_out = d->hwirq; unsigned long enable; - unsigned long flags; - u32 index, mask; - index = pin_out / 32; - mask = pin_out % 32; + if (pdc_version < PDC_VERSION_3_2) { + u32 index, mask; + + index = pin_out / 32; + mask = pin_out % 32; + + enable = pdc_reg_read(IRQ_ENABLE_BANK, index); + __assign_bit(mask, &enable, on); + pdc_reg_write(IRQ_ENABLE_BANK, index, enable); + } else { + enable = pdc_reg_read(IRQ_i_CFG, pin_out); + __assign_bit(IRQ_i_CFG_IRQ_ENABLE, &enable, on); + pdc_reg_write(IRQ_i_CFG, pin_out, enable); + } +} + +static void pdc_enable_intr(struct irq_data *d, bool on) +{ + unsigned long flags; raw_spin_lock_irqsave(&pdc_lock, flags); - enable = pdc_reg_read(IRQ_ENABLE_BANK, index); - __assign_bit(mask, &enable, on); - pdc_reg_write(IRQ_ENABLE_BANK, index, enable); + __pdc_enable_intr(d->hwirq, on); raw_spin_unlock_irqrestore(&pdc_lock, flags); } @@ -142,6 +166,7 @@ static int qcom_pdc_gic_set_type(struct irq_data *d, unsigned int type) } old_pdc_type = pdc_reg_read(IRQ_i_CFG, d->hwirq); + pdc_type |= (old_pdc_type & ~IRQ_i_CFG_TYPE_MASK); pdc_reg_write(IRQ_i_CFG, d->hwirq, pdc_type); ret = irq_chip_set_type_parent(d, type); @@ -246,7 +271,6 @@ static const struct irq_domain_ops qcom_pdc_ops = { static int pdc_setup_pin_mapping(struct device_node *np) { int ret, n, i; - u32 irq_index, reg_index, val; n = of_property_count_elems_of_size(np, "qcom,pdc-ranges", sizeof(u32)); if (n <= 0 || n % 3) @@ -276,29 +300,38 @@ static int pdc_setup_pin_mapping(struct device_node *np) if (ret) return ret; - for (i = 0; i < pdc_region[n].cnt; i++) { - reg_index = (i + pdc_region[n].pin_base) >> 5; - irq_index = (i + pdc_region[n].pin_base) & 0x1f; - val = pdc_reg_read(IRQ_ENABLE_BANK, reg_index); - val &= ~BIT(irq_index); - pdc_reg_write(IRQ_ENABLE_BANK, reg_index, val); - } + for (i = 0; i < pdc_region[n].cnt; i++) + __pdc_enable_intr(i + pdc_region[n].pin_base, 0); } return 0; } +#define QCOM_PDC_SIZE 0x30000 + static int qcom_pdc_init(struct device_node *node, struct device_node *parent) { struct irq_domain *parent_domain, *pdc_domain; + resource_size_t res_size; + struct resource res; int ret; - pdc_base = of_iomap(node, 0); + /* compat with old sm8150 DT which had very small region for PDC */ + if (of_address_to_resource(node, 0, &res)) + return -EINVAL; + + res_size = max_t(resource_size_t, resource_size(&res), QCOM_PDC_SIZE); + if (res_size > resource_size(&res)) + pr_warn("%pOF: invalid reg size, please fix DT\n", node); + + pdc_base = ioremap(res.start, res_size); if (!pdc_base) { pr_err("%pOF: unable to map PDC registers\n", node); return -ENXIO; } + pdc_version = pdc_reg_read(PDC_VERSION_REG, 0); + parent_domain = irq_find_host(parent); if (!parent_domain) { pr_err("%pOF: unable to find PDC's parent domain\n", node); diff --git a/drivers/leds/led-core.c b/drivers/leds/led-core.c index 04f9ea675f2c..214ed81eb0e9 100644 --- a/drivers/leds/led-core.c +++ b/drivers/leds/led-core.c @@ -479,10 +479,6 @@ int led_compose_name(struct device *dev, struct led_init_data *init_data, led_parse_fwnode_props(dev, fwnode, &props); - /* We want to label LEDs that can produce full range of colors - * as RGB, not multicolor */ - BUG_ON(props.color == LED_COLOR_ID_MULTI); - if (props.label) { /* * If init_data.devicename is NULL, then it indicates that diff --git a/drivers/mcb/mcb-core.c b/drivers/mcb/mcb-core.c index 978fdfc19a06..0cac5bead84f 100644 --- a/drivers/mcb/mcb-core.c +++ b/drivers/mcb/mcb-core.c @@ -387,17 +387,13 @@ EXPORT_SYMBOL_NS_GPL(mcb_free_dev, MCB); static int __mcb_bus_add_devices(struct device *dev, void *data) { - struct mcb_device *mdev = to_mcb_device(dev); int retval; - if (mdev->is_added) - return 0; - retval = device_attach(dev); - if (retval < 0) + if (retval < 0) { dev_err(dev, "Error adding device (%d)\n", retval); - - mdev->is_added = true; + return retval; + } return 0; } diff --git a/drivers/mcb/mcb-parse.c b/drivers/mcb/mcb-parse.c index 2aef990f379f..656b6b71c768 100644 --- a/drivers/mcb/mcb-parse.c +++ b/drivers/mcb/mcb-parse.c @@ -99,8 +99,6 @@ static int chameleon_parse_gdd(struct mcb_bus *bus, mdev->mem.end = mdev->mem.start + size - 1; mdev->mem.flags = IORESOURCE_MEM; - mdev->is_added = false; - ret = mcb_device_register(bus, mdev); if (ret < 0) goto err; diff --git a/drivers/md/dm-core.h b/drivers/md/dm-core.h index 0d93661f88d3..095b9b49aa82 100644 --- a/drivers/md/dm-core.h +++ b/drivers/md/dm-core.h @@ -214,6 +214,7 @@ struct dm_table { /* a list of devices used by this table */ struct list_head devices; + struct rw_semaphore devices_lock; /* events get handed up using this callback */ void (*event_fn)(void *data); diff --git a/drivers/md/dm-crypt.c b/drivers/md/dm-crypt.c index f2662c21a6df..5315fd261c23 100644 --- a/drivers/md/dm-crypt.c +++ b/drivers/md/dm-crypt.c @@ -753,7 +753,8 @@ static int crypt_iv_eboiv_gen(struct crypt_config *cc, u8 *iv, int err; u8 *buf; - reqsize = ALIGN(crypto_skcipher_reqsize(tfm), __alignof__(__le64)); + reqsize = sizeof(*req) + crypto_skcipher_reqsize(tfm); + reqsize = ALIGN(reqsize, __alignof__(__le64)); req = kmalloc(reqsize + cc->iv_size, GFP_NOIO); if (!req) diff --git a/drivers/md/dm-ioctl.c b/drivers/md/dm-ioctl.c index f5ed729a8e0c..21ebb6c39394 100644 --- a/drivers/md/dm-ioctl.c +++ b/drivers/md/dm-ioctl.c @@ -1630,6 +1630,8 @@ static void retrieve_deps(struct dm_table *table, struct dm_dev_internal *dd; struct dm_target_deps *deps; + down_read(&table->devices_lock); + deps = get_result_buffer(param, param_size, &len); /* @@ -1644,7 +1646,7 @@ static void retrieve_deps(struct dm_table *table, needed = struct_size(deps, dev, count); if (len < needed) { param->flags |= DM_BUFFER_FULL_FLAG; - return; + goto out; } /* @@ -1656,6 +1658,9 @@ static void retrieve_deps(struct dm_table *table, deps->dev[count++] = huge_encode_dev(dd->dm_dev->bdev->bd_dev); param->data_size = param->data_start + needed; + +out: + up_read(&table->devices_lock); } static int table_deps(struct file *filp, struct dm_ioctl *param, size_t param_size) diff --git a/drivers/md/dm-table.c b/drivers/md/dm-table.c index 7d208b2b1a19..37b48f63ae6a 100644 --- a/drivers/md/dm-table.c +++ b/drivers/md/dm-table.c @@ -135,6 +135,7 @@ int dm_table_create(struct dm_table **result, blk_mode_t mode, return -ENOMEM; INIT_LIST_HEAD(&t->devices); + init_rwsem(&t->devices_lock); if (!num_targets) num_targets = KEYS_PER_NODE; @@ -359,16 +360,20 @@ int __ref dm_get_device(struct dm_target *ti, const char *path, blk_mode_t mode, if (dev == disk_devt(t->md->disk)) return -EINVAL; + down_write(&t->devices_lock); + dd = find_device(&t->devices, dev); if (!dd) { dd = kmalloc(sizeof(*dd), GFP_KERNEL); - if (!dd) - return -ENOMEM; + if (!dd) { + r = -ENOMEM; + goto unlock_ret_r; + } r = dm_get_table_device(t->md, dev, mode, &dd->dm_dev); if (r) { kfree(dd); - return r; + goto unlock_ret_r; } refcount_set(&dd->count, 1); @@ -378,12 +383,17 @@ int __ref dm_get_device(struct dm_target *ti, const char *path, blk_mode_t mode, } else if (dd->dm_dev->mode != (mode | dd->dm_dev->mode)) { r = upgrade_mode(dd, mode, t->md); if (r) - return r; + goto unlock_ret_r; } refcount_inc(&dd->count); out: + up_write(&t->devices_lock); *result = dd->dm_dev; return 0; + +unlock_ret_r: + up_write(&t->devices_lock); + return r; } EXPORT_SYMBOL(dm_get_device); @@ -419,9 +429,12 @@ static int dm_set_device_limits(struct dm_target *ti, struct dm_dev *dev, void dm_put_device(struct dm_target *ti, struct dm_dev *d) { int found = 0; - struct list_head *devices = &ti->table->devices; + struct dm_table *t = ti->table; + struct list_head *devices = &t->devices; struct dm_dev_internal *dd; + down_write(&t->devices_lock); + list_for_each_entry(dd, devices, list) { if (dd->dm_dev == d) { found = 1; @@ -430,14 +443,17 @@ void dm_put_device(struct dm_target *ti, struct dm_dev *d) } if (!found) { DMERR("%s: device %s not in table devices list", - dm_device_name(ti->table->md), d->name); - return; + dm_device_name(t->md), d->name); + goto unlock_ret; } if (refcount_dec_and_test(&dd->count)) { - dm_put_table_device(ti->table->md, d); + dm_put_table_device(t->md, d); list_del(&dd->list); kfree(dd); } + +unlock_ret: + up_write(&t->devices_lock); } EXPORT_SYMBOL(dm_put_device); diff --git a/drivers/md/dm-zoned-target.c b/drivers/md/dm-zoned-target.c index ad8e670a2f9b..b487f7acc860 100644 --- a/drivers/md/dm-zoned-target.c +++ b/drivers/md/dm-zoned-target.c @@ -748,17 +748,16 @@ err: /* * Cleanup zoned device information. */ -static void dmz_put_zoned_device(struct dm_target *ti) +static void dmz_put_zoned_devices(struct dm_target *ti) { struct dmz_target *dmz = ti->private; int i; - for (i = 0; i < dmz->nr_ddevs; i++) { - if (dmz->ddev[i]) { + for (i = 0; i < dmz->nr_ddevs; i++) + if (dmz->ddev[i]) dm_put_device(ti, dmz->ddev[i]); - dmz->ddev[i] = NULL; - } - } + + kfree(dmz->ddev); } static int dmz_fixup_devices(struct dm_target *ti) @@ -948,7 +947,7 @@ err_bio: err_meta: dmz_dtr_metadata(dmz->metadata); err_dev: - dmz_put_zoned_device(ti); + dmz_put_zoned_devices(ti); err: kfree(dmz->dev); kfree(dmz); @@ -978,7 +977,7 @@ static void dmz_dtr(struct dm_target *ti) bioset_exit(&dmz->bio_set); - dmz_put_zoned_device(ti); + dmz_put_zoned_devices(ti); mutex_destroy(&dmz->chunk_lock); diff --git a/drivers/md/dm.c b/drivers/md/dm.c index f0f118ab20fa..64a1f306c96c 100644 --- a/drivers/md/dm.c +++ b/drivers/md/dm.c @@ -715,24 +715,6 @@ static void dm_put_live_table_fast(struct mapped_device *md) __releases(RCU) rcu_read_unlock(); } -static inline struct dm_table *dm_get_live_table_bio(struct mapped_device *md, - int *srcu_idx, blk_opf_t bio_opf) -{ - if (bio_opf & REQ_NOWAIT) - return dm_get_live_table_fast(md); - else - return dm_get_live_table(md, srcu_idx); -} - -static inline void dm_put_live_table_bio(struct mapped_device *md, int srcu_idx, - blk_opf_t bio_opf) -{ - if (bio_opf & REQ_NOWAIT) - dm_put_live_table_fast(md); - else - dm_put_live_table(md, srcu_idx); -} - static char *_dm_claim_ptr = "I belong to device-mapper"; /* @@ -1833,9 +1815,8 @@ static void dm_submit_bio(struct bio *bio) struct mapped_device *md = bio->bi_bdev->bd_disk->private_data; int srcu_idx; struct dm_table *map; - blk_opf_t bio_opf = bio->bi_opf; - map = dm_get_live_table_bio(md, &srcu_idx, bio_opf); + map = dm_get_live_table(md, &srcu_idx); /* If suspended, or map not yet available, queue this IO for later */ if (unlikely(test_bit(DMF_BLOCK_IO_FOR_SUSPEND, &md->flags)) || @@ -1851,7 +1832,7 @@ static void dm_submit_bio(struct bio *bio) dm_split_and_process_bio(md, map, bio); out: - dm_put_live_table_bio(md, srcu_idx, bio_opf); + dm_put_live_table(md, srcu_idx); } static bool dm_poll_dm_io(struct dm_io *io, struct io_comp_batch *iob, diff --git a/drivers/md/md.c b/drivers/md/md.c index 0fe7ab6e8ab9..a104a025084d 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -798,14 +798,14 @@ void mddev_unlock(struct mddev *mddev) } else mutex_unlock(&mddev->reconfig_mutex); + md_wakeup_thread(mddev->thread); + wake_up(&mddev->sb_wait); + list_for_each_entry_safe(rdev, tmp, &delete, same_set) { list_del_init(&rdev->same_set); kobject_del(&rdev->kobj); export_rdev(rdev, mddev); } - - md_wakeup_thread(mddev->thread); - wake_up(&mddev->sb_wait); } EXPORT_SYMBOL_GPL(mddev_unlock); @@ -2452,7 +2452,8 @@ static void export_rdev(struct md_rdev *rdev, struct mddev *mddev) if (test_bit(AutoDetected, &rdev->flags)) md_autodetect_dev(rdev->bdev->bd_dev); #endif - blkdev_put(rdev->bdev, mddev->external ? &claim_rdev : rdev); + blkdev_put(rdev->bdev, + test_bit(Holder, &rdev->flags) ? rdev : &claim_rdev); rdev->bdev = NULL; kobject_put(&rdev->kobj); } @@ -3632,6 +3633,7 @@ EXPORT_SYMBOL_GPL(md_rdev_init); static struct md_rdev *md_import_device(dev_t newdev, int super_format, int super_minor) { struct md_rdev *rdev; + struct md_rdev *holder; sector_t size; int err; @@ -3646,8 +3648,15 @@ static struct md_rdev *md_import_device(dev_t newdev, int super_format, int supe if (err) goto out_clear_rdev; + if (super_format == -2) { + holder = &claim_rdev; + } else { + holder = rdev; + set_bit(Holder, &rdev->flags); + } + rdev->bdev = blkdev_get_by_dev(newdev, BLK_OPEN_READ | BLK_OPEN_WRITE, - super_format == -2 ? &claim_rdev : rdev, NULL); + holder, NULL); if (IS_ERR(rdev->bdev)) { pr_warn("md: could not open device unknown-block(%u,%u).\n", MAJOR(newdev), MINOR(newdev)); @@ -3684,7 +3693,7 @@ static struct md_rdev *md_import_device(dev_t newdev, int super_format, int supe return rdev; out_blkdev_put: - blkdev_put(rdev->bdev, super_format == -2 ? &claim_rdev : rdev); + blkdev_put(rdev->bdev, holder); out_clear_rdev: md_rdev_clear(rdev); out_free_rdev: @@ -8256,7 +8265,7 @@ static void *md_seq_next(struct seq_file *seq, void *v, loff_t *pos) spin_unlock(&all_mddevs_lock); if (to_put) - mddev_put(mddev); + mddev_put(to_put); return next_mddev; } diff --git a/drivers/md/md.h b/drivers/md/md.h index 9bcb77bca963..7c9c13abd7ca 100644 --- a/drivers/md/md.h +++ b/drivers/md/md.h @@ -211,6 +211,9 @@ enum flag_bits { * check if there is collision between raid1 * serial bios. */ + Holder, /* rdev is used as holder while opening + * underlying disk exclusively. + */ }; static inline int is_badblock(struct md_rdev *rdev, sector_t s, int sectors, diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c index 4b30a1742162..2aabac773fe7 100644 --- a/drivers/md/raid1.c +++ b/drivers/md/raid1.c @@ -1837,12 +1837,11 @@ static int raid1_remove_disk(struct mddev *mddev, struct md_rdev *rdev) struct r1conf *conf = mddev->private; int err = 0; int number = rdev->raid_disk; + struct raid1_info *p = conf->mirrors + number; if (unlikely(number >= conf->raid_disks)) goto abort; - struct raid1_info *p = conf->mirrors + number; - if (rdev != p->rdev) p = conf->mirrors + conf->raid_disks + number; diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c index 4cb9c608ee19..284cd71bcc68 100644 --- a/drivers/md/raid5.c +++ b/drivers/md/raid5.c @@ -854,6 +854,13 @@ struct stripe_head *raid5_get_active_stripe(struct r5conf *conf, set_bit(R5_INACTIVE_BLOCKED, &conf->cache_state); r5l_wake_reclaim(conf->log, 0); + + /* release batch_last before wait to avoid risk of deadlock */ + if (ctx && ctx->batch_last) { + raid5_release_stripe(ctx->batch_last); + ctx->batch_last = NULL; + } + wait_event_lock_irq(conf->wait_for_stripe, is_inactive_blocked(conf, hash), *(conf->hash_locks + hash)); diff --git a/drivers/media/common/videobuf2/frame_vector.c b/drivers/media/common/videobuf2/frame_vector.c index 0f430ddc1f67..fd87747be9b1 100644 --- a/drivers/media/common/videobuf2/frame_vector.c +++ b/drivers/media/common/videobuf2/frame_vector.c @@ -31,6 +31,10 @@ * different type underlying the specified range of virtual addresses. * When the function isn't able to map a single page, it returns error. * + * Note that get_vaddr_frames() cannot follow VM_IO mappings. It used + * to be able to do that, but that could (racily) return non-refcounted + * pfns. + * * This function takes care of grabbing mmap_lock as necessary. */ int get_vaddr_frames(unsigned long start, unsigned int nr_frames, bool write, @@ -59,8 +63,6 @@ int get_vaddr_frames(unsigned long start, unsigned int nr_frames, bool write, if (likely(ret > 0)) return ret; - /* This used to (racily) return non-refcounted pfns. Let people know */ - WARN_ONCE(1, "get_vaddr_frames() cannot follow VM_IO mapping"); vec->nr_frames = 0; return ret ? ret : -EFAULT; } diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c index a1136fdfbed2..ec53abe2e84e 100644 --- a/drivers/media/i2c/imx219.c +++ b/drivers/media/i2c/imx219.c @@ -691,12 +691,12 @@ static int imx219_init_cfg(struct v4l2_subdev *sd, struct v4l2_mbus_framefmt *format; struct v4l2_rect *crop; - /* Initialize try_fmt */ + /* Initialize the format. */ format = v4l2_subdev_get_pad_format(sd, state, 0); imx219_update_pad_format(imx219, &supported_modes[0], format, MEDIA_BUS_FMT_SRGGB10_1X10); - /* Initialize crop rectangle. */ + /* Initialize the crop rectangle. */ crop = v4l2_subdev_get_pad_crop(sd, state, 0); crop->top = IMX219_PIXEL_ARRAY_TOP; crop->left = IMX219_PIXEL_ARRAY_LEFT; @@ -750,6 +750,7 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd, const struct imx219_mode *mode; int exposure_max, exposure_def, hblank; struct v4l2_mbus_framefmt *format; + struct v4l2_rect *crop; mode = v4l2_find_nearest_size(supported_modes, ARRAY_SIZE(supported_modes), @@ -757,10 +758,12 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd, fmt->format.width, fmt->format.height); imx219_update_pad_format(imx219, mode, &fmt->format, fmt->format.code); + format = v4l2_subdev_get_pad_format(sd, sd_state, 0); + crop = v4l2_subdev_get_pad_crop(sd, sd_state, 0); - if (imx219->mode == mode && format->code == fmt->format.code) - return 0; + *format = fmt->format; + *crop = mode->crop; if (fmt->which == V4L2_SUBDEV_FORMAT_ACTIVE) { imx219->mode = mode; @@ -788,8 +791,6 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd, hblank); } - *format = fmt->format; - return 0; } diff --git a/drivers/media/i2c/max9286.c b/drivers/media/i2c/max9286.c index 20e7c7cf5eeb..be84ff1e2b17 100644 --- a/drivers/media/i2c/max9286.c +++ b/drivers/media/i2c/max9286.c @@ -1110,7 +1110,6 @@ err_async: static void max9286_v4l2_unregister(struct max9286_priv *priv) { - fwnode_handle_put(priv->sd.fwnode); v4l2_ctrl_handler_free(&priv->ctrls); v4l2_async_unregister_subdev(&priv->sd); max9286_v4l2_notifier_unregister(priv); diff --git a/drivers/media/i2c/ov8858.c b/drivers/media/i2c/ov8858.c index 3af6125a2eee..4d9fd76e2f60 100644 --- a/drivers/media/i2c/ov8858.c +++ b/drivers/media/i2c/ov8858.c @@ -1850,9 +1850,9 @@ static int ov8858_parse_of(struct ov8858 *ov8858) } ret = v4l2_fwnode_endpoint_parse(endpoint, &vep); + fwnode_handle_put(endpoint); if (ret) { dev_err(dev, "Failed to parse endpoint: %d\n", ret); - fwnode_handle_put(endpoint); return ret; } @@ -1864,12 +1864,9 @@ static int ov8858_parse_of(struct ov8858 *ov8858) default: dev_err(dev, "Unsupported number of data lanes %u\n", ov8858->num_lanes); - fwnode_handle_put(endpoint); return -EINVAL; } - ov8858->subdev.fwnode = endpoint; - return 0; } @@ -1913,7 +1910,7 @@ static int ov8858_probe(struct i2c_client *client) ret = ov8858_init_ctrls(ov8858); if (ret) - goto err_put_fwnode; + return ret; sd = &ov8858->subdev; sd->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE | V4L2_SUBDEV_FL_HAS_EVENTS; @@ -1964,8 +1961,6 @@ err_clean_entity: media_entity_cleanup(&sd->entity); err_free_handler: v4l2_ctrl_handler_free(&ov8858->ctrl_handler); -err_put_fwnode: - fwnode_handle_put(ov8858->subdev.fwnode); return ret; } @@ -1978,7 +1973,6 @@ static void ov8858_remove(struct i2c_client *client) v4l2_async_unregister_subdev(sd); media_entity_cleanup(&sd->entity); v4l2_ctrl_handler_free(&ov8858->ctrl_handler); - fwnode_handle_put(ov8858->subdev.fwnode); pm_runtime_disable(&client->dev); if (!pm_runtime_status_suspended(&client->dev)) diff --git a/drivers/media/i2c/rdacm21.c b/drivers/media/i2c/rdacm21.c index a36a709243fd..3e22df36354f 100644 --- a/drivers/media/i2c/rdacm21.c +++ b/drivers/media/i2c/rdacm21.c @@ -608,7 +608,6 @@ static void rdacm21_remove(struct i2c_client *client) v4l2_async_unregister_subdev(&dev->sd); v4l2_ctrl_handler_free(&dev->ctrls); i2c_unregister_device(dev->isp); - fwnode_handle_put(dev->sd.fwnode); } static const struct of_device_id rdacm21_of_ids[] = { diff --git a/drivers/media/pci/bt8xx/bttv-risc.c b/drivers/media/pci/bt8xx/bttv-risc.c index 436baf6c8b08..241a696e374a 100644 --- a/drivers/media/pci/bt8xx/bttv-risc.c +++ b/drivers/media/pci/bt8xx/bttv-risc.c @@ -68,9 +68,7 @@ bttv_risc_packed(struct bttv *btv, struct btcx_riscmem *risc, sg = sglist; for (line = 0; line < store_lines; line++) { if ((line >= (store_lines - VCR_HACK_LINES)) && - (btv->opt_vcr_hack || - (V4L2_FIELD_HAS_BOTH(btv->field) || - btv->field == V4L2_FIELD_ALTERNATE))) + btv->opt_vcr_hack) continue; while (offset && offset >= sg_dma_len(sg)) { offset -= sg_dma_len(sg); diff --git a/drivers/media/pci/intel/Kconfig b/drivers/media/pci/intel/Kconfig index e113902fa806..ee4684159d3d 100644 --- a/drivers/media/pci/intel/Kconfig +++ b/drivers/media/pci/intel/Kconfig @@ -1,11 +1,19 @@ # SPDX-License-Identifier: GPL-2.0-only + +source "drivers/media/pci/intel/ipu3/Kconfig" +source "drivers/media/pci/intel/ivsc/Kconfig" + config IPU_BRIDGE - tristate + tristate "Intel IPU Bridge" depends on I2C && ACPI help - This is a helper module for the IPU bridge, which can be - used by ipu3 and other drivers. In order to handle module - dependencies, this is selected by each driver that needs it. + The IPU bridge is a helper library for Intel IPU drivers to + function on systems shipped with Windows. -source "drivers/media/pci/intel/ipu3/Kconfig" -source "drivers/media/pci/intel/ivsc/Kconfig" + Currently used by the ipu3-cio2 and atomisp drivers. + + Supported systems include: + + - Microsoft Surface models (except Surface Pro 3) + - The Lenovo Miix line (for example the 510, 520, 710 and 720) + - Dell 7285 diff --git a/drivers/media/pci/intel/ipu-bridge.c b/drivers/media/pci/intel/ipu-bridge.c index 1bde8b6e0b11..e38198e259c0 100644 --- a/drivers/media/pci/intel/ipu-bridge.c +++ b/drivers/media/pci/intel/ipu-bridge.c @@ -107,8 +107,10 @@ static struct acpi_device *ipu_bridge_get_ivsc_acpi_dev(struct acpi_device *adev for_each_acpi_dev_match(ivsc_adev, acpi_id->id, NULL, -1) /* camera sensor depends on IVSC in DSDT if exist */ for_each_acpi_consumer_dev(ivsc_adev, consumer) - if (consumer->handle == handle) + if (consumer->handle == handle) { + acpi_dev_put(consumer); return ivsc_adev; + } } return NULL; diff --git a/drivers/media/pci/intel/ipu3/Kconfig b/drivers/media/pci/intel/ipu3/Kconfig index 0951545eab21..c0a250daa927 100644 --- a/drivers/media/pci/intel/ipu3/Kconfig +++ b/drivers/media/pci/intel/ipu3/Kconfig @@ -2,13 +2,13 @@ config VIDEO_IPU3_CIO2 tristate "Intel ipu3-cio2 driver" depends on VIDEO_DEV && PCI + depends on IPU_BRIDGE || !IPU_BRIDGE depends on ACPI || COMPILE_TEST depends on X86 select MEDIA_CONTROLLER select VIDEO_V4L2_SUBDEV_API select V4L2_FWNODE select VIDEOBUF2_DMA_SG - select IPU_BRIDGE if CIO2_BRIDGE help This is the Intel IPU3 CIO2 CSI-2 receiver unit, found in Intel @@ -18,22 +18,3 @@ config VIDEO_IPU3_CIO2 Say Y or M here if you have a Skylake/Kaby Lake SoC with MIPI CSI-2 connected camera. The module will be called ipu3-cio2. - -config CIO2_BRIDGE - bool "IPU3 CIO2 Sensors Bridge" - depends on VIDEO_IPU3_CIO2 && ACPI - depends on I2C - help - This extension provides an API for the ipu3-cio2 driver to create - connections to cameras that are hidden in the SSDB buffer in ACPI. - It can be used to enable support for cameras in detachable / hybrid - devices that ship with Windows. - - Say Y here if your device is a detachable / hybrid laptop that comes - with Windows installed by the OEM, for example: - - - Microsoft Surface models (except Surface Pro 3) - - The Lenovo Miix line (for example the 510, 520, 710 and 720) - - Dell 7285 - - If in doubt, say N here. diff --git a/drivers/media/pci/intel/ivsc/Kconfig b/drivers/media/pci/intel/ivsc/Kconfig index 1ef1c4e3750d..a8cb981544f7 100644 --- a/drivers/media/pci/intel/ivsc/Kconfig +++ b/drivers/media/pci/intel/ivsc/Kconfig @@ -3,7 +3,10 @@ config INTEL_VSC tristate "Intel Visual Sensing Controller" - depends on INTEL_MEI && ACPI + depends on INTEL_MEI && ACPI && VIDEO_DEV + select MEDIA_CONTROLLER + select VIDEO_V4L2_SUBDEV_API + select V4L2_FWNODE help This adds support for Intel Visual Sensing Controller (IVSC). diff --git a/drivers/media/platform/intel/pxa_camera.c b/drivers/media/platform/intel/pxa_camera.c index 6e6caf50e11e..59b89e421dc2 100644 --- a/drivers/media/platform/intel/pxa_camera.c +++ b/drivers/media/platform/intel/pxa_camera.c @@ -2398,7 +2398,7 @@ static int pxa_camera_probe(struct platform_device *pdev) PXA_CAM_DRV_NAME, pcdev); if (err) { dev_err(&pdev->dev, "Camera interrupt register failed\n"); - goto exit_v4l2_device_unregister; + goto exit_deactivate; } pcdev->notifier.ops = &pxa_camera_sensor_ops; diff --git a/drivers/media/platform/marvell/Kconfig b/drivers/media/platform/marvell/Kconfig index ec1a16734a28..d6499ffe30e8 100644 --- a/drivers/media/platform/marvell/Kconfig +++ b/drivers/media/platform/marvell/Kconfig @@ -7,7 +7,7 @@ config VIDEO_CAFE_CCIC depends on V4L_PLATFORM_DRIVERS depends on PCI && I2C && VIDEO_DEV depends on COMMON_CLK - select VIDEO_OV7670 + select VIDEO_OV7670 if MEDIA_SUBDRV_AUTOSELECT && VIDEO_CAMERA_SENSOR select VIDEOBUF2_VMALLOC select VIDEOBUF2_DMA_CONTIG select VIDEOBUF2_DMA_SG @@ -22,7 +22,7 @@ config VIDEO_MMP_CAMERA depends on I2C && VIDEO_DEV depends on ARCH_MMP || COMPILE_TEST depends on COMMON_CLK - select VIDEO_OV7670 + select VIDEO_OV7670 if MEDIA_SUBDRV_AUTOSELECT && VIDEO_CAMERA_SENSOR select I2C_GPIO select VIDEOBUF2_VMALLOC select VIDEOBUF2_DMA_CONTIG diff --git a/drivers/media/platform/mediatek/vcodec/encoder/venc_vpu_if.c b/drivers/media/platform/mediatek/vcodec/encoder/venc_vpu_if.c index d299cc2962a5..ae6290d28f8e 100644 --- a/drivers/media/platform/mediatek/vcodec/encoder/venc_vpu_if.c +++ b/drivers/media/platform/mediatek/vcodec/encoder/venc_vpu_if.c @@ -138,7 +138,8 @@ int vpu_enc_init(struct venc_vpu_inst *vpu) vpu->ctx->vpu_inst = vpu; status = mtk_vcodec_fw_ipi_register(vpu->ctx->dev->fw_handler, vpu->id, - vpu_enc_ipi_handler, "venc", NULL); + vpu_enc_ipi_handler, "venc", + vpu->ctx->dev); if (status) { mtk_venc_err(vpu->ctx, "vpu_ipi_register fail %d", status); diff --git a/drivers/media/platform/nxp/imx-mipi-csis.c b/drivers/media/platform/nxp/imx-mipi-csis.c index 16f19a640130..5f93712bf485 100644 --- a/drivers/media/platform/nxp/imx-mipi-csis.c +++ b/drivers/media/platform/nxp/imx-mipi-csis.c @@ -1490,7 +1490,6 @@ err_cleanup: v4l2_async_unregister_subdev(&csis->sd); err_disable_clock: mipi_csis_clk_disable(csis); - fwnode_handle_put(csis->sd.fwnode); return ret; } @@ -1510,7 +1509,6 @@ static void mipi_csis_remove(struct platform_device *pdev) mipi_csis_clk_disable(csis); v4l2_subdev_cleanup(&csis->sd); media_entity_cleanup(&csis->sd.entity); - fwnode_handle_put(csis->sd.fwnode); pm_runtime_set_suspended(&pdev->dev); } diff --git a/drivers/media/platform/via/Kconfig b/drivers/media/platform/via/Kconfig index 8926eb0803b2..6e603c038248 100644 --- a/drivers/media/platform/via/Kconfig +++ b/drivers/media/platform/via/Kconfig @@ -7,7 +7,7 @@ config VIDEO_VIA_CAMERA depends on V4L_PLATFORM_DRIVERS depends on FB_VIA && VIDEO_DEV select VIDEOBUF2_DMA_SG - select VIDEO_OV7670 + select VIDEO_OV7670 if VIDEO_CAMERA_SENSOR help Driver support for the integrated camera controller in VIA Chrome9 chipsets. Currently only tested on OLPC xo-1.5 systems diff --git a/drivers/media/platform/xilinx/xilinx-vipp.c b/drivers/media/platform/xilinx/xilinx-vipp.c index 4285770fde18..996684a73038 100644 --- a/drivers/media/platform/xilinx/xilinx-vipp.c +++ b/drivers/media/platform/xilinx/xilinx-vipp.c @@ -55,11 +55,18 @@ xvip_graph_find_entity(struct xvip_composite_device *xdev, { struct xvip_graph_entity *entity; struct v4l2_async_connection *asd; - - list_for_each_entry(asd, &xdev->notifier.done_list, asc_entry) { - entity = to_xvip_entity(asd); - if (entity->asd.match.fwnode == fwnode) - return entity; + struct list_head *lists[] = { + &xdev->notifier.done_list, + &xdev->notifier.waiting_list + }; + unsigned int i; + + for (i = 0; i < ARRAY_SIZE(lists); i++) { + list_for_each_entry(asd, lists[i], asc_entry) { + entity = to_xvip_entity(asd); + if (entity->asd.match.fwnode == fwnode) + return entity; + } } return NULL; diff --git a/drivers/media/usb/em28xx/Kconfig b/drivers/media/usb/em28xx/Kconfig index b3c472b8c5a9..cb61fd6cc6c6 100644 --- a/drivers/media/usb/em28xx/Kconfig +++ b/drivers/media/usb/em28xx/Kconfig @@ -12,8 +12,8 @@ config VIDEO_EM28XX_V4L2 select VIDEO_SAA711X if MEDIA_SUBDRV_AUTOSELECT select VIDEO_TVP5150 if MEDIA_SUBDRV_AUTOSELECT select VIDEO_MSP3400 if MEDIA_SUBDRV_AUTOSELECT - select VIDEO_MT9V011 if MEDIA_SUBDRV_AUTOSELECT && MEDIA_CAMERA_SUPPORT - select VIDEO_OV2640 if MEDIA_SUBDRV_AUTOSELECT && MEDIA_CAMERA_SUPPORT + select VIDEO_MT9V011 if MEDIA_SUBDRV_AUTOSELECT && VIDEO_CAMERA_SENSOR + select VIDEO_OV2640 if MEDIA_SUBDRV_AUTOSELECT && VIDEO_CAMERA_SENSOR help This is a video4linux driver for Empia 28xx based TV cards. diff --git a/drivers/media/usb/go7007/Kconfig b/drivers/media/usb/go7007/Kconfig index 4ff79940ad8d..b2a15d9fb1f3 100644 --- a/drivers/media/usb/go7007/Kconfig +++ b/drivers/media/usb/go7007/Kconfig @@ -12,8 +12,8 @@ config VIDEO_GO7007 select VIDEO_TW2804 if MEDIA_SUBDRV_AUTOSELECT select VIDEO_TW9903 if MEDIA_SUBDRV_AUTOSELECT select VIDEO_TW9906 if MEDIA_SUBDRV_AUTOSELECT - select VIDEO_OV7640 if MEDIA_SUBDRV_AUTOSELECT && MEDIA_CAMERA_SUPPORT select VIDEO_UDA1342 if MEDIA_SUBDRV_AUTOSELECT + select VIDEO_OV7640 if MEDIA_SUBDRV_AUTOSELECT && VIDEO_CAMERA_SENSOR help This is a video4linux driver for the WIS GO7007 MPEG encoder chip. diff --git a/drivers/media/usb/uvc/uvc_ctrl.c b/drivers/media/usb/uvc/uvc_ctrl.c index 5e9d3da862dd..e59a463c2761 100644 --- a/drivers/media/usb/uvc/uvc_ctrl.c +++ b/drivers/media/usb/uvc/uvc_ctrl.c @@ -1402,6 +1402,9 @@ int uvc_query_v4l2_menu(struct uvc_video_chain *chain, query_menu->id = id; query_menu->index = index; + if (index >= BITS_PER_TYPE(mapping->menu_mask)) + return -EINVAL; + ret = mutex_lock_interruptible(&chain->ctrl_mutex); if (ret < 0) return -ERESTARTSYS; diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c index b92348ad61f6..31752c06d1f0 100644 --- a/drivers/media/v4l2-core/v4l2-subdev.c +++ b/drivers/media/v4l2-core/v4l2-subdev.c @@ -502,6 +502,13 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg, V4L2_SUBDEV_CLIENT_CAP_STREAMS; int rval; + /* + * If the streams API is not enabled, remove V4L2_SUBDEV_CAP_STREAMS. + * Remove this when the API is no longer experimental. + */ + if (!v4l2_subdev_enable_streams_api) + streams_subdev = false; + switch (cmd) { case VIDIOC_SUBDEV_QUERYCAP: { struct v4l2_subdev_capability *cap = arg; diff --git a/drivers/mfd/Kconfig b/drivers/mfd/Kconfig index aea95745c73f..90ce58fd629e 100644 --- a/drivers/mfd/Kconfig +++ b/drivers/mfd/Kconfig @@ -241,6 +241,7 @@ config MFD_CS42L43 tristate select MFD_CORE select REGMAP + select REGMAP_IRQ config MFD_CS42L43_I2C tristate "Cirrus Logic CS42L43 (I2C)" diff --git a/drivers/mfd/cs42l43.c b/drivers/mfd/cs42l43.c index 37b23e9bae82..7b6d07cbe6fc 100644 --- a/drivers/mfd/cs42l43.c +++ b/drivers/mfd/cs42l43.c @@ -1178,8 +1178,8 @@ err: } EXPORT_NS_GPL_DEV_PM_OPS(cs42l43_pm_ops, MFD_CS42L43) = { - SET_SYSTEM_SLEEP_PM_OPS(cs42l43_suspend, cs42l43_resume) - SET_RUNTIME_PM_OPS(cs42l43_runtime_suspend, cs42l43_runtime_resume, NULL) + SYSTEM_SLEEP_PM_OPS(cs42l43_suspend, cs42l43_resume) + RUNTIME_PM_OPS(cs42l43_runtime_suspend, cs42l43_runtime_resume, NULL) }; MODULE_DESCRIPTION("CS42L43 Core Driver"); diff --git a/drivers/misc/cardreader/rts5227.c b/drivers/misc/cardreader/rts5227.c index 3dae5e3a1697..cd512284bfb3 100644 --- a/drivers/misc/cardreader/rts5227.c +++ b/drivers/misc/cardreader/rts5227.c @@ -83,63 +83,20 @@ static void rts5227_fetch_vendor_settings(struct rtsx_pcr *pcr) static void rts5227_init_from_cfg(struct rtsx_pcr *pcr) { - struct pci_dev *pdev = pcr->pci; - int l1ss; - u32 lval; struct rtsx_cr_option *option = &pcr->option; - l1ss = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_L1SS); - if (!l1ss) - return; - - pci_read_config_dword(pdev, l1ss + PCI_L1SS_CTL1, &lval); - if (CHK_PCI_PID(pcr, 0x522A)) { - if (0 == (lval & 0x0F)) - rtsx_pci_enable_oobs_polling(pcr); - else + if (rtsx_check_dev_flag(pcr, ASPM_L1_1_EN | ASPM_L1_2_EN + | PM_L1_1_EN | PM_L1_2_EN)) rtsx_pci_disable_oobs_polling(pcr); + else + rtsx_pci_enable_oobs_polling(pcr); } - if (lval & PCI_L1SS_CTL1_ASPM_L1_1) - rtsx_set_dev_flag(pcr, ASPM_L1_1_EN); - else - rtsx_clear_dev_flag(pcr, ASPM_L1_1_EN); - - if (lval & PCI_L1SS_CTL1_ASPM_L1_2) - rtsx_set_dev_flag(pcr, ASPM_L1_2_EN); - else - rtsx_clear_dev_flag(pcr, ASPM_L1_2_EN); - - if (lval & PCI_L1SS_CTL1_PCIPM_L1_1) - rtsx_set_dev_flag(pcr, PM_L1_1_EN); - else - rtsx_clear_dev_flag(pcr, PM_L1_1_EN); - - if (lval & PCI_L1SS_CTL1_PCIPM_L1_2) - rtsx_set_dev_flag(pcr, PM_L1_2_EN); - else - rtsx_clear_dev_flag(pcr, PM_L1_2_EN); - if (option->ltr_en) { - u16 val; - - pcie_capability_read_word(pcr->pci, PCI_EXP_DEVCTL2, &val); - if (val & PCI_EXP_DEVCTL2_LTR_EN) { - option->ltr_enabled = true; - option->ltr_active = true; + if (option->ltr_enabled) rtsx_set_ltr_latency(pcr, option->ltr_active_latency); - } else { - option->ltr_enabled = false; - } } - - if (rtsx_check_dev_flag(pcr, ASPM_L1_1_EN | ASPM_L1_2_EN - | PM_L1_1_EN | PM_L1_2_EN)) - option->force_clkreq_0 = false; - else - option->force_clkreq_0 = true; - } static int rts5227_extra_init_hw(struct rtsx_pcr *pcr) @@ -195,7 +152,7 @@ static int rts5227_extra_init_hw(struct rtsx_pcr *pcr) } } - if (option->force_clkreq_0 && pcr->aspm_mode == ASPM_MODE_CFG) + if (option->force_clkreq_0) rtsx_pci_add_cmd(pcr, WRITE_REG_CMD, PETXCFG, FORCE_CLKREQ_DELINK_MASK, FORCE_CLKREQ_LOW); else diff --git a/drivers/misc/cardreader/rts5228.c b/drivers/misc/cardreader/rts5228.c index f4ab09439da7..0c7f10bcf6f1 100644 --- a/drivers/misc/cardreader/rts5228.c +++ b/drivers/misc/cardreader/rts5228.c @@ -386,59 +386,25 @@ static void rts5228_process_ocp(struct rtsx_pcr *pcr) static void rts5228_init_from_cfg(struct rtsx_pcr *pcr) { - struct pci_dev *pdev = pcr->pci; - int l1ss; - u32 lval; struct rtsx_cr_option *option = &pcr->option; - l1ss = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_L1SS); - if (!l1ss) - return; - - pci_read_config_dword(pdev, l1ss + PCI_L1SS_CTL1, &lval); - - if (0 == (lval & 0x0F)) - rtsx_pci_enable_oobs_polling(pcr); - else + if (rtsx_check_dev_flag(pcr, ASPM_L1_1_EN | ASPM_L1_2_EN + | PM_L1_1_EN | PM_L1_2_EN)) rtsx_pci_disable_oobs_polling(pcr); - - if (lval & PCI_L1SS_CTL1_ASPM_L1_1) - rtsx_set_dev_flag(pcr, ASPM_L1_1_EN); - else - rtsx_clear_dev_flag(pcr, ASPM_L1_1_EN); - - if (lval & PCI_L1SS_CTL1_ASPM_L1_2) - rtsx_set_dev_flag(pcr, ASPM_L1_2_EN); - else - rtsx_clear_dev_flag(pcr, ASPM_L1_2_EN); - - if (lval & PCI_L1SS_CTL1_PCIPM_L1_1) - rtsx_set_dev_flag(pcr, PM_L1_1_EN); else - rtsx_clear_dev_flag(pcr, PM_L1_1_EN); - - if (lval & PCI_L1SS_CTL1_PCIPM_L1_2) - rtsx_set_dev_flag(pcr, PM_L1_2_EN); - else - rtsx_clear_dev_flag(pcr, PM_L1_2_EN); + rtsx_pci_enable_oobs_polling(pcr); rtsx_pci_write_register(pcr, ASPM_FORCE_CTL, 0xFF, 0); - if (option->ltr_en) { - u16 val; - pcie_capability_read_word(pcr->pci, PCI_EXP_DEVCTL2, &val); - if (val & PCI_EXP_DEVCTL2_LTR_EN) { - option->ltr_enabled = true; - option->ltr_active = true; + if (option->ltr_en) { + if (option->ltr_enabled) rtsx_set_ltr_latency(pcr, option->ltr_active_latency); - } else { - option->ltr_enabled = false; - } } } static int rts5228_extra_init_hw(struct rtsx_pcr *pcr) { + struct rtsx_cr_option *option = &pcr->option; rtsx_pci_write_register(pcr, RTS5228_AUTOLOAD_CFG1, CD_RESUME_EN_MASK, CD_RESUME_EN_MASK); @@ -469,6 +435,17 @@ static int rts5228_extra_init_hw(struct rtsx_pcr *pcr) else rtsx_pci_write_register(pcr, PETXCFG, 0x30, 0x00); + /* + * If u_force_clkreq_0 is enabled, CLKREQ# PIN will be forced + * to drive low, and we forcibly request clock. + */ + if (option->force_clkreq_0) + rtsx_pci_write_register(pcr, PETXCFG, + FORCE_CLKREQ_DELINK_MASK, FORCE_CLKREQ_LOW); + else + rtsx_pci_write_register(pcr, PETXCFG, + FORCE_CLKREQ_DELINK_MASK, FORCE_CLKREQ_HIGH); + rtsx_pci_write_register(pcr, PWD_SUSPEND_EN, 0xFF, 0xFB); if (pcr->rtd3_en) { diff --git a/drivers/misc/cardreader/rts5249.c b/drivers/misc/cardreader/rts5249.c index 47ab72a43256..6c81040e18be 100644 --- a/drivers/misc/cardreader/rts5249.c +++ b/drivers/misc/cardreader/rts5249.c @@ -86,64 +86,22 @@ static void rtsx_base_fetch_vendor_settings(struct rtsx_pcr *pcr) static void rts5249_init_from_cfg(struct rtsx_pcr *pcr) { - struct pci_dev *pdev = pcr->pci; - int l1ss; struct rtsx_cr_option *option = &(pcr->option); - u32 lval; - - l1ss = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_L1SS); - if (!l1ss) - return; - - pci_read_config_dword(pdev, l1ss + PCI_L1SS_CTL1, &lval); if (CHK_PCI_PID(pcr, PID_524A) || CHK_PCI_PID(pcr, PID_525A)) { - if (0 == (lval & 0x0F)) - rtsx_pci_enable_oobs_polling(pcr); - else + if (rtsx_check_dev_flag(pcr, ASPM_L1_1_EN | ASPM_L1_2_EN + | PM_L1_1_EN | PM_L1_2_EN)) rtsx_pci_disable_oobs_polling(pcr); + else + rtsx_pci_enable_oobs_polling(pcr); } - - if (lval & PCI_L1SS_CTL1_ASPM_L1_1) - rtsx_set_dev_flag(pcr, ASPM_L1_1_EN); - - if (lval & PCI_L1SS_CTL1_ASPM_L1_2) - rtsx_set_dev_flag(pcr, ASPM_L1_2_EN); - - if (lval & PCI_L1SS_CTL1_PCIPM_L1_1) - rtsx_set_dev_flag(pcr, PM_L1_1_EN); - - if (lval & PCI_L1SS_CTL1_PCIPM_L1_2) - rtsx_set_dev_flag(pcr, PM_L1_2_EN); - if (option->ltr_en) { - u16 val; - - pcie_capability_read_word(pdev, PCI_EXP_DEVCTL2, &val); - if (val & PCI_EXP_DEVCTL2_LTR_EN) { - option->ltr_enabled = true; - option->ltr_active = true; + if (option->ltr_enabled) rtsx_set_ltr_latency(pcr, option->ltr_active_latency); - } else { - option->ltr_enabled = false; - } } } -static int rts5249_init_from_hw(struct rtsx_pcr *pcr) -{ - struct rtsx_cr_option *option = &(pcr->option); - - if (rtsx_check_dev_flag(pcr, ASPM_L1_1_EN | ASPM_L1_2_EN - | PM_L1_1_EN | PM_L1_2_EN)) - option->force_clkreq_0 = false; - else - option->force_clkreq_0 = true; - - return 0; -} - static void rts52xa_force_power_down(struct rtsx_pcr *pcr, u8 pm_state, bool runtime) { /* Set relink_time to 0 */ @@ -276,7 +234,6 @@ static int rts5249_extra_init_hw(struct rtsx_pcr *pcr) struct rtsx_cr_option *option = &(pcr->option); rts5249_init_from_cfg(pcr); - rts5249_init_from_hw(pcr); rtsx_pci_init_cmd(pcr); @@ -327,11 +284,12 @@ static int rts5249_extra_init_hw(struct rtsx_pcr *pcr) } } + /* * If u_force_clkreq_0 is enabled, CLKREQ# PIN will be forced * to drive low, and we forcibly request clock. */ - if (option->force_clkreq_0 && pcr->aspm_mode == ASPM_MODE_CFG) + if (option->force_clkreq_0) rtsx_pci_write_register(pcr, PETXCFG, FORCE_CLKREQ_DELINK_MASK, FORCE_CLKREQ_LOW); else diff --git a/drivers/misc/cardreader/rts5260.c b/drivers/misc/cardreader/rts5260.c index 79b18f6f73a8..d2d3a6ccb8f7 100644 --- a/drivers/misc/cardreader/rts5260.c +++ b/drivers/misc/cardreader/rts5260.c @@ -480,47 +480,19 @@ static void rts5260_pwr_saving_setting(struct rtsx_pcr *pcr) static void rts5260_init_from_cfg(struct rtsx_pcr *pcr) { - struct pci_dev *pdev = pcr->pci; - int l1ss; struct rtsx_cr_option *option = &pcr->option; - u32 lval; - - l1ss = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_L1SS); - if (!l1ss) - return; - - pci_read_config_dword(pdev, l1ss + PCI_L1SS_CTL1, &lval); - - if (lval & PCI_L1SS_CTL1_ASPM_L1_1) - rtsx_set_dev_flag(pcr, ASPM_L1_1_EN); - - if (lval & PCI_L1SS_CTL1_ASPM_L1_2) - rtsx_set_dev_flag(pcr, ASPM_L1_2_EN); - - if (lval & PCI_L1SS_CTL1_PCIPM_L1_1) - rtsx_set_dev_flag(pcr, PM_L1_1_EN); - - if (lval & PCI_L1SS_CTL1_PCIPM_L1_2) - rtsx_set_dev_flag(pcr, PM_L1_2_EN); rts5260_pwr_saving_setting(pcr); if (option->ltr_en) { - u16 val; - - pcie_capability_read_word(pdev, PCI_EXP_DEVCTL2, &val); - if (val & PCI_EXP_DEVCTL2_LTR_EN) { - option->ltr_enabled = true; - option->ltr_active = true; + if (option->ltr_enabled) rtsx_set_ltr_latency(pcr, option->ltr_active_latency); - } else { - option->ltr_enabled = false; - } } } static int rts5260_extra_init_hw(struct rtsx_pcr *pcr) { + struct rtsx_cr_option *option = &pcr->option; /* Set mcu_cnt to 7 to ensure data can be sampled properly */ rtsx_pci_write_register(pcr, 0xFC03, 0x7F, 0x07); @@ -539,6 +511,17 @@ static int rts5260_extra_init_hw(struct rtsx_pcr *pcr) rts5260_init_hw(pcr); + /* + * If u_force_clkreq_0 is enabled, CLKREQ# PIN will be forced + * to drive low, and we forcibly request clock. + */ + if (option->force_clkreq_0) + rtsx_pci_write_register(pcr, PETXCFG, + FORCE_CLKREQ_DELINK_MASK, FORCE_CLKREQ_LOW); + else + rtsx_pci_write_register(pcr, PETXCFG, + FORCE_CLKREQ_DELINK_MASK, FORCE_CLKREQ_HIGH); + rtsx_pci_write_register(pcr, pcr->reg_pm_ctrl3, 0x10, 0x00); return 0; diff --git a/drivers/misc/cardreader/rts5261.c b/drivers/misc/cardreader/rts5261.c index 94af6bf8a25a..67252512a132 100644 --- a/drivers/misc/cardreader/rts5261.c +++ b/drivers/misc/cardreader/rts5261.c @@ -454,54 +454,17 @@ static void rts5261_init_from_hw(struct rtsx_pcr *pcr) static void rts5261_init_from_cfg(struct rtsx_pcr *pcr) { - struct pci_dev *pdev = pcr->pci; - int l1ss; - u32 lval; struct rtsx_cr_option *option = &pcr->option; - l1ss = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_L1SS); - if (!l1ss) - return; - - pci_read_config_dword(pdev, l1ss + PCI_L1SS_CTL1, &lval); - - if (lval & PCI_L1SS_CTL1_ASPM_L1_1) - rtsx_set_dev_flag(pcr, ASPM_L1_1_EN); - else - rtsx_clear_dev_flag(pcr, ASPM_L1_1_EN); - - if (lval & PCI_L1SS_CTL1_ASPM_L1_2) - rtsx_set_dev_flag(pcr, ASPM_L1_2_EN); - else - rtsx_clear_dev_flag(pcr, ASPM_L1_2_EN); - - if (lval & PCI_L1SS_CTL1_PCIPM_L1_1) - rtsx_set_dev_flag(pcr, PM_L1_1_EN); - else - rtsx_clear_dev_flag(pcr, PM_L1_1_EN); - - if (lval & PCI_L1SS_CTL1_PCIPM_L1_2) - rtsx_set_dev_flag(pcr, PM_L1_2_EN); - else - rtsx_clear_dev_flag(pcr, PM_L1_2_EN); - - rtsx_pci_write_register(pcr, ASPM_FORCE_CTL, 0xFF, 0); if (option->ltr_en) { - u16 val; - - pcie_capability_read_word(pdev, PCI_EXP_DEVCTL2, &val); - if (val & PCI_EXP_DEVCTL2_LTR_EN) { - option->ltr_enabled = true; - option->ltr_active = true; + if (option->ltr_enabled) rtsx_set_ltr_latency(pcr, option->ltr_active_latency); - } else { - option->ltr_enabled = false; - } } } static int rts5261_extra_init_hw(struct rtsx_pcr *pcr) { + struct rtsx_cr_option *option = &pcr->option; u32 val; rtsx_pci_write_register(pcr, RTS5261_AUTOLOAD_CFG1, @@ -547,6 +510,17 @@ static int rts5261_extra_init_hw(struct rtsx_pcr *pcr) else rtsx_pci_write_register(pcr, PETXCFG, 0x30, 0x00); + /* + * If u_force_clkreq_0 is enabled, CLKREQ# PIN will be forced + * to drive low, and we forcibly request clock. + */ + if (option->force_clkreq_0) + rtsx_pci_write_register(pcr, PETXCFG, + FORCE_CLKREQ_DELINK_MASK, FORCE_CLKREQ_LOW); + else + rtsx_pci_write_register(pcr, PETXCFG, + FORCE_CLKREQ_DELINK_MASK, FORCE_CLKREQ_HIGH); + rtsx_pci_write_register(pcr, PWD_SUSPEND_EN, 0xFF, 0xFB); if (pcr->rtd3_en) { diff --git a/drivers/misc/cardreader/rtsx_pcr.c b/drivers/misc/cardreader/rtsx_pcr.c index a3f4b52bb159..a30751ad3733 100644 --- a/drivers/misc/cardreader/rtsx_pcr.c +++ b/drivers/misc/cardreader/rtsx_pcr.c @@ -1326,11 +1326,8 @@ static int rtsx_pci_init_hw(struct rtsx_pcr *pcr) return err; } - if (pcr->aspm_mode == ASPM_MODE_REG) { + if (pcr->aspm_mode == ASPM_MODE_REG) rtsx_pci_write_register(pcr, ASPM_FORCE_CTL, 0x30, 0x30); - rtsx_pci_write_register(pcr, PETXCFG, - FORCE_CLKREQ_DELINK_MASK, FORCE_CLKREQ_HIGH); - } /* No CD interrupt if probing driver with card inserted. * So we need to initialize pcr->card_exist here. @@ -1345,7 +1342,9 @@ static int rtsx_pci_init_hw(struct rtsx_pcr *pcr) static int rtsx_pci_init_chip(struct rtsx_pcr *pcr) { - int err; + struct rtsx_cr_option *option = &(pcr->option); + int err, l1ss; + u32 lval; u16 cfg_val; u8 val; @@ -1430,6 +1429,48 @@ static int rtsx_pci_init_chip(struct rtsx_pcr *pcr) pcr->aspm_enabled = true; } + l1ss = pci_find_ext_capability(pcr->pci, PCI_EXT_CAP_ID_L1SS); + if (l1ss) { + pci_read_config_dword(pcr->pci, l1ss + PCI_L1SS_CTL1, &lval); + + if (lval & PCI_L1SS_CTL1_ASPM_L1_1) + rtsx_set_dev_flag(pcr, ASPM_L1_1_EN); + else + rtsx_clear_dev_flag(pcr, ASPM_L1_1_EN); + + if (lval & PCI_L1SS_CTL1_ASPM_L1_2) + rtsx_set_dev_flag(pcr, ASPM_L1_2_EN); + else + rtsx_clear_dev_flag(pcr, ASPM_L1_2_EN); + + if (lval & PCI_L1SS_CTL1_PCIPM_L1_1) + rtsx_set_dev_flag(pcr, PM_L1_1_EN); + else + rtsx_clear_dev_flag(pcr, PM_L1_1_EN); + + if (lval & PCI_L1SS_CTL1_PCIPM_L1_2) + rtsx_set_dev_flag(pcr, PM_L1_2_EN); + else + rtsx_clear_dev_flag(pcr, PM_L1_2_EN); + + pcie_capability_read_word(pcr->pci, PCI_EXP_DEVCTL2, &cfg_val); + if (cfg_val & PCI_EXP_DEVCTL2_LTR_EN) { + option->ltr_enabled = true; + option->ltr_active = true; + } else { + option->ltr_enabled = false; + } + + if (rtsx_check_dev_flag(pcr, ASPM_L1_1_EN | ASPM_L1_2_EN + | PM_L1_1_EN | PM_L1_2_EN)) + option->force_clkreq_0 = false; + else + option->force_clkreq_0 = true; + } else { + option->ltr_enabled = false; + option->force_clkreq_0 = true; + } + if (pcr->ops->fetch_vendor_settings) pcr->ops->fetch_vendor_settings(pcr); diff --git a/drivers/mmc/core/block.c b/drivers/mmc/core/block.c index b5b414a71e0b..3a8f27c3e310 100644 --- a/drivers/mmc/core/block.c +++ b/drivers/mmc/core/block.c @@ -179,6 +179,7 @@ static void mmc_blk_rw_rq_prep(struct mmc_queue_req *mqrq, struct mmc_queue *mq); static void mmc_blk_hsq_req_done(struct mmc_request *mrq); static int mmc_spi_err_check(struct mmc_card *card); +static int mmc_blk_busy_cb(void *cb_data, bool *busy); static struct mmc_blk_data *mmc_blk_get(struct gendisk *disk) { @@ -470,7 +471,7 @@ static int __mmc_blk_ioctl_cmd(struct mmc_card *card, struct mmc_blk_data *md, struct mmc_data data = {}; struct mmc_request mrq = {}; struct scatterlist sg; - bool r1b_resp, use_r1b_resp = false; + bool r1b_resp; unsigned int busy_timeout_ms; int err; unsigned int target_part; @@ -551,8 +552,7 @@ static int __mmc_blk_ioctl_cmd(struct mmc_card *card, struct mmc_blk_data *md, busy_timeout_ms = idata->ic.cmd_timeout_ms ? : MMC_BLK_TIMEOUT_MS; r1b_resp = (cmd.flags & MMC_RSP_R1B) == MMC_RSP_R1B; if (r1b_resp) - use_r1b_resp = mmc_prepare_busy_cmd(card->host, &cmd, - busy_timeout_ms); + mmc_prepare_busy_cmd(card->host, &cmd, busy_timeout_ms); mmc_wait_for_req(card->host, &mrq); memcpy(&idata->ic.response, cmd.resp, sizeof(cmd.resp)); @@ -605,19 +605,28 @@ static int __mmc_blk_ioctl_cmd(struct mmc_card *card, struct mmc_blk_data *md, if (idata->ic.postsleep_min_us) usleep_range(idata->ic.postsleep_min_us, idata->ic.postsleep_max_us); - /* No need to poll when using HW busy detection. */ - if ((card->host->caps & MMC_CAP_WAIT_WHILE_BUSY) && use_r1b_resp) - return 0; - if (mmc_host_is_spi(card->host)) { if (idata->ic.write_flag || r1b_resp || cmd.flags & MMC_RSP_SPI_BUSY) return mmc_spi_err_check(card); return err; } - /* Ensure RPMB/R1B command has completed by polling with CMD13. */ - if (idata->rpmb || r1b_resp) - err = mmc_poll_for_busy(card, busy_timeout_ms, false, - MMC_BUSY_IO); + + /* + * Ensure RPMB, writes and R1B responses are completed by polling with + * CMD13. Note that, usually we don't need to poll when using HW busy + * detection, but here it's needed since some commands may indicate the + * error through the R1 status bits. + */ + if (idata->rpmb || idata->ic.write_flag || r1b_resp) { + struct mmc_blk_busy_data cb_data = { + .card = card, + }; + + err = __mmc_poll_for_busy(card->host, 0, busy_timeout_ms, + &mmc_blk_busy_cb, &cb_data); + + idata->ic.response[0] = cb_data.status; + } return err; } diff --git a/drivers/mmc/core/mmc.c b/drivers/mmc/core/mmc.c index 89cd48fcec79..4a4bab9aa726 100644 --- a/drivers/mmc/core/mmc.c +++ b/drivers/mmc/core/mmc.c @@ -104,7 +104,7 @@ static int mmc_decode_cid(struct mmc_card *card) case 3: /* MMC v3.1 - v3.3 */ case 4: /* MMC v4 */ card->cid.manfid = UNSTUFF_BITS(resp, 120, 8); - card->cid.oemid = UNSTUFF_BITS(resp, 104, 16); + card->cid.oemid = UNSTUFF_BITS(resp, 104, 8); card->cid.prod_name[0] = UNSTUFF_BITS(resp, 96, 8); card->cid.prod_name[1] = UNSTUFF_BITS(resp, 88, 8); card->cid.prod_name[2] = UNSTUFF_BITS(resp, 80, 8); diff --git a/drivers/mmc/core/sdio.c b/drivers/mmc/core/sdio.c index f64b9ac76a5c..5914516df2f7 100644 --- a/drivers/mmc/core/sdio.c +++ b/drivers/mmc/core/sdio.c @@ -1089,8 +1089,14 @@ static int mmc_sdio_resume(struct mmc_host *host) } err = mmc_sdio_reinit_card(host); } else if (mmc_card_wake_sdio_irq(host)) { - /* We may have switched to 1-bit mode during suspend */ + /* + * We may have switched to 1-bit mode during suspend, + * need to hold retuning, because tuning only supprt + * 4-bit mode or 8 bit mode. + */ + mmc_retune_hold_now(host); err = sdio_enable_4bit_bus(host->card); + mmc_retune_release(host); } if (err) diff --git a/drivers/mmc/host/mtk-sd.c b/drivers/mmc/host/mtk-sd.c index 5392200cfdf7..97f7c3d4be6e 100644 --- a/drivers/mmc/host/mtk-sd.c +++ b/drivers/mmc/host/mtk-sd.c @@ -669,11 +669,11 @@ static void msdc_reset_hw(struct msdc_host *host) u32 val; sdr_set_bits(host->base + MSDC_CFG, MSDC_CFG_RST); - readl_poll_timeout(host->base + MSDC_CFG, val, !(val & MSDC_CFG_RST), 0, 0); + readl_poll_timeout_atomic(host->base + MSDC_CFG, val, !(val & MSDC_CFG_RST), 0, 0); sdr_set_bits(host->base + MSDC_FIFOCS, MSDC_FIFOCS_CLR); - readl_poll_timeout(host->base + MSDC_FIFOCS, val, - !(val & MSDC_FIFOCS_CLR), 0, 0); + readl_poll_timeout_atomic(host->base + MSDC_FIFOCS, val, + !(val & MSDC_FIFOCS_CLR), 0, 0); val = readl(host->base + MSDC_INT); writel(val, host->base + MSDC_INT); diff --git a/drivers/mmc/host/sdhci-pci-gli.c b/drivers/mmc/host/sdhci-pci-gli.c index ae8c307b7aa7..109d4b010f97 100644 --- a/drivers/mmc/host/sdhci-pci-gli.c +++ b/drivers/mmc/host/sdhci-pci-gli.c @@ -1144,42 +1144,6 @@ static u32 sdhci_gl9750_readl(struct sdhci_host *host, int reg) return value; } -#ifdef CONFIG_PM_SLEEP -static int sdhci_pci_gli_resume(struct sdhci_pci_chip *chip) -{ - struct sdhci_pci_slot *slot = chip->slots[0]; - - pci_free_irq_vectors(slot->chip->pdev); - gli_pcie_enable_msi(slot); - - return sdhci_pci_resume_host(chip); -} - -static int sdhci_cqhci_gli_resume(struct sdhci_pci_chip *chip) -{ - struct sdhci_pci_slot *slot = chip->slots[0]; - int ret; - - ret = sdhci_pci_gli_resume(chip); - if (ret) - return ret; - - return cqhci_resume(slot->host->mmc); -} - -static int sdhci_cqhci_gli_suspend(struct sdhci_pci_chip *chip) -{ - struct sdhci_pci_slot *slot = chip->slots[0]; - int ret; - - ret = cqhci_suspend(slot->host->mmc); - if (ret) - return ret; - - return sdhci_suspend_host(slot->host); -} -#endif - static void gl9763e_hs400_enhanced_strobe(struct mmc_host *mmc, struct mmc_ios *ios) { @@ -1420,6 +1384,70 @@ static int gl9763e_runtime_resume(struct sdhci_pci_chip *chip) } #endif +#ifdef CONFIG_PM_SLEEP +static int sdhci_pci_gli_resume(struct sdhci_pci_chip *chip) +{ + struct sdhci_pci_slot *slot = chip->slots[0]; + + pci_free_irq_vectors(slot->chip->pdev); + gli_pcie_enable_msi(slot); + + return sdhci_pci_resume_host(chip); +} + +static int gl9763e_resume(struct sdhci_pci_chip *chip) +{ + struct sdhci_pci_slot *slot = chip->slots[0]; + int ret; + + ret = sdhci_pci_gli_resume(chip); + if (ret) + return ret; + + ret = cqhci_resume(slot->host->mmc); + if (ret) + return ret; + + /* + * Disable LPM negotiation to bring device back in sync + * with its runtime_pm state. + */ + gl9763e_set_low_power_negotiation(slot, false); + + return 0; +} + +static int gl9763e_suspend(struct sdhci_pci_chip *chip) +{ + struct sdhci_pci_slot *slot = chip->slots[0]; + int ret; + + /* + * Certain SoCs can suspend only with the bus in low- + * power state, notably x86 SoCs when using S0ix. + * Re-enable LPM negotiation to allow entering L1 state + * and entering system suspend. + */ + gl9763e_set_low_power_negotiation(slot, true); + + ret = cqhci_suspend(slot->host->mmc); + if (ret) + goto err_suspend; + + ret = sdhci_suspend_host(slot->host); + if (ret) + goto err_suspend_host; + + return 0; + +err_suspend_host: + cqhci_resume(slot->host->mmc); +err_suspend: + gl9763e_set_low_power_negotiation(slot, false); + return ret; +} +#endif + static int gli_probe_slot_gl9763e(struct sdhci_pci_slot *slot) { struct pci_dev *pdev = slot->chip->pdev; @@ -1527,8 +1555,8 @@ const struct sdhci_pci_fixes sdhci_gl9763e = { .probe_slot = gli_probe_slot_gl9763e, .ops = &sdhci_gl9763e_ops, #ifdef CONFIG_PM_SLEEP - .resume = sdhci_cqhci_gli_resume, - .suspend = sdhci_cqhci_gli_suspend, + .resume = gl9763e_resume, + .suspend = gl9763e_suspend, #endif #ifdef CONFIG_PM .runtime_suspend = gl9763e_runtime_suspend, diff --git a/drivers/mmc/host/sdhci-sprd.c b/drivers/mmc/host/sdhci-sprd.c index 649ae075e229..6b84ba27e6ab 100644 --- a/drivers/mmc/host/sdhci-sprd.c +++ b/drivers/mmc/host/sdhci-sprd.c @@ -644,6 +644,7 @@ static int sdhci_sprd_tuning(struct mmc_host *mmc, struct mmc_card *card, best_clk_sample = sdhci_sprd_get_best_clk_sample(mmc, value); if (best_clk_sample < 0) { dev_err(mmc_dev(host->mmc), "all tuning phase fail!\n"); + err = best_clk_sample; goto out; } diff --git a/drivers/mtd/maps/physmap-core.c b/drivers/mtd/maps/physmap-core.c index 78710fbc8e7f..fc8721339282 100644 --- a/drivers/mtd/maps/physmap-core.c +++ b/drivers/mtd/maps/physmap-core.c @@ -551,6 +551,17 @@ static int physmap_flash_probe(struct platform_device *dev) if (info->probe_type) { info->mtds[i] = do_map_probe(info->probe_type, &info->maps[i]); + + /* Fall back to mapping region as ROM */ + if (!info->mtds[i] && IS_ENABLED(CONFIG_MTD_ROM) && + strcmp(info->probe_type, "map_rom")) { + dev_warn(&dev->dev, + "map_probe() failed for type %s\n", + info->probe_type); + + info->mtds[i] = do_map_probe("map_rom", + &info->maps[i]); + } } else { int j; diff --git a/drivers/mtd/nand/raw/arasan-nand-controller.c b/drivers/mtd/nand/raw/arasan-nand-controller.c index 4621ec549cc7..a492051c46f5 100644 --- a/drivers/mtd/nand/raw/arasan-nand-controller.c +++ b/drivers/mtd/nand/raw/arasan-nand-controller.c @@ -515,6 +515,7 @@ static int anfc_write_page_hw_ecc(struct nand_chip *chip, const u8 *buf, struct mtd_info *mtd = nand_to_mtd(chip); unsigned int len = mtd->writesize + (oob_required ? mtd->oobsize : 0); dma_addr_t dma_addr; + u8 status; int ret; struct anfc_op nfc_op = { .pkt_reg = @@ -561,10 +562,21 @@ static int anfc_write_page_hw_ecc(struct nand_chip *chip, const u8 *buf, } /* Spare data is not protected */ - if (oob_required) + if (oob_required) { ret = nand_write_oob_std(chip, page); + if (ret) + return ret; + } - return ret; + /* Check write status on the chip side */ + ret = nand_status_op(chip, &status); + if (ret) + return ret; + + if (status & NAND_STATUS_FAIL) + return -EIO; + + return 0; } static int anfc_sel_write_page_hw_ecc(struct nand_chip *chip, const u8 *buf, diff --git a/drivers/mtd/nand/raw/marvell_nand.c b/drivers/mtd/nand/raw/marvell_nand.c index 2c94da7a3b3a..b841a81cb128 100644 --- a/drivers/mtd/nand/raw/marvell_nand.c +++ b/drivers/mtd/nand/raw/marvell_nand.c @@ -1165,6 +1165,7 @@ static int marvell_nfc_hw_ecc_hmg_do_write_page(struct nand_chip *chip, .ndcb[2] = NDCB2_ADDR5_PAGE(page), }; unsigned int oob_bytes = lt->spare_bytes + (raw ? lt->ecc_bytes : 0); + u8 status; int ret; /* NFCv2 needs more information about the operation being executed */ @@ -1198,7 +1199,18 @@ static int marvell_nfc_hw_ecc_hmg_do_write_page(struct nand_chip *chip, ret = marvell_nfc_wait_op(chip, PSEC_TO_MSEC(sdr->tPROG_max)); - return ret; + if (ret) + return ret; + + /* Check write status on the chip side */ + ret = nand_status_op(chip, &status); + if (ret) + return ret; + + if (status & NAND_STATUS_FAIL) + return -EIO; + + return 0; } static int marvell_nfc_hw_ecc_hmg_write_page_raw(struct nand_chip *chip, @@ -1627,6 +1639,7 @@ static int marvell_nfc_hw_ecc_bch_write_page(struct nand_chip *chip, int data_len = lt->data_bytes; int spare_len = lt->spare_bytes; int chunk, ret; + u8 status; marvell_nfc_select_target(chip, chip->cur_cs); @@ -1663,6 +1676,14 @@ static int marvell_nfc_hw_ecc_bch_write_page(struct nand_chip *chip, if (ret) return ret; + /* Check write status on the chip side */ + ret = nand_status_op(chip, &status); + if (ret) + return ret; + + if (status & NAND_STATUS_FAIL) + return -EIO; + return 0; } diff --git a/drivers/mtd/nand/raw/nand_base.c b/drivers/mtd/nand/raw/nand_base.c index d4b55155aeae..1fcac403cee6 100644 --- a/drivers/mtd/nand/raw/nand_base.c +++ b/drivers/mtd/nand/raw/nand_base.c @@ -5110,6 +5110,9 @@ static void rawnand_check_cont_read_support(struct nand_chip *chip) { struct mtd_info *mtd = nand_to_mtd(chip); + if (!chip->parameters.supports_read_cache) + return; + if (chip->read_retries) return; diff --git a/drivers/mtd/nand/raw/nand_jedec.c b/drivers/mtd/nand/raw/nand_jedec.c index 836757717660..b3cc8f360529 100644 --- a/drivers/mtd/nand/raw/nand_jedec.c +++ b/drivers/mtd/nand/raw/nand_jedec.c @@ -94,6 +94,9 @@ int nand_jedec_detect(struct nand_chip *chip) goto free_jedec_param_page; } + if (p->opt_cmd[0] & JEDEC_OPT_CMD_READ_CACHE) + chip->parameters.supports_read_cache = true; + memorg->pagesize = le32_to_cpu(p->byte_per_page); mtd->writesize = memorg->pagesize; diff --git a/drivers/mtd/nand/raw/nand_onfi.c b/drivers/mtd/nand/raw/nand_onfi.c index f15ef90aec8c..861975e44b55 100644 --- a/drivers/mtd/nand/raw/nand_onfi.c +++ b/drivers/mtd/nand/raw/nand_onfi.c @@ -303,6 +303,9 @@ int nand_onfi_detect(struct nand_chip *chip) ONFI_FEATURE_ADDR_TIMING_MODE, 1); } + if (le16_to_cpu(p->opt_cmd) & ONFI_OPT_CMD_READ_CACHE) + chip->parameters.supports_read_cache = true; + onfi = kzalloc(sizeof(*onfi), GFP_KERNEL); if (!onfi) { ret = -ENOMEM; diff --git a/drivers/mtd/nand/raw/pl35x-nand-controller.c b/drivers/mtd/nand/raw/pl35x-nand-controller.c index 8da5fee321b5..c506e92a3e45 100644 --- a/drivers/mtd/nand/raw/pl35x-nand-controller.c +++ b/drivers/mtd/nand/raw/pl35x-nand-controller.c @@ -511,6 +511,7 @@ static int pl35x_nand_write_page_hwecc(struct nand_chip *chip, u32 addr1 = 0, addr2 = 0, row; u32 cmd_addr; int i, ret; + u8 status; ret = pl35x_smc_set_ecc_mode(nfc, chip, PL35X_SMC_ECC_CFG_MODE_APB); if (ret) @@ -563,6 +564,14 @@ static int pl35x_nand_write_page_hwecc(struct nand_chip *chip, if (ret) goto disable_ecc_engine; + /* Check write status on the chip side */ + ret = nand_status_op(chip, &status); + if (ret) + goto disable_ecc_engine; + + if (status & NAND_STATUS_FAIL) + ret = -EIO; + disable_ecc_engine: pl35x_smc_set_ecc_mode(nfc, chip, PL35X_SMC_ECC_CFG_MODE_BYPASS); diff --git a/drivers/mtd/nand/raw/qcom_nandc.c b/drivers/mtd/nand/raw/qcom_nandc.c index 64499c1b3603..b079605c84d3 100644 --- a/drivers/mtd/nand/raw/qcom_nandc.c +++ b/drivers/mtd/nand/raw/qcom_nandc.c @@ -3444,7 +3444,7 @@ err_nandc_alloc: err_aon_clk: clk_disable_unprepare(nandc->core_clk); err_core_clk: - dma_unmap_resource(dev, res->start, resource_size(res), + dma_unmap_resource(dev, nandc->base_dma, resource_size(res), DMA_BIDIRECTIONAL, 0); return ret; } diff --git a/drivers/mtd/nand/spi/micron.c b/drivers/mtd/nand/spi/micron.c index 50b7295bc922..12601bc4227a 100644 --- a/drivers/mtd/nand/spi/micron.c +++ b/drivers/mtd/nand/spi/micron.c @@ -12,7 +12,7 @@ #define SPINAND_MFR_MICRON 0x2c -#define MICRON_STATUS_ECC_MASK GENMASK(7, 4) +#define MICRON_STATUS_ECC_MASK GENMASK(6, 4) #define MICRON_STATUS_ECC_NO_BITFLIPS (0 << 4) #define MICRON_STATUS_ECC_1TO3_BITFLIPS (1 << 4) #define MICRON_STATUS_ECC_4TO6_BITFLIPS (3 << 4) diff --git a/drivers/mtd/ubi/build.c b/drivers/mtd/ubi/build.c index 8b91a55ec0d2..8ee51e49fced 100644 --- a/drivers/mtd/ubi/build.c +++ b/drivers/mtd/ubi/build.c @@ -894,6 +894,13 @@ int ubi_attach_mtd_dev(struct mtd_info *mtd, int ubi_num, return -EINVAL; } + /* UBI cannot work on flashes with zero erasesize. */ + if (!mtd->erasesize) { + pr_err("ubi: refuse attaching mtd%d - zero erasesize flash is not supported\n", + mtd->index); + return -EINVAL; + } + if (ubi_num == UBI_DEV_NUM_AUTO) { /* Search for an empty slot in the @ubi_devices array */ for (ubi_num = 0; ubi_num < UBI_MAX_DEVICES; ubi_num++) diff --git a/drivers/net/bonding/bond_main.c b/drivers/net/bonding/bond_main.c index ed7212e61c54..51d47eda1c87 100644 --- a/drivers/net/bonding/bond_main.c +++ b/drivers/net/bonding/bond_main.c @@ -4023,7 +4023,7 @@ static inline const void *bond_pull_data(struct sk_buff *skb, if (likely(n <= hlen)) return data; else if (skb && likely(pskb_may_pull(skb, n))) - return skb->head; + return skb->data; return NULL; } diff --git a/drivers/net/can/Kconfig b/drivers/net/can/Kconfig index 649453a3c858..f8cde9f9f554 100644 --- a/drivers/net/can/Kconfig +++ b/drivers/net/can/Kconfig @@ -190,7 +190,7 @@ config CAN_SLCAN config CAN_SUN4I tristate "Allwinner A10 CAN controller" - depends on MACH_SUN4I || MACH_SUN7I || RISCV || COMPILE_TEST + depends on MACH_SUN4I || MACH_SUN7I || (RISCV && ARCH_SUNXI) || COMPILE_TEST help Say Y here if you want to use CAN controller found on Allwinner A10/A20/D1 SoCs. diff --git a/drivers/net/can/flexcan/flexcan-core.c b/drivers/net/can/flexcan/flexcan-core.c index add39e922b89..d15f85a40c1e 100644 --- a/drivers/net/can/flexcan/flexcan-core.c +++ b/drivers/net/can/flexcan/flexcan-core.c @@ -348,7 +348,7 @@ static struct flexcan_devtype_data fsl_imx8mp_devtype_data = { static struct flexcan_devtype_data fsl_imx93_devtype_data = { .quirks = FLEXCAN_QUIRK_DISABLE_RXFG | FLEXCAN_QUIRK_ENABLE_EACEN_RRS | FLEXCAN_QUIRK_DISABLE_MECR | FLEXCAN_QUIRK_USE_RX_MAILBOX | - FLEXCAN_QUIRK_BROKEN_PERR_STATE | FLEXCAN_QUIRK_AUTO_STOP_MODE | + FLEXCAN_QUIRK_BROKEN_PERR_STATE | FLEXCAN_QUIRK_SETUP_STOP_MODE_GPR | FLEXCAN_QUIRK_SUPPORT_FD | FLEXCAN_QUIRK_SUPPORT_ECC | FLEXCAN_QUIRK_SUPPORT_RX_MAILBOX | FLEXCAN_QUIRK_SUPPORT_RX_MAILBOX_RTR, @@ -544,11 +544,6 @@ static inline int flexcan_enter_stop_mode(struct flexcan_priv *priv) } else if (priv->devtype_data.quirks & FLEXCAN_QUIRK_SETUP_STOP_MODE_GPR) { regmap_update_bits(priv->stm.gpr, priv->stm.req_gpr, 1 << priv->stm.req_bit, 1 << priv->stm.req_bit); - } else if (priv->devtype_data.quirks & FLEXCAN_QUIRK_AUTO_STOP_MODE) { - /* For the auto stop mode, software do nothing, hardware will cover - * all the operation automatically after system go into low power mode. - */ - return 0; } return flexcan_low_power_enter_ack(priv); @@ -574,12 +569,6 @@ static inline int flexcan_exit_stop_mode(struct flexcan_priv *priv) reg_mcr &= ~FLEXCAN_MCR_SLF_WAK; priv->write(reg_mcr, ®s->mcr); - /* For the auto stop mode, hardware will exist stop mode - * automatically after system go out of low power mode. - */ - if (priv->devtype_data.quirks & FLEXCAN_QUIRK_AUTO_STOP_MODE) - return 0; - return flexcan_low_power_exit_ack(priv); } @@ -1994,13 +1983,18 @@ static int flexcan_setup_stop_mode(struct platform_device *pdev) ret = flexcan_setup_stop_mode_scfw(pdev); else if (priv->devtype_data.quirks & FLEXCAN_QUIRK_SETUP_STOP_MODE_GPR) ret = flexcan_setup_stop_mode_gpr(pdev); - else if (priv->devtype_data.quirks & FLEXCAN_QUIRK_AUTO_STOP_MODE) - ret = 0; else /* return 0 directly if doesn't support stop mode feature */ return 0; - if (ret) + /* If ret is -EINVAL, this means SoC claim to support stop mode, but + * dts file lack the stop mode property definition. For this case, + * directly return 0, this will skip the wakeup capable setting and + * will not block the driver probe. + */ + if (ret == -EINVAL) + return 0; + else if (ret) return ret; device_set_wakeup_capable(&pdev->dev, true); @@ -2320,16 +2314,8 @@ static int __maybe_unused flexcan_noirq_suspend(struct device *device) if (netif_running(dev)) { int err; - if (device_may_wakeup(device)) { + if (device_may_wakeup(device)) flexcan_enable_wakeup_irq(priv, true); - /* For auto stop mode, need to keep the clock on before - * system go into low power mode. After system go into - * low power mode, hardware will config the flexcan into - * stop mode, and gate off the clock automatically. - */ - if (priv->devtype_data.quirks & FLEXCAN_QUIRK_AUTO_STOP_MODE) - return 0; - } err = pm_runtime_force_suspend(device); if (err) @@ -2347,15 +2333,9 @@ static int __maybe_unused flexcan_noirq_resume(struct device *device) if (netif_running(dev)) { int err; - /* For the wakeup in auto stop mode, no need to gate on the - * clock here, hardware will do this automatically. - */ - if (!(device_may_wakeup(device) && - priv->devtype_data.quirks & FLEXCAN_QUIRK_AUTO_STOP_MODE)) { - err = pm_runtime_force_resume(device); - if (err) - return err; - } + err = pm_runtime_force_resume(device); + if (err) + return err; if (device_may_wakeup(device)) flexcan_enable_wakeup_irq(priv, false); diff --git a/drivers/net/can/flexcan/flexcan.h b/drivers/net/can/flexcan/flexcan.h index 91402977780b..025c3417031f 100644 --- a/drivers/net/can/flexcan/flexcan.h +++ b/drivers/net/can/flexcan/flexcan.h @@ -68,8 +68,6 @@ #define FLEXCAN_QUIRK_SUPPORT_RX_MAILBOX_RTR BIT(15) /* Device supports RX via FIFO */ #define FLEXCAN_QUIRK_SUPPORT_RX_FIFO BIT(16) -/* auto enter stop mode to support wakeup */ -#define FLEXCAN_QUIRK_AUTO_STOP_MODE BIT(17) struct flexcan_devtype_data { u32 quirks; /* quirks needed for different IP cores */ diff --git a/drivers/net/can/m_can/tcan4x5x-core.c b/drivers/net/can/m_can/tcan4x5x-core.c index 8a4143809d33..ae8c42f5debd 100644 --- a/drivers/net/can/m_can/tcan4x5x-core.c +++ b/drivers/net/can/m_can/tcan4x5x-core.c @@ -125,7 +125,7 @@ static const struct tcan4x5x_version_info tcan4x5x_versions[] = { }, [TCAN4553] = { .name = "4553", - .id2_register = 0x32353534, + .id2_register = 0x33353534, }, /* generic version with no id2_register at the end */ [TCAN4X5X] = { diff --git a/drivers/net/can/sja1000/sja1000.c b/drivers/net/can/sja1000/sja1000.c index 0ada0e160e93..743c2eb62b87 100644 --- a/drivers/net/can/sja1000/sja1000.c +++ b/drivers/net/can/sja1000/sja1000.c @@ -392,7 +392,13 @@ static irqreturn_t sja1000_reset_interrupt(int irq, void *dev_id) struct net_device *dev = (struct net_device *)dev_id; netdev_dbg(dev, "performing a soft reset upon overrun\n"); - sja1000_start(dev); + + netif_tx_lock(dev); + + can_free_echo_skb(dev, 0, NULL); + sja1000_set_mode(dev, CAN_MODE_START); + + netif_tx_unlock(dev); return IRQ_HANDLED; } diff --git a/drivers/net/dsa/bcm_sf2.c b/drivers/net/dsa/bcm_sf2.c index 72374b066f64..cd1f240c90f3 100644 --- a/drivers/net/dsa/bcm_sf2.c +++ b/drivers/net/dsa/bcm_sf2.c @@ -617,17 +617,16 @@ static int bcm_sf2_mdio_register(struct dsa_switch *ds) dn = of_find_compatible_node(NULL, NULL, "brcm,unimac-mdio"); priv->master_mii_bus = of_mdio_find_bus(dn); if (!priv->master_mii_bus) { - of_node_put(dn); - return -EPROBE_DEFER; + err = -EPROBE_DEFER; + goto err_of_node_put; } - get_device(&priv->master_mii_bus->dev); priv->master_mii_dn = dn; priv->slave_mii_bus = mdiobus_alloc(); if (!priv->slave_mii_bus) { - of_node_put(dn); - return -ENOMEM; + err = -ENOMEM; + goto err_put_master_mii_bus_dev; } priv->slave_mii_bus->priv = priv; @@ -684,11 +683,17 @@ static int bcm_sf2_mdio_register(struct dsa_switch *ds) } err = mdiobus_register(priv->slave_mii_bus); - if (err && dn) { - mdiobus_free(priv->slave_mii_bus); - of_node_put(dn); - } + if (err && dn) + goto err_free_slave_mii_bus; + return 0; + +err_free_slave_mii_bus: + mdiobus_free(priv->slave_mii_bus); +err_put_master_mii_bus_dev: + put_device(&priv->master_mii_bus->dev); +err_of_node_put: + of_node_put(dn); return err; } @@ -696,6 +701,7 @@ static void bcm_sf2_mdio_unregister(struct bcm_sf2_priv *priv) { mdiobus_unregister(priv->slave_mii_bus); mdiobus_free(priv->slave_mii_bus); + put_device(&priv->master_mii_bus->dev); of_node_put(priv->master_mii_dn); } diff --git a/drivers/net/dsa/mv88e6xxx/chip.c b/drivers/net/dsa/mv88e6xxx/chip.c index 52a99d8bada0..ab434a77b059 100644 --- a/drivers/net/dsa/mv88e6xxx/chip.c +++ b/drivers/net/dsa/mv88e6xxx/chip.c @@ -2958,14 +2958,16 @@ static void mv88e6xxx_hardware_reset(struct mv88e6xxx_chip *chip) * from the wrong location resulting in the switch booting * to wrong mode and inoperable. */ - mv88e6xxx_g1_wait_eeprom_done(chip); + if (chip->info->ops->get_eeprom) + mv88e6xxx_g2_eeprom_wait(chip); gpiod_set_value_cansleep(gpiod, 1); usleep_range(10000, 20000); gpiod_set_value_cansleep(gpiod, 0); usleep_range(10000, 20000); - mv88e6xxx_g1_wait_eeprom_done(chip); + if (chip->info->ops->get_eeprom) + mv88e6xxx_g2_eeprom_wait(chip); } } diff --git a/drivers/net/dsa/mv88e6xxx/global1.c b/drivers/net/dsa/mv88e6xxx/global1.c index 2fa55a643591..174c773b38c2 100644 --- a/drivers/net/dsa/mv88e6xxx/global1.c +++ b/drivers/net/dsa/mv88e6xxx/global1.c @@ -75,37 +75,6 @@ static int mv88e6xxx_g1_wait_init_ready(struct mv88e6xxx_chip *chip) return mv88e6xxx_g1_wait_bit(chip, MV88E6XXX_G1_STS, bit, 1); } -void mv88e6xxx_g1_wait_eeprom_done(struct mv88e6xxx_chip *chip) -{ - const unsigned long timeout = jiffies + 1 * HZ; - u16 val; - int err; - - /* Wait up to 1 second for the switch to finish reading the - * EEPROM. - */ - while (time_before(jiffies, timeout)) { - err = mv88e6xxx_g1_read(chip, MV88E6XXX_G1_STS, &val); - if (err) { - dev_err(chip->dev, "Error reading status"); - return; - } - - /* If the switch is still resetting, it may not - * respond on the bus, and so MDIO read returns - * 0xffff. Differentiate between that, and waiting for - * the EEPROM to be done by bit 0 being set. - */ - if (val != 0xffff && - val & BIT(MV88E6XXX_G1_STS_IRQ_EEPROM_DONE)) - return; - - usleep_range(1000, 2000); - } - - dev_err(chip->dev, "Timeout waiting for EEPROM done"); -} - /* Offset 0x01: Switch MAC Address Register Bytes 0 & 1 * Offset 0x02: Switch MAC Address Register Bytes 2 & 3 * Offset 0x03: Switch MAC Address Register Bytes 4 & 5 diff --git a/drivers/net/dsa/mv88e6xxx/global1.h b/drivers/net/dsa/mv88e6xxx/global1.h index c99ddd117fe6..1095261f5b49 100644 --- a/drivers/net/dsa/mv88e6xxx/global1.h +++ b/drivers/net/dsa/mv88e6xxx/global1.h @@ -282,7 +282,6 @@ int mv88e6xxx_g1_set_switch_mac(struct mv88e6xxx_chip *chip, u8 *addr); int mv88e6185_g1_reset(struct mv88e6xxx_chip *chip); int mv88e6352_g1_reset(struct mv88e6xxx_chip *chip); int mv88e6250_g1_reset(struct mv88e6xxx_chip *chip); -void mv88e6xxx_g1_wait_eeprom_done(struct mv88e6xxx_chip *chip); int mv88e6185_g1_ppu_enable(struct mv88e6xxx_chip *chip); int mv88e6185_g1_ppu_disable(struct mv88e6xxx_chip *chip); diff --git a/drivers/net/dsa/mv88e6xxx/global2.c b/drivers/net/dsa/mv88e6xxx/global2.c index 937a01f2ba75..b2b5f6ba438f 100644 --- a/drivers/net/dsa/mv88e6xxx/global2.c +++ b/drivers/net/dsa/mv88e6xxx/global2.c @@ -340,7 +340,7 @@ int mv88e6xxx_g2_pot_clear(struct mv88e6xxx_chip *chip) * Offset 0x15: EEPROM Addr (for 8-bit data access) */ -static int mv88e6xxx_g2_eeprom_wait(struct mv88e6xxx_chip *chip) +int mv88e6xxx_g2_eeprom_wait(struct mv88e6xxx_chip *chip) { int bit = __bf_shf(MV88E6XXX_G2_EEPROM_CMD_BUSY); int err; diff --git a/drivers/net/dsa/mv88e6xxx/global2.h b/drivers/net/dsa/mv88e6xxx/global2.h index 7e091965582b..d9434f7cae53 100644 --- a/drivers/net/dsa/mv88e6xxx/global2.h +++ b/drivers/net/dsa/mv88e6xxx/global2.h @@ -365,6 +365,7 @@ int mv88e6xxx_g2_trunk_clear(struct mv88e6xxx_chip *chip); int mv88e6xxx_g2_device_mapping_write(struct mv88e6xxx_chip *chip, int target, int port); +int mv88e6xxx_g2_eeprom_wait(struct mv88e6xxx_chip *chip); extern const struct mv88e6xxx_irq_ops mv88e6097_watchdog_ops; extern const struct mv88e6xxx_irq_ops mv88e6250_watchdog_ops; diff --git a/drivers/net/dsa/qca/qca8k-8xxx.c b/drivers/net/dsa/qca/qca8k-8xxx.c index de1dc22cf683..4ce68e655a63 100644 --- a/drivers/net/dsa/qca/qca8k-8xxx.c +++ b/drivers/net/dsa/qca/qca8k-8xxx.c @@ -505,8 +505,8 @@ qca8k_bulk_read(void *ctx, const void *reg_buf, size_t reg_len, void *val_buf, size_t val_len) { int i, count = val_len / sizeof(u32), ret; - u32 reg = *(u32 *)reg_buf & U16_MAX; struct qca8k_priv *priv = ctx; + u32 reg = *(u16 *)reg_buf; if (priv->mgmt_master && !qca8k_read_eth(priv, reg, val_buf, val_len)) @@ -527,8 +527,8 @@ qca8k_bulk_gather_write(void *ctx, const void *reg_buf, size_t reg_len, const void *val_buf, size_t val_len) { int i, count = val_len / sizeof(u32), ret; - u32 reg = *(u32 *)reg_buf & U16_MAX; struct qca8k_priv *priv = ctx; + u32 reg = *(u16 *)reg_buf; u32 *val = (u32 *)val_buf; if (priv->mgmt_master && @@ -666,6 +666,15 @@ qca8k_phy_eth_command(struct qca8k_priv *priv, bool read, int phy, goto err_read_skb; } + /* It seems that accessing the switch's internal PHYs via management + * packets still uses the MDIO bus within the switch internally, and + * these accesses can conflict with external MDIO accesses to other + * devices on the MDIO bus. + * We therefore need to lock the MDIO bus onto which the switch is + * connected. + */ + mutex_lock(&priv->bus->mdio_lock); + /* Actually start the request: * 1. Send mdio master packet * 2. Busy Wait for mdio master command @@ -678,6 +687,7 @@ qca8k_phy_eth_command(struct qca8k_priv *priv, bool read, int phy, mgmt_master = priv->mgmt_master; if (!mgmt_master) { mutex_unlock(&mgmt_eth_data->mutex); + mutex_unlock(&priv->bus->mdio_lock); ret = -EINVAL; goto err_mgmt_master; } @@ -765,6 +775,7 @@ exit: QCA8K_ETHERNET_TIMEOUT); mutex_unlock(&mgmt_eth_data->mutex); + mutex_unlock(&priv->bus->mdio_lock); return ret; diff --git a/drivers/net/dsa/sja1105/sja1105.h b/drivers/net/dsa/sja1105/sja1105.h index 0617d5ccd3ff..8c66d3bf61f0 100644 --- a/drivers/net/dsa/sja1105/sja1105.h +++ b/drivers/net/dsa/sja1105/sja1105.h @@ -266,6 +266,8 @@ struct sja1105_private { * the switch doesn't confuse them with one another. */ struct mutex mgmt_lock; + /* Serializes accesses to the FDB */ + struct mutex fdb_lock; /* PTP two-step TX timestamp ID, and its serialization lock */ spinlock_t ts_id_lock; u8 ts_id; diff --git a/drivers/net/dsa/sja1105/sja1105_dynamic_config.c b/drivers/net/dsa/sja1105/sja1105_dynamic_config.c index 7729d3f8b7f5..984c0e604e8d 100644 --- a/drivers/net/dsa/sja1105/sja1105_dynamic_config.c +++ b/drivers/net/dsa/sja1105/sja1105_dynamic_config.c @@ -1175,18 +1175,15 @@ const struct sja1105_dynamic_table_ops sja1110_dyn_ops[BLK_IDX_MAX_DYN] = { static int sja1105_dynamic_config_poll_valid(struct sja1105_private *priv, - struct sja1105_dyn_cmd *cmd, - const struct sja1105_dynamic_table_ops *ops) + const struct sja1105_dynamic_table_ops *ops, + void *entry, bool check_valident, + bool check_errors) { u8 packed_buf[SJA1105_MAX_DYN_CMD_SIZE] = {}; + struct sja1105_dyn_cmd cmd = {}; int rc; - /* We don't _need_ to read the full entry, just the command area which - * is a fixed SJA1105_SIZE_DYN_CMD. But our cmd_packing() API expects a - * buffer that contains the full entry too. Additionally, our API - * doesn't really know how many bytes into the buffer does the command - * area really begin. So just read back the whole entry. - */ + /* Read back the whole entry + command structure. */ rc = sja1105_xfer_buf(priv, SPI_READ, ops->addr, packed_buf, ops->packed_size); if (rc) @@ -1195,11 +1192,25 @@ sja1105_dynamic_config_poll_valid(struct sja1105_private *priv, /* Unpack the command structure, and return it to the caller in case it * needs to perform further checks on it (VALIDENT). */ - memset(cmd, 0, sizeof(*cmd)); - ops->cmd_packing(packed_buf, cmd, UNPACK); + ops->cmd_packing(packed_buf, &cmd, UNPACK); /* Hardware hasn't cleared VALID => still working on it */ - return cmd->valid ? -EAGAIN : 0; + if (cmd.valid) + return -EAGAIN; + + if (check_valident && !cmd.valident && !(ops->access & OP_VALID_ANYWAY)) + return -ENOENT; + + if (check_errors && cmd.errors) + return -EINVAL; + + /* Don't dereference possibly NULL pointer - maybe caller + * only wanted to see whether the entry existed or not. + */ + if (entry) + ops->entry_packing(packed_buf, entry, UNPACK); + + return 0; } /* Poll the dynamic config entry's control area until the hardware has @@ -1208,16 +1219,19 @@ sja1105_dynamic_config_poll_valid(struct sja1105_private *priv, */ static int sja1105_dynamic_config_wait_complete(struct sja1105_private *priv, - struct sja1105_dyn_cmd *cmd, - const struct sja1105_dynamic_table_ops *ops) + const struct sja1105_dynamic_table_ops *ops, + void *entry, bool check_valident, + bool check_errors) { - int rc; - - return read_poll_timeout(sja1105_dynamic_config_poll_valid, - rc, rc != -EAGAIN, - SJA1105_DYNAMIC_CONFIG_SLEEP_US, - SJA1105_DYNAMIC_CONFIG_TIMEOUT_US, - false, priv, cmd, ops); + int err, rc; + + err = read_poll_timeout(sja1105_dynamic_config_poll_valid, + rc, rc != -EAGAIN, + SJA1105_DYNAMIC_CONFIG_SLEEP_US, + SJA1105_DYNAMIC_CONFIG_TIMEOUT_US, + false, priv, ops, entry, check_valident, + check_errors); + return err < 0 ? err : rc; } /* Provides read access to the settings through the dynamic interface @@ -1286,25 +1300,14 @@ int sja1105_dynamic_config_read(struct sja1105_private *priv, mutex_lock(&priv->dynamic_config_lock); rc = sja1105_xfer_buf(priv, SPI_WRITE, ops->addr, packed_buf, ops->packed_size); - if (rc < 0) { - mutex_unlock(&priv->dynamic_config_lock); - return rc; - } - - rc = sja1105_dynamic_config_wait_complete(priv, &cmd, ops); - mutex_unlock(&priv->dynamic_config_lock); if (rc < 0) - return rc; + goto out; - if (!cmd.valident && !(ops->access & OP_VALID_ANYWAY)) - return -ENOENT; + rc = sja1105_dynamic_config_wait_complete(priv, ops, entry, true, false); +out: + mutex_unlock(&priv->dynamic_config_lock); - /* Don't dereference possibly NULL pointer - maybe caller - * only wanted to see whether the entry existed or not. - */ - if (entry) - ops->entry_packing(packed_buf, entry, UNPACK); - return 0; + return rc; } int sja1105_dynamic_config_write(struct sja1105_private *priv, @@ -1356,22 +1359,14 @@ int sja1105_dynamic_config_write(struct sja1105_private *priv, mutex_lock(&priv->dynamic_config_lock); rc = sja1105_xfer_buf(priv, SPI_WRITE, ops->addr, packed_buf, ops->packed_size); - if (rc < 0) { - mutex_unlock(&priv->dynamic_config_lock); - return rc; - } - - rc = sja1105_dynamic_config_wait_complete(priv, &cmd, ops); - mutex_unlock(&priv->dynamic_config_lock); if (rc < 0) - return rc; + goto out; - cmd = (struct sja1105_dyn_cmd) {0}; - ops->cmd_packing(packed_buf, &cmd, UNPACK); - if (cmd.errors) - return -EINVAL; + rc = sja1105_dynamic_config_wait_complete(priv, ops, NULL, false, true); +out: + mutex_unlock(&priv->dynamic_config_lock); - return 0; + return rc; } static u8 sja1105_crc8_add(u8 crc, u8 byte, u8 poly) diff --git a/drivers/net/dsa/sja1105/sja1105_main.c b/drivers/net/dsa/sja1105/sja1105_main.c index a23d980d28f5..1a367e64bc3b 100644 --- a/drivers/net/dsa/sja1105/sja1105_main.c +++ b/drivers/net/dsa/sja1105/sja1105_main.c @@ -1798,6 +1798,7 @@ static int sja1105_fdb_add(struct dsa_switch *ds, int port, struct dsa_db db) { struct sja1105_private *priv = ds->priv; + int rc; if (!vid) { switch (db.type) { @@ -1812,12 +1813,16 @@ static int sja1105_fdb_add(struct dsa_switch *ds, int port, } } - return priv->info->fdb_add_cmd(ds, port, addr, vid); + mutex_lock(&priv->fdb_lock); + rc = priv->info->fdb_add_cmd(ds, port, addr, vid); + mutex_unlock(&priv->fdb_lock); + + return rc; } -static int sja1105_fdb_del(struct dsa_switch *ds, int port, - const unsigned char *addr, u16 vid, - struct dsa_db db) +static int __sja1105_fdb_del(struct dsa_switch *ds, int port, + const unsigned char *addr, u16 vid, + struct dsa_db db) { struct sja1105_private *priv = ds->priv; @@ -1837,6 +1842,20 @@ static int sja1105_fdb_del(struct dsa_switch *ds, int port, return priv->info->fdb_del_cmd(ds, port, addr, vid); } +static int sja1105_fdb_del(struct dsa_switch *ds, int port, + const unsigned char *addr, u16 vid, + struct dsa_db db) +{ + struct sja1105_private *priv = ds->priv; + int rc; + + mutex_lock(&priv->fdb_lock); + rc = __sja1105_fdb_del(ds, port, addr, vid, db); + mutex_unlock(&priv->fdb_lock); + + return rc; +} + static int sja1105_fdb_dump(struct dsa_switch *ds, int port, dsa_fdb_dump_cb_t *cb, void *data) { @@ -1868,13 +1887,14 @@ static int sja1105_fdb_dump(struct dsa_switch *ds, int port, if (!(l2_lookup.destports & BIT(port))) continue; - /* We need to hide the FDB entry for unknown multicast */ - if (l2_lookup.macaddr == SJA1105_UNKNOWN_MULTICAST && - l2_lookup.mask_macaddr == SJA1105_UNKNOWN_MULTICAST) - continue; - u64_to_ether_addr(l2_lookup.macaddr, macaddr); + /* Hardware FDB is shared for fdb and mdb, "bridge fdb show" + * only wants to see unicast + */ + if (is_multicast_ether_addr(macaddr)) + continue; + /* We need to hide the dsa_8021q VLANs from the user. */ if (vid_is_dsa_8021q(l2_lookup.vlanid)) l2_lookup.vlanid = 0; @@ -1898,6 +1918,8 @@ static void sja1105_fast_age(struct dsa_switch *ds, int port) }; int i; + mutex_lock(&priv->fdb_lock); + for (i = 0; i < SJA1105_MAX_L2_LOOKUP_COUNT; i++) { struct sja1105_l2_lookup_entry l2_lookup = {0}; u8 macaddr[ETH_ALEN]; @@ -1911,7 +1933,7 @@ static void sja1105_fast_age(struct dsa_switch *ds, int port) if (rc) { dev_err(ds->dev, "Failed to read FDB: %pe\n", ERR_PTR(rc)); - return; + break; } if (!(l2_lookup.destports & BIT(port))) @@ -1923,14 +1945,16 @@ static void sja1105_fast_age(struct dsa_switch *ds, int port) u64_to_ether_addr(l2_lookup.macaddr, macaddr); - rc = sja1105_fdb_del(ds, port, macaddr, l2_lookup.vlanid, db); + rc = __sja1105_fdb_del(ds, port, macaddr, l2_lookup.vlanid, db); if (rc) { dev_err(ds->dev, "Failed to delete FDB entry %pM vid %lld: %pe\n", macaddr, l2_lookup.vlanid, ERR_PTR(rc)); - return; + break; } } + + mutex_unlock(&priv->fdb_lock); } static int sja1105_mdb_add(struct dsa_switch *ds, int port, @@ -2273,6 +2297,7 @@ int sja1105_static_config_reload(struct sja1105_private *priv, int rc, i; s64 now; + mutex_lock(&priv->fdb_lock); mutex_lock(&priv->mgmt_lock); mac = priv->static_config.tables[BLK_IDX_MAC_CONFIG].entries; @@ -2385,6 +2410,7 @@ int sja1105_static_config_reload(struct sja1105_private *priv, goto out; out: mutex_unlock(&priv->mgmt_lock); + mutex_unlock(&priv->fdb_lock); return rc; } @@ -2954,7 +2980,9 @@ static int sja1105_port_mcast_flood(struct sja1105_private *priv, int to, { struct sja1105_l2_lookup_entry *l2_lookup; struct sja1105_table *table; - int match; + int match, rc; + + mutex_lock(&priv->fdb_lock); table = &priv->static_config.tables[BLK_IDX_L2_LOOKUP]; l2_lookup = table->entries; @@ -2967,7 +2995,8 @@ static int sja1105_port_mcast_flood(struct sja1105_private *priv, int to, if (match == table->entry_count) { NL_SET_ERR_MSG_MOD(extack, "Could not find FDB entry for unknown multicast"); - return -ENOSPC; + rc = -ENOSPC; + goto out; } if (flags.val & BR_MCAST_FLOOD) @@ -2975,10 +3004,13 @@ static int sja1105_port_mcast_flood(struct sja1105_private *priv, int to, else l2_lookup[match].destports &= ~BIT(to); - return sja1105_dynamic_config_write(priv, BLK_IDX_L2_LOOKUP, - l2_lookup[match].index, - &l2_lookup[match], - true); + rc = sja1105_dynamic_config_write(priv, BLK_IDX_L2_LOOKUP, + l2_lookup[match].index, + &l2_lookup[match], true); +out: + mutex_unlock(&priv->fdb_lock); + + return rc; } static int sja1105_port_pre_bridge_flags(struct dsa_switch *ds, int port, @@ -3348,6 +3380,7 @@ static int sja1105_probe(struct spi_device *spi) mutex_init(&priv->ptp_data.lock); mutex_init(&priv->dynamic_config_lock); mutex_init(&priv->mgmt_lock); + mutex_init(&priv->fdb_lock); spin_lock_init(&priv->ts_id_lock); rc = sja1105_parse_dt(priv); diff --git a/drivers/net/ethernet/adi/adin1110.c b/drivers/net/ethernet/adi/adin1110.c index 1c009b485188..ca66b747b7c5 100644 --- a/drivers/net/ethernet/adi/adin1110.c +++ b/drivers/net/ethernet/adi/adin1110.c @@ -1385,7 +1385,7 @@ static int adin1110_fdb_add(struct adin1110_port_priv *port_priv, return -ENOMEM; other_port = priv->ports[!port_priv->nr]; - port_rules = adin1110_port_rules(port_priv, false, true); + port_rules = adin1110_port_rules(other_port, false, true); eth_broadcast_addr(mask); return adin1110_write_mac_address(other_port, mac_nr, (u8 *)fdb->addr, diff --git a/drivers/net/ethernet/amazon/ena/ena_netdev.c b/drivers/net/ethernet/amazon/ena/ena_netdev.c index ad32ca81f7ef..f955bde10cf9 100644 --- a/drivers/net/ethernet/amazon/ena/ena_netdev.c +++ b/drivers/net/ethernet/amazon/ena/ena_netdev.c @@ -1833,6 +1833,9 @@ static int ena_clean_rx_irq(struct ena_ring *rx_ring, struct napi_struct *napi, return work_done; error: + if (xdp_flags & ENA_XDP_REDIRECT) + xdp_do_flush(); + adapter = netdev_priv(rx_ring->netdev); if (rc == -ENOSPC) { diff --git a/drivers/net/ethernet/broadcom/asp2/bcmasp.c b/drivers/net/ethernet/broadcom/asp2/bcmasp.c index d63d321f3e7b..41a6098eb0c2 100644 --- a/drivers/net/ethernet/broadcom/asp2/bcmasp.c +++ b/drivers/net/ethernet/broadcom/asp2/bcmasp.c @@ -528,13 +528,16 @@ void bcmasp_netfilt_suspend(struct bcmasp_intf *intf) ASP_RX_FILTER_BLK_CTRL); } -void bcmasp_netfilt_get_all_active(struct bcmasp_intf *intf, u32 *rule_locs, - u32 *rule_cnt) +int bcmasp_netfilt_get_all_active(struct bcmasp_intf *intf, u32 *rule_locs, + u32 *rule_cnt) { struct bcmasp_priv *priv = intf->parent; int j = 0, i; for (i = 0; i < NUM_NET_FILTERS; i++) { + if (j == *rule_cnt) + return -EMSGSIZE; + if (!priv->net_filters[i].claimed || priv->net_filters[i].port != intf->port) continue; @@ -548,6 +551,8 @@ void bcmasp_netfilt_get_all_active(struct bcmasp_intf *intf, u32 *rule_locs, } *rule_cnt = j; + + return 0; } int bcmasp_netfilt_get_active(struct bcmasp_intf *intf) @@ -1300,6 +1305,7 @@ static int bcmasp_probe(struct platform_device *pdev) if (!intf) { dev_err(dev, "Cannot create eth interface %d\n", i); bcmasp_remove_intfs(priv); + of_node_put(intf_node); goto of_put_exit; } list_add_tail(&intf->list, &priv->intfs); diff --git a/drivers/net/ethernet/broadcom/asp2/bcmasp.h b/drivers/net/ethernet/broadcom/asp2/bcmasp.h index 5b512f7f5e94..ec90add6b03e 100644 --- a/drivers/net/ethernet/broadcom/asp2/bcmasp.h +++ b/drivers/net/ethernet/broadcom/asp2/bcmasp.h @@ -577,8 +577,8 @@ void bcmasp_netfilt_release(struct bcmasp_intf *intf, int bcmasp_netfilt_get_active(struct bcmasp_intf *intf); -void bcmasp_netfilt_get_all_active(struct bcmasp_intf *intf, u32 *rule_locs, - u32 *rule_cnt); +int bcmasp_netfilt_get_all_active(struct bcmasp_intf *intf, u32 *rule_locs, + u32 *rule_cnt); void bcmasp_netfilt_suspend(struct bcmasp_intf *intf); diff --git a/drivers/net/ethernet/broadcom/asp2/bcmasp_ethtool.c b/drivers/net/ethernet/broadcom/asp2/bcmasp_ethtool.c index c4f1604d5ab3..ce6a3d56fb23 100644 --- a/drivers/net/ethernet/broadcom/asp2/bcmasp_ethtool.c +++ b/drivers/net/ethernet/broadcom/asp2/bcmasp_ethtool.c @@ -335,7 +335,7 @@ static int bcmasp_get_rxnfc(struct net_device *dev, struct ethtool_rxnfc *cmd, err = bcmasp_flow_get(intf, cmd); break; case ETHTOOL_GRXCLSRLALL: - bcmasp_netfilt_get_all_active(intf, rule_locs, &cmd->rule_cnt); + err = bcmasp_netfilt_get_all_active(intf, rule_locs, &cmd->rule_cnt); cmd->data = NUM_NET_FILTERS; break; default: diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index 5cc0dbe12132..7551aa8068f8 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -2614,6 +2614,7 @@ static int bnxt_poll_nitroa0(struct napi_struct *napi, int budget) struct rx_cmp_ext *rxcmp1; u32 cp_cons, tmp_raw_cons; u32 raw_cons = cpr->cp_raw_cons; + bool flush_xdp = false; u32 rx_pkts = 0; u8 event = 0; @@ -2648,6 +2649,8 @@ static int bnxt_poll_nitroa0(struct napi_struct *napi, int budget) rx_pkts++; else if (rc == -EBUSY) /* partial completion */ break; + if (event & BNXT_REDIRECT_EVENT) + flush_xdp = true; } else if (unlikely(TX_CMP_TYPE(txcmp) == CMPL_BASE_TYPE_HWRM_DONE)) { bnxt_hwrm_handler(bp, txcmp); @@ -2667,6 +2670,8 @@ static int bnxt_poll_nitroa0(struct napi_struct *napi, int budget) if (event & BNXT_AGG_EVENT) bnxt_db_write(bp, &rxr->rx_agg_db, rxr->rx_agg_prod); + if (flush_xdp) + xdp_do_flush(); if (!bnxt_has_work(bp, cpr) && rx_pkts < budget) { napi_complete_done(napi, rx_pkts); diff --git a/drivers/net/ethernet/cadence/macb_main.c b/drivers/net/ethernet/cadence/macb_main.c index 31f664ee4d77..b940dcd3ace6 100644 --- a/drivers/net/ethernet/cadence/macb_main.c +++ b/drivers/net/ethernet/cadence/macb_main.c @@ -756,8 +756,6 @@ static void macb_mac_link_up(struct phylink_config *config, if (rx_pause) ctrl |= MACB_BIT(PAE); - macb_set_tx_clk(bp, speed); - /* Initialize rings & buffers as clearing MACB_BIT(TE) in link down * cleared the pipeline and control registers. */ @@ -777,6 +775,9 @@ static void macb_mac_link_up(struct phylink_config *config, spin_unlock_irqrestore(&bp->lock, flags); + if (!(bp->caps & MACB_CAPS_MACB_IS_EMAC)) + macb_set_tx_clk(bp, speed); + /* Enable Rx and Tx; Enable PTP unicast */ ctrl = macb_readl(bp, NCR); if (gem_has_ptp(bp)) diff --git a/drivers/net/ethernet/chelsio/inline_crypto/chtls/chtls_io.c b/drivers/net/ethernet/chelsio/inline_crypto/chtls/chtls_io.c index 5fc64e47568a..d567e42e1760 100644 --- a/drivers/net/ethernet/chelsio/inline_crypto/chtls/chtls_io.c +++ b/drivers/net/ethernet/chelsio/inline_crypto/chtls/chtls_io.c @@ -911,7 +911,7 @@ static int csk_wait_memory(struct chtls_dev *cdev, struct sock *sk, long *timeo_p) { DEFINE_WAIT_FUNC(wait, woken_wake_function); - int err = 0; + int ret, err = 0; long current_timeo; long vm_wait = 0; bool noblock; @@ -942,10 +942,13 @@ static int csk_wait_memory(struct chtls_dev *cdev, set_bit(SOCK_NOSPACE, &sk->sk_socket->flags); sk->sk_write_pending++; - sk_wait_event(sk, ¤t_timeo, sk->sk_err || - (sk->sk_shutdown & SEND_SHUTDOWN) || - (csk_mem_free(cdev, sk) && !vm_wait), &wait); + ret = sk_wait_event(sk, ¤t_timeo, sk->sk_err || + (sk->sk_shutdown & SEND_SHUTDOWN) || + (csk_mem_free(cdev, sk) && !vm_wait), + &wait); sk->sk_write_pending--; + if (ret < 0) + goto do_error; if (vm_wait) { vm_wait -= current_timeo; @@ -1348,6 +1351,7 @@ static int chtls_pt_recvmsg(struct sock *sk, struct msghdr *msg, size_t len, int copied = 0; int target; long timeo; + int ret; buffers_freed = 0; @@ -1423,7 +1427,11 @@ static int chtls_pt_recvmsg(struct sock *sk, struct msghdr *msg, size_t len, if (copied >= target) break; chtls_cleanup_rbuf(sk, copied); - sk_wait_data(sk, &timeo, NULL); + ret = sk_wait_data(sk, &timeo, NULL); + if (ret < 0) { + copied = copied ? : ret; + goto unlock; + } continue; found_ok_skb: if (!skb->len) { @@ -1518,6 +1526,8 @@ skip_copy: if (buffers_freed) chtls_cleanup_rbuf(sk, copied); + +unlock: release_sock(sk); return copied; } @@ -1534,6 +1544,7 @@ static int peekmsg(struct sock *sk, struct msghdr *msg, int copied = 0; size_t avail; /* amount of available data in current skb */ long timeo; + int ret; lock_sock(sk); timeo = sock_rcvtimeo(sk, flags & MSG_DONTWAIT); @@ -1585,7 +1596,12 @@ static int peekmsg(struct sock *sk, struct msghdr *msg, release_sock(sk); lock_sock(sk); } else { - sk_wait_data(sk, &timeo, NULL); + ret = sk_wait_data(sk, &timeo, NULL); + if (ret < 0) { + /* here 'copied' is 0 due to previous checks */ + copied = ret; + break; + } } if (unlikely(peek_seq != tp->copied_seq)) { @@ -1656,6 +1672,7 @@ int chtls_recvmsg(struct sock *sk, struct msghdr *msg, size_t len, int copied = 0; long timeo; int target; /* Read at least this many bytes */ + int ret; buffers_freed = 0; @@ -1747,7 +1764,11 @@ int chtls_recvmsg(struct sock *sk, struct msghdr *msg, size_t len, if (copied >= target) break; chtls_cleanup_rbuf(sk, copied); - sk_wait_data(sk, &timeo, NULL); + ret = sk_wait_data(sk, &timeo, NULL); + if (ret < 0) { + copied = copied ? : ret; + goto unlock; + } continue; found_ok_skb: @@ -1816,6 +1837,7 @@ skip_copy: if (buffers_freed) chtls_cleanup_rbuf(sk, copied); +unlock: release_sock(sk); return copied; } diff --git a/drivers/net/ethernet/engleder/tsnep_ethtool.c b/drivers/net/ethernet/engleder/tsnep_ethtool.c index 716815dad7d2..65ec1abc9442 100644 --- a/drivers/net/ethernet/engleder/tsnep_ethtool.c +++ b/drivers/net/ethernet/engleder/tsnep_ethtool.c @@ -300,10 +300,8 @@ static void tsnep_ethtool_get_channels(struct net_device *netdev, { struct tsnep_adapter *adapter = netdev_priv(netdev); - ch->max_rx = adapter->num_rx_queues; - ch->max_tx = adapter->num_tx_queues; - ch->rx_count = adapter->num_rx_queues; - ch->tx_count = adapter->num_tx_queues; + ch->max_combined = adapter->num_queues; + ch->combined_count = adapter->num_queues; } static int tsnep_ethtool_get_ts_info(struct net_device *netdev, diff --git a/drivers/net/ethernet/engleder/tsnep_main.c b/drivers/net/ethernet/engleder/tsnep_main.c index f61bd89734c5..8b992dc9bb52 100644 --- a/drivers/net/ethernet/engleder/tsnep_main.c +++ b/drivers/net/ethernet/engleder/tsnep_main.c @@ -87,8 +87,11 @@ static irqreturn_t tsnep_irq(int irq, void *arg) /* handle TX/RX queue 0 interrupt */ if ((active & adapter->queue[0].irq_mask) != 0) { - tsnep_disable_irq(adapter, adapter->queue[0].irq_mask); - napi_schedule(&adapter->queue[0].napi); + if (napi_schedule_prep(&adapter->queue[0].napi)) { + tsnep_disable_irq(adapter, adapter->queue[0].irq_mask); + /* schedule after masking to avoid races */ + __napi_schedule(&adapter->queue[0].napi); + } } return IRQ_HANDLED; @@ -99,8 +102,11 @@ static irqreturn_t tsnep_irq_txrx(int irq, void *arg) struct tsnep_queue *queue = arg; /* handle TX/RX queue interrupt */ - tsnep_disable_irq(queue->adapter, queue->irq_mask); - napi_schedule(&queue->napi); + if (napi_schedule_prep(&queue->napi)) { + tsnep_disable_irq(queue->adapter, queue->irq_mask); + /* schedule after masking to avoid races */ + __napi_schedule(&queue->napi); + } return IRQ_HANDLED; } @@ -1728,6 +1734,10 @@ static int tsnep_poll(struct napi_struct *napi, int budget) if (queue->tx) complete = tsnep_tx_poll(queue->tx, budget); + /* handle case where we are called by netpoll with a budget of 0 */ + if (unlikely(budget <= 0)) + return budget; + if (queue->rx) { done = queue->rx->xsk_pool ? tsnep_rx_poll_zc(queue->rx, napi, budget) : diff --git a/drivers/net/ethernet/google/gve/gve_rx.c b/drivers/net/ethernet/google/gve/gve_rx.c index d1da7413dc4d..e84a066aa1a4 100644 --- a/drivers/net/ethernet/google/gve/gve_rx.c +++ b/drivers/net/ethernet/google/gve/gve_rx.c @@ -146,7 +146,7 @@ static int gve_prefill_rx_pages(struct gve_rx_ring *rx) err = gve_rx_alloc_buffer(priv, &priv->pdev->dev, &rx->data.page_info[i], &rx->data.data_ring[i]); if (err) - goto alloc_err; + goto alloc_err_rda; } if (!rx->data.raw_addressing) { @@ -171,12 +171,26 @@ static int gve_prefill_rx_pages(struct gve_rx_ring *rx) return slots; alloc_err_qpl: + /* Fully free the copy pool pages. */ while (j--) { page_ref_sub(rx->qpl_copy_pool[j].page, rx->qpl_copy_pool[j].pagecnt_bias - 1); put_page(rx->qpl_copy_pool[j].page); } -alloc_err: + + /* Do not fully free QPL pages - only remove the bias added in this + * function with gve_setup_rx_buffer. + */ + while (i--) + page_ref_sub(rx->data.page_info[i].page, + rx->data.page_info[i].pagecnt_bias - 1); + + gve_unassign_qpl(priv, rx->data.qpl->id); + rx->data.qpl = NULL; + + return err; + +alloc_err_rda: while (i--) gve_rx_free_buffer(&priv->pdev->dev, &rx->data.page_info[i], diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3_enet.c b/drivers/net/ethernet/hisilicon/hns3/hns3_enet.c index b4895c7b3efd..cf50368441b7 100644 --- a/drivers/net/ethernet/hisilicon/hns3/hns3_enet.c +++ b/drivers/net/ethernet/hisilicon/hns3/hns3_enet.c @@ -3353,6 +3353,15 @@ static void hns3_set_default_feature(struct net_device *netdev) NETIF_F_HW_TC); netdev->hw_enc_features |= netdev->vlan_features | NETIF_F_TSO_MANGLEID; + + /* The device_version V3 hardware can't offload the checksum for IP in + * GRE packets, but can do it for NvGRE. So default to disable the + * checksum and GSO offload for GRE. + */ + if (ae_dev->dev_version > HNAE3_DEVICE_VERSION_V2) { + netdev->features &= ~NETIF_F_GSO_GRE; + netdev->features &= ~NETIF_F_GSO_GRE_CSUM; + } } static int hns3_alloc_buffer(struct hns3_enet_ring *ring, diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.c b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.c index 8ca368424436..c42574e29747 100644 --- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.c +++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.c @@ -3564,9 +3564,14 @@ static u32 hclge_check_event_cause(struct hclge_dev *hdev, u32 *clearval) static void hclge_clear_event_cause(struct hclge_dev *hdev, u32 event_type, u32 regclr) { +#define HCLGE_IMP_RESET_DELAY 5 + switch (event_type) { case HCLGE_VECTOR0_EVENT_PTP: case HCLGE_VECTOR0_EVENT_RST: + if (regclr == BIT(HCLGE_VECTOR0_IMPRESET_INT_B)) + mdelay(HCLGE_IMP_RESET_DELAY); + hclge_write_dev(&hdev->hw, HCLGE_MISC_RESET_STS_REG, regclr); break; case HCLGE_VECTOR0_EVENT_MBX: @@ -7348,6 +7353,12 @@ static int hclge_del_cls_flower(struct hnae3_handle *handle, ret = hclge_fd_tcam_config(hdev, HCLGE_FD_STAGE_1, true, rule->location, NULL, false); if (ret) { + /* if tcam config fail, set rule state to TO_DEL, + * so the rule will be deleted when periodic + * task being scheduled. + */ + hclge_update_fd_list(hdev, HCLGE_FD_TO_DEL, rule->location, NULL); + set_bit(HCLGE_STATE_FD_TBL_CHANGED, &hdev->state); spin_unlock_bh(&hdev->fd_rule_lock); return ret; } @@ -8824,7 +8835,7 @@ static void hclge_update_overflow_flags(struct hclge_vport *vport, if (mac_type == HCLGE_MAC_ADDR_UC) { if (is_all_added) vport->overflow_promisc_flags &= ~HNAE3_OVERFLOW_UPE; - else + else if (hclge_is_umv_space_full(vport, true)) vport->overflow_promisc_flags |= HNAE3_OVERFLOW_UPE; } else { if (is_all_added) diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.c b/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.c index 7a2f9233d695..a4d68fb216fb 100644 --- a/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.c +++ b/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.c @@ -1855,7 +1855,8 @@ static void hclgevf_periodic_service_task(struct hclgevf_dev *hdev) unsigned long delta = round_jiffies_relative(HZ); struct hnae3_handle *handle = &hdev->nic; - if (test_bit(HCLGEVF_STATE_RST_FAIL, &hdev->state)) + if (test_bit(HCLGEVF_STATE_RST_FAIL, &hdev->state) || + test_bit(HCLGE_COMM_STATE_CMD_DISABLE, &hdev->hw.hw.comm_state)) return; if (time_is_after_jiffies(hdev->last_serv_processed + HZ)) { diff --git a/drivers/net/ethernet/huawei/hinic/hinic_port.c b/drivers/net/ethernet/huawei/hinic/hinic_port.c index 9406237c461e..f81a43d2cdfc 100644 --- a/drivers/net/ethernet/huawei/hinic/hinic_port.c +++ b/drivers/net/ethernet/huawei/hinic/hinic_port.c @@ -456,9 +456,6 @@ int hinic_set_vlan_fliter(struct hinic_dev *nic_dev, u32 en) u16 out_size = sizeof(vlan_filter); int err; - if (!hwdev) - return -EINVAL; - vlan_filter.func_idx = HINIC_HWIF_FUNC_IDX(hwif); vlan_filter.enable = en; diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c index 832a2ae01950..a8d79ee350f8 100644 --- a/drivers/net/ethernet/ibm/ibmveth.c +++ b/drivers/net/ethernet/ibm/ibmveth.c @@ -1303,24 +1303,23 @@ static void ibmveth_rx_csum_helper(struct sk_buff *skb, * the user space for finding a flow. During this process, OVS computes * checksum on the first packet when CHECKSUM_PARTIAL flag is set. * - * So, re-compute TCP pseudo header checksum when configured for - * trunk mode. + * So, re-compute TCP pseudo header checksum. */ + if (iph_proto == IPPROTO_TCP) { struct tcphdr *tcph = (struct tcphdr *)(skb->data + iphlen); + if (tcph->check == 0x0000) { /* Recompute TCP pseudo header checksum */ - if (adapter->is_active_trunk) { - tcphdrlen = skb->len - iphlen; - if (skb_proto == ETH_P_IP) - tcph->check = - ~csum_tcpudp_magic(iph->saddr, - iph->daddr, tcphdrlen, iph_proto, 0); - else if (skb_proto == ETH_P_IPV6) - tcph->check = - ~csum_ipv6_magic(&iph6->saddr, - &iph6->daddr, tcphdrlen, iph_proto, 0); - } + tcphdrlen = skb->len - iphlen; + if (skb_proto == ETH_P_IP) + tcph->check = + ~csum_tcpudp_magic(iph->saddr, + iph->daddr, tcphdrlen, iph_proto, 0); + else if (skb_proto == ETH_P_IPV6) + tcph->check = + ~csum_ipv6_magic(&iph6->saddr, + &iph6->daddr, tcphdrlen, iph_proto, 0); /* Setup SKB fields for checksum offload */ skb_partial_csum_set(skb, iphlen, offsetof(struct tcphdr, check)); diff --git a/drivers/net/ethernet/intel/i40e/i40e_common.c b/drivers/net/ethernet/intel/i40e/i40e_common.c index eeef20f77106..1b493854f522 100644 --- a/drivers/net/ethernet/intel/i40e/i40e_common.c +++ b/drivers/net/ethernet/intel/i40e/i40e_common.c @@ -1082,7 +1082,7 @@ void i40e_clear_hw(struct i40e_hw *hw) I40E_PFLAN_QALLOC_FIRSTQ_SHIFT; j = (val & I40E_PFLAN_QALLOC_LASTQ_MASK) >> I40E_PFLAN_QALLOC_LASTQ_SHIFT; - if (val & I40E_PFLAN_QALLOC_VALID_MASK) + if (val & I40E_PFLAN_QALLOC_VALID_MASK && j >= base_queue) num_queues = (j - base_queue) + 1; else num_queues = 0; @@ -1092,7 +1092,7 @@ void i40e_clear_hw(struct i40e_hw *hw) I40E_PF_VT_PFALLOC_FIRSTVF_SHIFT; j = (val & I40E_PF_VT_PFALLOC_LASTVF_MASK) >> I40E_PF_VT_PFALLOC_LASTVF_SHIFT; - if (val & I40E_PF_VT_PFALLOC_VALID_MASK) + if (val & I40E_PF_VT_PFALLOC_VALID_MASK && j >= i) num_vfs = (j - i) + 1; else num_vfs = 0; diff --git a/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c b/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c index 8ea1a238dcef..d3d6415553ed 100644 --- a/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c +++ b/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c @@ -4475,9 +4475,7 @@ int i40e_ndo_set_vf_port_vlan(struct net_device *netdev, int vf_id, goto error_pvid; i40e_vlan_stripping_enable(vsi); - i40e_vc_reset_vf(vf, true); - /* During reset the VF got a new VSI, so refresh a pointer. */ - vsi = pf->vsi[vf->lan_vsi_idx]; + /* Locked once because multiple functions below iterate list */ spin_lock_bh(&vsi->mac_filter_hash_lock); @@ -4563,6 +4561,10 @@ int i40e_ndo_set_vf_port_vlan(struct net_device *netdev, int vf_id, */ vf->port_vlan_id = le16_to_cpu(vsi->info.pvid); + i40e_vc_reset_vf(vf, true); + /* During reset the VF got a new VSI, so refresh a pointer. */ + vsi = pf->vsi[vf->lan_vsi_idx]; + ret = i40e_config_vf_promiscuous_mode(vf, vsi->id, allmulti, alluni); if (ret) { dev_err(&pf->pdev->dev, "Unable to config vf promiscuous mode\n"); diff --git a/drivers/net/ethernet/intel/iavf/iavf.h b/drivers/net/ethernet/intel/iavf/iavf.h index 85fba85fbb23..e110ba346185 100644 --- a/drivers/net/ethernet/intel/iavf/iavf.h +++ b/drivers/net/ethernet/intel/iavf/iavf.h @@ -521,7 +521,7 @@ void iavf_down(struct iavf_adapter *adapter); int iavf_process_config(struct iavf_adapter *adapter); int iavf_parse_vf_resource_msg(struct iavf_adapter *adapter); void iavf_schedule_reset(struct iavf_adapter *adapter, u64 flags); -void iavf_schedule_request_stats(struct iavf_adapter *adapter); +void iavf_schedule_aq_request(struct iavf_adapter *adapter, u64 flags); void iavf_schedule_finish_config(struct iavf_adapter *adapter); void iavf_reset(struct iavf_adapter *adapter); void iavf_set_ethtool_ops(struct net_device *netdev); diff --git a/drivers/net/ethernet/intel/iavf/iavf_ethtool.c b/drivers/net/ethernet/intel/iavf/iavf_ethtool.c index a34303ad057d..90397293525f 100644 --- a/drivers/net/ethernet/intel/iavf/iavf_ethtool.c +++ b/drivers/net/ethernet/intel/iavf/iavf_ethtool.c @@ -362,7 +362,7 @@ static void iavf_get_ethtool_stats(struct net_device *netdev, unsigned int i; /* Explicitly request stats refresh */ - iavf_schedule_request_stats(adapter); + iavf_schedule_aq_request(adapter, IAVF_FLAG_AQ_REQUEST_STATS); iavf_add_ethtool_stats(&data, adapter, iavf_gstrings_stats); diff --git a/drivers/net/ethernet/intel/iavf/iavf_main.c b/drivers/net/ethernet/intel/iavf/iavf_main.c index 7b300c86ceda..6a2e6d64bc3a 100644 --- a/drivers/net/ethernet/intel/iavf/iavf_main.c +++ b/drivers/net/ethernet/intel/iavf/iavf_main.c @@ -314,15 +314,13 @@ void iavf_schedule_reset(struct iavf_adapter *adapter, u64 flags) } /** - * iavf_schedule_request_stats - Set the flags and schedule statistics request + * iavf_schedule_aq_request - Set the flags and schedule aq request * @adapter: board private structure - * - * Sets IAVF_FLAG_AQ_REQUEST_STATS flag so iavf_watchdog_task() will explicitly - * request and refresh ethtool stats + * @flags: requested aq flags **/ -void iavf_schedule_request_stats(struct iavf_adapter *adapter) +void iavf_schedule_aq_request(struct iavf_adapter *adapter, u64 flags) { - adapter->aq_required |= IAVF_FLAG_AQ_REQUEST_STATS; + adapter->aq_required |= flags; mod_delayed_work(adapter->wq, &adapter->watchdog_task, 0); } @@ -823,7 +821,7 @@ iavf_vlan_filter *iavf_add_vlan(struct iavf_adapter *adapter, list_add_tail(&f->list, &adapter->vlan_filter_list); f->state = IAVF_VLAN_ADD; adapter->num_vlan_filters++; - adapter->aq_required |= IAVF_FLAG_AQ_ADD_VLAN_FILTER; + iavf_schedule_aq_request(adapter, IAVF_FLAG_AQ_ADD_VLAN_FILTER); } clearout: @@ -845,7 +843,7 @@ static void iavf_del_vlan(struct iavf_adapter *adapter, struct iavf_vlan vlan) f = iavf_find_vlan(adapter, vlan); if (f) { f->state = IAVF_VLAN_REMOVE; - adapter->aq_required |= IAVF_FLAG_AQ_DEL_VLAN_FILTER; + iavf_schedule_aq_request(adapter, IAVF_FLAG_AQ_DEL_VLAN_FILTER); } spin_unlock_bh(&adapter->mac_vlan_list_lock); @@ -1421,7 +1419,8 @@ void iavf_down(struct iavf_adapter *adapter) iavf_clear_fdir_filters(adapter); iavf_clear_adv_rss_conf(adapter); - if (!(adapter->flags & IAVF_FLAG_PF_COMMS_FAILED)) { + if (!(adapter->flags & IAVF_FLAG_PF_COMMS_FAILED) && + !(test_bit(__IAVF_IN_REMOVE_TASK, &adapter->crit_section))) { /* cancel any current operation */ adapter->current_op = VIRTCHNL_OP_UNKNOWN; /* Schedule operations to close down the HW. Don't wait diff --git a/drivers/net/ethernet/intel/ice/ice_lag.c b/drivers/net/ethernet/intel/ice/ice_lag.c index 4f39863b5537..7b1256992dcf 100644 --- a/drivers/net/ethernet/intel/ice/ice_lag.c +++ b/drivers/net/ethernet/intel/ice/ice_lag.c @@ -2093,3 +2093,35 @@ lag_rebuild_out: } mutex_unlock(&pf->lag_mutex); } + +/** + * ice_lag_is_switchdev_running + * @pf: pointer to PF structure + * + * Check if switchdev is running on any of the interfaces connected to lag. + */ +bool ice_lag_is_switchdev_running(struct ice_pf *pf) +{ + struct ice_lag *lag = pf->lag; + struct net_device *tmp_nd; + + if (!ice_is_feature_supported(pf, ICE_F_SRIOV_LAG) || !lag) + return false; + + rcu_read_lock(); + for_each_netdev_in_bond_rcu(lag->upper_netdev, tmp_nd) { + struct ice_netdev_priv *priv = netdev_priv(tmp_nd); + + if (!netif_is_ice(tmp_nd) || !priv || !priv->vsi || + !priv->vsi->back) + continue; + + if (ice_is_switchdev_running(priv->vsi->back)) { + rcu_read_unlock(); + return true; + } + } + rcu_read_unlock(); + + return false; +} diff --git a/drivers/net/ethernet/intel/ice/ice_lag.h b/drivers/net/ethernet/intel/ice/ice_lag.h index 18075b82485a..facb6c894b6d 100644 --- a/drivers/net/ethernet/intel/ice/ice_lag.h +++ b/drivers/net/ethernet/intel/ice/ice_lag.h @@ -62,4 +62,5 @@ void ice_lag_move_new_vf_nodes(struct ice_vf *vf); int ice_init_lag(struct ice_pf *pf); void ice_deinit_lag(struct ice_pf *pf); void ice_lag_rebuild(struct ice_pf *pf); +bool ice_lag_is_switchdev_running(struct ice_pf *pf); #endif /* _ICE_LAG_H_ */ diff --git a/drivers/net/ethernet/intel/ice/ice_lib.c b/drivers/net/ethernet/intel/ice/ice_lib.c index 201570cd2e0b..73bbf06a76db 100644 --- a/drivers/net/ethernet/intel/ice/ice_lib.c +++ b/drivers/net/ethernet/intel/ice/ice_lib.c @@ -1201,8 +1201,7 @@ static void ice_set_rss_vsi_ctx(struct ice_vsi_ctx *ctxt, struct ice_vsi *vsi) ctxt->info.q_opt_rss = ((lut_type << ICE_AQ_VSI_Q_OPT_RSS_LUT_S) & ICE_AQ_VSI_Q_OPT_RSS_LUT_M) | - ((hash_type << ICE_AQ_VSI_Q_OPT_RSS_HASH_S) & - ICE_AQ_VSI_Q_OPT_RSS_HASH_M); + (hash_type & ICE_AQ_VSI_Q_OPT_RSS_HASH_M); } static void @@ -3575,6 +3574,12 @@ int ice_set_dflt_vsi(struct ice_vsi *vsi) dev = ice_pf_to_dev(vsi->back); + if (ice_lag_is_switchdev_running(vsi->back)) { + dev_dbg(dev, "VSI %d passed is a part of LAG containing interfaces in switchdev mode, nothing to do\n", + vsi->vsi_num); + return 0; + } + /* the VSI passed in is already the default VSI */ if (ice_is_vsi_dflt_vsi(vsi)) { dev_dbg(dev, "VSI %d passed in is already the default forwarding VSI, nothing to do\n", diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c index c8286adae946..7784135160fd 100644 --- a/drivers/net/ethernet/intel/ice/ice_main.c +++ b/drivers/net/ethernet/intel/ice/ice_main.c @@ -6,6 +6,7 @@ #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt #include +#include #include "ice.h" #include "ice_base.h" #include "ice_lib.h" @@ -4683,6 +4684,9 @@ static void ice_init_features(struct ice_pf *pf) static void ice_deinit_features(struct ice_pf *pf) { + if (ice_is_safe_mode(pf)) + return; + ice_deinit_lag(pf); if (test_bit(ICE_FLAG_DCB_CAPABLE, pf->flags)) ice_cfg_lldp_mib_change(&pf->hw, false); @@ -5014,6 +5018,20 @@ ice_probe(struct pci_dev *pdev, const struct pci_device_id __always_unused *ent) return -EINVAL; } + /* when under a kdump kernel initiate a reset before enabling the + * device in order to clear out any pending DMA transactions. These + * transactions can cause some systems to machine check when doing + * the pcim_enable_device() below. + */ + if (is_kdump_kernel()) { + pci_save_state(pdev); + pci_clear_master(pdev); + err = pcie_flr(pdev); + if (err) + return err; + pci_restore_state(pdev); + } + /* this driver uses devres, see * Documentation/driver-api/driver-model/devres.rst */ diff --git a/drivers/net/ethernet/intel/ice/ice_virtchnl.c b/drivers/net/ethernet/intel/ice/ice_virtchnl.c index b03426ac932b..db97353efd06 100644 --- a/drivers/net/ethernet/intel/ice/ice_virtchnl.c +++ b/drivers/net/ethernet/intel/ice/ice_virtchnl.c @@ -2617,12 +2617,14 @@ static int ice_vc_query_rxdid(struct ice_vf *vf) goto err; } - /* Read flexiflag registers to determine whether the - * corresponding RXDID is configured and supported or not. - * Since Legacy 16byte descriptor format is not supported, - * start from Legacy 32byte descriptor. + /* RXDIDs supported by DDP package can be read from the register + * to get the supported RXDID bitmap. But the legacy 32byte RXDID + * is not listed in DDP package, add it in the bitmap manually. + * Legacy 16byte descriptor is not supported. */ - for (i = ICE_RXDID_LEGACY_1; i < ICE_FLEX_DESC_RXDID_MAX_NUM; i++) { + rxdid->supported_rxdids |= BIT(ICE_RXDID_LEGACY_1); + + for (i = ICE_RXDID_FLEX_NIC; i < ICE_FLEX_DESC_RXDID_MAX_NUM; i++) { regval = rd32(hw, GLFLXP_RXDID_FLAGS(i, 0)); if ((regval >> GLFLXP_RXDID_FLAGS_FLEXIFLAG_4N_S) & GLFLXP_RXDID_FLAGS_FLEXIFLAG_4N_M) diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c index 13ba9c74bd84..76b34cee1da3 100644 --- a/drivers/net/ethernet/intel/igb/igb_main.c +++ b/drivers/net/ethernet/intel/igb/igb_main.c @@ -3827,8 +3827,11 @@ static int igb_enable_sriov(struct pci_dev *pdev, int num_vfs, bool reinit) } /* only call pci_enable_sriov() if no VFs are allocated already */ - if (!old_vfs) + if (!old_vfs) { err = pci_enable_sriov(pdev, adapter->vfs_allocated_count); + if (err) + goto err_out; + } goto out; diff --git a/drivers/net/ethernet/intel/igc/igc_ethtool.c b/drivers/net/ethernet/intel/igc/igc_ethtool.c index 93bce729be76..7ab6dd58e400 100644 --- a/drivers/net/ethernet/intel/igc/igc_ethtool.c +++ b/drivers/net/ethernet/intel/igc/igc_ethtool.c @@ -868,6 +868,18 @@ static void igc_ethtool_get_stats(struct net_device *netdev, spin_unlock(&adapter->stats64_lock); } +static int igc_ethtool_get_previous_rx_coalesce(struct igc_adapter *adapter) +{ + return (adapter->rx_itr_setting <= 3) ? + adapter->rx_itr_setting : adapter->rx_itr_setting >> 2; +} + +static int igc_ethtool_get_previous_tx_coalesce(struct igc_adapter *adapter) +{ + return (adapter->tx_itr_setting <= 3) ? + adapter->tx_itr_setting : adapter->tx_itr_setting >> 2; +} + static int igc_ethtool_get_coalesce(struct net_device *netdev, struct ethtool_coalesce *ec, struct kernel_ethtool_coalesce *kernel_coal, @@ -875,17 +887,8 @@ static int igc_ethtool_get_coalesce(struct net_device *netdev, { struct igc_adapter *adapter = netdev_priv(netdev); - if (adapter->rx_itr_setting <= 3) - ec->rx_coalesce_usecs = adapter->rx_itr_setting; - else - ec->rx_coalesce_usecs = adapter->rx_itr_setting >> 2; - - if (!(adapter->flags & IGC_FLAG_QUEUE_PAIRS)) { - if (adapter->tx_itr_setting <= 3) - ec->tx_coalesce_usecs = adapter->tx_itr_setting; - else - ec->tx_coalesce_usecs = adapter->tx_itr_setting >> 2; - } + ec->rx_coalesce_usecs = igc_ethtool_get_previous_rx_coalesce(adapter); + ec->tx_coalesce_usecs = igc_ethtool_get_previous_tx_coalesce(adapter); return 0; } @@ -910,8 +913,12 @@ static int igc_ethtool_set_coalesce(struct net_device *netdev, ec->tx_coalesce_usecs == 2) return -EINVAL; - if ((adapter->flags & IGC_FLAG_QUEUE_PAIRS) && ec->tx_coalesce_usecs) + if ((adapter->flags & IGC_FLAG_QUEUE_PAIRS) && + ec->tx_coalesce_usecs != igc_ethtool_get_previous_tx_coalesce(adapter)) { + NL_SET_ERR_MSG_MOD(extack, + "Queue Pair mode enabled, both Rx and Tx coalescing controlled by rx-usecs"); return -EINVAL; + } /* If ITR is disabled, disable DMAC */ if (ec->rx_coalesce_usecs == 0) { diff --git a/drivers/net/ethernet/intel/igc/igc_main.c b/drivers/net/ethernet/intel/igc/igc_main.c index 293b45717683..98de34d0ce07 100644 --- a/drivers/net/ethernet/intel/igc/igc_main.c +++ b/drivers/net/ethernet/intel/igc/igc_main.c @@ -6491,7 +6491,7 @@ static int igc_xdp_xmit(struct net_device *dev, int num_frames, struct igc_ring *ring; int i, drops; - if (unlikely(test_bit(__IGC_DOWN, &adapter->state))) + if (unlikely(!netif_carrier_ok(dev))) return -ENETDOWN; if (unlikely(flags & ~XDP_XMIT_FLAGS_MASK)) diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_ptp.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_ptp.c index 0310af851086..9339edbd9082 100644 --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_ptp.c +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_ptp.c @@ -979,6 +979,7 @@ static int ixgbe_ptp_set_timestamp_mode(struct ixgbe_adapter *adapter, u32 tsync_tx_ctl = IXGBE_TSYNCTXCTL_ENABLED; u32 tsync_rx_ctl = IXGBE_TSYNCRXCTL_ENABLED; u32 tsync_rx_mtrl = PTP_EV_PORT << 16; + u32 aflags = adapter->flags; bool is_l2 = false; u32 regval; @@ -996,20 +997,20 @@ static int ixgbe_ptp_set_timestamp_mode(struct ixgbe_adapter *adapter, case HWTSTAMP_FILTER_NONE: tsync_rx_ctl = 0; tsync_rx_mtrl = 0; - adapter->flags &= ~(IXGBE_FLAG_RX_HWTSTAMP_ENABLED | - IXGBE_FLAG_RX_HWTSTAMP_IN_REGISTER); + aflags &= ~(IXGBE_FLAG_RX_HWTSTAMP_ENABLED | + IXGBE_FLAG_RX_HWTSTAMP_IN_REGISTER); break; case HWTSTAMP_FILTER_PTP_V1_L4_SYNC: tsync_rx_ctl |= IXGBE_TSYNCRXCTL_TYPE_L4_V1; tsync_rx_mtrl |= IXGBE_RXMTRL_V1_SYNC_MSG; - adapter->flags |= (IXGBE_FLAG_RX_HWTSTAMP_ENABLED | - IXGBE_FLAG_RX_HWTSTAMP_IN_REGISTER); + aflags |= (IXGBE_FLAG_RX_HWTSTAMP_ENABLED | + IXGBE_FLAG_RX_HWTSTAMP_IN_REGISTER); break; case HWTSTAMP_FILTER_PTP_V1_L4_DELAY_REQ: tsync_rx_ctl |= IXGBE_TSYNCRXCTL_TYPE_L4_V1; tsync_rx_mtrl |= IXGBE_RXMTRL_V1_DELAY_REQ_MSG; - adapter->flags |= (IXGBE_FLAG_RX_HWTSTAMP_ENABLED | - IXGBE_FLAG_RX_HWTSTAMP_IN_REGISTER); + aflags |= (IXGBE_FLAG_RX_HWTSTAMP_ENABLED | + IXGBE_FLAG_RX_HWTSTAMP_IN_REGISTER); break; case HWTSTAMP_FILTER_PTP_V2_EVENT: case HWTSTAMP_FILTER_PTP_V2_L2_EVENT: @@ -1023,8 +1024,8 @@ static int ixgbe_ptp_set_timestamp_mode(struct ixgbe_adapter *adapter, tsync_rx_ctl |= IXGBE_TSYNCRXCTL_TYPE_EVENT_V2; is_l2 = true; config->rx_filter = HWTSTAMP_FILTER_PTP_V2_EVENT; - adapter->flags |= (IXGBE_FLAG_RX_HWTSTAMP_ENABLED | - IXGBE_FLAG_RX_HWTSTAMP_IN_REGISTER); + aflags |= (IXGBE_FLAG_RX_HWTSTAMP_ENABLED | + IXGBE_FLAG_RX_HWTSTAMP_IN_REGISTER); break; case HWTSTAMP_FILTER_PTP_V1_L4_EVENT: case HWTSTAMP_FILTER_NTP_ALL: @@ -1035,7 +1036,7 @@ static int ixgbe_ptp_set_timestamp_mode(struct ixgbe_adapter *adapter, if (hw->mac.type >= ixgbe_mac_X550) { tsync_rx_ctl |= IXGBE_TSYNCRXCTL_TYPE_ALL; config->rx_filter = HWTSTAMP_FILTER_ALL; - adapter->flags |= IXGBE_FLAG_RX_HWTSTAMP_ENABLED; + aflags |= IXGBE_FLAG_RX_HWTSTAMP_ENABLED; break; } fallthrough; @@ -1046,8 +1047,6 @@ static int ixgbe_ptp_set_timestamp_mode(struct ixgbe_adapter *adapter, * Delay_Req messages and hardware does not support * timestamping all packets => return error */ - adapter->flags &= ~(IXGBE_FLAG_RX_HWTSTAMP_ENABLED | - IXGBE_FLAG_RX_HWTSTAMP_IN_REGISTER); config->rx_filter = HWTSTAMP_FILTER_NONE; return -ERANGE; } @@ -1079,8 +1078,8 @@ static int ixgbe_ptp_set_timestamp_mode(struct ixgbe_adapter *adapter, IXGBE_TSYNCRXCTL_TYPE_ALL | IXGBE_TSYNCRXCTL_TSIP_UT_EN; config->rx_filter = HWTSTAMP_FILTER_ALL; - adapter->flags |= IXGBE_FLAG_RX_HWTSTAMP_ENABLED; - adapter->flags &= ~IXGBE_FLAG_RX_HWTSTAMP_IN_REGISTER; + aflags |= IXGBE_FLAG_RX_HWTSTAMP_ENABLED; + aflags &= ~IXGBE_FLAG_RX_HWTSTAMP_IN_REGISTER; is_l2 = true; break; default: @@ -1113,6 +1112,9 @@ static int ixgbe_ptp_set_timestamp_mode(struct ixgbe_adapter *adapter, IXGBE_WRITE_FLUSH(hw); + /* configure adapter flags only when HW is actually configured */ + adapter->flags = aflags; + /* clear TX/RX time stamp registers, just to be sure */ ixgbe_ptp_clear_tx_timestamp(adapter); IXGBE_READ_REG(hw, IXGBE_RXSTMPH); diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_sriov.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_sriov.c index 29cc60988071..ea88ac04ab9a 100644 --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_sriov.c +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_sriov.c @@ -28,6 +28,9 @@ static inline void ixgbe_alloc_vf_macvlans(struct ixgbe_adapter *adapter, struct vf_macvlans *mv_list; int num_vf_macvlans, i; + /* Initialize list of VF macvlans */ + INIT_LIST_HEAD(&adapter->vf_mvs.l); + num_vf_macvlans = hw->mac.num_rar_entries - (IXGBE_MAX_PF_MACVLANS + 1 + num_vfs); if (!num_vf_macvlans) @@ -36,8 +39,6 @@ static inline void ixgbe_alloc_vf_macvlans(struct ixgbe_adapter *adapter, mv_list = kcalloc(num_vf_macvlans, sizeof(struct vf_macvlans), GFP_KERNEL); if (mv_list) { - /* Initialize list of VF macvlans */ - INIT_LIST_HEAD(&adapter->vf_mvs.l); for (i = 0; i < num_vf_macvlans; i++) { mv_list[i].vf = -1; mv_list[i].free = true; diff --git a/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c b/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c index eb74ccddb440..21c3f9b015c8 100644 --- a/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c +++ b/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c @@ -5586,6 +5586,11 @@ static int mvpp2_ethtool_get_rxnfc(struct net_device *dev, break; case ETHTOOL_GRXCLSRLALL: for (i = 0; i < MVPP2_N_RFS_ENTRIES_PER_FLOW; i++) { + if (loc == info->rule_cnt) { + ret = -EMSGSIZE; + break; + } + if (port->rfs_rules[i]) rules[loc++] = i; } diff --git a/drivers/net/ethernet/marvell/octeon_ep/octep_main.c b/drivers/net/ethernet/marvell/octeon_ep/octep_main.c index 4424de2ffd70..5b46ca47c8e5 100644 --- a/drivers/net/ethernet/marvell/octeon_ep/octep_main.c +++ b/drivers/net/ethernet/marvell/octeon_ep/octep_main.c @@ -715,32 +715,31 @@ static netdev_tx_t octep_start_xmit(struct sk_buff *skb, hw_desc->dptr = tx_buffer->sglist_dma; } - /* Flush the hw descriptor before writing to doorbell */ - wmb(); - - /* Ring Doorbell to notify the NIC there is a new packet */ - writel(1, iq->doorbell_reg); + netdev_tx_sent_queue(iq->netdev_q, skb->len); + skb_tx_timestamp(skb); atomic_inc(&iq->instr_pending); wi++; if (wi == iq->max_count) wi = 0; iq->host_write_index = wi; + /* Flush the hw descriptor before writing to doorbell */ + wmb(); - netdev_tx_sent_queue(iq->netdev_q, skb->len); + /* Ring Doorbell to notify the NIC there is a new packet */ + writel(1, iq->doorbell_reg); iq->stats.instr_posted++; - skb_tx_timestamp(skb); return NETDEV_TX_OK; dma_map_sg_err: if (si > 0) { dma_unmap_single(iq->dev, sglist[0].dma_ptr[0], - sglist[0].len[0], DMA_TO_DEVICE); - sglist[0].len[0] = 0; + sglist[0].len[3], DMA_TO_DEVICE); + sglist[0].len[3] = 0; } while (si > 1) { dma_unmap_page(iq->dev, sglist[si >> 2].dma_ptr[si & 3], - sglist[si >> 2].len[si & 3], DMA_TO_DEVICE); - sglist[si >> 2].len[si & 3] = 0; + sglist[si >> 2].len[3 - (si & 3)], DMA_TO_DEVICE); + sglist[si >> 2].len[3 - (si & 3)] = 0; si--; } tx_buffer->gather = 0; diff --git a/drivers/net/ethernet/marvell/octeon_ep/octep_tx.c b/drivers/net/ethernet/marvell/octeon_ep/octep_tx.c index 5a520d37bea0..d0adb82d65c3 100644 --- a/drivers/net/ethernet/marvell/octeon_ep/octep_tx.c +++ b/drivers/net/ethernet/marvell/octeon_ep/octep_tx.c @@ -69,12 +69,12 @@ int octep_iq_process_completions(struct octep_iq *iq, u16 budget) compl_sg++; dma_unmap_single(iq->dev, tx_buffer->sglist[0].dma_ptr[0], - tx_buffer->sglist[0].len[0], DMA_TO_DEVICE); + tx_buffer->sglist[0].len[3], DMA_TO_DEVICE); i = 1; /* entry 0 is main skb, unmapped above */ while (frags--) { dma_unmap_page(iq->dev, tx_buffer->sglist[i >> 2].dma_ptr[i & 3], - tx_buffer->sglist[i >> 2].len[i & 3], DMA_TO_DEVICE); + tx_buffer->sglist[i >> 2].len[3 - (i & 3)], DMA_TO_DEVICE); i++; } @@ -131,13 +131,13 @@ static void octep_iq_free_pending(struct octep_iq *iq) dma_unmap_single(iq->dev, tx_buffer->sglist[0].dma_ptr[0], - tx_buffer->sglist[0].len[0], + tx_buffer->sglist[0].len[3], DMA_TO_DEVICE); i = 1; /* entry 0 is main skb, unmapped above */ while (frags--) { dma_unmap_page(iq->dev, tx_buffer->sglist[i >> 2].dma_ptr[i & 3], - tx_buffer->sglist[i >> 2].len[i & 3], DMA_TO_DEVICE); + tx_buffer->sglist[i >> 2].len[3 - (i & 3)], DMA_TO_DEVICE); i++; } diff --git a/drivers/net/ethernet/marvell/octeon_ep/octep_tx.h b/drivers/net/ethernet/marvell/octeon_ep/octep_tx.h index 2ef57980eb47..21e75ff9f5e7 100644 --- a/drivers/net/ethernet/marvell/octeon_ep/octep_tx.h +++ b/drivers/net/ethernet/marvell/octeon_ep/octep_tx.h @@ -17,7 +17,21 @@ #define TX_BUFTYPE_NET_SG 2 #define NUM_TX_BUFTYPES 3 -/* Hardware format for Scatter/Gather list */ +/* Hardware format for Scatter/Gather list + * + * 63 48|47 32|31 16|15 0 + * ----------------------------------------- + * | Len 0 | Len 1 | Len 2 | Len 3 | + * ----------------------------------------- + * | Ptr 0 | + * ----------------------------------------- + * | Ptr 1 | + * ----------------------------------------- + * | Ptr 2 | + * ----------------------------------------- + * | Ptr 3 | + * ----------------------------------------- + */ struct octep_tx_sglist_desc { u16 len[4]; dma_addr_t dma_ptr[4]; diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/cn10k.c b/drivers/net/ethernet/marvell/octeontx2/nic/cn10k.c index 826f691de259..a4a258da8dd5 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/cn10k.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/cn10k.c @@ -107,12 +107,13 @@ int cn10k_sq_aq_init(void *dev, u16 qidx, u16 sqb_aura) } #define NPA_MAX_BURST 16 -void cn10k_refill_pool_ptrs(void *dev, struct otx2_cq_queue *cq) +int cn10k_refill_pool_ptrs(void *dev, struct otx2_cq_queue *cq) { struct otx2_nic *pfvf = dev; + int cnt = cq->pool_ptrs; u64 ptrs[NPA_MAX_BURST]; - int num_ptrs = 1; dma_addr_t bufptr; + int num_ptrs = 1; /* Refill pool with new buffers */ while (cq->pool_ptrs) { @@ -131,6 +132,7 @@ void cn10k_refill_pool_ptrs(void *dev, struct otx2_cq_queue *cq) num_ptrs = 1; } } + return cnt - cq->pool_ptrs; } void cn10k_sqe_flush(void *dev, struct otx2_snd_queue *sq, int size, int qidx) diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/cn10k.h b/drivers/net/ethernet/marvell/octeontx2/nic/cn10k.h index 8ae96815865e..c1861f7de254 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/cn10k.h +++ b/drivers/net/ethernet/marvell/octeontx2/nic/cn10k.h @@ -24,7 +24,7 @@ static inline int mtu_to_dwrr_weight(struct otx2_nic *pfvf, int mtu) return weight; } -void cn10k_refill_pool_ptrs(void *dev, struct otx2_cq_queue *cq); +int cn10k_refill_pool_ptrs(void *dev, struct otx2_cq_queue *cq); void cn10k_sqe_flush(void *dev, struct otx2_snd_queue *sq, int size, int qidx); int cn10k_sq_aq_init(void *dev, u16 qidx, u16 sqb_aura); int cn10k_lmtst_init(struct otx2_nic *pfvf); diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/cn10k_macsec.c b/drivers/net/ethernet/marvell/octeontx2/nic/cn10k_macsec.c index 59b138214af2..6cc7a78968fc 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/cn10k_macsec.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/cn10k_macsec.c @@ -1357,10 +1357,12 @@ static int cn10k_mdo_upd_txsa(struct macsec_context *ctx) if (netif_running(secy->netdev)) { /* Keys cannot be changed after creation */ - err = cn10k_write_tx_sa_pn(pfvf, txsc, sa_num, - sw_tx_sa->next_pn); - if (err) - return err; + if (ctx->sa.update_pn) { + err = cn10k_write_tx_sa_pn(pfvf, txsc, sa_num, + sw_tx_sa->next_pn); + if (err) + return err; + } err = cn10k_mcs_link_tx_sa2sc(pfvf, secy, txsc, sa_num, sw_tx_sa->active); @@ -1529,6 +1531,9 @@ static int cn10k_mdo_upd_rxsa(struct macsec_context *ctx) if (err) return err; + if (!ctx->sa.update_pn) + return 0; + err = cn10k_mcs_write_rx_sa_pn(pfvf, rxsc, sa_num, rx_sa->next_pn); if (err) diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c index 8511906cb4e2..818ce76185b2 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c @@ -574,20 +574,8 @@ int otx2_alloc_rbuf(struct otx2_nic *pfvf, struct otx2_pool *pool, int otx2_alloc_buffer(struct otx2_nic *pfvf, struct otx2_cq_queue *cq, dma_addr_t *dma) { - if (unlikely(__otx2_alloc_rbuf(pfvf, cq->rbpool, dma))) { - struct refill_work *work; - struct delayed_work *dwork; - - work = &pfvf->refill_wrk[cq->cq_idx]; - dwork = &work->pool_refill_work; - /* Schedule a task if no other task is running */ - if (!cq->refill_task_sched) { - cq->refill_task_sched = true; - schedule_delayed_work(dwork, - msecs_to_jiffies(100)); - } + if (unlikely(__otx2_alloc_rbuf(pfvf, cq->rbpool, dma))) return -ENOMEM; - } return 0; } @@ -1082,39 +1070,20 @@ static int otx2_cq_init(struct otx2_nic *pfvf, u16 qidx) static void otx2_pool_refill_task(struct work_struct *work) { struct otx2_cq_queue *cq; - struct otx2_pool *rbpool; struct refill_work *wrk; - int qidx, free_ptrs = 0; struct otx2_nic *pfvf; - dma_addr_t bufptr; + int qidx; wrk = container_of(work, struct refill_work, pool_refill_work.work); pfvf = wrk->pf; qidx = wrk - pfvf->refill_wrk; cq = &pfvf->qset.cq[qidx]; - rbpool = cq->rbpool; - free_ptrs = cq->pool_ptrs; - while (cq->pool_ptrs) { - if (otx2_alloc_rbuf(pfvf, rbpool, &bufptr)) { - /* Schedule a WQ if we fails to free atleast half of the - * pointers else enable napi for this RQ. - */ - if (!((free_ptrs - cq->pool_ptrs) > free_ptrs / 2)) { - struct delayed_work *dwork; - - dwork = &wrk->pool_refill_work; - schedule_delayed_work(dwork, - msecs_to_jiffies(100)); - } else { - cq->refill_task_sched = false; - } - return; - } - pfvf->hw_ops->aura_freeptr(pfvf, qidx, bufptr + OTX2_HEAD_ROOM); - cq->pool_ptrs--; - } cq->refill_task_sched = false; + + local_bh_disable(); + napi_schedule(wrk->napi); + local_bh_enable(); } int otx2_config_nix_queues(struct otx2_nic *pfvf) @@ -1434,6 +1403,7 @@ int otx2_pool_init(struct otx2_nic *pfvf, u16 pool_id, return 0; } + pp_params.order = get_order(buf_size); pp_params.flags = PP_FLAG_PAGE_FRAG | PP_FLAG_DMA_MAP; pp_params.pool_size = min(OTX2_PAGE_POOL_SZ, numptrs); pp_params.nid = NUMA_NO_NODE; diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h index 4c6032ee7800..c04a8ee53a82 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h @@ -302,6 +302,7 @@ struct flr_work { struct refill_work { struct delayed_work pool_refill_work; struct otx2_nic *pf; + struct napi_struct *napi; }; /* PTPv2 originTimestamp structure */ @@ -370,7 +371,7 @@ struct dev_hw_ops { int (*sq_aq_init)(void *dev, u16 qidx, u16 sqb_aura); void (*sqe_flush)(void *dev, struct otx2_snd_queue *sq, int size, int qidx); - void (*refill_pool_ptrs)(void *dev, struct otx2_cq_queue *cq); + int (*refill_pool_ptrs)(void *dev, struct otx2_cq_queue *cq); void (*aura_freeptr)(void *dev, int aura, u64 buf); }; diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c index 70b9065f7d10..6daf4d58c25d 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c @@ -1943,6 +1943,10 @@ int otx2_stop(struct net_device *netdev) netif_tx_disable(netdev); + for (wrk = 0; wrk < pf->qset.cq_cnt; wrk++) + cancel_delayed_work_sync(&pf->refill_wrk[wrk].pool_refill_work); + devm_kfree(pf->dev, pf->refill_wrk); + otx2_free_hw_resources(pf); otx2_free_cints(pf, pf->hw.cint_cnt); otx2_disable_napi(pf); @@ -1950,9 +1954,6 @@ int otx2_stop(struct net_device *netdev) for (qidx = 0; qidx < netdev->num_tx_queues; qidx++) netdev_tx_reset_queue(netdev_get_tx_queue(netdev, qidx)); - for (wrk = 0; wrk < pf->qset.cq_cnt; wrk++) - cancel_delayed_work_sync(&pf->refill_wrk[wrk].pool_refill_work); - devm_kfree(pf->dev, pf->refill_wrk); kfree(qset->sq); kfree(qset->cq); diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c index e369baf11530..53b2a4ef5298 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c @@ -29,7 +29,8 @@ static bool otx2_xdp_rcv_pkt_handler(struct otx2_nic *pfvf, struct bpf_prog *prog, struct nix_cqe_rx_s *cqe, - struct otx2_cq_queue *cq); + struct otx2_cq_queue *cq, + bool *need_xdp_flush); static int otx2_nix_cq_op_status(struct otx2_nic *pfvf, struct otx2_cq_queue *cq) @@ -337,7 +338,7 @@ static bool otx2_check_rcv_errors(struct otx2_nic *pfvf, static void otx2_rcv_pkt_handler(struct otx2_nic *pfvf, struct napi_struct *napi, struct otx2_cq_queue *cq, - struct nix_cqe_rx_s *cqe) + struct nix_cqe_rx_s *cqe, bool *need_xdp_flush) { struct nix_rx_parse_s *parse = &cqe->parse; struct nix_rx_sg_s *sg = &cqe->sg; @@ -353,7 +354,7 @@ static void otx2_rcv_pkt_handler(struct otx2_nic *pfvf, } if (pfvf->xdp_prog) - if (otx2_xdp_rcv_pkt_handler(pfvf, pfvf->xdp_prog, cqe, cq)) + if (otx2_xdp_rcv_pkt_handler(pfvf, pfvf->xdp_prog, cqe, cq, need_xdp_flush)) return; skb = napi_get_frags(napi); @@ -388,6 +389,7 @@ static int otx2_rx_napi_handler(struct otx2_nic *pfvf, struct napi_struct *napi, struct otx2_cq_queue *cq, int budget) { + bool need_xdp_flush = false; struct nix_cqe_rx_s *cqe; int processed_cqe = 0; @@ -409,13 +411,15 @@ process_cqe: cq->cq_head++; cq->cq_head &= (cq->cqe_cnt - 1); - otx2_rcv_pkt_handler(pfvf, napi, cq, cqe); + otx2_rcv_pkt_handler(pfvf, napi, cq, cqe, &need_xdp_flush); cqe->hdr.cqe_type = NIX_XQE_TYPE_INVALID; cqe->sg.seg_addr = 0x00; processed_cqe++; cq->pend_cqe--; } + if (need_xdp_flush) + xdp_do_flush(); /* Free CQEs to HW */ otx2_write64(pfvf, NIX_LF_CQ_OP_DOOR, @@ -424,9 +428,10 @@ process_cqe: return processed_cqe; } -void otx2_refill_pool_ptrs(void *dev, struct otx2_cq_queue *cq) +int otx2_refill_pool_ptrs(void *dev, struct otx2_cq_queue *cq) { struct otx2_nic *pfvf = dev; + int cnt = cq->pool_ptrs; dma_addr_t bufptr; while (cq->pool_ptrs) { @@ -435,6 +440,8 @@ void otx2_refill_pool_ptrs(void *dev, struct otx2_cq_queue *cq) otx2_aura_freeptr(pfvf, cq->cq_idx, bufptr + OTX2_HEAD_ROOM); cq->pool_ptrs--; } + + return cnt - cq->pool_ptrs; } static int otx2_tx_napi_handler(struct otx2_nic *pfvf, @@ -521,6 +528,7 @@ int otx2_napi_handler(struct napi_struct *napi, int budget) struct otx2_cq_queue *cq; struct otx2_qset *qset; struct otx2_nic *pfvf; + int filled_cnt = -1; cq_poll = container_of(napi, struct otx2_cq_poll, napi); pfvf = (struct otx2_nic *)cq_poll->dev; @@ -541,7 +549,7 @@ int otx2_napi_handler(struct napi_struct *napi, int budget) } if (rx_cq && rx_cq->pool_ptrs) - pfvf->hw_ops->refill_pool_ptrs(pfvf, rx_cq); + filled_cnt = pfvf->hw_ops->refill_pool_ptrs(pfvf, rx_cq); /* Clear the IRQ */ otx2_write64(pfvf, NIX_LF_CINTX_INT(cq_poll->cint_idx), BIT_ULL(0)); @@ -561,9 +569,25 @@ int otx2_napi_handler(struct napi_struct *napi, int budget) otx2_config_irq_coalescing(pfvf, i); } - /* Re-enable interrupts */ - otx2_write64(pfvf, NIX_LF_CINTX_ENA_W1S(cq_poll->cint_idx), - BIT_ULL(0)); + if (unlikely(!filled_cnt)) { + struct refill_work *work; + struct delayed_work *dwork; + + work = &pfvf->refill_wrk[cq->cq_idx]; + dwork = &work->pool_refill_work; + /* Schedule a task if no other task is running */ + if (!cq->refill_task_sched) { + work->napi = napi; + cq->refill_task_sched = true; + schedule_delayed_work(dwork, + msecs_to_jiffies(100)); + } + } else { + /* Re-enable interrupts */ + otx2_write64(pfvf, + NIX_LF_CINTX_ENA_W1S(cq_poll->cint_idx), + BIT_ULL(0)); + } } return workdone; } @@ -1334,7 +1358,8 @@ bool otx2_xdp_sq_append_pkt(struct otx2_nic *pfvf, u64 iova, int len, u16 qidx) static bool otx2_xdp_rcv_pkt_handler(struct otx2_nic *pfvf, struct bpf_prog *prog, struct nix_cqe_rx_s *cqe, - struct otx2_cq_queue *cq) + struct otx2_cq_queue *cq, + bool *need_xdp_flush) { unsigned char *hard_start, *data; int qidx = cq->cq_idx; @@ -1371,8 +1396,10 @@ static bool otx2_xdp_rcv_pkt_handler(struct otx2_nic *pfvf, otx2_dma_unmap_page(pfvf, iova, pfvf->rbsize, DMA_FROM_DEVICE); - if (!err) + if (!err) { + *need_xdp_flush = true; return true; + } put_page(page); break; default: diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.h b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.h index 9e3bfbe5c480..a82ffca8ce1b 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.h +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.h @@ -170,6 +170,6 @@ void cn10k_sqe_flush(void *dev, struct otx2_snd_queue *sq, int size, int qidx); void otx2_sqe_flush(void *dev, struct otx2_snd_queue *sq, int size, int qidx); -void otx2_refill_pool_ptrs(void *dev, struct otx2_cq_queue *cq); -void cn10k_refill_pool_ptrs(void *dev, struct otx2_cq_queue *cq); +int otx2_refill_pool_ptrs(void *dev, struct otx2_cq_queue *cq); +int cn10k_refill_pool_ptrs(void *dev, struct otx2_cq_queue *cq); #endif /* OTX2_TXRX_H */ diff --git a/drivers/net/ethernet/marvell/sky2.h b/drivers/net/ethernet/marvell/sky2.h index ddec1627f1a7..8d0bacf4e49c 100644 --- a/drivers/net/ethernet/marvell/sky2.h +++ b/drivers/net/ethernet/marvell/sky2.h @@ -2195,7 +2195,7 @@ struct rx_ring_info { struct sk_buff *skb; dma_addr_t data_addr; DEFINE_DMA_UNMAP_LEN(data_size); - dma_addr_t frag_addr[ETH_JUMBO_MTU >> PAGE_SHIFT]; + dma_addr_t frag_addr[ETH_JUMBO_MTU >> PAGE_SHIFT ?: 1]; }; enum flow_control { diff --git a/drivers/net/ethernet/mediatek/mtk_eth_soc.c b/drivers/net/ethernet/mediatek/mtk_eth_soc.c index 6ad42e3b488f..20afe79f380a 100644 --- a/drivers/net/ethernet/mediatek/mtk_eth_soc.c +++ b/drivers/net/ethernet/mediatek/mtk_eth_soc.c @@ -2005,11 +2005,11 @@ static int mtk_poll_rx(struct napi_struct *napi, int budget, u8 *data, *new_data; struct mtk_rx_dma_v2 *rxd, trxd; int done = 0, bytes = 0; + dma_addr_t dma_addr = DMA_MAPPING_ERROR; while (done < budget) { unsigned int pktlen, *rxdcsum; struct net_device *netdev; - dma_addr_t dma_addr; u32 hash, reason; int mac = 0; @@ -2186,7 +2186,8 @@ release_desc: else rxd->rxd2 = RX_DMA_PREP_PLEN0(ring->buf_size); - if (MTK_HAS_CAPS(eth->soc->caps, MTK_36BIT_DMA)) + if (MTK_HAS_CAPS(eth->soc->caps, MTK_36BIT_DMA) && + likely(dma_addr != DMA_MAPPING_ERROR)) rxd->rxd2 |= RX_DMA_PREP_ADDR64(dma_addr); ring->calc_idx = idx; @@ -2994,6 +2995,9 @@ static int mtk_hwlro_get_fdir_all(struct net_device *dev, int i; for (i = 0; i < MTK_MAX_LRO_IP_CNT; i++) { + if (cnt == cmd->rule_cnt) + return -EMSGSIZE; + if (mac->hwlro_ip[i]) { rule_locs[cnt] = i; cnt++; @@ -3167,8 +3171,8 @@ static irqreturn_t mtk_handle_irq_rx(int irq, void *_eth) eth->rx_events++; if (likely(napi_schedule_prep(ð->rx_napi))) { - __napi_schedule(ð->rx_napi); mtk_rx_irq_disable(eth, eth->soc->txrx.rx_irq_done_mask); + __napi_schedule(ð->rx_napi); } return IRQ_HANDLED; @@ -3180,8 +3184,8 @@ static irqreturn_t mtk_handle_irq_tx(int irq, void *_eth) eth->tx_events++; if (likely(napi_schedule_prep(ð->tx_napi))) { - __napi_schedule(ð->tx_napi); mtk_tx_irq_disable(eth, MTK_TX_DONE_INT); + __napi_schedule(ð->tx_napi); } return IRQ_HANDLED; diff --git a/drivers/net/ethernet/mediatek/mtk_ppe_offload.c b/drivers/net/ethernet/mediatek/mtk_ppe_offload.c index a70a5417c173..a4efbeb16208 100644 --- a/drivers/net/ethernet/mediatek/mtk_ppe_offload.c +++ b/drivers/net/ethernet/mediatek/mtk_ppe_offload.c @@ -214,9 +214,11 @@ mtk_flow_set_output_device(struct mtk_eth *eth, struct mtk_foe_entry *foe, dsa_port = mtk_flow_get_dsa_port(&dev); if (dev == eth->netdev[0]) - pse_port = 1; + pse_port = PSE_GDM1_PORT; else if (dev == eth->netdev[1]) - pse_port = 2; + pse_port = PSE_GDM2_PORT; + else if (dev == eth->netdev[2]) + pse_port = PSE_GDM3_PORT; else return -EOPNOTSUPP; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/cmd.c b/drivers/net/ethernet/mellanox/mlx5/core/cmd.c index afb348579577..c22b0ad0c870 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/cmd.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/cmd.c @@ -2186,52 +2186,23 @@ static u16 cmdif_rev(struct mlx5_core_dev *dev) int mlx5_cmd_init(struct mlx5_core_dev *dev) { - int size = sizeof(struct mlx5_cmd_prot_block); - int align = roundup_pow_of_two(size); struct mlx5_cmd *cmd = &dev->cmd; - u32 cmd_l; - int err; - - cmd->pool = dma_pool_create("mlx5_cmd", mlx5_core_dma_dev(dev), size, align, 0); - if (!cmd->pool) - return -ENOMEM; - err = alloc_cmd_page(dev, cmd); - if (err) - goto err_free_pool; - - cmd_l = (u32)(cmd->dma); - if (cmd_l & 0xfff) { - mlx5_core_err(dev, "invalid command queue address\n"); - err = -ENOMEM; - goto err_cmd_page; - } cmd->checksum_disabled = 1; spin_lock_init(&cmd->alloc_lock); spin_lock_init(&cmd->token_lock); - create_msg_cache(dev); - set_wqname(dev); cmd->wq = create_singlethread_workqueue(cmd->wq_name); if (!cmd->wq) { mlx5_core_err(dev, "failed to create command workqueue\n"); - err = -ENOMEM; - goto err_cache; + return -ENOMEM; } mlx5_cmdif_debugfs_init(dev); return 0; - -err_cache: - destroy_msg_cache(dev); -err_cmd_page: - free_cmd_page(dev, cmd); -err_free_pool: - dma_pool_destroy(cmd->pool); - return err; } void mlx5_cmd_cleanup(struct mlx5_core_dev *dev) @@ -2240,15 +2211,15 @@ void mlx5_cmd_cleanup(struct mlx5_core_dev *dev) mlx5_cmdif_debugfs_cleanup(dev); destroy_workqueue(cmd->wq); - destroy_msg_cache(dev); - free_cmd_page(dev, cmd); - dma_pool_destroy(cmd->pool); } int mlx5_cmd_enable(struct mlx5_core_dev *dev) { + int size = sizeof(struct mlx5_cmd_prot_block); + int align = roundup_pow_of_two(size); struct mlx5_cmd *cmd = &dev->cmd; u32 cmd_h, cmd_l; + int err; memset(&cmd->vars, 0, sizeof(cmd->vars)); cmd->vars.cmdif_rev = cmdif_rev(dev); @@ -2281,10 +2252,21 @@ int mlx5_cmd_enable(struct mlx5_core_dev *dev) sema_init(&cmd->vars.pages_sem, 1); sema_init(&cmd->vars.throttle_sem, DIV_ROUND_UP(cmd->vars.max_reg_cmds, 2)); + cmd->pool = dma_pool_create("mlx5_cmd", mlx5_core_dma_dev(dev), size, align, 0); + if (!cmd->pool) + return -ENOMEM; + + err = alloc_cmd_page(dev, cmd); + if (err) + goto err_free_pool; + cmd_h = (u32)((u64)(cmd->dma) >> 32); cmd_l = (u32)(cmd->dma); - if (WARN_ON(cmd_l & 0xfff)) - return -EINVAL; + if (cmd_l & 0xfff) { + mlx5_core_err(dev, "invalid command queue address\n"); + err = -ENOMEM; + goto err_cmd_page; + } iowrite32be(cmd_h, &dev->iseg->cmdq_addr_h); iowrite32be(cmd_l, &dev->iseg->cmdq_addr_l_sz); @@ -2297,17 +2279,27 @@ int mlx5_cmd_enable(struct mlx5_core_dev *dev) cmd->mode = CMD_MODE_POLLING; cmd->allowed_opcode = CMD_ALLOWED_OPCODE_ALL; + create_msg_cache(dev); create_debugfs_files(dev); return 0; + +err_cmd_page: + free_cmd_page(dev, cmd); +err_free_pool: + dma_pool_destroy(cmd->pool); + return err; } void mlx5_cmd_disable(struct mlx5_core_dev *dev) { struct mlx5_cmd *cmd = &dev->cmd; - clean_debug_files(dev); flush_workqueue(cmd->wq); + clean_debug_files(dev); + destroy_msg_cache(dev); + free_cmd_page(dev, cmd); + dma_pool_destroy(cmd->pool); } void mlx5_cmd_set_state(struct mlx5_core_dev *dev, diff --git a/drivers/net/ethernet/mellanox/mlx5/core/diag/fw_tracer.c b/drivers/net/ethernet/mellanox/mlx5/core/diag/fw_tracer.c index 7c0f2adbea00..ad789349c06e 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/diag/fw_tracer.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/diag/fw_tracer.c @@ -848,7 +848,7 @@ static void mlx5_fw_tracer_ownership_change(struct work_struct *work) mlx5_core_dbg(tracer->dev, "FWTracer: ownership changed, current=(%d)\n", tracer->owner); if (tracer->owner) { - tracer->owner = false; + mlx5_fw_tracer_ownership_acquire(tracer); return; } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/rep/bridge.c b/drivers/net/ethernet/mellanox/mlx5/core/en/rep/bridge.c index 0fef853eab62..5d128c5b4529 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/rep/bridge.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/rep/bridge.c @@ -467,6 +467,17 @@ static int mlx5_esw_bridge_switchdev_event(struct notifier_block *nb, /* only handle the event on peers */ if (mlx5_esw_bridge_is_local(dev, rep, esw)) break; + + fdb_info = container_of(info, + struct switchdev_notifier_fdb_info, + info); + /* Mark for deletion to prevent the update wq task from + * spuriously refreshing the entry which would mark it again as + * offloaded in SW bridge. After this fallthrough to regular + * async delete code. + */ + mlx5_esw_bridge_fdb_mark_deleted(dev, vport_num, esw_owner_vhca_id, br_offloads, + fdb_info); fallthrough; case SWITCHDEV_FDB_ADD_TO_DEVICE: case SWITCHDEV_FDB_DEL_TO_DEVICE: diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/tc_tun_encap.c b/drivers/net/ethernet/mellanox/mlx5/core/en/tc_tun_encap.c index 1730f6a716ee..b10e40e1a9c1 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/tc_tun_encap.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/tc_tun_encap.c @@ -24,7 +24,8 @@ static int mlx5e_set_int_port_tunnel(struct mlx5e_priv *priv, route_dev = dev_get_by_index(dev_net(e->out_dev), e->route_dev_ifindex); - if (!route_dev || !netif_is_ovs_master(route_dev)) + if (!route_dev || !netif_is_ovs_master(route_dev) || + attr->parse_attr->filter_dev == e->out_dev) goto out; err = mlx5e_set_fwd_to_int_port_actions(priv, attr, e->route_dev_ifindex, diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c b/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c index 12f56d0db0af..8bed17d8fe56 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c @@ -874,11 +874,11 @@ int mlx5e_xdp_xmit(struct net_device *dev, int n, struct xdp_frame **frames, } out: - if (flags & XDP_XMIT_FLUSH) { - if (sq->mpwqe.wqe) - mlx5e_xdp_mpwqe_complete(sq); + if (sq->mpwqe.wqe) + mlx5e_xdp_mpwqe_complete(sq); + + if (flags & XDP_XMIT_FLUSH) mlx5e_xmit_xdp_doorbell(sq); - } return nxmit; } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/macsec.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/macsec.c index c9c1db971652..d4ebd8743114 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/macsec.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/macsec.c @@ -580,7 +580,7 @@ static int mlx5e_macsec_upd_txsa(struct macsec_context *ctx) goto out; } - if (tx_sa->next_pn != ctx_tx_sa->next_pn_halves.lower) { + if (ctx->sa.update_pn) { netdev_err(netdev, "MACsec offload: update TX sa %d PN isn't supported\n", assoc_num); err = -EINVAL; @@ -973,7 +973,7 @@ static int mlx5e_macsec_upd_rxsa(struct macsec_context *ctx) goto out; } - if (rx_sa->next_pn != ctx_rx_sa->next_pn_halves.lower) { + if (ctx->sa.update_pn) { netdev_err(ctx->netdev, "MACsec offload update RX sa %d PN isn't supported\n", assoc_num); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c index a2ae791538ed..acb40770cf0c 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c @@ -3952,13 +3952,14 @@ static int set_feature_rx_fcs(struct net_device *netdev, bool enable) struct mlx5e_channels *chs = &priv->channels; struct mlx5e_params new_params; int err; + bool rx_ts_over_crc = !enable; mutex_lock(&priv->state_lock); new_params = chs->params; new_params.scatter_fcs_en = enable; err = mlx5e_safe_switch_params(priv, &new_params, mlx5e_set_rx_port_ts_wrap, - &new_params.scatter_fcs_en, true); + &rx_ts_over_crc, true); mutex_unlock(&priv->state_lock); return err; } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c index 2fdb8895aecd..fd1cce542b68 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c @@ -701,7 +701,7 @@ mlx5e_rep_get_stats(struct net_device *dev, struct rtnl_link_stats64 *stats) /* update HW stats in background for next time */ mlx5e_queue_update_stats(priv); - memcpy(stats, &priv->stats.vf_vport, sizeof(*stats)); + mlx5e_stats_copy_rep_stats(stats, &priv->stats.rep_stats); } static int mlx5e_rep_change_mtu(struct net_device *netdev, int new_mtu) @@ -769,6 +769,7 @@ static int mlx5e_rep_max_nch_limit(struct mlx5_core_dev *mdev) static void mlx5e_build_rep_params(struct net_device *netdev) { + const bool take_rtnl = netdev->reg_state == NETREG_REGISTERED; struct mlx5e_priv *priv = netdev_priv(netdev); struct mlx5e_rep_priv *rpriv = priv->ppriv; struct mlx5_eswitch_rep *rep = rpriv->rep; @@ -794,8 +795,15 @@ static void mlx5e_build_rep_params(struct net_device *netdev) /* RQ */ mlx5e_build_rq_params(mdev, params); + /* If netdev is already registered (e.g. move from nic profile to uplink, + * RTNL lock must be held before triggering netdev notifiers. + */ + if (take_rtnl) + rtnl_lock(); /* update XDP supported features */ mlx5e_set_xdp_feature(netdev); + if (take_rtnl) + rtnl_unlock(); /* CQ moderation params */ params->rx_dim_enabled = MLX5_CAP_GEN(mdev, cq_moderation); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c index 3fd11b0761e0..8d9743a5e42c 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c @@ -457,26 +457,41 @@ static int mlx5e_alloc_rx_wqes(struct mlx5e_rq *rq, u16 ix, int wqe_bulk) static int mlx5e_refill_rx_wqes(struct mlx5e_rq *rq, u16 ix, int wqe_bulk) { int remaining = wqe_bulk; - int i = 0; + int total_alloc = 0; + int refill_alloc; + int refill; /* The WQE bulk is split into smaller bulks that are sized * according to the page pool cache refill size to avoid overflowing * the page pool cache due to too many page releases at once. */ do { - int refill = min_t(u16, rq->wqe.info.refill_unit, remaining); - int alloc_count; + refill = min_t(u16, rq->wqe.info.refill_unit, remaining); - mlx5e_free_rx_wqes(rq, ix + i, refill); - alloc_count = mlx5e_alloc_rx_wqes(rq, ix + i, refill); - i += alloc_count; - if (unlikely(alloc_count != refill)) - break; + mlx5e_free_rx_wqes(rq, ix + total_alloc, refill); + refill_alloc = mlx5e_alloc_rx_wqes(rq, ix + total_alloc, refill); + if (unlikely(refill_alloc != refill)) + goto err_free; + total_alloc += refill_alloc; remaining -= refill; } while (remaining); - return i; + return total_alloc; + +err_free: + mlx5e_free_rx_wqes(rq, ix, total_alloc + refill_alloc); + + for (int i = 0; i < total_alloc + refill; i++) { + int j = mlx5_wq_cyc_ctr2ix(&rq->wqe.wq, ix + i); + struct mlx5e_wqe_frag_info *frag; + + frag = get_frag(rq, j); + for (int k = 0; k < rq->wqe.info.num_frags; k++, frag++) + frag->flags |= BIT(MLX5E_WQE_FRAG_SKIP_RELEASE); + } + + return 0; } static void @@ -816,6 +831,8 @@ err_unmap: mlx5e_page_release_fragmented(rq, frag_page); } + bitmap_fill(wi->skip_release_bitmap, rq->mpwqe.pages_per_wqe); + err: rq->stats->buff_alloc_err++; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.h b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.h index 176fa5976259..477c547dcc04 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.h @@ -484,11 +484,20 @@ struct mlx5e_stats { struct mlx5e_vnic_env_stats vnic; struct mlx5e_vport_stats vport; struct mlx5e_pport_stats pport; - struct rtnl_link_stats64 vf_vport; struct mlx5e_pcie_stats pcie; struct mlx5e_rep_stats rep_stats; }; +static inline void mlx5e_stats_copy_rep_stats(struct rtnl_link_stats64 *vf_vport, + struct mlx5e_rep_stats *rep_stats) +{ + memset(vf_vport, 0, sizeof(*vf_vport)); + vf_vport->rx_packets = rep_stats->vport_rx_packets; + vf_vport->tx_packets = rep_stats->vport_tx_packets; + vf_vport->rx_bytes = rep_stats->vport_rx_bytes; + vf_vport->tx_bytes = rep_stats->vport_tx_bytes; +} + extern mlx5e_stats_grp_t mlx5e_nic_stats_grps[]; unsigned int mlx5e_nic_stats_grps_num(struct mlx5e_priv *priv); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c b/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c index c24828b688ac..c8590483ddc6 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c @@ -4972,7 +4972,8 @@ static int scan_tc_matchall_fdb_actions(struct mlx5e_priv *priv, if (err) return err; - rpriv->prev_vf_vport_stats = priv->stats.vf_vport; + mlx5e_stats_copy_rep_stats(&rpriv->prev_vf_vport_stats, + &priv->stats.rep_stats); break; default: NL_SET_ERR_MSG_MOD(extack, "mlx5 supports only police action for matchall"); @@ -5012,7 +5013,7 @@ void mlx5e_tc_stats_matchall(struct mlx5e_priv *priv, u64 dbytes; u64 dpkts; - cur_stats = priv->stats.vf_vport; + mlx5e_stats_copy_rep_stats(&cur_stats, &priv->stats.rep_stats); dpkts = cur_stats.rx_packets - rpriv->prev_vf_vport_stats.rx_packets; dbytes = cur_stats.rx_bytes - rpriv->prev_vf_vport_stats.rx_bytes; rpriv->prev_vf_vport_stats = cur_stats; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/esw/bridge.c b/drivers/net/ethernet/mellanox/mlx5/core/esw/bridge.c index e36294b7ade2..1b9bc32efd6f 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/esw/bridge.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/esw/bridge.c @@ -1748,6 +1748,28 @@ void mlx5_esw_bridge_fdb_update_used(struct net_device *dev, u16 vport_num, u16 entry->lastuse = jiffies; } +void mlx5_esw_bridge_fdb_mark_deleted(struct net_device *dev, u16 vport_num, u16 esw_owner_vhca_id, + struct mlx5_esw_bridge_offloads *br_offloads, + struct switchdev_notifier_fdb_info *fdb_info) +{ + struct mlx5_esw_bridge_fdb_entry *entry; + struct mlx5_esw_bridge *bridge; + + bridge = mlx5_esw_bridge_from_port_lookup(vport_num, esw_owner_vhca_id, br_offloads); + if (!bridge) + return; + + entry = mlx5_esw_bridge_fdb_lookup(bridge, fdb_info->addr, fdb_info->vid); + if (!entry) { + esw_debug(br_offloads->esw->dev, + "FDB mark deleted entry with specified key not found (MAC=%pM,vid=%u,vport=%u)\n", + fdb_info->addr, fdb_info->vid, vport_num); + return; + } + + entry->flags |= MLX5_ESW_BRIDGE_FLAG_DELETED; +} + void mlx5_esw_bridge_fdb_create(struct net_device *dev, u16 vport_num, u16 esw_owner_vhca_id, struct mlx5_esw_bridge_offloads *br_offloads, struct switchdev_notifier_fdb_info *fdb_info) @@ -1810,7 +1832,8 @@ void mlx5_esw_bridge_update(struct mlx5_esw_bridge_offloads *br_offloads) unsigned long lastuse = (unsigned long)mlx5_fc_query_lastuse(entry->ingress_counter); - if (entry->flags & MLX5_ESW_BRIDGE_FLAG_ADDED_BY_USER) + if (entry->flags & (MLX5_ESW_BRIDGE_FLAG_ADDED_BY_USER | + MLX5_ESW_BRIDGE_FLAG_DELETED)) continue; if (time_after(lastuse, entry->lastuse)) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/esw/bridge.h b/drivers/net/ethernet/mellanox/mlx5/core/esw/bridge.h index c2c7c70d99eb..d6f539161993 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/esw/bridge.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/esw/bridge.h @@ -62,6 +62,9 @@ int mlx5_esw_bridge_vport_peer_unlink(struct net_device *br_netdev, u16 vport_nu void mlx5_esw_bridge_fdb_update_used(struct net_device *dev, u16 vport_num, u16 esw_owner_vhca_id, struct mlx5_esw_bridge_offloads *br_offloads, struct switchdev_notifier_fdb_info *fdb_info); +void mlx5_esw_bridge_fdb_mark_deleted(struct net_device *dev, u16 vport_num, u16 esw_owner_vhca_id, + struct mlx5_esw_bridge_offloads *br_offloads, + struct switchdev_notifier_fdb_info *fdb_info); void mlx5_esw_bridge_fdb_create(struct net_device *dev, u16 vport_num, u16 esw_owner_vhca_id, struct mlx5_esw_bridge_offloads *br_offloads, struct switchdev_notifier_fdb_info *fdb_info); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/esw/bridge_priv.h b/drivers/net/ethernet/mellanox/mlx5/core/esw/bridge_priv.h index 4911cc32161b..7c251af566c6 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/esw/bridge_priv.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/esw/bridge_priv.h @@ -133,6 +133,7 @@ struct mlx5_esw_bridge_mdb_key { enum { MLX5_ESW_BRIDGE_FLAG_ADDED_BY_USER = BIT(0), MLX5_ESW_BRIDGE_FLAG_PEER = BIT(1), + MLX5_ESW_BRIDGE_FLAG_DELETED = BIT(2), }; enum { diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c index d4cde6555063..8d0b915a3121 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c @@ -1038,11 +1038,8 @@ const u32 *mlx5_esw_query_functions(struct mlx5_core_dev *dev) return ERR_PTR(err); } -static void mlx5_eswitch_event_handlers_register(struct mlx5_eswitch *esw) +static void mlx5_eswitch_event_handler_register(struct mlx5_eswitch *esw) { - MLX5_NB_INIT(&esw->nb, eswitch_vport_event, NIC_VPORT_CHANGE); - mlx5_eq_notifier_register(esw->dev, &esw->nb); - if (esw->mode == MLX5_ESWITCH_OFFLOADS && mlx5_eswitch_is_funcs_handler(esw->dev)) { MLX5_NB_INIT(&esw->esw_funcs.nb, mlx5_esw_funcs_changed_handler, ESW_FUNCTIONS_CHANGED); @@ -1050,13 +1047,11 @@ static void mlx5_eswitch_event_handlers_register(struct mlx5_eswitch *esw) } } -static void mlx5_eswitch_event_handlers_unregister(struct mlx5_eswitch *esw) +static void mlx5_eswitch_event_handler_unregister(struct mlx5_eswitch *esw) { if (esw->mode == MLX5_ESWITCH_OFFLOADS && mlx5_eswitch_is_funcs_handler(esw->dev)) mlx5_eq_notifier_unregister(esw->dev, &esw->esw_funcs.nb); - mlx5_eq_notifier_unregister(esw->dev, &esw->nb); - flush_workqueue(esw->work_queue); } @@ -1483,6 +1478,9 @@ int mlx5_eswitch_enable_locked(struct mlx5_eswitch *esw, int num_vfs) mlx5_eswitch_update_num_of_vfs(esw, num_vfs); + MLX5_NB_INIT(&esw->nb, eswitch_vport_event, NIC_VPORT_CHANGE); + mlx5_eq_notifier_register(esw->dev, &esw->nb); + if (esw->mode == MLX5_ESWITCH_LEGACY) { err = esw_legacy_enable(esw); } else { @@ -1495,7 +1493,7 @@ int mlx5_eswitch_enable_locked(struct mlx5_eswitch *esw, int num_vfs) esw->fdb_table.flags |= MLX5_ESW_FDB_CREATED; - mlx5_eswitch_event_handlers_register(esw); + mlx5_eswitch_event_handler_register(esw); esw_info(esw->dev, "Enable: mode(%s), nvfs(%d), necvfs(%d), active vports(%d)\n", esw->mode == MLX5_ESWITCH_LEGACY ? "LEGACY" : "OFFLOADS", @@ -1622,7 +1620,8 @@ void mlx5_eswitch_disable_locked(struct mlx5_eswitch *esw) */ mlx5_esw_mode_change_notify(esw, MLX5_ESWITCH_LEGACY); - mlx5_eswitch_event_handlers_unregister(esw); + mlx5_eq_notifier_unregister(esw->dev, &esw->nb); + mlx5_eswitch_event_handler_unregister(esw); esw_info(esw->dev, "Disable: mode(%s), nvfs(%d), necvfs(%d), active vports(%d)\n", esw->mode == MLX5_ESWITCH_LEGACY ? "LEGACY" : "OFFLOADS", diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_nve_vxlan.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum_nve_vxlan.c index bb8eeb86edf7..52c2fe3644d4 100644 --- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_nve_vxlan.c +++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_nve_vxlan.c @@ -310,8 +310,8 @@ const struct mlxsw_sp_nve_ops mlxsw_sp1_nve_vxlan_ops = { .fdb_clear_offload = mlxsw_sp_nve_vxlan_clear_offload, }; -static bool mlxsw_sp2_nve_vxlan_learning_set(struct mlxsw_sp *mlxsw_sp, - bool learning_en) +static int mlxsw_sp2_nve_vxlan_learning_set(struct mlxsw_sp *mlxsw_sp, + bool learning_en) { char tnpc_pl[MLXSW_REG_TNPC_LEN]; diff --git a/drivers/net/ethernet/microchip/Kconfig b/drivers/net/ethernet/microchip/Kconfig index 329e374b9539..43ba71e82260 100644 --- a/drivers/net/ethernet/microchip/Kconfig +++ b/drivers/net/ethernet/microchip/Kconfig @@ -46,6 +46,7 @@ config LAN743X tristate "LAN743x support" depends on PCI depends on PTP_1588_CLOCK_OPTIONAL + select PHYLIB select FIXED_PHY select CRC16 select CRC32 diff --git a/drivers/net/ethernet/microchip/vcap/vcap_api.c b/drivers/net/ethernet/microchip/vcap/vcap_api.c index 300fe1a93dce..ef980e4e5bc2 100644 --- a/drivers/net/ethernet/microchip/vcap/vcap_api.c +++ b/drivers/net/ethernet/microchip/vcap/vcap_api.c @@ -1021,18 +1021,32 @@ static struct vcap_rule_internal *vcap_dup_rule(struct vcap_rule_internal *ri, list_for_each_entry(ckf, &ri->data.keyfields, ctrl.list) { newckf = kmemdup(ckf, sizeof(*newckf), GFP_KERNEL); if (!newckf) - return ERR_PTR(-ENOMEM); + goto err; list_add_tail(&newckf->ctrl.list, &duprule->data.keyfields); } list_for_each_entry(caf, &ri->data.actionfields, ctrl.list) { newcaf = kmemdup(caf, sizeof(*newcaf), GFP_KERNEL); if (!newcaf) - return ERR_PTR(-ENOMEM); + goto err; list_add_tail(&newcaf->ctrl.list, &duprule->data.actionfields); } return duprule; + +err: + list_for_each_entry_safe(ckf, newckf, &duprule->data.keyfields, ctrl.list) { + list_del(&ckf->ctrl.list); + kfree(ckf); + } + + list_for_each_entry_safe(caf, newcaf, &duprule->data.actionfields, ctrl.list) { + list_del(&caf->ctrl.list); + kfree(caf); + } + + kfree(duprule); + return ERR_PTR(-ENOMEM); } static void vcap_apply_width(u8 *dst, int width, int bytes) diff --git a/drivers/net/ethernet/microchip/vcap/vcap_api_kunit.c b/drivers/net/ethernet/microchip/vcap/vcap_api_kunit.c index c07f25e791c7..fe4e166de8a0 100644 --- a/drivers/net/ethernet/microchip/vcap/vcap_api_kunit.c +++ b/drivers/net/ethernet/microchip/vcap/vcap_api_kunit.c @@ -243,10 +243,9 @@ static void vcap_test_api_init(struct vcap_admin *admin) } /* Helper function to create a rule of a specific size */ -static struct vcap_rule * -test_vcap_xn_rule_creator(struct kunit *test, int cid, enum vcap_user user, - u16 priority, - int id, int size, int expected_addr) +static void test_vcap_xn_rule_creator(struct kunit *test, int cid, + enum vcap_user user, u16 priority, + int id, int size, int expected_addr) { struct vcap_rule *rule; struct vcap_rule_internal *ri; @@ -311,7 +310,7 @@ test_vcap_xn_rule_creator(struct kunit *test, int cid, enum vcap_user user, ret = vcap_add_rule(rule); KUNIT_EXPECT_EQ(test, 0, ret); KUNIT_EXPECT_EQ(test, expected_addr, ri->addr); - return rule; + vcap_free_rule(rule); } /* Prepare testing rule deletion */ @@ -995,6 +994,16 @@ static void vcap_api_encode_rule_actionset_test(struct kunit *test) KUNIT_EXPECT_EQ(test, (u32)0x00000000, actwords[11]); } +static void vcap_free_ckf(struct vcap_rule *rule) +{ + struct vcap_client_keyfield *ckf, *next_ckf; + + list_for_each_entry_safe(ckf, next_ckf, &rule->keyfields, ctrl.list) { + list_del(&ckf->ctrl.list); + kfree(ckf); + } +} + static void vcap_api_rule_add_keyvalue_test(struct kunit *test) { struct vcap_admin admin = { @@ -1027,6 +1036,7 @@ static void vcap_api_rule_add_keyvalue_test(struct kunit *test) KUNIT_EXPECT_EQ(test, VCAP_FIELD_BIT, kf->ctrl.type); KUNIT_EXPECT_EQ(test, 0x0, kf->data.u1.value); KUNIT_EXPECT_EQ(test, 0x1, kf->data.u1.mask); + vcap_free_ckf(rule); INIT_LIST_HEAD(&rule->keyfields); ret = vcap_rule_add_key_bit(rule, VCAP_KF_LOOKUP_FIRST_IS, VCAP_BIT_1); @@ -1039,6 +1049,7 @@ static void vcap_api_rule_add_keyvalue_test(struct kunit *test) KUNIT_EXPECT_EQ(test, VCAP_FIELD_BIT, kf->ctrl.type); KUNIT_EXPECT_EQ(test, 0x1, kf->data.u1.value); KUNIT_EXPECT_EQ(test, 0x1, kf->data.u1.mask); + vcap_free_ckf(rule); INIT_LIST_HEAD(&rule->keyfields); ret = vcap_rule_add_key_bit(rule, VCAP_KF_LOOKUP_FIRST_IS, @@ -1052,6 +1063,7 @@ static void vcap_api_rule_add_keyvalue_test(struct kunit *test) KUNIT_EXPECT_EQ(test, VCAP_FIELD_BIT, kf->ctrl.type); KUNIT_EXPECT_EQ(test, 0x0, kf->data.u1.value); KUNIT_EXPECT_EQ(test, 0x0, kf->data.u1.mask); + vcap_free_ckf(rule); INIT_LIST_HEAD(&rule->keyfields); ret = vcap_rule_add_key_u32(rule, VCAP_KF_TYPE, 0x98765432, 0xff00ffab); @@ -1064,6 +1076,7 @@ static void vcap_api_rule_add_keyvalue_test(struct kunit *test) KUNIT_EXPECT_EQ(test, VCAP_FIELD_U32, kf->ctrl.type); KUNIT_EXPECT_EQ(test, 0x98765432, kf->data.u32.value); KUNIT_EXPECT_EQ(test, 0xff00ffab, kf->data.u32.mask); + vcap_free_ckf(rule); INIT_LIST_HEAD(&rule->keyfields); ret = vcap_rule_add_key_u128(rule, VCAP_KF_L3_IP6_SIP, &dip); @@ -1078,6 +1091,18 @@ static void vcap_api_rule_add_keyvalue_test(struct kunit *test) KUNIT_EXPECT_EQ(test, dip.value[idx], kf->data.u128.value[idx]); for (idx = 0; idx < ARRAY_SIZE(dip.mask); ++idx) KUNIT_EXPECT_EQ(test, dip.mask[idx], kf->data.u128.mask[idx]); + vcap_free_ckf(rule); +} + +static void vcap_free_caf(struct vcap_rule *rule) +{ + struct vcap_client_actionfield *caf, *next_caf; + + list_for_each_entry_safe(caf, next_caf, + &rule->actionfields, ctrl.list) { + list_del(&caf->ctrl.list); + kfree(caf); + } } static void vcap_api_rule_add_actionvalue_test(struct kunit *test) @@ -1105,6 +1130,7 @@ static void vcap_api_rule_add_actionvalue_test(struct kunit *test) KUNIT_EXPECT_EQ(test, VCAP_AF_POLICE_ENA, af->ctrl.action); KUNIT_EXPECT_EQ(test, VCAP_FIELD_BIT, af->ctrl.type); KUNIT_EXPECT_EQ(test, 0x0, af->data.u1.value); + vcap_free_caf(rule); INIT_LIST_HEAD(&rule->actionfields); ret = vcap_rule_add_action_bit(rule, VCAP_AF_POLICE_ENA, VCAP_BIT_1); @@ -1116,6 +1142,7 @@ static void vcap_api_rule_add_actionvalue_test(struct kunit *test) KUNIT_EXPECT_EQ(test, VCAP_AF_POLICE_ENA, af->ctrl.action); KUNIT_EXPECT_EQ(test, VCAP_FIELD_BIT, af->ctrl.type); KUNIT_EXPECT_EQ(test, 0x1, af->data.u1.value); + vcap_free_caf(rule); INIT_LIST_HEAD(&rule->actionfields); ret = vcap_rule_add_action_bit(rule, VCAP_AF_POLICE_ENA, VCAP_BIT_ANY); @@ -1127,6 +1154,7 @@ static void vcap_api_rule_add_actionvalue_test(struct kunit *test) KUNIT_EXPECT_EQ(test, VCAP_AF_POLICE_ENA, af->ctrl.action); KUNIT_EXPECT_EQ(test, VCAP_FIELD_BIT, af->ctrl.type); KUNIT_EXPECT_EQ(test, 0x0, af->data.u1.value); + vcap_free_caf(rule); INIT_LIST_HEAD(&rule->actionfields); ret = vcap_rule_add_action_u32(rule, VCAP_AF_TYPE, 0x98765432); @@ -1138,6 +1166,7 @@ static void vcap_api_rule_add_actionvalue_test(struct kunit *test) KUNIT_EXPECT_EQ(test, VCAP_AF_TYPE, af->ctrl.action); KUNIT_EXPECT_EQ(test, VCAP_FIELD_U32, af->ctrl.type); KUNIT_EXPECT_EQ(test, 0x98765432, af->data.u32.value); + vcap_free_caf(rule); INIT_LIST_HEAD(&rule->actionfields); ret = vcap_rule_add_action_u32(rule, VCAP_AF_MASK_MODE, 0xaabbccdd); @@ -1149,6 +1178,7 @@ static void vcap_api_rule_add_actionvalue_test(struct kunit *test) KUNIT_EXPECT_EQ(test, VCAP_AF_MASK_MODE, af->ctrl.action); KUNIT_EXPECT_EQ(test, VCAP_FIELD_U32, af->ctrl.type); KUNIT_EXPECT_EQ(test, 0xaabbccdd, af->data.u32.value); + vcap_free_caf(rule); } static void vcap_api_rule_find_keyset_basic_test(struct kunit *test) @@ -1408,6 +1438,10 @@ static void vcap_api_encode_rule_test(struct kunit *test) ret = list_empty(&is2_admin.rules); KUNIT_EXPECT_EQ(test, false, ret); KUNIT_EXPECT_EQ(test, 0, ret); + + vcap_enable_lookups(&test_vctrl, &test_netdev, 0, 0, + rule->cookie, false); + vcap_free_rule(rule); /* Check that the rule has been freed: tricky to access since this @@ -1418,6 +1452,8 @@ static void vcap_api_encode_rule_test(struct kunit *test) KUNIT_EXPECT_EQ(test, true, ret); ret = list_empty(&rule->actionfields); KUNIT_EXPECT_EQ(test, true, ret); + + vcap_del_rule(&test_vctrl, &test_netdev, id); } static void vcap_api_set_rule_counter_test(struct kunit *test) @@ -1561,6 +1597,11 @@ static void vcap_api_rule_insert_in_order_test(struct kunit *test) test_vcap_xn_rule_creator(test, 10000, VCAP_USER_QOS, 20, 400, 6, 774); test_vcap_xn_rule_creator(test, 10000, VCAP_USER_QOS, 30, 300, 3, 771); test_vcap_xn_rule_creator(test, 10000, VCAP_USER_QOS, 40, 200, 2, 768); + + vcap_del_rule(&test_vctrl, &test_netdev, 200); + vcap_del_rule(&test_vctrl, &test_netdev, 300); + vcap_del_rule(&test_vctrl, &test_netdev, 400); + vcap_del_rule(&test_vctrl, &test_netdev, 500); } static void vcap_api_rule_insert_reverse_order_test(struct kunit *test) @@ -1619,6 +1660,11 @@ static void vcap_api_rule_insert_reverse_order_test(struct kunit *test) ++idx; } KUNIT_EXPECT_EQ(test, 768, admin.last_used_addr); + + vcap_del_rule(&test_vctrl, &test_netdev, 500); + vcap_del_rule(&test_vctrl, &test_netdev, 400); + vcap_del_rule(&test_vctrl, &test_netdev, 300); + vcap_del_rule(&test_vctrl, &test_netdev, 200); } static void vcap_api_rule_remove_at_end_test(struct kunit *test) @@ -1819,6 +1865,9 @@ static void vcap_api_rule_remove_in_front_test(struct kunit *test) KUNIT_EXPECT_EQ(test, 786, test_init_start); KUNIT_EXPECT_EQ(test, 8, test_init_count); KUNIT_EXPECT_EQ(test, 794, admin.last_used_addr); + + vcap_del_rule(&test_vctrl, &test_netdev, 200); + vcap_del_rule(&test_vctrl, &test_netdev, 300); } static struct kunit_case vcap_api_rule_remove_test_cases[] = { diff --git a/drivers/net/ethernet/microsoft/mana/mana_en.c b/drivers/net/ethernet/microsoft/mana/mana_en.c index 4a16ebff3d1d..48ea4aeeea5d 100644 --- a/drivers/net/ethernet/microsoft/mana/mana_en.c +++ b/drivers/net/ethernet/microsoft/mana/mana_en.c @@ -91,63 +91,137 @@ static unsigned int mana_checksum_info(struct sk_buff *skb) return 0; } +static void mana_add_sge(struct mana_tx_package *tp, struct mana_skb_head *ash, + int sg_i, dma_addr_t da, int sge_len, u32 gpa_mkey) +{ + ash->dma_handle[sg_i] = da; + ash->size[sg_i] = sge_len; + + tp->wqe_req.sgl[sg_i].address = da; + tp->wqe_req.sgl[sg_i].mem_key = gpa_mkey; + tp->wqe_req.sgl[sg_i].size = sge_len; +} + static int mana_map_skb(struct sk_buff *skb, struct mana_port_context *apc, - struct mana_tx_package *tp) + struct mana_tx_package *tp, int gso_hs) { struct mana_skb_head *ash = (struct mana_skb_head *)skb->head; + int hsg = 1; /* num of SGEs of linear part */ struct gdma_dev *gd = apc->ac->gdma_dev; + int skb_hlen = skb_headlen(skb); + int sge0_len, sge1_len = 0; struct gdma_context *gc; struct device *dev; skb_frag_t *frag; dma_addr_t da; + int sg_i; int i; gc = gd->gdma_context; dev = gc->dev; - da = dma_map_single(dev, skb->data, skb_headlen(skb), DMA_TO_DEVICE); + if (gso_hs && gso_hs < skb_hlen) { + sge0_len = gso_hs; + sge1_len = skb_hlen - gso_hs; + } else { + sge0_len = skb_hlen; + } + + da = dma_map_single(dev, skb->data, sge0_len, DMA_TO_DEVICE); if (dma_mapping_error(dev, da)) return -ENOMEM; - ash->dma_handle[0] = da; - ash->size[0] = skb_headlen(skb); + mana_add_sge(tp, ash, 0, da, sge0_len, gd->gpa_mkey); - tp->wqe_req.sgl[0].address = ash->dma_handle[0]; - tp->wqe_req.sgl[0].mem_key = gd->gpa_mkey; - tp->wqe_req.sgl[0].size = ash->size[0]; + if (sge1_len) { + sg_i = 1; + da = dma_map_single(dev, skb->data + sge0_len, sge1_len, + DMA_TO_DEVICE); + if (dma_mapping_error(dev, da)) + goto frag_err; + + mana_add_sge(tp, ash, sg_i, da, sge1_len, gd->gpa_mkey); + hsg = 2; + } for (i = 0; i < skb_shinfo(skb)->nr_frags; i++) { + sg_i = hsg + i; + frag = &skb_shinfo(skb)->frags[i]; da = skb_frag_dma_map(dev, frag, 0, skb_frag_size(frag), DMA_TO_DEVICE); - if (dma_mapping_error(dev, da)) goto frag_err; - ash->dma_handle[i + 1] = da; - ash->size[i + 1] = skb_frag_size(frag); - - tp->wqe_req.sgl[i + 1].address = ash->dma_handle[i + 1]; - tp->wqe_req.sgl[i + 1].mem_key = gd->gpa_mkey; - tp->wqe_req.sgl[i + 1].size = ash->size[i + 1]; + mana_add_sge(tp, ash, sg_i, da, skb_frag_size(frag), + gd->gpa_mkey); } return 0; frag_err: - for (i = i - 1; i >= 0; i--) - dma_unmap_page(dev, ash->dma_handle[i + 1], ash->size[i + 1], + for (i = sg_i - 1; i >= hsg; i--) + dma_unmap_page(dev, ash->dma_handle[i], ash->size[i], DMA_TO_DEVICE); - dma_unmap_single(dev, ash->dma_handle[0], ash->size[0], DMA_TO_DEVICE); + for (i = hsg - 1; i >= 0; i--) + dma_unmap_single(dev, ash->dma_handle[i], ash->size[i], + DMA_TO_DEVICE); return -ENOMEM; } +/* Handle the case when GSO SKB linear length is too large. + * MANA NIC requires GSO packets to put only the packet header to SGE0. + * So, we need 2 SGEs for the skb linear part which contains more than the + * header. + * Return a positive value for the number of SGEs, or a negative value + * for an error. + */ +static int mana_fix_skb_head(struct net_device *ndev, struct sk_buff *skb, + int gso_hs) +{ + int num_sge = 1 + skb_shinfo(skb)->nr_frags; + int skb_hlen = skb_headlen(skb); + + if (gso_hs < skb_hlen) { + num_sge++; + } else if (gso_hs > skb_hlen) { + if (net_ratelimit()) + netdev_err(ndev, + "TX nonlinear head: hs:%d, skb_hlen:%d\n", + gso_hs, skb_hlen); + + return -EINVAL; + } + + return num_sge; +} + +/* Get the GSO packet's header size */ +static int mana_get_gso_hs(struct sk_buff *skb) +{ + int gso_hs; + + if (skb->encapsulation) { + gso_hs = skb_inner_tcp_all_headers(skb); + } else { + if (skb_shinfo(skb)->gso_type & SKB_GSO_UDP_L4) { + gso_hs = skb_transport_offset(skb) + + sizeof(struct udphdr); + } else { + gso_hs = skb_tcp_all_headers(skb); + } + } + + return gso_hs; +} + netdev_tx_t mana_start_xmit(struct sk_buff *skb, struct net_device *ndev) { enum mana_tx_pkt_format pkt_fmt = MANA_SHORT_PKT_FMT; struct mana_port_context *apc = netdev_priv(ndev); + int gso_hs = 0; /* zero for non-GSO pkts */ u16 txq_idx = skb_get_queue_mapping(skb); struct gdma_dev *gd = apc->ac->gdma_dev; bool ipv4 = false, ipv6 = false; @@ -159,7 +233,6 @@ netdev_tx_t mana_start_xmit(struct sk_buff *skb, struct net_device *ndev) struct mana_txq *txq; struct mana_cq *cq; int err, len; - u16 ihs; if (unlikely(!apc->port_is_up)) goto tx_drop; @@ -209,19 +282,6 @@ netdev_tx_t mana_start_xmit(struct sk_buff *skb, struct net_device *ndev) pkg.wqe_req.client_data_unit = 0; pkg.wqe_req.num_sge = 1 + skb_shinfo(skb)->nr_frags; - WARN_ON_ONCE(pkg.wqe_req.num_sge > MAX_TX_WQE_SGL_ENTRIES); - - if (pkg.wqe_req.num_sge <= ARRAY_SIZE(pkg.sgl_array)) { - pkg.wqe_req.sgl = pkg.sgl_array; - } else { - pkg.sgl_ptr = kmalloc_array(pkg.wqe_req.num_sge, - sizeof(struct gdma_sge), - GFP_ATOMIC); - if (!pkg.sgl_ptr) - goto tx_drop_count; - - pkg.wqe_req.sgl = pkg.sgl_ptr; - } if (skb->protocol == htons(ETH_P_IP)) ipv4 = true; @@ -229,6 +289,26 @@ netdev_tx_t mana_start_xmit(struct sk_buff *skb, struct net_device *ndev) ipv6 = true; if (skb_is_gso(skb)) { + int num_sge; + + gso_hs = mana_get_gso_hs(skb); + + num_sge = mana_fix_skb_head(ndev, skb, gso_hs); + if (num_sge > 0) + pkg.wqe_req.num_sge = num_sge; + else + goto tx_drop_count; + + u64_stats_update_begin(&tx_stats->syncp); + if (skb->encapsulation) { + tx_stats->tso_inner_packets++; + tx_stats->tso_inner_bytes += skb->len - gso_hs; + } else { + tx_stats->tso_packets++; + tx_stats->tso_bytes += skb->len - gso_hs; + } + u64_stats_update_end(&tx_stats->syncp); + pkg.tx_oob.s_oob.is_outer_ipv4 = ipv4; pkg.tx_oob.s_oob.is_outer_ipv6 = ipv6; @@ -252,28 +332,6 @@ netdev_tx_t mana_start_xmit(struct sk_buff *skb, struct net_device *ndev) &ipv6_hdr(skb)->daddr, 0, IPPROTO_TCP, 0); } - - if (skb->encapsulation) { - ihs = skb_inner_tcp_all_headers(skb); - u64_stats_update_begin(&tx_stats->syncp); - tx_stats->tso_inner_packets++; - tx_stats->tso_inner_bytes += skb->len - ihs; - u64_stats_update_end(&tx_stats->syncp); - } else { - if (skb_shinfo(skb)->gso_type & SKB_GSO_UDP_L4) { - ihs = skb_transport_offset(skb) + sizeof(struct udphdr); - } else { - ihs = skb_tcp_all_headers(skb); - if (ipv6_has_hopopt_jumbo(skb)) - ihs -= sizeof(struct hop_jumbo_hdr); - } - - u64_stats_update_begin(&tx_stats->syncp); - tx_stats->tso_packets++; - tx_stats->tso_bytes += skb->len - ihs; - u64_stats_update_end(&tx_stats->syncp); - } - } else if (skb->ip_summed == CHECKSUM_PARTIAL) { csum_type = mana_checksum_info(skb); @@ -296,11 +354,25 @@ netdev_tx_t mana_start_xmit(struct sk_buff *skb, struct net_device *ndev) } else { /* Can't do offload of this type of checksum */ if (skb_checksum_help(skb)) - goto free_sgl_ptr; + goto tx_drop_count; } } - if (mana_map_skb(skb, apc, &pkg)) { + WARN_ON_ONCE(pkg.wqe_req.num_sge > MAX_TX_WQE_SGL_ENTRIES); + + if (pkg.wqe_req.num_sge <= ARRAY_SIZE(pkg.sgl_array)) { + pkg.wqe_req.sgl = pkg.sgl_array; + } else { + pkg.sgl_ptr = kmalloc_array(pkg.wqe_req.num_sge, + sizeof(struct gdma_sge), + GFP_ATOMIC); + if (!pkg.sgl_ptr) + goto tx_drop_count; + + pkg.wqe_req.sgl = pkg.sgl_ptr; + } + + if (mana_map_skb(skb, apc, &pkg, gso_hs)) { u64_stats_update_begin(&tx_stats->syncp); tx_stats->mana_map_err++; u64_stats_update_end(&tx_stats->syncp); @@ -1258,11 +1330,16 @@ static void mana_unmap_skb(struct sk_buff *skb, struct mana_port_context *apc) struct mana_skb_head *ash = (struct mana_skb_head *)skb->head; struct gdma_context *gc = apc->ac->gdma_dev->gdma_context; struct device *dev = gc->dev; - int i; + int hsg, i; - dma_unmap_single(dev, ash->dma_handle[0], ash->size[0], DMA_TO_DEVICE); + /* Number of SGEs of linear part */ + hsg = (skb_is_gso(skb) && skb_headlen(skb) > ash->size[0]) ? 2 : 1; - for (i = 1; i < skb_shinfo(skb)->nr_frags + 1; i++) + for (i = 0; i < hsg; i++) + dma_unmap_single(dev, ash->dma_handle[i], ash->size[i], + DMA_TO_DEVICE); + + for (i = hsg; i < skb_shinfo(skb)->nr_frags + hsg; i++) dma_unmap_page(dev, ash->dma_handle[i], ash->size[i], DMA_TO_DEVICE); } @@ -1317,19 +1394,23 @@ static void mana_poll_tx_cq(struct mana_cq *cq) case CQE_TX_VPORT_IDX_OUT_OF_RANGE: case CQE_TX_VPORT_DISABLED: case CQE_TX_VLAN_TAGGING_VIOLATION: - WARN_ONCE(1, "TX: CQE error %d: ignored.\n", - cqe_oob->cqe_hdr.cqe_type); + if (net_ratelimit()) + netdev_err(ndev, "TX: CQE error %d\n", + cqe_oob->cqe_hdr.cqe_type); + apc->eth_stats.tx_cqe_err++; break; default: - /* If the CQE type is unexpected, log an error, assert, - * and go through the error path. + /* If the CQE type is unknown, log an error, + * and still free the SKB, update tail, etc. */ - WARN_ONCE(1, "TX: Unexpected CQE type %d: HW BUG?\n", - cqe_oob->cqe_hdr.cqe_type); + if (net_ratelimit()) + netdev_err(ndev, "TX: unknown CQE type %d\n", + cqe_oob->cqe_hdr.cqe_type); + apc->eth_stats.tx_cqe_unknown_type++; - return; + break; } if (WARN_ON_ONCE(txq->gdma_txq_id != completions[i].wq_num)) diff --git a/drivers/net/ethernet/netronome/nfp/flower/cmsg.c b/drivers/net/ethernet/netronome/nfp/flower/cmsg.c index f21cf1f40f98..153533cd8f08 100644 --- a/drivers/net/ethernet/netronome/nfp/flower/cmsg.c +++ b/drivers/net/ethernet/netronome/nfp/flower/cmsg.c @@ -210,6 +210,7 @@ nfp_flower_cmsg_merge_hint_rx(struct nfp_app *app, struct sk_buff *skb) unsigned int msg_len = nfp_flower_cmsg_get_data_len(skb); struct nfp_flower_cmsg_merge_hint *msg; struct nfp_fl_payload *sub_flows[2]; + struct nfp_flower_priv *priv; int err, i, flow_cnt; msg = nfp_flower_cmsg_get_data(skb); @@ -228,14 +229,15 @@ nfp_flower_cmsg_merge_hint_rx(struct nfp_app *app, struct sk_buff *skb) return; } - rtnl_lock(); + priv = app->priv; + mutex_lock(&priv->nfp_fl_lock); for (i = 0; i < flow_cnt; i++) { u32 ctx = be32_to_cpu(msg->flow[i].host_ctx); sub_flows[i] = nfp_flower_get_fl_payload_from_ctx(app, ctx); if (!sub_flows[i]) { nfp_flower_cmsg_warn(app, "Invalid flow in merge hint\n"); - goto err_rtnl_unlock; + goto err_mutex_unlock; } } @@ -244,8 +246,8 @@ nfp_flower_cmsg_merge_hint_rx(struct nfp_app *app, struct sk_buff *skb) if (err == -ENOMEM) nfp_flower_cmsg_warn(app, "Flow merge memory fail.\n"); -err_rtnl_unlock: - rtnl_unlock(); +err_mutex_unlock: + mutex_unlock(&priv->nfp_fl_lock); } static void diff --git a/drivers/net/ethernet/netronome/nfp/flower/conntrack.c b/drivers/net/ethernet/netronome/nfp/flower/conntrack.c index 2643c4b3ff1f..2967bab72505 100644 --- a/drivers/net/ethernet/netronome/nfp/flower/conntrack.c +++ b/drivers/net/ethernet/netronome/nfp/flower/conntrack.c @@ -2131,8 +2131,6 @@ nfp_fl_ct_offload_nft_flow(struct nfp_fl_ct_zone_entry *zt, struct flow_cls_offl struct nfp_fl_ct_flow_entry *ct_entry; struct netlink_ext_ack *extack = NULL; - ASSERT_RTNL(); - extack = flow->common.extack; switch (flow->command) { case FLOW_CLS_REPLACE: @@ -2178,9 +2176,13 @@ int nfp_fl_ct_handle_nft_flow(enum tc_setup_type type, void *type_data, void *cb switch (type) { case TC_SETUP_CLSFLOWER: - rtnl_lock(); + while (!mutex_trylock(&zt->priv->nfp_fl_lock)) { + if (!zt->nft) /* avoid deadlock */ + return err; + msleep(20); + } err = nfp_fl_ct_offload_nft_flow(zt, flow); - rtnl_unlock(); + mutex_unlock(&zt->priv->nfp_fl_lock); break; default: return -EOPNOTSUPP; @@ -2208,6 +2210,7 @@ int nfp_fl_ct_del_flow(struct nfp_fl_ct_map_entry *ct_map_ent) struct nfp_fl_ct_flow_entry *ct_entry; struct nfp_fl_ct_zone_entry *zt; struct rhashtable *m_table; + struct nf_flowtable *nft; if (!ct_map_ent) return -ENOENT; @@ -2226,8 +2229,12 @@ int nfp_fl_ct_del_flow(struct nfp_fl_ct_map_entry *ct_map_ent) if (ct_map_ent->cookie > 0) kfree(ct_map_ent); - if (!zt->pre_ct_count) { - zt->nft = NULL; + if (!zt->pre_ct_count && zt->nft) { + nft = zt->nft; + zt->nft = NULL; /* avoid deadlock */ + nf_flow_table_offload_del_cb(nft, + nfp_fl_ct_handle_nft_flow, + zt); nfp_fl_ct_clean_nft_entries(zt); } break; diff --git a/drivers/net/ethernet/netronome/nfp/flower/main.h b/drivers/net/ethernet/netronome/nfp/flower/main.h index 40372545148e..2b7c947ff4f2 100644 --- a/drivers/net/ethernet/netronome/nfp/flower/main.h +++ b/drivers/net/ethernet/netronome/nfp/flower/main.h @@ -297,6 +297,7 @@ struct nfp_fl_internal_ports { * @predt_list: List to keep track of decap pretun flows * @neigh_table: Table to keep track of neighbor entries * @predt_lock: Lock to serialise predt/neigh table updates + * @nfp_fl_lock: Lock to protect the flow offload operation */ struct nfp_flower_priv { struct nfp_app *app; @@ -339,6 +340,7 @@ struct nfp_flower_priv { struct list_head predt_list; struct rhashtable neigh_table; spinlock_t predt_lock; /* Lock to serialise predt/neigh table updates */ + struct mutex nfp_fl_lock; /* Protect the flow operation */ }; /** diff --git a/drivers/net/ethernet/netronome/nfp/flower/metadata.c b/drivers/net/ethernet/netronome/nfp/flower/metadata.c index 0f06ef6e24bf..80e4675582bf 100644 --- a/drivers/net/ethernet/netronome/nfp/flower/metadata.c +++ b/drivers/net/ethernet/netronome/nfp/flower/metadata.c @@ -528,6 +528,8 @@ int nfp_flower_metadata_init(struct nfp_app *app, u64 host_ctx_count, if (err) goto err_free_stats_ctx_table; + mutex_init(&priv->nfp_fl_lock); + err = rhashtable_init(&priv->ct_zone_table, &nfp_zone_table_params); if (err) goto err_free_merge_table; diff --git a/drivers/net/ethernet/netronome/nfp/flower/offload.c b/drivers/net/ethernet/netronome/nfp/flower/offload.c index c153f0575b92..0aceef9fe582 100644 --- a/drivers/net/ethernet/netronome/nfp/flower/offload.c +++ b/drivers/net/ethernet/netronome/nfp/flower/offload.c @@ -1009,8 +1009,6 @@ int nfp_flower_merge_offloaded_flows(struct nfp_app *app, u64 parent_ctx = 0; int err; - ASSERT_RTNL(); - if (sub_flow1 == sub_flow2 || nfp_flower_is_merge_flow(sub_flow1) || nfp_flower_is_merge_flow(sub_flow2)) @@ -1727,19 +1725,30 @@ static int nfp_flower_repr_offload(struct nfp_app *app, struct net_device *netdev, struct flow_cls_offload *flower) { + struct nfp_flower_priv *priv = app->priv; + int ret; + if (!eth_proto_is_802_3(flower->common.protocol)) return -EOPNOTSUPP; + mutex_lock(&priv->nfp_fl_lock); switch (flower->command) { case FLOW_CLS_REPLACE: - return nfp_flower_add_offload(app, netdev, flower); + ret = nfp_flower_add_offload(app, netdev, flower); + break; case FLOW_CLS_DESTROY: - return nfp_flower_del_offload(app, netdev, flower); + ret = nfp_flower_del_offload(app, netdev, flower); + break; case FLOW_CLS_STATS: - return nfp_flower_get_stats(app, netdev, flower); + ret = nfp_flower_get_stats(app, netdev, flower); + break; default: - return -EOPNOTSUPP; + ret = -EOPNOTSUPP; + break; } + mutex_unlock(&priv->nfp_fl_lock); + + return ret; } static int nfp_flower_setup_tc_block_cb(enum tc_setup_type type, @@ -1778,6 +1787,7 @@ static int nfp_flower_setup_tc_block(struct net_device *netdev, repr_priv = repr->app_priv; repr_priv->block_shared = f->block_shared; f->driver_block_list = &nfp_block_cb_list; + f->unlocked_driver_cb = true; switch (f->command) { case FLOW_BLOCK_BIND: @@ -1876,6 +1886,8 @@ nfp_flower_setup_indr_tc_block(struct net_device *netdev, struct Qdisc *sch, str nfp_flower_internal_port_can_offload(app, netdev))) return -EOPNOTSUPP; + f->unlocked_driver_cb = true; + switch (f->command) { case FLOW_BLOCK_BIND: cb_priv = nfp_flower_indr_block_cb_priv_lookup(app, netdev); diff --git a/drivers/net/ethernet/netronome/nfp/flower/qos_conf.c b/drivers/net/ethernet/netronome/nfp/flower/qos_conf.c index 99052a925d9e..e7180b4793c7 100644 --- a/drivers/net/ethernet/netronome/nfp/flower/qos_conf.c +++ b/drivers/net/ethernet/netronome/nfp/flower/qos_conf.c @@ -523,25 +523,31 @@ int nfp_flower_setup_qos_offload(struct nfp_app *app, struct net_device *netdev, { struct netlink_ext_ack *extack = flow->common.extack; struct nfp_flower_priv *fl_priv = app->priv; + int ret; if (!(fl_priv->flower_ext_feats & NFP_FL_FEATS_VF_RLIM)) { NL_SET_ERR_MSG_MOD(extack, "unsupported offload: loaded firmware does not support qos rate limit offload"); return -EOPNOTSUPP; } + mutex_lock(&fl_priv->nfp_fl_lock); switch (flow->command) { case TC_CLSMATCHALL_REPLACE: - return nfp_flower_install_rate_limiter(app, netdev, flow, - extack); + ret = nfp_flower_install_rate_limiter(app, netdev, flow, extack); + break; case TC_CLSMATCHALL_DESTROY: - return nfp_flower_remove_rate_limiter(app, netdev, flow, - extack); + ret = nfp_flower_remove_rate_limiter(app, netdev, flow, extack); + break; case TC_CLSMATCHALL_STATS: - return nfp_flower_stats_rate_limiter(app, netdev, flow, - extack); + ret = nfp_flower_stats_rate_limiter(app, netdev, flow, extack); + break; default: - return -EOPNOTSUPP; + ret = -EOPNOTSUPP; + break; } + mutex_unlock(&fl_priv->nfp_fl_lock); + + return ret; } /* Offload tc action, currently only for tc police */ diff --git a/drivers/net/ethernet/pensando/ionic/ionic_dev.h b/drivers/net/ethernet/pensando/ionic/ionic_dev.h index 6aac98bcb9f4..aae4131f146a 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_dev.h +++ b/drivers/net/ethernet/pensando/ionic/ionic_dev.h @@ -187,6 +187,7 @@ typedef void (*ionic_desc_cb)(struct ionic_queue *q, struct ionic_desc_info *desc_info, struct ionic_cq_info *cq_info, void *cb_arg); +#define IONIC_MAX_BUF_LEN ((u16)-1) #define IONIC_PAGE_SIZE PAGE_SIZE #define IONIC_PAGE_SPLIT_SZ (PAGE_SIZE / 2) #define IONIC_PAGE_GFP_MASK (GFP_ATOMIC | __GFP_NOWARN |\ diff --git a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c index 26798fc635db..44466e8c5d77 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c @@ -207,7 +207,8 @@ static struct sk_buff *ionic_rx_frags(struct ionic_queue *q, return NULL; } - frag_len = min_t(u16, len, IONIC_PAGE_SIZE - buf_info->page_offset); + frag_len = min_t(u16, len, min_t(u32, IONIC_MAX_BUF_LEN, + IONIC_PAGE_SIZE - buf_info->page_offset)); len -= frag_len; dma_sync_single_for_cpu(dev, @@ -452,7 +453,8 @@ void ionic_rx_fill(struct ionic_queue *q) /* fill main descriptor - buf[0] */ desc->addr = cpu_to_le64(buf_info->dma_addr + buf_info->page_offset); - frag_len = min_t(u16, len, IONIC_PAGE_SIZE - buf_info->page_offset); + frag_len = min_t(u16, len, min_t(u32, IONIC_MAX_BUF_LEN, + IONIC_PAGE_SIZE - buf_info->page_offset)); desc->len = cpu_to_le16(frag_len); remain_len -= frag_len; buf_info++; @@ -471,7 +473,9 @@ void ionic_rx_fill(struct ionic_queue *q) } sg_elem->addr = cpu_to_le64(buf_info->dma_addr + buf_info->page_offset); - frag_len = min_t(u16, remain_len, IONIC_PAGE_SIZE - buf_info->page_offset); + frag_len = min_t(u16, remain_len, min_t(u32, IONIC_MAX_BUF_LEN, + IONIC_PAGE_SIZE - + buf_info->page_offset)); sg_elem->len = cpu_to_le16(frag_len); remain_len -= frag_len; buf_info++; diff --git a/drivers/net/ethernet/qlogic/qed/qed_ll2.c b/drivers/net/ethernet/qlogic/qed/qed_ll2.c index 717a0b3f89bd..ab5ef254a748 100644 --- a/drivers/net/ethernet/qlogic/qed/qed_ll2.c +++ b/drivers/net/ethernet/qlogic/qed/qed_ll2.c @@ -113,7 +113,10 @@ static void qed_ll2b_complete_tx_packet(void *cxt, static int qed_ll2_alloc_buffer(struct qed_dev *cdev, u8 **data, dma_addr_t *phys_addr) { - *data = kmalloc(cdev->ll2->rx_size, GFP_ATOMIC); + size_t size = cdev->ll2->rx_size + NET_SKB_PAD + + SKB_DATA_ALIGN(sizeof(struct skb_shared_info)); + + *data = kmalloc(size, GFP_ATOMIC); if (!(*data)) { DP_INFO(cdev, "Failed to allocate LL2 buffer data\n"); return -ENOMEM; @@ -2589,7 +2592,7 @@ static int qed_ll2_start(struct qed_dev *cdev, struct qed_ll2_params *params) INIT_LIST_HEAD(&cdev->ll2->list); spin_lock_init(&cdev->ll2->lock); - cdev->ll2->rx_size = NET_SKB_PAD + ETH_HLEN + + cdev->ll2->rx_size = PRM_DMA_PAD_BYTES_NUM + ETH_HLEN + L1_CACHE_BYTES + params->mtu; /* Allocate memory for LL2. diff --git a/drivers/net/ethernet/qlogic/qed/qed_ll2.h b/drivers/net/ethernet/qlogic/qed/qed_ll2.h index 0bfc375161ed..a174c6fc626a 100644 --- a/drivers/net/ethernet/qlogic/qed/qed_ll2.h +++ b/drivers/net/ethernet/qlogic/qed/qed_ll2.h @@ -110,9 +110,9 @@ struct qed_ll2_info { enum core_tx_dest tx_dest; u8 tx_stats_en; bool main_func_queue; + struct qed_ll2_cbs cbs; struct qed_ll2_rx_queue rx_queue; struct qed_ll2_tx_queue tx_queue; - struct qed_ll2_cbs cbs; }; extern const struct qed_ll2_ops qed_ll2_ops_pass; diff --git a/drivers/net/ethernet/renesas/ravb_main.c b/drivers/net/ethernet/renesas/ravb_main.c index 7df9f9f8e134..0ef0b88b7145 100644 --- a/drivers/net/ethernet/renesas/ravb_main.c +++ b/drivers/net/ethernet/renesas/ravb_main.c @@ -2167,6 +2167,8 @@ static int ravb_close(struct net_device *ndev) of_phy_deregister_fixed_link(np); } + cancel_work_sync(&priv->work); + if (info->multi_irqs) { free_irq(priv->tx_irqs[RAVB_NC], ndev); free_irq(priv->rx_irqs[RAVB_NC], ndev); @@ -2891,8 +2893,6 @@ static int ravb_remove(struct platform_device *pdev) clk_disable_unprepare(priv->gptp_clk); clk_disable_unprepare(priv->refclk); - dma_free_coherent(ndev->dev.parent, priv->desc_bat_size, priv->desc_bat, - priv->desc_bat_dma); /* Set reset mode */ ravb_write(ndev, CCC_OPC_RESET, CCC); unregister_netdev(ndev); @@ -2900,6 +2900,8 @@ static int ravb_remove(struct platform_device *pdev) netif_napi_del(&priv->napi[RAVB_NC]); netif_napi_del(&priv->napi[RAVB_BE]); ravb_mdio_release(priv); + dma_free_coherent(ndev->dev.parent, priv->desc_bat_size, priv->desc_bat, + priv->desc_bat_dma); pm_runtime_put_sync(&pdev->dev); pm_runtime_disable(&pdev->dev); reset_control_assert(priv->rstc); diff --git a/drivers/net/ethernet/renesas/rswitch.c b/drivers/net/ethernet/renesas/rswitch.c index 6083b1c8e4fb..0fc0b6bea753 100644 --- a/drivers/net/ethernet/renesas/rswitch.c +++ b/drivers/net/ethernet/renesas/rswitch.c @@ -4,6 +4,7 @@ * Copyright (C) 2022 Renesas Electronics Corporation */ +#include #include #include #include @@ -799,6 +800,7 @@ static int rswitch_poll(struct napi_struct *napi, int budget) struct net_device *ndev = napi->dev; struct rswitch_private *priv; struct rswitch_device *rdev; + unsigned long flags; int quota = budget; rdev = netdev_priv(ndev); @@ -816,10 +818,12 @@ retry: netif_wake_subqueue(ndev, 0); - napi_complete(napi); - - rswitch_enadis_data_irq(priv, rdev->tx_queue->index, true); - rswitch_enadis_data_irq(priv, rdev->rx_queue->index, true); + if (napi_complete_done(napi, budget - quota)) { + spin_lock_irqsave(&priv->lock, flags); + rswitch_enadis_data_irq(priv, rdev->tx_queue->index, true); + rswitch_enadis_data_irq(priv, rdev->rx_queue->index, true); + spin_unlock_irqrestore(&priv->lock, flags); + } out: return budget - quota; @@ -835,8 +839,10 @@ static void rswitch_queue_interrupt(struct net_device *ndev) struct rswitch_device *rdev = netdev_priv(ndev); if (napi_schedule_prep(&rdev->napi)) { + spin_lock(&rdev->priv->lock); rswitch_enadis_data_irq(rdev->priv, rdev->tx_queue->index, false); rswitch_enadis_data_irq(rdev->priv, rdev->rx_queue->index, false); + spin_unlock(&rdev->priv->lock); __napi_schedule(&rdev->napi); } } @@ -1044,7 +1050,7 @@ static void rswitch_rmac_setting(struct rswitch_etha *etha, const u8 *mac) static void rswitch_etha_enable_mii(struct rswitch_etha *etha) { rswitch_modify(etha->addr, MPIC, MPIC_PSMCS_MASK | MPIC_PSMHT_MASK, - MPIC_PSMCS(0x05) | MPIC_PSMHT(0x06)); + MPIC_PSMCS(etha->psmcs) | MPIC_PSMHT(0x06)); rswitch_modify(etha->addr, MPSM, 0, MPSM_MFF_C45); } @@ -1248,7 +1254,7 @@ static void rswitch_adjust_link(struct net_device *ndev) phy_print_status(phydev); if (phydev->link) phy_power_on(rdev->serdes); - else + else if (rdev->serdes->power_count) phy_power_off(rdev->serdes); rdev->etha->link = phydev->link; @@ -1440,14 +1446,17 @@ static void rswitch_ether_port_deinit_all(struct rswitch_private *priv) static int rswitch_open(struct net_device *ndev) { struct rswitch_device *rdev = netdev_priv(ndev); + unsigned long flags; phy_start(ndev->phydev); napi_enable(&rdev->napi); netif_start_queue(ndev); + spin_lock_irqsave(&rdev->priv->lock, flags); rswitch_enadis_data_irq(rdev->priv, rdev->tx_queue->index, true); rswitch_enadis_data_irq(rdev->priv, rdev->rx_queue->index, true); + spin_unlock_irqrestore(&rdev->priv->lock, flags); if (bitmap_empty(rdev->priv->opened_ports, RSWITCH_NUM_PORTS)) iowrite32(GWCA_TS_IRQ_BIT, rdev->priv->addr + GWTSDIE); @@ -1461,6 +1470,7 @@ static int rswitch_stop(struct net_device *ndev) { struct rswitch_device *rdev = netdev_priv(ndev); struct rswitch_gwca_ts_info *ts_info, *ts_info2; + unsigned long flags; netif_tx_stop_all_queues(ndev); bitmap_clear(rdev->priv->opened_ports, rdev->port, 1); @@ -1476,8 +1486,10 @@ static int rswitch_stop(struct net_device *ndev) kfree(ts_info); } + spin_lock_irqsave(&rdev->priv->lock, flags); rswitch_enadis_data_irq(rdev->priv, rdev->tx_queue->index, false); rswitch_enadis_data_irq(rdev->priv, rdev->rx_queue->index, false); + spin_unlock_irqrestore(&rdev->priv->lock, flags); phy_stop(ndev->phydev); napi_disable(&rdev->napi); @@ -1682,6 +1694,12 @@ static void rswitch_etha_init(struct rswitch_private *priv, int index) etha->index = index; etha->addr = priv->addr + RSWITCH_ETHA_OFFSET + index * RSWITCH_ETHA_SIZE; etha->coma_addr = priv->addr; + + /* MPIC.PSMCS = (clk [MHz] / (MDC frequency [MHz] * 2) - 1. + * Calculating PSMCS value as MDC frequency = 2.5MHz. So, multiply + * both the numerator and the denominator by 10. + */ + etha->psmcs = clk_get_rate(priv->clk) / 100000 / (25 * 2) - 1; } static int rswitch_device_alloc(struct rswitch_private *priv, int index) @@ -1887,6 +1905,11 @@ static int renesas_eth_sw_probe(struct platform_device *pdev) priv = devm_kzalloc(&pdev->dev, sizeof(*priv), GFP_KERNEL); if (!priv) return -ENOMEM; + spin_lock_init(&priv->lock); + + priv->clk = devm_clk_get(&pdev->dev, NULL); + if (IS_ERR(priv->clk)) + return PTR_ERR(priv->clk); attr = soc_device_match(rswitch_soc_no_speed_change); if (attr) @@ -1941,15 +1964,17 @@ static void rswitch_deinit(struct rswitch_private *priv) rswitch_gwca_hw_deinit(priv); rcar_gen4_ptp_unregister(priv->ptp_priv); - for (i = 0; i < RSWITCH_NUM_PORTS; i++) { + rswitch_for_each_enabled_port(priv, i) { struct rswitch_device *rdev = priv->rdev[i]; - phy_exit(priv->rdev[i]->serdes); - rswitch_ether_port_deinit_one(rdev); unregister_netdev(rdev->ndev); - rswitch_device_free(priv, i); + rswitch_ether_port_deinit_one(rdev); + phy_exit(priv->rdev[i]->serdes); } + for (i = 0; i < RSWITCH_NUM_PORTS; i++) + rswitch_device_free(priv, i); + rswitch_gwca_ts_queue_free(priv); rswitch_gwca_linkfix_free(priv); diff --git a/drivers/net/ethernet/renesas/rswitch.h b/drivers/net/ethernet/renesas/rswitch.h index 54f397effbc6..04f49a7a5843 100644 --- a/drivers/net/ethernet/renesas/rswitch.h +++ b/drivers/net/ethernet/renesas/rswitch.h @@ -915,6 +915,7 @@ struct rswitch_etha { bool external_phy; struct mii_bus *mii; phy_interface_t phy_interface; + u32 psmcs; u8 mac_addr[MAX_ADDR_LEN]; int link; int speed; @@ -1011,6 +1012,9 @@ struct rswitch_private { struct rswitch_etha etha[RSWITCH_NUM_PORTS]; struct rswitch_mfwd mfwd; + spinlock_t lock; /* lock interrupt registers' control */ + struct clk *clk; + bool etha_no_runtime_change; bool gwca_halt; }; diff --git a/drivers/net/ethernet/sfc/tc.c b/drivers/net/ethernet/sfc/tc.c index 047322b04d4f..834f000ba1c4 100644 --- a/drivers/net/ethernet/sfc/tc.c +++ b/drivers/net/ethernet/sfc/tc.c @@ -136,6 +136,8 @@ static struct efx_tc_mac_pedit_action *efx_tc_flower_get_mac(struct efx_nic *efx if (old) { /* don't need our new entry */ kfree(ped); + if (IS_ERR(old)) /* oh dear, it's actually an error */ + return ERR_CAST(old); if (!refcount_inc_not_zero(&old->ref)) return ERR_PTR(-EAGAIN); /* existing entry found, ref taken */ @@ -602,6 +604,8 @@ static int efx_tc_flower_record_encap_match(struct efx_nic *efx, kfree(encap); if (pseudo) /* don't need our new pseudo either */ efx_tc_flower_release_encap_match(efx, pseudo); + if (IS_ERR(old)) /* oh dear, it's actually an error */ + return PTR_ERR(old); /* check old and new em_types are compatible */ switch (old->type) { case EFX_TC_EM_DIRECT: @@ -700,6 +704,8 @@ static struct efx_tc_recirc_id *efx_tc_get_recirc_id(struct efx_nic *efx, if (old) { /* don't need our new entry */ kfree(rid); + if (IS_ERR(old)) /* oh dear, it's actually an error */ + return ERR_CAST(old); if (!refcount_inc_not_zero(&old->ref)) return ERR_PTR(-EAGAIN); /* existing entry found */ @@ -1482,7 +1488,10 @@ static int efx_tc_flower_replace_foreign(struct efx_nic *efx, old = rhashtable_lookup_get_insert_fast(&efx->tc->match_action_ht, &rule->linkage, efx_tc_match_action_ht_params); - if (old) { + if (IS_ERR(old)) { + rc = PTR_ERR(old); + goto release; + } else if (old) { netif_dbg(efx, drv, efx->net_dev, "Ignoring already-offloaded rule (cookie %lx)\n", tc->cookie); @@ -1697,7 +1706,10 @@ static int efx_tc_flower_replace_lhs(struct efx_nic *efx, old = rhashtable_lookup_get_insert_fast(&efx->tc->lhs_rule_ht, &rule->linkage, efx_tc_lhs_rule_ht_params); - if (old) { + if (IS_ERR(old)) { + rc = PTR_ERR(old); + goto release; + } else if (old) { netif_dbg(efx, drv, efx->net_dev, "Already offloaded rule (cookie %lx)\n", tc->cookie); rc = -EEXIST; @@ -1858,7 +1870,10 @@ static int efx_tc_flower_replace(struct efx_nic *efx, old = rhashtable_lookup_get_insert_fast(&efx->tc->match_action_ht, &rule->linkage, efx_tc_match_action_ht_params); - if (old) { + if (IS_ERR(old)) { + rc = PTR_ERR(old); + goto release; + } else if (old) { netif_dbg(efx, drv, efx->net_dev, "Already offloaded rule (cookie %lx)\n", tc->cookie); NL_SET_ERR_MSG_MOD(extack, "Rule already offloaded"); diff --git a/drivers/net/ethernet/sfc/tc_conntrack.c b/drivers/net/ethernet/sfc/tc_conntrack.c index 8e06bfbcbea1..44bb57670340 100644 --- a/drivers/net/ethernet/sfc/tc_conntrack.c +++ b/drivers/net/ethernet/sfc/tc_conntrack.c @@ -298,7 +298,10 @@ static int efx_tc_ct_replace(struct efx_tc_ct_zone *ct_zone, old = rhashtable_lookup_get_insert_fast(&efx->tc->ct_ht, &conn->linkage, efx_tc_ct_ht_params); - if (old) { + if (IS_ERR(old)) { + rc = PTR_ERR(old); + goto release; + } else if (old) { netif_dbg(efx, drv, efx->net_dev, "Already offloaded conntrack (cookie %lx)\n", tc->cookie); rc = -EEXIST; @@ -482,6 +485,8 @@ struct efx_tc_ct_zone *efx_tc_ct_register_zone(struct efx_nic *efx, u16 zone, if (old) { /* don't need our new entry */ kfree(ct_zone); + if (IS_ERR(old)) /* oh dear, it's actually an error */ + return ERR_CAST(old); if (!refcount_inc_not_zero(&old->ref)) return ERR_PTR(-EAGAIN); /* existing entry found */ diff --git a/drivers/net/ethernet/sfc/tc_counters.c b/drivers/net/ethernet/sfc/tc_counters.c index 0fafb47ea082..c44088424323 100644 --- a/drivers/net/ethernet/sfc/tc_counters.c +++ b/drivers/net/ethernet/sfc/tc_counters.c @@ -236,6 +236,8 @@ struct efx_tc_counter_index *efx_tc_flower_get_counter_index( if (old) { /* don't need our new entry */ kfree(ctr); + if (IS_ERR(old)) /* oh dear, it's actually an error */ + return ERR_CAST(old); if (!refcount_inc_not_zero(&old->ref)) return ERR_PTR(-EAGAIN); /* existing entry found */ diff --git a/drivers/net/ethernet/sfc/tc_encap_actions.c b/drivers/net/ethernet/sfc/tc_encap_actions.c index 7e8bcdb222ad..87443f9dfd22 100644 --- a/drivers/net/ethernet/sfc/tc_encap_actions.c +++ b/drivers/net/ethernet/sfc/tc_encap_actions.c @@ -132,6 +132,8 @@ static int efx_bind_neigh(struct efx_nic *efx, /* don't need our new entry */ put_net_track(neigh->net, &neigh->ns_tracker); kfree(neigh); + if (IS_ERR(old)) /* oh dear, it's actually an error */ + return PTR_ERR(old); if (!refcount_inc_not_zero(&old->ref)) return -EAGAIN; /* existing entry found, ref taken */ @@ -640,6 +642,8 @@ struct efx_tc_encap_action *efx_tc_flower_create_encap_md( if (old) { /* don't need our new entry */ kfree(encap); + if (IS_ERR(old)) /* oh dear, it's actually an error */ + return ERR_CAST(old); if (!refcount_inc_not_zero(&old->ref)) return ERR_PTR(-EAGAIN); /* existing entry found, ref taken */ diff --git a/drivers/net/ethernet/stmicro/stmmac/common.h b/drivers/net/ethernet/stmicro/stmmac/common.h index 403cb397d4d3..1e996c29043d 100644 --- a/drivers/net/ethernet/stmicro/stmmac/common.h +++ b/drivers/net/ethernet/stmicro/stmmac/common.h @@ -70,7 +70,7 @@ struct stmmac_txq_stats { u64 tx_tso_frames; u64 tx_tso_nfrags; struct u64_stats_sync syncp; -}; +} ____cacheline_aligned_in_smp; struct stmmac_rxq_stats { u64 rx_bytes; @@ -79,7 +79,7 @@ struct stmmac_rxq_stats { u64 rx_normal_irq_n; u64 napi_poll; struct u64_stats_sync syncp; -}; +} ____cacheline_aligned_in_smp; /* Extra statistic and debug information exposed by ethtool */ struct stmmac_extra_stats { @@ -202,6 +202,9 @@ struct stmmac_extra_stats { unsigned long mtl_est_hlbf; unsigned long mtl_est_btre; unsigned long mtl_est_btrlm; + /* per queue statistics */ + struct stmmac_txq_stats txq_stats[MTL_MAX_TX_QUEUES]; + struct stmmac_rxq_stats rxq_stats[MTL_MAX_RX_QUEUES]; unsigned long rx_dropped; unsigned long rx_errors; unsigned long tx_dropped; diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac-stm32.c b/drivers/net/ethernet/stmicro/stmmac/dwmac-stm32.c index 26ea8c687881..a0e276783e65 100644 --- a/drivers/net/ethernet/stmicro/stmmac/dwmac-stm32.c +++ b/drivers/net/ethernet/stmicro/stmmac/dwmac-stm32.c @@ -104,6 +104,7 @@ struct stm32_ops { int (*parse_data)(struct stm32_dwmac *dwmac, struct device *dev); u32 syscfg_eth_mask; + bool clk_rx_enable_in_suspend; }; static int stm32_dwmac_init(struct plat_stmmacenet_data *plat_dat) @@ -121,7 +122,8 @@ static int stm32_dwmac_init(struct plat_stmmacenet_data *plat_dat) if (ret) return ret; - if (!dwmac->dev->power.is_suspended) { + if (!dwmac->ops->clk_rx_enable_in_suspend || + !dwmac->dev->power.is_suspended) { ret = clk_prepare_enable(dwmac->clk_rx); if (ret) { clk_disable_unprepare(dwmac->clk_tx); @@ -513,7 +515,8 @@ static struct stm32_ops stm32mp1_dwmac_data = { .suspend = stm32mp1_suspend, .resume = stm32mp1_resume, .parse_data = stm32mp1_parse_data, - .syscfg_eth_mask = SYSCFG_MP1_ETH_MASK + .syscfg_eth_mask = SYSCFG_MP1_ETH_MASK, + .clk_rx_enable_in_suspend = true }; static const struct of_device_id stm32_dwmac_match[] = { diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac-sun8i.c b/drivers/net/ethernet/stmicro/stmmac/dwmac-sun8i.c index 01e77368eef1..465ff1fd4785 100644 --- a/drivers/net/ethernet/stmicro/stmmac/dwmac-sun8i.c +++ b/drivers/net/ethernet/stmicro/stmmac/dwmac-sun8i.c @@ -441,8 +441,8 @@ static int sun8i_dwmac_dma_interrupt(struct stmmac_priv *priv, struct stmmac_extra_stats *x, u32 chan, u32 dir) { - struct stmmac_rx_queue *rx_q = &priv->dma_conf.rx_queue[chan]; - struct stmmac_tx_queue *tx_q = &priv->dma_conf.tx_queue[chan]; + struct stmmac_rxq_stats *rxq_stats = &priv->xstats.rxq_stats[chan]; + struct stmmac_txq_stats *txq_stats = &priv->xstats.txq_stats[chan]; int ret = 0; u32 v; @@ -455,9 +455,9 @@ static int sun8i_dwmac_dma_interrupt(struct stmmac_priv *priv, if (v & EMAC_TX_INT) { ret |= handle_tx; - u64_stats_update_begin(&tx_q->txq_stats.syncp); - tx_q->txq_stats.tx_normal_irq_n++; - u64_stats_update_end(&tx_q->txq_stats.syncp); + u64_stats_update_begin(&txq_stats->syncp); + txq_stats->tx_normal_irq_n++; + u64_stats_update_end(&txq_stats->syncp); } if (v & EMAC_TX_DMA_STOP_INT) @@ -479,9 +479,9 @@ static int sun8i_dwmac_dma_interrupt(struct stmmac_priv *priv, if (v & EMAC_RX_INT) { ret |= handle_rx; - u64_stats_update_begin(&rx_q->rxq_stats.syncp); - rx_q->rxq_stats.rx_normal_irq_n++; - u64_stats_update_end(&rx_q->rxq_stats.syncp); + u64_stats_update_begin(&rxq_stats->syncp); + rxq_stats->rx_normal_irq_n++; + u64_stats_update_end(&rxq_stats->syncp); } if (v & EMAC_RX_BUF_UA_INT) diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac4_lib.c b/drivers/net/ethernet/stmicro/stmmac/dwmac4_lib.c index 980e5f8a37ec..9470d3fd2ded 100644 --- a/drivers/net/ethernet/stmicro/stmmac/dwmac4_lib.c +++ b/drivers/net/ethernet/stmicro/stmmac/dwmac4_lib.c @@ -171,8 +171,8 @@ int dwmac4_dma_interrupt(struct stmmac_priv *priv, void __iomem *ioaddr, const struct dwmac4_addrs *dwmac4_addrs = priv->plat->dwmac4_addrs; u32 intr_status = readl(ioaddr + DMA_CHAN_STATUS(dwmac4_addrs, chan)); u32 intr_en = readl(ioaddr + DMA_CHAN_INTR_ENA(dwmac4_addrs, chan)); - struct stmmac_rx_queue *rx_q = &priv->dma_conf.rx_queue[chan]; - struct stmmac_tx_queue *tx_q = &priv->dma_conf.tx_queue[chan]; + struct stmmac_rxq_stats *rxq_stats = &priv->xstats.rxq_stats[chan]; + struct stmmac_txq_stats *txq_stats = &priv->xstats.txq_stats[chan]; int ret = 0; if (dir == DMA_DIR_RX) @@ -201,15 +201,15 @@ int dwmac4_dma_interrupt(struct stmmac_priv *priv, void __iomem *ioaddr, } /* TX/RX NORMAL interrupts */ if (likely(intr_status & DMA_CHAN_STATUS_RI)) { - u64_stats_update_begin(&rx_q->rxq_stats.syncp); - rx_q->rxq_stats.rx_normal_irq_n++; - u64_stats_update_end(&rx_q->rxq_stats.syncp); + u64_stats_update_begin(&rxq_stats->syncp); + rxq_stats->rx_normal_irq_n++; + u64_stats_update_end(&rxq_stats->syncp); ret |= handle_rx; } if (likely(intr_status & DMA_CHAN_STATUS_TI)) { - u64_stats_update_begin(&tx_q->txq_stats.syncp); - tx_q->txq_stats.tx_normal_irq_n++; - u64_stats_update_end(&tx_q->txq_stats.syncp); + u64_stats_update_begin(&txq_stats->syncp); + txq_stats->tx_normal_irq_n++; + u64_stats_update_end(&txq_stats->syncp); ret |= handle_tx; } diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac_lib.c b/drivers/net/ethernet/stmicro/stmmac/dwmac_lib.c index aaa09b16b016..7907d62d3437 100644 --- a/drivers/net/ethernet/stmicro/stmmac/dwmac_lib.c +++ b/drivers/net/ethernet/stmicro/stmmac/dwmac_lib.c @@ -162,8 +162,8 @@ static void show_rx_process_state(unsigned int status) int dwmac_dma_interrupt(struct stmmac_priv *priv, void __iomem *ioaddr, struct stmmac_extra_stats *x, u32 chan, u32 dir) { - struct stmmac_rx_queue *rx_q = &priv->dma_conf.rx_queue[chan]; - struct stmmac_tx_queue *tx_q = &priv->dma_conf.tx_queue[chan]; + struct stmmac_rxq_stats *rxq_stats = &priv->xstats.rxq_stats[chan]; + struct stmmac_txq_stats *txq_stats = &priv->xstats.txq_stats[chan]; int ret = 0; /* read the status register (CSR5) */ u32 intr_status = readl(ioaddr + DMA_STATUS); @@ -215,16 +215,16 @@ int dwmac_dma_interrupt(struct stmmac_priv *priv, void __iomem *ioaddr, u32 value = readl(ioaddr + DMA_INTR_ENA); /* to schedule NAPI on real RIE event. */ if (likely(value & DMA_INTR_ENA_RIE)) { - u64_stats_update_begin(&rx_q->rxq_stats.syncp); - rx_q->rxq_stats.rx_normal_irq_n++; - u64_stats_update_end(&rx_q->rxq_stats.syncp); + u64_stats_update_begin(&rxq_stats->syncp); + rxq_stats->rx_normal_irq_n++; + u64_stats_update_end(&rxq_stats->syncp); ret |= handle_rx; } } if (likely(intr_status & DMA_STATUS_TI)) { - u64_stats_update_begin(&tx_q->txq_stats.syncp); - tx_q->txq_stats.tx_normal_irq_n++; - u64_stats_update_end(&tx_q->txq_stats.syncp); + u64_stats_update_begin(&txq_stats->syncp); + txq_stats->tx_normal_irq_n++; + u64_stats_update_end(&txq_stats->syncp); ret |= handle_tx; } if (unlikely(intr_status & DMA_STATUS_ERI)) diff --git a/drivers/net/ethernet/stmicro/stmmac/dwxgmac2_dma.c b/drivers/net/ethernet/stmicro/stmmac/dwxgmac2_dma.c index fa69d64a8694..3cde695fec91 100644 --- a/drivers/net/ethernet/stmicro/stmmac/dwxgmac2_dma.c +++ b/drivers/net/ethernet/stmicro/stmmac/dwxgmac2_dma.c @@ -337,8 +337,8 @@ static int dwxgmac2_dma_interrupt(struct stmmac_priv *priv, struct stmmac_extra_stats *x, u32 chan, u32 dir) { - struct stmmac_rx_queue *rx_q = &priv->dma_conf.rx_queue[chan]; - struct stmmac_tx_queue *tx_q = &priv->dma_conf.tx_queue[chan]; + struct stmmac_rxq_stats *rxq_stats = &priv->xstats.rxq_stats[chan]; + struct stmmac_txq_stats *txq_stats = &priv->xstats.txq_stats[chan]; u32 intr_status = readl(ioaddr + XGMAC_DMA_CH_STATUS(chan)); u32 intr_en = readl(ioaddr + XGMAC_DMA_CH_INT_EN(chan)); int ret = 0; @@ -367,15 +367,15 @@ static int dwxgmac2_dma_interrupt(struct stmmac_priv *priv, /* TX/RX NORMAL interrupts */ if (likely(intr_status & XGMAC_NIS)) { if (likely(intr_status & XGMAC_RI)) { - u64_stats_update_begin(&rx_q->rxq_stats.syncp); - rx_q->rxq_stats.rx_normal_irq_n++; - u64_stats_update_end(&rx_q->rxq_stats.syncp); + u64_stats_update_begin(&rxq_stats->syncp); + rxq_stats->rx_normal_irq_n++; + u64_stats_update_end(&rxq_stats->syncp); ret |= handle_rx; } if (likely(intr_status & (XGMAC_TI | XGMAC_TBU))) { - u64_stats_update_begin(&tx_q->txq_stats.syncp); - tx_q->txq_stats.tx_normal_irq_n++; - u64_stats_update_end(&tx_q->txq_stats.syncp); + u64_stats_update_begin(&txq_stats->syncp); + txq_stats->tx_normal_irq_n++; + u64_stats_update_end(&txq_stats->syncp); ret |= handle_tx; } } diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac.h b/drivers/net/ethernet/stmicro/stmmac/stmmac.h index 3401e888a9f6..cd7a9768de5f 100644 --- a/drivers/net/ethernet/stmicro/stmmac/stmmac.h +++ b/drivers/net/ethernet/stmicro/stmmac/stmmac.h @@ -78,7 +78,6 @@ struct stmmac_tx_queue { dma_addr_t dma_tx_phy; dma_addr_t tx_tail_addr; u32 mss; - struct stmmac_txq_stats txq_stats; }; struct stmmac_rx_buffer { @@ -123,7 +122,6 @@ struct stmmac_rx_queue { unsigned int len; unsigned int error; } state; - struct stmmac_rxq_stats rxq_stats; }; struct stmmac_channel { diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_ethtool.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_ethtool.c index b7ac7abecdd3..6aa5c0556d22 100644 --- a/drivers/net/ethernet/stmicro/stmmac/stmmac_ethtool.c +++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_ethtool.c @@ -548,14 +548,14 @@ static void stmmac_get_per_qstats(struct stmmac_priv *priv, u64 *data) pos = data; for (q = 0; q < tx_cnt; q++) { - struct stmmac_tx_queue *tx_q = &priv->dma_conf.tx_queue[q]; + struct stmmac_txq_stats *txq_stats = &priv->xstats.txq_stats[q]; struct stmmac_txq_stats snapshot; data = pos; do { - start = u64_stats_fetch_begin(&tx_q->txq_stats.syncp); - snapshot = tx_q->txq_stats; - } while (u64_stats_fetch_retry(&tx_q->txq_stats.syncp, start)); + start = u64_stats_fetch_begin(&txq_stats->syncp); + snapshot = *txq_stats; + } while (u64_stats_fetch_retry(&txq_stats->syncp, start)); p = (char *)&snapshot + offsetof(struct stmmac_txq_stats, tx_pkt_n); for (stat = 0; stat < STMMAC_TXQ_STATS; stat++) { @@ -566,14 +566,14 @@ static void stmmac_get_per_qstats(struct stmmac_priv *priv, u64 *data) pos = data; for (q = 0; q < rx_cnt; q++) { - struct stmmac_rx_queue *rx_q = &priv->dma_conf.rx_queue[q]; + struct stmmac_rxq_stats *rxq_stats = &priv->xstats.rxq_stats[q]; struct stmmac_rxq_stats snapshot; data = pos; do { - start = u64_stats_fetch_begin(&rx_q->rxq_stats.syncp); - snapshot = rx_q->rxq_stats; - } while (u64_stats_fetch_retry(&rx_q->rxq_stats.syncp, start)); + start = u64_stats_fetch_begin(&rxq_stats->syncp); + snapshot = *rxq_stats; + } while (u64_stats_fetch_retry(&rxq_stats->syncp, start)); p = (char *)&snapshot + offsetof(struct stmmac_rxq_stats, rx_pkt_n); for (stat = 0; stat < STMMAC_RXQ_STATS; stat++) { @@ -637,14 +637,14 @@ static void stmmac_get_ethtool_stats(struct net_device *dev, pos = j; for (i = 0; i < rx_queues_count; i++) { - struct stmmac_rx_queue *rx_q = &priv->dma_conf.rx_queue[i]; + struct stmmac_rxq_stats *rxq_stats = &priv->xstats.rxq_stats[i]; struct stmmac_rxq_stats snapshot; j = pos; do { - start = u64_stats_fetch_begin(&rx_q->rxq_stats.syncp); - snapshot = rx_q->rxq_stats; - } while (u64_stats_fetch_retry(&rx_q->rxq_stats.syncp, start)); + start = u64_stats_fetch_begin(&rxq_stats->syncp); + snapshot = *rxq_stats; + } while (u64_stats_fetch_retry(&rxq_stats->syncp, start)); data[j++] += snapshot.rx_pkt_n; data[j++] += snapshot.rx_normal_irq_n; @@ -654,14 +654,14 @@ static void stmmac_get_ethtool_stats(struct net_device *dev, pos = j; for (i = 0; i < tx_queues_count; i++) { - struct stmmac_tx_queue *tx_q = &priv->dma_conf.tx_queue[i]; + struct stmmac_txq_stats *txq_stats = &priv->xstats.txq_stats[i]; struct stmmac_txq_stats snapshot; j = pos; do { - start = u64_stats_fetch_begin(&tx_q->txq_stats.syncp); - snapshot = tx_q->txq_stats; - } while (u64_stats_fetch_retry(&tx_q->txq_stats.syncp, start)); + start = u64_stats_fetch_begin(&txq_stats->syncp); + snapshot = *txq_stats; + } while (u64_stats_fetch_retry(&txq_stats->syncp, start)); data[j++] += snapshot.tx_pkt_n; data[j++] += snapshot.tx_normal_irq_n; diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c index 9a3182b9e767..ed1a5a31a491 100644 --- a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c +++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c @@ -2426,6 +2426,7 @@ static bool stmmac_xdp_xmit_zc(struct stmmac_priv *priv, u32 queue, u32 budget) { struct netdev_queue *nq = netdev_get_tx_queue(priv->dev, queue); struct stmmac_tx_queue *tx_q = &priv->dma_conf.tx_queue[queue]; + struct stmmac_txq_stats *txq_stats = &priv->xstats.txq_stats[queue]; struct xsk_buff_pool *pool = tx_q->xsk_pool; unsigned int entry = tx_q->cur_tx; struct dma_desc *tx_desc = NULL; @@ -2505,9 +2506,9 @@ static bool stmmac_xdp_xmit_zc(struct stmmac_priv *priv, u32 queue, u32 budget) tx_q->cur_tx = STMMAC_GET_ENTRY(tx_q->cur_tx, priv->dma_conf.dma_tx_size); entry = tx_q->cur_tx; } - flags = u64_stats_update_begin_irqsave(&tx_q->txq_stats.syncp); - tx_q->txq_stats.tx_set_ic_bit += tx_set_ic_bit; - u64_stats_update_end_irqrestore(&tx_q->txq_stats.syncp, flags); + flags = u64_stats_update_begin_irqsave(&txq_stats->syncp); + txq_stats->tx_set_ic_bit += tx_set_ic_bit; + u64_stats_update_end_irqrestore(&txq_stats->syncp, flags); if (tx_desc) { stmmac_flush_tx_descriptors(priv, queue); @@ -2547,6 +2548,7 @@ static void stmmac_bump_dma_threshold(struct stmmac_priv *priv, u32 chan) static int stmmac_tx_clean(struct stmmac_priv *priv, int budget, u32 queue) { struct stmmac_tx_queue *tx_q = &priv->dma_conf.tx_queue[queue]; + struct stmmac_txq_stats *txq_stats = &priv->xstats.txq_stats[queue]; unsigned int bytes_compl = 0, pkts_compl = 0; unsigned int entry, xmits = 0, count = 0; u32 tx_packets = 0, tx_errors = 0; @@ -2704,15 +2706,13 @@ static int stmmac_tx_clean(struct stmmac_priv *priv, int budget, u32 queue) /* We still have pending packets, let's call for a new scheduling */ if (tx_q->dirty_tx != tx_q->cur_tx) - hrtimer_start(&tx_q->txtimer, - STMMAC_COAL_TIMER(priv->tx_coal_timer[queue]), - HRTIMER_MODE_REL); + stmmac_tx_timer_arm(priv, queue); - flags = u64_stats_update_begin_irqsave(&tx_q->txq_stats.syncp); - tx_q->txq_stats.tx_packets += tx_packets; - tx_q->txq_stats.tx_pkt_n += tx_packets; - tx_q->txq_stats.tx_clean++; - u64_stats_update_end_irqrestore(&tx_q->txq_stats.syncp, flags); + flags = u64_stats_update_begin_irqsave(&txq_stats->syncp); + txq_stats->tx_packets += tx_packets; + txq_stats->tx_pkt_n += tx_packets; + txq_stats->tx_clean++; + u64_stats_update_end_irqrestore(&txq_stats->syncp, flags); priv->xstats.tx_errors += tx_errors; @@ -2995,9 +2995,13 @@ static int stmmac_init_dma_engine(struct stmmac_priv *priv) static void stmmac_tx_timer_arm(struct stmmac_priv *priv, u32 queue) { struct stmmac_tx_queue *tx_q = &priv->dma_conf.tx_queue[queue]; + u32 tx_coal_timer = priv->tx_coal_timer[queue]; + + if (!tx_coal_timer) + return; hrtimer_start(&tx_q->txtimer, - STMMAC_COAL_TIMER(priv->tx_coal_timer[queue]), + STMMAC_COAL_TIMER(tx_coal_timer), HRTIMER_MODE_REL); } @@ -4112,6 +4116,7 @@ static netdev_tx_t stmmac_tso_xmit(struct sk_buff *skb, struct net_device *dev) int nfrags = skb_shinfo(skb)->nr_frags; u32 queue = skb_get_queue_mapping(skb); unsigned int first_entry, tx_packets; + struct stmmac_txq_stats *txq_stats; int tmp_pay_len = 0, first_tx; struct stmmac_tx_queue *tx_q; bool has_vlan, set_ic; @@ -4122,6 +4127,7 @@ static netdev_tx_t stmmac_tso_xmit(struct sk_buff *skb, struct net_device *dev) int i; tx_q = &priv->dma_conf.tx_queue[queue]; + txq_stats = &priv->xstats.txq_stats[queue]; first_tx = tx_q->cur_tx; /* Compute header lengths */ @@ -4280,13 +4286,13 @@ static netdev_tx_t stmmac_tso_xmit(struct sk_buff *skb, struct net_device *dev) netif_tx_stop_queue(netdev_get_tx_queue(priv->dev, queue)); } - flags = u64_stats_update_begin_irqsave(&tx_q->txq_stats.syncp); - tx_q->txq_stats.tx_bytes += skb->len; - tx_q->txq_stats.tx_tso_frames++; - tx_q->txq_stats.tx_tso_nfrags += nfrags; + flags = u64_stats_update_begin_irqsave(&txq_stats->syncp); + txq_stats->tx_bytes += skb->len; + txq_stats->tx_tso_frames++; + txq_stats->tx_tso_nfrags += nfrags; if (set_ic) - tx_q->txq_stats.tx_set_ic_bit++; - u64_stats_update_end_irqrestore(&tx_q->txq_stats.syncp, flags); + txq_stats->tx_set_ic_bit++; + u64_stats_update_end_irqrestore(&txq_stats->syncp, flags); if (priv->sarc_type) stmmac_set_desc_sarc(priv, first, priv->sarc_type); @@ -4357,6 +4363,7 @@ static netdev_tx_t stmmac_xmit(struct sk_buff *skb, struct net_device *dev) u32 queue = skb_get_queue_mapping(skb); int nfrags = skb_shinfo(skb)->nr_frags; int gso = skb_shinfo(skb)->gso_type; + struct stmmac_txq_stats *txq_stats; struct dma_edesc *tbs_desc = NULL; struct dma_desc *desc, *first; struct stmmac_tx_queue *tx_q; @@ -4366,6 +4373,7 @@ static netdev_tx_t stmmac_xmit(struct sk_buff *skb, struct net_device *dev) dma_addr_t des; tx_q = &priv->dma_conf.tx_queue[queue]; + txq_stats = &priv->xstats.txq_stats[queue]; first_tx = tx_q->cur_tx; if (priv->tx_path_in_lpi_mode && priv->eee_sw_timer_en) @@ -4517,11 +4525,11 @@ static netdev_tx_t stmmac_xmit(struct sk_buff *skb, struct net_device *dev) netif_tx_stop_queue(netdev_get_tx_queue(priv->dev, queue)); } - flags = u64_stats_update_begin_irqsave(&tx_q->txq_stats.syncp); - tx_q->txq_stats.tx_bytes += skb->len; + flags = u64_stats_update_begin_irqsave(&txq_stats->syncp); + txq_stats->tx_bytes += skb->len; if (set_ic) - tx_q->txq_stats.tx_set_ic_bit++; - u64_stats_update_end_irqrestore(&tx_q->txq_stats.syncp, flags); + txq_stats->tx_set_ic_bit++; + u64_stats_update_end_irqrestore(&txq_stats->syncp, flags); if (priv->sarc_type) stmmac_set_desc_sarc(priv, first, priv->sarc_type); @@ -4728,6 +4736,7 @@ static unsigned int stmmac_rx_buf2_len(struct stmmac_priv *priv, static int stmmac_xdp_xmit_xdpf(struct stmmac_priv *priv, int queue, struct xdp_frame *xdpf, bool dma_map) { + struct stmmac_txq_stats *txq_stats = &priv->xstats.txq_stats[queue]; struct stmmac_tx_queue *tx_q = &priv->dma_conf.tx_queue[queue]; unsigned int entry = tx_q->cur_tx; struct dma_desc *tx_desc; @@ -4787,9 +4796,9 @@ static int stmmac_xdp_xmit_xdpf(struct stmmac_priv *priv, int queue, unsigned long flags; tx_q->tx_count_frames = 0; stmmac_set_tx_ic(priv, tx_desc); - flags = u64_stats_update_begin_irqsave(&tx_q->txq_stats.syncp); - tx_q->txq_stats.tx_set_ic_bit++; - u64_stats_update_end_irqrestore(&tx_q->txq_stats.syncp, flags); + flags = u64_stats_update_begin_irqsave(&txq_stats->syncp); + txq_stats->tx_set_ic_bit++; + u64_stats_update_end_irqrestore(&txq_stats->syncp, flags); } stmmac_enable_dma_transmission(priv, priv->ioaddr); @@ -4934,7 +4943,7 @@ static void stmmac_dispatch_skb_zc(struct stmmac_priv *priv, u32 queue, struct dma_desc *p, struct dma_desc *np, struct xdp_buff *xdp) { - struct stmmac_rx_queue *rx_q = &priv->dma_conf.rx_queue[queue]; + struct stmmac_rxq_stats *rxq_stats = &priv->xstats.rxq_stats[queue]; struct stmmac_channel *ch = &priv->channel[queue]; unsigned int len = xdp->data_end - xdp->data; enum pkt_hash_types hash_type; @@ -4964,10 +4973,10 @@ static void stmmac_dispatch_skb_zc(struct stmmac_priv *priv, u32 queue, skb_record_rx_queue(skb, queue); napi_gro_receive(&ch->rxtx_napi, skb); - flags = u64_stats_update_begin_irqsave(&rx_q->rxq_stats.syncp); - rx_q->rxq_stats.rx_pkt_n++; - rx_q->rxq_stats.rx_bytes += len; - u64_stats_update_end_irqrestore(&rx_q->rxq_stats.syncp, flags); + flags = u64_stats_update_begin_irqsave(&rxq_stats->syncp); + rxq_stats->rx_pkt_n++; + rxq_stats->rx_bytes += len; + u64_stats_update_end_irqrestore(&rxq_stats->syncp, flags); } static bool stmmac_rx_refill_zc(struct stmmac_priv *priv, u32 queue, u32 budget) @@ -5040,6 +5049,7 @@ static struct stmmac_xdp_buff *xsk_buff_to_stmmac_ctx(struct xdp_buff *xdp) static int stmmac_rx_zc(struct stmmac_priv *priv, int limit, u32 queue) { + struct stmmac_rxq_stats *rxq_stats = &priv->xstats.rxq_stats[queue]; struct stmmac_rx_queue *rx_q = &priv->dma_conf.rx_queue[queue]; unsigned int count = 0, error = 0, len = 0; int dirty = stmmac_rx_dirty(priv, queue); @@ -5203,9 +5213,9 @@ read_again: stmmac_finalize_xdp_rx(priv, xdp_status); - flags = u64_stats_update_begin_irqsave(&rx_q->rxq_stats.syncp); - rx_q->rxq_stats.rx_pkt_n += count; - u64_stats_update_end_irqrestore(&rx_q->rxq_stats.syncp, flags); + flags = u64_stats_update_begin_irqsave(&rxq_stats->syncp); + rxq_stats->rx_pkt_n += count; + u64_stats_update_end_irqrestore(&rxq_stats->syncp, flags); priv->xstats.rx_dropped += rx_dropped; priv->xstats.rx_errors += rx_errors; @@ -5233,6 +5243,7 @@ read_again: static int stmmac_rx(struct stmmac_priv *priv, int limit, u32 queue) { u32 rx_errors = 0, rx_dropped = 0, rx_bytes = 0, rx_packets = 0; + struct stmmac_rxq_stats *rxq_stats = &priv->xstats.rxq_stats[queue]; struct stmmac_rx_queue *rx_q = &priv->dma_conf.rx_queue[queue]; struct stmmac_channel *ch = &priv->channel[queue]; unsigned int count = 0, error = 0, len = 0; @@ -5494,11 +5505,11 @@ drain_data: stmmac_rx_refill(priv, queue); - flags = u64_stats_update_begin_irqsave(&rx_q->rxq_stats.syncp); - rx_q->rxq_stats.rx_packets += rx_packets; - rx_q->rxq_stats.rx_bytes += rx_bytes; - rx_q->rxq_stats.rx_pkt_n += count; - u64_stats_update_end_irqrestore(&rx_q->rxq_stats.syncp, flags); + flags = u64_stats_update_begin_irqsave(&rxq_stats->syncp); + rxq_stats->rx_packets += rx_packets; + rxq_stats->rx_bytes += rx_bytes; + rxq_stats->rx_pkt_n += count; + u64_stats_update_end_irqrestore(&rxq_stats->syncp, flags); priv->xstats.rx_dropped += rx_dropped; priv->xstats.rx_errors += rx_errors; @@ -5511,15 +5522,15 @@ static int stmmac_napi_poll_rx(struct napi_struct *napi, int budget) struct stmmac_channel *ch = container_of(napi, struct stmmac_channel, rx_napi); struct stmmac_priv *priv = ch->priv_data; - struct stmmac_rx_queue *rx_q; + struct stmmac_rxq_stats *rxq_stats; u32 chan = ch->index; unsigned long flags; int work_done; - rx_q = &priv->dma_conf.rx_queue[chan]; - flags = u64_stats_update_begin_irqsave(&rx_q->rxq_stats.syncp); - rx_q->rxq_stats.napi_poll++; - u64_stats_update_end_irqrestore(&rx_q->rxq_stats.syncp, flags); + rxq_stats = &priv->xstats.rxq_stats[chan]; + flags = u64_stats_update_begin_irqsave(&rxq_stats->syncp); + rxq_stats->napi_poll++; + u64_stats_update_end_irqrestore(&rxq_stats->syncp, flags); work_done = stmmac_rx(priv, budget, chan); if (work_done < budget && napi_complete_done(napi, work_done)) { @@ -5538,15 +5549,15 @@ static int stmmac_napi_poll_tx(struct napi_struct *napi, int budget) struct stmmac_channel *ch = container_of(napi, struct stmmac_channel, tx_napi); struct stmmac_priv *priv = ch->priv_data; - struct stmmac_tx_queue *tx_q; + struct stmmac_txq_stats *txq_stats; u32 chan = ch->index; unsigned long flags; int work_done; - tx_q = &priv->dma_conf.tx_queue[chan]; - flags = u64_stats_update_begin_irqsave(&tx_q->txq_stats.syncp); - tx_q->txq_stats.napi_poll++; - u64_stats_update_end_irqrestore(&tx_q->txq_stats.syncp, flags); + txq_stats = &priv->xstats.txq_stats[chan]; + flags = u64_stats_update_begin_irqsave(&txq_stats->syncp); + txq_stats->napi_poll++; + u64_stats_update_end_irqrestore(&txq_stats->syncp, flags); work_done = stmmac_tx_clean(priv, budget, chan); work_done = min(work_done, budget); @@ -5568,20 +5579,20 @@ static int stmmac_napi_poll_rxtx(struct napi_struct *napi, int budget) container_of(napi, struct stmmac_channel, rxtx_napi); struct stmmac_priv *priv = ch->priv_data; int rx_done, tx_done, rxtx_done; - struct stmmac_rx_queue *rx_q; - struct stmmac_tx_queue *tx_q; + struct stmmac_rxq_stats *rxq_stats; + struct stmmac_txq_stats *txq_stats; u32 chan = ch->index; unsigned long flags; - rx_q = &priv->dma_conf.rx_queue[chan]; - flags = u64_stats_update_begin_irqsave(&rx_q->rxq_stats.syncp); - rx_q->rxq_stats.napi_poll++; - u64_stats_update_end_irqrestore(&rx_q->rxq_stats.syncp, flags); + rxq_stats = &priv->xstats.rxq_stats[chan]; + flags = u64_stats_update_begin_irqsave(&rxq_stats->syncp); + rxq_stats->napi_poll++; + u64_stats_update_end_irqrestore(&rxq_stats->syncp, flags); - tx_q = &priv->dma_conf.tx_queue[chan]; - flags = u64_stats_update_begin_irqsave(&tx_q->txq_stats.syncp); - tx_q->txq_stats.napi_poll++; - u64_stats_update_end_irqrestore(&tx_q->txq_stats.syncp, flags); + txq_stats = &priv->xstats.txq_stats[chan]; + flags = u64_stats_update_begin_irqsave(&txq_stats->syncp); + txq_stats->napi_poll++; + u64_stats_update_end_irqrestore(&txq_stats->syncp, flags); tx_done = stmmac_tx_clean(priv, budget, chan); tx_done = min(tx_done, budget); @@ -5991,33 +6002,6 @@ static irqreturn_t stmmac_msi_intr_rx(int irq, void *data) return IRQ_HANDLED; } -#ifdef CONFIG_NET_POLL_CONTROLLER -/* Polling receive - used by NETCONSOLE and other diagnostic tools - * to allow network I/O with interrupts disabled. - */ -static void stmmac_poll_controller(struct net_device *dev) -{ - struct stmmac_priv *priv = netdev_priv(dev); - int i; - - /* If adapter is down, do nothing */ - if (test_bit(STMMAC_DOWN, &priv->state)) - return; - - if (priv->plat->flags & STMMAC_FLAG_MULTI_MSI_EN) { - for (i = 0; i < priv->plat->rx_queues_to_use; i++) - stmmac_msi_intr_rx(0, &priv->dma_conf.rx_queue[i]); - - for (i = 0; i < priv->plat->tx_queues_to_use; i++) - stmmac_msi_intr_tx(0, &priv->dma_conf.tx_queue[i]); - } else { - disable_irq(dev->irq); - stmmac_interrupt(dev->irq, dev); - enable_irq(dev->irq); - } -} -#endif - /** * stmmac_ioctl - Entry point for the Ioctl * @dev: Device pointer. @@ -6924,7 +6908,7 @@ static void stmmac_get_stats64(struct net_device *dev, struct rtnl_link_stats64 int q; for (q = 0; q < tx_cnt; q++) { - struct stmmac_txq_stats *txq_stats = &priv->dma_conf.tx_queue[q].txq_stats; + struct stmmac_txq_stats *txq_stats = &priv->xstats.txq_stats[q]; u64 tx_packets; u64 tx_bytes; @@ -6939,7 +6923,7 @@ static void stmmac_get_stats64(struct net_device *dev, struct rtnl_link_stats64 } for (q = 0; q < rx_cnt; q++) { - struct stmmac_rxq_stats *rxq_stats = &priv->dma_conf.rx_queue[q].rxq_stats; + struct stmmac_rxq_stats *rxq_stats = &priv->xstats.rxq_stats[q]; u64 rx_packets; u64 rx_bytes; @@ -6978,9 +6962,6 @@ static const struct net_device_ops stmmac_netdev_ops = { .ndo_get_stats64 = stmmac_get_stats64, .ndo_setup_tc = stmmac_setup_tc, .ndo_select_queue = stmmac_select_queue, -#ifdef CONFIG_NET_POLL_CONTROLLER - .ndo_poll_controller = stmmac_poll_controller, -#endif .ndo_set_mac_address = stmmac_set_mac_address, .ndo_vlan_rx_add_vid = stmmac_vlan_rx_add_vid, .ndo_vlan_rx_kill_vid = stmmac_vlan_rx_kill_vid, @@ -7340,9 +7321,9 @@ int stmmac_dvr_probe(struct device *device, priv->dev = ndev; for (i = 0; i < MTL_MAX_RX_QUEUES; i++) - u64_stats_init(&priv->dma_conf.rx_queue[i].rxq_stats.syncp); + u64_stats_init(&priv->xstats.rxq_stats[i].syncp); for (i = 0; i < MTL_MAX_TX_QUEUES; i++) - u64_stats_init(&priv->dma_conf.tx_queue[i].txq_stats.syncp); + u64_stats_init(&priv->xstats.txq_stats[i].syncp); stmmac_set_ethtool_ops(ndev); priv->pause = pause; diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_platform.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_platform.c index 0f28795e581c..2f0678f15fb7 100644 --- a/drivers/net/ethernet/stmicro/stmmac/stmmac_platform.c +++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_platform.c @@ -901,7 +901,7 @@ static int __maybe_unused stmmac_pltfr_resume(struct device *dev) struct platform_device *pdev = to_platform_device(dev); int ret; - ret = stmmac_pltfr_init(pdev, priv->plat->bsp_priv); + ret = stmmac_pltfr_init(pdev, priv->plat); if (ret) return ret; diff --git a/drivers/net/ethernet/ti/Kconfig b/drivers/net/ethernet/ti/Kconfig index 88b5b1b47779..cac61f5d3fd4 100644 --- a/drivers/net/ethernet/ti/Kconfig +++ b/drivers/net/ethernet/ti/Kconfig @@ -90,12 +90,16 @@ config TI_CPTS The unit can time stamp PTP UDP/IPv4 and Layer 2 packets, and the driver offers a PTP Hardware Clock. +config TI_K3_CPPI_DESC_POOL + tristate + config TI_K3_AM65_CPSW_NUSS tristate "TI K3 AM654x/J721E CPSW Ethernet driver" depends on ARCH_K3 && OF && TI_K3_UDMA_GLUE_LAYER select NET_DEVLINK select TI_DAVINCI_MDIO select PHYLINK + select TI_K3_CPPI_DESC_POOL imply PHY_TI_GMII_SEL depends on TI_K3_AM65_CPTS || !TI_K3_AM65_CPTS help @@ -187,6 +191,7 @@ config TI_ICSSG_PRUETH tristate "TI Gigabit PRU Ethernet driver" select PHYLIB select TI_ICSS_IEP + select TI_K3_CPPI_DESC_POOL depends on PRU_REMOTEPROC depends on ARCH_K3 && OF && TI_K3_UDMA_GLUE_LAYER help @@ -199,6 +204,7 @@ config TI_ICSSG_PRUETH config TI_ICSS_IEP tristate "TI PRU ICSS IEP driver" + depends on PTP_1588_CLOCK_OPTIONAL depends on TI_PRUSS default TI_PRUSS help diff --git a/drivers/net/ethernet/ti/Makefile b/drivers/net/ethernet/ti/Makefile index 34fd7a716ba6..67bed861f31d 100644 --- a/drivers/net/ethernet/ti/Makefile +++ b/drivers/net/ethernet/ti/Makefile @@ -24,14 +24,15 @@ keystone_netcp-y := netcp_core.o cpsw_ale.o obj-$(CONFIG_TI_KEYSTONE_NETCP_ETHSS) += keystone_netcp_ethss.o keystone_netcp_ethss-y := netcp_ethss.o netcp_sgmii.o netcp_xgbepcsr.o cpsw_ale.o +obj-$(CONFIG_TI_K3_CPPI_DESC_POOL) += k3-cppi-desc-pool.o + obj-$(CONFIG_TI_K3_AM65_CPSW_NUSS) += ti-am65-cpsw-nuss.o -ti-am65-cpsw-nuss-y := am65-cpsw-nuss.o cpsw_sl.o am65-cpsw-ethtool.o cpsw_ale.o k3-cppi-desc-pool.o am65-cpsw-qos.o +ti-am65-cpsw-nuss-y := am65-cpsw-nuss.o cpsw_sl.o am65-cpsw-ethtool.o cpsw_ale.o am65-cpsw-qos.o ti-am65-cpsw-nuss-$(CONFIG_TI_K3_AM65_CPSW_SWITCHDEV) += am65-cpsw-switchdev.o obj-$(CONFIG_TI_K3_AM65_CPTS) += am65-cpts.o obj-$(CONFIG_TI_ICSSG_PRUETH) += icssg-prueth.o -icssg-prueth-y := k3-cppi-desc-pool.o \ - icssg/icssg_prueth.o \ +icssg-prueth-y := icssg/icssg_prueth.o \ icssg/icssg_classifier.o \ icssg/icssg_queues.o \ icssg/icssg_config.o \ diff --git a/drivers/net/ethernet/ti/am65-cpsw-nuss.c b/drivers/net/ethernet/ti/am65-cpsw-nuss.c index bea6fc0f324c..24120605502f 100644 --- a/drivers/net/ethernet/ti/am65-cpsw-nuss.c +++ b/drivers/net/ethernet/ti/am65-cpsw-nuss.c @@ -1747,9 +1747,10 @@ static int am65_cpsw_nuss_init_tx_chns(struct am65_cpsw_common *common) } tx_chn->irq = k3_udma_glue_tx_get_irq(tx_chn->tx_chn); - if (tx_chn->irq <= 0) { + if (tx_chn->irq < 0) { dev_err(dev, "Failed to get tx dma irq %d\n", tx_chn->irq); + ret = tx_chn->irq; goto err; } diff --git a/drivers/net/ethernet/ti/icssg/icssg_config.c b/drivers/net/ethernet/ti/icssg/icssg_config.c index 933b84666574..b272361e378f 100644 --- a/drivers/net/ethernet/ti/icssg/icssg_config.c +++ b/drivers/net/ethernet/ti/icssg/icssg_config.c @@ -379,9 +379,9 @@ int icssg_config(struct prueth *prueth, struct prueth_emac *emac, int slice) /* Bitmask for ICSSG r30 commands */ static const struct icssg_r30_cmd emac_r32_bitmask[] = { - {{0xffff0004, 0xffff0100, 0xffff0100, EMAC_NONE}}, /* EMAC_PORT_DISABLE */ + {{0xffff0004, 0xffff0100, 0xffff0004, EMAC_NONE}}, /* EMAC_PORT_DISABLE */ {{0xfffb0040, 0xfeff0200, 0xfeff0200, EMAC_NONE}}, /* EMAC_PORT_BLOCK */ - {{0xffbb0000, 0xfcff0000, 0xdcff0000, EMAC_NONE}}, /* EMAC_PORT_FORWARD */ + {{0xffbb0000, 0xfcff0000, 0xdcfb0000, EMAC_NONE}}, /* EMAC_PORT_FORWARD */ {{0xffbb0000, 0xfcff0000, 0xfcff2000, EMAC_NONE}}, /* EMAC_PORT_FORWARD_WO_LEARNING */ {{0xffff0001, EMAC_NONE, EMAC_NONE, EMAC_NONE}}, /* ACCEPT ALL */ {{0xfffe0002, EMAC_NONE, EMAC_NONE, EMAC_NONE}}, /* ACCEPT TAGGED */ diff --git a/drivers/net/ethernet/ti/icssg/icssg_prueth.c b/drivers/net/ethernet/ti/icssg/icssg_prueth.c index 410612f43cbd..4914d0ef58e9 100644 --- a/drivers/net/ethernet/ti/icssg/icssg_prueth.c +++ b/drivers/net/ethernet/ti/icssg/icssg_prueth.c @@ -316,12 +316,12 @@ static int prueth_init_tx_chns(struct prueth_emac *emac) goto fail; } - tx_chn->irq = k3_udma_glue_tx_get_irq(tx_chn->tx_chn); - if (tx_chn->irq <= 0) { - ret = -EINVAL; + ret = k3_udma_glue_tx_get_irq(tx_chn->tx_chn); + if (ret < 0) { netdev_err(ndev, "failed to get tx irq\n"); goto fail; } + tx_chn->irq = ret; snprintf(tx_chn->name, sizeof(tx_chn->name), "%s-tx%d", dev_name(dev), tx_chn->id); diff --git a/drivers/net/ethernet/ti/icssg/icssg_stats.c b/drivers/net/ethernet/ti/icssg/icssg_stats.c index bb0b33927e3b..3dbadddd7e35 100644 --- a/drivers/net/ethernet/ti/icssg/icssg_stats.c +++ b/drivers/net/ethernet/ti/icssg/icssg_stats.c @@ -9,6 +9,9 @@ #include "icssg_stats.h" #include +#define ICSSG_TX_PACKET_OFFSET 0xA0 +#define ICSSG_TX_BYTE_OFFSET 0xEC + static u32 stats_base[] = { 0x54c, /* Slice 0 stats start */ 0xb18, /* Slice 1 stats start */ }; @@ -18,6 +21,7 @@ void emac_update_hardware_stats(struct prueth_emac *emac) struct prueth *prueth = emac->prueth; int slice = prueth_emac_slice(emac); u32 base = stats_base[slice]; + u32 tx_pkt_cnt = 0; u32 val; int i; @@ -29,7 +33,12 @@ void emac_update_hardware_stats(struct prueth_emac *emac) base + icssg_all_stats[i].offset, val); + if (icssg_all_stats[i].offset == ICSSG_TX_PACKET_OFFSET) + tx_pkt_cnt = val; + emac->stats[i] += val; + if (icssg_all_stats[i].offset == ICSSG_TX_BYTE_OFFSET) + emac->stats[i] -= tx_pkt_cnt * 8; } } diff --git a/drivers/net/ethernet/ti/k3-cppi-desc-pool.c b/drivers/net/ethernet/ti/k3-cppi-desc-pool.c index 38cc12f9f133..05cc7aab1ec8 100644 --- a/drivers/net/ethernet/ti/k3-cppi-desc-pool.c +++ b/drivers/net/ethernet/ti/k3-cppi-desc-pool.c @@ -39,6 +39,7 @@ void k3_cppi_desc_pool_destroy(struct k3_cppi_desc_pool *pool) gen_pool_destroy(pool->gen_pool); /* frees pool->name */ } +EXPORT_SYMBOL_GPL(k3_cppi_desc_pool_destroy); struct k3_cppi_desc_pool * k3_cppi_desc_pool_create_name(struct device *dev, size_t size, @@ -98,29 +99,38 @@ gen_pool_create_fail: devm_kfree(pool->dev, pool); return ERR_PTR(ret); } +EXPORT_SYMBOL_GPL(k3_cppi_desc_pool_create_name); dma_addr_t k3_cppi_desc_pool_virt2dma(struct k3_cppi_desc_pool *pool, void *addr) { return addr ? pool->dma_addr + (addr - pool->cpumem) : 0; } +EXPORT_SYMBOL_GPL(k3_cppi_desc_pool_virt2dma); void *k3_cppi_desc_pool_dma2virt(struct k3_cppi_desc_pool *pool, dma_addr_t dma) { return dma ? pool->cpumem + (dma - pool->dma_addr) : NULL; } +EXPORT_SYMBOL_GPL(k3_cppi_desc_pool_dma2virt); void *k3_cppi_desc_pool_alloc(struct k3_cppi_desc_pool *pool) { return (void *)gen_pool_alloc(pool->gen_pool, pool->desc_size); } +EXPORT_SYMBOL_GPL(k3_cppi_desc_pool_alloc); void k3_cppi_desc_pool_free(struct k3_cppi_desc_pool *pool, void *addr) { gen_pool_free(pool->gen_pool, (unsigned long)addr, pool->desc_size); } +EXPORT_SYMBOL_GPL(k3_cppi_desc_pool_free); size_t k3_cppi_desc_pool_avail(struct k3_cppi_desc_pool *pool) { return gen_pool_avail(pool->gen_pool) / pool->desc_size; } +EXPORT_SYMBOL_GPL(k3_cppi_desc_pool_avail); + +MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("TI K3 CPPI5 descriptors pool API"); diff --git a/drivers/net/ieee802154/ca8210.c b/drivers/net/ieee802154/ca8210.c index aebb19f1b3a4..4ec0dab38872 100644 --- a/drivers/net/ieee802154/ca8210.c +++ b/drivers/net/ieee802154/ca8210.c @@ -2740,7 +2740,6 @@ static int ca8210_register_ext_clock(struct spi_device *spi) struct device_node *np = spi->dev.of_node; struct ca8210_priv *priv = spi_get_drvdata(spi); struct ca8210_platform_data *pdata = spi->dev.platform_data; - int ret = 0; if (!np) return -EFAULT; @@ -2757,18 +2756,8 @@ static int ca8210_register_ext_clock(struct spi_device *spi) dev_crit(&spi->dev, "Failed to register external clk\n"); return PTR_ERR(priv->clk); } - ret = of_clk_add_provider(np, of_clk_src_simple_get, priv->clk); - if (ret) { - clk_unregister(priv->clk); - dev_crit( - &spi->dev, - "Failed to register external clock as clock provider\n" - ); - } else { - dev_info(&spi->dev, "External clock set as clock provider\n"); - } - return ret; + return of_clk_add_provider(np, of_clk_src_simple_get, priv->clk); } /** @@ -2780,8 +2769,8 @@ static void ca8210_unregister_ext_clock(struct spi_device *spi) { struct ca8210_priv *priv = spi_get_drvdata(spi); - if (!priv->clk) - return + if (IS_ERR_OR_NULL(priv->clk)) + return; of_clk_del_provider(spi->dev.of_node); clk_unregister(priv->clk); diff --git a/drivers/net/macsec.c b/drivers/net/macsec.c index b7e151439c48..c5cd4551c67c 100644 --- a/drivers/net/macsec.c +++ b/drivers/net/macsec.c @@ -2383,6 +2383,7 @@ static int macsec_upd_txsa(struct sk_buff *skb, struct genl_info *info) ctx.sa.assoc_num = assoc_num; ctx.sa.tx_sa = tx_sa; + ctx.sa.update_pn = !!prev_pn.full64; ctx.secy = secy; ret = macsec_offload(ops->mdo_upd_txsa, &ctx); @@ -2476,6 +2477,7 @@ static int macsec_upd_rxsa(struct sk_buff *skb, struct genl_info *info) ctx.sa.assoc_num = assoc_num; ctx.sa.rx_sa = rx_sa; + ctx.sa.update_pn = !!prev_pn.full64; ctx.secy = secy; ret = macsec_offload(ops->mdo_upd_rxsa, &ctx); diff --git a/drivers/net/mdio/mdio-mux.c b/drivers/net/mdio/mdio-mux.c index a881e3523328..bef4cce71287 100644 --- a/drivers/net/mdio/mdio-mux.c +++ b/drivers/net/mdio/mdio-mux.c @@ -55,6 +55,27 @@ out: return r; } +static int mdio_mux_read_c45(struct mii_bus *bus, int phy_id, int dev_addr, + int regnum) +{ + struct mdio_mux_child_bus *cb = bus->priv; + struct mdio_mux_parent_bus *pb = cb->parent; + int r; + + mutex_lock_nested(&pb->mii_bus->mdio_lock, MDIO_MUTEX_MUX); + r = pb->switch_fn(pb->current_child, cb->bus_number, pb->switch_data); + if (r) + goto out; + + pb->current_child = cb->bus_number; + + r = pb->mii_bus->read_c45(pb->mii_bus, phy_id, dev_addr, regnum); +out: + mutex_unlock(&pb->mii_bus->mdio_lock); + + return r; +} + /* * The parent bus' lock is used to order access to the switch_fn. */ @@ -80,6 +101,28 @@ out: return r; } +static int mdio_mux_write_c45(struct mii_bus *bus, int phy_id, int dev_addr, + int regnum, u16 val) +{ + struct mdio_mux_child_bus *cb = bus->priv; + struct mdio_mux_parent_bus *pb = cb->parent; + + int r; + + mutex_lock_nested(&pb->mii_bus->mdio_lock, MDIO_MUTEX_MUX); + r = pb->switch_fn(pb->current_child, cb->bus_number, pb->switch_data); + if (r) + goto out; + + pb->current_child = cb->bus_number; + + r = pb->mii_bus->write_c45(pb->mii_bus, phy_id, dev_addr, regnum, val); +out: + mutex_unlock(&pb->mii_bus->mdio_lock); + + return r; +} + static int parent_count; static void mdio_mux_uninit_children(struct mdio_mux_parent_bus *pb) @@ -173,6 +216,10 @@ int mdio_mux_init(struct device *dev, cb->mii_bus->parent = dev; cb->mii_bus->read = mdio_mux_read; cb->mii_bus->write = mdio_mux_write; + if (parent_bus->read_c45) + cb->mii_bus->read_c45 = mdio_mux_read_c45; + if (parent_bus->write_c45) + cb->mii_bus->write_c45 = mdio_mux_write_c45; r = of_mdiobus_register(cb->mii_bus, child_bus_node); if (r) { mdiobus_free(cb->mii_bus); diff --git a/drivers/net/phy/bcm7xxx.c b/drivers/net/phy/bcm7xxx.c index 8478b081c058..97638ba7ae85 100644 --- a/drivers/net/phy/bcm7xxx.c +++ b/drivers/net/phy/bcm7xxx.c @@ -894,6 +894,9 @@ static int bcm7xxx_28nm_probe(struct phy_device *phydev) .name = _name, \ /* PHY_BASIC_FEATURES */ \ .flags = PHY_IS_INTERNAL, \ + .get_sset_count = bcm_phy_get_sset_count, \ + .get_strings = bcm_phy_get_strings, \ + .get_stats = bcm7xxx_28nm_get_phy_stats, \ .probe = bcm7xxx_28nm_probe, \ .config_init = bcm7xxx_16nm_ephy_config_init, \ .config_aneg = genphy_config_aneg, \ diff --git a/drivers/net/phy/mscc/mscc_macsec.c b/drivers/net/phy/mscc/mscc_macsec.c index 018253a573b8..4f39ba63a9a9 100644 --- a/drivers/net/phy/mscc/mscc_macsec.c +++ b/drivers/net/phy/mscc/mscc_macsec.c @@ -849,6 +849,9 @@ static int vsc8584_macsec_upd_rxsa(struct macsec_context *ctx) struct macsec_flow *flow; int ret; + if (ctx->sa.update_pn) + return -EINVAL; + flow = vsc8584_macsec_find_flow(ctx, MACSEC_INGR); if (IS_ERR(flow)) return PTR_ERR(flow); @@ -900,6 +903,9 @@ static int vsc8584_macsec_upd_txsa(struct macsec_context *ctx) struct macsec_flow *flow; int ret; + if (ctx->sa.update_pn) + return -EINVAL; + flow = vsc8584_macsec_find_flow(ctx, MACSEC_EGR); if (IS_ERR(flow)) return PTR_ERR(flow); diff --git a/drivers/net/team/team.c b/drivers/net/team/team.c index e8b94580194e..508d9a392ab1 100644 --- a/drivers/net/team/team.c +++ b/drivers/net/team/team.c @@ -2115,7 +2115,12 @@ static const struct ethtool_ops team_ethtool_ops = { static void team_setup_by_port(struct net_device *dev, struct net_device *port_dev) { - dev->header_ops = port_dev->header_ops; + struct team *team = netdev_priv(dev); + + if (port_dev->type == ARPHRD_ETHER) + dev->header_ops = team->header_ops_cache; + else + dev->header_ops = port_dev->header_ops; dev->type = port_dev->type; dev->hard_header_len = port_dev->hard_header_len; dev->needed_headroom = port_dev->needed_headroom; @@ -2162,8 +2167,11 @@ static int team_dev_type_check_change(struct net_device *dev, static void team_setup(struct net_device *dev) { + struct team *team = netdev_priv(dev); + ether_setup(dev); dev->max_mtu = ETH_MAX_MTU; + team->header_ops_cache = dev->header_ops; dev->netdev_ops = &team_netdev_ops; dev->ethtool_ops = &team_ethtool_ops; diff --git a/drivers/net/thunderbolt/main.c b/drivers/net/thunderbolt/main.c index 0c1e8970ee58..0a53ec293d04 100644 --- a/drivers/net/thunderbolt/main.c +++ b/drivers/net/thunderbolt/main.c @@ -1049,12 +1049,11 @@ static bool tbnet_xmit_csum_and_map(struct tbnet *net, struct sk_buff *skb, *tucso = ~csum_tcpudp_magic(ip_hdr(skb)->saddr, ip_hdr(skb)->daddr, 0, ip_hdr(skb)->protocol, 0); - } else if (skb_is_gso_v6(skb)) { + } else if (skb_is_gso(skb) && skb_is_gso_v6(skb)) { tucso = dest + ((void *)&(tcp_hdr(skb)->check) - data); *tucso = ~csum_ipv6_magic(&ipv6_hdr(skb)->saddr, &ipv6_hdr(skb)->daddr, 0, IPPROTO_TCP, 0); - return false; } else if (protocol == htons(ETH_P_IPV6)) { tucso = dest + skb_checksum_start_offset(skb) + skb->csum_offset; *tucso = ~csum_ipv6_magic(&ipv6_hdr(skb)->saddr, diff --git a/drivers/net/tun.c b/drivers/net/tun.c index 89ab9efe522c..afa5497f7c35 100644 --- a/drivers/net/tun.c +++ b/drivers/net/tun.c @@ -3073,10 +3073,11 @@ static long __tun_chr_ioctl(struct file *file, unsigned int cmd, struct net *net = sock_net(&tfile->sk); struct tun_struct *tun; void __user* argp = (void __user*)arg; - unsigned int ifindex, carrier; + unsigned int carrier; struct ifreq ifr; kuid_t owner; kgid_t group; + int ifindex; int sndbuf; int vnet_hdr_sz; int le; @@ -3132,7 +3133,9 @@ static long __tun_chr_ioctl(struct file *file, unsigned int cmd, ret = -EFAULT; if (copy_from_user(&ifindex, argp, sizeof(ifindex))) goto unlock; - + ret = -EINVAL; + if (ifindex < 0) + goto unlock; ret = 0; tfile->ifindex = ifindex; goto unlock; diff --git a/drivers/net/usb/dm9601.c b/drivers/net/usb/dm9601.c index 48d7d278631e..99ec1d4a972d 100644 --- a/drivers/net/usb/dm9601.c +++ b/drivers/net/usb/dm9601.c @@ -222,13 +222,18 @@ static int dm9601_mdio_read(struct net_device *netdev, int phy_id, int loc) struct usbnet *dev = netdev_priv(netdev); __le16 res; + int err; if (phy_id) { netdev_dbg(dev->net, "Only internal phy supported\n"); return 0; } - dm_read_shared_word(dev, 1, loc, &res); + err = dm_read_shared_word(dev, 1, loc, &res); + if (err < 0) { + netdev_err(dev->net, "MDIO read error: %d\n", err); + return err; + } netdev_dbg(dev->net, "dm9601_mdio_read() phy_id=0x%02x, loc=0x%02x, returns=0x%04x\n", diff --git a/drivers/net/usb/r8152.c b/drivers/net/usb/r8152.c index 332c853ca99b..0c13d9950cd8 100644 --- a/drivers/net/usb/r8152.c +++ b/drivers/net/usb/r8152.c @@ -2636,6 +2636,9 @@ static int r8152_poll(struct napi_struct *napi, int budget) struct r8152 *tp = container_of(napi, struct r8152, napi); int work_done; + if (!budget) + return 0; + work_done = rx_bottom(tp, budget); if (work_done < budget) { diff --git a/drivers/net/usb/smsc75xx.c b/drivers/net/usb/smsc75xx.c index 5d6454fedb3f..78ad2da3ee29 100644 --- a/drivers/net/usb/smsc75xx.c +++ b/drivers/net/usb/smsc75xx.c @@ -90,7 +90,9 @@ static int __must_check __smsc75xx_read_reg(struct usbnet *dev, u32 index, ret = fn(dev, USB_VENDOR_REQUEST_READ_REGISTER, USB_DIR_IN | USB_TYPE_VENDOR | USB_RECIP_DEVICE, 0, index, &buf, 4); - if (unlikely(ret < 0)) { + if (unlikely(ret < 4)) { + ret = ret < 0 ? ret : -ENODATA; + netdev_warn(dev->net, "Failed to read reg index 0x%08x: %d\n", index, ret); return ret; diff --git a/drivers/net/usb/smsc95xx.c b/drivers/net/usb/smsc95xx.c index 563ecd27b93e..17da42fe605c 100644 --- a/drivers/net/usb/smsc95xx.c +++ b/drivers/net/usb/smsc95xx.c @@ -897,7 +897,7 @@ static int smsc95xx_reset(struct usbnet *dev) if (timeout >= 100) { netdev_warn(dev->net, "timeout waiting for completion of Lite Reset\n"); - return ret; + return -ETIMEDOUT; } ret = smsc95xx_set_mac_address(dev); diff --git a/drivers/net/veth.c b/drivers/net/veth.c index 9c6f4f83f22b..0deefd1573cf 100644 --- a/drivers/net/veth.c +++ b/drivers/net/veth.c @@ -1446,6 +1446,8 @@ static int veth_open(struct net_device *dev) netif_carrier_on(peer); } + veth_set_xdp_features(dev); + return 0; } diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index fe7f314d65c9..d67f742fbd4c 100644 --- a/drivers/net/virtio_net.c +++ b/drivers/net/virtio_net.c @@ -607,16 +607,16 @@ static void virtnet_rq_unmap(struct receive_queue *rq, void *buf, u32 len) --dma->ref; - if (dma->ref) { - if (dma->need_sync && len) { - offset = buf - (head + sizeof(*dma)); + if (dma->need_sync && len) { + offset = buf - (head + sizeof(*dma)); - virtqueue_dma_sync_single_range_for_cpu(rq->vq, dma->addr, offset, - len, DMA_FROM_DEVICE); - } + virtqueue_dma_sync_single_range_for_cpu(rq->vq, dma->addr, + offset, len, + DMA_FROM_DEVICE); + } + if (dma->ref) return; - } virtqueue_dma_unmap_single_attrs(rq->vq, dma->addr, dma->len, DMA_FROM_DEVICE, DMA_ATTR_SKIP_CPU_SYNC); diff --git a/drivers/net/vxlan/vxlan_core.c b/drivers/net/vxlan/vxlan_core.c index e463f59e95c2..5b5597073b00 100644 --- a/drivers/net/vxlan/vxlan_core.c +++ b/drivers/net/vxlan/vxlan_core.c @@ -4331,6 +4331,10 @@ static size_t vxlan_get_size(const struct net_device *dev) nla_total_size(sizeof(__u8)) + /* IFLA_VXLAN_REMCSUM_TX */ nla_total_size(sizeof(__u8)) + /* IFLA_VXLAN_REMCSUM_RX */ nla_total_size(sizeof(__u8)) + /* IFLA_VXLAN_LOCALBYPASS */ + nla_total_size(0) + /* IFLA_VXLAN_GBP */ + nla_total_size(0) + /* IFLA_VXLAN_GPE */ + nla_total_size(0) + /* IFLA_VXLAN_REMCSUM_NOPARTIAL */ + nla_total_size(sizeof(__u8)) + /* IFLA_VXLAN_VNIFILTER */ 0; } diff --git a/drivers/net/wan/fsl_ucc_hdlc.c b/drivers/net/wan/fsl_ucc_hdlc.c index 47c2ad7a3e42..fd50bb313b92 100644 --- a/drivers/net/wan/fsl_ucc_hdlc.c +++ b/drivers/net/wan/fsl_ucc_hdlc.c @@ -34,6 +34,8 @@ #define TDM_PPPOHT_SLIC_MAXIN #define RX_BD_ERRORS (R_CD_S | R_OV_S | R_CR_S | R_AB_S | R_NO_S | R_LG_S) +static int uhdlc_close(struct net_device *dev); + static struct ucc_tdm_info utdm_primary_info = { .uf_info = { .tsa = 0, @@ -708,6 +710,7 @@ static int uhdlc_open(struct net_device *dev) hdlc_device *hdlc = dev_to_hdlc(dev); struct ucc_hdlc_private *priv = hdlc->priv; struct ucc_tdm *utdm = priv->utdm; + int rc = 0; if (priv->hdlc_busy != 1) { if (request_irq(priv->ut_info->uf_info.irq, @@ -731,10 +734,13 @@ static int uhdlc_open(struct net_device *dev) napi_enable(&priv->napi); netdev_reset_queue(dev); netif_start_queue(dev); - hdlc_open(dev); + + rc = hdlc_open(dev); + if (rc) + uhdlc_close(dev); } - return 0; + return rc; } static void uhdlc_memclean(struct ucc_hdlc_private *priv) @@ -824,6 +830,8 @@ static int uhdlc_close(struct net_device *dev) netdev_reset_queue(dev); priv->hdlc_busy = 0; + hdlc_close(dev); + return 0; } diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/fwil_types.h b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/fwil_types.h index bece26741d3a..611d1a6aabb9 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/fwil_types.h +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/fwil_types.h @@ -442,7 +442,12 @@ struct brcmf_scan_params_v2_le { * fixed parameter portion is assumed, otherwise * ssid in the fixed portion is ignored */ - __le16 channel_list[1]; /* list of chanspecs */ + union { + __le16 padding; /* Reserve space for at least 1 entry for abort + * which uses an on stack brcmf_scan_params_v2_le + */ + DECLARE_FLEX_ARRAY(__le16, channel_list); /* chanspecs */ + }; }; struct brcmf_scan_results { @@ -702,7 +707,7 @@ struct brcmf_sta_info_le { struct brcmf_chanspec_list { __le32 count; /* # of entries */ - __le32 element[1]; /* variable length uint32 list */ + __le32 element[]; /* variable length uint32 list */ }; /* diff --git a/drivers/net/wireless/intel/iwlwifi/fw/error-dump.h b/drivers/net/wireless/intel/iwlwifi/fw/error-dump.h index f5e08988dc7b..06d6f7f66430 100644 --- a/drivers/net/wireless/intel/iwlwifi/fw/error-dump.h +++ b/drivers/net/wireless/intel/iwlwifi/fw/error-dump.h @@ -310,9 +310,9 @@ struct iwl_fw_ini_fifo_hdr { struct iwl_fw_ini_error_dump_range { __le32 range_data_size; union { - __le32 internal_base_addr; - __le64 dram_base_addr; - __le32 page_num; + __le32 internal_base_addr __packed; + __le64 dram_base_addr __packed; + __le32 page_num __packed; struct iwl_fw_ini_fifo_hdr fifo_hdr; struct iwl_cmd_header fw_pkt_hdr; }; diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/fw.c b/drivers/net/wireless/intel/iwlwifi/mvm/fw.c index 1f5db65a088d..1d5ee4330f29 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/fw.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/fw.c @@ -802,7 +802,7 @@ out: mvm->nvm_data->bands[0].n_channels = 1; mvm->nvm_data->bands[0].n_bitrates = 1; mvm->nvm_data->bands[0].bitrates = - (void *)((u8 *)mvm->nvm_data->channels + 1); + (void *)(mvm->nvm_data->channels + 1); mvm->nvm_data->bands[0].bitrates->hw_value = 10; } diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/mld-mac80211.c b/drivers/net/wireless/intel/iwlwifi/mvm/mld-mac80211.c index 8b6c641772ee..b719843e9457 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/mld-mac80211.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/mld-mac80211.c @@ -731,73 +731,78 @@ static void iwl_mvm_mld_vif_cfg_changed_station(struct iwl_mvm *mvm, mvmvif->associated = vif->cfg.assoc; - if (!(changes & BSS_CHANGED_ASSOC)) - return; - - if (vif->cfg.assoc) { - /* clear statistics to get clean beacon counter */ - iwl_mvm_request_statistics(mvm, true); - iwl_mvm_sf_update(mvm, vif, false); - iwl_mvm_power_vif_assoc(mvm, vif); - - for_each_mvm_vif_valid_link(mvmvif, i) { - memset(&mvmvif->link[i]->beacon_stats, 0, - sizeof(mvmvif->link[i]->beacon_stats)); + if (changes & BSS_CHANGED_ASSOC) { + if (vif->cfg.assoc) { + /* clear statistics to get clean beacon counter */ + iwl_mvm_request_statistics(mvm, true); + iwl_mvm_sf_update(mvm, vif, false); + iwl_mvm_power_vif_assoc(mvm, vif); + + for_each_mvm_vif_valid_link(mvmvif, i) { + memset(&mvmvif->link[i]->beacon_stats, 0, + sizeof(mvmvif->link[i]->beacon_stats)); + + if (vif->p2p) { + iwl_mvm_update_smps(mvm, vif, + IWL_MVM_SMPS_REQ_PROT, + IEEE80211_SMPS_DYNAMIC, i); + } + + rcu_read_lock(); + link_conf = rcu_dereference(vif->link_conf[i]); + if (link_conf && !link_conf->dtim_period) + protect = true; + rcu_read_unlock(); + } - if (vif->p2p) { - iwl_mvm_update_smps(mvm, vif, - IWL_MVM_SMPS_REQ_PROT, - IEEE80211_SMPS_DYNAMIC, i); + if (!test_bit(IWL_MVM_STATUS_IN_HW_RESTART, &mvm->status) && + protect) { + /* If we're not restarting and still haven't + * heard a beacon (dtim period unknown) then + * make sure we still have enough minimum time + * remaining in the time event, since the auth + * might actually have taken quite a while + * (especially for SAE) and so the remaining + * time could be small without us having heard + * a beacon yet. + */ + iwl_mvm_protect_assoc(mvm, vif, 0); } - rcu_read_lock(); - link_conf = rcu_dereference(vif->link_conf[i]); - if (link_conf && !link_conf->dtim_period) - protect = true; - rcu_read_unlock(); - } + iwl_mvm_sf_update(mvm, vif, false); + + /* FIXME: need to decide about misbehaving AP handling */ + iwl_mvm_power_vif_assoc(mvm, vif); + } else if (iwl_mvm_mld_vif_have_valid_ap_sta(mvmvif)) { + iwl_mvm_mei_host_disassociated(mvm); - if (!test_bit(IWL_MVM_STATUS_IN_HW_RESTART, &mvm->status) && - protect) { - /* If we're not restarting and still haven't - * heard a beacon (dtim period unknown) then - * make sure we still have enough minimum time - * remaining in the time event, since the auth - * might actually have taken quite a while - * (especially for SAE) and so the remaining - * time could be small without us having heard - * a beacon yet. + /* If update fails - SF might be running in associated + * mode while disassociated - which is forbidden. */ - iwl_mvm_protect_assoc(mvm, vif, 0); + ret = iwl_mvm_sf_update(mvm, vif, false); + WARN_ONCE(ret && + !test_bit(IWL_MVM_STATUS_HW_RESTART_REQUESTED, + &mvm->status), + "Failed to update SF upon disassociation\n"); + + /* If we get an assert during the connection (after the + * station has been added, but before the vif is set + * to associated), mac80211 will re-add the station and + * then configure the vif. Since the vif is not + * associated, we would remove the station here and + * this would fail the recovery. + */ + iwl_mvm_mld_vif_delete_all_stas(mvm, vif); } - iwl_mvm_sf_update(mvm, vif, false); - - /* FIXME: need to decide about misbehaving AP handling */ - iwl_mvm_power_vif_assoc(mvm, vif); - } else if (iwl_mvm_mld_vif_have_valid_ap_sta(mvmvif)) { - iwl_mvm_mei_host_disassociated(mvm); - - /* If update fails - SF might be running in associated - * mode while disassociated - which is forbidden. - */ - ret = iwl_mvm_sf_update(mvm, vif, false); - WARN_ONCE(ret && - !test_bit(IWL_MVM_STATUS_HW_RESTART_REQUESTED, - &mvm->status), - "Failed to update SF upon disassociation\n"); - - /* If we get an assert during the connection (after the - * station has been added, but before the vif is set - * to associated), mac80211 will re-add the station and - * then configure the vif. Since the vif is not - * associated, we would remove the station here and - * this would fail the recovery. - */ - iwl_mvm_mld_vif_delete_all_stas(mvm, vif); + iwl_mvm_bss_info_changed_station_assoc(mvm, vif, changes); } - iwl_mvm_bss_info_changed_station_assoc(mvm, vif, changes); + if (changes & BSS_CHANGED_PS) { + ret = iwl_mvm_power_update_mac(mvm); + if (ret) + IWL_ERR(mvm, "failed to update power mode\n"); + } } static void diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/scan.c b/drivers/net/wireless/intel/iwlwifi/mvm/scan.c index c1d9ce753468..3cbe2c0b8d6b 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/scan.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/scan.c @@ -2342,7 +2342,7 @@ iwl_mvm_scan_umac_fill_general_p_v12(struct iwl_mvm *mvm, if (gen_flags & IWL_UMAC_SCAN_GEN_FLAGS_V2_FRAGMENTED_LMAC2) gp->num_of_fragments[SCAN_HB_LMAC_IDX] = IWL_SCAN_NUM_OF_FRAGS; - if (version < 12) { + if (version < 16) { gp->scan_start_mac_or_link_id = scan_vif->id; } else { struct iwl_mvm_vif_link_info *link_info; diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/tx.c b/drivers/net/wireless/intel/iwlwifi/mvm/tx.c index 36d70d589aed..898dca393643 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/tx.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/tx.c @@ -1612,6 +1612,7 @@ static void iwl_mvm_rx_tx_cmd_single(struct iwl_mvm *mvm, iwl_trans_free_tx_cmd(mvm->trans, info->driver_data[1]); memset(&info->status, 0, sizeof(info->status)); + info->flags &= ~(IEEE80211_TX_STAT_ACK | IEEE80211_TX_STAT_TX_FILTERED); /* inform mac80211 about what happened with the frame */ switch (status & TX_STATUS_MSK) { @@ -1964,6 +1965,8 @@ static void iwl_mvm_tx_reclaim(struct iwl_mvm *mvm, int sta_id, int tid, */ if (!is_flush) info->flags |= IEEE80211_TX_STAT_ACK; + else + info->flags &= ~IEEE80211_TX_STAT_ACK; } /* diff --git a/drivers/net/wireless/marvell/mwifiex/11n_rxreorder.c b/drivers/net/wireless/marvell/mwifiex/11n_rxreorder.c index 391793a16adc..10690e82358b 100644 --- a/drivers/net/wireless/marvell/mwifiex/11n_rxreorder.c +++ b/drivers/net/wireless/marvell/mwifiex/11n_rxreorder.c @@ -918,9 +918,17 @@ void mwifiex_11n_rxba_sync_event(struct mwifiex_private *priv, mwifiex_dbg_dump(priv->adapter, EVT_D, "RXBA_SYNC event:", event_buf, len); - while (tlv_buf_left >= sizeof(*tlv_rxba)) { + while (tlv_buf_left > sizeof(*tlv_rxba)) { tlv_type = le16_to_cpu(tlv_rxba->header.type); tlv_len = le16_to_cpu(tlv_rxba->header.len); + if (size_add(sizeof(tlv_rxba->header), tlv_len) > tlv_buf_left) { + mwifiex_dbg(priv->adapter, WARN, + "TLV size (%zu) overflows event_buf buf_left=%d\n", + size_add(sizeof(tlv_rxba->header), tlv_len), + tlv_buf_left); + return; + } + if (tlv_type != TLV_TYPE_RXBA_SYNC) { mwifiex_dbg(priv->adapter, ERROR, "Wrong TLV id=0x%x\n", tlv_type); @@ -929,6 +937,14 @@ void mwifiex_11n_rxba_sync_event(struct mwifiex_private *priv, tlv_seq_num = le16_to_cpu(tlv_rxba->seq_num); tlv_bitmap_len = le16_to_cpu(tlv_rxba->bitmap_len); + if (size_add(sizeof(*tlv_rxba), tlv_bitmap_len) > tlv_buf_left) { + mwifiex_dbg(priv->adapter, WARN, + "TLV size (%zu) overflows event_buf buf_left=%d\n", + size_add(sizeof(*tlv_rxba), tlv_bitmap_len), + tlv_buf_left); + return; + } + mwifiex_dbg(priv->adapter, INFO, "%pM tid=%d seq_num=%d bitmap_len=%d\n", tlv_rxba->mac, tlv_rxba->tid, tlv_seq_num, @@ -965,8 +981,8 @@ void mwifiex_11n_rxba_sync_event(struct mwifiex_private *priv, } } - tlv_buf_left -= (sizeof(*tlv_rxba) + tlv_len); - tmp = (u8 *)tlv_rxba + tlv_len + sizeof(*tlv_rxba); + tlv_buf_left -= (sizeof(tlv_rxba->header) + tlv_len); + tmp = (u8 *)tlv_rxba + sizeof(tlv_rxba->header) + tlv_len; tlv_rxba = (struct mwifiex_ie_types_rxba_sync *)tmp; } } diff --git a/drivers/net/wireless/marvell/mwifiex/fw.h b/drivers/net/wireless/marvell/mwifiex/fw.h index f2168fac95ed..8e6db904e5b2 100644 --- a/drivers/net/wireless/marvell/mwifiex/fw.h +++ b/drivers/net/wireless/marvell/mwifiex/fw.h @@ -779,7 +779,7 @@ struct mwifiex_ie_types_rxba_sync { u8 reserved; __le16 seq_num; __le16 bitmap_len; - u8 bitmap[1]; + u8 bitmap[]; } __packed; struct chan_band_param_set { diff --git a/drivers/net/wireless/marvell/mwifiex/sta_rx.c b/drivers/net/wireless/marvell/mwifiex/sta_rx.c index 65420ad67416..257737137cd7 100644 --- a/drivers/net/wireless/marvell/mwifiex/sta_rx.c +++ b/drivers/net/wireless/marvell/mwifiex/sta_rx.c @@ -86,7 +86,8 @@ int mwifiex_process_rx_packet(struct mwifiex_private *priv, rx_pkt_len = le16_to_cpu(local_rx_pd->rx_pkt_length); rx_pkt_hdr = (void *)local_rx_pd + rx_pkt_off; - if (sizeof(*rx_pkt_hdr) + rx_pkt_off > skb->len) { + if (sizeof(rx_pkt_hdr->eth803_hdr) + sizeof(rfc1042_header) + + rx_pkt_off > skb->len) { mwifiex_dbg(priv->adapter, ERROR, "wrong rx packet offset: len=%d, rx_pkt_off=%d\n", skb->len, rx_pkt_off); @@ -95,12 +96,13 @@ int mwifiex_process_rx_packet(struct mwifiex_private *priv, return -1; } - if ((!memcmp(&rx_pkt_hdr->rfc1042_hdr, bridge_tunnel_header, - sizeof(bridge_tunnel_header))) || - (!memcmp(&rx_pkt_hdr->rfc1042_hdr, rfc1042_header, - sizeof(rfc1042_header)) && - ntohs(rx_pkt_hdr->rfc1042_hdr.snap_type) != ETH_P_AARP && - ntohs(rx_pkt_hdr->rfc1042_hdr.snap_type) != ETH_P_IPX)) { + if (sizeof(*rx_pkt_hdr) + rx_pkt_off <= skb->len && + ((!memcmp(&rx_pkt_hdr->rfc1042_hdr, bridge_tunnel_header, + sizeof(bridge_tunnel_header))) || + (!memcmp(&rx_pkt_hdr->rfc1042_hdr, rfc1042_header, + sizeof(rfc1042_header)) && + ntohs(rx_pkt_hdr->rfc1042_hdr.snap_type) != ETH_P_AARP && + ntohs(rx_pkt_hdr->rfc1042_hdr.snap_type) != ETH_P_IPX))) { /* * Replace the 803 header and rfc1042 header (llc/snap) with an * EthernetII header, keep the src/dst and snap_type diff --git a/drivers/net/wireless/mediatek/mt76/dma.c b/drivers/net/wireless/mediatek/mt76/dma.c index 05d9ab3ce819..dc8f4e157eb2 100644 --- a/drivers/net/wireless/mediatek/mt76/dma.c +++ b/drivers/net/wireless/mediatek/mt76/dma.c @@ -93,13 +93,13 @@ __mt76_get_rxwi(struct mt76_dev *dev) { struct mt76_txwi_cache *t = NULL; - spin_lock(&dev->wed_lock); + spin_lock_bh(&dev->wed_lock); if (!list_empty(&dev->rxwi_cache)) { t = list_first_entry(&dev->rxwi_cache, struct mt76_txwi_cache, list); list_del(&t->list); } - spin_unlock(&dev->wed_lock); + spin_unlock_bh(&dev->wed_lock); return t; } @@ -145,9 +145,9 @@ mt76_put_rxwi(struct mt76_dev *dev, struct mt76_txwi_cache *t) if (!t) return; - spin_lock(&dev->wed_lock); + spin_lock_bh(&dev->wed_lock); list_add(&t->list, &dev->rxwi_cache); - spin_unlock(&dev->wed_lock); + spin_unlock_bh(&dev->wed_lock); } EXPORT_SYMBOL_GPL(mt76_put_rxwi); diff --git a/drivers/net/wireless/mediatek/mt76/mt76x02_eeprom.c b/drivers/net/wireless/mediatek/mt76/mt76x02_eeprom.c index 0acabba2d1a5..5d402cf2951c 100644 --- a/drivers/net/wireless/mediatek/mt76/mt76x02_eeprom.c +++ b/drivers/net/wireless/mediatek/mt76/mt76x02_eeprom.c @@ -131,15 +131,8 @@ u8 mt76x02_get_lna_gain(struct mt76x02_dev *dev, s8 *lna_2g, s8 *lna_5g, struct ieee80211_channel *chan) { - u16 val; u8 lna; - val = mt76x02_eeprom_get(dev, MT_EE_NIC_CONF_1); - if (val & MT_EE_NIC_CONF_1_LNA_EXT_2G) - *lna_2g = 0; - if (val & MT_EE_NIC_CONF_1_LNA_EXT_5G) - memset(lna_5g, 0, sizeof(s8) * 3); - if (chan->band == NL80211_BAND_2GHZ) lna = *lna_2g; else if (chan->hw_value <= 64) diff --git a/drivers/net/wireless/mediatek/mt76/mt76x2/eeprom.c b/drivers/net/wireless/mediatek/mt76/mt76x2/eeprom.c index d5809408d1d3..8c01855885ce 100644 --- a/drivers/net/wireless/mediatek/mt76/mt76x2/eeprom.c +++ b/drivers/net/wireless/mediatek/mt76/mt76x2/eeprom.c @@ -256,7 +256,8 @@ void mt76x2_read_rx_gain(struct mt76x02_dev *dev) struct ieee80211_channel *chan = dev->mphy.chandef.chan; int channel = chan->hw_value; s8 lna_5g[3], lna_2g; - u8 lna; + bool use_lna; + u8 lna = 0; u16 val; if (chan->band == NL80211_BAND_2GHZ) @@ -275,7 +276,15 @@ void mt76x2_read_rx_gain(struct mt76x02_dev *dev) dev->cal.rx.mcu_gain |= (lna_5g[1] & 0xff) << 16; dev->cal.rx.mcu_gain |= (lna_5g[2] & 0xff) << 24; - lna = mt76x02_get_lna_gain(dev, &lna_2g, lna_5g, chan); + val = mt76x02_eeprom_get(dev, MT_EE_NIC_CONF_1); + if (chan->band == NL80211_BAND_2GHZ) + use_lna = !(val & MT_EE_NIC_CONF_1_LNA_EXT_2G); + else + use_lna = !(val & MT_EE_NIC_CONF_1_LNA_EXT_5G); + + if (use_lna) + lna = mt76x02_get_lna_gain(dev, &lna_2g, lna_5g, chan); + dev->cal.rx.lna_gain = mt76x02_sign_extend(lna, 8); } EXPORT_SYMBOL_GPL(mt76x2_read_rx_gain); diff --git a/drivers/net/wireless/realtek/rtw88/rtw8723d.h b/drivers/net/wireless/realtek/rtw88/rtw8723d.h index 3642a2c7f80c..2434e2480cbe 100644 --- a/drivers/net/wireless/realtek/rtw88/rtw8723d.h +++ b/drivers/net/wireless/realtek/rtw88/rtw8723d.h @@ -46,6 +46,7 @@ struct rtw8723du_efuse { u8 vender_id[2]; /* 0x100 */ u8 product_id[2]; /* 0x102 */ u8 usb_option; /* 0x104 */ + u8 res5[2]; /* 0x105 */ u8 mac_addr[ETH_ALEN]; /* 0x107 */ }; diff --git a/drivers/net/wwan/iosm/iosm_ipc_imem.c b/drivers/net/wwan/iosm/iosm_ipc_imem.c index 635301d677e1..829515a601b3 100644 --- a/drivers/net/wwan/iosm/iosm_ipc_imem.c +++ b/drivers/net/wwan/iosm/iosm_ipc_imem.c @@ -4,7 +4,6 @@ */ #include -#include #include "iosm_ipc_chnl_cfg.h" #include "iosm_ipc_devlink.h" @@ -632,11 +631,6 @@ static void ipc_imem_run_state_worker(struct work_struct *instance) /* Complete all memory stores after setting bit */ smp_mb__after_atomic(); - if (ipc_imem->pcie->pci->device == INTEL_CP_DEVICE_7560_ID) { - pm_runtime_mark_last_busy(ipc_imem->dev); - pm_runtime_put_autosuspend(ipc_imem->dev); - } - return; err_ipc_mux_deinit: @@ -1240,7 +1234,6 @@ void ipc_imem_cleanup(struct iosm_imem *ipc_imem) /* forward MDM_NOT_READY to listeners */ ipc_uevent_send(ipc_imem->dev, UEVENT_MDM_NOT_READY); - pm_runtime_get_sync(ipc_imem->dev); hrtimer_cancel(&ipc_imem->td_alloc_timer); hrtimer_cancel(&ipc_imem->tdupdate_timer); @@ -1426,16 +1419,6 @@ struct iosm_imem *ipc_imem_init(struct iosm_pcie *pcie, unsigned int device_id, set_bit(IOSM_DEVLINK_INIT, &ipc_imem->flag); } - - if (!pm_runtime_enabled(ipc_imem->dev)) - pm_runtime_enable(ipc_imem->dev); - - pm_runtime_set_autosuspend_delay(ipc_imem->dev, - IPC_MEM_AUTO_SUSPEND_DELAY_MS); - pm_runtime_use_autosuspend(ipc_imem->dev); - pm_runtime_allow(ipc_imem->dev); - pm_runtime_mark_last_busy(ipc_imem->dev); - return ipc_imem; devlink_channel_fail: ipc_devlink_deinit(ipc_imem->ipc_devlink); diff --git a/drivers/net/wwan/iosm/iosm_ipc_imem.h b/drivers/net/wwan/iosm/iosm_ipc_imem.h index 0144b45e2afb..5664ac507c90 100644 --- a/drivers/net/wwan/iosm/iosm_ipc_imem.h +++ b/drivers/net/wwan/iosm/iosm_ipc_imem.h @@ -103,8 +103,6 @@ struct ipc_chnl_cfg; #define FULLY_FUNCTIONAL 0 #define IOSM_DEVLINK_INIT 1 -#define IPC_MEM_AUTO_SUSPEND_DELAY_MS 5000 - /* List of the supported UL/DL pipes. */ enum ipc_mem_pipes { IPC_MEM_PIPE_0 = 0, diff --git a/drivers/net/wwan/iosm/iosm_ipc_pcie.c b/drivers/net/wwan/iosm/iosm_ipc_pcie.c index 3a259c9abefd..04517bd3325a 100644 --- a/drivers/net/wwan/iosm/iosm_ipc_pcie.c +++ b/drivers/net/wwan/iosm/iosm_ipc_pcie.c @@ -6,7 +6,6 @@ #include #include #include -#include #include #include "iosm_ipc_imem.h" @@ -438,8 +437,7 @@ static int __maybe_unused ipc_pcie_resume_cb(struct device *dev) return 0; } -static DEFINE_RUNTIME_DEV_PM_OPS(iosm_ipc_pm, ipc_pcie_suspend_cb, - ipc_pcie_resume_cb, NULL); +static SIMPLE_DEV_PM_OPS(iosm_ipc_pm, ipc_pcie_suspend_cb, ipc_pcie_resume_cb); static struct pci_driver iosm_ipc_driver = { .name = KBUILD_MODNAME, diff --git a/drivers/net/wwan/iosm/iosm_ipc_port.c b/drivers/net/wwan/iosm/iosm_ipc_port.c index 2ba1ddca3945..5d5b4183e14a 100644 --- a/drivers/net/wwan/iosm/iosm_ipc_port.c +++ b/drivers/net/wwan/iosm/iosm_ipc_port.c @@ -3,8 +3,6 @@ * Copyright (C) 2020-21 Intel Corporation. */ -#include - #include "iosm_ipc_chnl_cfg.h" #include "iosm_ipc_imem_ops.h" #include "iosm_ipc_port.h" @@ -15,16 +13,12 @@ static int ipc_port_ctrl_start(struct wwan_port *port) struct iosm_cdev *ipc_port = wwan_port_get_drvdata(port); int ret = 0; - pm_runtime_get_sync(ipc_port->ipc_imem->dev); ipc_port->channel = ipc_imem_sys_port_open(ipc_port->ipc_imem, ipc_port->chl_id, IPC_HP_CDEV_OPEN); if (!ipc_port->channel) ret = -EIO; - pm_runtime_mark_last_busy(ipc_port->ipc_imem->dev); - pm_runtime_put_autosuspend(ipc_port->ipc_imem->dev); - return ret; } @@ -33,24 +27,15 @@ static void ipc_port_ctrl_stop(struct wwan_port *port) { struct iosm_cdev *ipc_port = wwan_port_get_drvdata(port); - pm_runtime_get_sync(ipc_port->ipc_imem->dev); ipc_imem_sys_port_close(ipc_port->ipc_imem, ipc_port->channel); - pm_runtime_mark_last_busy(ipc_port->ipc_imem->dev); - pm_runtime_put_autosuspend(ipc_port->ipc_imem->dev); } /* transfer control data to modem */ static int ipc_port_ctrl_tx(struct wwan_port *port, struct sk_buff *skb) { struct iosm_cdev *ipc_port = wwan_port_get_drvdata(port); - int ret; - pm_runtime_get_sync(ipc_port->ipc_imem->dev); - ret = ipc_imem_sys_cdev_write(ipc_port, skb); - pm_runtime_mark_last_busy(ipc_port->ipc_imem->dev); - pm_runtime_put_autosuspend(ipc_port->ipc_imem->dev); - - return ret; + return ipc_imem_sys_cdev_write(ipc_port, skb); } static const struct wwan_port_ops ipc_wwan_ctrl_ops = { diff --git a/drivers/net/wwan/iosm/iosm_ipc_trace.c b/drivers/net/wwan/iosm/iosm_ipc_trace.c index 4368373797b6..eeecfa3d10c5 100644 --- a/drivers/net/wwan/iosm/iosm_ipc_trace.c +++ b/drivers/net/wwan/iosm/iosm_ipc_trace.c @@ -3,9 +3,7 @@ * Copyright (C) 2020-2021 Intel Corporation. */ -#include #include - #include "iosm_ipc_trace.h" /* sub buffer size and number of sub buffer */ @@ -99,8 +97,6 @@ static ssize_t ipc_trace_ctrl_file_write(struct file *filp, if (ret) return ret; - pm_runtime_get_sync(ipc_trace->ipc_imem->dev); - mutex_lock(&ipc_trace->trc_mutex); if (val == TRACE_ENABLE && ipc_trace->mode != TRACE_ENABLE) { ipc_trace->channel = ipc_imem_sys_port_open(ipc_trace->ipc_imem, @@ -121,10 +117,6 @@ static ssize_t ipc_trace_ctrl_file_write(struct file *filp, ret = count; unlock: mutex_unlock(&ipc_trace->trc_mutex); - - pm_runtime_mark_last_busy(ipc_trace->ipc_imem->dev); - pm_runtime_put_autosuspend(ipc_trace->ipc_imem->dev); - return ret; } diff --git a/drivers/net/wwan/iosm/iosm_ipc_wwan.c b/drivers/net/wwan/iosm/iosm_ipc_wwan.c index 93d17de08786..ff747fc79aaf 100644 --- a/drivers/net/wwan/iosm/iosm_ipc_wwan.c +++ b/drivers/net/wwan/iosm/iosm_ipc_wwan.c @@ -6,7 +6,6 @@ #include #include #include -#include #include #include #include @@ -52,13 +51,11 @@ static int ipc_wwan_link_open(struct net_device *netdev) struct iosm_netdev_priv *priv = wwan_netdev_drvpriv(netdev); struct iosm_wwan *ipc_wwan = priv->ipc_wwan; int if_id = priv->if_id; - int ret = 0; if (if_id < IP_MUX_SESSION_START || if_id >= ARRAY_SIZE(ipc_wwan->sub_netlist)) return -EINVAL; - pm_runtime_get_sync(ipc_wwan->ipc_imem->dev); /* get channel id */ priv->ch_id = ipc_imem_sys_wwan_open(ipc_wwan->ipc_imem, if_id); @@ -66,8 +63,7 @@ static int ipc_wwan_link_open(struct net_device *netdev) dev_err(ipc_wwan->dev, "cannot connect wwan0 & id %d to the IPC mem layer", if_id); - ret = -ENODEV; - goto err_out; + return -ENODEV; } /* enable tx path, DL data may follow */ @@ -76,11 +72,7 @@ static int ipc_wwan_link_open(struct net_device *netdev) dev_dbg(ipc_wwan->dev, "Channel id %d allocated to if_id %d", priv->ch_id, priv->if_id); -err_out: - pm_runtime_mark_last_busy(ipc_wwan->ipc_imem->dev); - pm_runtime_put_autosuspend(ipc_wwan->ipc_imem->dev); - - return ret; + return 0; } /* Bring-down the wwan net link */ @@ -90,12 +82,9 @@ static int ipc_wwan_link_stop(struct net_device *netdev) netif_stop_queue(netdev); - pm_runtime_get_sync(priv->ipc_wwan->ipc_imem->dev); ipc_imem_sys_wwan_close(priv->ipc_wwan->ipc_imem, priv->if_id, priv->ch_id); priv->ch_id = -1; - pm_runtime_mark_last_busy(priv->ipc_wwan->ipc_imem->dev); - pm_runtime_put_autosuspend(priv->ipc_wwan->ipc_imem->dev); return 0; } @@ -117,7 +106,6 @@ static netdev_tx_t ipc_wwan_link_transmit(struct sk_buff *skb, if_id >= ARRAY_SIZE(ipc_wwan->sub_netlist)) return -EINVAL; - pm_runtime_get(ipc_wwan->ipc_imem->dev); /* Send the SKB to device for transmission */ ret = ipc_imem_sys_wwan_transmit(ipc_wwan->ipc_imem, if_id, priv->ch_id, skb); @@ -131,14 +119,9 @@ static netdev_tx_t ipc_wwan_link_transmit(struct sk_buff *skb, ret = NETDEV_TX_BUSY; dev_err(ipc_wwan->dev, "unable to push packets"); } else { - pm_runtime_mark_last_busy(ipc_wwan->ipc_imem->dev); - pm_runtime_put_autosuspend(ipc_wwan->ipc_imem->dev); goto exit; } - pm_runtime_mark_last_busy(ipc_wwan->ipc_imem->dev); - pm_runtime_put_autosuspend(ipc_wwan->ipc_imem->dev); - return ret; exit: diff --git a/drivers/net/xen-netback/interface.c b/drivers/net/xen-netback/interface.c index f3f2c07423a6..fc3bb63b9ac3 100644 --- a/drivers/net/xen-netback/interface.c +++ b/drivers/net/xen-netback/interface.c @@ -41,8 +41,6 @@ #include #include -#define XENVIF_QUEUE_LENGTH 32 - /* Number of bytes allowed on the internal guest Rx queue. */ #define XENVIF_RX_QUEUE_BYTES (XEN_NETIF_RX_RING_SIZE/2 * PAGE_SIZE) @@ -530,8 +528,6 @@ struct xenvif *xenvif_alloc(struct device *parent, domid_t domid, dev->features = dev->hw_features | NETIF_F_RXCSUM; dev->ethtool_ops = &xenvif_ethtool_ops; - dev->tx_queue_len = XENVIF_QUEUE_LENGTH; - dev->min_mtu = ETH_MIN_MTU; dev->max_mtu = ETH_MAX_MTU - VLAN_ETH_HLEN; diff --git a/drivers/nvme/host/auth.c b/drivers/nvme/host/auth.c index daf5d144a8ea..064592a5d546 100644 --- a/drivers/nvme/host/auth.c +++ b/drivers/nvme/host/auth.c @@ -341,7 +341,7 @@ static int nvme_auth_process_dhchap_success1(struct nvme_ctrl *ctrl, struct nvmf_auth_dhchap_success1_data *data = chap->buf; size_t size = sizeof(*data); - if (chap->ctrl_key) + if (chap->s2) size += chap->hash_len; if (size > CHAP_BUF_SIZE) { @@ -825,7 +825,7 @@ static void nvme_queue_auth_work(struct work_struct *work) goto fail2; } - if (chap->ctrl_key) { + if (chap->s2) { /* DH-HMAC-CHAP Step 5: send success2 */ dev_dbg(ctrl->device, "%s: qid %d send success2\n", __func__, chap->qid); diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index f3a01b79148c..21783aa2ee8e 100644 --- a/drivers/nvme/host/core.c +++ b/drivers/nvme/host/core.c @@ -2245,25 +2245,8 @@ int nvme_enable_ctrl(struct nvme_ctrl *ctrl) else ctrl->ctrl_config = NVME_CC_CSS_NVM; - if (ctrl->cap & NVME_CAP_CRMS_CRWMS) { - u32 crto; - - ret = ctrl->ops->reg_read32(ctrl, NVME_REG_CRTO, &crto); - if (ret) { - dev_err(ctrl->device, "Reading CRTO failed (%d)\n", - ret); - return ret; - } - - if (ctrl->cap & NVME_CAP_CRMS_CRIMS) { - ctrl->ctrl_config |= NVME_CC_CRIME; - timeout = NVME_CRTO_CRIMT(crto); - } else { - timeout = NVME_CRTO_CRWMT(crto); - } - } else { - timeout = NVME_CAP_TIMEOUT(ctrl->cap); - } + if (ctrl->cap & NVME_CAP_CRMS_CRWMS && ctrl->cap & NVME_CAP_CRMS_CRIMS) + ctrl->ctrl_config |= NVME_CC_CRIME; ctrl->ctrl_config |= (NVME_CTRL_PAGE_SHIFT - 12) << NVME_CC_MPS_SHIFT; ctrl->ctrl_config |= NVME_CC_AMS_RR | NVME_CC_SHN_NONE; @@ -2277,6 +2260,39 @@ int nvme_enable_ctrl(struct nvme_ctrl *ctrl) if (ret) return ret; + /* CAP value may change after initial CC write */ + ret = ctrl->ops->reg_read64(ctrl, NVME_REG_CAP, &ctrl->cap); + if (ret) + return ret; + + timeout = NVME_CAP_TIMEOUT(ctrl->cap); + if (ctrl->cap & NVME_CAP_CRMS_CRWMS) { + u32 crto, ready_timeout; + + ret = ctrl->ops->reg_read32(ctrl, NVME_REG_CRTO, &crto); + if (ret) { + dev_err(ctrl->device, "Reading CRTO failed (%d)\n", + ret); + return ret; + } + + /* + * CRTO should always be greater or equal to CAP.TO, but some + * devices are known to get this wrong. Use the larger of the + * two values. + */ + if (ctrl->ctrl_config & NVME_CC_CRIME) + ready_timeout = NVME_CRTO_CRIMT(crto); + else + ready_timeout = NVME_CRTO_CRWMT(crto); + + if (ready_timeout < timeout) + dev_warn_once(ctrl->device, "bad crto:%x cap:%llx\n", + crto, ctrl->cap); + else + timeout = ready_timeout; + } + ctrl->ctrl_config |= NVME_CC_ENABLE; ret = ctrl->ops->reg_write32(ctrl, NVME_REG_CC, ctrl->ctrl_config); if (ret) diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c index 1cd2bf82319a..a15b37750d6e 100644 --- a/drivers/nvme/host/fc.c +++ b/drivers/nvme/host/fc.c @@ -1924,7 +1924,7 @@ char *nvme_fc_io_getuuid(struct nvmefc_fcp_req *req) struct nvme_fc_fcp_op *op = fcp_req_to_fcp_op(req); struct request *rq = op->rq; - if (!IS_ENABLED(CONFIG_BLK_CGROUP_FC_APPID) || !rq->bio) + if (!IS_ENABLED(CONFIG_BLK_CGROUP_FC_APPID) || !rq || !rq->bio) return NULL; return blkcg_get_fc_appid(rq->bio); } diff --git a/drivers/nvme/host/hwmon.c b/drivers/nvme/host/hwmon.c index 316f3e4ca7cc..8df73a0b3980 100644 --- a/drivers/nvme/host/hwmon.c +++ b/drivers/nvme/host/hwmon.c @@ -187,7 +187,7 @@ static umode_t nvme_hwmon_is_visible(const void *_data, return 0; } -static const struct hwmon_channel_info *nvme_hwmon_info[] = { +static const struct hwmon_channel_info *const nvme_hwmon_info[] = { HWMON_CHANNEL_INFO(chip, HWMON_C_REGISTER_TZ), HWMON_CHANNEL_INFO(temp, HWMON_T_INPUT | HWMON_T_MAX | HWMON_T_MIN | diff --git a/drivers/nvme/host/ioctl.c b/drivers/nvme/host/ioctl.c index d8ff796fd5f2..747c879e8982 100644 --- a/drivers/nvme/host/ioctl.c +++ b/drivers/nvme/host/ioctl.c @@ -108,9 +108,13 @@ static void *nvme_add_user_metadata(struct request *req, void __user *ubuf, if (!buf) goto out; - ret = -EFAULT; - if ((req_op(req) == REQ_OP_DRV_OUT) && copy_from_user(buf, ubuf, len)) - goto out_free_meta; + if (req_op(req) == REQ_OP_DRV_OUT) { + ret = -EFAULT; + if (copy_from_user(buf, ubuf, len)) + goto out_free_meta; + } else { + memset(buf, 0, len); + } bip = bio_integrity_alloc(bio, GFP_KERNEL, 1); if (IS_ERR(bip)) { diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c index 2f57da12d983..3f0c9ee09a12 100644 --- a/drivers/nvme/host/pci.c +++ b/drivers/nvme/host/pci.c @@ -2916,9 +2916,6 @@ static struct nvme_dev *nvme_pci_alloc_dev(struct pci_dev *pdev, struct nvme_dev *dev; int ret = -ENOMEM; - if (node == NUMA_NO_NODE) - set_dev_node(&pdev->dev, first_memory_node); - dev = kzalloc_node(sizeof(*dev), GFP_KERNEL, node); if (!dev) return ERR_PTR(-ENOMEM); @@ -3332,7 +3329,8 @@ static const struct pci_device_id nvme_id_table[] = { { PCI_VDEVICE(INTEL, 0x0a54), /* Intel P4500/P4600 */ .driver_data = NVME_QUIRK_STRIPE_SIZE | NVME_QUIRK_DEALLOCATE_ZEROES | - NVME_QUIRK_IGNORE_DEV_SUBNQN, }, + NVME_QUIRK_IGNORE_DEV_SUBNQN | + NVME_QUIRK_BOGUS_NID, }, { PCI_VDEVICE(INTEL, 0x0a55), /* Dell Express Flash P4600 */ .driver_data = NVME_QUIRK_STRIPE_SIZE | NVME_QUIRK_DEALLOCATE_ZEROES, }, diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c index 337a624a537c..a7fea4cbacd7 100644 --- a/drivers/nvme/host/rdma.c +++ b/drivers/nvme/host/rdma.c @@ -638,6 +638,9 @@ static void __nvme_rdma_stop_queue(struct nvme_rdma_queue *queue) static void nvme_rdma_stop_queue(struct nvme_rdma_queue *queue) { + if (!test_bit(NVME_RDMA_Q_ALLOCATED, &queue->flags)) + return; + mutex_lock(&queue->queue_lock); if (test_and_clear_bit(NVME_RDMA_Q_LIVE, &queue->flags)) __nvme_rdma_stop_queue(queue); diff --git a/drivers/nvme/target/fabrics-cmd-auth.c b/drivers/nvme/target/fabrics-cmd-auth.c index 586458f765f1..1d9854484e2e 100644 --- a/drivers/nvme/target/fabrics-cmd-auth.c +++ b/drivers/nvme/target/fabrics-cmd-auth.c @@ -333,19 +333,21 @@ done: __func__, ctrl->cntlid, req->sq->qid, status, req->error_loc); req->cqe->result.u64 = 0; - nvmet_req_complete(req, status); if (req->sq->dhchap_step != NVME_AUTH_DHCHAP_MESSAGE_SUCCESS2 && req->sq->dhchap_step != NVME_AUTH_DHCHAP_MESSAGE_FAILURE2) { unsigned long auth_expire_secs = ctrl->kato ? ctrl->kato : 120; mod_delayed_work(system_wq, &req->sq->auth_expired_work, auth_expire_secs * HZ); - return; + goto complete; } /* Final states, clear up variables */ nvmet_auth_sq_free(req->sq); if (req->sq->dhchap_step == NVME_AUTH_DHCHAP_MESSAGE_FAILURE2) nvmet_ctrl_fatal_error(ctrl); + +complete: + nvmet_req_complete(req, status); } static int nvmet_auth_challenge(struct nvmet_req *req, void *d, int al) @@ -514,11 +516,12 @@ void nvmet_execute_auth_receive(struct nvmet_req *req) kfree(d); done: req->cqe->result.u64 = 0; - nvmet_req_complete(req, status); + if (req->sq->dhchap_step == NVME_AUTH_DHCHAP_MESSAGE_SUCCESS2) nvmet_auth_sq_free(req->sq); else if (req->sq->dhchap_step == NVME_AUTH_DHCHAP_MESSAGE_FAILURE1) { nvmet_auth_sq_free(req->sq); nvmet_ctrl_fatal_error(ctrl); } + nvmet_req_complete(req, status); } diff --git a/drivers/nvme/target/tcp.c b/drivers/nvme/target/tcp.c index 868aa4de2e4c..197fc2ecb164 100644 --- a/drivers/nvme/target/tcp.c +++ b/drivers/nvme/target/tcp.c @@ -348,7 +348,7 @@ static void nvmet_tcp_build_pdu_iovec(struct nvmet_tcp_cmd *cmd) while (length) { u32 iov_len = min_t(u32, length, sg->length - sg_offset); - bvec_set_page(iov, sg_page(sg), sg->length, + bvec_set_page(iov, sg_page(sg), iov_len, sg->offset + sg_offset); length -= iov_len; @@ -372,6 +372,7 @@ static void nvmet_tcp_fatal_error(struct nvmet_tcp_queue *queue) static void nvmet_tcp_socket_error(struct nvmet_tcp_queue *queue, int status) { + queue->rcv_state = NVMET_TCP_RECV_ERR; if (status == -EPIPE || status == -ECONNRESET) kernel_sock_shutdown(queue->sock, SHUT_RDWR); else @@ -910,15 +911,11 @@ static int nvmet_tcp_handle_icreq(struct nvmet_tcp_queue *queue) iov.iov_len = sizeof(*icresp); ret = kernel_sendmsg(queue->sock, &msg, &iov, 1, iov.iov_len); if (ret < 0) - goto free_crypto; + return ret; /* queue removal will cleanup */ queue->state = NVMET_TCP_Q_LIVE; nvmet_prepare_receive_pdu(queue); return 0; -free_crypto: - if (queue->hdr_digest || queue->data_digest) - nvmet_tcp_free_crypto(queue); - return ret; } static void nvmet_tcp_handle_req_failure(struct nvmet_tcp_queue *queue, diff --git a/drivers/of/dynamic.c b/drivers/of/dynamic.c index 0a3483e247a8..f63250c650ca 100644 --- a/drivers/of/dynamic.c +++ b/drivers/of/dynamic.c @@ -890,13 +890,13 @@ int of_changeset_action(struct of_changeset *ocs, unsigned long action, { struct of_changeset_entry *ce; + if (WARN_ON(action >= ARRAY_SIZE(action_names))) + return -EINVAL; + ce = kzalloc(sizeof(*ce), GFP_KERNEL); if (!ce) return -ENOMEM; - if (WARN_ON(action >= ARRAY_SIZE(action_names))) - return -EINVAL; - /* get a reference to the node */ ce->action = action; ce->np = of_node_get(np); diff --git a/drivers/of/overlay.c b/drivers/of/overlay.c index dfb6fb962fc7..a9a292d6d59b 100644 --- a/drivers/of/overlay.c +++ b/drivers/of/overlay.c @@ -45,8 +45,8 @@ struct target { /** * struct fragment - info about fragment nodes in overlay expanded device tree - * @target: target of the overlay operation * @overlay: pointer to the __overlay__ node + * @target: target of the overlay operation */ struct fragment { struct device_node *overlay; diff --git a/drivers/parisc/ccio-dma.c b/drivers/parisc/ccio-dma.c index 509a4072d50a..9ce0d20a6c58 100644 --- a/drivers/parisc/ccio-dma.c +++ b/drivers/parisc/ccio-dma.c @@ -214,7 +214,7 @@ struct ioa_registers { struct ioc { struct ioa_registers __iomem *ioc_regs; /* I/O MMU base address */ u8 *res_map; /* resource map, bit == pdir entry */ - u64 *pdir_base; /* physical base address */ + __le64 *pdir_base; /* physical base address */ u32 pdir_size; /* bytes, function of IOV Space size */ u32 res_hint; /* next available IOVP - circular search */ @@ -339,7 +339,7 @@ ccio_alloc_range(struct ioc *ioc, struct device *dev, size_t size) BUG_ON(pages_needed == 0); BUG_ON((pages_needed * IOVP_SIZE) > DMA_CHUNK_SIZE); - DBG_RES("%s() size: %d pages_needed %d\n", + DBG_RES("%s() size: %zu pages_needed %d\n", __func__, size, pages_needed); /* @@ -427,7 +427,7 @@ ccio_free_range(struct ioc *ioc, dma_addr_t iova, unsigned long pages_mapped) BUG_ON((pages_mapped * IOVP_SIZE) > DMA_CHUNK_SIZE); BUG_ON(pages_mapped > BITS_PER_LONG); - DBG_RES("%s(): res_idx: %d pages_mapped %d\n", + DBG_RES("%s(): res_idx: %d pages_mapped %lu\n", __func__, res_idx, pages_mapped); #ifdef CCIO_COLLECT_STATS @@ -543,7 +543,7 @@ static u32 hint_lookup[] = { * index are bits 12:19 of the value returned by LCI. */ static void -ccio_io_pdir_entry(u64 *pdir_ptr, space_t sid, unsigned long vba, +ccio_io_pdir_entry(__le64 *pdir_ptr, space_t sid, unsigned long vba, unsigned long hints) { register unsigned long pa; @@ -719,7 +719,7 @@ ccio_map_single(struct device *dev, void *addr, size_t size, unsigned long flags; dma_addr_t iovp; dma_addr_t offset; - u64 *pdir_start; + __le64 *pdir_start; unsigned long hint = hint_lookup[(int)direction]; BUG_ON(!dev); @@ -746,8 +746,8 @@ ccio_map_single(struct device *dev, void *addr, size_t size, pdir_start = &(ioc->pdir_base[idx]); - DBG_RUN("%s() 0x%p -> 0x%lx size: %0x%x\n", - __func__, addr, (long)iovp | offset, size); + DBG_RUN("%s() %px -> %#lx size: %zu\n", + __func__, addr, (long)(iovp | offset), size); /* If not cacheline aligned, force SAFE_DMA on the whole mess */ if((size % L1_CACHE_BYTES) || ((unsigned long)addr % L1_CACHE_BYTES)) @@ -805,7 +805,7 @@ ccio_unmap_page(struct device *dev, dma_addr_t iova, size_t size, return; } - DBG_RUN("%s() iovp 0x%lx/%x\n", + DBG_RUN("%s() iovp %#lx/%zx\n", __func__, (long)iova, size); iova ^= offset; /* clear offset bits */ @@ -1283,7 +1283,7 @@ ccio_ioc_init(struct ioc *ioc) iova_space_size>>20, iov_order + PAGE_SHIFT); - ioc->pdir_base = (u64 *)__get_free_pages(GFP_KERNEL, + ioc->pdir_base = (__le64 *)__get_free_pages(GFP_KERNEL, get_order(ioc->pdir_size)); if(NULL == ioc->pdir_base) { panic("%s() could not allocate I/O Page Table\n", __func__); diff --git a/drivers/parisc/iommu-helpers.h b/drivers/parisc/iommu-helpers.h index 0905be256de0..c43f1a212a5c 100644 --- a/drivers/parisc/iommu-helpers.h +++ b/drivers/parisc/iommu-helpers.h @@ -14,13 +14,13 @@ static inline unsigned int iommu_fill_pdir(struct ioc *ioc, struct scatterlist *startsg, int nents, unsigned long hint, - void (*iommu_io_pdir_entry)(u64 *, space_t, unsigned long, + void (*iommu_io_pdir_entry)(__le64 *, space_t, unsigned long, unsigned long)) { struct scatterlist *dma_sg = startsg; /* pointer to current DMA */ unsigned int n_mappings = 0; unsigned long dma_offset = 0, dma_len = 0; - u64 *pdirp = NULL; + __le64 *pdirp = NULL; /* Horrible hack. For efficiency's sake, dma_sg starts one * entry below the true start (it is immediately incremented @@ -31,8 +31,8 @@ iommu_fill_pdir(struct ioc *ioc, struct scatterlist *startsg, int nents, unsigned long vaddr; long size; - DBG_RUN_SG(" %d : %08lx/%05x %p/%05x\n", nents, - (unsigned long)sg_dma_address(startsg), cnt, + DBG_RUN_SG(" %d : %08lx %p/%05x\n", nents, + (unsigned long)sg_dma_address(startsg), sg_virt(startsg), startsg->length ); diff --git a/drivers/parisc/iosapic.c b/drivers/parisc/iosapic.c index a7df764f1a72..a4011461189b 100644 --- a/drivers/parisc/iosapic.c +++ b/drivers/parisc/iosapic.c @@ -202,9 +202,9 @@ static inline void iosapic_write(void __iomem *iosapic, unsigned int reg, u32 va static DEFINE_SPINLOCK(iosapic_lock); -static inline void iosapic_eoi(void __iomem *addr, unsigned int data) +static inline void iosapic_eoi(__le32 __iomem *addr, __le32 data) { - __raw_writel(data, addr); + __raw_writel((__force u32)data, addr); } /* diff --git a/drivers/parisc/iosapic_private.h b/drivers/parisc/iosapic_private.h index 73ecc657ad95..bd8ff40162b4 100644 --- a/drivers/parisc/iosapic_private.h +++ b/drivers/parisc/iosapic_private.h @@ -118,8 +118,8 @@ struct iosapic_irt { struct vector_info { struct iosapic_info *iosapic; /* I/O SAPIC this vector is on */ struct irt_entry *irte; /* IRT entry */ - u32 __iomem *eoi_addr; /* precalculate EOI reg address */ - u32 eoi_data; /* IA64: ? PA: swapped txn_data */ + __le32 __iomem *eoi_addr; /* precalculate EOI reg address */ + __le32 eoi_data; /* IA64: ? PA: swapped txn_data */ int txn_irq; /* virtual IRQ number for processor */ ulong txn_addr; /* IA64: id_eid PA: partial HPA */ u32 txn_data; /* CPU interrupt bit */ diff --git a/drivers/parisc/sba_iommu.c b/drivers/parisc/sba_iommu.c index f6b510675318..05e7103d1d40 100644 --- a/drivers/parisc/sba_iommu.c +++ b/drivers/parisc/sba_iommu.c @@ -46,8 +46,6 @@ #include #include -#include /* for proc_mckinley_root */ -#include /* for proc_runway_root */ #include /* for PAGE0 */ #include /* for PDC_MODEL_* */ #include /* for is_pdc_pat() */ @@ -122,7 +120,7 @@ MODULE_PARM_DESC(sba_reserve_agpgart, "Reserve half of IO pdir as AGPGART"); #endif static struct proc_dir_entry *proc_runway_root __ro_after_init; -struct proc_dir_entry *proc_mckinley_root __ro_after_init; +static struct proc_dir_entry *proc_mckinley_root __ro_after_init; /************************************ ** SBA register read and write support @@ -204,7 +202,7 @@ static void sba_dump_pdir_entry(struct ioc *ioc, char *msg, uint pide) { /* start printing from lowest pde in rval */ - u64 *ptr = &(ioc->pdir_base[pide & (~0U * BITS_PER_LONG)]); + __le64 *ptr = &(ioc->pdir_base[pide & (~0U * BITS_PER_LONG)]); unsigned long *rptr = (unsigned long *) &(ioc->res_map[(pide >>3) & ~(sizeof(unsigned long) - 1)]); uint rcnt; @@ -571,7 +569,7 @@ typedef unsigned long space_t; */ static void -sba_io_pdir_entry(u64 *pdir_ptr, space_t sid, unsigned long vba, +sba_io_pdir_entry(__le64 *pdir_ptr, space_t sid, unsigned long vba, unsigned long hint) { u64 pa; /* physical address */ @@ -615,7 +613,7 @@ static void sba_mark_invalid(struct ioc *ioc, dma_addr_t iova, size_t byte_cnt) { u32 iovp = (u32) SBA_IOVP(ioc,iova); - u64 *pdir_ptr = &ioc->pdir_base[PDIR_INDEX(iovp)]; + __le64 *pdir_ptr = &ioc->pdir_base[PDIR_INDEX(iovp)]; #ifdef ASSERT_PDIR_SANITY /* Assert first pdir entry is set. @@ -716,7 +714,7 @@ sba_map_single(struct device *dev, void *addr, size_t size, unsigned long flags; dma_addr_t iovp; dma_addr_t offset; - u64 *pdir_start; + __le64 *pdir_start; int pide; ioc = GET_IOC(dev); @@ -1434,7 +1432,7 @@ sba_ioc_init(struct parisc_device *sba, struct ioc *ioc, int ioc_num) ioc->pdir_size = pdir_size = (iova_space_size/IOVP_SIZE) * sizeof(u64); - DBG_INIT("%s() hpa 0x%lx mem %ldMB IOV %dMB (%d bits)\n", + DBG_INIT("%s() hpa %px mem %ldMB IOV %dMB (%d bits)\n", __func__, ioc->ioc_hpa, (unsigned long) totalram_pages() >> (20 - PAGE_SHIFT), @@ -1471,7 +1469,7 @@ sba_ioc_init(struct parisc_device *sba, struct ioc *ioc, int ioc_num) ioc->iovp_mask = ~(iova_space_mask + PAGE_SIZE - 1); #endif - DBG_INIT("%s() IOV base 0x%lx mask 0x%0lx\n", + DBG_INIT("%s() IOV base %#lx mask %#0lx\n", __func__, ioc->ibase, ioc->imask); /* @@ -1583,7 +1581,7 @@ printk("sba_hw_init(): mem_boot 0x%x 0x%x 0x%x 0x%x\n", PAGE0->mem_boot.hpa, if (!IS_PLUTO(sba_dev->dev)) { ioc_ctl = READ_REG(sba_dev->sba_hpa+IOC_CTRL); - DBG_INIT("%s() hpa 0x%lx ioc_ctl 0x%Lx ->", + DBG_INIT("%s() hpa %px ioc_ctl 0x%Lx ->", __func__, sba_dev->sba_hpa, ioc_ctl); ioc_ctl &= ~(IOC_CTRL_RM | IOC_CTRL_NC | IOC_CTRL_CE); ioc_ctl |= IOC_CTRL_DD | IOC_CTRL_D4 | IOC_CTRL_TC; @@ -1668,14 +1666,14 @@ printk("sba_hw_init(): mem_boot 0x%x 0x%x 0x%x 0x%x\n", PAGE0->mem_boot.hpa, /* flush out the last writes */ READ_REG(sba_dev->ioc[i].ioc_hpa + ROPE7_CTL); - DBG_INIT(" ioc[%d] ROPE_CFG 0x%Lx ROPE_DBG 0x%Lx\n", + DBG_INIT(" ioc[%d] ROPE_CFG %#lx ROPE_DBG %lx\n", i, - READ_REG(sba_dev->ioc[i].ioc_hpa + 0x40), - READ_REG(sba_dev->ioc[i].ioc_hpa + 0x50) + (unsigned long) READ_REG(sba_dev->ioc[i].ioc_hpa + 0x40), + (unsigned long) READ_REG(sba_dev->ioc[i].ioc_hpa + 0x50) ); - DBG_INIT(" STATUS_CONTROL 0x%Lx FLUSH_CTRL 0x%Lx\n", - READ_REG(sba_dev->ioc[i].ioc_hpa + 0x108), - READ_REG(sba_dev->ioc[i].ioc_hpa + 0x400) + DBG_INIT(" STATUS_CONTROL %#lx FLUSH_CTRL %#lx\n", + (unsigned long) READ_REG(sba_dev->ioc[i].ioc_hpa + 0x108), + (unsigned long) READ_REG(sba_dev->ioc[i].ioc_hpa + 0x400) ); if (IS_PLUTO(sba_dev->dev)) { @@ -1739,7 +1737,7 @@ sba_common_init(struct sba_device *sba_dev) #ifdef ASSERT_PDIR_SANITY /* Mark first bit busy - ie no IOVA 0 */ sba_dev->ioc[i].res_map[0] = 0x80; - sba_dev->ioc[i].pdir_base[0] = 0xeeffc0addbba0080ULL; + sba_dev->ioc[i].pdir_base[0] = (__force __le64) 0xeeffc0addbba0080ULL; #endif /* Third (and last) part of PIRANHA BUG */ @@ -1899,9 +1897,7 @@ static int __init sba_driver_callback(struct parisc_device *dev) int i; char *version; void __iomem *sba_addr = ioremap(dev->hpa.start, SBA_FUNC_SIZE); -#ifdef CONFIG_PROC_FS - struct proc_dir_entry *root; -#endif + struct proc_dir_entry *root __maybe_unused; sba_dump_ranges(sba_addr); @@ -1967,7 +1963,6 @@ static int __init sba_driver_callback(struct parisc_device *dev) hppa_dma_ops = &sba_ops; -#ifdef CONFIG_PROC_FS switch (dev->id.hversion) { case PLUTO_MCKINLEY_PORT: if (!proc_mckinley_root) @@ -1985,7 +1980,6 @@ static int __init sba_driver_callback(struct parisc_device *dev) proc_create_single("sba_iommu", 0, root, sba_proc_info); proc_create_single("sba_iommu-bitmap", 0, root, sba_proc_bitmap_info); -#endif return 0; } diff --git a/drivers/pci/controller/dwc/pcie-qcom.c b/drivers/pci/controller/dwc/pcie-qcom.c index e2f29404c84e..64420ecc24d1 100644 --- a/drivers/pci/controller/dwc/pcie-qcom.c +++ b/drivers/pci/controller/dwc/pcie-qcom.c @@ -43,7 +43,6 @@ #define PARF_PHY_REFCLK 0x4c #define PARF_CONFIG_BITS 0x50 #define PARF_DBI_BASE_ADDR 0x168 -#define PARF_SLV_ADDR_SPACE_SIZE_2_3_3 0x16c /* Register offset specific to IP ver 2.3.3 */ #define PARF_MHI_CLOCK_RESET_CTRL 0x174 #define PARF_AXI_MSTR_WR_ADDR_HALT 0x178 #define PARF_AXI_MSTR_WR_ADDR_HALT_V2 0x1a8 @@ -797,8 +796,7 @@ static int qcom_pcie_post_init_2_3_3(struct qcom_pcie *pcie) u16 offset = dw_pcie_find_capability(pci, PCI_CAP_ID_EXP); u32 val; - writel(SLV_ADDR_SPACE_SZ, - pcie->parf + PARF_SLV_ADDR_SPACE_SIZE_2_3_3); + writel(SLV_ADDR_SPACE_SZ, pcie->parf + PARF_SLV_ADDR_SPACE_SIZE); val = readl(pcie->parf + PARF_PHY_CTRL); val &= ~PHY_TEST_PWR_DOWN; diff --git a/drivers/pci/of.c b/drivers/pci/of.c index 2af64bcb7da3..51e3dd0ea5ab 100644 --- a/drivers/pci/of.c +++ b/drivers/pci/of.c @@ -657,30 +657,33 @@ void of_pci_make_dev_node(struct pci_dev *pdev) cset = kmalloc(sizeof(*cset), GFP_KERNEL); if (!cset) - goto failed; + goto out_free_name; of_changeset_init(cset); np = of_changeset_create_node(cset, ppnode, name); if (!np) - goto failed; - np->data = cset; + goto out_destroy_cset; ret = of_pci_add_properties(pdev, cset, np); if (ret) - goto failed; + goto out_free_node; ret = of_changeset_apply(cset); if (ret) - goto failed; + goto out_free_node; + np->data = cset; pdev->dev.of_node = np; kfree(name); return; -failed: - if (np) - of_node_put(np); +out_free_node: + of_node_put(np); +out_destroy_cset: + of_changeset_destroy(cset); + kfree(cset); +out_free_name: kfree(name); } #endif diff --git a/drivers/pci/of_property.c b/drivers/pci/of_property.c index 710ec35ba4a1..c2c7334152bc 100644 --- a/drivers/pci/of_property.c +++ b/drivers/pci/of_property.c @@ -186,8 +186,8 @@ static int of_pci_prop_interrupts(struct pci_dev *pdev, static int of_pci_prop_intr_map(struct pci_dev *pdev, struct of_changeset *ocs, struct device_node *np) { + u32 i, addr_sz[OF_PCI_MAX_INT_PIN] = { 0 }, map_sz = 0; struct of_phandle_args out_irq[OF_PCI_MAX_INT_PIN]; - u32 i, addr_sz[OF_PCI_MAX_INT_PIN], map_sz = 0; __be32 laddr[OF_PCI_ADDRESS_CELLS] = { 0 }; u32 int_map_mask[] = { 0xffff00, 0, 0, 7 }; struct device_node *pnode; @@ -213,33 +213,44 @@ static int of_pci_prop_intr_map(struct pci_dev *pdev, struct of_changeset *ocs, out_irq[i].args[0] = pin; ret = of_irq_parse_raw(laddr, &out_irq[i]); if (ret) { - pci_err(pdev, "parse irq %d failed, ret %d", pin, ret); + out_irq[i].np = NULL; + pci_dbg(pdev, "parse irq %d failed, ret %d", pin, ret); continue; } - ret = of_property_read_u32(out_irq[i].np, "#address-cells", - &addr_sz[i]); - if (ret) - addr_sz[i] = 0; + of_property_read_u32(out_irq[i].np, "#address-cells", + &addr_sz[i]); } list_for_each_entry(child, &pdev->subordinate->devices, bus_list) { for (pin = 1; pin <= OF_PCI_MAX_INT_PIN; pin++) { i = pci_swizzle_interrupt_pin(child, pin) - 1; + if (!out_irq[i].np) + continue; map_sz += 5 + addr_sz[i] + out_irq[i].args_count; } } + /* + * Parsing interrupt failed for all pins. In this case, it does not + * need to generate interrupt-map property. + */ + if (!map_sz) + return 0; + int_map = kcalloc(map_sz, sizeof(u32), GFP_KERNEL); mapp = int_map; list_for_each_entry(child, &pdev->subordinate->devices, bus_list) { for (pin = 1; pin <= OF_PCI_MAX_INT_PIN; pin++) { + i = pci_swizzle_interrupt_pin(child, pin) - 1; + if (!out_irq[i].np) + continue; + *mapp = (child->bus->number << 16) | (child->devfn << 8); mapp += OF_PCI_ADDRESS_CELLS; *mapp = pin; mapp++; - i = pci_swizzle_interrupt_pin(child, pin) - 1; *mapp = out_irq[i].np->phandle; mapp++; if (addr_sz[i]) { diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c index a79c110c7e51..51ec9e7e784f 100644 --- a/drivers/pci/pci-driver.c +++ b/drivers/pci/pci-driver.c @@ -572,7 +572,19 @@ static void pci_pm_default_resume_early(struct pci_dev *pci_dev) static void pci_pm_bridge_power_up_actions(struct pci_dev *pci_dev) { - pci_bridge_wait_for_secondary_bus(pci_dev, "resume"); + int ret; + + ret = pci_bridge_wait_for_secondary_bus(pci_dev, "resume"); + if (ret) { + /* + * The downstream link failed to come up, so mark the + * devices below as disconnected to make sure we don't + * attempt to resume them. + */ + pci_walk_bus(pci_dev->subordinate, pci_dev_set_disconnected, + NULL); + return; + } /* * When powering on a bridge from D3cold, the whole hierarchy may be diff --git a/drivers/pci/pcie/aer.c b/drivers/pci/pcie/aer.c index e85ff946e8c8..9c8fd69ae5ad 100644 --- a/drivers/pci/pcie/aer.c +++ b/drivers/pci/pcie/aer.c @@ -229,6 +229,7 @@ int pcie_aer_is_native(struct pci_dev *dev) return pcie_ports_native || host->native_aer; } +EXPORT_SYMBOL_NS_GPL(pcie_aer_is_native, CXL); static int pci_enable_pcie_error_reporting(struct pci_dev *dev) { diff --git a/drivers/pci/pcie/portdrv.h b/drivers/pci/pcie/portdrv.h index 58a2b1a1cae4..1f3803bde7ee 100644 --- a/drivers/pci/pcie/portdrv.h +++ b/drivers/pci/pcie/portdrv.h @@ -29,10 +29,8 @@ extern bool pcie_ports_dpc_native; #ifdef CONFIG_PCIEAER int pcie_aer_init(void); -int pcie_aer_is_native(struct pci_dev *dev); #else static inline int pcie_aer_init(void) { return 0; } -static inline int pcie_aer_is_native(struct pci_dev *dev) { return 0; } #endif #ifdef CONFIG_HOTPLUG_PCI_PCIE diff --git a/drivers/perf/arm-cmn.c b/drivers/perf/arm-cmn.c index 913dc04b3a40..6b50bc551984 100644 --- a/drivers/perf/arm-cmn.c +++ b/drivers/perf/arm-cmn.c @@ -1972,7 +1972,7 @@ static irqreturn_t arm_cmn_handle_irq(int irq, void *dev_id) u64 delta; int i; - for (i = 0; i < CMN_DTM_NUM_COUNTERS; i++) { + for (i = 0; i < CMN_DT_NUM_COUNTERS; i++) { if (status & (1U << i)) { ret = IRQ_HANDLED; if (WARN_ON(!dtc->counters[i])) diff --git a/drivers/perf/riscv_pmu.c b/drivers/perf/riscv_pmu.c index 1f9a35f724f5..0dda70e1ef90 100644 --- a/drivers/perf/riscv_pmu.c +++ b/drivers/perf/riscv_pmu.c @@ -23,7 +23,8 @@ static bool riscv_perf_user_access(struct perf_event *event) return ((event->attr.type == PERF_TYPE_HARDWARE) || (event->attr.type == PERF_TYPE_HW_CACHE) || (event->attr.type == PERF_TYPE_RAW)) && - !!(event->hw.flags & PERF_EVENT_FLAG_USER_READ_CNT); + !!(event->hw.flags & PERF_EVENT_FLAG_USER_READ_CNT) && + (event->hw.idx != -1); } void arch_perf_update_userpage(struct perf_event *event, diff --git a/drivers/perf/riscv_pmu_sbi.c b/drivers/perf/riscv_pmu_sbi.c index 9a51053b1f99..96c7f670c8f0 100644 --- a/drivers/perf/riscv_pmu_sbi.c +++ b/drivers/perf/riscv_pmu_sbi.c @@ -510,16 +510,18 @@ static void pmu_sbi_set_scounteren(void *arg) { struct perf_event *event = (struct perf_event *)arg; - csr_write(CSR_SCOUNTEREN, - csr_read(CSR_SCOUNTEREN) | (1 << pmu_sbi_csr_index(event))); + if (event->hw.idx != -1) + csr_write(CSR_SCOUNTEREN, + csr_read(CSR_SCOUNTEREN) | (1 << pmu_sbi_csr_index(event))); } static void pmu_sbi_reset_scounteren(void *arg) { struct perf_event *event = (struct perf_event *)arg; - csr_write(CSR_SCOUNTEREN, - csr_read(CSR_SCOUNTEREN) & ~(1 << pmu_sbi_csr_index(event))); + if (event->hw.idx != -1) + csr_write(CSR_SCOUNTEREN, + csr_read(CSR_SCOUNTEREN) & ~(1 << pmu_sbi_csr_index(event))); } static void pmu_sbi_ctr_start(struct perf_event *event, u64 ival) @@ -541,7 +543,8 @@ static void pmu_sbi_ctr_start(struct perf_event *event, u64 ival) if ((hwc->flags & PERF_EVENT_FLAG_USER_ACCESS) && (hwc->flags & PERF_EVENT_FLAG_USER_READ_CNT)) - pmu_sbi_set_scounteren((void *)event); + on_each_cpu_mask(mm_cpumask(event->owner->mm), + pmu_sbi_set_scounteren, (void *)event, 1); } static void pmu_sbi_ctr_stop(struct perf_event *event, unsigned long flag) @@ -551,7 +554,8 @@ static void pmu_sbi_ctr_stop(struct perf_event *event, unsigned long flag) if ((hwc->flags & PERF_EVENT_FLAG_USER_ACCESS) && (hwc->flags & PERF_EVENT_FLAG_USER_READ_CNT)) - pmu_sbi_reset_scounteren((void *)event); + on_each_cpu_mask(mm_cpumask(event->owner->mm), + pmu_sbi_reset_scounteren, (void *)event, 1); ret = sbi_ecall(SBI_EXT_PMU, SBI_EXT_PMU_COUNTER_STOP, hwc->idx, 1, flag, 0, 0, 0); if (ret.error && (ret.error != SBI_ERR_ALREADY_STOPPED) && diff --git a/drivers/phy/freescale/phy-fsl-lynx-28g.c b/drivers/phy/freescale/phy-fsl-lynx-28g.c index 4f036c77284e..e2187767ce00 100644 --- a/drivers/phy/freescale/phy-fsl-lynx-28g.c +++ b/drivers/phy/freescale/phy-fsl-lynx-28g.c @@ -127,6 +127,10 @@ struct lynx_28g_lane { struct lynx_28g_priv { void __iomem *base; struct device *dev; + /* Serialize concurrent access to registers shared between lanes, + * like PCCn + */ + spinlock_t pcc_lock; struct lynx_28g_pll pll[LYNX_28G_NUM_PLL]; struct lynx_28g_lane lane[LYNX_28G_NUM_LANE]; @@ -397,6 +401,8 @@ static int lynx_28g_set_mode(struct phy *phy, enum phy_mode mode, int submode) if (powered_up) lynx_28g_power_off(phy); + spin_lock(&priv->pcc_lock); + switch (submode) { case PHY_INTERFACE_MODE_SGMII: case PHY_INTERFACE_MODE_1000BASEX: @@ -413,6 +419,8 @@ static int lynx_28g_set_mode(struct phy *phy, enum phy_mode mode, int submode) lane->interface = submode; out: + spin_unlock(&priv->pcc_lock); + /* Power up the lane if necessary */ if (powered_up) lynx_28g_power_on(phy); @@ -508,11 +516,12 @@ static void lynx_28g_cdr_lock_check(struct work_struct *work) for (i = 0; i < LYNX_28G_NUM_LANE; i++) { lane = &priv->lane[i]; - if (!lane->init) - continue; + mutex_lock(&lane->phy->mutex); - if (!lane->powered_up) + if (!lane->init || !lane->powered_up) { + mutex_unlock(&lane->phy->mutex); continue; + } rrstctl = lynx_28g_lane_read(lane, LNaRRSTCTL); if (!(rrstctl & LYNX_28G_LNaRRSTCTL_CDR_LOCK)) { @@ -521,6 +530,8 @@ static void lynx_28g_cdr_lock_check(struct work_struct *work) rrstctl = lynx_28g_lane_read(lane, LNaRRSTCTL); } while (!(rrstctl & LYNX_28G_LNaRRSTCTL_RST_DONE)); } + + mutex_unlock(&lane->phy->mutex); } queue_delayed_work(system_power_efficient_wq, &priv->cdr_check, msecs_to_jiffies(1000)); @@ -593,6 +604,7 @@ static int lynx_28g_probe(struct platform_device *pdev) dev_set_drvdata(dev, priv); + spin_lock_init(&priv->pcc_lock); INIT_DELAYED_WORK(&priv->cdr_check, lynx_28g_cdr_lock_check); queue_delayed_work(system_power_efficient_wq, &priv->cdr_check, @@ -604,6 +616,14 @@ static int lynx_28g_probe(struct platform_device *pdev) return PTR_ERR_OR_ZERO(provider); } +static void lynx_28g_remove(struct platform_device *pdev) +{ + struct device *dev = &pdev->dev; + struct lynx_28g_priv *priv = dev_get_drvdata(dev); + + cancel_delayed_work_sync(&priv->cdr_check); +} + static const struct of_device_id lynx_28g_of_match_table[] = { { .compatible = "fsl,lynx-28g" }, { }, @@ -612,6 +632,7 @@ MODULE_DEVICE_TABLE(of, lynx_28g_of_match_table); static struct platform_driver lynx_28g_driver = { .probe = lynx_28g_probe, + .remove_new = lynx_28g_remove, .driver = { .name = "lynx-28g", .of_match_table = lynx_28g_of_match_table, diff --git a/drivers/phy/motorola/phy-mapphone-mdm6600.c b/drivers/phy/motorola/phy-mapphone-mdm6600.c index 1d567604b650..376d023a0aa9 100644 --- a/drivers/phy/motorola/phy-mapphone-mdm6600.c +++ b/drivers/phy/motorola/phy-mapphone-mdm6600.c @@ -122,16 +122,10 @@ static int phy_mdm6600_power_on(struct phy *x) { struct phy_mdm6600 *ddata = phy_get_drvdata(x); struct gpio_desc *enable_gpio = ddata->ctrl_gpios[PHY_MDM6600_ENABLE]; - int error; if (!ddata->enabled) return -ENODEV; - error = pinctrl_pm_select_default_state(ddata->dev); - if (error) - dev_warn(ddata->dev, "%s: error with default_state: %i\n", - __func__, error); - gpiod_set_value_cansleep(enable_gpio, 1); /* Allow aggressive PM for USB, it's only needed for n_gsm port */ @@ -160,11 +154,6 @@ static int phy_mdm6600_power_off(struct phy *x) gpiod_set_value_cansleep(enable_gpio, 0); - error = pinctrl_pm_select_sleep_state(ddata->dev); - if (error) - dev_warn(ddata->dev, "%s: error with sleep_state: %i\n", - __func__, error); - return 0; } @@ -456,6 +445,7 @@ static void phy_mdm6600_device_power_off(struct phy_mdm6600 *ddata) { struct gpio_desc *reset_gpio = ddata->ctrl_gpios[PHY_MDM6600_RESET]; + int error; ddata->enabled = false; phy_mdm6600_cmd(ddata, PHY_MDM6600_CMD_BP_SHUTDOWN_REQ); @@ -471,6 +461,17 @@ static void phy_mdm6600_device_power_off(struct phy_mdm6600 *ddata) } else { dev_err(ddata->dev, "Timed out powering down\n"); } + + /* + * Keep reset gpio high with padconf internal pull-up resistor to + * prevent modem from waking up during deeper SoC idle states. The + * gpio bank lines can have glitches if not in the always-on wkup + * domain. + */ + error = pinctrl_pm_select_sleep_state(ddata->dev); + if (error) + dev_warn(ddata->dev, "%s: error with sleep_state: %i\n", + __func__, error); } static void phy_mdm6600_deferred_power_on(struct work_struct *work) @@ -571,12 +572,6 @@ static int phy_mdm6600_probe(struct platform_device *pdev) ddata->dev = &pdev->dev; platform_set_drvdata(pdev, ddata); - /* Active state selected in phy_mdm6600_power_on() */ - error = pinctrl_pm_select_sleep_state(ddata->dev); - if (error) - dev_warn(ddata->dev, "%s: error with sleep_state: %i\n", - __func__, error); - error = phy_mdm6600_init_lines(ddata); if (error) return error; @@ -627,10 +622,12 @@ idle: pm_runtime_put_autosuspend(ddata->dev); cleanup: - if (error < 0) + if (error < 0) { phy_mdm6600_device_power_off(ddata); - pm_runtime_disable(ddata->dev); - pm_runtime_dont_use_autosuspend(ddata->dev); + pm_runtime_disable(ddata->dev); + pm_runtime_dont_use_autosuspend(ddata->dev); + } + return error; } @@ -639,6 +636,7 @@ static void phy_mdm6600_remove(struct platform_device *pdev) struct phy_mdm6600 *ddata = platform_get_drvdata(pdev); struct gpio_desc *reset_gpio = ddata->ctrl_gpios[PHY_MDM6600_RESET]; + pm_runtime_get_noresume(ddata->dev); pm_runtime_dont_use_autosuspend(ddata->dev); pm_runtime_put_sync(ddata->dev); pm_runtime_disable(ddata->dev); diff --git a/drivers/phy/qualcomm/phy-qcom-apq8064-sata.c b/drivers/phy/qualcomm/phy-qcom-apq8064-sata.c index 8814f4322adf..3642a5d4f2f3 100644 --- a/drivers/phy/qualcomm/phy-qcom-apq8064-sata.c +++ b/drivers/phy/qualcomm/phy-qcom-apq8064-sata.c @@ -152,7 +152,7 @@ static int qcom_apq8064_sata_phy_init(struct phy *generic_phy) return ret; } - /* SATA phy calibrated succesfully, power up to functional mode */ + /* SATA phy calibrated successfully, power up to functional mode */ writel_relaxed(0x3E, base + SATA_PHY_POW_DWN_CTRL1); writel_relaxed(0x01, base + SATA_PHY_RX_IMCAL0); writel_relaxed(0x01, base + SATA_PHY_TX_IMCAL0); diff --git a/drivers/phy/qualcomm/phy-qcom-m31.c b/drivers/phy/qualcomm/phy-qcom-m31.c index ed08072ca032..5cb7e79b99b3 100644 --- a/drivers/phy/qualcomm/phy-qcom-m31.c +++ b/drivers/phy/qualcomm/phy-qcom-m31.c @@ -82,7 +82,7 @@ struct m31_priv_data { unsigned int nregs; }; -struct m31_phy_regs m31_ipq5332_regs[] = { +static struct m31_phy_regs m31_ipq5332_regs[] = { { USB_PHY_CFG0, UTMI_PHY_OVERRIDE_EN, @@ -172,8 +172,7 @@ static int m31usb_phy_init(struct phy *phy) ret = clk_prepare_enable(qphy->clk); if (ret) { - if (qphy->vreg) - regulator_disable(qphy->vreg); + regulator_disable(qphy->vreg); dev_err(&phy->dev, "failed to enable cfg ahb clock, %d\n", ret); return ret; } @@ -256,7 +255,7 @@ static int m31usb_phy_probe(struct platform_device *pdev) qphy->vreg = devm_regulator_get(dev, "vdda-phy"); if (IS_ERR(qphy->vreg)) - return dev_err_probe(dev, PTR_ERR(qphy->phy), + return dev_err_probe(dev, PTR_ERR(qphy->vreg), "failed to get vreg\n"); phy_set_drvdata(qphy->phy, qphy); diff --git a/drivers/phy/qualcomm/phy-qcom-qmp-combo.c b/drivers/phy/qualcomm/phy-qcom-qmp-combo.c index cbb28afce135..5e6fc8103e9d 100644 --- a/drivers/phy/qualcomm/phy-qcom-qmp-combo.c +++ b/drivers/phy/qualcomm/phy-qcom-qmp-combo.c @@ -859,10 +859,10 @@ static const struct qmp_phy_init_tbl sm8550_usb3_pcs_tbl[] = { QMP_PHY_INIT_CFG(QPHY_USB_V6_PCS_PCS_TX_RX_CONFIG, 0x0c), QMP_PHY_INIT_CFG(QPHY_USB_V6_PCS_EQ_CONFIG1, 0x4b), QMP_PHY_INIT_CFG(QPHY_USB_V6_PCS_EQ_CONFIG5, 0x10), - QMP_PHY_INIT_CFG(QPHY_USB_V6_PCS_USB3_POWER_STATE_CONFIG1, 0x68), }; static const struct qmp_phy_init_tbl sm8550_usb3_pcs_usb_tbl[] = { + QMP_PHY_INIT_CFG(QPHY_USB_V6_PCS_USB3_POWER_STATE_CONFIG1, 0x68), QMP_PHY_INIT_CFG(QPHY_USB_V6_PCS_USB3_LFPS_DET_HIGH_COUNT_VAL, 0xf8), QMP_PHY_INIT_CFG(QPHY_USB_V6_PCS_USB3_RXEQTRAINING_DFE_TIME_S2, 0x07), QMP_PHY_INIT_CFG(QPHY_USB_V6_PCS_USB3_RCVR_DTCT_DLY_U3_L, 0x40), @@ -2555,6 +2555,7 @@ static int qmp_combo_usb_power_on(struct phy *phy) void __iomem *tx2 = qmp->tx2; void __iomem *rx2 = qmp->rx2; void __iomem *pcs = qmp->pcs; + void __iomem *pcs_usb = qmp->pcs_usb; void __iomem *status; unsigned int val; int ret; @@ -2576,6 +2577,9 @@ static int qmp_combo_usb_power_on(struct phy *phy) qmp_combo_configure(pcs, cfg->pcs_tbl, cfg->pcs_tbl_num); + if (pcs_usb) + qmp_combo_configure(pcs_usb, cfg->pcs_usb_tbl, cfg->pcs_usb_tbl_num); + if (cfg->has_pwrdn_delay) usleep_range(10, 20); diff --git a/drivers/phy/qualcomm/phy-qcom-qmp-pcs-usb-v6.h b/drivers/phy/qualcomm/phy-qcom-qmp-pcs-usb-v6.h index 9510e63ba9d8..c38530d6776b 100644 --- a/drivers/phy/qualcomm/phy-qcom-qmp-pcs-usb-v6.h +++ b/drivers/phy/qualcomm/phy-qcom-qmp-pcs-usb-v6.h @@ -12,7 +12,7 @@ #define QPHY_USB_V6_PCS_LOCK_DETECT_CONFIG3 0xcc #define QPHY_USB_V6_PCS_LOCK_DETECT_CONFIG6 0xd8 #define QPHY_USB_V6_PCS_REFGEN_REQ_CONFIG1 0xdc -#define QPHY_USB_V6_PCS_USB3_POWER_STATE_CONFIG1 0x90 +#define QPHY_USB_V6_PCS_POWER_STATE_CONFIG1 0x90 #define QPHY_USB_V6_PCS_RX_SIGDET_LVL 0x188 #define QPHY_USB_V6_PCS_RCVR_DTCT_DLY_P1U2_L 0x190 #define QPHY_USB_V6_PCS_RCVR_DTCT_DLY_P1U2_H 0x194 @@ -23,6 +23,7 @@ #define QPHY_USB_V6_PCS_EQ_CONFIG1 0x1dc #define QPHY_USB_V6_PCS_EQ_CONFIG5 0x1ec +#define QPHY_USB_V6_PCS_USB3_POWER_STATE_CONFIG1 0x00 #define QPHY_USB_V6_PCS_USB3_LFPS_DET_HIGH_COUNT_VAL 0x18 #define QPHY_USB_V6_PCS_USB3_RXEQTRAINING_DFE_TIME_S2 0x3c #define QPHY_USB_V6_PCS_USB3_RCVR_DTCT_DLY_U3_L 0x40 diff --git a/drivers/phy/qualcomm/phy-qcom-qmp-usb.c b/drivers/phy/qualcomm/phy-qcom-qmp-usb.c index 0130bb8e809a..c69577601ae0 100644 --- a/drivers/phy/qualcomm/phy-qcom-qmp-usb.c +++ b/drivers/phy/qualcomm/phy-qcom-qmp-usb.c @@ -1112,8 +1112,6 @@ static const struct qmp_phy_init_tbl sc8280xp_usb3_uniphy_pcs_tbl[] = { QMP_PHY_INIT_CFG(QPHY_V5_PCS_RCVR_DTCT_DLY_P1U2_H, 0x03), QMP_PHY_INIT_CFG(QPHY_V5_PCS_RX_SIGDET_LVL, 0xaa), QMP_PHY_INIT_CFG(QPHY_V5_PCS_PCS_TX_RX_CONFIG, 0x0c), - QMP_PHY_INIT_CFG(QPHY_V5_PCS_USB3_RXEQTRAINING_DFE_TIME_S2, 0x07), - QMP_PHY_INIT_CFG(QPHY_V5_PCS_USB3_LFPS_DET_HIGH_COUNT_VAL, 0xf8), QMP_PHY_INIT_CFG(QPHY_V5_PCS_CDR_RESET_TIME, 0x0a), QMP_PHY_INIT_CFG(QPHY_V5_PCS_ALIGN_DETECT_CONFIG1, 0x88), QMP_PHY_INIT_CFG(QPHY_V5_PCS_ALIGN_DETECT_CONFIG2, 0x13), @@ -1122,6 +1120,11 @@ static const struct qmp_phy_init_tbl sc8280xp_usb3_uniphy_pcs_tbl[] = { QMP_PHY_INIT_CFG(QPHY_V5_PCS_REFGEN_REQ_CONFIG1, 0x21), }; +static const struct qmp_phy_init_tbl sc8280xp_usb3_uniphy_pcs_usb_tbl[] = { + QMP_PHY_INIT_CFG(QPHY_V5_PCS_USB3_RXEQTRAINING_DFE_TIME_S2, 0x07), + QMP_PHY_INIT_CFG(QPHY_V5_PCS_USB3_LFPS_DET_HIGH_COUNT_VAL, 0xf8), +}; + static const struct qmp_phy_init_tbl sa8775p_usb3_uniphy_pcs_tbl[] = { QMP_PHY_INIT_CFG(QPHY_V5_PCS_LOCK_DETECT_CONFIG1, 0xc4), QMP_PHY_INIT_CFG(QPHY_V5_PCS_LOCK_DETECT_CONFIG2, 0x89), @@ -1131,9 +1134,6 @@ static const struct qmp_phy_init_tbl sa8775p_usb3_uniphy_pcs_tbl[] = { QMP_PHY_INIT_CFG(QPHY_V5_PCS_RCVR_DTCT_DLY_P1U2_H, 0x03), QMP_PHY_INIT_CFG(QPHY_V5_PCS_RX_SIGDET_LVL, 0xaa), QMP_PHY_INIT_CFG(QPHY_V5_PCS_PCS_TX_RX_CONFIG, 0x0c), - QMP_PHY_INIT_CFG(QPHY_V5_PCS_USB3_RXEQTRAINING_DFE_TIME_S2, 0x07), - QMP_PHY_INIT_CFG(QPHY_V5_PCS_USB3_LFPS_DET_HIGH_COUNT_VAL, 0xf8), - QMP_PHY_INIT_CFG(QPHY_V5_PCS_USB3_POWER_STATE_CONFIG1, 0x6f), QMP_PHY_INIT_CFG(QPHY_V5_PCS_CDR_RESET_TIME, 0x0a), QMP_PHY_INIT_CFG(QPHY_V5_PCS_ALIGN_DETECT_CONFIG1, 0x88), QMP_PHY_INIT_CFG(QPHY_V5_PCS_ALIGN_DETECT_CONFIG2, 0x13), @@ -1142,6 +1142,12 @@ static const struct qmp_phy_init_tbl sa8775p_usb3_uniphy_pcs_tbl[] = { QMP_PHY_INIT_CFG(QPHY_V5_PCS_REFGEN_REQ_CONFIG1, 0x21), }; +static const struct qmp_phy_init_tbl sa8775p_usb3_uniphy_pcs_usb_tbl[] = { + QMP_PHY_INIT_CFG(QPHY_V5_PCS_USB3_RXEQTRAINING_DFE_TIME_S2, 0x07), + QMP_PHY_INIT_CFG(QPHY_V5_PCS_USB3_LFPS_DET_HIGH_COUNT_VAL, 0xf8), + QMP_PHY_INIT_CFG(QPHY_V5_PCS_USB3_POWER_STATE_CONFIG1, 0x6f), +}; + struct qmp_usb_offsets { u16 serdes; u16 pcs; @@ -1383,6 +1389,8 @@ static const struct qmp_phy_cfg sa8775p_usb3_uniphy_cfg = { .rx_tbl_num = ARRAY_SIZE(sc8280xp_usb3_uniphy_rx_tbl), .pcs_tbl = sa8775p_usb3_uniphy_pcs_tbl, .pcs_tbl_num = ARRAY_SIZE(sa8775p_usb3_uniphy_pcs_tbl), + .pcs_usb_tbl = sa8775p_usb3_uniphy_pcs_usb_tbl, + .pcs_usb_tbl_num = ARRAY_SIZE(sa8775p_usb3_uniphy_pcs_usb_tbl), .clk_list = qmp_v4_phy_clk_l, .num_clks = ARRAY_SIZE(qmp_v4_phy_clk_l), .reset_list = qcm2290_usb3phy_reset_l, @@ -1405,6 +1413,8 @@ static const struct qmp_phy_cfg sc8280xp_usb3_uniphy_cfg = { .rx_tbl_num = ARRAY_SIZE(sc8280xp_usb3_uniphy_rx_tbl), .pcs_tbl = sc8280xp_usb3_uniphy_pcs_tbl, .pcs_tbl_num = ARRAY_SIZE(sc8280xp_usb3_uniphy_pcs_tbl), + .pcs_usb_tbl = sc8280xp_usb3_uniphy_pcs_usb_tbl, + .pcs_usb_tbl_num = ARRAY_SIZE(sc8280xp_usb3_uniphy_pcs_usb_tbl), .clk_list = qmp_v4_phy_clk_l, .num_clks = ARRAY_SIZE(qmp_v4_phy_clk_l), .reset_list = qcm2290_usb3phy_reset_l, @@ -1703,6 +1713,7 @@ static int qmp_usb_power_on(struct phy *phy) void __iomem *tx = qmp->tx; void __iomem *rx = qmp->rx; void __iomem *pcs = qmp->pcs; + void __iomem *pcs_usb = qmp->pcs_usb; void __iomem *status; unsigned int val; int ret; @@ -1726,6 +1737,9 @@ static int qmp_usb_power_on(struct phy *phy) qmp_usb_configure(pcs, cfg->pcs_tbl, cfg->pcs_tbl_num); + if (pcs_usb) + qmp_usb_configure(pcs_usb, cfg->pcs_usb_tbl, cfg->pcs_usb_tbl_num); + if (cfg->has_pwrdn_delay) usleep_range(10, 20); diff --git a/drivers/phy/realtek/Kconfig b/drivers/phy/realtek/Kconfig index 650e20ed69af..75ac7e7c31ae 100644 --- a/drivers/phy/realtek/Kconfig +++ b/drivers/phy/realtek/Kconfig @@ -2,6 +2,9 @@ # # Phy drivers for Realtek platforms # + +if ARCH_REALTEK || COMPILE_TEST + config PHY_RTK_RTD_USB2PHY tristate "Realtek RTD USB2 PHY Transceiver Driver" depends on USB_SUPPORT @@ -25,3 +28,5 @@ config PHY_RTK_RTD_USB3PHY The DHC (digital home center) RTD series SoCs used the Synopsys DWC3 USB IP. This driver will do the PHY initialization of the parameters. + +endif # ARCH_REALTEK || COMPILE_TEST diff --git a/drivers/phy/realtek/phy-rtk-usb2.c b/drivers/phy/realtek/phy-rtk-usb2.c index 5e7ee060b404..aedc78bd37f7 100644 --- a/drivers/phy/realtek/phy-rtk-usb2.c +++ b/drivers/phy/realtek/phy-rtk-usb2.c @@ -853,17 +853,11 @@ static inline void create_debug_files(struct rtk_phy *rtk_phy) rtk_phy->debug_dir = debugfs_create_dir(dev_name(rtk_phy->dev), phy_debug_root); - if (!rtk_phy->debug_dir) - return; - if (!debugfs_create_file("parameter", 0444, rtk_phy->debug_dir, rtk_phy, - &rtk_usb2_parameter_fops)) - goto file_error; + debugfs_create_file("parameter", 0444, rtk_phy->debug_dir, rtk_phy, + &rtk_usb2_parameter_fops); return; - -file_error: - debugfs_remove_recursive(rtk_phy->debug_dir); } static inline void remove_debug_files(struct rtk_phy *rtk_phy) diff --git a/drivers/phy/realtek/phy-rtk-usb3.c b/drivers/phy/realtek/phy-rtk-usb3.c index 7881f908aade..dfb3122f3f11 100644 --- a/drivers/phy/realtek/phy-rtk-usb3.c +++ b/drivers/phy/realtek/phy-rtk-usb3.c @@ -416,17 +416,11 @@ static inline void create_debug_files(struct rtk_phy *rtk_phy) return; rtk_phy->debug_dir = debugfs_create_dir(dev_name(rtk_phy->dev), phy_debug_root); - if (!rtk_phy->debug_dir) - return; - if (!debugfs_create_file("parameter", 0444, rtk_phy->debug_dir, rtk_phy, - &rtk_usb3_parameter_fops)) - goto file_error; + debugfs_create_file("parameter", 0444, rtk_phy->debug_dir, rtk_phy, + &rtk_usb3_parameter_fops); return; - -file_error: - debugfs_remove_recursive(rtk_phy->debug_dir); } static inline void remove_debug_files(struct rtk_phy *rtk_phy) diff --git a/drivers/pinctrl/nuvoton/pinctrl-wpcm450.c b/drivers/pinctrl/nuvoton/pinctrl-wpcm450.c index 2d1c1652cfd9..8a9961ac8712 100644 --- a/drivers/pinctrl/nuvoton/pinctrl-wpcm450.c +++ b/drivers/pinctrl/nuvoton/pinctrl-wpcm450.c @@ -1062,13 +1062,13 @@ static int wpcm450_gpio_register(struct platform_device *pdev, if (ret < 0) return ret; - gpio = &pctrl->gpio_bank[reg]; - gpio->pctrl = pctrl; - if (reg >= WPCM450_NUM_BANKS) return dev_err_probe(dev, -EINVAL, "GPIO index %d out of range!\n", reg); + gpio = &pctrl->gpio_bank[reg]; + gpio->pctrl = pctrl; + bank = &wpcm450_banks[reg]; gpio->bank = bank; diff --git a/drivers/pinctrl/pinctrl-lantiq.h b/drivers/pinctrl/pinctrl-lantiq.h index efb25fc34f14..1ac49ae638de 100644 --- a/drivers/pinctrl/pinctrl-lantiq.h +++ b/drivers/pinctrl/pinctrl-lantiq.h @@ -198,5 +198,4 @@ enum ltq_pin { extern int ltq_pinctrl_register(struct platform_device *pdev, struct ltq_pinmux_info *info); -extern int ltq_pinctrl_unregister(struct platform_device *pdev); #endif /* __PINCTRL_LANTIQ_H */ diff --git a/drivers/pinctrl/qcom/pinctrl-lpass-lpi.c b/drivers/pinctrl/qcom/pinctrl-lpass-lpi.c index e5a418026ba3..0b2839d27fd6 100644 --- a/drivers/pinctrl/qcom/pinctrl-lpass-lpi.c +++ b/drivers/pinctrl/qcom/pinctrl-lpass-lpi.c @@ -32,7 +32,8 @@ struct lpi_pinctrl { char __iomem *tlmm_base; char __iomem *slew_base; struct clk_bulk_data clks[MAX_LPI_NUM_CLKS]; - struct mutex slew_access_lock; + /* Protects from concurrent register updates */ + struct mutex lock; DECLARE_BITMAP(ever_gpio, MAX_NR_GPIO); const struct lpi_pinctrl_variant_data *data; }; @@ -103,6 +104,7 @@ static int lpi_gpio_set_mux(struct pinctrl_dev *pctldev, unsigned int function, if (WARN_ON(i == g->nfuncs)) return -EINVAL; + mutex_lock(&pctrl->lock); val = lpi_gpio_read(pctrl, pin, LPI_GPIO_CFG_REG); /* @@ -128,6 +130,7 @@ static int lpi_gpio_set_mux(struct pinctrl_dev *pctldev, unsigned int function, u32p_replace_bits(&val, i, LPI_GPIO_FUNCTION_MASK); lpi_gpio_write(pctrl, pin, LPI_GPIO_CFG_REG, val); + mutex_unlock(&pctrl->lock); return 0; } @@ -233,14 +236,14 @@ static int lpi_config_set(struct pinctrl_dev *pctldev, unsigned int group, if (slew_offset == LPI_NO_SLEW) break; - mutex_lock(&pctrl->slew_access_lock); + mutex_lock(&pctrl->lock); sval = ioread32(pctrl->slew_base + LPI_SLEW_RATE_CTL_REG); sval &= ~(LPI_SLEW_RATE_MASK << slew_offset); sval |= arg << slew_offset; iowrite32(sval, pctrl->slew_base + LPI_SLEW_RATE_CTL_REG); - mutex_unlock(&pctrl->slew_access_lock); + mutex_unlock(&pctrl->lock); break; default: return -EINVAL; @@ -256,6 +259,7 @@ static int lpi_config_set(struct pinctrl_dev *pctldev, unsigned int group, lpi_gpio_write(pctrl, group, LPI_GPIO_VALUE_REG, val); } + mutex_lock(&pctrl->lock); val = lpi_gpio_read(pctrl, group, LPI_GPIO_CFG_REG); u32p_replace_bits(&val, pullup, LPI_GPIO_PULL_MASK); @@ -264,6 +268,7 @@ static int lpi_config_set(struct pinctrl_dev *pctldev, unsigned int group, u32p_replace_bits(&val, output_enabled, LPI_GPIO_OE_MASK); lpi_gpio_write(pctrl, group, LPI_GPIO_CFG_REG, val); + mutex_unlock(&pctrl->lock); return 0; } @@ -461,7 +466,7 @@ int lpi_pinctrl_probe(struct platform_device *pdev) pctrl->chip.label = dev_name(dev); pctrl->chip.can_sleep = false; - mutex_init(&pctrl->slew_access_lock); + mutex_init(&pctrl->lock); pctrl->ctrl = devm_pinctrl_register(dev, &pctrl->desc, pctrl); if (IS_ERR(pctrl->ctrl)) { @@ -483,7 +488,7 @@ int lpi_pinctrl_probe(struct platform_device *pdev) return 0; err_pinctrl: - mutex_destroy(&pctrl->slew_access_lock); + mutex_destroy(&pctrl->lock); clk_bulk_disable_unprepare(MAX_LPI_NUM_CLKS, pctrl->clks); return ret; @@ -495,7 +500,7 @@ int lpi_pinctrl_remove(struct platform_device *pdev) struct lpi_pinctrl *pctrl = platform_get_drvdata(pdev); int i; - mutex_destroy(&pctrl->slew_access_lock); + mutex_destroy(&pctrl->lock); clk_bulk_disable_unprepare(MAX_LPI_NUM_CLKS, pctrl->clks); for (i = 0; i < pctrl->data->npins; i++) diff --git a/drivers/pinctrl/renesas/Kconfig b/drivers/pinctrl/renesas/Kconfig index 77730dc548ed..c8d519ca53eb 100644 --- a/drivers/pinctrl/renesas/Kconfig +++ b/drivers/pinctrl/renesas/Kconfig @@ -235,6 +235,7 @@ config PINCTRL_RZN1 depends on OF depends on ARCH_RZN1 || COMPILE_TEST select GENERIC_PINCONF + select PINMUX help This selects pinctrl driver for Renesas RZ/N1 devices. diff --git a/drivers/pinctrl/starfive/pinctrl-starfive-jh7110-aon.c b/drivers/pinctrl/starfive/pinctrl-starfive-jh7110-aon.c index 4bfe3aa57f8a..cf42e204cbf0 100644 --- a/drivers/pinctrl/starfive/pinctrl-starfive-jh7110-aon.c +++ b/drivers/pinctrl/starfive/pinctrl-starfive-jh7110-aon.c @@ -31,6 +31,8 @@ #define JH7110_AON_NGPIO 4 #define JH7110_AON_GC_BASE 64 +#define JH7110_AON_REGS_NUM 37 + /* registers */ #define JH7110_AON_DOEN 0x0 #define JH7110_AON_DOUT 0x4 @@ -145,6 +147,7 @@ static const struct jh7110_pinctrl_soc_info jh7110_aon_pinctrl_info = { .gpi_mask = GENMASK(3, 0), .gpioin_reg_base = JH7110_AON_GPIOIN, .irq_reg = &jh7110_aon_irq_reg, + .nsaved_regs = JH7110_AON_REGS_NUM, .jh7110_set_one_pin_mux = jh7110_aon_set_one_pin_mux, .jh7110_get_padcfg_base = jh7110_aon_get_padcfg_base, .jh7110_gpio_irq_handler = jh7110_aon_irq_handler, @@ -165,6 +168,7 @@ static struct platform_driver jh7110_aon_pinctrl_driver = { .driver = { .name = "starfive-jh7110-aon-pinctrl", .of_match_table = jh7110_aon_pinctrl_of_match, + .pm = pm_sleep_ptr(&jh7110_pinctrl_pm_ops), }, }; module_platform_driver(jh7110_aon_pinctrl_driver); diff --git a/drivers/pinctrl/starfive/pinctrl-starfive-jh7110-sys.c b/drivers/pinctrl/starfive/pinctrl-starfive-jh7110-sys.c index 20c85db1cd3a..03c2ad808d61 100644 --- a/drivers/pinctrl/starfive/pinctrl-starfive-jh7110-sys.c +++ b/drivers/pinctrl/starfive/pinctrl-starfive-jh7110-sys.c @@ -31,6 +31,8 @@ #define JH7110_SYS_NGPIO 64 #define JH7110_SYS_GC_BASE 0 +#define JH7110_SYS_REGS_NUM 174 + /* registers */ #define JH7110_SYS_DOEN 0x000 #define JH7110_SYS_DOUT 0x040 @@ -417,6 +419,7 @@ static const struct jh7110_pinctrl_soc_info jh7110_sys_pinctrl_info = { .gpi_mask = GENMASK(6, 0), .gpioin_reg_base = JH7110_SYS_GPIOIN, .irq_reg = &jh7110_sys_irq_reg, + .nsaved_regs = JH7110_SYS_REGS_NUM, .jh7110_set_one_pin_mux = jh7110_sys_set_one_pin_mux, .jh7110_get_padcfg_base = jh7110_sys_get_padcfg_base, .jh7110_gpio_irq_handler = jh7110_sys_irq_handler, @@ -437,6 +440,7 @@ static struct platform_driver jh7110_sys_pinctrl_driver = { .driver = { .name = "starfive-jh7110-sys-pinctrl", .of_match_table = jh7110_sys_pinctrl_of_match, + .pm = pm_sleep_ptr(&jh7110_pinctrl_pm_ops), }, }; module_platform_driver(jh7110_sys_pinctrl_driver); diff --git a/drivers/pinctrl/starfive/pinctrl-starfive-jh7110.c b/drivers/pinctrl/starfive/pinctrl-starfive-jh7110.c index b9081805c8f6..640f827a9b2c 100644 --- a/drivers/pinctrl/starfive/pinctrl-starfive-jh7110.c +++ b/drivers/pinctrl/starfive/pinctrl-starfive-jh7110.c @@ -872,6 +872,13 @@ int jh7110_pinctrl_probe(struct platform_device *pdev) if (!sfp) return -ENOMEM; +#if IS_ENABLED(CONFIG_PM_SLEEP) + sfp->saved_regs = devm_kcalloc(dev, info->nsaved_regs, + sizeof(*sfp->saved_regs), GFP_KERNEL); + if (!sfp->saved_regs) + return -ENOMEM; +#endif + sfp->base = devm_platform_ioremap_resource(pdev, 0); if (IS_ERR(sfp->base)) return PTR_ERR(sfp->base); @@ -967,14 +974,45 @@ int jh7110_pinctrl_probe(struct platform_device *pdev) if (ret) return dev_err_probe(dev, ret, "could not register gpiochip\n"); - irq_domain_set_pm_device(sfp->gc.irq.domain, dev); - dev_info(dev, "StarFive GPIO chip registered %d GPIOs\n", sfp->gc.ngpio); return pinctrl_enable(sfp->pctl); } EXPORT_SYMBOL_GPL(jh7110_pinctrl_probe); +static int jh7110_pinctrl_suspend(struct device *dev) +{ + struct jh7110_pinctrl *sfp = dev_get_drvdata(dev); + unsigned long flags; + unsigned int i; + + raw_spin_lock_irqsave(&sfp->lock, flags); + for (i = 0 ; i < sfp->info->nsaved_regs ; i++) + sfp->saved_regs[i] = readl_relaxed(sfp->base + 4 * i); + + raw_spin_unlock_irqrestore(&sfp->lock, flags); + return 0; +} + +static int jh7110_pinctrl_resume(struct device *dev) +{ + struct jh7110_pinctrl *sfp = dev_get_drvdata(dev); + unsigned long flags; + unsigned int i; + + raw_spin_lock_irqsave(&sfp->lock, flags); + for (i = 0 ; i < sfp->info->nsaved_regs ; i++) + writel_relaxed(sfp->saved_regs[i], sfp->base + 4 * i); + + raw_spin_unlock_irqrestore(&sfp->lock, flags); + return 0; +} + +const struct dev_pm_ops jh7110_pinctrl_pm_ops = { + LATE_SYSTEM_SLEEP_PM_OPS(jh7110_pinctrl_suspend, jh7110_pinctrl_resume) +}; +EXPORT_SYMBOL_GPL(jh7110_pinctrl_pm_ops); + MODULE_DESCRIPTION("Pinctrl driver for the StarFive JH7110 SoC"); MODULE_AUTHOR("Emil Renner Berthing "); MODULE_AUTHOR("Jianlong Huang "); diff --git a/drivers/pinctrl/starfive/pinctrl-starfive-jh7110.h b/drivers/pinctrl/starfive/pinctrl-starfive-jh7110.h index 3f20b7ff96dd..a33d0d4e1382 100644 --- a/drivers/pinctrl/starfive/pinctrl-starfive-jh7110.h +++ b/drivers/pinctrl/starfive/pinctrl-starfive-jh7110.h @@ -21,6 +21,7 @@ struct jh7110_pinctrl { /* register read/write mutex */ struct mutex mutex; const struct jh7110_pinctrl_soc_info *info; + u32 *saved_regs; }; struct jh7110_gpio_irq_reg { @@ -50,6 +51,8 @@ struct jh7110_pinctrl_soc_info { const struct jh7110_gpio_irq_reg *irq_reg; + unsigned int nsaved_regs; + /* generic pinmux */ int (*jh7110_set_one_pin_mux)(struct jh7110_pinctrl *sfp, unsigned int pin, @@ -66,5 +69,6 @@ void jh7110_set_gpiomux(struct jh7110_pinctrl *sfp, unsigned int pin, unsigned int din, u32 dout, u32 doen); int jh7110_pinctrl_probe(struct platform_device *pdev); struct jh7110_pinctrl *jh7110_from_irq_desc(struct irq_desc *desc); +extern const struct dev_pm_ops jh7110_pinctrl_pm_ops; #endif /* __PINCTRL_STARFIVE_JH7110_H__ */ diff --git a/drivers/pinctrl/tegra/pinctrl-tegra.c b/drivers/pinctrl/tegra/pinctrl-tegra.c index cfeda5b3e048..734c71ef005b 100644 --- a/drivers/pinctrl/tegra/pinctrl-tegra.c +++ b/drivers/pinctrl/tegra/pinctrl-tegra.c @@ -96,7 +96,6 @@ static const struct cfg_param { {"nvidia,slew-rate-falling", TEGRA_PINCONF_PARAM_SLEW_RATE_FALLING}, {"nvidia,slew-rate-rising", TEGRA_PINCONF_PARAM_SLEW_RATE_RISING}, {"nvidia,drive-type", TEGRA_PINCONF_PARAM_DRIVE_TYPE}, - {"nvidia,function", TEGRA_PINCONF_PARAM_FUNCTION}, }; static int tegra_pinctrl_dt_subnode_to_map(struct pinctrl_dev *pctldev, @@ -471,12 +470,6 @@ static int tegra_pinconf_reg(struct tegra_pmx *pmx, *bit = g->drvtype_bit; *width = 2; break; - case TEGRA_PINCONF_PARAM_FUNCTION: - *bank = g->mux_bank; - *reg = g->mux_reg; - *bit = g->mux_bit; - *width = 2; - break; default: dev_err(pmx->dev, "Invalid config param %04x\n", param); return -ENOTSUPP; @@ -640,16 +633,8 @@ static void tegra_pinconf_group_dbg_show(struct pinctrl_dev *pctldev, val >>= bit; val &= (1 << width) - 1; - if (cfg_params[i].param == TEGRA_PINCONF_PARAM_FUNCTION) { - u8 idx = pmx->soc->groups[group].funcs[val]; - - seq_printf(s, "\n\t%s=%s", - strip_prefix(cfg_params[i].property), - pmx->functions[idx].name); - } else { - seq_printf(s, "\n\t%s=%u", - strip_prefix(cfg_params[i].property), val); - } + seq_printf(s, "\n\t%s=%u", + strip_prefix(cfg_params[i].property), val); } } diff --git a/drivers/pinctrl/tegra/pinctrl-tegra.h b/drivers/pinctrl/tegra/pinctrl-tegra.h index e728efeaa4de..b3289bdf727d 100644 --- a/drivers/pinctrl/tegra/pinctrl-tegra.h +++ b/drivers/pinctrl/tegra/pinctrl-tegra.h @@ -54,8 +54,6 @@ enum tegra_pinconf_param { TEGRA_PINCONF_PARAM_SLEW_RATE_RISING, /* argument: Integer, range is HW-dependant */ TEGRA_PINCONF_PARAM_DRIVE_TYPE, - /* argument: pinmux settings */ - TEGRA_PINCONF_PARAM_FUNCTION, }; enum tegra_pinconf_pull { diff --git a/drivers/platform/mellanox/Kconfig b/drivers/platform/mellanox/Kconfig index 382793e73a60..f7dfa0e785fd 100644 --- a/drivers/platform/mellanox/Kconfig +++ b/drivers/platform/mellanox/Kconfig @@ -60,6 +60,7 @@ config MLXBF_BOOTCTL tristate "Mellanox BlueField Firmware Boot Control driver" depends on ARM64 depends on ACPI + depends on NET help The Mellanox BlueField firmware implements functionality to request swapping the primary and alternate eMMC boot partition, @@ -80,8 +81,8 @@ config MLXBF_PMC config NVSW_SN2201 tristate "Nvidia SN2201 platform driver support" - depends on HWMON - depends on I2C + depends on HWMON && I2C + depends on ACPI || COMPILE_TEST select REGMAP_I2C help This driver provides support for the Nvidia SN2201 platform. diff --git a/drivers/platform/mellanox/mlxbf-pmc.c b/drivers/platform/mellanox/mlxbf-pmc.c index be967d797c28..2d4bbe99959e 100644 --- a/drivers/platform/mellanox/mlxbf-pmc.c +++ b/drivers/platform/mellanox/mlxbf-pmc.c @@ -191,6 +191,7 @@ static const struct mlxbf_pmc_events mlxbf_pmc_smgen_events[] = { }; static const struct mlxbf_pmc_events mlxbf_pmc_trio_events_1[] = { + { 0x0, "DISABLE" }, { 0xa0, "TPIO_DATA_BEAT" }, { 0xa1, "TDMA_DATA_BEAT" }, { 0xa2, "MAP_DATA_BEAT" }, @@ -214,6 +215,7 @@ static const struct mlxbf_pmc_events mlxbf_pmc_trio_events_1[] = { }; static const struct mlxbf_pmc_events mlxbf_pmc_trio_events_2[] = { + { 0x0, "DISABLE" }, { 0xa0, "TPIO_DATA_BEAT" }, { 0xa1, "TDMA_DATA_BEAT" }, { 0xa2, "MAP_DATA_BEAT" }, @@ -246,6 +248,7 @@ static const struct mlxbf_pmc_events mlxbf_pmc_trio_events_2[] = { }; static const struct mlxbf_pmc_events mlxbf_pmc_ecc_events[] = { + { 0x0, "DISABLE" }, { 0x100, "ECC_SINGLE_ERROR_CNT" }, { 0x104, "ECC_DOUBLE_ERROR_CNT" }, { 0x114, "SERR_INJ" }, @@ -258,6 +261,7 @@ static const struct mlxbf_pmc_events mlxbf_pmc_ecc_events[] = { }; static const struct mlxbf_pmc_events mlxbf_pmc_mss_events[] = { + { 0x0, "DISABLE" }, { 0xc0, "RXREQ_MSS" }, { 0xc1, "RXDAT_MSS" }, { 0xc2, "TXRSP_MSS" }, @@ -265,6 +269,7 @@ static const struct mlxbf_pmc_events mlxbf_pmc_mss_events[] = { }; static const struct mlxbf_pmc_events mlxbf_pmc_hnf_events[] = { + { 0x0, "DISABLE" }, { 0x45, "HNF_REQUESTS" }, { 0x46, "HNF_REJECTS" }, { 0x47, "ALL_BUSY" }, @@ -323,6 +328,7 @@ static const struct mlxbf_pmc_events mlxbf_pmc_hnf_events[] = { }; static const struct mlxbf_pmc_events mlxbf_pmc_hnfnet_events[] = { + { 0x0, "DISABLE" }, { 0x12, "CDN_REQ" }, { 0x13, "DDN_REQ" }, { 0x14, "NDN_REQ" }, @@ -892,7 +898,7 @@ static int mlxbf_pmc_read_event(int blk_num, uint32_t cnt_num, bool is_l3, uint64_t *result) { uint32_t perfcfg_offset, perfval_offset; - uint64_t perfmon_cfg, perfevt, perfctl; + uint64_t perfmon_cfg, perfevt; if (cnt_num >= pmc->block[blk_num].counters) return -EINVAL; @@ -904,25 +910,6 @@ static int mlxbf_pmc_read_event(int blk_num, uint32_t cnt_num, bool is_l3, perfval_offset = perfcfg_offset + pmc->block[blk_num].counters * MLXBF_PMC_REG_SIZE; - /* Set counter in "read" mode */ - perfmon_cfg = FIELD_PREP(MLXBF_PMC_PERFMON_CONFIG_ADDR, - MLXBF_PMC_PERFCTL); - perfmon_cfg |= FIELD_PREP(MLXBF_PMC_PERFMON_CONFIG_STROBE, 1); - perfmon_cfg |= FIELD_PREP(MLXBF_PMC_PERFMON_CONFIG_WR_R_B, 0); - - if (mlxbf_pmc_write(pmc->block[blk_num].mmio_base + perfcfg_offset, - MLXBF_PMC_WRITE_REG_64, perfmon_cfg)) - return -EFAULT; - - /* Check if the counter is enabled */ - - if (mlxbf_pmc_read(pmc->block[blk_num].mmio_base + perfval_offset, - MLXBF_PMC_READ_REG_64, &perfctl)) - return -EFAULT; - - if (!FIELD_GET(MLXBF_PMC_PERFCTL_EN0, perfctl)) - return -EINVAL; - /* Set counter in "read" mode */ perfmon_cfg = FIELD_PREP(MLXBF_PMC_PERFMON_CONFIG_ADDR, MLXBF_PMC_PERFEVT); @@ -1008,7 +995,7 @@ static ssize_t mlxbf_pmc_counter_show(struct device *dev, } else return -EINVAL; - return sprintf(buf, "0x%llx\n", value); + return sysfs_emit(buf, "0x%llx\n", value); } /* Store function for "counter" sysfs files */ @@ -1078,13 +1065,13 @@ static ssize_t mlxbf_pmc_event_show(struct device *dev, err = mlxbf_pmc_read_event(blk_num, cnt_num, is_l3, &evt_num); if (err) - return sprintf(buf, "No event being monitored\n"); + return sysfs_emit(buf, "No event being monitored\n"); evt_name = mlxbf_pmc_get_event_name(pmc->block_name[blk_num], evt_num); if (!evt_name) return -EINVAL; - return sprintf(buf, "0x%llx: %s\n", evt_num, evt_name); + return sysfs_emit(buf, "0x%llx: %s\n", evt_num, evt_name); } /* Store function for "event" sysfs files */ @@ -1139,9 +1126,9 @@ static ssize_t mlxbf_pmc_event_list_show(struct device *dev, return -EINVAL; for (i = 0, buf[0] = '\0'; i < size; ++i) { - len += sprintf(e_info, "0x%x: %s\n", events[i].evt_num, - events[i].evt_name); - if (len > PAGE_SIZE) + len += snprintf(e_info, sizeof(e_info), "0x%x: %s\n", + events[i].evt_num, events[i].evt_name); + if (len >= PAGE_SIZE) break; strcat(buf, e_info); ret = len; @@ -1168,7 +1155,7 @@ static ssize_t mlxbf_pmc_enable_show(struct device *dev, value = FIELD_GET(MLXBF_PMC_L3C_PERF_CNT_CFG_EN, perfcnt_cfg); - return sprintf(buf, "%d\n", value); + return sysfs_emit(buf, "%d\n", value); } /* Store function for "enable" sysfs files - only for l3cache */ diff --git a/drivers/platform/mellanox/mlxbf-tmfifo.c b/drivers/platform/mellanox/mlxbf-tmfifo.c index b600b77d91ef..ab7d7a1235b8 100644 --- a/drivers/platform/mellanox/mlxbf-tmfifo.c +++ b/drivers/platform/mellanox/mlxbf-tmfifo.c @@ -53,12 +53,13 @@ struct mlxbf_tmfifo; /** - * mlxbf_tmfifo_vring - Structure of the TmFifo virtual ring + * struct mlxbf_tmfifo_vring - Structure of the TmFifo virtual ring * @va: virtual address of the ring * @dma: dma address of the ring * @vq: pointer to the virtio virtqueue * @desc: current descriptor of the pending packet * @desc_head: head descriptor of the pending packet + * @drop_desc: dummy desc for packet dropping * @cur_len: processed length of the current descriptor * @rem_len: remaining length of the pending packet * @pkt_len: total length of the pending packet @@ -75,6 +76,7 @@ struct mlxbf_tmfifo_vring { struct virtqueue *vq; struct vring_desc *desc; struct vring_desc *desc_head; + struct vring_desc drop_desc; int cur_len; int rem_len; u32 pkt_len; @@ -86,6 +88,14 @@ struct mlxbf_tmfifo_vring { struct mlxbf_tmfifo *fifo; }; +/* Check whether vring is in drop mode. */ +#define IS_VRING_DROP(_r) ({ \ + typeof(_r) (r) = (_r); \ + (r->desc_head == &r->drop_desc ? true : false); }) + +/* A stub length to drop maximum length packet. */ +#define VRING_DROP_DESC_MAX_LEN GENMASK(15, 0) + /* Interrupt types. */ enum { MLXBF_TM_RX_LWM_IRQ, @@ -103,12 +113,13 @@ enum { }; /** - * mlxbf_tmfifo_vdev - Structure of the TmFifo virtual device + * struct mlxbf_tmfifo_vdev - Structure of the TmFifo virtual device * @vdev: virtio device, in which the vdev.id.device field has the * VIRTIO_ID_xxx id to distinguish the virtual device. * @status: status of the device * @features: supported features of the device * @vrings: array of tmfifo vrings of this device + * @config: non-anonymous union for cons and net * @config.cons: virtual console config - * select if vdev.id.device is VIRTIO_ID_CONSOLE * @config.net: virtual network config - @@ -128,7 +139,7 @@ struct mlxbf_tmfifo_vdev { }; /** - * mlxbf_tmfifo_irq_info - Structure of the interrupt information + * struct mlxbf_tmfifo_irq_info - Structure of the interrupt information * @fifo: pointer to the tmfifo structure * @irq: interrupt number * @index: index into the interrupt array @@ -140,7 +151,7 @@ struct mlxbf_tmfifo_irq_info { }; /** - * mlxbf_tmfifo_io - Structure of the TmFifo IO resource (for both rx & tx) + * struct mlxbf_tmfifo_io - Structure of the TmFifo IO resource (for both rx & tx) * @ctl: control register offset (TMFIFO_RX_CTL / TMFIFO_TX_CTL) * @sts: status register offset (TMFIFO_RX_STS / TMFIFO_TX_STS) * @data: data register offset (TMFIFO_RX_DATA / TMFIFO_TX_DATA) @@ -152,7 +163,7 @@ struct mlxbf_tmfifo_io { }; /** - * mlxbf_tmfifo - Structure of the TmFifo + * struct mlxbf_tmfifo - Structure of the TmFifo * @vdev: array of the virtual devices running over the TmFifo * @lock: lock to protect the TmFifo access * @res0: mapped resource block 0 @@ -188,7 +199,7 @@ struct mlxbf_tmfifo { }; /** - * mlxbf_tmfifo_msg_hdr - Structure of the TmFifo message header + * struct mlxbf_tmfifo_msg_hdr - Structure of the TmFifo message header * @type: message type * @len: payload length in network byte order. Messages sent into the FIFO * will be read by the other side as data stream in the same byte order. @@ -198,6 +209,7 @@ struct mlxbf_tmfifo { struct mlxbf_tmfifo_msg_hdr { u8 type; __be16 len; + /* private: */ u8 unused[5]; } __packed __aligned(sizeof(u64)); @@ -214,7 +226,7 @@ static u8 mlxbf_tmfifo_net_default_mac[ETH_ALEN] = { static efi_char16_t mlxbf_tmfifo_efi_name[] = L"RshimMacAddr"; /* Maximum L2 header length. */ -#define MLXBF_TMFIFO_NET_L2_OVERHEAD 36 +#define MLXBF_TMFIFO_NET_L2_OVERHEAD (ETH_HLEN + VLAN_HLEN) /* Supported virtio-net features. */ #define MLXBF_TMFIFO_NET_FEATURES \ @@ -262,6 +274,7 @@ static int mlxbf_tmfifo_alloc_vrings(struct mlxbf_tmfifo *fifo, vring->align = SMP_CACHE_BYTES; vring->index = i; vring->vdev_id = tm_vdev->vdev.id.device; + vring->drop_desc.len = VRING_DROP_DESC_MAX_LEN; dev = &tm_vdev->vdev.dev; size = vring_size(vring->num, vring->align); @@ -367,7 +380,7 @@ static u32 mlxbf_tmfifo_get_pkt_len(struct mlxbf_tmfifo_vring *vring, return len; } -static void mlxbf_tmfifo_release_pending_pkt(struct mlxbf_tmfifo_vring *vring) +static void mlxbf_tmfifo_release_pkt(struct mlxbf_tmfifo_vring *vring) { struct vring_desc *desc_head; u32 len = 0; @@ -596,19 +609,26 @@ static void mlxbf_tmfifo_rxtx_word(struct mlxbf_tmfifo_vring *vring, if (vring->cur_len + sizeof(u64) <= len) { /* The whole word. */ - if (is_rx) - memcpy(addr + vring->cur_len, &data, sizeof(u64)); - else - memcpy(&data, addr + vring->cur_len, sizeof(u64)); + if (is_rx) { + if (!IS_VRING_DROP(vring)) + memcpy(addr + vring->cur_len, &data, + sizeof(u64)); + } else { + memcpy(&data, addr + vring->cur_len, + sizeof(u64)); + } vring->cur_len += sizeof(u64); } else { /* Leftover bytes. */ - if (is_rx) - memcpy(addr + vring->cur_len, &data, - len - vring->cur_len); - else + if (is_rx) { + if (!IS_VRING_DROP(vring)) + memcpy(addr + vring->cur_len, &data, + len - vring->cur_len); + } else { + data = 0; memcpy(&data, addr + vring->cur_len, len - vring->cur_len); + } vring->cur_len = len; } @@ -625,13 +645,14 @@ static void mlxbf_tmfifo_rxtx_word(struct mlxbf_tmfifo_vring *vring, * flag is set. */ static void mlxbf_tmfifo_rxtx_header(struct mlxbf_tmfifo_vring *vring, - struct vring_desc *desc, + struct vring_desc **desc, bool is_rx, bool *vring_change) { struct mlxbf_tmfifo *fifo = vring->fifo; struct virtio_net_config *config; struct mlxbf_tmfifo_msg_hdr hdr; int vdev_id, hdr_len; + bool drop_rx = false; /* Read/Write packet header. */ if (is_rx) { @@ -651,8 +672,8 @@ static void mlxbf_tmfifo_rxtx_header(struct mlxbf_tmfifo_vring *vring, if (ntohs(hdr.len) > __virtio16_to_cpu(virtio_legacy_is_little_endian(), config->mtu) + - MLXBF_TMFIFO_NET_L2_OVERHEAD) - return; + MLXBF_TMFIFO_NET_L2_OVERHEAD) + drop_rx = true; } else { vdev_id = VIRTIO_ID_CONSOLE; hdr_len = 0; @@ -667,16 +688,25 @@ static void mlxbf_tmfifo_rxtx_header(struct mlxbf_tmfifo_vring *vring, if (!tm_dev2) return; - vring->desc = desc; + vring->desc = *desc; vring = &tm_dev2->vrings[MLXBF_TMFIFO_VRING_RX]; *vring_change = true; } + + if (drop_rx && !IS_VRING_DROP(vring)) { + if (vring->desc_head) + mlxbf_tmfifo_release_pkt(vring); + *desc = &vring->drop_desc; + vring->desc_head = *desc; + vring->desc = *desc; + } + vring->pkt_len = ntohs(hdr.len) + hdr_len; } else { /* Network virtio has an extra header. */ hdr_len = (vring->vdev_id == VIRTIO_ID_NET) ? sizeof(struct virtio_net_hdr) : 0; - vring->pkt_len = mlxbf_tmfifo_get_pkt_len(vring, desc); + vring->pkt_len = mlxbf_tmfifo_get_pkt_len(vring, *desc); hdr.type = (vring->vdev_id == VIRTIO_ID_NET) ? VIRTIO_ID_NET : VIRTIO_ID_CONSOLE; hdr.len = htons(vring->pkt_len - hdr_len); @@ -709,15 +739,23 @@ static bool mlxbf_tmfifo_rxtx_one_desc(struct mlxbf_tmfifo_vring *vring, /* Get the descriptor of the next packet. */ if (!vring->desc) { desc = mlxbf_tmfifo_get_next_pkt(vring, is_rx); - if (!desc) - return false; + if (!desc) { + /* Drop next Rx packet to avoid stuck. */ + if (is_rx) { + desc = &vring->drop_desc; + vring->desc_head = desc; + vring->desc = desc; + } else { + return false; + } + } } else { desc = vring->desc; } /* Beginning of a packet. Start to Rx/Tx packet header. */ if (vring->pkt_len == 0) { - mlxbf_tmfifo_rxtx_header(vring, desc, is_rx, &vring_change); + mlxbf_tmfifo_rxtx_header(vring, &desc, is_rx, &vring_change); (*avail)--; /* Return if new packet is for another ring. */ @@ -743,17 +781,24 @@ static bool mlxbf_tmfifo_rxtx_one_desc(struct mlxbf_tmfifo_vring *vring, vring->rem_len -= len; /* Get the next desc on the chain. */ - if (vring->rem_len > 0 && + if (!IS_VRING_DROP(vring) && vring->rem_len > 0 && (virtio16_to_cpu(vdev, desc->flags) & VRING_DESC_F_NEXT)) { idx = virtio16_to_cpu(vdev, desc->next); desc = &vr->desc[idx]; goto mlxbf_tmfifo_desc_done; } - /* Done and release the pending packet. */ - mlxbf_tmfifo_release_pending_pkt(vring); + /* Done and release the packet. */ desc = NULL; fifo->vring[is_rx] = NULL; + if (!IS_VRING_DROP(vring)) { + mlxbf_tmfifo_release_pkt(vring); + } else { + vring->pkt_len = 0; + vring->desc_head = NULL; + vring->desc = NULL; + return false; + } /* * Make sure the load/store are in order before @@ -933,7 +978,7 @@ static void mlxbf_tmfifo_virtio_del_vqs(struct virtio_device *vdev) /* Release the pending packet. */ if (vring->desc) - mlxbf_tmfifo_release_pending_pkt(vring); + mlxbf_tmfifo_release_pkt(vring); vq = vring->vq; if (vq) { vring->vq = NULL; diff --git a/drivers/platform/surface/surface_platform_profile.c b/drivers/platform/surface/surface_platform_profile.c index f433a13c3689..a5a3941b3f43 100644 --- a/drivers/platform/surface/surface_platform_profile.c +++ b/drivers/platform/surface/surface_platform_profile.c @@ -159,8 +159,7 @@ static int surface_platform_profile_probe(struct ssam_device *sdev) set_bit(PLATFORM_PROFILE_BALANCED_PERFORMANCE, tpd->handler.choices); set_bit(PLATFORM_PROFILE_PERFORMANCE, tpd->handler.choices); - platform_profile_register(&tpd->handler); - return 0; + return platform_profile_register(&tpd->handler); } static void surface_platform_profile_remove(struct ssam_device *sdev) diff --git a/drivers/platform/x86/apple-gmux.c b/drivers/platform/x86/apple-gmux.c index cadbb557a108..1417e230edbd 100644 --- a/drivers/platform/x86/apple-gmux.c +++ b/drivers/platform/x86/apple-gmux.c @@ -105,6 +105,8 @@ struct apple_gmux_config { #define GMUX_BRIGHTNESS_MASK 0x00ffffff #define GMUX_MAX_BRIGHTNESS GMUX_BRIGHTNESS_MASK +# define MMIO_GMUX_MAX_BRIGHTNESS 0xffff + static u8 gmux_pio_read8(struct apple_gmux_data *gmux_data, int port) { return inb(gmux_data->iostart + port); @@ -857,7 +859,17 @@ get_version: memset(&props, 0, sizeof(props)); props.type = BACKLIGHT_PLATFORM; - props.max_brightness = gmux_read32(gmux_data, GMUX_PORT_MAX_BRIGHTNESS); + + /* + * All MMIO gmux's have 0xffff as max brightness, but some iMacs incorrectly + * report 0x03ff, despite the firmware being happy to set 0xffff as the brightness + * at boot. Force 0xffff for all MMIO gmux's so they all have the correct brightness + * range. + */ + if (type == APPLE_GMUX_TYPE_MMIO) + props.max_brightness = MMIO_GMUX_MAX_BRIGHTNESS; + else + props.max_brightness = gmux_read32(gmux_data, GMUX_PORT_MAX_BRIGHTNESS); #if IS_REACHABLE(CONFIG_ACPI_VIDEO) register_bdev = acpi_video_get_backlight_type() == acpi_backlight_apple_gmux; diff --git a/drivers/platform/x86/asus-nb-wmi.c b/drivers/platform/x86/asus-nb-wmi.c index fdf7da06af30..df1db54d4e18 100644 --- a/drivers/platform/x86/asus-nb-wmi.c +++ b/drivers/platform/x86/asus-nb-wmi.c @@ -478,6 +478,15 @@ static const struct dmi_system_id asus_quirks[] = { }, .driver_data = &quirk_asus_tablet_mode, }, + { + .callback = dmi_matched, + .ident = "ASUS ROG FLOW X16", + .matches = { + DMI_MATCH(DMI_SYS_VENDOR, "ASUSTeK COMPUTER INC."), + DMI_MATCH(DMI_PRODUCT_NAME, "GV601V"), + }, + .driver_data = &quirk_asus_tablet_mode, + }, { .callback = dmi_matched, .ident = "ASUS VivoBook E410MA", @@ -522,6 +531,9 @@ static void asus_nb_wmi_quirks(struct asus_wmi_driver *driver) static const struct key_entry asus_nb_wmi_keymap[] = { { KE_KEY, ASUS_WMI_BRN_DOWN, { KEY_BRIGHTNESSDOWN } }, { KE_KEY, ASUS_WMI_BRN_UP, { KEY_BRIGHTNESSUP } }, + { KE_KEY, 0x2a, { KEY_SELECTIVE_SCREENSHOT } }, + { KE_IGNORE, 0x2b, }, /* PrintScreen (also send via PS/2) on newer models */ + { KE_IGNORE, 0x2c, }, /* CapsLock (also send via PS/2) on newer models */ { KE_KEY, 0x30, { KEY_VOLUMEUP } }, { KE_KEY, 0x31, { KEY_VOLUMEDOWN } }, { KE_KEY, 0x32, { KEY_MUTE } }, diff --git a/drivers/platform/x86/asus-wmi.c b/drivers/platform/x86/asus-wmi.c index 9f8cea5f9615..19bfd30861aa 100644 --- a/drivers/platform/x86/asus-wmi.c +++ b/drivers/platform/x86/asus-wmi.c @@ -3826,7 +3826,6 @@ static void asus_wmi_handle_event_code(int code, struct asus_wmi *asus) { unsigned int key_value = 1; bool autorelease = 1; - int orig_code = code; if (asus->driver->key_filter) { asus->driver->key_filter(asus->driver, &code, &key_value, @@ -3835,16 +3834,10 @@ static void asus_wmi_handle_event_code(int code, struct asus_wmi *asus) return; } - if (code >= NOTIFY_BRNUP_MIN && code <= NOTIFY_BRNUP_MAX) - code = ASUS_WMI_BRN_UP; - else if (code >= NOTIFY_BRNDOWN_MIN && code <= NOTIFY_BRNDOWN_MAX) - code = ASUS_WMI_BRN_DOWN; - - if (code == ASUS_WMI_BRN_DOWN || code == ASUS_WMI_BRN_UP) { - if (acpi_video_get_backlight_type() == acpi_backlight_vendor) { - asus_wmi_backlight_notify(asus, orig_code); - return; - } + if (acpi_video_get_backlight_type() == acpi_backlight_vendor && + code >= NOTIFY_BRNUP_MIN && code <= NOTIFY_BRNDOWN_MAX) { + asus_wmi_backlight_notify(asus, code); + return; } if (code == NOTIFY_KBD_BRTUP) { diff --git a/drivers/platform/x86/asus-wmi.h b/drivers/platform/x86/asus-wmi.h index a478ebfd34df..fc41d1b1bb7f 100644 --- a/drivers/platform/x86/asus-wmi.h +++ b/drivers/platform/x86/asus-wmi.h @@ -18,7 +18,7 @@ #include #define ASUS_WMI_KEY_IGNORE (-1) -#define ASUS_WMI_BRN_DOWN 0x20 +#define ASUS_WMI_BRN_DOWN 0x2e #define ASUS_WMI_BRN_UP 0x2f struct module; diff --git a/drivers/platform/x86/hp/hp-bioscfg/bioscfg.c b/drivers/platform/x86/hp/hp-bioscfg/bioscfg.c index 8c4f9e12f018..5798b49ddaba 100644 --- a/drivers/platform/x86/hp/hp-bioscfg/bioscfg.c +++ b/drivers/platform/x86/hp/hp-bioscfg/bioscfg.c @@ -659,7 +659,7 @@ static int hp_init_bios_package_attribute(enum hp_wmi_data_type attr_type, const char *guid, int min_elements, int instance_id) { - struct kobject *attr_name_kobj; + struct kobject *attr_name_kobj, *duplicate; union acpi_object *elements; struct kset *temp_kset; @@ -704,8 +704,11 @@ static int hp_init_bios_package_attribute(enum hp_wmi_data_type attr_type, } /* All duplicate attributes found are ignored */ - if (kset_find_obj(temp_kset, str_value)) { + duplicate = kset_find_obj(temp_kset, str_value); + if (duplicate) { pr_debug("Duplicate attribute name found - %s\n", str_value); + /* kset_find_obj() returns a reference */ + kobject_put(duplicate); goto pack_attr_exit; } @@ -768,7 +771,7 @@ static int hp_init_bios_buffer_attribute(enum hp_wmi_data_type attr_type, const char *guid, int min_elements, int instance_id) { - struct kobject *attr_name_kobj; + struct kobject *attr_name_kobj, *duplicate; struct kset *temp_kset; char str[MAX_BUFF_SIZE]; @@ -794,8 +797,11 @@ static int hp_init_bios_buffer_attribute(enum hp_wmi_data_type attr_type, temp_kset = bioscfg_drv.main_dir_kset; /* All duplicate attributes found are ignored */ - if (kset_find_obj(temp_kset, str)) { + duplicate = kset_find_obj(temp_kset, str); + if (duplicate) { pr_debug("Duplicate attribute name found - %s\n", str); + /* kset_find_obj() returns a reference */ + kobject_put(duplicate); goto buff_attr_exit; } diff --git a/drivers/platform/x86/hp/hp-wmi.c b/drivers/platform/x86/hp/hp-wmi.c index e76e5458db35..8ebb7be52ee7 100644 --- a/drivers/platform/x86/hp/hp-wmi.c +++ b/drivers/platform/x86/hp/hp-wmi.c @@ -1548,7 +1548,13 @@ static const struct dev_pm_ops hp_wmi_pm_ops = { .restore = hp_wmi_resume_handler, }; -static struct platform_driver hp_wmi_driver = { +/* + * hp_wmi_bios_remove() lives in .exit.text. For drivers registered via + * module_platform_driver_probe() this is ok because they cannot get unbound at + * runtime. So mark the driver struct with __refdata to prevent modpost + * triggering a section mismatch warning. + */ +static struct platform_driver hp_wmi_driver __refdata = { .driver = { .name = "hp-wmi", .pm = &hp_wmi_pm_ops, diff --git a/drivers/platform/x86/intel/ifs/runtest.c b/drivers/platform/x86/intel/ifs/runtest.c index 1061eb7ec399..43c864add778 100644 --- a/drivers/platform/x86/intel/ifs/runtest.c +++ b/drivers/platform/x86/intel/ifs/runtest.c @@ -331,14 +331,15 @@ int do_core_test(int cpu, struct device *dev) switch (test->test_num) { case IFS_TYPE_SAF: if (!ifsd->loaded) - return -EPERM; - ifs_test_core(cpu, dev); + ret = -EPERM; + else + ifs_test_core(cpu, dev); break; case IFS_TYPE_ARRAY_BIST: ifs_array_test_core(cpu, dev); break; default: - return -EINVAL; + ret = -EINVAL; } out: cpus_read_unlock(); diff --git a/drivers/platform/x86/intel/uncore-frequency/uncore-frequency-common.c b/drivers/platform/x86/intel/uncore-frequency/uncore-frequency-common.c index 1152deaa0078..33ab207493e3 100644 --- a/drivers/platform/x86/intel/uncore-frequency/uncore-frequency-common.c +++ b/drivers/platform/x86/intel/uncore-frequency/uncore-frequency-common.c @@ -176,7 +176,7 @@ show_uncore_data(initial_max_freq_khz); static int create_attr_group(struct uncore_data *data, char *name) { - int ret, index = 0; + int ret, freq, index = 0; init_attribute_rw(max_freq_khz); init_attribute_rw(min_freq_khz); @@ -197,7 +197,11 @@ static int create_attr_group(struct uncore_data *data, char *name) data->uncore_attrs[index++] = &data->min_freq_khz_dev_attr.attr; data->uncore_attrs[index++] = &data->initial_min_freq_khz_dev_attr.attr; data->uncore_attrs[index++] = &data->initial_max_freq_khz_dev_attr.attr; - data->uncore_attrs[index++] = &data->current_freq_khz_dev_attr.attr; + + ret = uncore_read_freq(data, &freq); + if (!ret) + data->uncore_attrs[index++] = &data->current_freq_khz_dev_attr.attr; + data->uncore_attrs[index] = NULL; data->uncore_attr_group.name = name; diff --git a/drivers/platform/x86/intel_scu_ipc.c b/drivers/platform/x86/intel_scu_ipc.c index 6851d10d6582..a68df4133403 100644 --- a/drivers/platform/x86/intel_scu_ipc.c +++ b/drivers/platform/x86/intel_scu_ipc.c @@ -19,6 +19,7 @@ #include #include #include +#include #include #include @@ -231,19 +232,15 @@ static inline u32 ipc_data_readl(struct intel_scu_ipc_dev *scu, u32 offset) /* Wait till scu status is busy */ static inline int busy_loop(struct intel_scu_ipc_dev *scu) { - unsigned long end = jiffies + IPC_TIMEOUT; - - do { - u32 status; - - status = ipc_read_status(scu); - if (!(status & IPC_STATUS_BUSY)) - return (status & IPC_STATUS_ERR) ? -EIO : 0; + u8 status; + int err; - usleep_range(50, 100); - } while (time_before(jiffies, end)); + err = readx_poll_timeout(ipc_read_status, scu, status, !(status & IPC_STATUS_BUSY), + 100, jiffies_to_usecs(IPC_TIMEOUT)); + if (err) + return err; - return -ETIMEDOUT; + return (status & IPC_STATUS_ERR) ? -EIO : 0; } /* Wait till ipc ioc interrupt is received or timeout in 10 HZ */ @@ -251,10 +248,12 @@ static inline int ipc_wait_for_interrupt(struct intel_scu_ipc_dev *scu) { int status; - if (!wait_for_completion_timeout(&scu->cmd_complete, IPC_TIMEOUT)) - return -ETIMEDOUT; + wait_for_completion_timeout(&scu->cmd_complete, IPC_TIMEOUT); status = ipc_read_status(scu); + if (status & IPC_STATUS_BUSY) + return -ETIMEDOUT; + if (status & IPC_STATUS_ERR) return -EIO; @@ -266,6 +265,24 @@ static int intel_scu_ipc_check_status(struct intel_scu_ipc_dev *scu) return scu->irq > 0 ? ipc_wait_for_interrupt(scu) : busy_loop(scu); } +static struct intel_scu_ipc_dev *intel_scu_ipc_get(struct intel_scu_ipc_dev *scu) +{ + u8 status; + + if (!scu) + scu = ipcdev; + if (!scu) + return ERR_PTR(-ENODEV); + + status = ipc_read_status(scu); + if (status & IPC_STATUS_BUSY) { + dev_dbg(&scu->dev, "device is busy\n"); + return ERR_PTR(-EBUSY); + } + + return scu; +} + /* Read/Write power control(PMIC in Langwell, MSIC in PenWell) registers */ static int pwr_reg_rdwr(struct intel_scu_ipc_dev *scu, u16 *addr, u8 *data, u32 count, u32 op, u32 id) @@ -279,11 +296,10 @@ static int pwr_reg_rdwr(struct intel_scu_ipc_dev *scu, u16 *addr, u8 *data, memset(cbuf, 0, sizeof(cbuf)); mutex_lock(&ipclock); - if (!scu) - scu = ipcdev; - if (!scu) { + scu = intel_scu_ipc_get(scu); + if (IS_ERR(scu)) { mutex_unlock(&ipclock); - return -ENODEV; + return PTR_ERR(scu); } for (nc = 0; nc < count; nc++, offset += 2) { @@ -438,13 +454,12 @@ int intel_scu_ipc_dev_simple_command(struct intel_scu_ipc_dev *scu, int cmd, int err; mutex_lock(&ipclock); - if (!scu) - scu = ipcdev; - if (!scu) { + scu = intel_scu_ipc_get(scu); + if (IS_ERR(scu)) { mutex_unlock(&ipclock); - return -ENODEV; + return PTR_ERR(scu); } - scu = ipcdev; + cmdval = sub << 12 | cmd; ipc_command(scu, cmdval); err = intel_scu_ipc_check_status(scu); @@ -484,11 +499,10 @@ int intel_scu_ipc_dev_command_with_size(struct intel_scu_ipc_dev *scu, int cmd, return -EINVAL; mutex_lock(&ipclock); - if (!scu) - scu = ipcdev; - if (!scu) { + scu = intel_scu_ipc_get(scu); + if (IS_ERR(scu)) { mutex_unlock(&ipclock); - return -ENODEV; + return PTR_ERR(scu); } memcpy(inbuf, in, inlen); diff --git a/drivers/platform/x86/mlx-platform.c b/drivers/platform/x86/mlx-platform.c index 3d96dbf79a72..a2ffe4157df1 100644 --- a/drivers/platform/x86/mlx-platform.c +++ b/drivers/platform/x86/mlx-platform.c @@ -6514,6 +6514,7 @@ static int mlxplat_i2c_main_init(struct mlxplat_priv *priv) return 0; fail_mlxplat_i2c_mux_topology_init: + platform_device_unregister(priv->pdev_i2c); fail_platform_i2c_register: fail_mlxplat_mlxcpld_verify_bus_topology: return err; @@ -6521,6 +6522,7 @@ fail_mlxplat_mlxcpld_verify_bus_topology: static void mlxplat_i2c_main_exit(struct mlxplat_priv *priv) { + mlxplat_pre_exit(priv); mlxplat_i2c_mux_topology_exit(priv); if (priv->pdev_i2c) platform_device_unregister(priv->pdev_i2c); @@ -6597,7 +6599,7 @@ static int mlxplat_probe(struct platform_device *pdev) fail_register_reboot_notifier: fail_regcache_sync: - mlxplat_pre_exit(priv); + mlxplat_i2c_main_exit(priv); fail_mlxplat_i2c_main_init: fail_regmap_write: fail_alloc: @@ -6614,7 +6616,6 @@ static int mlxplat_remove(struct platform_device *pdev) pm_power_off = NULL; if (mlxplat_reboot_nb) unregister_reboot_notifier(mlxplat_reboot_nb); - mlxplat_pre_exit(priv); mlxplat_i2c_main_exit(priv); mlxplat_post_exit(); return 0; diff --git a/drivers/platform/x86/msi-ec.c b/drivers/platform/x86/msi-ec.c index f26a3121092f..492eb383ee7a 100644 --- a/drivers/platform/x86/msi-ec.c +++ b/drivers/platform/x86/msi-ec.c @@ -276,14 +276,13 @@ static struct msi_ec_conf CONF2 __initdata = { static const char * const ALLOWED_FW_3[] __initconst = { "1592EMS1.111", - "E1592IMS.10C", NULL }; static struct msi_ec_conf CONF3 __initdata = { .allowed_fw = ALLOWED_FW_3, .charge_control = { - .address = 0xef, + .address = 0xd7, .offset_start = 0x8a, .offset_end = 0x80, .range_min = 0x8a, diff --git a/drivers/platform/x86/think-lmi.c b/drivers/platform/x86/think-lmi.c index 79346881cadb..aee869769843 100644 --- a/drivers/platform/x86/think-lmi.c +++ b/drivers/platform/x86/think-lmi.c @@ -1248,6 +1248,24 @@ static void tlmi_release_attr(void) kset_unregister(tlmi_priv.authentication_kset); } +static int tlmi_validate_setting_name(struct kset *attribute_kset, char *name) +{ + struct kobject *duplicate; + + if (!strcmp(name, "Reserved")) + return -EINVAL; + + duplicate = kset_find_obj(attribute_kset, name); + if (duplicate) { + pr_debug("Duplicate attribute name found - %s\n", name); + /* kset_find_obj() returns a reference */ + kobject_put(duplicate); + return -EBUSY; + } + + return 0; +} + static int tlmi_sysfs_init(void) { int i, ret; @@ -1276,10 +1294,8 @@ static int tlmi_sysfs_init(void) continue; /* check for duplicate or reserved values */ - if (kset_find_obj(tlmi_priv.attribute_kset, tlmi_priv.setting[i]->display_name) || - !strcmp(tlmi_priv.setting[i]->display_name, "Reserved")) { - pr_debug("duplicate or reserved attribute name found - %s\n", - tlmi_priv.setting[i]->display_name); + if (tlmi_validate_setting_name(tlmi_priv.attribute_kset, + tlmi_priv.setting[i]->display_name) < 0) { kfree(tlmi_priv.setting[i]->possible_values); kfree(tlmi_priv.setting[i]); tlmi_priv.setting[i] = NULL; diff --git a/drivers/platform/x86/thinkpad_acpi.c b/drivers/platform/x86/thinkpad_acpi.c index d70c89d32534..41584427dc32 100644 --- a/drivers/platform/x86/thinkpad_acpi.c +++ b/drivers/platform/x86/thinkpad_acpi.c @@ -4116,9 +4116,11 @@ static void hotkey_resume(void) { tpacpi_disable_brightness_delay(); + mutex_lock(&hotkey_mutex); if (hotkey_status_set(true) < 0 || hotkey_mask_set(hotkey_acpi_mask) < 0) pr_err("error while attempting to reset the event firmware interface\n"); + mutex_unlock(&hotkey_mutex); tpacpi_send_radiosw_update(); tpacpi_input_send_tabletsw(); diff --git a/drivers/platform/x86/touchscreen_dmi.c b/drivers/platform/x86/touchscreen_dmi.c index f9301a9382e7..0c6733772698 100644 --- a/drivers/platform/x86/touchscreen_dmi.c +++ b/drivers/platform/x86/touchscreen_dmi.c @@ -42,6 +42,21 @@ static const struct ts_dmi_data archos_101_cesium_educ_data = { .properties = archos_101_cesium_educ_props, }; +static const struct property_entry bush_bush_windows_tablet_props[] = { + PROPERTY_ENTRY_U32("touchscreen-size-x", 1850), + PROPERTY_ENTRY_U32("touchscreen-size-y", 1280), + PROPERTY_ENTRY_BOOL("touchscreen-swapped-x-y"), + PROPERTY_ENTRY_U32("silead,max-fingers", 10), + PROPERTY_ENTRY_BOOL("silead,home-button"), + PROPERTY_ENTRY_STRING("firmware-name", "gsl1680-bush-bush-windows-tablet.fw"), + { } +}; + +static const struct ts_dmi_data bush_bush_windows_tablet_data = { + .acpi_name = "MSSL1680:00", + .properties = bush_bush_windows_tablet_props, +}; + static const struct property_entry chuwi_hi8_props[] = { PROPERTY_ENTRY_U32("touchscreen-size-x", 1665), PROPERTY_ENTRY_U32("touchscreen-size-y", 1140), @@ -756,6 +771,21 @@ static const struct ts_dmi_data pipo_w11_data = { .properties = pipo_w11_props, }; +static const struct property_entry positivo_c4128b_props[] = { + PROPERTY_ENTRY_U32("touchscreen-min-x", 4), + PROPERTY_ENTRY_U32("touchscreen-min-y", 13), + PROPERTY_ENTRY_U32("touchscreen-size-x", 1915), + PROPERTY_ENTRY_U32("touchscreen-size-y", 1269), + PROPERTY_ENTRY_STRING("firmware-name", "gsl1680-positivo-c4128b.fw"), + PROPERTY_ENTRY_U32("silead,max-fingers", 10), + { } +}; + +static const struct ts_dmi_data positivo_c4128b_data = { + .acpi_name = "MSSL1680:00", + .properties = positivo_c4128b_props, +}; + static const struct property_entry pov_mobii_wintab_p800w_v20_props[] = { PROPERTY_ENTRY_U32("touchscreen-min-x", 32), PROPERTY_ENTRY_U32("touchscreen-min-y", 16), @@ -1070,6 +1100,13 @@ const struct dmi_system_id touchscreen_dmi_table[] = { DMI_MATCH(DMI_PRODUCT_NAME, "ARCHOS 101 Cesium Educ"), }, }, + { + /* Bush Windows tablet */ + .driver_data = (void *)&bush_bush_windows_tablet_data, + .matches = { + DMI_MATCH(DMI_PRODUCT_NAME, "Bush Windows tablet"), + }, + }, { /* Chuwi Hi8 */ .driver_data = (void *)&chuwi_hi8_data, @@ -1480,6 +1517,14 @@ const struct dmi_system_id touchscreen_dmi_table[] = { DMI_MATCH(DMI_BIOS_VERSION, "MOMO.G.WI71C.MABMRBA02"), }, }, + { + /* Positivo C4128B */ + .driver_data = (void *)&positivo_c4128b_data, + .matches = { + DMI_MATCH(DMI_SYS_VENDOR, "Positivo Tecnologia SA"), + DMI_MATCH(DMI_PRODUCT_NAME, "C4128B-1"), + }, + }, { /* Point of View mobii wintab p800w (v2.0) */ .driver_data = (void *)&pov_mobii_wintab_p800w_v20_data, diff --git a/drivers/pmdomain/Makefile b/drivers/pmdomain/Makefile new file mode 100644 index 000000000000..666753676e5c --- /dev/null +++ b/drivers/pmdomain/Makefile @@ -0,0 +1,17 @@ +# SPDX-License-Identifier: GPL-2.0-only +obj-y += actions/ +obj-y += amlogic/ +obj-y += apple/ +obj-y += bcm/ +obj-y += imx/ +obj-y += mediatek/ +obj-y += qcom/ +obj-y += renesas/ +obj-y += rockchip/ +obj-y += samsung/ +obj-y += st/ +obj-y += starfive/ +obj-y += sunxi/ +obj-y += tegra/ +obj-y += ti/ +obj-y += xilinx/ diff --git a/drivers/pmdomain/actions/Makefile b/drivers/pmdomain/actions/Makefile new file mode 100644 index 000000000000..7e8aa473d12d --- /dev/null +++ b/drivers/pmdomain/actions/Makefile @@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0+ +obj-$(CONFIG_OWL_PM_DOMAINS_HELPER) += owl-sps-helper.o +obj-$(CONFIG_OWL_PM_DOMAINS) += owl-sps.o diff --git a/drivers/pmdomain/actions/owl-sps-helper.c b/drivers/pmdomain/actions/owl-sps-helper.c new file mode 100644 index 000000000000..e3f36603dd53 --- /dev/null +++ b/drivers/pmdomain/actions/owl-sps-helper.c @@ -0,0 +1,48 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* + * Actions Semi Owl Smart Power System (SPS) shared helpers + * + * Copyright 2012 Actions Semi Inc. + * Author: Actions Semi, Inc. + * + * Copyright (c) 2017 Andreas Färber + */ + +#include +#include +#include + +#define OWL_SPS_PG_CTL 0x0 + +int owl_sps_set_pg(void __iomem *base, u32 pwr_mask, u32 ack_mask, bool enable) +{ + u32 val; + bool ack; + int timeout; + + val = readl(base + OWL_SPS_PG_CTL); + ack = val & ack_mask; + if (ack == enable) + return 0; + + if (enable) + val |= pwr_mask; + else + val &= ~pwr_mask; + + writel(val, base + OWL_SPS_PG_CTL); + + for (timeout = 5000; timeout > 0; timeout -= 50) { + val = readl(base + OWL_SPS_PG_CTL); + if ((val & ack_mask) == (enable ? ack_mask : 0)) + break; + udelay(50); + } + if (timeout <= 0) + return -ETIMEDOUT; + + udelay(10); + + return 0; +} +EXPORT_SYMBOL_GPL(owl_sps_set_pg); diff --git a/drivers/pmdomain/actions/owl-sps.c b/drivers/pmdomain/actions/owl-sps.c new file mode 100644 index 000000000000..73a9e0bb7e8e --- /dev/null +++ b/drivers/pmdomain/actions/owl-sps.c @@ -0,0 +1,320 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* + * Actions Semi Owl Smart Power System (SPS) + * + * Copyright 2012 Actions Semi Inc. + * Author: Actions Semi, Inc. + * + * Copyright (c) 2017 Andreas Färber + */ + +#include +#include +#include +#include +#include +#include +#include + +struct owl_sps_domain_info { + const char *name; + int pwr_bit; + int ack_bit; + unsigned int genpd_flags; +}; + +struct owl_sps_info { + unsigned num_domains; + const struct owl_sps_domain_info *domains; +}; + +struct owl_sps { + struct device *dev; + const struct owl_sps_info *info; + void __iomem *base; + struct genpd_onecell_data genpd_data; + struct generic_pm_domain *domains[]; +}; + +#define to_owl_pd(gpd) container_of(gpd, struct owl_sps_domain, genpd) + +struct owl_sps_domain { + struct generic_pm_domain genpd; + const struct owl_sps_domain_info *info; + struct owl_sps *sps; +}; + +static int owl_sps_set_power(struct owl_sps_domain *pd, bool enable) +{ + u32 pwr_mask, ack_mask; + + ack_mask = BIT(pd->info->ack_bit); + pwr_mask = BIT(pd->info->pwr_bit); + + return owl_sps_set_pg(pd->sps->base, pwr_mask, ack_mask, enable); +} + +static int owl_sps_power_on(struct generic_pm_domain *domain) +{ + struct owl_sps_domain *pd = to_owl_pd(domain); + + dev_dbg(pd->sps->dev, "%s power on", pd->info->name); + + return owl_sps_set_power(pd, true); +} + +static int owl_sps_power_off(struct generic_pm_domain *domain) +{ + struct owl_sps_domain *pd = to_owl_pd(domain); + + dev_dbg(pd->sps->dev, "%s power off", pd->info->name); + + return owl_sps_set_power(pd, false); +} + +static int owl_sps_init_domain(struct owl_sps *sps, int index) +{ + struct owl_sps_domain *pd; + + pd = devm_kzalloc(sps->dev, sizeof(*pd), GFP_KERNEL); + if (!pd) + return -ENOMEM; + + pd->info = &sps->info->domains[index]; + pd->sps = sps; + + pd->genpd.name = pd->info->name; + pd->genpd.power_on = owl_sps_power_on; + pd->genpd.power_off = owl_sps_power_off; + pd->genpd.flags = pd->info->genpd_flags; + pm_genpd_init(&pd->genpd, NULL, false); + + sps->genpd_data.domains[index] = &pd->genpd; + + return 0; +} + +static int owl_sps_probe(struct platform_device *pdev) +{ + const struct of_device_id *match; + const struct owl_sps_info *sps_info; + struct owl_sps *sps; + int i, ret; + + if (!pdev->dev.of_node) { + dev_err(&pdev->dev, "no device node\n"); + return -ENODEV; + } + + match = of_match_device(pdev->dev.driver->of_match_table, &pdev->dev); + if (!match || !match->data) { + dev_err(&pdev->dev, "unknown compatible or missing data\n"); + return -EINVAL; + } + + sps_info = match->data; + + sps = devm_kzalloc(&pdev->dev, + struct_size(sps, domains, sps_info->num_domains), + GFP_KERNEL); + if (!sps) + return -ENOMEM; + + sps->base = of_io_request_and_map(pdev->dev.of_node, 0, "owl-sps"); + if (IS_ERR(sps->base)) { + dev_err(&pdev->dev, "failed to map sps registers\n"); + return PTR_ERR(sps->base); + } + + sps->dev = &pdev->dev; + sps->info = sps_info; + sps->genpd_data.domains = sps->domains; + sps->genpd_data.num_domains = sps_info->num_domains; + + for (i = 0; i < sps_info->num_domains; i++) { + ret = owl_sps_init_domain(sps, i); + if (ret) + return ret; + } + + ret = of_genpd_add_provider_onecell(pdev->dev.of_node, &sps->genpd_data); + if (ret) { + dev_err(&pdev->dev, "failed to add provider (%d)", ret); + return ret; + } + + return 0; +} + +static const struct owl_sps_domain_info s500_sps_domains[] = { + [S500_PD_VDE] = { + .name = "VDE", + .pwr_bit = 0, + .ack_bit = 16, + }, + [S500_PD_VCE_SI] = { + .name = "VCE_SI", + .pwr_bit = 1, + .ack_bit = 17, + }, + [S500_PD_USB2_1] = { + .name = "USB2_1", + .pwr_bit = 2, + .ack_bit = 18, + }, + [S500_PD_CPU2] = { + .name = "CPU2", + .pwr_bit = 5, + .ack_bit = 21, + .genpd_flags = GENPD_FLAG_ALWAYS_ON, + }, + [S500_PD_CPU3] = { + .name = "CPU3", + .pwr_bit = 6, + .ack_bit = 22, + .genpd_flags = GENPD_FLAG_ALWAYS_ON, + }, + [S500_PD_DMA] = { + .name = "DMA", + .pwr_bit = 8, + .ack_bit = 12, + }, + [S500_PD_DS] = { + .name = "DS", + .pwr_bit = 9, + .ack_bit = 13, + }, + [S500_PD_USB3] = { + .name = "USB3", + .pwr_bit = 10, + .ack_bit = 14, + }, + [S500_PD_USB2_0] = { + .name = "USB2_0", + .pwr_bit = 11, + .ack_bit = 15, + }, +}; + +static const struct owl_sps_info s500_sps_info = { + .num_domains = ARRAY_SIZE(s500_sps_domains), + .domains = s500_sps_domains, +}; + +static const struct owl_sps_domain_info s700_sps_domains[] = { + [S700_PD_VDE] = { + .name = "VDE", + .pwr_bit = 0, + }, + [S700_PD_VCE_SI] = { + .name = "VCE_SI", + .pwr_bit = 1, + }, + [S700_PD_USB2_1] = { + .name = "USB2_1", + .pwr_bit = 2, + }, + [S700_PD_HDE] = { + .name = "HDE", + .pwr_bit = 7, + }, + [S700_PD_DMA] = { + .name = "DMA", + .pwr_bit = 8, + }, + [S700_PD_DS] = { + .name = "DS", + .pwr_bit = 9, + }, + [S700_PD_USB3] = { + .name = "USB3", + .pwr_bit = 10, + }, + [S700_PD_USB2_0] = { + .name = "USB2_0", + .pwr_bit = 11, + }, +}; + +static const struct owl_sps_info s700_sps_info = { + .num_domains = ARRAY_SIZE(s700_sps_domains), + .domains = s700_sps_domains, +}; + +static const struct owl_sps_domain_info s900_sps_domains[] = { + [S900_PD_GPU_B] = { + .name = "GPU_B", + .pwr_bit = 3, + }, + [S900_PD_VCE] = { + .name = "VCE", + .pwr_bit = 4, + }, + [S900_PD_SENSOR] = { + .name = "SENSOR", + .pwr_bit = 5, + }, + [S900_PD_VDE] = { + .name = "VDE", + .pwr_bit = 6, + }, + [S900_PD_HDE] = { + .name = "HDE", + .pwr_bit = 7, + }, + [S900_PD_USB3] = { + .name = "USB3", + .pwr_bit = 8, + }, + [S900_PD_DDR0] = { + .name = "DDR0", + .pwr_bit = 9, + }, + [S900_PD_DDR1] = { + .name = "DDR1", + .pwr_bit = 10, + }, + [S900_PD_DE] = { + .name = "DE", + .pwr_bit = 13, + }, + [S900_PD_NAND] = { + .name = "NAND", + .pwr_bit = 14, + }, + [S900_PD_USB2_H0] = { + .name = "USB2_H0", + .pwr_bit = 15, + }, + [S900_PD_USB2_H1] = { + .name = "USB2_H1", + .pwr_bit = 16, + }, +}; + +static const struct owl_sps_info s900_sps_info = { + .num_domains = ARRAY_SIZE(s900_sps_domains), + .domains = s900_sps_domains, +}; + +static const struct of_device_id owl_sps_of_matches[] = { + { .compatible = "actions,s500-sps", .data = &s500_sps_info }, + { .compatible = "actions,s700-sps", .data = &s700_sps_info }, + { .compatible = "actions,s900-sps", .data = &s900_sps_info }, + { } +}; + +static struct platform_driver owl_sps_platform_driver = { + .probe = owl_sps_probe, + .driver = { + .name = "owl-sps", + .of_match_table = owl_sps_of_matches, + .suppress_bind_attrs = true, + }, +}; + +static int __init owl_sps_init(void) +{ + return platform_driver_register(&owl_sps_platform_driver); +} +postcore_initcall(owl_sps_init); diff --git a/drivers/pmdomain/amlogic/Makefile b/drivers/pmdomain/amlogic/Makefile new file mode 100644 index 000000000000..3d58abd574f9 --- /dev/null +++ b/drivers/pmdomain/amlogic/Makefile @@ -0,0 +1,4 @@ +# SPDX-License-Identifier: GPL-2.0-only +obj-$(CONFIG_MESON_GX_PM_DOMAINS) += meson-gx-pwrc-vpu.o +obj-$(CONFIG_MESON_EE_PM_DOMAINS) += meson-ee-pwrc.o +obj-$(CONFIG_MESON_SECURE_PM_DOMAINS) += meson-secure-pwrc.o diff --git a/drivers/pmdomain/amlogic/meson-ee-pwrc.c b/drivers/pmdomain/amlogic/meson-ee-pwrc.c new file mode 100644 index 000000000000..cfb796d40d9d --- /dev/null +++ b/drivers/pmdomain/amlogic/meson-ee-pwrc.c @@ -0,0 +1,635 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* + * Copyright (c) 2019 BayLibre, SAS + * Author: Neil Armstrong + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +/* AO Offsets */ + +#define GX_AO_RTI_GEN_PWR_SLEEP0 (0x3a << 2) +#define GX_AO_RTI_GEN_PWR_ISO0 (0x3b << 2) + +/* + * Meson8/Meson8b/Meson8m2 only expose the power management registers of the + * AO-bus as syscon. 0x3a from GX translates to 0x02, 0x3b translates to 0x03 + * and so on. + */ +#define MESON8_AO_RTI_GEN_PWR_SLEEP0 (0x02 << 2) +#define MESON8_AO_RTI_GEN_PWR_ISO0 (0x03 << 2) + +/* HHI Offsets */ + +#define HHI_MEM_PD_REG0 (0x40 << 2) +#define HHI_VPU_MEM_PD_REG0 (0x41 << 2) +#define HHI_VPU_MEM_PD_REG1 (0x42 << 2) +#define HHI_VPU_MEM_PD_REG3 (0x43 << 2) +#define HHI_VPU_MEM_PD_REG4 (0x44 << 2) +#define HHI_AUDIO_MEM_PD_REG0 (0x45 << 2) +#define HHI_NANOQ_MEM_PD_REG0 (0x46 << 2) +#define HHI_NANOQ_MEM_PD_REG1 (0x47 << 2) +#define HHI_VPU_MEM_PD_REG2 (0x4d << 2) + +#define G12A_HHI_NANOQ_MEM_PD_REG0 (0x43 << 2) +#define G12A_HHI_NANOQ_MEM_PD_REG1 (0x44 << 2) + +struct meson_ee_pwrc; +struct meson_ee_pwrc_domain; + +struct meson_ee_pwrc_mem_domain { + unsigned int reg; + unsigned int mask; +}; + +struct meson_ee_pwrc_top_domain { + unsigned int sleep_reg; + unsigned int sleep_mask; + unsigned int iso_reg; + unsigned int iso_mask; +}; + +struct meson_ee_pwrc_domain_desc { + char *name; + unsigned int reset_names_count; + unsigned int clk_names_count; + struct meson_ee_pwrc_top_domain *top_pd; + unsigned int mem_pd_count; + struct meson_ee_pwrc_mem_domain *mem_pd; + bool (*is_powered_off)(struct meson_ee_pwrc_domain *pwrc_domain); +}; + +struct meson_ee_pwrc_domain_data { + unsigned int count; + struct meson_ee_pwrc_domain_desc *domains; +}; + +/* TOP Power Domains */ + +static struct meson_ee_pwrc_top_domain gx_pwrc_vpu = { + .sleep_reg = GX_AO_RTI_GEN_PWR_SLEEP0, + .sleep_mask = BIT(8), + .iso_reg = GX_AO_RTI_GEN_PWR_SLEEP0, + .iso_mask = BIT(9), +}; + +static struct meson_ee_pwrc_top_domain meson8_pwrc_vpu = { + .sleep_reg = MESON8_AO_RTI_GEN_PWR_SLEEP0, + .sleep_mask = BIT(8), + .iso_reg = MESON8_AO_RTI_GEN_PWR_SLEEP0, + .iso_mask = BIT(9), +}; + +#define SM1_EE_PD(__bit) \ + { \ + .sleep_reg = GX_AO_RTI_GEN_PWR_SLEEP0, \ + .sleep_mask = BIT(__bit), \ + .iso_reg = GX_AO_RTI_GEN_PWR_ISO0, \ + .iso_mask = BIT(__bit), \ + } + +static struct meson_ee_pwrc_top_domain sm1_pwrc_vpu = SM1_EE_PD(8); +static struct meson_ee_pwrc_top_domain sm1_pwrc_nna = SM1_EE_PD(16); +static struct meson_ee_pwrc_top_domain sm1_pwrc_usb = SM1_EE_PD(17); +static struct meson_ee_pwrc_top_domain sm1_pwrc_pci = SM1_EE_PD(18); +static struct meson_ee_pwrc_top_domain sm1_pwrc_ge2d = SM1_EE_PD(19); + +static struct meson_ee_pwrc_top_domain g12a_pwrc_nna = { + .sleep_reg = GX_AO_RTI_GEN_PWR_SLEEP0, + .sleep_mask = BIT(16) | BIT(17), + .iso_reg = GX_AO_RTI_GEN_PWR_ISO0, + .iso_mask = BIT(16) | BIT(17), +}; + +/* Memory PD Domains */ + +#define VPU_MEMPD(__reg) \ + { __reg, GENMASK(1, 0) }, \ + { __reg, GENMASK(3, 2) }, \ + { __reg, GENMASK(5, 4) }, \ + { __reg, GENMASK(7, 6) }, \ + { __reg, GENMASK(9, 8) }, \ + { __reg, GENMASK(11, 10) }, \ + { __reg, GENMASK(13, 12) }, \ + { __reg, GENMASK(15, 14) }, \ + { __reg, GENMASK(17, 16) }, \ + { __reg, GENMASK(19, 18) }, \ + { __reg, GENMASK(21, 20) }, \ + { __reg, GENMASK(23, 22) }, \ + { __reg, GENMASK(25, 24) }, \ + { __reg, GENMASK(27, 26) }, \ + { __reg, GENMASK(29, 28) }, \ + { __reg, GENMASK(31, 30) } + +#define VPU_HHI_MEMPD(__reg) \ + { __reg, BIT(8) }, \ + { __reg, BIT(9) }, \ + { __reg, BIT(10) }, \ + { __reg, BIT(11) }, \ + { __reg, BIT(12) }, \ + { __reg, BIT(13) }, \ + { __reg, BIT(14) }, \ + { __reg, BIT(15) } + +static struct meson_ee_pwrc_mem_domain axg_pwrc_mem_vpu[] = { + VPU_MEMPD(HHI_VPU_MEM_PD_REG0), + VPU_HHI_MEMPD(HHI_MEM_PD_REG0), +}; + +static struct meson_ee_pwrc_mem_domain g12a_pwrc_mem_vpu[] = { + VPU_MEMPD(HHI_VPU_MEM_PD_REG0), + VPU_MEMPD(HHI_VPU_MEM_PD_REG1), + VPU_MEMPD(HHI_VPU_MEM_PD_REG2), + VPU_HHI_MEMPD(HHI_MEM_PD_REG0), +}; + +static struct meson_ee_pwrc_mem_domain gxbb_pwrc_mem_vpu[] = { + VPU_MEMPD(HHI_VPU_MEM_PD_REG0), + VPU_MEMPD(HHI_VPU_MEM_PD_REG1), + VPU_HHI_MEMPD(HHI_MEM_PD_REG0), +}; + +static struct meson_ee_pwrc_mem_domain meson_pwrc_mem_eth[] = { + { HHI_MEM_PD_REG0, GENMASK(3, 2) }, +}; + +static struct meson_ee_pwrc_mem_domain meson8_pwrc_audio_dsp_mem[] = { + { HHI_MEM_PD_REG0, GENMASK(1, 0) }, +}; + +static struct meson_ee_pwrc_mem_domain meson8_pwrc_mem_vpu[] = { + VPU_MEMPD(HHI_VPU_MEM_PD_REG0), + VPU_MEMPD(HHI_VPU_MEM_PD_REG1), + VPU_HHI_MEMPD(HHI_MEM_PD_REG0), +}; + +static struct meson_ee_pwrc_mem_domain sm1_pwrc_mem_vpu[] = { + VPU_MEMPD(HHI_VPU_MEM_PD_REG0), + VPU_MEMPD(HHI_VPU_MEM_PD_REG1), + VPU_MEMPD(HHI_VPU_MEM_PD_REG2), + VPU_MEMPD(HHI_VPU_MEM_PD_REG3), + { HHI_VPU_MEM_PD_REG4, GENMASK(1, 0) }, + { HHI_VPU_MEM_PD_REG4, GENMASK(3, 2) }, + { HHI_VPU_MEM_PD_REG4, GENMASK(5, 4) }, + { HHI_VPU_MEM_PD_REG4, GENMASK(7, 6) }, + VPU_HHI_MEMPD(HHI_MEM_PD_REG0), +}; + +static struct meson_ee_pwrc_mem_domain sm1_pwrc_mem_nna[] = { + { HHI_NANOQ_MEM_PD_REG0, 0xff }, + { HHI_NANOQ_MEM_PD_REG1, 0xff }, +}; + +static struct meson_ee_pwrc_mem_domain sm1_pwrc_mem_usb[] = { + { HHI_MEM_PD_REG0, GENMASK(31, 30) }, +}; + +static struct meson_ee_pwrc_mem_domain sm1_pwrc_mem_pcie[] = { + { HHI_MEM_PD_REG0, GENMASK(29, 26) }, +}; + +static struct meson_ee_pwrc_mem_domain sm1_pwrc_mem_ge2d[] = { + { HHI_MEM_PD_REG0, GENMASK(25, 18) }, +}; + +static struct meson_ee_pwrc_mem_domain axg_pwrc_mem_audio[] = { + { HHI_MEM_PD_REG0, GENMASK(5, 4) }, +}; + +static struct meson_ee_pwrc_mem_domain sm1_pwrc_mem_audio[] = { + { HHI_MEM_PD_REG0, GENMASK(5, 4) }, + { HHI_AUDIO_MEM_PD_REG0, GENMASK(1, 0) }, + { HHI_AUDIO_MEM_PD_REG0, GENMASK(3, 2) }, + { HHI_AUDIO_MEM_PD_REG0, GENMASK(5, 4) }, + { HHI_AUDIO_MEM_PD_REG0, GENMASK(7, 6) }, + { HHI_AUDIO_MEM_PD_REG0, GENMASK(13, 12) }, + { HHI_AUDIO_MEM_PD_REG0, GENMASK(15, 14) }, + { HHI_AUDIO_MEM_PD_REG0, GENMASK(17, 16) }, + { HHI_AUDIO_MEM_PD_REG0, GENMASK(19, 18) }, + { HHI_AUDIO_MEM_PD_REG0, GENMASK(21, 20) }, + { HHI_AUDIO_MEM_PD_REG0, GENMASK(23, 22) }, + { HHI_AUDIO_MEM_PD_REG0, GENMASK(25, 24) }, + { HHI_AUDIO_MEM_PD_REG0, GENMASK(27, 26) }, +}; + +static struct meson_ee_pwrc_mem_domain g12a_pwrc_mem_nna[] = { + { G12A_HHI_NANOQ_MEM_PD_REG0, GENMASK(31, 0) }, + { G12A_HHI_NANOQ_MEM_PD_REG1, GENMASK(23, 0) }, +}; + +#define VPU_PD(__name, __top_pd, __mem, __is_pwr_off, __resets, __clks) \ + { \ + .name = __name, \ + .reset_names_count = __resets, \ + .clk_names_count = __clks, \ + .top_pd = __top_pd, \ + .mem_pd_count = ARRAY_SIZE(__mem), \ + .mem_pd = __mem, \ + .is_powered_off = __is_pwr_off, \ + } + +#define TOP_PD(__name, __top_pd, __mem, __is_pwr_off) \ + { \ + .name = __name, \ + .top_pd = __top_pd, \ + .mem_pd_count = ARRAY_SIZE(__mem), \ + .mem_pd = __mem, \ + .is_powered_off = __is_pwr_off, \ + } + +#define MEM_PD(__name, __mem) \ + TOP_PD(__name, NULL, __mem, NULL) + +static bool pwrc_ee_is_powered_off(struct meson_ee_pwrc_domain *pwrc_domain); + +static struct meson_ee_pwrc_domain_desc axg_pwrc_domains[] = { + [PWRC_AXG_VPU_ID] = VPU_PD("VPU", &gx_pwrc_vpu, axg_pwrc_mem_vpu, + pwrc_ee_is_powered_off, 5, 2), + [PWRC_AXG_ETHERNET_MEM_ID] = MEM_PD("ETH", meson_pwrc_mem_eth), + [PWRC_AXG_AUDIO_ID] = MEM_PD("AUDIO", axg_pwrc_mem_audio), +}; + +static struct meson_ee_pwrc_domain_desc g12a_pwrc_domains[] = { + [PWRC_G12A_VPU_ID] = VPU_PD("VPU", &gx_pwrc_vpu, g12a_pwrc_mem_vpu, + pwrc_ee_is_powered_off, 11, 2), + [PWRC_G12A_ETH_ID] = MEM_PD("ETH", meson_pwrc_mem_eth), + [PWRC_G12A_NNA_ID] = TOP_PD("NNA", &g12a_pwrc_nna, g12a_pwrc_mem_nna, + pwrc_ee_is_powered_off), +}; + +static struct meson_ee_pwrc_domain_desc gxbb_pwrc_domains[] = { + [PWRC_GXBB_VPU_ID] = VPU_PD("VPU", &gx_pwrc_vpu, gxbb_pwrc_mem_vpu, + pwrc_ee_is_powered_off, 12, 2), + [PWRC_GXBB_ETHERNET_MEM_ID] = MEM_PD("ETH", meson_pwrc_mem_eth), +}; + +static struct meson_ee_pwrc_domain_desc meson8_pwrc_domains[] = { + [PWRC_MESON8_VPU_ID] = VPU_PD("VPU", &meson8_pwrc_vpu, + meson8_pwrc_mem_vpu, + pwrc_ee_is_powered_off, 0, 1), + [PWRC_MESON8_ETHERNET_MEM_ID] = MEM_PD("ETHERNET_MEM", + meson_pwrc_mem_eth), + [PWRC_MESON8_AUDIO_DSP_MEM_ID] = MEM_PD("AUDIO_DSP_MEM", + meson8_pwrc_audio_dsp_mem), +}; + +static struct meson_ee_pwrc_domain_desc meson8b_pwrc_domains[] = { + [PWRC_MESON8_VPU_ID] = VPU_PD("VPU", &meson8_pwrc_vpu, + meson8_pwrc_mem_vpu, + pwrc_ee_is_powered_off, 11, 1), + [PWRC_MESON8_ETHERNET_MEM_ID] = MEM_PD("ETHERNET_MEM", + meson_pwrc_mem_eth), + [PWRC_MESON8_AUDIO_DSP_MEM_ID] = MEM_PD("AUDIO_DSP_MEM", + meson8_pwrc_audio_dsp_mem), +}; + +static struct meson_ee_pwrc_domain_desc sm1_pwrc_domains[] = { + [PWRC_SM1_VPU_ID] = VPU_PD("VPU", &sm1_pwrc_vpu, sm1_pwrc_mem_vpu, + pwrc_ee_is_powered_off, 11, 2), + [PWRC_SM1_NNA_ID] = TOP_PD("NNA", &sm1_pwrc_nna, sm1_pwrc_mem_nna, + pwrc_ee_is_powered_off), + [PWRC_SM1_USB_ID] = TOP_PD("USB", &sm1_pwrc_usb, sm1_pwrc_mem_usb, + pwrc_ee_is_powered_off), + [PWRC_SM1_PCIE_ID] = TOP_PD("PCI", &sm1_pwrc_pci, sm1_pwrc_mem_pcie, + pwrc_ee_is_powered_off), + [PWRC_SM1_GE2D_ID] = TOP_PD("GE2D", &sm1_pwrc_ge2d, sm1_pwrc_mem_ge2d, + pwrc_ee_is_powered_off), + [PWRC_SM1_AUDIO_ID] = MEM_PD("AUDIO", sm1_pwrc_mem_audio), + [PWRC_SM1_ETH_ID] = MEM_PD("ETH", meson_pwrc_mem_eth), +}; + +struct meson_ee_pwrc_domain { + struct generic_pm_domain base; + bool enabled; + struct meson_ee_pwrc *pwrc; + struct meson_ee_pwrc_domain_desc desc; + struct clk_bulk_data *clks; + int num_clks; + struct reset_control *rstc; + int num_rstc; +}; + +struct meson_ee_pwrc { + struct regmap *regmap_ao; + struct regmap *regmap_hhi; + struct meson_ee_pwrc_domain *domains; + struct genpd_onecell_data xlate; +}; + +static bool pwrc_ee_is_powered_off(struct meson_ee_pwrc_domain *pwrc_domain) +{ + u32 reg; + + regmap_read(pwrc_domain->pwrc->regmap_ao, + pwrc_domain->desc.top_pd->sleep_reg, ®); + + return (reg & pwrc_domain->desc.top_pd->sleep_mask); +} + +static int meson_ee_pwrc_off(struct generic_pm_domain *domain) +{ + struct meson_ee_pwrc_domain *pwrc_domain = + container_of(domain, struct meson_ee_pwrc_domain, base); + int i; + + if (pwrc_domain->desc.top_pd) + regmap_update_bits(pwrc_domain->pwrc->regmap_ao, + pwrc_domain->desc.top_pd->sleep_reg, + pwrc_domain->desc.top_pd->sleep_mask, + pwrc_domain->desc.top_pd->sleep_mask); + udelay(20); + + for (i = 0 ; i < pwrc_domain->desc.mem_pd_count ; ++i) + regmap_update_bits(pwrc_domain->pwrc->regmap_hhi, + pwrc_domain->desc.mem_pd[i].reg, + pwrc_domain->desc.mem_pd[i].mask, + pwrc_domain->desc.mem_pd[i].mask); + + udelay(20); + + if (pwrc_domain->desc.top_pd) + regmap_update_bits(pwrc_domain->pwrc->regmap_ao, + pwrc_domain->desc.top_pd->iso_reg, + pwrc_domain->desc.top_pd->iso_mask, + pwrc_domain->desc.top_pd->iso_mask); + + if (pwrc_domain->num_clks) { + msleep(20); + clk_bulk_disable_unprepare(pwrc_domain->num_clks, + pwrc_domain->clks); + } + + return 0; +} + +static int meson_ee_pwrc_on(struct generic_pm_domain *domain) +{ + struct meson_ee_pwrc_domain *pwrc_domain = + container_of(domain, struct meson_ee_pwrc_domain, base); + int i, ret; + + if (pwrc_domain->desc.top_pd) + regmap_update_bits(pwrc_domain->pwrc->regmap_ao, + pwrc_domain->desc.top_pd->sleep_reg, + pwrc_domain->desc.top_pd->sleep_mask, 0); + udelay(20); + + for (i = 0 ; i < pwrc_domain->desc.mem_pd_count ; ++i) + regmap_update_bits(pwrc_domain->pwrc->regmap_hhi, + pwrc_domain->desc.mem_pd[i].reg, + pwrc_domain->desc.mem_pd[i].mask, 0); + + udelay(20); + + ret = reset_control_assert(pwrc_domain->rstc); + if (ret) + return ret; + + if (pwrc_domain->desc.top_pd) + regmap_update_bits(pwrc_domain->pwrc->regmap_ao, + pwrc_domain->desc.top_pd->iso_reg, + pwrc_domain->desc.top_pd->iso_mask, 0); + + ret = reset_control_deassert(pwrc_domain->rstc); + if (ret) + return ret; + + return clk_bulk_prepare_enable(pwrc_domain->num_clks, + pwrc_domain->clks); +} + +static int meson_ee_pwrc_init_domain(struct platform_device *pdev, + struct meson_ee_pwrc *pwrc, + struct meson_ee_pwrc_domain *dom) +{ + int ret; + + dom->pwrc = pwrc; + dom->num_rstc = dom->desc.reset_names_count; + dom->num_clks = dom->desc.clk_names_count; + + if (dom->num_rstc) { + int count = reset_control_get_count(&pdev->dev); + + if (count != dom->num_rstc) + dev_warn(&pdev->dev, "Invalid resets count %d for domain %s\n", + count, dom->desc.name); + + dom->rstc = devm_reset_control_array_get_exclusive(&pdev->dev); + if (IS_ERR(dom->rstc)) + return PTR_ERR(dom->rstc); + } + + if (dom->num_clks) { + int ret = devm_clk_bulk_get_all(&pdev->dev, &dom->clks); + if (ret < 0) + return ret; + + if (dom->num_clks != ret) { + dev_warn(&pdev->dev, "Invalid clocks count %d for domain %s\n", + ret, dom->desc.name); + dom->num_clks = ret; + } + } + + dom->base.name = dom->desc.name; + dom->base.power_on = meson_ee_pwrc_on; + dom->base.power_off = meson_ee_pwrc_off; + + /* + * TOFIX: This is a special case for the VPU power domain, which can + * be enabled previously by the bootloader. In this case the VPU + * pipeline may be functional but no driver maybe never attach + * to this power domain, and if the domain is disabled it could + * cause system errors. This is why the pm_domain_always_on_gov + * is used here. + * For the same reason, the clocks should be enabled in case + * we need to power the domain off, otherwise the internal clocks + * prepare/enable counters won't be in sync. + */ + if (dom->num_clks && dom->desc.is_powered_off && !dom->desc.is_powered_off(dom)) { + ret = clk_bulk_prepare_enable(dom->num_clks, dom->clks); + if (ret) + return ret; + + dom->base.flags = GENPD_FLAG_ALWAYS_ON; + ret = pm_genpd_init(&dom->base, NULL, false); + if (ret) + return ret; + } else { + ret = pm_genpd_init(&dom->base, NULL, + (dom->desc.is_powered_off ? + dom->desc.is_powered_off(dom) : true)); + if (ret) + return ret; + } + + return 0; +} + +static int meson_ee_pwrc_probe(struct platform_device *pdev) +{ + const struct meson_ee_pwrc_domain_data *match; + struct regmap *regmap_ao, *regmap_hhi; + struct device_node *parent_np; + struct meson_ee_pwrc *pwrc; + int i, ret; + + match = of_device_get_match_data(&pdev->dev); + if (!match) { + dev_err(&pdev->dev, "failed to get match data\n"); + return -ENODEV; + } + + pwrc = devm_kzalloc(&pdev->dev, sizeof(*pwrc), GFP_KERNEL); + if (!pwrc) + return -ENOMEM; + + pwrc->xlate.domains = devm_kcalloc(&pdev->dev, match->count, + sizeof(*pwrc->xlate.domains), + GFP_KERNEL); + if (!pwrc->xlate.domains) + return -ENOMEM; + + pwrc->domains = devm_kcalloc(&pdev->dev, match->count, + sizeof(*pwrc->domains), GFP_KERNEL); + if (!pwrc->domains) + return -ENOMEM; + + pwrc->xlate.num_domains = match->count; + + parent_np = of_get_parent(pdev->dev.of_node); + regmap_hhi = syscon_node_to_regmap(parent_np); + of_node_put(parent_np); + if (IS_ERR(regmap_hhi)) { + dev_err(&pdev->dev, "failed to get HHI regmap\n"); + return PTR_ERR(regmap_hhi); + } + + regmap_ao = syscon_regmap_lookup_by_phandle(pdev->dev.of_node, + "amlogic,ao-sysctrl"); + if (IS_ERR(regmap_ao)) { + dev_err(&pdev->dev, "failed to get AO regmap\n"); + return PTR_ERR(regmap_ao); + } + + pwrc->regmap_ao = regmap_ao; + pwrc->regmap_hhi = regmap_hhi; + + platform_set_drvdata(pdev, pwrc); + + for (i = 0 ; i < match->count ; ++i) { + struct meson_ee_pwrc_domain *dom = &pwrc->domains[i]; + + memcpy(&dom->desc, &match->domains[i], sizeof(dom->desc)); + + ret = meson_ee_pwrc_init_domain(pdev, pwrc, dom); + if (ret) + return ret; + + pwrc->xlate.domains[i] = &dom->base; + } + + return of_genpd_add_provider_onecell(pdev->dev.of_node, &pwrc->xlate); +} + +static void meson_ee_pwrc_shutdown(struct platform_device *pdev) +{ + struct meson_ee_pwrc *pwrc = platform_get_drvdata(pdev); + int i; + + for (i = 0 ; i < pwrc->xlate.num_domains ; ++i) { + struct meson_ee_pwrc_domain *dom = &pwrc->domains[i]; + + if (dom->desc.is_powered_off && !dom->desc.is_powered_off(dom)) + meson_ee_pwrc_off(&dom->base); + } +} + +static struct meson_ee_pwrc_domain_data meson_ee_g12a_pwrc_data = { + .count = ARRAY_SIZE(g12a_pwrc_domains), + .domains = g12a_pwrc_domains, +}; + +static struct meson_ee_pwrc_domain_data meson_ee_axg_pwrc_data = { + .count = ARRAY_SIZE(axg_pwrc_domains), + .domains = axg_pwrc_domains, +}; + +static struct meson_ee_pwrc_domain_data meson_ee_gxbb_pwrc_data = { + .count = ARRAY_SIZE(gxbb_pwrc_domains), + .domains = gxbb_pwrc_domains, +}; + +static struct meson_ee_pwrc_domain_data meson_ee_m8_pwrc_data = { + .count = ARRAY_SIZE(meson8_pwrc_domains), + .domains = meson8_pwrc_domains, +}; + +static struct meson_ee_pwrc_domain_data meson_ee_m8b_pwrc_data = { + .count = ARRAY_SIZE(meson8b_pwrc_domains), + .domains = meson8b_pwrc_domains, +}; + +static struct meson_ee_pwrc_domain_data meson_ee_sm1_pwrc_data = { + .count = ARRAY_SIZE(sm1_pwrc_domains), + .domains = sm1_pwrc_domains, +}; + +static const struct of_device_id meson_ee_pwrc_match_table[] = { + { + .compatible = "amlogic,meson8-pwrc", + .data = &meson_ee_m8_pwrc_data, + }, + { + .compatible = "amlogic,meson8b-pwrc", + .data = &meson_ee_m8b_pwrc_data, + }, + { + .compatible = "amlogic,meson8m2-pwrc", + .data = &meson_ee_m8b_pwrc_data, + }, + { + .compatible = "amlogic,meson-axg-pwrc", + .data = &meson_ee_axg_pwrc_data, + }, + { + .compatible = "amlogic,meson-gxbb-pwrc", + .data = &meson_ee_gxbb_pwrc_data, + }, + { + .compatible = "amlogic,meson-g12a-pwrc", + .data = &meson_ee_g12a_pwrc_data, + }, + { + .compatible = "amlogic,meson-sm1-pwrc", + .data = &meson_ee_sm1_pwrc_data, + }, + { /* sentinel */ } +}; +MODULE_DEVICE_TABLE(of, meson_ee_pwrc_match_table); + +static struct platform_driver meson_ee_pwrc_driver = { + .probe = meson_ee_pwrc_probe, + .shutdown = meson_ee_pwrc_shutdown, + .driver = { + .name = "meson_ee_pwrc", + .of_match_table = meson_ee_pwrc_match_table, + }, +}; +module_platform_driver(meson_ee_pwrc_driver); +MODULE_LICENSE("GPL v2"); diff --git a/drivers/pmdomain/amlogic/meson-gx-pwrc-vpu.c b/drivers/pmdomain/amlogic/meson-gx-pwrc-vpu.c new file mode 100644 index 000000000000..33df520eab95 --- /dev/null +++ b/drivers/pmdomain/amlogic/meson-gx-pwrc-vpu.c @@ -0,0 +1,379 @@ +/* + * Copyright (c) 2017 BayLibre, SAS + * Author: Neil Armstrong + * + * SPDX-License-Identifier: GPL-2.0+ + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +/* AO Offsets */ + +#define AO_RTI_GEN_PWR_SLEEP0 (0x3a << 2) + +#define GEN_PWR_VPU_HDMI BIT(8) +#define GEN_PWR_VPU_HDMI_ISO BIT(9) + +/* HHI Offsets */ + +#define HHI_MEM_PD_REG0 (0x40 << 2) +#define HHI_VPU_MEM_PD_REG0 (0x41 << 2) +#define HHI_VPU_MEM_PD_REG1 (0x42 << 2) +#define HHI_VPU_MEM_PD_REG2 (0x4d << 2) + +struct meson_gx_pwrc_vpu { + struct generic_pm_domain genpd; + struct regmap *regmap_ao; + struct regmap *regmap_hhi; + struct reset_control *rstc; + struct clk *vpu_clk; + struct clk *vapb_clk; +}; + +static inline +struct meson_gx_pwrc_vpu *genpd_to_pd(struct generic_pm_domain *d) +{ + return container_of(d, struct meson_gx_pwrc_vpu, genpd); +} + +static int meson_gx_pwrc_vpu_power_off(struct generic_pm_domain *genpd) +{ + struct meson_gx_pwrc_vpu *pd = genpd_to_pd(genpd); + int i; + + regmap_update_bits(pd->regmap_ao, AO_RTI_GEN_PWR_SLEEP0, + GEN_PWR_VPU_HDMI_ISO, GEN_PWR_VPU_HDMI_ISO); + udelay(20); + + /* Power Down Memories */ + for (i = 0; i < 32; i += 2) { + regmap_update_bits(pd->regmap_hhi, HHI_VPU_MEM_PD_REG0, + 0x3 << i, 0x3 << i); + udelay(5); + } + for (i = 0; i < 32; i += 2) { + regmap_update_bits(pd->regmap_hhi, HHI_VPU_MEM_PD_REG1, + 0x3 << i, 0x3 << i); + udelay(5); + } + for (i = 8; i < 16; i++) { + regmap_update_bits(pd->regmap_hhi, HHI_MEM_PD_REG0, + BIT(i), BIT(i)); + udelay(5); + } + udelay(20); + + regmap_update_bits(pd->regmap_ao, AO_RTI_GEN_PWR_SLEEP0, + GEN_PWR_VPU_HDMI, GEN_PWR_VPU_HDMI); + + msleep(20); + + clk_disable_unprepare(pd->vpu_clk); + clk_disable_unprepare(pd->vapb_clk); + + return 0; +} + +static int meson_g12a_pwrc_vpu_power_off(struct generic_pm_domain *genpd) +{ + struct meson_gx_pwrc_vpu *pd = genpd_to_pd(genpd); + int i; + + regmap_update_bits(pd->regmap_ao, AO_RTI_GEN_PWR_SLEEP0, + GEN_PWR_VPU_HDMI_ISO, GEN_PWR_VPU_HDMI_ISO); + udelay(20); + + /* Power Down Memories */ + for (i = 0; i < 32; i += 2) { + regmap_update_bits(pd->regmap_hhi, HHI_VPU_MEM_PD_REG0, + 0x3 << i, 0x3 << i); + udelay(5); + } + for (i = 0; i < 32; i += 2) { + regmap_update_bits(pd->regmap_hhi, HHI_VPU_MEM_PD_REG1, + 0x3 << i, 0x3 << i); + udelay(5); + } + for (i = 0; i < 32; i += 2) { + regmap_update_bits(pd->regmap_hhi, HHI_VPU_MEM_PD_REG2, + 0x3 << i, 0x3 << i); + udelay(5); + } + for (i = 8; i < 16; i++) { + regmap_update_bits(pd->regmap_hhi, HHI_MEM_PD_REG0, + BIT(i), BIT(i)); + udelay(5); + } + udelay(20); + + regmap_update_bits(pd->regmap_ao, AO_RTI_GEN_PWR_SLEEP0, + GEN_PWR_VPU_HDMI, GEN_PWR_VPU_HDMI); + + msleep(20); + + clk_disable_unprepare(pd->vpu_clk); + clk_disable_unprepare(pd->vapb_clk); + + return 0; +} + +static int meson_gx_pwrc_vpu_setup_clk(struct meson_gx_pwrc_vpu *pd) +{ + int ret; + + ret = clk_prepare_enable(pd->vpu_clk); + if (ret) + return ret; + + ret = clk_prepare_enable(pd->vapb_clk); + if (ret) + clk_disable_unprepare(pd->vpu_clk); + + return ret; +} + +static int meson_gx_pwrc_vpu_power_on(struct generic_pm_domain *genpd) +{ + struct meson_gx_pwrc_vpu *pd = genpd_to_pd(genpd); + int ret; + int i; + + regmap_update_bits(pd->regmap_ao, AO_RTI_GEN_PWR_SLEEP0, + GEN_PWR_VPU_HDMI, 0); + udelay(20); + + /* Power Up Memories */ + for (i = 0; i < 32; i += 2) { + regmap_update_bits(pd->regmap_hhi, HHI_VPU_MEM_PD_REG0, + 0x3 << i, 0); + udelay(5); + } + + for (i = 0; i < 32; i += 2) { + regmap_update_bits(pd->regmap_hhi, HHI_VPU_MEM_PD_REG1, + 0x3 << i, 0); + udelay(5); + } + + for (i = 8; i < 16; i++) { + regmap_update_bits(pd->regmap_hhi, HHI_MEM_PD_REG0, + BIT(i), 0); + udelay(5); + } + udelay(20); + + ret = reset_control_assert(pd->rstc); + if (ret) + return ret; + + regmap_update_bits(pd->regmap_ao, AO_RTI_GEN_PWR_SLEEP0, + GEN_PWR_VPU_HDMI_ISO, 0); + + ret = reset_control_deassert(pd->rstc); + if (ret) + return ret; + + ret = meson_gx_pwrc_vpu_setup_clk(pd); + if (ret) + return ret; + + return 0; +} + +static int meson_g12a_pwrc_vpu_power_on(struct generic_pm_domain *genpd) +{ + struct meson_gx_pwrc_vpu *pd = genpd_to_pd(genpd); + int ret; + int i; + + regmap_update_bits(pd->regmap_ao, AO_RTI_GEN_PWR_SLEEP0, + GEN_PWR_VPU_HDMI, 0); + udelay(20); + + /* Power Up Memories */ + for (i = 0; i < 32; i += 2) { + regmap_update_bits(pd->regmap_hhi, HHI_VPU_MEM_PD_REG0, + 0x3 << i, 0); + udelay(5); + } + + for (i = 0; i < 32; i += 2) { + regmap_update_bits(pd->regmap_hhi, HHI_VPU_MEM_PD_REG1, + 0x3 << i, 0); + udelay(5); + } + + for (i = 0; i < 32; i += 2) { + regmap_update_bits(pd->regmap_hhi, HHI_VPU_MEM_PD_REG2, + 0x3 << i, 0); + udelay(5); + } + + for (i = 8; i < 16; i++) { + regmap_update_bits(pd->regmap_hhi, HHI_MEM_PD_REG0, + BIT(i), 0); + udelay(5); + } + udelay(20); + + ret = reset_control_assert(pd->rstc); + if (ret) + return ret; + + regmap_update_bits(pd->regmap_ao, AO_RTI_GEN_PWR_SLEEP0, + GEN_PWR_VPU_HDMI_ISO, 0); + + ret = reset_control_deassert(pd->rstc); + if (ret) + return ret; + + ret = meson_gx_pwrc_vpu_setup_clk(pd); + if (ret) + return ret; + + return 0; +} + +static bool meson_gx_pwrc_vpu_get_power(struct meson_gx_pwrc_vpu *pd) +{ + u32 reg; + + regmap_read(pd->regmap_ao, AO_RTI_GEN_PWR_SLEEP0, ®); + + return (reg & GEN_PWR_VPU_HDMI); +} + +static struct meson_gx_pwrc_vpu vpu_hdmi_pd = { + .genpd = { + .name = "vpu_hdmi", + .power_off = meson_gx_pwrc_vpu_power_off, + .power_on = meson_gx_pwrc_vpu_power_on, + }, +}; + +static struct meson_gx_pwrc_vpu vpu_hdmi_pd_g12a = { + .genpd = { + .name = "vpu_hdmi", + .power_off = meson_g12a_pwrc_vpu_power_off, + .power_on = meson_g12a_pwrc_vpu_power_on, + }, +}; + +static int meson_gx_pwrc_vpu_probe(struct platform_device *pdev) +{ + const struct meson_gx_pwrc_vpu *vpu_pd_match; + struct regmap *regmap_ao, *regmap_hhi; + struct meson_gx_pwrc_vpu *vpu_pd; + struct device_node *parent_np; + struct reset_control *rstc; + struct clk *vpu_clk; + struct clk *vapb_clk; + bool powered_off; + int ret; + + vpu_pd_match = of_device_get_match_data(&pdev->dev); + if (!vpu_pd_match) { + dev_err(&pdev->dev, "failed to get match data\n"); + return -ENODEV; + } + + vpu_pd = devm_kzalloc(&pdev->dev, sizeof(*vpu_pd), GFP_KERNEL); + if (!vpu_pd) + return -ENOMEM; + + memcpy(vpu_pd, vpu_pd_match, sizeof(*vpu_pd)); + + parent_np = of_get_parent(pdev->dev.of_node); + regmap_ao = syscon_node_to_regmap(parent_np); + of_node_put(parent_np); + if (IS_ERR(regmap_ao)) { + dev_err(&pdev->dev, "failed to get regmap\n"); + return PTR_ERR(regmap_ao); + } + + regmap_hhi = syscon_regmap_lookup_by_phandle(pdev->dev.of_node, + "amlogic,hhi-sysctrl"); + if (IS_ERR(regmap_hhi)) { + dev_err(&pdev->dev, "failed to get HHI regmap\n"); + return PTR_ERR(regmap_hhi); + } + + rstc = devm_reset_control_array_get_exclusive(&pdev->dev); + if (IS_ERR(rstc)) + return dev_err_probe(&pdev->dev, PTR_ERR(rstc), + "failed to get reset lines\n"); + + vpu_clk = devm_clk_get(&pdev->dev, "vpu"); + if (IS_ERR(vpu_clk)) { + dev_err(&pdev->dev, "vpu clock request failed\n"); + return PTR_ERR(vpu_clk); + } + + vapb_clk = devm_clk_get(&pdev->dev, "vapb"); + if (IS_ERR(vapb_clk)) { + dev_err(&pdev->dev, "vapb clock request failed\n"); + return PTR_ERR(vapb_clk); + } + + vpu_pd->regmap_ao = regmap_ao; + vpu_pd->regmap_hhi = regmap_hhi; + vpu_pd->rstc = rstc; + vpu_pd->vpu_clk = vpu_clk; + vpu_pd->vapb_clk = vapb_clk; + + platform_set_drvdata(pdev, vpu_pd); + + powered_off = meson_gx_pwrc_vpu_get_power(vpu_pd); + + /* If already powered, sync the clock states */ + if (!powered_off) { + ret = meson_gx_pwrc_vpu_setup_clk(vpu_pd); + if (ret) + return ret; + } + + vpu_pd->genpd.flags = GENPD_FLAG_ALWAYS_ON; + pm_genpd_init(&vpu_pd->genpd, NULL, powered_off); + + return of_genpd_add_provider_simple(pdev->dev.of_node, + &vpu_pd->genpd); +} + +static void meson_gx_pwrc_vpu_shutdown(struct platform_device *pdev) +{ + struct meson_gx_pwrc_vpu *vpu_pd = platform_get_drvdata(pdev); + bool powered_off; + + powered_off = meson_gx_pwrc_vpu_get_power(vpu_pd); + if (!powered_off) + vpu_pd->genpd.power_off(&vpu_pd->genpd); +} + +static const struct of_device_id meson_gx_pwrc_vpu_match_table[] = { + { .compatible = "amlogic,meson-gx-pwrc-vpu", .data = &vpu_hdmi_pd }, + { + .compatible = "amlogic,meson-g12a-pwrc-vpu", + .data = &vpu_hdmi_pd_g12a + }, + { /* sentinel */ } +}; +MODULE_DEVICE_TABLE(of, meson_gx_pwrc_vpu_match_table); + +static struct platform_driver meson_gx_pwrc_vpu_driver = { + .probe = meson_gx_pwrc_vpu_probe, + .shutdown = meson_gx_pwrc_vpu_shutdown, + .driver = { + .name = "meson_gx_pwrc_vpu", + .of_match_table = meson_gx_pwrc_vpu_match_table, + }, +}; +module_platform_driver(meson_gx_pwrc_vpu_driver); +MODULE_LICENSE("GPL v2"); diff --git a/drivers/pmdomain/amlogic/meson-secure-pwrc.c b/drivers/pmdomain/amlogic/meson-secure-pwrc.c new file mode 100644 index 000000000000..89c881c56cd7 --- /dev/null +++ b/drivers/pmdomain/amlogic/meson-secure-pwrc.c @@ -0,0 +1,257 @@ +// SPDX-License-Identifier: (GPL-2.0+ OR MIT) +/* + * Copyright (c) 2019 Amlogic, Inc. + * Author: Jianxin Pan + */ + +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#define PWRC_ON 1 +#define PWRC_OFF 0 + +struct meson_secure_pwrc_domain { + struct generic_pm_domain base; + unsigned int index; + struct meson_secure_pwrc *pwrc; +}; + +struct meson_secure_pwrc { + struct meson_secure_pwrc_domain *domains; + struct genpd_onecell_data xlate; + struct meson_sm_firmware *fw; +}; + +struct meson_secure_pwrc_domain_desc { + unsigned int index; + unsigned int flags; + char *name; + bool (*is_off)(struct meson_secure_pwrc_domain *pwrc_domain); +}; + +struct meson_secure_pwrc_domain_data { + unsigned int count; + struct meson_secure_pwrc_domain_desc *domains; +}; + +static bool pwrc_secure_is_off(struct meson_secure_pwrc_domain *pwrc_domain) +{ + int is_off = 1; + + if (meson_sm_call(pwrc_domain->pwrc->fw, SM_A1_PWRC_GET, &is_off, + pwrc_domain->index, 0, 0, 0, 0) < 0) + pr_err("failed to get power domain status\n"); + + return is_off; +} + +static int meson_secure_pwrc_off(struct generic_pm_domain *domain) +{ + int ret = 0; + struct meson_secure_pwrc_domain *pwrc_domain = + container_of(domain, struct meson_secure_pwrc_domain, base); + + if (meson_sm_call(pwrc_domain->pwrc->fw, SM_A1_PWRC_SET, NULL, + pwrc_domain->index, PWRC_OFF, 0, 0, 0) < 0) { + pr_err("failed to set power domain off\n"); + ret = -EINVAL; + } + + return ret; +} + +static int meson_secure_pwrc_on(struct generic_pm_domain *domain) +{ + int ret = 0; + struct meson_secure_pwrc_domain *pwrc_domain = + container_of(domain, struct meson_secure_pwrc_domain, base); + + if (meson_sm_call(pwrc_domain->pwrc->fw, SM_A1_PWRC_SET, NULL, + pwrc_domain->index, PWRC_ON, 0, 0, 0) < 0) { + pr_err("failed to set power domain on\n"); + ret = -EINVAL; + } + + return ret; +} + +#define SEC_PD(__name, __flag) \ +[PWRC_##__name##_ID] = \ +{ \ + .name = #__name, \ + .index = PWRC_##__name##_ID, \ + .is_off = pwrc_secure_is_off, \ + .flags = __flag, \ +} + +static struct meson_secure_pwrc_domain_desc a1_pwrc_domains[] = { + SEC_PD(DSPA, 0), + SEC_PD(DSPB, 0), + /* UART should keep working in ATF after suspend and before resume */ + SEC_PD(UART, GENPD_FLAG_ALWAYS_ON), + /* DMC is for DDR PHY ana/dig and DMC, and should be always on */ + SEC_PD(DMC, GENPD_FLAG_ALWAYS_ON), + SEC_PD(I2C, 0), + SEC_PD(PSRAM, 0), + SEC_PD(ACODEC, 0), + SEC_PD(AUDIO, 0), + SEC_PD(OTP, 0), + SEC_PD(DMA, GENPD_FLAG_ALWAYS_ON | GENPD_FLAG_IRQ_SAFE), + SEC_PD(SD_EMMC, 0), + SEC_PD(RAMA, 0), + /* SRAMB is used as ATF runtime memory, and should be always on */ + SEC_PD(RAMB, GENPD_FLAG_ALWAYS_ON), + SEC_PD(IR, 0), + SEC_PD(SPICC, 0), + SEC_PD(SPIFC, 0), + SEC_PD(USB, 0), + /* NIC is for the Arm NIC-400 interconnect, and should be always on */ + SEC_PD(NIC, GENPD_FLAG_ALWAYS_ON), + SEC_PD(PDMIN, 0), + SEC_PD(RSA, 0), +}; + +static struct meson_secure_pwrc_domain_desc c3_pwrc_domains[] = { + SEC_PD(C3_NNA, 0), + SEC_PD(C3_AUDIO, GENPD_FLAG_ALWAYS_ON), + SEC_PD(C3_SDIOA, GENPD_FLAG_ALWAYS_ON), + SEC_PD(C3_EMMC, GENPD_FLAG_ALWAYS_ON), + SEC_PD(C3_USB_COMB, GENPD_FLAG_ALWAYS_ON), + SEC_PD(C3_SDCARD, GENPD_FLAG_ALWAYS_ON), + SEC_PD(C3_ETH, GENPD_FLAG_ALWAYS_ON), + SEC_PD(C3_GE2D, GENPD_FLAG_ALWAYS_ON), + SEC_PD(C3_CVE, GENPD_FLAG_ALWAYS_ON), + SEC_PD(C3_GDC_WRAP, GENPD_FLAG_ALWAYS_ON), + SEC_PD(C3_ISP_TOP, GENPD_FLAG_ALWAYS_ON), + SEC_PD(C3_MIPI_ISP_WRAP, GENPD_FLAG_ALWAYS_ON), + SEC_PD(C3_VCODEC, 0), +}; + +static struct meson_secure_pwrc_domain_desc s4_pwrc_domains[] = { + SEC_PD(S4_DOS_HEVC, 0), + SEC_PD(S4_DOS_VDEC, 0), + SEC_PD(S4_VPU_HDMI, 0), + SEC_PD(S4_USB_COMB, 0), + SEC_PD(S4_GE2D, 0), + /* ETH is for ethernet online wakeup, and should be always on */ + SEC_PD(S4_ETH, GENPD_FLAG_ALWAYS_ON), + SEC_PD(S4_DEMOD, 0), + SEC_PD(S4_AUDIO, 0), +}; + +static int meson_secure_pwrc_probe(struct platform_device *pdev) +{ + int i; + struct device_node *sm_np; + struct meson_secure_pwrc *pwrc; + const struct meson_secure_pwrc_domain_data *match; + + match = of_device_get_match_data(&pdev->dev); + if (!match) { + dev_err(&pdev->dev, "failed to get match data\n"); + return -ENODEV; + } + + sm_np = of_find_compatible_node(NULL, NULL, "amlogic,meson-gxbb-sm"); + if (!sm_np) { + dev_err(&pdev->dev, "no secure-monitor node\n"); + return -ENODEV; + } + + pwrc = devm_kzalloc(&pdev->dev, sizeof(*pwrc), GFP_KERNEL); + if (!pwrc) { + of_node_put(sm_np); + return -ENOMEM; + } + + pwrc->fw = meson_sm_get(sm_np); + of_node_put(sm_np); + if (!pwrc->fw) + return -EPROBE_DEFER; + + pwrc->xlate.domains = devm_kcalloc(&pdev->dev, match->count, + sizeof(*pwrc->xlate.domains), + GFP_KERNEL); + if (!pwrc->xlate.domains) + return -ENOMEM; + + pwrc->domains = devm_kcalloc(&pdev->dev, match->count, + sizeof(*pwrc->domains), GFP_KERNEL); + if (!pwrc->domains) + return -ENOMEM; + + pwrc->xlate.num_domains = match->count; + platform_set_drvdata(pdev, pwrc); + + for (i = 0 ; i < match->count ; ++i) { + struct meson_secure_pwrc_domain *dom = &pwrc->domains[i]; + + if (!match->domains[i].name) + continue; + + dom->pwrc = pwrc; + dom->index = match->domains[i].index; + dom->base.name = match->domains[i].name; + dom->base.flags = match->domains[i].flags; + dom->base.power_on = meson_secure_pwrc_on; + dom->base.power_off = meson_secure_pwrc_off; + + pm_genpd_init(&dom->base, NULL, match->domains[i].is_off(dom)); + + pwrc->xlate.domains[i] = &dom->base; + } + + return of_genpd_add_provider_onecell(pdev->dev.of_node, &pwrc->xlate); +} + +static struct meson_secure_pwrc_domain_data meson_secure_a1_pwrc_data = { + .domains = a1_pwrc_domains, + .count = ARRAY_SIZE(a1_pwrc_domains), +}; + +static struct meson_secure_pwrc_domain_data amlogic_secure_c3_pwrc_data = { + .domains = c3_pwrc_domains, + .count = ARRAY_SIZE(c3_pwrc_domains), +}; + +static struct meson_secure_pwrc_domain_data meson_secure_s4_pwrc_data = { + .domains = s4_pwrc_domains, + .count = ARRAY_SIZE(s4_pwrc_domains), +}; + +static const struct of_device_id meson_secure_pwrc_match_table[] = { + { + .compatible = "amlogic,meson-a1-pwrc", + .data = &meson_secure_a1_pwrc_data, + }, + { + .compatible = "amlogic,c3-pwrc", + .data = &amlogic_secure_c3_pwrc_data, + }, + { + .compatible = "amlogic,meson-s4-pwrc", + .data = &meson_secure_s4_pwrc_data, + }, + { /* sentinel */ } +}; +MODULE_DEVICE_TABLE(of, meson_secure_pwrc_match_table); + +static struct platform_driver meson_secure_pwrc_driver = { + .probe = meson_secure_pwrc_probe, + .driver = { + .name = "meson_secure_pwrc", + .of_match_table = meson_secure_pwrc_match_table, + }, +}; +module_platform_driver(meson_secure_pwrc_driver); +MODULE_LICENSE("Dual MIT/GPL"); diff --git a/drivers/pmdomain/apple/Makefile b/drivers/pmdomain/apple/Makefile new file mode 100644 index 000000000000..53665af630be --- /dev/null +++ b/drivers/pmdomain/apple/Makefile @@ -0,0 +1,2 @@ +# SPDX-License-Identifier: GPL-2.0-only +obj-$(CONFIG_APPLE_PMGR_PWRSTATE) += pmgr-pwrstate.o diff --git a/drivers/pmdomain/apple/pmgr-pwrstate.c b/drivers/pmdomain/apple/pmgr-pwrstate.c new file mode 100644 index 000000000000..d62a776c89a1 --- /dev/null +++ b/drivers/pmdomain/apple/pmgr-pwrstate.c @@ -0,0 +1,326 @@ +// SPDX-License-Identifier: GPL-2.0-only OR MIT +/* + * Apple SoC PMGR device power state driver + * + * Copyright The Asahi Linux Contributors + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#define APPLE_PMGR_RESET BIT(31) +#define APPLE_PMGR_AUTO_ENABLE BIT(28) +#define APPLE_PMGR_PS_AUTO GENMASK(27, 24) +#define APPLE_PMGR_PS_MIN GENMASK(19, 16) +#define APPLE_PMGR_PARENT_OFF BIT(11) +#define APPLE_PMGR_DEV_DISABLE BIT(10) +#define APPLE_PMGR_WAS_CLKGATED BIT(9) +#define APPLE_PMGR_WAS_PWRGATED BIT(8) +#define APPLE_PMGR_PS_ACTUAL GENMASK(7, 4) +#define APPLE_PMGR_PS_TARGET GENMASK(3, 0) + +#define APPLE_PMGR_FLAGS (APPLE_PMGR_WAS_CLKGATED | APPLE_PMGR_WAS_PWRGATED) + +#define APPLE_PMGR_PS_ACTIVE 0xf +#define APPLE_PMGR_PS_CLKGATE 0x4 +#define APPLE_PMGR_PS_PWRGATE 0x0 + +#define APPLE_PMGR_PS_SET_TIMEOUT 100 +#define APPLE_PMGR_RESET_TIME 1 + +struct apple_pmgr_ps { + struct device *dev; + struct generic_pm_domain genpd; + struct reset_controller_dev rcdev; + struct regmap *regmap; + u32 offset; + u32 min_state; +}; + +#define genpd_to_apple_pmgr_ps(_genpd) container_of(_genpd, struct apple_pmgr_ps, genpd) +#define rcdev_to_apple_pmgr_ps(_rcdev) container_of(_rcdev, struct apple_pmgr_ps, rcdev) + +static int apple_pmgr_ps_set(struct generic_pm_domain *genpd, u32 pstate, bool auto_enable) +{ + int ret; + struct apple_pmgr_ps *ps = genpd_to_apple_pmgr_ps(genpd); + u32 reg; + + ret = regmap_read(ps->regmap, ps->offset, ®); + if (ret < 0) + return ret; + + /* Resets are synchronous, and only work if the device is powered and clocked. */ + if (reg & APPLE_PMGR_RESET && pstate != APPLE_PMGR_PS_ACTIVE) + dev_err(ps->dev, "PS %s: powering off with RESET active\n", + genpd->name); + + reg &= ~(APPLE_PMGR_AUTO_ENABLE | APPLE_PMGR_FLAGS | APPLE_PMGR_PS_TARGET); + reg |= FIELD_PREP(APPLE_PMGR_PS_TARGET, pstate); + + dev_dbg(ps->dev, "PS %s: pwrstate = 0x%x: 0x%x\n", genpd->name, pstate, reg); + + regmap_write(ps->regmap, ps->offset, reg); + + ret = regmap_read_poll_timeout_atomic( + ps->regmap, ps->offset, reg, + (FIELD_GET(APPLE_PMGR_PS_ACTUAL, reg) == pstate), 1, + APPLE_PMGR_PS_SET_TIMEOUT); + if (ret < 0) + dev_err(ps->dev, "PS %s: Failed to reach power state 0x%x (now: 0x%x)\n", + genpd->name, pstate, reg); + + if (auto_enable) { + /* Not all devices implement this; this is a no-op where not implemented. */ + reg &= ~APPLE_PMGR_FLAGS; + reg |= APPLE_PMGR_AUTO_ENABLE; + regmap_write(ps->regmap, ps->offset, reg); + } + + return ret; +} + +static bool apple_pmgr_ps_is_active(struct apple_pmgr_ps *ps) +{ + u32 reg = 0; + + regmap_read(ps->regmap, ps->offset, ®); + /* + * We consider domains as active if they are actually on, or if they have auto-PM + * enabled and the intended target is on. + */ + return (FIELD_GET(APPLE_PMGR_PS_ACTUAL, reg) == APPLE_PMGR_PS_ACTIVE || + (FIELD_GET(APPLE_PMGR_PS_TARGET, reg) == APPLE_PMGR_PS_ACTIVE && + reg & APPLE_PMGR_AUTO_ENABLE)); +} + +static int apple_pmgr_ps_power_on(struct generic_pm_domain *genpd) +{ + return apple_pmgr_ps_set(genpd, APPLE_PMGR_PS_ACTIVE, true); +} + +static int apple_pmgr_ps_power_off(struct generic_pm_domain *genpd) +{ + return apple_pmgr_ps_set(genpd, APPLE_PMGR_PS_PWRGATE, false); +} + +static int apple_pmgr_reset_assert(struct reset_controller_dev *rcdev, unsigned long id) +{ + struct apple_pmgr_ps *ps = rcdev_to_apple_pmgr_ps(rcdev); + unsigned long flags; + + spin_lock_irqsave(&ps->genpd.slock, flags); + + if (ps->genpd.status == GENPD_STATE_OFF) + dev_err(ps->dev, "PS 0x%x: asserting RESET while powered down\n", ps->offset); + + dev_dbg(ps->dev, "PS 0x%x: assert reset\n", ps->offset); + /* Quiesce device before asserting reset */ + regmap_update_bits(ps->regmap, ps->offset, APPLE_PMGR_FLAGS | APPLE_PMGR_DEV_DISABLE, + APPLE_PMGR_DEV_DISABLE); + regmap_update_bits(ps->regmap, ps->offset, APPLE_PMGR_FLAGS | APPLE_PMGR_RESET, + APPLE_PMGR_RESET); + + spin_unlock_irqrestore(&ps->genpd.slock, flags); + + return 0; +} + +static int apple_pmgr_reset_deassert(struct reset_controller_dev *rcdev, unsigned long id) +{ + struct apple_pmgr_ps *ps = rcdev_to_apple_pmgr_ps(rcdev); + unsigned long flags; + + spin_lock_irqsave(&ps->genpd.slock, flags); + + dev_dbg(ps->dev, "PS 0x%x: deassert reset\n", ps->offset); + regmap_update_bits(ps->regmap, ps->offset, APPLE_PMGR_FLAGS | APPLE_PMGR_RESET, 0); + regmap_update_bits(ps->regmap, ps->offset, APPLE_PMGR_FLAGS | APPLE_PMGR_DEV_DISABLE, 0); + + if (ps->genpd.status == GENPD_STATE_OFF) + dev_err(ps->dev, "PS 0x%x: RESET was deasserted while powered down\n", ps->offset); + + spin_unlock_irqrestore(&ps->genpd.slock, flags); + + return 0; +} + +static int apple_pmgr_reset_reset(struct reset_controller_dev *rcdev, unsigned long id) +{ + int ret; + + ret = apple_pmgr_reset_assert(rcdev, id); + if (ret) + return ret; + + usleep_range(APPLE_PMGR_RESET_TIME, 2 * APPLE_PMGR_RESET_TIME); + + return apple_pmgr_reset_deassert(rcdev, id); +} + +static int apple_pmgr_reset_status(struct reset_controller_dev *rcdev, unsigned long id) +{ + struct apple_pmgr_ps *ps = rcdev_to_apple_pmgr_ps(rcdev); + u32 reg = 0; + + regmap_read(ps->regmap, ps->offset, ®); + + return !!(reg & APPLE_PMGR_RESET); +} + +const struct reset_control_ops apple_pmgr_reset_ops = { + .assert = apple_pmgr_reset_assert, + .deassert = apple_pmgr_reset_deassert, + .reset = apple_pmgr_reset_reset, + .status = apple_pmgr_reset_status, +}; + +static int apple_pmgr_reset_xlate(struct reset_controller_dev *rcdev, + const struct of_phandle_args *reset_spec) +{ + return 0; +} + +static int apple_pmgr_ps_probe(struct platform_device *pdev) +{ + struct device *dev = &pdev->dev; + struct device_node *node = dev->of_node; + struct apple_pmgr_ps *ps; + struct regmap *regmap; + struct of_phandle_iterator it; + int ret; + const char *name; + bool active; + + regmap = syscon_node_to_regmap(node->parent); + if (IS_ERR(regmap)) + return PTR_ERR(regmap); + + ps = devm_kzalloc(dev, sizeof(*ps), GFP_KERNEL); + if (!ps) + return -ENOMEM; + + ps->dev = dev; + ps->regmap = regmap; + + ret = of_property_read_string(node, "label", &name); + if (ret < 0) { + dev_err(dev, "missing label property\n"); + return ret; + } + + ret = of_property_read_u32(node, "reg", &ps->offset); + if (ret < 0) { + dev_err(dev, "missing reg property\n"); + return ret; + } + + ps->genpd.flags |= GENPD_FLAG_IRQ_SAFE; + ps->genpd.name = name; + ps->genpd.power_on = apple_pmgr_ps_power_on; + ps->genpd.power_off = apple_pmgr_ps_power_off; + + ret = of_property_read_u32(node, "apple,min-state", &ps->min_state); + if (ret == 0 && ps->min_state <= APPLE_PMGR_PS_ACTIVE) + regmap_update_bits(regmap, ps->offset, APPLE_PMGR_FLAGS | APPLE_PMGR_PS_MIN, + FIELD_PREP(APPLE_PMGR_PS_MIN, ps->min_state)); + + active = apple_pmgr_ps_is_active(ps); + if (of_property_read_bool(node, "apple,always-on")) { + ps->genpd.flags |= GENPD_FLAG_ALWAYS_ON; + if (!active) { + dev_warn(dev, "always-on domain %s is not on at boot\n", name); + /* Turn it on so pm_genpd_init does not fail */ + active = apple_pmgr_ps_power_on(&ps->genpd) == 0; + } + } + + /* Turn on auto-PM if the domain is already on */ + if (active) + regmap_update_bits(regmap, ps->offset, APPLE_PMGR_FLAGS | APPLE_PMGR_AUTO_ENABLE, + APPLE_PMGR_AUTO_ENABLE); + + ret = pm_genpd_init(&ps->genpd, NULL, !active); + if (ret < 0) { + dev_err(dev, "pm_genpd_init failed\n"); + return ret; + } + + ret = of_genpd_add_provider_simple(node, &ps->genpd); + if (ret < 0) { + dev_err(dev, "of_genpd_add_provider_simple failed\n"); + return ret; + } + + of_for_each_phandle(&it, ret, node, "power-domains", "#power-domain-cells", -1) { + struct of_phandle_args parent, child; + + parent.np = it.node; + parent.args_count = of_phandle_iterator_args(&it, parent.args, MAX_PHANDLE_ARGS); + child.np = node; + child.args_count = 0; + ret = of_genpd_add_subdomain(&parent, &child); + + if (ret == -EPROBE_DEFER) { + of_node_put(parent.np); + goto err_remove; + } else if (ret < 0) { + dev_err(dev, "failed to add to parent domain: %d (%s -> %s)\n", + ret, it.node->name, node->name); + of_node_put(parent.np); + goto err_remove; + } + } + + /* + * Do not participate in regular PM; parent power domains are handled via the + * genpd hierarchy. + */ + pm_genpd_remove_device(dev); + + ps->rcdev.owner = THIS_MODULE; + ps->rcdev.nr_resets = 1; + ps->rcdev.ops = &apple_pmgr_reset_ops; + ps->rcdev.of_node = dev->of_node; + ps->rcdev.of_reset_n_cells = 0; + ps->rcdev.of_xlate = apple_pmgr_reset_xlate; + + ret = devm_reset_controller_register(dev, &ps->rcdev); + if (ret < 0) + goto err_remove; + + return 0; +err_remove: + of_genpd_del_provider(node); + pm_genpd_remove(&ps->genpd); + return ret; +} + +static const struct of_device_id apple_pmgr_ps_of_match[] = { + { .compatible = "apple,pmgr-pwrstate" }, + {} +}; + +MODULE_DEVICE_TABLE(of, apple_pmgr_ps_of_match); + +static struct platform_driver apple_pmgr_ps_driver = { + .probe = apple_pmgr_ps_probe, + .driver = { + .name = "apple-pmgr-pwrstate", + .of_match_table = apple_pmgr_ps_of_match, + }, +}; + +MODULE_AUTHOR("Hector Martin "); +MODULE_DESCRIPTION("PMGR power state driver for Apple SoCs"); + +module_platform_driver(apple_pmgr_ps_driver); diff --git a/drivers/pmdomain/bcm/Makefile b/drivers/pmdomain/bcm/Makefile new file mode 100644 index 000000000000..6bfbe4e4db13 --- /dev/null +++ b/drivers/pmdomain/bcm/Makefile @@ -0,0 +1,5 @@ +# SPDX-License-Identifier: GPL-2.0-only +obj-$(CONFIG_BCM_PMB) += bcm-pmb.o +obj-$(CONFIG_BCM2835_POWER) += bcm2835-power.o +obj-$(CONFIG_BCM63XX_POWER) += bcm63xx-power.o +obj-$(CONFIG_RASPBERRYPI_POWER) += raspberrypi-power.o diff --git a/drivers/pmdomain/bcm/bcm-pmb.c b/drivers/pmdomain/bcm/bcm-pmb.c new file mode 100644 index 000000000000..a72ba26ecf9d --- /dev/null +++ b/drivers/pmdomain/bcm/bcm-pmb.c @@ -0,0 +1,363 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * Copyright (c) 2013 Broadcom + * Copyright (C) 2020 Rafał Miłecki + */ + +#include +#include +#include +#include +#include +#include +#include + +#define BPCM_ID_REG 0x00 +#define BPCM_CAPABILITIES 0x04 +#define BPCM_CAP_NUM_ZONES 0x000000ff +#define BPCM_CAP_SR_REG_BITS 0x0000ff00 +#define BPCM_CAP_PLLTYPE 0x00030000 +#define BPCM_CAP_UBUS 0x00080000 +#define BPCM_CONTROL 0x08 +#define BPCM_STATUS 0x0c +#define BPCM_ROSC_CONTROL 0x10 +#define BPCM_ROSC_THRESH_H 0x14 +#define BPCM_ROSC_THRESHOLD_BCM6838 0x14 +#define BPCM_ROSC_THRESH_S 0x18 +#define BPCM_ROSC_COUNT_BCM6838 0x18 +#define BPCM_ROSC_COUNT 0x1c +#define BPCM_PWD_CONTROL_BCM6838 0x1c +#define BPCM_PWD_CONTROL 0x20 +#define BPCM_SR_CONTROL_BCM6838 0x20 +#define BPCM_PWD_ACCUM_CONTROL 0x24 +#define BPCM_SR_CONTROL 0x28 +#define BPCM_GLOBAL_CONTROL 0x2c +#define BPCM_MISC_CONTROL 0x30 +#define BPCM_MISC_CONTROL2 0x34 +#define BPCM_SGPHY_CNTL 0x38 +#define BPCM_SGPHY_STATUS 0x3c +#define BPCM_ZONE0 0x40 +#define BPCM_ZONE_CONTROL 0x00 +#define BPCM_ZONE_CONTROL_MANUAL_CLK_EN 0x00000001 +#define BPCM_ZONE_CONTROL_MANUAL_RESET_CTL 0x00000002 +#define BPCM_ZONE_CONTROL_FREQ_SCALE_USED 0x00000004 /* R/O */ +#define BPCM_ZONE_CONTROL_DPG_CAPABLE 0x00000008 /* R/O */ +#define BPCM_ZONE_CONTROL_MANUAL_MEM_PWR 0x00000030 +#define BPCM_ZONE_CONTROL_MANUAL_ISO_CTL 0x00000040 +#define BPCM_ZONE_CONTROL_MANUAL_CTL 0x00000080 +#define BPCM_ZONE_CONTROL_DPG_CTL_EN 0x00000100 +#define BPCM_ZONE_CONTROL_PWR_DN_REQ 0x00000200 +#define BPCM_ZONE_CONTROL_PWR_UP_REQ 0x00000400 +#define BPCM_ZONE_CONTROL_MEM_PWR_CTL_EN 0x00000800 +#define BPCM_ZONE_CONTROL_BLK_RESET_ASSERT 0x00001000 +#define BPCM_ZONE_CONTROL_MEM_STBY 0x00002000 +#define BPCM_ZONE_CONTROL_RESERVED 0x0007c000 +#define BPCM_ZONE_CONTROL_PWR_CNTL_STATE 0x00f80000 +#define BPCM_ZONE_CONTROL_FREQ_SCALAR_DYN_SEL 0x01000000 /* R/O */ +#define BPCM_ZONE_CONTROL_PWR_OFF_STATE 0x02000000 /* R/O */ +#define BPCM_ZONE_CONTROL_PWR_ON_STATE 0x04000000 /* R/O */ +#define BPCM_ZONE_CONTROL_PWR_GOOD 0x08000000 /* R/O */ +#define BPCM_ZONE_CONTROL_DPG_PWR_STATE 0x10000000 /* R/O */ +#define BPCM_ZONE_CONTROL_MEM_PWR_STATE 0x20000000 /* R/O */ +#define BPCM_ZONE_CONTROL_ISO_STATE 0x40000000 /* R/O */ +#define BPCM_ZONE_CONTROL_RESET_STATE 0x80000000 /* R/O */ +#define BPCM_ZONE_CONFIG1 0x04 +#define BPCM_ZONE_CONFIG2 0x08 +#define BPCM_ZONE_FREQ_SCALAR_CONTROL 0x0c +#define BPCM_ZONE_SIZE 0x10 + +struct bcm_pmb { + struct device *dev; + void __iomem *base; + spinlock_t lock; + bool little_endian; + struct genpd_onecell_data genpd_onecell_data; +}; + +struct bcm_pmb_pd_data { + const char * const name; + int id; + u8 bus; + u8 device; +}; + +struct bcm_pmb_pm_domain { + struct bcm_pmb *pmb; + const struct bcm_pmb_pd_data *data; + struct generic_pm_domain genpd; +}; + +static int bcm_pmb_bpcm_read(struct bcm_pmb *pmb, int bus, u8 device, + int offset, u32 *val) +{ + void __iomem *base = pmb->base + bus * 0x20; + unsigned long flags; + int err; + + spin_lock_irqsave(&pmb->lock, flags); + err = bpcm_rd(base, device, offset, val); + spin_unlock_irqrestore(&pmb->lock, flags); + + if (!err) + *val = pmb->little_endian ? le32_to_cpu(*val) : be32_to_cpu(*val); + + return err; +} + +static int bcm_pmb_bpcm_write(struct bcm_pmb *pmb, int bus, u8 device, + int offset, u32 val) +{ + void __iomem *base = pmb->base + bus * 0x20; + unsigned long flags; + int err; + + val = pmb->little_endian ? cpu_to_le32(val) : cpu_to_be32(val); + + spin_lock_irqsave(&pmb->lock, flags); + err = bpcm_wr(base, device, offset, val); + spin_unlock_irqrestore(&pmb->lock, flags); + + return err; +} + +static int bcm_pmb_power_off_zone(struct bcm_pmb *pmb, int bus, u8 device, + int zone) +{ + int offset; + u32 val; + int err; + + offset = BPCM_ZONE0 + zone * BPCM_ZONE_SIZE + BPCM_ZONE_CONTROL; + + err = bcm_pmb_bpcm_read(pmb, bus, device, offset, &val); + if (err) + return err; + + val |= BPCM_ZONE_CONTROL_PWR_DN_REQ; + val &= ~BPCM_ZONE_CONTROL_PWR_UP_REQ; + + err = bcm_pmb_bpcm_write(pmb, bus, device, offset, val); + + return err; +} + +static int bcm_pmb_power_on_zone(struct bcm_pmb *pmb, int bus, u8 device, + int zone) +{ + int offset; + u32 val; + int err; + + offset = BPCM_ZONE0 + zone * BPCM_ZONE_SIZE + BPCM_ZONE_CONTROL; + + err = bcm_pmb_bpcm_read(pmb, bus, device, offset, &val); + if (err) + return err; + + if (!(val & BPCM_ZONE_CONTROL_PWR_ON_STATE)) { + val &= ~BPCM_ZONE_CONTROL_PWR_DN_REQ; + val |= BPCM_ZONE_CONTROL_DPG_CTL_EN; + val |= BPCM_ZONE_CONTROL_PWR_UP_REQ; + val |= BPCM_ZONE_CONTROL_MEM_PWR_CTL_EN; + val |= BPCM_ZONE_CONTROL_BLK_RESET_ASSERT; + + err = bcm_pmb_bpcm_write(pmb, bus, device, offset, val); + } + + return err; +} + +static int bcm_pmb_power_off_device(struct bcm_pmb *pmb, int bus, u8 device) +{ + int offset; + u32 val; + int err; + + /* Entire device can be powered off by powering off the 0th zone */ + offset = BPCM_ZONE0 + BPCM_ZONE_CONTROL; + + err = bcm_pmb_bpcm_read(pmb, bus, device, offset, &val); + if (err) + return err; + + if (!(val & BPCM_ZONE_CONTROL_PWR_OFF_STATE)) { + val = BPCM_ZONE_CONTROL_PWR_DN_REQ; + + err = bcm_pmb_bpcm_write(pmb, bus, device, offset, val); + } + + return err; +} + +static int bcm_pmb_power_on_device(struct bcm_pmb *pmb, int bus, u8 device) +{ + u32 val; + int err; + int i; + + err = bcm_pmb_bpcm_read(pmb, bus, device, BPCM_CAPABILITIES, &val); + if (err) + return err; + + for (i = 0; i < (val & BPCM_CAP_NUM_ZONES); i++) { + err = bcm_pmb_power_on_zone(pmb, bus, device, i); + if (err) + return err; + } + + return err; +} + +static int bcm_pmb_power_on_sata(struct bcm_pmb *pmb, int bus, u8 device) +{ + int err; + + err = bcm_pmb_power_on_zone(pmb, bus, device, 0); + if (err) + return err; + + /* Does not apply to the BCM963158 */ + err = bcm_pmb_bpcm_write(pmb, bus, device, BPCM_MISC_CONTROL, 0); + if (err) + return err; + + err = bcm_pmb_bpcm_write(pmb, bus, device, BPCM_SR_CONTROL, 0xffffffff); + if (err) + return err; + + err = bcm_pmb_bpcm_write(pmb, bus, device, BPCM_SR_CONTROL, 0); + + return err; +} + +static int bcm_pmb_power_on(struct generic_pm_domain *genpd) +{ + struct bcm_pmb_pm_domain *pd = container_of(genpd, struct bcm_pmb_pm_domain, genpd); + const struct bcm_pmb_pd_data *data = pd->data; + struct bcm_pmb *pmb = pd->pmb; + + switch (data->id) { + case BCM_PMB_PCIE0: + case BCM_PMB_PCIE1: + case BCM_PMB_PCIE2: + return bcm_pmb_power_on_zone(pmb, data->bus, data->device, 0); + case BCM_PMB_HOST_USB: + return bcm_pmb_power_on_device(pmb, data->bus, data->device); + case BCM_PMB_SATA: + return bcm_pmb_power_on_sata(pmb, data->bus, data->device); + default: + dev_err(pmb->dev, "unsupported device id: %d\n", data->id); + return -EINVAL; + } +} + +static int bcm_pmb_power_off(struct generic_pm_domain *genpd) +{ + struct bcm_pmb_pm_domain *pd = container_of(genpd, struct bcm_pmb_pm_domain, genpd); + const struct bcm_pmb_pd_data *data = pd->data; + struct bcm_pmb *pmb = pd->pmb; + + switch (data->id) { + case BCM_PMB_PCIE0: + case BCM_PMB_PCIE1: + case BCM_PMB_PCIE2: + return bcm_pmb_power_off_zone(pmb, data->bus, data->device, 0); + case BCM_PMB_HOST_USB: + return bcm_pmb_power_off_device(pmb, data->bus, data->device); + default: + dev_err(pmb->dev, "unsupported device id: %d\n", data->id); + return -EINVAL; + } +} + +static int bcm_pmb_probe(struct platform_device *pdev) +{ + struct device *dev = &pdev->dev; + const struct bcm_pmb_pd_data *table; + const struct bcm_pmb_pd_data *e; + struct bcm_pmb *pmb; + int max_id; + int err; + + pmb = devm_kzalloc(dev, sizeof(*pmb), GFP_KERNEL); + if (!pmb) + return -ENOMEM; + + pmb->dev = dev; + + pmb->base = devm_platform_ioremap_resource(pdev, 0); + if (IS_ERR(pmb->base)) + return PTR_ERR(pmb->base); + + spin_lock_init(&pmb->lock); + + pmb->little_endian = !of_device_is_big_endian(dev->of_node); + + table = of_device_get_match_data(dev); + if (!table) + return -EINVAL; + + max_id = 0; + for (e = table; e->name; e++) + max_id = max(max_id, e->id); + + pmb->genpd_onecell_data.num_domains = max_id + 1; + pmb->genpd_onecell_data.domains = + devm_kcalloc(dev, pmb->genpd_onecell_data.num_domains, + sizeof(struct generic_pm_domain *), GFP_KERNEL); + if (!pmb->genpd_onecell_data.domains) + return -ENOMEM; + + for (e = table; e->name; e++) { + struct bcm_pmb_pm_domain *pd = devm_kzalloc(dev, sizeof(*pd), GFP_KERNEL); + + if (!pd) + return -ENOMEM; + + pd->pmb = pmb; + pd->data = e; + pd->genpd.name = e->name; + pd->genpd.power_on = bcm_pmb_power_on; + pd->genpd.power_off = bcm_pmb_power_off; + + pm_genpd_init(&pd->genpd, NULL, true); + pmb->genpd_onecell_data.domains[e->id] = &pd->genpd; + } + + err = of_genpd_add_provider_onecell(dev->of_node, &pmb->genpd_onecell_data); + if (err) { + dev_err(dev, "failed to add genpd provider: %d\n", err); + return err; + } + + return 0; +} + +static const struct bcm_pmb_pd_data bcm_pmb_bcm4908_data[] = { + { .name = "pcie2", .id = BCM_PMB_PCIE2, .bus = 0, .device = 2, }, + { .name = "pcie0", .id = BCM_PMB_PCIE0, .bus = 1, .device = 14, }, + { .name = "pcie1", .id = BCM_PMB_PCIE1, .bus = 1, .device = 15, }, + { .name = "usb", .id = BCM_PMB_HOST_USB, .bus = 1, .device = 17, }, + { }, +}; + +static const struct bcm_pmb_pd_data bcm_pmb_bcm63138_data[] = { + { .name = "sata", .id = BCM_PMB_SATA, .bus = 0, .device = 3, }, + { }, +}; + +static const struct of_device_id bcm_pmb_of_match[] = { + { .compatible = "brcm,bcm4908-pmb", .data = &bcm_pmb_bcm4908_data, }, + { .compatible = "brcm,bcm63138-pmb", .data = &bcm_pmb_bcm63138_data, }, + { }, +}; + +static struct platform_driver bcm_pmb_driver = { + .driver = { + .name = "bcm-pmb", + .of_match_table = bcm_pmb_of_match, + }, + .probe = bcm_pmb_probe, +}; + +builtin_platform_driver(bcm_pmb_driver); diff --git a/drivers/pmdomain/bcm/bcm2835-power.c b/drivers/pmdomain/bcm/bcm2835-power.c new file mode 100644 index 000000000000..1a179d4e011c --- /dev/null +++ b/drivers/pmdomain/bcm/bcm2835-power.c @@ -0,0 +1,713 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* + * Power domain driver for Broadcom BCM2835 + * + * Copyright (C) 2018 Broadcom + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#define PM_GNRIC 0x00 +#define PM_AUDIO 0x04 +#define PM_STATUS 0x18 +#define PM_RSTC 0x1c +#define PM_RSTS 0x20 +#define PM_WDOG 0x24 +#define PM_PADS0 0x28 +#define PM_PADS2 0x2c +#define PM_PADS3 0x30 +#define PM_PADS4 0x34 +#define PM_PADS5 0x38 +#define PM_PADS6 0x3c +#define PM_CAM0 0x44 +#define PM_CAM0_LDOHPEN BIT(2) +#define PM_CAM0_LDOLPEN BIT(1) +#define PM_CAM0_CTRLEN BIT(0) + +#define PM_CAM1 0x48 +#define PM_CAM1_LDOHPEN BIT(2) +#define PM_CAM1_LDOLPEN BIT(1) +#define PM_CAM1_CTRLEN BIT(0) + +#define PM_CCP2TX 0x4c +#define PM_CCP2TX_LDOEN BIT(1) +#define PM_CCP2TX_CTRLEN BIT(0) + +#define PM_DSI0 0x50 +#define PM_DSI0_LDOHPEN BIT(2) +#define PM_DSI0_LDOLPEN BIT(1) +#define PM_DSI0_CTRLEN BIT(0) + +#define PM_DSI1 0x54 +#define PM_DSI1_LDOHPEN BIT(2) +#define PM_DSI1_LDOLPEN BIT(1) +#define PM_DSI1_CTRLEN BIT(0) + +#define PM_HDMI 0x58 +#define PM_HDMI_RSTDR BIT(19) +#define PM_HDMI_LDOPD BIT(1) +#define PM_HDMI_CTRLEN BIT(0) + +#define PM_USB 0x5c +/* The power gates must be enabled with this bit before enabling the LDO in the + * USB block. + */ +#define PM_USB_CTRLEN BIT(0) + +#define PM_PXLDO 0x60 +#define PM_PXBG 0x64 +#define PM_DFT 0x68 +#define PM_SMPS 0x6c +#define PM_XOSC 0x70 +#define PM_SPAREW 0x74 +#define PM_SPARER 0x78 +#define PM_AVS_RSTDR 0x7c +#define PM_AVS_STAT 0x80 +#define PM_AVS_EVENT 0x84 +#define PM_AVS_INTEN 0x88 +#define PM_DUMMY 0xfc + +#define PM_IMAGE 0x108 +#define PM_GRAFX 0x10c +#define PM_PROC 0x110 +#define PM_ENAB BIT(12) +#define PM_ISPRSTN BIT(8) +#define PM_H264RSTN BIT(7) +#define PM_PERIRSTN BIT(6) +#define PM_V3DRSTN BIT(6) +#define PM_ISFUNC BIT(5) +#define PM_MRDONE BIT(4) +#define PM_MEMREP BIT(3) +#define PM_ISPOW BIT(2) +#define PM_POWOK BIT(1) +#define PM_POWUP BIT(0) +#define PM_INRUSH_SHIFT 13 +#define PM_INRUSH_3_5_MA 0 +#define PM_INRUSH_5_MA 1 +#define PM_INRUSH_10_MA 2 +#define PM_INRUSH_20_MA 3 +#define PM_INRUSH_MASK (3 << PM_INRUSH_SHIFT) + +#define PM_PASSWORD 0x5a000000 + +#define PM_WDOG_TIME_SET 0x000fffff +#define PM_RSTC_WRCFG_CLR 0xffffffcf +#define PM_RSTS_HADWRH_SET 0x00000040 +#define PM_RSTC_WRCFG_SET 0x00000030 +#define PM_RSTC_WRCFG_FULL_RESET 0x00000020 +#define PM_RSTC_RESET 0x00000102 + +#define PM_READ(reg) readl(power->base + (reg)) +#define PM_WRITE(reg, val) writel(PM_PASSWORD | (val), power->base + (reg)) + +#define ASB_BRDG_VERSION 0x00 +#define ASB_CPR_CTRL 0x04 + +#define ASB_V3D_S_CTRL 0x08 +#define ASB_V3D_M_CTRL 0x0c +#define ASB_ISP_S_CTRL 0x10 +#define ASB_ISP_M_CTRL 0x14 +#define ASB_H264_S_CTRL 0x18 +#define ASB_H264_M_CTRL 0x1c + +#define ASB_REQ_STOP BIT(0) +#define ASB_ACK BIT(1) +#define ASB_EMPTY BIT(2) +#define ASB_FULL BIT(3) + +#define ASB_AXI_BRDG_ID 0x20 + +#define BCM2835_BRDG_ID 0x62726467 + +struct bcm2835_power_domain { + struct generic_pm_domain base; + struct bcm2835_power *power; + u32 domain; + struct clk *clk; +}; + +struct bcm2835_power { + struct device *dev; + /* PM registers. */ + void __iomem *base; + /* AXI Async bridge registers. */ + void __iomem *asb; + /* RPiVid bridge registers. */ + void __iomem *rpivid_asb; + + struct genpd_onecell_data pd_xlate; + struct bcm2835_power_domain domains[BCM2835_POWER_DOMAIN_COUNT]; + struct reset_controller_dev reset; +}; + +static int bcm2835_asb_control(struct bcm2835_power *power, u32 reg, bool enable) +{ + void __iomem *base = power->asb; + u64 start; + u32 val; + + switch (reg) { + case 0: + return 0; + case ASB_V3D_S_CTRL: + case ASB_V3D_M_CTRL: + if (power->rpivid_asb) + base = power->rpivid_asb; + break; + } + + start = ktime_get_ns(); + + /* Enable the module's async AXI bridges. */ + if (enable) { + val = readl(base + reg) & ~ASB_REQ_STOP; + } else { + val = readl(base + reg) | ASB_REQ_STOP; + } + writel(PM_PASSWORD | val, base + reg); + + while (readl(base + reg) & ASB_ACK) { + cpu_relax(); + if (ktime_get_ns() - start >= 1000) + return -ETIMEDOUT; + } + + return 0; +} + +static int bcm2835_asb_enable(struct bcm2835_power *power, u32 reg) +{ + return bcm2835_asb_control(power, reg, true); +} + +static int bcm2835_asb_disable(struct bcm2835_power *power, u32 reg) +{ + return bcm2835_asb_control(power, reg, false); +} + +static int bcm2835_power_power_off(struct bcm2835_power_domain *pd, u32 pm_reg) +{ + struct bcm2835_power *power = pd->power; + + /* We don't run this on BCM2711 */ + if (power->rpivid_asb) + return 0; + + /* Enable functional isolation */ + PM_WRITE(pm_reg, PM_READ(pm_reg) & ~PM_ISFUNC); + + /* Enable electrical isolation */ + PM_WRITE(pm_reg, PM_READ(pm_reg) & ~PM_ISPOW); + + /* Open the power switches. */ + PM_WRITE(pm_reg, PM_READ(pm_reg) & ~PM_POWUP); + + return 0; +} + +static int bcm2835_power_power_on(struct bcm2835_power_domain *pd, u32 pm_reg) +{ + struct bcm2835_power *power = pd->power; + struct device *dev = power->dev; + u64 start; + int ret; + int inrush; + bool powok; + + /* We don't run this on BCM2711 */ + if (power->rpivid_asb) + return 0; + + /* If it was already powered on by the fw, leave it that way. */ + if (PM_READ(pm_reg) & PM_POWUP) + return 0; + + /* Enable power. Allowing too much current at once may result + * in POWOK never getting set, so start low and ramp it up as + * necessary to succeed. + */ + powok = false; + for (inrush = PM_INRUSH_3_5_MA; inrush <= PM_INRUSH_20_MA; inrush++) { + PM_WRITE(pm_reg, + (PM_READ(pm_reg) & ~PM_INRUSH_MASK) | + (inrush << PM_INRUSH_SHIFT) | + PM_POWUP); + + start = ktime_get_ns(); + while (!(powok = !!(PM_READ(pm_reg) & PM_POWOK))) { + cpu_relax(); + if (ktime_get_ns() - start >= 3000) + break; + } + } + if (!powok) { + dev_err(dev, "Timeout waiting for %s power OK\n", + pd->base.name); + ret = -ETIMEDOUT; + goto err_disable_powup; + } + + /* Disable electrical isolation */ + PM_WRITE(pm_reg, PM_READ(pm_reg) | PM_ISPOW); + + /* Repair memory */ + PM_WRITE(pm_reg, PM_READ(pm_reg) | PM_MEMREP); + start = ktime_get_ns(); + while (!(PM_READ(pm_reg) & PM_MRDONE)) { + cpu_relax(); + if (ktime_get_ns() - start >= 1000) { + dev_err(dev, "Timeout waiting for %s memory repair\n", + pd->base.name); + ret = -ETIMEDOUT; + goto err_disable_ispow; + } + } + + /* Disable functional isolation */ + PM_WRITE(pm_reg, PM_READ(pm_reg) | PM_ISFUNC); + + return 0; + +err_disable_ispow: + PM_WRITE(pm_reg, PM_READ(pm_reg) & ~PM_ISPOW); +err_disable_powup: + PM_WRITE(pm_reg, PM_READ(pm_reg) & ~(PM_POWUP | PM_INRUSH_MASK)); + return ret; +} + +static int bcm2835_asb_power_on(struct bcm2835_power_domain *pd, + u32 pm_reg, + u32 asb_m_reg, + u32 asb_s_reg, + u32 reset_flags) +{ + struct bcm2835_power *power = pd->power; + int ret; + + ret = clk_prepare_enable(pd->clk); + if (ret) { + dev_err(power->dev, "Failed to enable clock for %s\n", + pd->base.name); + return ret; + } + + /* Wait 32 clocks for reset to propagate, 1 us will be enough */ + udelay(1); + + clk_disable_unprepare(pd->clk); + + /* Deassert the resets. */ + PM_WRITE(pm_reg, PM_READ(pm_reg) | reset_flags); + + ret = clk_prepare_enable(pd->clk); + if (ret) { + dev_err(power->dev, "Failed to enable clock for %s\n", + pd->base.name); + goto err_enable_resets; + } + + ret = bcm2835_asb_enable(power, asb_m_reg); + if (ret) { + dev_err(power->dev, "Failed to enable ASB master for %s\n", + pd->base.name); + goto err_disable_clk; + } + ret = bcm2835_asb_enable(power, asb_s_reg); + if (ret) { + dev_err(power->dev, "Failed to enable ASB slave for %s\n", + pd->base.name); + goto err_disable_asb_master; + } + + return 0; + +err_disable_asb_master: + bcm2835_asb_disable(power, asb_m_reg); +err_disable_clk: + clk_disable_unprepare(pd->clk); +err_enable_resets: + PM_WRITE(pm_reg, PM_READ(pm_reg) & ~reset_flags); + return ret; +} + +static int bcm2835_asb_power_off(struct bcm2835_power_domain *pd, + u32 pm_reg, + u32 asb_m_reg, + u32 asb_s_reg, + u32 reset_flags) +{ + struct bcm2835_power *power = pd->power; + int ret; + + ret = bcm2835_asb_disable(power, asb_s_reg); + if (ret) { + dev_warn(power->dev, "Failed to disable ASB slave for %s\n", + pd->base.name); + return ret; + } + ret = bcm2835_asb_disable(power, asb_m_reg); + if (ret) { + dev_warn(power->dev, "Failed to disable ASB master for %s\n", + pd->base.name); + bcm2835_asb_enable(power, asb_s_reg); + return ret; + } + + clk_disable_unprepare(pd->clk); + + /* Assert the resets. */ + PM_WRITE(pm_reg, PM_READ(pm_reg) & ~reset_flags); + + return 0; +} + +static int bcm2835_power_pd_power_on(struct generic_pm_domain *domain) +{ + struct bcm2835_power_domain *pd = + container_of(domain, struct bcm2835_power_domain, base); + struct bcm2835_power *power = pd->power; + + switch (pd->domain) { + case BCM2835_POWER_DOMAIN_GRAFX: + return bcm2835_power_power_on(pd, PM_GRAFX); + + case BCM2835_POWER_DOMAIN_GRAFX_V3D: + return bcm2835_asb_power_on(pd, PM_GRAFX, + ASB_V3D_M_CTRL, ASB_V3D_S_CTRL, + PM_V3DRSTN); + + case BCM2835_POWER_DOMAIN_IMAGE: + return bcm2835_power_power_on(pd, PM_IMAGE); + + case BCM2835_POWER_DOMAIN_IMAGE_PERI: + return bcm2835_asb_power_on(pd, PM_IMAGE, + 0, 0, + PM_PERIRSTN); + + case BCM2835_POWER_DOMAIN_IMAGE_ISP: + return bcm2835_asb_power_on(pd, PM_IMAGE, + ASB_ISP_M_CTRL, ASB_ISP_S_CTRL, + PM_ISPRSTN); + + case BCM2835_POWER_DOMAIN_IMAGE_H264: + return bcm2835_asb_power_on(pd, PM_IMAGE, + ASB_H264_M_CTRL, ASB_H264_S_CTRL, + PM_H264RSTN); + + case BCM2835_POWER_DOMAIN_USB: + PM_WRITE(PM_USB, PM_USB_CTRLEN); + return 0; + + case BCM2835_POWER_DOMAIN_DSI0: + PM_WRITE(PM_DSI0, PM_DSI0_CTRLEN); + PM_WRITE(PM_DSI0, PM_DSI0_CTRLEN | PM_DSI0_LDOHPEN); + return 0; + + case BCM2835_POWER_DOMAIN_DSI1: + PM_WRITE(PM_DSI1, PM_DSI1_CTRLEN); + PM_WRITE(PM_DSI1, PM_DSI1_CTRLEN | PM_DSI1_LDOHPEN); + return 0; + + case BCM2835_POWER_DOMAIN_CCP2TX: + PM_WRITE(PM_CCP2TX, PM_CCP2TX_CTRLEN); + PM_WRITE(PM_CCP2TX, PM_CCP2TX_CTRLEN | PM_CCP2TX_LDOEN); + return 0; + + case BCM2835_POWER_DOMAIN_HDMI: + PM_WRITE(PM_HDMI, PM_READ(PM_HDMI) | PM_HDMI_RSTDR); + PM_WRITE(PM_HDMI, PM_READ(PM_HDMI) | PM_HDMI_CTRLEN); + PM_WRITE(PM_HDMI, PM_READ(PM_HDMI) & ~PM_HDMI_LDOPD); + usleep_range(100, 200); + PM_WRITE(PM_HDMI, PM_READ(PM_HDMI) & ~PM_HDMI_RSTDR); + return 0; + + default: + dev_err(power->dev, "Invalid domain %d\n", pd->domain); + return -EINVAL; + } +} + +static int bcm2835_power_pd_power_off(struct generic_pm_domain *domain) +{ + struct bcm2835_power_domain *pd = + container_of(domain, struct bcm2835_power_domain, base); + struct bcm2835_power *power = pd->power; + + switch (pd->domain) { + case BCM2835_POWER_DOMAIN_GRAFX: + return bcm2835_power_power_off(pd, PM_GRAFX); + + case BCM2835_POWER_DOMAIN_GRAFX_V3D: + return bcm2835_asb_power_off(pd, PM_GRAFX, + ASB_V3D_M_CTRL, ASB_V3D_S_CTRL, + PM_V3DRSTN); + + case BCM2835_POWER_DOMAIN_IMAGE: + return bcm2835_power_power_off(pd, PM_IMAGE); + + case BCM2835_POWER_DOMAIN_IMAGE_PERI: + return bcm2835_asb_power_off(pd, PM_IMAGE, + 0, 0, + PM_PERIRSTN); + + case BCM2835_POWER_DOMAIN_IMAGE_ISP: + return bcm2835_asb_power_off(pd, PM_IMAGE, + ASB_ISP_M_CTRL, ASB_ISP_S_CTRL, + PM_ISPRSTN); + + case BCM2835_POWER_DOMAIN_IMAGE_H264: + return bcm2835_asb_power_off(pd, PM_IMAGE, + ASB_H264_M_CTRL, ASB_H264_S_CTRL, + PM_H264RSTN); + + case BCM2835_POWER_DOMAIN_USB: + PM_WRITE(PM_USB, 0); + return 0; + + case BCM2835_POWER_DOMAIN_DSI0: + PM_WRITE(PM_DSI0, PM_DSI0_CTRLEN); + PM_WRITE(PM_DSI0, 0); + return 0; + + case BCM2835_POWER_DOMAIN_DSI1: + PM_WRITE(PM_DSI1, PM_DSI1_CTRLEN); + PM_WRITE(PM_DSI1, 0); + return 0; + + case BCM2835_POWER_DOMAIN_CCP2TX: + PM_WRITE(PM_CCP2TX, PM_CCP2TX_CTRLEN); + PM_WRITE(PM_CCP2TX, 0); + return 0; + + case BCM2835_POWER_DOMAIN_HDMI: + PM_WRITE(PM_HDMI, PM_READ(PM_HDMI) | PM_HDMI_LDOPD); + PM_WRITE(PM_HDMI, PM_READ(PM_HDMI) & ~PM_HDMI_CTRLEN); + return 0; + + default: + dev_err(power->dev, "Invalid domain %d\n", pd->domain); + return -EINVAL; + } +} + +static int +bcm2835_init_power_domain(struct bcm2835_power *power, + int pd_xlate_index, const char *name) +{ + struct device *dev = power->dev; + struct bcm2835_power_domain *dom = &power->domains[pd_xlate_index]; + + dom->clk = devm_clk_get(dev->parent, name); + if (IS_ERR(dom->clk)) { + int ret = PTR_ERR(dom->clk); + + if (ret == -EPROBE_DEFER) + return ret; + + /* Some domains don't have a clk, so make sure that we + * don't deref an error pointer later. + */ + dom->clk = NULL; + } + + dom->base.name = name; + dom->base.power_on = bcm2835_power_pd_power_on; + dom->base.power_off = bcm2835_power_pd_power_off; + + dom->domain = pd_xlate_index; + dom->power = power; + + /* XXX: on/off at boot? */ + pm_genpd_init(&dom->base, NULL, true); + + power->pd_xlate.domains[pd_xlate_index] = &dom->base; + + return 0; +} + +/** bcm2835_reset_reset - Resets a block that has a reset line in the + * PM block. + * + * The consumer of the reset controller must have the power domain up + * -- there's no reset ability with the power domain down. To reset + * the sub-block, we just disable its access to memory through the + * ASB, reset, and re-enable. + */ +static int bcm2835_reset_reset(struct reset_controller_dev *rcdev, + unsigned long id) +{ + struct bcm2835_power *power = container_of(rcdev, struct bcm2835_power, + reset); + struct bcm2835_power_domain *pd; + int ret; + + switch (id) { + case BCM2835_RESET_V3D: + pd = &power->domains[BCM2835_POWER_DOMAIN_GRAFX_V3D]; + break; + case BCM2835_RESET_H264: + pd = &power->domains[BCM2835_POWER_DOMAIN_IMAGE_H264]; + break; + case BCM2835_RESET_ISP: + pd = &power->domains[BCM2835_POWER_DOMAIN_IMAGE_ISP]; + break; + default: + dev_err(power->dev, "Bad reset id %ld\n", id); + return -EINVAL; + } + + ret = bcm2835_power_pd_power_off(&pd->base); + if (ret) + return ret; + + return bcm2835_power_pd_power_on(&pd->base); +} + +static int bcm2835_reset_status(struct reset_controller_dev *rcdev, + unsigned long id) +{ + struct bcm2835_power *power = container_of(rcdev, struct bcm2835_power, + reset); + + switch (id) { + case BCM2835_RESET_V3D: + return !PM_READ(PM_GRAFX & PM_V3DRSTN); + case BCM2835_RESET_H264: + return !PM_READ(PM_IMAGE & PM_H264RSTN); + case BCM2835_RESET_ISP: + return !PM_READ(PM_IMAGE & PM_ISPRSTN); + default: + return -EINVAL; + } +} + +static const struct reset_control_ops bcm2835_reset_ops = { + .reset = bcm2835_reset_reset, + .status = bcm2835_reset_status, +}; + +static const char *const power_domain_names[] = { + [BCM2835_POWER_DOMAIN_GRAFX] = "grafx", + [BCM2835_POWER_DOMAIN_GRAFX_V3D] = "v3d", + + [BCM2835_POWER_DOMAIN_IMAGE] = "image", + [BCM2835_POWER_DOMAIN_IMAGE_PERI] = "peri_image", + [BCM2835_POWER_DOMAIN_IMAGE_H264] = "h264", + [BCM2835_POWER_DOMAIN_IMAGE_ISP] = "isp", + + [BCM2835_POWER_DOMAIN_USB] = "usb", + [BCM2835_POWER_DOMAIN_DSI0] = "dsi0", + [BCM2835_POWER_DOMAIN_DSI1] = "dsi1", + [BCM2835_POWER_DOMAIN_CAM0] = "cam0", + [BCM2835_POWER_DOMAIN_CAM1] = "cam1", + [BCM2835_POWER_DOMAIN_CCP2TX] = "ccp2tx", + [BCM2835_POWER_DOMAIN_HDMI] = "hdmi", +}; + +static int bcm2835_power_probe(struct platform_device *pdev) +{ + struct bcm2835_pm *pm = dev_get_drvdata(pdev->dev.parent); + struct device *dev = &pdev->dev; + struct bcm2835_power *power; + static const struct { + int parent, child; + } domain_deps[] = { + { BCM2835_POWER_DOMAIN_GRAFX, BCM2835_POWER_DOMAIN_GRAFX_V3D }, + { BCM2835_POWER_DOMAIN_IMAGE, BCM2835_POWER_DOMAIN_IMAGE_PERI }, + { BCM2835_POWER_DOMAIN_IMAGE, BCM2835_POWER_DOMAIN_IMAGE_H264 }, + { BCM2835_POWER_DOMAIN_IMAGE, BCM2835_POWER_DOMAIN_IMAGE_ISP }, + { BCM2835_POWER_DOMAIN_IMAGE_PERI, BCM2835_POWER_DOMAIN_USB }, + { BCM2835_POWER_DOMAIN_IMAGE_PERI, BCM2835_POWER_DOMAIN_CAM0 }, + { BCM2835_POWER_DOMAIN_IMAGE_PERI, BCM2835_POWER_DOMAIN_CAM1 }, + }; + int ret = 0, i; + u32 id; + + power = devm_kzalloc(dev, sizeof(*power), GFP_KERNEL); + if (!power) + return -ENOMEM; + platform_set_drvdata(pdev, power); + + power->dev = dev; + power->base = pm->base; + power->asb = pm->asb; + power->rpivid_asb = pm->rpivid_asb; + + id = readl(power->asb + ASB_AXI_BRDG_ID); + if (id != BCM2835_BRDG_ID /* "BRDG" */) { + dev_err(dev, "ASB register ID returned 0x%08x\n", id); + return -ENODEV; + } + + if (power->rpivid_asb) { + id = readl(power->rpivid_asb + ASB_AXI_BRDG_ID); + if (id != BCM2835_BRDG_ID /* "BRDG" */) { + dev_err(dev, "RPiVid ASB register ID returned 0x%08x\n", + id); + return -ENODEV; + } + } + + power->pd_xlate.domains = devm_kcalloc(dev, + ARRAY_SIZE(power_domain_names), + sizeof(*power->pd_xlate.domains), + GFP_KERNEL); + if (!power->pd_xlate.domains) + return -ENOMEM; + + power->pd_xlate.num_domains = ARRAY_SIZE(power_domain_names); + + for (i = 0; i < ARRAY_SIZE(power_domain_names); i++) { + ret = bcm2835_init_power_domain(power, i, power_domain_names[i]); + if (ret) + goto fail; + } + + for (i = 0; i < ARRAY_SIZE(domain_deps); i++) { + pm_genpd_add_subdomain(&power->domains[domain_deps[i].parent].base, + &power->domains[domain_deps[i].child].base); + } + + power->reset.owner = THIS_MODULE; + power->reset.nr_resets = BCM2835_RESET_COUNT; + power->reset.ops = &bcm2835_reset_ops; + power->reset.of_node = dev->parent->of_node; + + ret = devm_reset_controller_register(dev, &power->reset); + if (ret) + goto fail; + + of_genpd_add_provider_onecell(dev->parent->of_node, &power->pd_xlate); + + dev_info(dev, "Broadcom BCM2835 power domains driver"); + return 0; + +fail: + for (i = 0; i < ARRAY_SIZE(power_domain_names); i++) { + struct generic_pm_domain *dom = &power->domains[i].base; + + if (dom->name) + pm_genpd_remove(dom); + } + return ret; +} + +static struct platform_driver bcm2835_power_driver = { + .probe = bcm2835_power_probe, + .driver = { + .name = "bcm2835-power", + }, +}; +module_platform_driver(bcm2835_power_driver); + +MODULE_AUTHOR("Eric Anholt "); +MODULE_DESCRIPTION("Driver for Broadcom BCM2835 PM power domains and reset"); diff --git a/drivers/pmdomain/bcm/bcm63xx-power.c b/drivers/pmdomain/bcm/bcm63xx-power.c new file mode 100644 index 000000000000..98b0c2430dbc --- /dev/null +++ b/drivers/pmdomain/bcm/bcm63xx-power.c @@ -0,0 +1,375 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * BCM63xx Power Domain Controller Driver + * + * Copyright (C) 2020 Álvaro Fernández Rojas + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +struct bcm63xx_power_dev { + struct generic_pm_domain genpd; + struct bcm63xx_power *power; + uint32_t mask; +}; + +struct bcm63xx_power { + void __iomem *base; + spinlock_t lock; + struct bcm63xx_power_dev *dev; + struct genpd_onecell_data genpd_data; + struct generic_pm_domain **genpd; +}; + +struct bcm63xx_power_data { + const char * const name; + uint8_t bit; + unsigned int flags; +}; + +static int bcm63xx_power_get_state(struct bcm63xx_power_dev *pmd, bool *is_on) +{ + struct bcm63xx_power *power = pmd->power; + + if (!pmd->mask) { + *is_on = false; + return -EINVAL; + } + + *is_on = !(__raw_readl(power->base) & pmd->mask); + + return 0; +} + +static int bcm63xx_power_set_state(struct bcm63xx_power_dev *pmd, bool on) +{ + struct bcm63xx_power *power = pmd->power; + unsigned long flags; + uint32_t val; + + if (!pmd->mask) + return -EINVAL; + + spin_lock_irqsave(&power->lock, flags); + val = __raw_readl(power->base); + if (on) + val &= ~pmd->mask; + else + val |= pmd->mask; + __raw_writel(val, power->base); + spin_unlock_irqrestore(&power->lock, flags); + + return 0; +} + +static int bcm63xx_power_on(struct generic_pm_domain *genpd) +{ + struct bcm63xx_power_dev *pmd = container_of(genpd, + struct bcm63xx_power_dev, genpd); + + return bcm63xx_power_set_state(pmd, true); +} + +static int bcm63xx_power_off(struct generic_pm_domain *genpd) +{ + struct bcm63xx_power_dev *pmd = container_of(genpd, + struct bcm63xx_power_dev, genpd); + + return bcm63xx_power_set_state(pmd, false); +} + +static int bcm63xx_power_probe(struct platform_device *pdev) +{ + struct device *dev = &pdev->dev; + struct device_node *np = dev->of_node; + const struct bcm63xx_power_data *entry, *table; + struct bcm63xx_power *power; + unsigned int ndom; + uint8_t max_bit = 0; + int ret; + + power = devm_kzalloc(dev, sizeof(*power), GFP_KERNEL); + if (!power) + return -ENOMEM; + + power->base = devm_platform_ioremap_resource(pdev, 0); + if (IS_ERR(power->base)) + return PTR_ERR(power->base); + + table = of_device_get_match_data(dev); + if (!table) + return -EINVAL; + + power->genpd_data.num_domains = 0; + ndom = 0; + for (entry = table; entry->name; entry++) { + max_bit = max(max_bit, entry->bit); + ndom++; + } + + if (!ndom) + return -ENODEV; + + power->genpd_data.num_domains = max_bit + 1; + + power->dev = devm_kcalloc(dev, power->genpd_data.num_domains, + sizeof(struct bcm63xx_power_dev), + GFP_KERNEL); + if (!power->dev) + return -ENOMEM; + + power->genpd = devm_kcalloc(dev, power->genpd_data.num_domains, + sizeof(struct generic_pm_domain *), + GFP_KERNEL); + if (!power->genpd) + return -ENOMEM; + + power->genpd_data.domains = power->genpd; + + ndom = 0; + for (entry = table; entry->name; entry++) { + struct bcm63xx_power_dev *pmd = &power->dev[ndom]; + bool is_on; + + pmd->power = power; + pmd->mask = BIT(entry->bit); + pmd->genpd.name = entry->name; + pmd->genpd.flags = entry->flags; + + ret = bcm63xx_power_get_state(pmd, &is_on); + if (ret) + dev_warn(dev, "unable to get current state for %s\n", + pmd->genpd.name); + + pmd->genpd.power_on = bcm63xx_power_on; + pmd->genpd.power_off = bcm63xx_power_off; + + pm_genpd_init(&pmd->genpd, NULL, !is_on); + power->genpd[entry->bit] = &pmd->genpd; + + ndom++; + } + + spin_lock_init(&power->lock); + + ret = of_genpd_add_provider_onecell(np, &power->genpd_data); + if (ret) { + dev_err(dev, "failed to register genpd driver: %d\n", ret); + return ret; + } + + dev_info(dev, "registered %u power domains\n", ndom); + + return 0; +} + +static const struct bcm63xx_power_data bcm6318_power_domains[] = { + { + .name = "pcie", + .bit = BCM6318_POWER_DOMAIN_PCIE, + }, { + .name = "usb", + .bit = BCM6318_POWER_DOMAIN_USB, + }, { + .name = "ephy0", + .bit = BCM6318_POWER_DOMAIN_EPHY0, + }, { + .name = "ephy1", + .bit = BCM6318_POWER_DOMAIN_EPHY1, + }, { + .name = "ephy2", + .bit = BCM6318_POWER_DOMAIN_EPHY2, + }, { + .name = "ephy3", + .bit = BCM6318_POWER_DOMAIN_EPHY3, + }, { + .name = "ldo2p5", + .bit = BCM6318_POWER_DOMAIN_LDO2P5, + .flags = GENPD_FLAG_ALWAYS_ON, + }, { + .name = "ldo2p9", + .bit = BCM6318_POWER_DOMAIN_LDO2P9, + .flags = GENPD_FLAG_ALWAYS_ON, + }, { + .name = "sw1p0", + .bit = BCM6318_POWER_DOMAIN_SW1P0, + .flags = GENPD_FLAG_ALWAYS_ON, + }, { + .name = "pad", + .bit = BCM6318_POWER_DOMAIN_PAD, + .flags = GENPD_FLAG_ALWAYS_ON, + }, { + /* sentinel */ + }, +}; + +static const struct bcm63xx_power_data bcm6328_power_domains[] = { + { + .name = "adsl2-mips", + .bit = BCM6328_POWER_DOMAIN_ADSL2_MIPS, + }, { + .name = "adsl2-phy", + .bit = BCM6328_POWER_DOMAIN_ADSL2_PHY, + }, { + .name = "adsl2-afe", + .bit = BCM6328_POWER_DOMAIN_ADSL2_AFE, + }, { + .name = "sar", + .bit = BCM6328_POWER_DOMAIN_SAR, + }, { + .name = "pcm", + .bit = BCM6328_POWER_DOMAIN_PCM, + }, { + .name = "usbd", + .bit = BCM6328_POWER_DOMAIN_USBD, + }, { + .name = "usbh", + .bit = BCM6328_POWER_DOMAIN_USBH, + }, { + .name = "pcie", + .bit = BCM6328_POWER_DOMAIN_PCIE, + }, { + .name = "robosw", + .bit = BCM6328_POWER_DOMAIN_ROBOSW, + }, { + .name = "ephy", + .bit = BCM6328_POWER_DOMAIN_EPHY, + }, { + /* sentinel */ + }, +}; + +static const struct bcm63xx_power_data bcm6362_power_domains[] = { + { + .name = "sar", + .bit = BCM6362_POWER_DOMAIN_SAR, + }, { + .name = "ipsec", + .bit = BCM6362_POWER_DOMAIN_IPSEC, + }, { + .name = "mips", + .bit = BCM6362_POWER_DOMAIN_MIPS, + .flags = GENPD_FLAG_ALWAYS_ON, + }, { + .name = "dect", + .bit = BCM6362_POWER_DOMAIN_DECT, + }, { + .name = "usbh", + .bit = BCM6362_POWER_DOMAIN_USBH, + }, { + .name = "usbd", + .bit = BCM6362_POWER_DOMAIN_USBD, + }, { + .name = "robosw", + .bit = BCM6362_POWER_DOMAIN_ROBOSW, + }, { + .name = "pcm", + .bit = BCM6362_POWER_DOMAIN_PCM, + }, { + .name = "periph", + .bit = BCM6362_POWER_DOMAIN_PERIPH, + .flags = GENPD_FLAG_ALWAYS_ON, + }, { + .name = "adsl-phy", + .bit = BCM6362_POWER_DOMAIN_ADSL_PHY, + }, { + .name = "gmii-pads", + .bit = BCM6362_POWER_DOMAIN_GMII_PADS, + }, { + .name = "fap", + .bit = BCM6362_POWER_DOMAIN_FAP, + }, { + .name = "pcie", + .bit = BCM6362_POWER_DOMAIN_PCIE, + }, { + .name = "wlan-pads", + .bit = BCM6362_POWER_DOMAIN_WLAN_PADS, + }, { + /* sentinel */ + }, +}; + +static const struct bcm63xx_power_data bcm63268_power_domains[] = { + { + .name = "sar", + .bit = BCM63268_POWER_DOMAIN_SAR, + }, { + .name = "ipsec", + .bit = BCM63268_POWER_DOMAIN_IPSEC, + }, { + .name = "mips", + .bit = BCM63268_POWER_DOMAIN_MIPS, + .flags = GENPD_FLAG_ALWAYS_ON, + }, { + .name = "dect", + .bit = BCM63268_POWER_DOMAIN_DECT, + }, { + .name = "usbh", + .bit = BCM63268_POWER_DOMAIN_USBH, + }, { + .name = "usbd", + .bit = BCM63268_POWER_DOMAIN_USBD, + }, { + .name = "robosw", + .bit = BCM63268_POWER_DOMAIN_ROBOSW, + }, { + .name = "pcm", + .bit = BCM63268_POWER_DOMAIN_PCM, + }, { + .name = "periph", + .bit = BCM63268_POWER_DOMAIN_PERIPH, + .flags = GENPD_FLAG_ALWAYS_ON, + }, { + .name = "vdsl-phy", + .bit = BCM63268_POWER_DOMAIN_VDSL_PHY, + }, { + .name = "vdsl-mips", + .bit = BCM63268_POWER_DOMAIN_VDSL_MIPS, + }, { + .name = "fap", + .bit = BCM63268_POWER_DOMAIN_FAP, + }, { + .name = "pcie", + .bit = BCM63268_POWER_DOMAIN_PCIE, + }, { + .name = "wlan-pads", + .bit = BCM63268_POWER_DOMAIN_WLAN_PADS, + }, { + /* sentinel */ + }, +}; + +static const struct of_device_id bcm63xx_power_of_match[] = { + { + .compatible = "brcm,bcm6318-power-controller", + .data = &bcm6318_power_domains, + }, { + .compatible = "brcm,bcm6328-power-controller", + .data = &bcm6328_power_domains, + }, { + .compatible = "brcm,bcm6362-power-controller", + .data = &bcm6362_power_domains, + }, { + .compatible = "brcm,bcm63268-power-controller", + .data = &bcm63268_power_domains, + }, { + /* sentinel */ + } +}; + +static struct platform_driver bcm63xx_power_driver = { + .driver = { + .name = "bcm63xx-power-controller", + .of_match_table = bcm63xx_power_of_match, + }, + .probe = bcm63xx_power_probe, +}; +builtin_platform_driver(bcm63xx_power_driver); diff --git a/drivers/pmdomain/bcm/raspberrypi-power.c b/drivers/pmdomain/bcm/raspberrypi-power.c new file mode 100644 index 000000000000..06196ebfe03b --- /dev/null +++ b/drivers/pmdomain/bcm/raspberrypi-power.c @@ -0,0 +1,245 @@ +// SPDX-License-Identifier: GPL-2.0 +/* (C) 2015 Pengutronix, Alexander Aring + * + * Authors: + * Alexander Aring + * Eric Anholt + */ + +#include +#include +#include +#include +#include +#include + +/* + * Firmware indices for the old power domains interface. Only a few + * of them were actually implemented. + */ +#define RPI_OLD_POWER_DOMAIN_USB 3 +#define RPI_OLD_POWER_DOMAIN_V3D 10 + +struct rpi_power_domain { + u32 domain; + bool enabled; + bool old_interface; + struct generic_pm_domain base; + struct rpi_firmware *fw; +}; + +struct rpi_power_domains { + bool has_new_interface; + struct genpd_onecell_data xlate; + struct rpi_firmware *fw; + struct rpi_power_domain domains[RPI_POWER_DOMAIN_COUNT]; +}; + +/* + * Packet definition used by RPI_FIRMWARE_SET_POWER_STATE and + * RPI_FIRMWARE_SET_DOMAIN_STATE + */ +struct rpi_power_domain_packet { + u32 domain; + u32 on; +}; + +/* + * Asks the firmware to enable or disable power on a specific power + * domain. + */ +static int rpi_firmware_set_power(struct rpi_power_domain *rpi_domain, bool on) +{ + struct rpi_power_domain_packet packet; + + packet.domain = rpi_domain->domain; + packet.on = on; + return rpi_firmware_property(rpi_domain->fw, + rpi_domain->old_interface ? + RPI_FIRMWARE_SET_POWER_STATE : + RPI_FIRMWARE_SET_DOMAIN_STATE, + &packet, sizeof(packet)); +} + +static int rpi_domain_off(struct generic_pm_domain *domain) +{ + struct rpi_power_domain *rpi_domain = + container_of(domain, struct rpi_power_domain, base); + + return rpi_firmware_set_power(rpi_domain, false); +} + +static int rpi_domain_on(struct generic_pm_domain *domain) +{ + struct rpi_power_domain *rpi_domain = + container_of(domain, struct rpi_power_domain, base); + + return rpi_firmware_set_power(rpi_domain, true); +} + +static void rpi_common_init_power_domain(struct rpi_power_domains *rpi_domains, + int xlate_index, const char *name) +{ + struct rpi_power_domain *dom = &rpi_domains->domains[xlate_index]; + + dom->fw = rpi_domains->fw; + + dom->base.name = name; + dom->base.power_on = rpi_domain_on; + dom->base.power_off = rpi_domain_off; + + /* + * Treat all power domains as off at boot. + * + * The firmware itself may be keeping some domains on, but + * from Linux's perspective all we control is the refcounts + * that we give to the firmware, and we can't ask the firmware + * to turn off something that we haven't ourselves turned on. + */ + pm_genpd_init(&dom->base, NULL, true); + + rpi_domains->xlate.domains[xlate_index] = &dom->base; +} + +static void rpi_init_power_domain(struct rpi_power_domains *rpi_domains, + int xlate_index, const char *name) +{ + struct rpi_power_domain *dom = &rpi_domains->domains[xlate_index]; + + if (!rpi_domains->has_new_interface) + return; + + /* The DT binding index is the firmware's domain index minus one. */ + dom->domain = xlate_index + 1; + + rpi_common_init_power_domain(rpi_domains, xlate_index, name); +} + +static void rpi_init_old_power_domain(struct rpi_power_domains *rpi_domains, + int xlate_index, int domain, + const char *name) +{ + struct rpi_power_domain *dom = &rpi_domains->domains[xlate_index]; + + dom->old_interface = true; + dom->domain = domain; + + rpi_common_init_power_domain(rpi_domains, xlate_index, name); +} + +/* + * Detects whether the firmware supports the new power domains interface. + * + * The firmware doesn't actually return an error on an unknown tag, + * and just skips over it, so we do the detection by putting an + * unexpected value in the return field and checking if it was + * unchanged. + */ +static bool +rpi_has_new_domain_support(struct rpi_power_domains *rpi_domains) +{ + struct rpi_power_domain_packet packet; + int ret; + + packet.domain = RPI_POWER_DOMAIN_ARM; + packet.on = ~0; + + ret = rpi_firmware_property(rpi_domains->fw, + RPI_FIRMWARE_GET_DOMAIN_STATE, + &packet, sizeof(packet)); + + return ret == 0 && packet.on != ~0; +} + +static int rpi_power_probe(struct platform_device *pdev) +{ + struct device_node *fw_np; + struct device *dev = &pdev->dev; + struct rpi_power_domains *rpi_domains; + + rpi_domains = devm_kzalloc(dev, sizeof(*rpi_domains), GFP_KERNEL); + if (!rpi_domains) + return -ENOMEM; + + rpi_domains->xlate.domains = + devm_kcalloc(dev, + RPI_POWER_DOMAIN_COUNT, + sizeof(*rpi_domains->xlate.domains), + GFP_KERNEL); + if (!rpi_domains->xlate.domains) + return -ENOMEM; + + rpi_domains->xlate.num_domains = RPI_POWER_DOMAIN_COUNT; + + fw_np = of_parse_phandle(pdev->dev.of_node, "firmware", 0); + if (!fw_np) { + dev_err(&pdev->dev, "no firmware node\n"); + return -ENODEV; + } + + rpi_domains->fw = devm_rpi_firmware_get(&pdev->dev, fw_np); + of_node_put(fw_np); + if (!rpi_domains->fw) + return -EPROBE_DEFER; + + rpi_domains->has_new_interface = + rpi_has_new_domain_support(rpi_domains); + + rpi_init_power_domain(rpi_domains, RPI_POWER_DOMAIN_I2C0, "I2C0"); + rpi_init_power_domain(rpi_domains, RPI_POWER_DOMAIN_I2C1, "I2C1"); + rpi_init_power_domain(rpi_domains, RPI_POWER_DOMAIN_I2C2, "I2C2"); + rpi_init_power_domain(rpi_domains, RPI_POWER_DOMAIN_VIDEO_SCALER, + "VIDEO_SCALER"); + rpi_init_power_domain(rpi_domains, RPI_POWER_DOMAIN_VPU1, "VPU1"); + rpi_init_power_domain(rpi_domains, RPI_POWER_DOMAIN_HDMI, "HDMI"); + + /* + * Use the old firmware interface for USB power, so that we + * can turn it on even if the firmware hasn't been updated. + */ + rpi_init_old_power_domain(rpi_domains, RPI_POWER_DOMAIN_USB, + RPI_OLD_POWER_DOMAIN_USB, "USB"); + + rpi_init_power_domain(rpi_domains, RPI_POWER_DOMAIN_VEC, "VEC"); + rpi_init_power_domain(rpi_domains, RPI_POWER_DOMAIN_JPEG, "JPEG"); + rpi_init_power_domain(rpi_domains, RPI_POWER_DOMAIN_H264, "H264"); + rpi_init_power_domain(rpi_domains, RPI_POWER_DOMAIN_V3D, "V3D"); + rpi_init_power_domain(rpi_domains, RPI_POWER_DOMAIN_ISP, "ISP"); + rpi_init_power_domain(rpi_domains, RPI_POWER_DOMAIN_UNICAM0, "UNICAM0"); + rpi_init_power_domain(rpi_domains, RPI_POWER_DOMAIN_UNICAM1, "UNICAM1"); + rpi_init_power_domain(rpi_domains, RPI_POWER_DOMAIN_CCP2RX, "CCP2RX"); + rpi_init_power_domain(rpi_domains, RPI_POWER_DOMAIN_CSI2, "CSI2"); + rpi_init_power_domain(rpi_domains, RPI_POWER_DOMAIN_CPI, "CPI"); + rpi_init_power_domain(rpi_domains, RPI_POWER_DOMAIN_DSI0, "DSI0"); + rpi_init_power_domain(rpi_domains, RPI_POWER_DOMAIN_DSI1, "DSI1"); + rpi_init_power_domain(rpi_domains, RPI_POWER_DOMAIN_TRANSPOSER, + "TRANSPOSER"); + rpi_init_power_domain(rpi_domains, RPI_POWER_DOMAIN_CCP2TX, "CCP2TX"); + rpi_init_power_domain(rpi_domains, RPI_POWER_DOMAIN_CDP, "CDP"); + rpi_init_power_domain(rpi_domains, RPI_POWER_DOMAIN_ARM, "ARM"); + + of_genpd_add_provider_onecell(dev->of_node, &rpi_domains->xlate); + + platform_set_drvdata(pdev, rpi_domains); + + return 0; +} + +static const struct of_device_id rpi_power_of_match[] = { + { .compatible = "raspberrypi,bcm2835-power", }, + {}, +}; +MODULE_DEVICE_TABLE(of, rpi_power_of_match); + +static struct platform_driver rpi_power_driver = { + .driver = { + .name = "raspberrypi-power", + .of_match_table = rpi_power_of_match, + }, + .probe = rpi_power_probe, +}; +builtin_platform_driver(rpi_power_driver); + +MODULE_AUTHOR("Alexander Aring "); +MODULE_AUTHOR("Eric Anholt "); +MODULE_DESCRIPTION("Raspberry Pi power domain driver"); diff --git a/drivers/pmdomain/imx/Makefile b/drivers/pmdomain/imx/Makefile new file mode 100644 index 000000000000..52d2629014a7 --- /dev/null +++ b/drivers/pmdomain/imx/Makefile @@ -0,0 +1,8 @@ +# SPDX-License-Identifier: GPL-2.0-only +obj-$(CONFIG_HAVE_IMX_GPC) += gpc.o +obj-$(CONFIG_IMX_GPCV2_PM_DOMAINS) += gpcv2.o +obj-$(CONFIG_IMX_SCU_PD) += scu-pd.o +obj-$(CONFIG_IMX8M_BLK_CTRL) += imx8m-blk-ctrl.o +obj-$(CONFIG_IMX8M_BLK_CTRL) += imx8mp-blk-ctrl.o +obj-$(CONFIG_SOC_IMX9) += imx93-pd.o +obj-$(CONFIG_IMX9_BLK_CTRL) += imx93-blk-ctrl.o diff --git a/drivers/pmdomain/imx/gpc.c b/drivers/pmdomain/imx/gpc.c new file mode 100644 index 000000000000..90a8b2c0676f --- /dev/null +++ b/drivers/pmdomain/imx/gpc.c @@ -0,0 +1,554 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* + * Copyright 2015-2017 Pengutronix, Lucas Stach + * Copyright 2011-2013 Freescale Semiconductor, Inc. + */ + +#include +#include +#include +#include +#include +#include +#include +#include + +#define GPC_CNTR 0x000 + +#define GPC_PGC_CTRL_OFFS 0x0 +#define GPC_PGC_PUPSCR_OFFS 0x4 +#define GPC_PGC_PDNSCR_OFFS 0x8 +#define GPC_PGC_SW2ISO_SHIFT 0x8 +#define GPC_PGC_SW_SHIFT 0x0 + +#define GPC_PGC_PCI_PDN 0x200 +#define GPC_PGC_PCI_SR 0x20c + +#define GPC_PGC_GPU_PDN 0x260 +#define GPC_PGC_GPU_PUPSCR 0x264 +#define GPC_PGC_GPU_PDNSCR 0x268 +#define GPC_PGC_GPU_SR 0x26c + +#define GPC_PGC_DISP_PDN 0x240 +#define GPC_PGC_DISP_SR 0x24c + +#define GPU_VPU_PUP_REQ BIT(1) +#define GPU_VPU_PDN_REQ BIT(0) + +#define GPC_CLK_MAX 7 + +#define PGC_DOMAIN_FLAG_NO_PD BIT(0) + +struct imx_pm_domain { + struct generic_pm_domain base; + struct regmap *regmap; + struct regulator *supply; + struct clk *clk[GPC_CLK_MAX]; + int num_clks; + unsigned int reg_offs; + signed char cntr_pdn_bit; + unsigned int ipg_rate_mhz; +}; + +static inline struct imx_pm_domain * +to_imx_pm_domain(struct generic_pm_domain *genpd) +{ + return container_of(genpd, struct imx_pm_domain, base); +} + +static int imx6_pm_domain_power_off(struct generic_pm_domain *genpd) +{ + struct imx_pm_domain *pd = to_imx_pm_domain(genpd); + int iso, iso2sw; + u32 val; + + /* Read ISO and ISO2SW power down delays */ + regmap_read(pd->regmap, pd->reg_offs + GPC_PGC_PDNSCR_OFFS, &val); + iso = val & 0x3f; + iso2sw = (val >> 8) & 0x3f; + + /* Gate off domain when powered down */ + regmap_update_bits(pd->regmap, pd->reg_offs + GPC_PGC_CTRL_OFFS, + 0x1, 0x1); + + /* Request GPC to power down domain */ + val = BIT(pd->cntr_pdn_bit); + regmap_update_bits(pd->regmap, GPC_CNTR, val, val); + + /* Wait ISO + ISO2SW IPG clock cycles */ + udelay(DIV_ROUND_UP(iso + iso2sw, pd->ipg_rate_mhz)); + + if (pd->supply) + regulator_disable(pd->supply); + + return 0; +} + +static int imx6_pm_domain_power_on(struct generic_pm_domain *genpd) +{ + struct imx_pm_domain *pd = to_imx_pm_domain(genpd); + int i, ret; + u32 val, req; + + if (pd->supply) { + ret = regulator_enable(pd->supply); + if (ret) { + pr_err("%s: failed to enable regulator: %d\n", + __func__, ret); + return ret; + } + } + + /* Enable reset clocks for all devices in the domain */ + for (i = 0; i < pd->num_clks; i++) + clk_prepare_enable(pd->clk[i]); + + /* Gate off domain when powered down */ + regmap_update_bits(pd->regmap, pd->reg_offs + GPC_PGC_CTRL_OFFS, + 0x1, 0x1); + + /* Request GPC to power up domain */ + req = BIT(pd->cntr_pdn_bit + 1); + regmap_update_bits(pd->regmap, GPC_CNTR, req, req); + + /* Wait for the PGC to handle the request */ + ret = regmap_read_poll_timeout(pd->regmap, GPC_CNTR, val, !(val & req), + 1, 50); + if (ret) + pr_err("powerup request on domain %s timed out\n", genpd->name); + + /* Wait for reset to propagate through peripherals */ + usleep_range(5, 10); + + /* Disable reset clocks for all devices in the domain */ + for (i = 0; i < pd->num_clks; i++) + clk_disable_unprepare(pd->clk[i]); + + return 0; +} + +static int imx_pgc_get_clocks(struct device *dev, struct imx_pm_domain *domain) +{ + int i, ret; + + for (i = 0; ; i++) { + struct clk *clk = of_clk_get(dev->of_node, i); + if (IS_ERR(clk)) + break; + if (i >= GPC_CLK_MAX) { + dev_err(dev, "more than %d clocks\n", GPC_CLK_MAX); + ret = -EINVAL; + goto clk_err; + } + domain->clk[i] = clk; + } + domain->num_clks = i; + + return 0; + +clk_err: + while (i--) + clk_put(domain->clk[i]); + + return ret; +} + +static void imx_pgc_put_clocks(struct imx_pm_domain *domain) +{ + int i; + + for (i = domain->num_clks - 1; i >= 0; i--) + clk_put(domain->clk[i]); +} + +static int imx_pgc_parse_dt(struct device *dev, struct imx_pm_domain *domain) +{ + /* try to get the domain supply regulator */ + domain->supply = devm_regulator_get_optional(dev, "power"); + if (IS_ERR(domain->supply)) { + if (PTR_ERR(domain->supply) == -ENODEV) + domain->supply = NULL; + else + return PTR_ERR(domain->supply); + } + + /* try to get all clocks needed for reset propagation */ + return imx_pgc_get_clocks(dev, domain); +} + +static int imx_pgc_power_domain_probe(struct platform_device *pdev) +{ + struct imx_pm_domain *domain = pdev->dev.platform_data; + struct device *dev = &pdev->dev; + int ret; + + /* if this PD is associated with a DT node try to parse it */ + if (dev->of_node) { + ret = imx_pgc_parse_dt(dev, domain); + if (ret) + return ret; + } + + /* initially power on the domain */ + if (domain->base.power_on) + domain->base.power_on(&domain->base); + + if (IS_ENABLED(CONFIG_PM_GENERIC_DOMAINS)) { + pm_genpd_init(&domain->base, NULL, false); + ret = of_genpd_add_provider_simple(dev->of_node, &domain->base); + if (ret) + goto genpd_err; + } + + device_link_add(dev, dev->parent, DL_FLAG_AUTOREMOVE_CONSUMER); + + return 0; + +genpd_err: + pm_genpd_remove(&domain->base); + imx_pgc_put_clocks(domain); + + return ret; +} + +static int imx_pgc_power_domain_remove(struct platform_device *pdev) +{ + struct imx_pm_domain *domain = pdev->dev.platform_data; + + if (IS_ENABLED(CONFIG_PM_GENERIC_DOMAINS)) { + of_genpd_del_provider(pdev->dev.of_node); + pm_genpd_remove(&domain->base); + imx_pgc_put_clocks(domain); + } + + return 0; +} + +static const struct platform_device_id imx_pgc_power_domain_id[] = { + { "imx-pgc-power-domain"}, + { }, +}; + +static struct platform_driver imx_pgc_power_domain_driver = { + .driver = { + .name = "imx-pgc-pd", + }, + .probe = imx_pgc_power_domain_probe, + .remove = imx_pgc_power_domain_remove, + .id_table = imx_pgc_power_domain_id, +}; +builtin_platform_driver(imx_pgc_power_domain_driver) + +#define GPC_PGC_DOMAIN_ARM 0 +#define GPC_PGC_DOMAIN_PU 1 +#define GPC_PGC_DOMAIN_DISPLAY 2 +#define GPC_PGC_DOMAIN_PCI 3 + +static struct genpd_power_state imx6_pm_domain_pu_state = { + .power_off_latency_ns = 25000, + .power_on_latency_ns = 2000000, +}; + +static struct imx_pm_domain imx_gpc_domains[] = { + [GPC_PGC_DOMAIN_ARM] = { + .base = { + .name = "ARM", + .flags = GENPD_FLAG_ALWAYS_ON, + }, + }, + [GPC_PGC_DOMAIN_PU] = { + .base = { + .name = "PU", + .power_off = imx6_pm_domain_power_off, + .power_on = imx6_pm_domain_power_on, + .states = &imx6_pm_domain_pu_state, + .state_count = 1, + }, + .reg_offs = 0x260, + .cntr_pdn_bit = 0, + }, + [GPC_PGC_DOMAIN_DISPLAY] = { + .base = { + .name = "DISPLAY", + .power_off = imx6_pm_domain_power_off, + .power_on = imx6_pm_domain_power_on, + }, + .reg_offs = 0x240, + .cntr_pdn_bit = 4, + }, + [GPC_PGC_DOMAIN_PCI] = { + .base = { + .name = "PCI", + .power_off = imx6_pm_domain_power_off, + .power_on = imx6_pm_domain_power_on, + }, + .reg_offs = 0x200, + .cntr_pdn_bit = 6, + }, +}; + +struct imx_gpc_dt_data { + int num_domains; + bool err009619_present; + bool err006287_present; +}; + +static const struct imx_gpc_dt_data imx6q_dt_data = { + .num_domains = 2, + .err009619_present = false, + .err006287_present = false, +}; + +static const struct imx_gpc_dt_data imx6qp_dt_data = { + .num_domains = 2, + .err009619_present = true, + .err006287_present = false, +}; + +static const struct imx_gpc_dt_data imx6sl_dt_data = { + .num_domains = 3, + .err009619_present = false, + .err006287_present = true, +}; + +static const struct imx_gpc_dt_data imx6sx_dt_data = { + .num_domains = 4, + .err009619_present = false, + .err006287_present = false, +}; + +static const struct of_device_id imx_gpc_dt_ids[] = { + { .compatible = "fsl,imx6q-gpc", .data = &imx6q_dt_data }, + { .compatible = "fsl,imx6qp-gpc", .data = &imx6qp_dt_data }, + { .compatible = "fsl,imx6sl-gpc", .data = &imx6sl_dt_data }, + { .compatible = "fsl,imx6sx-gpc", .data = &imx6sx_dt_data }, + { } +}; + +static const struct regmap_range yes_ranges[] = { + regmap_reg_range(GPC_CNTR, GPC_CNTR), + regmap_reg_range(GPC_PGC_PCI_PDN, GPC_PGC_PCI_SR), + regmap_reg_range(GPC_PGC_GPU_PDN, GPC_PGC_GPU_SR), + regmap_reg_range(GPC_PGC_DISP_PDN, GPC_PGC_DISP_SR), +}; + +static const struct regmap_access_table access_table = { + .yes_ranges = yes_ranges, + .n_yes_ranges = ARRAY_SIZE(yes_ranges), +}; + +static const struct regmap_config imx_gpc_regmap_config = { + .reg_bits = 32, + .val_bits = 32, + .reg_stride = 4, + .rd_table = &access_table, + .wr_table = &access_table, + .max_register = 0x2ac, + .fast_io = true, +}; + +static struct generic_pm_domain *imx_gpc_onecell_domains[] = { + &imx_gpc_domains[GPC_PGC_DOMAIN_ARM].base, + &imx_gpc_domains[GPC_PGC_DOMAIN_PU].base, +}; + +static struct genpd_onecell_data imx_gpc_onecell_data = { + .domains = imx_gpc_onecell_domains, + .num_domains = 2, +}; + +static int imx_gpc_old_dt_init(struct device *dev, struct regmap *regmap, + unsigned int num_domains) +{ + struct imx_pm_domain *domain; + int i, ret; + + for (i = 0; i < num_domains; i++) { + domain = &imx_gpc_domains[i]; + domain->regmap = regmap; + domain->ipg_rate_mhz = 66; + + if (i == 1) { + domain->supply = devm_regulator_get(dev, "pu"); + if (IS_ERR(domain->supply)) + return PTR_ERR(domain->supply); + + ret = imx_pgc_get_clocks(dev, domain); + if (ret) + goto clk_err; + + domain->base.power_on(&domain->base); + } + } + + for (i = 0; i < num_domains; i++) + pm_genpd_init(&imx_gpc_domains[i].base, NULL, false); + + if (IS_ENABLED(CONFIG_PM_GENERIC_DOMAINS)) { + ret = of_genpd_add_provider_onecell(dev->of_node, + &imx_gpc_onecell_data); + if (ret) + goto genpd_err; + } + + return 0; + +genpd_err: + for (i = 0; i < num_domains; i++) + pm_genpd_remove(&imx_gpc_domains[i].base); + imx_pgc_put_clocks(&imx_gpc_domains[GPC_PGC_DOMAIN_PU]); +clk_err: + return ret; +} + +static int imx_gpc_probe(struct platform_device *pdev) +{ + const struct of_device_id *of_id = + of_match_device(imx_gpc_dt_ids, &pdev->dev); + const struct imx_gpc_dt_data *of_id_data = of_id->data; + struct device_node *pgc_node; + struct regmap *regmap; + void __iomem *base; + int ret; + + pgc_node = of_get_child_by_name(pdev->dev.of_node, "pgc"); + + /* bail out if DT too old and doesn't provide the necessary info */ + if (!of_property_read_bool(pdev->dev.of_node, "#power-domain-cells") && + !pgc_node) + return 0; + + base = devm_platform_ioremap_resource(pdev, 0); + if (IS_ERR(base)) + return PTR_ERR(base); + + regmap = devm_regmap_init_mmio_clk(&pdev->dev, NULL, base, + &imx_gpc_regmap_config); + if (IS_ERR(regmap)) { + ret = PTR_ERR(regmap); + dev_err(&pdev->dev, "failed to init regmap: %d\n", + ret); + return ret; + } + + /* + * Disable PU power down by runtime PM if ERR009619 is present. + * + * The PRE clock will be paused for several cycles when turning on the + * PU domain LDO from power down state. If PRE is in use at that time, + * the IPU/PRG cannot get the correct display data from the PRE. + * + * This is not a concern when the whole system enters suspend state, so + * it's safe to power down PU in this case. + */ + if (of_id_data->err009619_present) + imx_gpc_domains[GPC_PGC_DOMAIN_PU].base.flags |= + GENPD_FLAG_RPM_ALWAYS_ON; + + /* Keep DISP always on if ERR006287 is present */ + if (of_id_data->err006287_present) + imx_gpc_domains[GPC_PGC_DOMAIN_DISPLAY].base.flags |= + GENPD_FLAG_ALWAYS_ON; + + if (!pgc_node) { + ret = imx_gpc_old_dt_init(&pdev->dev, regmap, + of_id_data->num_domains); + if (ret) + return ret; + } else { + struct imx_pm_domain *domain; + struct platform_device *pd_pdev; + struct device_node *np; + struct clk *ipg_clk; + unsigned int ipg_rate_mhz; + int domain_index; + + ipg_clk = devm_clk_get(&pdev->dev, "ipg"); + if (IS_ERR(ipg_clk)) + return PTR_ERR(ipg_clk); + ipg_rate_mhz = clk_get_rate(ipg_clk) / 1000000; + + for_each_child_of_node(pgc_node, np) { + ret = of_property_read_u32(np, "reg", &domain_index); + if (ret) { + of_node_put(np); + return ret; + } + if (domain_index >= of_id_data->num_domains) + continue; + + pd_pdev = platform_device_alloc("imx-pgc-power-domain", + domain_index); + if (!pd_pdev) { + of_node_put(np); + return -ENOMEM; + } + + ret = platform_device_add_data(pd_pdev, + &imx_gpc_domains[domain_index], + sizeof(imx_gpc_domains[domain_index])); + if (ret) { + platform_device_put(pd_pdev); + of_node_put(np); + return ret; + } + domain = pd_pdev->dev.platform_data; + domain->regmap = regmap; + domain->ipg_rate_mhz = ipg_rate_mhz; + + pd_pdev->dev.parent = &pdev->dev; + pd_pdev->dev.of_node = np; + + ret = platform_device_add(pd_pdev); + if (ret) { + platform_device_put(pd_pdev); + of_node_put(np); + return ret; + } + } + } + + return 0; +} + +static int imx_gpc_remove(struct platform_device *pdev) +{ + struct device_node *pgc_node; + int ret; + + pgc_node = of_get_child_by_name(pdev->dev.of_node, "pgc"); + + /* bail out if DT too old and doesn't provide the necessary info */ + if (!of_property_read_bool(pdev->dev.of_node, "#power-domain-cells") && + !pgc_node) + return 0; + + /* + * If the old DT binding is used the toplevel driver needs to + * de-register the power domains + */ + if (!pgc_node) { + of_genpd_del_provider(pdev->dev.of_node); + + ret = pm_genpd_remove(&imx_gpc_domains[GPC_PGC_DOMAIN_PU].base); + if (ret) + return ret; + imx_pgc_put_clocks(&imx_gpc_domains[GPC_PGC_DOMAIN_PU]); + + ret = pm_genpd_remove(&imx_gpc_domains[GPC_PGC_DOMAIN_ARM].base); + if (ret) + return ret; + } + + return 0; +} + +static struct platform_driver imx_gpc_driver = { + .driver = { + .name = "imx-gpc", + .of_match_table = imx_gpc_dt_ids, + }, + .probe = imx_gpc_probe, + .remove = imx_gpc_remove, +}; +builtin_platform_driver(imx_gpc_driver) diff --git a/drivers/pmdomain/imx/gpcv2.c b/drivers/pmdomain/imx/gpcv2.c new file mode 100644 index 000000000000..fbd3d92f8cd8 --- /dev/null +++ b/drivers/pmdomain/imx/gpcv2.c @@ -0,0 +1,1550 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* + * Copyright 2017 Impinj, Inc + * Author: Andrey Smirnov + * + * Based on the code of analogus driver: + * + * Copyright 2015-2017 Pengutronix, Lucas Stach + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#define GPC_LPCR_A_CORE_BSC 0x000 + +#define GPC_PGC_CPU_MAPPING 0x0ec +#define IMX8MP_GPC_PGC_CPU_MAPPING 0x1cc + +#define IMX7_USB_HSIC_PHY_A_CORE_DOMAIN BIT(6) +#define IMX7_USB_OTG2_PHY_A_CORE_DOMAIN BIT(5) +#define IMX7_USB_OTG1_PHY_A_CORE_DOMAIN BIT(4) +#define IMX7_PCIE_PHY_A_CORE_DOMAIN BIT(3) +#define IMX7_MIPI_PHY_A_CORE_DOMAIN BIT(2) + +#define IMX8M_PCIE2_A53_DOMAIN BIT(15) +#define IMX8M_MIPI_CSI2_A53_DOMAIN BIT(14) +#define IMX8M_MIPI_CSI1_A53_DOMAIN BIT(13) +#define IMX8M_DISP_A53_DOMAIN BIT(12) +#define IMX8M_HDMI_A53_DOMAIN BIT(11) +#define IMX8M_VPU_A53_DOMAIN BIT(10) +#define IMX8M_GPU_A53_DOMAIN BIT(9) +#define IMX8M_DDR2_A53_DOMAIN BIT(8) +#define IMX8M_DDR1_A53_DOMAIN BIT(7) +#define IMX8M_OTG2_A53_DOMAIN BIT(5) +#define IMX8M_OTG1_A53_DOMAIN BIT(4) +#define IMX8M_PCIE1_A53_DOMAIN BIT(3) +#define IMX8M_MIPI_A53_DOMAIN BIT(2) + +#define IMX8MM_VPUH1_A53_DOMAIN BIT(15) +#define IMX8MM_VPUG2_A53_DOMAIN BIT(14) +#define IMX8MM_VPUG1_A53_DOMAIN BIT(13) +#define IMX8MM_DISPMIX_A53_DOMAIN BIT(12) +#define IMX8MM_VPUMIX_A53_DOMAIN BIT(10) +#define IMX8MM_GPUMIX_A53_DOMAIN BIT(9) +#define IMX8MM_GPU_A53_DOMAIN (BIT(8) | BIT(11)) +#define IMX8MM_DDR1_A53_DOMAIN BIT(7) +#define IMX8MM_OTG2_A53_DOMAIN BIT(5) +#define IMX8MM_OTG1_A53_DOMAIN BIT(4) +#define IMX8MM_PCIE_A53_DOMAIN BIT(3) +#define IMX8MM_MIPI_A53_DOMAIN BIT(2) + +#define IMX8MN_DISPMIX_A53_DOMAIN BIT(12) +#define IMX8MN_GPUMIX_A53_DOMAIN BIT(9) +#define IMX8MN_DDR1_A53_DOMAIN BIT(7) +#define IMX8MN_OTG1_A53_DOMAIN BIT(4) +#define IMX8MN_MIPI_A53_DOMAIN BIT(2) + +#define IMX8MP_MEDIA_ISPDWP_A53_DOMAIN BIT(20) +#define IMX8MP_HSIOMIX_A53_DOMAIN BIT(19) +#define IMX8MP_MIPI_PHY2_A53_DOMAIN BIT(18) +#define IMX8MP_HDMI_PHY_A53_DOMAIN BIT(17) +#define IMX8MP_HDMIMIX_A53_DOMAIN BIT(16) +#define IMX8MP_VPU_VC8000E_A53_DOMAIN BIT(15) +#define IMX8MP_VPU_G2_A53_DOMAIN BIT(14) +#define IMX8MP_VPU_G1_A53_DOMAIN BIT(13) +#define IMX8MP_MEDIAMIX_A53_DOMAIN BIT(12) +#define IMX8MP_GPU3D_A53_DOMAIN BIT(11) +#define IMX8MP_VPUMIX_A53_DOMAIN BIT(10) +#define IMX8MP_GPUMIX_A53_DOMAIN BIT(9) +#define IMX8MP_GPU2D_A53_DOMAIN BIT(8) +#define IMX8MP_AUDIOMIX_A53_DOMAIN BIT(7) +#define IMX8MP_MLMIX_A53_DOMAIN BIT(6) +#define IMX8MP_USB2_PHY_A53_DOMAIN BIT(5) +#define IMX8MP_USB1_PHY_A53_DOMAIN BIT(4) +#define IMX8MP_PCIE_PHY_A53_DOMAIN BIT(3) +#define IMX8MP_MIPI_PHY1_A53_DOMAIN BIT(2) + +#define IMX8MP_GPC_PU_PGC_SW_PUP_REQ 0x0d8 +#define IMX8MP_GPC_PU_PGC_SW_PDN_REQ 0x0e4 + +#define GPC_PU_PGC_SW_PUP_REQ 0x0f8 +#define GPC_PU_PGC_SW_PDN_REQ 0x104 + +#define IMX7_USB_HSIC_PHY_SW_Pxx_REQ BIT(4) +#define IMX7_USB_OTG2_PHY_SW_Pxx_REQ BIT(3) +#define IMX7_USB_OTG1_PHY_SW_Pxx_REQ BIT(2) +#define IMX7_PCIE_PHY_SW_Pxx_REQ BIT(1) +#define IMX7_MIPI_PHY_SW_Pxx_REQ BIT(0) + +#define IMX8M_PCIE2_SW_Pxx_REQ BIT(13) +#define IMX8M_MIPI_CSI2_SW_Pxx_REQ BIT(12) +#define IMX8M_MIPI_CSI1_SW_Pxx_REQ BIT(11) +#define IMX8M_DISP_SW_Pxx_REQ BIT(10) +#define IMX8M_HDMI_SW_Pxx_REQ BIT(9) +#define IMX8M_VPU_SW_Pxx_REQ BIT(8) +#define IMX8M_GPU_SW_Pxx_REQ BIT(7) +#define IMX8M_DDR2_SW_Pxx_REQ BIT(6) +#define IMX8M_DDR1_SW_Pxx_REQ BIT(5) +#define IMX8M_OTG2_SW_Pxx_REQ BIT(3) +#define IMX8M_OTG1_SW_Pxx_REQ BIT(2) +#define IMX8M_PCIE1_SW_Pxx_REQ BIT(1) +#define IMX8M_MIPI_SW_Pxx_REQ BIT(0) + +#define IMX8MM_VPUH1_SW_Pxx_REQ BIT(13) +#define IMX8MM_VPUG2_SW_Pxx_REQ BIT(12) +#define IMX8MM_VPUG1_SW_Pxx_REQ BIT(11) +#define IMX8MM_DISPMIX_SW_Pxx_REQ BIT(10) +#define IMX8MM_VPUMIX_SW_Pxx_REQ BIT(8) +#define IMX8MM_GPUMIX_SW_Pxx_REQ BIT(7) +#define IMX8MM_GPU_SW_Pxx_REQ (BIT(6) | BIT(9)) +#define IMX8MM_DDR1_SW_Pxx_REQ BIT(5) +#define IMX8MM_OTG2_SW_Pxx_REQ BIT(3) +#define IMX8MM_OTG1_SW_Pxx_REQ BIT(2) +#define IMX8MM_PCIE_SW_Pxx_REQ BIT(1) +#define IMX8MM_MIPI_SW_Pxx_REQ BIT(0) + +#define IMX8MN_DISPMIX_SW_Pxx_REQ BIT(10) +#define IMX8MN_GPUMIX_SW_Pxx_REQ BIT(7) +#define IMX8MN_DDR1_SW_Pxx_REQ BIT(5) +#define IMX8MN_OTG1_SW_Pxx_REQ BIT(2) +#define IMX8MN_MIPI_SW_Pxx_REQ BIT(0) + +#define IMX8MP_DDRMIX_Pxx_REQ BIT(19) +#define IMX8MP_MEDIA_ISP_DWP_Pxx_REQ BIT(18) +#define IMX8MP_HSIOMIX_Pxx_REQ BIT(17) +#define IMX8MP_MIPI_PHY2_Pxx_REQ BIT(16) +#define IMX8MP_HDMI_PHY_Pxx_REQ BIT(15) +#define IMX8MP_HDMIMIX_Pxx_REQ BIT(14) +#define IMX8MP_VPU_VC8K_Pxx_REQ BIT(13) +#define IMX8MP_VPU_G2_Pxx_REQ BIT(12) +#define IMX8MP_VPU_G1_Pxx_REQ BIT(11) +#define IMX8MP_MEDIMIX_Pxx_REQ BIT(10) +#define IMX8MP_GPU_3D_Pxx_REQ BIT(9) +#define IMX8MP_VPU_MIX_SHARE_LOGIC_Pxx_REQ BIT(8) +#define IMX8MP_GPU_SHARE_LOGIC_Pxx_REQ BIT(7) +#define IMX8MP_GPU_2D_Pxx_REQ BIT(6) +#define IMX8MP_AUDIOMIX_Pxx_REQ BIT(5) +#define IMX8MP_MLMIX_Pxx_REQ BIT(4) +#define IMX8MP_USB2_PHY_Pxx_REQ BIT(3) +#define IMX8MP_USB1_PHY_Pxx_REQ BIT(2) +#define IMX8MP_PCIE_PHY_SW_Pxx_REQ BIT(1) +#define IMX8MP_MIPI_PHY1_SW_Pxx_REQ BIT(0) + +#define GPC_M4_PU_PDN_FLG 0x1bc + +#define IMX8MP_GPC_PU_PWRHSK 0x190 +#define GPC_PU_PWRHSK 0x1fc + +#define IMX8M_GPU_HSK_PWRDNACKN BIT(26) +#define IMX8M_VPU_HSK_PWRDNACKN BIT(25) +#define IMX8M_DISP_HSK_PWRDNACKN BIT(24) +#define IMX8M_GPU_HSK_PWRDNREQN BIT(6) +#define IMX8M_VPU_HSK_PWRDNREQN BIT(5) +#define IMX8M_DISP_HSK_PWRDNREQN BIT(4) + +#define IMX8MM_GPUMIX_HSK_PWRDNACKN BIT(29) +#define IMX8MM_GPU_HSK_PWRDNACKN (BIT(27) | BIT(28)) +#define IMX8MM_VPUMIX_HSK_PWRDNACKN BIT(26) +#define IMX8MM_DISPMIX_HSK_PWRDNACKN BIT(25) +#define IMX8MM_HSIO_HSK_PWRDNACKN (BIT(23) | BIT(24)) +#define IMX8MM_GPUMIX_HSK_PWRDNREQN BIT(11) +#define IMX8MM_GPU_HSK_PWRDNREQN (BIT(9) | BIT(10)) +#define IMX8MM_VPUMIX_HSK_PWRDNREQN BIT(8) +#define IMX8MM_DISPMIX_HSK_PWRDNREQN BIT(7) +#define IMX8MM_HSIO_HSK_PWRDNREQN (BIT(5) | BIT(6)) + +#define IMX8MN_GPUMIX_HSK_PWRDNACKN (BIT(29) | BIT(27)) +#define IMX8MN_DISPMIX_HSK_PWRDNACKN BIT(25) +#define IMX8MN_HSIO_HSK_PWRDNACKN BIT(23) +#define IMX8MN_GPUMIX_HSK_PWRDNREQN (BIT(11) | BIT(9)) +#define IMX8MN_DISPMIX_HSK_PWRDNREQN BIT(7) +#define IMX8MN_HSIO_HSK_PWRDNREQN BIT(5) + +#define IMX8MP_MEDIAMIX_PWRDNACKN BIT(30) +#define IMX8MP_HDMIMIX_PWRDNACKN BIT(29) +#define IMX8MP_HSIOMIX_PWRDNACKN BIT(28) +#define IMX8MP_VPUMIX_PWRDNACKN BIT(26) +#define IMX8MP_GPUMIX_PWRDNACKN BIT(25) +#define IMX8MP_MLMIX_PWRDNACKN (BIT(23) | BIT(24)) +#define IMX8MP_AUDIOMIX_PWRDNACKN (BIT(20) | BIT(31)) +#define IMX8MP_MEDIAMIX_PWRDNREQN BIT(14) +#define IMX8MP_HDMIMIX_PWRDNREQN BIT(13) +#define IMX8MP_HSIOMIX_PWRDNREQN BIT(12) +#define IMX8MP_VPUMIX_PWRDNREQN BIT(10) +#define IMX8MP_GPUMIX_PWRDNREQN BIT(9) +#define IMX8MP_MLMIX_PWRDNREQN (BIT(7) | BIT(8)) +#define IMX8MP_AUDIOMIX_PWRDNREQN (BIT(4) | BIT(15)) + +/* + * The PGC offset values in Reference Manual + * (Rev. 1, 01/2018 and the older ones) GPC chapter's + * GPC_PGC memory map are incorrect, below offset + * values are from design RTL. + */ +#define IMX7_PGC_MIPI 16 +#define IMX7_PGC_PCIE 17 +#define IMX7_PGC_USB_HSIC 20 + +#define IMX8M_PGC_MIPI 16 +#define IMX8M_PGC_PCIE1 17 +#define IMX8M_PGC_OTG1 18 +#define IMX8M_PGC_OTG2 19 +#define IMX8M_PGC_DDR1 21 +#define IMX8M_PGC_GPU 23 +#define IMX8M_PGC_VPU 24 +#define IMX8M_PGC_DISP 26 +#define IMX8M_PGC_MIPI_CSI1 27 +#define IMX8M_PGC_MIPI_CSI2 28 +#define IMX8M_PGC_PCIE2 29 + +#define IMX8MM_PGC_MIPI 16 +#define IMX8MM_PGC_PCIE 17 +#define IMX8MM_PGC_OTG1 18 +#define IMX8MM_PGC_OTG2 19 +#define IMX8MM_PGC_DDR1 21 +#define IMX8MM_PGC_GPU2D 22 +#define IMX8MM_PGC_GPUMIX 23 +#define IMX8MM_PGC_VPUMIX 24 +#define IMX8MM_PGC_GPU3D 25 +#define IMX8MM_PGC_DISPMIX 26 +#define IMX8MM_PGC_VPUG1 27 +#define IMX8MM_PGC_VPUG2 28 +#define IMX8MM_PGC_VPUH1 29 + +#define IMX8MN_PGC_MIPI 16 +#define IMX8MN_PGC_OTG1 18 +#define IMX8MN_PGC_DDR1 21 +#define IMX8MN_PGC_GPUMIX 23 +#define IMX8MN_PGC_DISPMIX 26 + +#define IMX8MP_PGC_NOC 9 +#define IMX8MP_PGC_MIPI1 12 +#define IMX8MP_PGC_PCIE 13 +#define IMX8MP_PGC_USB1 14 +#define IMX8MP_PGC_USB2 15 +#define IMX8MP_PGC_MLMIX 16 +#define IMX8MP_PGC_AUDIOMIX 17 +#define IMX8MP_PGC_GPU2D 18 +#define IMX8MP_PGC_GPUMIX 19 +#define IMX8MP_PGC_VPUMIX 20 +#define IMX8MP_PGC_GPU3D 21 +#define IMX8MP_PGC_MEDIAMIX 22 +#define IMX8MP_PGC_VPU_G1 23 +#define IMX8MP_PGC_VPU_G2 24 +#define IMX8MP_PGC_VPU_VC8000E 25 +#define IMX8MP_PGC_HDMIMIX 26 +#define IMX8MP_PGC_HDMI 27 +#define IMX8MP_PGC_MIPI2 28 +#define IMX8MP_PGC_HSIOMIX 29 +#define IMX8MP_PGC_MEDIA_ISP_DWP 30 +#define IMX8MP_PGC_DDRMIX 31 + +#define GPC_PGC_CTRL(n) (0x800 + (n) * 0x40) +#define GPC_PGC_SR(n) (GPC_PGC_CTRL(n) + 0xc) + +#define GPC_PGC_CTRL_PCR BIT(0) + +struct imx_pgc_regs { + u16 map; + u16 pup; + u16 pdn; + u16 hsk; +}; + +struct imx_pgc_domain { + struct generic_pm_domain genpd; + struct regmap *regmap; + const struct imx_pgc_regs *regs; + struct regulator *regulator; + struct reset_control *reset; + struct clk_bulk_data *clks; + int num_clks; + + unsigned long pgc; + + const struct { + u32 pxx; + u32 map; + u32 hskreq; + u32 hskack; + } bits; + + const int voltage; + const bool keep_clocks; + struct device *dev; + + unsigned int pgc_sw_pup_reg; + unsigned int pgc_sw_pdn_reg; +}; + +struct imx_pgc_domain_data { + const struct imx_pgc_domain *domains; + size_t domains_num; + const struct regmap_access_table *reg_access_table; + const struct imx_pgc_regs *pgc_regs; +}; + +static inline struct imx_pgc_domain * +to_imx_pgc_domain(struct generic_pm_domain *genpd) +{ + return container_of(genpd, struct imx_pgc_domain, genpd); +} + +static int imx_pgc_power_up(struct generic_pm_domain *genpd) +{ + struct imx_pgc_domain *domain = to_imx_pgc_domain(genpd); + u32 reg_val, pgc; + int ret; + + ret = pm_runtime_get_sync(domain->dev); + if (ret < 0) { + pm_runtime_put_noidle(domain->dev); + return ret; + } + + if (!IS_ERR(domain->regulator)) { + ret = regulator_enable(domain->regulator); + if (ret) { + dev_err(domain->dev, + "failed to enable regulator: %pe\n", + ERR_PTR(ret)); + goto out_put_pm; + } + } + + reset_control_assert(domain->reset); + + /* Enable reset clocks for all devices in the domain */ + ret = clk_bulk_prepare_enable(domain->num_clks, domain->clks); + if (ret) { + dev_err(domain->dev, "failed to enable reset clocks\n"); + goto out_regulator_disable; + } + + /* delays for reset to propagate */ + udelay(5); + + if (domain->bits.pxx) { + /* request the domain to power up */ + regmap_update_bits(domain->regmap, domain->regs->pup, + domain->bits.pxx, domain->bits.pxx); + /* + * As per "5.5.9.4 Example Code 4" in IMX7DRM.pdf wait + * for PUP_REQ/PDN_REQ bit to be cleared + */ + ret = regmap_read_poll_timeout(domain->regmap, + domain->regs->pup, reg_val, + !(reg_val & domain->bits.pxx), + 0, USEC_PER_MSEC); + if (ret) { + dev_err(domain->dev, "failed to command PGC\n"); + goto out_clk_disable; + } + + /* disable power control */ + for_each_set_bit(pgc, &domain->pgc, 32) { + regmap_clear_bits(domain->regmap, GPC_PGC_CTRL(pgc), + GPC_PGC_CTRL_PCR); + } + } + + /* delay for reset to propagate */ + udelay(5); + + reset_control_deassert(domain->reset); + + /* request the ADB400 to power up */ + if (domain->bits.hskreq) { + regmap_update_bits(domain->regmap, domain->regs->hsk, + domain->bits.hskreq, domain->bits.hskreq); + + /* + * ret = regmap_read_poll_timeout(domain->regmap, domain->regs->hsk, reg_val, + * (reg_val & domain->bits.hskack), 0, + * USEC_PER_MSEC); + * Technically we need the commented code to wait handshake. But that needs + * the BLK-CTL module BUS clk-en bit being set. + * + * There is a separate BLK-CTL module and we will have such a driver for it, + * that driver will set the BUS clk-en bit and handshake will be triggered + * automatically there. Just add a delay and suppose the handshake finish + * after that. + */ + } + + /* Disable reset clocks for all devices in the domain */ + if (!domain->keep_clocks) + clk_bulk_disable_unprepare(domain->num_clks, domain->clks); + + return 0; + +out_clk_disable: + clk_bulk_disable_unprepare(domain->num_clks, domain->clks); +out_regulator_disable: + if (!IS_ERR(domain->regulator)) + regulator_disable(domain->regulator); +out_put_pm: + pm_runtime_put(domain->dev); + + return ret; +} + +static int imx_pgc_power_down(struct generic_pm_domain *genpd) +{ + struct imx_pgc_domain *domain = to_imx_pgc_domain(genpd); + u32 reg_val, pgc; + int ret; + + /* Enable reset clocks for all devices in the domain */ + if (!domain->keep_clocks) { + ret = clk_bulk_prepare_enable(domain->num_clks, domain->clks); + if (ret) { + dev_err(domain->dev, "failed to enable reset clocks\n"); + return ret; + } + } + + /* request the ADB400 to power down */ + if (domain->bits.hskreq) { + regmap_clear_bits(domain->regmap, domain->regs->hsk, + domain->bits.hskreq); + + ret = regmap_read_poll_timeout(domain->regmap, domain->regs->hsk, + reg_val, + !(reg_val & domain->bits.hskack), + 0, USEC_PER_MSEC); + if (ret) { + dev_err(domain->dev, "failed to power down ADB400\n"); + goto out_clk_disable; + } + } + + if (domain->bits.pxx) { + /* enable power control */ + for_each_set_bit(pgc, &domain->pgc, 32) { + regmap_update_bits(domain->regmap, GPC_PGC_CTRL(pgc), + GPC_PGC_CTRL_PCR, GPC_PGC_CTRL_PCR); + } + + /* request the domain to power down */ + regmap_update_bits(domain->regmap, domain->regs->pdn, + domain->bits.pxx, domain->bits.pxx); + /* + * As per "5.5.9.4 Example Code 4" in IMX7DRM.pdf wait + * for PUP_REQ/PDN_REQ bit to be cleared + */ + ret = regmap_read_poll_timeout(domain->regmap, + domain->regs->pdn, reg_val, + !(reg_val & domain->bits.pxx), + 0, USEC_PER_MSEC); + if (ret) { + dev_err(domain->dev, "failed to command PGC\n"); + goto out_clk_disable; + } + } + + /* Disable reset clocks for all devices in the domain */ + clk_bulk_disable_unprepare(domain->num_clks, domain->clks); + + if (!IS_ERR(domain->regulator)) { + ret = regulator_disable(domain->regulator); + if (ret) { + dev_err(domain->dev, + "failed to disable regulator: %pe\n", + ERR_PTR(ret)); + return ret; + } + } + + pm_runtime_put_sync_suspend(domain->dev); + + return 0; + +out_clk_disable: + if (!domain->keep_clocks) + clk_bulk_disable_unprepare(domain->num_clks, domain->clks); + + return ret; +} + +static const struct imx_pgc_domain imx7_pgc_domains[] = { + [IMX7_POWER_DOMAIN_MIPI_PHY] = { + .genpd = { + .name = "mipi-phy", + }, + .bits = { + .pxx = IMX7_MIPI_PHY_SW_Pxx_REQ, + .map = IMX7_MIPI_PHY_A_CORE_DOMAIN, + }, + .voltage = 1000000, + .pgc = BIT(IMX7_PGC_MIPI), + }, + + [IMX7_POWER_DOMAIN_PCIE_PHY] = { + .genpd = { + .name = "pcie-phy", + }, + .bits = { + .pxx = IMX7_PCIE_PHY_SW_Pxx_REQ, + .map = IMX7_PCIE_PHY_A_CORE_DOMAIN, + }, + .voltage = 1000000, + .pgc = BIT(IMX7_PGC_PCIE), + }, + + [IMX7_POWER_DOMAIN_USB_HSIC_PHY] = { + .genpd = { + .name = "usb-hsic-phy", + }, + .bits = { + .pxx = IMX7_USB_HSIC_PHY_SW_Pxx_REQ, + .map = IMX7_USB_HSIC_PHY_A_CORE_DOMAIN, + }, + .voltage = 1200000, + .pgc = BIT(IMX7_PGC_USB_HSIC), + }, +}; + +static const struct regmap_range imx7_yes_ranges[] = { + regmap_reg_range(GPC_LPCR_A_CORE_BSC, + GPC_M4_PU_PDN_FLG), + regmap_reg_range(GPC_PGC_CTRL(IMX7_PGC_MIPI), + GPC_PGC_SR(IMX7_PGC_MIPI)), + regmap_reg_range(GPC_PGC_CTRL(IMX7_PGC_PCIE), + GPC_PGC_SR(IMX7_PGC_PCIE)), + regmap_reg_range(GPC_PGC_CTRL(IMX7_PGC_USB_HSIC), + GPC_PGC_SR(IMX7_PGC_USB_HSIC)), +}; + +static const struct regmap_access_table imx7_access_table = { + .yes_ranges = imx7_yes_ranges, + .n_yes_ranges = ARRAY_SIZE(imx7_yes_ranges), +}; + +static const struct imx_pgc_regs imx7_pgc_regs = { + .map = GPC_PGC_CPU_MAPPING, + .pup = GPC_PU_PGC_SW_PUP_REQ, + .pdn = GPC_PU_PGC_SW_PDN_REQ, + .hsk = GPC_PU_PWRHSK, +}; + +static const struct imx_pgc_domain_data imx7_pgc_domain_data = { + .domains = imx7_pgc_domains, + .domains_num = ARRAY_SIZE(imx7_pgc_domains), + .reg_access_table = &imx7_access_table, + .pgc_regs = &imx7_pgc_regs, +}; + +static const struct imx_pgc_domain imx8m_pgc_domains[] = { + [IMX8M_POWER_DOMAIN_MIPI] = { + .genpd = { + .name = "mipi", + }, + .bits = { + .pxx = IMX8M_MIPI_SW_Pxx_REQ, + .map = IMX8M_MIPI_A53_DOMAIN, + }, + .pgc = BIT(IMX8M_PGC_MIPI), + }, + + [IMX8M_POWER_DOMAIN_PCIE1] = { + .genpd = { + .name = "pcie1", + }, + .bits = { + .pxx = IMX8M_PCIE1_SW_Pxx_REQ, + .map = IMX8M_PCIE1_A53_DOMAIN, + }, + .pgc = BIT(IMX8M_PGC_PCIE1), + }, + + [IMX8M_POWER_DOMAIN_USB_OTG1] = { + .genpd = { + .name = "usb-otg1", + }, + .bits = { + .pxx = IMX8M_OTG1_SW_Pxx_REQ, + .map = IMX8M_OTG1_A53_DOMAIN, + }, + .pgc = BIT(IMX8M_PGC_OTG1), + }, + + [IMX8M_POWER_DOMAIN_USB_OTG2] = { + .genpd = { + .name = "usb-otg2", + }, + .bits = { + .pxx = IMX8M_OTG2_SW_Pxx_REQ, + .map = IMX8M_OTG2_A53_DOMAIN, + }, + .pgc = BIT(IMX8M_PGC_OTG2), + }, + + [IMX8M_POWER_DOMAIN_DDR1] = { + .genpd = { + .name = "ddr1", + }, + .bits = { + .pxx = IMX8M_DDR1_SW_Pxx_REQ, + .map = IMX8M_DDR2_A53_DOMAIN, + }, + .pgc = BIT(IMX8M_PGC_DDR1), + }, + + [IMX8M_POWER_DOMAIN_GPU] = { + .genpd = { + .name = "gpu", + }, + .bits = { + .pxx = IMX8M_GPU_SW_Pxx_REQ, + .map = IMX8M_GPU_A53_DOMAIN, + .hskreq = IMX8M_GPU_HSK_PWRDNREQN, + .hskack = IMX8M_GPU_HSK_PWRDNACKN, + }, + .pgc = BIT(IMX8M_PGC_GPU), + }, + + [IMX8M_POWER_DOMAIN_VPU] = { + .genpd = { + .name = "vpu", + }, + .bits = { + .pxx = IMX8M_VPU_SW_Pxx_REQ, + .map = IMX8M_VPU_A53_DOMAIN, + .hskreq = IMX8M_VPU_HSK_PWRDNREQN, + .hskack = IMX8M_VPU_HSK_PWRDNACKN, + }, + .pgc = BIT(IMX8M_PGC_VPU), + .keep_clocks = true, + }, + + [IMX8M_POWER_DOMAIN_DISP] = { + .genpd = { + .name = "disp", + }, + .bits = { + .pxx = IMX8M_DISP_SW_Pxx_REQ, + .map = IMX8M_DISP_A53_DOMAIN, + .hskreq = IMX8M_DISP_HSK_PWRDNREQN, + .hskack = IMX8M_DISP_HSK_PWRDNACKN, + }, + .pgc = BIT(IMX8M_PGC_DISP), + }, + + [IMX8M_POWER_DOMAIN_MIPI_CSI1] = { + .genpd = { + .name = "mipi-csi1", + }, + .bits = { + .pxx = IMX8M_MIPI_CSI1_SW_Pxx_REQ, + .map = IMX8M_MIPI_CSI1_A53_DOMAIN, + }, + .pgc = BIT(IMX8M_PGC_MIPI_CSI1), + }, + + [IMX8M_POWER_DOMAIN_MIPI_CSI2] = { + .genpd = { + .name = "mipi-csi2", + }, + .bits = { + .pxx = IMX8M_MIPI_CSI2_SW_Pxx_REQ, + .map = IMX8M_MIPI_CSI2_A53_DOMAIN, + }, + .pgc = BIT(IMX8M_PGC_MIPI_CSI2), + }, + + [IMX8M_POWER_DOMAIN_PCIE2] = { + .genpd = { + .name = "pcie2", + }, + .bits = { + .pxx = IMX8M_PCIE2_SW_Pxx_REQ, + .map = IMX8M_PCIE2_A53_DOMAIN, + }, + .pgc = BIT(IMX8M_PGC_PCIE2), + }, +}; + +static const struct regmap_range imx8m_yes_ranges[] = { + regmap_reg_range(GPC_LPCR_A_CORE_BSC, + GPC_PU_PWRHSK), + regmap_reg_range(GPC_PGC_CTRL(IMX8M_PGC_MIPI), + GPC_PGC_SR(IMX8M_PGC_MIPI)), + regmap_reg_range(GPC_PGC_CTRL(IMX8M_PGC_PCIE1), + GPC_PGC_SR(IMX8M_PGC_PCIE1)), + regmap_reg_range(GPC_PGC_CTRL(IMX8M_PGC_OTG1), + GPC_PGC_SR(IMX8M_PGC_OTG1)), + regmap_reg_range(GPC_PGC_CTRL(IMX8M_PGC_OTG2), + GPC_PGC_SR(IMX8M_PGC_OTG2)), + regmap_reg_range(GPC_PGC_CTRL(IMX8M_PGC_DDR1), + GPC_PGC_SR(IMX8M_PGC_DDR1)), + regmap_reg_range(GPC_PGC_CTRL(IMX8M_PGC_GPU), + GPC_PGC_SR(IMX8M_PGC_GPU)), + regmap_reg_range(GPC_PGC_CTRL(IMX8M_PGC_VPU), + GPC_PGC_SR(IMX8M_PGC_VPU)), + regmap_reg_range(GPC_PGC_CTRL(IMX8M_PGC_DISP), + GPC_PGC_SR(IMX8M_PGC_DISP)), + regmap_reg_range(GPC_PGC_CTRL(IMX8M_PGC_MIPI_CSI1), + GPC_PGC_SR(IMX8M_PGC_MIPI_CSI1)), + regmap_reg_range(GPC_PGC_CTRL(IMX8M_PGC_MIPI_CSI2), + GPC_PGC_SR(IMX8M_PGC_MIPI_CSI2)), + regmap_reg_range(GPC_PGC_CTRL(IMX8M_PGC_PCIE2), + GPC_PGC_SR(IMX8M_PGC_PCIE2)), +}; + +static const struct regmap_access_table imx8m_access_table = { + .yes_ranges = imx8m_yes_ranges, + .n_yes_ranges = ARRAY_SIZE(imx8m_yes_ranges), +}; + +static const struct imx_pgc_domain_data imx8m_pgc_domain_data = { + .domains = imx8m_pgc_domains, + .domains_num = ARRAY_SIZE(imx8m_pgc_domains), + .reg_access_table = &imx8m_access_table, + .pgc_regs = &imx7_pgc_regs, +}; + +static const struct imx_pgc_domain imx8mm_pgc_domains[] = { + [IMX8MM_POWER_DOMAIN_HSIOMIX] = { + .genpd = { + .name = "hsiomix", + }, + .bits = { + .pxx = 0, /* no power sequence control */ + .map = 0, /* no power sequence control */ + .hskreq = IMX8MM_HSIO_HSK_PWRDNREQN, + .hskack = IMX8MM_HSIO_HSK_PWRDNACKN, + }, + .keep_clocks = true, + }, + + [IMX8MM_POWER_DOMAIN_PCIE] = { + .genpd = { + .name = "pcie", + }, + .bits = { + .pxx = IMX8MM_PCIE_SW_Pxx_REQ, + .map = IMX8MM_PCIE_A53_DOMAIN, + }, + .pgc = BIT(IMX8MM_PGC_PCIE), + }, + + [IMX8MM_POWER_DOMAIN_OTG1] = { + .genpd = { + .name = "usb-otg1", + .flags = GENPD_FLAG_ACTIVE_WAKEUP, + }, + .bits = { + .pxx = IMX8MM_OTG1_SW_Pxx_REQ, + .map = IMX8MM_OTG1_A53_DOMAIN, + }, + .pgc = BIT(IMX8MM_PGC_OTG1), + }, + + [IMX8MM_POWER_DOMAIN_OTG2] = { + .genpd = { + .name = "usb-otg2", + .flags = GENPD_FLAG_ACTIVE_WAKEUP, + }, + .bits = { + .pxx = IMX8MM_OTG2_SW_Pxx_REQ, + .map = IMX8MM_OTG2_A53_DOMAIN, + }, + .pgc = BIT(IMX8MM_PGC_OTG2), + }, + + [IMX8MM_POWER_DOMAIN_GPUMIX] = { + .genpd = { + .name = "gpumix", + }, + .bits = { + .pxx = IMX8MM_GPUMIX_SW_Pxx_REQ, + .map = IMX8MM_GPUMIX_A53_DOMAIN, + .hskreq = IMX8MM_GPUMIX_HSK_PWRDNREQN, + .hskack = IMX8MM_GPUMIX_HSK_PWRDNACKN, + }, + .pgc = BIT(IMX8MM_PGC_GPUMIX), + .keep_clocks = true, + }, + + [IMX8MM_POWER_DOMAIN_GPU] = { + .genpd = { + .name = "gpu", + }, + .bits = { + .pxx = IMX8MM_GPU_SW_Pxx_REQ, + .map = IMX8MM_GPU_A53_DOMAIN, + .hskreq = IMX8MM_GPU_HSK_PWRDNREQN, + .hskack = IMX8MM_GPU_HSK_PWRDNACKN, + }, + .pgc = BIT(IMX8MM_PGC_GPU2D) | BIT(IMX8MM_PGC_GPU3D), + }, + + [IMX8MM_POWER_DOMAIN_VPUMIX] = { + .genpd = { + .name = "vpumix", + }, + .bits = { + .pxx = IMX8MM_VPUMIX_SW_Pxx_REQ, + .map = IMX8MM_VPUMIX_A53_DOMAIN, + .hskreq = IMX8MM_VPUMIX_HSK_PWRDNREQN, + .hskack = IMX8MM_VPUMIX_HSK_PWRDNACKN, + }, + .pgc = BIT(IMX8MM_PGC_VPUMIX), + .keep_clocks = true, + }, + + [IMX8MM_POWER_DOMAIN_VPUG1] = { + .genpd = { + .name = "vpu-g1", + }, + .bits = { + .pxx = IMX8MM_VPUG1_SW_Pxx_REQ, + .map = IMX8MM_VPUG1_A53_DOMAIN, + }, + .pgc = BIT(IMX8MM_PGC_VPUG1), + }, + + [IMX8MM_POWER_DOMAIN_VPUG2] = { + .genpd = { + .name = "vpu-g2", + }, + .bits = { + .pxx = IMX8MM_VPUG2_SW_Pxx_REQ, + .map = IMX8MM_VPUG2_A53_DOMAIN, + }, + .pgc = BIT(IMX8MM_PGC_VPUG2), + }, + + [IMX8MM_POWER_DOMAIN_VPUH1] = { + .genpd = { + .name = "vpu-h1", + }, + .bits = { + .pxx = IMX8MM_VPUH1_SW_Pxx_REQ, + .map = IMX8MM_VPUH1_A53_DOMAIN, + }, + .pgc = BIT(IMX8MM_PGC_VPUH1), + .keep_clocks = true, + }, + + [IMX8MM_POWER_DOMAIN_DISPMIX] = { + .genpd = { + .name = "dispmix", + }, + .bits = { + .pxx = IMX8MM_DISPMIX_SW_Pxx_REQ, + .map = IMX8MM_DISPMIX_A53_DOMAIN, + .hskreq = IMX8MM_DISPMIX_HSK_PWRDNREQN, + .hskack = IMX8MM_DISPMIX_HSK_PWRDNACKN, + }, + .pgc = BIT(IMX8MM_PGC_DISPMIX), + .keep_clocks = true, + }, + + [IMX8MM_POWER_DOMAIN_MIPI] = { + .genpd = { + .name = "mipi", + }, + .bits = { + .pxx = IMX8MM_MIPI_SW_Pxx_REQ, + .map = IMX8MM_MIPI_A53_DOMAIN, + }, + .pgc = BIT(IMX8MM_PGC_MIPI), + }, +}; + +static const struct regmap_range imx8mm_yes_ranges[] = { + regmap_reg_range(GPC_LPCR_A_CORE_BSC, + GPC_PU_PWRHSK), + regmap_reg_range(GPC_PGC_CTRL(IMX8MM_PGC_MIPI), + GPC_PGC_SR(IMX8MM_PGC_MIPI)), + regmap_reg_range(GPC_PGC_CTRL(IMX8MM_PGC_PCIE), + GPC_PGC_SR(IMX8MM_PGC_PCIE)), + regmap_reg_range(GPC_PGC_CTRL(IMX8MM_PGC_OTG1), + GPC_PGC_SR(IMX8MM_PGC_OTG1)), + regmap_reg_range(GPC_PGC_CTRL(IMX8MM_PGC_OTG2), + GPC_PGC_SR(IMX8MM_PGC_OTG2)), + regmap_reg_range(GPC_PGC_CTRL(IMX8MM_PGC_DDR1), + GPC_PGC_SR(IMX8MM_PGC_DDR1)), + regmap_reg_range(GPC_PGC_CTRL(IMX8MM_PGC_GPU2D), + GPC_PGC_SR(IMX8MM_PGC_GPU2D)), + regmap_reg_range(GPC_PGC_CTRL(IMX8MM_PGC_GPUMIX), + GPC_PGC_SR(IMX8MM_PGC_GPUMIX)), + regmap_reg_range(GPC_PGC_CTRL(IMX8MM_PGC_VPUMIX), + GPC_PGC_SR(IMX8MM_PGC_VPUMIX)), + regmap_reg_range(GPC_PGC_CTRL(IMX8MM_PGC_GPU3D), + GPC_PGC_SR(IMX8MM_PGC_GPU3D)), + regmap_reg_range(GPC_PGC_CTRL(IMX8MM_PGC_DISPMIX), + GPC_PGC_SR(IMX8MM_PGC_DISPMIX)), + regmap_reg_range(GPC_PGC_CTRL(IMX8MM_PGC_VPUG1), + GPC_PGC_SR(IMX8MM_PGC_VPUG1)), + regmap_reg_range(GPC_PGC_CTRL(IMX8MM_PGC_VPUG2), + GPC_PGC_SR(IMX8MM_PGC_VPUG2)), + regmap_reg_range(GPC_PGC_CTRL(IMX8MM_PGC_VPUH1), + GPC_PGC_SR(IMX8MM_PGC_VPUH1)), +}; + +static const struct regmap_access_table imx8mm_access_table = { + .yes_ranges = imx8mm_yes_ranges, + .n_yes_ranges = ARRAY_SIZE(imx8mm_yes_ranges), +}; + +static const struct imx_pgc_domain_data imx8mm_pgc_domain_data = { + .domains = imx8mm_pgc_domains, + .domains_num = ARRAY_SIZE(imx8mm_pgc_domains), + .reg_access_table = &imx8mm_access_table, + .pgc_regs = &imx7_pgc_regs, +}; + +static const struct imx_pgc_domain imx8mp_pgc_domains[] = { + [IMX8MP_POWER_DOMAIN_MIPI_PHY1] = { + .genpd = { + .name = "mipi-phy1", + }, + .bits = { + .pxx = IMX8MP_MIPI_PHY1_SW_Pxx_REQ, + .map = IMX8MP_MIPI_PHY1_A53_DOMAIN, + }, + .pgc = BIT(IMX8MP_PGC_MIPI1), + }, + + [IMX8MP_POWER_DOMAIN_PCIE_PHY] = { + .genpd = { + .name = "pcie-phy1", + }, + .bits = { + .pxx = IMX8MP_PCIE_PHY_SW_Pxx_REQ, + .map = IMX8MP_PCIE_PHY_A53_DOMAIN, + }, + .pgc = BIT(IMX8MP_PGC_PCIE), + }, + + [IMX8MP_POWER_DOMAIN_USB1_PHY] = { + .genpd = { + .name = "usb-otg1", + }, + .bits = { + .pxx = IMX8MP_USB1_PHY_Pxx_REQ, + .map = IMX8MP_USB1_PHY_A53_DOMAIN, + }, + .pgc = BIT(IMX8MP_PGC_USB1), + }, + + [IMX8MP_POWER_DOMAIN_USB2_PHY] = { + .genpd = { + .name = "usb-otg2", + }, + .bits = { + .pxx = IMX8MP_USB2_PHY_Pxx_REQ, + .map = IMX8MP_USB2_PHY_A53_DOMAIN, + }, + .pgc = BIT(IMX8MP_PGC_USB2), + }, + + [IMX8MP_POWER_DOMAIN_MLMIX] = { + .genpd = { + .name = "mlmix", + }, + .bits = { + .pxx = IMX8MP_MLMIX_Pxx_REQ, + .map = IMX8MP_MLMIX_A53_DOMAIN, + .hskreq = IMX8MP_MLMIX_PWRDNREQN, + .hskack = IMX8MP_MLMIX_PWRDNACKN, + }, + .pgc = BIT(IMX8MP_PGC_MLMIX), + .keep_clocks = true, + }, + + [IMX8MP_POWER_DOMAIN_AUDIOMIX] = { + .genpd = { + .name = "audiomix", + }, + .bits = { + .pxx = IMX8MP_AUDIOMIX_Pxx_REQ, + .map = IMX8MP_AUDIOMIX_A53_DOMAIN, + .hskreq = IMX8MP_AUDIOMIX_PWRDNREQN, + .hskack = IMX8MP_AUDIOMIX_PWRDNACKN, + }, + .pgc = BIT(IMX8MP_PGC_AUDIOMIX), + .keep_clocks = true, + }, + + [IMX8MP_POWER_DOMAIN_GPU2D] = { + .genpd = { + .name = "gpu2d", + }, + .bits = { + .pxx = IMX8MP_GPU_2D_Pxx_REQ, + .map = IMX8MP_GPU2D_A53_DOMAIN, + }, + .pgc = BIT(IMX8MP_PGC_GPU2D), + }, + + [IMX8MP_POWER_DOMAIN_GPUMIX] = { + .genpd = { + .name = "gpumix", + }, + .bits = { + .pxx = IMX8MP_GPU_SHARE_LOGIC_Pxx_REQ, + .map = IMX8MP_GPUMIX_A53_DOMAIN, + .hskreq = IMX8MP_GPUMIX_PWRDNREQN, + .hskack = IMX8MP_GPUMIX_PWRDNACKN, + }, + .pgc = BIT(IMX8MP_PGC_GPUMIX), + .keep_clocks = true, + }, + + [IMX8MP_POWER_DOMAIN_VPUMIX] = { + .genpd = { + .name = "vpumix", + }, + .bits = { + .pxx = IMX8MP_VPU_MIX_SHARE_LOGIC_Pxx_REQ, + .map = IMX8MP_VPUMIX_A53_DOMAIN, + .hskreq = IMX8MP_VPUMIX_PWRDNREQN, + .hskack = IMX8MP_VPUMIX_PWRDNACKN, + }, + .pgc = BIT(IMX8MP_PGC_VPUMIX), + .keep_clocks = true, + }, + + [IMX8MP_POWER_DOMAIN_GPU3D] = { + .genpd = { + .name = "gpu3d", + }, + .bits = { + .pxx = IMX8MP_GPU_3D_Pxx_REQ, + .map = IMX8MP_GPU3D_A53_DOMAIN, + }, + .pgc = BIT(IMX8MP_PGC_GPU3D), + }, + + [IMX8MP_POWER_DOMAIN_MEDIAMIX] = { + .genpd = { + .name = "mediamix", + }, + .bits = { + .pxx = IMX8MP_MEDIMIX_Pxx_REQ, + .map = IMX8MP_MEDIAMIX_A53_DOMAIN, + .hskreq = IMX8MP_MEDIAMIX_PWRDNREQN, + .hskack = IMX8MP_MEDIAMIX_PWRDNACKN, + }, + .pgc = BIT(IMX8MP_PGC_MEDIAMIX), + .keep_clocks = true, + }, + + [IMX8MP_POWER_DOMAIN_VPU_G1] = { + .genpd = { + .name = "vpu-g1", + }, + .bits = { + .pxx = IMX8MP_VPU_G1_Pxx_REQ, + .map = IMX8MP_VPU_G1_A53_DOMAIN, + }, + .pgc = BIT(IMX8MP_PGC_VPU_G1), + }, + + [IMX8MP_POWER_DOMAIN_VPU_G2] = { + .genpd = { + .name = "vpu-g2", + }, + .bits = { + .pxx = IMX8MP_VPU_G2_Pxx_REQ, + .map = IMX8MP_VPU_G2_A53_DOMAIN + }, + .pgc = BIT(IMX8MP_PGC_VPU_G2), + }, + + [IMX8MP_POWER_DOMAIN_VPU_VC8000E] = { + .genpd = { + .name = "vpu-h1", + }, + .bits = { + .pxx = IMX8MP_VPU_VC8K_Pxx_REQ, + .map = IMX8MP_VPU_VC8000E_A53_DOMAIN, + }, + .pgc = BIT(IMX8MP_PGC_VPU_VC8000E), + }, + + [IMX8MP_POWER_DOMAIN_HDMIMIX] = { + .genpd = { + .name = "hdmimix", + }, + .bits = { + .pxx = IMX8MP_HDMIMIX_Pxx_REQ, + .map = IMX8MP_HDMIMIX_A53_DOMAIN, + .hskreq = IMX8MP_HDMIMIX_PWRDNREQN, + .hskack = IMX8MP_HDMIMIX_PWRDNACKN, + }, + .pgc = BIT(IMX8MP_PGC_HDMIMIX), + .keep_clocks = true, + }, + + [IMX8MP_POWER_DOMAIN_HDMI_PHY] = { + .genpd = { + .name = "hdmi-phy", + }, + .bits = { + .pxx = IMX8MP_HDMI_PHY_Pxx_REQ, + .map = IMX8MP_HDMI_PHY_A53_DOMAIN, + }, + .pgc = BIT(IMX8MP_PGC_HDMI), + }, + + [IMX8MP_POWER_DOMAIN_MIPI_PHY2] = { + .genpd = { + .name = "mipi-phy2", + }, + .bits = { + .pxx = IMX8MP_MIPI_PHY2_Pxx_REQ, + .map = IMX8MP_MIPI_PHY2_A53_DOMAIN, + }, + .pgc = BIT(IMX8MP_PGC_MIPI2), + }, + + [IMX8MP_POWER_DOMAIN_HSIOMIX] = { + .genpd = { + .name = "hsiomix", + }, + .bits = { + .pxx = IMX8MP_HSIOMIX_Pxx_REQ, + .map = IMX8MP_HSIOMIX_A53_DOMAIN, + .hskreq = IMX8MP_HSIOMIX_PWRDNREQN, + .hskack = IMX8MP_HSIOMIX_PWRDNACKN, + }, + .pgc = BIT(IMX8MP_PGC_HSIOMIX), + .keep_clocks = true, + }, + + [IMX8MP_POWER_DOMAIN_MEDIAMIX_ISPDWP] = { + .genpd = { + .name = "mediamix-isp-dwp", + }, + .bits = { + .pxx = IMX8MP_MEDIA_ISP_DWP_Pxx_REQ, + .map = IMX8MP_MEDIA_ISPDWP_A53_DOMAIN, + }, + .pgc = BIT(IMX8MP_PGC_MEDIA_ISP_DWP), + }, +}; + +static const struct regmap_range imx8mp_yes_ranges[] = { + regmap_reg_range(GPC_LPCR_A_CORE_BSC, + IMX8MP_GPC_PGC_CPU_MAPPING), + regmap_reg_range(GPC_PGC_CTRL(IMX8MP_PGC_NOC), + GPC_PGC_SR(IMX8MP_PGC_NOC)), + regmap_reg_range(GPC_PGC_CTRL(IMX8MP_PGC_MIPI1), + GPC_PGC_SR(IMX8MP_PGC_MIPI1)), + regmap_reg_range(GPC_PGC_CTRL(IMX8MP_PGC_PCIE), + GPC_PGC_SR(IMX8MP_PGC_PCIE)), + regmap_reg_range(GPC_PGC_CTRL(IMX8MP_PGC_USB1), + GPC_PGC_SR(IMX8MP_PGC_USB1)), + regmap_reg_range(GPC_PGC_CTRL(IMX8MP_PGC_USB2), + GPC_PGC_SR(IMX8MP_PGC_USB2)), + regmap_reg_range(GPC_PGC_CTRL(IMX8MP_PGC_MLMIX), + GPC_PGC_SR(IMX8MP_PGC_MLMIX)), + regmap_reg_range(GPC_PGC_CTRL(IMX8MP_PGC_AUDIOMIX), + GPC_PGC_SR(IMX8MP_PGC_AUDIOMIX)), + regmap_reg_range(GPC_PGC_CTRL(IMX8MP_PGC_GPU2D), + GPC_PGC_SR(IMX8MP_PGC_GPU2D)), + regmap_reg_range(GPC_PGC_CTRL(IMX8MP_PGC_GPUMIX), + GPC_PGC_SR(IMX8MP_PGC_GPUMIX)), + regmap_reg_range(GPC_PGC_CTRL(IMX8MP_PGC_VPUMIX), + GPC_PGC_SR(IMX8MP_PGC_VPUMIX)), + regmap_reg_range(GPC_PGC_CTRL(IMX8MP_PGC_GPU3D), + GPC_PGC_SR(IMX8MP_PGC_GPU3D)), + regmap_reg_range(GPC_PGC_CTRL(IMX8MP_PGC_MEDIAMIX), + GPC_PGC_SR(IMX8MP_PGC_MEDIAMIX)), + regmap_reg_range(GPC_PGC_CTRL(IMX8MP_PGC_VPU_G1), + GPC_PGC_SR(IMX8MP_PGC_VPU_G1)), + regmap_reg_range(GPC_PGC_CTRL(IMX8MP_PGC_VPU_G2), + GPC_PGC_SR(IMX8MP_PGC_VPU_G2)), + regmap_reg_range(GPC_PGC_CTRL(IMX8MP_PGC_VPU_VC8000E), + GPC_PGC_SR(IMX8MP_PGC_VPU_VC8000E)), + regmap_reg_range(GPC_PGC_CTRL(IMX8MP_PGC_HDMIMIX), + GPC_PGC_SR(IMX8MP_PGC_HDMIMIX)), + regmap_reg_range(GPC_PGC_CTRL(IMX8MP_PGC_HDMI), + GPC_PGC_SR(IMX8MP_PGC_HDMI)), + regmap_reg_range(GPC_PGC_CTRL(IMX8MP_PGC_MIPI2), + GPC_PGC_SR(IMX8MP_PGC_MIPI2)), + regmap_reg_range(GPC_PGC_CTRL(IMX8MP_PGC_HSIOMIX), + GPC_PGC_SR(IMX8MP_PGC_HSIOMIX)), + regmap_reg_range(GPC_PGC_CTRL(IMX8MP_PGC_MEDIA_ISP_DWP), + GPC_PGC_SR(IMX8MP_PGC_MEDIA_ISP_DWP)), + regmap_reg_range(GPC_PGC_CTRL(IMX8MP_PGC_DDRMIX), + GPC_PGC_SR(IMX8MP_PGC_DDRMIX)), +}; + +static const struct regmap_access_table imx8mp_access_table = { + .yes_ranges = imx8mp_yes_ranges, + .n_yes_ranges = ARRAY_SIZE(imx8mp_yes_ranges), +}; + +static const struct imx_pgc_regs imx8mp_pgc_regs = { + .map = IMX8MP_GPC_PGC_CPU_MAPPING, + .pup = IMX8MP_GPC_PU_PGC_SW_PUP_REQ, + .pdn = IMX8MP_GPC_PU_PGC_SW_PDN_REQ, + .hsk = IMX8MP_GPC_PU_PWRHSK, +}; +static const struct imx_pgc_domain_data imx8mp_pgc_domain_data = { + .domains = imx8mp_pgc_domains, + .domains_num = ARRAY_SIZE(imx8mp_pgc_domains), + .reg_access_table = &imx8mp_access_table, + .pgc_regs = &imx8mp_pgc_regs, +}; + +static const struct imx_pgc_domain imx8mn_pgc_domains[] = { + [IMX8MN_POWER_DOMAIN_HSIOMIX] = { + .genpd = { + .name = "hsiomix", + }, + .bits = { + .pxx = 0, /* no power sequence control */ + .map = 0, /* no power sequence control */ + .hskreq = IMX8MN_HSIO_HSK_PWRDNREQN, + .hskack = IMX8MN_HSIO_HSK_PWRDNACKN, + }, + .keep_clocks = true, + }, + + [IMX8MN_POWER_DOMAIN_OTG1] = { + .genpd = { + .name = "usb-otg1", + .flags = GENPD_FLAG_ACTIVE_WAKEUP, + }, + .bits = { + .pxx = IMX8MN_OTG1_SW_Pxx_REQ, + .map = IMX8MN_OTG1_A53_DOMAIN, + }, + .pgc = BIT(IMX8MN_PGC_OTG1), + }, + + [IMX8MN_POWER_DOMAIN_GPUMIX] = { + .genpd = { + .name = "gpumix", + }, + .bits = { + .pxx = IMX8MN_GPUMIX_SW_Pxx_REQ, + .map = IMX8MN_GPUMIX_A53_DOMAIN, + .hskreq = IMX8MN_GPUMIX_HSK_PWRDNREQN, + .hskack = IMX8MN_GPUMIX_HSK_PWRDNACKN, + }, + .pgc = BIT(IMX8MN_PGC_GPUMIX), + .keep_clocks = true, + }, + + [IMX8MN_POWER_DOMAIN_DISPMIX] = { + .genpd = { + .name = "dispmix", + }, + .bits = { + .pxx = IMX8MN_DISPMIX_SW_Pxx_REQ, + .map = IMX8MN_DISPMIX_A53_DOMAIN, + .hskreq = IMX8MN_DISPMIX_HSK_PWRDNREQN, + .hskack = IMX8MN_DISPMIX_HSK_PWRDNACKN, + }, + .pgc = BIT(IMX8MN_PGC_DISPMIX), + .keep_clocks = true, + }, + + [IMX8MN_POWER_DOMAIN_MIPI] = { + .genpd = { + .name = "mipi", + }, + .bits = { + .pxx = IMX8MN_MIPI_SW_Pxx_REQ, + .map = IMX8MN_MIPI_A53_DOMAIN, + }, + .pgc = BIT(IMX8MN_PGC_MIPI), + }, +}; + +static const struct regmap_range imx8mn_yes_ranges[] = { + regmap_reg_range(GPC_LPCR_A_CORE_BSC, + GPC_PU_PWRHSK), + regmap_reg_range(GPC_PGC_CTRL(IMX8MN_PGC_MIPI), + GPC_PGC_SR(IMX8MN_PGC_MIPI)), + regmap_reg_range(GPC_PGC_CTRL(IMX8MN_PGC_OTG1), + GPC_PGC_SR(IMX8MN_PGC_OTG1)), + regmap_reg_range(GPC_PGC_CTRL(IMX8MN_PGC_DDR1), + GPC_PGC_SR(IMX8MN_PGC_DDR1)), + regmap_reg_range(GPC_PGC_CTRL(IMX8MN_PGC_GPUMIX), + GPC_PGC_SR(IMX8MN_PGC_GPUMIX)), + regmap_reg_range(GPC_PGC_CTRL(IMX8MN_PGC_DISPMIX), + GPC_PGC_SR(IMX8MN_PGC_DISPMIX)), +}; + +static const struct regmap_access_table imx8mn_access_table = { + .yes_ranges = imx8mn_yes_ranges, + .n_yes_ranges = ARRAY_SIZE(imx8mn_yes_ranges), +}; + +static const struct imx_pgc_domain_data imx8mn_pgc_domain_data = { + .domains = imx8mn_pgc_domains, + .domains_num = ARRAY_SIZE(imx8mn_pgc_domains), + .reg_access_table = &imx8mn_access_table, + .pgc_regs = &imx7_pgc_regs, +}; + +static int imx_pgc_domain_probe(struct platform_device *pdev) +{ + struct imx_pgc_domain *domain = pdev->dev.platform_data; + int ret; + + domain->dev = &pdev->dev; + + domain->regulator = devm_regulator_get_optional(domain->dev, "power"); + if (IS_ERR(domain->regulator)) { + if (PTR_ERR(domain->regulator) != -ENODEV) + return dev_err_probe(domain->dev, PTR_ERR(domain->regulator), + "Failed to get domain's regulator\n"); + } else if (domain->voltage) { + regulator_set_voltage(domain->regulator, + domain->voltage, domain->voltage); + } + + domain->num_clks = devm_clk_bulk_get_all(domain->dev, &domain->clks); + if (domain->num_clks < 0) + return dev_err_probe(domain->dev, domain->num_clks, + "Failed to get domain's clocks\n"); + + domain->reset = devm_reset_control_array_get_optional_exclusive(domain->dev); + if (IS_ERR(domain->reset)) + return dev_err_probe(domain->dev, PTR_ERR(domain->reset), + "Failed to get domain's resets\n"); + + pm_runtime_enable(domain->dev); + + if (domain->bits.map) + regmap_update_bits(domain->regmap, domain->regs->map, + domain->bits.map, domain->bits.map); + + ret = pm_genpd_init(&domain->genpd, NULL, true); + if (ret) { + dev_err(domain->dev, "Failed to init power domain\n"); + goto out_domain_unmap; + } + + if (IS_ENABLED(CONFIG_LOCKDEP) && + of_property_read_bool(domain->dev->of_node, "power-domains")) + lockdep_set_subclass(&domain->genpd.mlock, 1); + + ret = of_genpd_add_provider_simple(domain->dev->of_node, + &domain->genpd); + if (ret) { + dev_err(domain->dev, "Failed to add genpd provider\n"); + goto out_genpd_remove; + } + + return 0; + +out_genpd_remove: + pm_genpd_remove(&domain->genpd); +out_domain_unmap: + if (domain->bits.map) + regmap_update_bits(domain->regmap, domain->regs->map, + domain->bits.map, 0); + pm_runtime_disable(domain->dev); + + return ret; +} + +static int imx_pgc_domain_remove(struct platform_device *pdev) +{ + struct imx_pgc_domain *domain = pdev->dev.platform_data; + + of_genpd_del_provider(domain->dev->of_node); + pm_genpd_remove(&domain->genpd); + + if (domain->bits.map) + regmap_update_bits(domain->regmap, domain->regs->map, + domain->bits.map, 0); + + pm_runtime_disable(domain->dev); + + return 0; +} + +#ifdef CONFIG_PM_SLEEP +static int imx_pgc_domain_suspend(struct device *dev) +{ + int ret; + + /* + * This may look strange, but is done so the generic PM_SLEEP code + * can power down our domain and more importantly power it up again + * after resume, without tripping over our usage of runtime PM to + * power up/down the nested domains. + */ + ret = pm_runtime_get_sync(dev); + if (ret < 0) { + pm_runtime_put_noidle(dev); + return ret; + } + + return 0; +} + +static int imx_pgc_domain_resume(struct device *dev) +{ + return pm_runtime_put(dev); +} +#endif + +static const struct dev_pm_ops imx_pgc_domain_pm_ops = { + SET_SYSTEM_SLEEP_PM_OPS(imx_pgc_domain_suspend, imx_pgc_domain_resume) +}; + +static const struct platform_device_id imx_pgc_domain_id[] = { + { "imx-pgc-domain", }, + { }, +}; + +static struct platform_driver imx_pgc_domain_driver = { + .driver = { + .name = "imx-pgc", + .pm = &imx_pgc_domain_pm_ops, + }, + .probe = imx_pgc_domain_probe, + .remove = imx_pgc_domain_remove, + .id_table = imx_pgc_domain_id, +}; +builtin_platform_driver(imx_pgc_domain_driver) + +static int imx_gpcv2_probe(struct platform_device *pdev) +{ + const struct imx_pgc_domain_data *domain_data = + of_device_get_match_data(&pdev->dev); + + struct regmap_config regmap_config = { + .reg_bits = 32, + .val_bits = 32, + .reg_stride = 4, + .rd_table = domain_data->reg_access_table, + .wr_table = domain_data->reg_access_table, + .max_register = SZ_4K, + }; + struct device *dev = &pdev->dev; + struct device_node *pgc_np, *np; + struct regmap *regmap; + void __iomem *base; + int ret; + + pgc_np = of_get_child_by_name(dev->of_node, "pgc"); + if (!pgc_np) { + dev_err(dev, "No power domains specified in DT\n"); + return -EINVAL; + } + + base = devm_platform_ioremap_resource(pdev, 0); + if (IS_ERR(base)) + return PTR_ERR(base); + + regmap = devm_regmap_init_mmio(dev, base, ®map_config); + if (IS_ERR(regmap)) { + ret = PTR_ERR(regmap); + dev_err(dev, "failed to init regmap (%d)\n", ret); + return ret; + } + + for_each_child_of_node(pgc_np, np) { + struct platform_device *pd_pdev; + struct imx_pgc_domain *domain; + u32 domain_index; + + if (!of_device_is_available(np)) + continue; + + ret = of_property_read_u32(np, "reg", &domain_index); + if (ret) { + dev_err(dev, "Failed to read 'reg' property\n"); + of_node_put(np); + return ret; + } + + if (domain_index >= domain_data->domains_num) { + dev_warn(dev, + "Domain index %d is out of bounds\n", + domain_index); + continue; + } + + pd_pdev = platform_device_alloc("imx-pgc-domain", + domain_index); + if (!pd_pdev) { + dev_err(dev, "Failed to allocate platform device\n"); + of_node_put(np); + return -ENOMEM; + } + + ret = platform_device_add_data(pd_pdev, + &domain_data->domains[domain_index], + sizeof(domain_data->domains[domain_index])); + if (ret) { + platform_device_put(pd_pdev); + of_node_put(np); + return ret; + } + + domain = pd_pdev->dev.platform_data; + domain->regmap = regmap; + domain->regs = domain_data->pgc_regs; + + domain->genpd.power_on = imx_pgc_power_up; + domain->genpd.power_off = imx_pgc_power_down; + + pd_pdev->dev.parent = dev; + device_set_node(&pd_pdev->dev, of_fwnode_handle(np)); + + ret = platform_device_add(pd_pdev); + if (ret) { + platform_device_put(pd_pdev); + of_node_put(np); + return ret; + } + } + + return 0; +} + +static const struct of_device_id imx_gpcv2_dt_ids[] = { + { .compatible = "fsl,imx7d-gpc", .data = &imx7_pgc_domain_data, }, + { .compatible = "fsl,imx8mm-gpc", .data = &imx8mm_pgc_domain_data, }, + { .compatible = "fsl,imx8mn-gpc", .data = &imx8mn_pgc_domain_data, }, + { .compatible = "fsl,imx8mp-gpc", .data = &imx8mp_pgc_domain_data, }, + { .compatible = "fsl,imx8mq-gpc", .data = &imx8m_pgc_domain_data, }, + { } +}; + +static struct platform_driver imx_gpc_driver = { + .driver = { + .name = "imx-gpcv2", + .of_match_table = imx_gpcv2_dt_ids, + }, + .probe = imx_gpcv2_probe, +}; +builtin_platform_driver(imx_gpc_driver) diff --git a/drivers/pmdomain/imx/imx8m-blk-ctrl.c b/drivers/pmdomain/imx/imx8m-blk-ctrl.c new file mode 100644 index 000000000000..cc5ef6e2f0a8 --- /dev/null +++ b/drivers/pmdomain/imx/imx8m-blk-ctrl.c @@ -0,0 +1,899 @@ +// SPDX-License-Identifier: GPL-2.0+ + +/* + * Copyright 2021 Pengutronix, Lucas Stach + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include + +#define BLK_SFT_RSTN 0x0 +#define BLK_CLK_EN 0x4 +#define BLK_MIPI_RESET_DIV 0x8 /* Mini/Nano/Plus DISPLAY_BLK_CTRL only */ + +struct imx8m_blk_ctrl_domain; + +struct imx8m_blk_ctrl { + struct device *dev; + struct notifier_block power_nb; + struct device *bus_power_dev; + struct regmap *regmap; + struct imx8m_blk_ctrl_domain *domains; + struct genpd_onecell_data onecell_data; +}; + +struct imx8m_blk_ctrl_domain_data { + const char *name; + const char * const *clk_names; + const char * const *path_names; + const char *gpc_name; + int num_clks; + int num_paths; + u32 rst_mask; + u32 clk_mask; + + /* + * i.MX8M Mini, Nano and Plus have a third DISPLAY_BLK_CTRL register + * which is used to control the reset for the MIPI Phy. + * Since it's only present in certain circumstances, + * an if-statement should be used before setting and clearing this + * register. + */ + u32 mipi_phy_rst_mask; +}; + +#define DOMAIN_MAX_CLKS 4 +#define DOMAIN_MAX_PATHS 4 + +struct imx8m_blk_ctrl_domain { + struct generic_pm_domain genpd; + const struct imx8m_blk_ctrl_domain_data *data; + struct clk_bulk_data clks[DOMAIN_MAX_CLKS]; + struct icc_bulk_data paths[DOMAIN_MAX_PATHS]; + struct device *power_dev; + struct imx8m_blk_ctrl *bc; + int num_paths; +}; + +struct imx8m_blk_ctrl_data { + int max_reg; + notifier_fn_t power_notifier_fn; + const struct imx8m_blk_ctrl_domain_data *domains; + int num_domains; +}; + +static inline struct imx8m_blk_ctrl_domain * +to_imx8m_blk_ctrl_domain(struct generic_pm_domain *genpd) +{ + return container_of(genpd, struct imx8m_blk_ctrl_domain, genpd); +} + +static int imx8m_blk_ctrl_power_on(struct generic_pm_domain *genpd) +{ + struct imx8m_blk_ctrl_domain *domain = to_imx8m_blk_ctrl_domain(genpd); + const struct imx8m_blk_ctrl_domain_data *data = domain->data; + struct imx8m_blk_ctrl *bc = domain->bc; + int ret; + + /* make sure bus domain is awake */ + ret = pm_runtime_get_sync(bc->bus_power_dev); + if (ret < 0) { + pm_runtime_put_noidle(bc->bus_power_dev); + dev_err(bc->dev, "failed to power up bus domain\n"); + return ret; + } + + /* put devices into reset */ + regmap_clear_bits(bc->regmap, BLK_SFT_RSTN, data->rst_mask); + if (data->mipi_phy_rst_mask) + regmap_clear_bits(bc->regmap, BLK_MIPI_RESET_DIV, data->mipi_phy_rst_mask); + + /* enable upstream and blk-ctrl clocks to allow reset to propagate */ + ret = clk_bulk_prepare_enable(data->num_clks, domain->clks); + if (ret) { + dev_err(bc->dev, "failed to enable clocks\n"); + goto bus_put; + } + regmap_set_bits(bc->regmap, BLK_CLK_EN, data->clk_mask); + + /* power up upstream GPC domain */ + ret = pm_runtime_get_sync(domain->power_dev); + if (ret < 0) { + dev_err(bc->dev, "failed to power up peripheral domain\n"); + goto clk_disable; + } + + /* wait for reset to propagate */ + udelay(5); + + /* release reset */ + regmap_set_bits(bc->regmap, BLK_SFT_RSTN, data->rst_mask); + if (data->mipi_phy_rst_mask) + regmap_set_bits(bc->regmap, BLK_MIPI_RESET_DIV, data->mipi_phy_rst_mask); + + ret = icc_bulk_set_bw(domain->num_paths, domain->paths); + if (ret) + dev_err(bc->dev, "failed to set icc bw\n"); + + /* disable upstream clocks */ + clk_bulk_disable_unprepare(data->num_clks, domain->clks); + + return 0; + +clk_disable: + clk_bulk_disable_unprepare(data->num_clks, domain->clks); +bus_put: + pm_runtime_put(bc->bus_power_dev); + + return ret; +} + +static int imx8m_blk_ctrl_power_off(struct generic_pm_domain *genpd) +{ + struct imx8m_blk_ctrl_domain *domain = to_imx8m_blk_ctrl_domain(genpd); + const struct imx8m_blk_ctrl_domain_data *data = domain->data; + struct imx8m_blk_ctrl *bc = domain->bc; + + /* put devices into reset and disable clocks */ + if (data->mipi_phy_rst_mask) + regmap_clear_bits(bc->regmap, BLK_MIPI_RESET_DIV, data->mipi_phy_rst_mask); + + regmap_clear_bits(bc->regmap, BLK_SFT_RSTN, data->rst_mask); + regmap_clear_bits(bc->regmap, BLK_CLK_EN, data->clk_mask); + + /* power down upstream GPC domain */ + pm_runtime_put(domain->power_dev); + + /* allow bus domain to suspend */ + pm_runtime_put(bc->bus_power_dev); + + return 0; +} + +static struct lock_class_key blk_ctrl_genpd_lock_class; + +static int imx8m_blk_ctrl_probe(struct platform_device *pdev) +{ + const struct imx8m_blk_ctrl_data *bc_data; + struct device *dev = &pdev->dev; + struct imx8m_blk_ctrl *bc; + void __iomem *base; + int i, ret; + + struct regmap_config regmap_config = { + .reg_bits = 32, + .val_bits = 32, + .reg_stride = 4, + }; + + bc = devm_kzalloc(dev, sizeof(*bc), GFP_KERNEL); + if (!bc) + return -ENOMEM; + + bc->dev = dev; + + bc_data = of_device_get_match_data(dev); + + base = devm_platform_ioremap_resource(pdev, 0); + if (IS_ERR(base)) + return PTR_ERR(base); + + regmap_config.max_register = bc_data->max_reg; + bc->regmap = devm_regmap_init_mmio(dev, base, ®map_config); + if (IS_ERR(bc->regmap)) + return dev_err_probe(dev, PTR_ERR(bc->regmap), + "failed to init regmap\n"); + + bc->domains = devm_kcalloc(dev, bc_data->num_domains, + sizeof(struct imx8m_blk_ctrl_domain), + GFP_KERNEL); + if (!bc->domains) + return -ENOMEM; + + bc->onecell_data.num_domains = bc_data->num_domains; + bc->onecell_data.domains = + devm_kcalloc(dev, bc_data->num_domains, + sizeof(struct generic_pm_domain *), GFP_KERNEL); + if (!bc->onecell_data.domains) + return -ENOMEM; + + bc->bus_power_dev = dev_pm_domain_attach_by_name(dev, "bus"); + if (IS_ERR(bc->bus_power_dev)) { + if (PTR_ERR(bc->bus_power_dev) == -ENODEV) + return dev_err_probe(dev, -EPROBE_DEFER, + "failed to attach power domain \"bus\"\n"); + else + return dev_err_probe(dev, PTR_ERR(bc->bus_power_dev), + "failed to attach power domain \"bus\"\n"); + } + + for (i = 0; i < bc_data->num_domains; i++) { + const struct imx8m_blk_ctrl_domain_data *data = &bc_data->domains[i]; + struct imx8m_blk_ctrl_domain *domain = &bc->domains[i]; + int j; + + domain->data = data; + domain->num_paths = data->num_paths; + + for (j = 0; j < data->num_clks; j++) + domain->clks[j].id = data->clk_names[j]; + + for (j = 0; j < data->num_paths; j++) { + domain->paths[j].name = data->path_names[j]; + /* Fake value for now, just let ICC could configure NoC mode/priority */ + domain->paths[j].avg_bw = 1; + domain->paths[j].peak_bw = 1; + } + + ret = devm_of_icc_bulk_get(dev, data->num_paths, domain->paths); + if (ret) { + if (ret != -EPROBE_DEFER) { + dev_warn_once(dev, "Could not get interconnect paths, NoC will stay unconfigured!\n"); + domain->num_paths = 0; + } else { + dev_err_probe(dev, ret, "failed to get noc entries\n"); + goto cleanup_pds; + } + } + + ret = devm_clk_bulk_get(dev, data->num_clks, domain->clks); + if (ret) { + dev_err_probe(dev, ret, "failed to get clock\n"); + goto cleanup_pds; + } + + domain->power_dev = + dev_pm_domain_attach_by_name(dev, data->gpc_name); + if (IS_ERR(domain->power_dev)) { + dev_err_probe(dev, PTR_ERR(domain->power_dev), + "failed to attach power domain \"%s\"\n", + data->gpc_name); + ret = PTR_ERR(domain->power_dev); + goto cleanup_pds; + } + + domain->genpd.name = data->name; + domain->genpd.power_on = imx8m_blk_ctrl_power_on; + domain->genpd.power_off = imx8m_blk_ctrl_power_off; + domain->bc = bc; + + ret = pm_genpd_init(&domain->genpd, NULL, true); + if (ret) { + dev_err_probe(dev, ret, + "failed to init power domain \"%s\"\n", + data->gpc_name); + dev_pm_domain_detach(domain->power_dev, true); + goto cleanup_pds; + } + + /* + * We use runtime PM to trigger power on/off of the upstream GPC + * domain, as a strict hierarchical parent/child power domain + * setup doesn't allow us to meet the sequencing requirements. + * This means we have nested locking of genpd locks, without the + * nesting being visible at the genpd level, so we need a + * separate lock class to make lockdep aware of the fact that + * this are separate domain locks that can be nested without a + * self-deadlock. + */ + lockdep_set_class(&domain->genpd.mlock, + &blk_ctrl_genpd_lock_class); + + bc->onecell_data.domains[i] = &domain->genpd; + } + + ret = of_genpd_add_provider_onecell(dev->of_node, &bc->onecell_data); + if (ret) { + dev_err_probe(dev, ret, "failed to add power domain provider\n"); + goto cleanup_pds; + } + + bc->power_nb.notifier_call = bc_data->power_notifier_fn; + ret = dev_pm_genpd_add_notifier(bc->bus_power_dev, &bc->power_nb); + if (ret) { + dev_err_probe(dev, ret, "failed to add power notifier\n"); + goto cleanup_provider; + } + + dev_set_drvdata(dev, bc); + + ret = devm_of_platform_populate(dev); + if (ret) + goto cleanup_provider; + + return 0; + +cleanup_provider: + of_genpd_del_provider(dev->of_node); +cleanup_pds: + for (i--; i >= 0; i--) { + pm_genpd_remove(&bc->domains[i].genpd); + dev_pm_domain_detach(bc->domains[i].power_dev, true); + } + + dev_pm_domain_detach(bc->bus_power_dev, true); + + return ret; +} + +static int imx8m_blk_ctrl_remove(struct platform_device *pdev) +{ + struct imx8m_blk_ctrl *bc = dev_get_drvdata(&pdev->dev); + int i; + + of_genpd_del_provider(pdev->dev.of_node); + + for (i = 0; bc->onecell_data.num_domains; i++) { + struct imx8m_blk_ctrl_domain *domain = &bc->domains[i]; + + pm_genpd_remove(&domain->genpd); + dev_pm_domain_detach(domain->power_dev, true); + } + + dev_pm_genpd_remove_notifier(bc->bus_power_dev); + + dev_pm_domain_detach(bc->bus_power_dev, true); + + return 0; +} + +#ifdef CONFIG_PM_SLEEP +static int imx8m_blk_ctrl_suspend(struct device *dev) +{ + struct imx8m_blk_ctrl *bc = dev_get_drvdata(dev); + int ret, i; + + /* + * This may look strange, but is done so the generic PM_SLEEP code + * can power down our domains and more importantly power them up again + * after resume, without tripping over our usage of runtime PM to + * control the upstream GPC domains. Things happen in the right order + * in the system suspend/resume paths due to the device parent/child + * hierarchy. + */ + ret = pm_runtime_get_sync(bc->bus_power_dev); + if (ret < 0) { + pm_runtime_put_noidle(bc->bus_power_dev); + return ret; + } + + for (i = 0; i < bc->onecell_data.num_domains; i++) { + struct imx8m_blk_ctrl_domain *domain = &bc->domains[i]; + + ret = pm_runtime_get_sync(domain->power_dev); + if (ret < 0) { + pm_runtime_put_noidle(domain->power_dev); + goto out_fail; + } + } + + return 0; + +out_fail: + for (i--; i >= 0; i--) + pm_runtime_put(bc->domains[i].power_dev); + + pm_runtime_put(bc->bus_power_dev); + + return ret; +} + +static int imx8m_blk_ctrl_resume(struct device *dev) +{ + struct imx8m_blk_ctrl *bc = dev_get_drvdata(dev); + int i; + + for (i = 0; i < bc->onecell_data.num_domains; i++) + pm_runtime_put(bc->domains[i].power_dev); + + pm_runtime_put(bc->bus_power_dev); + + return 0; +} +#endif + +static const struct dev_pm_ops imx8m_blk_ctrl_pm_ops = { + SET_SYSTEM_SLEEP_PM_OPS(imx8m_blk_ctrl_suspend, imx8m_blk_ctrl_resume) +}; + +static int imx8mm_vpu_power_notifier(struct notifier_block *nb, + unsigned long action, void *data) +{ + struct imx8m_blk_ctrl *bc = container_of(nb, struct imx8m_blk_ctrl, + power_nb); + + if (action != GENPD_NOTIFY_ON && action != GENPD_NOTIFY_PRE_OFF) + return NOTIFY_OK; + + /* + * The ADB in the VPUMIX domain has no separate reset and clock + * enable bits, but is ungated together with the VPU clocks. To + * allow the handshake with the GPC to progress we put the VPUs + * in reset and ungate the clocks. + */ + regmap_clear_bits(bc->regmap, BLK_SFT_RSTN, BIT(0) | BIT(1) | BIT(2)); + regmap_set_bits(bc->regmap, BLK_CLK_EN, BIT(0) | BIT(1) | BIT(2)); + + if (action == GENPD_NOTIFY_ON) { + /* + * On power up we have no software backchannel to the GPC to + * wait for the ADB handshake to happen, so we just delay for a + * bit. On power down the GPC driver waits for the handshake. + */ + udelay(5); + + /* set "fuse" bits to enable the VPUs */ + regmap_set_bits(bc->regmap, 0x8, 0xffffffff); + regmap_set_bits(bc->regmap, 0xc, 0xffffffff); + regmap_set_bits(bc->regmap, 0x10, 0xffffffff); + regmap_set_bits(bc->regmap, 0x14, 0xffffffff); + } + + return NOTIFY_OK; +} + +static const struct imx8m_blk_ctrl_domain_data imx8mm_vpu_blk_ctl_domain_data[] = { + [IMX8MM_VPUBLK_PD_G1] = { + .name = "vpublk-g1", + .clk_names = (const char *[]){ "g1", }, + .num_clks = 1, + .gpc_name = "g1", + .rst_mask = BIT(1), + .clk_mask = BIT(1), + }, + [IMX8MM_VPUBLK_PD_G2] = { + .name = "vpublk-g2", + .clk_names = (const char *[]){ "g2", }, + .num_clks = 1, + .gpc_name = "g2", + .rst_mask = BIT(0), + .clk_mask = BIT(0), + }, + [IMX8MM_VPUBLK_PD_H1] = { + .name = "vpublk-h1", + .clk_names = (const char *[]){ "h1", }, + .num_clks = 1, + .gpc_name = "h1", + .rst_mask = BIT(2), + .clk_mask = BIT(2), + }, +}; + +static const struct imx8m_blk_ctrl_data imx8mm_vpu_blk_ctl_dev_data = { + .max_reg = 0x18, + .power_notifier_fn = imx8mm_vpu_power_notifier, + .domains = imx8mm_vpu_blk_ctl_domain_data, + .num_domains = ARRAY_SIZE(imx8mm_vpu_blk_ctl_domain_data), +}; + +static const struct imx8m_blk_ctrl_domain_data imx8mp_vpu_blk_ctl_domain_data[] = { + [IMX8MP_VPUBLK_PD_G1] = { + .name = "vpublk-g1", + .clk_names = (const char *[]){ "g1", }, + .num_clks = 1, + .gpc_name = "g1", + .rst_mask = BIT(1), + .clk_mask = BIT(1), + .path_names = (const char *[]){"g1"}, + .num_paths = 1, + }, + [IMX8MP_VPUBLK_PD_G2] = { + .name = "vpublk-g2", + .clk_names = (const char *[]){ "g2", }, + .num_clks = 1, + .gpc_name = "g2", + .rst_mask = BIT(0), + .clk_mask = BIT(0), + .path_names = (const char *[]){"g2"}, + .num_paths = 1, + }, + [IMX8MP_VPUBLK_PD_VC8000E] = { + .name = "vpublk-vc8000e", + .clk_names = (const char *[]){ "vc8000e", }, + .num_clks = 1, + .gpc_name = "vc8000e", + .rst_mask = BIT(2), + .clk_mask = BIT(2), + .path_names = (const char *[]){"vc8000e"}, + .num_paths = 1, + }, +}; + +static const struct imx8m_blk_ctrl_data imx8mp_vpu_blk_ctl_dev_data = { + .max_reg = 0x18, + .power_notifier_fn = imx8mm_vpu_power_notifier, + .domains = imx8mp_vpu_blk_ctl_domain_data, + .num_domains = ARRAY_SIZE(imx8mp_vpu_blk_ctl_domain_data), +}; + +static int imx8mm_disp_power_notifier(struct notifier_block *nb, + unsigned long action, void *data) +{ + struct imx8m_blk_ctrl *bc = container_of(nb, struct imx8m_blk_ctrl, + power_nb); + + if (action != GENPD_NOTIFY_ON && action != GENPD_NOTIFY_PRE_OFF) + return NOTIFY_OK; + + /* Enable bus clock and deassert bus reset */ + regmap_set_bits(bc->regmap, BLK_CLK_EN, BIT(12)); + regmap_set_bits(bc->regmap, BLK_SFT_RSTN, BIT(6)); + + /* + * On power up we have no software backchannel to the GPC to + * wait for the ADB handshake to happen, so we just delay for a + * bit. On power down the GPC driver waits for the handshake. + */ + if (action == GENPD_NOTIFY_ON) + udelay(5); + + + return NOTIFY_OK; +} + +static const struct imx8m_blk_ctrl_domain_data imx8mm_disp_blk_ctl_domain_data[] = { + [IMX8MM_DISPBLK_PD_CSI_BRIDGE] = { + .name = "dispblk-csi-bridge", + .clk_names = (const char *[]){ "csi-bridge-axi", "csi-bridge-apb", + "csi-bridge-core", }, + .num_clks = 3, + .gpc_name = "csi-bridge", + .rst_mask = BIT(0) | BIT(1) | BIT(2), + .clk_mask = BIT(0) | BIT(1) | BIT(2) | BIT(3) | BIT(4) | BIT(5), + }, + [IMX8MM_DISPBLK_PD_LCDIF] = { + .name = "dispblk-lcdif", + .clk_names = (const char *[]){ "lcdif-axi", "lcdif-apb", "lcdif-pix", }, + .num_clks = 3, + .gpc_name = "lcdif", + .clk_mask = BIT(6) | BIT(7), + }, + [IMX8MM_DISPBLK_PD_MIPI_DSI] = { + .name = "dispblk-mipi-dsi", + .clk_names = (const char *[]){ "dsi-pclk", "dsi-ref", }, + .num_clks = 2, + .gpc_name = "mipi-dsi", + .rst_mask = BIT(5), + .clk_mask = BIT(8) | BIT(9), + .mipi_phy_rst_mask = BIT(17), + }, + [IMX8MM_DISPBLK_PD_MIPI_CSI] = { + .name = "dispblk-mipi-csi", + .clk_names = (const char *[]){ "csi-aclk", "csi-pclk" }, + .num_clks = 2, + .gpc_name = "mipi-csi", + .rst_mask = BIT(3) | BIT(4), + .clk_mask = BIT(10) | BIT(11), + .mipi_phy_rst_mask = BIT(16), + }, +}; + +static const struct imx8m_blk_ctrl_data imx8mm_disp_blk_ctl_dev_data = { + .max_reg = 0x2c, + .power_notifier_fn = imx8mm_disp_power_notifier, + .domains = imx8mm_disp_blk_ctl_domain_data, + .num_domains = ARRAY_SIZE(imx8mm_disp_blk_ctl_domain_data), +}; + + +static int imx8mn_disp_power_notifier(struct notifier_block *nb, + unsigned long action, void *data) +{ + struct imx8m_blk_ctrl *bc = container_of(nb, struct imx8m_blk_ctrl, + power_nb); + + if (action != GENPD_NOTIFY_ON && action != GENPD_NOTIFY_PRE_OFF) + return NOTIFY_OK; + + /* Enable bus clock and deassert bus reset */ + regmap_set_bits(bc->regmap, BLK_CLK_EN, BIT(8)); + regmap_set_bits(bc->regmap, BLK_SFT_RSTN, BIT(8)); + + /* + * On power up we have no software backchannel to the GPC to + * wait for the ADB handshake to happen, so we just delay for a + * bit. On power down the GPC driver waits for the handshake. + */ + if (action == GENPD_NOTIFY_ON) + udelay(5); + + + return NOTIFY_OK; +} + +static const struct imx8m_blk_ctrl_domain_data imx8mn_disp_blk_ctl_domain_data[] = { + [IMX8MN_DISPBLK_PD_MIPI_DSI] = { + .name = "dispblk-mipi-dsi", + .clk_names = (const char *[]){ "dsi-pclk", "dsi-ref", }, + .num_clks = 2, + .gpc_name = "mipi-dsi", + .rst_mask = BIT(0) | BIT(1), + .clk_mask = BIT(0) | BIT(1), + .mipi_phy_rst_mask = BIT(17), + }, + [IMX8MN_DISPBLK_PD_MIPI_CSI] = { + .name = "dispblk-mipi-csi", + .clk_names = (const char *[]){ "csi-aclk", "csi-pclk" }, + .num_clks = 2, + .gpc_name = "mipi-csi", + .rst_mask = BIT(2) | BIT(3), + .clk_mask = BIT(2) | BIT(3), + .mipi_phy_rst_mask = BIT(16), + }, + [IMX8MN_DISPBLK_PD_LCDIF] = { + .name = "dispblk-lcdif", + .clk_names = (const char *[]){ "lcdif-axi", "lcdif-apb", "lcdif-pix", }, + .num_clks = 3, + .gpc_name = "lcdif", + .rst_mask = BIT(4) | BIT(5), + .clk_mask = BIT(4) | BIT(5), + }, + [IMX8MN_DISPBLK_PD_ISI] = { + .name = "dispblk-isi", + .clk_names = (const char *[]){ "disp_axi", "disp_apb", "disp_axi_root", + "disp_apb_root"}, + .num_clks = 4, + .gpc_name = "isi", + .rst_mask = BIT(6) | BIT(7), + .clk_mask = BIT(6) | BIT(7), + }, +}; + +static const struct imx8m_blk_ctrl_data imx8mn_disp_blk_ctl_dev_data = { + .max_reg = 0x84, + .power_notifier_fn = imx8mn_disp_power_notifier, + .domains = imx8mn_disp_blk_ctl_domain_data, + .num_domains = ARRAY_SIZE(imx8mn_disp_blk_ctl_domain_data), +}; + +#define LCDIF_ARCACHE_CTRL 0x4c +#define LCDIF_1_RD_HURRY GENMASK(15, 13) +#define LCDIF_0_RD_HURRY GENMASK(12, 10) + +static int imx8mp_media_power_notifier(struct notifier_block *nb, + unsigned long action, void *data) +{ + struct imx8m_blk_ctrl *bc = container_of(nb, struct imx8m_blk_ctrl, + power_nb); + + if (action != GENPD_NOTIFY_ON && action != GENPD_NOTIFY_PRE_OFF) + return NOTIFY_OK; + + /* Enable bus clock and deassert bus reset */ + regmap_set_bits(bc->regmap, BLK_CLK_EN, BIT(8)); + regmap_set_bits(bc->regmap, BLK_SFT_RSTN, BIT(8)); + + if (action == GENPD_NOTIFY_ON) { + /* + * On power up we have no software backchannel to the GPC to + * wait for the ADB handshake to happen, so we just delay for a + * bit. On power down the GPC driver waits for the handshake. + */ + udelay(5); + + /* + * Set panic read hurry level for both LCDIF interfaces to + * maximum priority to minimize chances of display FIFO + * underflow. + */ + regmap_set_bits(bc->regmap, LCDIF_ARCACHE_CTRL, + FIELD_PREP(LCDIF_1_RD_HURRY, 7) | + FIELD_PREP(LCDIF_0_RD_HURRY, 7)); + } + + return NOTIFY_OK; +} + +/* + * From i.MX 8M Plus Applications Processor Reference Manual, Rev. 1, + * section 13.2.2, 13.2.3 + * isp-ahb and dwe are not in Figure 13-5. Media BLK_CTRL Clocks + */ +static const struct imx8m_blk_ctrl_domain_data imx8mp_media_blk_ctl_domain_data[] = { + [IMX8MP_MEDIABLK_PD_MIPI_DSI_1] = { + .name = "mediablk-mipi-dsi-1", + .clk_names = (const char *[]){ "apb", "phy", }, + .num_clks = 2, + .gpc_name = "mipi-dsi1", + .rst_mask = BIT(0) | BIT(1), + .clk_mask = BIT(0) | BIT(1), + .mipi_phy_rst_mask = BIT(17), + }, + [IMX8MP_MEDIABLK_PD_MIPI_CSI2_1] = { + .name = "mediablk-mipi-csi2-1", + .clk_names = (const char *[]){ "apb", "cam1" }, + .num_clks = 2, + .gpc_name = "mipi-csi1", + .rst_mask = BIT(2) | BIT(3), + .clk_mask = BIT(2) | BIT(3), + .mipi_phy_rst_mask = BIT(16), + }, + [IMX8MP_MEDIABLK_PD_LCDIF_1] = { + .name = "mediablk-lcdif-1", + .clk_names = (const char *[]){ "disp1", "apb", "axi", }, + .num_clks = 3, + .gpc_name = "lcdif1", + .rst_mask = BIT(4) | BIT(5) | BIT(23), + .clk_mask = BIT(4) | BIT(5) | BIT(23), + .path_names = (const char *[]){"lcdif-rd", "lcdif-wr"}, + .num_paths = 2, + }, + [IMX8MP_MEDIABLK_PD_ISI] = { + .name = "mediablk-isi", + .clk_names = (const char *[]){ "axi", "apb" }, + .num_clks = 2, + .gpc_name = "isi", + .rst_mask = BIT(6) | BIT(7), + .clk_mask = BIT(6) | BIT(7), + .path_names = (const char *[]){"isi0", "isi1", "isi2"}, + .num_paths = 3, + }, + [IMX8MP_MEDIABLK_PD_MIPI_CSI2_2] = { + .name = "mediablk-mipi-csi2-2", + .clk_names = (const char *[]){ "apb", "cam2" }, + .num_clks = 2, + .gpc_name = "mipi-csi2", + .rst_mask = BIT(9) | BIT(10), + .clk_mask = BIT(9) | BIT(10), + .mipi_phy_rst_mask = BIT(30), + }, + [IMX8MP_MEDIABLK_PD_LCDIF_2] = { + .name = "mediablk-lcdif-2", + .clk_names = (const char *[]){ "disp2", "apb", "axi", }, + .num_clks = 3, + .gpc_name = "lcdif2", + .rst_mask = BIT(11) | BIT(12) | BIT(24), + .clk_mask = BIT(11) | BIT(12) | BIT(24), + .path_names = (const char *[]){"lcdif-rd", "lcdif-wr"}, + .num_paths = 2, + }, + [IMX8MP_MEDIABLK_PD_ISP] = { + .name = "mediablk-isp", + .clk_names = (const char *[]){ "isp", "axi", "apb" }, + .num_clks = 3, + .gpc_name = "isp", + .rst_mask = BIT(16) | BIT(17) | BIT(18), + .clk_mask = BIT(16) | BIT(17) | BIT(18), + .path_names = (const char *[]){"isp0", "isp1"}, + .num_paths = 2, + }, + [IMX8MP_MEDIABLK_PD_DWE] = { + .name = "mediablk-dwe", + .clk_names = (const char *[]){ "axi", "apb" }, + .num_clks = 2, + .gpc_name = "dwe", + .rst_mask = BIT(19) | BIT(20) | BIT(21), + .clk_mask = BIT(19) | BIT(20) | BIT(21), + .path_names = (const char *[]){"dwe"}, + .num_paths = 1, + }, + [IMX8MP_MEDIABLK_PD_MIPI_DSI_2] = { + .name = "mediablk-mipi-dsi-2", + .clk_names = (const char *[]){ "phy", }, + .num_clks = 1, + .gpc_name = "mipi-dsi2", + .rst_mask = BIT(22), + .clk_mask = BIT(22), + .mipi_phy_rst_mask = BIT(29), + }, +}; + +static const struct imx8m_blk_ctrl_data imx8mp_media_blk_ctl_dev_data = { + .max_reg = 0x138, + .power_notifier_fn = imx8mp_media_power_notifier, + .domains = imx8mp_media_blk_ctl_domain_data, + .num_domains = ARRAY_SIZE(imx8mp_media_blk_ctl_domain_data), +}; + +static int imx8mq_vpu_power_notifier(struct notifier_block *nb, + unsigned long action, void *data) +{ + struct imx8m_blk_ctrl *bc = container_of(nb, struct imx8m_blk_ctrl, + power_nb); + + if (action != GENPD_NOTIFY_ON && action != GENPD_NOTIFY_PRE_OFF) + return NOTIFY_OK; + + /* + * The ADB in the VPUMIX domain has no separate reset and clock + * enable bits, but is ungated and reset together with the VPUs. The + * reset and clock enable inputs to the ADB is a logical OR of the + * VPU bits. In order to set the G2 fuse bits, the G2 clock must + * also be enabled. + */ + regmap_set_bits(bc->regmap, BLK_SFT_RSTN, BIT(0) | BIT(1)); + regmap_set_bits(bc->regmap, BLK_CLK_EN, BIT(0) | BIT(1)); + + if (action == GENPD_NOTIFY_ON) { + /* + * On power up we have no software backchannel to the GPC to + * wait for the ADB handshake to happen, so we just delay for a + * bit. On power down the GPC driver waits for the handshake. + */ + udelay(5); + + /* set "fuse" bits to enable the VPUs */ + regmap_set_bits(bc->regmap, 0x8, 0xffffffff); + regmap_set_bits(bc->regmap, 0xc, 0xffffffff); + regmap_set_bits(bc->regmap, 0x10, 0xffffffff); + } + + return NOTIFY_OK; +} + +static const struct imx8m_blk_ctrl_domain_data imx8mq_vpu_blk_ctl_domain_data[] = { + [IMX8MQ_VPUBLK_PD_G1] = { + .name = "vpublk-g1", + .clk_names = (const char *[]){ "g1", }, + .num_clks = 1, + .gpc_name = "g1", + .rst_mask = BIT(1), + .clk_mask = BIT(1), + }, + [IMX8MQ_VPUBLK_PD_G2] = { + .name = "vpublk-g2", + .clk_names = (const char *[]){ "g2", }, + .num_clks = 1, + .gpc_name = "g2", + .rst_mask = BIT(0), + .clk_mask = BIT(0), + }, +}; + +static const struct imx8m_blk_ctrl_data imx8mq_vpu_blk_ctl_dev_data = { + .max_reg = 0x14, + .power_notifier_fn = imx8mq_vpu_power_notifier, + .domains = imx8mq_vpu_blk_ctl_domain_data, + .num_domains = ARRAY_SIZE(imx8mq_vpu_blk_ctl_domain_data), +}; + +static const struct of_device_id imx8m_blk_ctrl_of_match[] = { + { + .compatible = "fsl,imx8mm-vpu-blk-ctrl", + .data = &imx8mm_vpu_blk_ctl_dev_data + }, { + .compatible = "fsl,imx8mm-disp-blk-ctrl", + .data = &imx8mm_disp_blk_ctl_dev_data + }, { + .compatible = "fsl,imx8mn-disp-blk-ctrl", + .data = &imx8mn_disp_blk_ctl_dev_data + }, { + .compatible = "fsl,imx8mp-media-blk-ctrl", + .data = &imx8mp_media_blk_ctl_dev_data + }, { + .compatible = "fsl,imx8mq-vpu-blk-ctrl", + .data = &imx8mq_vpu_blk_ctl_dev_data + }, { + .compatible = "fsl,imx8mp-vpu-blk-ctrl", + .data = &imx8mp_vpu_blk_ctl_dev_data + }, { + /* Sentinel */ + } +}; +MODULE_DEVICE_TABLE(of, imx8m_blk_ctrl_of_match); + +static struct platform_driver imx8m_blk_ctrl_driver = { + .probe = imx8m_blk_ctrl_probe, + .remove = imx8m_blk_ctrl_remove, + .driver = { + .name = "imx8m-blk-ctrl", + .pm = &imx8m_blk_ctrl_pm_ops, + .of_match_table = imx8m_blk_ctrl_of_match, + }, +}; +module_platform_driver(imx8m_blk_ctrl_driver); +MODULE_LICENSE("GPL"); diff --git a/drivers/pmdomain/imx/imx8mp-blk-ctrl.c b/drivers/pmdomain/imx/imx8mp-blk-ctrl.c new file mode 100644 index 000000000000..c6ac32c1a8c1 --- /dev/null +++ b/drivers/pmdomain/imx/imx8mp-blk-ctrl.c @@ -0,0 +1,867 @@ +// SPDX-License-Identifier: GPL-2.0+ + +/* + * Copyright 2022 Pengutronix, Lucas Stach + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +#define GPR_REG0 0x0 +#define PCIE_CLOCK_MODULE_EN BIT(0) +#define USB_CLOCK_MODULE_EN BIT(1) +#define PCIE_PHY_APB_RST BIT(4) +#define PCIE_PHY_INIT_RST BIT(5) +#define GPR_REG1 0x4 +#define PLL_LOCK BIT(13) +#define GPR_REG2 0x8 +#define P_PLL_MASK GENMASK(5, 0) +#define M_PLL_MASK GENMASK(15, 6) +#define S_PLL_MASK GENMASK(18, 16) +#define GPR_REG3 0xc +#define PLL_CKE BIT(17) +#define PLL_RST BIT(31) + +struct imx8mp_blk_ctrl_domain; + +struct imx8mp_blk_ctrl { + struct device *dev; + struct notifier_block power_nb; + struct device *bus_power_dev; + struct regmap *regmap; + struct imx8mp_blk_ctrl_domain *domains; + struct genpd_onecell_data onecell_data; + void (*power_off) (struct imx8mp_blk_ctrl *bc, struct imx8mp_blk_ctrl_domain *domain); + void (*power_on) (struct imx8mp_blk_ctrl *bc, struct imx8mp_blk_ctrl_domain *domain); +}; + +struct imx8mp_blk_ctrl_domain_data { + const char *name; + const char * const *clk_names; + int num_clks; + const char * const *path_names; + int num_paths; + const char *gpc_name; +}; + +#define DOMAIN_MAX_CLKS 2 +#define DOMAIN_MAX_PATHS 3 + +struct imx8mp_blk_ctrl_domain { + struct generic_pm_domain genpd; + const struct imx8mp_blk_ctrl_domain_data *data; + struct clk_bulk_data clks[DOMAIN_MAX_CLKS]; + struct icc_bulk_data paths[DOMAIN_MAX_PATHS]; + struct device *power_dev; + struct imx8mp_blk_ctrl *bc; + int num_paths; + int id; +}; + +struct imx8mp_blk_ctrl_data { + int max_reg; + int (*probe) (struct imx8mp_blk_ctrl *bc); + notifier_fn_t power_notifier_fn; + void (*power_off) (struct imx8mp_blk_ctrl *bc, struct imx8mp_blk_ctrl_domain *domain); + void (*power_on) (struct imx8mp_blk_ctrl *bc, struct imx8mp_blk_ctrl_domain *domain); + const struct imx8mp_blk_ctrl_domain_data *domains; + int num_domains; +}; + +static inline struct imx8mp_blk_ctrl_domain * +to_imx8mp_blk_ctrl_domain(struct generic_pm_domain *genpd) +{ + return container_of(genpd, struct imx8mp_blk_ctrl_domain, genpd); +} + +struct clk_hsio_pll { + struct clk_hw hw; + struct regmap *regmap; +}; + +static inline struct clk_hsio_pll *to_clk_hsio_pll(struct clk_hw *hw) +{ + return container_of(hw, struct clk_hsio_pll, hw); +} + +static int clk_hsio_pll_prepare(struct clk_hw *hw) +{ + struct clk_hsio_pll *clk = to_clk_hsio_pll(hw); + u32 val; + + /* set the PLL configuration */ + regmap_update_bits(clk->regmap, GPR_REG2, + P_PLL_MASK | M_PLL_MASK | S_PLL_MASK, + FIELD_PREP(P_PLL_MASK, 12) | + FIELD_PREP(M_PLL_MASK, 800) | + FIELD_PREP(S_PLL_MASK, 4)); + + /* de-assert PLL reset */ + regmap_update_bits(clk->regmap, GPR_REG3, PLL_RST, PLL_RST); + + /* enable PLL */ + regmap_update_bits(clk->regmap, GPR_REG3, PLL_CKE, PLL_CKE); + + return regmap_read_poll_timeout(clk->regmap, GPR_REG1, val, + val & PLL_LOCK, 10, 100); +} + +static void clk_hsio_pll_unprepare(struct clk_hw *hw) +{ + struct clk_hsio_pll *clk = to_clk_hsio_pll(hw); + + regmap_update_bits(clk->regmap, GPR_REG3, PLL_RST | PLL_CKE, 0); +} + +static int clk_hsio_pll_is_prepared(struct clk_hw *hw) +{ + struct clk_hsio_pll *clk = to_clk_hsio_pll(hw); + + return regmap_test_bits(clk->regmap, GPR_REG1, PLL_LOCK); +} + +static unsigned long clk_hsio_pll_recalc_rate(struct clk_hw *hw, + unsigned long parent_rate) +{ + return 100000000; +} + +static const struct clk_ops clk_hsio_pll_ops = { + .prepare = clk_hsio_pll_prepare, + .unprepare = clk_hsio_pll_unprepare, + .is_prepared = clk_hsio_pll_is_prepared, + .recalc_rate = clk_hsio_pll_recalc_rate, +}; + +static int imx8mp_hsio_blk_ctrl_probe(struct imx8mp_blk_ctrl *bc) +{ + struct clk_hsio_pll *clk_hsio_pll; + struct clk_hw *hw; + struct clk_init_data init = {}; + int ret; + + clk_hsio_pll = devm_kzalloc(bc->dev, sizeof(*clk_hsio_pll), GFP_KERNEL); + if (!clk_hsio_pll) + return -ENOMEM; + + init.name = "hsio_pll"; + init.ops = &clk_hsio_pll_ops; + init.parent_names = (const char *[]){"osc_24m"}; + init.num_parents = 1; + + clk_hsio_pll->regmap = bc->regmap; + clk_hsio_pll->hw.init = &init; + + hw = &clk_hsio_pll->hw; + ret = devm_clk_hw_register(bc->bus_power_dev, hw); + if (ret) + return ret; + + return devm_of_clk_add_hw_provider(bc->dev, of_clk_hw_simple_get, hw); +} + +static void imx8mp_hsio_blk_ctrl_power_on(struct imx8mp_blk_ctrl *bc, + struct imx8mp_blk_ctrl_domain *domain) +{ + switch (domain->id) { + case IMX8MP_HSIOBLK_PD_USB: + regmap_set_bits(bc->regmap, GPR_REG0, USB_CLOCK_MODULE_EN); + break; + case IMX8MP_HSIOBLK_PD_PCIE: + regmap_set_bits(bc->regmap, GPR_REG0, PCIE_CLOCK_MODULE_EN); + break; + case IMX8MP_HSIOBLK_PD_PCIE_PHY: + regmap_set_bits(bc->regmap, GPR_REG0, + PCIE_PHY_APB_RST | PCIE_PHY_INIT_RST); + break; + default: + break; + } +} + +static void imx8mp_hsio_blk_ctrl_power_off(struct imx8mp_blk_ctrl *bc, + struct imx8mp_blk_ctrl_domain *domain) +{ + switch (domain->id) { + case IMX8MP_HSIOBLK_PD_USB: + regmap_clear_bits(bc->regmap, GPR_REG0, USB_CLOCK_MODULE_EN); + break; + case IMX8MP_HSIOBLK_PD_PCIE: + regmap_clear_bits(bc->regmap, GPR_REG0, PCIE_CLOCK_MODULE_EN); + break; + case IMX8MP_HSIOBLK_PD_PCIE_PHY: + regmap_clear_bits(bc->regmap, GPR_REG0, + PCIE_PHY_APB_RST | PCIE_PHY_INIT_RST); + break; + default: + break; + } +} + +static int imx8mp_hsio_power_notifier(struct notifier_block *nb, + unsigned long action, void *data) +{ + struct imx8mp_blk_ctrl *bc = container_of(nb, struct imx8mp_blk_ctrl, + power_nb); + struct clk_bulk_data *usb_clk = bc->domains[IMX8MP_HSIOBLK_PD_USB].clks; + int num_clks = bc->domains[IMX8MP_HSIOBLK_PD_USB].data->num_clks; + int ret; + + switch (action) { + case GENPD_NOTIFY_ON: + /* + * enable USB clock for a moment for the power-on ADB handshake + * to proceed + */ + ret = clk_bulk_prepare_enable(num_clks, usb_clk); + if (ret) + return NOTIFY_BAD; + regmap_set_bits(bc->regmap, GPR_REG0, USB_CLOCK_MODULE_EN); + + udelay(5); + + regmap_clear_bits(bc->regmap, GPR_REG0, USB_CLOCK_MODULE_EN); + clk_bulk_disable_unprepare(num_clks, usb_clk); + break; + case GENPD_NOTIFY_PRE_OFF: + /* enable USB clock for the power-down ADB handshake to work */ + ret = clk_bulk_prepare_enable(num_clks, usb_clk); + if (ret) + return NOTIFY_BAD; + + regmap_set_bits(bc->regmap, GPR_REG0, USB_CLOCK_MODULE_EN); + break; + case GENPD_NOTIFY_OFF: + clk_bulk_disable_unprepare(num_clks, usb_clk); + break; + default: + break; + } + + return NOTIFY_OK; +} + +static const struct imx8mp_blk_ctrl_domain_data imx8mp_hsio_domain_data[] = { + [IMX8MP_HSIOBLK_PD_USB] = { + .name = "hsioblk-usb", + .clk_names = (const char *[]){ "usb" }, + .num_clks = 1, + .gpc_name = "usb", + .path_names = (const char *[]){"usb1", "usb2"}, + .num_paths = 2, + }, + [IMX8MP_HSIOBLK_PD_USB_PHY1] = { + .name = "hsioblk-usb-phy1", + .gpc_name = "usb-phy1", + }, + [IMX8MP_HSIOBLK_PD_USB_PHY2] = { + .name = "hsioblk-usb-phy2", + .gpc_name = "usb-phy2", + }, + [IMX8MP_HSIOBLK_PD_PCIE] = { + .name = "hsioblk-pcie", + .clk_names = (const char *[]){ "pcie" }, + .num_clks = 1, + .gpc_name = "pcie", + .path_names = (const char *[]){"noc-pcie", "pcie"}, + .num_paths = 2, + }, + [IMX8MP_HSIOBLK_PD_PCIE_PHY] = { + .name = "hsioblk-pcie-phy", + .gpc_name = "pcie-phy", + }, +}; + +static const struct imx8mp_blk_ctrl_data imx8mp_hsio_blk_ctl_dev_data = { + .max_reg = 0x24, + .probe = imx8mp_hsio_blk_ctrl_probe, + .power_on = imx8mp_hsio_blk_ctrl_power_on, + .power_off = imx8mp_hsio_blk_ctrl_power_off, + .power_notifier_fn = imx8mp_hsio_power_notifier, + .domains = imx8mp_hsio_domain_data, + .num_domains = ARRAY_SIZE(imx8mp_hsio_domain_data), +}; + +#define HDMI_RTX_RESET_CTL0 0x20 +#define HDMI_RTX_CLK_CTL0 0x40 +#define HDMI_RTX_CLK_CTL1 0x50 +#define HDMI_RTX_CLK_CTL2 0x60 +#define HDMI_RTX_CLK_CTL3 0x70 +#define HDMI_RTX_CLK_CTL4 0x80 +#define HDMI_TX_CONTROL0 0x200 +#define HDMI_LCDIF_NOC_HURRY_MASK GENMASK(14, 12) + +static void imx8mp_hdmi_blk_ctrl_power_on(struct imx8mp_blk_ctrl *bc, + struct imx8mp_blk_ctrl_domain *domain) +{ + switch (domain->id) { + case IMX8MP_HDMIBLK_PD_IRQSTEER: + regmap_set_bits(bc->regmap, HDMI_RTX_CLK_CTL0, BIT(9)); + regmap_set_bits(bc->regmap, HDMI_RTX_RESET_CTL0, BIT(16)); + break; + case IMX8MP_HDMIBLK_PD_LCDIF: + regmap_set_bits(bc->regmap, HDMI_RTX_CLK_CTL0, + BIT(16) | BIT(17) | BIT(18) | + BIT(19) | BIT(20)); + regmap_set_bits(bc->regmap, HDMI_RTX_CLK_CTL1, BIT(11)); + regmap_set_bits(bc->regmap, HDMI_RTX_RESET_CTL0, + BIT(4) | BIT(5) | BIT(6)); + regmap_set_bits(bc->regmap, HDMI_TX_CONTROL0, + FIELD_PREP(HDMI_LCDIF_NOC_HURRY_MASK, 7)); + break; + case IMX8MP_HDMIBLK_PD_PAI: + regmap_set_bits(bc->regmap, HDMI_RTX_CLK_CTL1, BIT(17)); + regmap_set_bits(bc->regmap, HDMI_RTX_RESET_CTL0, BIT(18)); + break; + case IMX8MP_HDMIBLK_PD_PVI: + regmap_set_bits(bc->regmap, HDMI_RTX_CLK_CTL1, BIT(28)); + regmap_set_bits(bc->regmap, HDMI_RTX_RESET_CTL0, BIT(22)); + break; + case IMX8MP_HDMIBLK_PD_TRNG: + regmap_set_bits(bc->regmap, HDMI_RTX_CLK_CTL1, BIT(27) | BIT(30)); + regmap_set_bits(bc->regmap, HDMI_RTX_RESET_CTL0, BIT(20)); + break; + case IMX8MP_HDMIBLK_PD_HDMI_TX: + regmap_set_bits(bc->regmap, HDMI_RTX_CLK_CTL0, + BIT(2) | BIT(4) | BIT(5)); + regmap_set_bits(bc->regmap, HDMI_RTX_CLK_CTL1, + BIT(12) | BIT(13) | BIT(14) | BIT(15) | BIT(16) | + BIT(18) | BIT(19) | BIT(20) | BIT(21)); + regmap_set_bits(bc->regmap, HDMI_RTX_RESET_CTL0, + BIT(7) | BIT(10) | BIT(11)); + regmap_set_bits(bc->regmap, HDMI_TX_CONTROL0, BIT(1)); + break; + case IMX8MP_HDMIBLK_PD_HDMI_TX_PHY: + regmap_set_bits(bc->regmap, HDMI_RTX_CLK_CTL0, BIT(7)); + regmap_set_bits(bc->regmap, HDMI_RTX_CLK_CTL1, BIT(22) | BIT(24)); + regmap_set_bits(bc->regmap, HDMI_RTX_RESET_CTL0, BIT(12)); + regmap_clear_bits(bc->regmap, HDMI_TX_CONTROL0, BIT(3)); + break; + case IMX8MP_HDMIBLK_PD_HDCP: + regmap_set_bits(bc->regmap, HDMI_RTX_CLK_CTL0, BIT(11)); + break; + case IMX8MP_HDMIBLK_PD_HRV: + regmap_set_bits(bc->regmap, HDMI_RTX_CLK_CTL1, BIT(3) | BIT(4) | BIT(5)); + regmap_set_bits(bc->regmap, HDMI_RTX_RESET_CTL0, BIT(15)); + break; + default: + break; + } +} + +static void imx8mp_hdmi_blk_ctrl_power_off(struct imx8mp_blk_ctrl *bc, + struct imx8mp_blk_ctrl_domain *domain) +{ + switch (domain->id) { + case IMX8MP_HDMIBLK_PD_IRQSTEER: + regmap_clear_bits(bc->regmap, HDMI_RTX_CLK_CTL0, BIT(9)); + regmap_clear_bits(bc->regmap, HDMI_RTX_RESET_CTL0, BIT(16)); + break; + case IMX8MP_HDMIBLK_PD_LCDIF: + regmap_clear_bits(bc->regmap, HDMI_RTX_RESET_CTL0, + BIT(4) | BIT(5) | BIT(6)); + regmap_clear_bits(bc->regmap, HDMI_RTX_CLK_CTL1, BIT(11)); + regmap_clear_bits(bc->regmap, HDMI_RTX_CLK_CTL0, + BIT(16) | BIT(17) | BIT(18) | + BIT(19) | BIT(20)); + break; + case IMX8MP_HDMIBLK_PD_PAI: + regmap_clear_bits(bc->regmap, HDMI_RTX_RESET_CTL0, BIT(18)); + regmap_clear_bits(bc->regmap, HDMI_RTX_CLK_CTL1, BIT(17)); + break; + case IMX8MP_HDMIBLK_PD_PVI: + regmap_clear_bits(bc->regmap, HDMI_RTX_RESET_CTL0, BIT(22)); + regmap_clear_bits(bc->regmap, HDMI_RTX_CLK_CTL1, BIT(28)); + break; + case IMX8MP_HDMIBLK_PD_TRNG: + regmap_clear_bits(bc->regmap, HDMI_RTX_RESET_CTL0, BIT(20)); + regmap_clear_bits(bc->regmap, HDMI_RTX_CLK_CTL1, BIT(27) | BIT(30)); + break; + case IMX8MP_HDMIBLK_PD_HDMI_TX: + regmap_clear_bits(bc->regmap, HDMI_TX_CONTROL0, BIT(1)); + regmap_clear_bits(bc->regmap, HDMI_RTX_RESET_CTL0, + BIT(7) | BIT(10) | BIT(11)); + regmap_clear_bits(bc->regmap, HDMI_RTX_CLK_CTL1, + BIT(12) | BIT(13) | BIT(14) | BIT(15) | BIT(16) | + BIT(18) | BIT(19) | BIT(20) | BIT(21)); + regmap_clear_bits(bc->regmap, HDMI_RTX_CLK_CTL0, + BIT(2) | BIT(4) | BIT(5)); + break; + case IMX8MP_HDMIBLK_PD_HDMI_TX_PHY: + regmap_set_bits(bc->regmap, HDMI_TX_CONTROL0, BIT(3)); + regmap_clear_bits(bc->regmap, HDMI_RTX_RESET_CTL0, BIT(12)); + regmap_clear_bits(bc->regmap, HDMI_RTX_CLK_CTL0, BIT(7)); + regmap_clear_bits(bc->regmap, HDMI_RTX_CLK_CTL1, BIT(22) | BIT(24)); + break; + case IMX8MP_HDMIBLK_PD_HDCP: + regmap_clear_bits(bc->regmap, HDMI_RTX_CLK_CTL0, BIT(11)); + break; + case IMX8MP_HDMIBLK_PD_HRV: + regmap_clear_bits(bc->regmap, HDMI_RTX_RESET_CTL0, BIT(15)); + regmap_clear_bits(bc->regmap, HDMI_RTX_CLK_CTL1, BIT(3) | BIT(4) | BIT(5)); + break; + default: + break; + } +} + +static int imx8mp_hdmi_power_notifier(struct notifier_block *nb, + unsigned long action, void *data) +{ + struct imx8mp_blk_ctrl *bc = container_of(nb, struct imx8mp_blk_ctrl, + power_nb); + + if (action != GENPD_NOTIFY_ON) + return NOTIFY_OK; + + /* + * Contrary to other blk-ctrls the reset and clock don't clear when the + * power domain is powered down. To ensure the proper reset pulsing, + * first clear them all to asserted state, then enable the bus clocks + * and then release the ADB reset. + */ + regmap_write(bc->regmap, HDMI_RTX_RESET_CTL0, 0x0); + regmap_write(bc->regmap, HDMI_RTX_CLK_CTL0, 0x0); + regmap_write(bc->regmap, HDMI_RTX_CLK_CTL1, 0x0); + regmap_set_bits(bc->regmap, HDMI_RTX_CLK_CTL0, + BIT(0) | BIT(1) | BIT(10)); + regmap_set_bits(bc->regmap, HDMI_RTX_RESET_CTL0, BIT(0)); + + /* + * On power up we have no software backchannel to the GPC to + * wait for the ADB handshake to happen, so we just delay for a + * bit. On power down the GPC driver waits for the handshake. + */ + udelay(5); + + return NOTIFY_OK; +} + +static const struct imx8mp_blk_ctrl_domain_data imx8mp_hdmi_domain_data[] = { + [IMX8MP_HDMIBLK_PD_IRQSTEER] = { + .name = "hdmiblk-irqsteer", + .clk_names = (const char *[]){ "apb" }, + .num_clks = 1, + .gpc_name = "irqsteer", + }, + [IMX8MP_HDMIBLK_PD_LCDIF] = { + .name = "hdmiblk-lcdif", + .clk_names = (const char *[]){ "axi", "apb" }, + .num_clks = 2, + .gpc_name = "lcdif", + .path_names = (const char *[]){"lcdif-hdmi"}, + .num_paths = 1, + }, + [IMX8MP_HDMIBLK_PD_PAI] = { + .name = "hdmiblk-pai", + .clk_names = (const char *[]){ "apb" }, + .num_clks = 1, + .gpc_name = "pai", + }, + [IMX8MP_HDMIBLK_PD_PVI] = { + .name = "hdmiblk-pvi", + .clk_names = (const char *[]){ "apb" }, + .num_clks = 1, + .gpc_name = "pvi", + }, + [IMX8MP_HDMIBLK_PD_TRNG] = { + .name = "hdmiblk-trng", + .clk_names = (const char *[]){ "apb" }, + .num_clks = 1, + .gpc_name = "trng", + }, + [IMX8MP_HDMIBLK_PD_HDMI_TX] = { + .name = "hdmiblk-hdmi-tx", + .clk_names = (const char *[]){ "apb", "ref_266m" }, + .num_clks = 2, + .gpc_name = "hdmi-tx", + }, + [IMX8MP_HDMIBLK_PD_HDMI_TX_PHY] = { + .name = "hdmiblk-hdmi-tx-phy", + .clk_names = (const char *[]){ "apb", "ref_24m" }, + .num_clks = 2, + .gpc_name = "hdmi-tx-phy", + }, + [IMX8MP_HDMIBLK_PD_HRV] = { + .name = "hdmiblk-hrv", + .clk_names = (const char *[]){ "axi", "apb" }, + .num_clks = 2, + .gpc_name = "hrv", + .path_names = (const char *[]){"hrv"}, + .num_paths = 1, + }, + [IMX8MP_HDMIBLK_PD_HDCP] = { + .name = "hdmiblk-hdcp", + .clk_names = (const char *[]){ "axi", "apb" }, + .num_clks = 2, + .gpc_name = "hdcp", + .path_names = (const char *[]){"hdcp"}, + .num_paths = 1, + }, +}; + +static const struct imx8mp_blk_ctrl_data imx8mp_hdmi_blk_ctl_dev_data = { + .max_reg = 0x23c, + .power_on = imx8mp_hdmi_blk_ctrl_power_on, + .power_off = imx8mp_hdmi_blk_ctrl_power_off, + .power_notifier_fn = imx8mp_hdmi_power_notifier, + .domains = imx8mp_hdmi_domain_data, + .num_domains = ARRAY_SIZE(imx8mp_hdmi_domain_data), +}; + +static int imx8mp_blk_ctrl_power_on(struct generic_pm_domain *genpd) +{ + struct imx8mp_blk_ctrl_domain *domain = to_imx8mp_blk_ctrl_domain(genpd); + const struct imx8mp_blk_ctrl_domain_data *data = domain->data; + struct imx8mp_blk_ctrl *bc = domain->bc; + int ret; + + /* make sure bus domain is awake */ + ret = pm_runtime_resume_and_get(bc->bus_power_dev); + if (ret < 0) { + dev_err(bc->dev, "failed to power up bus domain\n"); + return ret; + } + + /* enable upstream clocks */ + ret = clk_bulk_prepare_enable(data->num_clks, domain->clks); + if (ret) { + dev_err(bc->dev, "failed to enable clocks\n"); + goto bus_put; + } + + /* domain specific blk-ctrl manipulation */ + bc->power_on(bc, domain); + + /* power up upstream GPC domain */ + ret = pm_runtime_resume_and_get(domain->power_dev); + if (ret < 0) { + dev_err(bc->dev, "failed to power up peripheral domain\n"); + goto clk_disable; + } + + ret = icc_bulk_set_bw(domain->num_paths, domain->paths); + if (ret) + dev_err(bc->dev, "failed to set icc bw\n"); + + clk_bulk_disable_unprepare(data->num_clks, domain->clks); + + return 0; + +clk_disable: + clk_bulk_disable_unprepare(data->num_clks, domain->clks); +bus_put: + pm_runtime_put(bc->bus_power_dev); + + return ret; +} + +static int imx8mp_blk_ctrl_power_off(struct generic_pm_domain *genpd) +{ + struct imx8mp_blk_ctrl_domain *domain = to_imx8mp_blk_ctrl_domain(genpd); + const struct imx8mp_blk_ctrl_domain_data *data = domain->data; + struct imx8mp_blk_ctrl *bc = domain->bc; + int ret; + + ret = clk_bulk_prepare_enable(data->num_clks, domain->clks); + if (ret) { + dev_err(bc->dev, "failed to enable clocks\n"); + return ret; + } + + /* domain specific blk-ctrl manipulation */ + bc->power_off(bc, domain); + + clk_bulk_disable_unprepare(data->num_clks, domain->clks); + + /* power down upstream GPC domain */ + pm_runtime_put(domain->power_dev); + + /* allow bus domain to suspend */ + pm_runtime_put(bc->bus_power_dev); + + return 0; +} + +static struct lock_class_key blk_ctrl_genpd_lock_class; + +static int imx8mp_blk_ctrl_probe(struct platform_device *pdev) +{ + const struct imx8mp_blk_ctrl_data *bc_data; + struct device *dev = &pdev->dev; + struct imx8mp_blk_ctrl *bc; + void __iomem *base; + int num_domains, i, ret; + + struct regmap_config regmap_config = { + .reg_bits = 32, + .val_bits = 32, + .reg_stride = 4, + }; + + bc = devm_kzalloc(dev, sizeof(*bc), GFP_KERNEL); + if (!bc) + return -ENOMEM; + + bc->dev = dev; + + bc_data = of_device_get_match_data(dev); + num_domains = bc_data->num_domains; + + base = devm_platform_ioremap_resource(pdev, 0); + if (IS_ERR(base)) + return PTR_ERR(base); + + regmap_config.max_register = bc_data->max_reg; + bc->regmap = devm_regmap_init_mmio(dev, base, ®map_config); + if (IS_ERR(bc->regmap)) + return dev_err_probe(dev, PTR_ERR(bc->regmap), + "failed to init regmap\n"); + + bc->domains = devm_kcalloc(dev, num_domains, + sizeof(struct imx8mp_blk_ctrl_domain), + GFP_KERNEL); + if (!bc->domains) + return -ENOMEM; + + bc->onecell_data.num_domains = num_domains; + bc->onecell_data.domains = + devm_kcalloc(dev, num_domains, + sizeof(struct generic_pm_domain *), GFP_KERNEL); + if (!bc->onecell_data.domains) + return -ENOMEM; + + bc->bus_power_dev = dev_pm_domain_attach_by_name(dev, "bus"); + if (IS_ERR(bc->bus_power_dev)) + return dev_err_probe(dev, PTR_ERR(bc->bus_power_dev), + "failed to attach bus power domain\n"); + + bc->power_off = bc_data->power_off; + bc->power_on = bc_data->power_on; + + for (i = 0; i < num_domains; i++) { + const struct imx8mp_blk_ctrl_domain_data *data = &bc_data->domains[i]; + struct imx8mp_blk_ctrl_domain *domain = &bc->domains[i]; + int j; + + domain->data = data; + domain->num_paths = data->num_paths; + + for (j = 0; j < data->num_clks; j++) + domain->clks[j].id = data->clk_names[j]; + + for (j = 0; j < data->num_paths; j++) { + domain->paths[j].name = data->path_names[j]; + /* Fake value for now, just let ICC could configure NoC mode/priority */ + domain->paths[j].avg_bw = 1; + domain->paths[j].peak_bw = 1; + } + + ret = devm_of_icc_bulk_get(dev, data->num_paths, domain->paths); + if (ret) { + if (ret != -EPROBE_DEFER) { + dev_warn_once(dev, "Could not get interconnect paths, NoC will stay unconfigured!\n"); + domain->num_paths = 0; + } else { + dev_err_probe(dev, ret, "failed to get noc entries\n"); + goto cleanup_pds; + } + } + + ret = devm_clk_bulk_get(dev, data->num_clks, domain->clks); + if (ret) { + dev_err_probe(dev, ret, "failed to get clock\n"); + goto cleanup_pds; + } + + domain->power_dev = + dev_pm_domain_attach_by_name(dev, data->gpc_name); + if (IS_ERR(domain->power_dev)) { + dev_err_probe(dev, PTR_ERR(domain->power_dev), + "failed to attach power domain %s\n", + data->gpc_name); + ret = PTR_ERR(domain->power_dev); + goto cleanup_pds; + } + + domain->genpd.name = data->name; + domain->genpd.power_on = imx8mp_blk_ctrl_power_on; + domain->genpd.power_off = imx8mp_blk_ctrl_power_off; + domain->bc = bc; + domain->id = i; + + ret = pm_genpd_init(&domain->genpd, NULL, true); + if (ret) { + dev_err_probe(dev, ret, "failed to init power domain\n"); + dev_pm_domain_detach(domain->power_dev, true); + goto cleanup_pds; + } + + /* + * We use runtime PM to trigger power on/off of the upstream GPC + * domain, as a strict hierarchical parent/child power domain + * setup doesn't allow us to meet the sequencing requirements. + * This means we have nested locking of genpd locks, without the + * nesting being visible at the genpd level, so we need a + * separate lock class to make lockdep aware of the fact that + * this are separate domain locks that can be nested without a + * self-deadlock. + */ + lockdep_set_class(&domain->genpd.mlock, + &blk_ctrl_genpd_lock_class); + + bc->onecell_data.domains[i] = &domain->genpd; + } + + ret = of_genpd_add_provider_onecell(dev->of_node, &bc->onecell_data); + if (ret) { + dev_err_probe(dev, ret, "failed to add power domain provider\n"); + goto cleanup_pds; + } + + bc->power_nb.notifier_call = bc_data->power_notifier_fn; + ret = dev_pm_genpd_add_notifier(bc->bus_power_dev, &bc->power_nb); + if (ret) { + dev_err_probe(dev, ret, "failed to add power notifier\n"); + goto cleanup_provider; + } + + if (bc_data->probe) { + ret = bc_data->probe(bc); + if (ret) + goto cleanup_provider; + } + + dev_set_drvdata(dev, bc); + + return 0; + +cleanup_provider: + of_genpd_del_provider(dev->of_node); +cleanup_pds: + for (i--; i >= 0; i--) { + pm_genpd_remove(&bc->domains[i].genpd); + dev_pm_domain_detach(bc->domains[i].power_dev, true); + } + + dev_pm_domain_detach(bc->bus_power_dev, true); + + return ret; +} + +static int imx8mp_blk_ctrl_remove(struct platform_device *pdev) +{ + struct imx8mp_blk_ctrl *bc = dev_get_drvdata(&pdev->dev); + int i; + + of_genpd_del_provider(pdev->dev.of_node); + + for (i = 0; bc->onecell_data.num_domains; i++) { + struct imx8mp_blk_ctrl_domain *domain = &bc->domains[i]; + + pm_genpd_remove(&domain->genpd); + dev_pm_domain_detach(domain->power_dev, true); + } + + dev_pm_genpd_remove_notifier(bc->bus_power_dev); + + dev_pm_domain_detach(bc->bus_power_dev, true); + + return 0; +} + +#ifdef CONFIG_PM_SLEEP +static int imx8mp_blk_ctrl_suspend(struct device *dev) +{ + struct imx8mp_blk_ctrl *bc = dev_get_drvdata(dev); + int ret, i; + + /* + * This may look strange, but is done so the generic PM_SLEEP code + * can power down our domains and more importantly power them up again + * after resume, without tripping over our usage of runtime PM to + * control the upstream GPC domains. Things happen in the right order + * in the system suspend/resume paths due to the device parent/child + * hierarchy. + */ + ret = pm_runtime_get_sync(bc->bus_power_dev); + if (ret < 0) { + pm_runtime_put_noidle(bc->bus_power_dev); + return ret; + } + + for (i = 0; i < bc->onecell_data.num_domains; i++) { + struct imx8mp_blk_ctrl_domain *domain = &bc->domains[i]; + + ret = pm_runtime_get_sync(domain->power_dev); + if (ret < 0) { + pm_runtime_put_noidle(domain->power_dev); + goto out_fail; + } + } + + return 0; + +out_fail: + for (i--; i >= 0; i--) + pm_runtime_put(bc->domains[i].power_dev); + + pm_runtime_put(bc->bus_power_dev); + + return ret; +} + +static int imx8mp_blk_ctrl_resume(struct device *dev) +{ + struct imx8mp_blk_ctrl *bc = dev_get_drvdata(dev); + int i; + + for (i = 0; i < bc->onecell_data.num_domains; i++) + pm_runtime_put(bc->domains[i].power_dev); + + pm_runtime_put(bc->bus_power_dev); + + return 0; +} +#endif + +static const struct dev_pm_ops imx8mp_blk_ctrl_pm_ops = { + SET_SYSTEM_SLEEP_PM_OPS(imx8mp_blk_ctrl_suspend, + imx8mp_blk_ctrl_resume) +}; + +static const struct of_device_id imx8mp_blk_ctrl_of_match[] = { + { + .compatible = "fsl,imx8mp-hsio-blk-ctrl", + .data = &imx8mp_hsio_blk_ctl_dev_data, + }, { + .compatible = "fsl,imx8mp-hdmi-blk-ctrl", + .data = &imx8mp_hdmi_blk_ctl_dev_data, + }, { + /* Sentinel */ + } +}; +MODULE_DEVICE_TABLE(of, imx8mp_blk_ctrl_of_match); + +static struct platform_driver imx8mp_blk_ctrl_driver = { + .probe = imx8mp_blk_ctrl_probe, + .remove = imx8mp_blk_ctrl_remove, + .driver = { + .name = "imx8mp-blk-ctrl", + .pm = &imx8mp_blk_ctrl_pm_ops, + .of_match_table = imx8mp_blk_ctrl_of_match, + }, +}; +module_platform_driver(imx8mp_blk_ctrl_driver); +MODULE_LICENSE("GPL"); diff --git a/drivers/pmdomain/imx/imx93-blk-ctrl.c b/drivers/pmdomain/imx/imx93-blk-ctrl.c new file mode 100644 index 000000000000..40bd90f8b977 --- /dev/null +++ b/drivers/pmdomain/imx/imx93-blk-ctrl.c @@ -0,0 +1,451 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright 2022 NXP, Peng Fan + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +#define BLK_SFT_RSTN 0x0 +#define BLK_CLK_EN 0x4 +#define BLK_MAX_CLKS 4 + +#define DOMAIN_MAX_CLKS 4 + +#define LCDIF_QOS_REG 0xC +#define LCDIF_DEFAULT_QOS_OFF 12 +#define LCDIF_CFG_QOS_OFF 8 + +#define PXP_QOS_REG 0x10 +#define PXP_R_DEFAULT_QOS_OFF 28 +#define PXP_R_CFG_QOS_OFF 24 +#define PXP_W_DEFAULT_QOS_OFF 20 +#define PXP_W_CFG_QOS_OFF 16 + +#define ISI_CACHE_REG 0x14 + +#define ISI_QOS_REG 0x1C +#define ISI_V_DEFAULT_QOS_OFF 28 +#define ISI_V_CFG_QOS_OFF 24 +#define ISI_U_DEFAULT_QOS_OFF 20 +#define ISI_U_CFG_QOS_OFF 16 +#define ISI_Y_R_DEFAULT_QOS_OFF 12 +#define ISI_Y_R_CFG_QOS_OFF 8 +#define ISI_Y_W_DEFAULT_QOS_OFF 4 +#define ISI_Y_W_CFG_QOS_OFF 0 + +#define PRIO_MASK 0xF + +#define PRIO(X) (X) + +struct imx93_blk_ctrl_domain; + +struct imx93_blk_ctrl { + struct device *dev; + struct regmap *regmap; + int num_clks; + struct clk_bulk_data clks[BLK_MAX_CLKS]; + struct imx93_blk_ctrl_domain *domains; + struct genpd_onecell_data onecell_data; +}; + +#define DOMAIN_MAX_QOS 4 + +struct imx93_blk_ctrl_qos { + u32 reg; + u32 cfg_off; + u32 default_prio; + u32 cfg_prio; +}; + +struct imx93_blk_ctrl_domain_data { + const char *name; + const char * const *clk_names; + int num_clks; + u32 rst_mask; + u32 clk_mask; + int num_qos; + struct imx93_blk_ctrl_qos qos[DOMAIN_MAX_QOS]; +}; + +struct imx93_blk_ctrl_domain { + struct generic_pm_domain genpd; + const struct imx93_blk_ctrl_domain_data *data; + struct clk_bulk_data clks[DOMAIN_MAX_CLKS]; + struct imx93_blk_ctrl *bc; +}; + +struct imx93_blk_ctrl_data { + const struct imx93_blk_ctrl_domain_data *domains; + int num_domains; + const char * const *clk_names; + int num_clks; + const struct regmap_access_table *reg_access_table; +}; + +static inline struct imx93_blk_ctrl_domain * +to_imx93_blk_ctrl_domain(struct generic_pm_domain *genpd) +{ + return container_of(genpd, struct imx93_blk_ctrl_domain, genpd); +} + +static int imx93_blk_ctrl_set_qos(struct imx93_blk_ctrl_domain *domain) +{ + const struct imx93_blk_ctrl_domain_data *data = domain->data; + struct imx93_blk_ctrl *bc = domain->bc; + const struct imx93_blk_ctrl_qos *qos; + u32 val, mask; + int i; + + for (i = 0; i < data->num_qos; i++) { + qos = &data->qos[i]; + + mask = PRIO_MASK << qos->cfg_off; + mask |= PRIO_MASK << (qos->cfg_off + 4); + val = qos->cfg_prio << qos->cfg_off; + val |= qos->default_prio << (qos->cfg_off + 4); + + regmap_write_bits(bc->regmap, qos->reg, mask, val); + + dev_dbg(bc->dev, "data->qos[i].reg 0x%x 0x%x\n", qos->reg, val); + } + + return 0; +} + +static int imx93_blk_ctrl_power_on(struct generic_pm_domain *genpd) +{ + struct imx93_blk_ctrl_domain *domain = to_imx93_blk_ctrl_domain(genpd); + const struct imx93_blk_ctrl_domain_data *data = domain->data; + struct imx93_blk_ctrl *bc = domain->bc; + int ret; + + ret = clk_bulk_prepare_enable(bc->num_clks, bc->clks); + if (ret) { + dev_err(bc->dev, "failed to enable bus clocks\n"); + return ret; + } + + ret = clk_bulk_prepare_enable(data->num_clks, domain->clks); + if (ret) { + clk_bulk_disable_unprepare(bc->num_clks, bc->clks); + dev_err(bc->dev, "failed to enable clocks\n"); + return ret; + } + + ret = pm_runtime_get_sync(bc->dev); + if (ret < 0) { + pm_runtime_put_noidle(bc->dev); + dev_err(bc->dev, "failed to power up domain\n"); + goto disable_clk; + } + + /* ungate clk */ + regmap_clear_bits(bc->regmap, BLK_CLK_EN, data->clk_mask); + + /* release reset */ + regmap_set_bits(bc->regmap, BLK_SFT_RSTN, data->rst_mask); + + dev_dbg(bc->dev, "pd_on: name: %s\n", genpd->name); + + return imx93_blk_ctrl_set_qos(domain); + +disable_clk: + clk_bulk_disable_unprepare(data->num_clks, domain->clks); + + clk_bulk_disable_unprepare(bc->num_clks, bc->clks); + + return ret; +} + +static int imx93_blk_ctrl_power_off(struct generic_pm_domain *genpd) +{ + struct imx93_blk_ctrl_domain *domain = to_imx93_blk_ctrl_domain(genpd); + const struct imx93_blk_ctrl_domain_data *data = domain->data; + struct imx93_blk_ctrl *bc = domain->bc; + + dev_dbg(bc->dev, "pd_off: name: %s\n", genpd->name); + + regmap_clear_bits(bc->regmap, BLK_SFT_RSTN, data->rst_mask); + regmap_set_bits(bc->regmap, BLK_CLK_EN, data->clk_mask); + + pm_runtime_put(bc->dev); + + clk_bulk_disable_unprepare(data->num_clks, domain->clks); + + clk_bulk_disable_unprepare(bc->num_clks, bc->clks); + + return 0; +} + +static struct lock_class_key blk_ctrl_genpd_lock_class; + +static int imx93_blk_ctrl_probe(struct platform_device *pdev) +{ + struct device *dev = &pdev->dev; + const struct imx93_blk_ctrl_data *bc_data = of_device_get_match_data(dev); + struct imx93_blk_ctrl *bc; + void __iomem *base; + int i, ret; + + struct regmap_config regmap_config = { + .reg_bits = 32, + .val_bits = 32, + .reg_stride = 4, + .rd_table = bc_data->reg_access_table, + .wr_table = bc_data->reg_access_table, + .max_register = SZ_4K, + }; + + bc = devm_kzalloc(dev, sizeof(*bc), GFP_KERNEL); + if (!bc) + return -ENOMEM; + + bc->dev = dev; + + base = devm_platform_ioremap_resource(pdev, 0); + if (IS_ERR(base)) + return PTR_ERR(base); + + bc->regmap = devm_regmap_init_mmio(dev, base, ®map_config); + if (IS_ERR(bc->regmap)) + return dev_err_probe(dev, PTR_ERR(bc->regmap), + "failed to init regmap\n"); + + bc->domains = devm_kcalloc(dev, bc_data->num_domains, + sizeof(struct imx93_blk_ctrl_domain), + GFP_KERNEL); + if (!bc->domains) + return -ENOMEM; + + bc->onecell_data.num_domains = bc_data->num_domains; + bc->onecell_data.domains = + devm_kcalloc(dev, bc_data->num_domains, + sizeof(struct generic_pm_domain *), GFP_KERNEL); + if (!bc->onecell_data.domains) + return -ENOMEM; + + for (i = 0; i < bc_data->num_clks; i++) + bc->clks[i].id = bc_data->clk_names[i]; + bc->num_clks = bc_data->num_clks; + + ret = devm_clk_bulk_get(dev, bc->num_clks, bc->clks); + if (ret) { + dev_err_probe(dev, ret, "failed to get bus clock\n"); + return ret; + } + + for (i = 0; i < bc_data->num_domains; i++) { + const struct imx93_blk_ctrl_domain_data *data = &bc_data->domains[i]; + struct imx93_blk_ctrl_domain *domain = &bc->domains[i]; + int j; + + domain->data = data; + + for (j = 0; j < data->num_clks; j++) + domain->clks[j].id = data->clk_names[j]; + + ret = devm_clk_bulk_get(dev, data->num_clks, domain->clks); + if (ret) { + dev_err_probe(dev, ret, "failed to get clock\n"); + goto cleanup_pds; + } + + domain->genpd.name = data->name; + domain->genpd.power_on = imx93_blk_ctrl_power_on; + domain->genpd.power_off = imx93_blk_ctrl_power_off; + domain->bc = bc; + + ret = pm_genpd_init(&domain->genpd, NULL, true); + if (ret) { + dev_err_probe(dev, ret, "failed to init power domain\n"); + goto cleanup_pds; + } + + /* + * We use runtime PM to trigger power on/off of the upstream GPC + * domain, as a strict hierarchical parent/child power domain + * setup doesn't allow us to meet the sequencing requirements. + * This means we have nested locking of genpd locks, without the + * nesting being visible at the genpd level, so we need a + * separate lock class to make lockdep aware of the fact that + * this are separate domain locks that can be nested without a + * self-deadlock. + */ + lockdep_set_class(&domain->genpd.mlock, + &blk_ctrl_genpd_lock_class); + + bc->onecell_data.domains[i] = &domain->genpd; + } + + pm_runtime_enable(dev); + + ret = of_genpd_add_provider_onecell(dev->of_node, &bc->onecell_data); + if (ret) { + dev_err_probe(dev, ret, "failed to add power domain provider\n"); + goto cleanup_pds; + } + + dev_set_drvdata(dev, bc); + + return 0; + +cleanup_pds: + for (i--; i >= 0; i--) + pm_genpd_remove(&bc->domains[i].genpd); + + return ret; +} + +static int imx93_blk_ctrl_remove(struct platform_device *pdev) +{ + struct imx93_blk_ctrl *bc = dev_get_drvdata(&pdev->dev); + int i; + + of_genpd_del_provider(pdev->dev.of_node); + + for (i = 0; bc->onecell_data.num_domains; i++) { + struct imx93_blk_ctrl_domain *domain = &bc->domains[i]; + + pm_genpd_remove(&domain->genpd); + } + + return 0; +} + +static const struct imx93_blk_ctrl_domain_data imx93_media_blk_ctl_domain_data[] = { + [IMX93_MEDIABLK_PD_MIPI_DSI] = { + .name = "mediablk-mipi-dsi", + .clk_names = (const char *[]){ "dsi" }, + .num_clks = 1, + .rst_mask = BIT(11) | BIT(12), + .clk_mask = BIT(11) | BIT(12), + }, + [IMX93_MEDIABLK_PD_MIPI_CSI] = { + .name = "mediablk-mipi-csi", + .clk_names = (const char *[]){ "cam", "csi" }, + .num_clks = 2, + .rst_mask = BIT(9) | BIT(10), + .clk_mask = BIT(9) | BIT(10), + }, + [IMX93_MEDIABLK_PD_PXP] = { + .name = "mediablk-pxp", + .clk_names = (const char *[]){ "pxp" }, + .num_clks = 1, + .rst_mask = BIT(7) | BIT(8), + .clk_mask = BIT(7) | BIT(8), + .num_qos = 2, + .qos = { + { + .reg = PXP_QOS_REG, + .cfg_off = PXP_R_CFG_QOS_OFF, + .default_prio = PRIO(3), + .cfg_prio = PRIO(6), + }, { + .reg = PXP_QOS_REG, + .cfg_off = PXP_W_CFG_QOS_OFF, + .default_prio = PRIO(3), + .cfg_prio = PRIO(6), + } + } + }, + [IMX93_MEDIABLK_PD_LCDIF] = { + .name = "mediablk-lcdif", + .clk_names = (const char *[]){ "disp", "lcdif" }, + .num_clks = 2, + .rst_mask = BIT(4) | BIT(5) | BIT(6), + .clk_mask = BIT(4) | BIT(5) | BIT(6), + .num_qos = 1, + .qos = { + { + .reg = LCDIF_QOS_REG, + .cfg_off = LCDIF_CFG_QOS_OFF, + .default_prio = PRIO(3), + .cfg_prio = PRIO(7), + } + } + }, + [IMX93_MEDIABLK_PD_ISI] = { + .name = "mediablk-isi", + .clk_names = (const char *[]){ "isi" }, + .num_clks = 1, + .rst_mask = BIT(2) | BIT(3), + .clk_mask = BIT(2) | BIT(3), + .num_qos = 4, + .qos = { + { + .reg = ISI_QOS_REG, + .cfg_off = ISI_Y_W_CFG_QOS_OFF, + .default_prio = PRIO(3), + .cfg_prio = PRIO(7), + }, { + .reg = ISI_QOS_REG, + .cfg_off = ISI_Y_R_CFG_QOS_OFF, + .default_prio = PRIO(3), + .cfg_prio = PRIO(7), + }, { + .reg = ISI_QOS_REG, + .cfg_off = ISI_U_CFG_QOS_OFF, + .default_prio = PRIO(3), + .cfg_prio = PRIO(7), + }, { + .reg = ISI_QOS_REG, + .cfg_off = ISI_V_CFG_QOS_OFF, + .default_prio = PRIO(3), + .cfg_prio = PRIO(7), + } + } + }, +}; + +static const struct regmap_range imx93_media_blk_ctl_yes_ranges[] = { + regmap_reg_range(BLK_SFT_RSTN, BLK_CLK_EN), + regmap_reg_range(LCDIF_QOS_REG, ISI_CACHE_REG), + regmap_reg_range(ISI_QOS_REG, ISI_QOS_REG), +}; + +static const struct regmap_access_table imx93_media_blk_ctl_access_table = { + .yes_ranges = imx93_media_blk_ctl_yes_ranges, + .n_yes_ranges = ARRAY_SIZE(imx93_media_blk_ctl_yes_ranges), +}; + +static const struct imx93_blk_ctrl_data imx93_media_blk_ctl_dev_data = { + .domains = imx93_media_blk_ctl_domain_data, + .num_domains = ARRAY_SIZE(imx93_media_blk_ctl_domain_data), + .clk_names = (const char *[]){ "axi", "apb", "nic", }, + .num_clks = 3, + .reg_access_table = &imx93_media_blk_ctl_access_table, +}; + +static const struct of_device_id imx93_blk_ctrl_of_match[] = { + { + .compatible = "fsl,imx93-media-blk-ctrl", + .data = &imx93_media_blk_ctl_dev_data + }, { + /* Sentinel */ + } +}; +MODULE_DEVICE_TABLE(of, imx93_blk_ctrl_of_match); + +static struct platform_driver imx93_blk_ctrl_driver = { + .probe = imx93_blk_ctrl_probe, + .remove = imx93_blk_ctrl_remove, + .driver = { + .name = "imx93-blk-ctrl", + .of_match_table = imx93_blk_ctrl_of_match, + }, +}; +module_platform_driver(imx93_blk_ctrl_driver); + +MODULE_AUTHOR("Peng Fan "); +MODULE_DESCRIPTION("i.MX93 BLK CTRL driver"); +MODULE_LICENSE("GPL"); diff --git a/drivers/pmdomain/imx/imx93-pd.c b/drivers/pmdomain/imx/imx93-pd.c new file mode 100644 index 000000000000..b9e60d136875 --- /dev/null +++ b/drivers/pmdomain/imx/imx93-pd.c @@ -0,0 +1,176 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright 2022 NXP + */ + +#include +#include +#include +#include +#include +#include +#include + +#define MIX_SLICE_SW_CTRL_OFF 0x20 +#define SLICE_SW_CTRL_PSW_CTRL_OFF_MASK BIT(4) +#define SLICE_SW_CTRL_PDN_SOFT_MASK BIT(31) + +#define MIX_FUNC_STAT_OFF 0xB4 + +#define FUNC_STAT_PSW_STAT_MASK BIT(0) +#define FUNC_STAT_RST_STAT_MASK BIT(2) +#define FUNC_STAT_ISO_STAT_MASK BIT(4) + +struct imx93_power_domain { + struct generic_pm_domain genpd; + struct device *dev; + void __iomem *addr; + struct clk_bulk_data *clks; + int num_clks; + bool init_off; +}; + +#define to_imx93_pd(_genpd) container_of(_genpd, struct imx93_power_domain, genpd) + +static int imx93_pd_on(struct generic_pm_domain *genpd) +{ + struct imx93_power_domain *domain = to_imx93_pd(genpd); + void __iomem *addr = domain->addr; + u32 val; + int ret; + + ret = clk_bulk_prepare_enable(domain->num_clks, domain->clks); + if (ret) { + dev_err(domain->dev, "failed to enable clocks for domain: %s\n", genpd->name); + return ret; + } + + val = readl(addr + MIX_SLICE_SW_CTRL_OFF); + val &= ~SLICE_SW_CTRL_PDN_SOFT_MASK; + writel(val, addr + MIX_SLICE_SW_CTRL_OFF); + + ret = readl_poll_timeout(addr + MIX_FUNC_STAT_OFF, val, + !(val & FUNC_STAT_ISO_STAT_MASK), 1, 10000); + if (ret) { + dev_err(domain->dev, "pd_on timeout: name: %s, stat: %x\n", genpd->name, val); + return ret; + } + + return 0; +} + +static int imx93_pd_off(struct generic_pm_domain *genpd) +{ + struct imx93_power_domain *domain = to_imx93_pd(genpd); + void __iomem *addr = domain->addr; + int ret; + u32 val; + + /* Power off MIX */ + val = readl(addr + MIX_SLICE_SW_CTRL_OFF); + val |= SLICE_SW_CTRL_PDN_SOFT_MASK; + writel(val, addr + MIX_SLICE_SW_CTRL_OFF); + + ret = readl_poll_timeout(addr + MIX_FUNC_STAT_OFF, val, + val & FUNC_STAT_PSW_STAT_MASK, 1, 1000); + if (ret) { + dev_err(domain->dev, "pd_off timeout: name: %s, stat: %x\n", genpd->name, val); + return ret; + } + + clk_bulk_disable_unprepare(domain->num_clks, domain->clks); + + return 0; +}; + +static int imx93_pd_remove(struct platform_device *pdev) +{ + struct imx93_power_domain *domain = platform_get_drvdata(pdev); + struct device *dev = &pdev->dev; + struct device_node *np = dev->of_node; + + if (!domain->init_off) + clk_bulk_disable_unprepare(domain->num_clks, domain->clks); + + of_genpd_del_provider(np); + pm_genpd_remove(&domain->genpd); + + return 0; +} + +static int imx93_pd_probe(struct platform_device *pdev) +{ + struct device *dev = &pdev->dev; + struct device_node *np = dev->of_node; + struct imx93_power_domain *domain; + int ret; + + domain = devm_kzalloc(dev, sizeof(*domain), GFP_KERNEL); + if (!domain) + return -ENOMEM; + + domain->addr = devm_platform_ioremap_resource(pdev, 0); + if (IS_ERR(domain->addr)) + return PTR_ERR(domain->addr); + + domain->num_clks = devm_clk_bulk_get_all(dev, &domain->clks); + if (domain->num_clks < 0) + return dev_err_probe(dev, domain->num_clks, "Failed to get domain's clocks\n"); + + domain->genpd.name = dev_name(dev); + domain->genpd.power_off = imx93_pd_off; + domain->genpd.power_on = imx93_pd_on; + domain->dev = dev; + + domain->init_off = readl(domain->addr + MIX_FUNC_STAT_OFF) & FUNC_STAT_ISO_STAT_MASK; + /* Just to sync the status of hardware */ + if (!domain->init_off) { + ret = clk_bulk_prepare_enable(domain->num_clks, domain->clks); + if (ret) { + dev_err(domain->dev, "failed to enable clocks for domain: %s\n", + domain->genpd.name); + return ret; + } + } + + ret = pm_genpd_init(&domain->genpd, NULL, domain->init_off); + if (ret) + goto err_clk_unprepare; + + platform_set_drvdata(pdev, domain); + + ret = of_genpd_add_provider_simple(np, &domain->genpd); + if (ret) + goto err_genpd_remove; + + return 0; + +err_genpd_remove: + pm_genpd_remove(&domain->genpd); + +err_clk_unprepare: + if (!domain->init_off) + clk_bulk_disable_unprepare(domain->num_clks, domain->clks); + + return ret; +} + +static const struct of_device_id imx93_pd_ids[] = { + { .compatible = "fsl,imx93-src-slice" }, + { } +}; +MODULE_DEVICE_TABLE(of, imx93_pd_ids); + +static struct platform_driver imx93_power_domain_driver = { + .driver = { + .name = "imx93_power_domain", + .of_match_table = imx93_pd_ids, + }, + .probe = imx93_pd_probe, + .remove = imx93_pd_remove, +}; +module_platform_driver(imx93_power_domain_driver); + +MODULE_AUTHOR("Peng Fan "); +MODULE_DESCRIPTION("NXP i.MX93 power domain driver"); +MODULE_LICENSE("GPL"); diff --git a/drivers/pmdomain/imx/scu-pd.c b/drivers/pmdomain/imx/scu-pd.c new file mode 100644 index 000000000000..891c1d925a9d --- /dev/null +++ b/drivers/pmdomain/imx/scu-pd.c @@ -0,0 +1,551 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* + * Copyright (C) 2016 Freescale Semiconductor, Inc. + * Copyright 2017-2018 NXP + * Dong Aisheng + * + * Implementation of the SCU based Power Domains + * + * NOTE: a better implementation suggested by Ulf Hansson is using a + * single global power domain and implement the ->attach|detach_dev() + * callback for the genpd and use the regular of_genpd_add_provider_simple(). + * From within the ->attach_dev(), we could get the OF node for + * the device that is being attached and then parse the power-domain + * cell containing the "resource id" and store that in the per device + * struct generic_pm_domain_data (we have void pointer there for + * storing these kind of things). + * + * Additionally, we need to implement the ->stop() and ->start() + * callbacks of genpd, which is where you "power on/off" devices, + * rather than using the above ->power_on|off() callbacks. + * + * However, there're two known issues: + * 1. The ->attach_dev() of power domain infrastructure still does + * not support multi domains case as the struct device *dev passed + * in is a virtual PD device, it does not help for parsing the real + * device resource id from device tree, so it's unware of which + * real sub power domain of device should be attached. + * + * The framework needs some proper extension to support multi power + * domain cases. + * + * Update: Genpd assigns the ->of_node for the virtual device before it + * invokes ->attach_dev() callback, hence parsing for device resources via + * DT should work fine. + * + * 2. It also breaks most of current drivers as the driver probe sequence + * behavior changed if removing ->power_on|off() callback and use + * ->start() and ->stop() instead. genpd_dev_pm_attach will only power + * up the domain and attach device, but will not call .start() which + * relies on device runtime pm. That means the device power is still + * not up before running driver probe function. For SCU enabled + * platforms, all device drivers accessing registers/clock without power + * domain enabled will trigger a HW access error. That means we need fix + * most drivers probe sequence with proper runtime pm. + * + * Update: Runtime PM support isn't necessary. Instead, this can easily be + * fixed in drivers by adding a call to dev_pm_domain_start() during probe. + * + * In summary, the second part needs to be addressed via minor updates to the + * relevant drivers, before the "single global power domain" model can be used. + * + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +/* SCU Power Mode Protocol definition */ +struct imx_sc_msg_req_set_resource_power_mode { + struct imx_sc_rpc_msg hdr; + u16 resource; + u8 mode; +} __packed __aligned(4); + +struct req_get_resource_mode { + u16 resource; +}; + +struct resp_get_resource_mode { + u8 mode; +}; + +struct imx_sc_msg_req_get_resource_power_mode { + struct imx_sc_rpc_msg hdr; + union { + struct req_get_resource_mode req; + struct resp_get_resource_mode resp; + } data; +} __packed __aligned(4); + +#define IMX_SCU_PD_NAME_SIZE 20 +struct imx_sc_pm_domain { + struct generic_pm_domain pd; + char name[IMX_SCU_PD_NAME_SIZE]; + u32 rsrc; +}; + +struct imx_sc_pd_range { + char *name; + u32 rsrc; + u8 num; + + /* add domain index */ + bool postfix; + u8 start_from; +}; + +struct imx_sc_pd_soc { + const struct imx_sc_pd_range *pd_ranges; + u8 num_ranges; +}; + +static int imx_con_rsrc; + +/* Align with the IMX_SC_PM_PW_MODE_[OFF,STBY,LP,ON] macros */ +static const char * const imx_sc_pm_mode[] = { + "IMX_SC_PM_PW_MODE_OFF", + "IMX_SC_PM_PW_MODE_STBY", + "IMX_SC_PM_PW_MODE_LP", + "IMX_SC_PM_PW_MODE_ON" +}; + +static const struct imx_sc_pd_range imx8qxp_scu_pd_ranges[] = { + /* LSIO SS */ + { "pwm", IMX_SC_R_PWM_0, 8, true, 0 }, + { "gpio", IMX_SC_R_GPIO_0, 8, true, 0 }, + { "gpt", IMX_SC_R_GPT_0, 5, true, 0 }, + { "kpp", IMX_SC_R_KPP, 1, false, 0 }, + { "fspi", IMX_SC_R_FSPI_0, 2, true, 0 }, + { "mu_a", IMX_SC_R_MU_0A, 14, true, 0 }, + { "mu_b", IMX_SC_R_MU_5B, 9, true, 5 }, + + /* CONN SS */ + { "usb", IMX_SC_R_USB_0, 2, true, 0 }, + { "usb0phy", IMX_SC_R_USB_0_PHY, 1, false, 0 }, + { "usb1phy", IMX_SC_R_USB_1_PHY, 1, false, 0}, + { "usb2", IMX_SC_R_USB_2, 1, false, 0 }, + { "usb2phy", IMX_SC_R_USB_2_PHY, 1, false, 0 }, + { "sdhc", IMX_SC_R_SDHC_0, 3, true, 0 }, + { "enet", IMX_SC_R_ENET_0, 2, true, 0 }, + { "nand", IMX_SC_R_NAND, 1, false, 0 }, + { "mlb", IMX_SC_R_MLB_0, 1, true, 0 }, + + /* AUDIO SS */ + { "audio-pll0", IMX_SC_R_AUDIO_PLL_0, 1, false, 0 }, + { "audio-pll1", IMX_SC_R_AUDIO_PLL_1, 1, false, 0 }, + { "audio-clk-0", IMX_SC_R_AUDIO_CLK_0, 1, false, 0 }, + { "audio-clk-1", IMX_SC_R_AUDIO_CLK_1, 1, false, 0 }, + { "mclk-out-0", IMX_SC_R_MCLK_OUT_0, 1, false, 0 }, + { "mclk-out-1", IMX_SC_R_MCLK_OUT_1, 1, false, 0 }, + { "dma0-ch", IMX_SC_R_DMA_0_CH0, 32, true, 0 }, + { "dma1-ch", IMX_SC_R_DMA_1_CH0, 16, true, 0 }, + { "dma2-ch-0", IMX_SC_R_DMA_2_CH0, 5, true, 0 }, + { "dma2-ch-1", IMX_SC_R_DMA_2_CH5, 27, true, 0 }, + { "dma3-ch", IMX_SC_R_DMA_3_CH0, 32, true, 0 }, + { "asrc0", IMX_SC_R_ASRC_0, 1, false, 0 }, + { "asrc1", IMX_SC_R_ASRC_1, 1, false, 0 }, + { "esai0", IMX_SC_R_ESAI_0, 1, false, 0 }, + { "esai1", IMX_SC_R_ESAI_1, 1, false, 0 }, + { "spdif0", IMX_SC_R_SPDIF_0, 1, false, 0 }, + { "spdif1", IMX_SC_R_SPDIF_1, 1, false, 0 }, + { "sai", IMX_SC_R_SAI_0, 3, true, 0 }, + { "sai3", IMX_SC_R_SAI_3, 1, false, 0 }, + { "sai4", IMX_SC_R_SAI_4, 1, false, 0 }, + { "sai5", IMX_SC_R_SAI_5, 1, false, 0 }, + { "sai6", IMX_SC_R_SAI_6, 1, false, 0 }, + { "sai7", IMX_SC_R_SAI_7, 1, false, 0 }, + { "amix", IMX_SC_R_AMIX, 1, false, 0 }, + { "mqs0", IMX_SC_R_MQS_0, 1, false, 0 }, + { "dsp", IMX_SC_R_DSP, 1, false, 0 }, + { "dsp-ram", IMX_SC_R_DSP_RAM, 1, false, 0 }, + + /* DMA SS */ + { "can", IMX_SC_R_CAN_0, 3, true, 0 }, + { "ftm", IMX_SC_R_FTM_0, 2, true, 0 }, + { "lpi2c", IMX_SC_R_I2C_0, 5, true, 0 }, + { "adc", IMX_SC_R_ADC_0, 2, true, 0 }, + { "lcd", IMX_SC_R_LCD_0, 1, true, 0 }, + { "lcd-pll", IMX_SC_R_ELCDIF_PLL, 1, true, 0 }, + { "lcd0-pwm", IMX_SC_R_LCD_0_PWM_0, 1, true, 0 }, + { "lpuart", IMX_SC_R_UART_0, 5, true, 0 }, + { "sim", IMX_SC_R_EMVSIM_0, 2, true, 0 }, + { "lpspi", IMX_SC_R_SPI_0, 4, true, 0 }, + { "irqstr_dsp", IMX_SC_R_IRQSTR_DSP, 1, false, 0 }, + + /* VPU SS */ + { "vpu", IMX_SC_R_VPU, 1, false, 0 }, + { "vpu-pid", IMX_SC_R_VPU_PID0, 8, true, 0 }, + { "vpu-dec0", IMX_SC_R_VPU_DEC_0, 1, false, 0 }, + { "vpu-enc0", IMX_SC_R_VPU_ENC_0, 1, false, 0 }, + { "vpu-enc1", IMX_SC_R_VPU_ENC_1, 1, false, 0 }, + { "vpu-mu0", IMX_SC_R_VPU_MU_0, 1, false, 0 }, + { "vpu-mu1", IMX_SC_R_VPU_MU_1, 1, false, 0 }, + { "vpu-mu2", IMX_SC_R_VPU_MU_2, 1, false, 0 }, + + /* GPU SS */ + { "gpu0-pid", IMX_SC_R_GPU_0_PID0, 4, true, 0 }, + { "gpu1-pid", IMX_SC_R_GPU_1_PID0, 4, true, 0 }, + + + /* HSIO SS */ + { "pcie-a", IMX_SC_R_PCIE_A, 1, false, 0 }, + { "serdes-0", IMX_SC_R_SERDES_0, 1, false, 0 }, + { "pcie-b", IMX_SC_R_PCIE_B, 1, false, 0 }, + { "serdes-1", IMX_SC_R_SERDES_1, 1, false, 0 }, + { "sata-0", IMX_SC_R_SATA_0, 1, false, 0 }, + { "hsio-gpio", IMX_SC_R_HSIO_GPIO, 1, false, 0 }, + + /* MIPI SS */ + { "mipi0", IMX_SC_R_MIPI_0, 1, false, 0 }, + { "mipi0-pwm0", IMX_SC_R_MIPI_0_PWM_0, 1, false, 0 }, + { "mipi0-i2c", IMX_SC_R_MIPI_0_I2C_0, 2, true, 0 }, + + { "mipi1", IMX_SC_R_MIPI_1, 1, false, 0 }, + { "mipi1-pwm0", IMX_SC_R_MIPI_1_PWM_0, 1, false, 0 }, + { "mipi1-i2c", IMX_SC_R_MIPI_1_I2C_0, 2, true, 0 }, + + /* LVDS SS */ + { "lvds0", IMX_SC_R_LVDS_0, 1, false, 0 }, + { "lvds0-pwm", IMX_SC_R_LVDS_0_PWM_0, 1, false, 0 }, + { "lvds0-lpi2c", IMX_SC_R_LVDS_0_I2C_0, 2, true, 0 }, + { "lvds1", IMX_SC_R_LVDS_1, 1, false, 0 }, + { "lvds1-pwm", IMX_SC_R_LVDS_1_PWM_0, 1, false, 0 }, + { "lvds1-lpi2c", IMX_SC_R_LVDS_1_I2C_0, 2, true, 0 }, + + { "mipi1", IMX_SC_R_MIPI_1, 1, 0 }, + { "mipi1-pwm0", IMX_SC_R_MIPI_1_PWM_0, 1, 0 }, + { "mipi1-i2c", IMX_SC_R_MIPI_1_I2C_0, 2, 1 }, + { "lvds1", IMX_SC_R_LVDS_1, 1, 0 }, + + /* DC SS */ + { "dc0", IMX_SC_R_DC_0, 1, false, 0 }, + { "dc0-pll", IMX_SC_R_DC_0_PLL_0, 2, true, 0 }, + { "dc0-video", IMX_SC_R_DC_0_VIDEO0, 2, true, 0 }, + + { "dc1", IMX_SC_R_DC_1, 1, false, 0 }, + { "dc1-pll", IMX_SC_R_DC_1_PLL_0, 2, true, 0 }, + { "dc1-video", IMX_SC_R_DC_1_VIDEO0, 2, true, 0 }, + + /* CM40 SS */ + { "cm40-i2c", IMX_SC_R_M4_0_I2C, 1, false, 0 }, + { "cm40-intmux", IMX_SC_R_M4_0_INTMUX, 1, false, 0 }, + { "cm40-pid", IMX_SC_R_M4_0_PID0, 5, true, 0}, + { "cm40-mu-a1", IMX_SC_R_M4_0_MU_1A, 1, false, 0}, + { "cm40-lpuart", IMX_SC_R_M4_0_UART, 1, false, 0}, + + /* CM41 SS */ + { "cm41-i2c", IMX_SC_R_M4_1_I2C, 1, false, 0 }, + { "cm41-intmux", IMX_SC_R_M4_1_INTMUX, 1, false, 0 }, + { "cm41-pid", IMX_SC_R_M4_1_PID0, 5, true, 0}, + { "cm41-mu-a1", IMX_SC_R_M4_1_MU_1A, 1, false, 0}, + { "cm41-lpuart", IMX_SC_R_M4_1_UART, 1, false, 0}, + + /* CM41 SS */ + { "cm41_i2c", IMX_SC_R_M4_1_I2C, 1, false, 0 }, + { "cm41_intmux", IMX_SC_R_M4_1_INTMUX, 1, false, 0 }, + + /* DB SS */ + { "perf", IMX_SC_R_PERF, 1, false, 0}, + + /* IMAGE SS */ + { "img-jpegdec-mp", IMX_SC_R_MJPEG_DEC_MP, 1, false, 0 }, + { "img-jpegdec-s0", IMX_SC_R_MJPEG_DEC_S0, 4, true, 0 }, + { "img-jpegenc-mp", IMX_SC_R_MJPEG_ENC_MP, 1, false, 0 }, + { "img-jpegenc-s0", IMX_SC_R_MJPEG_ENC_S0, 4, true, 0 }, + + /* SECO SS */ + { "seco_mu", IMX_SC_R_SECO_MU_2, 3, true, 2}, + + /* V2X SS */ + { "v2x_mu", IMX_SC_R_V2X_MU_0, 2, true, 0}, + { "v2x_mu", IMX_SC_R_V2X_MU_2, 1, true, 2}, + { "v2x_mu", IMX_SC_R_V2X_MU_3, 2, true, 3}, + { "img-pdma", IMX_SC_R_ISI_CH0, 8, true, 0 }, + { "img-csi0", IMX_SC_R_CSI_0, 1, false, 0 }, + { "img-csi0-i2c0", IMX_SC_R_CSI_0_I2C_0, 1, false, 0 }, + { "img-csi0-pwm0", IMX_SC_R_CSI_0_PWM_0, 1, false, 0 }, + { "img-csi1", IMX_SC_R_CSI_1, 1, false, 0 }, + { "img-csi1-i2c0", IMX_SC_R_CSI_1_I2C_0, 1, false, 0 }, + { "img-csi1-pwm0", IMX_SC_R_CSI_1_PWM_0, 1, false, 0 }, + { "img-parallel", IMX_SC_R_PI_0, 1, false, 0 }, + { "img-parallel-i2c0", IMX_SC_R_PI_0_I2C_0, 1, false, 0 }, + { "img-parallel-pwm0", IMX_SC_R_PI_0_PWM_0, 2, true, 0 }, + { "img-parallel-pll", IMX_SC_R_PI_0_PLL, 1, false, 0 }, + + /* HDMI TX SS */ + { "hdmi-tx", IMX_SC_R_HDMI, 1, false, 0}, + { "hdmi-tx-i2s", IMX_SC_R_HDMI_I2S, 1, false, 0}, + { "hdmi-tx-i2c0", IMX_SC_R_HDMI_I2C_0, 1, false, 0}, + { "hdmi-tx-pll0", IMX_SC_R_HDMI_PLL_0, 1, false, 0}, + { "hdmi-tx-pll1", IMX_SC_R_HDMI_PLL_1, 1, false, 0}, + + /* HDMI RX SS */ + { "hdmi-rx", IMX_SC_R_HDMI_RX, 1, false, 0}, + { "hdmi-rx-pwm", IMX_SC_R_HDMI_RX_PWM_0, 1, false, 0}, + { "hdmi-rx-i2c0", IMX_SC_R_HDMI_RX_I2C_0, 1, false, 0}, + { "hdmi-rx-bypass", IMX_SC_R_HDMI_RX_BYPASS, 1, false, 0}, + + /* SECURITY SS */ + { "sec-jr", IMX_SC_R_CAAM_JR2, 2, true, 2}, + + /* BOARD SS */ + { "board", IMX_SC_R_BOARD_R0, 8, true, 0}, +}; + +static const struct imx_sc_pd_soc imx8qxp_scu_pd = { + .pd_ranges = imx8qxp_scu_pd_ranges, + .num_ranges = ARRAY_SIZE(imx8qxp_scu_pd_ranges), +}; + +static struct imx_sc_ipc *pm_ipc_handle; + +static inline struct imx_sc_pm_domain * +to_imx_sc_pd(struct generic_pm_domain *genpd) +{ + return container_of(genpd, struct imx_sc_pm_domain, pd); +} + +static void imx_sc_pd_get_console_rsrc(void) +{ + struct of_phandle_args specs; + int ret; + + if (!of_stdout) + return; + + ret = of_parse_phandle_with_args(of_stdout, "power-domains", + "#power-domain-cells", + 0, &specs); + if (ret) + return; + + imx_con_rsrc = specs.args[0]; +} + +static int imx_sc_get_pd_power(struct device *dev, u32 rsrc) +{ + struct imx_sc_msg_req_get_resource_power_mode msg; + struct imx_sc_rpc_msg *hdr = &msg.hdr; + int ret; + + hdr->ver = IMX_SC_RPC_VERSION; + hdr->svc = IMX_SC_RPC_SVC_PM; + hdr->func = IMX_SC_PM_FUNC_GET_RESOURCE_POWER_MODE; + hdr->size = 2; + + msg.data.req.resource = rsrc; + + ret = imx_scu_call_rpc(pm_ipc_handle, &msg, true); + if (ret) + dev_err(dev, "failed to get power resource %d mode, ret %d\n", + rsrc, ret); + + return msg.data.resp.mode; +} + +static int imx_sc_pd_power(struct generic_pm_domain *domain, bool power_on) +{ + struct imx_sc_msg_req_set_resource_power_mode msg; + struct imx_sc_rpc_msg *hdr = &msg.hdr; + struct imx_sc_pm_domain *pd; + int ret; + + pd = to_imx_sc_pd(domain); + + hdr->ver = IMX_SC_RPC_VERSION; + hdr->svc = IMX_SC_RPC_SVC_PM; + hdr->func = IMX_SC_PM_FUNC_SET_RESOURCE_POWER_MODE; + hdr->size = 2; + + msg.resource = pd->rsrc; + msg.mode = power_on ? IMX_SC_PM_PW_MODE_ON : IMX_SC_PM_PW_MODE_LP; + + /* keep uart console power on for no_console_suspend */ + if (imx_con_rsrc == pd->rsrc && !console_suspend_enabled && !power_on) + return -EBUSY; + + ret = imx_scu_call_rpc(pm_ipc_handle, &msg, true); + if (ret) + dev_err(&domain->dev, "failed to power %s resource %d ret %d\n", + power_on ? "up" : "off", pd->rsrc, ret); + + return ret; +} + +static int imx_sc_pd_power_on(struct generic_pm_domain *domain) +{ + return imx_sc_pd_power(domain, true); +} + +static int imx_sc_pd_power_off(struct generic_pm_domain *domain) +{ + return imx_sc_pd_power(domain, false); +} + +static struct generic_pm_domain *imx_scu_pd_xlate(struct of_phandle_args *spec, + void *data) +{ + struct generic_pm_domain *domain = ERR_PTR(-ENOENT); + struct genpd_onecell_data *pd_data = data; + unsigned int i; + + for (i = 0; i < pd_data->num_domains; i++) { + struct imx_sc_pm_domain *sc_pd; + + sc_pd = to_imx_sc_pd(pd_data->domains[i]); + if (sc_pd->rsrc == spec->args[0]) { + domain = &sc_pd->pd; + break; + } + } + + return domain; +} + +static struct imx_sc_pm_domain * +imx_scu_add_pm_domain(struct device *dev, int idx, + const struct imx_sc_pd_range *pd_ranges) +{ + struct imx_sc_pm_domain *sc_pd; + bool is_off; + int mode, ret; + + if (!imx_sc_rm_is_resource_owned(pm_ipc_handle, pd_ranges->rsrc + idx)) + return NULL; + + sc_pd = devm_kzalloc(dev, sizeof(*sc_pd), GFP_KERNEL); + if (!sc_pd) + return ERR_PTR(-ENOMEM); + + sc_pd->rsrc = pd_ranges->rsrc + idx; + sc_pd->pd.power_off = imx_sc_pd_power_off; + sc_pd->pd.power_on = imx_sc_pd_power_on; + + if (pd_ranges->postfix) + snprintf(sc_pd->name, sizeof(sc_pd->name), + "%s%i", pd_ranges->name, pd_ranges->start_from + idx); + else + snprintf(sc_pd->name, sizeof(sc_pd->name), + "%s", pd_ranges->name); + + sc_pd->pd.name = sc_pd->name; + if (imx_con_rsrc == sc_pd->rsrc) + sc_pd->pd.flags = GENPD_FLAG_RPM_ALWAYS_ON; + + mode = imx_sc_get_pd_power(dev, pd_ranges->rsrc + idx); + if (mode == IMX_SC_PM_PW_MODE_ON) + is_off = false; + else + is_off = true; + + dev_dbg(dev, "%s : %s\n", sc_pd->name, imx_sc_pm_mode[mode]); + + if (sc_pd->rsrc >= IMX_SC_R_LAST) { + dev_warn(dev, "invalid pd %s rsrc id %d found", + sc_pd->name, sc_pd->rsrc); + + devm_kfree(dev, sc_pd); + return NULL; + } + + ret = pm_genpd_init(&sc_pd->pd, NULL, is_off); + if (ret) { + dev_warn(dev, "failed to init pd %s rsrc id %d", + sc_pd->name, sc_pd->rsrc); + devm_kfree(dev, sc_pd); + return NULL; + } + + return sc_pd; +} + +static int imx_scu_init_pm_domains(struct device *dev, + const struct imx_sc_pd_soc *pd_soc) +{ + const struct imx_sc_pd_range *pd_ranges = pd_soc->pd_ranges; + struct generic_pm_domain **domains; + struct genpd_onecell_data *pd_data; + struct imx_sc_pm_domain *sc_pd; + u32 count = 0; + int i, j; + + for (i = 0; i < pd_soc->num_ranges; i++) + count += pd_ranges[i].num; + + domains = devm_kcalloc(dev, count, sizeof(*domains), GFP_KERNEL); + if (!domains) + return -ENOMEM; + + pd_data = devm_kzalloc(dev, sizeof(*pd_data), GFP_KERNEL); + if (!pd_data) + return -ENOMEM; + + count = 0; + for (i = 0; i < pd_soc->num_ranges; i++) { + for (j = 0; j < pd_ranges[i].num; j++) { + sc_pd = imx_scu_add_pm_domain(dev, j, &pd_ranges[i]); + if (IS_ERR_OR_NULL(sc_pd)) + continue; + + domains[count++] = &sc_pd->pd; + dev_dbg(dev, "added power domain %s\n", sc_pd->pd.name); + } + } + + pd_data->domains = domains; + pd_data->num_domains = count; + pd_data->xlate = imx_scu_pd_xlate; + + of_genpd_add_provider_onecell(dev->of_node, pd_data); + + return 0; +} + +static int imx_sc_pd_probe(struct platform_device *pdev) +{ + const struct imx_sc_pd_soc *pd_soc; + int ret; + + ret = imx_scu_get_handle(&pm_ipc_handle); + if (ret) + return ret; + + pd_soc = of_device_get_match_data(&pdev->dev); + if (!pd_soc) + return -ENODEV; + + imx_sc_pd_get_console_rsrc(); + + return imx_scu_init_pm_domains(&pdev->dev, pd_soc); +} + +static const struct of_device_id imx_sc_pd_match[] = { + { .compatible = "fsl,imx8qxp-scu-pd", &imx8qxp_scu_pd}, + { .compatible = "fsl,scu-pd", &imx8qxp_scu_pd}, + { /* sentinel */ } +}; + +static struct platform_driver imx_sc_pd_driver = { + .driver = { + .name = "imx-scu-pd", + .of_match_table = imx_sc_pd_match, + .suppress_bind_attrs = true, + }, + .probe = imx_sc_pd_probe, +}; +builtin_platform_driver(imx_sc_pd_driver); + +MODULE_AUTHOR("Dong Aisheng "); +MODULE_DESCRIPTION("IMX SCU Power Domain driver"); +MODULE_LICENSE("GPL v2"); diff --git a/drivers/pmdomain/mediatek/Makefile b/drivers/pmdomain/mediatek/Makefile new file mode 100644 index 000000000000..8cde09e654b3 --- /dev/null +++ b/drivers/pmdomain/mediatek/Makefile @@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0-only +obj-$(CONFIG_MTK_SCPSYS) += mtk-scpsys.o +obj-$(CONFIG_MTK_SCPSYS_PM_DOMAINS) += mtk-pm-domains.o diff --git a/drivers/pmdomain/mediatek/mt6795-pm-domains.h b/drivers/pmdomain/mediatek/mt6795-pm-domains.h new file mode 100644 index 000000000000..ef07c9dfdd9b --- /dev/null +++ b/drivers/pmdomain/mediatek/mt6795-pm-domains.h @@ -0,0 +1,112 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ + +#ifndef __SOC_MEDIATEK_MT6795_PM_DOMAINS_H +#define __SOC_MEDIATEK_MT6795_PM_DOMAINS_H + +#include "mtk-pm-domains.h" +#include + +/* + * MT6795 power domain support + */ + +static const struct scpsys_domain_data scpsys_domain_data_mt6795[] = { + [MT6795_POWER_DOMAIN_VDEC] = { + .name = "vdec", + .sta_mask = PWR_STATUS_VDEC, + .ctl_offs = SPM_VDE_PWR_CON, + .pwr_sta_offs = SPM_PWR_STATUS, + .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + }, + [MT6795_POWER_DOMAIN_VENC] = { + .name = "venc", + .sta_mask = PWR_STATUS_VENC, + .ctl_offs = SPM_VEN_PWR_CON, + .pwr_sta_offs = SPM_PWR_STATUS, + .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = GENMASK(15, 12), + }, + [MT6795_POWER_DOMAIN_ISP] = { + .name = "isp", + .sta_mask = PWR_STATUS_ISP, + .ctl_offs = SPM_ISP_PWR_CON, + .pwr_sta_offs = SPM_PWR_STATUS, + .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = GENMASK(13, 12), + }, + [MT6795_POWER_DOMAIN_MM] = { + .name = "mm", + .sta_mask = PWR_STATUS_DISP, + .ctl_offs = SPM_DIS_PWR_CON, + .pwr_sta_offs = SPM_PWR_STATUS, + .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .bp_infracfg = { + BUS_PROT_UPDATE_TOPAXI(MT8173_TOP_AXI_PROT_EN_MM_M0 | + MT8173_TOP_AXI_PROT_EN_MM_M1), + }, + }, + [MT6795_POWER_DOMAIN_MJC] = { + .name = "mjc", + .sta_mask = BIT(20), + .ctl_offs = 0x298, + .pwr_sta_offs = SPM_PWR_STATUS, + .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = GENMASK(15, 12), + }, + [MT6795_POWER_DOMAIN_AUDIO] = { + .name = "audio", + .sta_mask = PWR_STATUS_AUDIO, + .ctl_offs = SPM_AUDIO_PWR_CON, + .pwr_sta_offs = SPM_PWR_STATUS, + .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = GENMASK(15, 12), + }, + [MT6795_POWER_DOMAIN_MFG_ASYNC] = { + .name = "mfg_async", + .sta_mask = PWR_STATUS_MFG_ASYNC, + .ctl_offs = SPM_MFG_ASYNC_PWR_CON, + .pwr_sta_offs = SPM_PWR_STATUS, + .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = 0, + }, + [MT6795_POWER_DOMAIN_MFG_2D] = { + .name = "mfg_2d", + .sta_mask = PWR_STATUS_MFG_2D, + .ctl_offs = SPM_MFG_2D_PWR_CON, + .pwr_sta_offs = SPM_PWR_STATUS, + .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = GENMASK(13, 12), + }, + [MT6795_POWER_DOMAIN_MFG] = { + .name = "mfg", + .sta_mask = PWR_STATUS_MFG, + .ctl_offs = SPM_MFG_PWR_CON, + .pwr_sta_offs = SPM_PWR_STATUS, + .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, + .sram_pdn_bits = GENMASK(13, 8), + .sram_pdn_ack_bits = GENMASK(21, 16), + .bp_infracfg = { + BUS_PROT_UPDATE_TOPAXI(MT8173_TOP_AXI_PROT_EN_MFG_S | + MT8173_TOP_AXI_PROT_EN_MFG_M0 | + MT8173_TOP_AXI_PROT_EN_MFG_M1 | + MT8173_TOP_AXI_PROT_EN_MFG_SNOOP_OUT), + }, + }, +}; + +static const struct scpsys_soc_data mt6795_scpsys_data = { + .domains_data = scpsys_domain_data_mt6795, + .num_domains = ARRAY_SIZE(scpsys_domain_data_mt6795), +}; + +#endif /* __SOC_MEDIATEK_MT6795_PM_DOMAINS_H */ diff --git a/drivers/pmdomain/mediatek/mt8167-pm-domains.h b/drivers/pmdomain/mediatek/mt8167-pm-domains.h new file mode 100644 index 000000000000..4d6c32759606 --- /dev/null +++ b/drivers/pmdomain/mediatek/mt8167-pm-domains.h @@ -0,0 +1,105 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ + +#ifndef __SOC_MEDIATEK_MT8167_PM_DOMAINS_H +#define __SOC_MEDIATEK_MT8167_PM_DOMAINS_H + +#include "mtk-pm-domains.h" +#include + +#define MT8167_PWR_STATUS_MFG_2D BIT(24) +#define MT8167_PWR_STATUS_MFG_ASYNC BIT(25) + +/* + * MT8167 power domain support + */ + +static const struct scpsys_domain_data scpsys_domain_data_mt8167[] = { + [MT8167_POWER_DOMAIN_MM] = { + .name = "mm", + .sta_mask = PWR_STATUS_DISP, + .ctl_offs = SPM_DIS_PWR_CON, + .pwr_sta_offs = SPM_PWR_STATUS, + .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .bp_infracfg = { + BUS_PROT_UPDATE_TOPAXI(MT8167_TOP_AXI_PROT_EN_MM_EMI | + MT8167_TOP_AXI_PROT_EN_MCU_MM), + }, + .caps = MTK_SCPD_ACTIVE_WAKEUP, + }, + [MT8167_POWER_DOMAIN_VDEC] = { + .name = "vdec", + .sta_mask = PWR_STATUS_VDEC, + .ctl_offs = SPM_VDE_PWR_CON, + .pwr_sta_offs = SPM_PWR_STATUS, + .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .caps = MTK_SCPD_ACTIVE_WAKEUP, + }, + [MT8167_POWER_DOMAIN_ISP] = { + .name = "isp", + .sta_mask = PWR_STATUS_ISP, + .ctl_offs = SPM_ISP_PWR_CON, + .pwr_sta_offs = SPM_PWR_STATUS, + .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = GENMASK(13, 12), + .caps = MTK_SCPD_ACTIVE_WAKEUP, + }, + [MT8167_POWER_DOMAIN_MFG_ASYNC] = { + .name = "mfg_async", + .sta_mask = MT8167_PWR_STATUS_MFG_ASYNC, + .ctl_offs = SPM_MFG_ASYNC_PWR_CON, + .pwr_sta_offs = SPM_PWR_STATUS, + .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, + .sram_pdn_bits = 0, + .sram_pdn_ack_bits = 0, + .bp_infracfg = { + BUS_PROT_UPDATE_TOPAXI(MT8167_TOP_AXI_PROT_EN_MCU_MFG | + MT8167_TOP_AXI_PROT_EN_MFG_EMI), + }, + }, + [MT8167_POWER_DOMAIN_MFG_2D] = { + .name = "mfg_2d", + .sta_mask = MT8167_PWR_STATUS_MFG_2D, + .ctl_offs = SPM_MFG_2D_PWR_CON, + .pwr_sta_offs = SPM_PWR_STATUS, + .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = GENMASK(15, 12), + }, + [MT8167_POWER_DOMAIN_MFG] = { + .name = "mfg", + .sta_mask = PWR_STATUS_MFG, + .ctl_offs = SPM_MFG_PWR_CON, + .pwr_sta_offs = SPM_PWR_STATUS, + .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = GENMASK(15, 12), + }, + [MT8167_POWER_DOMAIN_CONN] = { + .name = "conn", + .sta_mask = PWR_STATUS_CONN, + .ctl_offs = SPM_CONN_PWR_CON, + .pwr_sta_offs = SPM_PWR_STATUS, + .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = 0, + .caps = MTK_SCPD_ACTIVE_WAKEUP, + .bp_infracfg = { + BUS_PROT_UPDATE_TOPAXI(MT8167_TOP_AXI_PROT_EN_CONN_EMI | + MT8167_TOP_AXI_PROT_EN_CONN_MCU | + MT8167_TOP_AXI_PROT_EN_MCU_CONN), + }, + }, +}; + +static const struct scpsys_soc_data mt8167_scpsys_data = { + .domains_data = scpsys_domain_data_mt8167, + .num_domains = ARRAY_SIZE(scpsys_domain_data_mt8167), +}; + +#endif /* __SOC_MEDIATEK_MT8167_PM_DOMAINS_H */ + diff --git a/drivers/pmdomain/mediatek/mt8173-pm-domains.h b/drivers/pmdomain/mediatek/mt8173-pm-domains.h new file mode 100644 index 000000000000..1a5dc63b7357 --- /dev/null +++ b/drivers/pmdomain/mediatek/mt8173-pm-domains.h @@ -0,0 +1,123 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ + +#ifndef __SOC_MEDIATEK_MT8173_PM_DOMAINS_H +#define __SOC_MEDIATEK_MT8173_PM_DOMAINS_H + +#include "mtk-pm-domains.h" +#include + +/* + * MT8173 power domain support + */ + +static const struct scpsys_domain_data scpsys_domain_data_mt8173[] = { + [MT8173_POWER_DOMAIN_VDEC] = { + .name = "vdec", + .sta_mask = PWR_STATUS_VDEC, + .ctl_offs = SPM_VDE_PWR_CON, + .pwr_sta_offs = SPM_PWR_STATUS, + .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + }, + [MT8173_POWER_DOMAIN_VENC] = { + .name = "venc", + .sta_mask = PWR_STATUS_VENC, + .ctl_offs = SPM_VEN_PWR_CON, + .pwr_sta_offs = SPM_PWR_STATUS, + .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = GENMASK(15, 12), + }, + [MT8173_POWER_DOMAIN_ISP] = { + .name = "isp", + .sta_mask = PWR_STATUS_ISP, + .ctl_offs = SPM_ISP_PWR_CON, + .pwr_sta_offs = SPM_PWR_STATUS, + .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = GENMASK(13, 12), + }, + [MT8173_POWER_DOMAIN_MM] = { + .name = "mm", + .sta_mask = PWR_STATUS_DISP, + .ctl_offs = SPM_DIS_PWR_CON, + .pwr_sta_offs = SPM_PWR_STATUS, + .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .bp_infracfg = { + BUS_PROT_UPDATE_TOPAXI(MT8173_TOP_AXI_PROT_EN_MM_M0 | + MT8173_TOP_AXI_PROT_EN_MM_M1), + }, + }, + [MT8173_POWER_DOMAIN_VENC_LT] = { + .name = "venc_lt", + .sta_mask = PWR_STATUS_VENC_LT, + .ctl_offs = SPM_VEN2_PWR_CON, + .pwr_sta_offs = SPM_PWR_STATUS, + .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = GENMASK(15, 12), + }, + [MT8173_POWER_DOMAIN_AUDIO] = { + .name = "audio", + .sta_mask = PWR_STATUS_AUDIO, + .ctl_offs = SPM_AUDIO_PWR_CON, + .pwr_sta_offs = SPM_PWR_STATUS, + .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = GENMASK(15, 12), + }, + [MT8173_POWER_DOMAIN_USB] = { + .name = "usb", + .sta_mask = PWR_STATUS_USB, + .ctl_offs = SPM_USB_PWR_CON, + .pwr_sta_offs = SPM_PWR_STATUS, + .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = GENMASK(15, 12), + .caps = MTK_SCPD_ACTIVE_WAKEUP, + }, + [MT8173_POWER_DOMAIN_MFG_ASYNC] = { + .name = "mfg_async", + .sta_mask = PWR_STATUS_MFG_ASYNC, + .ctl_offs = SPM_MFG_ASYNC_PWR_CON, + .pwr_sta_offs = SPM_PWR_STATUS, + .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = 0, + .caps = MTK_SCPD_DOMAIN_SUPPLY, + }, + [MT8173_POWER_DOMAIN_MFG_2D] = { + .name = "mfg_2d", + .sta_mask = PWR_STATUS_MFG_2D, + .ctl_offs = SPM_MFG_2D_PWR_CON, + .pwr_sta_offs = SPM_PWR_STATUS, + .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = GENMASK(13, 12), + }, + [MT8173_POWER_DOMAIN_MFG] = { + .name = "mfg", + .sta_mask = PWR_STATUS_MFG, + .ctl_offs = SPM_MFG_PWR_CON, + .pwr_sta_offs = SPM_PWR_STATUS, + .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND, + .sram_pdn_bits = GENMASK(13, 8), + .sram_pdn_ack_bits = GENMASK(21, 16), + .bp_infracfg = { + BUS_PROT_UPDATE_TOPAXI(MT8173_TOP_AXI_PROT_EN_MFG_S | + MT8173_TOP_AXI_PROT_EN_MFG_M0 | + MT8173_TOP_AXI_PROT_EN_MFG_M1 | + MT8173_TOP_AXI_PROT_EN_MFG_SNOOP_OUT), + }, + }, +}; + +static const struct scpsys_soc_data mt8173_scpsys_data = { + .domains_data = scpsys_domain_data_mt8173, + .num_domains = ARRAY_SIZE(scpsys_domain_data_mt8173), +}; + +#endif /* __SOC_MEDIATEK_MT8173_PM_DOMAINS_H */ diff --git a/drivers/pmdomain/mediatek/mt8183-pm-domains.h b/drivers/pmdomain/mediatek/mt8183-pm-domains.h new file mode 100644 index 000000000000..99de67fe5de8 --- /dev/null +++ b/drivers/pmdomain/mediatek/mt8183-pm-domains.h @@ -0,0 +1,266 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ + +#ifndef __SOC_MEDIATEK_MT8183_PM_DOMAINS_H +#define __SOC_MEDIATEK_MT8183_PM_DOMAINS_H + +#include "mtk-pm-domains.h" +#include + +/* + * MT8183 power domain support + */ + +static const struct scpsys_domain_data scpsys_domain_data_mt8183[] = { + [MT8183_POWER_DOMAIN_AUDIO] = { + .name = "audio", + .sta_mask = PWR_STATUS_AUDIO, + .ctl_offs = 0x0314, + .pwr_sta_offs = 0x0180, + .pwr_sta2nd_offs = 0x0184, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = GENMASK(15, 12), + }, + [MT8183_POWER_DOMAIN_CONN] = { + .name = "conn", + .sta_mask = PWR_STATUS_CONN, + .ctl_offs = 0x032c, + .pwr_sta_offs = 0x0180, + .pwr_sta2nd_offs = 0x0184, + .sram_pdn_bits = 0, + .sram_pdn_ack_bits = 0, + .bp_infracfg = { + BUS_PROT_WR(MT8183_TOP_AXI_PROT_EN_CONN, MT8183_TOP_AXI_PROT_EN_SET, + MT8183_TOP_AXI_PROT_EN_CLR, MT8183_TOP_AXI_PROT_EN_STA1), + }, + }, + [MT8183_POWER_DOMAIN_MFG_ASYNC] = { + .name = "mfg_async", + .sta_mask = PWR_STATUS_MFG_ASYNC, + .ctl_offs = 0x0334, + .pwr_sta_offs = 0x0180, + .pwr_sta2nd_offs = 0x0184, + .sram_pdn_bits = 0, + .sram_pdn_ack_bits = 0, + .caps = MTK_SCPD_DOMAIN_SUPPLY, + }, + [MT8183_POWER_DOMAIN_MFG] = { + .name = "mfg", + .sta_mask = PWR_STATUS_MFG, + .ctl_offs = 0x0338, + .pwr_sta_offs = 0x0180, + .pwr_sta2nd_offs = 0x0184, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .caps = MTK_SCPD_DOMAIN_SUPPLY, + }, + [MT8183_POWER_DOMAIN_MFG_CORE0] = { + .name = "mfg_core0", + .sta_mask = BIT(7), + .ctl_offs = 0x034c, + .pwr_sta_offs = 0x0180, + .pwr_sta2nd_offs = 0x0184, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + }, + [MT8183_POWER_DOMAIN_MFG_CORE1] = { + .name = "mfg_core1", + .sta_mask = BIT(20), + .ctl_offs = 0x0310, + .pwr_sta_offs = 0x0180, + .pwr_sta2nd_offs = 0x0184, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + }, + [MT8183_POWER_DOMAIN_MFG_2D] = { + .name = "mfg_2d", + .sta_mask = PWR_STATUS_MFG_2D, + .ctl_offs = 0x0348, + .pwr_sta_offs = 0x0180, + .pwr_sta2nd_offs = 0x0184, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .bp_infracfg = { + BUS_PROT_WR(MT8183_TOP_AXI_PROT_EN_1_MFG, MT8183_TOP_AXI_PROT_EN_1_SET, + MT8183_TOP_AXI_PROT_EN_1_CLR, MT8183_TOP_AXI_PROT_EN_STA1_1), + BUS_PROT_WR(MT8183_TOP_AXI_PROT_EN_MFG, MT8183_TOP_AXI_PROT_EN_SET, + MT8183_TOP_AXI_PROT_EN_CLR, MT8183_TOP_AXI_PROT_EN_STA1), + }, + }, + [MT8183_POWER_DOMAIN_DISP] = { + .name = "disp", + .sta_mask = PWR_STATUS_DISP, + .ctl_offs = 0x030c, + .pwr_sta_offs = 0x0180, + .pwr_sta2nd_offs = 0x0184, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .bp_infracfg = { + BUS_PROT_WR(MT8183_TOP_AXI_PROT_EN_1_DISP, MT8183_TOP_AXI_PROT_EN_1_SET, + MT8183_TOP_AXI_PROT_EN_1_CLR, MT8183_TOP_AXI_PROT_EN_STA1_1), + BUS_PROT_WR(MT8183_TOP_AXI_PROT_EN_DISP, MT8183_TOP_AXI_PROT_EN_SET, + MT8183_TOP_AXI_PROT_EN_CLR, MT8183_TOP_AXI_PROT_EN_STA1), + }, + .bp_smi = { + BUS_PROT_WR(MT8183_SMI_COMMON_SMI_CLAMP_DISP, + MT8183_SMI_COMMON_CLAMP_EN_SET, + MT8183_SMI_COMMON_CLAMP_EN_CLR, + MT8183_SMI_COMMON_CLAMP_EN), + }, + }, + [MT8183_POWER_DOMAIN_CAM] = { + .name = "cam", + .sta_mask = BIT(25), + .ctl_offs = 0x0344, + .pwr_sta_offs = 0x0180, + .pwr_sta2nd_offs = 0x0184, + .sram_pdn_bits = GENMASK(9, 8), + .sram_pdn_ack_bits = GENMASK(13, 12), + .bp_infracfg = { + BUS_PROT_WR(MT8183_TOP_AXI_PROT_EN_MM_CAM, MT8183_TOP_AXI_PROT_EN_MM_SET, + MT8183_TOP_AXI_PROT_EN_MM_CLR, MT8183_TOP_AXI_PROT_EN_MM_STA1), + BUS_PROT_WR(MT8183_TOP_AXI_PROT_EN_CAM, MT8183_TOP_AXI_PROT_EN_SET, + MT8183_TOP_AXI_PROT_EN_CLR, MT8183_TOP_AXI_PROT_EN_STA1), + BUS_PROT_WR_IGN(MT8183_TOP_AXI_PROT_EN_MM_CAM_2ND, + MT8183_TOP_AXI_PROT_EN_MM_SET, + MT8183_TOP_AXI_PROT_EN_MM_CLR, + MT8183_TOP_AXI_PROT_EN_MM_STA1), + }, + .bp_smi = { + BUS_PROT_WR(MT8183_SMI_COMMON_SMI_CLAMP_CAM, + MT8183_SMI_COMMON_CLAMP_EN_SET, + MT8183_SMI_COMMON_CLAMP_EN_CLR, + MT8183_SMI_COMMON_CLAMP_EN), + }, + }, + [MT8183_POWER_DOMAIN_ISP] = { + .name = "isp", + .sta_mask = PWR_STATUS_ISP, + .ctl_offs = 0x0308, + .pwr_sta_offs = 0x0180, + .pwr_sta2nd_offs = 0x0184, + .sram_pdn_bits = GENMASK(9, 8), + .sram_pdn_ack_bits = GENMASK(13, 12), + .bp_infracfg = { + BUS_PROT_WR(MT8183_TOP_AXI_PROT_EN_MM_ISP, + MT8183_TOP_AXI_PROT_EN_MM_SET, + MT8183_TOP_AXI_PROT_EN_MM_CLR, + MT8183_TOP_AXI_PROT_EN_MM_STA1), + BUS_PROT_WR_IGN(MT8183_TOP_AXI_PROT_EN_MM_ISP_2ND, + MT8183_TOP_AXI_PROT_EN_MM_SET, + MT8183_TOP_AXI_PROT_EN_MM_CLR, + MT8183_TOP_AXI_PROT_EN_MM_STA1), + }, + .bp_smi = { + BUS_PROT_WR(MT8183_SMI_COMMON_SMI_CLAMP_ISP, + MT8183_SMI_COMMON_CLAMP_EN_SET, + MT8183_SMI_COMMON_CLAMP_EN_CLR, + MT8183_SMI_COMMON_CLAMP_EN), + }, + }, + [MT8183_POWER_DOMAIN_VDEC] = { + .name = "vdec", + .sta_mask = BIT(31), + .ctl_offs = 0x0300, + .pwr_sta_offs = 0x0180, + .pwr_sta2nd_offs = 0x0184, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .bp_smi = { + BUS_PROT_WR(MT8183_SMI_COMMON_SMI_CLAMP_VDEC, + MT8183_SMI_COMMON_CLAMP_EN_SET, + MT8183_SMI_COMMON_CLAMP_EN_CLR, + MT8183_SMI_COMMON_CLAMP_EN), + }, + }, + [MT8183_POWER_DOMAIN_VENC] = { + .name = "venc", + .sta_mask = PWR_STATUS_VENC, + .ctl_offs = 0x0304, + .pwr_sta_offs = 0x0180, + .pwr_sta2nd_offs = 0x0184, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = GENMASK(15, 12), + .bp_smi = { + BUS_PROT_WR(MT8183_SMI_COMMON_SMI_CLAMP_VENC, + MT8183_SMI_COMMON_CLAMP_EN_SET, + MT8183_SMI_COMMON_CLAMP_EN_CLR, + MT8183_SMI_COMMON_CLAMP_EN), + }, + }, + [MT8183_POWER_DOMAIN_VPU_TOP] = { + .name = "vpu_top", + .sta_mask = BIT(26), + .ctl_offs = 0x0324, + .pwr_sta_offs = 0x0180, + .pwr_sta2nd_offs = 0x0184, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .bp_infracfg = { + BUS_PROT_WR(MT8183_TOP_AXI_PROT_EN_MM_VPU_TOP, + MT8183_TOP_AXI_PROT_EN_MM_SET, + MT8183_TOP_AXI_PROT_EN_MM_CLR, + MT8183_TOP_AXI_PROT_EN_MM_STA1), + BUS_PROT_WR(MT8183_TOP_AXI_PROT_EN_VPU_TOP, + MT8183_TOP_AXI_PROT_EN_SET, + MT8183_TOP_AXI_PROT_EN_CLR, + MT8183_TOP_AXI_PROT_EN_STA1), + BUS_PROT_WR(MT8183_TOP_AXI_PROT_EN_MM_VPU_TOP_2ND, + MT8183_TOP_AXI_PROT_EN_MM_SET, + MT8183_TOP_AXI_PROT_EN_MM_CLR, + MT8183_TOP_AXI_PROT_EN_MM_STA1), + }, + .bp_smi = { + BUS_PROT_WR(MT8183_SMI_COMMON_SMI_CLAMP_VPU_TOP, + MT8183_SMI_COMMON_CLAMP_EN_SET, + MT8183_SMI_COMMON_CLAMP_EN_CLR, + MT8183_SMI_COMMON_CLAMP_EN), + }, + }, + [MT8183_POWER_DOMAIN_VPU_CORE0] = { + .name = "vpu_core0", + .sta_mask = BIT(27), + .ctl_offs = 0x33c, + .pwr_sta_offs = 0x0180, + .pwr_sta2nd_offs = 0x0184, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = GENMASK(13, 12), + .bp_infracfg = { + BUS_PROT_WR(MT8183_TOP_AXI_PROT_EN_MCU_VPU_CORE0, + MT8183_TOP_AXI_PROT_EN_MCU_SET, + MT8183_TOP_AXI_PROT_EN_MCU_CLR, + MT8183_TOP_AXI_PROT_EN_MCU_STA1), + BUS_PROT_WR(MT8183_TOP_AXI_PROT_EN_MCU_VPU_CORE0_2ND, + MT8183_TOP_AXI_PROT_EN_MCU_SET, + MT8183_TOP_AXI_PROT_EN_MCU_CLR, + MT8183_TOP_AXI_PROT_EN_MCU_STA1), + }, + .caps = MTK_SCPD_SRAM_ISO, + }, + [MT8183_POWER_DOMAIN_VPU_CORE1] = { + .name = "vpu_core1", + .sta_mask = BIT(28), + .ctl_offs = 0x0340, + .pwr_sta_offs = 0x0180, + .pwr_sta2nd_offs = 0x0184, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = GENMASK(13, 12), + .bp_infracfg = { + BUS_PROT_WR(MT8183_TOP_AXI_PROT_EN_MCU_VPU_CORE1, + MT8183_TOP_AXI_PROT_EN_MCU_SET, + MT8183_TOP_AXI_PROT_EN_MCU_CLR, + MT8183_TOP_AXI_PROT_EN_MCU_STA1), + BUS_PROT_WR(MT8183_TOP_AXI_PROT_EN_MCU_VPU_CORE1_2ND, + MT8183_TOP_AXI_PROT_EN_MCU_SET, + MT8183_TOP_AXI_PROT_EN_MCU_CLR, + MT8183_TOP_AXI_PROT_EN_MCU_STA1), + }, + .caps = MTK_SCPD_SRAM_ISO, + }, +}; + +static const struct scpsys_soc_data mt8183_scpsys_data = { + .domains_data = scpsys_domain_data_mt8183, + .num_domains = ARRAY_SIZE(scpsys_domain_data_mt8183), +}; + +#endif /* __SOC_MEDIATEK_MT8183_PM_DOMAINS_H */ diff --git a/drivers/pmdomain/mediatek/mt8186-pm-domains.h b/drivers/pmdomain/mediatek/mt8186-pm-domains.h new file mode 100644 index 000000000000..fce86f79c505 --- /dev/null +++ b/drivers/pmdomain/mediatek/mt8186-pm-domains.h @@ -0,0 +1,342 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022 MediaTek Inc. + * Author: Chun-Jie Chen + */ + +#ifndef __SOC_MEDIATEK_MT8186_PM_DOMAINS_H +#define __SOC_MEDIATEK_MT8186_PM_DOMAINS_H + +#include "mtk-pm-domains.h" +#include + +/* + * MT8186 power domain support + */ + +static const struct scpsys_domain_data scpsys_domain_data_mt8186[] = { + [MT8186_POWER_DOMAIN_MFG0] = { + .name = "mfg0", + .sta_mask = BIT(2), + .ctl_offs = 0x308, + .pwr_sta_offs = 0x16C, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = BIT(8), + .sram_pdn_ack_bits = BIT(12), + .caps = MTK_SCPD_KEEP_DEFAULT_OFF | MTK_SCPD_DOMAIN_SUPPLY, + }, + [MT8186_POWER_DOMAIN_MFG1] = { + .name = "mfg1", + .sta_mask = BIT(3), + .ctl_offs = 0x30c, + .pwr_sta_offs = 0x16C, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = BIT(8), + .sram_pdn_ack_bits = BIT(12), + .bp_infracfg = { + BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_1_MFG1_STEP1, + MT8186_TOP_AXI_PROT_EN_1_SET, + MT8186_TOP_AXI_PROT_EN_1_CLR, + MT8186_TOP_AXI_PROT_EN_1_STA), + BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_MFG1_STEP2, + MT8186_TOP_AXI_PROT_EN_SET, + MT8186_TOP_AXI_PROT_EN_CLR, + MT8186_TOP_AXI_PROT_EN_STA), + BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_MFG1_STEP3, + MT8186_TOP_AXI_PROT_EN_SET, + MT8186_TOP_AXI_PROT_EN_CLR, + MT8186_TOP_AXI_PROT_EN_STA), + BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_1_MFG1_STEP4, + MT8186_TOP_AXI_PROT_EN_1_SET, + MT8186_TOP_AXI_PROT_EN_1_CLR, + MT8186_TOP_AXI_PROT_EN_1_STA), + }, + .caps = MTK_SCPD_KEEP_DEFAULT_OFF | MTK_SCPD_DOMAIN_SUPPLY, + }, + [MT8186_POWER_DOMAIN_MFG2] = { + .name = "mfg2", + .sta_mask = BIT(4), + .ctl_offs = 0x310, + .pwr_sta_offs = 0x16C, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = BIT(8), + .sram_pdn_ack_bits = BIT(12), + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8186_POWER_DOMAIN_MFG3] = { + .name = "mfg3", + .sta_mask = BIT(5), + .ctl_offs = 0x314, + .pwr_sta_offs = 0x16C, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = BIT(8), + .sram_pdn_ack_bits = BIT(12), + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8186_POWER_DOMAIN_SSUSB] = { + .name = "ssusb", + .sta_mask = BIT(20), + .ctl_offs = 0x9F0, + .pwr_sta_offs = 0x16C, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = BIT(8), + .sram_pdn_ack_bits = BIT(12), + .caps = MTK_SCPD_ACTIVE_WAKEUP, + }, + [MT8186_POWER_DOMAIN_SSUSB_P1] = { + .name = "ssusb_p1", + .sta_mask = BIT(19), + .ctl_offs = 0x9F4, + .pwr_sta_offs = 0x16C, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = BIT(8), + .sram_pdn_ack_bits = BIT(12), + .caps = MTK_SCPD_ACTIVE_WAKEUP, + }, + [MT8186_POWER_DOMAIN_DIS] = { + .name = "dis", + .sta_mask = BIT(21), + .ctl_offs = 0x354, + .pwr_sta_offs = 0x16C, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = BIT(8), + .sram_pdn_ack_bits = BIT(12), + .bp_infracfg = { + BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_1_DIS_STEP1, + MT8186_TOP_AXI_PROT_EN_1_SET, + MT8186_TOP_AXI_PROT_EN_1_CLR, + MT8186_TOP_AXI_PROT_EN_1_STA), + BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_DIS_STEP2, + MT8186_TOP_AXI_PROT_EN_SET, + MT8186_TOP_AXI_PROT_EN_CLR, + MT8186_TOP_AXI_PROT_EN_STA), + }, + }, + [MT8186_POWER_DOMAIN_IMG] = { + .name = "img", + .sta_mask = BIT(13), + .ctl_offs = 0x334, + .pwr_sta_offs = 0x16C, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = BIT(8), + .sram_pdn_ack_bits = BIT(12), + .bp_infracfg = { + BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_1_IMG_STEP1, + MT8186_TOP_AXI_PROT_EN_1_SET, + MT8186_TOP_AXI_PROT_EN_1_CLR, + MT8186_TOP_AXI_PROT_EN_1_STA), + BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_1_IMG_STEP2, + MT8186_TOP_AXI_PROT_EN_1_SET, + MT8186_TOP_AXI_PROT_EN_1_CLR, + MT8186_TOP_AXI_PROT_EN_1_STA), + }, + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8186_POWER_DOMAIN_IMG2] = { + .name = "img2", + .sta_mask = BIT(14), + .ctl_offs = 0x338, + .pwr_sta_offs = 0x16C, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = BIT(8), + .sram_pdn_ack_bits = BIT(12), + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8186_POWER_DOMAIN_IPE] = { + .name = "ipe", + .sta_mask = BIT(15), + .ctl_offs = 0x33C, + .pwr_sta_offs = 0x16C, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = BIT(8), + .sram_pdn_ack_bits = BIT(12), + .bp_infracfg = { + BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_1_IPE_STEP1, + MT8186_TOP_AXI_PROT_EN_1_SET, + MT8186_TOP_AXI_PROT_EN_1_CLR, + MT8186_TOP_AXI_PROT_EN_1_STA), + BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_1_IPE_STEP2, + MT8186_TOP_AXI_PROT_EN_1_SET, + MT8186_TOP_AXI_PROT_EN_1_CLR, + MT8186_TOP_AXI_PROT_EN_1_STA), + }, + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8186_POWER_DOMAIN_CAM] = { + .name = "cam", + .sta_mask = BIT(23), + .ctl_offs = 0x35C, + .pwr_sta_offs = 0x16C, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = BIT(8), + .sram_pdn_ack_bits = BIT(12), + .bp_infracfg = { + BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_1_CAM_STEP1, + MT8186_TOP_AXI_PROT_EN_1_SET, + MT8186_TOP_AXI_PROT_EN_1_CLR, + MT8186_TOP_AXI_PROT_EN_1_STA), + BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_1_CAM_STEP2, + MT8186_TOP_AXI_PROT_EN_1_SET, + MT8186_TOP_AXI_PROT_EN_1_CLR, + MT8186_TOP_AXI_PROT_EN_1_STA), + }, + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8186_POWER_DOMAIN_CAM_RAWA] = { + .name = "cam_rawa", + .sta_mask = BIT(24), + .ctl_offs = 0x360, + .pwr_sta_offs = 0x16C, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = BIT(8), + .sram_pdn_ack_bits = BIT(12), + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8186_POWER_DOMAIN_CAM_RAWB] = { + .name = "cam_rawb", + .sta_mask = BIT(25), + .ctl_offs = 0x364, + .pwr_sta_offs = 0x16C, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = BIT(8), + .sram_pdn_ack_bits = BIT(12), + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8186_POWER_DOMAIN_VENC] = { + .name = "venc", + .sta_mask = BIT(18), + .ctl_offs = 0x348, + .pwr_sta_offs = 0x16C, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = BIT(8), + .sram_pdn_ack_bits = BIT(12), + .bp_infracfg = { + BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_1_VENC_STEP1, + MT8186_TOP_AXI_PROT_EN_1_SET, + MT8186_TOP_AXI_PROT_EN_1_CLR, + MT8186_TOP_AXI_PROT_EN_1_STA), + BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_1_VENC_STEP2, + MT8186_TOP_AXI_PROT_EN_1_SET, + MT8186_TOP_AXI_PROT_EN_1_CLR, + MT8186_TOP_AXI_PROT_EN_1_STA), + }, + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8186_POWER_DOMAIN_VDEC] = { + .name = "vdec", + .sta_mask = BIT(16), + .ctl_offs = 0x340, + .pwr_sta_offs = 0x16C, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = BIT(8), + .sram_pdn_ack_bits = BIT(12), + .bp_infracfg = { + BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_1_VDEC_STEP1, + MT8186_TOP_AXI_PROT_EN_1_SET, + MT8186_TOP_AXI_PROT_EN_1_CLR, + MT8186_TOP_AXI_PROT_EN_1_STA), + BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_1_VDEC_STEP2, + MT8186_TOP_AXI_PROT_EN_1_SET, + MT8186_TOP_AXI_PROT_EN_1_CLR, + MT8186_TOP_AXI_PROT_EN_1_STA), + }, + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8186_POWER_DOMAIN_WPE] = { + .name = "wpe", + .sta_mask = BIT(0), + .ctl_offs = 0x3F8, + .pwr_sta_offs = 0x16C, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = BIT(8), + .sram_pdn_ack_bits = BIT(12), + .bp_infracfg = { + BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_2_WPE_STEP1, + MT8186_TOP_AXI_PROT_EN_2_SET, + MT8186_TOP_AXI_PROT_EN_2_CLR, + MT8186_TOP_AXI_PROT_EN_2_STA), + BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_2_WPE_STEP2, + MT8186_TOP_AXI_PROT_EN_2_SET, + MT8186_TOP_AXI_PROT_EN_2_CLR, + MT8186_TOP_AXI_PROT_EN_2_STA), + }, + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8186_POWER_DOMAIN_CONN_ON] = { + .name = "conn_on", + .sta_mask = BIT(1), + .ctl_offs = 0x304, + .pwr_sta_offs = 0x16C, + .pwr_sta2nd_offs = 0x170, + .bp_infracfg = { + BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_1_CONN_ON_STEP1, + MT8186_TOP_AXI_PROT_EN_1_SET, + MT8186_TOP_AXI_PROT_EN_1_CLR, + MT8186_TOP_AXI_PROT_EN_1_STA), + BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_CONN_ON_STEP2, + MT8186_TOP_AXI_PROT_EN_SET, + MT8186_TOP_AXI_PROT_EN_CLR, + MT8186_TOP_AXI_PROT_EN_STA), + BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_CONN_ON_STEP3, + MT8186_TOP_AXI_PROT_EN_SET, + MT8186_TOP_AXI_PROT_EN_CLR, + MT8186_TOP_AXI_PROT_EN_STA), + BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_CONN_ON_STEP4, + MT8186_TOP_AXI_PROT_EN_SET, + MT8186_TOP_AXI_PROT_EN_CLR, + MT8186_TOP_AXI_PROT_EN_STA), + }, + .caps = MTK_SCPD_KEEP_DEFAULT_OFF | MTK_SCPD_ACTIVE_WAKEUP, + }, + [MT8186_POWER_DOMAIN_CSIRX_TOP] = { + .name = "csirx_top", + .sta_mask = BIT(6), + .ctl_offs = 0x318, + .pwr_sta_offs = 0x16C, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = BIT(8), + .sram_pdn_ack_bits = BIT(12), + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8186_POWER_DOMAIN_ADSP_AO] = { + .name = "adsp_ao", + .sta_mask = BIT(17), + .ctl_offs = 0x9FC, + .pwr_sta_offs = 0x16C, + .pwr_sta2nd_offs = 0x170, + }, + [MT8186_POWER_DOMAIN_ADSP_INFRA] = { + .name = "adsp_infra", + .sta_mask = BIT(10), + .ctl_offs = 0x9F8, + .pwr_sta_offs = 0x16C, + .pwr_sta2nd_offs = 0x170, + }, + [MT8186_POWER_DOMAIN_ADSP_TOP] = { + .name = "adsp_top", + .sta_mask = BIT(31), + .ctl_offs = 0x3E4, + .pwr_sta_offs = 0x16C, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = BIT(8), + .sram_pdn_ack_bits = BIT(12), + .bp_infracfg = { + BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_3_ADSP_TOP_STEP1, + MT8186_TOP_AXI_PROT_EN_3_SET, + MT8186_TOP_AXI_PROT_EN_3_CLR, + MT8186_TOP_AXI_PROT_EN_3_STA), + BUS_PROT_WR_IGN(MT8186_TOP_AXI_PROT_EN_3_ADSP_TOP_STEP2, + MT8186_TOP_AXI_PROT_EN_3_SET, + MT8186_TOP_AXI_PROT_EN_3_CLR, + MT8186_TOP_AXI_PROT_EN_3_STA), + }, + .caps = MTK_SCPD_SRAM_ISO | MTK_SCPD_ACTIVE_WAKEUP, + }, +}; + +static const struct scpsys_soc_data mt8186_scpsys_data = { + .domains_data = scpsys_domain_data_mt8186, + .num_domains = ARRAY_SIZE(scpsys_domain_data_mt8186), +}; + +#endif /* __SOC_MEDIATEK_MT8186_PM_DOMAINS_H */ diff --git a/drivers/pmdomain/mediatek/mt8188-pm-domains.h b/drivers/pmdomain/mediatek/mt8188-pm-domains.h new file mode 100644 index 000000000000..0692cb444ed0 --- /dev/null +++ b/drivers/pmdomain/mediatek/mt8188-pm-domains.h @@ -0,0 +1,623 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022 MediaTek Inc. + * Author: Garmin Chang + */ + +#ifndef __SOC_MEDIATEK_MT8188_PM_DOMAINS_H +#define __SOC_MEDIATEK_MT8188_PM_DOMAINS_H + +#include "mtk-pm-domains.h" +#include + +/* + * MT8188 power domain support + */ + +static const struct scpsys_domain_data scpsys_domain_data_mt8188[] = { + [MT8188_POWER_DOMAIN_MFG0] = { + .name = "mfg0", + .sta_mask = BIT(1), + .ctl_offs = 0x300, + .pwr_sta_offs = 0x174, + .pwr_sta2nd_offs = 0x178, + .sram_pdn_bits = BIT(8), + .sram_pdn_ack_bits = BIT(12), + .caps = MTK_SCPD_KEEP_DEFAULT_OFF | MTK_SCPD_DOMAIN_SUPPLY, + }, + [MT8188_POWER_DOMAIN_MFG1] = { + .name = "mfg1", + .sta_mask = BIT(2), + .ctl_offs = 0x304, + .pwr_sta_offs = 0x174, + .pwr_sta2nd_offs = 0x178, + .sram_pdn_bits = BIT(8), + .sram_pdn_ack_bits = BIT(12), + .bp_infracfg = { + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MFG1_STEP1, + MT8188_TOP_AXI_PROT_EN_SET, + MT8188_TOP_AXI_PROT_EN_CLR, + MT8188_TOP_AXI_PROT_EN_STA), + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_2_MFG1_STEP2, + MT8188_TOP_AXI_PROT_EN_2_SET, + MT8188_TOP_AXI_PROT_EN_2_CLR, + MT8188_TOP_AXI_PROT_EN_2_STA), + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_1_MFG1_STEP3, + MT8188_TOP_AXI_PROT_EN_1_SET, + MT8188_TOP_AXI_PROT_EN_1_CLR, + MT8188_TOP_AXI_PROT_EN_1_STA), + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_2_MFG1_STEP4, + MT8188_TOP_AXI_PROT_EN_2_SET, + MT8188_TOP_AXI_PROT_EN_2_CLR, + MT8188_TOP_AXI_PROT_EN_2_STA), + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MFG1_STEP5, + MT8188_TOP_AXI_PROT_EN_SET, + MT8188_TOP_AXI_PROT_EN_CLR, + MT8188_TOP_AXI_PROT_EN_STA), + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_MFG1_STEP6, + MT8188_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_SET, + MT8188_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_CLR, + MT8188_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_STA), + }, + .caps = MTK_SCPD_KEEP_DEFAULT_OFF | MTK_SCPD_DOMAIN_SUPPLY, + }, + [MT8188_POWER_DOMAIN_MFG2] = { + .name = "mfg2", + .sta_mask = BIT(3), + .ctl_offs = 0x308, + .pwr_sta_offs = 0x174, + .pwr_sta2nd_offs = 0x178, + .sram_pdn_bits = BIT(8), + .sram_pdn_ack_bits = BIT(12), + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8188_POWER_DOMAIN_MFG3] = { + .name = "mfg3", + .sta_mask = BIT(4), + .ctl_offs = 0x30C, + .pwr_sta_offs = 0x174, + .pwr_sta2nd_offs = 0x178, + .sram_pdn_bits = BIT(8), + .sram_pdn_ack_bits = BIT(12), + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8188_POWER_DOMAIN_MFG4] = { + .name = "mfg4", + .sta_mask = BIT(5), + .ctl_offs = 0x310, + .pwr_sta_offs = 0x174, + .pwr_sta2nd_offs = 0x178, + .sram_pdn_bits = BIT(8), + .sram_pdn_ack_bits = BIT(12), + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8188_POWER_DOMAIN_PEXTP_MAC_P0] = { + .name = "pextp_mac_p0", + .sta_mask = BIT(10), + .ctl_offs = 0x324, + .pwr_sta_offs = 0x174, + .pwr_sta2nd_offs = 0x178, + .sram_pdn_bits = BIT(8), + .sram_pdn_ack_bits = BIT(12), + .bp_infracfg = { + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_PEXTP_MAC_P0_STEP1, + MT8188_TOP_AXI_PROT_EN_SET, + MT8188_TOP_AXI_PROT_EN_CLR, + MT8188_TOP_AXI_PROT_EN_STA), + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_INFRA_VDNR_PEXTP_MAC_P0_STEP2, + MT8188_TOP_AXI_PROT_EN_INFRA_VDNR_SET, + MT8188_TOP_AXI_PROT_EN_INFRA_VDNR_CLR, + MT8188_TOP_AXI_PROT_EN_INFRA_VDNR_STA), + }, + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8188_POWER_DOMAIN_PEXTP_PHY_TOP] = { + .name = "pextp_phy_top", + .sta_mask = BIT(12), + .ctl_offs = 0x328, + .pwr_sta_offs = 0x174, + .pwr_sta2nd_offs = 0x178, + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8188_POWER_DOMAIN_CSIRX_TOP] = { + .name = "pextp_csirx_top", + .sta_mask = BIT(17), + .ctl_offs = 0x3C4, + .pwr_sta_offs = 0x174, + .pwr_sta2nd_offs = 0x178, + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8188_POWER_DOMAIN_ETHER] = { + .name = "ether", + .sta_mask = BIT(1), + .ctl_offs = 0x338, + .pwr_sta_offs = 0x16C, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = BIT(8), + .sram_pdn_ack_bits = BIT(12), + .bp_infracfg = { + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_INFRA_VDNR_ETHER_STEP1, + MT8188_TOP_AXI_PROT_EN_INFRA_VDNR_SET, + MT8188_TOP_AXI_PROT_EN_INFRA_VDNR_CLR, + MT8188_TOP_AXI_PROT_EN_INFRA_VDNR_STA), + }, + .caps = MTK_SCPD_KEEP_DEFAULT_OFF | MTK_SCPD_ACTIVE_WAKEUP, + }, + [MT8188_POWER_DOMAIN_HDMI_TX] = { + .name = "hdmi_tx", + .sta_mask = BIT(18), + .ctl_offs = 0x37C, + .pwr_sta_offs = 0x16C, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = BIT(8), + .sram_pdn_ack_bits = BIT(12), + .bp_infracfg = { + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_INFRA_VDNR_HDMI_TX_STEP1, + MT8188_TOP_AXI_PROT_EN_INFRA_VDNR_SET, + MT8188_TOP_AXI_PROT_EN_INFRA_VDNR_CLR, + MT8188_TOP_AXI_PROT_EN_INFRA_VDNR_STA), + }, + .caps = MTK_SCPD_KEEP_DEFAULT_OFF | MTK_SCPD_ACTIVE_WAKEUP, + }, + [MT8188_POWER_DOMAIN_ADSP_AO] = { + .name = "adsp_ao", + .sta_mask = BIT(10), + .ctl_offs = 0x35C, + .pwr_sta_offs = 0x16C, + .pwr_sta2nd_offs = 0x170, + .bp_infracfg = { + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_2_ADSP_AO_STEP1, + MT8188_TOP_AXI_PROT_EN_2_SET, + MT8188_TOP_AXI_PROT_EN_2_CLR, + MT8188_TOP_AXI_PROT_EN_2_STA), + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_2_ADSP_AO_STEP2, + MT8188_TOP_AXI_PROT_EN_2_SET, + MT8188_TOP_AXI_PROT_EN_2_CLR, + MT8188_TOP_AXI_PROT_EN_2_STA), + }, + .caps = MTK_SCPD_ALWAYS_ON, + }, + [MT8188_POWER_DOMAIN_ADSP_INFRA] = { + .name = "adsp_infra", + .sta_mask = BIT(9), + .ctl_offs = 0x358, + .pwr_sta_offs = 0x16C, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = BIT(8), + .sram_pdn_ack_bits = BIT(12), + .bp_infracfg = { + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_2_ADSP_INFRA_STEP1, + MT8188_TOP_AXI_PROT_EN_2_SET, + MT8188_TOP_AXI_PROT_EN_2_CLR, + MT8188_TOP_AXI_PROT_EN_2_STA), + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_2_ADSP_INFRA_STEP2, + MT8188_TOP_AXI_PROT_EN_2_SET, + MT8188_TOP_AXI_PROT_EN_2_CLR, + MT8188_TOP_AXI_PROT_EN_2_STA), + }, + .caps = MTK_SCPD_SRAM_ISO | MTK_SCPD_ALWAYS_ON, + }, + [MT8188_POWER_DOMAIN_ADSP] = { + .name = "adsp", + .sta_mask = BIT(8), + .ctl_offs = 0x354, + .pwr_sta_offs = 0x16C, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = BIT(8), + .sram_pdn_ack_bits = BIT(12), + .bp_infracfg = { + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_2_ADSP_STEP1, + MT8188_TOP_AXI_PROT_EN_2_SET, + MT8188_TOP_AXI_PROT_EN_2_CLR, + MT8188_TOP_AXI_PROT_EN_2_STA), + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_2_ADSP_STEP2, + MT8188_TOP_AXI_PROT_EN_2_SET, + MT8188_TOP_AXI_PROT_EN_2_CLR, + MT8188_TOP_AXI_PROT_EN_2_STA), + }, + .caps = MTK_SCPD_KEEP_DEFAULT_OFF | MTK_SCPD_SRAM_ISO | MTK_SCPD_ACTIVE_WAKEUP, + }, + [MT8188_POWER_DOMAIN_AUDIO] = { + .name = "audio", + .sta_mask = BIT(6), + .ctl_offs = 0x34C, + .pwr_sta_offs = 0x16C, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = BIT(8), + .sram_pdn_ack_bits = BIT(12), + .bp_infracfg = { + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_2_AUDIO_STEP1, + MT8188_TOP_AXI_PROT_EN_2_SET, + MT8188_TOP_AXI_PROT_EN_2_CLR, + MT8188_TOP_AXI_PROT_EN_2_STA), + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_2_AUDIO_STEP2, + MT8188_TOP_AXI_PROT_EN_2_SET, + MT8188_TOP_AXI_PROT_EN_2_CLR, + MT8188_TOP_AXI_PROT_EN_2_STA), + }, + .caps = MTK_SCPD_KEEP_DEFAULT_OFF | MTK_SCPD_ACTIVE_WAKEUP, + }, + [MT8188_POWER_DOMAIN_AUDIO_ASRC] = { + .name = "audio_asrc", + .sta_mask = BIT(7), + .ctl_offs = 0x350, + .pwr_sta_offs = 0x16C, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = BIT(8), + .sram_pdn_ack_bits = BIT(12), + .bp_infracfg = { + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_2_AUDIO_ASRC_STEP1, + MT8188_TOP_AXI_PROT_EN_2_SET, + MT8188_TOP_AXI_PROT_EN_2_CLR, + MT8188_TOP_AXI_PROT_EN_2_STA), + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_2_AUDIO_ASRC_STEP2, + MT8188_TOP_AXI_PROT_EN_2_SET, + MT8188_TOP_AXI_PROT_EN_2_CLR, + MT8188_TOP_AXI_PROT_EN_2_STA), + }, + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8188_POWER_DOMAIN_VPPSYS0] = { + .name = "vppsys0", + .sta_mask = BIT(11), + .ctl_offs = 0x360, + .pwr_sta_offs = 0x16C, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = BIT(8), + .sram_pdn_ack_bits = BIT(12), + .bp_infracfg = { + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_VPPSYS0_STEP1, + MT8188_TOP_AXI_PROT_EN_SET, + MT8188_TOP_AXI_PROT_EN_CLR, + MT8188_TOP_AXI_PROT_EN_STA), + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_2_VPPSYS0_STEP2, + MT8188_TOP_AXI_PROT_EN_MM_2_SET, + MT8188_TOP_AXI_PROT_EN_MM_2_CLR, + MT8188_TOP_AXI_PROT_EN_MM_2_STA), + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_VPPSYS0_STEP3, + MT8188_TOP_AXI_PROT_EN_SET, + MT8188_TOP_AXI_PROT_EN_CLR, + MT8188_TOP_AXI_PROT_EN_STA), + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_2_VPPSYS0_STEP4, + MT8188_TOP_AXI_PROT_EN_MM_2_SET, + MT8188_TOP_AXI_PROT_EN_MM_2_CLR, + MT8188_TOP_AXI_PROT_EN_MM_2_STA), + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_VPPSYS0_STEP5, + MT8188_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_SET, + MT8188_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_CLR, + MT8188_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_STA), + }, + }, + [MT8188_POWER_DOMAIN_VDOSYS0] = { + .name = "vdosys0", + .sta_mask = BIT(13), + .ctl_offs = 0x368, + .pwr_sta_offs = 0x16C, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = BIT(8), + .sram_pdn_ack_bits = BIT(12), + .bp_infracfg = { + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_VDOSYS0_STEP1, + MT8188_TOP_AXI_PROT_EN_MM_SET, + MT8188_TOP_AXI_PROT_EN_MM_CLR, + MT8188_TOP_AXI_PROT_EN_MM_STA), + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_VDOSYS0_STEP2, + MT8188_TOP_AXI_PROT_EN_SET, + MT8188_TOP_AXI_PROT_EN_CLR, + MT8188_TOP_AXI_PROT_EN_STA), + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_VDOSYS0_STEP3, + MT8188_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_SET, + MT8188_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_CLR, + MT8188_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_STA), + }, + }, + [MT8188_POWER_DOMAIN_VDOSYS1] = { + .name = "vdosys1", + .sta_mask = BIT(14), + .ctl_offs = 0x36C, + .pwr_sta_offs = 0x16C, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = BIT(8), + .sram_pdn_ack_bits = BIT(12), + .bp_infracfg = { + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_VDOSYS1_STEP1, + MT8188_TOP_AXI_PROT_EN_MM_SET, + MT8188_TOP_AXI_PROT_EN_MM_CLR, + MT8188_TOP_AXI_PROT_EN_MM_STA), + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_VDOSYS1_STEP2, + MT8188_TOP_AXI_PROT_EN_MM_SET, + MT8188_TOP_AXI_PROT_EN_MM_CLR, + MT8188_TOP_AXI_PROT_EN_MM_STA), + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_2_VDOSYS1_STEP3, + MT8188_TOP_AXI_PROT_EN_MM_2_SET, + MT8188_TOP_AXI_PROT_EN_MM_2_CLR, + MT8188_TOP_AXI_PROT_EN_MM_2_STA), + }, + }, + [MT8188_POWER_DOMAIN_DP_TX] = { + .name = "dp_tx", + .sta_mask = BIT(16), + .ctl_offs = 0x374, + .pwr_sta_offs = 0x16C, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = BIT(8), + .sram_pdn_ack_bits = BIT(12), + .bp_infracfg = { + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_INFRA_VDNR_DP_TX_STEP1, + MT8188_TOP_AXI_PROT_EN_INFRA_VDNR_SET, + MT8188_TOP_AXI_PROT_EN_INFRA_VDNR_CLR, + MT8188_TOP_AXI_PROT_EN_INFRA_VDNR_STA), + }, + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8188_POWER_DOMAIN_EDP_TX] = { + .name = "edp_tx", + .sta_mask = BIT(17), + .ctl_offs = 0x378, + .pwr_sta_offs = 0x16C, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = BIT(8), + .sram_pdn_ack_bits = BIT(12), + .bp_infracfg = { + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_INFRA_VDNR_EDP_TX_STEP1, + MT8188_TOP_AXI_PROT_EN_INFRA_VDNR_SET, + MT8188_TOP_AXI_PROT_EN_INFRA_VDNR_CLR, + MT8188_TOP_AXI_PROT_EN_INFRA_VDNR_STA), + }, + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8188_POWER_DOMAIN_VPPSYS1] = { + .name = "vppsys1", + .sta_mask = BIT(12), + .ctl_offs = 0x364, + .pwr_sta_offs = 0x16C, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = BIT(8), + .sram_pdn_ack_bits = BIT(12), + .bp_infracfg = { + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_VPPSYS1_STEP1, + MT8188_TOP_AXI_PROT_EN_MM_SET, + MT8188_TOP_AXI_PROT_EN_MM_CLR, + MT8188_TOP_AXI_PROT_EN_MM_STA), + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_VPPSYS1_STEP2, + MT8188_TOP_AXI_PROT_EN_MM_SET, + MT8188_TOP_AXI_PROT_EN_MM_CLR, + MT8188_TOP_AXI_PROT_EN_MM_STA), + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_2_VPPSYS1_STEP3, + MT8188_TOP_AXI_PROT_EN_MM_2_SET, + MT8188_TOP_AXI_PROT_EN_MM_2_CLR, + MT8188_TOP_AXI_PROT_EN_MM_2_STA), + }, + }, + [MT8188_POWER_DOMAIN_WPE] = { + .name = "wpe", + .sta_mask = BIT(15), + .ctl_offs = 0x370, + .pwr_sta_offs = 0x16C, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = BIT(8), + .sram_pdn_ack_bits = BIT(12), + .bp_infracfg = { + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_2_WPE_STEP1, + MT8188_TOP_AXI_PROT_EN_MM_2_SET, + MT8188_TOP_AXI_PROT_EN_MM_2_CLR, + MT8188_TOP_AXI_PROT_EN_MM_2_STA), + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_2_WPE_STEP2, + MT8188_TOP_AXI_PROT_EN_MM_2_SET, + MT8188_TOP_AXI_PROT_EN_MM_2_CLR, + MT8188_TOP_AXI_PROT_EN_MM_2_STA), + }, + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8188_POWER_DOMAIN_VDEC0] = { + .name = "vdec0", + .sta_mask = BIT(19), + .ctl_offs = 0x380, + .pwr_sta_offs = 0x16C, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = BIT(8), + .sram_pdn_ack_bits = BIT(12), + .bp_infracfg = { + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_VDEC0_STEP1, + MT8188_TOP_AXI_PROT_EN_MM_SET, + MT8188_TOP_AXI_PROT_EN_MM_CLR, + MT8188_TOP_AXI_PROT_EN_MM_STA), + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_2_VDEC0_STEP2, + MT8188_TOP_AXI_PROT_EN_MM_2_SET, + MT8188_TOP_AXI_PROT_EN_MM_2_CLR, + MT8188_TOP_AXI_PROT_EN_MM_2_STA), + }, + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8188_POWER_DOMAIN_VDEC1] = { + .name = "vdec1", + .sta_mask = BIT(20), + .ctl_offs = 0x384, + .pwr_sta_offs = 0x16C, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = BIT(8), + .sram_pdn_ack_bits = BIT(12), + .bp_infracfg = { + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_VDEC1_STEP1, + MT8188_TOP_AXI_PROT_EN_MM_SET, + MT8188_TOP_AXI_PROT_EN_MM_CLR, + MT8188_TOP_AXI_PROT_EN_MM_STA), + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_VDEC1_STEP2, + MT8188_TOP_AXI_PROT_EN_MM_SET, + MT8188_TOP_AXI_PROT_EN_MM_CLR, + MT8188_TOP_AXI_PROT_EN_MM_STA), + }, + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8188_POWER_DOMAIN_VENC] = { + .name = "venc", + .sta_mask = BIT(22), + .ctl_offs = 0x38C, + .pwr_sta_offs = 0x16C, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = BIT(8), + .sram_pdn_ack_bits = BIT(12), + .bp_infracfg = { + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_VENC_STEP1, + MT8188_TOP_AXI_PROT_EN_MM_SET, + MT8188_TOP_AXI_PROT_EN_MM_CLR, + MT8188_TOP_AXI_PROT_EN_MM_STA), + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_VENC_STEP2, + MT8188_TOP_AXI_PROT_EN_MM_SET, + MT8188_TOP_AXI_PROT_EN_MM_CLR, + MT8188_TOP_AXI_PROT_EN_MM_STA), + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_2_VENC_STEP3, + MT8188_TOP_AXI_PROT_EN_MM_2_SET, + MT8188_TOP_AXI_PROT_EN_MM_2_CLR, + MT8188_TOP_AXI_PROT_EN_MM_2_STA), + }, + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8188_POWER_DOMAIN_IMG_VCORE] = { + .name = "vcore", + .sta_mask = BIT(28), + .ctl_offs = 0x3A4, + .pwr_sta_offs = 0x16C, + .pwr_sta2nd_offs = 0x170, + .bp_infracfg = { + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_IMG_VCORE_STEP1, + MT8188_TOP_AXI_PROT_EN_MM_SET, + MT8188_TOP_AXI_PROT_EN_MM_CLR, + MT8188_TOP_AXI_PROT_EN_MM_STA), + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_IMG_VCORE_STEP2, + MT8188_TOP_AXI_PROT_EN_MM_SET, + MT8188_TOP_AXI_PROT_EN_MM_CLR, + MT8188_TOP_AXI_PROT_EN_MM_STA), + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_2_IMG_VCORE_STEP3, + MT8188_TOP_AXI_PROT_EN_MM_2_SET, + MT8188_TOP_AXI_PROT_EN_MM_2_CLR, + MT8188_TOP_AXI_PROT_EN_MM_2_STA), + }, + .caps = MTK_SCPD_KEEP_DEFAULT_OFF | MTK_SCPD_DOMAIN_SUPPLY, + }, + [MT8188_POWER_DOMAIN_IMG_MAIN] = { + .name = "img_main", + .sta_mask = BIT(29), + .ctl_offs = 0x3A8, + .pwr_sta_offs = 0x16C, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = BIT(8), + .sram_pdn_ack_bits = BIT(12), + .bp_infracfg = { + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_2_IMG_MAIN_STEP1, + MT8188_TOP_AXI_PROT_EN_MM_2_SET, + MT8188_TOP_AXI_PROT_EN_MM_2_CLR, + MT8188_TOP_AXI_PROT_EN_MM_2_STA), + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_2_IMG_MAIN_STEP2, + MT8188_TOP_AXI_PROT_EN_MM_2_SET, + MT8188_TOP_AXI_PROT_EN_MM_2_CLR, + MT8188_TOP_AXI_PROT_EN_MM_2_STA), + }, + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8188_POWER_DOMAIN_DIP] = { + .name = "dip", + .sta_mask = BIT(30), + .ctl_offs = 0x3AC, + .pwr_sta_offs = 0x16C, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = BIT(8), + .sram_pdn_ack_bits = BIT(12), + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8188_POWER_DOMAIN_IPE] = { + .name = "ipe", + .sta_mask = BIT(31), + .ctl_offs = 0x3B0, + .pwr_sta_offs = 0x16C, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = BIT(8), + .sram_pdn_ack_bits = BIT(12), + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8188_POWER_DOMAIN_CAM_VCORE] = { + .name = "cam_vcore", + .sta_mask = BIT(27), + .ctl_offs = 0x3A0, + .pwr_sta_offs = 0x16C, + .pwr_sta2nd_offs = 0x170, + .bp_infracfg = { + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_CAM_VCORE_STEP1, + MT8188_TOP_AXI_PROT_EN_MM_SET, + MT8188_TOP_AXI_PROT_EN_MM_CLR, + MT8188_TOP_AXI_PROT_EN_MM_STA), + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_2_CAM_VCORE_STEP2, + MT8188_TOP_AXI_PROT_EN_2_SET, + MT8188_TOP_AXI_PROT_EN_2_CLR, + MT8188_TOP_AXI_PROT_EN_2_STA), + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_1_CAM_VCORE_STEP3, + MT8188_TOP_AXI_PROT_EN_1_SET, + MT8188_TOP_AXI_PROT_EN_1_CLR, + MT8188_TOP_AXI_PROT_EN_1_STA), + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_CAM_VCORE_STEP4, + MT8188_TOP_AXI_PROT_EN_MM_SET, + MT8188_TOP_AXI_PROT_EN_MM_CLR, + MT8188_TOP_AXI_PROT_EN_MM_STA), + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_2_CAM_VCORE_STEP5, + MT8188_TOP_AXI_PROT_EN_MM_2_SET, + MT8188_TOP_AXI_PROT_EN_MM_2_CLR, + MT8188_TOP_AXI_PROT_EN_MM_2_STA), + }, + .caps = MTK_SCPD_KEEP_DEFAULT_OFF | MTK_SCPD_DOMAIN_SUPPLY, + }, + [MT8188_POWER_DOMAIN_CAM_MAIN] = { + .name = "cam_main", + .sta_mask = BIT(24), + .ctl_offs = 0x394, + .pwr_sta_offs = 0x16C, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = BIT(8), + .sram_pdn_ack_bits = BIT(12), + .bp_infracfg = { + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_2_CAM_MAIN_STEP1, + MT8188_TOP_AXI_PROT_EN_MM_2_SET, + MT8188_TOP_AXI_PROT_EN_MM_2_CLR, + MT8188_TOP_AXI_PROT_EN_MM_2_STA), + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_2_CAM_MAIN_STEP2, + MT8188_TOP_AXI_PROT_EN_2_SET, + MT8188_TOP_AXI_PROT_EN_2_CLR, + MT8188_TOP_AXI_PROT_EN_2_STA), + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_MM_2_CAM_MAIN_STEP3, + MT8188_TOP_AXI_PROT_EN_MM_2_SET, + MT8188_TOP_AXI_PROT_EN_MM_2_CLR, + MT8188_TOP_AXI_PROT_EN_MM_2_STA), + BUS_PROT_WR(MT8188_TOP_AXI_PROT_EN_2_CAM_MAIN_STEP4, + MT8188_TOP_AXI_PROT_EN_2_SET, + MT8188_TOP_AXI_PROT_EN_2_CLR, + MT8188_TOP_AXI_PROT_EN_2_STA), + }, + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8188_POWER_DOMAIN_CAM_SUBA] = { + .name = "cam_suba", + .sta_mask = BIT(25), + .ctl_offs = 0x398, + .pwr_sta_offs = 0x16C, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = BIT(8), + .sram_pdn_ack_bits = BIT(12), + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8188_POWER_DOMAIN_CAM_SUBB] = { + .name = "cam_subb", + .sta_mask = BIT(26), + .ctl_offs = 0x39C, + .pwr_sta_offs = 0x16C, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = BIT(8), + .sram_pdn_ack_bits = BIT(12), + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, +}; + +static const struct scpsys_soc_data mt8188_scpsys_data = { + .domains_data = scpsys_domain_data_mt8188, + .num_domains = ARRAY_SIZE(scpsys_domain_data_mt8188), +}; + +#endif /* __SOC_MEDIATEK_MT8188_PM_DOMAINS_H */ diff --git a/drivers/pmdomain/mediatek/mt8192-pm-domains.h b/drivers/pmdomain/mediatek/mt8192-pm-domains.h new file mode 100644 index 000000000000..b97b2051920f --- /dev/null +++ b/drivers/pmdomain/mediatek/mt8192-pm-domains.h @@ -0,0 +1,355 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ + +#ifndef __SOC_MEDIATEK_MT8192_PM_DOMAINS_H +#define __SOC_MEDIATEK_MT8192_PM_DOMAINS_H + +#include "mtk-pm-domains.h" +#include + +/* + * MT8192 power domain support + */ + +static const struct scpsys_domain_data scpsys_domain_data_mt8192[] = { + [MT8192_POWER_DOMAIN_AUDIO] = { + .name = "audio", + .sta_mask = BIT(21), + .ctl_offs = 0x0354, + .pwr_sta_offs = 0x016c, + .pwr_sta2nd_offs = 0x0170, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .bp_infracfg = { + BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_2_AUDIO, + MT8192_TOP_AXI_PROT_EN_2_SET, + MT8192_TOP_AXI_PROT_EN_2_CLR, + MT8192_TOP_AXI_PROT_EN_2_STA1), + }, + }, + [MT8192_POWER_DOMAIN_CONN] = { + .name = "conn", + .sta_mask = PWR_STATUS_CONN, + .ctl_offs = 0x0304, + .pwr_sta_offs = 0x016c, + .pwr_sta2nd_offs = 0x0170, + .sram_pdn_bits = 0, + .sram_pdn_ack_bits = 0, + .bp_infracfg = { + BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_CONN, + MT8192_TOP_AXI_PROT_EN_SET, + MT8192_TOP_AXI_PROT_EN_CLR, + MT8192_TOP_AXI_PROT_EN_STA1), + BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_CONN_2ND, + MT8192_TOP_AXI_PROT_EN_SET, + MT8192_TOP_AXI_PROT_EN_CLR, + MT8192_TOP_AXI_PROT_EN_STA1), + BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_1_CONN, + MT8192_TOP_AXI_PROT_EN_1_SET, + MT8192_TOP_AXI_PROT_EN_1_CLR, + MT8192_TOP_AXI_PROT_EN_1_STA1), + }, + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8192_POWER_DOMAIN_MFG0] = { + .name = "mfg0", + .sta_mask = BIT(2), + .ctl_offs = 0x0308, + .pwr_sta_offs = 0x016c, + .pwr_sta2nd_offs = 0x0170, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .caps = MTK_SCPD_DOMAIN_SUPPLY, + }, + [MT8192_POWER_DOMAIN_MFG1] = { + .name = "mfg1", + .sta_mask = BIT(3), + .ctl_offs = 0x030c, + .pwr_sta_offs = 0x016c, + .pwr_sta2nd_offs = 0x0170, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .bp_infracfg = { + BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_1_MFG1, + MT8192_TOP_AXI_PROT_EN_1_SET, + MT8192_TOP_AXI_PROT_EN_1_CLR, + MT8192_TOP_AXI_PROT_EN_1_STA1), + BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_2_MFG1, + MT8192_TOP_AXI_PROT_EN_2_SET, + MT8192_TOP_AXI_PROT_EN_2_CLR, + MT8192_TOP_AXI_PROT_EN_2_STA1), + BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_MFG1, + MT8192_TOP_AXI_PROT_EN_SET, + MT8192_TOP_AXI_PROT_EN_CLR, + MT8192_TOP_AXI_PROT_EN_STA1), + BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_2_MFG1_2ND, + MT8192_TOP_AXI_PROT_EN_2_SET, + MT8192_TOP_AXI_PROT_EN_2_CLR, + MT8192_TOP_AXI_PROT_EN_2_STA1), + }, + .caps = MTK_SCPD_DOMAIN_SUPPLY, + }, + [MT8192_POWER_DOMAIN_MFG2] = { + .name = "mfg2", + .sta_mask = BIT(4), + .ctl_offs = 0x0310, + .pwr_sta_offs = 0x016c, + .pwr_sta2nd_offs = 0x0170, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + }, + [MT8192_POWER_DOMAIN_MFG3] = { + .name = "mfg3", + .sta_mask = BIT(5), + .ctl_offs = 0x0314, + .pwr_sta_offs = 0x016c, + .pwr_sta2nd_offs = 0x0170, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + }, + [MT8192_POWER_DOMAIN_MFG4] = { + .name = "mfg4", + .sta_mask = BIT(6), + .ctl_offs = 0x0318, + .pwr_sta_offs = 0x016c, + .pwr_sta2nd_offs = 0x0170, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + }, + [MT8192_POWER_DOMAIN_MFG5] = { + .name = "mfg5", + .sta_mask = BIT(7), + .ctl_offs = 0x031c, + .pwr_sta_offs = 0x016c, + .pwr_sta2nd_offs = 0x0170, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + }, + [MT8192_POWER_DOMAIN_MFG6] = { + .name = "mfg6", + .sta_mask = BIT(8), + .ctl_offs = 0x0320, + .pwr_sta_offs = 0x016c, + .pwr_sta2nd_offs = 0x0170, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + }, + [MT8192_POWER_DOMAIN_DISP] = { + .name = "disp", + .sta_mask = BIT(20), + .ctl_offs = 0x0350, + .pwr_sta_offs = 0x016c, + .pwr_sta2nd_offs = 0x0170, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .bp_infracfg = { + BUS_PROT_WR_IGN(MT8192_TOP_AXI_PROT_EN_MM_DISP, + MT8192_TOP_AXI_PROT_EN_MM_SET, + MT8192_TOP_AXI_PROT_EN_MM_CLR, + MT8192_TOP_AXI_PROT_EN_MM_STA1), + BUS_PROT_WR_IGN(MT8192_TOP_AXI_PROT_EN_MM_2_DISP, + MT8192_TOP_AXI_PROT_EN_MM_2_SET, + MT8192_TOP_AXI_PROT_EN_MM_2_CLR, + MT8192_TOP_AXI_PROT_EN_MM_2_STA1), + BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_DISP, + MT8192_TOP_AXI_PROT_EN_SET, + MT8192_TOP_AXI_PROT_EN_CLR, + MT8192_TOP_AXI_PROT_EN_STA1), + BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_MM_DISP_2ND, + MT8192_TOP_AXI_PROT_EN_MM_SET, + MT8192_TOP_AXI_PROT_EN_MM_CLR, + MT8192_TOP_AXI_PROT_EN_MM_STA1), + BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_MM_2_DISP_2ND, + MT8192_TOP_AXI_PROT_EN_MM_2_SET, + MT8192_TOP_AXI_PROT_EN_MM_2_CLR, + MT8192_TOP_AXI_PROT_EN_MM_2_STA1), + }, + }, + [MT8192_POWER_DOMAIN_IPE] = { + .name = "ipe", + .sta_mask = BIT(14), + .ctl_offs = 0x0338, + .pwr_sta_offs = 0x016c, + .pwr_sta2nd_offs = 0x0170, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .bp_infracfg = { + BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_MM_IPE, + MT8192_TOP_AXI_PROT_EN_MM_SET, + MT8192_TOP_AXI_PROT_EN_MM_CLR, + MT8192_TOP_AXI_PROT_EN_MM_STA1), + BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_MM_IPE_2ND, + MT8192_TOP_AXI_PROT_EN_MM_SET, + MT8192_TOP_AXI_PROT_EN_MM_CLR, + MT8192_TOP_AXI_PROT_EN_MM_STA1), + }, + }, + [MT8192_POWER_DOMAIN_ISP] = { + .name = "isp", + .sta_mask = BIT(12), + .ctl_offs = 0x0330, + .pwr_sta_offs = 0x016c, + .pwr_sta2nd_offs = 0x0170, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .bp_infracfg = { + BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_MM_2_ISP, + MT8192_TOP_AXI_PROT_EN_MM_2_SET, + MT8192_TOP_AXI_PROT_EN_MM_2_CLR, + MT8192_TOP_AXI_PROT_EN_MM_2_STA1), + BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_MM_2_ISP_2ND, + MT8192_TOP_AXI_PROT_EN_MM_2_SET, + MT8192_TOP_AXI_PROT_EN_MM_2_CLR, + MT8192_TOP_AXI_PROT_EN_MM_2_STA1), + }, + }, + [MT8192_POWER_DOMAIN_ISP2] = { + .name = "isp2", + .sta_mask = BIT(13), + .ctl_offs = 0x0334, + .pwr_sta_offs = 0x016c, + .pwr_sta2nd_offs = 0x0170, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .bp_infracfg = { + BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_MM_ISP2, + MT8192_TOP_AXI_PROT_EN_MM_SET, + MT8192_TOP_AXI_PROT_EN_MM_CLR, + MT8192_TOP_AXI_PROT_EN_MM_STA1), + BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_MM_ISP2_2ND, + MT8192_TOP_AXI_PROT_EN_MM_SET, + MT8192_TOP_AXI_PROT_EN_MM_CLR, + MT8192_TOP_AXI_PROT_EN_MM_STA1), + }, + }, + [MT8192_POWER_DOMAIN_MDP] = { + .name = "mdp", + .sta_mask = BIT(19), + .ctl_offs = 0x034c, + .pwr_sta_offs = 0x016c, + .pwr_sta2nd_offs = 0x0170, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .bp_infracfg = { + BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_MM_2_MDP, + MT8192_TOP_AXI_PROT_EN_MM_2_SET, + MT8192_TOP_AXI_PROT_EN_MM_2_CLR, + MT8192_TOP_AXI_PROT_EN_MM_2_STA1), + BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_MM_2_MDP_2ND, + MT8192_TOP_AXI_PROT_EN_MM_2_SET, + MT8192_TOP_AXI_PROT_EN_MM_2_CLR, + MT8192_TOP_AXI_PROT_EN_MM_2_STA1), + }, + }, + [MT8192_POWER_DOMAIN_VENC] = { + .name = "venc", + .sta_mask = BIT(17), + .ctl_offs = 0x0344, + .pwr_sta_offs = 0x016c, + .pwr_sta2nd_offs = 0x0170, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .bp_infracfg = { + BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_MM_VENC, + MT8192_TOP_AXI_PROT_EN_MM_SET, + MT8192_TOP_AXI_PROT_EN_MM_CLR, + MT8192_TOP_AXI_PROT_EN_MM_STA1), + BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_MM_VENC_2ND, + MT8192_TOP_AXI_PROT_EN_MM_SET, + MT8192_TOP_AXI_PROT_EN_MM_CLR, + MT8192_TOP_AXI_PROT_EN_MM_STA1), + }, + }, + [MT8192_POWER_DOMAIN_VDEC] = { + .name = "vdec", + .sta_mask = BIT(15), + .ctl_offs = 0x033c, + .pwr_sta_offs = 0x016c, + .pwr_sta2nd_offs = 0x0170, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .bp_infracfg = { + BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_MM_VDEC, + MT8192_TOP_AXI_PROT_EN_MM_SET, + MT8192_TOP_AXI_PROT_EN_MM_CLR, + MT8192_TOP_AXI_PROT_EN_MM_STA1), + BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_MM_VDEC_2ND, + MT8192_TOP_AXI_PROT_EN_MM_SET, + MT8192_TOP_AXI_PROT_EN_MM_CLR, + MT8192_TOP_AXI_PROT_EN_MM_STA1), + }, + }, + [MT8192_POWER_DOMAIN_VDEC2] = { + .name = "vdec2", + .sta_mask = BIT(16), + .ctl_offs = 0x0340, + .pwr_sta_offs = 0x016c, + .pwr_sta2nd_offs = 0x0170, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + }, + [MT8192_POWER_DOMAIN_CAM] = { + .name = "cam", + .sta_mask = BIT(23), + .ctl_offs = 0x035c, + .pwr_sta_offs = 0x016c, + .pwr_sta2nd_offs = 0x0170, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .bp_infracfg = { + BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_2_CAM, + MT8192_TOP_AXI_PROT_EN_2_SET, + MT8192_TOP_AXI_PROT_EN_2_CLR, + MT8192_TOP_AXI_PROT_EN_2_STA1), + BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_MM_CAM, + MT8192_TOP_AXI_PROT_EN_MM_SET, + MT8192_TOP_AXI_PROT_EN_MM_CLR, + MT8192_TOP_AXI_PROT_EN_MM_STA1), + BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_1_CAM, + MT8192_TOP_AXI_PROT_EN_1_SET, + MT8192_TOP_AXI_PROT_EN_1_CLR, + MT8192_TOP_AXI_PROT_EN_1_STA1), + BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_MM_CAM_2ND, + MT8192_TOP_AXI_PROT_EN_MM_SET, + MT8192_TOP_AXI_PROT_EN_MM_CLR, + MT8192_TOP_AXI_PROT_EN_MM_STA1), + BUS_PROT_WR(MT8192_TOP_AXI_PROT_EN_VDNR_CAM, + MT8192_TOP_AXI_PROT_EN_VDNR_SET, + MT8192_TOP_AXI_PROT_EN_VDNR_CLR, + MT8192_TOP_AXI_PROT_EN_VDNR_STA1), + }, + }, + [MT8192_POWER_DOMAIN_CAM_RAWA] = { + .name = "cam_rawa", + .sta_mask = BIT(24), + .ctl_offs = 0x0360, + .pwr_sta_offs = 0x016c, + .pwr_sta2nd_offs = 0x0170, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + }, + [MT8192_POWER_DOMAIN_CAM_RAWB] = { + .name = "cam_rawb", + .sta_mask = BIT(25), + .ctl_offs = 0x0364, + .pwr_sta_offs = 0x016c, + .pwr_sta2nd_offs = 0x0170, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + }, + [MT8192_POWER_DOMAIN_CAM_RAWC] = { + .name = "cam_rawc", + .sta_mask = BIT(26), + .ctl_offs = 0x0368, + .pwr_sta_offs = 0x016c, + .pwr_sta2nd_offs = 0x0170, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + }, +}; + +static const struct scpsys_soc_data mt8192_scpsys_data = { + .domains_data = scpsys_domain_data_mt8192, + .num_domains = ARRAY_SIZE(scpsys_domain_data_mt8192), +}; + +#endif /* __SOC_MEDIATEK_MT8192_PM_DOMAINS_H */ diff --git a/drivers/pmdomain/mediatek/mt8195-pm-domains.h b/drivers/pmdomain/mediatek/mt8195-pm-domains.h new file mode 100644 index 000000000000..d7387ea1b9c9 --- /dev/null +++ b/drivers/pmdomain/mediatek/mt8195-pm-domains.h @@ -0,0 +1,613 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2021 MediaTek Inc. + * Author: Chun-Jie Chen + */ + +#ifndef __SOC_MEDIATEK_MT8195_PM_DOMAINS_H +#define __SOC_MEDIATEK_MT8195_PM_DOMAINS_H + +#include "mtk-pm-domains.h" +#include + +/* + * MT8195 power domain support + */ + +static const struct scpsys_domain_data scpsys_domain_data_mt8195[] = { + [MT8195_POWER_DOMAIN_PCIE_MAC_P0] = { + .name = "pcie_mac_p0", + .sta_mask = BIT(11), + .ctl_offs = 0x328, + .pwr_sta_offs = 0x174, + .pwr_sta2nd_offs = 0x178, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .bp_infracfg = { + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_VDNR_PCIE_MAC_P0, + MT8195_TOP_AXI_PROT_EN_VDNR_SET, + MT8195_TOP_AXI_PROT_EN_VDNR_CLR, + MT8195_TOP_AXI_PROT_EN_VDNR_STA1), + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_VDNR_1_PCIE_MAC_P0, + MT8195_TOP_AXI_PROT_EN_VDNR_1_SET, + MT8195_TOP_AXI_PROT_EN_VDNR_1_CLR, + MT8195_TOP_AXI_PROT_EN_VDNR_1_STA1), + }, + }, + [MT8195_POWER_DOMAIN_PCIE_MAC_P1] = { + .name = "pcie_mac_p1", + .sta_mask = BIT(12), + .ctl_offs = 0x32C, + .pwr_sta_offs = 0x174, + .pwr_sta2nd_offs = 0x178, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .bp_infracfg = { + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_VDNR_PCIE_MAC_P1, + MT8195_TOP_AXI_PROT_EN_VDNR_SET, + MT8195_TOP_AXI_PROT_EN_VDNR_CLR, + MT8195_TOP_AXI_PROT_EN_VDNR_STA1), + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_VDNR_1_PCIE_MAC_P1, + MT8195_TOP_AXI_PROT_EN_VDNR_1_SET, + MT8195_TOP_AXI_PROT_EN_VDNR_1_CLR, + MT8195_TOP_AXI_PROT_EN_VDNR_1_STA1), + }, + }, + [MT8195_POWER_DOMAIN_PCIE_PHY] = { + .name = "pcie_phy", + .sta_mask = BIT(13), + .ctl_offs = 0x330, + .pwr_sta_offs = 0x174, + .pwr_sta2nd_offs = 0x178, + .caps = MTK_SCPD_ACTIVE_WAKEUP, + }, + [MT8195_POWER_DOMAIN_SSUSB_PCIE_PHY] = { + .name = "ssusb_pcie_phy", + .sta_mask = BIT(14), + .ctl_offs = 0x334, + .pwr_sta_offs = 0x174, + .pwr_sta2nd_offs = 0x178, + .caps = MTK_SCPD_ACTIVE_WAKEUP | MTK_SCPD_ALWAYS_ON, + }, + [MT8195_POWER_DOMAIN_CSI_RX_TOP] = { + .name = "csi_rx_top", + .sta_mask = BIT(18), + .ctl_offs = 0x3C4, + .pwr_sta_offs = 0x174, + .pwr_sta2nd_offs = 0x178, + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8195_POWER_DOMAIN_ETHER] = { + .name = "ether", + .sta_mask = BIT(3), + .ctl_offs = 0x344, + .pwr_sta_offs = 0x16c, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .caps = MTK_SCPD_ACTIVE_WAKEUP, + }, + [MT8195_POWER_DOMAIN_ADSP] = { + .name = "adsp", + .sta_mask = BIT(10), + .ctl_offs = 0x360, + .pwr_sta_offs = 0x16c, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .bp_infracfg = { + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_2_ADSP, + MT8195_TOP_AXI_PROT_EN_2_SET, + MT8195_TOP_AXI_PROT_EN_2_CLR, + MT8195_TOP_AXI_PROT_EN_2_STA1), + }, + .caps = MTK_SCPD_SRAM_ISO | MTK_SCPD_ACTIVE_WAKEUP, + }, + [MT8195_POWER_DOMAIN_AUDIO] = { + .name = "audio", + .sta_mask = BIT(8), + .ctl_offs = 0x358, + .pwr_sta_offs = 0x16c, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .bp_infracfg = { + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_2_AUDIO, + MT8195_TOP_AXI_PROT_EN_2_SET, + MT8195_TOP_AXI_PROT_EN_2_CLR, + MT8195_TOP_AXI_PROT_EN_2_STA1), + }, + }, + [MT8195_POWER_DOMAIN_MFG0] = { + .name = "mfg0", + .sta_mask = BIT(1), + .ctl_offs = 0x300, + .pwr_sta_offs = 0x174, + .pwr_sta2nd_offs = 0x178, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .caps = MTK_SCPD_KEEP_DEFAULT_OFF | MTK_SCPD_DOMAIN_SUPPLY, + }, + [MT8195_POWER_DOMAIN_MFG1] = { + .name = "mfg1", + .sta_mask = BIT(2), + .ctl_offs = 0x304, + .pwr_sta_offs = 0x174, + .pwr_sta2nd_offs = 0x178, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .bp_infracfg = { + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MFG1, + MT8195_TOP_AXI_PROT_EN_SET, + MT8195_TOP_AXI_PROT_EN_CLR, + MT8195_TOP_AXI_PROT_EN_STA1), + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_2_MFG1, + MT8195_TOP_AXI_PROT_EN_2_SET, + MT8195_TOP_AXI_PROT_EN_2_CLR, + MT8195_TOP_AXI_PROT_EN_2_STA1), + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_1_MFG1, + MT8195_TOP_AXI_PROT_EN_1_SET, + MT8195_TOP_AXI_PROT_EN_1_CLR, + MT8195_TOP_AXI_PROT_EN_1_STA1), + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_2_MFG1_2ND, + MT8195_TOP_AXI_PROT_EN_2_SET, + MT8195_TOP_AXI_PROT_EN_2_CLR, + MT8195_TOP_AXI_PROT_EN_2_STA1), + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MFG1_2ND, + MT8195_TOP_AXI_PROT_EN_SET, + MT8195_TOP_AXI_PROT_EN_CLR, + MT8195_TOP_AXI_PROT_EN_STA1), + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_MFG1, + MT8195_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_SET, + MT8195_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_CLR, + MT8195_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_STA1), + }, + .caps = MTK_SCPD_KEEP_DEFAULT_OFF | MTK_SCPD_DOMAIN_SUPPLY, + }, + [MT8195_POWER_DOMAIN_MFG2] = { + .name = "mfg2", + .sta_mask = BIT(3), + .ctl_offs = 0x308, + .pwr_sta_offs = 0x174, + .pwr_sta2nd_offs = 0x178, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8195_POWER_DOMAIN_MFG3] = { + .name = "mfg3", + .sta_mask = BIT(4), + .ctl_offs = 0x30C, + .pwr_sta_offs = 0x174, + .pwr_sta2nd_offs = 0x178, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8195_POWER_DOMAIN_MFG4] = { + .name = "mfg4", + .sta_mask = BIT(5), + .ctl_offs = 0x310, + .pwr_sta_offs = 0x174, + .pwr_sta2nd_offs = 0x178, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8195_POWER_DOMAIN_MFG5] = { + .name = "mfg5", + .sta_mask = BIT(6), + .ctl_offs = 0x314, + .pwr_sta_offs = 0x174, + .pwr_sta2nd_offs = 0x178, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8195_POWER_DOMAIN_MFG6] = { + .name = "mfg6", + .sta_mask = BIT(7), + .ctl_offs = 0x318, + .pwr_sta_offs = 0x174, + .pwr_sta2nd_offs = 0x178, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8195_POWER_DOMAIN_VPPSYS0] = { + .name = "vppsys0", + .sta_mask = BIT(11), + .ctl_offs = 0x364, + .pwr_sta_offs = 0x16c, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .bp_infracfg = { + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_VPPSYS0, + MT8195_TOP_AXI_PROT_EN_SET, + MT8195_TOP_AXI_PROT_EN_CLR, + MT8195_TOP_AXI_PROT_EN_STA1), + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_2_VPPSYS0, + MT8195_TOP_AXI_PROT_EN_MM_2_SET, + MT8195_TOP_AXI_PROT_EN_MM_2_CLR, + MT8195_TOP_AXI_PROT_EN_MM_2_STA1), + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_VPPSYS0_2ND, + MT8195_TOP_AXI_PROT_EN_SET, + MT8195_TOP_AXI_PROT_EN_CLR, + MT8195_TOP_AXI_PROT_EN_STA1), + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_2_VPPSYS0_2ND, + MT8195_TOP_AXI_PROT_EN_MM_2_SET, + MT8195_TOP_AXI_PROT_EN_MM_2_CLR, + MT8195_TOP_AXI_PROT_EN_MM_2_STA1), + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_VPPSYS0, + MT8195_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_SET, + MT8195_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_CLR, + MT8195_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_STA1), + }, + }, + [MT8195_POWER_DOMAIN_VDOSYS0] = { + .name = "vdosys0", + .sta_mask = BIT(13), + .ctl_offs = 0x36C, + .pwr_sta_offs = 0x16c, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .bp_infracfg = { + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_VDOSYS0, + MT8195_TOP_AXI_PROT_EN_MM_SET, + MT8195_TOP_AXI_PROT_EN_MM_CLR, + MT8195_TOP_AXI_PROT_EN_MM_STA1), + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_VDOSYS0, + MT8195_TOP_AXI_PROT_EN_SET, + MT8195_TOP_AXI_PROT_EN_CLR, + MT8195_TOP_AXI_PROT_EN_STA1), + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_VDOSYS0, + MT8195_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_SET, + MT8195_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_CLR, + MT8195_TOP_AXI_PROT_EN_SUB_INFRA_VDNR_STA1), + }, + }, + [MT8195_POWER_DOMAIN_VPPSYS1] = { + .name = "vppsys1", + .sta_mask = BIT(12), + .ctl_offs = 0x368, + .pwr_sta_offs = 0x16c, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .bp_infracfg = { + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_VPPSYS1, + MT8195_TOP_AXI_PROT_EN_MM_SET, + MT8195_TOP_AXI_PROT_EN_MM_CLR, + MT8195_TOP_AXI_PROT_EN_MM_STA1), + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_VPPSYS1_2ND, + MT8195_TOP_AXI_PROT_EN_MM_SET, + MT8195_TOP_AXI_PROT_EN_MM_CLR, + MT8195_TOP_AXI_PROT_EN_MM_STA1), + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_2_VPPSYS1, + MT8195_TOP_AXI_PROT_EN_MM_2_SET, + MT8195_TOP_AXI_PROT_EN_MM_2_CLR, + MT8195_TOP_AXI_PROT_EN_MM_2_STA1), + }, + }, + [MT8195_POWER_DOMAIN_VDOSYS1] = { + .name = "vdosys1", + .sta_mask = BIT(14), + .ctl_offs = 0x370, + .pwr_sta_offs = 0x16c, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .bp_infracfg = { + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_VDOSYS1, + MT8195_TOP_AXI_PROT_EN_MM_SET, + MT8195_TOP_AXI_PROT_EN_MM_CLR, + MT8195_TOP_AXI_PROT_EN_MM_STA1), + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_VDOSYS1_2ND, + MT8195_TOP_AXI_PROT_EN_MM_SET, + MT8195_TOP_AXI_PROT_EN_MM_CLR, + MT8195_TOP_AXI_PROT_EN_MM_STA1), + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_2_VDOSYS1, + MT8195_TOP_AXI_PROT_EN_MM_2_SET, + MT8195_TOP_AXI_PROT_EN_MM_2_CLR, + MT8195_TOP_AXI_PROT_EN_MM_2_STA1), + }, + }, + [MT8195_POWER_DOMAIN_DP_TX] = { + .name = "dp_tx", + .sta_mask = BIT(16), + .ctl_offs = 0x378, + .pwr_sta_offs = 0x16c, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .bp_infracfg = { + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_VDNR_1_DP_TX, + MT8195_TOP_AXI_PROT_EN_VDNR_1_SET, + MT8195_TOP_AXI_PROT_EN_VDNR_1_CLR, + MT8195_TOP_AXI_PROT_EN_VDNR_1_STA1), + }, + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8195_POWER_DOMAIN_EPD_TX] = { + .name = "epd_tx", + .sta_mask = BIT(17), + .ctl_offs = 0x37C, + .pwr_sta_offs = 0x16c, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .bp_infracfg = { + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_VDNR_1_EPD_TX, + MT8195_TOP_AXI_PROT_EN_VDNR_1_SET, + MT8195_TOP_AXI_PROT_EN_VDNR_1_CLR, + MT8195_TOP_AXI_PROT_EN_VDNR_1_STA1), + }, + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8195_POWER_DOMAIN_HDMI_TX] = { + .name = "hdmi_tx", + .sta_mask = BIT(18), + .ctl_offs = 0x380, + .pwr_sta_offs = 0x16c, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .caps = MTK_SCPD_KEEP_DEFAULT_OFF | MTK_SCPD_ACTIVE_WAKEUP, + }, + [MT8195_POWER_DOMAIN_WPESYS] = { + .name = "wpesys", + .sta_mask = BIT(15), + .ctl_offs = 0x374, + .pwr_sta_offs = 0x16c, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .bp_infracfg = { + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_2_WPESYS, + MT8195_TOP_AXI_PROT_EN_MM_2_SET, + MT8195_TOP_AXI_PROT_EN_MM_2_CLR, + MT8195_TOP_AXI_PROT_EN_MM_2_STA1), + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_WPESYS, + MT8195_TOP_AXI_PROT_EN_MM_SET, + MT8195_TOP_AXI_PROT_EN_MM_CLR, + MT8195_TOP_AXI_PROT_EN_MM_STA1), + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_2_WPESYS_2ND, + MT8195_TOP_AXI_PROT_EN_MM_2_SET, + MT8195_TOP_AXI_PROT_EN_MM_2_CLR, + MT8195_TOP_AXI_PROT_EN_MM_2_STA1), + }, + }, + [MT8195_POWER_DOMAIN_VDEC0] = { + .name = "vdec0", + .sta_mask = BIT(20), + .ctl_offs = 0x388, + .pwr_sta_offs = 0x16c, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .bp_infracfg = { + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_VDEC0, + MT8195_TOP_AXI_PROT_EN_MM_SET, + MT8195_TOP_AXI_PROT_EN_MM_CLR, + MT8195_TOP_AXI_PROT_EN_MM_STA1), + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_2_VDEC0, + MT8195_TOP_AXI_PROT_EN_MM_2_SET, + MT8195_TOP_AXI_PROT_EN_MM_2_CLR, + MT8195_TOP_AXI_PROT_EN_MM_2_STA1), + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_VDEC0_2ND, + MT8195_TOP_AXI_PROT_EN_MM_SET, + MT8195_TOP_AXI_PROT_EN_MM_CLR, + MT8195_TOP_AXI_PROT_EN_MM_STA1), + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_2_VDEC0_2ND, + MT8195_TOP_AXI_PROT_EN_MM_2_SET, + MT8195_TOP_AXI_PROT_EN_MM_2_CLR, + MT8195_TOP_AXI_PROT_EN_MM_2_STA1), + }, + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8195_POWER_DOMAIN_VDEC1] = { + .name = "vdec1", + .sta_mask = BIT(21), + .ctl_offs = 0x38C, + .pwr_sta_offs = 0x16c, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .bp_infracfg = { + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_VDEC1, + MT8195_TOP_AXI_PROT_EN_MM_SET, + MT8195_TOP_AXI_PROT_EN_MM_CLR, + MT8195_TOP_AXI_PROT_EN_MM_STA1), + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_VDEC1_2ND, + MT8195_TOP_AXI_PROT_EN_MM_SET, + MT8195_TOP_AXI_PROT_EN_MM_CLR, + MT8195_TOP_AXI_PROT_EN_MM_STA1), + }, + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8195_POWER_DOMAIN_VDEC2] = { + .name = "vdec2", + .sta_mask = BIT(22), + .ctl_offs = 0x390, + .pwr_sta_offs = 0x16c, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .bp_infracfg = { + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_2_VDEC2, + MT8195_TOP_AXI_PROT_EN_MM_2_SET, + MT8195_TOP_AXI_PROT_EN_MM_2_CLR, + MT8195_TOP_AXI_PROT_EN_MM_2_STA1), + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_2_VDEC2_2ND, + MT8195_TOP_AXI_PROT_EN_MM_2_SET, + MT8195_TOP_AXI_PROT_EN_MM_2_CLR, + MT8195_TOP_AXI_PROT_EN_MM_2_STA1), + }, + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8195_POWER_DOMAIN_VENC] = { + .name = "venc", + .sta_mask = BIT(23), + .ctl_offs = 0x394, + .pwr_sta_offs = 0x16c, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .bp_infracfg = { + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_VENC, + MT8195_TOP_AXI_PROT_EN_MM_SET, + MT8195_TOP_AXI_PROT_EN_MM_CLR, + MT8195_TOP_AXI_PROT_EN_MM_STA1), + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_VENC_2ND, + MT8195_TOP_AXI_PROT_EN_MM_SET, + MT8195_TOP_AXI_PROT_EN_MM_CLR, + MT8195_TOP_AXI_PROT_EN_MM_STA1), + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_2_VENC, + MT8195_TOP_AXI_PROT_EN_MM_2_SET, + MT8195_TOP_AXI_PROT_EN_MM_2_CLR, + MT8195_TOP_AXI_PROT_EN_MM_2_STA1), + }, + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8195_POWER_DOMAIN_VENC_CORE1] = { + .name = "venc_core1", + .sta_mask = BIT(24), + .ctl_offs = 0x398, + .pwr_sta_offs = 0x16c, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .bp_infracfg = { + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_VENC_CORE1, + MT8195_TOP_AXI_PROT_EN_MM_SET, + MT8195_TOP_AXI_PROT_EN_MM_CLR, + MT8195_TOP_AXI_PROT_EN_MM_STA1), + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_2_VENC_CORE1, + MT8195_TOP_AXI_PROT_EN_MM_2_SET, + MT8195_TOP_AXI_PROT_EN_MM_2_CLR, + MT8195_TOP_AXI_PROT_EN_MM_2_STA1), + }, + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8195_POWER_DOMAIN_IMG] = { + .name = "img", + .sta_mask = BIT(29), + .ctl_offs = 0x3AC, + .pwr_sta_offs = 0x16c, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .bp_infracfg = { + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_IMG, + MT8195_TOP_AXI_PROT_EN_MM_SET, + MT8195_TOP_AXI_PROT_EN_MM_CLR, + MT8195_TOP_AXI_PROT_EN_MM_STA1), + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_IMG_2ND, + MT8195_TOP_AXI_PROT_EN_MM_SET, + MT8195_TOP_AXI_PROT_EN_MM_CLR, + MT8195_TOP_AXI_PROT_EN_MM_STA1), + }, + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8195_POWER_DOMAIN_DIP] = { + .name = "dip", + .sta_mask = BIT(30), + .ctl_offs = 0x3B0, + .pwr_sta_offs = 0x16c, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8195_POWER_DOMAIN_IPE] = { + .name = "ipe", + .sta_mask = BIT(31), + .ctl_offs = 0x3B4, + .pwr_sta_offs = 0x16c, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .bp_infracfg = { + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_IPE, + MT8195_TOP_AXI_PROT_EN_MM_SET, + MT8195_TOP_AXI_PROT_EN_MM_CLR, + MT8195_TOP_AXI_PROT_EN_MM_STA1), + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_2_IPE, + MT8195_TOP_AXI_PROT_EN_MM_2_SET, + MT8195_TOP_AXI_PROT_EN_MM_2_CLR, + MT8195_TOP_AXI_PROT_EN_MM_2_STA1), + }, + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8195_POWER_DOMAIN_CAM] = { + .name = "cam", + .sta_mask = BIT(25), + .ctl_offs = 0x39C, + .pwr_sta_offs = 0x16c, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .bp_infracfg = { + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_2_CAM, + MT8195_TOP_AXI_PROT_EN_2_SET, + MT8195_TOP_AXI_PROT_EN_2_CLR, + MT8195_TOP_AXI_PROT_EN_2_STA1), + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_CAM, + MT8195_TOP_AXI_PROT_EN_MM_SET, + MT8195_TOP_AXI_PROT_EN_MM_CLR, + MT8195_TOP_AXI_PROT_EN_MM_STA1), + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_1_CAM, + MT8195_TOP_AXI_PROT_EN_1_SET, + MT8195_TOP_AXI_PROT_EN_1_CLR, + MT8195_TOP_AXI_PROT_EN_1_STA1), + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_CAM_2ND, + MT8195_TOP_AXI_PROT_EN_MM_SET, + MT8195_TOP_AXI_PROT_EN_MM_CLR, + MT8195_TOP_AXI_PROT_EN_MM_STA1), + BUS_PROT_WR(MT8195_TOP_AXI_PROT_EN_MM_2_CAM, + MT8195_TOP_AXI_PROT_EN_MM_2_SET, + MT8195_TOP_AXI_PROT_EN_MM_2_CLR, + MT8195_TOP_AXI_PROT_EN_MM_2_STA1), + }, + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8195_POWER_DOMAIN_CAM_RAWA] = { + .name = "cam_rawa", + .sta_mask = BIT(26), + .ctl_offs = 0x3A0, + .pwr_sta_offs = 0x16c, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8195_POWER_DOMAIN_CAM_RAWB] = { + .name = "cam_rawb", + .sta_mask = BIT(27), + .ctl_offs = 0x3A4, + .pwr_sta_offs = 0x16c, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, + [MT8195_POWER_DOMAIN_CAM_MRAW] = { + .name = "cam_mraw", + .sta_mask = BIT(28), + .ctl_offs = 0x3A8, + .pwr_sta_offs = 0x16c, + .pwr_sta2nd_offs = 0x170, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .caps = MTK_SCPD_KEEP_DEFAULT_OFF, + }, +}; + +static const struct scpsys_soc_data mt8195_scpsys_data = { + .domains_data = scpsys_domain_data_mt8195, + .num_domains = ARRAY_SIZE(scpsys_domain_data_mt8195), +}; + +#endif /* __SOC_MEDIATEK_MT8195_PM_DOMAINS_H */ diff --git a/drivers/pmdomain/mediatek/mtk-pm-domains.c b/drivers/pmdomain/mediatek/mtk-pm-domains.c new file mode 100644 index 000000000000..ee962804b830 --- /dev/null +++ b/drivers/pmdomain/mediatek/mtk-pm-domains.c @@ -0,0 +1,688 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2020 Collabora Ltd. + */ +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "mt6795-pm-domains.h" +#include "mt8167-pm-domains.h" +#include "mt8173-pm-domains.h" +#include "mt8183-pm-domains.h" +#include "mt8186-pm-domains.h" +#include "mt8188-pm-domains.h" +#include "mt8192-pm-domains.h" +#include "mt8195-pm-domains.h" + +#define MTK_POLL_DELAY_US 10 +#define MTK_POLL_TIMEOUT USEC_PER_SEC + +#define PWR_RST_B_BIT BIT(0) +#define PWR_ISO_BIT BIT(1) +#define PWR_ON_BIT BIT(2) +#define PWR_ON_2ND_BIT BIT(3) +#define PWR_CLK_DIS_BIT BIT(4) +#define PWR_SRAM_CLKISO_BIT BIT(5) +#define PWR_SRAM_ISOINT_B_BIT BIT(6) + +struct scpsys_domain { + struct generic_pm_domain genpd; + const struct scpsys_domain_data *data; + struct scpsys *scpsys; + int num_clks; + struct clk_bulk_data *clks; + int num_subsys_clks; + struct clk_bulk_data *subsys_clks; + struct regmap *infracfg; + struct regmap *smi; + struct regulator *supply; +}; + +struct scpsys { + struct device *dev; + struct regmap *base; + const struct scpsys_soc_data *soc_data; + struct genpd_onecell_data pd_data; + struct generic_pm_domain *domains[]; +}; + +#define to_scpsys_domain(gpd) container_of(gpd, struct scpsys_domain, genpd) + +static bool scpsys_domain_is_on(struct scpsys_domain *pd) +{ + struct scpsys *scpsys = pd->scpsys; + u32 status, status2; + + regmap_read(scpsys->base, pd->data->pwr_sta_offs, &status); + status &= pd->data->sta_mask; + + regmap_read(scpsys->base, pd->data->pwr_sta2nd_offs, &status2); + status2 &= pd->data->sta_mask; + + /* A domain is on when both status bits are set. */ + return status && status2; +} + +static int scpsys_sram_enable(struct scpsys_domain *pd) +{ + u32 pdn_ack = pd->data->sram_pdn_ack_bits; + struct scpsys *scpsys = pd->scpsys; + unsigned int tmp; + int ret; + + regmap_clear_bits(scpsys->base, pd->data->ctl_offs, pd->data->sram_pdn_bits); + + /* Either wait until SRAM_PDN_ACK all 1 or 0 */ + ret = regmap_read_poll_timeout(scpsys->base, pd->data->ctl_offs, tmp, + (tmp & pdn_ack) == 0, MTK_POLL_DELAY_US, MTK_POLL_TIMEOUT); + if (ret < 0) + return ret; + + if (MTK_SCPD_CAPS(pd, MTK_SCPD_SRAM_ISO)) { + regmap_set_bits(scpsys->base, pd->data->ctl_offs, PWR_SRAM_ISOINT_B_BIT); + udelay(1); + regmap_clear_bits(scpsys->base, pd->data->ctl_offs, PWR_SRAM_CLKISO_BIT); + } + + return 0; +} + +static int scpsys_sram_disable(struct scpsys_domain *pd) +{ + u32 pdn_ack = pd->data->sram_pdn_ack_bits; + struct scpsys *scpsys = pd->scpsys; + unsigned int tmp; + + if (MTK_SCPD_CAPS(pd, MTK_SCPD_SRAM_ISO)) { + regmap_set_bits(scpsys->base, pd->data->ctl_offs, PWR_SRAM_CLKISO_BIT); + udelay(1); + regmap_clear_bits(scpsys->base, pd->data->ctl_offs, PWR_SRAM_ISOINT_B_BIT); + } + + regmap_set_bits(scpsys->base, pd->data->ctl_offs, pd->data->sram_pdn_bits); + + /* Either wait until SRAM_PDN_ACK all 1 or 0 */ + return regmap_read_poll_timeout(scpsys->base, pd->data->ctl_offs, tmp, + (tmp & pdn_ack) == pdn_ack, MTK_POLL_DELAY_US, + MTK_POLL_TIMEOUT); +} + +static int _scpsys_bus_protect_enable(const struct scpsys_bus_prot_data *bpd, struct regmap *regmap) +{ + int i, ret; + + for (i = 0; i < SPM_MAX_BUS_PROT_DATA; i++) { + u32 val, mask = bpd[i].bus_prot_mask; + + if (!mask) + break; + + if (bpd[i].bus_prot_reg_update) + regmap_set_bits(regmap, bpd[i].bus_prot_set, mask); + else + regmap_write(regmap, bpd[i].bus_prot_set, mask); + + ret = regmap_read_poll_timeout(regmap, bpd[i].bus_prot_sta, + val, (val & mask) == mask, + MTK_POLL_DELAY_US, MTK_POLL_TIMEOUT); + if (ret) + return ret; + } + + return 0; +} + +static int scpsys_bus_protect_enable(struct scpsys_domain *pd) +{ + int ret; + + ret = _scpsys_bus_protect_enable(pd->data->bp_infracfg, pd->infracfg); + if (ret) + return ret; + + return _scpsys_bus_protect_enable(pd->data->bp_smi, pd->smi); +} + +static int _scpsys_bus_protect_disable(const struct scpsys_bus_prot_data *bpd, + struct regmap *regmap) +{ + int i, ret; + + for (i = SPM_MAX_BUS_PROT_DATA - 1; i >= 0; i--) { + u32 val, mask = bpd[i].bus_prot_mask; + + if (!mask) + continue; + + if (bpd[i].bus_prot_reg_update) + regmap_clear_bits(regmap, bpd[i].bus_prot_clr, mask); + else + regmap_write(regmap, bpd[i].bus_prot_clr, mask); + + if (bpd[i].ignore_clr_ack) + continue; + + ret = regmap_read_poll_timeout(regmap, bpd[i].bus_prot_sta, + val, !(val & mask), + MTK_POLL_DELAY_US, MTK_POLL_TIMEOUT); + if (ret) + return ret; + } + + return 0; +} + +static int scpsys_bus_protect_disable(struct scpsys_domain *pd) +{ + int ret; + + ret = _scpsys_bus_protect_disable(pd->data->bp_smi, pd->smi); + if (ret) + return ret; + + return _scpsys_bus_protect_disable(pd->data->bp_infracfg, pd->infracfg); +} + +static int scpsys_regulator_enable(struct regulator *supply) +{ + return supply ? regulator_enable(supply) : 0; +} + +static int scpsys_regulator_disable(struct regulator *supply) +{ + return supply ? regulator_disable(supply) : 0; +} + +static int scpsys_power_on(struct generic_pm_domain *genpd) +{ + struct scpsys_domain *pd = container_of(genpd, struct scpsys_domain, genpd); + struct scpsys *scpsys = pd->scpsys; + bool tmp; + int ret; + + ret = scpsys_regulator_enable(pd->supply); + if (ret) + return ret; + + ret = clk_bulk_prepare_enable(pd->num_clks, pd->clks); + if (ret) + goto err_reg; + + if (pd->data->ext_buck_iso_offs && MTK_SCPD_CAPS(pd, MTK_SCPD_EXT_BUCK_ISO)) + regmap_clear_bits(scpsys->base, pd->data->ext_buck_iso_offs, + pd->data->ext_buck_iso_mask); + + /* subsys power on */ + regmap_set_bits(scpsys->base, pd->data->ctl_offs, PWR_ON_BIT); + regmap_set_bits(scpsys->base, pd->data->ctl_offs, PWR_ON_2ND_BIT); + + /* wait until PWR_ACK = 1 */ + ret = readx_poll_timeout(scpsys_domain_is_on, pd, tmp, tmp, MTK_POLL_DELAY_US, + MTK_POLL_TIMEOUT); + if (ret < 0) + goto err_pwr_ack; + + regmap_clear_bits(scpsys->base, pd->data->ctl_offs, PWR_CLK_DIS_BIT); + regmap_clear_bits(scpsys->base, pd->data->ctl_offs, PWR_ISO_BIT); + regmap_set_bits(scpsys->base, pd->data->ctl_offs, PWR_RST_B_BIT); + + ret = clk_bulk_prepare_enable(pd->num_subsys_clks, pd->subsys_clks); + if (ret) + goto err_pwr_ack; + + ret = scpsys_sram_enable(pd); + if (ret < 0) + goto err_disable_subsys_clks; + + ret = scpsys_bus_protect_disable(pd); + if (ret < 0) + goto err_disable_sram; + + return 0; + +err_disable_sram: + scpsys_sram_disable(pd); +err_disable_subsys_clks: + clk_bulk_disable_unprepare(pd->num_subsys_clks, pd->subsys_clks); +err_pwr_ack: + clk_bulk_disable_unprepare(pd->num_clks, pd->clks); +err_reg: + scpsys_regulator_disable(pd->supply); + return ret; +} + +static int scpsys_power_off(struct generic_pm_domain *genpd) +{ + struct scpsys_domain *pd = container_of(genpd, struct scpsys_domain, genpd); + struct scpsys *scpsys = pd->scpsys; + bool tmp; + int ret; + + ret = scpsys_bus_protect_enable(pd); + if (ret < 0) + return ret; + + ret = scpsys_sram_disable(pd); + if (ret < 0) + return ret; + + if (pd->data->ext_buck_iso_offs && MTK_SCPD_CAPS(pd, MTK_SCPD_EXT_BUCK_ISO)) + regmap_set_bits(scpsys->base, pd->data->ext_buck_iso_offs, + pd->data->ext_buck_iso_mask); + + clk_bulk_disable_unprepare(pd->num_subsys_clks, pd->subsys_clks); + + /* subsys power off */ + regmap_set_bits(scpsys->base, pd->data->ctl_offs, PWR_ISO_BIT); + regmap_set_bits(scpsys->base, pd->data->ctl_offs, PWR_CLK_DIS_BIT); + regmap_clear_bits(scpsys->base, pd->data->ctl_offs, PWR_RST_B_BIT); + regmap_clear_bits(scpsys->base, pd->data->ctl_offs, PWR_ON_2ND_BIT); + regmap_clear_bits(scpsys->base, pd->data->ctl_offs, PWR_ON_BIT); + + /* wait until PWR_ACK = 0 */ + ret = readx_poll_timeout(scpsys_domain_is_on, pd, tmp, !tmp, MTK_POLL_DELAY_US, + MTK_POLL_TIMEOUT); + if (ret < 0) + return ret; + + clk_bulk_disable_unprepare(pd->num_clks, pd->clks); + + scpsys_regulator_disable(pd->supply); + + return 0; +} + +static struct +generic_pm_domain *scpsys_add_one_domain(struct scpsys *scpsys, struct device_node *node) +{ + const struct scpsys_domain_data *domain_data; + struct scpsys_domain *pd; + struct device_node *root_node = scpsys->dev->of_node; + struct device_node *smi_node; + struct property *prop; + const char *clk_name; + int i, ret, num_clks; + struct clk *clk; + int clk_ind = 0; + u32 id; + + ret = of_property_read_u32(node, "reg", &id); + if (ret) { + dev_err(scpsys->dev, "%pOF: failed to retrieve domain id from reg: %d\n", + node, ret); + return ERR_PTR(-EINVAL); + } + + if (id >= scpsys->soc_data->num_domains) { + dev_err(scpsys->dev, "%pOF: invalid domain id %d\n", node, id); + return ERR_PTR(-EINVAL); + } + + domain_data = &scpsys->soc_data->domains_data[id]; + if (domain_data->sta_mask == 0) { + dev_err(scpsys->dev, "%pOF: undefined domain id %d\n", node, id); + return ERR_PTR(-EINVAL); + } + + pd = devm_kzalloc(scpsys->dev, sizeof(*pd), GFP_KERNEL); + if (!pd) + return ERR_PTR(-ENOMEM); + + pd->data = domain_data; + pd->scpsys = scpsys; + + if (MTK_SCPD_CAPS(pd, MTK_SCPD_DOMAIN_SUPPLY)) { + /* + * Find regulator in current power domain node. + * devm_regulator_get() finds regulator in a node and its child + * node, so set of_node to current power domain node then change + * back to original node after regulator is found for current + * power domain node. + */ + scpsys->dev->of_node = node; + pd->supply = devm_regulator_get(scpsys->dev, "domain"); + scpsys->dev->of_node = root_node; + if (IS_ERR(pd->supply)) { + dev_err_probe(scpsys->dev, PTR_ERR(pd->supply), + "%pOF: failed to get power supply.\n", + node); + return ERR_CAST(pd->supply); + } + } + + pd->infracfg = syscon_regmap_lookup_by_phandle_optional(node, "mediatek,infracfg"); + if (IS_ERR(pd->infracfg)) + return ERR_CAST(pd->infracfg); + + smi_node = of_parse_phandle(node, "mediatek,smi", 0); + if (smi_node) { + pd->smi = device_node_to_regmap(smi_node); + of_node_put(smi_node); + if (IS_ERR(pd->smi)) + return ERR_CAST(pd->smi); + } + + num_clks = of_clk_get_parent_count(node); + if (num_clks > 0) { + /* Calculate number of subsys_clks */ + of_property_for_each_string(node, "clock-names", prop, clk_name) { + char *subsys; + + subsys = strchr(clk_name, '-'); + if (subsys) + pd->num_subsys_clks++; + else + pd->num_clks++; + } + + pd->clks = devm_kcalloc(scpsys->dev, pd->num_clks, sizeof(*pd->clks), GFP_KERNEL); + if (!pd->clks) + return ERR_PTR(-ENOMEM); + + pd->subsys_clks = devm_kcalloc(scpsys->dev, pd->num_subsys_clks, + sizeof(*pd->subsys_clks), GFP_KERNEL); + if (!pd->subsys_clks) + return ERR_PTR(-ENOMEM); + + } + + for (i = 0; i < pd->num_clks; i++) { + clk = of_clk_get(node, i); + if (IS_ERR(clk)) { + ret = PTR_ERR(clk); + dev_err_probe(scpsys->dev, ret, + "%pOF: failed to get clk at index %d\n", node, i); + goto err_put_clocks; + } + + pd->clks[clk_ind++].clk = clk; + } + + for (i = 0; i < pd->num_subsys_clks; i++) { + clk = of_clk_get(node, i + clk_ind); + if (IS_ERR(clk)) { + ret = PTR_ERR(clk); + dev_err_probe(scpsys->dev, ret, + "%pOF: failed to get clk at index %d\n", node, + i + clk_ind); + goto err_put_subsys_clocks; + } + + pd->subsys_clks[i].clk = clk; + } + + /* + * Initially turn on all domains to make the domains usable + * with !CONFIG_PM and to get the hardware in sync with the + * software. The unused domains will be switched off during + * late_init time. + */ + if (MTK_SCPD_CAPS(pd, MTK_SCPD_KEEP_DEFAULT_OFF)) { + if (scpsys_domain_is_on(pd)) + dev_warn(scpsys->dev, + "%pOF: A default off power domain has been ON\n", node); + } else { + ret = scpsys_power_on(&pd->genpd); + if (ret < 0) { + dev_err(scpsys->dev, "%pOF: failed to power on domain: %d\n", node, ret); + goto err_put_subsys_clocks; + } + + if (MTK_SCPD_CAPS(pd, MTK_SCPD_ALWAYS_ON)) + pd->genpd.flags |= GENPD_FLAG_ALWAYS_ON; + } + + if (scpsys->domains[id]) { + ret = -EINVAL; + dev_err(scpsys->dev, + "power domain with id %d already exists, check your device-tree\n", id); + goto err_put_subsys_clocks; + } + + if (!pd->data->name) + pd->genpd.name = node->name; + else + pd->genpd.name = pd->data->name; + + pd->genpd.power_off = scpsys_power_off; + pd->genpd.power_on = scpsys_power_on; + + if (MTK_SCPD_CAPS(pd, MTK_SCPD_ACTIVE_WAKEUP)) + pd->genpd.flags |= GENPD_FLAG_ACTIVE_WAKEUP; + + if (MTK_SCPD_CAPS(pd, MTK_SCPD_KEEP_DEFAULT_OFF)) + pm_genpd_init(&pd->genpd, NULL, true); + else + pm_genpd_init(&pd->genpd, NULL, false); + + scpsys->domains[id] = &pd->genpd; + + return scpsys->pd_data.domains[id]; + +err_put_subsys_clocks: + clk_bulk_put(pd->num_subsys_clks, pd->subsys_clks); +err_put_clocks: + clk_bulk_put(pd->num_clks, pd->clks); + return ERR_PTR(ret); +} + +static int scpsys_add_subdomain(struct scpsys *scpsys, struct device_node *parent) +{ + struct generic_pm_domain *child_pd, *parent_pd; + struct device_node *child; + int ret; + + for_each_child_of_node(parent, child) { + u32 id; + + ret = of_property_read_u32(parent, "reg", &id); + if (ret) { + dev_err(scpsys->dev, "%pOF: failed to get parent domain id\n", child); + goto err_put_node; + } + + if (!scpsys->pd_data.domains[id]) { + ret = -EINVAL; + dev_err(scpsys->dev, "power domain with id %d does not exist\n", id); + goto err_put_node; + } + + parent_pd = scpsys->pd_data.domains[id]; + + child_pd = scpsys_add_one_domain(scpsys, child); + if (IS_ERR(child_pd)) { + ret = PTR_ERR(child_pd); + dev_err_probe(scpsys->dev, ret, "%pOF: failed to get child domain id\n", + child); + goto err_put_node; + } + + ret = pm_genpd_add_subdomain(parent_pd, child_pd); + if (ret) { + dev_err(scpsys->dev, "failed to add %s subdomain to parent %s\n", + child_pd->name, parent_pd->name); + goto err_put_node; + } else { + dev_dbg(scpsys->dev, "%s add subdomain: %s\n", parent_pd->name, + child_pd->name); + } + + /* recursive call to add all subdomains */ + ret = scpsys_add_subdomain(scpsys, child); + if (ret) + goto err_put_node; + } + + return 0; + +err_put_node: + of_node_put(child); + return ret; +} + +static void scpsys_remove_one_domain(struct scpsys_domain *pd) +{ + int ret; + + if (scpsys_domain_is_on(pd)) + scpsys_power_off(&pd->genpd); + + /* + * We're in the error cleanup already, so we only complain, + * but won't emit another error on top of the original one. + */ + ret = pm_genpd_remove(&pd->genpd); + if (ret < 0) + dev_err(pd->scpsys->dev, + "failed to remove domain '%s' : %d - state may be inconsistent\n", + pd->genpd.name, ret); + + clk_bulk_put(pd->num_clks, pd->clks); + clk_bulk_put(pd->num_subsys_clks, pd->subsys_clks); +} + +static void scpsys_domain_cleanup(struct scpsys *scpsys) +{ + struct generic_pm_domain *genpd; + struct scpsys_domain *pd; + int i; + + for (i = scpsys->pd_data.num_domains - 1; i >= 0; i--) { + genpd = scpsys->pd_data.domains[i]; + if (genpd) { + pd = to_scpsys_domain(genpd); + scpsys_remove_one_domain(pd); + } + } +} + +static const struct of_device_id scpsys_of_match[] = { + { + .compatible = "mediatek,mt6795-power-controller", + .data = &mt6795_scpsys_data, + }, + { + .compatible = "mediatek,mt8167-power-controller", + .data = &mt8167_scpsys_data, + }, + { + .compatible = "mediatek,mt8173-power-controller", + .data = &mt8173_scpsys_data, + }, + { + .compatible = "mediatek,mt8183-power-controller", + .data = &mt8183_scpsys_data, + }, + { + .compatible = "mediatek,mt8186-power-controller", + .data = &mt8186_scpsys_data, + }, + { + .compatible = "mediatek,mt8188-power-controller", + .data = &mt8188_scpsys_data, + }, + { + .compatible = "mediatek,mt8192-power-controller", + .data = &mt8192_scpsys_data, + }, + { + .compatible = "mediatek,mt8195-power-controller", + .data = &mt8195_scpsys_data, + }, + { } +}; + +static int scpsys_probe(struct platform_device *pdev) +{ + struct device *dev = &pdev->dev; + struct device_node *np = dev->of_node; + const struct scpsys_soc_data *soc; + struct device_node *node; + struct device *parent; + struct scpsys *scpsys; + int ret; + + soc = of_device_get_match_data(&pdev->dev); + if (!soc) { + dev_err(&pdev->dev, "no power controller data\n"); + return -EINVAL; + } + + scpsys = devm_kzalloc(dev, struct_size(scpsys, domains, soc->num_domains), GFP_KERNEL); + if (!scpsys) + return -ENOMEM; + + scpsys->dev = dev; + scpsys->soc_data = soc; + + scpsys->pd_data.domains = scpsys->domains; + scpsys->pd_data.num_domains = soc->num_domains; + + parent = dev->parent; + if (!parent) { + dev_err(dev, "no parent for syscon devices\n"); + return -ENODEV; + } + + scpsys->base = syscon_node_to_regmap(parent->of_node); + if (IS_ERR(scpsys->base)) { + dev_err(dev, "no regmap available\n"); + return PTR_ERR(scpsys->base); + } + + ret = -ENODEV; + for_each_available_child_of_node(np, node) { + struct generic_pm_domain *domain; + + domain = scpsys_add_one_domain(scpsys, node); + if (IS_ERR(domain)) { + ret = PTR_ERR(domain); + of_node_put(node); + goto err_cleanup_domains; + } + + ret = scpsys_add_subdomain(scpsys, node); + if (ret) { + of_node_put(node); + goto err_cleanup_domains; + } + } + + if (ret) { + dev_dbg(dev, "no power domains present\n"); + return ret; + } + + ret = of_genpd_add_provider_onecell(np, &scpsys->pd_data); + if (ret) { + dev_err(dev, "failed to add provider: %d\n", ret); + goto err_cleanup_domains; + } + + return 0; + +err_cleanup_domains: + scpsys_domain_cleanup(scpsys); + return ret; +} + +static struct platform_driver scpsys_pm_domain_driver = { + .probe = scpsys_probe, + .driver = { + .name = "mtk-power-controller", + .suppress_bind_attrs = true, + .of_match_table = scpsys_of_match, + }, +}; +builtin_platform_driver(scpsys_pm_domain_driver); diff --git a/drivers/pmdomain/mediatek/mtk-pm-domains.h b/drivers/pmdomain/mediatek/mtk-pm-domains.h new file mode 100644 index 000000000000..5ec53ee073c4 --- /dev/null +++ b/drivers/pmdomain/mediatek/mtk-pm-domains.h @@ -0,0 +1,111 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ + +#ifndef __SOC_MEDIATEK_MTK_PM_DOMAINS_H +#define __SOC_MEDIATEK_MTK_PM_DOMAINS_H + +#define MTK_SCPD_ACTIVE_WAKEUP BIT(0) +#define MTK_SCPD_FWAIT_SRAM BIT(1) +#define MTK_SCPD_SRAM_ISO BIT(2) +#define MTK_SCPD_KEEP_DEFAULT_OFF BIT(3) +#define MTK_SCPD_DOMAIN_SUPPLY BIT(4) +/* can't set MTK_SCPD_KEEP_DEFAULT_OFF at the same time */ +#define MTK_SCPD_ALWAYS_ON BIT(5) +#define MTK_SCPD_EXT_BUCK_ISO BIT(6) +#define MTK_SCPD_CAPS(_scpd, _x) ((_scpd)->data->caps & (_x)) + +#define SPM_VDE_PWR_CON 0x0210 +#define SPM_MFG_PWR_CON 0x0214 +#define SPM_VEN_PWR_CON 0x0230 +#define SPM_ISP_PWR_CON 0x0238 +#define SPM_DIS_PWR_CON 0x023c +#define SPM_CONN_PWR_CON 0x0280 +#define SPM_VEN2_PWR_CON 0x0298 +#define SPM_AUDIO_PWR_CON 0x029c +#define SPM_MFG_2D_PWR_CON 0x02c0 +#define SPM_MFG_ASYNC_PWR_CON 0x02c4 +#define SPM_USB_PWR_CON 0x02cc + +#define SPM_PWR_STATUS 0x060c +#define SPM_PWR_STATUS_2ND 0x0610 + +#define PWR_STATUS_CONN BIT(1) +#define PWR_STATUS_DISP BIT(3) +#define PWR_STATUS_MFG BIT(4) +#define PWR_STATUS_ISP BIT(5) +#define PWR_STATUS_VDEC BIT(7) +#define PWR_STATUS_VENC_LT BIT(20) +#define PWR_STATUS_VENC BIT(21) +#define PWR_STATUS_MFG_2D BIT(22) +#define PWR_STATUS_MFG_ASYNC BIT(23) +#define PWR_STATUS_AUDIO BIT(24) +#define PWR_STATUS_USB BIT(25) + +#define SPM_MAX_BUS_PROT_DATA 6 + +#define _BUS_PROT(_mask, _set, _clr, _sta, _update, _ignore) { \ + .bus_prot_mask = (_mask), \ + .bus_prot_set = _set, \ + .bus_prot_clr = _clr, \ + .bus_prot_sta = _sta, \ + .bus_prot_reg_update = _update, \ + .ignore_clr_ack = _ignore, \ + } + +#define BUS_PROT_WR(_mask, _set, _clr, _sta) \ + _BUS_PROT(_mask, _set, _clr, _sta, false, false) + +#define BUS_PROT_WR_IGN(_mask, _set, _clr, _sta) \ + _BUS_PROT(_mask, _set, _clr, _sta, false, true) + +#define BUS_PROT_UPDATE(_mask, _set, _clr, _sta) \ + _BUS_PROT(_mask, _set, _clr, _sta, true, false) + +#define BUS_PROT_UPDATE_TOPAXI(_mask) \ + BUS_PROT_UPDATE(_mask, \ + INFRA_TOPAXI_PROTECTEN, \ + INFRA_TOPAXI_PROTECTEN, \ + INFRA_TOPAXI_PROTECTSTA1) + +struct scpsys_bus_prot_data { + u32 bus_prot_mask; + u32 bus_prot_set; + u32 bus_prot_clr; + u32 bus_prot_sta; + bool bus_prot_reg_update; + bool ignore_clr_ack; +}; + +/** + * struct scpsys_domain_data - scp domain data for power on/off flow + * @name: The name of the power domain. + * @sta_mask: The mask for power on/off status bit. + * @ctl_offs: The offset for main power control register. + * @sram_pdn_bits: The mask for sram power control bits. + * @sram_pdn_ack_bits: The mask for sram power control acked bits. + * @ext_buck_iso_offs: The offset for external buck isolation + * @ext_buck_iso_mask: The mask for external buck isolation + * @caps: The flag for active wake-up action. + * @bp_infracfg: bus protection for infracfg subsystem + * @bp_smi: bus protection for smi subsystem + */ +struct scpsys_domain_data { + const char *name; + u32 sta_mask; + int ctl_offs; + u32 sram_pdn_bits; + u32 sram_pdn_ack_bits; + int ext_buck_iso_offs; + u32 ext_buck_iso_mask; + u8 caps; + const struct scpsys_bus_prot_data bp_infracfg[SPM_MAX_BUS_PROT_DATA]; + const struct scpsys_bus_prot_data bp_smi[SPM_MAX_BUS_PROT_DATA]; + int pwr_sta_offs; + int pwr_sta2nd_offs; +}; + +struct scpsys_soc_data { + const struct scpsys_domain_data *domains_data; + int num_domains; +}; + +#endif /* __SOC_MEDIATEK_MTK_PM_DOMAINS_H */ diff --git a/drivers/pmdomain/mediatek/mtk-scpsys.c b/drivers/pmdomain/mediatek/mtk-scpsys.c new file mode 100644 index 000000000000..b374d01fdac7 --- /dev/null +++ b/drivers/pmdomain/mediatek/mtk-scpsys.c @@ -0,0 +1,1147 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2015 Pengutronix, Sascha Hauer + */ +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include + +#define MTK_POLL_DELAY_US 10 +#define MTK_POLL_TIMEOUT USEC_PER_SEC + +#define MTK_SCPD_ACTIVE_WAKEUP BIT(0) +#define MTK_SCPD_FWAIT_SRAM BIT(1) +#define MTK_SCPD_CAPS(_scpd, _x) ((_scpd)->data->caps & (_x)) + +#define SPM_VDE_PWR_CON 0x0210 +#define SPM_MFG_PWR_CON 0x0214 +#define SPM_VEN_PWR_CON 0x0230 +#define SPM_ISP_PWR_CON 0x0238 +#define SPM_DIS_PWR_CON 0x023c +#define SPM_CONN_PWR_CON 0x0280 +#define SPM_VEN2_PWR_CON 0x0298 +#define SPM_AUDIO_PWR_CON 0x029c /* MT8173, MT2712 */ +#define SPM_BDP_PWR_CON 0x029c /* MT2701 */ +#define SPM_ETH_PWR_CON 0x02a0 +#define SPM_HIF_PWR_CON 0x02a4 +#define SPM_IFR_MSC_PWR_CON 0x02a8 +#define SPM_MFG_2D_PWR_CON 0x02c0 +#define SPM_MFG_ASYNC_PWR_CON 0x02c4 +#define SPM_USB_PWR_CON 0x02cc +#define SPM_USB2_PWR_CON 0x02d4 /* MT2712 */ +#define SPM_ETHSYS_PWR_CON 0x02e0 /* MT7622 */ +#define SPM_HIF0_PWR_CON 0x02e4 /* MT7622 */ +#define SPM_HIF1_PWR_CON 0x02e8 /* MT7622 */ +#define SPM_WB_PWR_CON 0x02ec /* MT7622 */ + +#define SPM_PWR_STATUS 0x060c +#define SPM_PWR_STATUS_2ND 0x0610 + +#define PWR_RST_B_BIT BIT(0) +#define PWR_ISO_BIT BIT(1) +#define PWR_ON_BIT BIT(2) +#define PWR_ON_2ND_BIT BIT(3) +#define PWR_CLK_DIS_BIT BIT(4) + +#define PWR_STATUS_CONN BIT(1) +#define PWR_STATUS_DISP BIT(3) +#define PWR_STATUS_MFG BIT(4) +#define PWR_STATUS_ISP BIT(5) +#define PWR_STATUS_VDEC BIT(7) +#define PWR_STATUS_BDP BIT(14) +#define PWR_STATUS_ETH BIT(15) +#define PWR_STATUS_HIF BIT(16) +#define PWR_STATUS_IFR_MSC BIT(17) +#define PWR_STATUS_USB2 BIT(19) /* MT2712 */ +#define PWR_STATUS_VENC_LT BIT(20) +#define PWR_STATUS_VENC BIT(21) +#define PWR_STATUS_MFG_2D BIT(22) /* MT8173 */ +#define PWR_STATUS_MFG_ASYNC BIT(23) /* MT8173 */ +#define PWR_STATUS_AUDIO BIT(24) /* MT8173, MT2712 */ +#define PWR_STATUS_USB BIT(25) /* MT8173, MT2712 */ +#define PWR_STATUS_ETHSYS BIT(24) /* MT7622 */ +#define PWR_STATUS_HIF0 BIT(25) /* MT7622 */ +#define PWR_STATUS_HIF1 BIT(26) /* MT7622 */ +#define PWR_STATUS_WB BIT(27) /* MT7622 */ + +enum clk_id { + CLK_NONE, + CLK_MM, + CLK_MFG, + CLK_VENC, + CLK_VENC_LT, + CLK_ETHIF, + CLK_VDEC, + CLK_HIFSEL, + CLK_JPGDEC, + CLK_AUDIO, + CLK_MAX, +}; + +static const char * const clk_names[] = { + NULL, + "mm", + "mfg", + "venc", + "venc_lt", + "ethif", + "vdec", + "hif_sel", + "jpgdec", + "audio", + NULL, +}; + +#define MAX_CLKS 3 + +/** + * struct scp_domain_data - scp domain data for power on/off flow + * @name: The domain name. + * @sta_mask: The mask for power on/off status bit. + * @ctl_offs: The offset for main power control register. + * @sram_pdn_bits: The mask for sram power control bits. + * @sram_pdn_ack_bits: The mask for sram power control acked bits. + * @bus_prot_mask: The mask for single step bus protection. + * @clk_id: The basic clocks required by this power domain. + * @caps: The flag for active wake-up action. + */ +struct scp_domain_data { + const char *name; + u32 sta_mask; + int ctl_offs; + u32 sram_pdn_bits; + u32 sram_pdn_ack_bits; + u32 bus_prot_mask; + enum clk_id clk_id[MAX_CLKS]; + u8 caps; +}; + +struct scp; + +struct scp_domain { + struct generic_pm_domain genpd; + struct scp *scp; + struct clk *clk[MAX_CLKS]; + const struct scp_domain_data *data; + struct regulator *supply; +}; + +struct scp_ctrl_reg { + int pwr_sta_offs; + int pwr_sta2nd_offs; +}; + +struct scp { + struct scp_domain *domains; + struct genpd_onecell_data pd_data; + struct device *dev; + void __iomem *base; + struct regmap *infracfg; + struct scp_ctrl_reg ctrl_reg; + bool bus_prot_reg_update; +}; + +struct scp_subdomain { + int origin; + int subdomain; +}; + +struct scp_soc_data { + const struct scp_domain_data *domains; + int num_domains; + const struct scp_subdomain *subdomains; + int num_subdomains; + const struct scp_ctrl_reg regs; + bool bus_prot_reg_update; +}; + +static int scpsys_domain_is_on(struct scp_domain *scpd) +{ + struct scp *scp = scpd->scp; + + u32 status = readl(scp->base + scp->ctrl_reg.pwr_sta_offs) & + scpd->data->sta_mask; + u32 status2 = readl(scp->base + scp->ctrl_reg.pwr_sta2nd_offs) & + scpd->data->sta_mask; + + /* + * A domain is on when both status bits are set. If only one is set + * return an error. This happens while powering up a domain + */ + + if (status && status2) + return true; + if (!status && !status2) + return false; + + return -EINVAL; +} + +static int scpsys_regulator_enable(struct scp_domain *scpd) +{ + if (!scpd->supply) + return 0; + + return regulator_enable(scpd->supply); +} + +static int scpsys_regulator_disable(struct scp_domain *scpd) +{ + if (!scpd->supply) + return 0; + + return regulator_disable(scpd->supply); +} + +static void scpsys_clk_disable(struct clk *clk[], int max_num) +{ + int i; + + for (i = max_num - 1; i >= 0; i--) + clk_disable_unprepare(clk[i]); +} + +static int scpsys_clk_enable(struct clk *clk[], int max_num) +{ + int i, ret = 0; + + for (i = 0; i < max_num && clk[i]; i++) { + ret = clk_prepare_enable(clk[i]); + if (ret) { + scpsys_clk_disable(clk, i); + break; + } + } + + return ret; +} + +static int scpsys_sram_enable(struct scp_domain *scpd, void __iomem *ctl_addr) +{ + u32 val; + u32 pdn_ack = scpd->data->sram_pdn_ack_bits; + int tmp; + + val = readl(ctl_addr); + val &= ~scpd->data->sram_pdn_bits; + writel(val, ctl_addr); + + /* Either wait until SRAM_PDN_ACK all 0 or have a force wait */ + if (MTK_SCPD_CAPS(scpd, MTK_SCPD_FWAIT_SRAM)) { + /* + * Currently, MTK_SCPD_FWAIT_SRAM is necessary only for + * MT7622_POWER_DOMAIN_WB and thus just a trivial setup + * is applied here. + */ + usleep_range(12000, 12100); + } else { + /* Either wait until SRAM_PDN_ACK all 1 or 0 */ + int ret = readl_poll_timeout(ctl_addr, tmp, + (tmp & pdn_ack) == 0, + MTK_POLL_DELAY_US, MTK_POLL_TIMEOUT); + if (ret < 0) + return ret; + } + + return 0; +} + +static int scpsys_sram_disable(struct scp_domain *scpd, void __iomem *ctl_addr) +{ + u32 val; + u32 pdn_ack = scpd->data->sram_pdn_ack_bits; + int tmp; + + val = readl(ctl_addr); + val |= scpd->data->sram_pdn_bits; + writel(val, ctl_addr); + + /* Either wait until SRAM_PDN_ACK all 1 or 0 */ + return readl_poll_timeout(ctl_addr, tmp, + (tmp & pdn_ack) == pdn_ack, + MTK_POLL_DELAY_US, MTK_POLL_TIMEOUT); +} + +static int scpsys_bus_protect_enable(struct scp_domain *scpd) +{ + struct scp *scp = scpd->scp; + + if (!scpd->data->bus_prot_mask) + return 0; + + return mtk_infracfg_set_bus_protection(scp->infracfg, + scpd->data->bus_prot_mask, + scp->bus_prot_reg_update); +} + +static int scpsys_bus_protect_disable(struct scp_domain *scpd) +{ + struct scp *scp = scpd->scp; + + if (!scpd->data->bus_prot_mask) + return 0; + + return mtk_infracfg_clear_bus_protection(scp->infracfg, + scpd->data->bus_prot_mask, + scp->bus_prot_reg_update); +} + +static int scpsys_power_on(struct generic_pm_domain *genpd) +{ + struct scp_domain *scpd = container_of(genpd, struct scp_domain, genpd); + struct scp *scp = scpd->scp; + void __iomem *ctl_addr = scp->base + scpd->data->ctl_offs; + u32 val; + int ret, tmp; + + ret = scpsys_regulator_enable(scpd); + if (ret < 0) + return ret; + + ret = scpsys_clk_enable(scpd->clk, MAX_CLKS); + if (ret) + goto err_clk; + + /* subsys power on */ + val = readl(ctl_addr); + val |= PWR_ON_BIT; + writel(val, ctl_addr); + val |= PWR_ON_2ND_BIT; + writel(val, ctl_addr); + + /* wait until PWR_ACK = 1 */ + ret = readx_poll_timeout(scpsys_domain_is_on, scpd, tmp, tmp > 0, + MTK_POLL_DELAY_US, MTK_POLL_TIMEOUT); + if (ret < 0) + goto err_pwr_ack; + + val &= ~PWR_CLK_DIS_BIT; + writel(val, ctl_addr); + + val &= ~PWR_ISO_BIT; + writel(val, ctl_addr); + + val |= PWR_RST_B_BIT; + writel(val, ctl_addr); + + ret = scpsys_sram_enable(scpd, ctl_addr); + if (ret < 0) + goto err_pwr_ack; + + ret = scpsys_bus_protect_disable(scpd); + if (ret < 0) + goto err_pwr_ack; + + return 0; + +err_pwr_ack: + scpsys_clk_disable(scpd->clk, MAX_CLKS); +err_clk: + scpsys_regulator_disable(scpd); + + dev_err(scp->dev, "Failed to power on domain %s\n", genpd->name); + + return ret; +} + +static int scpsys_power_off(struct generic_pm_domain *genpd) +{ + struct scp_domain *scpd = container_of(genpd, struct scp_domain, genpd); + struct scp *scp = scpd->scp; + void __iomem *ctl_addr = scp->base + scpd->data->ctl_offs; + u32 val; + int ret, tmp; + + ret = scpsys_bus_protect_enable(scpd); + if (ret < 0) + goto out; + + ret = scpsys_sram_disable(scpd, ctl_addr); + if (ret < 0) + goto out; + + /* subsys power off */ + val = readl(ctl_addr); + val |= PWR_ISO_BIT; + writel(val, ctl_addr); + + val &= ~PWR_RST_B_BIT; + writel(val, ctl_addr); + + val |= PWR_CLK_DIS_BIT; + writel(val, ctl_addr); + + val &= ~PWR_ON_BIT; + writel(val, ctl_addr); + + val &= ~PWR_ON_2ND_BIT; + writel(val, ctl_addr); + + /* wait until PWR_ACK = 0 */ + ret = readx_poll_timeout(scpsys_domain_is_on, scpd, tmp, tmp == 0, + MTK_POLL_DELAY_US, MTK_POLL_TIMEOUT); + if (ret < 0) + goto out; + + scpsys_clk_disable(scpd->clk, MAX_CLKS); + + ret = scpsys_regulator_disable(scpd); + if (ret < 0) + goto out; + + return 0; + +out: + dev_err(scp->dev, "Failed to power off domain %s\n", genpd->name); + + return ret; +} + +static void init_clks(struct platform_device *pdev, struct clk **clk) +{ + int i; + + for (i = CLK_NONE + 1; i < CLK_MAX; i++) + clk[i] = devm_clk_get(&pdev->dev, clk_names[i]); +} + +static struct scp *init_scp(struct platform_device *pdev, + const struct scp_domain_data *scp_domain_data, int num, + const struct scp_ctrl_reg *scp_ctrl_reg, + bool bus_prot_reg_update) +{ + struct genpd_onecell_data *pd_data; + struct resource *res; + int i, j; + struct scp *scp; + struct clk *clk[CLK_MAX]; + + scp = devm_kzalloc(&pdev->dev, sizeof(*scp), GFP_KERNEL); + if (!scp) + return ERR_PTR(-ENOMEM); + + scp->ctrl_reg.pwr_sta_offs = scp_ctrl_reg->pwr_sta_offs; + scp->ctrl_reg.pwr_sta2nd_offs = scp_ctrl_reg->pwr_sta2nd_offs; + + scp->bus_prot_reg_update = bus_prot_reg_update; + + scp->dev = &pdev->dev; + + res = platform_get_resource(pdev, IORESOURCE_MEM, 0); + scp->base = devm_ioremap_resource(&pdev->dev, res); + if (IS_ERR(scp->base)) + return ERR_CAST(scp->base); + + scp->domains = devm_kcalloc(&pdev->dev, + num, sizeof(*scp->domains), GFP_KERNEL); + if (!scp->domains) + return ERR_PTR(-ENOMEM); + + pd_data = &scp->pd_data; + + pd_data->domains = devm_kcalloc(&pdev->dev, + num, sizeof(*pd_data->domains), GFP_KERNEL); + if (!pd_data->domains) + return ERR_PTR(-ENOMEM); + + scp->infracfg = syscon_regmap_lookup_by_phandle(pdev->dev.of_node, + "infracfg"); + if (IS_ERR(scp->infracfg)) { + dev_err(&pdev->dev, "Cannot find infracfg controller: %ld\n", + PTR_ERR(scp->infracfg)); + return ERR_CAST(scp->infracfg); + } + + for (i = 0; i < num; i++) { + struct scp_domain *scpd = &scp->domains[i]; + const struct scp_domain_data *data = &scp_domain_data[i]; + + scpd->supply = devm_regulator_get_optional(&pdev->dev, data->name); + if (IS_ERR(scpd->supply)) { + if (PTR_ERR(scpd->supply) == -ENODEV) + scpd->supply = NULL; + else + return ERR_CAST(scpd->supply); + } + } + + pd_data->num_domains = num; + + init_clks(pdev, clk); + + for (i = 0; i < num; i++) { + struct scp_domain *scpd = &scp->domains[i]; + struct generic_pm_domain *genpd = &scpd->genpd; + const struct scp_domain_data *data = &scp_domain_data[i]; + + pd_data->domains[i] = genpd; + scpd->scp = scp; + + scpd->data = data; + + for (j = 0; j < MAX_CLKS && data->clk_id[j]; j++) { + struct clk *c = clk[data->clk_id[j]]; + + if (IS_ERR(c)) { + dev_err(&pdev->dev, "%s: clk unavailable\n", + data->name); + return ERR_CAST(c); + } + + scpd->clk[j] = c; + } + + genpd->name = data->name; + genpd->power_off = scpsys_power_off; + genpd->power_on = scpsys_power_on; + if (MTK_SCPD_CAPS(scpd, MTK_SCPD_ACTIVE_WAKEUP)) + genpd->flags |= GENPD_FLAG_ACTIVE_WAKEUP; + } + + return scp; +} + +static void mtk_register_power_domains(struct platform_device *pdev, + struct scp *scp, int num) +{ + struct genpd_onecell_data *pd_data; + int i, ret; + + for (i = 0; i < num; i++) { + struct scp_domain *scpd = &scp->domains[i]; + struct generic_pm_domain *genpd = &scpd->genpd; + bool on; + + /* + * Initially turn on all domains to make the domains usable + * with !CONFIG_PM and to get the hardware in sync with the + * software. The unused domains will be switched off during + * late_init time. + */ + on = !WARN_ON(genpd->power_on(genpd) < 0); + + pm_genpd_init(genpd, NULL, !on); + } + + /* + * We are not allowed to fail here since there is no way to unregister + * a power domain. Once registered above we have to keep the domains + * valid. + */ + + pd_data = &scp->pd_data; + + ret = of_genpd_add_provider_onecell(pdev->dev.of_node, pd_data); + if (ret) + dev_err(&pdev->dev, "Failed to add OF provider: %d\n", ret); +} + +/* + * MT2701 power domain support + */ + +static const struct scp_domain_data scp_domain_data_mt2701[] = { + [MT2701_POWER_DOMAIN_CONN] = { + .name = "conn", + .sta_mask = PWR_STATUS_CONN, + .ctl_offs = SPM_CONN_PWR_CON, + .bus_prot_mask = MT2701_TOP_AXI_PROT_EN_CONN_M | + MT2701_TOP_AXI_PROT_EN_CONN_S, + .clk_id = {CLK_NONE}, + .caps = MTK_SCPD_ACTIVE_WAKEUP, + }, + [MT2701_POWER_DOMAIN_DISP] = { + .name = "disp", + .sta_mask = PWR_STATUS_DISP, + .ctl_offs = SPM_DIS_PWR_CON, + .sram_pdn_bits = GENMASK(11, 8), + .clk_id = {CLK_MM}, + .bus_prot_mask = MT2701_TOP_AXI_PROT_EN_MM_M0, + .caps = MTK_SCPD_ACTIVE_WAKEUP, + }, + [MT2701_POWER_DOMAIN_MFG] = { + .name = "mfg", + .sta_mask = PWR_STATUS_MFG, + .ctl_offs = SPM_MFG_PWR_CON, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .clk_id = {CLK_MFG}, + .caps = MTK_SCPD_ACTIVE_WAKEUP, + }, + [MT2701_POWER_DOMAIN_VDEC] = { + .name = "vdec", + .sta_mask = PWR_STATUS_VDEC, + .ctl_offs = SPM_VDE_PWR_CON, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .clk_id = {CLK_MM}, + .caps = MTK_SCPD_ACTIVE_WAKEUP, + }, + [MT2701_POWER_DOMAIN_ISP] = { + .name = "isp", + .sta_mask = PWR_STATUS_ISP, + .ctl_offs = SPM_ISP_PWR_CON, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = GENMASK(13, 12), + .clk_id = {CLK_MM}, + .caps = MTK_SCPD_ACTIVE_WAKEUP, + }, + [MT2701_POWER_DOMAIN_BDP] = { + .name = "bdp", + .sta_mask = PWR_STATUS_BDP, + .ctl_offs = SPM_BDP_PWR_CON, + .sram_pdn_bits = GENMASK(11, 8), + .clk_id = {CLK_NONE}, + .caps = MTK_SCPD_ACTIVE_WAKEUP, + }, + [MT2701_POWER_DOMAIN_ETH] = { + .name = "eth", + .sta_mask = PWR_STATUS_ETH, + .ctl_offs = SPM_ETH_PWR_CON, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = GENMASK(15, 12), + .clk_id = {CLK_ETHIF}, + .caps = MTK_SCPD_ACTIVE_WAKEUP, + }, + [MT2701_POWER_DOMAIN_HIF] = { + .name = "hif", + .sta_mask = PWR_STATUS_HIF, + .ctl_offs = SPM_HIF_PWR_CON, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = GENMASK(15, 12), + .clk_id = {CLK_ETHIF}, + .caps = MTK_SCPD_ACTIVE_WAKEUP, + }, + [MT2701_POWER_DOMAIN_IFR_MSC] = { + .name = "ifr_msc", + .sta_mask = PWR_STATUS_IFR_MSC, + .ctl_offs = SPM_IFR_MSC_PWR_CON, + .clk_id = {CLK_NONE}, + .caps = MTK_SCPD_ACTIVE_WAKEUP, + }, +}; + +/* + * MT2712 power domain support + */ +static const struct scp_domain_data scp_domain_data_mt2712[] = { + [MT2712_POWER_DOMAIN_MM] = { + .name = "mm", + .sta_mask = PWR_STATUS_DISP, + .ctl_offs = SPM_DIS_PWR_CON, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .clk_id = {CLK_MM}, + .caps = MTK_SCPD_ACTIVE_WAKEUP, + }, + [MT2712_POWER_DOMAIN_VDEC] = { + .name = "vdec", + .sta_mask = PWR_STATUS_VDEC, + .ctl_offs = SPM_VDE_PWR_CON, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .clk_id = {CLK_MM, CLK_VDEC}, + .caps = MTK_SCPD_ACTIVE_WAKEUP, + }, + [MT2712_POWER_DOMAIN_VENC] = { + .name = "venc", + .sta_mask = PWR_STATUS_VENC, + .ctl_offs = SPM_VEN_PWR_CON, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = GENMASK(15, 12), + .clk_id = {CLK_MM, CLK_VENC, CLK_JPGDEC}, + .caps = MTK_SCPD_ACTIVE_WAKEUP, + }, + [MT2712_POWER_DOMAIN_ISP] = { + .name = "isp", + .sta_mask = PWR_STATUS_ISP, + .ctl_offs = SPM_ISP_PWR_CON, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = GENMASK(13, 12), + .clk_id = {CLK_MM}, + .caps = MTK_SCPD_ACTIVE_WAKEUP, + }, + [MT2712_POWER_DOMAIN_AUDIO] = { + .name = "audio", + .sta_mask = PWR_STATUS_AUDIO, + .ctl_offs = SPM_AUDIO_PWR_CON, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = GENMASK(15, 12), + .clk_id = {CLK_AUDIO}, + .caps = MTK_SCPD_ACTIVE_WAKEUP, + }, + [MT2712_POWER_DOMAIN_USB] = { + .name = "usb", + .sta_mask = PWR_STATUS_USB, + .ctl_offs = SPM_USB_PWR_CON, + .sram_pdn_bits = GENMASK(10, 8), + .sram_pdn_ack_bits = GENMASK(14, 12), + .clk_id = {CLK_NONE}, + .caps = MTK_SCPD_ACTIVE_WAKEUP, + }, + [MT2712_POWER_DOMAIN_USB2] = { + .name = "usb2", + .sta_mask = PWR_STATUS_USB2, + .ctl_offs = SPM_USB2_PWR_CON, + .sram_pdn_bits = GENMASK(10, 8), + .sram_pdn_ack_bits = GENMASK(14, 12), + .clk_id = {CLK_NONE}, + .caps = MTK_SCPD_ACTIVE_WAKEUP, + }, + [MT2712_POWER_DOMAIN_MFG] = { + .name = "mfg", + .sta_mask = PWR_STATUS_MFG, + .ctl_offs = SPM_MFG_PWR_CON, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(16, 16), + .clk_id = {CLK_MFG}, + .bus_prot_mask = BIT(14) | BIT(21) | BIT(23), + .caps = MTK_SCPD_ACTIVE_WAKEUP, + }, + [MT2712_POWER_DOMAIN_MFG_SC1] = { + .name = "mfg_sc1", + .sta_mask = BIT(22), + .ctl_offs = 0x02c0, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(16, 16), + .clk_id = {CLK_NONE}, + .caps = MTK_SCPD_ACTIVE_WAKEUP, + }, + [MT2712_POWER_DOMAIN_MFG_SC2] = { + .name = "mfg_sc2", + .sta_mask = BIT(23), + .ctl_offs = 0x02c4, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(16, 16), + .clk_id = {CLK_NONE}, + .caps = MTK_SCPD_ACTIVE_WAKEUP, + }, + [MT2712_POWER_DOMAIN_MFG_SC3] = { + .name = "mfg_sc3", + .sta_mask = BIT(30), + .ctl_offs = 0x01f8, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(16, 16), + .clk_id = {CLK_NONE}, + .caps = MTK_SCPD_ACTIVE_WAKEUP, + }, +}; + +static const struct scp_subdomain scp_subdomain_mt2712[] = { + {MT2712_POWER_DOMAIN_MM, MT2712_POWER_DOMAIN_VDEC}, + {MT2712_POWER_DOMAIN_MM, MT2712_POWER_DOMAIN_VENC}, + {MT2712_POWER_DOMAIN_MM, MT2712_POWER_DOMAIN_ISP}, + {MT2712_POWER_DOMAIN_MFG, MT2712_POWER_DOMAIN_MFG_SC1}, + {MT2712_POWER_DOMAIN_MFG_SC1, MT2712_POWER_DOMAIN_MFG_SC2}, + {MT2712_POWER_DOMAIN_MFG_SC2, MT2712_POWER_DOMAIN_MFG_SC3}, +}; + +/* + * MT6797 power domain support + */ + +static const struct scp_domain_data scp_domain_data_mt6797[] = { + [MT6797_POWER_DOMAIN_VDEC] = { + .name = "vdec", + .sta_mask = BIT(7), + .ctl_offs = 0x300, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .clk_id = {CLK_VDEC}, + }, + [MT6797_POWER_DOMAIN_VENC] = { + .name = "venc", + .sta_mask = BIT(21), + .ctl_offs = 0x304, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = GENMASK(15, 12), + .clk_id = {CLK_NONE}, + }, + [MT6797_POWER_DOMAIN_ISP] = { + .name = "isp", + .sta_mask = BIT(5), + .ctl_offs = 0x308, + .sram_pdn_bits = GENMASK(9, 8), + .sram_pdn_ack_bits = GENMASK(13, 12), + .clk_id = {CLK_NONE}, + }, + [MT6797_POWER_DOMAIN_MM] = { + .name = "mm", + .sta_mask = BIT(3), + .ctl_offs = 0x30C, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .clk_id = {CLK_MM}, + .bus_prot_mask = (BIT(1) | BIT(2)), + }, + [MT6797_POWER_DOMAIN_AUDIO] = { + .name = "audio", + .sta_mask = BIT(24), + .ctl_offs = 0x314, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = GENMASK(15, 12), + .clk_id = {CLK_NONE}, + }, + [MT6797_POWER_DOMAIN_MFG_ASYNC] = { + .name = "mfg_async", + .sta_mask = BIT(13), + .ctl_offs = 0x334, + .sram_pdn_bits = 0, + .sram_pdn_ack_bits = 0, + .clk_id = {CLK_MFG}, + }, + [MT6797_POWER_DOMAIN_MJC] = { + .name = "mjc", + .sta_mask = BIT(20), + .ctl_offs = 0x310, + .sram_pdn_bits = GENMASK(8, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .clk_id = {CLK_NONE}, + }, +}; + +#define SPM_PWR_STATUS_MT6797 0x0180 +#define SPM_PWR_STATUS_2ND_MT6797 0x0184 + +static const struct scp_subdomain scp_subdomain_mt6797[] = { + {MT6797_POWER_DOMAIN_MM, MT6797_POWER_DOMAIN_VDEC}, + {MT6797_POWER_DOMAIN_MM, MT6797_POWER_DOMAIN_ISP}, + {MT6797_POWER_DOMAIN_MM, MT6797_POWER_DOMAIN_VENC}, + {MT6797_POWER_DOMAIN_MM, MT6797_POWER_DOMAIN_MJC}, +}; + +/* + * MT7622 power domain support + */ + +static const struct scp_domain_data scp_domain_data_mt7622[] = { + [MT7622_POWER_DOMAIN_ETHSYS] = { + .name = "ethsys", + .sta_mask = PWR_STATUS_ETHSYS, + .ctl_offs = SPM_ETHSYS_PWR_CON, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = GENMASK(15, 12), + .clk_id = {CLK_NONE}, + .bus_prot_mask = MT7622_TOP_AXI_PROT_EN_ETHSYS, + .caps = MTK_SCPD_ACTIVE_WAKEUP, + }, + [MT7622_POWER_DOMAIN_HIF0] = { + .name = "hif0", + .sta_mask = PWR_STATUS_HIF0, + .ctl_offs = SPM_HIF0_PWR_CON, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = GENMASK(15, 12), + .clk_id = {CLK_HIFSEL}, + .bus_prot_mask = MT7622_TOP_AXI_PROT_EN_HIF0, + .caps = MTK_SCPD_ACTIVE_WAKEUP, + }, + [MT7622_POWER_DOMAIN_HIF1] = { + .name = "hif1", + .sta_mask = PWR_STATUS_HIF1, + .ctl_offs = SPM_HIF1_PWR_CON, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = GENMASK(15, 12), + .clk_id = {CLK_HIFSEL}, + .bus_prot_mask = MT7622_TOP_AXI_PROT_EN_HIF1, + .caps = MTK_SCPD_ACTIVE_WAKEUP, + }, + [MT7622_POWER_DOMAIN_WB] = { + .name = "wb", + .sta_mask = PWR_STATUS_WB, + .ctl_offs = SPM_WB_PWR_CON, + .sram_pdn_bits = 0, + .sram_pdn_ack_bits = 0, + .clk_id = {CLK_NONE}, + .bus_prot_mask = MT7622_TOP_AXI_PROT_EN_WB, + .caps = MTK_SCPD_ACTIVE_WAKEUP | MTK_SCPD_FWAIT_SRAM, + }, +}; + +/* + * MT7623A power domain support + */ + +static const struct scp_domain_data scp_domain_data_mt7623a[] = { + [MT7623A_POWER_DOMAIN_CONN] = { + .name = "conn", + .sta_mask = PWR_STATUS_CONN, + .ctl_offs = SPM_CONN_PWR_CON, + .bus_prot_mask = MT2701_TOP_AXI_PROT_EN_CONN_M | + MT2701_TOP_AXI_PROT_EN_CONN_S, + .clk_id = {CLK_NONE}, + .caps = MTK_SCPD_ACTIVE_WAKEUP, + }, + [MT7623A_POWER_DOMAIN_ETH] = { + .name = "eth", + .sta_mask = PWR_STATUS_ETH, + .ctl_offs = SPM_ETH_PWR_CON, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = GENMASK(15, 12), + .clk_id = {CLK_ETHIF}, + .caps = MTK_SCPD_ACTIVE_WAKEUP, + }, + [MT7623A_POWER_DOMAIN_HIF] = { + .name = "hif", + .sta_mask = PWR_STATUS_HIF, + .ctl_offs = SPM_HIF_PWR_CON, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = GENMASK(15, 12), + .clk_id = {CLK_ETHIF}, + .caps = MTK_SCPD_ACTIVE_WAKEUP, + }, + [MT7623A_POWER_DOMAIN_IFR_MSC] = { + .name = "ifr_msc", + .sta_mask = PWR_STATUS_IFR_MSC, + .ctl_offs = SPM_IFR_MSC_PWR_CON, + .clk_id = {CLK_NONE}, + .caps = MTK_SCPD_ACTIVE_WAKEUP, + }, +}; + +/* + * MT8173 power domain support + */ + +static const struct scp_domain_data scp_domain_data_mt8173[] = { + [MT8173_POWER_DOMAIN_VDEC] = { + .name = "vdec", + .sta_mask = PWR_STATUS_VDEC, + .ctl_offs = SPM_VDE_PWR_CON, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .clk_id = {CLK_MM}, + }, + [MT8173_POWER_DOMAIN_VENC] = { + .name = "venc", + .sta_mask = PWR_STATUS_VENC, + .ctl_offs = SPM_VEN_PWR_CON, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = GENMASK(15, 12), + .clk_id = {CLK_MM, CLK_VENC}, + }, + [MT8173_POWER_DOMAIN_ISP] = { + .name = "isp", + .sta_mask = PWR_STATUS_ISP, + .ctl_offs = SPM_ISP_PWR_CON, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = GENMASK(13, 12), + .clk_id = {CLK_MM}, + }, + [MT8173_POWER_DOMAIN_MM] = { + .name = "mm", + .sta_mask = PWR_STATUS_DISP, + .ctl_offs = SPM_DIS_PWR_CON, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = GENMASK(12, 12), + .clk_id = {CLK_MM}, + .bus_prot_mask = MT8173_TOP_AXI_PROT_EN_MM_M0 | + MT8173_TOP_AXI_PROT_EN_MM_M1, + }, + [MT8173_POWER_DOMAIN_VENC_LT] = { + .name = "venc_lt", + .sta_mask = PWR_STATUS_VENC_LT, + .ctl_offs = SPM_VEN2_PWR_CON, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = GENMASK(15, 12), + .clk_id = {CLK_MM, CLK_VENC_LT}, + }, + [MT8173_POWER_DOMAIN_AUDIO] = { + .name = "audio", + .sta_mask = PWR_STATUS_AUDIO, + .ctl_offs = SPM_AUDIO_PWR_CON, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = GENMASK(15, 12), + .clk_id = {CLK_NONE}, + }, + [MT8173_POWER_DOMAIN_USB] = { + .name = "usb", + .sta_mask = PWR_STATUS_USB, + .ctl_offs = SPM_USB_PWR_CON, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = GENMASK(15, 12), + .clk_id = {CLK_NONE}, + .caps = MTK_SCPD_ACTIVE_WAKEUP, + }, + [MT8173_POWER_DOMAIN_MFG_ASYNC] = { + .name = "mfg_async", + .sta_mask = PWR_STATUS_MFG_ASYNC, + .ctl_offs = SPM_MFG_ASYNC_PWR_CON, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = 0, + .clk_id = {CLK_MFG}, + }, + [MT8173_POWER_DOMAIN_MFG_2D] = { + .name = "mfg_2d", + .sta_mask = PWR_STATUS_MFG_2D, + .ctl_offs = SPM_MFG_2D_PWR_CON, + .sram_pdn_bits = GENMASK(11, 8), + .sram_pdn_ack_bits = GENMASK(13, 12), + .clk_id = {CLK_NONE}, + }, + [MT8173_POWER_DOMAIN_MFG] = { + .name = "mfg", + .sta_mask = PWR_STATUS_MFG, + .ctl_offs = SPM_MFG_PWR_CON, + .sram_pdn_bits = GENMASK(13, 8), + .sram_pdn_ack_bits = GENMASK(21, 16), + .clk_id = {CLK_NONE}, + .bus_prot_mask = MT8173_TOP_AXI_PROT_EN_MFG_S | + MT8173_TOP_AXI_PROT_EN_MFG_M0 | + MT8173_TOP_AXI_PROT_EN_MFG_M1 | + MT8173_TOP_AXI_PROT_EN_MFG_SNOOP_OUT, + }, +}; + +static const struct scp_subdomain scp_subdomain_mt8173[] = { + {MT8173_POWER_DOMAIN_MFG_ASYNC, MT8173_POWER_DOMAIN_MFG_2D}, + {MT8173_POWER_DOMAIN_MFG_2D, MT8173_POWER_DOMAIN_MFG}, +}; + +static const struct scp_soc_data mt2701_data = { + .domains = scp_domain_data_mt2701, + .num_domains = ARRAY_SIZE(scp_domain_data_mt2701), + .regs = { + .pwr_sta_offs = SPM_PWR_STATUS, + .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND + }, + .bus_prot_reg_update = true, +}; + +static const struct scp_soc_data mt2712_data = { + .domains = scp_domain_data_mt2712, + .num_domains = ARRAY_SIZE(scp_domain_data_mt2712), + .subdomains = scp_subdomain_mt2712, + .num_subdomains = ARRAY_SIZE(scp_subdomain_mt2712), + .regs = { + .pwr_sta_offs = SPM_PWR_STATUS, + .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND + }, + .bus_prot_reg_update = false, +}; + +static const struct scp_soc_data mt6797_data = { + .domains = scp_domain_data_mt6797, + .num_domains = ARRAY_SIZE(scp_domain_data_mt6797), + .subdomains = scp_subdomain_mt6797, + .num_subdomains = ARRAY_SIZE(scp_subdomain_mt6797), + .regs = { + .pwr_sta_offs = SPM_PWR_STATUS_MT6797, + .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND_MT6797 + }, + .bus_prot_reg_update = true, +}; + +static const struct scp_soc_data mt7622_data = { + .domains = scp_domain_data_mt7622, + .num_domains = ARRAY_SIZE(scp_domain_data_mt7622), + .regs = { + .pwr_sta_offs = SPM_PWR_STATUS, + .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND + }, + .bus_prot_reg_update = true, +}; + +static const struct scp_soc_data mt7623a_data = { + .domains = scp_domain_data_mt7623a, + .num_domains = ARRAY_SIZE(scp_domain_data_mt7623a), + .regs = { + .pwr_sta_offs = SPM_PWR_STATUS, + .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND + }, + .bus_prot_reg_update = true, +}; + +static const struct scp_soc_data mt8173_data = { + .domains = scp_domain_data_mt8173, + .num_domains = ARRAY_SIZE(scp_domain_data_mt8173), + .subdomains = scp_subdomain_mt8173, + .num_subdomains = ARRAY_SIZE(scp_subdomain_mt8173), + .regs = { + .pwr_sta_offs = SPM_PWR_STATUS, + .pwr_sta2nd_offs = SPM_PWR_STATUS_2ND + }, + .bus_prot_reg_update = true, +}; + +/* + * scpsys driver init + */ + +static const struct of_device_id of_scpsys_match_tbl[] = { + { + .compatible = "mediatek,mt2701-scpsys", + .data = &mt2701_data, + }, { + .compatible = "mediatek,mt2712-scpsys", + .data = &mt2712_data, + }, { + .compatible = "mediatek,mt6797-scpsys", + .data = &mt6797_data, + }, { + .compatible = "mediatek,mt7622-scpsys", + .data = &mt7622_data, + }, { + .compatible = "mediatek,mt7623a-scpsys", + .data = &mt7623a_data, + }, { + .compatible = "mediatek,mt8173-scpsys", + .data = &mt8173_data, + }, { + /* sentinel */ + } +}; + +static int scpsys_probe(struct platform_device *pdev) +{ + const struct scp_subdomain *sd; + const struct scp_soc_data *soc; + struct scp *scp; + struct genpd_onecell_data *pd_data; + int i, ret; + + soc = of_device_get_match_data(&pdev->dev); + + scp = init_scp(pdev, soc->domains, soc->num_domains, &soc->regs, + soc->bus_prot_reg_update); + if (IS_ERR(scp)) + return PTR_ERR(scp); + + mtk_register_power_domains(pdev, scp, soc->num_domains); + + pd_data = &scp->pd_data; + + for (i = 0, sd = soc->subdomains; i < soc->num_subdomains; i++, sd++) { + ret = pm_genpd_add_subdomain(pd_data->domains[sd->origin], + pd_data->domains[sd->subdomain]); + if (ret && IS_ENABLED(CONFIG_PM)) + dev_err(&pdev->dev, "Failed to add subdomain: %d\n", + ret); + } + + return 0; +} + +static struct platform_driver scpsys_drv = { + .probe = scpsys_probe, + .driver = { + .name = "mtk-scpsys", + .suppress_bind_attrs = true, + .owner = THIS_MODULE, + .of_match_table = of_scpsys_match_tbl, + }, +}; +builtin_platform_driver(scpsys_drv); diff --git a/drivers/pmdomain/qcom/Makefile b/drivers/pmdomain/qcom/Makefile new file mode 100644 index 000000000000..403dfc5af095 --- /dev/null +++ b/drivers/pmdomain/qcom/Makefile @@ -0,0 +1,4 @@ +# SPDX-License-Identifier: GPL-2.0 +obj-$(CONFIG_QCOM_CPR) += cpr.o +obj-$(CONFIG_QCOM_RPMPD) += rpmpd.o +obj-$(CONFIG_QCOM_RPMHPD) += rpmhpd.o diff --git a/drivers/pmdomain/qcom/cpr.c b/drivers/pmdomain/qcom/cpr.c new file mode 100644 index 000000000000..94a3f0977212 --- /dev/null +++ b/drivers/pmdomain/qcom/cpr.c @@ -0,0 +1,1756 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (c) 2013-2015, The Linux Foundation. All rights reserved. + * Copyright (c) 2019, Linaro Limited + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +/* Register Offsets for RB-CPR and Bit Definitions */ + +/* RBCPR Version Register */ +#define REG_RBCPR_VERSION 0 +#define RBCPR_VER_2 0x02 +#define FLAGS_IGNORE_1ST_IRQ_STATUS BIT(0) + +/* RBCPR Gate Count and Target Registers */ +#define REG_RBCPR_GCNT_TARGET(n) (0x60 + 4 * (n)) + +#define RBCPR_GCNT_TARGET_TARGET_SHIFT 0 +#define RBCPR_GCNT_TARGET_TARGET_MASK GENMASK(11, 0) +#define RBCPR_GCNT_TARGET_GCNT_SHIFT 12 +#define RBCPR_GCNT_TARGET_GCNT_MASK GENMASK(9, 0) + +/* RBCPR Timer Control */ +#define REG_RBCPR_TIMER_INTERVAL 0x44 +#define REG_RBIF_TIMER_ADJUST 0x4c + +#define RBIF_TIMER_ADJ_CONS_UP_MASK GENMASK(3, 0) +#define RBIF_TIMER_ADJ_CONS_UP_SHIFT 0 +#define RBIF_TIMER_ADJ_CONS_DOWN_MASK GENMASK(3, 0) +#define RBIF_TIMER_ADJ_CONS_DOWN_SHIFT 4 +#define RBIF_TIMER_ADJ_CLAMP_INT_MASK GENMASK(7, 0) +#define RBIF_TIMER_ADJ_CLAMP_INT_SHIFT 8 + +/* RBCPR Config Register */ +#define REG_RBIF_LIMIT 0x48 +#define RBIF_LIMIT_CEILING_MASK GENMASK(5, 0) +#define RBIF_LIMIT_CEILING_SHIFT 6 +#define RBIF_LIMIT_FLOOR_BITS 6 +#define RBIF_LIMIT_FLOOR_MASK GENMASK(5, 0) + +#define RBIF_LIMIT_CEILING_DEFAULT RBIF_LIMIT_CEILING_MASK +#define RBIF_LIMIT_FLOOR_DEFAULT 0 + +#define REG_RBIF_SW_VLEVEL 0x94 +#define RBIF_SW_VLEVEL_DEFAULT 0x20 + +#define REG_RBCPR_STEP_QUOT 0x80 +#define RBCPR_STEP_QUOT_STEPQUOT_MASK GENMASK(7, 0) +#define RBCPR_STEP_QUOT_IDLE_CLK_MASK GENMASK(3, 0) +#define RBCPR_STEP_QUOT_IDLE_CLK_SHIFT 8 + +/* RBCPR Control Register */ +#define REG_RBCPR_CTL 0x90 + +#define RBCPR_CTL_LOOP_EN BIT(0) +#define RBCPR_CTL_TIMER_EN BIT(3) +#define RBCPR_CTL_SW_AUTO_CONT_ACK_EN BIT(5) +#define RBCPR_CTL_SW_AUTO_CONT_NACK_DN_EN BIT(6) +#define RBCPR_CTL_COUNT_MODE BIT(10) +#define RBCPR_CTL_UP_THRESHOLD_MASK GENMASK(3, 0) +#define RBCPR_CTL_UP_THRESHOLD_SHIFT 24 +#define RBCPR_CTL_DN_THRESHOLD_MASK GENMASK(3, 0) +#define RBCPR_CTL_DN_THRESHOLD_SHIFT 28 + +/* RBCPR Ack/Nack Response */ +#define REG_RBIF_CONT_ACK_CMD 0x98 +#define REG_RBIF_CONT_NACK_CMD 0x9c + +/* RBCPR Result status Register */ +#define REG_RBCPR_RESULT_0 0xa0 + +#define RBCPR_RESULT0_BUSY_SHIFT 19 +#define RBCPR_RESULT0_BUSY_MASK BIT(RBCPR_RESULT0_BUSY_SHIFT) +#define RBCPR_RESULT0_ERROR_LT0_SHIFT 18 +#define RBCPR_RESULT0_ERROR_SHIFT 6 +#define RBCPR_RESULT0_ERROR_MASK GENMASK(11, 0) +#define RBCPR_RESULT0_ERROR_STEPS_SHIFT 2 +#define RBCPR_RESULT0_ERROR_STEPS_MASK GENMASK(3, 0) +#define RBCPR_RESULT0_STEP_UP_SHIFT 1 + +/* RBCPR Interrupt Control Register */ +#define REG_RBIF_IRQ_EN(n) (0x100 + 4 * (n)) +#define REG_RBIF_IRQ_CLEAR 0x110 +#define REG_RBIF_IRQ_STATUS 0x114 + +#define CPR_INT_DONE BIT(0) +#define CPR_INT_MIN BIT(1) +#define CPR_INT_DOWN BIT(2) +#define CPR_INT_MID BIT(3) +#define CPR_INT_UP BIT(4) +#define CPR_INT_MAX BIT(5) +#define CPR_INT_CLAMP BIT(6) +#define CPR_INT_ALL (CPR_INT_DONE | CPR_INT_MIN | CPR_INT_DOWN | \ + CPR_INT_MID | CPR_INT_UP | CPR_INT_MAX | CPR_INT_CLAMP) +#define CPR_INT_DEFAULT (CPR_INT_UP | CPR_INT_DOWN) + +#define CPR_NUM_RING_OSC 8 + +/* CPR eFuse parameters */ +#define CPR_FUSE_TARGET_QUOT_BITS_MASK GENMASK(11, 0) + +#define CPR_FUSE_MIN_QUOT_DIFF 50 + +#define FUSE_REVISION_UNKNOWN (-1) + +enum voltage_change_dir { + NO_CHANGE, + DOWN, + UP, +}; + +struct cpr_fuse { + char *ring_osc; + char *init_voltage; + char *quotient; + char *quotient_offset; +}; + +struct fuse_corner_data { + int ref_uV; + int max_uV; + int min_uV; + int max_volt_scale; + int max_quot_scale; + /* fuse quot */ + int quot_offset; + int quot_scale; + int quot_adjust; + /* fuse quot_offset */ + int quot_offset_scale; + int quot_offset_adjust; +}; + +struct cpr_fuses { + int init_voltage_step; + int init_voltage_width; + struct fuse_corner_data *fuse_corner_data; +}; + +struct corner_data { + unsigned int fuse_corner; + unsigned long freq; +}; + +struct cpr_desc { + unsigned int num_fuse_corners; + int min_diff_quot; + int *step_quot; + + unsigned int timer_delay_us; + unsigned int timer_cons_up; + unsigned int timer_cons_down; + unsigned int up_threshold; + unsigned int down_threshold; + unsigned int idle_clocks; + unsigned int gcnt_us; + unsigned int vdd_apc_step_up_limit; + unsigned int vdd_apc_step_down_limit; + unsigned int clamp_timer_interval; + + struct cpr_fuses cpr_fuses; + bool reduce_to_fuse_uV; + bool reduce_to_corner_uV; +}; + +struct acc_desc { + unsigned int enable_reg; + u32 enable_mask; + + struct reg_sequence *config; + struct reg_sequence *settings; + int num_regs_per_fuse; +}; + +struct cpr_acc_desc { + const struct cpr_desc *cpr_desc; + const struct acc_desc *acc_desc; +}; + +struct fuse_corner { + int min_uV; + int max_uV; + int uV; + int quot; + int step_quot; + const struct reg_sequence *accs; + int num_accs; + unsigned long max_freq; + u8 ring_osc_idx; +}; + +struct corner { + int min_uV; + int max_uV; + int uV; + int last_uV; + int quot_adjust; + u32 save_ctl; + u32 save_irq; + unsigned long freq; + struct fuse_corner *fuse_corner; +}; + +struct cpr_drv { + unsigned int num_corners; + unsigned int ref_clk_khz; + + struct generic_pm_domain pd; + struct device *dev; + struct device *attached_cpu_dev; + struct mutex lock; + void __iomem *base; + struct corner *corner; + struct regulator *vdd_apc; + struct clk *cpu_clk; + struct regmap *tcsr; + bool loop_disabled; + u32 gcnt; + unsigned long flags; + + struct fuse_corner *fuse_corners; + struct corner *corners; + + const struct cpr_desc *desc; + const struct acc_desc *acc_desc; + const struct cpr_fuse *cpr_fuses; + + struct dentry *debugfs; +}; + +static bool cpr_is_allowed(struct cpr_drv *drv) +{ + return !drv->loop_disabled; +} + +static void cpr_write(struct cpr_drv *drv, u32 offset, u32 value) +{ + writel_relaxed(value, drv->base + offset); +} + +static u32 cpr_read(struct cpr_drv *drv, u32 offset) +{ + return readl_relaxed(drv->base + offset); +} + +static void +cpr_masked_write(struct cpr_drv *drv, u32 offset, u32 mask, u32 value) +{ + u32 val; + + val = readl_relaxed(drv->base + offset); + val &= ~mask; + val |= value & mask; + writel_relaxed(val, drv->base + offset); +} + +static void cpr_irq_clr(struct cpr_drv *drv) +{ + cpr_write(drv, REG_RBIF_IRQ_CLEAR, CPR_INT_ALL); +} + +static void cpr_irq_clr_nack(struct cpr_drv *drv) +{ + cpr_irq_clr(drv); + cpr_write(drv, REG_RBIF_CONT_NACK_CMD, 1); +} + +static void cpr_irq_clr_ack(struct cpr_drv *drv) +{ + cpr_irq_clr(drv); + cpr_write(drv, REG_RBIF_CONT_ACK_CMD, 1); +} + +static void cpr_irq_set(struct cpr_drv *drv, u32 int_bits) +{ + cpr_write(drv, REG_RBIF_IRQ_EN(0), int_bits); +} + +static void cpr_ctl_modify(struct cpr_drv *drv, u32 mask, u32 value) +{ + cpr_masked_write(drv, REG_RBCPR_CTL, mask, value); +} + +static void cpr_ctl_enable(struct cpr_drv *drv, struct corner *corner) +{ + u32 val, mask; + const struct cpr_desc *desc = drv->desc; + + /* Program Consecutive Up & Down */ + val = desc->timer_cons_down << RBIF_TIMER_ADJ_CONS_DOWN_SHIFT; + val |= desc->timer_cons_up << RBIF_TIMER_ADJ_CONS_UP_SHIFT; + mask = RBIF_TIMER_ADJ_CONS_UP_MASK | RBIF_TIMER_ADJ_CONS_DOWN_MASK; + cpr_masked_write(drv, REG_RBIF_TIMER_ADJUST, mask, val); + cpr_masked_write(drv, REG_RBCPR_CTL, + RBCPR_CTL_SW_AUTO_CONT_NACK_DN_EN | + RBCPR_CTL_SW_AUTO_CONT_ACK_EN, + corner->save_ctl); + cpr_irq_set(drv, corner->save_irq); + + if (cpr_is_allowed(drv) && corner->max_uV > corner->min_uV) + val = RBCPR_CTL_LOOP_EN; + else + val = 0; + cpr_ctl_modify(drv, RBCPR_CTL_LOOP_EN, val); +} + +static void cpr_ctl_disable(struct cpr_drv *drv) +{ + cpr_irq_set(drv, 0); + cpr_ctl_modify(drv, RBCPR_CTL_SW_AUTO_CONT_NACK_DN_EN | + RBCPR_CTL_SW_AUTO_CONT_ACK_EN, 0); + cpr_masked_write(drv, REG_RBIF_TIMER_ADJUST, + RBIF_TIMER_ADJ_CONS_UP_MASK | + RBIF_TIMER_ADJ_CONS_DOWN_MASK, 0); + cpr_irq_clr(drv); + cpr_write(drv, REG_RBIF_CONT_ACK_CMD, 1); + cpr_write(drv, REG_RBIF_CONT_NACK_CMD, 1); + cpr_ctl_modify(drv, RBCPR_CTL_LOOP_EN, 0); +} + +static bool cpr_ctl_is_enabled(struct cpr_drv *drv) +{ + u32 reg_val; + + reg_val = cpr_read(drv, REG_RBCPR_CTL); + return reg_val & RBCPR_CTL_LOOP_EN; +} + +static bool cpr_ctl_is_busy(struct cpr_drv *drv) +{ + u32 reg_val; + + reg_val = cpr_read(drv, REG_RBCPR_RESULT_0); + return reg_val & RBCPR_RESULT0_BUSY_MASK; +} + +static void cpr_corner_save(struct cpr_drv *drv, struct corner *corner) +{ + corner->save_ctl = cpr_read(drv, REG_RBCPR_CTL); + corner->save_irq = cpr_read(drv, REG_RBIF_IRQ_EN(0)); +} + +static void cpr_corner_restore(struct cpr_drv *drv, struct corner *corner) +{ + u32 gcnt, ctl, irq, ro_sel, step_quot; + struct fuse_corner *fuse = corner->fuse_corner; + const struct cpr_desc *desc = drv->desc; + int i; + + ro_sel = fuse->ring_osc_idx; + gcnt = drv->gcnt; + gcnt |= fuse->quot - corner->quot_adjust; + + /* Program the step quotient and idle clocks */ + step_quot = desc->idle_clocks << RBCPR_STEP_QUOT_IDLE_CLK_SHIFT; + step_quot |= fuse->step_quot & RBCPR_STEP_QUOT_STEPQUOT_MASK; + cpr_write(drv, REG_RBCPR_STEP_QUOT, step_quot); + + /* Clear the target quotient value and gate count of all ROs */ + for (i = 0; i < CPR_NUM_RING_OSC; i++) + cpr_write(drv, REG_RBCPR_GCNT_TARGET(i), 0); + + cpr_write(drv, REG_RBCPR_GCNT_TARGET(ro_sel), gcnt); + ctl = corner->save_ctl; + cpr_write(drv, REG_RBCPR_CTL, ctl); + irq = corner->save_irq; + cpr_irq_set(drv, irq); + dev_dbg(drv->dev, "gcnt = %#08x, ctl = %#08x, irq = %#08x\n", gcnt, + ctl, irq); +} + +static void cpr_set_acc(struct regmap *tcsr, struct fuse_corner *f, + struct fuse_corner *end) +{ + if (f == end) + return; + + if (f < end) { + for (f += 1; f <= end; f++) + regmap_multi_reg_write(tcsr, f->accs, f->num_accs); + } else { + for (f -= 1; f >= end; f--) + regmap_multi_reg_write(tcsr, f->accs, f->num_accs); + } +} + +static int cpr_pre_voltage(struct cpr_drv *drv, + struct fuse_corner *fuse_corner, + enum voltage_change_dir dir) +{ + struct fuse_corner *prev_fuse_corner = drv->corner->fuse_corner; + + if (drv->tcsr && dir == DOWN) + cpr_set_acc(drv->tcsr, prev_fuse_corner, fuse_corner); + + return 0; +} + +static int cpr_post_voltage(struct cpr_drv *drv, + struct fuse_corner *fuse_corner, + enum voltage_change_dir dir) +{ + struct fuse_corner *prev_fuse_corner = drv->corner->fuse_corner; + + if (drv->tcsr && dir == UP) + cpr_set_acc(drv->tcsr, prev_fuse_corner, fuse_corner); + + return 0; +} + +static int cpr_scale_voltage(struct cpr_drv *drv, struct corner *corner, + int new_uV, enum voltage_change_dir dir) +{ + int ret; + struct fuse_corner *fuse_corner = corner->fuse_corner; + + ret = cpr_pre_voltage(drv, fuse_corner, dir); + if (ret) + return ret; + + ret = regulator_set_voltage(drv->vdd_apc, new_uV, new_uV); + if (ret) { + dev_err_ratelimited(drv->dev, "failed to set apc voltage %d\n", + new_uV); + return ret; + } + + ret = cpr_post_voltage(drv, fuse_corner, dir); + if (ret) + return ret; + + return 0; +} + +static unsigned int cpr_get_cur_perf_state(struct cpr_drv *drv) +{ + return drv->corner ? drv->corner - drv->corners + 1 : 0; +} + +static int cpr_scale(struct cpr_drv *drv, enum voltage_change_dir dir) +{ + u32 val, error_steps, reg_mask; + int last_uV, new_uV, step_uV, ret; + struct corner *corner; + const struct cpr_desc *desc = drv->desc; + + if (dir != UP && dir != DOWN) + return 0; + + step_uV = regulator_get_linear_step(drv->vdd_apc); + if (!step_uV) + return -EINVAL; + + corner = drv->corner; + + val = cpr_read(drv, REG_RBCPR_RESULT_0); + + error_steps = val >> RBCPR_RESULT0_ERROR_STEPS_SHIFT; + error_steps &= RBCPR_RESULT0_ERROR_STEPS_MASK; + last_uV = corner->last_uV; + + if (dir == UP) { + if (desc->clamp_timer_interval && + error_steps < desc->up_threshold) { + /* + * Handle the case where another measurement started + * after the interrupt was triggered due to a core + * exiting from power collapse. + */ + error_steps = max(desc->up_threshold, + desc->vdd_apc_step_up_limit); + } + + if (last_uV >= corner->max_uV) { + cpr_irq_clr_nack(drv); + + /* Maximize the UP threshold */ + reg_mask = RBCPR_CTL_UP_THRESHOLD_MASK; + reg_mask <<= RBCPR_CTL_UP_THRESHOLD_SHIFT; + val = reg_mask; + cpr_ctl_modify(drv, reg_mask, val); + + /* Disable UP interrupt */ + cpr_irq_set(drv, CPR_INT_DEFAULT & ~CPR_INT_UP); + + return 0; + } + + if (error_steps > desc->vdd_apc_step_up_limit) + error_steps = desc->vdd_apc_step_up_limit; + + /* Calculate new voltage */ + new_uV = last_uV + error_steps * step_uV; + new_uV = min(new_uV, corner->max_uV); + + dev_dbg(drv->dev, + "UP: -> new_uV: %d last_uV: %d perf state: %u\n", + new_uV, last_uV, cpr_get_cur_perf_state(drv)); + } else { + if (desc->clamp_timer_interval && + error_steps < desc->down_threshold) { + /* + * Handle the case where another measurement started + * after the interrupt was triggered due to a core + * exiting from power collapse. + */ + error_steps = max(desc->down_threshold, + desc->vdd_apc_step_down_limit); + } + + if (last_uV <= corner->min_uV) { + cpr_irq_clr_nack(drv); + + /* Enable auto nack down */ + reg_mask = RBCPR_CTL_SW_AUTO_CONT_NACK_DN_EN; + val = RBCPR_CTL_SW_AUTO_CONT_NACK_DN_EN; + + cpr_ctl_modify(drv, reg_mask, val); + + /* Disable DOWN interrupt */ + cpr_irq_set(drv, CPR_INT_DEFAULT & ~CPR_INT_DOWN); + + return 0; + } + + if (error_steps > desc->vdd_apc_step_down_limit) + error_steps = desc->vdd_apc_step_down_limit; + + /* Calculate new voltage */ + new_uV = last_uV - error_steps * step_uV; + new_uV = max(new_uV, corner->min_uV); + + dev_dbg(drv->dev, + "DOWN: -> new_uV: %d last_uV: %d perf state: %u\n", + new_uV, last_uV, cpr_get_cur_perf_state(drv)); + } + + ret = cpr_scale_voltage(drv, corner, new_uV, dir); + if (ret) { + cpr_irq_clr_nack(drv); + return ret; + } + drv->corner->last_uV = new_uV; + + if (dir == UP) { + /* Disable auto nack down */ + reg_mask = RBCPR_CTL_SW_AUTO_CONT_NACK_DN_EN; + val = 0; + } else { + /* Restore default threshold for UP */ + reg_mask = RBCPR_CTL_UP_THRESHOLD_MASK; + reg_mask <<= RBCPR_CTL_UP_THRESHOLD_SHIFT; + val = desc->up_threshold; + val <<= RBCPR_CTL_UP_THRESHOLD_SHIFT; + } + + cpr_ctl_modify(drv, reg_mask, val); + + /* Re-enable default interrupts */ + cpr_irq_set(drv, CPR_INT_DEFAULT); + + /* Ack */ + cpr_irq_clr_ack(drv); + + return 0; +} + +static irqreturn_t cpr_irq_handler(int irq, void *dev) +{ + struct cpr_drv *drv = dev; + const struct cpr_desc *desc = drv->desc; + irqreturn_t ret = IRQ_HANDLED; + u32 val; + + mutex_lock(&drv->lock); + + val = cpr_read(drv, REG_RBIF_IRQ_STATUS); + if (drv->flags & FLAGS_IGNORE_1ST_IRQ_STATUS) + val = cpr_read(drv, REG_RBIF_IRQ_STATUS); + + dev_dbg(drv->dev, "IRQ_STATUS = %#02x\n", val); + + if (!cpr_ctl_is_enabled(drv)) { + dev_dbg(drv->dev, "CPR is disabled\n"); + ret = IRQ_NONE; + } else if (cpr_ctl_is_busy(drv) && !desc->clamp_timer_interval) { + dev_dbg(drv->dev, "CPR measurement is not ready\n"); + } else if (!cpr_is_allowed(drv)) { + val = cpr_read(drv, REG_RBCPR_CTL); + dev_err_ratelimited(drv->dev, + "Interrupt broken? RBCPR_CTL = %#02x\n", + val); + ret = IRQ_NONE; + } else { + /* + * Following sequence of handling is as per each IRQ's + * priority + */ + if (val & CPR_INT_UP) { + cpr_scale(drv, UP); + } else if (val & CPR_INT_DOWN) { + cpr_scale(drv, DOWN); + } else if (val & CPR_INT_MIN) { + cpr_irq_clr_nack(drv); + } else if (val & CPR_INT_MAX) { + cpr_irq_clr_nack(drv); + } else if (val & CPR_INT_MID) { + /* RBCPR_CTL_SW_AUTO_CONT_ACK_EN is enabled */ + dev_dbg(drv->dev, "IRQ occurred for Mid Flag\n"); + } else { + dev_dbg(drv->dev, + "IRQ occurred for unknown flag (%#08x)\n", val); + } + + /* Save register values for the corner */ + cpr_corner_save(drv, drv->corner); + } + + mutex_unlock(&drv->lock); + + return ret; +} + +static int cpr_enable(struct cpr_drv *drv) +{ + int ret; + + ret = regulator_enable(drv->vdd_apc); + if (ret) + return ret; + + mutex_lock(&drv->lock); + + if (cpr_is_allowed(drv) && drv->corner) { + cpr_irq_clr(drv); + cpr_corner_restore(drv, drv->corner); + cpr_ctl_enable(drv, drv->corner); + } + + mutex_unlock(&drv->lock); + + return 0; +} + +static int cpr_disable(struct cpr_drv *drv) +{ + mutex_lock(&drv->lock); + + if (cpr_is_allowed(drv)) { + cpr_ctl_disable(drv); + cpr_irq_clr(drv); + } + + mutex_unlock(&drv->lock); + + return regulator_disable(drv->vdd_apc); +} + +static int cpr_config(struct cpr_drv *drv) +{ + int i; + u32 val, gcnt; + struct corner *corner; + const struct cpr_desc *desc = drv->desc; + + /* Disable interrupt and CPR */ + cpr_write(drv, REG_RBIF_IRQ_EN(0), 0); + cpr_write(drv, REG_RBCPR_CTL, 0); + + /* Program the default HW ceiling, floor and vlevel */ + val = (RBIF_LIMIT_CEILING_DEFAULT & RBIF_LIMIT_CEILING_MASK) + << RBIF_LIMIT_CEILING_SHIFT; + val |= RBIF_LIMIT_FLOOR_DEFAULT & RBIF_LIMIT_FLOOR_MASK; + cpr_write(drv, REG_RBIF_LIMIT, val); + cpr_write(drv, REG_RBIF_SW_VLEVEL, RBIF_SW_VLEVEL_DEFAULT); + + /* + * Clear the target quotient value and gate count of all + * ring oscillators + */ + for (i = 0; i < CPR_NUM_RING_OSC; i++) + cpr_write(drv, REG_RBCPR_GCNT_TARGET(i), 0); + + /* Init and save gcnt */ + gcnt = (drv->ref_clk_khz * desc->gcnt_us) / 1000; + gcnt = gcnt & RBCPR_GCNT_TARGET_GCNT_MASK; + gcnt <<= RBCPR_GCNT_TARGET_GCNT_SHIFT; + drv->gcnt = gcnt; + + /* Program the delay count for the timer */ + val = (drv->ref_clk_khz * desc->timer_delay_us) / 1000; + cpr_write(drv, REG_RBCPR_TIMER_INTERVAL, val); + dev_dbg(drv->dev, "Timer count: %#0x (for %d us)\n", val, + desc->timer_delay_us); + + /* Program Consecutive Up & Down */ + val = desc->timer_cons_down << RBIF_TIMER_ADJ_CONS_DOWN_SHIFT; + val |= desc->timer_cons_up << RBIF_TIMER_ADJ_CONS_UP_SHIFT; + val |= desc->clamp_timer_interval << RBIF_TIMER_ADJ_CLAMP_INT_SHIFT; + cpr_write(drv, REG_RBIF_TIMER_ADJUST, val); + + /* Program the control register */ + val = desc->up_threshold << RBCPR_CTL_UP_THRESHOLD_SHIFT; + val |= desc->down_threshold << RBCPR_CTL_DN_THRESHOLD_SHIFT; + val |= RBCPR_CTL_TIMER_EN | RBCPR_CTL_COUNT_MODE; + val |= RBCPR_CTL_SW_AUTO_CONT_ACK_EN; + cpr_write(drv, REG_RBCPR_CTL, val); + + for (i = 0; i < drv->num_corners; i++) { + corner = &drv->corners[i]; + corner->save_ctl = val; + corner->save_irq = CPR_INT_DEFAULT; + } + + cpr_irq_set(drv, CPR_INT_DEFAULT); + + val = cpr_read(drv, REG_RBCPR_VERSION); + if (val <= RBCPR_VER_2) + drv->flags |= FLAGS_IGNORE_1ST_IRQ_STATUS; + + return 0; +} + +static int cpr_set_performance_state(struct generic_pm_domain *domain, + unsigned int state) +{ + struct cpr_drv *drv = container_of(domain, struct cpr_drv, pd); + struct corner *corner, *end; + enum voltage_change_dir dir; + int ret = 0, new_uV; + + mutex_lock(&drv->lock); + + dev_dbg(drv->dev, "%s: setting perf state: %u (prev state: %u)\n", + __func__, state, cpr_get_cur_perf_state(drv)); + + /* + * Determine new corner we're going to. + * Remove one since lowest performance state is 1. + */ + corner = drv->corners + state - 1; + end = &drv->corners[drv->num_corners - 1]; + if (corner > end || corner < drv->corners) { + ret = -EINVAL; + goto unlock; + } + + /* Determine direction */ + if (drv->corner > corner) + dir = DOWN; + else if (drv->corner < corner) + dir = UP; + else + dir = NO_CHANGE; + + if (cpr_is_allowed(drv)) + new_uV = corner->last_uV; + else + new_uV = corner->uV; + + if (cpr_is_allowed(drv)) + cpr_ctl_disable(drv); + + ret = cpr_scale_voltage(drv, corner, new_uV, dir); + if (ret) + goto unlock; + + if (cpr_is_allowed(drv)) { + cpr_irq_clr(drv); + if (drv->corner != corner) + cpr_corner_restore(drv, corner); + cpr_ctl_enable(drv, corner); + } + + drv->corner = corner; + +unlock: + mutex_unlock(&drv->lock); + + return ret; +} + +static int +cpr_populate_ring_osc_idx(struct cpr_drv *drv) +{ + struct fuse_corner *fuse = drv->fuse_corners; + struct fuse_corner *end = fuse + drv->desc->num_fuse_corners; + const struct cpr_fuse *fuses = drv->cpr_fuses; + u32 data; + int ret; + + for (; fuse < end; fuse++, fuses++) { + ret = nvmem_cell_read_variable_le_u32(drv->dev, fuses->ring_osc, &data); + if (ret) + return ret; + fuse->ring_osc_idx = data; + } + + return 0; +} + +static int cpr_read_fuse_uV(const struct cpr_desc *desc, + const struct fuse_corner_data *fdata, + const char *init_v_efuse, + int step_volt, + struct cpr_drv *drv) +{ + int step_size_uV, steps, uV; + u32 bits = 0; + int ret; + + ret = nvmem_cell_read_variable_le_u32(drv->dev, init_v_efuse, &bits); + if (ret) + return ret; + + steps = bits & ~BIT(desc->cpr_fuses.init_voltage_width - 1); + /* Not two's complement.. instead highest bit is sign bit */ + if (bits & BIT(desc->cpr_fuses.init_voltage_width - 1)) + steps = -steps; + + step_size_uV = desc->cpr_fuses.init_voltage_step; + + uV = fdata->ref_uV + steps * step_size_uV; + return DIV_ROUND_UP(uV, step_volt) * step_volt; +} + +static int cpr_fuse_corner_init(struct cpr_drv *drv) +{ + const struct cpr_desc *desc = drv->desc; + const struct cpr_fuse *fuses = drv->cpr_fuses; + const struct acc_desc *acc_desc = drv->acc_desc; + int i; + unsigned int step_volt; + struct fuse_corner_data *fdata; + struct fuse_corner *fuse, *end; + int uV; + const struct reg_sequence *accs; + int ret; + + accs = acc_desc->settings; + + step_volt = regulator_get_linear_step(drv->vdd_apc); + if (!step_volt) + return -EINVAL; + + /* Populate fuse_corner members */ + fuse = drv->fuse_corners; + end = &fuse[desc->num_fuse_corners - 1]; + fdata = desc->cpr_fuses.fuse_corner_data; + + for (i = 0; fuse <= end; fuse++, fuses++, i++, fdata++) { + /* + * Update SoC voltages: platforms might choose a different + * regulators than the one used to characterize the algorithms + * (ie, init_voltage_step). + */ + fdata->min_uV = roundup(fdata->min_uV, step_volt); + fdata->max_uV = roundup(fdata->max_uV, step_volt); + + /* Populate uV */ + uV = cpr_read_fuse_uV(desc, fdata, fuses->init_voltage, + step_volt, drv); + if (uV < 0) + return uV; + + fuse->min_uV = fdata->min_uV; + fuse->max_uV = fdata->max_uV; + fuse->uV = clamp(uV, fuse->min_uV, fuse->max_uV); + + if (fuse == end) { + /* + * Allow the highest fuse corner's PVS voltage to + * define the ceiling voltage for that corner in order + * to support SoC's in which variable ceiling values + * are required. + */ + end->max_uV = max(end->max_uV, end->uV); + } + + /* Populate target quotient by scaling */ + ret = nvmem_cell_read_variable_le_u32(drv->dev, fuses->quotient, &fuse->quot); + if (ret) + return ret; + + fuse->quot *= fdata->quot_scale; + fuse->quot += fdata->quot_offset; + fuse->quot += fdata->quot_adjust; + fuse->step_quot = desc->step_quot[fuse->ring_osc_idx]; + + /* Populate acc settings */ + fuse->accs = accs; + fuse->num_accs = acc_desc->num_regs_per_fuse; + accs += acc_desc->num_regs_per_fuse; + } + + /* + * Restrict all fuse corner PVS voltages based upon per corner + * ceiling and floor voltages. + */ + for (fuse = drv->fuse_corners, i = 0; fuse <= end; fuse++, i++) { + if (fuse->uV > fuse->max_uV) + fuse->uV = fuse->max_uV; + else if (fuse->uV < fuse->min_uV) + fuse->uV = fuse->min_uV; + + ret = regulator_is_supported_voltage(drv->vdd_apc, + fuse->min_uV, + fuse->min_uV); + if (!ret) { + dev_err(drv->dev, + "min uV: %d (fuse corner: %d) not supported by regulator\n", + fuse->min_uV, i); + return -EINVAL; + } + + ret = regulator_is_supported_voltage(drv->vdd_apc, + fuse->max_uV, + fuse->max_uV); + if (!ret) { + dev_err(drv->dev, + "max uV: %d (fuse corner: %d) not supported by regulator\n", + fuse->max_uV, i); + return -EINVAL; + } + + dev_dbg(drv->dev, + "fuse corner %d: [%d %d %d] RO%hhu quot %d squot %d\n", + i, fuse->min_uV, fuse->uV, fuse->max_uV, + fuse->ring_osc_idx, fuse->quot, fuse->step_quot); + } + + return 0; +} + +static int cpr_calculate_scaling(const char *quot_offset, + struct cpr_drv *drv, + const struct fuse_corner_data *fdata, + const struct corner *corner) +{ + u32 quot_diff = 0; + unsigned long freq_diff; + int scaling; + const struct fuse_corner *fuse, *prev_fuse; + int ret; + + fuse = corner->fuse_corner; + prev_fuse = fuse - 1; + + if (quot_offset) { + ret = nvmem_cell_read_variable_le_u32(drv->dev, quot_offset, "_diff); + if (ret) + return ret; + + quot_diff *= fdata->quot_offset_scale; + quot_diff += fdata->quot_offset_adjust; + } else { + quot_diff = fuse->quot - prev_fuse->quot; + } + + freq_diff = fuse->max_freq - prev_fuse->max_freq; + freq_diff /= 1000000; /* Convert to MHz */ + scaling = 1000 * quot_diff / freq_diff; + return min(scaling, fdata->max_quot_scale); +} + +static int cpr_interpolate(const struct corner *corner, int step_volt, + const struct fuse_corner_data *fdata) +{ + unsigned long f_high, f_low, f_diff; + int uV_high, uV_low, uV; + u64 temp, temp_limit; + const struct fuse_corner *fuse, *prev_fuse; + + fuse = corner->fuse_corner; + prev_fuse = fuse - 1; + + f_high = fuse->max_freq; + f_low = prev_fuse->max_freq; + uV_high = fuse->uV; + uV_low = prev_fuse->uV; + f_diff = fuse->max_freq - corner->freq; + + /* + * Don't interpolate in the wrong direction. This could happen + * if the adjusted fuse voltage overlaps with the previous fuse's + * adjusted voltage. + */ + if (f_high <= f_low || uV_high <= uV_low || f_high <= corner->freq) + return corner->uV; + + temp = f_diff * (uV_high - uV_low); + temp = div64_ul(temp, f_high - f_low); + + /* + * max_volt_scale has units of uV/MHz while freq values + * have units of Hz. Divide by 1000000 to convert to. + */ + temp_limit = f_diff * fdata->max_volt_scale; + do_div(temp_limit, 1000000); + + uV = uV_high - min(temp, temp_limit); + return roundup(uV, step_volt); +} + +static unsigned int cpr_get_fuse_corner(struct dev_pm_opp *opp) +{ + struct device_node *np; + unsigned int fuse_corner = 0; + + np = dev_pm_opp_get_of_node(opp); + if (of_property_read_u32(np, "qcom,opp-fuse-level", &fuse_corner)) + pr_err("%s: missing 'qcom,opp-fuse-level' property\n", + __func__); + + of_node_put(np); + + return fuse_corner; +} + +static unsigned long cpr_get_opp_hz_for_req(struct dev_pm_opp *ref, + struct device *cpu_dev) +{ + u64 rate = 0; + struct device_node *ref_np; + struct device_node *desc_np; + struct device_node *child_np = NULL; + struct device_node *child_req_np = NULL; + + desc_np = dev_pm_opp_of_get_opp_desc_node(cpu_dev); + if (!desc_np) + return 0; + + ref_np = dev_pm_opp_get_of_node(ref); + if (!ref_np) + goto out_ref; + + do { + of_node_put(child_req_np); + child_np = of_get_next_available_child(desc_np, child_np); + child_req_np = of_parse_phandle(child_np, "required-opps", 0); + } while (child_np && child_req_np != ref_np); + + if (child_np && child_req_np == ref_np) + of_property_read_u64(child_np, "opp-hz", &rate); + + of_node_put(child_req_np); + of_node_put(child_np); + of_node_put(ref_np); +out_ref: + of_node_put(desc_np); + + return (unsigned long) rate; +} + +static int cpr_corner_init(struct cpr_drv *drv) +{ + const struct cpr_desc *desc = drv->desc; + const struct cpr_fuse *fuses = drv->cpr_fuses; + int i, level, scaling = 0; + unsigned int fnum, fc; + const char *quot_offset; + struct fuse_corner *fuse, *prev_fuse; + struct corner *corner, *end; + struct corner_data *cdata; + const struct fuse_corner_data *fdata; + bool apply_scaling; + unsigned long freq_diff, freq_diff_mhz; + unsigned long freq; + int step_volt = regulator_get_linear_step(drv->vdd_apc); + struct dev_pm_opp *opp; + + if (!step_volt) + return -EINVAL; + + corner = drv->corners; + end = &corner[drv->num_corners - 1]; + + cdata = devm_kcalloc(drv->dev, drv->num_corners, + sizeof(struct corner_data), + GFP_KERNEL); + if (!cdata) + return -ENOMEM; + + /* + * Store maximum frequency for each fuse corner based on the frequency + * plan + */ + for (level = 1; level <= drv->num_corners; level++) { + opp = dev_pm_opp_find_level_exact(&drv->pd.dev, level); + if (IS_ERR(opp)) + return -EINVAL; + fc = cpr_get_fuse_corner(opp); + if (!fc) { + dev_pm_opp_put(opp); + return -EINVAL; + } + fnum = fc - 1; + freq = cpr_get_opp_hz_for_req(opp, drv->attached_cpu_dev); + if (!freq) { + dev_pm_opp_put(opp); + return -EINVAL; + } + cdata[level - 1].fuse_corner = fnum; + cdata[level - 1].freq = freq; + + fuse = &drv->fuse_corners[fnum]; + dev_dbg(drv->dev, "freq: %lu level: %u fuse level: %u\n", + freq, dev_pm_opp_get_level(opp) - 1, fnum); + if (freq > fuse->max_freq) + fuse->max_freq = freq; + dev_pm_opp_put(opp); + } + + /* + * Get the quotient adjustment scaling factor, according to: + * + * scaling = min(1000 * (QUOT(corner_N) - QUOT(corner_N-1)) + * / (freq(corner_N) - freq(corner_N-1)), max_factor) + * + * QUOT(corner_N): quotient read from fuse for fuse corner N + * QUOT(corner_N-1): quotient read from fuse for fuse corner (N - 1) + * freq(corner_N): max frequency in MHz supported by fuse corner N + * freq(corner_N-1): max frequency in MHz supported by fuse corner + * (N - 1) + * + * Then walk through the corners mapped to each fuse corner + * and calculate the quotient adjustment for each one using the + * following formula: + * + * quot_adjust = (freq_max - freq_corner) * scaling / 1000 + * + * freq_max: max frequency in MHz supported by the fuse corner + * freq_corner: frequency in MHz corresponding to the corner + * scaling: calculated from above equation + * + * + * + + + * | v | + * q | f c o | f c + * u | c l | c + * o | f t | f + * t | c a | c + * | c f g | c f + * | e | + * +--------------- +---------------- + * 0 1 2 3 4 5 6 0 1 2 3 4 5 6 + * corner corner + * + * c = corner + * f = fuse corner + * + */ + for (apply_scaling = false, i = 0; corner <= end; corner++, i++) { + fnum = cdata[i].fuse_corner; + fdata = &desc->cpr_fuses.fuse_corner_data[fnum]; + quot_offset = fuses[fnum].quotient_offset; + fuse = &drv->fuse_corners[fnum]; + if (fnum) + prev_fuse = &drv->fuse_corners[fnum - 1]; + else + prev_fuse = NULL; + + corner->fuse_corner = fuse; + corner->freq = cdata[i].freq; + corner->uV = fuse->uV; + + if (prev_fuse && cdata[i - 1].freq == prev_fuse->max_freq) { + scaling = cpr_calculate_scaling(quot_offset, drv, + fdata, corner); + if (scaling < 0) + return scaling; + + apply_scaling = true; + } else if (corner->freq == fuse->max_freq) { + /* This is a fuse corner; don't scale anything */ + apply_scaling = false; + } + + if (apply_scaling) { + freq_diff = fuse->max_freq - corner->freq; + freq_diff_mhz = freq_diff / 1000000; + corner->quot_adjust = scaling * freq_diff_mhz / 1000; + + corner->uV = cpr_interpolate(corner, step_volt, fdata); + } + + corner->max_uV = fuse->max_uV; + corner->min_uV = fuse->min_uV; + corner->uV = clamp(corner->uV, corner->min_uV, corner->max_uV); + corner->last_uV = corner->uV; + + /* Reduce the ceiling voltage if needed */ + if (desc->reduce_to_corner_uV && corner->uV < corner->max_uV) + corner->max_uV = corner->uV; + else if (desc->reduce_to_fuse_uV && fuse->uV < corner->max_uV) + corner->max_uV = max(corner->min_uV, fuse->uV); + + dev_dbg(drv->dev, "corner %d: [%d %d %d] quot %d\n", i, + corner->min_uV, corner->uV, corner->max_uV, + fuse->quot - corner->quot_adjust); + } + + return 0; +} + +static const struct cpr_fuse *cpr_get_fuses(struct cpr_drv *drv) +{ + const struct cpr_desc *desc = drv->desc; + struct cpr_fuse *fuses; + int i; + + fuses = devm_kcalloc(drv->dev, desc->num_fuse_corners, + sizeof(struct cpr_fuse), + GFP_KERNEL); + if (!fuses) + return ERR_PTR(-ENOMEM); + + for (i = 0; i < desc->num_fuse_corners; i++) { + char tbuf[32]; + + snprintf(tbuf, 32, "cpr_ring_osc%d", i + 1); + fuses[i].ring_osc = devm_kstrdup(drv->dev, tbuf, GFP_KERNEL); + if (!fuses[i].ring_osc) + return ERR_PTR(-ENOMEM); + + snprintf(tbuf, 32, "cpr_init_voltage%d", i + 1); + fuses[i].init_voltage = devm_kstrdup(drv->dev, tbuf, + GFP_KERNEL); + if (!fuses[i].init_voltage) + return ERR_PTR(-ENOMEM); + + snprintf(tbuf, 32, "cpr_quotient%d", i + 1); + fuses[i].quotient = devm_kstrdup(drv->dev, tbuf, GFP_KERNEL); + if (!fuses[i].quotient) + return ERR_PTR(-ENOMEM); + + snprintf(tbuf, 32, "cpr_quotient_offset%d", i + 1); + fuses[i].quotient_offset = devm_kstrdup(drv->dev, tbuf, + GFP_KERNEL); + if (!fuses[i].quotient_offset) + return ERR_PTR(-ENOMEM); + } + + return fuses; +} + +static void cpr_set_loop_allowed(struct cpr_drv *drv) +{ + drv->loop_disabled = false; +} + +static int cpr_init_parameters(struct cpr_drv *drv) +{ + const struct cpr_desc *desc = drv->desc; + struct clk *clk; + + clk = clk_get(drv->dev, "ref"); + if (IS_ERR(clk)) + return PTR_ERR(clk); + + drv->ref_clk_khz = clk_get_rate(clk) / 1000; + clk_put(clk); + + if (desc->timer_cons_up > RBIF_TIMER_ADJ_CONS_UP_MASK || + desc->timer_cons_down > RBIF_TIMER_ADJ_CONS_DOWN_MASK || + desc->up_threshold > RBCPR_CTL_UP_THRESHOLD_MASK || + desc->down_threshold > RBCPR_CTL_DN_THRESHOLD_MASK || + desc->idle_clocks > RBCPR_STEP_QUOT_IDLE_CLK_MASK || + desc->clamp_timer_interval > RBIF_TIMER_ADJ_CLAMP_INT_MASK) + return -EINVAL; + + dev_dbg(drv->dev, "up threshold = %u, down threshold = %u\n", + desc->up_threshold, desc->down_threshold); + + return 0; +} + +static int cpr_find_initial_corner(struct cpr_drv *drv) +{ + unsigned long rate; + const struct corner *end; + struct corner *iter; + unsigned int i = 0; + + if (!drv->cpu_clk) { + dev_err(drv->dev, "cannot get rate from NULL clk\n"); + return -EINVAL; + } + + end = &drv->corners[drv->num_corners - 1]; + rate = clk_get_rate(drv->cpu_clk); + + /* + * Some bootloaders set a CPU clock frequency that is not defined + * in the OPP table. When running at an unlisted frequency, + * cpufreq_online() will change to the OPP which has the lowest + * frequency, at or above the unlisted frequency. + * Since cpufreq_online() always "rounds up" in the case of an + * unlisted frequency, this function always "rounds down" in case + * of an unlisted frequency. That way, when cpufreq_online() + * triggers the first ever call to cpr_set_performance_state(), + * it will correctly determine the direction as UP. + */ + for (iter = drv->corners; iter <= end; iter++) { + if (iter->freq > rate) + break; + i++; + if (iter->freq == rate) { + drv->corner = iter; + break; + } + if (iter->freq < rate) + drv->corner = iter; + } + + if (!drv->corner) { + dev_err(drv->dev, "boot up corner not found\n"); + return -EINVAL; + } + + dev_dbg(drv->dev, "boot up perf state: %u\n", i); + + return 0; +} + +static const struct cpr_desc qcs404_cpr_desc = { + .num_fuse_corners = 3, + .min_diff_quot = CPR_FUSE_MIN_QUOT_DIFF, + .step_quot = (int []){ 25, 25, 25, }, + .timer_delay_us = 5000, + .timer_cons_up = 0, + .timer_cons_down = 2, + .up_threshold = 1, + .down_threshold = 3, + .idle_clocks = 15, + .gcnt_us = 1, + .vdd_apc_step_up_limit = 1, + .vdd_apc_step_down_limit = 1, + .cpr_fuses = { + .init_voltage_step = 8000, + .init_voltage_width = 6, + .fuse_corner_data = (struct fuse_corner_data[]){ + /* fuse corner 0 */ + { + .ref_uV = 1224000, + .max_uV = 1224000, + .min_uV = 1048000, + .max_volt_scale = 0, + .max_quot_scale = 0, + .quot_offset = 0, + .quot_scale = 1, + .quot_adjust = 0, + .quot_offset_scale = 5, + .quot_offset_adjust = 0, + }, + /* fuse corner 1 */ + { + .ref_uV = 1288000, + .max_uV = 1288000, + .min_uV = 1048000, + .max_volt_scale = 2000, + .max_quot_scale = 1400, + .quot_offset = 0, + .quot_scale = 1, + .quot_adjust = -20, + .quot_offset_scale = 5, + .quot_offset_adjust = 0, + }, + /* fuse corner 2 */ + { + .ref_uV = 1352000, + .max_uV = 1384000, + .min_uV = 1088000, + .max_volt_scale = 2000, + .max_quot_scale = 1400, + .quot_offset = 0, + .quot_scale = 1, + .quot_adjust = 0, + .quot_offset_scale = 5, + .quot_offset_adjust = 0, + }, + }, + }, +}; + +static const struct acc_desc qcs404_acc_desc = { + .settings = (struct reg_sequence[]){ + { 0xb120, 0x1041040 }, + { 0xb124, 0x41 }, + { 0xb120, 0x0 }, + { 0xb124, 0x0 }, + { 0xb120, 0x0 }, + { 0xb124, 0x0 }, + }, + .config = (struct reg_sequence[]){ + { 0xb138, 0xff }, + { 0xb130, 0x5555 }, + }, + .num_regs_per_fuse = 2, +}; + +static const struct cpr_acc_desc qcs404_cpr_acc_desc = { + .cpr_desc = &qcs404_cpr_desc, + .acc_desc = &qcs404_acc_desc, +}; + +static unsigned int cpr_get_performance_state(struct generic_pm_domain *genpd, + struct dev_pm_opp *opp) +{ + return dev_pm_opp_get_level(opp); +} + +static int cpr_power_off(struct generic_pm_domain *domain) +{ + struct cpr_drv *drv = container_of(domain, struct cpr_drv, pd); + + return cpr_disable(drv); +} + +static int cpr_power_on(struct generic_pm_domain *domain) +{ + struct cpr_drv *drv = container_of(domain, struct cpr_drv, pd); + + return cpr_enable(drv); +} + +static int cpr_pd_attach_dev(struct generic_pm_domain *domain, + struct device *dev) +{ + struct cpr_drv *drv = container_of(domain, struct cpr_drv, pd); + const struct acc_desc *acc_desc = drv->acc_desc; + int ret = 0; + + mutex_lock(&drv->lock); + + dev_dbg(drv->dev, "attach callback for: %s\n", dev_name(dev)); + + /* + * This driver only supports scaling voltage for a CPU cluster + * where all CPUs in the cluster share a single regulator. + * Therefore, save the struct device pointer only for the first + * CPU device that gets attached. There is no need to do any + * additional initialization when further CPUs get attached. + */ + if (drv->attached_cpu_dev) + goto unlock; + + /* + * cpr_scale_voltage() requires the direction (if we are changing + * to a higher or lower OPP). The first time + * cpr_set_performance_state() is called, there is no previous + * performance state defined. Therefore, we call + * cpr_find_initial_corner() that gets the CPU clock frequency + * set by the bootloader, so that we can determine the direction + * the first time cpr_set_performance_state() is called. + */ + drv->cpu_clk = devm_clk_get(dev, NULL); + if (IS_ERR(drv->cpu_clk)) { + ret = PTR_ERR(drv->cpu_clk); + if (ret != -EPROBE_DEFER) + dev_err(drv->dev, "could not get cpu clk: %d\n", ret); + goto unlock; + } + drv->attached_cpu_dev = dev; + + dev_dbg(drv->dev, "using cpu clk from: %s\n", + dev_name(drv->attached_cpu_dev)); + + /* + * Everything related to (virtual) corners has to be initialized + * here, when attaching to the power domain, since we need to know + * the maximum frequency for each fuse corner, and this is only + * available after the cpufreq driver has attached to us. + * The reason for this is that we need to know the highest + * frequency associated with each fuse corner. + */ + ret = dev_pm_opp_get_opp_count(&drv->pd.dev); + if (ret < 0) { + dev_err(drv->dev, "could not get OPP count\n"); + goto unlock; + } + drv->num_corners = ret; + + if (drv->num_corners < 2) { + dev_err(drv->dev, "need at least 2 OPPs to use CPR\n"); + ret = -EINVAL; + goto unlock; + } + + drv->corners = devm_kcalloc(drv->dev, drv->num_corners, + sizeof(*drv->corners), + GFP_KERNEL); + if (!drv->corners) { + ret = -ENOMEM; + goto unlock; + } + + ret = cpr_corner_init(drv); + if (ret) + goto unlock; + + cpr_set_loop_allowed(drv); + + ret = cpr_init_parameters(drv); + if (ret) + goto unlock; + + /* Configure CPR HW but keep it disabled */ + ret = cpr_config(drv); + if (ret) + goto unlock; + + ret = cpr_find_initial_corner(drv); + if (ret) + goto unlock; + + if (acc_desc->config) + regmap_multi_reg_write(drv->tcsr, acc_desc->config, + acc_desc->num_regs_per_fuse); + + /* Enable ACC if required */ + if (acc_desc->enable_mask) + regmap_update_bits(drv->tcsr, acc_desc->enable_reg, + acc_desc->enable_mask, + acc_desc->enable_mask); + + dev_info(drv->dev, "driver initialized with %u OPPs\n", + drv->num_corners); + +unlock: + mutex_unlock(&drv->lock); + + return ret; +} + +static int cpr_debug_info_show(struct seq_file *s, void *unused) +{ + u32 gcnt, ro_sel, ctl, irq_status, reg, error_steps; + u32 step_dn, step_up, error, error_lt0, busy; + struct cpr_drv *drv = s->private; + struct fuse_corner *fuse_corner; + struct corner *corner; + + corner = drv->corner; + fuse_corner = corner->fuse_corner; + + seq_printf(s, "corner, current_volt = %d uV\n", + corner->last_uV); + + ro_sel = fuse_corner->ring_osc_idx; + gcnt = cpr_read(drv, REG_RBCPR_GCNT_TARGET(ro_sel)); + seq_printf(s, "rbcpr_gcnt_target (%u) = %#02X\n", ro_sel, gcnt); + + ctl = cpr_read(drv, REG_RBCPR_CTL); + seq_printf(s, "rbcpr_ctl = %#02X\n", ctl); + + irq_status = cpr_read(drv, REG_RBIF_IRQ_STATUS); + seq_printf(s, "rbcpr_irq_status = %#02X\n", irq_status); + + reg = cpr_read(drv, REG_RBCPR_RESULT_0); + seq_printf(s, "rbcpr_result_0 = %#02X\n", reg); + + step_dn = reg & 0x01; + step_up = (reg >> RBCPR_RESULT0_STEP_UP_SHIFT) & 0x01; + seq_printf(s, " [step_dn = %u", step_dn); + + seq_printf(s, ", step_up = %u", step_up); + + error_steps = (reg >> RBCPR_RESULT0_ERROR_STEPS_SHIFT) + & RBCPR_RESULT0_ERROR_STEPS_MASK; + seq_printf(s, ", error_steps = %u", error_steps); + + error = (reg >> RBCPR_RESULT0_ERROR_SHIFT) & RBCPR_RESULT0_ERROR_MASK; + seq_printf(s, ", error = %u", error); + + error_lt0 = (reg >> RBCPR_RESULT0_ERROR_LT0_SHIFT) & 0x01; + seq_printf(s, ", error_lt_0 = %u", error_lt0); + + busy = (reg >> RBCPR_RESULT0_BUSY_SHIFT) & 0x01; + seq_printf(s, ", busy = %u]\n", busy); + + return 0; +} +DEFINE_SHOW_ATTRIBUTE(cpr_debug_info); + +static void cpr_debugfs_init(struct cpr_drv *drv) +{ + drv->debugfs = debugfs_create_dir("qcom_cpr", NULL); + + debugfs_create_file("debug_info", 0444, drv->debugfs, + drv, &cpr_debug_info_fops); +} + +static int cpr_probe(struct platform_device *pdev) +{ + struct device *dev = &pdev->dev; + struct cpr_drv *drv; + int irq, ret; + const struct cpr_acc_desc *data; + struct device_node *np; + u32 cpr_rev = FUSE_REVISION_UNKNOWN; + + data = of_device_get_match_data(dev); + if (!data || !data->cpr_desc || !data->acc_desc) + return -EINVAL; + + drv = devm_kzalloc(dev, sizeof(*drv), GFP_KERNEL); + if (!drv) + return -ENOMEM; + drv->dev = dev; + drv->desc = data->cpr_desc; + drv->acc_desc = data->acc_desc; + + drv->fuse_corners = devm_kcalloc(dev, drv->desc->num_fuse_corners, + sizeof(*drv->fuse_corners), + GFP_KERNEL); + if (!drv->fuse_corners) + return -ENOMEM; + + np = of_parse_phandle(dev->of_node, "acc-syscon", 0); + if (!np) + return -ENODEV; + + drv->tcsr = syscon_node_to_regmap(np); + of_node_put(np); + if (IS_ERR(drv->tcsr)) + return PTR_ERR(drv->tcsr); + + drv->base = devm_platform_ioremap_resource(pdev, 0); + if (IS_ERR(drv->base)) + return PTR_ERR(drv->base); + + irq = platform_get_irq(pdev, 0); + if (irq < 0) + return -EINVAL; + + drv->vdd_apc = devm_regulator_get(dev, "vdd-apc"); + if (IS_ERR(drv->vdd_apc)) + return PTR_ERR(drv->vdd_apc); + + /* + * Initialize fuse corners, since it simply depends + * on data in efuses. + * Everything related to (virtual) corners has to be + * initialized after attaching to the power domain, + * since it depends on the CPU's OPP table. + */ + ret = nvmem_cell_read_variable_le_u32(dev, "cpr_fuse_revision", &cpr_rev); + if (ret) + return ret; + + drv->cpr_fuses = cpr_get_fuses(drv); + if (IS_ERR(drv->cpr_fuses)) + return PTR_ERR(drv->cpr_fuses); + + ret = cpr_populate_ring_osc_idx(drv); + if (ret) + return ret; + + ret = cpr_fuse_corner_init(drv); + if (ret) + return ret; + + mutex_init(&drv->lock); + + ret = devm_request_threaded_irq(dev, irq, NULL, + cpr_irq_handler, + IRQF_ONESHOT | IRQF_TRIGGER_RISING, + "cpr", drv); + if (ret) + return ret; + + drv->pd.name = devm_kstrdup_const(dev, dev->of_node->full_name, + GFP_KERNEL); + if (!drv->pd.name) + return -EINVAL; + + drv->pd.power_off = cpr_power_off; + drv->pd.power_on = cpr_power_on; + drv->pd.set_performance_state = cpr_set_performance_state; + drv->pd.opp_to_performance_state = cpr_get_performance_state; + drv->pd.attach_dev = cpr_pd_attach_dev; + + ret = pm_genpd_init(&drv->pd, NULL, true); + if (ret) + return ret; + + ret = of_genpd_add_provider_simple(dev->of_node, &drv->pd); + if (ret) + goto err_remove_genpd; + + platform_set_drvdata(pdev, drv); + cpr_debugfs_init(drv); + + return 0; + +err_remove_genpd: + pm_genpd_remove(&drv->pd); + return ret; +} + +static int cpr_remove(struct platform_device *pdev) +{ + struct cpr_drv *drv = platform_get_drvdata(pdev); + + if (cpr_is_allowed(drv)) { + cpr_ctl_disable(drv); + cpr_irq_set(drv, 0); + } + + of_genpd_del_provider(pdev->dev.of_node); + pm_genpd_remove(&drv->pd); + + debugfs_remove_recursive(drv->debugfs); + + return 0; +} + +static const struct of_device_id cpr_match_table[] = { + { .compatible = "qcom,qcs404-cpr", .data = &qcs404_cpr_acc_desc }, + { } +}; +MODULE_DEVICE_TABLE(of, cpr_match_table); + +static struct platform_driver cpr_driver = { + .probe = cpr_probe, + .remove = cpr_remove, + .driver = { + .name = "qcom-cpr", + .of_match_table = cpr_match_table, + }, +}; +module_platform_driver(cpr_driver); + +MODULE_DESCRIPTION("Core Power Reduction (CPR) driver"); +MODULE_LICENSE("GPL v2"); diff --git a/drivers/pmdomain/qcom/rpmhpd.c b/drivers/pmdomain/qcom/rpmhpd.c new file mode 100644 index 000000000000..a87e336d5e33 --- /dev/null +++ b/drivers/pmdomain/qcom/rpmhpd.c @@ -0,0 +1,886 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2018, The Linux Foundation. All rights reserved.*/ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#define domain_to_rpmhpd(domain) container_of(domain, struct rpmhpd, pd) + +#define RPMH_ARC_MAX_LEVELS 16 + +/** + * struct rpmhpd - top level RPMh power domain resource data structure + * @dev: rpmh power domain controller device + * @pd: generic_pm_domain corresponding to the power domain + * @parent: generic_pm_domain corresponding to the parent's power domain + * @peer: A peer power domain in case Active only Voting is + * supported + * @active_only: True if it represents an Active only peer + * @corner: current corner + * @active_corner: current active corner + * @enable_corner: lowest non-zero corner + * @level: An array of level (vlvl) to corner (hlvl) mappings + * derived from cmd-db + * @level_count: Number of levels supported by the power domain. max + * being 16 (0 - 15) + * @enabled: true if the power domain is enabled + * @res_name: Resource name used for cmd-db lookup + * @addr: Resource address as looped up using resource name from + * cmd-db + * @state_synced: Indicator that sync_state has been invoked for the rpmhpd resource + */ +struct rpmhpd { + struct device *dev; + struct generic_pm_domain pd; + struct generic_pm_domain *parent; + struct rpmhpd *peer; + const bool active_only; + unsigned int corner; + unsigned int active_corner; + unsigned int enable_corner; + u32 level[RPMH_ARC_MAX_LEVELS]; + size_t level_count; + bool enabled; + const char *res_name; + u32 addr; + bool state_synced; +}; + +struct rpmhpd_desc { + struct rpmhpd **rpmhpds; + size_t num_pds; +}; + +static DEFINE_MUTEX(rpmhpd_lock); + +/* RPMH powerdomains */ + +static struct rpmhpd cx_ao; +static struct rpmhpd mx; +static struct rpmhpd mx_ao; +static struct rpmhpd cx = { + .pd = { .name = "cx", }, + .peer = &cx_ao, + .res_name = "cx.lvl", +}; + +static struct rpmhpd cx_ao = { + .pd = { .name = "cx_ao", }, + .active_only = true, + .peer = &cx, + .res_name = "cx.lvl", +}; + +static struct rpmhpd cx_ao_w_mx_parent; +static struct rpmhpd cx_w_mx_parent = { + .pd = { .name = "cx", }, + .peer = &cx_ao_w_mx_parent, + .parent = &mx.pd, + .res_name = "cx.lvl", +}; + +static struct rpmhpd cx_ao_w_mx_parent = { + .pd = { .name = "cx_ao", }, + .active_only = true, + .peer = &cx_w_mx_parent, + .parent = &mx_ao.pd, + .res_name = "cx.lvl", +}; + +static struct rpmhpd ebi = { + .pd = { .name = "ebi", }, + .res_name = "ebi.lvl", +}; + +static struct rpmhpd gfx = { + .pd = { .name = "gfx", }, + .res_name = "gfx.lvl", +}; + +static struct rpmhpd lcx = { + .pd = { .name = "lcx", }, + .res_name = "lcx.lvl", +}; + +static struct rpmhpd lmx = { + .pd = { .name = "lmx", }, + .res_name = "lmx.lvl", +}; + +static struct rpmhpd mmcx_ao; +static struct rpmhpd mmcx = { + .pd = { .name = "mmcx", }, + .peer = &mmcx_ao, + .res_name = "mmcx.lvl", +}; + +static struct rpmhpd mmcx_ao = { + .pd = { .name = "mmcx_ao", }, + .active_only = true, + .peer = &mmcx, + .res_name = "mmcx.lvl", +}; + +static struct rpmhpd mmcx_ao_w_cx_parent; +static struct rpmhpd mmcx_w_cx_parent = { + .pd = { .name = "mmcx", }, + .peer = &mmcx_ao_w_cx_parent, + .parent = &cx.pd, + .res_name = "mmcx.lvl", +}; + +static struct rpmhpd mmcx_ao_w_cx_parent = { + .pd = { .name = "mmcx_ao", }, + .active_only = true, + .peer = &mmcx_w_cx_parent, + .parent = &cx_ao.pd, + .res_name = "mmcx.lvl", +}; + +static struct rpmhpd mss = { + .pd = { .name = "mss", }, + .res_name = "mss.lvl", +}; + +static struct rpmhpd mx_ao; +static struct rpmhpd mx = { + .pd = { .name = "mx", }, + .peer = &mx_ao, + .res_name = "mx.lvl", +}; + +static struct rpmhpd mx_ao = { + .pd = { .name = "mx_ao", }, + .active_only = true, + .peer = &mx, + .res_name = "mx.lvl", +}; + +static struct rpmhpd mxc_ao; +static struct rpmhpd mxc = { + .pd = { .name = "mxc", }, + .peer = &mxc_ao, + .res_name = "mxc.lvl", +}; + +static struct rpmhpd mxc_ao = { + .pd = { .name = "mxc_ao", }, + .active_only = true, + .peer = &mxc, + .res_name = "mxc.lvl", +}; + +static struct rpmhpd nsp = { + .pd = { .name = "nsp", }, + .res_name = "nsp.lvl", +}; + +static struct rpmhpd nsp0 = { + .pd = { .name = "nsp0", }, + .res_name = "nsp0.lvl", +}; + +static struct rpmhpd nsp1 = { + .pd = { .name = "nsp1", }, + .res_name = "nsp1.lvl", +}; + +static struct rpmhpd qphy = { + .pd = { .name = "qphy", }, + .res_name = "qphy.lvl", +}; + +/* SA8540P RPMH powerdomains */ +static struct rpmhpd *sa8540p_rpmhpds[] = { + [SC8280XP_CX] = &cx, + [SC8280XP_CX_AO] = &cx_ao, + [SC8280XP_EBI] = &ebi, + [SC8280XP_GFX] = &gfx, + [SC8280XP_LCX] = &lcx, + [SC8280XP_LMX] = &lmx, + [SC8280XP_MMCX] = &mmcx, + [SC8280XP_MMCX_AO] = &mmcx_ao, + [SC8280XP_MX] = &mx, + [SC8280XP_MX_AO] = &mx_ao, + [SC8280XP_NSP] = &nsp, +}; + +static const struct rpmhpd_desc sa8540p_desc = { + .rpmhpds = sa8540p_rpmhpds, + .num_pds = ARRAY_SIZE(sa8540p_rpmhpds), +}; + +/* SA8775P RPMH power domains */ +static struct rpmhpd *sa8775p_rpmhpds[] = { + [SA8775P_CX] = &cx, + [SA8775P_CX_AO] = &cx_ao, + [SA8775P_EBI] = &ebi, + [SA8775P_GFX] = &gfx, + [SA8775P_LCX] = &lcx, + [SA8775P_LMX] = &lmx, + [SA8775P_MMCX] = &mmcx, + [SA8775P_MMCX_AO] = &mmcx_ao, + [SA8775P_MXC] = &mxc, + [SA8775P_MXC_AO] = &mxc_ao, + [SA8775P_MX] = &mx, + [SA8775P_MX_AO] = &mx_ao, + [SA8775P_NSP0] = &nsp0, + [SA8775P_NSP1] = &nsp1, +}; + +static const struct rpmhpd_desc sa8775p_desc = { + .rpmhpds = sa8775p_rpmhpds, + .num_pds = ARRAY_SIZE(sa8775p_rpmhpds), +}; + +/* SDM670 RPMH powerdomains */ +static struct rpmhpd *sdm670_rpmhpds[] = { + [SDM670_CX] = &cx_w_mx_parent, + [SDM670_CX_AO] = &cx_ao_w_mx_parent, + [SDM670_GFX] = &gfx, + [SDM670_LCX] = &lcx, + [SDM670_LMX] = &lmx, + [SDM670_MSS] = &mss, + [SDM670_MX] = &mx, + [SDM670_MX_AO] = &mx_ao, +}; + +static const struct rpmhpd_desc sdm670_desc = { + .rpmhpds = sdm670_rpmhpds, + .num_pds = ARRAY_SIZE(sdm670_rpmhpds), +}; + +/* SDM845 RPMH powerdomains */ +static struct rpmhpd *sdm845_rpmhpds[] = { + [SDM845_CX] = &cx_w_mx_parent, + [SDM845_CX_AO] = &cx_ao_w_mx_parent, + [SDM845_EBI] = &ebi, + [SDM845_GFX] = &gfx, + [SDM845_LCX] = &lcx, + [SDM845_LMX] = &lmx, + [SDM845_MSS] = &mss, + [SDM845_MX] = &mx, + [SDM845_MX_AO] = &mx_ao, +}; + +static const struct rpmhpd_desc sdm845_desc = { + .rpmhpds = sdm845_rpmhpds, + .num_pds = ARRAY_SIZE(sdm845_rpmhpds), +}; + +/* SDX55 RPMH powerdomains */ +static struct rpmhpd *sdx55_rpmhpds[] = { + [SDX55_CX] = &cx_w_mx_parent, + [SDX55_MSS] = &mss, + [SDX55_MX] = &mx, +}; + +static const struct rpmhpd_desc sdx55_desc = { + .rpmhpds = sdx55_rpmhpds, + .num_pds = ARRAY_SIZE(sdx55_rpmhpds), +}; + +/* SDX65 RPMH powerdomains */ +static struct rpmhpd *sdx65_rpmhpds[] = { + [SDX65_CX] = &cx_w_mx_parent, + [SDX65_CX_AO] = &cx_ao_w_mx_parent, + [SDX65_MSS] = &mss, + [SDX65_MX] = &mx, + [SDX65_MX_AO] = &mx_ao, + [SDX65_MXC] = &mxc, +}; + +static const struct rpmhpd_desc sdx65_desc = { + .rpmhpds = sdx65_rpmhpds, + .num_pds = ARRAY_SIZE(sdx65_rpmhpds), +}; + +/* SDX75 RPMH powerdomains */ +static struct rpmhpd *sdx75_rpmhpds[] = { + [RPMHPD_CX] = &cx, + [RPMHPD_CX_AO] = &cx_ao, + [RPMHPD_MSS] = &mss, + [RPMHPD_MX] = &mx, + [RPMHPD_MX_AO] = &mx_ao, + [RPMHPD_MXC] = &mxc, +}; + +static const struct rpmhpd_desc sdx75_desc = { + .rpmhpds = sdx75_rpmhpds, + .num_pds = ARRAY_SIZE(sdx75_rpmhpds), +}; + +/* SM6350 RPMH powerdomains */ +static struct rpmhpd *sm6350_rpmhpds[] = { + [SM6350_CX] = &cx_w_mx_parent, + [SM6350_GFX] = &gfx, + [SM6350_LCX] = &lcx, + [SM6350_LMX] = &lmx, + [SM6350_MSS] = &mss, + [SM6350_MX] = &mx, +}; + +static const struct rpmhpd_desc sm6350_desc = { + .rpmhpds = sm6350_rpmhpds, + .num_pds = ARRAY_SIZE(sm6350_rpmhpds), +}; + +/* SM8150 RPMH powerdomains */ +static struct rpmhpd *sm8150_rpmhpds[] = { + [SM8150_CX] = &cx_w_mx_parent, + [SM8150_CX_AO] = &cx_ao_w_mx_parent, + [SM8150_EBI] = &ebi, + [SM8150_GFX] = &gfx, + [SM8150_LCX] = &lcx, + [SM8150_LMX] = &lmx, + [SM8150_MMCX] = &mmcx, + [SM8150_MMCX_AO] = &mmcx_ao, + [SM8150_MSS] = &mss, + [SM8150_MX] = &mx, + [SM8150_MX_AO] = &mx_ao, +}; + +static const struct rpmhpd_desc sm8150_desc = { + .rpmhpds = sm8150_rpmhpds, + .num_pds = ARRAY_SIZE(sm8150_rpmhpds), +}; + +static struct rpmhpd *sa8155p_rpmhpds[] = { + [SA8155P_CX] = &cx_w_mx_parent, + [SA8155P_CX_AO] = &cx_ao_w_mx_parent, + [SA8155P_EBI] = &ebi, + [SA8155P_GFX] = &gfx, + [SA8155P_MSS] = &mss, + [SA8155P_MX] = &mx, + [SA8155P_MX_AO] = &mx_ao, +}; + +static const struct rpmhpd_desc sa8155p_desc = { + .rpmhpds = sa8155p_rpmhpds, + .num_pds = ARRAY_SIZE(sa8155p_rpmhpds), +}; + +/* SM8250 RPMH powerdomains */ +static struct rpmhpd *sm8250_rpmhpds[] = { + [RPMHPD_CX] = &cx_w_mx_parent, + [RPMHPD_CX_AO] = &cx_ao_w_mx_parent, + [RPMHPD_EBI] = &ebi, + [RPMHPD_GFX] = &gfx, + [RPMHPD_LCX] = &lcx, + [RPMHPD_LMX] = &lmx, + [RPMHPD_MMCX] = &mmcx, + [RPMHPD_MMCX_AO] = &mmcx_ao, + [RPMHPD_MX] = &mx, + [RPMHPD_MX_AO] = &mx_ao, +}; + +static const struct rpmhpd_desc sm8250_desc = { + .rpmhpds = sm8250_rpmhpds, + .num_pds = ARRAY_SIZE(sm8250_rpmhpds), +}; + +/* SM8350 Power domains */ +static struct rpmhpd *sm8350_rpmhpds[] = { + [RPMHPD_CX] = &cx_w_mx_parent, + [RPMHPD_CX_AO] = &cx_ao_w_mx_parent, + [RPMHPD_EBI] = &ebi, + [RPMHPD_GFX] = &gfx, + [RPMHPD_LCX] = &lcx, + [RPMHPD_LMX] = &lmx, + [RPMHPD_MMCX] = &mmcx, + [RPMHPD_MMCX_AO] = &mmcx_ao, + [RPMHPD_MSS] = &mss, + [RPMHPD_MX] = &mx, + [RPMHPD_MX_AO] = &mx_ao, + [RPMHPD_MXC] = &mxc, + [RPMHPD_MXC_AO] = &mxc_ao, +}; + +static const struct rpmhpd_desc sm8350_desc = { + .rpmhpds = sm8350_rpmhpds, + .num_pds = ARRAY_SIZE(sm8350_rpmhpds), +}; + +/* SM8450 RPMH powerdomains */ +static struct rpmhpd *sm8450_rpmhpds[] = { + [RPMHPD_CX] = &cx, + [RPMHPD_CX_AO] = &cx_ao, + [RPMHPD_EBI] = &ebi, + [RPMHPD_GFX] = &gfx, + [RPMHPD_LCX] = &lcx, + [RPMHPD_LMX] = &lmx, + [RPMHPD_MMCX] = &mmcx_w_cx_parent, + [RPMHPD_MMCX_AO] = &mmcx_ao_w_cx_parent, + [RPMHPD_MSS] = &mss, + [RPMHPD_MX] = &mx, + [RPMHPD_MX_AO] = &mx_ao, + [RPMHPD_MXC] = &mxc, + [RPMHPD_MXC_AO] = &mxc_ao, +}; + +static const struct rpmhpd_desc sm8450_desc = { + .rpmhpds = sm8450_rpmhpds, + .num_pds = ARRAY_SIZE(sm8450_rpmhpds), +}; + +/* SM8550 RPMH powerdomains */ +static struct rpmhpd *sm8550_rpmhpds[] = { + [RPMHPD_CX] = &cx, + [RPMHPD_CX_AO] = &cx_ao, + [RPMHPD_EBI] = &ebi, + [RPMHPD_GFX] = &gfx, + [RPMHPD_LCX] = &lcx, + [RPMHPD_LMX] = &lmx, + [RPMHPD_MMCX] = &mmcx_w_cx_parent, + [RPMHPD_MMCX_AO] = &mmcx_ao_w_cx_parent, + [RPMHPD_MSS] = &mss, + [RPMHPD_MX] = &mx, + [RPMHPD_MX_AO] = &mx_ao, + [RPMHPD_MXC] = &mxc, + [RPMHPD_MXC_AO] = &mxc_ao, + [RPMHPD_NSP] = &nsp, +}; + +static const struct rpmhpd_desc sm8550_desc = { + .rpmhpds = sm8550_rpmhpds, + .num_pds = ARRAY_SIZE(sm8550_rpmhpds), +}; + +/* QDU1000/QRU1000 RPMH powerdomains */ +static struct rpmhpd *qdu1000_rpmhpds[] = { + [QDU1000_CX] = &cx, + [QDU1000_EBI] = &ebi, + [QDU1000_MSS] = &mss, + [QDU1000_MX] = &mx, +}; + +static const struct rpmhpd_desc qdu1000_desc = { + .rpmhpds = qdu1000_rpmhpds, + .num_pds = ARRAY_SIZE(qdu1000_rpmhpds), +}; + +/* SC7180 RPMH powerdomains */ +static struct rpmhpd *sc7180_rpmhpds[] = { + [SC7180_CX] = &cx_w_mx_parent, + [SC7180_CX_AO] = &cx_ao_w_mx_parent, + [SC7180_GFX] = &gfx, + [SC7180_LCX] = &lcx, + [SC7180_LMX] = &lmx, + [SC7180_MSS] = &mss, + [SC7180_MX] = &mx, + [SC7180_MX_AO] = &mx_ao, +}; + +static const struct rpmhpd_desc sc7180_desc = { + .rpmhpds = sc7180_rpmhpds, + .num_pds = ARRAY_SIZE(sc7180_rpmhpds), +}; + +/* SC7280 RPMH powerdomains */ +static struct rpmhpd *sc7280_rpmhpds[] = { + [SC7280_CX] = &cx, + [SC7280_CX_AO] = &cx_ao, + [SC7280_EBI] = &ebi, + [SC7280_GFX] = &gfx, + [SC7280_LCX] = &lcx, + [SC7280_LMX] = &lmx, + [SC7280_MSS] = &mss, + [SC7280_MX] = &mx, + [SC7280_MX_AO] = &mx_ao, +}; + +static const struct rpmhpd_desc sc7280_desc = { + .rpmhpds = sc7280_rpmhpds, + .num_pds = ARRAY_SIZE(sc7280_rpmhpds), +}; + +/* SC8180x RPMH powerdomains */ +static struct rpmhpd *sc8180x_rpmhpds[] = { + [SC8180X_CX] = &cx_w_mx_parent, + [SC8180X_CX_AO] = &cx_ao_w_mx_parent, + [SC8180X_EBI] = &ebi, + [SC8180X_GFX] = &gfx, + [SC8180X_LCX] = &lcx, + [SC8180X_LMX] = &lmx, + [SC8180X_MMCX] = &mmcx, + [SC8180X_MMCX_AO] = &mmcx_ao, + [SC8180X_MSS] = &mss, + [SC8180X_MX] = &mx, + [SC8180X_MX_AO] = &mx_ao, +}; + +static const struct rpmhpd_desc sc8180x_desc = { + .rpmhpds = sc8180x_rpmhpds, + .num_pds = ARRAY_SIZE(sc8180x_rpmhpds), +}; + +/* SC8280xp RPMH powerdomains */ +static struct rpmhpd *sc8280xp_rpmhpds[] = { + [SC8280XP_CX] = &cx, + [SC8280XP_CX_AO] = &cx_ao, + [SC8280XP_EBI] = &ebi, + [SC8280XP_GFX] = &gfx, + [SC8280XP_LCX] = &lcx, + [SC8280XP_LMX] = &lmx, + [SC8280XP_MMCX] = &mmcx, + [SC8280XP_MMCX_AO] = &mmcx_ao, + [SC8280XP_MX] = &mx, + [SC8280XP_MX_AO] = &mx_ao, + [SC8280XP_NSP] = &nsp, + [SC8280XP_QPHY] = &qphy, +}; + +static const struct rpmhpd_desc sc8280xp_desc = { + .rpmhpds = sc8280xp_rpmhpds, + .num_pds = ARRAY_SIZE(sc8280xp_rpmhpds), +}; + +static const struct of_device_id rpmhpd_match_table[] = { + { .compatible = "qcom,qdu1000-rpmhpd", .data = &qdu1000_desc }, + { .compatible = "qcom,sa8155p-rpmhpd", .data = &sa8155p_desc }, + { .compatible = "qcom,sa8540p-rpmhpd", .data = &sa8540p_desc }, + { .compatible = "qcom,sa8775p-rpmhpd", .data = &sa8775p_desc }, + { .compatible = "qcom,sc7180-rpmhpd", .data = &sc7180_desc }, + { .compatible = "qcom,sc7280-rpmhpd", .data = &sc7280_desc }, + { .compatible = "qcom,sc8180x-rpmhpd", .data = &sc8180x_desc }, + { .compatible = "qcom,sc8280xp-rpmhpd", .data = &sc8280xp_desc }, + { .compatible = "qcom,sdm670-rpmhpd", .data = &sdm670_desc }, + { .compatible = "qcom,sdm845-rpmhpd", .data = &sdm845_desc }, + { .compatible = "qcom,sdx55-rpmhpd", .data = &sdx55_desc}, + { .compatible = "qcom,sdx65-rpmhpd", .data = &sdx65_desc}, + { .compatible = "qcom,sdx75-rpmhpd", .data = &sdx75_desc}, + { .compatible = "qcom,sm6350-rpmhpd", .data = &sm6350_desc }, + { .compatible = "qcom,sm8150-rpmhpd", .data = &sm8150_desc }, + { .compatible = "qcom,sm8250-rpmhpd", .data = &sm8250_desc }, + { .compatible = "qcom,sm8350-rpmhpd", .data = &sm8350_desc }, + { .compatible = "qcom,sm8450-rpmhpd", .data = &sm8450_desc }, + { .compatible = "qcom,sm8550-rpmhpd", .data = &sm8550_desc }, + { } +}; +MODULE_DEVICE_TABLE(of, rpmhpd_match_table); + +static int rpmhpd_send_corner(struct rpmhpd *pd, int state, + unsigned int corner, bool sync) +{ + struct tcs_cmd cmd = { + .addr = pd->addr, + .data = corner, + }; + + /* + * Wait for an ack only when we are increasing the + * perf state of the power domain + */ + if (sync) + return rpmh_write(pd->dev, state, &cmd, 1); + else + return rpmh_write_async(pd->dev, state, &cmd, 1); +} + +static void to_active_sleep(struct rpmhpd *pd, unsigned int corner, + unsigned int *active, unsigned int *sleep) +{ + *active = corner; + + if (pd->active_only) + *sleep = 0; + else + *sleep = *active; +} + +/* + * This function is used to aggregate the votes across the active only + * resources and its peers. The aggregated votes are sent to RPMh as + * ACTIVE_ONLY votes (which take effect immediately), as WAKE_ONLY votes + * (applied by RPMh on system wakeup) and as SLEEP votes (applied by RPMh + * on system sleep). + * We send ACTIVE_ONLY votes for resources without any peers. For others, + * which have an active only peer, all 3 votes are sent. + */ +static int rpmhpd_aggregate_corner(struct rpmhpd *pd, unsigned int corner) +{ + int ret; + struct rpmhpd *peer = pd->peer; + unsigned int active_corner, sleep_corner; + unsigned int this_active_corner = 0, this_sleep_corner = 0; + unsigned int peer_active_corner = 0, peer_sleep_corner = 0; + + if (pd->state_synced) { + to_active_sleep(pd, corner, &this_active_corner, &this_sleep_corner); + } else { + /* Clamp to highest corner if sync_state hasn't happened */ + this_active_corner = pd->level_count - 1; + this_sleep_corner = pd->level_count - 1; + } + + if (peer && peer->enabled) + to_active_sleep(peer, peer->corner, &peer_active_corner, + &peer_sleep_corner); + + active_corner = max(this_active_corner, peer_active_corner); + + ret = rpmhpd_send_corner(pd, RPMH_ACTIVE_ONLY_STATE, active_corner, + active_corner > pd->active_corner); + if (ret) + return ret; + + pd->active_corner = active_corner; + + if (peer) { + peer->active_corner = active_corner; + + ret = rpmhpd_send_corner(pd, RPMH_WAKE_ONLY_STATE, + active_corner, false); + if (ret) + return ret; + + sleep_corner = max(this_sleep_corner, peer_sleep_corner); + + return rpmhpd_send_corner(pd, RPMH_SLEEP_STATE, sleep_corner, + false); + } + + return ret; +} + +static int rpmhpd_power_on(struct generic_pm_domain *domain) +{ + struct rpmhpd *pd = domain_to_rpmhpd(domain); + unsigned int corner; + int ret; + + mutex_lock(&rpmhpd_lock); + + corner = max(pd->corner, pd->enable_corner); + ret = rpmhpd_aggregate_corner(pd, corner); + if (!ret) + pd->enabled = true; + + mutex_unlock(&rpmhpd_lock); + + return ret; +} + +static int rpmhpd_power_off(struct generic_pm_domain *domain) +{ + struct rpmhpd *pd = domain_to_rpmhpd(domain); + int ret; + + mutex_lock(&rpmhpd_lock); + + ret = rpmhpd_aggregate_corner(pd, 0); + if (!ret) + pd->enabled = false; + + mutex_unlock(&rpmhpd_lock); + + return ret; +} + +static int rpmhpd_set_performance_state(struct generic_pm_domain *domain, + unsigned int level) +{ + struct rpmhpd *pd = domain_to_rpmhpd(domain); + int ret = 0, i; + + mutex_lock(&rpmhpd_lock); + + for (i = 0; i < pd->level_count; i++) + if (level <= pd->level[i]) + break; + + /* + * If the level requested is more than that supported by the + * max corner, just set it to max anyway. + */ + if (i == pd->level_count) + i--; + + if (pd->enabled) { + /* Ensure that the domain isn't turn off */ + if (i < pd->enable_corner) + i = pd->enable_corner; + + ret = rpmhpd_aggregate_corner(pd, i); + if (ret) + goto out; + } + + pd->corner = i; +out: + mutex_unlock(&rpmhpd_lock); + + return ret; +} + +static unsigned int rpmhpd_get_performance_state(struct generic_pm_domain *genpd, + struct dev_pm_opp *opp) +{ + return dev_pm_opp_get_level(opp); +} + +static int rpmhpd_update_level_mapping(struct rpmhpd *rpmhpd) +{ + int i; + const u16 *buf; + + buf = cmd_db_read_aux_data(rpmhpd->res_name, &rpmhpd->level_count); + if (IS_ERR(buf)) + return PTR_ERR(buf); + + /* 2 bytes used for each command DB aux data entry */ + rpmhpd->level_count >>= 1; + + if (rpmhpd->level_count > RPMH_ARC_MAX_LEVELS) + return -EINVAL; + + for (i = 0; i < rpmhpd->level_count; i++) { + rpmhpd->level[i] = buf[i]; + + /* Remember the first corner with non-zero level */ + if (!rpmhpd->level[rpmhpd->enable_corner] && rpmhpd->level[i]) + rpmhpd->enable_corner = i; + + /* + * The AUX data may be zero padded. These 0 valued entries at + * the end of the map must be ignored. + */ + if (i > 0 && rpmhpd->level[i] == 0) { + rpmhpd->level_count = i; + break; + } + pr_debug("%s: ARC hlvl=%2d --> vlvl=%4u\n", rpmhpd->res_name, i, + rpmhpd->level[i]); + } + + return 0; +} + +static int rpmhpd_probe(struct platform_device *pdev) +{ + int i, ret; + size_t num_pds; + struct device *dev = &pdev->dev; + struct genpd_onecell_data *data; + struct rpmhpd **rpmhpds; + const struct rpmhpd_desc *desc; + + desc = of_device_get_match_data(dev); + if (!desc) + return -EINVAL; + + rpmhpds = desc->rpmhpds; + num_pds = desc->num_pds; + + data = devm_kzalloc(dev, sizeof(*data), GFP_KERNEL); + if (!data) + return -ENOMEM; + + data->domains = devm_kcalloc(dev, num_pds, sizeof(*data->domains), + GFP_KERNEL); + if (!data->domains) + return -ENOMEM; + + data->num_domains = num_pds; + + for (i = 0; i < num_pds; i++) { + if (!rpmhpds[i]) + continue; + + rpmhpds[i]->dev = dev; + rpmhpds[i]->addr = cmd_db_read_addr(rpmhpds[i]->res_name); + if (!rpmhpds[i]->addr) { + dev_err(dev, "Could not find RPMh address for resource %s\n", + rpmhpds[i]->res_name); + return -ENODEV; + } + + ret = cmd_db_read_slave_id(rpmhpds[i]->res_name); + if (ret != CMD_DB_HW_ARC) { + dev_err(dev, "RPMh slave ID mismatch\n"); + return -EINVAL; + } + + ret = rpmhpd_update_level_mapping(rpmhpds[i]); + if (ret) + return ret; + + rpmhpds[i]->pd.power_off = rpmhpd_power_off; + rpmhpds[i]->pd.power_on = rpmhpd_power_on; + rpmhpds[i]->pd.set_performance_state = rpmhpd_set_performance_state; + rpmhpds[i]->pd.opp_to_performance_state = rpmhpd_get_performance_state; + pm_genpd_init(&rpmhpds[i]->pd, NULL, true); + + data->domains[i] = &rpmhpds[i]->pd; + } + + /* Add subdomains */ + for (i = 0; i < num_pds; i++) { + if (!rpmhpds[i]) + continue; + if (rpmhpds[i]->parent) + pm_genpd_add_subdomain(rpmhpds[i]->parent, + &rpmhpds[i]->pd); + } + + return of_genpd_add_provider_onecell(pdev->dev.of_node, data); +} + +static void rpmhpd_sync_state(struct device *dev) +{ + const struct rpmhpd_desc *desc = of_device_get_match_data(dev); + struct rpmhpd **rpmhpds = desc->rpmhpds; + unsigned int corner; + struct rpmhpd *pd; + unsigned int i; + int ret; + + mutex_lock(&rpmhpd_lock); + for (i = 0; i < desc->num_pds; i++) { + pd = rpmhpds[i]; + if (!pd) + continue; + + pd->state_synced = true; + if (pd->enabled) + corner = max(pd->corner, pd->enable_corner); + else + corner = 0; + + ret = rpmhpd_aggregate_corner(pd, corner); + if (ret) + dev_err(dev, "failed to sync %s\n", pd->res_name); + } + mutex_unlock(&rpmhpd_lock); +} + +static struct platform_driver rpmhpd_driver = { + .driver = { + .name = "qcom-rpmhpd", + .of_match_table = rpmhpd_match_table, + .suppress_bind_attrs = true, + .sync_state = rpmhpd_sync_state, + }, + .probe = rpmhpd_probe, +}; + +static int __init rpmhpd_init(void) +{ + return platform_driver_register(&rpmhpd_driver); +} +core_initcall(rpmhpd_init); + +MODULE_DESCRIPTION("Qualcomm Technologies, Inc. RPMh Power Domain Driver"); +MODULE_LICENSE("GPL v2"); diff --git a/drivers/pmdomain/qcom/rpmpd.c b/drivers/pmdomain/qcom/rpmpd.c new file mode 100644 index 000000000000..3135dd1dafe0 --- /dev/null +++ b/drivers/pmdomain/qcom/rpmpd.c @@ -0,0 +1,1023 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2017-2018, The Linux Foundation. All rights reserved. */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +#define domain_to_rpmpd(domain) container_of(domain, struct rpmpd, pd) + +/* Resource types: + * RPMPD_X is X encoded as a little-endian, lower-case, ASCII string */ +#define RPMPD_SMPA 0x61706d73 +#define RPMPD_LDOA 0x616f646c +#define RPMPD_SMPB 0x62706d73 +#define RPMPD_LDOB 0x626f646c +#define RPMPD_RWCX 0x78637772 +#define RPMPD_RWMX 0x786d7772 +#define RPMPD_RWLC 0x636c7772 +#define RPMPD_RWLM 0x6d6c7772 +#define RPMPD_RWSC 0x63737772 +#define RPMPD_RWSM 0x6d737772 +#define RPMPD_RWGX 0x78677772 + +/* Operation Keys */ +#define KEY_CORNER 0x6e726f63 /* corn */ +#define KEY_ENABLE 0x6e657773 /* swen */ +#define KEY_FLOOR_CORNER 0x636676 /* vfc */ +#define KEY_FLOOR_LEVEL 0x6c6676 /* vfl */ +#define KEY_LEVEL 0x6c766c76 /* vlvl */ + +#define MAX_CORNER_RPMPD_STATE 6 + +struct rpmpd_req { + __le32 key; + __le32 nbytes; + __le32 value; +}; + +struct rpmpd { + struct generic_pm_domain pd; + struct generic_pm_domain *parent; + struct rpmpd *peer; + const bool active_only; + unsigned int corner; + bool enabled; + const int res_type; + const int res_id; + struct qcom_smd_rpm *rpm; + unsigned int max_state; + __le32 key; + bool state_synced; +}; + +struct rpmpd_desc { + struct rpmpd **rpmpds; + size_t num_pds; + unsigned int max_state; +}; + +static DEFINE_MUTEX(rpmpd_lock); + +/* CX */ +static struct rpmpd cx_rwcx0_lvl_ao; +static struct rpmpd cx_rwcx0_lvl = { + .pd = { .name = "cx", }, + .peer = &cx_rwcx0_lvl_ao, + .res_type = RPMPD_RWCX, + .res_id = 0, + .key = KEY_LEVEL, +}; + +static struct rpmpd cx_rwcx0_lvl_ao = { + .pd = { .name = "cx_ao", }, + .peer = &cx_rwcx0_lvl, + .active_only = true, + .res_type = RPMPD_RWCX, + .res_id = 0, + .key = KEY_LEVEL, +}; + +static struct rpmpd cx_s1a_corner_ao; +static struct rpmpd cx_s1a_corner = { + .pd = { .name = "cx", }, + .peer = &cx_s1a_corner_ao, + .res_type = RPMPD_SMPA, + .res_id = 1, + .key = KEY_CORNER, +}; + +static struct rpmpd cx_s1a_corner_ao = { + .pd = { .name = "cx_ao", }, + .peer = &cx_s1a_corner, + .active_only = true, + .res_type = RPMPD_SMPA, + .res_id = 1, + .key = KEY_CORNER, +}; + +static struct rpmpd cx_s2a_corner_ao; +static struct rpmpd cx_s2a_corner = { + .pd = { .name = "cx", }, + .peer = &cx_s2a_corner_ao, + .res_type = RPMPD_SMPA, + .res_id = 2, + .key = KEY_CORNER, +}; + +static struct rpmpd cx_s2a_corner_ao = { + .pd = { .name = "cx_ao", }, + .peer = &cx_s2a_corner, + .active_only = true, + .res_type = RPMPD_SMPA, + .res_id = 2, + .key = KEY_CORNER, +}; + +static struct rpmpd cx_s2a_lvl_ao; +static struct rpmpd cx_s2a_lvl = { + .pd = { .name = "cx", }, + .peer = &cx_s2a_lvl_ao, + .res_type = RPMPD_SMPA, + .res_id = 2, + .key = KEY_LEVEL, +}; + +static struct rpmpd cx_s2a_lvl_ao = { + .pd = { .name = "cx_ao", }, + .peer = &cx_s2a_lvl, + .active_only = true, + .res_type = RPMPD_SMPA, + .res_id = 2, + .key = KEY_LEVEL, +}; + +static struct rpmpd cx_s3a_lvl_ao; +static struct rpmpd cx_s3a_lvl = { + .pd = { .name = "cx", }, + .peer = &cx_s3a_lvl_ao, + .res_type = RPMPD_SMPA, + .res_id = 3, + .key = KEY_LEVEL, +}; + +static struct rpmpd cx_s3a_lvl_ao = { + .pd = { .name = "cx_ao", }, + .peer = &cx_s3a_lvl, + .active_only = true, + .res_type = RPMPD_SMPA, + .res_id = 3, + .key = KEY_LEVEL, +}; + +static struct rpmpd cx_rwcx0_vfl = { + .pd = { .name = "cx_vfl", }, + .res_type = RPMPD_RWCX, + .res_id = 0, + .key = KEY_FLOOR_LEVEL, +}; + +static struct rpmpd cx_rwsc2_vfl = { + .pd = { .name = "cx_vfl", }, + .res_type = RPMPD_RWSC, + .res_id = 2, + .key = KEY_FLOOR_LEVEL, +}; + +static struct rpmpd cx_s1a_vfc = { + .pd = { .name = "cx_vfc", }, + .res_type = RPMPD_SMPA, + .res_id = 1, + .key = KEY_FLOOR_CORNER, +}; + +static struct rpmpd cx_s2a_vfc = { + .pd = { .name = "cx_vfc", }, + .res_type = RPMPD_SMPA, + .res_id = 2, + .key = KEY_FLOOR_CORNER, +}; + +static struct rpmpd cx_s2a_vfl = { + .pd = { .name = "cx_vfl", }, + .res_type = RPMPD_SMPA, + .res_id = 2, + .key = KEY_FLOOR_LEVEL, +}; + +static struct rpmpd cx_s3a_vfl = { + .pd = { .name = "cx_vfl", }, + .res_type = RPMPD_SMPA, + .res_id = 3, + .key = KEY_FLOOR_LEVEL, +}; + +/* G(F)X */ +static struct rpmpd gfx_s2b_corner = { + .pd = { .name = "gfx", }, + .res_type = RPMPD_SMPB, + .res_id = 2, + .key = KEY_CORNER, +}; + +static struct rpmpd gfx_s2b_vfc = { + .pd = { .name = "gfx_vfc", }, + .res_type = RPMPD_SMPB, + .res_id = 2, + .key = KEY_FLOOR_CORNER, +}; + +static struct rpmpd mx_rwmx0_lvl; +static struct rpmpd gx_rwgx0_lvl_ao; +static struct rpmpd gx_rwgx0_lvl = { + .pd = { .name = "gx", }, + .peer = &gx_rwgx0_lvl_ao, + .res_type = RPMPD_RWGX, + .parent = &mx_rwmx0_lvl.pd, + .res_id = 0, + .key = KEY_LEVEL, +}; + +static struct rpmpd mx_rwmx0_lvl_ao; +static struct rpmpd gx_rwgx0_lvl_ao = { + .pd = { .name = "gx_ao", }, + .peer = &gx_rwgx0_lvl, + .parent = &mx_rwmx0_lvl_ao.pd, + .active_only = true, + .res_type = RPMPD_RWGX, + .res_id = 0, + .key = KEY_LEVEL, +}; + +/* MX */ +static struct rpmpd mx_l3a_corner_ao; +static struct rpmpd mx_l3a_corner = { + .pd = { .name = "mx", }, + .peer = &mx_l3a_corner_ao, + .res_type = RPMPD_LDOA, + .res_id = 3, + .key = KEY_CORNER, +}; + +static struct rpmpd mx_l3a_corner_ao = { + .pd = { .name = "mx_ao", }, + .peer = &mx_l3a_corner, + .active_only = true, + .res_type = RPMPD_LDOA, + .res_id = 3, + .key = KEY_CORNER, +}; + +static struct rpmpd mx_l12a_lvl_ao; +static struct rpmpd mx_l12a_lvl = { + .pd = { .name = "mx", }, + .peer = &mx_l12a_lvl_ao, + .res_type = RPMPD_LDOA, + .res_id = 12, + .key = KEY_LEVEL, +}; + +static struct rpmpd mx_l12a_lvl_ao = { + .pd = { .name = "mx_ao", }, + .peer = &mx_l12a_lvl, + .active_only = true, + .res_type = RPMPD_LDOA, + .res_id = 12, + .key = KEY_LEVEL, +}; + +static struct rpmpd mx_s2a_corner_ao; +static struct rpmpd mx_s2a_corner = { + .pd = { .name = "mx", }, + .peer = &mx_s2a_corner_ao, + .res_type = RPMPD_SMPA, + .res_id = 2, + .key = KEY_CORNER, +}; + +static struct rpmpd mx_s2a_corner_ao = { + .pd = { .name = "mx_ao", }, + .peer = &mx_s2a_corner, + .active_only = true, + .res_type = RPMPD_SMPA, + .res_id = 2, + .key = KEY_CORNER, +}; + +static struct rpmpd mx_rwmx0_lvl_ao; +static struct rpmpd mx_rwmx0_lvl = { + .pd = { .name = "mx", }, + .peer = &mx_rwmx0_lvl_ao, + .res_type = RPMPD_RWMX, + .res_id = 0, + .key = KEY_LEVEL, +}; + +static struct rpmpd mx_rwmx0_lvl_ao = { + .pd = { .name = "mx_ao", }, + .peer = &mx_rwmx0_lvl, + .active_only = true, + .res_type = RPMPD_RWMX, + .res_id = 0, + .key = KEY_LEVEL, +}; + +static struct rpmpd mx_s6a_lvl_ao; +static struct rpmpd mx_s6a_lvl = { + .pd = { .name = "mx", }, + .peer = &mx_s6a_lvl_ao, + .res_type = RPMPD_SMPA, + .res_id = 6, + .key = KEY_LEVEL, +}; + +static struct rpmpd mx_s6a_lvl_ao = { + .pd = { .name = "mx_ao", }, + .peer = &mx_s6a_lvl, + .active_only = true, + .res_type = RPMPD_SMPA, + .res_id = 6, + .key = KEY_LEVEL, +}; + +static struct rpmpd mx_s7a_lvl_ao; +static struct rpmpd mx_s7a_lvl = { + .pd = { .name = "mx", }, + .peer = &mx_s7a_lvl_ao, + .res_type = RPMPD_SMPA, + .res_id = 7, + .key = KEY_LEVEL, +}; + +static struct rpmpd mx_s7a_lvl_ao = { + .pd = { .name = "mx_ao", }, + .peer = &mx_s7a_lvl, + .active_only = true, + .res_type = RPMPD_SMPA, + .res_id = 7, + .key = KEY_LEVEL, +}; + +static struct rpmpd mx_l12a_vfl = { + .pd = { .name = "mx_vfl", }, + .res_type = RPMPD_LDOA, + .res_id = 12, + .key = KEY_FLOOR_LEVEL, +}; + +static struct rpmpd mx_rwmx0_vfl = { + .pd = { .name = "mx_vfl", }, + .res_type = RPMPD_RWMX, + .res_id = 0, + .key = KEY_FLOOR_LEVEL, +}; + +static struct rpmpd mx_rwsm6_vfl = { + .pd = { .name = "mx_vfl", }, + .res_type = RPMPD_RWSM, + .res_id = 6, + .key = KEY_FLOOR_LEVEL, +}; + +/* MD */ +static struct rpmpd md_s1a_corner_ao; +static struct rpmpd md_s1a_corner = { + .pd = { .name = "md", }, + .peer = &md_s1a_corner_ao, + .res_type = RPMPD_SMPA, + .res_id = 1, + .key = KEY_CORNER, +}; + +static struct rpmpd md_s1a_corner_ao = { + .pd = { .name = "md_ao", }, + .peer = &md_s1a_corner, + .active_only = true, + .res_type = RPMPD_SMPA, + .res_id = 1, + .key = KEY_CORNER, +}; + +static struct rpmpd md_s1a_lvl_ao; +static struct rpmpd md_s1a_lvl = { + .pd = { .name = "md", }, + .peer = &md_s1a_lvl_ao, + .res_type = RPMPD_SMPA, + .res_id = 1, + .key = KEY_LEVEL, +}; + +static struct rpmpd md_s1a_lvl_ao = { + .pd = { .name = "md_ao", }, + .peer = &md_s1a_lvl, + .active_only = true, + .res_type = RPMPD_SMPA, + .res_id = 1, + .key = KEY_LEVEL, +}; + +static struct rpmpd md_s1a_vfc = { + .pd = { .name = "md_vfc", }, + .res_type = RPMPD_SMPA, + .res_id = 1, + .key = KEY_FLOOR_CORNER, +}; + +/* LPI_CX */ +static struct rpmpd lpi_cx_rwlc0_lvl = { + .pd = { .name = "lpi_cx", }, + .res_type = RPMPD_RWLC, + .res_id = 0, + .key = KEY_LEVEL, +}; + +static struct rpmpd lpi_cx_rwlc0_vfl = { + .pd = { .name = "lpi_cx_vfl", }, + .res_type = RPMPD_RWLC, + .res_id = 0, + .key = KEY_FLOOR_LEVEL, +}; + +/* LPI_MX */ +static struct rpmpd lpi_mx_rwlm0_lvl = { + .pd = { .name = "lpi_mx", }, + .res_type = RPMPD_RWLM, + .res_id = 0, + .key = KEY_LEVEL, +}; + +static struct rpmpd lpi_mx_rwlm0_vfl = { + .pd = { .name = "lpi_mx_vfl", }, + .res_type = RPMPD_RWLM, + .res_id = 0, + .key = KEY_FLOOR_LEVEL, +}; + +/* SSC_CX */ +static struct rpmpd ssc_cx_l26a_corner = { + .pd = { .name = "ssc_cx", }, + .res_type = RPMPD_LDOA, + .res_id = 26, + .key = KEY_CORNER, +}; + +static struct rpmpd ssc_cx_rwlc0_lvl = { + .pd = { .name = "ssc_cx", }, + .res_type = RPMPD_RWLC, + .res_id = 0, + .key = KEY_LEVEL, +}; + +static struct rpmpd ssc_cx_rwsc0_lvl = { + .pd = { .name = "ssc_cx", }, + .res_type = RPMPD_RWSC, + .res_id = 0, + .key = KEY_LEVEL, +}; + +static struct rpmpd ssc_cx_l26a_vfc = { + .pd = { .name = "ssc_cx_vfc", }, + .res_type = RPMPD_LDOA, + .res_id = 26, + .key = KEY_FLOOR_CORNER, +}; + +static struct rpmpd ssc_cx_rwlc0_vfl = { + .pd = { .name = "ssc_cx_vfl", }, + .res_type = RPMPD_RWLC, + .res_id = 0, + .key = KEY_FLOOR_LEVEL, +}; + +static struct rpmpd ssc_cx_rwsc0_vfl = { + .pd = { .name = "ssc_cx_vfl", }, + .res_type = RPMPD_RWSC, + .res_id = 0, + .key = KEY_FLOOR_LEVEL, +}; + +/* SSC_MX */ +static struct rpmpd ssc_mx_rwlm0_lvl = { + .pd = { .name = "ssc_mx", }, + .res_type = RPMPD_RWLM, + .res_id = 0, + .key = KEY_LEVEL, +}; + +static struct rpmpd ssc_mx_rwsm0_lvl = { + .pd = { .name = "ssc_mx", }, + .res_type = RPMPD_RWSM, + .res_id = 0, + .key = KEY_LEVEL, +}; + +static struct rpmpd ssc_mx_rwlm0_vfl = { + .pd = { .name = "ssc_mx_vfl", }, + .res_type = RPMPD_RWLM, + .res_id = 0, + .key = KEY_FLOOR_LEVEL, +}; + +static struct rpmpd ssc_mx_rwsm0_vfl = { + .pd = { .name = "ssc_mx_vfl", }, + .res_type = RPMPD_RWSM, + .res_id = 0, + .key = KEY_FLOOR_LEVEL, +}; + +static struct rpmpd *mdm9607_rpmpds[] = { + [MDM9607_VDDCX] = &cx_s3a_lvl, + [MDM9607_VDDCX_AO] = &cx_s3a_lvl_ao, + [MDM9607_VDDCX_VFL] = &cx_s3a_vfl, + [MDM9607_VDDMX] = &mx_l12a_lvl, + [MDM9607_VDDMX_AO] = &mx_l12a_lvl_ao, + [MDM9607_VDDMX_VFL] = &mx_l12a_vfl, +}; + +static const struct rpmpd_desc mdm9607_desc = { + .rpmpds = mdm9607_rpmpds, + .num_pds = ARRAY_SIZE(mdm9607_rpmpds), + .max_state = RPM_SMD_LEVEL_TURBO, +}; + +static struct rpmpd *msm8226_rpmpds[] = { + [MSM8226_VDDCX] = &cx_s1a_corner, + [MSM8226_VDDCX_AO] = &cx_s1a_corner_ao, + [MSM8226_VDDCX_VFC] = &cx_s1a_vfc, +}; + +static const struct rpmpd_desc msm8226_desc = { + .rpmpds = msm8226_rpmpds, + .num_pds = ARRAY_SIZE(msm8226_rpmpds), + .max_state = MAX_CORNER_RPMPD_STATE, +}; + +static struct rpmpd *msm8939_rpmpds[] = { + [MSM8939_VDDMDCX] = &md_s1a_corner, + [MSM8939_VDDMDCX_AO] = &md_s1a_corner_ao, + [MSM8939_VDDMDCX_VFC] = &md_s1a_vfc, + [MSM8939_VDDCX] = &cx_s2a_corner, + [MSM8939_VDDCX_AO] = &cx_s2a_corner_ao, + [MSM8939_VDDCX_VFC] = &cx_s2a_vfc, + [MSM8939_VDDMX] = &mx_l3a_corner, + [MSM8939_VDDMX_AO] = &mx_l3a_corner_ao, +}; + +static const struct rpmpd_desc msm8939_desc = { + .rpmpds = msm8939_rpmpds, + .num_pds = ARRAY_SIZE(msm8939_rpmpds), + .max_state = MAX_CORNER_RPMPD_STATE, +}; + +static struct rpmpd *msm8916_rpmpds[] = { + [MSM8916_VDDCX] = &cx_s1a_corner, + [MSM8916_VDDCX_AO] = &cx_s1a_corner_ao, + [MSM8916_VDDCX_VFC] = &cx_s1a_vfc, + [MSM8916_VDDMX] = &mx_l3a_corner, + [MSM8916_VDDMX_AO] = &mx_l3a_corner_ao, +}; + +static const struct rpmpd_desc msm8916_desc = { + .rpmpds = msm8916_rpmpds, + .num_pds = ARRAY_SIZE(msm8916_rpmpds), + .max_state = MAX_CORNER_RPMPD_STATE, +}; + +static struct rpmpd *msm8953_rpmpds[] = { + [MSM8953_VDDMD] = &md_s1a_lvl, + [MSM8953_VDDMD_AO] = &md_s1a_lvl_ao, + [MSM8953_VDDCX] = &cx_s2a_lvl, + [MSM8953_VDDCX_AO] = &cx_s2a_lvl_ao, + [MSM8953_VDDCX_VFL] = &cx_s2a_vfl, + [MSM8953_VDDMX] = &mx_s7a_lvl, + [MSM8953_VDDMX_AO] = &mx_s7a_lvl_ao, +}; + +static const struct rpmpd_desc msm8953_desc = { + .rpmpds = msm8953_rpmpds, + .num_pds = ARRAY_SIZE(msm8953_rpmpds), + .max_state = RPM_SMD_LEVEL_TURBO, +}; + +static struct rpmpd *msm8976_rpmpds[] = { + [MSM8976_VDDCX] = &cx_s2a_lvl, + [MSM8976_VDDCX_AO] = &cx_s2a_lvl_ao, + [MSM8976_VDDCX_VFL] = &cx_rwsc2_vfl, + [MSM8976_VDDMX] = &mx_s6a_lvl, + [MSM8976_VDDMX_AO] = &mx_s6a_lvl_ao, + [MSM8976_VDDMX_VFL] = &mx_rwsm6_vfl, +}; + +static const struct rpmpd_desc msm8976_desc = { + .rpmpds = msm8976_rpmpds, + .num_pds = ARRAY_SIZE(msm8976_rpmpds), + .max_state = RPM_SMD_LEVEL_TURBO_HIGH, +}; + +static struct rpmpd *msm8994_rpmpds[] = { + [MSM8994_VDDCX] = &cx_s1a_corner, + [MSM8994_VDDCX_AO] = &cx_s1a_corner_ao, + [MSM8994_VDDCX_VFC] = &cx_s1a_vfc, + [MSM8994_VDDMX] = &mx_s2a_corner, + [MSM8994_VDDMX_AO] = &mx_s2a_corner_ao, + + /* Attention! *Some* 8994 boards with pm8004 may use SMPC here! */ + [MSM8994_VDDGFX] = &gfx_s2b_corner, + [MSM8994_VDDGFX_VFC] = &gfx_s2b_vfc, +}; + +static const struct rpmpd_desc msm8994_desc = { + .rpmpds = msm8994_rpmpds, + .num_pds = ARRAY_SIZE(msm8994_rpmpds), + .max_state = MAX_CORNER_RPMPD_STATE, +}; + +static struct rpmpd *msm8996_rpmpds[] = { + [MSM8996_VDDCX] = &cx_s1a_corner, + [MSM8996_VDDCX_AO] = &cx_s1a_corner_ao, + [MSM8996_VDDCX_VFC] = &cx_s1a_vfc, + [MSM8996_VDDMX] = &mx_s2a_corner, + [MSM8996_VDDMX_AO] = &mx_s2a_corner_ao, + [MSM8996_VDDSSCX] = &ssc_cx_l26a_corner, + [MSM8996_VDDSSCX_VFC] = &ssc_cx_l26a_vfc, +}; + +static const struct rpmpd_desc msm8996_desc = { + .rpmpds = msm8996_rpmpds, + .num_pds = ARRAY_SIZE(msm8996_rpmpds), + .max_state = MAX_CORNER_RPMPD_STATE, +}; + +static struct rpmpd *msm8998_rpmpds[] = { + [MSM8998_VDDCX] = &cx_rwcx0_lvl, + [MSM8998_VDDCX_AO] = &cx_rwcx0_lvl_ao, + [MSM8998_VDDCX_VFL] = &cx_rwcx0_vfl, + [MSM8998_VDDMX] = &mx_rwmx0_lvl, + [MSM8998_VDDMX_AO] = &mx_rwmx0_lvl_ao, + [MSM8998_VDDMX_VFL] = &mx_rwmx0_vfl, + [MSM8998_SSCCX] = &ssc_cx_rwsc0_lvl, + [MSM8998_SSCCX_VFL] = &ssc_cx_rwsc0_vfl, + [MSM8998_SSCMX] = &ssc_mx_rwsm0_lvl, + [MSM8998_SSCMX_VFL] = &ssc_mx_rwsm0_vfl, +}; + +static const struct rpmpd_desc msm8998_desc = { + .rpmpds = msm8998_rpmpds, + .num_pds = ARRAY_SIZE(msm8998_rpmpds), + .max_state = RPM_SMD_LEVEL_BINNING, +}; + +static struct rpmpd *qcs404_rpmpds[] = { + [QCS404_VDDMX] = &mx_rwmx0_lvl, + [QCS404_VDDMX_AO] = &mx_rwmx0_lvl_ao, + [QCS404_VDDMX_VFL] = &mx_rwmx0_vfl, + [QCS404_LPICX] = &lpi_cx_rwlc0_lvl, + [QCS404_LPICX_VFL] = &lpi_cx_rwlc0_vfl, + [QCS404_LPIMX] = &lpi_mx_rwlm0_lvl, + [QCS404_LPIMX_VFL] = &lpi_mx_rwlm0_vfl, +}; + +static const struct rpmpd_desc qcs404_desc = { + .rpmpds = qcs404_rpmpds, + .num_pds = ARRAY_SIZE(qcs404_rpmpds), + .max_state = RPM_SMD_LEVEL_BINNING, +}; + +static struct rpmpd *sdm660_rpmpds[] = { + [SDM660_VDDCX] = &cx_rwcx0_lvl, + [SDM660_VDDCX_AO] = &cx_rwcx0_lvl_ao, + [SDM660_VDDCX_VFL] = &cx_rwcx0_vfl, + [SDM660_VDDMX] = &mx_rwmx0_lvl, + [SDM660_VDDMX_AO] = &mx_rwmx0_lvl_ao, + [SDM660_VDDMX_VFL] = &mx_rwmx0_vfl, + [SDM660_SSCCX] = &ssc_cx_rwlc0_lvl, + [SDM660_SSCCX_VFL] = &ssc_cx_rwlc0_vfl, + [SDM660_SSCMX] = &ssc_mx_rwlm0_lvl, + [SDM660_SSCMX_VFL] = &ssc_mx_rwlm0_vfl, +}; + +static const struct rpmpd_desc sdm660_desc = { + .rpmpds = sdm660_rpmpds, + .num_pds = ARRAY_SIZE(sdm660_rpmpds), + .max_state = RPM_SMD_LEVEL_TURBO, +}; + +static struct rpmpd *sm6115_rpmpds[] = { + [SM6115_VDDCX] = &cx_rwcx0_lvl, + [SM6115_VDDCX_AO] = &cx_rwcx0_lvl_ao, + [SM6115_VDDCX_VFL] = &cx_rwcx0_vfl, + [SM6115_VDDMX] = &mx_rwmx0_lvl, + [SM6115_VDDMX_AO] = &mx_rwmx0_lvl_ao, + [SM6115_VDDMX_VFL] = &mx_rwmx0_vfl, + [SM6115_VDD_LPI_CX] = &lpi_cx_rwlc0_lvl, + [SM6115_VDD_LPI_MX] = &lpi_mx_rwlm0_lvl, +}; + +static const struct rpmpd_desc sm6115_desc = { + .rpmpds = sm6115_rpmpds, + .num_pds = ARRAY_SIZE(sm6115_rpmpds), + .max_state = RPM_SMD_LEVEL_TURBO_NO_CPR, +}; + +static struct rpmpd *sm6125_rpmpds[] = { + [SM6125_VDDCX] = &cx_rwcx0_lvl, + [SM6125_VDDCX_AO] = &cx_rwcx0_lvl_ao, + [SM6125_VDDCX_VFL] = &cx_rwcx0_vfl, + [SM6125_VDDMX] = &mx_rwmx0_lvl, + [SM6125_VDDMX_AO] = &mx_rwmx0_lvl_ao, + [SM6125_VDDMX_VFL] = &mx_rwmx0_vfl, +}; + +static const struct rpmpd_desc sm6125_desc = { + .rpmpds = sm6125_rpmpds, + .num_pds = ARRAY_SIZE(sm6125_rpmpds), + .max_state = RPM_SMD_LEVEL_BINNING, +}; + +static struct rpmpd *sm6375_rpmpds[] = { + [SM6375_VDDCX] = &cx_rwcx0_lvl, + [SM6375_VDDCX_AO] = &cx_rwcx0_lvl_ao, + [SM6375_VDDCX_VFL] = &cx_rwcx0_vfl, + [SM6375_VDDMX] = &mx_rwmx0_lvl, + [SM6375_VDDMX_AO] = &mx_rwmx0_lvl_ao, + [SM6375_VDDMX_VFL] = &mx_rwmx0_vfl, + [SM6375_VDDGX] = &gx_rwgx0_lvl, + [SM6375_VDDGX_AO] = &gx_rwgx0_lvl_ao, + [SM6375_VDD_LPI_CX] = &lpi_cx_rwlc0_lvl, + [SM6375_VDD_LPI_MX] = &lpi_mx_rwlm0_lvl, +}; + +static const struct rpmpd_desc sm6375_desc = { + .rpmpds = sm6375_rpmpds, + .num_pds = ARRAY_SIZE(sm6375_rpmpds), + .max_state = RPM_SMD_LEVEL_TURBO_NO_CPR, +}; + +static struct rpmpd *qcm2290_rpmpds[] = { + [QCM2290_VDDCX] = &cx_rwcx0_lvl, + [QCM2290_VDDCX_AO] = &cx_rwcx0_lvl_ao, + [QCM2290_VDDCX_VFL] = &cx_rwcx0_vfl, + [QCM2290_VDDMX] = &mx_rwmx0_lvl, + [QCM2290_VDDMX_AO] = &mx_rwmx0_lvl_ao, + [QCM2290_VDDMX_VFL] = &mx_rwmx0_vfl, + [QCM2290_VDD_LPI_CX] = &lpi_cx_rwlc0_lvl, + [QCM2290_VDD_LPI_MX] = &lpi_mx_rwlm0_lvl, +}; + +static const struct rpmpd_desc qcm2290_desc = { + .rpmpds = qcm2290_rpmpds, + .num_pds = ARRAY_SIZE(qcm2290_rpmpds), + .max_state = RPM_SMD_LEVEL_TURBO_NO_CPR, +}; + +static const struct of_device_id rpmpd_match_table[] = { + { .compatible = "qcom,mdm9607-rpmpd", .data = &mdm9607_desc }, + { .compatible = "qcom,msm8226-rpmpd", .data = &msm8226_desc }, + { .compatible = "qcom,msm8909-rpmpd", .data = &msm8916_desc }, + { .compatible = "qcom,msm8916-rpmpd", .data = &msm8916_desc }, + { .compatible = "qcom,msm8939-rpmpd", .data = &msm8939_desc }, + { .compatible = "qcom,msm8953-rpmpd", .data = &msm8953_desc }, + { .compatible = "qcom,msm8976-rpmpd", .data = &msm8976_desc }, + { .compatible = "qcom,msm8994-rpmpd", .data = &msm8994_desc }, + { .compatible = "qcom,msm8996-rpmpd", .data = &msm8996_desc }, + { .compatible = "qcom,msm8998-rpmpd", .data = &msm8998_desc }, + { .compatible = "qcom,qcm2290-rpmpd", .data = &qcm2290_desc }, + { .compatible = "qcom,qcs404-rpmpd", .data = &qcs404_desc }, + { .compatible = "qcom,sdm660-rpmpd", .data = &sdm660_desc }, + { .compatible = "qcom,sm6115-rpmpd", .data = &sm6115_desc }, + { .compatible = "qcom,sm6125-rpmpd", .data = &sm6125_desc }, + { .compatible = "qcom,sm6375-rpmpd", .data = &sm6375_desc }, + { } +}; +MODULE_DEVICE_TABLE(of, rpmpd_match_table); + +static int rpmpd_send_enable(struct rpmpd *pd, bool enable) +{ + struct rpmpd_req req = { + .key = KEY_ENABLE, + .nbytes = cpu_to_le32(sizeof(u32)), + .value = cpu_to_le32(enable), + }; + + return qcom_rpm_smd_write(pd->rpm, QCOM_SMD_RPM_ACTIVE_STATE, + pd->res_type, pd->res_id, &req, sizeof(req)); +} + +static int rpmpd_send_corner(struct rpmpd *pd, int state, unsigned int corner) +{ + struct rpmpd_req req = { + .key = pd->key, + .nbytes = cpu_to_le32(sizeof(u32)), + .value = cpu_to_le32(corner), + }; + + return qcom_rpm_smd_write(pd->rpm, state, pd->res_type, pd->res_id, + &req, sizeof(req)); +}; + +static void to_active_sleep(struct rpmpd *pd, unsigned int corner, + unsigned int *active, unsigned int *sleep) +{ + *active = corner; + + if (pd->active_only) + *sleep = 0; + else + *sleep = *active; +} + +static int rpmpd_aggregate_corner(struct rpmpd *pd) +{ + int ret; + struct rpmpd *peer = pd->peer; + unsigned int active_corner, sleep_corner; + unsigned int this_active_corner = 0, this_sleep_corner = 0; + unsigned int peer_active_corner = 0, peer_sleep_corner = 0; + + /* Clamp to the highest corner/level if sync_state isn't done yet */ + if (!pd->state_synced) + this_active_corner = this_sleep_corner = pd->max_state - 1; + else + to_active_sleep(pd, pd->corner, &this_active_corner, &this_sleep_corner); + + if (peer && peer->enabled) + to_active_sleep(peer, peer->corner, &peer_active_corner, + &peer_sleep_corner); + + active_corner = max(this_active_corner, peer_active_corner); + + ret = rpmpd_send_corner(pd, QCOM_SMD_RPM_ACTIVE_STATE, active_corner); + if (ret) + return ret; + + sleep_corner = max(this_sleep_corner, peer_sleep_corner); + + return rpmpd_send_corner(pd, QCOM_SMD_RPM_SLEEP_STATE, sleep_corner); +} + +static int rpmpd_power_on(struct generic_pm_domain *domain) +{ + int ret; + struct rpmpd *pd = domain_to_rpmpd(domain); + + mutex_lock(&rpmpd_lock); + + ret = rpmpd_send_enable(pd, true); + if (ret) + goto out; + + pd->enabled = true; + + if (pd->corner) + ret = rpmpd_aggregate_corner(pd); + +out: + mutex_unlock(&rpmpd_lock); + + return ret; +} + +static int rpmpd_power_off(struct generic_pm_domain *domain) +{ + int ret; + struct rpmpd *pd = domain_to_rpmpd(domain); + + mutex_lock(&rpmpd_lock); + + ret = rpmpd_send_enable(pd, false); + if (!ret) + pd->enabled = false; + + mutex_unlock(&rpmpd_lock); + + return ret; +} + +static int rpmpd_set_performance(struct generic_pm_domain *domain, + unsigned int state) +{ + int ret = 0; + struct rpmpd *pd = domain_to_rpmpd(domain); + + if (state > pd->max_state) + state = pd->max_state; + + mutex_lock(&rpmpd_lock); + + pd->corner = state; + + /* Always send updates for vfc and vfl */ + if (!pd->enabled && pd->key != cpu_to_le32(KEY_FLOOR_CORNER) && + pd->key != cpu_to_le32(KEY_FLOOR_LEVEL)) + goto out; + + ret = rpmpd_aggregate_corner(pd); + +out: + mutex_unlock(&rpmpd_lock); + + return ret; +} + +static unsigned int rpmpd_get_performance(struct generic_pm_domain *genpd, + struct dev_pm_opp *opp) +{ + return dev_pm_opp_get_level(opp); +} + +static int rpmpd_probe(struct platform_device *pdev) +{ + int i; + size_t num; + struct genpd_onecell_data *data; + struct qcom_smd_rpm *rpm; + struct rpmpd **rpmpds; + const struct rpmpd_desc *desc; + + rpm = dev_get_drvdata(pdev->dev.parent); + if (!rpm) { + dev_err(&pdev->dev, "Unable to retrieve handle to RPM\n"); + return -ENODEV; + } + + desc = of_device_get_match_data(&pdev->dev); + if (!desc) + return -EINVAL; + + rpmpds = desc->rpmpds; + num = desc->num_pds; + + data = devm_kzalloc(&pdev->dev, sizeof(*data), GFP_KERNEL); + if (!data) + return -ENOMEM; + + data->domains = devm_kcalloc(&pdev->dev, num, sizeof(*data->domains), + GFP_KERNEL); + if (!data->domains) + return -ENOMEM; + + data->num_domains = num; + + for (i = 0; i < num; i++) { + if (!rpmpds[i]) { + dev_warn(&pdev->dev, "rpmpds[] with empty entry at index=%d\n", + i); + continue; + } + + rpmpds[i]->rpm = rpm; + rpmpds[i]->max_state = desc->max_state; + rpmpds[i]->pd.power_off = rpmpd_power_off; + rpmpds[i]->pd.power_on = rpmpd_power_on; + rpmpds[i]->pd.set_performance_state = rpmpd_set_performance; + rpmpds[i]->pd.opp_to_performance_state = rpmpd_get_performance; + pm_genpd_init(&rpmpds[i]->pd, NULL, true); + + data->domains[i] = &rpmpds[i]->pd; + } + + /* Add subdomains */ + for (i = 0; i < num; i++) { + if (!rpmpds[i]) + continue; + + if (rpmpds[i]->parent) + pm_genpd_add_subdomain(rpmpds[i]->parent, &rpmpds[i]->pd); + } + + return of_genpd_add_provider_onecell(pdev->dev.of_node, data); +} + +static void rpmpd_sync_state(struct device *dev) +{ + const struct rpmpd_desc *desc = of_device_get_match_data(dev); + struct rpmpd **rpmpds = desc->rpmpds; + struct rpmpd *pd; + unsigned int i; + int ret; + + mutex_lock(&rpmpd_lock); + for (i = 0; i < desc->num_pds; i++) { + pd = rpmpds[i]; + if (!pd) + continue; + + pd->state_synced = true; + + if (!pd->enabled) + pd->corner = 0; + + ret = rpmpd_aggregate_corner(pd); + if (ret) + dev_err(dev, "failed to sync %s: %d\n", pd->pd.name, ret); + } + mutex_unlock(&rpmpd_lock); +} + +static struct platform_driver rpmpd_driver = { + .driver = { + .name = "qcom-rpmpd", + .of_match_table = rpmpd_match_table, + .suppress_bind_attrs = true, + .sync_state = rpmpd_sync_state, + }, + .probe = rpmpd_probe, +}; + +static int __init rpmpd_init(void) +{ + return platform_driver_register(&rpmpd_driver); +} +core_initcall(rpmpd_init); + +MODULE_DESCRIPTION("Qualcomm Technologies, Inc. RPM Power Domain Driver"); +MODULE_LICENSE("GPL v2"); diff --git a/drivers/pmdomain/renesas/Makefile b/drivers/pmdomain/renesas/Makefile new file mode 100644 index 000000000000..e306e396fc8c --- /dev/null +++ b/drivers/pmdomain/renesas/Makefile @@ -0,0 +1,30 @@ +# SPDX-License-Identifier: GPL-2.0 +# SoC +obj-$(CONFIG_SYSC_R8A7742) += r8a7742-sysc.o +obj-$(CONFIG_SYSC_R8A7743) += r8a7743-sysc.o +obj-$(CONFIG_SYSC_R8A7745) += r8a7745-sysc.o +obj-$(CONFIG_SYSC_R8A77470) += r8a77470-sysc.o +obj-$(CONFIG_SYSC_R8A774A1) += r8a774a1-sysc.o +obj-$(CONFIG_SYSC_R8A774B1) += r8a774b1-sysc.o +obj-$(CONFIG_SYSC_R8A774C0) += r8a774c0-sysc.o +obj-$(CONFIG_SYSC_R8A774E1) += r8a774e1-sysc.o +obj-$(CONFIG_SYSC_R8A7779) += r8a7779-sysc.o +obj-$(CONFIG_SYSC_R8A7790) += r8a7790-sysc.o +obj-$(CONFIG_SYSC_R8A7791) += r8a7791-sysc.o +obj-$(CONFIG_SYSC_R8A7792) += r8a7792-sysc.o +obj-$(CONFIG_SYSC_R8A7794) += r8a7794-sysc.o +obj-$(CONFIG_SYSC_R8A7795) += r8a7795-sysc.o +obj-$(CONFIG_SYSC_R8A77960) += r8a7796-sysc.o +obj-$(CONFIG_SYSC_R8A77961) += r8a7796-sysc.o +obj-$(CONFIG_SYSC_R8A77965) += r8a77965-sysc.o +obj-$(CONFIG_SYSC_R8A77970) += r8a77970-sysc.o +obj-$(CONFIG_SYSC_R8A77980) += r8a77980-sysc.o +obj-$(CONFIG_SYSC_R8A77990) += r8a77990-sysc.o +obj-$(CONFIG_SYSC_R8A77995) += r8a77995-sysc.o +obj-$(CONFIG_SYSC_R8A779A0) += r8a779a0-sysc.o +obj-$(CONFIG_SYSC_R8A779F0) += r8a779f0-sysc.o +obj-$(CONFIG_SYSC_R8A779G0) += r8a779g0-sysc.o +# Family +obj-$(CONFIG_SYSC_RCAR) += rcar-sysc.o +obj-$(CONFIG_SYSC_RCAR_GEN4) += rcar-gen4-sysc.o +obj-$(CONFIG_SYSC_RMOBILE) += rmobile-sysc.o diff --git a/drivers/pmdomain/renesas/r8a7742-sysc.c b/drivers/pmdomain/renesas/r8a7742-sysc.c new file mode 100644 index 000000000000..219a675f83f4 --- /dev/null +++ b/drivers/pmdomain/renesas/r8a7742-sysc.c @@ -0,0 +1,42 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Renesas RZ/G1H System Controller + * + * Copyright (C) 2020 Renesas Electronics Corp. + */ + +#include + +#include + +#include "rcar-sysc.h" + +static const struct rcar_sysc_area r8a7742_areas[] __initconst = { + { "always-on", 0, 0, R8A7742_PD_ALWAYS_ON, -1, PD_ALWAYS_ON }, + { "ca15-scu", 0x180, 0, R8A7742_PD_CA15_SCU, R8A7742_PD_ALWAYS_ON, + PD_SCU }, + { "ca15-cpu0", 0x40, 0, R8A7742_PD_CA15_CPU0, R8A7742_PD_CA15_SCU, + PD_CPU_NOCR }, + { "ca15-cpu1", 0x40, 1, R8A7742_PD_CA15_CPU1, R8A7742_PD_CA15_SCU, + PD_CPU_NOCR }, + { "ca15-cpu2", 0x40, 2, R8A7742_PD_CA15_CPU2, R8A7742_PD_CA15_SCU, + PD_CPU_NOCR }, + { "ca15-cpu3", 0x40, 3, R8A7742_PD_CA15_CPU3, R8A7742_PD_CA15_SCU, + PD_CPU_NOCR }, + { "ca7-scu", 0x100, 0, R8A7742_PD_CA7_SCU, R8A7742_PD_ALWAYS_ON, + PD_SCU }, + { "ca7-cpu0", 0x1c0, 0, R8A7742_PD_CA7_CPU0, R8A7742_PD_CA7_SCU, + PD_CPU_NOCR }, + { "ca7-cpu1", 0x1c0, 1, R8A7742_PD_CA7_CPU1, R8A7742_PD_CA7_SCU, + PD_CPU_NOCR }, + { "ca7-cpu2", 0x1c0, 2, R8A7742_PD_CA7_CPU2, R8A7742_PD_CA7_SCU, + PD_CPU_NOCR }, + { "ca7-cpu3", 0x1c0, 3, R8A7742_PD_CA7_CPU3, R8A7742_PD_CA7_SCU, + PD_CPU_NOCR }, + { "rgx", 0xc0, 0, R8A7742_PD_RGX, R8A7742_PD_ALWAYS_ON }, +}; + +const struct rcar_sysc_info r8a7742_sysc_info __initconst = { + .areas = r8a7742_areas, + .num_areas = ARRAY_SIZE(r8a7742_areas), +}; diff --git a/drivers/pmdomain/renesas/r8a7743-sysc.c b/drivers/pmdomain/renesas/r8a7743-sysc.c new file mode 100644 index 000000000000..4e2c0ab951b3 --- /dev/null +++ b/drivers/pmdomain/renesas/r8a7743-sysc.c @@ -0,0 +1,28 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Renesas RZ/G1M System Controller + * + * Copyright (C) 2016 Cogent Embedded Inc. + */ + +#include + +#include + +#include "rcar-sysc.h" + +static const struct rcar_sysc_area r8a7743_areas[] __initconst = { + { "always-on", 0, 0, R8A7743_PD_ALWAYS_ON, -1, PD_ALWAYS_ON }, + { "ca15-scu", 0x180, 0, R8A7743_PD_CA15_SCU, R8A7743_PD_ALWAYS_ON, + PD_SCU }, + { "ca15-cpu0", 0x40, 0, R8A7743_PD_CA15_CPU0, R8A7743_PD_CA15_SCU, + PD_CPU_NOCR }, + { "ca15-cpu1", 0x40, 1, R8A7743_PD_CA15_CPU1, R8A7743_PD_CA15_SCU, + PD_CPU_NOCR }, + { "sgx", 0xc0, 0, R8A7743_PD_SGX, R8A7743_PD_ALWAYS_ON }, +}; + +const struct rcar_sysc_info r8a7743_sysc_info __initconst = { + .areas = r8a7743_areas, + .num_areas = ARRAY_SIZE(r8a7743_areas), +}; diff --git a/drivers/pmdomain/renesas/r8a7745-sysc.c b/drivers/pmdomain/renesas/r8a7745-sysc.c new file mode 100644 index 000000000000..865821a2f0c6 --- /dev/null +++ b/drivers/pmdomain/renesas/r8a7745-sysc.c @@ -0,0 +1,28 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Renesas RZ/G1E System Controller + * + * Copyright (C) 2016 Cogent Embedded Inc. + */ + +#include + +#include + +#include "rcar-sysc.h" + +static const struct rcar_sysc_area r8a7745_areas[] __initconst = { + { "always-on", 0, 0, R8A7745_PD_ALWAYS_ON, -1, PD_ALWAYS_ON }, + { "ca7-scu", 0x100, 0, R8A7745_PD_CA7_SCU, R8A7745_PD_ALWAYS_ON, + PD_SCU }, + { "ca7-cpu0", 0x1c0, 0, R8A7745_PD_CA7_CPU0, R8A7745_PD_CA7_SCU, + PD_CPU_NOCR }, + { "ca7-cpu1", 0x1c0, 1, R8A7745_PD_CA7_CPU1, R8A7745_PD_CA7_SCU, + PD_CPU_NOCR }, + { "sgx", 0xc0, 0, R8A7745_PD_SGX, R8A7745_PD_ALWAYS_ON }, +}; + +const struct rcar_sysc_info r8a7745_sysc_info __initconst = { + .areas = r8a7745_areas, + .num_areas = ARRAY_SIZE(r8a7745_areas), +}; diff --git a/drivers/pmdomain/renesas/r8a77470-sysc.c b/drivers/pmdomain/renesas/r8a77470-sysc.c new file mode 100644 index 000000000000..1eeb8018df50 --- /dev/null +++ b/drivers/pmdomain/renesas/r8a77470-sysc.c @@ -0,0 +1,28 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Renesas RZ/G1C System Controller + * + * Copyright (C) 2018 Renesas Electronics Corp. + */ + +#include + +#include + +#include "rcar-sysc.h" + +static const struct rcar_sysc_area r8a77470_areas[] __initconst = { + { "always-on", 0, 0, R8A77470_PD_ALWAYS_ON, -1, PD_ALWAYS_ON }, + { "ca7-scu", 0x100, 0, R8A77470_PD_CA7_SCU, R8A77470_PD_ALWAYS_ON, + PD_SCU }, + { "ca7-cpu0", 0x1c0, 0, R8A77470_PD_CA7_CPU0, R8A77470_PD_CA7_SCU, + PD_CPU_NOCR }, + { "ca7-cpu1", 0x1c0, 1, R8A77470_PD_CA7_CPU1, R8A77470_PD_CA7_SCU, + PD_CPU_NOCR }, + { "sgx", 0xc0, 0, R8A77470_PD_SGX, R8A77470_PD_ALWAYS_ON }, +}; + +const struct rcar_sysc_info r8a77470_sysc_info __initconst = { + .areas = r8a77470_areas, + .num_areas = ARRAY_SIZE(r8a77470_areas), +}; diff --git a/drivers/pmdomain/renesas/r8a774a1-sysc.c b/drivers/pmdomain/renesas/r8a774a1-sysc.c new file mode 100644 index 000000000000..38ac2c689ff0 --- /dev/null +++ b/drivers/pmdomain/renesas/r8a774a1-sysc.c @@ -0,0 +1,44 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Renesas RZ/G2M System Controller + * Copyright (C) 2018 Renesas Electronics Corp. + * + * Based on Renesas R-Car M3-W System Controller + * Copyright (C) 2016 Glider bvba + */ + +#include + +#include + +#include "rcar-sysc.h" + +static const struct rcar_sysc_area r8a774a1_areas[] __initconst = { + { "always-on", 0, 0, R8A774A1_PD_ALWAYS_ON, -1, PD_ALWAYS_ON }, + { "ca57-scu", 0x1c0, 0, R8A774A1_PD_CA57_SCU, R8A774A1_PD_ALWAYS_ON, + PD_SCU }, + { "ca57-cpu0", 0x80, 0, R8A774A1_PD_CA57_CPU0, R8A774A1_PD_CA57_SCU, + PD_CPU_NOCR }, + { "ca57-cpu1", 0x80, 1, R8A774A1_PD_CA57_CPU1, R8A774A1_PD_CA57_SCU, + PD_CPU_NOCR }, + { "ca53-scu", 0x140, 0, R8A774A1_PD_CA53_SCU, R8A774A1_PD_ALWAYS_ON, + PD_SCU }, + { "ca53-cpu0", 0x200, 0, R8A774A1_PD_CA53_CPU0, R8A774A1_PD_CA53_SCU, + PD_CPU_NOCR }, + { "ca53-cpu1", 0x200, 1, R8A774A1_PD_CA53_CPU1, R8A774A1_PD_CA53_SCU, + PD_CPU_NOCR }, + { "ca53-cpu2", 0x200, 2, R8A774A1_PD_CA53_CPU2, R8A774A1_PD_CA53_SCU, + PD_CPU_NOCR }, + { "ca53-cpu3", 0x200, 3, R8A774A1_PD_CA53_CPU3, R8A774A1_PD_CA53_SCU, + PD_CPU_NOCR }, + { "a3vc", 0x380, 0, R8A774A1_PD_A3VC, R8A774A1_PD_ALWAYS_ON }, + { "a2vc0", 0x3c0, 0, R8A774A1_PD_A2VC0, R8A774A1_PD_A3VC }, + { "a2vc1", 0x3c0, 1, R8A774A1_PD_A2VC1, R8A774A1_PD_A3VC }, + { "3dg-a", 0x100, 0, R8A774A1_PD_3DG_A, R8A774A1_PD_ALWAYS_ON }, + { "3dg-b", 0x100, 1, R8A774A1_PD_3DG_B, R8A774A1_PD_3DG_A }, +}; + +const struct rcar_sysc_info r8a774a1_sysc_info __initconst = { + .areas = r8a774a1_areas, + .num_areas = ARRAY_SIZE(r8a774a1_areas), +}; diff --git a/drivers/pmdomain/renesas/r8a774b1-sysc.c b/drivers/pmdomain/renesas/r8a774b1-sysc.c new file mode 100644 index 000000000000..5f97ff26f3f8 --- /dev/null +++ b/drivers/pmdomain/renesas/r8a774b1-sysc.c @@ -0,0 +1,37 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Renesas RZ/G2N System Controller + * Copyright (C) 2019 Renesas Electronics Corp. + * + * Based on Renesas R-Car M3-W System Controller + * Copyright (C) 2016 Glider bvba + */ + +#include +#include + +#include + +#include "rcar-sysc.h" + +static const struct rcar_sysc_area r8a774b1_areas[] __initconst = { + { "always-on", 0, 0, R8A774B1_PD_ALWAYS_ON, -1, PD_ALWAYS_ON }, + { "ca57-scu", 0x1c0, 0, R8A774B1_PD_CA57_SCU, R8A774B1_PD_ALWAYS_ON, + PD_SCU }, + { "ca57-cpu0", 0x80, 0, R8A774B1_PD_CA57_CPU0, R8A774B1_PD_CA57_SCU, + PD_CPU_NOCR }, + { "ca57-cpu1", 0x80, 1, R8A774B1_PD_CA57_CPU1, R8A774B1_PD_CA57_SCU, + PD_CPU_NOCR }, + { "a3vc", 0x380, 0, R8A774B1_PD_A3VC, R8A774B1_PD_ALWAYS_ON }, + { "a3vp", 0x340, 0, R8A774B1_PD_A3VP, R8A774B1_PD_ALWAYS_ON }, + { "a2vc1", 0x3c0, 1, R8A774B1_PD_A2VC1, R8A774B1_PD_A3VC }, + { "3dg-a", 0x100, 0, R8A774B1_PD_3DG_A, R8A774B1_PD_ALWAYS_ON }, + { "3dg-b", 0x100, 1, R8A774B1_PD_3DG_B, R8A774B1_PD_3DG_A }, +}; + +const struct rcar_sysc_info r8a774b1_sysc_info __initconst = { + .areas = r8a774b1_areas, + .num_areas = ARRAY_SIZE(r8a774b1_areas), + .extmask_offs = 0x2f8, + .extmask_val = BIT(0), +}; diff --git a/drivers/pmdomain/renesas/r8a774c0-sysc.c b/drivers/pmdomain/renesas/r8a774c0-sysc.c new file mode 100644 index 000000000000..c1c216f7d073 --- /dev/null +++ b/drivers/pmdomain/renesas/r8a774c0-sysc.c @@ -0,0 +1,55 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Renesas RZ/G2E System Controller + * Copyright (C) 2018 Renesas Electronics Corp. + * + * Based on Renesas R-Car E3 System Controller + */ + +#include +#include +#include + +#include + +#include "rcar-sysc.h" + +static struct rcar_sysc_area r8a774c0_areas[] __initdata = { + { "always-on", 0, 0, R8A774C0_PD_ALWAYS_ON, -1, PD_ALWAYS_ON }, + { "ca53-scu", 0x140, 0, R8A774C0_PD_CA53_SCU, R8A774C0_PD_ALWAYS_ON, + PD_SCU }, + { "ca53-cpu0", 0x200, 0, R8A774C0_PD_CA53_CPU0, R8A774C0_PD_CA53_SCU, + PD_CPU_NOCR }, + { "ca53-cpu1", 0x200, 1, R8A774C0_PD_CA53_CPU1, R8A774C0_PD_CA53_SCU, + PD_CPU_NOCR }, + { "a3vc", 0x380, 0, R8A774C0_PD_A3VC, R8A774C0_PD_ALWAYS_ON }, + { "a2vc1", 0x3c0, 1, R8A774C0_PD_A2VC1, R8A774C0_PD_A3VC }, + { "3dg-a", 0x100, 0, R8A774C0_PD_3DG_A, R8A774C0_PD_ALWAYS_ON }, + { "3dg-b", 0x100, 1, R8A774C0_PD_3DG_B, R8A774C0_PD_3DG_A }, +}; + +/* Fixups for RZ/G2E ES1.0 revision */ +static const struct soc_device_attribute r8a774c0[] __initconst = { + { .soc_id = "r8a774c0", .revision = "ES1.0" }, + { /* sentinel */ } +}; + +static int __init r8a774c0_sysc_init(void) +{ + if (soc_device_match(r8a774c0)) { + /* Fix incorrect 3DG hierarchy */ + swap(r8a774c0_areas[6], r8a774c0_areas[7]); + r8a774c0_areas[6].parent = R8A774C0_PD_ALWAYS_ON; + r8a774c0_areas[7].parent = R8A774C0_PD_3DG_B; + } + + return 0; +} + +const struct rcar_sysc_info r8a774c0_sysc_info __initconst = { + .init = r8a774c0_sysc_init, + .areas = r8a774c0_areas, + .num_areas = ARRAY_SIZE(r8a774c0_areas), + .extmask_offs = 0x2f8, + .extmask_val = BIT(0), +}; diff --git a/drivers/pmdomain/renesas/r8a774e1-sysc.c b/drivers/pmdomain/renesas/r8a774e1-sysc.c new file mode 100644 index 000000000000..18449f746455 --- /dev/null +++ b/drivers/pmdomain/renesas/r8a774e1-sysc.c @@ -0,0 +1,43 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Renesas RZ/G2H System Controller + * Copyright (C) 2020 Renesas Electronics Corp. + * + * Based on Renesas R-Car H3 System Controller + * Copyright (C) 2016-2017 Glider bvba + */ + +#include + +#include + +#include "rcar-sysc.h" + +static const struct rcar_sysc_area r8a774e1_areas[] __initconst = { + { "always-on", 0, 0, R8A774E1_PD_ALWAYS_ON, -1, PD_ALWAYS_ON }, + { "ca57-scu", 0x1c0, 0, R8A774E1_PD_CA57_SCU, R8A774E1_PD_ALWAYS_ON, PD_SCU }, + { "ca57-cpu0", 0x80, 0, R8A774E1_PD_CA57_CPU0, R8A774E1_PD_CA57_SCU, PD_CPU_NOCR }, + { "ca57-cpu1", 0x80, 1, R8A774E1_PD_CA57_CPU1, R8A774E1_PD_CA57_SCU, PD_CPU_NOCR }, + { "ca57-cpu2", 0x80, 2, R8A774E1_PD_CA57_CPU2, R8A774E1_PD_CA57_SCU, PD_CPU_NOCR }, + { "ca57-cpu3", 0x80, 3, R8A774E1_PD_CA57_CPU3, R8A774E1_PD_CA57_SCU, PD_CPU_NOCR }, + { "ca53-scu", 0x140, 0, R8A774E1_PD_CA53_SCU, R8A774E1_PD_ALWAYS_ON, PD_SCU }, + { "ca53-cpu0", 0x200, 0, R8A774E1_PD_CA53_CPU0, R8A774E1_PD_CA53_SCU, PD_CPU_NOCR }, + { "ca53-cpu1", 0x200, 1, R8A774E1_PD_CA53_CPU1, R8A774E1_PD_CA53_SCU, PD_CPU_NOCR }, + { "ca53-cpu2", 0x200, 2, R8A774E1_PD_CA53_CPU2, R8A774E1_PD_CA53_SCU, PD_CPU_NOCR }, + { "ca53-cpu3", 0x200, 3, R8A774E1_PD_CA53_CPU3, R8A774E1_PD_CA53_SCU, PD_CPU_NOCR }, + { "a3vp", 0x340, 0, R8A774E1_PD_A3VP, R8A774E1_PD_ALWAYS_ON }, + { "a3vc", 0x380, 0, R8A774E1_PD_A3VC, R8A774E1_PD_ALWAYS_ON }, + { "a2vc1", 0x3c0, 1, R8A774E1_PD_A2VC1, R8A774E1_PD_A3VC }, + { "3dg-a", 0x100, 0, R8A774E1_PD_3DG_A, R8A774E1_PD_ALWAYS_ON }, + { "3dg-b", 0x100, 1, R8A774E1_PD_3DG_B, R8A774E1_PD_3DG_A }, + { "3dg-c", 0x100, 2, R8A774E1_PD_3DG_C, R8A774E1_PD_3DG_B }, + { "3dg-d", 0x100, 3, R8A774E1_PD_3DG_D, R8A774E1_PD_3DG_C }, + { "3dg-e", 0x100, 4, R8A774E1_PD_3DG_E, R8A774E1_PD_3DG_D }, +}; + +const struct rcar_sysc_info r8a774e1_sysc_info __initconst = { + .areas = r8a774e1_areas, + .num_areas = ARRAY_SIZE(r8a774e1_areas), + .extmask_offs = 0x2f8, + .extmask_val = BIT(0), +}; diff --git a/drivers/pmdomain/renesas/r8a7779-sysc.c b/drivers/pmdomain/renesas/r8a7779-sysc.c new file mode 100644 index 000000000000..e24a7151d55f --- /dev/null +++ b/drivers/pmdomain/renesas/r8a7779-sysc.c @@ -0,0 +1,30 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Renesas R-Car H1 System Controller + * + * Copyright (C) 2016 Glider bvba + */ + +#include + +#include + +#include "rcar-sysc.h" + +static const struct rcar_sysc_area r8a7779_areas[] __initconst = { + { "always-on", 0, 0, R8A7779_PD_ALWAYS_ON, -1, PD_ALWAYS_ON }, + { "arm1", 0x40, 1, R8A7779_PD_ARM1, R8A7779_PD_ALWAYS_ON, + PD_CPU_CR }, + { "arm2", 0x40, 2, R8A7779_PD_ARM2, R8A7779_PD_ALWAYS_ON, + PD_CPU_CR }, + { "arm3", 0x40, 3, R8A7779_PD_ARM3, R8A7779_PD_ALWAYS_ON, + PD_CPU_CR }, + { "sgx", 0xc0, 0, R8A7779_PD_SGX, R8A7779_PD_ALWAYS_ON }, + { "vdp", 0x100, 0, R8A7779_PD_VDP, R8A7779_PD_ALWAYS_ON }, + { "imp", 0x140, 0, R8A7779_PD_IMP, R8A7779_PD_ALWAYS_ON }, +}; + +const struct rcar_sysc_info r8a7779_sysc_info __initconst = { + .areas = r8a7779_areas, + .num_areas = ARRAY_SIZE(r8a7779_areas), +}; diff --git a/drivers/pmdomain/renesas/r8a7790-sysc.c b/drivers/pmdomain/renesas/r8a7790-sysc.c new file mode 100644 index 000000000000..b9afe7f6245b --- /dev/null +++ b/drivers/pmdomain/renesas/r8a7790-sysc.c @@ -0,0 +1,44 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Renesas R-Car H2 System Controller + * + * Copyright (C) 2016 Glider bvba + */ + +#include + +#include + +#include "rcar-sysc.h" + +static const struct rcar_sysc_area r8a7790_areas[] __initconst = { + { "always-on", 0, 0, R8A7790_PD_ALWAYS_ON, -1, PD_ALWAYS_ON }, + { "ca15-scu", 0x180, 0, R8A7790_PD_CA15_SCU, R8A7790_PD_ALWAYS_ON, + PD_SCU }, + { "ca15-cpu0", 0x40, 0, R8A7790_PD_CA15_CPU0, R8A7790_PD_CA15_SCU, + PD_CPU_NOCR }, + { "ca15-cpu1", 0x40, 1, R8A7790_PD_CA15_CPU1, R8A7790_PD_CA15_SCU, + PD_CPU_NOCR }, + { "ca15-cpu2", 0x40, 2, R8A7790_PD_CA15_CPU2, R8A7790_PD_CA15_SCU, + PD_CPU_NOCR }, + { "ca15-cpu3", 0x40, 3, R8A7790_PD_CA15_CPU3, R8A7790_PD_CA15_SCU, + PD_CPU_NOCR }, + { "ca7-scu", 0x100, 0, R8A7790_PD_CA7_SCU, R8A7790_PD_ALWAYS_ON, + PD_SCU }, + { "ca7-cpu0", 0x1c0, 0, R8A7790_PD_CA7_CPU0, R8A7790_PD_CA7_SCU, + PD_CPU_NOCR }, + { "ca7-cpu1", 0x1c0, 1, R8A7790_PD_CA7_CPU1, R8A7790_PD_CA7_SCU, + PD_CPU_NOCR }, + { "ca7-cpu2", 0x1c0, 2, R8A7790_PD_CA7_CPU2, R8A7790_PD_CA7_SCU, + PD_CPU_NOCR }, + { "ca7-cpu3", 0x1c0, 3, R8A7790_PD_CA7_CPU3, R8A7790_PD_CA7_SCU, + PD_CPU_NOCR }, + { "sh-4a", 0x80, 0, R8A7790_PD_SH_4A, R8A7790_PD_ALWAYS_ON }, + { "rgx", 0xc0, 0, R8A7790_PD_RGX, R8A7790_PD_ALWAYS_ON }, + { "imp", 0x140, 0, R8A7790_PD_IMP, R8A7790_PD_ALWAYS_ON }, +}; + +const struct rcar_sysc_info r8a7790_sysc_info __initconst = { + .areas = r8a7790_areas, + .num_areas = ARRAY_SIZE(r8a7790_areas), +}; diff --git a/drivers/pmdomain/renesas/r8a7791-sysc.c b/drivers/pmdomain/renesas/r8a7791-sysc.c new file mode 100644 index 000000000000..f00fa24522a3 --- /dev/null +++ b/drivers/pmdomain/renesas/r8a7791-sysc.c @@ -0,0 +1,29 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Renesas R-Car M2-W/N System Controller + * + * Copyright (C) 2016 Glider bvba + */ + +#include + +#include + +#include "rcar-sysc.h" + +static const struct rcar_sysc_area r8a7791_areas[] __initconst = { + { "always-on", 0, 0, R8A7791_PD_ALWAYS_ON, -1, PD_ALWAYS_ON }, + { "ca15-scu", 0x180, 0, R8A7791_PD_CA15_SCU, R8A7791_PD_ALWAYS_ON, + PD_SCU }, + { "ca15-cpu0", 0x40, 0, R8A7791_PD_CA15_CPU0, R8A7791_PD_CA15_SCU, + PD_CPU_NOCR }, + { "ca15-cpu1", 0x40, 1, R8A7791_PD_CA15_CPU1, R8A7791_PD_CA15_SCU, + PD_CPU_NOCR }, + { "sh-4a", 0x80, 0, R8A7791_PD_SH_4A, R8A7791_PD_ALWAYS_ON }, + { "sgx", 0xc0, 0, R8A7791_PD_SGX, R8A7791_PD_ALWAYS_ON }, +}; + +const struct rcar_sysc_info r8a7791_sysc_info __initconst = { + .areas = r8a7791_areas, + .num_areas = ARRAY_SIZE(r8a7791_areas), +}; diff --git a/drivers/pmdomain/renesas/r8a7792-sysc.c b/drivers/pmdomain/renesas/r8a7792-sysc.c new file mode 100644 index 000000000000..60aae242c43f --- /dev/null +++ b/drivers/pmdomain/renesas/r8a7792-sysc.c @@ -0,0 +1,30 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Renesas R-Car V2H (R8A7792) System Controller + * + * Copyright (C) 2016 Cogent Embedded Inc. + */ + +#include +#include + +#include + +#include "rcar-sysc.h" + +static const struct rcar_sysc_area r8a7792_areas[] __initconst = { + { "always-on", 0, 0, R8A7792_PD_ALWAYS_ON, -1, PD_ALWAYS_ON }, + { "ca15-scu", 0x180, 0, R8A7792_PD_CA15_SCU, R8A7792_PD_ALWAYS_ON, + PD_SCU }, + { "ca15-cpu0", 0x40, 0, R8A7792_PD_CA15_CPU0, R8A7792_PD_CA15_SCU, + PD_CPU_NOCR }, + { "ca15-cpu1", 0x40, 1, R8A7792_PD_CA15_CPU1, R8A7792_PD_CA15_SCU, + PD_CPU_NOCR }, + { "sgx", 0xc0, 0, R8A7792_PD_SGX, R8A7792_PD_ALWAYS_ON }, + { "imp", 0x140, 0, R8A7792_PD_IMP, R8A7792_PD_ALWAYS_ON }, +}; + +const struct rcar_sysc_info r8a7792_sysc_info __initconst = { + .areas = r8a7792_areas, + .num_areas = ARRAY_SIZE(r8a7792_areas), +}; diff --git a/drivers/pmdomain/renesas/r8a7794-sysc.c b/drivers/pmdomain/renesas/r8a7794-sysc.c new file mode 100644 index 000000000000..72ef4e85458f --- /dev/null +++ b/drivers/pmdomain/renesas/r8a7794-sysc.c @@ -0,0 +1,29 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Renesas R-Car E2 System Controller + * + * Copyright (C) 2016 Glider bvba + */ + +#include + +#include + +#include "rcar-sysc.h" + +static const struct rcar_sysc_area r8a7794_areas[] __initconst = { + { "always-on", 0, 0, R8A7794_PD_ALWAYS_ON, -1, PD_ALWAYS_ON }, + { "ca7-scu", 0x100, 0, R8A7794_PD_CA7_SCU, R8A7794_PD_ALWAYS_ON, + PD_SCU }, + { "ca7-cpu0", 0x1c0, 0, R8A7794_PD_CA7_CPU0, R8A7794_PD_CA7_SCU, + PD_CPU_NOCR }, + { "ca7-cpu1", 0x1c0, 1, R8A7794_PD_CA7_CPU1, R8A7794_PD_CA7_SCU, + PD_CPU_NOCR }, + { "sh-4a", 0x80, 0, R8A7794_PD_SH_4A, R8A7794_PD_ALWAYS_ON }, + { "sgx", 0xc0, 0, R8A7794_PD_SGX, R8A7794_PD_ALWAYS_ON }, +}; + +const struct rcar_sysc_info r8a7794_sysc_info __initconst = { + .areas = r8a7794_areas, + .num_areas = ARRAY_SIZE(r8a7794_areas), +}; diff --git a/drivers/pmdomain/renesas/r8a7795-sysc.c b/drivers/pmdomain/renesas/r8a7795-sysc.c new file mode 100644 index 000000000000..cbe1ff0fc583 --- /dev/null +++ b/drivers/pmdomain/renesas/r8a7795-sysc.c @@ -0,0 +1,86 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Renesas R-Car H3 System Controller + * + * Copyright (C) 2016-2017 Glider bvba + */ + +#include +#include +#include + +#include + +#include "rcar-sysc.h" + +static struct rcar_sysc_area r8a7795_areas[] __initdata = { + { "always-on", 0, 0, R8A7795_PD_ALWAYS_ON, -1, PD_ALWAYS_ON }, + { "ca57-scu", 0x1c0, 0, R8A7795_PD_CA57_SCU, R8A7795_PD_ALWAYS_ON, + PD_SCU }, + { "ca57-cpu0", 0x80, 0, R8A7795_PD_CA57_CPU0, R8A7795_PD_CA57_SCU, + PD_CPU_NOCR }, + { "ca57-cpu1", 0x80, 1, R8A7795_PD_CA57_CPU1, R8A7795_PD_CA57_SCU, + PD_CPU_NOCR }, + { "ca57-cpu2", 0x80, 2, R8A7795_PD_CA57_CPU2, R8A7795_PD_CA57_SCU, + PD_CPU_NOCR }, + { "ca57-cpu3", 0x80, 3, R8A7795_PD_CA57_CPU3, R8A7795_PD_CA57_SCU, + PD_CPU_NOCR }, + { "ca53-scu", 0x140, 0, R8A7795_PD_CA53_SCU, R8A7795_PD_ALWAYS_ON, + PD_SCU }, + { "ca53-cpu0", 0x200, 0, R8A7795_PD_CA53_CPU0, R8A7795_PD_CA53_SCU, + PD_CPU_NOCR }, + { "ca53-cpu1", 0x200, 1, R8A7795_PD_CA53_CPU1, R8A7795_PD_CA53_SCU, + PD_CPU_NOCR }, + { "ca53-cpu2", 0x200, 2, R8A7795_PD_CA53_CPU2, R8A7795_PD_CA53_SCU, + PD_CPU_NOCR }, + { "ca53-cpu3", 0x200, 3, R8A7795_PD_CA53_CPU3, R8A7795_PD_CA53_SCU, + PD_CPU_NOCR }, + { "a3vp", 0x340, 0, R8A7795_PD_A3VP, R8A7795_PD_ALWAYS_ON }, + { "cr7", 0x240, 0, R8A7795_PD_CR7, R8A7795_PD_ALWAYS_ON }, + { "a3vc", 0x380, 0, R8A7795_PD_A3VC, R8A7795_PD_ALWAYS_ON }, + { "a2vc1", 0x3c0, 1, R8A7795_PD_A2VC1, R8A7795_PD_A3VC }, + { "3dg-a", 0x100, 0, R8A7795_PD_3DG_A, R8A7795_PD_ALWAYS_ON }, + { "3dg-b", 0x100, 1, R8A7795_PD_3DG_B, R8A7795_PD_3DG_A }, + { "3dg-c", 0x100, 2, R8A7795_PD_3DG_C, R8A7795_PD_3DG_B }, + { "3dg-d", 0x100, 3, R8A7795_PD_3DG_D, R8A7795_PD_3DG_C }, + { "3dg-e", 0x100, 4, R8A7795_PD_3DG_E, R8A7795_PD_3DG_D }, + { "a3ir", 0x180, 0, R8A7795_PD_A3IR, R8A7795_PD_ALWAYS_ON }, +}; + + + /* + * Fixups for R-Car H3 revisions + */ + +#define NO_EXTMASK BIT(1) /* Missing SYSCEXTMASK register */ + +static const struct soc_device_attribute r8a7795_quirks_match[] __initconst = { + { + .soc_id = "r8a7795", .revision = "ES2.*", + .data = (void *)(NO_EXTMASK), + }, + { /* sentinel */ } +}; + +static int __init r8a7795_sysc_init(void) +{ + const struct soc_device_attribute *attr; + u32 quirks = 0; + + attr = soc_device_match(r8a7795_quirks_match); + if (attr) + quirks = (uintptr_t)attr->data; + + if (quirks & NO_EXTMASK) + r8a7795_sysc_info.extmask_val = 0; + + return 0; +} + +struct rcar_sysc_info r8a7795_sysc_info __initdata = { + .init = r8a7795_sysc_init, + .areas = r8a7795_areas, + .num_areas = ARRAY_SIZE(r8a7795_areas), + .extmask_offs = 0x2f8, + .extmask_val = BIT(0), +}; diff --git a/drivers/pmdomain/renesas/r8a7796-sysc.c b/drivers/pmdomain/renesas/r8a7796-sysc.c new file mode 100644 index 000000000000..471bd5b3b6ad --- /dev/null +++ b/drivers/pmdomain/renesas/r8a7796-sysc.c @@ -0,0 +1,67 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Renesas R-Car M3-W/W+ System Controller + * + * Copyright (C) 2016 Glider bvba + * Copyright (C) 2018-2019 Renesas Electronics Corporation + */ + +#include +#include + +#include + +#include "rcar-sysc.h" + +static struct rcar_sysc_area r8a7796_areas[] __initdata = { + { "always-on", 0, 0, R8A7796_PD_ALWAYS_ON, -1, PD_ALWAYS_ON }, + { "ca57-scu", 0x1c0, 0, R8A7796_PD_CA57_SCU, R8A7796_PD_ALWAYS_ON, + PD_SCU }, + { "ca57-cpu0", 0x80, 0, R8A7796_PD_CA57_CPU0, R8A7796_PD_CA57_SCU, + PD_CPU_NOCR }, + { "ca57-cpu1", 0x80, 1, R8A7796_PD_CA57_CPU1, R8A7796_PD_CA57_SCU, + PD_CPU_NOCR }, + { "ca53-scu", 0x140, 0, R8A7796_PD_CA53_SCU, R8A7796_PD_ALWAYS_ON, + PD_SCU }, + { "ca53-cpu0", 0x200, 0, R8A7796_PD_CA53_CPU0, R8A7796_PD_CA53_SCU, + PD_CPU_NOCR }, + { "ca53-cpu1", 0x200, 1, R8A7796_PD_CA53_CPU1, R8A7796_PD_CA53_SCU, + PD_CPU_NOCR }, + { "ca53-cpu2", 0x200, 2, R8A7796_PD_CA53_CPU2, R8A7796_PD_CA53_SCU, + PD_CPU_NOCR }, + { "ca53-cpu3", 0x200, 3, R8A7796_PD_CA53_CPU3, R8A7796_PD_CA53_SCU, + PD_CPU_NOCR }, + { "cr7", 0x240, 0, R8A7796_PD_CR7, R8A7796_PD_ALWAYS_ON }, + { "a3vc", 0x380, 0, R8A7796_PD_A3VC, R8A7796_PD_ALWAYS_ON }, + { "a2vc0", 0x3c0, 0, R8A7796_PD_A2VC0, R8A7796_PD_A3VC }, + { "a2vc1", 0x3c0, 1, R8A7796_PD_A2VC1, R8A7796_PD_A3VC }, + { "3dg-a", 0x100, 0, R8A7796_PD_3DG_A, R8A7796_PD_ALWAYS_ON }, + { "3dg-b", 0x100, 1, R8A7796_PD_3DG_B, R8A7796_PD_3DG_A }, + { "a3ir", 0x180, 0, R8A7796_PD_A3IR, R8A7796_PD_ALWAYS_ON }, +}; + + +#ifdef CONFIG_SYSC_R8A77960 +const struct rcar_sysc_info r8a77960_sysc_info __initconst = { + .areas = r8a7796_areas, + .num_areas = ARRAY_SIZE(r8a7796_areas), +}; +#endif /* CONFIG_SYSC_R8A77960 */ + +#ifdef CONFIG_SYSC_R8A77961 +static int __init r8a77961_sysc_init(void) +{ + rcar_sysc_nullify(r8a7796_areas, ARRAY_SIZE(r8a7796_areas), + R8A7796_PD_A2VC0); + + return 0; +} + +const struct rcar_sysc_info r8a77961_sysc_info __initconst = { + .init = r8a77961_sysc_init, + .areas = r8a7796_areas, + .num_areas = ARRAY_SIZE(r8a7796_areas), + .extmask_offs = 0x2f8, + .extmask_val = BIT(0), +}; +#endif /* CONFIG_SYSC_R8A77961 */ diff --git a/drivers/pmdomain/renesas/r8a77965-sysc.c b/drivers/pmdomain/renesas/r8a77965-sysc.c new file mode 100644 index 000000000000..ff0b0d116992 --- /dev/null +++ b/drivers/pmdomain/renesas/r8a77965-sysc.c @@ -0,0 +1,38 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Renesas R-Car M3-N System Controller + * Copyright (C) 2018 Jacopo Mondi + * + * Based on Renesas R-Car M3-W System Controller + * Copyright (C) 2016 Glider bvba + */ + +#include +#include + +#include + +#include "rcar-sysc.h" + +static const struct rcar_sysc_area r8a77965_areas[] __initconst = { + { "always-on", 0, 0, R8A77965_PD_ALWAYS_ON, -1, PD_ALWAYS_ON }, + { "ca57-scu", 0x1c0, 0, R8A77965_PD_CA57_SCU, R8A77965_PD_ALWAYS_ON, + PD_SCU }, + { "ca57-cpu0", 0x80, 0, R8A77965_PD_CA57_CPU0, R8A77965_PD_CA57_SCU, + PD_CPU_NOCR }, + { "ca57-cpu1", 0x80, 1, R8A77965_PD_CA57_CPU1, R8A77965_PD_CA57_SCU, + PD_CPU_NOCR }, + { "cr7", 0x240, 0, R8A77965_PD_CR7, R8A77965_PD_ALWAYS_ON }, + { "a3vc", 0x380, 0, R8A77965_PD_A3VC, R8A77965_PD_ALWAYS_ON }, + { "a3vp", 0x340, 0, R8A77965_PD_A3VP, R8A77965_PD_ALWAYS_ON }, + { "a2vc1", 0x3c0, 1, R8A77965_PD_A2VC1, R8A77965_PD_A3VC }, + { "3dg-a", 0x100, 0, R8A77965_PD_3DG_A, R8A77965_PD_ALWAYS_ON }, + { "3dg-b", 0x100, 1, R8A77965_PD_3DG_B, R8A77965_PD_3DG_A }, +}; + +const struct rcar_sysc_info r8a77965_sysc_info __initconst = { + .areas = r8a77965_areas, + .num_areas = ARRAY_SIZE(r8a77965_areas), + .extmask_offs = 0x2f8, + .extmask_val = BIT(0), +}; diff --git a/drivers/pmdomain/renesas/r8a77970-sysc.c b/drivers/pmdomain/renesas/r8a77970-sysc.c new file mode 100644 index 000000000000..706258250600 --- /dev/null +++ b/drivers/pmdomain/renesas/r8a77970-sysc.c @@ -0,0 +1,37 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Renesas R-Car V3M System Controller + * + * Copyright (C) 2017 Cogent Embedded Inc. + */ + +#include +#include + +#include + +#include "rcar-sysc.h" + +static const struct rcar_sysc_area r8a77970_areas[] __initconst = { + { "always-on", 0, 0, R8A77970_PD_ALWAYS_ON, -1, PD_ALWAYS_ON }, + { "ca53-scu", 0x140, 0, R8A77970_PD_CA53_SCU, R8A77970_PD_ALWAYS_ON, + PD_SCU }, + { "ca53-cpu0", 0x200, 0, R8A77970_PD_CA53_CPU0, R8A77970_PD_CA53_SCU, + PD_CPU_NOCR }, + { "ca53-cpu1", 0x200, 1, R8A77970_PD_CA53_CPU1, R8A77970_PD_CA53_SCU, + PD_CPU_NOCR }, + { "a3ir", 0x180, 0, R8A77970_PD_A3IR, R8A77970_PD_ALWAYS_ON }, + { "a2ir0", 0x400, 0, R8A77970_PD_A2IR0, R8A77970_PD_A3IR }, + { "a2ir1", 0x400, 1, R8A77970_PD_A2IR1, R8A77970_PD_A3IR }, + { "a2dp", 0x400, 2, R8A77970_PD_A2DP, R8A77970_PD_A3IR }, + { "a2cn", 0x400, 3, R8A77970_PD_A2CN, R8A77970_PD_A3IR }, + { "a2sc0", 0x400, 4, R8A77970_PD_A2SC0, R8A77970_PD_A3IR }, + { "a2sc1", 0x400, 5, R8A77970_PD_A2SC1, R8A77970_PD_A3IR }, +}; + +const struct rcar_sysc_info r8a77970_sysc_info __initconst = { + .areas = r8a77970_areas, + .num_areas = ARRAY_SIZE(r8a77970_areas), + .extmask_offs = 0x1b0, + .extmask_val = BIT(0), +}; diff --git a/drivers/pmdomain/renesas/r8a77980-sysc.c b/drivers/pmdomain/renesas/r8a77980-sysc.c new file mode 100644 index 000000000000..39ca84a67daa --- /dev/null +++ b/drivers/pmdomain/renesas/r8a77980-sysc.c @@ -0,0 +1,54 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Renesas R-Car V3H System Controller + * + * Copyright (C) 2018 Renesas Electronics Corp. + * Copyright (C) 2018 Cogent Embedded, Inc. + */ + +#include +#include + +#include + +#include "rcar-sysc.h" + +static const struct rcar_sysc_area r8a77980_areas[] __initconst = { + { "always-on", 0, 0, R8A77980_PD_ALWAYS_ON, -1, PD_ALWAYS_ON }, + { "ca53-scu", 0x140, 0, R8A77980_PD_CA53_SCU, R8A77980_PD_ALWAYS_ON, + PD_SCU }, + { "ca53-cpu0", 0x200, 0, R8A77980_PD_CA53_CPU0, R8A77980_PD_CA53_SCU, + PD_CPU_NOCR }, + { "ca53-cpu1", 0x200, 1, R8A77980_PD_CA53_CPU1, R8A77980_PD_CA53_SCU, + PD_CPU_NOCR }, + { "ca53-cpu2", 0x200, 2, R8A77980_PD_CA53_CPU2, R8A77980_PD_CA53_SCU, + PD_CPU_NOCR }, + { "ca53-cpu3", 0x200, 3, R8A77980_PD_CA53_CPU3, R8A77980_PD_CA53_SCU, + PD_CPU_NOCR }, + { "cr7", 0x240, 0, R8A77980_PD_CR7, R8A77980_PD_ALWAYS_ON }, + { "a3ir", 0x180, 0, R8A77980_PD_A3IR, R8A77980_PD_ALWAYS_ON }, + { "a2ir0", 0x400, 0, R8A77980_PD_A2IR0, R8A77980_PD_A3IR }, + { "a2ir1", 0x400, 1, R8A77980_PD_A2IR1, R8A77980_PD_A3IR }, + { "a2ir2", 0x400, 2, R8A77980_PD_A2IR2, R8A77980_PD_A3IR }, + { "a2ir3", 0x400, 3, R8A77980_PD_A2IR3, R8A77980_PD_A3IR }, + { "a2ir4", 0x400, 4, R8A77980_PD_A2IR4, R8A77980_PD_A3IR }, + { "a2ir5", 0x400, 5, R8A77980_PD_A2IR5, R8A77980_PD_A3IR }, + { "a2sc0", 0x400, 6, R8A77980_PD_A2SC0, R8A77980_PD_A3IR }, + { "a2sc1", 0x400, 7, R8A77980_PD_A2SC1, R8A77980_PD_A3IR }, + { "a2sc2", 0x400, 8, R8A77980_PD_A2SC2, R8A77980_PD_A3IR }, + { "a2sc3", 0x400, 9, R8A77980_PD_A2SC3, R8A77980_PD_A3IR }, + { "a2sc4", 0x400, 10, R8A77980_PD_A2SC4, R8A77980_PD_A3IR }, + { "a2dp0", 0x400, 11, R8A77980_PD_A2DP0, R8A77980_PD_A3IR }, + { "a2dp1", 0x400, 12, R8A77980_PD_A2DP1, R8A77980_PD_A3IR }, + { "a2cn", 0x400, 13, R8A77980_PD_A2CN, R8A77980_PD_A3IR }, + { "a3vip0", 0x2c0, 0, R8A77980_PD_A3VIP0, R8A77980_PD_ALWAYS_ON }, + { "a3vip1", 0x300, 0, R8A77980_PD_A3VIP1, R8A77980_PD_ALWAYS_ON }, + { "a3vip2", 0x280, 0, R8A77980_PD_A3VIP2, R8A77980_PD_ALWAYS_ON }, +}; + +const struct rcar_sysc_info r8a77980_sysc_info __initconst = { + .areas = r8a77980_areas, + .num_areas = ARRAY_SIZE(r8a77980_areas), + .extmask_offs = 0x138, + .extmask_val = BIT(0), +}; diff --git a/drivers/pmdomain/renesas/r8a77990-sysc.c b/drivers/pmdomain/renesas/r8a77990-sysc.c new file mode 100644 index 000000000000..9f92737dc352 --- /dev/null +++ b/drivers/pmdomain/renesas/r8a77990-sysc.c @@ -0,0 +1,55 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Renesas R-Car E3 System Controller + * + * Copyright (C) 2018 Renesas Electronics Corp. + */ + +#include +#include +#include + +#include + +#include "rcar-sysc.h" + +static struct rcar_sysc_area r8a77990_areas[] __initdata = { + { "always-on", 0, 0, R8A77990_PD_ALWAYS_ON, -1, PD_ALWAYS_ON }, + { "ca53-scu", 0x140, 0, R8A77990_PD_CA53_SCU, R8A77990_PD_ALWAYS_ON, + PD_SCU }, + { "ca53-cpu0", 0x200, 0, R8A77990_PD_CA53_CPU0, R8A77990_PD_CA53_SCU, + PD_CPU_NOCR }, + { "ca53-cpu1", 0x200, 1, R8A77990_PD_CA53_CPU1, R8A77990_PD_CA53_SCU, + PD_CPU_NOCR }, + { "cr7", 0x240, 0, R8A77990_PD_CR7, R8A77990_PD_ALWAYS_ON }, + { "a3vc", 0x380, 0, R8A77990_PD_A3VC, R8A77990_PD_ALWAYS_ON }, + { "a2vc1", 0x3c0, 1, R8A77990_PD_A2VC1, R8A77990_PD_A3VC }, + { "3dg-a", 0x100, 0, R8A77990_PD_3DG_A, R8A77990_PD_ALWAYS_ON }, + { "3dg-b", 0x100, 1, R8A77990_PD_3DG_B, R8A77990_PD_3DG_A }, +}; + +/* Fixups for R-Car E3 ES1.0 revision */ +static const struct soc_device_attribute r8a77990[] __initconst = { + { .soc_id = "r8a77990", .revision = "ES1.0" }, + { /* sentinel */ } +}; + +static int __init r8a77990_sysc_init(void) +{ + if (soc_device_match(r8a77990)) { + /* Fix incorrect 3DG hierarchy */ + swap(r8a77990_areas[7], r8a77990_areas[8]); + r8a77990_areas[7].parent = R8A77990_PD_ALWAYS_ON; + r8a77990_areas[8].parent = R8A77990_PD_3DG_B; + } + + return 0; +} + +const struct rcar_sysc_info r8a77990_sysc_info __initconst = { + .init = r8a77990_sysc_init, + .areas = r8a77990_areas, + .num_areas = ARRAY_SIZE(r8a77990_areas), + .extmask_offs = 0x2f8, + .extmask_val = BIT(0), +}; diff --git a/drivers/pmdomain/renesas/r8a77995-sysc.c b/drivers/pmdomain/renesas/r8a77995-sysc.c new file mode 100644 index 000000000000..efcc67e3d76d --- /dev/null +++ b/drivers/pmdomain/renesas/r8a77995-sysc.c @@ -0,0 +1,26 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Renesas R-Car D3 System Controller + * + * Copyright (C) 2017 Glider bvba + */ + +#include + +#include + +#include "rcar-sysc.h" + +static const struct rcar_sysc_area r8a77995_areas[] __initconst = { + { "always-on", 0, 0, R8A77995_PD_ALWAYS_ON, -1, PD_ALWAYS_ON }, + { "ca53-scu", 0x140, 0, R8A77995_PD_CA53_SCU, R8A77995_PD_ALWAYS_ON, + PD_SCU }, + { "ca53-cpu0", 0x200, 0, R8A77995_PD_CA53_CPU0, R8A77995_PD_CA53_SCU, + PD_CPU_NOCR }, +}; + + +const struct rcar_sysc_info r8a77995_sysc_info __initconst = { + .areas = r8a77995_areas, + .num_areas = ARRAY_SIZE(r8a77995_areas), +}; diff --git a/drivers/pmdomain/renesas/r8a779a0-sysc.c b/drivers/pmdomain/renesas/r8a779a0-sysc.c new file mode 100644 index 000000000000..04f1bc322ae7 --- /dev/null +++ b/drivers/pmdomain/renesas/r8a779a0-sysc.c @@ -0,0 +1,76 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Renesas R-Car V3U System Controller + * + * Copyright (C) 2020 Renesas Electronics Corp. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +#include "rcar-gen4-sysc.h" + +static struct rcar_gen4_sysc_area r8a779a0_areas[] __initdata = { + { "always-on", R8A779A0_PD_ALWAYS_ON, -1, PD_ALWAYS_ON }, + { "a3e0", R8A779A0_PD_A3E0, R8A779A0_PD_ALWAYS_ON, PD_SCU }, + { "a3e1", R8A779A0_PD_A3E1, R8A779A0_PD_ALWAYS_ON, PD_SCU }, + { "a2e0d0", R8A779A0_PD_A2E0D0, R8A779A0_PD_A3E0, PD_SCU }, + { "a2e0d1", R8A779A0_PD_A2E0D1, R8A779A0_PD_A3E0, PD_SCU }, + { "a2e1d0", R8A779A0_PD_A2E1D0, R8A779A0_PD_A3E1, PD_SCU }, + { "a2e1d1", R8A779A0_PD_A2E1D1, R8A779A0_PD_A3E1, PD_SCU }, + { "a1e0d0c0", R8A779A0_PD_A1E0D0C0, R8A779A0_PD_A2E0D0, PD_CPU_NOCR }, + { "a1e0d0c1", R8A779A0_PD_A1E0D0C1, R8A779A0_PD_A2E0D0, PD_CPU_NOCR }, + { "a1e0d1c0", R8A779A0_PD_A1E0D1C0, R8A779A0_PD_A2E0D1, PD_CPU_NOCR }, + { "a1e0d1c1", R8A779A0_PD_A1E0D1C1, R8A779A0_PD_A2E0D1, PD_CPU_NOCR }, + { "a1e1d0c0", R8A779A0_PD_A1E1D0C0, R8A779A0_PD_A2E1D0, PD_CPU_NOCR }, + { "a1e1d0c1", R8A779A0_PD_A1E1D0C1, R8A779A0_PD_A2E1D0, PD_CPU_NOCR }, + { "a1e1d1c0", R8A779A0_PD_A1E1D1C0, R8A779A0_PD_A2E1D1, PD_CPU_NOCR }, + { "a1e1d1c1", R8A779A0_PD_A1E1D1C1, R8A779A0_PD_A2E1D1, PD_CPU_NOCR }, + { "3dg-a", R8A779A0_PD_3DG_A, R8A779A0_PD_ALWAYS_ON }, + { "3dg-b", R8A779A0_PD_3DG_B, R8A779A0_PD_3DG_A }, + { "a3vip0", R8A779A0_PD_A3VIP0, R8A779A0_PD_ALWAYS_ON }, + { "a3vip1", R8A779A0_PD_A3VIP1, R8A779A0_PD_ALWAYS_ON }, + { "a3vip3", R8A779A0_PD_A3VIP3, R8A779A0_PD_ALWAYS_ON }, + { "a3vip2", R8A779A0_PD_A3VIP2, R8A779A0_PD_ALWAYS_ON }, + { "a3isp01", R8A779A0_PD_A3ISP01, R8A779A0_PD_ALWAYS_ON }, + { "a3isp23", R8A779A0_PD_A3ISP23, R8A779A0_PD_ALWAYS_ON }, + { "a3ir", R8A779A0_PD_A3IR, R8A779A0_PD_ALWAYS_ON }, + { "a2cn0", R8A779A0_PD_A2CN0, R8A779A0_PD_A3IR }, + { "a2imp01", R8A779A0_PD_A2IMP01, R8A779A0_PD_A3IR }, + { "a2dp0", R8A779A0_PD_A2DP0, R8A779A0_PD_A3IR }, + { "a2cv0", R8A779A0_PD_A2CV0, R8A779A0_PD_A3IR }, + { "a2cv1", R8A779A0_PD_A2CV1, R8A779A0_PD_A3IR }, + { "a2cv4", R8A779A0_PD_A2CV4, R8A779A0_PD_A3IR }, + { "a2cv6", R8A779A0_PD_A2CV6, R8A779A0_PD_A3IR }, + { "a2cn2", R8A779A0_PD_A2CN2, R8A779A0_PD_A3IR }, + { "a2imp23", R8A779A0_PD_A2IMP23, R8A779A0_PD_A3IR }, + { "a2dp1", R8A779A0_PD_A2DP1, R8A779A0_PD_A3IR }, + { "a2cv2", R8A779A0_PD_A2CV2, R8A779A0_PD_A3IR }, + { "a2cv3", R8A779A0_PD_A2CV3, R8A779A0_PD_A3IR }, + { "a2cv5", R8A779A0_PD_A2CV5, R8A779A0_PD_A3IR }, + { "a2cv7", R8A779A0_PD_A2CV7, R8A779A0_PD_A3IR }, + { "a2cn1", R8A779A0_PD_A2CN1, R8A779A0_PD_A3IR }, + { "a1cnn0", R8A779A0_PD_A1CNN0, R8A779A0_PD_A2CN0 }, + { "a1cnn2", R8A779A0_PD_A1CNN2, R8A779A0_PD_A2CN2 }, + { "a1dsp0", R8A779A0_PD_A1DSP0, R8A779A0_PD_A2CN2 }, + { "a1cnn1", R8A779A0_PD_A1CNN1, R8A779A0_PD_A2CN1 }, + { "a1dsp1", R8A779A0_PD_A1DSP1, R8A779A0_PD_A2CN1 }, +}; + +const struct rcar_gen4_sysc_info r8a779a0_sysc_info __initconst = { + .areas = r8a779a0_areas, + .num_areas = ARRAY_SIZE(r8a779a0_areas), +}; diff --git a/drivers/pmdomain/renesas/r8a779f0-sysc.c b/drivers/pmdomain/renesas/r8a779f0-sysc.c new file mode 100644 index 000000000000..5602aa6bd7ed --- /dev/null +++ b/drivers/pmdomain/renesas/r8a779f0-sysc.c @@ -0,0 +1,47 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Renesas R-Car S4-8 System Controller + * + * Copyright (C) 2021 Renesas Electronics Corp. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +#include "rcar-gen4-sysc.h" + +static struct rcar_gen4_sysc_area r8a779f0_areas[] __initdata = { + { "always-on", R8A779F0_PD_ALWAYS_ON, -1, PD_ALWAYS_ON }, + { "a3e0", R8A779F0_PD_A3E0, R8A779F0_PD_ALWAYS_ON, PD_SCU }, + { "a3e1", R8A779F0_PD_A3E1, R8A779F0_PD_ALWAYS_ON, PD_SCU }, + { "a2e0d0", R8A779F0_PD_A2E0D0, R8A779F0_PD_A3E0, PD_SCU }, + { "a2e0d1", R8A779F0_PD_A2E0D1, R8A779F0_PD_A3E0, PD_SCU }, + { "a2e1d0", R8A779F0_PD_A2E1D0, R8A779F0_PD_A3E1, PD_SCU }, + { "a2e1d1", R8A779F0_PD_A2E1D1, R8A779F0_PD_A3E1, PD_SCU }, + { "a1e0d0c0", R8A779F0_PD_A1E0D0C0, R8A779F0_PD_A2E0D0, PD_CPU_NOCR }, + { "a1e0d0c1", R8A779F0_PD_A1E0D0C1, R8A779F0_PD_A2E0D0, PD_CPU_NOCR }, + { "a1e0d1c0", R8A779F0_PD_A1E0D1C0, R8A779F0_PD_A2E0D1, PD_CPU_NOCR }, + { "a1e0d1c1", R8A779F0_PD_A1E0D1C1, R8A779F0_PD_A2E0D1, PD_CPU_NOCR }, + { "a1e1d0c0", R8A779F0_PD_A1E1D0C0, R8A779F0_PD_A2E1D0, PD_CPU_NOCR }, + { "a1e1d0c1", R8A779F0_PD_A1E1D0C1, R8A779F0_PD_A2E1D0, PD_CPU_NOCR }, + { "a1e1d1c0", R8A779F0_PD_A1E1D1C0, R8A779F0_PD_A2E1D1, PD_CPU_NOCR }, + { "a1e1d1c1", R8A779F0_PD_A1E1D1C1, R8A779F0_PD_A2E1D1, PD_CPU_NOCR }, +}; + +const struct rcar_gen4_sysc_info r8a779f0_sysc_info __initconst = { + .areas = r8a779f0_areas, + .num_areas = ARRAY_SIZE(r8a779f0_areas), +}; diff --git a/drivers/pmdomain/renesas/r8a779g0-sysc.c b/drivers/pmdomain/renesas/r8a779g0-sysc.c new file mode 100644 index 000000000000..b932eba1b804 --- /dev/null +++ b/drivers/pmdomain/renesas/r8a779g0-sysc.c @@ -0,0 +1,63 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Renesas R-Car V4H System Controller + * + * Copyright (C) 2022 Renesas Electronics Corp. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +#include "rcar-gen4-sysc.h" + +static struct rcar_gen4_sysc_area r8a779g0_areas[] __initdata = { + { "always-on", R8A779G0_PD_ALWAYS_ON, -1, PD_ALWAYS_ON }, + { "a3e0", R8A779G0_PD_A3E0, R8A779G0_PD_ALWAYS_ON, PD_SCU }, + { "a2e0d0", R8A779G0_PD_A2E0D0, R8A779G0_PD_A3E0, PD_SCU }, + { "a2e0d1", R8A779G0_PD_A2E0D1, R8A779G0_PD_A3E0, PD_SCU }, + { "a1e0d0c0", R8A779G0_PD_A1E0D0C0, R8A779G0_PD_A2E0D0, PD_CPU_NOCR }, + { "a1e0d0c1", R8A779G0_PD_A1E0D0C1, R8A779G0_PD_A2E0D0, PD_CPU_NOCR }, + { "a1e0d1c0", R8A779G0_PD_A1E0D1C0, R8A779G0_PD_A2E0D1, PD_CPU_NOCR }, + { "a1e0d1c1", R8A779G0_PD_A1E0D1C1, R8A779G0_PD_A2E0D1, PD_CPU_NOCR }, + { "a33dga", R8A779G0_PD_A33DGA, R8A779G0_PD_ALWAYS_ON }, + { "a23dgb", R8A779G0_PD_A23DGB, R8A779G0_PD_A33DGA }, + { "a3vip0", R8A779G0_PD_A3VIP0, R8A779G0_PD_ALWAYS_ON }, + { "a3vip1", R8A779G0_PD_A3VIP1, R8A779G0_PD_ALWAYS_ON }, + { "a3vip2", R8A779G0_PD_A3VIP2, R8A779G0_PD_ALWAYS_ON }, + { "a3dul", R8A779G0_PD_A3DUL, R8A779G0_PD_ALWAYS_ON }, + { "a3isp0", R8A779G0_PD_A3ISP0, R8A779G0_PD_ALWAYS_ON }, + { "a3isp1", R8A779G0_PD_A3ISP1, R8A779G0_PD_ALWAYS_ON }, + { "a3ir", R8A779G0_PD_A3IR, R8A779G0_PD_ALWAYS_ON }, + { "a2cn0", R8A779G0_PD_A2CN0, R8A779G0_PD_A3IR }, + { "a1cnn0", R8A779G0_PD_A1CNN0, R8A779G0_PD_A2CN0 }, + { "a1dsp0", R8A779G0_PD_A1DSP0, R8A779G0_PD_A2CN0 }, + { "a1dsp1", R8A779G0_PD_A1DSP1, R8A779G0_PD_A2CN0 }, + { "a1dsp2", R8A779G0_PD_A1DSP2, R8A779G0_PD_A2CN0 }, + { "a1dsp3", R8A779G0_PD_A1DSP3, R8A779G0_PD_A2CN0 }, + { "a2imp01", R8A779G0_PD_A2IMP01, R8A779G0_PD_A3IR }, + { "a2imp23", R8A779G0_PD_A2IMP23, R8A779G0_PD_A3IR }, + { "a2psc", R8A779G0_PD_A2PSC, R8A779G0_PD_A3IR }, + { "a2dma", R8A779G0_PD_A2DMA, R8A779G0_PD_A3IR }, + { "a2cv0", R8A779G0_PD_A2CV0, R8A779G0_PD_A3IR }, + { "a2cv1", R8A779G0_PD_A2CV1, R8A779G0_PD_A3IR }, + { "a2cv2", R8A779G0_PD_A2CV2, R8A779G0_PD_A3IR }, + { "a2cv3", R8A779G0_PD_A2CV3, R8A779G0_PD_A3IR }, +}; + +const struct rcar_gen4_sysc_info r8a779g0_sysc_info __initconst = { + .areas = r8a779g0_areas, + .num_areas = ARRAY_SIZE(r8a779g0_areas), +}; diff --git a/drivers/pmdomain/renesas/rcar-gen4-sysc.c b/drivers/pmdomain/renesas/rcar-gen4-sysc.c new file mode 100644 index 000000000000..9e5e6e077abc --- /dev/null +++ b/drivers/pmdomain/renesas/rcar-gen4-sysc.c @@ -0,0 +1,379 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * R-Car Gen4 SYSC Power management support + * + * Copyright (C) 2021 Renesas Electronics Corp. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "rcar-gen4-sysc.h" + +/* SYSC Common */ +#define SYSCSR 0x000 /* SYSC Status Register */ +#define SYSCPONSR(x) (0x800 + ((x) * 0x4)) /* Power-ON Status Register 0 */ +#define SYSCPOFFSR(x) (0x808 + ((x) * 0x4)) /* Power-OFF Status Register */ +#define SYSCISCR(x) (0x810 + ((x) * 0x4)) /* Interrupt Status/Clear Register */ +#define SYSCIER(x) (0x820 + ((x) * 0x4)) /* Interrupt Enable Register */ +#define SYSCIMR(x) (0x830 + ((x) * 0x4)) /* Interrupt Mask Register */ + +/* Power Domain Registers */ +#define PDRSR(n) (0x1000 + ((n) * 0x40)) +#define PDRONCR(n) (0x1004 + ((n) * 0x40)) +#define PDROFFCR(n) (0x1008 + ((n) * 0x40)) +#define PDRESR(n) (0x100C + ((n) * 0x40)) + +/* PWRON/PWROFF */ +#define PWRON_PWROFF BIT(0) /* Power-ON/OFF request */ + +/* PDRESR */ +#define PDRESR_ERR BIT(0) + +/* PDRSR */ +#define PDRSR_OFF BIT(0) /* Power-OFF state */ +#define PDRSR_ON BIT(4) /* Power-ON state */ +#define PDRSR_OFF_STATE BIT(8) /* Processing Power-OFF sequence */ +#define PDRSR_ON_STATE BIT(12) /* Processing Power-ON sequence */ + +#define SYSCSR_BUSY GENMASK(1, 0) /* All bit sets is not busy */ + +#define SYSCSR_TIMEOUT 10000 +#define SYSCSR_DELAY_US 10 + +#define PDRESR_RETRIES 1000 +#define PDRESR_DELAY_US 10 + +#define SYSCISR_TIMEOUT 10000 +#define SYSCISR_DELAY_US 10 + +#define RCAR_GEN4_PD_ALWAYS_ON 64 +#define NUM_DOMAINS_EACH_REG BITS_PER_TYPE(u32) + +static void __iomem *rcar_gen4_sysc_base; +static DEFINE_SPINLOCK(rcar_gen4_sysc_lock); /* SMP CPUs + I/O devices */ + +static int rcar_gen4_sysc_pwr_on_off(u8 pdr, bool on) +{ + unsigned int reg_offs; + u32 val; + int ret; + + if (on) + reg_offs = PDRONCR(pdr); + else + reg_offs = PDROFFCR(pdr); + + /* Wait until SYSC is ready to accept a power request */ + ret = readl_poll_timeout_atomic(rcar_gen4_sysc_base + SYSCSR, val, + (val & SYSCSR_BUSY) == SYSCSR_BUSY, + SYSCSR_DELAY_US, SYSCSR_TIMEOUT); + if (ret < 0) + return -EAGAIN; + + /* Submit power shutoff or power resume request */ + iowrite32(PWRON_PWROFF, rcar_gen4_sysc_base + reg_offs); + + return 0; +} + +static int clear_irq_flags(unsigned int reg_idx, unsigned int isr_mask) +{ + u32 val; + int ret; + + iowrite32(isr_mask, rcar_gen4_sysc_base + SYSCISCR(reg_idx)); + + ret = readl_poll_timeout_atomic(rcar_gen4_sysc_base + SYSCISCR(reg_idx), + val, !(val & isr_mask), + SYSCISR_DELAY_US, SYSCISR_TIMEOUT); + if (ret < 0) { + pr_err("\n %s : Can not clear IRQ flags in SYSCISCR", __func__); + return -EIO; + } + + return 0; +} + +static int rcar_gen4_sysc_power(u8 pdr, bool on) +{ + unsigned int isr_mask; + unsigned int reg_idx, bit_idx; + unsigned int status; + unsigned long flags; + int ret = 0; + u32 val; + int k; + + spin_lock_irqsave(&rcar_gen4_sysc_lock, flags); + + reg_idx = pdr / NUM_DOMAINS_EACH_REG; + bit_idx = pdr % NUM_DOMAINS_EACH_REG; + + isr_mask = BIT(bit_idx); + + /* + * The interrupt source needs to be enabled, but masked, to prevent the + * CPU from receiving it. + */ + iowrite32(ioread32(rcar_gen4_sysc_base + SYSCIER(reg_idx)) | isr_mask, + rcar_gen4_sysc_base + SYSCIER(reg_idx)); + iowrite32(ioread32(rcar_gen4_sysc_base + SYSCIMR(reg_idx)) | isr_mask, + rcar_gen4_sysc_base + SYSCIMR(reg_idx)); + + ret = clear_irq_flags(reg_idx, isr_mask); + if (ret) + goto out; + + /* Submit power shutoff or resume request until it was accepted */ + for (k = 0; k < PDRESR_RETRIES; k++) { + ret = rcar_gen4_sysc_pwr_on_off(pdr, on); + if (ret) + goto out; + + status = ioread32(rcar_gen4_sysc_base + PDRESR(pdr)); + if (!(status & PDRESR_ERR)) + break; + + udelay(PDRESR_DELAY_US); + } + + if (k == PDRESR_RETRIES) { + ret = -EIO; + goto out; + } + + /* Wait until the power shutoff or resume request has completed * */ + ret = readl_poll_timeout_atomic(rcar_gen4_sysc_base + SYSCISCR(reg_idx), + val, (val & isr_mask), + SYSCISR_DELAY_US, SYSCISR_TIMEOUT); + if (ret < 0) { + ret = -EIO; + goto out; + } + + /* Clear interrupt flags */ + ret = clear_irq_flags(reg_idx, isr_mask); + if (ret) + goto out; + + out: + spin_unlock_irqrestore(&rcar_gen4_sysc_lock, flags); + + pr_debug("sysc power %s domain %d: %08x -> %d\n", on ? "on" : "off", + pdr, ioread32(rcar_gen4_sysc_base + SYSCISCR(reg_idx)), ret); + return ret; +} + +static bool rcar_gen4_sysc_power_is_off(u8 pdr) +{ + unsigned int st; + + st = ioread32(rcar_gen4_sysc_base + PDRSR(pdr)); + + if (st & PDRSR_OFF) + return true; + + return false; +} + +struct rcar_gen4_sysc_pd { + struct generic_pm_domain genpd; + u8 pdr; + unsigned int flags; + char name[]; +}; + +static inline struct rcar_gen4_sysc_pd *to_rcar_gen4_pd(struct generic_pm_domain *d) +{ + return container_of(d, struct rcar_gen4_sysc_pd, genpd); +} + +static int rcar_gen4_sysc_pd_power_off(struct generic_pm_domain *genpd) +{ + struct rcar_gen4_sysc_pd *pd = to_rcar_gen4_pd(genpd); + + pr_debug("%s: %s\n", __func__, genpd->name); + return rcar_gen4_sysc_power(pd->pdr, false); +} + +static int rcar_gen4_sysc_pd_power_on(struct generic_pm_domain *genpd) +{ + struct rcar_gen4_sysc_pd *pd = to_rcar_gen4_pd(genpd); + + pr_debug("%s: %s\n", __func__, genpd->name); + return rcar_gen4_sysc_power(pd->pdr, true); +} + +static int __init rcar_gen4_sysc_pd_setup(struct rcar_gen4_sysc_pd *pd) +{ + struct generic_pm_domain *genpd = &pd->genpd; + const char *name = pd->genpd.name; + int error; + + if (pd->flags & PD_CPU) { + /* + * This domain contains a CPU core and therefore it should + * only be turned off if the CPU is not in use. + */ + pr_debug("PM domain %s contains %s\n", name, "CPU"); + genpd->flags |= GENPD_FLAG_ALWAYS_ON; + } else if (pd->flags & PD_SCU) { + /* + * This domain contains an SCU and cache-controller, and + * therefore it should only be turned off if the CPU cores are + * not in use. + */ + pr_debug("PM domain %s contains %s\n", name, "SCU"); + genpd->flags |= GENPD_FLAG_ALWAYS_ON; + } else if (pd->flags & PD_NO_CR) { + /* + * This domain cannot be turned off. + */ + genpd->flags |= GENPD_FLAG_ALWAYS_ON; + } + + if (!(pd->flags & (PD_CPU | PD_SCU))) { + /* Enable Clock Domain for I/O devices */ + genpd->flags |= GENPD_FLAG_PM_CLK | GENPD_FLAG_ACTIVE_WAKEUP; + genpd->attach_dev = cpg_mssr_attach_dev; + genpd->detach_dev = cpg_mssr_detach_dev; + } + + genpd->power_off = rcar_gen4_sysc_pd_power_off; + genpd->power_on = rcar_gen4_sysc_pd_power_on; + + if (pd->flags & (PD_CPU | PD_NO_CR)) { + /* Skip CPUs (handled by SMP code) and areas without control */ + pr_debug("%s: Not touching %s\n", __func__, genpd->name); + goto finalize; + } + + if (!rcar_gen4_sysc_power_is_off(pd->pdr)) { + pr_debug("%s: %s is already powered\n", __func__, genpd->name); + goto finalize; + } + + rcar_gen4_sysc_power(pd->pdr, true); + +finalize: + error = pm_genpd_init(genpd, &simple_qos_governor, false); + if (error) + pr_err("Failed to init PM domain %s: %d\n", name, error); + + return error; +} + +static const struct of_device_id rcar_gen4_sysc_matches[] __initconst = { +#ifdef CONFIG_SYSC_R8A779A0 + { .compatible = "renesas,r8a779a0-sysc", .data = &r8a779a0_sysc_info }, +#endif +#ifdef CONFIG_SYSC_R8A779F0 + { .compatible = "renesas,r8a779f0-sysc", .data = &r8a779f0_sysc_info }, +#endif +#ifdef CONFIG_SYSC_R8A779G0 + { .compatible = "renesas,r8a779g0-sysc", .data = &r8a779g0_sysc_info }, +#endif + { /* sentinel */ } +}; + +struct rcar_gen4_pm_domains { + struct genpd_onecell_data onecell_data; + struct generic_pm_domain *domains[RCAR_GEN4_PD_ALWAYS_ON + 1]; +}; + +static struct genpd_onecell_data *rcar_gen4_sysc_onecell_data; + +static int __init rcar_gen4_sysc_pd_init(void) +{ + const struct rcar_gen4_sysc_info *info; + const struct of_device_id *match; + struct rcar_gen4_pm_domains *domains; + struct device_node *np; + void __iomem *base; + unsigned int i; + int error; + + np = of_find_matching_node_and_match(NULL, rcar_gen4_sysc_matches, &match); + if (!np) + return -ENODEV; + + info = match->data; + + base = of_iomap(np, 0); + if (!base) { + pr_warn("%pOF: Cannot map regs\n", np); + error = -ENOMEM; + goto out_put; + } + + rcar_gen4_sysc_base = base; + + domains = kzalloc(sizeof(*domains), GFP_KERNEL); + if (!domains) { + error = -ENOMEM; + goto out_put; + } + + domains->onecell_data.domains = domains->domains; + domains->onecell_data.num_domains = ARRAY_SIZE(domains->domains); + rcar_gen4_sysc_onecell_data = &domains->onecell_data; + + for (i = 0; i < info->num_areas; i++) { + const struct rcar_gen4_sysc_area *area = &info->areas[i]; + struct rcar_gen4_sysc_pd *pd; + size_t n; + + if (!area->name) { + /* Skip NULLified area */ + continue; + } + + n = strlen(area->name) + 1; + pd = kzalloc(sizeof(*pd) + n, GFP_KERNEL); + if (!pd) { + error = -ENOMEM; + goto out_put; + } + + memcpy(pd->name, area->name, n); + pd->genpd.name = pd->name; + pd->pdr = area->pdr; + pd->flags = area->flags; + + error = rcar_gen4_sysc_pd_setup(pd); + if (error) + goto out_put; + + domains->domains[area->pdr] = &pd->genpd; + + if (area->parent < 0) + continue; + + error = pm_genpd_add_subdomain(domains->domains[area->parent], + &pd->genpd); + if (error) { + pr_warn("Failed to add PM subdomain %s to parent %u\n", + area->name, area->parent); + goto out_put; + } + } + + error = of_genpd_add_provider_onecell(np, &domains->onecell_data); + +out_put: + of_node_put(np); + return error; +} +early_initcall(rcar_gen4_sysc_pd_init); diff --git a/drivers/pmdomain/renesas/rcar-gen4-sysc.h b/drivers/pmdomain/renesas/rcar-gen4-sysc.h new file mode 100644 index 000000000000..388cfa8f8f9f --- /dev/null +++ b/drivers/pmdomain/renesas/rcar-gen4-sysc.h @@ -0,0 +1,44 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * R-Car Gen4 System Controller + * + * Copyright (C) 2021 Renesas Electronics Corp. + */ +#ifndef __SOC_RENESAS_RCAR_GEN4_SYSC_H__ +#define __SOC_RENESAS_RCAR_GEN4_SYSC_H__ + +#include + +/* + * Power Domain flags + */ +#define PD_CPU BIT(0) /* Area contains main CPU core */ +#define PD_SCU BIT(1) /* Area contains SCU and L2 cache */ +#define PD_NO_CR BIT(2) /* Area lacks PWR{ON,OFF}CR registers */ + +#define PD_CPU_NOCR (PD_CPU | PD_NO_CR) /* CPU area lacks CR */ +#define PD_ALWAYS_ON PD_NO_CR /* Always-on area */ + +/* + * Description of a Power Area + */ +struct rcar_gen4_sysc_area { + const char *name; + u8 pdr; /* PDRn */ + s8 parent; /* -1 if none */ + u8 flags; /* See PD_* */ +}; + +/* + * SoC-specific Power Area Description + */ +struct rcar_gen4_sysc_info { + const struct rcar_gen4_sysc_area *areas; + unsigned int num_areas; +}; + +extern const struct rcar_gen4_sysc_info r8a779a0_sysc_info; +extern const struct rcar_gen4_sysc_info r8a779f0_sysc_info; +extern const struct rcar_gen4_sysc_info r8a779g0_sysc_info; + +#endif /* __SOC_RENESAS_RCAR_GEN4_SYSC_H__ */ diff --git a/drivers/pmdomain/renesas/rcar-sysc.c b/drivers/pmdomain/renesas/rcar-sysc.c new file mode 100644 index 000000000000..eed47696e825 --- /dev/null +++ b/drivers/pmdomain/renesas/rcar-sysc.c @@ -0,0 +1,494 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * R-Car SYSC Power management support + * + * Copyright (C) 2014 Magnus Damm + * Copyright (C) 2015-2017 Glider bvba + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "rcar-sysc.h" + +/* SYSC Common */ +#define SYSCSR 0x00 /* SYSC Status Register */ +#define SYSCISR 0x04 /* Interrupt Status Register */ +#define SYSCISCR 0x08 /* Interrupt Status Clear Register */ +#define SYSCIER 0x0c /* Interrupt Enable Register */ +#define SYSCIMR 0x10 /* Interrupt Mask Register */ + +/* SYSC Status Register */ +#define SYSCSR_PONENB 1 /* Ready for power resume requests */ +#define SYSCSR_POFFENB 0 /* Ready for power shutoff requests */ + +/* + * Power Control Register Offsets inside the register block for each domain + * Note: The "CR" registers for ARM cores exist on H1 only + * Use WFI to power off, CPG/APMU to resume ARM cores on R-Car Gen2 + * Use PSCI on R-Car Gen3 + */ +#define PWRSR_OFFS 0x00 /* Power Status Register */ +#define PWROFFCR_OFFS 0x04 /* Power Shutoff Control Register */ +#define PWROFFSR_OFFS 0x08 /* Power Shutoff Status Register */ +#define PWRONCR_OFFS 0x0c /* Power Resume Control Register */ +#define PWRONSR_OFFS 0x10 /* Power Resume Status Register */ +#define PWRER_OFFS 0x14 /* Power Shutoff/Resume Error */ + + +#define SYSCSR_TIMEOUT 100 +#define SYSCSR_DELAY_US 1 + +#define PWRER_RETRIES 100 +#define PWRER_DELAY_US 1 + +#define SYSCISR_TIMEOUT 1000 +#define SYSCISR_DELAY_US 1 + +#define RCAR_PD_ALWAYS_ON 32 /* Always-on power area */ + +struct rcar_sysc_ch { + u16 chan_offs; + u8 chan_bit; + u8 isr_bit; +}; + +static void __iomem *rcar_sysc_base; +static DEFINE_SPINLOCK(rcar_sysc_lock); /* SMP CPUs + I/O devices */ +static u32 rcar_sysc_extmask_offs, rcar_sysc_extmask_val; + +static int rcar_sysc_pwr_on_off(const struct rcar_sysc_ch *sysc_ch, bool on) +{ + unsigned int sr_bit, reg_offs; + u32 val; + int ret; + + if (on) { + sr_bit = SYSCSR_PONENB; + reg_offs = PWRONCR_OFFS; + } else { + sr_bit = SYSCSR_POFFENB; + reg_offs = PWROFFCR_OFFS; + } + + /* Wait until SYSC is ready to accept a power request */ + ret = readl_poll_timeout_atomic(rcar_sysc_base + SYSCSR, val, + val & BIT(sr_bit), SYSCSR_DELAY_US, + SYSCSR_TIMEOUT); + if (ret) + return -EAGAIN; + + /* Submit power shutoff or power resume request */ + iowrite32(BIT(sysc_ch->chan_bit), + rcar_sysc_base + sysc_ch->chan_offs + reg_offs); + + return 0; +} + +static int rcar_sysc_power(const struct rcar_sysc_ch *sysc_ch, bool on) +{ + unsigned int isr_mask = BIT(sysc_ch->isr_bit); + unsigned int chan_mask = BIT(sysc_ch->chan_bit); + unsigned int status, k; + unsigned long flags; + int ret; + + spin_lock_irqsave(&rcar_sysc_lock, flags); + + /* + * Mask external power requests for CPU or 3DG domains + */ + if (rcar_sysc_extmask_val) { + iowrite32(rcar_sysc_extmask_val, + rcar_sysc_base + rcar_sysc_extmask_offs); + } + + /* + * The interrupt source needs to be enabled, but masked, to prevent the + * CPU from receiving it. + */ + iowrite32(ioread32(rcar_sysc_base + SYSCIMR) | isr_mask, + rcar_sysc_base + SYSCIMR); + iowrite32(ioread32(rcar_sysc_base + SYSCIER) | isr_mask, + rcar_sysc_base + SYSCIER); + + iowrite32(isr_mask, rcar_sysc_base + SYSCISCR); + + /* Submit power shutoff or resume request until it was accepted */ + for (k = 0; k < PWRER_RETRIES; k++) { + ret = rcar_sysc_pwr_on_off(sysc_ch, on); + if (ret) + goto out; + + status = ioread32(rcar_sysc_base + + sysc_ch->chan_offs + PWRER_OFFS); + if (!(status & chan_mask)) + break; + + udelay(PWRER_DELAY_US); + } + + if (k == PWRER_RETRIES) { + ret = -EIO; + goto out; + } + + /* Wait until the power shutoff or resume request has completed * */ + ret = readl_poll_timeout_atomic(rcar_sysc_base + SYSCISR, status, + status & isr_mask, SYSCISR_DELAY_US, + SYSCISR_TIMEOUT); + if (ret) + ret = -EIO; + + iowrite32(isr_mask, rcar_sysc_base + SYSCISCR); + + out: + if (rcar_sysc_extmask_val) + iowrite32(0, rcar_sysc_base + rcar_sysc_extmask_offs); + + spin_unlock_irqrestore(&rcar_sysc_lock, flags); + + pr_debug("sysc power %s domain %d: %08x -> %d\n", on ? "on" : "off", + sysc_ch->isr_bit, ioread32(rcar_sysc_base + SYSCISR), ret); + return ret; +} + +static bool rcar_sysc_power_is_off(const struct rcar_sysc_ch *sysc_ch) +{ + unsigned int st; + + st = ioread32(rcar_sysc_base + sysc_ch->chan_offs + PWRSR_OFFS); + if (st & BIT(sysc_ch->chan_bit)) + return true; + + return false; +} + +struct rcar_sysc_pd { + struct generic_pm_domain genpd; + struct rcar_sysc_ch ch; + unsigned int flags; + char name[]; +}; + +static inline struct rcar_sysc_pd *to_rcar_pd(struct generic_pm_domain *d) +{ + return container_of(d, struct rcar_sysc_pd, genpd); +} + +static int rcar_sysc_pd_power_off(struct generic_pm_domain *genpd) +{ + struct rcar_sysc_pd *pd = to_rcar_pd(genpd); + + pr_debug("%s: %s\n", __func__, genpd->name); + return rcar_sysc_power(&pd->ch, false); +} + +static int rcar_sysc_pd_power_on(struct generic_pm_domain *genpd) +{ + struct rcar_sysc_pd *pd = to_rcar_pd(genpd); + + pr_debug("%s: %s\n", __func__, genpd->name); + return rcar_sysc_power(&pd->ch, true); +} + +static bool has_cpg_mstp; + +static int __init rcar_sysc_pd_setup(struct rcar_sysc_pd *pd) +{ + struct generic_pm_domain *genpd = &pd->genpd; + const char *name = pd->genpd.name; + int error; + + if (pd->flags & PD_CPU) { + /* + * This domain contains a CPU core and therefore it should + * only be turned off if the CPU is not in use. + */ + pr_debug("PM domain %s contains %s\n", name, "CPU"); + genpd->flags |= GENPD_FLAG_ALWAYS_ON; + } else if (pd->flags & PD_SCU) { + /* + * This domain contains an SCU and cache-controller, and + * therefore it should only be turned off if the CPU cores are + * not in use. + */ + pr_debug("PM domain %s contains %s\n", name, "SCU"); + genpd->flags |= GENPD_FLAG_ALWAYS_ON; + } else if (pd->flags & PD_NO_CR) { + /* + * This domain cannot be turned off. + */ + genpd->flags |= GENPD_FLAG_ALWAYS_ON; + } + + if (!(pd->flags & (PD_CPU | PD_SCU))) { + /* Enable Clock Domain for I/O devices */ + genpd->flags |= GENPD_FLAG_PM_CLK | GENPD_FLAG_ACTIVE_WAKEUP; + if (has_cpg_mstp) { + genpd->attach_dev = cpg_mstp_attach_dev; + genpd->detach_dev = cpg_mstp_detach_dev; + } else { + genpd->attach_dev = cpg_mssr_attach_dev; + genpd->detach_dev = cpg_mssr_detach_dev; + } + } + + genpd->power_off = rcar_sysc_pd_power_off; + genpd->power_on = rcar_sysc_pd_power_on; + + if (pd->flags & (PD_CPU | PD_NO_CR)) { + /* Skip CPUs (handled by SMP code) and areas without control */ + pr_debug("%s: Not touching %s\n", __func__, genpd->name); + goto finalize; + } + + if (!rcar_sysc_power_is_off(&pd->ch)) { + pr_debug("%s: %s is already powered\n", __func__, genpd->name); + goto finalize; + } + + rcar_sysc_power(&pd->ch, true); + +finalize: + error = pm_genpd_init(genpd, &simple_qos_governor, false); + if (error) + pr_err("Failed to init PM domain %s: %d\n", name, error); + + return error; +} + +static const struct of_device_id rcar_sysc_matches[] __initconst = { +#ifdef CONFIG_SYSC_R8A7742 + { .compatible = "renesas,r8a7742-sysc", .data = &r8a7742_sysc_info }, +#endif +#ifdef CONFIG_SYSC_R8A7743 + { .compatible = "renesas,r8a7743-sysc", .data = &r8a7743_sysc_info }, + /* RZ/G1N is identical to RZ/G2M w.r.t. power domains. */ + { .compatible = "renesas,r8a7744-sysc", .data = &r8a7743_sysc_info }, +#endif +#ifdef CONFIG_SYSC_R8A7745 + { .compatible = "renesas,r8a7745-sysc", .data = &r8a7745_sysc_info }, +#endif +#ifdef CONFIG_SYSC_R8A77470 + { .compatible = "renesas,r8a77470-sysc", .data = &r8a77470_sysc_info }, +#endif +#ifdef CONFIG_SYSC_R8A774A1 + { .compatible = "renesas,r8a774a1-sysc", .data = &r8a774a1_sysc_info }, +#endif +#ifdef CONFIG_SYSC_R8A774B1 + { .compatible = "renesas,r8a774b1-sysc", .data = &r8a774b1_sysc_info }, +#endif +#ifdef CONFIG_SYSC_R8A774C0 + { .compatible = "renesas,r8a774c0-sysc", .data = &r8a774c0_sysc_info }, +#endif +#ifdef CONFIG_SYSC_R8A774E1 + { .compatible = "renesas,r8a774e1-sysc", .data = &r8a774e1_sysc_info }, +#endif +#ifdef CONFIG_SYSC_R8A7779 + { .compatible = "renesas,r8a7779-sysc", .data = &r8a7779_sysc_info }, +#endif +#ifdef CONFIG_SYSC_R8A7790 + { .compatible = "renesas,r8a7790-sysc", .data = &r8a7790_sysc_info }, +#endif +#ifdef CONFIG_SYSC_R8A7791 + { .compatible = "renesas,r8a7791-sysc", .data = &r8a7791_sysc_info }, + /* R-Car M2-N is identical to R-Car M2-W w.r.t. power domains. */ + { .compatible = "renesas,r8a7793-sysc", .data = &r8a7791_sysc_info }, +#endif +#ifdef CONFIG_SYSC_R8A7792 + { .compatible = "renesas,r8a7792-sysc", .data = &r8a7792_sysc_info }, +#endif +#ifdef CONFIG_SYSC_R8A7794 + { .compatible = "renesas,r8a7794-sysc", .data = &r8a7794_sysc_info }, +#endif +#ifdef CONFIG_SYSC_R8A7795 + { .compatible = "renesas,r8a7795-sysc", .data = &r8a7795_sysc_info }, +#endif +#ifdef CONFIG_SYSC_R8A77960 + { .compatible = "renesas,r8a7796-sysc", .data = &r8a77960_sysc_info }, +#endif +#ifdef CONFIG_SYSC_R8A77961 + { .compatible = "renesas,r8a77961-sysc", .data = &r8a77961_sysc_info }, +#endif +#ifdef CONFIG_SYSC_R8A77965 + { .compatible = "renesas,r8a77965-sysc", .data = &r8a77965_sysc_info }, +#endif +#ifdef CONFIG_SYSC_R8A77970 + { .compatible = "renesas,r8a77970-sysc", .data = &r8a77970_sysc_info }, +#endif +#ifdef CONFIG_SYSC_R8A77980 + { .compatible = "renesas,r8a77980-sysc", .data = &r8a77980_sysc_info }, +#endif +#ifdef CONFIG_SYSC_R8A77990 + { .compatible = "renesas,r8a77990-sysc", .data = &r8a77990_sysc_info }, +#endif +#ifdef CONFIG_SYSC_R8A77995 + { .compatible = "renesas,r8a77995-sysc", .data = &r8a77995_sysc_info }, +#endif + { /* sentinel */ } +}; + +struct rcar_pm_domains { + struct genpd_onecell_data onecell_data; + struct generic_pm_domain *domains[RCAR_PD_ALWAYS_ON + 1]; +}; + +static struct genpd_onecell_data *rcar_sysc_onecell_data; + +static int __init rcar_sysc_pd_init(void) +{ + const struct rcar_sysc_info *info; + const struct of_device_id *match; + struct rcar_pm_domains *domains; + struct device_node *np; + void __iomem *base; + unsigned int i; + int error; + + np = of_find_matching_node_and_match(NULL, rcar_sysc_matches, &match); + if (!np) + return -ENODEV; + + info = match->data; + + if (info->init) { + error = info->init(); + if (error) + goto out_put; + } + + has_cpg_mstp = of_find_compatible_node(NULL, NULL, + "renesas,cpg-mstp-clocks"); + + base = of_iomap(np, 0); + if (!base) { + pr_warn("%pOF: Cannot map regs\n", np); + error = -ENOMEM; + goto out_put; + } + + rcar_sysc_base = base; + + /* Optional External Request Mask Register */ + rcar_sysc_extmask_offs = info->extmask_offs; + rcar_sysc_extmask_val = info->extmask_val; + + domains = kzalloc(sizeof(*domains), GFP_KERNEL); + if (!domains) { + error = -ENOMEM; + goto out_put; + } + + domains->onecell_data.domains = domains->domains; + domains->onecell_data.num_domains = ARRAY_SIZE(domains->domains); + rcar_sysc_onecell_data = &domains->onecell_data; + + for (i = 0; i < info->num_areas; i++) { + const struct rcar_sysc_area *area = &info->areas[i]; + struct rcar_sysc_pd *pd; + size_t n; + + if (!area->name) { + /* Skip NULLified area */ + continue; + } + + n = strlen(area->name) + 1; + pd = kzalloc(sizeof(*pd) + n, GFP_KERNEL); + if (!pd) { + error = -ENOMEM; + goto out_put; + } + + memcpy(pd->name, area->name, n); + pd->genpd.name = pd->name; + pd->ch.chan_offs = area->chan_offs; + pd->ch.chan_bit = area->chan_bit; + pd->ch.isr_bit = area->isr_bit; + pd->flags = area->flags; + + error = rcar_sysc_pd_setup(pd); + if (error) + goto out_put; + + domains->domains[area->isr_bit] = &pd->genpd; + + if (area->parent < 0) + continue; + + error = pm_genpd_add_subdomain(domains->domains[area->parent], + &pd->genpd); + if (error) { + pr_warn("Failed to add PM subdomain %s to parent %u\n", + area->name, area->parent); + goto out_put; + } + } + + error = of_genpd_add_provider_onecell(np, &domains->onecell_data); + if (!error) + fwnode_dev_initialized(of_fwnode_handle(np), true); + +out_put: + of_node_put(np); + return error; +} +early_initcall(rcar_sysc_pd_init); + +void __init rcar_sysc_nullify(struct rcar_sysc_area *areas, + unsigned int num_areas, u8 id) +{ + unsigned int i; + + for (i = 0; i < num_areas; i++) + if (areas[i].isr_bit == id) { + areas[i].name = NULL; + return; + } +} + +#ifdef CONFIG_ARCH_R8A7779 +static int rcar_sysc_power_cpu(unsigned int idx, bool on) +{ + struct generic_pm_domain *genpd; + struct rcar_sysc_pd *pd; + unsigned int i; + + if (!rcar_sysc_onecell_data) + return -ENODEV; + + for (i = 0; i < rcar_sysc_onecell_data->num_domains; i++) { + genpd = rcar_sysc_onecell_data->domains[i]; + if (!genpd) + continue; + + pd = to_rcar_pd(genpd); + if (!(pd->flags & PD_CPU) || pd->ch.chan_bit != idx) + continue; + + return rcar_sysc_power(&pd->ch, on); + } + + return -ENOENT; +} + +int rcar_sysc_power_down_cpu(unsigned int cpu) +{ + return rcar_sysc_power_cpu(cpu, false); +} + +int rcar_sysc_power_up_cpu(unsigned int cpu) +{ + return rcar_sysc_power_cpu(cpu, true); +} +#endif /* CONFIG_ARCH_R8A7779 */ diff --git a/drivers/pmdomain/renesas/rcar-sysc.h b/drivers/pmdomain/renesas/rcar-sysc.h new file mode 100644 index 000000000000..266c599a0a9b --- /dev/null +++ b/drivers/pmdomain/renesas/rcar-sysc.h @@ -0,0 +1,82 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Renesas R-Car System Controller + * + * Copyright (C) 2016 Glider bvba + */ +#ifndef __SOC_RENESAS_RCAR_SYSC_H__ +#define __SOC_RENESAS_RCAR_SYSC_H__ + +#include + + +/* + * Power Domain flags + */ +#define PD_CPU BIT(0) /* Area contains main CPU core */ +#define PD_SCU BIT(1) /* Area contains SCU and L2 cache */ +#define PD_NO_CR BIT(2) /* Area lacks PWR{ON,OFF}CR registers */ + +#define PD_CPU_CR PD_CPU /* CPU area has CR (R-Car H1) */ +#define PD_CPU_NOCR PD_CPU | PD_NO_CR /* CPU area lacks CR (R-Car Gen2/3) */ +#define PD_ALWAYS_ON PD_NO_CR /* Always-on area */ + + +/* + * Description of a Power Area + */ + +struct rcar_sysc_area { + const char *name; + u16 chan_offs; /* Offset of PWRSR register for this area */ + u8 chan_bit; /* Bit in PWR* (except for PWRUP in PWRSR) */ + u8 isr_bit; /* Bit in SYSCI*R */ + s8 parent; /* -1 if none */ + u8 flags; /* See PD_* */ +}; + + +/* + * SoC-specific Power Area Description + */ + +struct rcar_sysc_info { + int (*init)(void); /* Optional */ + const struct rcar_sysc_area *areas; + unsigned int num_areas; + /* Optional External Request Mask Register */ + u32 extmask_offs; /* SYSCEXTMASK register offset */ + u32 extmask_val; /* SYSCEXTMASK register mask value */ +}; + +extern const struct rcar_sysc_info r8a7742_sysc_info; +extern const struct rcar_sysc_info r8a7743_sysc_info; +extern const struct rcar_sysc_info r8a7745_sysc_info; +extern const struct rcar_sysc_info r8a77470_sysc_info; +extern const struct rcar_sysc_info r8a774a1_sysc_info; +extern const struct rcar_sysc_info r8a774b1_sysc_info; +extern const struct rcar_sysc_info r8a774c0_sysc_info; +extern const struct rcar_sysc_info r8a774e1_sysc_info; +extern const struct rcar_sysc_info r8a7779_sysc_info; +extern const struct rcar_sysc_info r8a7790_sysc_info; +extern const struct rcar_sysc_info r8a7791_sysc_info; +extern const struct rcar_sysc_info r8a7792_sysc_info; +extern const struct rcar_sysc_info r8a7794_sysc_info; +extern struct rcar_sysc_info r8a7795_sysc_info; +extern const struct rcar_sysc_info r8a77960_sysc_info; +extern const struct rcar_sysc_info r8a77961_sysc_info; +extern const struct rcar_sysc_info r8a77965_sysc_info; +extern const struct rcar_sysc_info r8a77970_sysc_info; +extern const struct rcar_sysc_info r8a77980_sysc_info; +extern const struct rcar_sysc_info r8a77990_sysc_info; +extern const struct rcar_sysc_info r8a77995_sysc_info; + + + /* + * Helpers for fixing up power area tables depending on SoC revision + */ + +extern void rcar_sysc_nullify(struct rcar_sysc_area *areas, + unsigned int num_areas, u8 id); + +#endif /* __SOC_RENESAS_RCAR_SYSC_H__ */ diff --git a/drivers/pmdomain/renesas/rmobile-sysc.c b/drivers/pmdomain/renesas/rmobile-sysc.c new file mode 100644 index 000000000000..912daadaa10d --- /dev/null +++ b/drivers/pmdomain/renesas/rmobile-sysc.c @@ -0,0 +1,343 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * rmobile power management support + * + * Copyright (C) 2012 Renesas Solutions Corp. + * Copyright (C) 2012 Kuninori Morimoto + * Copyright (C) 2014 Glider bvba + * + * based on pm-sh7372.c + * Copyright (C) 2011 Magnus Damm + */ +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +/* SYSC */ +#define SPDCR 0x08 /* SYS Power Down Control Register */ +#define SWUCR 0x14 /* SYS Wakeup Control Register */ +#define PSTR 0x80 /* Power Status Register */ + +#define PSTR_RETRIES 100 +#define PSTR_DELAY_US 10 + +struct rmobile_pm_domain { + struct generic_pm_domain genpd; + struct dev_power_governor *gov; + int (*suspend)(void); + void __iomem *base; + unsigned int bit_shift; +}; + +static inline +struct rmobile_pm_domain *to_rmobile_pd(struct generic_pm_domain *d) +{ + return container_of(d, struct rmobile_pm_domain, genpd); +} + +static int rmobile_pd_power_down(struct generic_pm_domain *genpd) +{ + struct rmobile_pm_domain *rmobile_pd = to_rmobile_pd(genpd); + unsigned int mask = BIT(rmobile_pd->bit_shift); + u32 val; + + if (rmobile_pd->suspend) { + int ret = rmobile_pd->suspend(); + + if (ret) + return ret; + } + + if (readl(rmobile_pd->base + PSTR) & mask) { + writel(mask, rmobile_pd->base + SPDCR); + + readl_poll_timeout_atomic(rmobile_pd->base + SPDCR, val, + !(val & mask), 0, PSTR_RETRIES); + } + + pr_debug("%s: Power off, 0x%08x -> PSTR = 0x%08x\n", genpd->name, mask, + readl(rmobile_pd->base + PSTR)); + + return 0; +} + +static int __rmobile_pd_power_up(struct rmobile_pm_domain *rmobile_pd) +{ + unsigned int val, mask = BIT(rmobile_pd->bit_shift); + int ret = 0; + + if (readl(rmobile_pd->base + PSTR) & mask) + return ret; + + writel(mask, rmobile_pd->base + SWUCR); + + ret = readl_poll_timeout_atomic(rmobile_pd->base + SWUCR, val, + (val & mask), PSTR_DELAY_US, + PSTR_RETRIES * PSTR_DELAY_US); + + pr_debug("%s: Power on, 0x%08x -> PSTR = 0x%08x\n", + rmobile_pd->genpd.name, mask, + readl(rmobile_pd->base + PSTR)); + + return ret; +} + +static int rmobile_pd_power_up(struct generic_pm_domain *genpd) +{ + return __rmobile_pd_power_up(to_rmobile_pd(genpd)); +} + +static void rmobile_init_pm_domain(struct rmobile_pm_domain *rmobile_pd) +{ + struct generic_pm_domain *genpd = &rmobile_pd->genpd; + struct dev_power_governor *gov = rmobile_pd->gov; + + genpd->flags |= GENPD_FLAG_PM_CLK | GENPD_FLAG_ACTIVE_WAKEUP; + genpd->attach_dev = cpg_mstp_attach_dev; + genpd->detach_dev = cpg_mstp_detach_dev; + + if (!(genpd->flags & GENPD_FLAG_ALWAYS_ON)) { + genpd->power_off = rmobile_pd_power_down; + genpd->power_on = rmobile_pd_power_up; + __rmobile_pd_power_up(rmobile_pd); + } + + pm_genpd_init(genpd, gov ? : &simple_qos_governor, false); +} + +static int rmobile_pd_suspend_console(void) +{ + /* + * Serial consoles make use of SCIF hardware located in this domain, + * hence keep the power domain on if "no_console_suspend" is set. + */ + return console_suspend_enabled ? 0 : -EBUSY; +} + +enum pd_types { + PD_NORMAL, + PD_CPU, + PD_CONSOLE, + PD_DEBUG, + PD_MEMCTL, +}; + +#define MAX_NUM_SPECIAL_PDS 16 + +static struct special_pd { + struct device_node *pd; + enum pd_types type; +} special_pds[MAX_NUM_SPECIAL_PDS] __initdata; + +static unsigned int num_special_pds __initdata; + +static const struct of_device_id special_ids[] __initconst = { + { .compatible = "arm,coresight-etm3x", .data = (void *)PD_DEBUG }, + { .compatible = "renesas,dbsc-r8a73a4", .data = (void *)PD_MEMCTL, }, + { .compatible = "renesas,dbsc3-r8a7740", .data = (void *)PD_MEMCTL, }, + { .compatible = "renesas,sbsc-sh73a0", .data = (void *)PD_MEMCTL, }, + { /* sentinel */ }, +}; + +static void __init add_special_pd(struct device_node *np, enum pd_types type) +{ + unsigned int i; + struct device_node *pd; + + pd = of_parse_phandle(np, "power-domains", 0); + if (!pd) + return; + + for (i = 0; i < num_special_pds; i++) + if (pd == special_pds[i].pd && type == special_pds[i].type) { + of_node_put(pd); + return; + } + + if (num_special_pds == ARRAY_SIZE(special_pds)) { + pr_warn("Too many special PM domains\n"); + of_node_put(pd); + return; + } + + pr_debug("Special PM domain %pOFn type %d for %pOF\n", pd, type, np); + + special_pds[num_special_pds].pd = pd; + special_pds[num_special_pds].type = type; + num_special_pds++; +} + +static void __init get_special_pds(void) +{ + struct device_node *np; + const struct of_device_id *id; + + /* PM domains containing CPUs */ + for_each_of_cpu_node(np) + add_special_pd(np, PD_CPU); + + /* PM domain containing console */ + if (of_stdout) + add_special_pd(of_stdout, PD_CONSOLE); + + /* PM domains containing other special devices */ + for_each_matching_node_and_match(np, special_ids, &id) + add_special_pd(np, (enum pd_types)id->data); +} + +static void __init put_special_pds(void) +{ + unsigned int i; + + for (i = 0; i < num_special_pds; i++) + of_node_put(special_pds[i].pd); +} + +static enum pd_types __init pd_type(const struct device_node *pd) +{ + unsigned int i; + + for (i = 0; i < num_special_pds; i++) + if (pd == special_pds[i].pd) + return special_pds[i].type; + + return PD_NORMAL; +} + +static void __init rmobile_setup_pm_domain(struct device_node *np, + struct rmobile_pm_domain *pd) +{ + const char *name = pd->genpd.name; + + switch (pd_type(np)) { + case PD_CPU: + /* + * This domain contains the CPU core and therefore it should + * only be turned off if the CPU is not in use. + */ + pr_debug("PM domain %s contains CPU\n", name); + pd->genpd.flags |= GENPD_FLAG_ALWAYS_ON; + break; + + case PD_CONSOLE: + pr_debug("PM domain %s contains serial console\n", name); + pd->gov = &pm_domain_always_on_gov; + pd->suspend = rmobile_pd_suspend_console; + break; + + case PD_DEBUG: + /* + * This domain contains the Coresight-ETM hardware block and + * therefore it should only be turned off if the debug module + * is not in use. + */ + pr_debug("PM domain %s contains Coresight-ETM\n", name); + pd->genpd.flags |= GENPD_FLAG_ALWAYS_ON; + break; + + case PD_MEMCTL: + /* + * This domain contains a memory-controller and therefore it + * should only be turned off if memory is not in use. + */ + pr_debug("PM domain %s contains MEMCTL\n", name); + pd->genpd.flags |= GENPD_FLAG_ALWAYS_ON; + break; + + case PD_NORMAL: + if (pd->bit_shift == ~0) { + /* Top-level always-on domain */ + pr_debug("PM domain %s is always-on domain\n", name); + pd->genpd.flags |= GENPD_FLAG_ALWAYS_ON; + } + break; + } + + rmobile_init_pm_domain(pd); +} + +static int __init rmobile_add_pm_domains(void __iomem *base, + struct device_node *parent, + struct generic_pm_domain *genpd_parent) +{ + struct device_node *np; + + for_each_child_of_node(parent, np) { + struct rmobile_pm_domain *pd; + u32 idx = ~0; + + if (of_property_read_u32(np, "reg", &idx)) { + /* always-on domain */ + } + + pd = kzalloc(sizeof(*pd), GFP_KERNEL); + if (!pd) { + of_node_put(np); + return -ENOMEM; + } + + pd->genpd.name = np->name; + pd->base = base; + pd->bit_shift = idx; + + rmobile_setup_pm_domain(np, pd); + if (genpd_parent) + pm_genpd_add_subdomain(genpd_parent, &pd->genpd); + of_genpd_add_provider_simple(np, &pd->genpd); + + rmobile_add_pm_domains(base, np, &pd->genpd); + } + return 0; +} + +static int __init rmobile_init_pm_domains(void) +{ + struct device_node *np, *pmd; + bool scanned = false; + void __iomem *base; + int ret = 0; + + for_each_compatible_node(np, NULL, "renesas,sysc-rmobile") { + base = of_iomap(np, 0); + if (!base) { + pr_warn("%pOF cannot map reg 0\n", np); + continue; + } + + pmd = of_get_child_by_name(np, "pm-domains"); + if (!pmd) { + iounmap(base); + pr_warn("%pOF lacks pm-domains node\n", np); + continue; + } + + if (!scanned) { + /* Find PM domains containing special blocks */ + get_special_pds(); + scanned = true; + } + + ret = rmobile_add_pm_domains(base, pmd, NULL); + of_node_put(pmd); + if (ret) { + of_node_put(np); + break; + } + + fwnode_dev_initialized(of_fwnode_handle(np), true); + } + + put_special_pds(); + + return ret; +} + +core_initcall(rmobile_init_pm_domains); diff --git a/drivers/pmdomain/rockchip/Makefile b/drivers/pmdomain/rockchip/Makefile new file mode 100644 index 000000000000..8fb9d88a3492 --- /dev/null +++ b/drivers/pmdomain/rockchip/Makefile @@ -0,0 +1,2 @@ +# SPDX-License-Identifier: GPL-2.0-only +obj-$(CONFIG_ROCKCHIP_PM_DOMAINS) += pm-domains.o diff --git a/drivers/pmdomain/rockchip/pm-domains.c b/drivers/pmdomain/rockchip/pm-domains.c new file mode 100644 index 000000000000..d5d3ecb38283 --- /dev/null +++ b/drivers/pmdomain/rockchip/pm-domains.c @@ -0,0 +1,1396 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Rockchip Generic power domain support. + * + * Copyright (c) 2015 ROCKCHIP, Co. Ltd. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +struct rockchip_domain_info { + const char *name; + int pwr_mask; + int status_mask; + int req_mask; + int idle_mask; + int ack_mask; + bool active_wakeup; + int pwr_w_mask; + int req_w_mask; + int mem_status_mask; + int repair_status_mask; + u32 pwr_offset; + u32 mem_offset; + u32 req_offset; +}; + +struct rockchip_pmu_info { + u32 pwr_offset; + u32 status_offset; + u32 req_offset; + u32 idle_offset; + u32 ack_offset; + u32 mem_pwr_offset; + u32 chain_status_offset; + u32 mem_status_offset; + u32 repair_status_offset; + + u32 core_pwrcnt_offset; + u32 gpu_pwrcnt_offset; + + unsigned int core_power_transition_time; + unsigned int gpu_power_transition_time; + + int num_domains; + const struct rockchip_domain_info *domain_info; +}; + +#define MAX_QOS_REGS_NUM 5 +#define QOS_PRIORITY 0x08 +#define QOS_MODE 0x0c +#define QOS_BANDWIDTH 0x10 +#define QOS_SATURATION 0x14 +#define QOS_EXTCONTROL 0x18 + +struct rockchip_pm_domain { + struct generic_pm_domain genpd; + const struct rockchip_domain_info *info; + struct rockchip_pmu *pmu; + int num_qos; + struct regmap **qos_regmap; + u32 *qos_save_regs[MAX_QOS_REGS_NUM]; + int num_clks; + struct clk_bulk_data *clks; +}; + +struct rockchip_pmu { + struct device *dev; + struct regmap *regmap; + const struct rockchip_pmu_info *info; + struct mutex mutex; /* mutex lock for pmu */ + struct genpd_onecell_data genpd_data; + struct generic_pm_domain *domains[]; +}; + +#define to_rockchip_pd(gpd) container_of(gpd, struct rockchip_pm_domain, genpd) + +#define DOMAIN(_name, pwr, status, req, idle, ack, wakeup) \ +{ \ + .name = _name, \ + .pwr_mask = (pwr), \ + .status_mask = (status), \ + .req_mask = (req), \ + .idle_mask = (idle), \ + .ack_mask = (ack), \ + .active_wakeup = (wakeup), \ +} + +#define DOMAIN_M(_name, pwr, status, req, idle, ack, wakeup) \ +{ \ + .name = _name, \ + .pwr_w_mask = (pwr) << 16, \ + .pwr_mask = (pwr), \ + .status_mask = (status), \ + .req_w_mask = (req) << 16, \ + .req_mask = (req), \ + .idle_mask = (idle), \ + .ack_mask = (ack), \ + .active_wakeup = wakeup, \ +} + +#define DOMAIN_M_O_R(_name, p_offset, pwr, status, m_offset, m_status, r_status, r_offset, req, idle, ack, wakeup) \ +{ \ + .name = _name, \ + .pwr_offset = p_offset, \ + .pwr_w_mask = (pwr) << 16, \ + .pwr_mask = (pwr), \ + .status_mask = (status), \ + .mem_offset = m_offset, \ + .mem_status_mask = (m_status), \ + .repair_status_mask = (r_status), \ + .req_offset = r_offset, \ + .req_w_mask = (req) << 16, \ + .req_mask = (req), \ + .idle_mask = (idle), \ + .ack_mask = (ack), \ + .active_wakeup = wakeup, \ +} + +#define DOMAIN_RK3036(_name, req, ack, idle, wakeup) \ +{ \ + .name = _name, \ + .req_mask = (req), \ + .req_w_mask = (req) << 16, \ + .ack_mask = (ack), \ + .idle_mask = (idle), \ + .active_wakeup = wakeup, \ +} + +#define DOMAIN_PX30(name, pwr, status, req, wakeup) \ + DOMAIN_M(name, pwr, status, req, (req) << 16, req, wakeup) + +#define DOMAIN_RV1126(name, pwr, req, idle, wakeup) \ + DOMAIN_M(name, pwr, pwr, req, idle, idle, wakeup) + +#define DOMAIN_RK3288(name, pwr, status, req, wakeup) \ + DOMAIN(name, pwr, status, req, req, (req) << 16, wakeup) + +#define DOMAIN_RK3328(name, pwr, status, req, wakeup) \ + DOMAIN_M(name, pwr, pwr, req, (req) << 10, req, wakeup) + +#define DOMAIN_RK3368(name, pwr, status, req, wakeup) \ + DOMAIN(name, pwr, status, req, (req) << 16, req, wakeup) + +#define DOMAIN_RK3399(name, pwr, status, req, wakeup) \ + DOMAIN(name, pwr, status, req, req, req, wakeup) + +#define DOMAIN_RK3568(name, pwr, req, wakeup) \ + DOMAIN_M(name, pwr, pwr, req, req, req, wakeup) + +/* + * Dynamic Memory Controller may need to coordinate with us -- see + * rockchip_pmu_block(). + * + * dmc_pmu_mutex protects registration-time races, so DMC driver doesn't try to + * block() while we're initializing the PMU. + */ +static DEFINE_MUTEX(dmc_pmu_mutex); +static struct rockchip_pmu *dmc_pmu; + +/* + * Block PMU transitions and make sure they don't interfere with ARM Trusted + * Firmware operations. There are two conflicts, noted in the comments below. + * + * Caller must unblock PMU transitions via rockchip_pmu_unblock(). + */ +int rockchip_pmu_block(void) +{ + struct rockchip_pmu *pmu; + struct generic_pm_domain *genpd; + struct rockchip_pm_domain *pd; + int i, ret; + + mutex_lock(&dmc_pmu_mutex); + + /* No PMU (yet)? Then we just block rockchip_pmu_probe(). */ + if (!dmc_pmu) + return 0; + pmu = dmc_pmu; + + /* + * mutex blocks all idle transitions: we can't touch the + * PMU_BUS_IDLE_REQ (our ".idle_offset") register while ARM Trusted + * Firmware might be using it. + */ + mutex_lock(&pmu->mutex); + + /* + * Power domain clocks: Per Rockchip, we *must* keep certain clocks + * enabled for the duration of power-domain transitions. Most + * transitions are handled by this driver, but some cases (in + * particular, DRAM DVFS / memory-controller idle) must be handled by + * firmware. Firmware can handle most clock management via a special + * "ungate" register (PMU_CRU_GATEDIS_CON0), but unfortunately, this + * doesn't handle PLLs. We can assist this transition by doing the + * clock management on behalf of firmware. + */ + for (i = 0; i < pmu->genpd_data.num_domains; i++) { + genpd = pmu->genpd_data.domains[i]; + if (genpd) { + pd = to_rockchip_pd(genpd); + ret = clk_bulk_enable(pd->num_clks, pd->clks); + if (ret < 0) { + dev_err(pmu->dev, + "failed to enable clks for domain '%s': %d\n", + genpd->name, ret); + goto err; + } + } + } + + return 0; + +err: + for (i = i - 1; i >= 0; i--) { + genpd = pmu->genpd_data.domains[i]; + if (genpd) { + pd = to_rockchip_pd(genpd); + clk_bulk_disable(pd->num_clks, pd->clks); + } + } + mutex_unlock(&pmu->mutex); + mutex_unlock(&dmc_pmu_mutex); + + return ret; +} +EXPORT_SYMBOL_GPL(rockchip_pmu_block); + +/* Unblock PMU transitions. */ +void rockchip_pmu_unblock(void) +{ + struct rockchip_pmu *pmu; + struct generic_pm_domain *genpd; + struct rockchip_pm_domain *pd; + int i; + + if (dmc_pmu) { + pmu = dmc_pmu; + for (i = 0; i < pmu->genpd_data.num_domains; i++) { + genpd = pmu->genpd_data.domains[i]; + if (genpd) { + pd = to_rockchip_pd(genpd); + clk_bulk_disable(pd->num_clks, pd->clks); + } + } + + mutex_unlock(&pmu->mutex); + } + + mutex_unlock(&dmc_pmu_mutex); +} +EXPORT_SYMBOL_GPL(rockchip_pmu_unblock); + +#define DOMAIN_RK3588(name, p_offset, pwr, status, m_offset, m_status, r_status, r_offset, req, idle, wakeup) \ + DOMAIN_M_O_R(name, p_offset, pwr, status, m_offset, m_status, r_status, r_offset, req, idle, idle, wakeup) + +static bool rockchip_pmu_domain_is_idle(struct rockchip_pm_domain *pd) +{ + struct rockchip_pmu *pmu = pd->pmu; + const struct rockchip_domain_info *pd_info = pd->info; + unsigned int val; + + regmap_read(pmu->regmap, pmu->info->idle_offset, &val); + return (val & pd_info->idle_mask) == pd_info->idle_mask; +} + +static unsigned int rockchip_pmu_read_ack(struct rockchip_pmu *pmu) +{ + unsigned int val; + + regmap_read(pmu->regmap, pmu->info->ack_offset, &val); + return val; +} + +static int rockchip_pmu_set_idle_request(struct rockchip_pm_domain *pd, + bool idle) +{ + const struct rockchip_domain_info *pd_info = pd->info; + struct generic_pm_domain *genpd = &pd->genpd; + struct rockchip_pmu *pmu = pd->pmu; + u32 pd_req_offset = pd_info->req_offset; + unsigned int target_ack; + unsigned int val; + bool is_idle; + int ret; + + if (pd_info->req_mask == 0) + return 0; + else if (pd_info->req_w_mask) + regmap_write(pmu->regmap, pmu->info->req_offset + pd_req_offset, + idle ? (pd_info->req_mask | pd_info->req_w_mask) : + pd_info->req_w_mask); + else + regmap_update_bits(pmu->regmap, pmu->info->req_offset + pd_req_offset, + pd_info->req_mask, idle ? -1U : 0); + + wmb(); + + /* Wait util idle_ack = 1 */ + target_ack = idle ? pd_info->ack_mask : 0; + ret = readx_poll_timeout_atomic(rockchip_pmu_read_ack, pmu, val, + (val & pd_info->ack_mask) == target_ack, + 0, 10000); + if (ret) { + dev_err(pmu->dev, + "failed to get ack on domain '%s', val=0x%x\n", + genpd->name, val); + return ret; + } + + ret = readx_poll_timeout_atomic(rockchip_pmu_domain_is_idle, pd, + is_idle, is_idle == idle, 0, 10000); + if (ret) { + dev_err(pmu->dev, + "failed to set idle on domain '%s', val=%d\n", + genpd->name, is_idle); + return ret; + } + + return 0; +} + +static int rockchip_pmu_save_qos(struct rockchip_pm_domain *pd) +{ + int i; + + for (i = 0; i < pd->num_qos; i++) { + regmap_read(pd->qos_regmap[i], + QOS_PRIORITY, + &pd->qos_save_regs[0][i]); + regmap_read(pd->qos_regmap[i], + QOS_MODE, + &pd->qos_save_regs[1][i]); + regmap_read(pd->qos_regmap[i], + QOS_BANDWIDTH, + &pd->qos_save_regs[2][i]); + regmap_read(pd->qos_regmap[i], + QOS_SATURATION, + &pd->qos_save_regs[3][i]); + regmap_read(pd->qos_regmap[i], + QOS_EXTCONTROL, + &pd->qos_save_regs[4][i]); + } + return 0; +} + +static int rockchip_pmu_restore_qos(struct rockchip_pm_domain *pd) +{ + int i; + + for (i = 0; i < pd->num_qos; i++) { + regmap_write(pd->qos_regmap[i], + QOS_PRIORITY, + pd->qos_save_regs[0][i]); + regmap_write(pd->qos_regmap[i], + QOS_MODE, + pd->qos_save_regs[1][i]); + regmap_write(pd->qos_regmap[i], + QOS_BANDWIDTH, + pd->qos_save_regs[2][i]); + regmap_write(pd->qos_regmap[i], + QOS_SATURATION, + pd->qos_save_regs[3][i]); + regmap_write(pd->qos_regmap[i], + QOS_EXTCONTROL, + pd->qos_save_regs[4][i]); + } + + return 0; +} + +static bool rockchip_pmu_domain_is_on(struct rockchip_pm_domain *pd) +{ + struct rockchip_pmu *pmu = pd->pmu; + unsigned int val; + + if (pd->info->repair_status_mask) { + regmap_read(pmu->regmap, pmu->info->repair_status_offset, &val); + /* 1'b1: power on, 1'b0: power off */ + return val & pd->info->repair_status_mask; + } + + /* check idle status for idle-only domains */ + if (pd->info->status_mask == 0) + return !rockchip_pmu_domain_is_idle(pd); + + regmap_read(pmu->regmap, pmu->info->status_offset, &val); + + /* 1'b0: power on, 1'b1: power off */ + return !(val & pd->info->status_mask); +} + +static bool rockchip_pmu_domain_is_mem_on(struct rockchip_pm_domain *pd) +{ + struct rockchip_pmu *pmu = pd->pmu; + unsigned int val; + + regmap_read(pmu->regmap, + pmu->info->mem_status_offset + pd->info->mem_offset, &val); + + /* 1'b0: power on, 1'b1: power off */ + return !(val & pd->info->mem_status_mask); +} + +static bool rockchip_pmu_domain_is_chain_on(struct rockchip_pm_domain *pd) +{ + struct rockchip_pmu *pmu = pd->pmu; + unsigned int val; + + regmap_read(pmu->regmap, + pmu->info->chain_status_offset + pd->info->mem_offset, &val); + + /* 1'b1: power on, 1'b0: power off */ + return val & pd->info->mem_status_mask; +} + +static int rockchip_pmu_domain_mem_reset(struct rockchip_pm_domain *pd) +{ + struct rockchip_pmu *pmu = pd->pmu; + struct generic_pm_domain *genpd = &pd->genpd; + bool is_on; + int ret = 0; + + ret = readx_poll_timeout_atomic(rockchip_pmu_domain_is_chain_on, pd, is_on, + is_on == true, 0, 10000); + if (ret) { + dev_err(pmu->dev, + "failed to get chain status '%s', target_on=1, val=%d\n", + genpd->name, is_on); + goto error; + } + + udelay(20); + + regmap_write(pmu->regmap, pmu->info->mem_pwr_offset + pd->info->pwr_offset, + (pd->info->pwr_mask | pd->info->pwr_w_mask)); + wmb(); + + ret = readx_poll_timeout_atomic(rockchip_pmu_domain_is_mem_on, pd, is_on, + is_on == false, 0, 10000); + if (ret) { + dev_err(pmu->dev, + "failed to get mem status '%s', target_on=0, val=%d\n", + genpd->name, is_on); + goto error; + } + + regmap_write(pmu->regmap, pmu->info->mem_pwr_offset + pd->info->pwr_offset, + pd->info->pwr_w_mask); + wmb(); + + ret = readx_poll_timeout_atomic(rockchip_pmu_domain_is_mem_on, pd, is_on, + is_on == true, 0, 10000); + if (ret) { + dev_err(pmu->dev, + "failed to get mem status '%s', target_on=1, val=%d\n", + genpd->name, is_on); + } + +error: + return ret; +} + +static void rockchip_do_pmu_set_power_domain(struct rockchip_pm_domain *pd, + bool on) +{ + struct rockchip_pmu *pmu = pd->pmu; + struct generic_pm_domain *genpd = &pd->genpd; + u32 pd_pwr_offset = pd->info->pwr_offset; + bool is_on, is_mem_on = false; + + if (pd->info->pwr_mask == 0) + return; + + if (on && pd->info->mem_status_mask) + is_mem_on = rockchip_pmu_domain_is_mem_on(pd); + + if (pd->info->pwr_w_mask) + regmap_write(pmu->regmap, pmu->info->pwr_offset + pd_pwr_offset, + on ? pd->info->pwr_w_mask : + (pd->info->pwr_mask | pd->info->pwr_w_mask)); + else + regmap_update_bits(pmu->regmap, pmu->info->pwr_offset + pd_pwr_offset, + pd->info->pwr_mask, on ? 0 : -1U); + + wmb(); + + if (is_mem_on && rockchip_pmu_domain_mem_reset(pd)) + return; + + if (readx_poll_timeout_atomic(rockchip_pmu_domain_is_on, pd, is_on, + is_on == on, 0, 10000)) { + dev_err(pmu->dev, + "failed to set domain '%s', val=%d\n", + genpd->name, is_on); + return; + } +} + +static int rockchip_pd_power(struct rockchip_pm_domain *pd, bool power_on) +{ + struct rockchip_pmu *pmu = pd->pmu; + int ret; + + mutex_lock(&pmu->mutex); + + if (rockchip_pmu_domain_is_on(pd) != power_on) { + ret = clk_bulk_enable(pd->num_clks, pd->clks); + if (ret < 0) { + dev_err(pmu->dev, "failed to enable clocks\n"); + mutex_unlock(&pmu->mutex); + return ret; + } + + if (!power_on) { + rockchip_pmu_save_qos(pd); + + /* if powering down, idle request to NIU first */ + rockchip_pmu_set_idle_request(pd, true); + } + + rockchip_do_pmu_set_power_domain(pd, power_on); + + if (power_on) { + /* if powering up, leave idle mode */ + rockchip_pmu_set_idle_request(pd, false); + + rockchip_pmu_restore_qos(pd); + } + + clk_bulk_disable(pd->num_clks, pd->clks); + } + + mutex_unlock(&pmu->mutex); + return 0; +} + +static int rockchip_pd_power_on(struct generic_pm_domain *domain) +{ + struct rockchip_pm_domain *pd = to_rockchip_pd(domain); + + return rockchip_pd_power(pd, true); +} + +static int rockchip_pd_power_off(struct generic_pm_domain *domain) +{ + struct rockchip_pm_domain *pd = to_rockchip_pd(domain); + + return rockchip_pd_power(pd, false); +} + +static int rockchip_pd_attach_dev(struct generic_pm_domain *genpd, + struct device *dev) +{ + struct clk *clk; + int i; + int error; + + dev_dbg(dev, "attaching to power domain '%s'\n", genpd->name); + + error = pm_clk_create(dev); + if (error) { + dev_err(dev, "pm_clk_create failed %d\n", error); + return error; + } + + i = 0; + while ((clk = of_clk_get(dev->of_node, i++)) && !IS_ERR(clk)) { + dev_dbg(dev, "adding clock '%pC' to list of PM clocks\n", clk); + error = pm_clk_add_clk(dev, clk); + if (error) { + dev_err(dev, "pm_clk_add_clk failed %d\n", error); + clk_put(clk); + pm_clk_destroy(dev); + return error; + } + } + + return 0; +} + +static void rockchip_pd_detach_dev(struct generic_pm_domain *genpd, + struct device *dev) +{ + dev_dbg(dev, "detaching from power domain '%s'\n", genpd->name); + + pm_clk_destroy(dev); +} + +static int rockchip_pm_add_one_domain(struct rockchip_pmu *pmu, + struct device_node *node) +{ + const struct rockchip_domain_info *pd_info; + struct rockchip_pm_domain *pd; + struct device_node *qos_node; + int i, j; + u32 id; + int error; + + error = of_property_read_u32(node, "reg", &id); + if (error) { + dev_err(pmu->dev, + "%pOFn: failed to retrieve domain id (reg): %d\n", + node, error); + return -EINVAL; + } + + if (id >= pmu->info->num_domains) { + dev_err(pmu->dev, "%pOFn: invalid domain id %d\n", + node, id); + return -EINVAL; + } + /* RK3588 has domains with two parents (RKVDEC0/RKVDEC1) */ + if (pmu->genpd_data.domains[id]) + return 0; + + pd_info = &pmu->info->domain_info[id]; + if (!pd_info) { + dev_err(pmu->dev, "%pOFn: undefined domain id %d\n", + node, id); + return -EINVAL; + } + + pd = devm_kzalloc(pmu->dev, sizeof(*pd), GFP_KERNEL); + if (!pd) + return -ENOMEM; + + pd->info = pd_info; + pd->pmu = pmu; + + pd->num_clks = of_clk_get_parent_count(node); + if (pd->num_clks > 0) { + pd->clks = devm_kcalloc(pmu->dev, pd->num_clks, + sizeof(*pd->clks), GFP_KERNEL); + if (!pd->clks) + return -ENOMEM; + } else { + dev_dbg(pmu->dev, "%pOFn: doesn't have clocks: %d\n", + node, pd->num_clks); + pd->num_clks = 0; + } + + for (i = 0; i < pd->num_clks; i++) { + pd->clks[i].clk = of_clk_get(node, i); + if (IS_ERR(pd->clks[i].clk)) { + error = PTR_ERR(pd->clks[i].clk); + dev_err(pmu->dev, + "%pOFn: failed to get clk at index %d: %d\n", + node, i, error); + return error; + } + } + + error = clk_bulk_prepare(pd->num_clks, pd->clks); + if (error) + goto err_put_clocks; + + pd->num_qos = of_count_phandle_with_args(node, "pm_qos", + NULL); + + if (pd->num_qos > 0) { + pd->qos_regmap = devm_kcalloc(pmu->dev, pd->num_qos, + sizeof(*pd->qos_regmap), + GFP_KERNEL); + if (!pd->qos_regmap) { + error = -ENOMEM; + goto err_unprepare_clocks; + } + + for (j = 0; j < MAX_QOS_REGS_NUM; j++) { + pd->qos_save_regs[j] = devm_kcalloc(pmu->dev, + pd->num_qos, + sizeof(u32), + GFP_KERNEL); + if (!pd->qos_save_regs[j]) { + error = -ENOMEM; + goto err_unprepare_clocks; + } + } + + for (j = 0; j < pd->num_qos; j++) { + qos_node = of_parse_phandle(node, "pm_qos", j); + if (!qos_node) { + error = -ENODEV; + goto err_unprepare_clocks; + } + pd->qos_regmap[j] = syscon_node_to_regmap(qos_node); + if (IS_ERR(pd->qos_regmap[j])) { + error = -ENODEV; + of_node_put(qos_node); + goto err_unprepare_clocks; + } + of_node_put(qos_node); + } + } + + if (pd->info->name) + pd->genpd.name = pd->info->name; + else + pd->genpd.name = kbasename(node->full_name); + pd->genpd.power_off = rockchip_pd_power_off; + pd->genpd.power_on = rockchip_pd_power_on; + pd->genpd.attach_dev = rockchip_pd_attach_dev; + pd->genpd.detach_dev = rockchip_pd_detach_dev; + pd->genpd.flags = GENPD_FLAG_PM_CLK; + if (pd_info->active_wakeup) + pd->genpd.flags |= GENPD_FLAG_ACTIVE_WAKEUP; + pm_genpd_init(&pd->genpd, NULL, + !rockchip_pmu_domain_is_on(pd) || + (pd->info->mem_status_mask && !rockchip_pmu_domain_is_mem_on(pd))); + + pmu->genpd_data.domains[id] = &pd->genpd; + return 0; + +err_unprepare_clocks: + clk_bulk_unprepare(pd->num_clks, pd->clks); +err_put_clocks: + clk_bulk_put(pd->num_clks, pd->clks); + return error; +} + +static void rockchip_pm_remove_one_domain(struct rockchip_pm_domain *pd) +{ + int ret; + + /* + * We're in the error cleanup already, so we only complain, + * but won't emit another error on top of the original one. + */ + ret = pm_genpd_remove(&pd->genpd); + if (ret < 0) + dev_err(pd->pmu->dev, "failed to remove domain '%s' : %d - state may be inconsistent\n", + pd->genpd.name, ret); + + clk_bulk_unprepare(pd->num_clks, pd->clks); + clk_bulk_put(pd->num_clks, pd->clks); + + /* protect the zeroing of pm->num_clks */ + mutex_lock(&pd->pmu->mutex); + pd->num_clks = 0; + mutex_unlock(&pd->pmu->mutex); + + /* devm will free our memory */ +} + +static void rockchip_pm_domain_cleanup(struct rockchip_pmu *pmu) +{ + struct generic_pm_domain *genpd; + struct rockchip_pm_domain *pd; + int i; + + for (i = 0; i < pmu->genpd_data.num_domains; i++) { + genpd = pmu->genpd_data.domains[i]; + if (genpd) { + pd = to_rockchip_pd(genpd); + rockchip_pm_remove_one_domain(pd); + } + } + + /* devm will free our memory */ +} + +static void rockchip_configure_pd_cnt(struct rockchip_pmu *pmu, + u32 domain_reg_offset, + unsigned int count) +{ + /* First configure domain power down transition count ... */ + regmap_write(pmu->regmap, domain_reg_offset, count); + /* ... and then power up count. */ + regmap_write(pmu->regmap, domain_reg_offset + 4, count); +} + +static int rockchip_pm_add_subdomain(struct rockchip_pmu *pmu, + struct device_node *parent) +{ + struct device_node *np; + struct generic_pm_domain *child_domain, *parent_domain; + int error; + + for_each_child_of_node(parent, np) { + u32 idx; + + error = of_property_read_u32(parent, "reg", &idx); + if (error) { + dev_err(pmu->dev, + "%pOFn: failed to retrieve domain id (reg): %d\n", + parent, error); + goto err_out; + } + parent_domain = pmu->genpd_data.domains[idx]; + + error = rockchip_pm_add_one_domain(pmu, np); + if (error) { + dev_err(pmu->dev, "failed to handle node %pOFn: %d\n", + np, error); + goto err_out; + } + + error = of_property_read_u32(np, "reg", &idx); + if (error) { + dev_err(pmu->dev, + "%pOFn: failed to retrieve domain id (reg): %d\n", + np, error); + goto err_out; + } + child_domain = pmu->genpd_data.domains[idx]; + + error = pm_genpd_add_subdomain(parent_domain, child_domain); + if (error) { + dev_err(pmu->dev, "%s failed to add subdomain %s: %d\n", + parent_domain->name, child_domain->name, error); + goto err_out; + } else { + dev_dbg(pmu->dev, "%s add subdomain: %s\n", + parent_domain->name, child_domain->name); + } + + rockchip_pm_add_subdomain(pmu, np); + } + + return 0; + +err_out: + of_node_put(np); + return error; +} + +static int rockchip_pm_domain_probe(struct platform_device *pdev) +{ + struct device *dev = &pdev->dev; + struct device_node *np = dev->of_node; + struct device_node *node; + struct device *parent; + struct rockchip_pmu *pmu; + const struct of_device_id *match; + const struct rockchip_pmu_info *pmu_info; + int error; + + if (!np) { + dev_err(dev, "device tree node not found\n"); + return -ENODEV; + } + + match = of_match_device(dev->driver->of_match_table, dev); + if (!match || !match->data) { + dev_err(dev, "missing pmu data\n"); + return -EINVAL; + } + + pmu_info = match->data; + + pmu = devm_kzalloc(dev, + struct_size(pmu, domains, pmu_info->num_domains), + GFP_KERNEL); + if (!pmu) + return -ENOMEM; + + pmu->dev = &pdev->dev; + mutex_init(&pmu->mutex); + + pmu->info = pmu_info; + + pmu->genpd_data.domains = pmu->domains; + pmu->genpd_data.num_domains = pmu_info->num_domains; + + parent = dev->parent; + if (!parent) { + dev_err(dev, "no parent for syscon devices\n"); + return -ENODEV; + } + + pmu->regmap = syscon_node_to_regmap(parent->of_node); + if (IS_ERR(pmu->regmap)) { + dev_err(dev, "no regmap available\n"); + return PTR_ERR(pmu->regmap); + } + + /* + * Configure power up and down transition delays for CORE + * and GPU domains. + */ + if (pmu_info->core_power_transition_time) + rockchip_configure_pd_cnt(pmu, pmu_info->core_pwrcnt_offset, + pmu_info->core_power_transition_time); + if (pmu_info->gpu_pwrcnt_offset) + rockchip_configure_pd_cnt(pmu, pmu_info->gpu_pwrcnt_offset, + pmu_info->gpu_power_transition_time); + + error = -ENODEV; + + /* + * Prevent any rockchip_pmu_block() from racing with the remainder of + * setup (clocks, register initialization). + */ + mutex_lock(&dmc_pmu_mutex); + + for_each_available_child_of_node(np, node) { + error = rockchip_pm_add_one_domain(pmu, node); + if (error) { + dev_err(dev, "failed to handle node %pOFn: %d\n", + node, error); + of_node_put(node); + goto err_out; + } + + error = rockchip_pm_add_subdomain(pmu, node); + if (error < 0) { + dev_err(dev, "failed to handle subdomain node %pOFn: %d\n", + node, error); + of_node_put(node); + goto err_out; + } + } + + if (error) { + dev_dbg(dev, "no power domains defined\n"); + goto err_out; + } + + error = of_genpd_add_provider_onecell(np, &pmu->genpd_data); + if (error) { + dev_err(dev, "failed to add provider: %d\n", error); + goto err_out; + } + + /* We only expect one PMU. */ + if (!WARN_ON_ONCE(dmc_pmu)) + dmc_pmu = pmu; + + mutex_unlock(&dmc_pmu_mutex); + + return 0; + +err_out: + rockchip_pm_domain_cleanup(pmu); + mutex_unlock(&dmc_pmu_mutex); + return error; +} + +static const struct rockchip_domain_info px30_pm_domains[] = { + [PX30_PD_USB] = DOMAIN_PX30("usb", BIT(5), BIT(5), BIT(10), false), + [PX30_PD_SDCARD] = DOMAIN_PX30("sdcard", BIT(8), BIT(8), BIT(9), false), + [PX30_PD_GMAC] = DOMAIN_PX30("gmac", BIT(10), BIT(10), BIT(6), false), + [PX30_PD_MMC_NAND] = DOMAIN_PX30("mmc_nand", BIT(11), BIT(11), BIT(5), false), + [PX30_PD_VPU] = DOMAIN_PX30("vpu", BIT(12), BIT(12), BIT(14), false), + [PX30_PD_VO] = DOMAIN_PX30("vo", BIT(13), BIT(13), BIT(7), false), + [PX30_PD_VI] = DOMAIN_PX30("vi", BIT(14), BIT(14), BIT(8), false), + [PX30_PD_GPU] = DOMAIN_PX30("gpu", BIT(15), BIT(15), BIT(2), false), +}; + +static const struct rockchip_domain_info rv1126_pm_domains[] = { + [RV1126_PD_VEPU] = DOMAIN_RV1126("vepu", BIT(2), BIT(9), BIT(9), false), + [RV1126_PD_VI] = DOMAIN_RV1126("vi", BIT(4), BIT(6), BIT(6), false), + [RV1126_PD_VO] = DOMAIN_RV1126("vo", BIT(5), BIT(7), BIT(7), false), + [RV1126_PD_ISPP] = DOMAIN_RV1126("ispp", BIT(1), BIT(8), BIT(8), false), + [RV1126_PD_VDPU] = DOMAIN_RV1126("vdpu", BIT(3), BIT(10), BIT(10), false), + [RV1126_PD_NVM] = DOMAIN_RV1126("nvm", BIT(7), BIT(11), BIT(11), false), + [RV1126_PD_SDIO] = DOMAIN_RV1126("sdio", BIT(8), BIT(13), BIT(13), false), + [RV1126_PD_USB] = DOMAIN_RV1126("usb", BIT(9), BIT(15), BIT(15), false), +}; + +static const struct rockchip_domain_info rk3036_pm_domains[] = { + [RK3036_PD_MSCH] = DOMAIN_RK3036("msch", BIT(14), BIT(23), BIT(30), true), + [RK3036_PD_CORE] = DOMAIN_RK3036("core", BIT(13), BIT(17), BIT(24), false), + [RK3036_PD_PERI] = DOMAIN_RK3036("peri", BIT(12), BIT(18), BIT(25), false), + [RK3036_PD_VIO] = DOMAIN_RK3036("vio", BIT(11), BIT(19), BIT(26), false), + [RK3036_PD_VPU] = DOMAIN_RK3036("vpu", BIT(10), BIT(20), BIT(27), false), + [RK3036_PD_GPU] = DOMAIN_RK3036("gpu", BIT(9), BIT(21), BIT(28), false), + [RK3036_PD_SYS] = DOMAIN_RK3036("sys", BIT(8), BIT(22), BIT(29), false), +}; + +static const struct rockchip_domain_info rk3066_pm_domains[] = { + [RK3066_PD_GPU] = DOMAIN("gpu", BIT(9), BIT(9), BIT(3), BIT(24), BIT(29), false), + [RK3066_PD_VIDEO] = DOMAIN("video", BIT(8), BIT(8), BIT(4), BIT(23), BIT(28), false), + [RK3066_PD_VIO] = DOMAIN("vio", BIT(7), BIT(7), BIT(5), BIT(22), BIT(27), false), + [RK3066_PD_PERI] = DOMAIN("peri", BIT(6), BIT(6), BIT(2), BIT(25), BIT(30), false), + [RK3066_PD_CPU] = DOMAIN("cpu", 0, BIT(5), BIT(1), BIT(26), BIT(31), false), +}; + +static const struct rockchip_domain_info rk3128_pm_domains[] = { + [RK3128_PD_CORE] = DOMAIN_RK3288("core", BIT(0), BIT(0), BIT(4), false), + [RK3128_PD_MSCH] = DOMAIN_RK3288("msch", 0, 0, BIT(6), true), + [RK3128_PD_VIO] = DOMAIN_RK3288("vio", BIT(3), BIT(3), BIT(2), false), + [RK3128_PD_VIDEO] = DOMAIN_RK3288("video", BIT(2), BIT(2), BIT(1), false), + [RK3128_PD_GPU] = DOMAIN_RK3288("gpu", BIT(1), BIT(1), BIT(3), false), +}; + +static const struct rockchip_domain_info rk3188_pm_domains[] = { + [RK3188_PD_GPU] = DOMAIN("gpu", BIT(9), BIT(9), BIT(3), BIT(24), BIT(29), false), + [RK3188_PD_VIDEO] = DOMAIN("video", BIT(8), BIT(8), BIT(4), BIT(23), BIT(28), false), + [RK3188_PD_VIO] = DOMAIN("vio", BIT(7), BIT(7), BIT(5), BIT(22), BIT(27), false), + [RK3188_PD_PERI] = DOMAIN("peri", BIT(6), BIT(6), BIT(2), BIT(25), BIT(30), false), + [RK3188_PD_CPU] = DOMAIN("cpu", BIT(5), BIT(5), BIT(1), BIT(26), BIT(31), false), +}; + +static const struct rockchip_domain_info rk3228_pm_domains[] = { + [RK3228_PD_CORE] = DOMAIN_RK3036("core", BIT(0), BIT(0), BIT(16), true), + [RK3228_PD_MSCH] = DOMAIN_RK3036("msch", BIT(1), BIT(1), BIT(17), true), + [RK3228_PD_BUS] = DOMAIN_RK3036("bus", BIT(2), BIT(2), BIT(18), true), + [RK3228_PD_SYS] = DOMAIN_RK3036("sys", BIT(3), BIT(3), BIT(19), true), + [RK3228_PD_VIO] = DOMAIN_RK3036("vio", BIT(4), BIT(4), BIT(20), false), + [RK3228_PD_VOP] = DOMAIN_RK3036("vop", BIT(5), BIT(5), BIT(21), false), + [RK3228_PD_VPU] = DOMAIN_RK3036("vpu", BIT(6), BIT(6), BIT(22), false), + [RK3228_PD_RKVDEC] = DOMAIN_RK3036("vdec", BIT(7), BIT(7), BIT(23), false), + [RK3228_PD_GPU] = DOMAIN_RK3036("gpu", BIT(8), BIT(8), BIT(24), false), + [RK3228_PD_PERI] = DOMAIN_RK3036("peri", BIT(9), BIT(9), BIT(25), true), + [RK3228_PD_GMAC] = DOMAIN_RK3036("gmac", BIT(10), BIT(10), BIT(26), false), +}; + +static const struct rockchip_domain_info rk3288_pm_domains[] = { + [RK3288_PD_VIO] = DOMAIN_RK3288("vio", BIT(7), BIT(7), BIT(4), false), + [RK3288_PD_HEVC] = DOMAIN_RK3288("hevc", BIT(14), BIT(10), BIT(9), false), + [RK3288_PD_VIDEO] = DOMAIN_RK3288("video", BIT(8), BIT(8), BIT(3), false), + [RK3288_PD_GPU] = DOMAIN_RK3288("gpu", BIT(9), BIT(9), BIT(2), false), +}; + +static const struct rockchip_domain_info rk3328_pm_domains[] = { + [RK3328_PD_CORE] = DOMAIN_RK3328("core", 0, BIT(0), BIT(0), false), + [RK3328_PD_GPU] = DOMAIN_RK3328("gpu", 0, BIT(1), BIT(1), false), + [RK3328_PD_BUS] = DOMAIN_RK3328("bus", 0, BIT(2), BIT(2), true), + [RK3328_PD_MSCH] = DOMAIN_RK3328("msch", 0, BIT(3), BIT(3), true), + [RK3328_PD_PERI] = DOMAIN_RK3328("peri", 0, BIT(4), BIT(4), true), + [RK3328_PD_VIDEO] = DOMAIN_RK3328("video", 0, BIT(5), BIT(5), false), + [RK3328_PD_HEVC] = DOMAIN_RK3328("hevc", 0, BIT(6), BIT(6), false), + [RK3328_PD_VIO] = DOMAIN_RK3328("vio", 0, BIT(8), BIT(8), false), + [RK3328_PD_VPU] = DOMAIN_RK3328("vpu", 0, BIT(9), BIT(9), false), +}; + +static const struct rockchip_domain_info rk3366_pm_domains[] = { + [RK3366_PD_PERI] = DOMAIN_RK3368("peri", BIT(10), BIT(10), BIT(6), true), + [RK3366_PD_VIO] = DOMAIN_RK3368("vio", BIT(14), BIT(14), BIT(8), false), + [RK3366_PD_VIDEO] = DOMAIN_RK3368("video", BIT(13), BIT(13), BIT(7), false), + [RK3366_PD_RKVDEC] = DOMAIN_RK3368("vdec", BIT(11), BIT(11), BIT(7), false), + [RK3366_PD_WIFIBT] = DOMAIN_RK3368("wifibt", BIT(8), BIT(8), BIT(9), false), + [RK3366_PD_VPU] = DOMAIN_RK3368("vpu", BIT(12), BIT(12), BIT(7), false), + [RK3366_PD_GPU] = DOMAIN_RK3368("gpu", BIT(15), BIT(15), BIT(2), false), +}; + +static const struct rockchip_domain_info rk3368_pm_domains[] = { + [RK3368_PD_PERI] = DOMAIN_RK3368("peri", BIT(13), BIT(12), BIT(6), true), + [RK3368_PD_VIO] = DOMAIN_RK3368("vio", BIT(15), BIT(14), BIT(8), false), + [RK3368_PD_VIDEO] = DOMAIN_RK3368("video", BIT(14), BIT(13), BIT(7), false), + [RK3368_PD_GPU_0] = DOMAIN_RK3368("gpu_0", BIT(16), BIT(15), BIT(2), false), + [RK3368_PD_GPU_1] = DOMAIN_RK3368("gpu_1", BIT(17), BIT(16), BIT(2), false), +}; + +static const struct rockchip_domain_info rk3399_pm_domains[] = { + [RK3399_PD_TCPD0] = DOMAIN_RK3399("tcpd0", BIT(8), BIT(8), 0, false), + [RK3399_PD_TCPD1] = DOMAIN_RK3399("tcpd1", BIT(9), BIT(9), 0, false), + [RK3399_PD_CCI] = DOMAIN_RK3399("cci", BIT(10), BIT(10), 0, true), + [RK3399_PD_CCI0] = DOMAIN_RK3399("cci0", 0, 0, BIT(15), true), + [RK3399_PD_CCI1] = DOMAIN_RK3399("cci1", 0, 0, BIT(16), true), + [RK3399_PD_PERILP] = DOMAIN_RK3399("perilp", BIT(11), BIT(11), BIT(1), true), + [RK3399_PD_PERIHP] = DOMAIN_RK3399("perihp", BIT(12), BIT(12), BIT(2), true), + [RK3399_PD_CENTER] = DOMAIN_RK3399("center", BIT(13), BIT(13), BIT(14), true), + [RK3399_PD_VIO] = DOMAIN_RK3399("vio", BIT(14), BIT(14), BIT(17), false), + [RK3399_PD_GPU] = DOMAIN_RK3399("gpu", BIT(15), BIT(15), BIT(0), false), + [RK3399_PD_VCODEC] = DOMAIN_RK3399("vcodec", BIT(16), BIT(16), BIT(3), false), + [RK3399_PD_VDU] = DOMAIN_RK3399("vdu", BIT(17), BIT(17), BIT(4), false), + [RK3399_PD_RGA] = DOMAIN_RK3399("rga", BIT(18), BIT(18), BIT(5), false), + [RK3399_PD_IEP] = DOMAIN_RK3399("iep", BIT(19), BIT(19), BIT(6), false), + [RK3399_PD_VO] = DOMAIN_RK3399("vo", BIT(20), BIT(20), 0, false), + [RK3399_PD_VOPB] = DOMAIN_RK3399("vopb", 0, 0, BIT(7), false), + [RK3399_PD_VOPL] = DOMAIN_RK3399("vopl", 0, 0, BIT(8), false), + [RK3399_PD_ISP0] = DOMAIN_RK3399("isp0", BIT(22), BIT(22), BIT(9), false), + [RK3399_PD_ISP1] = DOMAIN_RK3399("isp1", BIT(23), BIT(23), BIT(10), false), + [RK3399_PD_HDCP] = DOMAIN_RK3399("hdcp", BIT(24), BIT(24), BIT(11), false), + [RK3399_PD_GMAC] = DOMAIN_RK3399("gmac", BIT(25), BIT(25), BIT(23), true), + [RK3399_PD_EMMC] = DOMAIN_RK3399("emmc", BIT(26), BIT(26), BIT(24), true), + [RK3399_PD_USB3] = DOMAIN_RK3399("usb3", BIT(27), BIT(27), BIT(12), true), + [RK3399_PD_EDP] = DOMAIN_RK3399("edp", BIT(28), BIT(28), BIT(22), false), + [RK3399_PD_GIC] = DOMAIN_RK3399("gic", BIT(29), BIT(29), BIT(27), true), + [RK3399_PD_SD] = DOMAIN_RK3399("sd", BIT(30), BIT(30), BIT(28), true), + [RK3399_PD_SDIOAUDIO] = DOMAIN_RK3399("sdioaudio", BIT(31), BIT(31), BIT(29), true), +}; + +static const struct rockchip_domain_info rk3568_pm_domains[] = { + [RK3568_PD_NPU] = DOMAIN_RK3568("npu", BIT(1), BIT(2), false), + [RK3568_PD_GPU] = DOMAIN_RK3568("gpu", BIT(0), BIT(1), false), + [RK3568_PD_VI] = DOMAIN_RK3568("vi", BIT(6), BIT(3), false), + [RK3568_PD_VO] = DOMAIN_RK3568("vo", BIT(7), BIT(4), false), + [RK3568_PD_RGA] = DOMAIN_RK3568("rga", BIT(5), BIT(5), false), + [RK3568_PD_VPU] = DOMAIN_RK3568("vpu", BIT(2), BIT(6), false), + [RK3568_PD_RKVDEC] = DOMAIN_RK3568("vdec", BIT(4), BIT(8), false), + [RK3568_PD_RKVENC] = DOMAIN_RK3568("venc", BIT(3), BIT(7), false), + [RK3568_PD_PIPE] = DOMAIN_RK3568("pipe", BIT(8), BIT(11), false), +}; + +static const struct rockchip_domain_info rk3588_pm_domains[] = { + [RK3588_PD_GPU] = DOMAIN_RK3588("gpu", 0x0, BIT(0), 0, 0x0, 0, BIT(1), 0x0, BIT(0), BIT(0), false), + [RK3588_PD_NPU] = DOMAIN_RK3588("npu", 0x0, BIT(1), BIT(1), 0x0, 0, 0, 0x0, 0, 0, false), + [RK3588_PD_VCODEC] = DOMAIN_RK3588("vcodec", 0x0, BIT(2), BIT(2), 0x0, 0, 0, 0x0, 0, 0, false), + [RK3588_PD_NPUTOP] = DOMAIN_RK3588("nputop", 0x0, BIT(3), 0, 0x0, BIT(11), BIT(2), 0x0, BIT(1), BIT(1), false), + [RK3588_PD_NPU1] = DOMAIN_RK3588("npu1", 0x0, BIT(4), 0, 0x0, BIT(12), BIT(3), 0x0, BIT(2), BIT(2), false), + [RK3588_PD_NPU2] = DOMAIN_RK3588("npu2", 0x0, BIT(5), 0, 0x0, BIT(13), BIT(4), 0x0, BIT(3), BIT(3), false), + [RK3588_PD_VENC0] = DOMAIN_RK3588("venc0", 0x0, BIT(6), 0, 0x0, BIT(14), BIT(5), 0x0, BIT(4), BIT(4), false), + [RK3588_PD_VENC1] = DOMAIN_RK3588("venc1", 0x0, BIT(7), 0, 0x0, BIT(15), BIT(6), 0x0, BIT(5), BIT(5), false), + [RK3588_PD_RKVDEC0] = DOMAIN_RK3588("rkvdec0", 0x0, BIT(8), 0, 0x0, BIT(16), BIT(7), 0x0, BIT(6), BIT(6), false), + [RK3588_PD_RKVDEC1] = DOMAIN_RK3588("rkvdec1", 0x0, BIT(9), 0, 0x0, BIT(17), BIT(8), 0x0, BIT(7), BIT(7), false), + [RK3588_PD_VDPU] = DOMAIN_RK3588("vdpu", 0x0, BIT(10), 0, 0x0, BIT(18), BIT(9), 0x0, BIT(8), BIT(8), false), + [RK3588_PD_RGA30] = DOMAIN_RK3588("rga30", 0x0, BIT(11), 0, 0x0, BIT(19), BIT(10), 0x0, 0, 0, false), + [RK3588_PD_AV1] = DOMAIN_RK3588("av1", 0x0, BIT(12), 0, 0x0, BIT(20), BIT(11), 0x0, BIT(9), BIT(9), false), + [RK3588_PD_VI] = DOMAIN_RK3588("vi", 0x0, BIT(13), 0, 0x0, BIT(21), BIT(12), 0x0, BIT(10), BIT(10), false), + [RK3588_PD_FEC] = DOMAIN_RK3588("fec", 0x0, BIT(14), 0, 0x0, BIT(22), BIT(13), 0x0, 0, 0, false), + [RK3588_PD_ISP1] = DOMAIN_RK3588("isp1", 0x0, BIT(15), 0, 0x0, BIT(23), BIT(14), 0x0, BIT(11), BIT(11), false), + [RK3588_PD_RGA31] = DOMAIN_RK3588("rga31", 0x4, BIT(0), 0, 0x0, BIT(24), BIT(15), 0x0, BIT(12), BIT(12), false), + [RK3588_PD_VOP] = DOMAIN_RK3588("vop", 0x4, BIT(1), 0, 0x0, BIT(25), BIT(16), 0x0, BIT(13) | BIT(14), BIT(13) | BIT(14), false), + [RK3588_PD_VO0] = DOMAIN_RK3588("vo0", 0x4, BIT(2), 0, 0x0, BIT(26), BIT(17), 0x0, BIT(15), BIT(15), false), + [RK3588_PD_VO1] = DOMAIN_RK3588("vo1", 0x4, BIT(3), 0, 0x0, BIT(27), BIT(18), 0x4, BIT(0), BIT(16), false), + [RK3588_PD_AUDIO] = DOMAIN_RK3588("audio", 0x4, BIT(4), 0, 0x0, BIT(28), BIT(19), 0x4, BIT(1), BIT(17), false), + [RK3588_PD_PHP] = DOMAIN_RK3588("php", 0x4, BIT(5), 0, 0x0, BIT(29), BIT(20), 0x4, BIT(5), BIT(21), false), + [RK3588_PD_GMAC] = DOMAIN_RK3588("gmac", 0x4, BIT(6), 0, 0x0, BIT(30), BIT(21), 0x0, 0, 0, false), + [RK3588_PD_PCIE] = DOMAIN_RK3588("pcie", 0x4, BIT(7), 0, 0x0, BIT(31), BIT(22), 0x0, 0, 0, true), + [RK3588_PD_NVM] = DOMAIN_RK3588("nvm", 0x4, BIT(8), BIT(24), 0x4, 0, 0, 0x4, BIT(2), BIT(18), false), + [RK3588_PD_NVM0] = DOMAIN_RK3588("nvm0", 0x4, BIT(9), 0, 0x4, BIT(1), BIT(23), 0x0, 0, 0, false), + [RK3588_PD_SDIO] = DOMAIN_RK3588("sdio", 0x4, BIT(10), 0, 0x4, BIT(2), BIT(24), 0x4, BIT(3), BIT(19), false), + [RK3588_PD_USB] = DOMAIN_RK3588("usb", 0x4, BIT(11), 0, 0x4, BIT(3), BIT(25), 0x4, BIT(4), BIT(20), true), + [RK3588_PD_SDMMC] = DOMAIN_RK3588("sdmmc", 0x4, BIT(13), 0, 0x4, BIT(5), BIT(26), 0x0, 0, 0, false), +}; + +static const struct rockchip_pmu_info px30_pmu = { + .pwr_offset = 0x18, + .status_offset = 0x20, + .req_offset = 0x64, + .idle_offset = 0x6c, + .ack_offset = 0x6c, + + .num_domains = ARRAY_SIZE(px30_pm_domains), + .domain_info = px30_pm_domains, +}; + +static const struct rockchip_pmu_info rk3036_pmu = { + .req_offset = 0x148, + .idle_offset = 0x14c, + .ack_offset = 0x14c, + + .num_domains = ARRAY_SIZE(rk3036_pm_domains), + .domain_info = rk3036_pm_domains, +}; + +static const struct rockchip_pmu_info rk3066_pmu = { + .pwr_offset = 0x08, + .status_offset = 0x0c, + .req_offset = 0x38, /* PMU_MISC_CON1 */ + .idle_offset = 0x0c, + .ack_offset = 0x0c, + + .num_domains = ARRAY_SIZE(rk3066_pm_domains), + .domain_info = rk3066_pm_domains, +}; + +static const struct rockchip_pmu_info rk3128_pmu = { + .pwr_offset = 0x04, + .status_offset = 0x08, + .req_offset = 0x0c, + .idle_offset = 0x10, + .ack_offset = 0x10, + + .num_domains = ARRAY_SIZE(rk3128_pm_domains), + .domain_info = rk3128_pm_domains, +}; + +static const struct rockchip_pmu_info rk3188_pmu = { + .pwr_offset = 0x08, + .status_offset = 0x0c, + .req_offset = 0x38, /* PMU_MISC_CON1 */ + .idle_offset = 0x0c, + .ack_offset = 0x0c, + + .num_domains = ARRAY_SIZE(rk3188_pm_domains), + .domain_info = rk3188_pm_domains, +}; + +static const struct rockchip_pmu_info rk3228_pmu = { + .req_offset = 0x40c, + .idle_offset = 0x488, + .ack_offset = 0x488, + + .num_domains = ARRAY_SIZE(rk3228_pm_domains), + .domain_info = rk3228_pm_domains, +}; + +static const struct rockchip_pmu_info rk3288_pmu = { + .pwr_offset = 0x08, + .status_offset = 0x0c, + .req_offset = 0x10, + .idle_offset = 0x14, + .ack_offset = 0x14, + + .core_pwrcnt_offset = 0x34, + .gpu_pwrcnt_offset = 0x3c, + + .core_power_transition_time = 24, /* 1us */ + .gpu_power_transition_time = 24, /* 1us */ + + .num_domains = ARRAY_SIZE(rk3288_pm_domains), + .domain_info = rk3288_pm_domains, +}; + +static const struct rockchip_pmu_info rk3328_pmu = { + .req_offset = 0x414, + .idle_offset = 0x484, + .ack_offset = 0x484, + + .num_domains = ARRAY_SIZE(rk3328_pm_domains), + .domain_info = rk3328_pm_domains, +}; + +static const struct rockchip_pmu_info rk3366_pmu = { + .pwr_offset = 0x0c, + .status_offset = 0x10, + .req_offset = 0x3c, + .idle_offset = 0x40, + .ack_offset = 0x40, + + .core_pwrcnt_offset = 0x48, + .gpu_pwrcnt_offset = 0x50, + + .core_power_transition_time = 24, + .gpu_power_transition_time = 24, + + .num_domains = ARRAY_SIZE(rk3366_pm_domains), + .domain_info = rk3366_pm_domains, +}; + +static const struct rockchip_pmu_info rk3368_pmu = { + .pwr_offset = 0x0c, + .status_offset = 0x10, + .req_offset = 0x3c, + .idle_offset = 0x40, + .ack_offset = 0x40, + + .core_pwrcnt_offset = 0x48, + .gpu_pwrcnt_offset = 0x50, + + .core_power_transition_time = 24, + .gpu_power_transition_time = 24, + + .num_domains = ARRAY_SIZE(rk3368_pm_domains), + .domain_info = rk3368_pm_domains, +}; + +static const struct rockchip_pmu_info rk3399_pmu = { + .pwr_offset = 0x14, + .status_offset = 0x18, + .req_offset = 0x60, + .idle_offset = 0x64, + .ack_offset = 0x68, + + /* ARM Trusted Firmware manages power transition times */ + + .num_domains = ARRAY_SIZE(rk3399_pm_domains), + .domain_info = rk3399_pm_domains, +}; + +static const struct rockchip_pmu_info rk3568_pmu = { + .pwr_offset = 0xa0, + .status_offset = 0x98, + .req_offset = 0x50, + .idle_offset = 0x68, + .ack_offset = 0x60, + + .num_domains = ARRAY_SIZE(rk3568_pm_domains), + .domain_info = rk3568_pm_domains, +}; + +static const struct rockchip_pmu_info rk3588_pmu = { + .pwr_offset = 0x14c, + .status_offset = 0x180, + .req_offset = 0x10c, + .idle_offset = 0x120, + .ack_offset = 0x118, + .mem_pwr_offset = 0x1a0, + .chain_status_offset = 0x1f0, + .mem_status_offset = 0x1f8, + .repair_status_offset = 0x290, + + .num_domains = ARRAY_SIZE(rk3588_pm_domains), + .domain_info = rk3588_pm_domains, +}; + +static const struct rockchip_pmu_info rv1126_pmu = { + .pwr_offset = 0x110, + .status_offset = 0x108, + .req_offset = 0xc0, + .idle_offset = 0xd8, + .ack_offset = 0xd0, + + .num_domains = ARRAY_SIZE(rv1126_pm_domains), + .domain_info = rv1126_pm_domains, +}; + +static const struct of_device_id rockchip_pm_domain_dt_match[] = { + { + .compatible = "rockchip,px30-power-controller", + .data = (void *)&px30_pmu, + }, + { + .compatible = "rockchip,rk3036-power-controller", + .data = (void *)&rk3036_pmu, + }, + { + .compatible = "rockchip,rk3066-power-controller", + .data = (void *)&rk3066_pmu, + }, + { + .compatible = "rockchip,rk3128-power-controller", + .data = (void *)&rk3128_pmu, + }, + { + .compatible = "rockchip,rk3188-power-controller", + .data = (void *)&rk3188_pmu, + }, + { + .compatible = "rockchip,rk3228-power-controller", + .data = (void *)&rk3228_pmu, + }, + { + .compatible = "rockchip,rk3288-power-controller", + .data = (void *)&rk3288_pmu, + }, + { + .compatible = "rockchip,rk3328-power-controller", + .data = (void *)&rk3328_pmu, + }, + { + .compatible = "rockchip,rk3366-power-controller", + .data = (void *)&rk3366_pmu, + }, + { + .compatible = "rockchip,rk3368-power-controller", + .data = (void *)&rk3368_pmu, + }, + { + .compatible = "rockchip,rk3399-power-controller", + .data = (void *)&rk3399_pmu, + }, + { + .compatible = "rockchip,rk3568-power-controller", + .data = (void *)&rk3568_pmu, + }, + { + .compatible = "rockchip,rk3588-power-controller", + .data = (void *)&rk3588_pmu, + }, + { + .compatible = "rockchip,rv1126-power-controller", + .data = (void *)&rv1126_pmu, + }, + { /* sentinel */ }, +}; + +static struct platform_driver rockchip_pm_domain_driver = { + .probe = rockchip_pm_domain_probe, + .driver = { + .name = "rockchip-pm-domain", + .of_match_table = rockchip_pm_domain_dt_match, + /* + * We can't forcibly eject devices from the power + * domain, so we can't really remove power domains + * once they were added. + */ + .suppress_bind_attrs = true, + }, +}; + +static int __init rockchip_pm_domain_drv_register(void) +{ + return platform_driver_register(&rockchip_pm_domain_driver); +} +postcore_initcall(rockchip_pm_domain_drv_register); diff --git a/drivers/pmdomain/samsung/Makefile b/drivers/pmdomain/samsung/Makefile new file mode 100644 index 000000000000..397aa5908c1d --- /dev/null +++ b/drivers/pmdomain/samsung/Makefile @@ -0,0 +1,2 @@ +# SPDX-License-Identifier: GPL-2.0-only +obj-$(CONFIG_EXYNOS_PM_DOMAINS) += exynos-pm-domains.o diff --git a/drivers/pmdomain/samsung/exynos-pm-domains.c b/drivers/pmdomain/samsung/exynos-pm-domains.c new file mode 100644 index 000000000000..9b502e8751d1 --- /dev/null +++ b/drivers/pmdomain/samsung/exynos-pm-domains.c @@ -0,0 +1,167 @@ +// SPDX-License-Identifier: GPL-2.0 +// +// Exynos Generic power domain support. +// +// Copyright (c) 2012 Samsung Electronics Co., Ltd. +// http://www.samsung.com +// +// Implementation of Exynos specific power domain control which is used in +// conjunction with runtime-pm. Support for both device-tree and non-device-tree +// based power domain support is included. + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +struct exynos_pm_domain_config { + /* Value for LOCAL_PWR_CFG and STATUS fields for each domain */ + u32 local_pwr_cfg; +}; + +/* + * Exynos specific wrapper around the generic power domain + */ +struct exynos_pm_domain { + void __iomem *base; + struct generic_pm_domain pd; + u32 local_pwr_cfg; +}; + +static int exynos_pd_power(struct generic_pm_domain *domain, bool power_on) +{ + struct exynos_pm_domain *pd; + void __iomem *base; + u32 timeout, pwr; + char *op; + + pd = container_of(domain, struct exynos_pm_domain, pd); + base = pd->base; + + pwr = power_on ? pd->local_pwr_cfg : 0; + writel_relaxed(pwr, base); + + /* Wait max 1ms */ + timeout = 10; + + while ((readl_relaxed(base + 0x4) & pd->local_pwr_cfg) != pwr) { + if (!timeout) { + op = (power_on) ? "enable" : "disable"; + pr_err("Power domain %s %s failed\n", domain->name, op); + return -ETIMEDOUT; + } + timeout--; + cpu_relax(); + usleep_range(80, 100); + } + + return 0; +} + +static int exynos_pd_power_on(struct generic_pm_domain *domain) +{ + return exynos_pd_power(domain, true); +} + +static int exynos_pd_power_off(struct generic_pm_domain *domain) +{ + return exynos_pd_power(domain, false); +} + +static const struct exynos_pm_domain_config exynos4210_cfg = { + .local_pwr_cfg = 0x7, +}; + +static const struct exynos_pm_domain_config exynos5433_cfg = { + .local_pwr_cfg = 0xf, +}; + +static const struct of_device_id exynos_pm_domain_of_match[] = { + { + .compatible = "samsung,exynos4210-pd", + .data = &exynos4210_cfg, + }, { + .compatible = "samsung,exynos5433-pd", + .data = &exynos5433_cfg, + }, + { }, +}; + +static const char *exynos_get_domain_name(struct device_node *node) +{ + const char *name; + + if (of_property_read_string(node, "label", &name) < 0) + name = kbasename(node->full_name); + return kstrdup_const(name, GFP_KERNEL); +} + +static int exynos_pd_probe(struct platform_device *pdev) +{ + const struct exynos_pm_domain_config *pm_domain_cfg; + struct device *dev = &pdev->dev; + struct device_node *np = dev->of_node; + struct of_phandle_args child, parent; + struct exynos_pm_domain *pd; + int on, ret; + + pm_domain_cfg = of_device_get_match_data(dev); + pd = devm_kzalloc(dev, sizeof(*pd), GFP_KERNEL); + if (!pd) + return -ENOMEM; + + pd->pd.name = exynos_get_domain_name(np); + if (!pd->pd.name) + return -ENOMEM; + + pd->base = of_iomap(np, 0); + if (!pd->base) { + kfree_const(pd->pd.name); + return -ENODEV; + } + + pd->pd.power_off = exynos_pd_power_off; + pd->pd.power_on = exynos_pd_power_on; + pd->local_pwr_cfg = pm_domain_cfg->local_pwr_cfg; + + on = readl_relaxed(pd->base + 0x4) & pd->local_pwr_cfg; + + pm_genpd_init(&pd->pd, NULL, !on); + ret = of_genpd_add_provider_simple(np, &pd->pd); + + if (ret == 0 && of_parse_phandle_with_args(np, "power-domains", + "#power-domain-cells", 0, &parent) == 0) { + child.np = np; + child.args_count = 0; + + if (of_genpd_add_subdomain(&parent, &child)) + pr_warn("%pOF failed to add subdomain: %pOF\n", + parent.np, child.np); + else + pr_info("%pOF has as child subdomain: %pOF.\n", + parent.np, child.np); + } + + pm_runtime_enable(dev); + return ret; +} + +static struct platform_driver exynos_pd_driver = { + .probe = exynos_pd_probe, + .driver = { + .name = "exynos-pd", + .of_match_table = exynos_pm_domain_of_match, + .suppress_bind_attrs = true, + } +}; + +static __init int exynos4_pm_init_power_domain(void) +{ + return platform_driver_register(&exynos_pd_driver); +} +core_initcall(exynos4_pm_init_power_domain); diff --git a/drivers/pmdomain/st/Makefile b/drivers/pmdomain/st/Makefile new file mode 100644 index 000000000000..8fa5f9855460 --- /dev/null +++ b/drivers/pmdomain/st/Makefile @@ -0,0 +1,2 @@ +# SPDX-License-Identifier: GPL-2.0-only +obj-$(CONFIG_ARCH_U8500) += ste-ux500-pm-domain.o diff --git a/drivers/pmdomain/st/ste-ux500-pm-domain.c b/drivers/pmdomain/st/ste-ux500-pm-domain.c new file mode 100644 index 000000000000..3d4f111ed156 --- /dev/null +++ b/drivers/pmdomain/st/ste-ux500-pm-domain.c @@ -0,0 +1,94 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2014 Linaro Ltd. + * + * Author: Ulf Hansson + * + * Implements PM domains using the generic PM domain for ux500. + */ +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +static int pd_power_off(struct generic_pm_domain *domain) +{ + /* + * Handle the gating of the PM domain regulator here. + * + * Drivers/subsystems handling devices in the PM domain needs to perform + * register context save/restore from their respective runtime PM + * callbacks, to be able to enable PM domain gating/ungating. + */ + return 0; +} + +static int pd_power_on(struct generic_pm_domain *domain) +{ + /* + * Handle the ungating of the PM domain regulator here. + * + * Drivers/subsystems handling devices in the PM domain needs to perform + * register context save/restore from their respective runtime PM + * callbacks, to be able to enable PM domain gating/ungating. + */ + return 0; +} + +static struct generic_pm_domain ux500_pm_domain_vape = { + .name = "VAPE", + .power_off = pd_power_off, + .power_on = pd_power_on, +}; + +static struct generic_pm_domain *ux500_pm_domains[NR_DOMAINS] = { + [DOMAIN_VAPE] = &ux500_pm_domain_vape, +}; + +static const struct of_device_id ux500_pm_domain_matches[] = { + { .compatible = "stericsson,ux500-pm-domains", }, + { }, +}; + +static int ux500_pm_domains_probe(struct platform_device *pdev) +{ + struct device_node *np = pdev->dev.of_node; + struct genpd_onecell_data *genpd_data; + int i; + + if (!np) + return -ENODEV; + + genpd_data = kzalloc(sizeof(*genpd_data), GFP_KERNEL); + if (!genpd_data) + return -ENOMEM; + + genpd_data->domains = ux500_pm_domains; + genpd_data->num_domains = ARRAY_SIZE(ux500_pm_domains); + + for (i = 0; i < ARRAY_SIZE(ux500_pm_domains); ++i) + pm_genpd_init(ux500_pm_domains[i], NULL, false); + + of_genpd_add_provider_onecell(np, genpd_data); + return 0; +} + +static struct platform_driver ux500_pm_domains_driver = { + .probe = ux500_pm_domains_probe, + .driver = { + .name = "ux500_pm_domains", + .of_match_table = ux500_pm_domain_matches, + }, +}; + +static int __init ux500_pm_domains_init(void) +{ + return platform_driver_register(&ux500_pm_domains_driver); +} +arch_initcall(ux500_pm_domains_init); diff --git a/drivers/pmdomain/starfive/Makefile b/drivers/pmdomain/starfive/Makefile new file mode 100644 index 000000000000..975bba2a29a9 --- /dev/null +++ b/drivers/pmdomain/starfive/Makefile @@ -0,0 +1,2 @@ +# SPDX-License-Identifier: GPL-2.0 +obj-$(CONFIG_JH71XX_PMU) += jh71xx-pmu.o diff --git a/drivers/pmdomain/starfive/jh71xx-pmu.c b/drivers/pmdomain/starfive/jh71xx-pmu.c new file mode 100644 index 000000000000..7d5f50d71c0d --- /dev/null +++ b/drivers/pmdomain/starfive/jh71xx-pmu.c @@ -0,0 +1,383 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * StarFive JH71XX PMU (Power Management Unit) Controller Driver + * + * Copyright (C) 2022 StarFive Technology Co., Ltd. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +/* register offset */ +#define JH71XX_PMU_SW_TURN_ON_POWER 0x0C +#define JH71XX_PMU_SW_TURN_OFF_POWER 0x10 +#define JH71XX_PMU_SW_ENCOURAGE 0x44 +#define JH71XX_PMU_TIMER_INT_MASK 0x48 +#define JH71XX_PMU_CURR_POWER_MODE 0x80 +#define JH71XX_PMU_EVENT_STATUS 0x88 +#define JH71XX_PMU_INT_STATUS 0x8C + +/* sw encourage cfg */ +#define JH71XX_PMU_SW_ENCOURAGE_EN_LO 0x05 +#define JH71XX_PMU_SW_ENCOURAGE_EN_HI 0x50 +#define JH71XX_PMU_SW_ENCOURAGE_DIS_LO 0x0A +#define JH71XX_PMU_SW_ENCOURAGE_DIS_HI 0xA0 +#define JH71XX_PMU_SW_ENCOURAGE_ON 0xFF + +/* pmu int status */ +#define JH71XX_PMU_INT_SEQ_DONE BIT(0) +#define JH71XX_PMU_INT_HW_REQ BIT(1) +#define JH71XX_PMU_INT_SW_FAIL GENMASK(3, 2) +#define JH71XX_PMU_INT_HW_FAIL GENMASK(5, 4) +#define JH71XX_PMU_INT_PCH_FAIL GENMASK(8, 6) +#define JH71XX_PMU_INT_ALL_MASK GENMASK(8, 0) + +/* + * The time required for switching power status is based on the time + * to turn on the largest domain's power, which is at microsecond level + */ +#define JH71XX_PMU_TIMEOUT_US 100 + +struct jh71xx_domain_info { + const char * const name; + unsigned int flags; + u8 bit; +}; + +struct jh71xx_pmu_match_data { + const struct jh71xx_domain_info *domain_info; + int num_domains; +}; + +struct jh71xx_pmu { + struct device *dev; + const struct jh71xx_pmu_match_data *match_data; + void __iomem *base; + struct generic_pm_domain **genpd; + struct genpd_onecell_data genpd_data; + int irq; + spinlock_t lock; /* protects pmu reg */ +}; + +struct jh71xx_pmu_dev { + const struct jh71xx_domain_info *domain_info; + struct jh71xx_pmu *pmu; + struct generic_pm_domain genpd; +}; + +static int jh71xx_pmu_get_state(struct jh71xx_pmu_dev *pmd, u32 mask, bool *is_on) +{ + struct jh71xx_pmu *pmu = pmd->pmu; + + if (!mask) + return -EINVAL; + + *is_on = readl(pmu->base + JH71XX_PMU_CURR_POWER_MODE) & mask; + + return 0; +} + +static int jh71xx_pmu_set_state(struct jh71xx_pmu_dev *pmd, u32 mask, bool on) +{ + struct jh71xx_pmu *pmu = pmd->pmu; + unsigned long flags; + u32 val; + u32 mode; + u32 encourage_lo; + u32 encourage_hi; + bool is_on; + int ret; + + ret = jh71xx_pmu_get_state(pmd, mask, &is_on); + if (ret) { + dev_dbg(pmu->dev, "unable to get current state for %s\n", + pmd->genpd.name); + return ret; + } + + if (is_on == on) { + dev_dbg(pmu->dev, "pm domain [%s] is already %sable status.\n", + pmd->genpd.name, on ? "en" : "dis"); + return 0; + } + + spin_lock_irqsave(&pmu->lock, flags); + + /* + * The PMU accepts software encourage to switch power mode in the following 2 steps: + * + * 1.Configure the register SW_TURN_ON_POWER (offset 0x0c) by writing 1 to + * the bit corresponding to the power domain that will be turned on + * and writing 0 to the others. + * Likewise, configure the register SW_TURN_OFF_POWER (offset 0x10) by + * writing 1 to the bit corresponding to the power domain that will be + * turned off and writing 0 to the others. + */ + if (on) { + mode = JH71XX_PMU_SW_TURN_ON_POWER; + encourage_lo = JH71XX_PMU_SW_ENCOURAGE_EN_LO; + encourage_hi = JH71XX_PMU_SW_ENCOURAGE_EN_HI; + } else { + mode = JH71XX_PMU_SW_TURN_OFF_POWER; + encourage_lo = JH71XX_PMU_SW_ENCOURAGE_DIS_LO; + encourage_hi = JH71XX_PMU_SW_ENCOURAGE_DIS_HI; + } + + writel(mask, pmu->base + mode); + + /* + * 2.Write SW encourage command sequence to the Software Encourage Reg (offset 0x44) + * First write SW_MODE_ENCOURAGE_ON to JH71XX_PMU_SW_ENCOURAGE. This will reset + * the state machine which parses the command sequence. This register must be + * written every time software wants to power on/off a domain. + * Then write the lower bits of the command sequence, followed by the upper + * bits. The sequence differs between powering on & off a domain. + */ + writel(JH71XX_PMU_SW_ENCOURAGE_ON, pmu->base + JH71XX_PMU_SW_ENCOURAGE); + writel(encourage_lo, pmu->base + JH71XX_PMU_SW_ENCOURAGE); + writel(encourage_hi, pmu->base + JH71XX_PMU_SW_ENCOURAGE); + + spin_unlock_irqrestore(&pmu->lock, flags); + + /* Wait for the power domain bit to be enabled / disabled */ + if (on) { + ret = readl_poll_timeout_atomic(pmu->base + JH71XX_PMU_CURR_POWER_MODE, + val, val & mask, + 1, JH71XX_PMU_TIMEOUT_US); + } else { + ret = readl_poll_timeout_atomic(pmu->base + JH71XX_PMU_CURR_POWER_MODE, + val, !(val & mask), + 1, JH71XX_PMU_TIMEOUT_US); + } + + if (ret) { + dev_err(pmu->dev, "%s: failed to power %s\n", + pmd->genpd.name, on ? "on" : "off"); + return -ETIMEDOUT; + } + + return 0; +} + +static int jh71xx_pmu_on(struct generic_pm_domain *genpd) +{ + struct jh71xx_pmu_dev *pmd = container_of(genpd, + struct jh71xx_pmu_dev, genpd); + u32 pwr_mask = BIT(pmd->domain_info->bit); + + return jh71xx_pmu_set_state(pmd, pwr_mask, true); +} + +static int jh71xx_pmu_off(struct generic_pm_domain *genpd) +{ + struct jh71xx_pmu_dev *pmd = container_of(genpd, + struct jh71xx_pmu_dev, genpd); + u32 pwr_mask = BIT(pmd->domain_info->bit); + + return jh71xx_pmu_set_state(pmd, pwr_mask, false); +} + +static void jh71xx_pmu_int_enable(struct jh71xx_pmu *pmu, u32 mask, bool enable) +{ + u32 val; + unsigned long flags; + + spin_lock_irqsave(&pmu->lock, flags); + val = readl(pmu->base + JH71XX_PMU_TIMER_INT_MASK); + + if (enable) + val &= ~mask; + else + val |= mask; + + writel(val, pmu->base + JH71XX_PMU_TIMER_INT_MASK); + spin_unlock_irqrestore(&pmu->lock, flags); +} + +static irqreturn_t jh71xx_pmu_interrupt(int irq, void *data) +{ + struct jh71xx_pmu *pmu = data; + u32 val; + + val = readl(pmu->base + JH71XX_PMU_INT_STATUS); + + if (val & JH71XX_PMU_INT_SEQ_DONE) + dev_dbg(pmu->dev, "sequence done.\n"); + if (val & JH71XX_PMU_INT_HW_REQ) + dev_dbg(pmu->dev, "hardware encourage requestion.\n"); + if (val & JH71XX_PMU_INT_SW_FAIL) + dev_err(pmu->dev, "software encourage fail.\n"); + if (val & JH71XX_PMU_INT_HW_FAIL) + dev_err(pmu->dev, "hardware encourage fail.\n"); + if (val & JH71XX_PMU_INT_PCH_FAIL) + dev_err(pmu->dev, "p-channel fail event.\n"); + + /* clear interrupts */ + writel(val, pmu->base + JH71XX_PMU_INT_STATUS); + writel(val, pmu->base + JH71XX_PMU_EVENT_STATUS); + + return IRQ_HANDLED; +} + +static int jh71xx_pmu_init_domain(struct jh71xx_pmu *pmu, int index) +{ + struct jh71xx_pmu_dev *pmd; + u32 pwr_mask; + int ret; + bool is_on = false; + + pmd = devm_kzalloc(pmu->dev, sizeof(*pmd), GFP_KERNEL); + if (!pmd) + return -ENOMEM; + + pmd->domain_info = &pmu->match_data->domain_info[index]; + pmd->pmu = pmu; + pwr_mask = BIT(pmd->domain_info->bit); + + pmd->genpd.name = pmd->domain_info->name; + pmd->genpd.flags = pmd->domain_info->flags; + + ret = jh71xx_pmu_get_state(pmd, pwr_mask, &is_on); + if (ret) + dev_warn(pmu->dev, "unable to get current state for %s\n", + pmd->genpd.name); + + pmd->genpd.power_on = jh71xx_pmu_on; + pmd->genpd.power_off = jh71xx_pmu_off; + pm_genpd_init(&pmd->genpd, NULL, !is_on); + + pmu->genpd_data.domains[index] = &pmd->genpd; + + return 0; +} + +static int jh71xx_pmu_probe(struct platform_device *pdev) +{ + struct device *dev = &pdev->dev; + struct device_node *np = dev->of_node; + const struct jh71xx_pmu_match_data *match_data; + struct jh71xx_pmu *pmu; + unsigned int i; + int ret; + + pmu = devm_kzalloc(dev, sizeof(*pmu), GFP_KERNEL); + if (!pmu) + return -ENOMEM; + + pmu->base = devm_platform_ioremap_resource(pdev, 0); + if (IS_ERR(pmu->base)) + return PTR_ERR(pmu->base); + + pmu->irq = platform_get_irq(pdev, 0); + if (pmu->irq < 0) + return pmu->irq; + + ret = devm_request_irq(dev, pmu->irq, jh71xx_pmu_interrupt, + 0, pdev->name, pmu); + if (ret) + dev_err(dev, "failed to request irq\n"); + + match_data = of_device_get_match_data(dev); + if (!match_data) + return -EINVAL; + + pmu->genpd = devm_kcalloc(dev, match_data->num_domains, + sizeof(struct generic_pm_domain *), + GFP_KERNEL); + if (!pmu->genpd) + return -ENOMEM; + + pmu->dev = dev; + pmu->match_data = match_data; + pmu->genpd_data.domains = pmu->genpd; + pmu->genpd_data.num_domains = match_data->num_domains; + + for (i = 0; i < match_data->num_domains; i++) { + ret = jh71xx_pmu_init_domain(pmu, i); + if (ret) { + dev_err(dev, "failed to initialize power domain\n"); + return ret; + } + } + + spin_lock_init(&pmu->lock); + jh71xx_pmu_int_enable(pmu, JH71XX_PMU_INT_ALL_MASK & ~JH71XX_PMU_INT_PCH_FAIL, true); + + ret = of_genpd_add_provider_onecell(np, &pmu->genpd_data); + if (ret) { + dev_err(dev, "failed to register genpd driver: %d\n", ret); + return ret; + } + + dev_dbg(dev, "registered %u power domains\n", i); + + return 0; +} + +static const struct jh71xx_domain_info jh7110_power_domains[] = { + [JH7110_PD_SYSTOP] = { + .name = "SYSTOP", + .bit = 0, + .flags = GENPD_FLAG_ALWAYS_ON, + }, + [JH7110_PD_CPU] = { + .name = "CPU", + .bit = 1, + .flags = GENPD_FLAG_ALWAYS_ON, + }, + [JH7110_PD_GPUA] = { + .name = "GPUA", + .bit = 2, + }, + [JH7110_PD_VDEC] = { + .name = "VDEC", + .bit = 3, + }, + [JH7110_PD_VOUT] = { + .name = "VOUT", + .bit = 4, + }, + [JH7110_PD_ISP] = { + .name = "ISP", + .bit = 5, + }, + [JH7110_PD_VENC] = { + .name = "VENC", + .bit = 6, + }, +}; + +static const struct jh71xx_pmu_match_data jh7110_pmu = { + .num_domains = ARRAY_SIZE(jh7110_power_domains), + .domain_info = jh7110_power_domains, +}; + +static const struct of_device_id jh71xx_pmu_of_match[] = { + { + .compatible = "starfive,jh7110-pmu", + .data = (void *)&jh7110_pmu, + }, { + /* sentinel */ + } +}; + +static struct platform_driver jh71xx_pmu_driver = { + .probe = jh71xx_pmu_probe, + .driver = { + .name = "jh71xx-pmu", + .of_match_table = jh71xx_pmu_of_match, + .suppress_bind_attrs = true, + }, +}; +builtin_platform_driver(jh71xx_pmu_driver); + +MODULE_AUTHOR("Walker Chen "); +MODULE_DESCRIPTION("StarFive JH71XX PMU Driver"); +MODULE_LICENSE("GPL"); diff --git a/drivers/pmdomain/sunxi/Makefile b/drivers/pmdomain/sunxi/Makefile new file mode 100644 index 000000000000..ec1d7a2fb21d --- /dev/null +++ b/drivers/pmdomain/sunxi/Makefile @@ -0,0 +1,2 @@ +# SPDX-License-Identifier: GPL-2.0-only +obj-$(CONFIG_SUN20I_PPU) += sun20i-ppu.o diff --git a/drivers/pmdomain/sunxi/sun20i-ppu.c b/drivers/pmdomain/sunxi/sun20i-ppu.c new file mode 100644 index 000000000000..8700f9dd5f75 --- /dev/null +++ b/drivers/pmdomain/sunxi/sun20i-ppu.c @@ -0,0 +1,207 @@ +// SPDX-License-Identifier: GPL-2.0-only + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#define PD_STATE_ON 1 +#define PD_STATE_OFF 2 + +#define PD_RSTN_REG 0x00 +#define PD_CLK_GATE_REG 0x04 +#define PD_PWROFF_GATE_REG 0x08 +#define PD_PSW_ON_REG 0x0c +#define PD_PSW_OFF_REG 0x10 +#define PD_PSW_DELAY_REG 0x14 +#define PD_OFF_DELAY_REG 0x18 +#define PD_ON_DELAY_REG 0x1c +#define PD_COMMAND_REG 0x20 +#define PD_STATUS_REG 0x24 +#define PD_STATUS_COMPLETE BIT(1) +#define PD_STATUS_BUSY BIT(3) +#define PD_STATUS_STATE GENMASK(17, 16) +#define PD_ACTIVE_CTRL_REG 0x2c +#define PD_GATE_STATUS_REG 0x30 +#define PD_RSTN_STATUS BIT(0) +#define PD_CLK_GATE_STATUS BIT(1) +#define PD_PWROFF_GATE_STATUS BIT(2) +#define PD_PSW_STATUS_REG 0x34 + +#define PD_REGS_SIZE 0x80 + +struct sun20i_ppu_desc { + const char *const *names; + unsigned int num_domains; +}; + +struct sun20i_ppu_pd { + struct generic_pm_domain genpd; + void __iomem *base; +}; + +#define to_sun20i_ppu_pd(_genpd) \ + container_of(_genpd, struct sun20i_ppu_pd, genpd) + +static bool sun20i_ppu_pd_is_on(const struct sun20i_ppu_pd *pd) +{ + u32 status = readl(pd->base + PD_STATUS_REG); + + return FIELD_GET(PD_STATUS_STATE, status) == PD_STATE_ON; +} + +static int sun20i_ppu_pd_set_power(const struct sun20i_ppu_pd *pd, bool power_on) +{ + u32 state, status; + int ret; + + if (sun20i_ppu_pd_is_on(pd) == power_on) + return 0; + + /* Wait for the power controller to be idle. */ + ret = readl_poll_timeout(pd->base + PD_STATUS_REG, status, + !(status & PD_STATUS_BUSY), 100, 1000); + if (ret) + return ret; + + state = power_on ? PD_STATE_ON : PD_STATE_OFF; + writel(state, pd->base + PD_COMMAND_REG); + + /* Wait for the state transition to complete. */ + ret = readl_poll_timeout(pd->base + PD_STATUS_REG, status, + FIELD_GET(PD_STATUS_STATE, status) == state && + (status & PD_STATUS_COMPLETE), 100, 1000); + if (ret) + return ret; + + /* Clear the completion flag. */ + writel(status, pd->base + PD_STATUS_REG); + + return 0; +} + +static int sun20i_ppu_pd_power_on(struct generic_pm_domain *genpd) +{ + const struct sun20i_ppu_pd *pd = to_sun20i_ppu_pd(genpd); + + return sun20i_ppu_pd_set_power(pd, true); +} + +static int sun20i_ppu_pd_power_off(struct generic_pm_domain *genpd) +{ + const struct sun20i_ppu_pd *pd = to_sun20i_ppu_pd(genpd); + + return sun20i_ppu_pd_set_power(pd, false); +} + +static int sun20i_ppu_probe(struct platform_device *pdev) +{ + const struct sun20i_ppu_desc *desc; + struct device *dev = &pdev->dev; + struct genpd_onecell_data *ppu; + struct sun20i_ppu_pd *pds; + struct reset_control *rst; + void __iomem *base; + struct clk *clk; + int ret; + + desc = of_device_get_match_data(dev); + if (!desc) + return -EINVAL; + + pds = devm_kcalloc(dev, desc->num_domains, sizeof(*pds), GFP_KERNEL); + if (!pds) + return -ENOMEM; + + ppu = devm_kzalloc(dev, sizeof(*ppu), GFP_KERNEL); + if (!ppu) + return -ENOMEM; + + ppu->domains = devm_kcalloc(dev, desc->num_domains, + sizeof(*ppu->domains), GFP_KERNEL); + if (!ppu->domains) + return -ENOMEM; + + ppu->num_domains = desc->num_domains; + platform_set_drvdata(pdev, ppu); + + base = devm_platform_ioremap_resource(pdev, 0); + if (IS_ERR(base)) + return PTR_ERR(base); + + clk = devm_clk_get_enabled(dev, NULL); + if (IS_ERR(clk)) + return PTR_ERR(clk); + + rst = devm_reset_control_get_exclusive(dev, NULL); + if (IS_ERR(rst)) + return PTR_ERR(rst); + + ret = reset_control_deassert(rst); + if (ret) + return ret; + + for (unsigned int i = 0; i < ppu->num_domains; ++i) { + struct sun20i_ppu_pd *pd = &pds[i]; + + pd->genpd.name = desc->names[i]; + pd->genpd.power_off = sun20i_ppu_pd_power_off; + pd->genpd.power_on = sun20i_ppu_pd_power_on; + pd->base = base + PD_REGS_SIZE * i; + + ret = pm_genpd_init(&pd->genpd, NULL, sun20i_ppu_pd_is_on(pd)); + if (ret) { + dev_warn(dev, "Failed to add '%s' domain: %d\n", + pd->genpd.name, ret); + continue; + } + + ppu->domains[i] = &pd->genpd; + } + + ret = of_genpd_add_provider_onecell(dev->of_node, ppu); + if (ret) + dev_warn(dev, "Failed to add provider: %d\n", ret); + + return 0; +} + +static const char *const sun20i_d1_ppu_pd_names[] = { + "CPU", + "VE", + "DSP", +}; + +static const struct sun20i_ppu_desc sun20i_d1_ppu_desc = { + .names = sun20i_d1_ppu_pd_names, + .num_domains = ARRAY_SIZE(sun20i_d1_ppu_pd_names), +}; + +static const struct of_device_id sun20i_ppu_of_match[] = { + { + .compatible = "allwinner,sun20i-d1-ppu", + .data = &sun20i_d1_ppu_desc, + }, + { } +}; +MODULE_DEVICE_TABLE(of, sun20i_ppu_of_match); + +static struct platform_driver sun20i_ppu_driver = { + .probe = sun20i_ppu_probe, + .driver = { + .name = "sun20i-ppu", + .of_match_table = sun20i_ppu_of_match, + /* Power domains cannot be removed while they are in use. */ + .suppress_bind_attrs = true, + }, +}; +module_platform_driver(sun20i_ppu_driver); + +MODULE_AUTHOR("Samuel Holland "); +MODULE_DESCRIPTION("Allwinner D1 PPU power domain driver"); +MODULE_LICENSE("GPL"); diff --git a/drivers/pmdomain/tegra/Makefile b/drivers/pmdomain/tegra/Makefile new file mode 100644 index 000000000000..ec8acfd2c77c --- /dev/null +++ b/drivers/pmdomain/tegra/Makefile @@ -0,0 +1,2 @@ +# SPDX-License-Identifier: GPL-2.0 +obj-$(CONFIG_SOC_TEGRA_POWERGATE_BPMP) += powergate-bpmp.o diff --git a/drivers/pmdomain/tegra/powergate-bpmp.c b/drivers/pmdomain/tegra/powergate-bpmp.c new file mode 100644 index 000000000000..179ed895c279 --- /dev/null +++ b/drivers/pmdomain/tegra/powergate-bpmp.c @@ -0,0 +1,361 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2016-2017, NVIDIA CORPORATION. All rights reserved + */ + +#include +#include +#include +#include + +#include +#include + +struct tegra_powergate_info { + unsigned int id; + char *name; +}; + +struct tegra_powergate { + struct generic_pm_domain genpd; + struct tegra_bpmp *bpmp; + unsigned int id; +}; + +static inline struct tegra_powergate * +to_tegra_powergate(struct generic_pm_domain *genpd) +{ + return container_of(genpd, struct tegra_powergate, genpd); +} + +static int tegra_bpmp_powergate_set_state(struct tegra_bpmp *bpmp, + unsigned int id, u32 state) +{ + struct mrq_pg_request request; + struct tegra_bpmp_message msg; + int err; + + memset(&request, 0, sizeof(request)); + request.cmd = CMD_PG_SET_STATE; + request.id = id; + request.set_state.state = state; + + memset(&msg, 0, sizeof(msg)); + msg.mrq = MRQ_PG; + msg.tx.data = &request; + msg.tx.size = sizeof(request); + + err = tegra_bpmp_transfer(bpmp, &msg); + if (err < 0) + return err; + else if (msg.rx.ret < 0) + return -EINVAL; + + return 0; +} + +static int tegra_bpmp_powergate_get_state(struct tegra_bpmp *bpmp, + unsigned int id) +{ + struct mrq_pg_response response; + struct mrq_pg_request request; + struct tegra_bpmp_message msg; + int err; + + memset(&request, 0, sizeof(request)); + request.cmd = CMD_PG_GET_STATE; + request.id = id; + + memset(&response, 0, sizeof(response)); + + memset(&msg, 0, sizeof(msg)); + msg.mrq = MRQ_PG; + msg.tx.data = &request; + msg.tx.size = sizeof(request); + msg.rx.data = &response; + msg.rx.size = sizeof(response); + + err = tegra_bpmp_transfer(bpmp, &msg); + if (err < 0) + return PG_STATE_OFF; + else if (msg.rx.ret < 0) + return -EINVAL; + + return response.get_state.state; +} + +static int tegra_bpmp_powergate_get_max_id(struct tegra_bpmp *bpmp) +{ + struct mrq_pg_response response; + struct mrq_pg_request request; + struct tegra_bpmp_message msg; + int err; + + memset(&request, 0, sizeof(request)); + request.cmd = CMD_PG_GET_MAX_ID; + + memset(&response, 0, sizeof(response)); + + memset(&msg, 0, sizeof(msg)); + msg.mrq = MRQ_PG; + msg.tx.data = &request; + msg.tx.size = sizeof(request); + msg.rx.data = &response; + msg.rx.size = sizeof(response); + + err = tegra_bpmp_transfer(bpmp, &msg); + if (err < 0) + return err; + else if (msg.rx.ret < 0) + return -EINVAL; + + return response.get_max_id.max_id; +} + +static char *tegra_bpmp_powergate_get_name(struct tegra_bpmp *bpmp, + unsigned int id) +{ + struct mrq_pg_response response; + struct mrq_pg_request request; + struct tegra_bpmp_message msg; + int err; + + memset(&request, 0, sizeof(request)); + request.cmd = CMD_PG_GET_NAME; + request.id = id; + + memset(&response, 0, sizeof(response)); + + memset(&msg, 0, sizeof(msg)); + msg.mrq = MRQ_PG; + msg.tx.data = &request; + msg.tx.size = sizeof(request); + msg.rx.data = &response; + msg.rx.size = sizeof(response); + + err = tegra_bpmp_transfer(bpmp, &msg); + if (err < 0 || msg.rx.ret < 0) + return NULL; + + return kstrdup(response.get_name.name, GFP_KERNEL); +} + +static inline bool tegra_bpmp_powergate_is_powered(struct tegra_bpmp *bpmp, + unsigned int id) +{ + return tegra_bpmp_powergate_get_state(bpmp, id) != PG_STATE_OFF; +} + +static int tegra_powergate_power_on(struct generic_pm_domain *domain) +{ + struct tegra_powergate *powergate = to_tegra_powergate(domain); + struct tegra_bpmp *bpmp = powergate->bpmp; + + return tegra_bpmp_powergate_set_state(bpmp, powergate->id, + PG_STATE_ON); +} + +static int tegra_powergate_power_off(struct generic_pm_domain *domain) +{ + struct tegra_powergate *powergate = to_tegra_powergate(domain); + struct tegra_bpmp *bpmp = powergate->bpmp; + + return tegra_bpmp_powergate_set_state(bpmp, powergate->id, + PG_STATE_OFF); +} + +static struct tegra_powergate * +tegra_powergate_add(struct tegra_bpmp *bpmp, + const struct tegra_powergate_info *info) +{ + struct tegra_powergate *powergate; + bool off; + int err; + + off = !tegra_bpmp_powergate_is_powered(bpmp, info->id); + + powergate = devm_kzalloc(bpmp->dev, sizeof(*powergate), GFP_KERNEL); + if (!powergate) + return ERR_PTR(-ENOMEM); + + powergate->id = info->id; + powergate->bpmp = bpmp; + + powergate->genpd.name = kstrdup(info->name, GFP_KERNEL); + powergate->genpd.power_on = tegra_powergate_power_on; + powergate->genpd.power_off = tegra_powergate_power_off; + + err = pm_genpd_init(&powergate->genpd, NULL, off); + if (err < 0) { + kfree(powergate->genpd.name); + return ERR_PTR(err); + } + + return powergate; +} + +static void tegra_powergate_remove(struct tegra_powergate *powergate) +{ + struct generic_pm_domain *genpd = &powergate->genpd; + struct tegra_bpmp *bpmp = powergate->bpmp; + int err; + + err = pm_genpd_remove(genpd); + if (err < 0) + dev_err(bpmp->dev, "failed to remove power domain %s: %d\n", + genpd->name, err); + + kfree(genpd->name); +} + +static int +tegra_bpmp_probe_powergates(struct tegra_bpmp *bpmp, + struct tegra_powergate_info **powergatesp) +{ + struct tegra_powergate_info *powergates; + unsigned int max_id, id, count = 0; + unsigned int num_holes = 0; + int err; + + err = tegra_bpmp_powergate_get_max_id(bpmp); + if (err < 0) + return err; + + max_id = err; + + dev_dbg(bpmp->dev, "maximum powergate ID: %u\n", max_id); + + powergates = kcalloc(max_id + 1, sizeof(*powergates), GFP_KERNEL); + if (!powergates) + return -ENOMEM; + + for (id = 0; id <= max_id; id++) { + struct tegra_powergate_info *info = &powergates[count]; + + info->name = tegra_bpmp_powergate_get_name(bpmp, id); + if (!info->name || info->name[0] == '\0') { + num_holes++; + continue; + } + + info->id = id; + count++; + } + + dev_dbg(bpmp->dev, "holes: %u\n", num_holes); + + *powergatesp = powergates; + + return count; +} + +static int tegra_bpmp_add_powergates(struct tegra_bpmp *bpmp, + struct tegra_powergate_info *powergates, + unsigned int count) +{ + struct genpd_onecell_data *genpd = &bpmp->genpd; + struct generic_pm_domain **domains; + struct tegra_powergate *powergate; + unsigned int i; + int err; + + domains = kcalloc(count, sizeof(*domains), GFP_KERNEL); + if (!domains) + return -ENOMEM; + + for (i = 0; i < count; i++) { + powergate = tegra_powergate_add(bpmp, &powergates[i]); + if (IS_ERR(powergate)) { + err = PTR_ERR(powergate); + goto remove; + } + + dev_dbg(bpmp->dev, "added power domain %s\n", + powergate->genpd.name); + domains[i] = &powergate->genpd; + } + + genpd->num_domains = count; + genpd->domains = domains; + + return 0; + +remove: + while (i--) { + powergate = to_tegra_powergate(domains[i]); + tegra_powergate_remove(powergate); + } + + kfree(domains); + return err; +} + +static void tegra_bpmp_remove_powergates(struct tegra_bpmp *bpmp) +{ + struct genpd_onecell_data *genpd = &bpmp->genpd; + unsigned int i = genpd->num_domains; + struct tegra_powergate *powergate; + + while (i--) { + dev_dbg(bpmp->dev, "removing power domain %s\n", + genpd->domains[i]->name); + powergate = to_tegra_powergate(genpd->domains[i]); + tegra_powergate_remove(powergate); + } +} + +static struct generic_pm_domain * +tegra_powergate_xlate(struct of_phandle_args *spec, void *data) +{ + struct generic_pm_domain *domain = ERR_PTR(-ENOENT); + struct genpd_onecell_data *genpd = data; + unsigned int i; + + for (i = 0; i < genpd->num_domains; i++) { + struct tegra_powergate *powergate; + + powergate = to_tegra_powergate(genpd->domains[i]); + if (powergate->id == spec->args[0]) { + domain = &powergate->genpd; + break; + } + } + + return domain; +} + +int tegra_bpmp_init_powergates(struct tegra_bpmp *bpmp) +{ + struct device_node *np = bpmp->dev->of_node; + struct tegra_powergate_info *powergates; + struct device *dev = bpmp->dev; + unsigned int count, i; + int err; + + err = tegra_bpmp_probe_powergates(bpmp, &powergates); + if (err < 0) + return err; + + count = err; + + dev_dbg(dev, "%u power domains probed\n", count); + + err = tegra_bpmp_add_powergates(bpmp, powergates, count); + if (err < 0) + goto free; + + bpmp->genpd.xlate = tegra_powergate_xlate; + + err = of_genpd_add_provider_onecell(np, &bpmp->genpd); + if (err < 0) { + dev_err(dev, "failed to add power domain provider: %d\n", err); + tegra_bpmp_remove_powergates(bpmp); + } + +free: + for (i = 0; i < count; i++) + kfree(powergates[i].name); + + kfree(powergates); + return err; +} diff --git a/drivers/pmdomain/ti/Makefile b/drivers/pmdomain/ti/Makefile new file mode 100644 index 000000000000..69580afbb436 --- /dev/null +++ b/drivers/pmdomain/ti/Makefile @@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0 +obj-$(CONFIG_ARCH_OMAP2PLUS) += omap_prm.o +obj-$(CONFIG_TI_SCI_PM_DOMAINS) += ti_sci_pm_domains.o diff --git a/drivers/pmdomain/ti/omap_prm.c b/drivers/pmdomain/ti/omap_prm.c new file mode 100644 index 000000000000..c2feae3a634c --- /dev/null +++ b/drivers/pmdomain/ti/omap_prm.c @@ -0,0 +1,989 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * OMAP2+ PRM driver + * + * Copyright (C) 2019 Texas Instruments Incorporated - http://www.ti.com/ + * Tero Kristo + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +enum omap_prm_domain_mode { + OMAP_PRMD_OFF, + OMAP_PRMD_RETENTION, + OMAP_PRMD_ON_INACTIVE, + OMAP_PRMD_ON_ACTIVE, +}; + +struct omap_prm_domain_map { + unsigned int usable_modes; /* Mask of hardware supported modes */ + unsigned long statechange:1; /* Optional low-power state change */ + unsigned long logicretstate:1; /* Optional logic off mode */ +}; + +struct omap_prm_domain { + struct device *dev; + struct omap_prm *prm; + struct generic_pm_domain pd; + u16 pwrstctrl; + u16 pwrstst; + const struct omap_prm_domain_map *cap; + u32 pwrstctrl_saved; + unsigned int uses_pm_clk:1; +}; + +struct omap_rst_map { + s8 rst; + s8 st; +}; + +struct omap_prm_data { + u32 base; + const char *name; + const char *clkdm_name; + u16 pwrstctrl; + u16 pwrstst; + const struct omap_prm_domain_map *dmap; + u16 rstctrl; + u16 rstst; + const struct omap_rst_map *rstmap; + u8 flags; +}; + +struct omap_prm { + const struct omap_prm_data *data; + void __iomem *base; + struct omap_prm_domain *prmd; +}; + +struct omap_reset_data { + struct reset_controller_dev rcdev; + struct omap_prm *prm; + u32 mask; + spinlock_t lock; + struct clockdomain *clkdm; + struct device *dev; +}; + +#define genpd_to_prm_domain(gpd) container_of(gpd, struct omap_prm_domain, pd) +#define to_omap_reset_data(p) container_of((p), struct omap_reset_data, rcdev) + +#define OMAP_MAX_RESETS 8 +#define OMAP_RESET_MAX_WAIT 10000 + +#define OMAP_PRM_HAS_RSTCTRL BIT(0) +#define OMAP_PRM_HAS_RSTST BIT(1) +#define OMAP_PRM_HAS_NO_CLKDM BIT(2) +#define OMAP_PRM_RET_WHEN_IDLE BIT(3) + +#define OMAP_PRM_HAS_RESETS (OMAP_PRM_HAS_RSTCTRL | OMAP_PRM_HAS_RSTST) + +#define PRM_STATE_MAX_WAIT 10000 +#define PRM_LOGICRETSTATE BIT(2) +#define PRM_LOWPOWERSTATECHANGE BIT(4) +#define PRM_POWERSTATE_MASK OMAP_PRMD_ON_ACTIVE + +#define PRM_ST_INTRANSITION BIT(20) + +static const struct omap_prm_domain_map omap_prm_all = { + .usable_modes = BIT(OMAP_PRMD_ON_ACTIVE) | BIT(OMAP_PRMD_ON_INACTIVE) | + BIT(OMAP_PRMD_RETENTION) | BIT(OMAP_PRMD_OFF), + .statechange = 1, + .logicretstate = 1, +}; + +static const struct omap_prm_domain_map omap_prm_noinact = { + .usable_modes = BIT(OMAP_PRMD_ON_ACTIVE) | BIT(OMAP_PRMD_RETENTION) | + BIT(OMAP_PRMD_OFF), + .statechange = 1, + .logicretstate = 1, +}; + +static const struct omap_prm_domain_map omap_prm_nooff = { + .usable_modes = BIT(OMAP_PRMD_ON_ACTIVE) | BIT(OMAP_PRMD_ON_INACTIVE) | + BIT(OMAP_PRMD_RETENTION), + .statechange = 1, + .logicretstate = 1, +}; + +static const struct omap_prm_domain_map omap_prm_onoff_noauto = { + .usable_modes = BIT(OMAP_PRMD_ON_ACTIVE) | BIT(OMAP_PRMD_OFF), + .statechange = 1, +}; + +static const struct omap_prm_domain_map omap_prm_alwon = { + .usable_modes = BIT(OMAP_PRMD_ON_ACTIVE), +}; + +static const struct omap_prm_domain_map omap_prm_reton = { + .usable_modes = BIT(OMAP_PRMD_ON_ACTIVE) | BIT(OMAP_PRMD_RETENTION), + .statechange = 1, + .logicretstate = 1, +}; + +static const struct omap_rst_map rst_map_0[] = { + { .rst = 0, .st = 0 }, + { .rst = -1 }, +}; + +static const struct omap_rst_map rst_map_01[] = { + { .rst = 0, .st = 0 }, + { .rst = 1, .st = 1 }, + { .rst = -1 }, +}; + +static const struct omap_rst_map rst_map_012[] = { + { .rst = 0, .st = 0 }, + { .rst = 1, .st = 1 }, + { .rst = 2, .st = 2 }, + { .rst = -1 }, +}; + +static const struct omap_prm_data omap4_prm_data[] = { + { + .name = "mpu", .base = 0x4a306300, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_reton, + }, + { + .name = "tesla", .base = 0x4a306400, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_noinact, + .rstctrl = 0x10, .rstst = 0x14, .rstmap = rst_map_01 + }, + { + .name = "abe", .base = 0x4a306500, + .pwrstctrl = 0, .pwrstst = 0x4, .dmap = &omap_prm_all, + }, + { + .name = "always_on_core", .base = 0x4a306600, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_alwon, + }, + { + .name = "core", .base = 0x4a306700, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_reton, + .rstctrl = 0x210, .rstst = 0x214, .clkdm_name = "ducati", + .rstmap = rst_map_012, + .flags = OMAP_PRM_RET_WHEN_IDLE, + }, + { + .name = "ivahd", .base = 0x4a306f00, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_noinact, + .rstctrl = 0x10, .rstst = 0x14, .rstmap = rst_map_012 + }, + { + .name = "cam", .base = 0x4a307000, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto, + }, + { + .name = "dss", .base = 0x4a307100, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_noinact + }, + { + .name = "gfx", .base = 0x4a307200, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto + }, + { + .name = "l3init", .base = 0x4a307300, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_reton + }, + { + .name = "l4per", .base = 0x4a307400, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_reton, + .flags = OMAP_PRM_RET_WHEN_IDLE, + }, + { + .name = "cefuse", .base = 0x4a307600, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto + }, + { + .name = "wkup", .base = 0x4a307700, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_alwon + }, + { + .name = "emu", .base = 0x4a307900, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto + }, + { + .name = "device", .base = 0x4a307b00, + .rstctrl = 0x0, .rstst = 0x4, .rstmap = rst_map_01, + .flags = OMAP_PRM_HAS_RSTCTRL | OMAP_PRM_HAS_NO_CLKDM + }, + { }, +}; + +static const struct omap_prm_data omap5_prm_data[] = { + { + .name = "mpu", .base = 0x4ae06300, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_reton, + }, + { + .name = "dsp", .base = 0x4ae06400, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_noinact, + .rstctrl = 0x10, .rstst = 0x14, .rstmap = rst_map_01 + }, + { + .name = "abe", .base = 0x4ae06500, + .pwrstctrl = 0, .pwrstst = 0x4, .dmap = &omap_prm_nooff, + }, + { + .name = "coreaon", .base = 0x4ae06600, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_alwon + }, + { + .name = "core", .base = 0x4ae06700, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_reton, + .rstctrl = 0x210, .rstst = 0x214, .clkdm_name = "ipu", + .rstmap = rst_map_012 + }, + { + .name = "iva", .base = 0x4ae07200, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_noinact, + .rstctrl = 0x10, .rstst = 0x14, .rstmap = rst_map_012 + }, + { + .name = "cam", .base = 0x4ae07300, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto + }, + { + .name = "dss", .base = 0x4ae07400, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_noinact + }, + { + .name = "gpu", .base = 0x4ae07500, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto + }, + { + .name = "l3init", .base = 0x4ae07600, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_reton + }, + { + .name = "custefuse", .base = 0x4ae07700, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto + }, + { + .name = "wkupaon", .base = 0x4ae07800, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_alwon + }, + { + .name = "emu", .base = 0x4ae07a00, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto + }, + { + .name = "device", .base = 0x4ae07c00, + .rstctrl = 0x0, .rstst = 0x4, .rstmap = rst_map_01, + .flags = OMAP_PRM_HAS_RSTCTRL | OMAP_PRM_HAS_NO_CLKDM + }, + { }, +}; + +static const struct omap_prm_data dra7_prm_data[] = { + { + .name = "mpu", .base = 0x4ae06300, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_reton, + }, + { + .name = "dsp1", .base = 0x4ae06400, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto, + .rstctrl = 0x10, .rstst = 0x14, .rstmap = rst_map_01, + }, + { + .name = "ipu", .base = 0x4ae06500, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto, + .rstctrl = 0x10, .rstst = 0x14, .rstmap = rst_map_012, + .clkdm_name = "ipu1" + }, + { + .name = "coreaon", .base = 0x4ae06628, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_alwon, + }, + { + .name = "core", .base = 0x4ae06700, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_alwon, + .rstctrl = 0x210, .rstst = 0x214, .rstmap = rst_map_012, + .clkdm_name = "ipu2" + }, + { + .name = "iva", .base = 0x4ae06f00, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto, + .rstctrl = 0x10, .rstst = 0x14, .rstmap = rst_map_012, + }, + { + .name = "cam", .base = 0x4ae07000, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto, + }, + { + .name = "dss", .base = 0x4ae07100, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto, + }, + { + .name = "gpu", .base = 0x4ae07200, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto, + }, + { + .name = "l3init", .base = 0x4ae07300, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_alwon, + .rstctrl = 0x10, .rstst = 0x14, .rstmap = rst_map_01, + .clkdm_name = "pcie" + }, + { + .name = "l4per", .base = 0x4ae07400, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_alwon, + }, + { + .name = "custefuse", .base = 0x4ae07600, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto, + }, + { + .name = "wkupaon", .base = 0x4ae07724, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_alwon, + }, + { + .name = "emu", .base = 0x4ae07900, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto, + }, + { + .name = "dsp2", .base = 0x4ae07b00, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto, + .rstctrl = 0x10, .rstst = 0x14, .rstmap = rst_map_01 + }, + { + .name = "eve1", .base = 0x4ae07b40, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto, + .rstctrl = 0x10, .rstst = 0x14, .rstmap = rst_map_01 + }, + { + .name = "eve2", .base = 0x4ae07b80, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto, + .rstctrl = 0x10, .rstst = 0x14, .rstmap = rst_map_01 + }, + { + .name = "eve3", .base = 0x4ae07bc0, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto, + .rstctrl = 0x10, .rstst = 0x14, .rstmap = rst_map_01 + }, + { + .name = "eve4", .base = 0x4ae07c00, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto, + .rstctrl = 0x10, .rstst = 0x14, .rstmap = rst_map_01 + }, + { + .name = "rtc", .base = 0x4ae07c60, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_alwon, + }, + { + .name = "vpe", .base = 0x4ae07c80, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto, + }, + { }, +}; + +static const struct omap_rst_map am3_per_rst_map[] = { + { .rst = 1 }, + { .rst = -1 }, +}; + +static const struct omap_rst_map am3_wkup_rst_map[] = { + { .rst = 3, .st = 5 }, + { .rst = -1 }, +}; + +static const struct omap_prm_data am3_prm_data[] = { + { + .name = "per", .base = 0x44e00c00, + .pwrstctrl = 0xc, .pwrstst = 0x8, .dmap = &omap_prm_noinact, + .rstctrl = 0x0, .rstmap = am3_per_rst_map, + .flags = OMAP_PRM_HAS_RSTCTRL, .clkdm_name = "pruss_ocp" + }, + { + .name = "wkup", .base = 0x44e00d00, + .pwrstctrl = 0x4, .pwrstst = 0x4, .dmap = &omap_prm_alwon, + .rstctrl = 0x0, .rstst = 0xc, .rstmap = am3_wkup_rst_map, + .flags = OMAP_PRM_HAS_RSTCTRL | OMAP_PRM_HAS_NO_CLKDM + }, + { + .name = "mpu", .base = 0x44e00e00, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_noinact, + }, + { + .name = "device", .base = 0x44e00f00, + .rstctrl = 0x0, .rstst = 0x8, .rstmap = rst_map_01, + .flags = OMAP_PRM_HAS_RSTCTRL | OMAP_PRM_HAS_NO_CLKDM + }, + { + .name = "rtc", .base = 0x44e01000, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_alwon, + }, + { + .name = "gfx", .base = 0x44e01100, + .pwrstctrl = 0, .pwrstst = 0x10, .dmap = &omap_prm_noinact, + .rstctrl = 0x4, .rstst = 0x14, .rstmap = rst_map_0, .clkdm_name = "gfx_l3", + }, + { + .name = "cefuse", .base = 0x44e01200, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto, + }, + { }, +}; + +static const struct omap_rst_map am4_per_rst_map[] = { + { .rst = 1, .st = 0 }, + { .rst = -1 }, +}; + +static const struct omap_rst_map am4_device_rst_map[] = { + { .rst = 0, .st = 1 }, + { .rst = 1, .st = 0 }, + { .rst = -1 }, +}; + +static const struct omap_prm_data am4_prm_data[] = { + { + .name = "mpu", .base = 0x44df0300, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_noinact, + }, + { + .name = "gfx", .base = 0x44df0400, + .pwrstctrl = 0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto, + .rstctrl = 0x10, .rstst = 0x14, .rstmap = rst_map_0, .clkdm_name = "gfx_l3", + }, + { + .name = "rtc", .base = 0x44df0500, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_alwon, + }, + { + .name = "tamper", .base = 0x44df0600, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_alwon, + }, + { + .name = "cefuse", .base = 0x44df0700, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_onoff_noauto, + }, + { + .name = "per", .base = 0x44df0800, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_noinact, + .rstctrl = 0x10, .rstst = 0x14, .rstmap = am4_per_rst_map, + .clkdm_name = "pruss_ocp" + }, + { + .name = "wkup", .base = 0x44df2000, + .pwrstctrl = 0x0, .pwrstst = 0x4, .dmap = &omap_prm_alwon, + .rstctrl = 0x10, .rstst = 0x14, .rstmap = am3_wkup_rst_map, + .flags = OMAP_PRM_HAS_NO_CLKDM + }, + { + .name = "device", .base = 0x44df4000, + .rstctrl = 0x0, .rstst = 0x4, .rstmap = am4_device_rst_map, + .flags = OMAP_PRM_HAS_RSTCTRL | OMAP_PRM_HAS_NO_CLKDM + }, + { }, +}; + +static const struct of_device_id omap_prm_id_table[] = { + { .compatible = "ti,omap4-prm-inst", .data = omap4_prm_data }, + { .compatible = "ti,omap5-prm-inst", .data = omap5_prm_data }, + { .compatible = "ti,dra7-prm-inst", .data = dra7_prm_data }, + { .compatible = "ti,am3-prm-inst", .data = am3_prm_data }, + { .compatible = "ti,am4-prm-inst", .data = am4_prm_data }, + { }, +}; + +#ifdef DEBUG +static void omap_prm_domain_show_state(struct omap_prm_domain *prmd, + const char *desc) +{ + dev_dbg(prmd->dev, "%s %s: %08x/%08x\n", + prmd->pd.name, desc, + readl_relaxed(prmd->prm->base + prmd->pwrstctrl), + readl_relaxed(prmd->prm->base + prmd->pwrstst)); +} +#else +static inline void omap_prm_domain_show_state(struct omap_prm_domain *prmd, + const char *desc) +{ +} +#endif + +static int omap_prm_domain_power_on(struct generic_pm_domain *domain) +{ + struct omap_prm_domain *prmd; + int ret; + u32 v, mode; + + prmd = genpd_to_prm_domain(domain); + if (!prmd->cap) + return 0; + + omap_prm_domain_show_state(prmd, "on: previous state"); + + if (prmd->pwrstctrl_saved) + v = prmd->pwrstctrl_saved; + else + v = readl_relaxed(prmd->prm->base + prmd->pwrstctrl); + + if (prmd->prm->data->flags & OMAP_PRM_RET_WHEN_IDLE) + mode = OMAP_PRMD_RETENTION; + else + mode = OMAP_PRMD_ON_ACTIVE; + + writel_relaxed((v & ~PRM_POWERSTATE_MASK) | mode, + prmd->prm->base + prmd->pwrstctrl); + + /* wait for the transition bit to get cleared */ + ret = readl_relaxed_poll_timeout(prmd->prm->base + prmd->pwrstst, + v, !(v & PRM_ST_INTRANSITION), 1, + PRM_STATE_MAX_WAIT); + if (ret) + dev_err(prmd->dev, "%s: %s timed out\n", + prmd->pd.name, __func__); + + omap_prm_domain_show_state(prmd, "on: new state"); + + return ret; +} + +/* No need to check for holes in the mask for the lowest mode */ +static int omap_prm_domain_find_lowest(struct omap_prm_domain *prmd) +{ + return __ffs(prmd->cap->usable_modes); +} + +static int omap_prm_domain_power_off(struct generic_pm_domain *domain) +{ + struct omap_prm_domain *prmd; + int ret; + u32 v; + + prmd = genpd_to_prm_domain(domain); + if (!prmd->cap) + return 0; + + omap_prm_domain_show_state(prmd, "off: previous state"); + + v = readl_relaxed(prmd->prm->base + prmd->pwrstctrl); + prmd->pwrstctrl_saved = v; + + v &= ~PRM_POWERSTATE_MASK; + v |= omap_prm_domain_find_lowest(prmd); + + if (prmd->cap->statechange) + v |= PRM_LOWPOWERSTATECHANGE; + if (prmd->cap->logicretstate) + v &= ~PRM_LOGICRETSTATE; + else + v |= PRM_LOGICRETSTATE; + + writel_relaxed(v, prmd->prm->base + prmd->pwrstctrl); + + /* wait for the transition bit to get cleared */ + ret = readl_relaxed_poll_timeout(prmd->prm->base + prmd->pwrstst, + v, !(v & PRM_ST_INTRANSITION), 1, + PRM_STATE_MAX_WAIT); + if (ret) + dev_warn(prmd->dev, "%s: %s timed out\n", + __func__, prmd->pd.name); + + omap_prm_domain_show_state(prmd, "off: new state"); + + return 0; +} + +/* + * Note that ti-sysc already manages the module clocks separately so + * no need to manage those. Interconnect instances need clocks managed + * for simple-pm-bus. + */ +static int omap_prm_domain_attach_clock(struct device *dev, + struct omap_prm_domain *prmd) +{ + struct device_node *np = dev->of_node; + int error; + + if (!of_device_is_compatible(np, "simple-pm-bus")) + return 0; + + if (!of_property_read_bool(np, "clocks")) + return 0; + + error = pm_clk_create(dev); + if (error) + return error; + + error = of_pm_clk_add_clks(dev); + if (error < 0) { + pm_clk_destroy(dev); + return error; + } + + prmd->uses_pm_clk = 1; + + return 0; +} + +static int omap_prm_domain_attach_dev(struct generic_pm_domain *domain, + struct device *dev) +{ + struct generic_pm_domain_data *genpd_data; + struct of_phandle_args pd_args; + struct omap_prm_domain *prmd; + struct device_node *np; + int ret; + + prmd = genpd_to_prm_domain(domain); + np = dev->of_node; + + ret = of_parse_phandle_with_args(np, "power-domains", + "#power-domain-cells", 0, &pd_args); + if (ret < 0) + return ret; + + if (pd_args.args_count != 0) + dev_warn(dev, "%s: unusupported #power-domain-cells: %i\n", + prmd->pd.name, pd_args.args_count); + + genpd_data = dev_gpd_data(dev); + genpd_data->data = NULL; + + ret = omap_prm_domain_attach_clock(dev, prmd); + if (ret) + return ret; + + return 0; +} + +static void omap_prm_domain_detach_dev(struct generic_pm_domain *domain, + struct device *dev) +{ + struct generic_pm_domain_data *genpd_data; + struct omap_prm_domain *prmd; + + prmd = genpd_to_prm_domain(domain); + if (prmd->uses_pm_clk) + pm_clk_destroy(dev); + genpd_data = dev_gpd_data(dev); + genpd_data->data = NULL; +} + +static int omap_prm_domain_init(struct device *dev, struct omap_prm *prm) +{ + struct omap_prm_domain *prmd; + struct device_node *np = dev->of_node; + const struct omap_prm_data *data; + const char *name; + int error; + + if (!of_property_present(dev->of_node, "#power-domain-cells")) + return 0; + + of_node_put(dev->of_node); + + prmd = devm_kzalloc(dev, sizeof(*prmd), GFP_KERNEL); + if (!prmd) + return -ENOMEM; + + data = prm->data; + name = devm_kasprintf(dev, GFP_KERNEL, "prm_%s", + data->name); + + prmd->dev = dev; + prmd->prm = prm; + prmd->cap = prmd->prm->data->dmap; + prmd->pwrstctrl = prmd->prm->data->pwrstctrl; + prmd->pwrstst = prmd->prm->data->pwrstst; + + prmd->pd.name = name; + prmd->pd.power_on = omap_prm_domain_power_on; + prmd->pd.power_off = omap_prm_domain_power_off; + prmd->pd.attach_dev = omap_prm_domain_attach_dev; + prmd->pd.detach_dev = omap_prm_domain_detach_dev; + prmd->pd.flags = GENPD_FLAG_PM_CLK; + + pm_genpd_init(&prmd->pd, NULL, true); + error = of_genpd_add_provider_simple(np, &prmd->pd); + if (error) + pm_genpd_remove(&prmd->pd); + else + prm->prmd = prmd; + + return error; +} + +static bool _is_valid_reset(struct omap_reset_data *reset, unsigned long id) +{ + if (reset->mask & BIT(id)) + return true; + + return false; +} + +static int omap_reset_get_st_bit(struct omap_reset_data *reset, + unsigned long id) +{ + const struct omap_rst_map *map = reset->prm->data->rstmap; + + while (map->rst >= 0) { + if (map->rst == id) + return map->st; + + map++; + } + + return id; +} + +static int omap_reset_status(struct reset_controller_dev *rcdev, + unsigned long id) +{ + struct omap_reset_data *reset = to_omap_reset_data(rcdev); + u32 v; + int st_bit = omap_reset_get_st_bit(reset, id); + bool has_rstst = reset->prm->data->rstst || + (reset->prm->data->flags & OMAP_PRM_HAS_RSTST); + + /* Check if we have rstst */ + if (!has_rstst) + return -ENOTSUPP; + + /* Check if hw reset line is asserted */ + v = readl_relaxed(reset->prm->base + reset->prm->data->rstctrl); + if (v & BIT(id)) + return 1; + + /* + * Check reset status, high value means reset sequence has been + * completed successfully so we can return 0 here (reset deasserted) + */ + v = readl_relaxed(reset->prm->base + reset->prm->data->rstst); + v >>= st_bit; + v &= 1; + + return !v; +} + +static int omap_reset_assert(struct reset_controller_dev *rcdev, + unsigned long id) +{ + struct omap_reset_data *reset = to_omap_reset_data(rcdev); + u32 v; + unsigned long flags; + + /* assert the reset control line */ + spin_lock_irqsave(&reset->lock, flags); + v = readl_relaxed(reset->prm->base + reset->prm->data->rstctrl); + v |= 1 << id; + writel_relaxed(v, reset->prm->base + reset->prm->data->rstctrl); + spin_unlock_irqrestore(&reset->lock, flags); + + return 0; +} + +static int omap_reset_deassert(struct reset_controller_dev *rcdev, + unsigned long id) +{ + struct omap_reset_data *reset = to_omap_reset_data(rcdev); + u32 v; + int st_bit; + bool has_rstst; + unsigned long flags; + struct ti_prm_platform_data *pdata = dev_get_platdata(reset->dev); + int ret = 0; + + /* Nothing to do if the reset is already deasserted */ + if (!omap_reset_status(rcdev, id)) + return 0; + + has_rstst = reset->prm->data->rstst || + (reset->prm->data->flags & OMAP_PRM_HAS_RSTST); + + if (has_rstst) { + st_bit = omap_reset_get_st_bit(reset, id); + + /* Clear the reset status by writing 1 to the status bit */ + v = 1 << st_bit; + writel_relaxed(v, reset->prm->base + reset->prm->data->rstst); + } + + if (reset->clkdm) + pdata->clkdm_deny_idle(reset->clkdm); + + /* de-assert the reset control line */ + spin_lock_irqsave(&reset->lock, flags); + v = readl_relaxed(reset->prm->base + reset->prm->data->rstctrl); + v &= ~(1 << id); + writel_relaxed(v, reset->prm->base + reset->prm->data->rstctrl); + spin_unlock_irqrestore(&reset->lock, flags); + + /* wait for the reset bit to clear */ + ret = readl_relaxed_poll_timeout_atomic(reset->prm->base + + reset->prm->data->rstctrl, + v, !(v & BIT(id)), 1, + OMAP_RESET_MAX_WAIT); + if (ret) + pr_err("%s: timedout waiting for %s:%lu\n", __func__, + reset->prm->data->name, id); + + /* wait for the status to be set */ + if (has_rstst) { + ret = readl_relaxed_poll_timeout_atomic(reset->prm->base + + reset->prm->data->rstst, + v, v & BIT(st_bit), 1, + OMAP_RESET_MAX_WAIT); + if (ret) + pr_err("%s: timedout waiting for %s:%lu\n", __func__, + reset->prm->data->name, id); + } + + if (reset->clkdm) + pdata->clkdm_allow_idle(reset->clkdm); + + return ret; +} + +static const struct reset_control_ops omap_reset_ops = { + .assert = omap_reset_assert, + .deassert = omap_reset_deassert, + .status = omap_reset_status, +}; + +static int omap_prm_reset_xlate(struct reset_controller_dev *rcdev, + const struct of_phandle_args *reset_spec) +{ + struct omap_reset_data *reset = to_omap_reset_data(rcdev); + + if (!_is_valid_reset(reset, reset_spec->args[0])) + return -EINVAL; + + return reset_spec->args[0]; +} + +static int omap_prm_reset_init(struct platform_device *pdev, + struct omap_prm *prm) +{ + struct omap_reset_data *reset; + const struct omap_rst_map *map; + struct ti_prm_platform_data *pdata = dev_get_platdata(&pdev->dev); + char buf[32]; + u32 v; + + /* + * Check if we have controllable resets. If either rstctrl is non-zero + * or OMAP_PRM_HAS_RSTCTRL flag is set, we have reset control register + * for the domain. + */ + if (!prm->data->rstctrl && !(prm->data->flags & OMAP_PRM_HAS_RSTCTRL)) + return 0; + + /* Check if we have the pdata callbacks in place */ + if (!pdata || !pdata->clkdm_lookup || !pdata->clkdm_deny_idle || + !pdata->clkdm_allow_idle) + return -EINVAL; + + map = prm->data->rstmap; + if (!map) + return -EINVAL; + + reset = devm_kzalloc(&pdev->dev, sizeof(*reset), GFP_KERNEL); + if (!reset) + return -ENOMEM; + + reset->rcdev.owner = THIS_MODULE; + reset->rcdev.ops = &omap_reset_ops; + reset->rcdev.of_node = pdev->dev.of_node; + reset->rcdev.nr_resets = OMAP_MAX_RESETS; + reset->rcdev.of_xlate = omap_prm_reset_xlate; + reset->rcdev.of_reset_n_cells = 1; + reset->dev = &pdev->dev; + spin_lock_init(&reset->lock); + + reset->prm = prm; + + sprintf(buf, "%s_clkdm", prm->data->clkdm_name ? prm->data->clkdm_name : + prm->data->name); + + if (!(prm->data->flags & OMAP_PRM_HAS_NO_CLKDM)) { + reset->clkdm = pdata->clkdm_lookup(buf); + if (!reset->clkdm) + return -EINVAL; + } + + while (map->rst >= 0) { + reset->mask |= BIT(map->rst); + map++; + } + + /* Quirk handling to assert rst_map_012 bits on reset and avoid errors */ + if (prm->data->rstmap == rst_map_012) { + v = readl_relaxed(reset->prm->base + reset->prm->data->rstctrl); + if ((v & reset->mask) != reset->mask) { + dev_dbg(&pdev->dev, "Asserting all resets: %08x\n", v); + writel_relaxed(reset->mask, reset->prm->base + + reset->prm->data->rstctrl); + } + } + + return devm_reset_controller_register(&pdev->dev, &reset->rcdev); +} + +static int omap_prm_probe(struct platform_device *pdev) +{ + struct resource *res; + const struct omap_prm_data *data; + struct omap_prm *prm; + int ret; + + data = of_device_get_match_data(&pdev->dev); + if (!data) + return -ENOTSUPP; + + prm = devm_kzalloc(&pdev->dev, sizeof(*prm), GFP_KERNEL); + if (!prm) + return -ENOMEM; + + prm->base = devm_platform_get_and_ioremap_resource(pdev, 0, &res); + if (IS_ERR(prm->base)) + return PTR_ERR(prm->base); + + while (data->base != res->start) { + if (!data->base) + return -EINVAL; + data++; + } + + prm->data = data; + + ret = omap_prm_domain_init(&pdev->dev, prm); + if (ret) + return ret; + + ret = omap_prm_reset_init(pdev, prm); + if (ret) + goto err_domain; + + return 0; + +err_domain: + of_genpd_del_provider(pdev->dev.of_node); + pm_genpd_remove(&prm->prmd->pd); + + return ret; +} + +static struct platform_driver omap_prm_driver = { + .probe = omap_prm_probe, + .driver = { + .name = KBUILD_MODNAME, + .of_match_table = omap_prm_id_table, + }, +}; +builtin_platform_driver(omap_prm_driver); diff --git a/drivers/pmdomain/ti/ti_sci_pm_domains.c b/drivers/pmdomain/ti/ti_sci_pm_domains.c new file mode 100644 index 000000000000..34645104fe45 --- /dev/null +++ b/drivers/pmdomain/ti/ti_sci_pm_domains.c @@ -0,0 +1,204 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * TI SCI Generic Power Domain Driver + * + * Copyright (C) 2015-2017 Texas Instruments Incorporated - http://www.ti.com/ + * J Keerthy + * Dave Gerlach + */ + +#include +#include +#include +#include +#include +#include +#include +#include + +/** + * struct ti_sci_genpd_provider: holds common TI SCI genpd provider data + * @ti_sci: handle to TI SCI protocol driver that provides ops to + * communicate with system control processor. + * @dev: pointer to dev for the driver for devm allocs + * @pd_list: list of all the power domains on the device + * @data: onecell data for genpd core + */ +struct ti_sci_genpd_provider { + const struct ti_sci_handle *ti_sci; + struct device *dev; + struct list_head pd_list; + struct genpd_onecell_data data; +}; + +/** + * struct ti_sci_pm_domain: TI specific data needed for power domain + * @idx: index of the device that identifies it with the system + * control processor. + * @exclusive: Permissions for exclusive request or shared request of the + * device. + * @pd: generic_pm_domain for use with the genpd framework + * @node: link for the genpd list + * @parent: link to the parent TI SCI genpd provider + */ +struct ti_sci_pm_domain { + int idx; + u8 exclusive; + struct generic_pm_domain pd; + struct list_head node; + struct ti_sci_genpd_provider *parent; +}; + +#define genpd_to_ti_sci_pd(gpd) container_of(gpd, struct ti_sci_pm_domain, pd) + +/* + * ti_sci_pd_power_off(): genpd power down hook + * @domain: pointer to the powerdomain to power off + */ +static int ti_sci_pd_power_off(struct generic_pm_domain *domain) +{ + struct ti_sci_pm_domain *pd = genpd_to_ti_sci_pd(domain); + const struct ti_sci_handle *ti_sci = pd->parent->ti_sci; + + return ti_sci->ops.dev_ops.put_device(ti_sci, pd->idx); +} + +/* + * ti_sci_pd_power_on(): genpd power up hook + * @domain: pointer to the powerdomain to power on + */ +static int ti_sci_pd_power_on(struct generic_pm_domain *domain) +{ + struct ti_sci_pm_domain *pd = genpd_to_ti_sci_pd(domain); + const struct ti_sci_handle *ti_sci = pd->parent->ti_sci; + + if (pd->exclusive) + return ti_sci->ops.dev_ops.get_device_exclusive(ti_sci, + pd->idx); + else + return ti_sci->ops.dev_ops.get_device(ti_sci, pd->idx); +} + +/* + * ti_sci_pd_xlate(): translation service for TI SCI genpds + * @genpdspec: DT identification data for the genpd + * @data: genpd core data for all the powerdomains on the device + */ +static struct generic_pm_domain *ti_sci_pd_xlate( + struct of_phandle_args *genpdspec, + void *data) +{ + struct genpd_onecell_data *genpd_data = data; + unsigned int idx = genpdspec->args[0]; + + if (genpdspec->args_count != 1 && genpdspec->args_count != 2) + return ERR_PTR(-EINVAL); + + if (idx >= genpd_data->num_domains) { + pr_err("%s: invalid domain index %u\n", __func__, idx); + return ERR_PTR(-EINVAL); + } + + if (!genpd_data->domains[idx]) + return ERR_PTR(-ENOENT); + + genpd_to_ti_sci_pd(genpd_data->domains[idx])->exclusive = + genpdspec->args[1]; + + return genpd_data->domains[idx]; +} + +static const struct of_device_id ti_sci_pm_domain_matches[] = { + { .compatible = "ti,sci-pm-domain", }, + { }, +}; +MODULE_DEVICE_TABLE(of, ti_sci_pm_domain_matches); + +static int ti_sci_pm_domain_probe(struct platform_device *pdev) +{ + struct device *dev = &pdev->dev; + struct ti_sci_genpd_provider *pd_provider; + struct ti_sci_pm_domain *pd; + struct device_node *np; + struct of_phandle_args args; + int ret; + u32 max_id = 0; + int index; + + pd_provider = devm_kzalloc(dev, sizeof(*pd_provider), GFP_KERNEL); + if (!pd_provider) + return -ENOMEM; + + pd_provider->ti_sci = devm_ti_sci_get_handle(dev); + if (IS_ERR(pd_provider->ti_sci)) + return PTR_ERR(pd_provider->ti_sci); + + pd_provider->dev = dev; + + INIT_LIST_HEAD(&pd_provider->pd_list); + + /* Find highest device ID used for power domains */ + for_each_node_with_property(np, "power-domains") { + index = 0; + + while (1) { + ret = of_parse_phandle_with_args(np, "power-domains", + "#power-domain-cells", + index, &args); + if (ret) + break; + + if (args.args_count >= 1 && args.np == dev->of_node) { + if (args.args[0] > max_id) + max_id = args.args[0]; + + pd = devm_kzalloc(dev, sizeof(*pd), GFP_KERNEL); + if (!pd) + return -ENOMEM; + + pd->pd.name = devm_kasprintf(dev, GFP_KERNEL, + "pd:%d", + args.args[0]); + if (!pd->pd.name) + return -ENOMEM; + + pd->pd.power_off = ti_sci_pd_power_off; + pd->pd.power_on = ti_sci_pd_power_on; + pd->idx = args.args[0]; + pd->parent = pd_provider; + + pm_genpd_init(&pd->pd, NULL, true); + + list_add(&pd->node, &pd_provider->pd_list); + } + index++; + } + } + + pd_provider->data.domains = + devm_kcalloc(dev, max_id + 1, + sizeof(*pd_provider->data.domains), + GFP_KERNEL); + if (!pd_provider->data.domains) + return -ENOMEM; + + pd_provider->data.num_domains = max_id + 1; + pd_provider->data.xlate = ti_sci_pd_xlate; + + list_for_each_entry(pd, &pd_provider->pd_list, node) + pd_provider->data.domains[pd->idx] = &pd->pd; + + return of_genpd_add_provider_onecell(dev->of_node, &pd_provider->data); +} + +static struct platform_driver ti_sci_pm_domains_driver = { + .probe = ti_sci_pm_domain_probe, + .driver = { + .name = "ti_sci_pm_domains", + .of_match_table = ti_sci_pm_domain_matches, + }, +}; +module_platform_driver(ti_sci_pm_domains_driver); +MODULE_LICENSE("GPL v2"); +MODULE_DESCRIPTION("TI System Control Interface (SCI) Power Domain driver"); +MODULE_AUTHOR("Dave Gerlach"); diff --git a/drivers/pmdomain/xilinx/Makefile b/drivers/pmdomain/xilinx/Makefile new file mode 100644 index 000000000000..a706ab699cfa --- /dev/null +++ b/drivers/pmdomain/xilinx/Makefile @@ -0,0 +1,2 @@ +# SPDX-License-Identifier: GPL-2.0 +obj-$(CONFIG_ZYNQMP_PM_DOMAINS) += zynqmp-pm-domains.o diff --git a/drivers/pmdomain/xilinx/zynqmp-pm-domains.c b/drivers/pmdomain/xilinx/zynqmp-pm-domains.c new file mode 100644 index 000000000000..69d03ad4cf1e --- /dev/null +++ b/drivers/pmdomain/xilinx/zynqmp-pm-domains.c @@ -0,0 +1,322 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * ZynqMP Generic PM domain support + * + * Copyright (C) 2015-2019 Xilinx, Inc. + * + * Davorin Mista + * Jolly Shah + * Rajan Vaja + */ + +#include +#include +#include +#include +#include +#include +#include + +#include + +#define ZYNQMP_NUM_DOMAINS (100) + +static int min_capability; + +/** + * struct zynqmp_pm_domain - Wrapper around struct generic_pm_domain + * @gpd: Generic power domain + * @node_id: PM node ID corresponding to device inside PM domain + * @requested: The PM node mapped to the PM domain has been requested + */ +struct zynqmp_pm_domain { + struct generic_pm_domain gpd; + u32 node_id; + bool requested; +}; + +#define to_zynqmp_pm_domain(pm_domain) \ + container_of(pm_domain, struct zynqmp_pm_domain, gpd) + +/** + * zynqmp_gpd_is_active_wakeup_path() - Check if device is in wakeup source + * path + * @dev: Device to check for wakeup source path + * @not_used: Data member (not required) + * + * This function is checks device's child hierarchy and checks if any device is + * set as wakeup source. + * + * Return: 1 if device is in wakeup source path else 0 + */ +static int zynqmp_gpd_is_active_wakeup_path(struct device *dev, void *not_used) +{ + int may_wakeup; + + may_wakeup = device_may_wakeup(dev); + if (may_wakeup) + return may_wakeup; + + return device_for_each_child(dev, NULL, + zynqmp_gpd_is_active_wakeup_path); +} + +/** + * zynqmp_gpd_power_on() - Power on PM domain + * @domain: Generic PM domain + * + * This function is called before devices inside a PM domain are resumed, to + * power on PM domain. + * + * Return: 0 on success, error code otherwise + */ +static int zynqmp_gpd_power_on(struct generic_pm_domain *domain) +{ + struct zynqmp_pm_domain *pd = to_zynqmp_pm_domain(domain); + int ret; + + ret = zynqmp_pm_set_requirement(pd->node_id, + ZYNQMP_PM_CAPABILITY_ACCESS, + ZYNQMP_PM_MAX_QOS, + ZYNQMP_PM_REQUEST_ACK_BLOCKING); + if (ret) { + dev_err(&domain->dev, + "failed to set requirement to 0x%x for PM node id %d: %d\n", + ZYNQMP_PM_CAPABILITY_ACCESS, pd->node_id, ret); + return ret; + } + + dev_dbg(&domain->dev, "set requirement to 0x%x for PM node id %d\n", + ZYNQMP_PM_CAPABILITY_ACCESS, pd->node_id); + + return 0; +} + +/** + * zynqmp_gpd_power_off() - Power off PM domain + * @domain: Generic PM domain + * + * This function is called after devices inside a PM domain are suspended, to + * power off PM domain. + * + * Return: 0 on success, error code otherwise + */ +static int zynqmp_gpd_power_off(struct generic_pm_domain *domain) +{ + struct zynqmp_pm_domain *pd = to_zynqmp_pm_domain(domain); + int ret; + struct pm_domain_data *pdd, *tmp; + u32 capabilities = min_capability; + bool may_wakeup; + + /* If domain is already released there is nothing to be done */ + if (!pd->requested) { + dev_dbg(&domain->dev, "PM node id %d is already released\n", + pd->node_id); + return 0; + } + + list_for_each_entry_safe(pdd, tmp, &domain->dev_list, list_node) { + /* If device is in wakeup path, set capability to WAKEUP */ + may_wakeup = zynqmp_gpd_is_active_wakeup_path(pdd->dev, NULL); + if (may_wakeup) { + dev_dbg(pdd->dev, "device is in wakeup path in %s\n", + domain->name); + capabilities = ZYNQMP_PM_CAPABILITY_WAKEUP; + break; + } + } + + ret = zynqmp_pm_set_requirement(pd->node_id, capabilities, 0, + ZYNQMP_PM_REQUEST_ACK_NO); + if (ret) { + dev_err(&domain->dev, + "failed to set requirement to 0x%x for PM node id %d: %d\n", + capabilities, pd->node_id, ret); + return ret; + } + + dev_dbg(&domain->dev, "set requirement to 0x%x for PM node id %d\n", + capabilities, pd->node_id); + + return 0; +} + +/** + * zynqmp_gpd_attach_dev() - Attach device to the PM domain + * @domain: Generic PM domain + * @dev: Device to attach + * + * Return: 0 on success, error code otherwise + */ +static int zynqmp_gpd_attach_dev(struct generic_pm_domain *domain, + struct device *dev) +{ + struct zynqmp_pm_domain *pd = to_zynqmp_pm_domain(domain); + struct device_link *link; + int ret; + + link = device_link_add(dev, &domain->dev, DL_FLAG_SYNC_STATE_ONLY); + if (!link) + dev_dbg(&domain->dev, "failed to create device link for %s\n", + dev_name(dev)); + + /* If this is not the first device to attach there is nothing to do */ + if (domain->device_count) + return 0; + + ret = zynqmp_pm_request_node(pd->node_id, 0, 0, + ZYNQMP_PM_REQUEST_ACK_BLOCKING); + if (ret) { + dev_err(&domain->dev, "%s request failed for node %d: %d\n", + domain->name, pd->node_id, ret); + return ret; + } + + pd->requested = true; + + dev_dbg(&domain->dev, "%s requested PM node id %d\n", + dev_name(dev), pd->node_id); + + return 0; +} + +/** + * zynqmp_gpd_detach_dev() - Detach device from the PM domain + * @domain: Generic PM domain + * @dev: Device to detach + */ +static void zynqmp_gpd_detach_dev(struct generic_pm_domain *domain, + struct device *dev) +{ + struct zynqmp_pm_domain *pd = to_zynqmp_pm_domain(domain); + int ret; + + /* If this is not the last device to detach there is nothing to do */ + if (domain->device_count) + return; + + ret = zynqmp_pm_release_node(pd->node_id); + if (ret) { + dev_err(&domain->dev, "failed to release PM node id %d: %d\n", + pd->node_id, ret); + return; + } + + pd->requested = false; + + dev_dbg(&domain->dev, "%s released PM node id %d\n", + dev_name(dev), pd->node_id); +} + +static struct generic_pm_domain *zynqmp_gpd_xlate + (struct of_phandle_args *genpdspec, void *data) +{ + struct genpd_onecell_data *genpd_data = data; + unsigned int i, idx = genpdspec->args[0]; + struct zynqmp_pm_domain *pd; + + pd = to_zynqmp_pm_domain(genpd_data->domains[0]); + + if (genpdspec->args_count != 1) + return ERR_PTR(-EINVAL); + + /* Check for existing pm domains */ + for (i = 0; i < ZYNQMP_NUM_DOMAINS; i++) { + if (pd[i].node_id == idx) + goto done; + } + + /* + * Add index in empty node_id of power domain list as no existing + * power domain found for current index. + */ + for (i = 0; i < ZYNQMP_NUM_DOMAINS; i++) { + if (pd[i].node_id == 0) { + pd[i].node_id = idx; + break; + } + } + +done: + if (!genpd_data->domains[i] || i == ZYNQMP_NUM_DOMAINS) + return ERR_PTR(-ENOENT); + + return genpd_data->domains[i]; +} + +static int zynqmp_gpd_probe(struct platform_device *pdev) +{ + int i; + struct genpd_onecell_data *zynqmp_pd_data; + struct generic_pm_domain **domains; + struct zynqmp_pm_domain *pd; + struct device *dev = &pdev->dev; + + pd = devm_kcalloc(dev, ZYNQMP_NUM_DOMAINS, sizeof(*pd), GFP_KERNEL); + if (!pd) + return -ENOMEM; + + zynqmp_pd_data = devm_kzalloc(dev, sizeof(*zynqmp_pd_data), GFP_KERNEL); + if (!zynqmp_pd_data) + return -ENOMEM; + + zynqmp_pd_data->xlate = zynqmp_gpd_xlate; + + domains = devm_kcalloc(dev, ZYNQMP_NUM_DOMAINS, sizeof(*domains), + GFP_KERNEL); + if (!domains) + return -ENOMEM; + + if (!of_device_is_compatible(dev->parent->of_node, + "xlnx,zynqmp-firmware")) + min_capability = ZYNQMP_PM_CAPABILITY_UNUSABLE; + + for (i = 0; i < ZYNQMP_NUM_DOMAINS; i++, pd++) { + pd->node_id = 0; + pd->gpd.name = kasprintf(GFP_KERNEL, "domain%d", i); + pd->gpd.power_off = zynqmp_gpd_power_off; + pd->gpd.power_on = zynqmp_gpd_power_on; + pd->gpd.attach_dev = zynqmp_gpd_attach_dev; + pd->gpd.detach_dev = zynqmp_gpd_detach_dev; + + domains[i] = &pd->gpd; + + /* Mark all PM domains as initially powered off */ + pm_genpd_init(&pd->gpd, NULL, true); + } + + zynqmp_pd_data->domains = domains; + zynqmp_pd_data->num_domains = ZYNQMP_NUM_DOMAINS; + of_genpd_add_provider_onecell(dev->parent->of_node, zynqmp_pd_data); + + return 0; +} + +static int zynqmp_gpd_remove(struct platform_device *pdev) +{ + of_genpd_del_provider(pdev->dev.parent->of_node); + + return 0; +} + +static void zynqmp_gpd_sync_state(struct device *dev) +{ + int ret; + + ret = zynqmp_pm_init_finalize(); + if (ret) + dev_warn(dev, "failed to release power management to firmware\n"); +} + +static struct platform_driver zynqmp_power_domain_driver = { + .driver = { + .name = "zynqmp_power_controller", + .sync_state = zynqmp_gpd_sync_state, + }, + .probe = zynqmp_gpd_probe, + .remove = zynqmp_gpd_remove, +}; +module_platform_driver(zynqmp_power_domain_driver); + +MODULE_ALIAS("platform:zynqmp_power_controller"); diff --git a/drivers/power/reset/Kconfig b/drivers/power/reset/Kconfig index 59e1ebb7842e..411e00b255d6 100644 --- a/drivers/power/reset/Kconfig +++ b/drivers/power/reset/Kconfig @@ -300,7 +300,7 @@ config NVMEM_REBOOT_MODE config POWER_MLXBF tristate "Mellanox BlueField power handling driver" - depends on (GPIO_MLXBF2 && ACPI) + depends on (GPIO_MLXBF2 || GPIO_MLXBF3) && ACPI help This driver supports reset or low power mode handling for Mellanox BlueField. diff --git a/drivers/power/reset/pwr-mlxbf.c b/drivers/power/reset/pwr-mlxbf.c index 12dedf841a44..de35d24bb7ef 100644 --- a/drivers/power/reset/pwr-mlxbf.c +++ b/drivers/power/reset/pwr-mlxbf.c @@ -1,4 +1,4 @@ -// SPDX-License-Identifier: GPL-2.0-only or BSD-3-Clause +// SPDX-License-Identifier: GPL-2.0-only OR BSD-3-Clause /* * Copyright (c) 2022 NVIDIA CORPORATION & AFFILIATES. diff --git a/drivers/power/reset/vexpress-poweroff.c b/drivers/power/reset/vexpress-poweroff.c index 447ffdacddf9..17064d7b19f6 100644 --- a/drivers/power/reset/vexpress-poweroff.c +++ b/drivers/power/reset/vexpress-poweroff.c @@ -121,7 +121,7 @@ static int vexpress_reset_probe(struct platform_device *pdev) return PTR_ERR(regmap); dev_set_drvdata(&pdev->dev, regmap); - switch ((enum vexpress_reset_func)match->data) { + switch ((uintptr_t)match->data) { case FUNC_SHUTDOWN: vexpress_power_off_device = &pdev->dev; pm_power_off = vexpress_power_off; diff --git a/drivers/power/supply/Kconfig b/drivers/power/supply/Kconfig index 663a1c423806..a61bb1283e19 100644 --- a/drivers/power/supply/Kconfig +++ b/drivers/power/supply/Kconfig @@ -769,6 +769,7 @@ config BATTERY_RT5033 config CHARGER_RT5033 tristate "RT5033 battery charger support" depends on MFD_RT5033 + depends on EXTCON || !EXTCON help This adds support for battery charger in Richtek RT5033 PMIC. The device supports pre-charge mode, fast charge mode and diff --git a/drivers/power/supply/ab8500_btemp.c b/drivers/power/supply/ab8500_btemp.c index 6f83e99d2eb7..ce36d6ca3422 100644 --- a/drivers/power/supply/ab8500_btemp.c +++ b/drivers/power/supply/ab8500_btemp.c @@ -115,7 +115,6 @@ struct ab8500_btemp { static enum power_supply_property ab8500_btemp_props[] = { POWER_SUPPLY_PROP_PRESENT, POWER_SUPPLY_PROP_ONLINE, - POWER_SUPPLY_PROP_TECHNOLOGY, POWER_SUPPLY_PROP_TEMP, }; @@ -532,12 +531,6 @@ static int ab8500_btemp_get_property(struct power_supply *psy, else val->intval = 1; break; - case POWER_SUPPLY_PROP_TECHNOLOGY: - if (di->bm->bi) - val->intval = di->bm->bi->technology; - else - val->intval = POWER_SUPPLY_TECHNOLOGY_UNKNOWN; - break; case POWER_SUPPLY_PROP_TEMP: val->intval = ab8500_btemp_get_temp(di); break; @@ -662,7 +655,7 @@ static char *supply_interface[] = { static const struct power_supply_desc ab8500_btemp_desc = { .name = "ab8500_btemp", - .type = POWER_SUPPLY_TYPE_BATTERY, + .type = POWER_SUPPLY_TYPE_UNKNOWN, .properties = ab8500_btemp_props, .num_properties = ARRAY_SIZE(ab8500_btemp_props), .get_property = ab8500_btemp_get_property, diff --git a/drivers/power/supply/ab8500_chargalg.c b/drivers/power/supply/ab8500_chargalg.c index ea4ad61d4c7e..2205ea0834a6 100644 --- a/drivers/power/supply/ab8500_chargalg.c +++ b/drivers/power/supply/ab8500_chargalg.c @@ -1720,7 +1720,7 @@ static char *supply_interface[] = { static const struct power_supply_desc ab8500_chargalg_desc = { .name = "ab8500_chargalg", - .type = POWER_SUPPLY_TYPE_BATTERY, + .type = POWER_SUPPLY_TYPE_UNKNOWN, .properties = ab8500_chargalg_props, .num_properties = ARRAY_SIZE(ab8500_chargalg_props), .get_property = ab8500_chargalg_get_property, diff --git a/drivers/power/supply/mt6370-charger.c b/drivers/power/supply/mt6370-charger.c index f27dae5043f5..a9641bd3d8cf 100644 --- a/drivers/power/supply/mt6370-charger.c +++ b/drivers/power/supply/mt6370-charger.c @@ -324,7 +324,7 @@ static int mt6370_chg_toggle_cfo(struct mt6370_priv *priv) if (fl_strobe) { dev_err(priv->dev, "Flash led is still in strobe mode\n"); - return ret; + return -EINVAL; } /* cfo off */ diff --git a/drivers/power/supply/power_supply_sysfs.c b/drivers/power/supply/power_supply_sysfs.c index 06e5b6b0e255..d483a81560ab 100644 --- a/drivers/power/supply/power_supply_sysfs.c +++ b/drivers/power/supply/power_supply_sysfs.c @@ -482,6 +482,13 @@ int power_supply_uevent(const struct device *dev, struct kobj_uevent_env *env) if (ret) return ret; + /* + * Kernel generates KOBJ_REMOVE uevent in device removal path, after + * resources have been freed. Exit early to avoid use-after-free. + */ + if (psy->removing) + return 0; + prop_buf = (char *)get_zeroed_page(GFP_KERNEL); if (!prop_buf) return -ENOMEM; diff --git a/drivers/power/supply/qcom_battmgr.c b/drivers/power/supply/qcom_battmgr.c index de77df97b3a4..ec163d1bcd18 100644 --- a/drivers/power/supply/qcom_battmgr.c +++ b/drivers/power/supply/qcom_battmgr.c @@ -105,7 +105,7 @@ struct qcom_battmgr_property_request { struct qcom_battmgr_update_request { struct pmic_glink_hdr hdr; - u32 battery_id; + __le32 battery_id; }; struct qcom_battmgr_charge_time_request { @@ -1282,9 +1282,9 @@ static void qcom_battmgr_enable_worker(struct work_struct *work) { struct qcom_battmgr *battmgr = container_of(work, struct qcom_battmgr, enable_work); struct qcom_battmgr_enable_request req = { - .hdr.owner = PMIC_GLINK_OWNER_BATTMGR, - .hdr.type = PMIC_GLINK_NOTIFY, - .hdr.opcode = BATTMGR_REQUEST_NOTIFICATION, + .hdr.owner = cpu_to_le32(PMIC_GLINK_OWNER_BATTMGR), + .hdr.type = cpu_to_le32(PMIC_GLINK_NOTIFY), + .hdr.opcode = cpu_to_le32(BATTMGR_REQUEST_NOTIFICATION), }; int ret; diff --git a/drivers/power/supply/rk817_charger.c b/drivers/power/supply/rk817_charger.c index 8328bcea1a29..f64daf5a41d9 100644 --- a/drivers/power/supply/rk817_charger.c +++ b/drivers/power/supply/rk817_charger.c @@ -1045,6 +1045,13 @@ static void rk817_charging_monitor(struct work_struct *work) queue_delayed_work(system_wq, &charger->work, msecs_to_jiffies(8000)); } +static void rk817_cleanup_node(void *data) +{ + struct device_node *node = data; + + of_node_put(node); +} + static int rk817_charger_probe(struct platform_device *pdev) { struct rk808 *rk808 = dev_get_drvdata(pdev->dev.parent); @@ -1061,11 +1068,13 @@ static int rk817_charger_probe(struct platform_device *pdev) if (!node) return -ENODEV; + ret = devm_add_action_or_reset(&pdev->dev, rk817_cleanup_node, node); + if (ret) + return ret; + charger = devm_kzalloc(&pdev->dev, sizeof(*charger), GFP_KERNEL); - if (!charger) { - of_node_put(node); + if (!charger) return -ENOMEM; - } charger->rk808 = rk808; @@ -1211,3 +1220,4 @@ MODULE_DESCRIPTION("Battery power supply driver for RK817 PMIC"); MODULE_AUTHOR("Maya Matuszczyk "); MODULE_AUTHOR("Chris Morgan "); MODULE_LICENSE("GPL"); +MODULE_ALIAS("platform:rk817-charger"); diff --git a/drivers/power/supply/rt9467-charger.c b/drivers/power/supply/rt9467-charger.c index 683adb18253d..fdfdc83ab045 100644 --- a/drivers/power/supply/rt9467-charger.c +++ b/drivers/power/supply/rt9467-charger.c @@ -598,8 +598,8 @@ static int rt9467_run_aicl(struct rt9467_chg_data *data) reinit_completion(&data->aicl_done); ret = wait_for_completion_timeout(&data->aicl_done, msecs_to_jiffies(3500)); - if (ret) - return ret; + if (ret == 0) + return -ETIMEDOUT; ret = rt9467_get_value_from_ranges(data, F_IAICR, RT9467_RANGE_IAICR, &aicr_get); if (ret) { diff --git a/drivers/power/supply/ucs1002_power.c b/drivers/power/supply/ucs1002_power.c index 954feba6600b..7970843a4f48 100644 --- a/drivers/power/supply/ucs1002_power.c +++ b/drivers/power/supply/ucs1002_power.c @@ -384,7 +384,8 @@ static int ucs1002_get_property(struct power_supply *psy, case POWER_SUPPLY_PROP_USB_TYPE: return ucs1002_get_usb_type(info, val); case POWER_SUPPLY_PROP_HEALTH: - return val->intval = info->health; + val->intval = info->health; + return 0; case POWER_SUPPLY_PROP_PRESENT: val->intval = info->present; return 0; diff --git a/drivers/ptp/ptp_ocp.c b/drivers/ptp/ptp_ocp.c index 20a974ced8d6..a7a6947ab4bc 100644 --- a/drivers/ptp/ptp_ocp.c +++ b/drivers/ptp/ptp_ocp.c @@ -3998,7 +3998,6 @@ ptp_ocp_device_init(struct ptp_ocp *bp, struct pci_dev *pdev) return 0; out: - ptp_ocp_dev_release(&bp->dev); put_device(&bp->dev); return err; } diff --git a/drivers/regulator/core.c b/drivers/regulator/core.c index d8e1caaf207e..3137e40fcd3e 100644 --- a/drivers/regulator/core.c +++ b/drivers/regulator/core.c @@ -5542,6 +5542,8 @@ regulator_register(struct device *dev, goto rinse; } device_initialize(&rdev->dev); + dev_set_drvdata(&rdev->dev, rdev); + rdev->dev.class = ®ulator_class; spin_lock_init(&rdev->err_lock); /* @@ -5603,11 +5605,9 @@ regulator_register(struct device *dev, rdev->supply_name = regulator_desc->supply_name; /* register with sysfs */ - rdev->dev.class = ®ulator_class; rdev->dev.parent = config->dev; dev_set_name(&rdev->dev, "regulator.%lu", (unsigned long) atomic_inc_return(®ulator_no)); - dev_set_drvdata(&rdev->dev, rdev); /* set regulator constraints */ if (init_data) @@ -5724,15 +5724,11 @@ wash: mutex_lock(®ulator_list_mutex); regulator_ena_gpio_free(rdev); mutex_unlock(®ulator_list_mutex); - put_device(&rdev->dev); - rdev = NULL; clean: if (dangling_of_gpiod) gpiod_put(config->ena_gpiod); - if (rdev && rdev->dev.of_node) - of_node_put(rdev->dev.of_node); - kfree(rdev); kfree(config); + put_device(&rdev->dev); rinse: if (dangling_cfg_gpiod) gpiod_put(cfg->ena_gpiod); diff --git a/drivers/regulator/helpers.c b/drivers/regulator/helpers.c index 5ad5f3b3a6b5..d49268336553 100644 --- a/drivers/regulator/helpers.c +++ b/drivers/regulator/helpers.c @@ -197,7 +197,7 @@ int regulator_set_voltage_sel_pickable_regmap(struct regulator_dev *rdev, sel += rdev->desc->linear_ranges[i].min_sel; range = rdev->desc->linear_range_selectors_bitfield[i]; - range <<= ffs(rdev->desc->vsel_mask) - 1; + range <<= ffs(rdev->desc->vsel_range_mask) - 1; if (rdev->desc->vsel_reg == rdev->desc->vsel_range_reg) { ret = regmap_update_bits(rdev->regmap, diff --git a/drivers/regulator/mt6358-regulator.c b/drivers/regulator/mt6358-regulator.c index b9cda2210c33..65fbd95f1dbb 100644 --- a/drivers/regulator/mt6358-regulator.c +++ b/drivers/regulator/mt6358-regulator.c @@ -43,7 +43,7 @@ struct mt6358_regulator_info { .desc = { \ .name = #vreg, \ .of_match = of_match_ptr(match), \ - .ops = &mt6358_volt_range_ops, \ + .ops = &mt6358_buck_ops, \ .type = REGULATOR_VOLTAGE, \ .id = MT6358_ID_##vreg, \ .owner = THIS_MODULE, \ @@ -139,7 +139,7 @@ struct mt6358_regulator_info { .desc = { \ .name = #vreg, \ .of_match = of_match_ptr(match), \ - .ops = &mt6358_volt_range_ops, \ + .ops = &mt6358_buck_ops, \ .type = REGULATOR_VOLTAGE, \ .id = MT6366_ID_##vreg, \ .owner = THIS_MODULE, \ @@ -450,7 +450,7 @@ static unsigned int mt6358_regulator_get_mode(struct regulator_dev *rdev) } } -static const struct regulator_ops mt6358_volt_range_ops = { +static const struct regulator_ops mt6358_buck_ops = { .list_voltage = regulator_list_voltage_linear, .map_voltage = regulator_map_voltage_linear, .set_voltage_sel = regulator_set_voltage_sel_regmap, @@ -464,6 +464,18 @@ static const struct regulator_ops mt6358_volt_range_ops = { .get_mode = mt6358_regulator_get_mode, }; +static const struct regulator_ops mt6358_volt_range_ops = { + .list_voltage = regulator_list_voltage_linear, + .map_voltage = regulator_map_voltage_linear, + .set_voltage_sel = regulator_set_voltage_sel_regmap, + .get_voltage_sel = mt6358_get_buck_voltage_sel, + .set_voltage_time_sel = regulator_set_voltage_time_sel, + .enable = regulator_enable_regmap, + .disable = regulator_disable_regmap, + .is_enabled = regulator_is_enabled_regmap, + .get_status = mt6358_get_status, +}; + static const struct regulator_ops mt6358_volt_table_ops = { .list_voltage = regulator_list_voltage_table, .map_voltage = regulator_map_voltage_iterate, diff --git a/drivers/s390/cio/css.c b/drivers/s390/cio/css.c index 3ef636935a54..3ff46fc694f8 100644 --- a/drivers/s390/cio/css.c +++ b/drivers/s390/cio/css.c @@ -233,17 +233,19 @@ struct subchannel *css_alloc_subchannel(struct subchannel_id schid, */ ret = dma_set_coherent_mask(&sch->dev, DMA_BIT_MASK(31)); if (ret) - goto err; + goto err_lock; /* * But we don't have such restrictions imposed on the stuff that * is handled by the streaming API. */ ret = dma_set_mask(&sch->dev, DMA_BIT_MASK(64)); if (ret) - goto err; + goto err_lock; return sch; +err_lock: + kfree(sch->lock); err: kfree(sch); return ERR_PTR(ret); diff --git a/drivers/s390/net/Kconfig b/drivers/s390/net/Kconfig index 74760c1a163b..4902d45e929c 100644 --- a/drivers/s390/net/Kconfig +++ b/drivers/s390/net/Kconfig @@ -102,7 +102,7 @@ config CCWGROUP config ISM tristate "Support for ISM vPCI Adapter" - depends on PCI && SMC + depends on PCI default n help Select this option if you want to use the Internal Shared Memory diff --git a/drivers/s390/scsi/zfcp_aux.c b/drivers/s390/scsi/zfcp_aux.c index df782646e856..ab2f35bc294d 100644 --- a/drivers/s390/scsi/zfcp_aux.c +++ b/drivers/s390/scsi/zfcp_aux.c @@ -518,12 +518,12 @@ struct zfcp_port *zfcp_port_enqueue(struct zfcp_adapter *adapter, u64 wwpn, if (port) { put_device(&port->dev); retval = -EEXIST; - goto err_out; + goto err_put; } port = kzalloc(sizeof(struct zfcp_port), GFP_KERNEL); if (!port) - goto err_out; + goto err_put; rwlock_init(&port->unit_list_lock); INIT_LIST_HEAD(&port->unit_list); @@ -546,7 +546,7 @@ struct zfcp_port *zfcp_port_enqueue(struct zfcp_adapter *adapter, u64 wwpn, if (dev_set_name(&port->dev, "0x%016llx", (unsigned long long)wwpn)) { kfree(port); - goto err_out; + goto err_put; } retval = -EINVAL; @@ -563,7 +563,8 @@ struct zfcp_port *zfcp_port_enqueue(struct zfcp_adapter *adapter, u64 wwpn, return port; -err_out: +err_put: zfcp_ccw_adapter_put(adapter); +err_out: return ERR_PTR(retval); } diff --git a/drivers/scsi/fnic/fnic.h b/drivers/scsi/fnic/fnic.h index 93c68931a593..22cef283b2b9 100644 --- a/drivers/scsi/fnic/fnic.h +++ b/drivers/scsi/fnic/fnic.h @@ -27,7 +27,7 @@ #define DRV_NAME "fnic" #define DRV_DESCRIPTION "Cisco FCoE HBA Driver" -#define DRV_VERSION "1.6.0.56" +#define DRV_VERSION "1.6.0.57" #define PFX DRV_NAME ": " #define DFX DRV_NAME "%d: " @@ -237,6 +237,8 @@ struct fnic { unsigned int cq_count; struct mutex sgreset_mutex; + spinlock_t sgreset_lock; /* lock for sgreset */ + struct scsi_cmnd *sgreset_sc; struct dentry *fnic_stats_debugfs_host; struct dentry *fnic_stats_debugfs_file; struct dentry *fnic_reset_debugfs_file; diff --git a/drivers/scsi/fnic/fnic_io.h b/drivers/scsi/fnic/fnic_io.h index f4c8769df312..5895ead20e14 100644 --- a/drivers/scsi/fnic/fnic_io.h +++ b/drivers/scsi/fnic/fnic_io.h @@ -52,6 +52,8 @@ struct fnic_io_req { unsigned long start_time; /* in jiffies */ struct completion *abts_done; /* completion for abts */ struct completion *dr_done; /* completion for device reset */ + unsigned int tag; + struct scsi_cmnd *sc; /* midlayer's cmd pointer */ }; enum fnic_port_speeds { diff --git a/drivers/scsi/fnic/fnic_main.c b/drivers/scsi/fnic/fnic_main.c index 984bc5fc55e2..f27f9319e0b2 100644 --- a/drivers/scsi/fnic/fnic_main.c +++ b/drivers/scsi/fnic/fnic_main.c @@ -754,6 +754,8 @@ static int fnic_probe(struct pci_dev *pdev, const struct pci_device_id *ent) for (i = 0; i < FNIC_IO_LOCKS; i++) spin_lock_init(&fnic->io_req_lock[i]); + spin_lock_init(&fnic->sgreset_lock); + err = -ENOMEM; fnic->io_req_pool = mempool_create_slab_pool(2, fnic_io_req_cache); if (!fnic->io_req_pool) diff --git a/drivers/scsi/fnic/fnic_scsi.c b/drivers/scsi/fnic/fnic_scsi.c index 9761b2c9db48..416d81954819 100644 --- a/drivers/scsi/fnic/fnic_scsi.c +++ b/drivers/scsi/fnic/fnic_scsi.c @@ -1047,9 +1047,9 @@ static void fnic_fcpio_itmf_cmpl_handler(struct fnic *fnic, { u8 type; u8 hdr_status; - struct fcpio_tag tag; + struct fcpio_tag ftag; u32 id; - struct scsi_cmnd *sc; + struct scsi_cmnd *sc = NULL; struct fnic_io_req *io_req; struct fnic_stats *fnic_stats = &fnic->fnic_stats; struct abort_stats *abts_stats = &fnic->fnic_stats.abts_stats; @@ -1058,27 +1058,43 @@ static void fnic_fcpio_itmf_cmpl_handler(struct fnic *fnic, unsigned long flags; spinlock_t *io_lock; unsigned long start_time; + unsigned int tag; - fcpio_header_dec(&desc->hdr, &type, &hdr_status, &tag); - fcpio_tag_id_dec(&tag, &id); + fcpio_header_dec(&desc->hdr, &type, &hdr_status, &ftag); + fcpio_tag_id_dec(&ftag, &id); - if ((id & FNIC_TAG_MASK) >= fnic->fnic_max_tag_id) { + tag = id & FNIC_TAG_MASK; + if (tag == fnic->fnic_max_tag_id) { + if (!(id & FNIC_TAG_DEV_RST)) { + shost_printk(KERN_ERR, fnic->lport->host, + "Tag out of range id 0x%x hdr status = %s\n", + id, fnic_fcpio_status_to_str(hdr_status)); + return; + } + } else if (tag > fnic->fnic_max_tag_id) { shost_printk(KERN_ERR, fnic->lport->host, - "Tag out of range tag %x hdr status = %s\n", - id, fnic_fcpio_status_to_str(hdr_status)); + "Tag out of range tag 0x%x hdr status = %s\n", + tag, fnic_fcpio_status_to_str(hdr_status)); return; } - sc = scsi_host_find_tag(fnic->lport->host, id & FNIC_TAG_MASK); + if ((tag == fnic->fnic_max_tag_id) && (id & FNIC_TAG_DEV_RST)) { + sc = fnic->sgreset_sc; + io_lock = &fnic->sgreset_lock; + } else { + sc = scsi_host_find_tag(fnic->lport->host, id & FNIC_TAG_MASK); + io_lock = fnic_io_lock_hash(fnic, sc); + } + WARN_ON_ONCE(!sc); if (!sc) { atomic64_inc(&fnic_stats->io_stats.sc_null); shost_printk(KERN_ERR, fnic->lport->host, "itmf_cmpl sc is null - hdr status = %s tag = 0x%x\n", - fnic_fcpio_status_to_str(hdr_status), id); + fnic_fcpio_status_to_str(hdr_status), tag); return; } - io_lock = fnic_io_lock_hash(fnic, sc); + spin_lock_irqsave(io_lock, flags); io_req = fnic_priv(sc)->io_req; WARN_ON_ONCE(!io_req); @@ -1089,7 +1105,7 @@ static void fnic_fcpio_itmf_cmpl_handler(struct fnic *fnic, shost_printk(KERN_ERR, fnic->lport->host, "itmf_cmpl io_req is null - " "hdr status = %s tag = 0x%x sc 0x%p\n", - fnic_fcpio_status_to_str(hdr_status), id, sc); + fnic_fcpio_status_to_str(hdr_status), tag, sc); return; } start_time = io_req->start_time; @@ -1938,6 +1954,10 @@ static inline int fnic_queue_dr_io_req(struct fnic *fnic, struct scsi_lun fc_lun; int ret = 0; unsigned long intr_flags; + unsigned int tag = scsi_cmd_to_rq(sc)->tag; + + if (tag == SCSI_NO_TAG) + tag = io_req->tag; spin_lock_irqsave(host->host_lock, intr_flags); if (unlikely(fnic_chk_state_flags_locked(fnic, @@ -1964,7 +1984,8 @@ static inline int fnic_queue_dr_io_req(struct fnic *fnic, /* fill in the lun info */ int_to_scsilun(sc->device->lun, &fc_lun); - fnic_queue_wq_copy_desc_itmf(wq, scsi_cmd_to_rq(sc)->tag | FNIC_TAG_DEV_RST, + tag |= FNIC_TAG_DEV_RST; + fnic_queue_wq_copy_desc_itmf(wq, tag, 0, FCPIO_ITMF_LUN_RESET, SCSI_NO_TAG, fc_lun.scsi_lun, io_req->port_id, fnic->config.ra_tov, fnic->config.ed_tov); @@ -2146,8 +2167,7 @@ static int fnic_clean_pending_aborts(struct fnic *fnic, .ret = SUCCESS, }; - if (new_sc) - iter_data.lr_sc = lr_sc; + iter_data.lr_sc = lr_sc; scsi_host_busy_iter(fnic->lport->host, fnic_pending_aborts_iter, &iter_data); @@ -2230,8 +2250,14 @@ int fnic_device_reset(struct scsi_cmnd *sc) mutex_lock(&fnic->sgreset_mutex); tag = fnic->fnic_max_tag_id; new_sc = 1; - } - io_lock = fnic_io_lock_hash(fnic, sc); + fnic->sgreset_sc = sc; + io_lock = &fnic->sgreset_lock; + FNIC_SCSI_DBG(KERN_INFO, fnic->lport->host, + "fcid: 0x%x lun: 0x%llx flags: 0x%x tag: 0x%x Issuing sgreset\n", + rport->port_id, sc->device->lun, fnic_priv(sc)->flags, tag); + } else + io_lock = fnic_io_lock_hash(fnic, sc); + spin_lock_irqsave(io_lock, flags); io_req = fnic_priv(sc)->io_req; @@ -2247,6 +2273,8 @@ int fnic_device_reset(struct scsi_cmnd *sc) } memset(io_req, 0, sizeof(*io_req)); io_req->port_id = rport->port_id; + io_req->tag = tag; + io_req->sc = sc; fnic_priv(sc)->io_req = io_req; } io_req->dr_done = &tm_done; @@ -2400,8 +2428,10 @@ fnic_device_reset_end: (u64)sc->cmnd[4] << 8 | sc->cmnd[5]), fnic_flags_and_state(sc)); - if (new_sc) + if (new_sc) { + fnic->sgreset_sc = NULL; mutex_unlock(&fnic->sgreset_mutex); + } FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host, "Returning from device reset %s\n", diff --git a/drivers/scsi/iscsi_tcp.c b/drivers/scsi/iscsi_tcp.c index 9ab8555180a3..8e14cea15f98 100644 --- a/drivers/scsi/iscsi_tcp.c +++ b/drivers/scsi/iscsi_tcp.c @@ -724,6 +724,10 @@ iscsi_sw_tcp_conn_bind(struct iscsi_cls_session *cls_session, return -EEXIST; } + err = -EINVAL; + if (!sk_is_tcp(sock->sk)) + goto free_socket; + err = iscsi_conn_bind(cls_session, cls_conn, is_leading); if (err) goto free_socket; diff --git a/drivers/scsi/lpfc/lpfc_debugfs.c b/drivers/scsi/lpfc/lpfc_debugfs.c index 7f9b221e7c34..ea9b42225e62 100644 --- a/drivers/scsi/lpfc/lpfc_debugfs.c +++ b/drivers/scsi/lpfc/lpfc_debugfs.c @@ -6073,7 +6073,7 @@ lpfc_debugfs_initialize(struct lpfc_vport *vport) phba->hba_debugfs_root, phba, &lpfc_debugfs_op_multixripools); - if (!phba->debug_multixri_pools) { + if (IS_ERR(phba->debug_multixri_pools)) { lpfc_printf_vlog(vport, KERN_ERR, LOG_INIT, "0527 Cannot create debugfs multixripools\n"); goto debug_failed; @@ -6085,7 +6085,7 @@ lpfc_debugfs_initialize(struct lpfc_vport *vport) debugfs_create_file(name, S_IFREG | 0644, phba->hba_debugfs_root, phba, &lpfc_cgn_buffer_op); - if (!phba->debug_cgn_buffer) { + if (IS_ERR(phba->debug_cgn_buffer)) { lpfc_printf_vlog(vport, KERN_ERR, LOG_INIT, "6527 Cannot create debugfs " "cgn_buffer\n"); @@ -6098,7 +6098,7 @@ lpfc_debugfs_initialize(struct lpfc_vport *vport) debugfs_create_file(name, S_IFREG | 0644, phba->hba_debugfs_root, phba, &lpfc_rx_monitor_op); - if (!phba->debug_rx_monitor) { + if (IS_ERR(phba->debug_rx_monitor)) { lpfc_printf_vlog(vport, KERN_ERR, LOG_INIT, "6528 Cannot create debugfs " "rx_monitor\n"); @@ -6111,7 +6111,7 @@ lpfc_debugfs_initialize(struct lpfc_vport *vport) debugfs_create_file(name, 0644, phba->hba_debugfs_root, phba, &lpfc_debugfs_ras_log); - if (!phba->debug_ras_log) { + if (IS_ERR(phba->debug_ras_log)) { lpfc_printf_vlog(vport, KERN_ERR, LOG_INIT, "6148 Cannot create debugfs" " ras_log\n"); @@ -6132,7 +6132,7 @@ lpfc_debugfs_initialize(struct lpfc_vport *vport) debugfs_create_file(name, S_IFREG | 0644, phba->hba_debugfs_root, phba, &lpfc_debugfs_op_lockstat); - if (!phba->debug_lockstat) { + if (IS_ERR(phba->debug_lockstat)) { lpfc_printf_vlog(vport, KERN_ERR, LOG_INIT, "4610 Can't create debugfs lockstat\n"); goto debug_failed; @@ -6358,7 +6358,7 @@ nvmeio_off: debugfs_create_file(name, 0644, vport->vport_debugfs_root, vport, &lpfc_debugfs_op_scsistat); - if (!vport->debug_scsistat) { + if (IS_ERR(vport->debug_scsistat)) { lpfc_printf_vlog(vport, KERN_ERR, LOG_INIT, "4611 Cannot create debugfs scsistat\n"); goto debug_failed; @@ -6369,7 +6369,7 @@ nvmeio_off: debugfs_create_file(name, 0644, vport->vport_debugfs_root, vport, &lpfc_debugfs_op_ioktime); - if (!vport->debug_ioktime) { + if (IS_ERR(vport->debug_ioktime)) { lpfc_printf_vlog(vport, KERN_ERR, LOG_INIT, "0815 Cannot create debugfs ioktime\n"); goto debug_failed; diff --git a/drivers/scsi/lpfc/lpfc_hbadisc.c b/drivers/scsi/lpfc/lpfc_hbadisc.c index 51afb60859eb..5154eeaee0ec 100644 --- a/drivers/scsi/lpfc/lpfc_hbadisc.c +++ b/drivers/scsi/lpfc/lpfc_hbadisc.c @@ -199,11 +199,12 @@ lpfc_dev_loss_tmo_callbk(struct fc_rport *rport) /* Only 1 thread can drop the initial node reference. If * another thread has set NLP_DROPPED, this thread is done. */ - if (!(ndlp->nlp_flag & NLP_DROPPED)) { + if (!(ndlp->fc4_xpt_flags & NVME_XPT_REGD) && + !(ndlp->nlp_flag & NLP_DROPPED)) { ndlp->nlp_flag |= NLP_DROPPED; spin_unlock_irqrestore(&ndlp->lock, iflags); lpfc_nlp_put(ndlp); - spin_lock_irqsave(&ndlp->lock, iflags); + return; } spin_unlock_irqrestore(&ndlp->lock, iflags); diff --git a/drivers/scsi/lpfc/lpfc_nvme.c b/drivers/scsi/lpfc/lpfc_nvme.c index 39acbcb7ec66..96e11a26c297 100644 --- a/drivers/scsi/lpfc/lpfc_nvme.c +++ b/drivers/scsi/lpfc/lpfc_nvme.c @@ -228,8 +228,7 @@ lpfc_nvme_remoteport_delete(struct nvme_fc_remote_port *remoteport) spin_unlock_irq(&ndlp->lock); /* On a devloss timeout event, one more put is executed provided the - * NVME and SCSI rport unregister requests are complete. If the vport - * is unloading, this extra put is executed by lpfc_drop_node. + * NVME and SCSI rport unregister requests are complete. */ if (!(ndlp->fc4_xpt_flags & fc4_xpt_flags)) lpfc_disc_state_machine(vport, ndlp, NULL, NLP_EVT_DEVICE_RM); @@ -2567,11 +2566,7 @@ lpfc_nvme_rescan_port(struct lpfc_vport *vport, struct lpfc_nodelist *ndlp) * nvme_transport perspective. Loss of an rport just means IO cannot * be sent and recovery is completely up to the initator. * For now, the driver just unbinds the DID and port_role so that - * no further IO can be issued. Changes are planned for later. - * - * Notes - the ndlp reference count is not decremented here since - * since there is no nvme_transport api for devloss. Node ref count - * is only adjusted in driver unload. + * no further IO can be issued. */ void lpfc_nvme_unregister_port(struct lpfc_vport *vport, struct lpfc_nodelist *ndlp) @@ -2646,6 +2641,21 @@ lpfc_nvme_unregister_port(struct lpfc_vport *vport, struct lpfc_nodelist *ndlp) "6167 NVME unregister failed %d " "port_state x%x\n", ret, remoteport->port_state); + + if (vport->load_flag & FC_UNLOADING) { + /* Only 1 thread can drop the initial node + * reference. Check if another thread has set + * NLP_DROPPED. + */ + spin_lock_irq(&ndlp->lock); + if (!(ndlp->nlp_flag & NLP_DROPPED)) { + ndlp->nlp_flag |= NLP_DROPPED; + spin_unlock_irq(&ndlp->lock); + lpfc_nlp_put(ndlp); + return; + } + spin_unlock_irq(&ndlp->lock); + } } } return; diff --git a/drivers/scsi/megaraid/megaraid_sas.h b/drivers/scsi/megaraid/megaraid_sas.h index 3554f6b07727..94abba57582d 100644 --- a/drivers/scsi/megaraid/megaraid_sas.h +++ b/drivers/scsi/megaraid/megaraid_sas.h @@ -2332,7 +2332,7 @@ struct megasas_instance { u32 support_morethan256jbod; /* FW support for more than 256 PD/JBOD */ bool use_seqnum_jbod_fp; /* Added for PD sequence */ bool smp_affinity_enable; - spinlock_t crashdump_lock; + struct mutex crashdump_lock; struct megasas_register_set __iomem *reg_set; u32 __iomem *reply_post_host_index_addr[MR_MAX_MSIX_REG_ARRAY]; diff --git a/drivers/scsi/megaraid/megaraid_sas_base.c b/drivers/scsi/megaraid/megaraid_sas_base.c index b9d46dcb5210..e1aa667dae66 100644 --- a/drivers/scsi/megaraid/megaraid_sas_base.c +++ b/drivers/scsi/megaraid/megaraid_sas_base.c @@ -3271,14 +3271,13 @@ fw_crash_buffer_store(struct device *cdev, struct megasas_instance *instance = (struct megasas_instance *) shost->hostdata; int val = 0; - unsigned long flags; if (kstrtoint(buf, 0, &val) != 0) return -EINVAL; - spin_lock_irqsave(&instance->crashdump_lock, flags); + mutex_lock(&instance->crashdump_lock); instance->fw_crash_buffer_offset = val; - spin_unlock_irqrestore(&instance->crashdump_lock, flags); + mutex_unlock(&instance->crashdump_lock); return strlen(buf); } @@ -3293,24 +3292,23 @@ fw_crash_buffer_show(struct device *cdev, unsigned long dmachunk = CRASH_DMA_BUF_SIZE; unsigned long chunk_left_bytes; unsigned long src_addr; - unsigned long flags; u32 buff_offset; - spin_lock_irqsave(&instance->crashdump_lock, flags); + mutex_lock(&instance->crashdump_lock); buff_offset = instance->fw_crash_buffer_offset; if (!instance->crash_dump_buf || !((instance->fw_crash_state == AVAILABLE) || (instance->fw_crash_state == COPYING))) { dev_err(&instance->pdev->dev, "Firmware crash dump is not available\n"); - spin_unlock_irqrestore(&instance->crashdump_lock, flags); + mutex_unlock(&instance->crashdump_lock); return -EINVAL; } if (buff_offset > (instance->fw_crash_buffer_size * dmachunk)) { dev_err(&instance->pdev->dev, "Firmware crash dump offset is out of range\n"); - spin_unlock_irqrestore(&instance->crashdump_lock, flags); + mutex_unlock(&instance->crashdump_lock); return 0; } @@ -3322,7 +3320,7 @@ fw_crash_buffer_show(struct device *cdev, src_addr = (unsigned long)instance->crash_buf[buff_offset / dmachunk] + (buff_offset % dmachunk); memcpy(buf, (void *)src_addr, size); - spin_unlock_irqrestore(&instance->crashdump_lock, flags); + mutex_unlock(&instance->crashdump_lock); return size; } @@ -3347,7 +3345,6 @@ fw_crash_state_store(struct device *cdev, struct megasas_instance *instance = (struct megasas_instance *) shost->hostdata; int val = 0; - unsigned long flags; if (kstrtoint(buf, 0, &val) != 0) return -EINVAL; @@ -3361,9 +3358,9 @@ fw_crash_state_store(struct device *cdev, instance->fw_crash_state = val; if ((val == COPIED) || (val == COPY_ERROR)) { - spin_lock_irqsave(&instance->crashdump_lock, flags); + mutex_lock(&instance->crashdump_lock); megasas_free_host_crash_buffer(instance); - spin_unlock_irqrestore(&instance->crashdump_lock, flags); + mutex_unlock(&instance->crashdump_lock); if (val == COPY_ERROR) dev_info(&instance->pdev->dev, "application failed to " "copy Firmware crash dump\n"); @@ -7422,7 +7419,7 @@ static inline void megasas_init_ctrl_params(struct megasas_instance *instance) init_waitqueue_head(&instance->int_cmd_wait_q); init_waitqueue_head(&instance->abort_cmd_wait_q); - spin_lock_init(&instance->crashdump_lock); + mutex_init(&instance->crashdump_lock); spin_lock_init(&instance->mfi_pool_lock); spin_lock_init(&instance->hba_lock); spin_lock_init(&instance->stream_lock); diff --git a/drivers/scsi/mpt3sas/mpt3sas_scsih.c b/drivers/scsi/mpt3sas/mpt3sas_scsih.c index c3c1f466fe01..605013d3ee83 100644 --- a/drivers/scsi/mpt3sas/mpt3sas_scsih.c +++ b/drivers/scsi/mpt3sas/mpt3sas_scsih.c @@ -12913,8 +12913,10 @@ _mpt3sas_init(void) mpt3sas_ctl_init(hbas_to_enumerate); error = pci_register_driver(&mpt3sas_driver); - if (error) + if (error) { + mpt3sas_ctl_exit(hbas_to_enumerate); scsih_exit(); + } return error; } diff --git a/drivers/scsi/pm8001/pm8001_hwi.c b/drivers/scsi/pm8001/pm8001_hwi.c index 33053db5a713..90069c7b1642 100644 --- a/drivers/scsi/pm8001/pm8001_hwi.c +++ b/drivers/scsi/pm8001/pm8001_hwi.c @@ -4180,7 +4180,7 @@ pm8001_chip_phy_start_req(struct pm8001_hba_info *pm8001_ha, u8 phy_id) payload.sas_identify.dev_type = SAS_END_DEVICE; payload.sas_identify.initiator_bits = SAS_PROTOCOL_ALL; memcpy(payload.sas_identify.sas_addr, - pm8001_ha->sas_addr, SAS_ADDR_SIZE); + &pm8001_ha->phy[phy_id].dev_sas_addr, SAS_ADDR_SIZE); payload.sas_identify.phy_id = phy_id; return pm8001_mpi_build_cmd(pm8001_ha, 0, opcode, &payload, diff --git a/drivers/scsi/pm8001/pm8001_init.c b/drivers/scsi/pm8001/pm8001_init.c index 5e5ce1e74c3b..443a3176c6c0 100644 --- a/drivers/scsi/pm8001/pm8001_init.c +++ b/drivers/scsi/pm8001/pm8001_init.c @@ -273,7 +273,6 @@ static irqreturn_t pm8001_interrupt_handler_intx(int irq, void *dev_id) return ret; } -static u32 pm8001_setup_irq(struct pm8001_hba_info *pm8001_ha); static u32 pm8001_request_irq(struct pm8001_hba_info *pm8001_ha); /** @@ -294,13 +293,6 @@ static int pm8001_alloc(struct pm8001_hba_info *pm8001_ha, pm8001_dbg(pm8001_ha, INIT, "pm8001_alloc: PHY:%x\n", pm8001_ha->chip->n_phy); - /* Setup Interrupt */ - rc = pm8001_setup_irq(pm8001_ha); - if (rc) { - pm8001_dbg(pm8001_ha, FAIL, - "pm8001_setup_irq failed [ret: %d]\n", rc); - goto err_out; - } /* Request Interrupt */ rc = pm8001_request_irq(pm8001_ha); if (rc) @@ -1031,47 +1023,38 @@ static u32 pm8001_request_msix(struct pm8001_hba_info *pm8001_ha) } #endif -static u32 pm8001_setup_irq(struct pm8001_hba_info *pm8001_ha) -{ - struct pci_dev *pdev; - - pdev = pm8001_ha->pdev; - -#ifdef PM8001_USE_MSIX - if (pci_find_capability(pdev, PCI_CAP_ID_MSIX)) - return pm8001_setup_msix(pm8001_ha); - pm8001_dbg(pm8001_ha, INIT, "MSIX not supported!!!\n"); -#endif - return 0; -} - /** * pm8001_request_irq - register interrupt * @pm8001_ha: our ha struct. */ static u32 pm8001_request_irq(struct pm8001_hba_info *pm8001_ha) { - struct pci_dev *pdev; + struct pci_dev *pdev = pm8001_ha->pdev; +#ifdef PM8001_USE_MSIX int rc; - pdev = pm8001_ha->pdev; + if (pci_find_capability(pdev, PCI_CAP_ID_MSIX)) { + rc = pm8001_setup_msix(pm8001_ha); + if (rc) { + pm8001_dbg(pm8001_ha, FAIL, + "pm8001_setup_irq failed [ret: %d]\n", rc); + return rc; + } -#ifdef PM8001_USE_MSIX - if (pdev->msix_cap && pci_msi_enabled()) - return pm8001_request_msix(pm8001_ha); - else { - pm8001_dbg(pm8001_ha, INIT, "MSIX not supported!!!\n"); - goto intx; + if (pdev->msix_cap && pci_msi_enabled()) + return pm8001_request_msix(pm8001_ha); } + + pm8001_dbg(pm8001_ha, INIT, "MSIX not supported!!!\n"); #endif -intx: /* initialize the INT-X interrupt */ pm8001_ha->irq_vector[0].irq_id = 0; pm8001_ha->irq_vector[0].drv_inst = pm8001_ha; - rc = request_irq(pdev->irq, pm8001_interrupt_handler_intx, IRQF_SHARED, - pm8001_ha->name, SHOST_TO_SAS_HA(pm8001_ha->shost)); - return rc; + + return request_irq(pdev->irq, pm8001_interrupt_handler_intx, + IRQF_SHARED, pm8001_ha->name, + SHOST_TO_SAS_HA(pm8001_ha->shost)); } /** diff --git a/drivers/scsi/pm8001/pm80xx_hwi.c b/drivers/scsi/pm8001/pm80xx_hwi.c index f6857632dc7c..3afd9443c425 100644 --- a/drivers/scsi/pm8001/pm80xx_hwi.c +++ b/drivers/scsi/pm8001/pm80xx_hwi.c @@ -3671,10 +3671,12 @@ static int mpi_set_controller_config_resp(struct pm8001_hba_info *pm8001_ha, (struct set_ctrl_cfg_resp *)(piomb + 4); u32 status = le32_to_cpu(pPayload->status); u32 err_qlfr_pgcd = le32_to_cpu(pPayload->err_qlfr_pgcd); + u32 tag = le32_to_cpu(pPayload->tag); pm8001_dbg(pm8001_ha, MSG, "SET CONTROLLER RESP: status 0x%x qlfr_pgcd 0x%x\n", status, err_qlfr_pgcd); + pm8001_tag_free(pm8001_ha, tag); return 0; } @@ -4671,7 +4673,7 @@ pm80xx_chip_phy_start_req(struct pm8001_hba_info *pm8001_ha, u8 phy_id) payload.sas_identify.dev_type = SAS_END_DEVICE; payload.sas_identify.initiator_bits = SAS_PROTOCOL_ALL; memcpy(payload.sas_identify.sas_addr, - &pm8001_ha->sas_addr, SAS_ADDR_SIZE); + &pm8001_ha->phy[phy_id].dev_sas_addr, SAS_ADDR_SIZE); payload.sas_identify.phy_id = phy_id; return pm8001_mpi_build_cmd(pm8001_ha, 0, opcode, &payload, diff --git a/drivers/scsi/ppa.c b/drivers/scsi/ppa.c index 19f0b93fa3d8..d592ee9170c1 100644 --- a/drivers/scsi/ppa.c +++ b/drivers/scsi/ppa.c @@ -307,9 +307,9 @@ static int ppa_out(ppa_struct *dev, char *buffer, int len) case PPA_EPP_8: epp_reset(ppb); w_ctr(ppb, 0x4); - if (dev->mode == PPA_EPP_32 && !(((long) buffer | len) & 0x01)) + if (dev->mode == PPA_EPP_32 && !(((long) buffer | len) & 0x03)) outsl(ppb + 4, buffer, len >> 2); - else if (dev->mode == PPA_EPP_16 && !(((long) buffer | len) & 0x03)) + else if (dev->mode == PPA_EPP_16 && !(((long) buffer | len) & 0x01)) outsw(ppb + 4, buffer, len >> 1); else outsb(ppb + 4, buffer, len); diff --git a/drivers/scsi/qedf/qedf_io.c b/drivers/scsi/qedf/qedf_io.c index 4750ec5789a8..10fe3383855c 100644 --- a/drivers/scsi/qedf/qedf_io.c +++ b/drivers/scsi/qedf/qedf_io.c @@ -1904,6 +1904,7 @@ int qedf_initiate_abts(struct qedf_ioreq *io_req, bool return_scsi_cmd_on_abts) goto drop_rdata_kref; } + spin_lock_irqsave(&fcport->rport_lock, flags); if (!test_bit(QEDF_CMD_OUTSTANDING, &io_req->flags) || test_bit(QEDF_CMD_IN_CLEANUP, &io_req->flags) || test_bit(QEDF_CMD_IN_ABORT, &io_req->flags)) { @@ -1911,17 +1912,20 @@ int qedf_initiate_abts(struct qedf_ioreq *io_req, bool return_scsi_cmd_on_abts) "io_req xid=0x%x sc_cmd=%p already in cleanup or abort processing or already completed.\n", io_req->xid, io_req->sc_cmd); rc = 1; + spin_unlock_irqrestore(&fcport->rport_lock, flags); goto drop_rdata_kref; } + /* Set the command type to abort */ + io_req->cmd_type = QEDF_ABTS; + spin_unlock_irqrestore(&fcport->rport_lock, flags); + kref_get(&io_req->refcount); xid = io_req->xid; qedf->control_requests++; qedf->packet_aborts++; - /* Set the command type to abort */ - io_req->cmd_type = QEDF_ABTS; io_req->return_scsi_cmd_on_abts = return_scsi_cmd_on_abts; set_bit(QEDF_CMD_IN_ABORT, &io_req->flags); @@ -2210,7 +2214,9 @@ process_els: refcount, fcport, fcport->rdata->ids.port_id); /* Cleanup cmds re-use the same TID as the original I/O */ + spin_lock_irqsave(&fcport->rport_lock, flags); io_req->cmd_type = QEDF_CLEANUP; + spin_unlock_irqrestore(&fcport->rport_lock, flags); io_req->return_scsi_cmd_on_abts = return_scsi_cmd_on_abts; init_completion(&io_req->cleanup_done); diff --git a/drivers/scsi/qedf/qedf_main.c b/drivers/scsi/qedf/qedf_main.c index 7825765c936c..91f3f1d7098e 100644 --- a/drivers/scsi/qedf/qedf_main.c +++ b/drivers/scsi/qedf/qedf_main.c @@ -2805,6 +2805,8 @@ void qedf_process_cqe(struct qedf_ctx *qedf, struct fcoe_cqe *cqe) struct qedf_ioreq *io_req; struct qedf_rport *fcport; u32 comp_type; + u8 io_comp_type; + unsigned long flags; comp_type = (cqe->cqe_data >> FCOE_CQE_CQE_TYPE_SHIFT) & FCOE_CQE_CQE_TYPE_MASK; @@ -2838,11 +2840,14 @@ void qedf_process_cqe(struct qedf_ctx *qedf, struct fcoe_cqe *cqe) return; } + spin_lock_irqsave(&fcport->rport_lock, flags); + io_comp_type = io_req->cmd_type; + spin_unlock_irqrestore(&fcport->rport_lock, flags); switch (comp_type) { case FCOE_GOOD_COMPLETION_CQE_TYPE: atomic_inc(&fcport->free_sqes); - switch (io_req->cmd_type) { + switch (io_comp_type) { case QEDF_SCSI_CMD: qedf_scsi_completion(qedf, cqe, io_req); break; diff --git a/drivers/scsi/qla2xxx/qla_dfs.c b/drivers/scsi/qla2xxx/qla_dfs.c index f060e593685d..a7a364760b80 100644 --- a/drivers/scsi/qla2xxx/qla_dfs.c +++ b/drivers/scsi/qla2xxx/qla_dfs.c @@ -116,7 +116,7 @@ qla2x00_dfs_create_rport(scsi_qla_host_t *vha, struct fc_port *fp) sprintf(wwn, "pn-%016llx", wwn_to_u64(fp->port_name)); fp->dfs_rport_dir = debugfs_create_dir(wwn, vha->dfs_rport_root); - if (!fp->dfs_rport_dir) + if (IS_ERR(fp->dfs_rport_dir)) return; if (NVME_TARGET(vha->hw, fp)) debugfs_create_file("dev_loss_tmo", 0600, fp->dfs_rport_dir, @@ -708,14 +708,14 @@ create_nodes: if (IS_QLA27XX(ha) || IS_QLA83XX(ha) || IS_QLA28XX(ha)) { ha->tgt.dfs_naqp = debugfs_create_file("naqp", 0400, ha->dfs_dir, vha, &dfs_naqp_ops); - if (!ha->tgt.dfs_naqp) { + if (IS_ERR(ha->tgt.dfs_naqp)) { ql_log(ql_log_warn, vha, 0xd011, "Unable to create debugFS naqp node.\n"); goto out; } } vha->dfs_rport_root = debugfs_create_dir("rports", ha->dfs_dir); - if (!vha->dfs_rport_root) { + if (IS_ERR(vha->dfs_rport_root)) { ql_log(ql_log_warn, vha, 0xd012, "Unable to create debugFS rports node.\n"); goto out; diff --git a/drivers/scsi/qla2xxx/qla_inline.h b/drivers/scsi/qla2xxx/qla_inline.h index 0556969f6dc1..a4a56ab0ba74 100644 --- a/drivers/scsi/qla2xxx/qla_inline.h +++ b/drivers/scsi/qla2xxx/qla_inline.h @@ -577,7 +577,7 @@ fcport_is_bigger(fc_port_t *fcport) static inline struct qla_qpair * qla_mapq_nvme_select_qpair(struct qla_hw_data *ha, struct qla_qpair *qpair) { - int cpuid = smp_processor_id(); + int cpuid = raw_smp_processor_id(); if (qpair->cpuid != cpuid && ha->qp_cpu_map[cpuid]) { diff --git a/drivers/scsi/qla2xxx/qla_isr.c b/drivers/scsi/qla2xxx/qla_isr.c index e98788191897..d48007e18288 100644 --- a/drivers/scsi/qla2xxx/qla_isr.c +++ b/drivers/scsi/qla2xxx/qla_isr.c @@ -3965,7 +3965,7 @@ void qla24xx_process_response_queue(struct scsi_qla_host *vha, if (!ha->flags.fw_started) return; - if (rsp->qpair->cpuid != smp_processor_id() || !rsp->qpair->rcv_intr) { + if (rsp->qpair->cpuid != raw_smp_processor_id() || !rsp->qpair->rcv_intr) { rsp->qpair->rcv_intr = 1; if (!rsp->qpair->cpu_mapped) @@ -4468,7 +4468,7 @@ qla2xxx_msix_rsp_q(int irq, void *dev_id) } ha = qpair->hw; - queue_work_on(smp_processor_id(), ha->wq, &qpair->q_work); + queue_work(ha->wq, &qpair->q_work); return IRQ_HANDLED; } @@ -4494,7 +4494,7 @@ qla2xxx_msix_rsp_q_hs(int irq, void *dev_id) wrt_reg_dword(®->hccr, HCCRX_CLR_RISC_INT); spin_unlock_irqrestore(&ha->hardware_lock, flags); - queue_work_on(smp_processor_id(), ha->wq, &qpair->q_work); + queue_work(ha->wq, &qpair->q_work); return IRQ_HANDLED; } diff --git a/drivers/scsi/qla2xxx/qla_nvme.c b/drivers/scsi/qla2xxx/qla_nvme.c index db753d712991..a8ddf356e662 100644 --- a/drivers/scsi/qla2xxx/qla_nvme.c +++ b/drivers/scsi/qla2xxx/qla_nvme.c @@ -399,14 +399,14 @@ static int qla_nvme_xmt_ls_rsp(struct nvme_fc_local_port *lport, nvme->u.nvme.dl = 0; nvme->u.nvme.timeout_sec = 0; nvme->u.nvme.cmd_dma = fd_resp->rspdma; - nvme->u.nvme.cmd_len = fd_resp->rsplen; + nvme->u.nvme.cmd_len = cpu_to_le32(fd_resp->rsplen); nvme->u.nvme.rsp_len = 0; nvme->u.nvme.rsp_dma = 0; nvme->u.nvme.exchange_address = uctx->exchange_address; nvme->u.nvme.nport_handle = uctx->nport_handle; nvme->u.nvme.ox_id = uctx->ox_id; dma_sync_single_for_device(&ha->pdev->dev, nvme->u.nvme.cmd_dma, - le32_to_cpu(fd_resp->rsplen), DMA_TO_DEVICE); + fd_resp->rsplen, DMA_TO_DEVICE); ql_dbg(ql_dbg_unsol, vha, 0x2122, "Unsol lsreq portid=%06x %8phC exchange_address 0x%x ox_id 0x%x hdl 0x%x\n", @@ -504,13 +504,13 @@ static int qla_nvme_ls_req(struct nvme_fc_local_port *lport, nvme->u.nvme.desc = fd; nvme->u.nvme.dir = 0; nvme->u.nvme.dl = 0; - nvme->u.nvme.cmd_len = fd->rqstlen; - nvme->u.nvme.rsp_len = fd->rsplen; + nvme->u.nvme.cmd_len = cpu_to_le32(fd->rqstlen); + nvme->u.nvme.rsp_len = cpu_to_le32(fd->rsplen); nvme->u.nvme.rsp_dma = fd->rspdma; nvme->u.nvme.timeout_sec = fd->timeout; nvme->u.nvme.cmd_dma = fd->rqstdma; dma_sync_single_for_device(&ha->pdev->dev, nvme->u.nvme.cmd_dma, - le32_to_cpu(fd->rqstlen), DMA_TO_DEVICE); + fd->rqstlen, DMA_TO_DEVICE); rval = qla2x00_start_sp(sp); if (rval != QLA_SUCCESS) { diff --git a/drivers/scsi/qla2xxx/qla_os.c b/drivers/scsi/qla2xxx/qla_os.c index 50db08265c51..dcae09a37d49 100644 --- a/drivers/scsi/qla2xxx/qla_os.c +++ b/drivers/scsi/qla2xxx/qla_os.c @@ -4953,7 +4953,7 @@ qla2x00_mem_free(struct qla_hw_data *ha) ha->gid_list = NULL; ha->gid_list_dma = 0; - if (!list_empty(&ha->base_qpair->dsd_list)) { + if (ha->base_qpair && !list_empty(&ha->base_qpair->dsd_list)) { struct dsd_dma *dsd_ptr, *tdsd_ptr; /* clean up allocated prev pool */ diff --git a/drivers/scsi/qla2xxx/qla_target.c b/drivers/scsi/qla2xxx/qla_target.c index 2b815a9928ea..2ef2dbac0db2 100644 --- a/drivers/scsi/qla2xxx/qla_target.c +++ b/drivers/scsi/qla2xxx/qla_target.c @@ -4425,8 +4425,7 @@ static int qlt_handle_cmd_for_atio(struct scsi_qla_host *vha, queue_work_on(cmd->se_cmd.cpuid, qla_tgt_wq, &cmd->work); } else if (ha->msix_count) { if (cmd->atio.u.isp24.fcp_cmnd.rddata) - queue_work_on(smp_processor_id(), qla_tgt_wq, - &cmd->work); + queue_work(qla_tgt_wq, &cmd->work); else queue_work_on(cmd->se_cmd.cpuid, qla_tgt_wq, &cmd->work); diff --git a/drivers/scsi/qla2xxx/tcm_qla2xxx.c b/drivers/scsi/qla2xxx/tcm_qla2xxx.c index 3b5ba4b47b3b..68a0e6a2fb6e 100644 --- a/drivers/scsi/qla2xxx/tcm_qla2xxx.c +++ b/drivers/scsi/qla2xxx/tcm_qla2xxx.c @@ -310,7 +310,7 @@ static void tcm_qla2xxx_free_cmd(struct qla_tgt_cmd *cmd) cmd->trc_flags |= TRC_CMD_DONE; INIT_WORK(&cmd->work, tcm_qla2xxx_complete_free); - queue_work_on(smp_processor_id(), tcm_qla2xxx_free_wq, &cmd->work); + queue_work(tcm_qla2xxx_free_wq, &cmd->work); } /* @@ -547,7 +547,7 @@ static void tcm_qla2xxx_handle_data(struct qla_tgt_cmd *cmd) cmd->trc_flags |= TRC_DATA_IN; cmd->cmd_in_wq = 1; INIT_WORK(&cmd->work, tcm_qla2xxx_handle_data_work); - queue_work_on(smp_processor_id(), tcm_qla2xxx_free_wq, &cmd->work); + queue_work(tcm_qla2xxx_free_wq, &cmd->work); } static int tcm_qla2xxx_chk_dif_tags(uint32_t tag) diff --git a/drivers/scsi/scsi.c b/drivers/scsi/scsi.c index d0911bc28663..89367c4bf0ef 100644 --- a/drivers/scsi/scsi.c +++ b/drivers/scsi/scsi.c @@ -613,6 +613,17 @@ void scsi_cdl_check(struct scsi_device *sdev) bool cdl_supported; unsigned char *buf; + /* + * Support for CDL was defined in SPC-5. Ignore devices reporting an + * lower SPC version. This also avoids problems with old drives choking + * on MAINTENANCE_IN / MI_REPORT_SUPPORTED_OPERATION_CODES with a + * service action specified, as done in scsi_cdl_check_cmd(). + */ + if (sdev->scsi_level < SCSI_SPC_5) { + sdev->cdl_supported = 0; + return; + } + buf = kmalloc(SCSI_CDL_CHECK_BUF_LEN, GFP_KERNEL); if (!buf) { sdev->cdl_supported = 0; diff --git a/drivers/scsi/scsi_scan.c b/drivers/scsi/scsi_scan.c index 52014b2d39e1..44680f65ea14 100644 --- a/drivers/scsi/scsi_scan.c +++ b/drivers/scsi/scsi_scan.c @@ -822,7 +822,7 @@ static int scsi_probe_lun(struct scsi_device *sdev, unsigned char *inq_result, * device is attached at LUN 0 (SCSI_SCAN_TARGET_PRESENT) so * non-zero LUNs can be scanned. */ - sdev->scsi_level = inq_result[2] & 0x07; + sdev->scsi_level = inq_result[2] & 0x0f; if (sdev->scsi_level >= 2 || (sdev->scsi_level == 1 && (inq_result[3] & 0x0f) == 1)) sdev->scsi_level++; @@ -1619,12 +1619,25 @@ int scsi_add_device(struct Scsi_Host *host, uint channel, } EXPORT_SYMBOL(scsi_add_device); -void scsi_rescan_device(struct scsi_device *sdev) +int scsi_rescan_device(struct scsi_device *sdev) { struct device *dev = &sdev->sdev_gendev; + int ret = 0; device_lock(dev); + /* + * Bail out if the device or its queue are not running. Otherwise, + * the rescan may block waiting for commands to be executed, with us + * holding the device lock. This can result in a potential deadlock + * in the power management core code when system resume is on-going. + */ + if (sdev->sdev_state != SDEV_RUNNING || + blk_queue_pm_only(sdev->request_queue)) { + ret = -EWOULDBLOCK; + goto unlock; + } + scsi_attach_vpd(sdev); scsi_cdl_check(sdev); @@ -1638,7 +1651,11 @@ void scsi_rescan_device(struct scsi_device *sdev) drv->rescan(dev); module_put(dev->driver->owner); } + +unlock: device_unlock(dev); + + return ret; } EXPORT_SYMBOL(scsi_rescan_device); diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c index c92a317ba547..83b6a3f3863b 100644 --- a/drivers/scsi/sd.c +++ b/drivers/scsi/sd.c @@ -201,18 +201,32 @@ cache_type_store(struct device *dev, struct device_attribute *attr, } static ssize_t -manage_start_stop_show(struct device *dev, struct device_attribute *attr, - char *buf) +manage_start_stop_show(struct device *dev, + struct device_attribute *attr, char *buf) { struct scsi_disk *sdkp = to_scsi_disk(dev); struct scsi_device *sdp = sdkp->device; - return sprintf(buf, "%u\n", sdp->manage_start_stop); + return sysfs_emit(buf, "%u\n", + sdp->manage_system_start_stop && + sdp->manage_runtime_start_stop); } +static DEVICE_ATTR_RO(manage_start_stop); static ssize_t -manage_start_stop_store(struct device *dev, struct device_attribute *attr, - const char *buf, size_t count) +manage_system_start_stop_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + struct scsi_disk *sdkp = to_scsi_disk(dev); + struct scsi_device *sdp = sdkp->device; + + return sysfs_emit(buf, "%u\n", sdp->manage_system_start_stop); +} + +static ssize_t +manage_system_start_stop_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t count) { struct scsi_disk *sdkp = to_scsi_disk(dev); struct scsi_device *sdp = sdkp->device; @@ -224,11 +238,42 @@ manage_start_stop_store(struct device *dev, struct device_attribute *attr, if (kstrtobool(buf, &v)) return -EINVAL; - sdp->manage_start_stop = v; + sdp->manage_system_start_stop = v; return count; } -static DEVICE_ATTR_RW(manage_start_stop); +static DEVICE_ATTR_RW(manage_system_start_stop); + +static ssize_t +manage_runtime_start_stop_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + struct scsi_disk *sdkp = to_scsi_disk(dev); + struct scsi_device *sdp = sdkp->device; + + return sysfs_emit(buf, "%u\n", sdp->manage_runtime_start_stop); +} + +static ssize_t +manage_runtime_start_stop_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t count) +{ + struct scsi_disk *sdkp = to_scsi_disk(dev); + struct scsi_device *sdp = sdkp->device; + bool v; + + if (!capable(CAP_SYS_ADMIN)) + return -EACCES; + + if (kstrtobool(buf, &v)) + return -EINVAL; + + sdp->manage_runtime_start_stop = v; + + return count; +} +static DEVICE_ATTR_RW(manage_runtime_start_stop); static ssize_t allow_restart_show(struct device *dev, struct device_attribute *attr, char *buf) @@ -560,6 +605,8 @@ static struct attribute *sd_disk_attrs[] = { &dev_attr_FUA.attr, &dev_attr_allow_restart.attr, &dev_attr_manage_start_stop.attr, + &dev_attr_manage_system_start_stop.attr, + &dev_attr_manage_runtime_start_stop.attr, &dev_attr_protection_type.attr, &dev_attr_protection_mode.attr, &dev_attr_app_tag_own.attr, @@ -3694,7 +3741,8 @@ static int sd_remove(struct device *dev) device_del(&sdkp->disk_dev); del_gendisk(sdkp->disk); - sd_shutdown(dev); + if (!sdkp->suspended) + sd_shutdown(dev); put_disk(sdkp->disk); return 0; @@ -3771,13 +3819,20 @@ static void sd_shutdown(struct device *dev) sd_sync_cache(sdkp, NULL); } - if (system_state != SYSTEM_RESTART && sdkp->device->manage_start_stop) { + if (system_state != SYSTEM_RESTART && + sdkp->device->manage_system_start_stop) { sd_printk(KERN_NOTICE, sdkp, "Stopping disk\n"); sd_start_stop_device(sdkp, 0); } } -static int sd_suspend_common(struct device *dev, bool ignore_stop_errors) +static inline bool sd_do_start_stop(struct scsi_device *sdev, bool runtime) +{ + return (sdev->manage_system_start_stop && !runtime) || + (sdev->manage_runtime_start_stop && runtime); +} + +static int sd_suspend_common(struct device *dev, bool runtime) { struct scsi_disk *sdkp = dev_get_drvdata(dev); struct scsi_sense_hdr sshdr; @@ -3809,15 +3864,18 @@ static int sd_suspend_common(struct device *dev, bool ignore_stop_errors) } } - if (sdkp->device->manage_start_stop) { + if (sd_do_start_stop(sdkp->device, runtime)) { if (!sdkp->device->silence_suspend) sd_printk(KERN_NOTICE, sdkp, "Stopping disk\n"); /* an error is not worth aborting a system sleep */ ret = sd_start_stop_device(sdkp, 0); - if (ignore_stop_errors) + if (!runtime) ret = 0; } + if (!ret) + sdkp->suspended = true; + return ret; } @@ -3826,15 +3884,15 @@ static int sd_suspend_system(struct device *dev) if (pm_runtime_suspended(dev)) return 0; - return sd_suspend_common(dev, true); + return sd_suspend_common(dev, false); } static int sd_suspend_runtime(struct device *dev) { - return sd_suspend_common(dev, false); + return sd_suspend_common(dev, true); } -static int sd_resume(struct device *dev) +static int sd_resume(struct device *dev, bool runtime) { struct scsi_disk *sdkp = dev_get_drvdata(dev); int ret = 0; @@ -3842,16 +3900,21 @@ static int sd_resume(struct device *dev) if (!sdkp) /* E.g.: runtime resume at the start of sd_probe() */ return 0; - if (!sdkp->device->manage_start_stop) + if (!sd_do_start_stop(sdkp->device, runtime)) { + sdkp->suspended = false; return 0; + } if (!sdkp->device->no_start_on_resume) { sd_printk(KERN_NOTICE, sdkp, "Starting disk\n"); ret = sd_start_stop_device(sdkp, 1); } - if (!ret) + if (!ret) { opal_unlock_from_suspend(sdkp->opal_dev); + sdkp->suspended = false; + } + return ret; } @@ -3860,7 +3923,7 @@ static int sd_resume_system(struct device *dev) if (pm_runtime_suspended(dev)) return 0; - return sd_resume(dev); + return sd_resume(dev, false); } static int sd_resume_runtime(struct device *dev) @@ -3887,7 +3950,7 @@ static int sd_resume_runtime(struct device *dev) "Failed to clear sense data\n"); } - return sd_resume(dev); + return sd_resume(dev, true); } static const struct dev_pm_ops sd_pm_ops = { diff --git a/drivers/scsi/sd.h b/drivers/scsi/sd.h index 5eea762f84d1..409dda5350d1 100644 --- a/drivers/scsi/sd.h +++ b/drivers/scsi/sd.h @@ -131,6 +131,7 @@ struct scsi_disk { u8 provisioning_mode; u8 zeroing_mode; u8 nr_actuators; /* Number of actuators */ + bool suspended; /* Disk is suspended (stopped) */ unsigned ATO : 1; /* state of disk ATO bit */ unsigned cache_override : 1; /* temp override of WCE,RCD */ unsigned WCE : 1; /* state of disk WCE bit */ diff --git a/drivers/soc/imx/soc-imx8m.c b/drivers/soc/imx/soc-imx8m.c index 1dcd243df567..ec87d9d878f3 100644 --- a/drivers/soc/imx/soc-imx8m.c +++ b/drivers/soc/imx/soc-imx8m.c @@ -100,6 +100,7 @@ static void __init imx8mm_soc_uid(void) { void __iomem *ocotp_base; struct device_node *np; + struct clk *clk; u32 offset = of_machine_is_compatible("fsl,imx8mp") ? IMX8MP_OCOTP_UID_OFFSET : 0; @@ -109,11 +110,20 @@ static void __init imx8mm_soc_uid(void) ocotp_base = of_iomap(np, 0); WARN_ON(!ocotp_base); + clk = of_clk_get_by_name(np, NULL); + if (IS_ERR(clk)) { + WARN_ON(IS_ERR(clk)); + return; + } + + clk_prepare_enable(clk); soc_uid = readl_relaxed(ocotp_base + OCOTP_UID_HIGH + offset); soc_uid <<= 32; soc_uid |= readl_relaxed(ocotp_base + OCOTP_UID_LOW + offset); + clk_disable_unprepare(clk); + clk_put(clk); iounmap(ocotp_base); of_node_put(np); } diff --git a/drivers/soc/loongson/Kconfig b/drivers/soc/loongson/Kconfig index 314e13bb3e01..368344943a93 100644 --- a/drivers/soc/loongson/Kconfig +++ b/drivers/soc/loongson/Kconfig @@ -20,6 +20,7 @@ config LOONGSON2_GUTS config LOONGSON2_PM bool "Loongson-2 SoC Power Management Controller Driver" depends on LOONGARCH && OF + depends on INPUT=y help The Loongson-2's power management controller was ACPI, supports ACPI S2Idle (Suspend To Idle), ACPI S3 (Suspend To RAM), ACPI S4 (Suspend To diff --git a/drivers/soc/loongson/loongson2_guts.c b/drivers/soc/loongson/loongson2_guts.c index bace4bc8e03b..9a469779eea7 100644 --- a/drivers/soc/loongson/loongson2_guts.c +++ b/drivers/soc/loongson/loongson2_guts.c @@ -70,7 +70,7 @@ static const struct loongson2_soc_die_attr *loongson2_soc_die_match( if (matches->svr == (svr & matches->mask)) return matches; matches++; - }; + } return NULL; } @@ -94,7 +94,6 @@ static int loongson2_guts_probe(struct platform_device *pdev) { struct device_node *root, *np = pdev->dev.of_node; struct device *dev = &pdev->dev; - struct resource *res; const struct loongson2_soc_die_attr *soc_die; const char *machine; u32 svr; @@ -106,8 +105,7 @@ static int loongson2_guts_probe(struct platform_device *pdev) guts->little_endian = of_property_read_bool(np, "little-endian"); - res = platform_get_resource(pdev, IORESOURCE_MEM, 0); - guts->regs = ioremap(res->start, res->end - res->start + 1); + guts->regs = devm_platform_ioremap_resource(pdev, 0); if (IS_ERR(guts->regs)) return PTR_ERR(guts->regs); diff --git a/drivers/soc/loongson/loongson2_pm.c b/drivers/soc/loongson/loongson2_pm.c index 796add6e8b63..b8e5e1e3528a 100644 --- a/drivers/soc/loongson/loongson2_pm.c +++ b/drivers/soc/loongson/loongson2_pm.c @@ -11,6 +11,7 @@ #include #include #include +#include #include #include #include @@ -192,12 +193,16 @@ static int loongson2_pm_probe(struct platform_device *pdev) if (loongson_sysconf.suspend_addr) suspend_set_ops(&loongson2_suspend_ops); + /* Populate children */ + retval = devm_of_platform_populate(dev); + if (retval) + dev_err(dev, "Error populating children, reboot and poweroff might not work properly\n"); + return 0; } static const struct of_device_id loongson2_pm_match[] = { { .compatible = "loongson,ls2k0500-pmc", }, - { .compatible = "loongson,ls2k1000-pmc", }, {}, }; diff --git a/drivers/soc/renesas/Kconfig b/drivers/soc/renesas/Kconfig index 5a75ab64d1ed..12040ce116a5 100644 --- a/drivers/soc/renesas/Kconfig +++ b/drivers/soc/renesas/Kconfig @@ -333,6 +333,7 @@ if RISCV config ARCH_R9A07G043 bool "RISC-V Platform support for RZ/Five" + depends on NONPORTABLE select ARCH_RZG2L select AX45MP_L2_CACHE if RISCV_DMA_NONCOHERENT select DMA_GLOBAL_POOL diff --git a/drivers/soundwire/Makefile b/drivers/soundwire/Makefile index c3d3ab3262d3..657f5888a77b 100644 --- a/drivers/soundwire/Makefile +++ b/drivers/soundwire/Makefile @@ -15,6 +15,10 @@ ifdef CONFIG_DEBUG_FS soundwire-bus-y += debugfs.o endif +ifdef CONFIG_IRQ_DOMAIN +soundwire-bus-y += irq.o +endif + #AMD driver soundwire-amd-y := amd_manager.o obj-$(CONFIG_SOUNDWIRE_AMD) += soundwire-amd.o diff --git a/drivers/soundwire/bus.c b/drivers/soundwire/bus.c index 1720031f35a3..0e7bc3c40f9d 100644 --- a/drivers/soundwire/bus.c +++ b/drivers/soundwire/bus.c @@ -3,13 +3,13 @@ #include #include -#include #include #include #include #include #include #include "bus.h" +#include "irq.h" #include "sysfs_local.h" static DEFINE_IDA(sdw_bus_ida); @@ -25,23 +25,6 @@ static int sdw_get_id(struct sdw_bus *bus) return 0; } -static int sdw_irq_map(struct irq_domain *h, unsigned int virq, - irq_hw_number_t hw) -{ - struct sdw_bus *bus = h->host_data; - - irq_set_chip_data(virq, bus); - irq_set_chip(virq, &bus->irq_chip); - irq_set_nested_thread(virq, 1); - irq_set_noprobe(virq); - - return 0; -} - -static const struct irq_domain_ops sdw_domain_ops = { - .map = sdw_irq_map, -}; - /** * sdw_bus_master_add() - add a bus Master instance * @bus: bus instance @@ -168,13 +151,9 @@ int sdw_bus_master_add(struct sdw_bus *bus, struct device *parent, bus->params.curr_bank = SDW_BANK0; bus->params.next_bank = SDW_BANK1; - bus->irq_chip.name = dev_name(bus->dev); - bus->domain = irq_domain_create_linear(fwnode, SDW_MAX_DEVICES, - &sdw_domain_ops, bus); - if (!bus->domain) { - dev_err(bus->dev, "Failed to add IRQ domain\n"); - return -EINVAL; - } + ret = sdw_irq_create(bus, fwnode); + if (ret) + return ret; return 0; } @@ -213,7 +192,7 @@ void sdw_bus_master_delete(struct sdw_bus *bus) { device_for_each_child(bus->dev, NULL, sdw_delete_slave); - irq_domain_remove(bus->domain); + sdw_irq_delete(bus); sdw_master_device_del(bus); diff --git a/drivers/soundwire/bus_type.c b/drivers/soundwire/bus_type.c index fafbc284e82d..9fa93bb923d7 100644 --- a/drivers/soundwire/bus_type.c +++ b/drivers/soundwire/bus_type.c @@ -7,6 +7,7 @@ #include #include #include "bus.h" +#include "irq.h" #include "sysfs_local.h" /** @@ -122,11 +123,8 @@ static int sdw_drv_probe(struct device *dev) if (drv->ops && drv->ops->read_prop) drv->ops->read_prop(slave); - if (slave->prop.use_domain_irq) { - slave->irq = irq_create_mapping(slave->bus->domain, slave->dev_num); - if (!slave->irq) - dev_warn(dev, "Failed to map IRQ\n"); - } + if (slave->prop.use_domain_irq) + sdw_irq_create_mapping(slave); /* init the sysfs as we have properties now */ ret = sdw_slave_sysfs_init(slave); @@ -176,8 +174,7 @@ static int sdw_drv_remove(struct device *dev) slave->probed = false; if (slave->prop.use_domain_irq) - irq_dispose_mapping(irq_find_mapping(slave->bus->domain, - slave->dev_num)); + sdw_irq_dispose_mapping(slave); mutex_unlock(&slave->sdw_dev_lock); diff --git a/drivers/soundwire/irq.c b/drivers/soundwire/irq.c new file mode 100644 index 000000000000..0c08cebb1235 --- /dev/null +++ b/drivers/soundwire/irq.c @@ -0,0 +1,59 @@ +// SPDX-License-Identifier: GPL-2.0 +// Copyright (C) 2023 Cirrus Logic, Inc. and +// Cirrus Logic International Semiconductor Ltd. + +#include +#include +#include +#include +#include +#include "irq.h" + +static int sdw_irq_map(struct irq_domain *h, unsigned int virq, + irq_hw_number_t hw) +{ + struct sdw_bus *bus = h->host_data; + + irq_set_chip_data(virq, bus); + irq_set_chip(virq, &bus->irq_chip); + irq_set_nested_thread(virq, 1); + irq_set_noprobe(virq); + + return 0; +} + +static const struct irq_domain_ops sdw_domain_ops = { + .map = sdw_irq_map, +}; + +int sdw_irq_create(struct sdw_bus *bus, + struct fwnode_handle *fwnode) +{ + bus->irq_chip.name = dev_name(bus->dev); + + bus->domain = irq_domain_create_linear(fwnode, SDW_MAX_DEVICES, + &sdw_domain_ops, bus); + if (!bus->domain) { + dev_err(bus->dev, "Failed to add IRQ domain\n"); + return -EINVAL; + } + + return 0; +} + +void sdw_irq_delete(struct sdw_bus *bus) +{ + irq_domain_remove(bus->domain); +} + +void sdw_irq_create_mapping(struct sdw_slave *slave) +{ + slave->irq = irq_create_mapping(slave->bus->domain, slave->dev_num); + if (!slave->irq) + dev_warn(&slave->dev, "Failed to map IRQ\n"); +} + +void sdw_irq_dispose_mapping(struct sdw_slave *slave) +{ + irq_dispose_mapping(irq_find_mapping(slave->bus->domain, slave->dev_num)); +} diff --git a/drivers/soundwire/irq.h b/drivers/soundwire/irq.h new file mode 100644 index 000000000000..58a58046d92b --- /dev/null +++ b/drivers/soundwire/irq.h @@ -0,0 +1,43 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (C) 2023 Cirrus Logic, Inc. and + * Cirrus Logic International Semiconductor Ltd. + */ + +#ifndef __SDW_IRQ_H +#define __SDW_IRQ_H + +#include +#include + +#if IS_ENABLED(CONFIG_IRQ_DOMAIN) + +int sdw_irq_create(struct sdw_bus *bus, + struct fwnode_handle *fwnode); +void sdw_irq_delete(struct sdw_bus *bus); +void sdw_irq_create_mapping(struct sdw_slave *slave); +void sdw_irq_dispose_mapping(struct sdw_slave *slave); + +#else /* CONFIG_IRQ_DOMAIN */ + +static inline int sdw_irq_create(struct sdw_bus *bus, + struct fwnode_handle *fwnode) +{ + return 0; +} + +static inline void sdw_irq_delete(struct sdw_bus *bus) +{ +} + +static inline void sdw_irq_create_mapping(struct sdw_slave *slave) +{ +} + +static inline void sdw_irq_dispose_mapping(struct sdw_slave *slave) +{ +} + +#endif /* CONFIG_IRQ_DOMAIN */ + +#endif /* __SDW_IRQ_H */ diff --git a/drivers/spi/spi-cs42l43.c b/drivers/spi/spi-cs42l43.c index 453a9b37ce78..d239fc5a49cc 100644 --- a/drivers/spi/spi-cs42l43.c +++ b/drivers/spi/spi-cs42l43.c @@ -256,7 +256,6 @@ static int cs42l43_spi_probe(struct platform_device *pdev) ret = devm_spi_register_controller(priv->dev, priv->ctlr); if (ret) { - pm_runtime_disable(priv->dev); dev_err(priv->dev, "Failed to register SPI controller: %d\n", ret); } diff --git a/drivers/spi/spi-gxp.c b/drivers/spi/spi-gxp.c index fd2fac236bbd..3aff5a166c94 100644 --- a/drivers/spi/spi-gxp.c +++ b/drivers/spi/spi-gxp.c @@ -194,7 +194,7 @@ static ssize_t gxp_spi_write(struct gxp_spi_chip *chip, const struct spi_mem_op return ret; } - return write_len; + return 0; } static int do_gxp_exec_mem_op(struct spi_mem *mem, const struct spi_mem_op *op) diff --git a/drivers/spi/spi-imx.c b/drivers/spi/spi-imx.c index a8a74c7cb79f..498e35c8db2c 100644 --- a/drivers/spi/spi-imx.c +++ b/drivers/spi/spi-imx.c @@ -662,7 +662,7 @@ static int mx51_ecspi_prepare_transfer(struct spi_imx_data *spi_imx, if (spi_imx->count >= 512) ctrl |= 0xFFF << MX51_ECSPI_CTRL_BL_OFFSET; else - ctrl |= (spi_imx->count*8 - 1) + ctrl |= (spi_imx->count * spi_imx->bits_per_word - 1) << MX51_ECSPI_CTRL_BL_OFFSET; } diff --git a/drivers/spi/spi-intel-pci.c b/drivers/spi/spi-intel-pci.c index a7381e774b95..57d767a68e7b 100644 --- a/drivers/spi/spi-intel-pci.c +++ b/drivers/spi/spi-intel-pci.c @@ -72,6 +72,7 @@ static const struct pci_device_id intel_spi_pci_ids[] = { { PCI_VDEVICE(INTEL, 0x4da4), (unsigned long)&bxt_info }, { PCI_VDEVICE(INTEL, 0x51a4), (unsigned long)&cnl_info }, { PCI_VDEVICE(INTEL, 0x54a4), (unsigned long)&cnl_info }, + { PCI_VDEVICE(INTEL, 0x5794), (unsigned long)&cnl_info }, { PCI_VDEVICE(INTEL, 0x7a24), (unsigned long)&cnl_info }, { PCI_VDEVICE(INTEL, 0x7aa4), (unsigned long)&cnl_info }, { PCI_VDEVICE(INTEL, 0x7e23), (unsigned long)&cnl_info }, diff --git a/drivers/spi/spi-npcm-fiu.c b/drivers/spi/spi-npcm-fiu.c index 0ca21ff0e9cc..e42248519688 100644 --- a/drivers/spi/spi-npcm-fiu.c +++ b/drivers/spi/spi-npcm-fiu.c @@ -353,8 +353,9 @@ static int npcm_fiu_uma_read(struct spi_mem *mem, uma_cfg |= ilog2(op->cmd.buswidth); uma_cfg |= ilog2(op->addr.buswidth) << NPCM_FIU_UMA_CFG_ADBPCK_SHIFT; - uma_cfg |= ilog2(op->dummy.buswidth) - << NPCM_FIU_UMA_CFG_DBPCK_SHIFT; + if (op->dummy.nbytes) + uma_cfg |= ilog2(op->dummy.buswidth) + << NPCM_FIU_UMA_CFG_DBPCK_SHIFT; uma_cfg |= ilog2(op->data.buswidth) << NPCM_FIU_UMA_CFG_RDBPCK_SHIFT; uma_cfg |= op->dummy.nbytes << NPCM_FIU_UMA_CFG_DBSIZ_SHIFT; diff --git a/drivers/spi/spi-nxp-fspi.c b/drivers/spi/spi-nxp-fspi.c index 45a4acc95661..c964f41dcc42 100644 --- a/drivers/spi/spi-nxp-fspi.c +++ b/drivers/spi/spi-nxp-fspi.c @@ -1084,6 +1084,13 @@ static int nxp_fspi_default_setup(struct nxp_fspi *f) fspi_writel(f, FSPI_AHBCR_PREF_EN | FSPI_AHBCR_RDADDROPT, base + FSPI_AHBCR); + /* Reset the FLSHxCR1 registers. */ + reg = FSPI_FLSHXCR1_TCSH(0x3) | FSPI_FLSHXCR1_TCSS(0x3); + fspi_writel(f, reg, base + FSPI_FLSHA1CR1); + fspi_writel(f, reg, base + FSPI_FLSHA2CR1); + fspi_writel(f, reg, base + FSPI_FLSHB1CR1); + fspi_writel(f, reg, base + FSPI_FLSHB2CR1); + /* AHB Read - Set lut sequence ID for all CS. */ fspi_writel(f, SEQID_LUT, base + FSPI_FLSHA1CR2); fspi_writel(f, SEQID_LUT, base + FSPI_FLSHA2CR2); diff --git a/drivers/spi/spi-stm32.c b/drivers/spi/spi-stm32.c index b6d66caba4c0..ef665f470c5b 100644 --- a/drivers/spi/spi-stm32.c +++ b/drivers/spi/spi-stm32.c @@ -277,6 +277,7 @@ struct stm32_spi_cfg { * @fifo_size: size of the embedded fifo in bytes * @cur_midi: master inter-data idleness in ns * @cur_speed: speed configured in Hz + * @cur_half_period: time of a half bit in us * @cur_bpw: number of bits in a single SPI data frame * @cur_fthlv: fifo threshold level (data frames in a single data packet) * @cur_comm: SPI communication mode @@ -304,6 +305,7 @@ struct stm32_spi { unsigned int cur_midi; unsigned int cur_speed; + unsigned int cur_half_period; unsigned int cur_bpw; unsigned int cur_fthlv; unsigned int cur_comm; @@ -468,6 +470,8 @@ static int stm32_spi_prepare_mbr(struct stm32_spi *spi, u32 speed_hz, spi->cur_speed = spi->clk_rate / (1 << mbrdiv); + spi->cur_half_period = DIV_ROUND_CLOSEST(USEC_PER_SEC, 2 * spi->cur_speed); + return mbrdiv - 1; } @@ -709,6 +713,10 @@ static void stm32h7_spi_disable(struct stm32_spi *spi) return; } + /* Add a delay to make sure that transmission is ended. */ + if (spi->cur_half_period) + udelay(spi->cur_half_period); + if (spi->cur_usedma && spi->dma_tx) dmaengine_terminate_async(spi->dma_tx); if (spi->cur_usedma && spi->dma_rx) diff --git a/drivers/spi/spi-zynqmp-gqspi.c b/drivers/spi/spi-zynqmp-gqspi.c index 94d9a33d9af5..9a46b2478f4e 100644 --- a/drivers/spi/spi-zynqmp-gqspi.c +++ b/drivers/spi/spi-zynqmp-gqspi.c @@ -1340,9 +1340,9 @@ static int zynqmp_qspi_probe(struct platform_device *pdev) return 0; clk_dis_all: - pm_runtime_put_sync(&pdev->dev); - pm_runtime_set_suspended(&pdev->dev); pm_runtime_disable(&pdev->dev); + pm_runtime_put_noidle(&pdev->dev); + pm_runtime_set_suspended(&pdev->dev); clk_disable_unprepare(xqspi->refclk); clk_dis_pclk: clk_disable_unprepare(xqspi->pclk); @@ -1366,11 +1366,15 @@ static void zynqmp_qspi_remove(struct platform_device *pdev) { struct zynqmp_qspi *xqspi = platform_get_drvdata(pdev); + pm_runtime_get_sync(&pdev->dev); + zynqmp_gqspi_write(xqspi, GQSPI_EN_OFST, 0x0); + + pm_runtime_disable(&pdev->dev); + pm_runtime_put_noidle(&pdev->dev); + pm_runtime_set_suspended(&pdev->dev); clk_disable_unprepare(xqspi->refclk); clk_disable_unprepare(xqspi->pclk); - pm_runtime_set_suspended(&pdev->dev); - pm_runtime_disable(&pdev->dev); } MODULE_DEVICE_TABLE(of, zynqmp_qspi_of_match); diff --git a/drivers/staging/media/atomisp/Kconfig b/drivers/staging/media/atomisp/Kconfig index 5d8917160d41..75c985da75b5 100644 --- a/drivers/staging/media/atomisp/Kconfig +++ b/drivers/staging/media/atomisp/Kconfig @@ -12,12 +12,12 @@ menuconfig INTEL_ATOMISP config VIDEO_ATOMISP tristate "Intel Atom Image Signal Processor Driver" depends on VIDEO_DEV && INTEL_ATOMISP + depends on IPU_BRIDGE depends on MEDIA_PCI_SUPPORT depends on PMIC_OPREGION depends on I2C select V4L2_FWNODE select IOSF_MBI - select IPU_BRIDGE select VIDEOBUF2_VMALLOC select VIDEO_V4L2_SUBDEV_API help diff --git a/drivers/staging/media/tegra-video/vi.c b/drivers/staging/media/tegra-video/vi.c index e98b3010520e..94171e62dee9 100644 --- a/drivers/staging/media/tegra-video/vi.c +++ b/drivers/staging/media/tegra-video/vi.c @@ -1455,17 +1455,18 @@ static int __maybe_unused vi_runtime_suspend(struct device *dev) } /* - * Graph Management + * Find the entity matching a given fwnode in an v4l2_async_notifier list */ static struct tegra_vi_graph_entity * -tegra_vi_graph_find_entity(struct tegra_vi_channel *chan, +tegra_vi_graph_find_entity(struct list_head *list, const struct fwnode_handle *fwnode) { struct tegra_vi_graph_entity *entity; struct v4l2_async_connection *asd; - list_for_each_entry(asd, &chan->notifier.done_list, asc_entry) { + list_for_each_entry(asd, list, asc_entry) { entity = to_tegra_vi_graph_entity(asd); + if (entity->asd.match.fwnode == fwnode) return entity; } @@ -1532,7 +1533,8 @@ static int tegra_vi_graph_build(struct tegra_vi_channel *chan, } /* find the remote entity from notifier list */ - ent = tegra_vi_graph_find_entity(chan, link.remote_node); + ent = tegra_vi_graph_find_entity(&chan->notifier.done_list, + link.remote_node); if (!ent) { dev_err(vi->dev, "no entity found for %pOF\n", to_of_node(link.remote_node)); @@ -1664,7 +1666,8 @@ static int tegra_vi_graph_notify_bound(struct v4l2_async_notifier *notifier, * Locate the entity corresponding to the bound subdev and store the * subdev pointer. */ - entity = tegra_vi_graph_find_entity(chan, subdev->fwnode); + entity = tegra_vi_graph_find_entity(&chan->notifier.waiting_list, + subdev->fwnode); if (!entity) { dev_err(vi->dev, "no entity for subdev %s\n", subdev->name); return -EINVAL; @@ -1713,7 +1716,8 @@ static int tegra_vi_graph_parse_one(struct tegra_vi_channel *chan, /* skip entities that are already processed */ if (device_match_fwnode(vi->dev, remote) || - tegra_vi_graph_find_entity(chan, remote)) { + tegra_vi_graph_find_entity(&chan->notifier.waiting_list, + remote)) { fwnode_handle_put(remote); continue; } diff --git a/drivers/target/target_core_configfs.c b/drivers/target/target_core_configfs.c index 936e5ff1b209..d5860c1c1f46 100644 --- a/drivers/target/target_core_configfs.c +++ b/drivers/target/target_core_configfs.c @@ -1392,16 +1392,16 @@ static ssize_t target_wwn_vendor_id_store(struct config_item *item, /* +2 to allow for a trailing (stripped) '\n' and null-terminator */ unsigned char buf[INQUIRY_VENDOR_LEN + 2]; char *stripped = NULL; - size_t len; + ssize_t len; ssize_t ret; - len = strlcpy(buf, page, sizeof(buf)); - if (len < sizeof(buf)) { + len = strscpy(buf, page, sizeof(buf)); + if (len > 0) { /* Strip any newline added from userspace. */ stripped = strstrip(buf); len = strlen(stripped); } - if (len > INQUIRY_VENDOR_LEN) { + if (len < 0 || len > INQUIRY_VENDOR_LEN) { pr_err("Emulated T10 Vendor Identification exceeds" " INQUIRY_VENDOR_LEN: " __stringify(INQUIRY_VENDOR_LEN) "\n"); @@ -1448,16 +1448,16 @@ static ssize_t target_wwn_product_id_store(struct config_item *item, /* +2 to allow for a trailing (stripped) '\n' and null-terminator */ unsigned char buf[INQUIRY_MODEL_LEN + 2]; char *stripped = NULL; - size_t len; + ssize_t len; ssize_t ret; - len = strlcpy(buf, page, sizeof(buf)); - if (len < sizeof(buf)) { + len = strscpy(buf, page, sizeof(buf)); + if (len > 0) { /* Strip any newline added from userspace. */ stripped = strstrip(buf); len = strlen(stripped); } - if (len > INQUIRY_MODEL_LEN) { + if (len < 0 || len > INQUIRY_MODEL_LEN) { pr_err("Emulated T10 Vendor exceeds INQUIRY_MODEL_LEN: " __stringify(INQUIRY_MODEL_LEN) "\n"); @@ -1504,16 +1504,16 @@ static ssize_t target_wwn_revision_store(struct config_item *item, /* +2 to allow for a trailing (stripped) '\n' and null-terminator */ unsigned char buf[INQUIRY_REVISION_LEN + 2]; char *stripped = NULL; - size_t len; + ssize_t len; ssize_t ret; - len = strlcpy(buf, page, sizeof(buf)); - if (len < sizeof(buf)) { + len = strscpy(buf, page, sizeof(buf)); + if (len > 0) { /* Strip any newline added from userspace. */ stripped = strstrip(buf); len = strlen(stripped); } - if (len > INQUIRY_REVISION_LEN) { + if (len < 0 || len > INQUIRY_REVISION_LEN) { pr_err("Emulated T10 Revision exceeds INQUIRY_REVISION_LEN: " __stringify(INQUIRY_REVISION_LEN) "\n"); diff --git a/drivers/target/target_core_device.c b/drivers/target/target_core_device.c index b7ac60f4a219..b6523d4b9259 100644 --- a/drivers/target/target_core_device.c +++ b/drivers/target/target_core_device.c @@ -843,7 +843,6 @@ sector_t target_to_linux_sector(struct se_device *dev, sector_t lb) EXPORT_SYMBOL(target_to_linux_sector); struct devices_idr_iter { - struct config_item *prev_item; int (*fn)(struct se_device *dev, void *data); void *data; }; @@ -853,11 +852,9 @@ static int target_devices_idr_iter(int id, void *p, void *data) { struct devices_idr_iter *iter = data; struct se_device *dev = p; + struct config_item *item; int ret; - config_item_put(iter->prev_item); - iter->prev_item = NULL; - /* * We add the device early to the idr, so it can be used * by backend modules during configuration. We do not want @@ -867,12 +864,13 @@ static int target_devices_idr_iter(int id, void *p, void *data) if (!target_dev_configured(dev)) return 0; - iter->prev_item = config_item_get_unless_zero(&dev->dev_group.cg_item); - if (!iter->prev_item) + item = config_item_get_unless_zero(&dev->dev_group.cg_item); + if (!item) return 0; mutex_unlock(&device_mutex); ret = iter->fn(dev, iter->data); + config_item_put(item); mutex_lock(&device_mutex); return ret; @@ -895,7 +893,6 @@ int target_for_each_device(int (*fn)(struct se_device *dev, void *data), mutex_lock(&device_mutex); ret = idr_for_each(&devices_idr, target_devices_idr_iter, &iter); mutex_unlock(&device_mutex); - config_item_put(iter.prev_item); return ret; } diff --git a/drivers/target/target_core_transport.c b/drivers/target/target_core_transport.c index 687adc9e086c..0686882bcbda 100644 --- a/drivers/target/target_core_transport.c +++ b/drivers/target/target_core_transport.c @@ -264,6 +264,7 @@ void target_free_cmd_counter(struct target_cmd_counter *cmd_cnt) percpu_ref_put(&cmd_cnt->refcnt); percpu_ref_exit(&cmd_cnt->refcnt); + kfree(cmd_cnt); } EXPORT_SYMBOL_GPL(target_free_cmd_counter); diff --git a/drivers/tee/amdtee/core.c b/drivers/tee/amdtee/core.c index 372d64756ed6..3c15f6a9e91c 100644 --- a/drivers/tee/amdtee/core.c +++ b/drivers/tee/amdtee/core.c @@ -217,12 +217,12 @@ unlock: return rc; } +/* mutex must be held by caller */ static void destroy_session(struct kref *ref) { struct amdtee_session *sess = container_of(ref, struct amdtee_session, refcount); - mutex_lock(&session_list_mutex); list_del(&sess->list_node); mutex_unlock(&session_list_mutex); kfree(sess); @@ -272,7 +272,8 @@ int amdtee_open_session(struct tee_context *ctx, if (arg->ret != TEEC_SUCCESS) { pr_err("open_session failed %d\n", arg->ret); handle_unload_ta(ta_handle); - kref_put(&sess->refcount, destroy_session); + kref_put_mutex(&sess->refcount, destroy_session, + &session_list_mutex); goto out; } @@ -290,7 +291,8 @@ int amdtee_open_session(struct tee_context *ctx, pr_err("reached maximum session count %d\n", TEE_NUM_SESSIONS); handle_close_session(ta_handle, session_info); handle_unload_ta(ta_handle); - kref_put(&sess->refcount, destroy_session); + kref_put_mutex(&sess->refcount, destroy_session, + &session_list_mutex); rc = -ENOMEM; goto out; } @@ -331,7 +333,7 @@ int amdtee_close_session(struct tee_context *ctx, u32 session) handle_close_session(ta_handle, session_info); handle_unload_ta(ta_handle); - kref_put(&sess->refcount, destroy_session); + kref_put_mutex(&sess->refcount, destroy_session, &session_list_mutex); return 0; } diff --git a/drivers/tee/optee/optee_private.h b/drivers/tee/optee/optee_private.h index 72685ee0d53f..6bb5cae09688 100644 --- a/drivers/tee/optee/optee_private.h +++ b/drivers/tee/optee/optee_private.h @@ -238,8 +238,6 @@ int optee_notif_send(struct optee *optee, u_int key); u32 optee_supp_thrd_req(struct tee_context *ctx, u32 func, size_t num_params, struct tee_param *param); -int optee_supp_read(struct tee_context *ctx, void __user *buf, size_t len); -int optee_supp_write(struct tee_context *ctx, void __user *buf, size_t len); void optee_supp_init(struct optee_supp *supp); void optee_supp_uninit(struct optee_supp *supp); void optee_supp_release(struct optee_supp *supp); diff --git a/drivers/tee/tee_private.h b/drivers/tee/tee_private.h index 409cadcc1cff..754e11dcb240 100644 --- a/drivers/tee/tee_private.h +++ b/drivers/tee/tee_private.h @@ -47,8 +47,6 @@ struct tee_device { struct tee_shm_pool *pool; }; -int tee_shm_init(void); - int tee_shm_get_fd(struct tee_shm *shm); bool tee_device_get(struct tee_device *teedev); diff --git a/drivers/thermal/thermal_core.c b/drivers/thermal/thermal_core.c index 8717a3343512..58533ea75cd9 100644 --- a/drivers/thermal/thermal_core.c +++ b/drivers/thermal/thermal_core.c @@ -348,12 +348,14 @@ static void handle_thermal_trip(struct thermal_zone_device *tz, int trip_id) struct thermal_trip trip; /* Ignore disabled trip points */ - if (test_bit(trip_id, &tz->trips_disabled) || - trip.temperature == THERMAL_TEMP_INVALID) + if (test_bit(trip_id, &tz->trips_disabled)) return; __thermal_zone_get_trip(tz, trip_id, &trip); + if (trip.temperature == THERMAL_TEMP_INVALID) + return; + if (tz->last_temperature != THERMAL_TEMP_INVALID) { if (tz->last_temperature < trip.temperature && tz->temperature >= trip.temperature) diff --git a/drivers/thermal/thermal_of.c b/drivers/thermal/thermal_of.c index 4ca905723429..1e0655b63259 100644 --- a/drivers/thermal/thermal_of.c +++ b/drivers/thermal/thermal_of.c @@ -37,8 +37,10 @@ static int of_find_trip_id(struct device_node *np, struct device_node *trip) */ for_each_child_of_node(trips, t) { - if (t == trip) + if (t == trip) { + of_node_put(t); goto out; + } i++; } @@ -401,8 +403,10 @@ static int thermal_of_for_each_cooling_maps(struct thermal_zone_device *tz, for_each_child_of_node(cm_np, child) { ret = thermal_of_for_each_cooling_device(tz_np, child, tz, cdev, action); - if (ret) + if (ret) { + of_node_put(child); break; + } } of_node_put(cm_np); diff --git a/drivers/thermal/thermal_sysfs.c b/drivers/thermal/thermal_sysfs.c index 6c20c9f90a05..4e6a97db894e 100644 --- a/drivers/thermal/thermal_sysfs.c +++ b/drivers/thermal/thermal_sysfs.c @@ -185,9 +185,6 @@ trip_point_hyst_store(struct device *dev, struct device_attribute *attr, if (sscanf(attr->attr.name, "trip_point_%d_hyst", &trip_id) != 1) return -EINVAL; - if (kstrtoint(buf, 10, &trip.hysteresis)) - return -EINVAL; - mutex_lock(&tz->lock); if (!device_is_registered(dev)) { @@ -198,7 +195,11 @@ trip_point_hyst_store(struct device *dev, struct device_attribute *attr, ret = __thermal_zone_get_trip(tz, trip_id, &trip); if (ret) goto unlock; - + + ret = kstrtoint(buf, 10, &trip.hysteresis); + if (ret) + goto unlock; + ret = thermal_zone_set_trip(tz, trip_id, &trip); unlock: mutex_unlock(&tz->lock); diff --git a/drivers/thermal/ti-soc-thermal/ti-thermal-common.c b/drivers/thermal/ti-soc-thermal/ti-thermal-common.c index 6ba2613627e1..0cf0826b805a 100644 --- a/drivers/thermal/ti-soc-thermal/ti-thermal-common.c +++ b/drivers/thermal/ti-soc-thermal/ti-thermal-common.c @@ -110,7 +110,8 @@ static inline int __ti_thermal_get_temp(struct thermal_zone_device *tz, int *tem } static int __ti_thermal_get_trend(struct thermal_zone_device *tz, - struct thermal_trip *trip, enum thermal_trend *trend) + const struct thermal_trip *trip, + enum thermal_trend *trend) { struct ti_thermal_data *data = thermal_zone_device_priv(tz); struct ti_bandgap *bgp; diff --git a/drivers/thunderbolt/icm.c b/drivers/thunderbolt/icm.c index dbdcad8d73bf..d8b9c734abd3 100644 --- a/drivers/thunderbolt/icm.c +++ b/drivers/thunderbolt/icm.c @@ -41,6 +41,7 @@ #define PHY_PORT_CS1_LINK_STATE_SHIFT 26 #define ICM_TIMEOUT 5000 /* ms */ +#define ICM_RETRIES 3 #define ICM_APPROVE_TIMEOUT 10000 /* ms */ #define ICM_MAX_LINK 4 @@ -296,10 +297,9 @@ static bool icm_copy(struct tb_cfg_request *req, const struct ctl_pkg *pkg) static int icm_request(struct tb *tb, const void *request, size_t request_size, void *response, size_t response_size, size_t npackets, - unsigned int timeout_msec) + int retries, unsigned int timeout_msec) { struct icm *icm = tb_priv(tb); - int retries = 3; do { struct tb_cfg_request *req; @@ -410,7 +410,7 @@ static int icm_fr_get_route(struct tb *tb, u8 link, u8 depth, u64 *route) return -ENOMEM; ret = icm_request(tb, &request, sizeof(request), switches, - sizeof(*switches), npackets, ICM_TIMEOUT); + sizeof(*switches), npackets, ICM_RETRIES, ICM_TIMEOUT); if (ret) goto err_free; @@ -463,7 +463,7 @@ icm_fr_driver_ready(struct tb *tb, enum tb_security_level *security_level, memset(&reply, 0, sizeof(reply)); ret = icm_request(tb, &request, sizeof(request), &reply, sizeof(reply), - 1, ICM_TIMEOUT); + 1, ICM_RETRIES, ICM_TIMEOUT); if (ret) return ret; @@ -488,7 +488,7 @@ static int icm_fr_approve_switch(struct tb *tb, struct tb_switch *sw) memset(&reply, 0, sizeof(reply)); /* Use larger timeout as establishing tunnels can take some time */ ret = icm_request(tb, &request, sizeof(request), &reply, sizeof(reply), - 1, ICM_APPROVE_TIMEOUT); + 1, ICM_RETRIES, ICM_APPROVE_TIMEOUT); if (ret) return ret; @@ -515,7 +515,7 @@ static int icm_fr_add_switch_key(struct tb *tb, struct tb_switch *sw) memset(&reply, 0, sizeof(reply)); ret = icm_request(tb, &request, sizeof(request), &reply, sizeof(reply), - 1, ICM_TIMEOUT); + 1, ICM_RETRIES, ICM_TIMEOUT); if (ret) return ret; @@ -543,7 +543,7 @@ static int icm_fr_challenge_switch_key(struct tb *tb, struct tb_switch *sw, memset(&reply, 0, sizeof(reply)); ret = icm_request(tb, &request, sizeof(request), &reply, sizeof(reply), - 1, ICM_TIMEOUT); + 1, ICM_RETRIES, ICM_TIMEOUT); if (ret) return ret; @@ -577,7 +577,7 @@ static int icm_fr_approve_xdomain_paths(struct tb *tb, struct tb_xdomain *xd, memset(&reply, 0, sizeof(reply)); ret = icm_request(tb, &request, sizeof(request), &reply, sizeof(reply), - 1, ICM_TIMEOUT); + 1, ICM_RETRIES, ICM_TIMEOUT); if (ret) return ret; @@ -1020,7 +1020,7 @@ icm_tr_driver_ready(struct tb *tb, enum tb_security_level *security_level, memset(&reply, 0, sizeof(reply)); ret = icm_request(tb, &request, sizeof(request), &reply, sizeof(reply), - 1, 20000); + 1, 10, 2000); if (ret) return ret; @@ -1053,7 +1053,7 @@ static int icm_tr_approve_switch(struct tb *tb, struct tb_switch *sw) memset(&reply, 0, sizeof(reply)); ret = icm_request(tb, &request, sizeof(request), &reply, sizeof(reply), - 1, ICM_APPROVE_TIMEOUT); + 1, ICM_RETRIES, ICM_APPROVE_TIMEOUT); if (ret) return ret; @@ -1081,7 +1081,7 @@ static int icm_tr_add_switch_key(struct tb *tb, struct tb_switch *sw) memset(&reply, 0, sizeof(reply)); ret = icm_request(tb, &request, sizeof(request), &reply, sizeof(reply), - 1, ICM_TIMEOUT); + 1, ICM_RETRIES, ICM_TIMEOUT); if (ret) return ret; @@ -1110,7 +1110,7 @@ static int icm_tr_challenge_switch_key(struct tb *tb, struct tb_switch *sw, memset(&reply, 0, sizeof(reply)); ret = icm_request(tb, &request, sizeof(request), &reply, sizeof(reply), - 1, ICM_TIMEOUT); + 1, ICM_RETRIES, ICM_TIMEOUT); if (ret) return ret; @@ -1144,7 +1144,7 @@ static int icm_tr_approve_xdomain_paths(struct tb *tb, struct tb_xdomain *xd, memset(&reply, 0, sizeof(reply)); ret = icm_request(tb, &request, sizeof(request), &reply, sizeof(reply), - 1, ICM_TIMEOUT); + 1, ICM_RETRIES, ICM_TIMEOUT); if (ret) return ret; @@ -1170,7 +1170,7 @@ static int icm_tr_xdomain_tear_down(struct tb *tb, struct tb_xdomain *xd, memset(&reply, 0, sizeof(reply)); ret = icm_request(tb, &request, sizeof(request), &reply, sizeof(reply), - 1, ICM_TIMEOUT); + 1, ICM_RETRIES, ICM_TIMEOUT); if (ret) return ret; @@ -1496,7 +1496,7 @@ icm_ar_driver_ready(struct tb *tb, enum tb_security_level *security_level, memset(&reply, 0, sizeof(reply)); ret = icm_request(tb, &request, sizeof(request), &reply, sizeof(reply), - 1, ICM_TIMEOUT); + 1, ICM_RETRIES, ICM_TIMEOUT); if (ret) return ret; @@ -1522,7 +1522,7 @@ static int icm_ar_get_route(struct tb *tb, u8 link, u8 depth, u64 *route) memset(&reply, 0, sizeof(reply)); ret = icm_request(tb, &request, sizeof(request), &reply, sizeof(reply), - 1, ICM_TIMEOUT); + 1, ICM_RETRIES, ICM_TIMEOUT); if (ret) return ret; @@ -1543,7 +1543,7 @@ static int icm_ar_get_boot_acl(struct tb *tb, uuid_t *uuids, size_t nuuids) memset(&reply, 0, sizeof(reply)); ret = icm_request(tb, &request, sizeof(request), &reply, sizeof(reply), - 1, ICM_TIMEOUT); + 1, ICM_RETRIES, ICM_TIMEOUT); if (ret) return ret; @@ -1604,7 +1604,7 @@ static int icm_ar_set_boot_acl(struct tb *tb, const uuid_t *uuids, memset(&reply, 0, sizeof(reply)); ret = icm_request(tb, &request, sizeof(request), &reply, sizeof(reply), - 1, ICM_TIMEOUT); + 1, ICM_RETRIES, ICM_TIMEOUT); if (ret) return ret; @@ -1626,7 +1626,7 @@ icm_icl_driver_ready(struct tb *tb, enum tb_security_level *security_level, memset(&reply, 0, sizeof(reply)); ret = icm_request(tb, &request, sizeof(request), &reply, sizeof(reply), - 1, 20000); + 1, ICM_RETRIES, 20000); if (ret) return ret; @@ -2298,7 +2298,7 @@ static int icm_usb4_switch_op(struct tb_switch *sw, u16 opcode, u32 *metadata, memset(&reply, 0, sizeof(reply)); ret = icm_request(tb, &request, sizeof(request), &reply, sizeof(reply), - 1, ICM_TIMEOUT); + 1, ICM_RETRIES, ICM_TIMEOUT); if (ret) return ret; diff --git a/drivers/thunderbolt/switch.c b/drivers/thunderbolt/switch.c index 43171cc1cc2d..bd5815f8f23b 100644 --- a/drivers/thunderbolt/switch.c +++ b/drivers/thunderbolt/switch.c @@ -2725,6 +2725,13 @@ int tb_switch_lane_bonding_enable(struct tb_switch *sw) !tb_port_is_width_supported(down, TB_LINK_WIDTH_DUAL)) return 0; + /* + * Both lanes need to be in CL0. Here we assume lane 0 already be in + * CL0 and check just for lane 1. + */ + if (tb_wait_for_port(down->dual_link_port, false) <= 0) + return -ENOTCONN; + ret = tb_port_lane_bonding_enable(up); if (ret) { tb_port_warn(up, "failed to enable lane bonding\n"); diff --git a/drivers/thunderbolt/tb.c b/drivers/thunderbolt/tb.c index dd0a1ef8cf12..27bd6ca6f99e 100644 --- a/drivers/thunderbolt/tb.c +++ b/drivers/thunderbolt/tb.c @@ -1907,14 +1907,14 @@ static void tb_handle_dp_bandwidth_request(struct work_struct *work) in = &sw->ports[ev->port]; if (!tb_port_is_dpin(in)) { tb_port_warn(in, "bandwidth request to non-DP IN adapter\n"); - goto unlock; + goto put_sw; } tb_port_dbg(in, "handling bandwidth allocation request\n"); if (!usb4_dp_port_bandwidth_mode_enabled(in)) { tb_port_warn(in, "bandwidth allocation mode not enabled\n"); - goto unlock; + goto put_sw; } ret = usb4_dp_port_requested_bandwidth(in); @@ -1923,7 +1923,7 @@ static void tb_handle_dp_bandwidth_request(struct work_struct *work) tb_port_dbg(in, "no bandwidth request active\n"); else tb_port_warn(in, "failed to read requested bandwidth\n"); - goto unlock; + goto put_sw; } requested_bw = ret; @@ -1932,7 +1932,7 @@ static void tb_handle_dp_bandwidth_request(struct work_struct *work) tunnel = tb_find_tunnel(tb, TB_TUNNEL_DP, in, NULL); if (!tunnel) { tb_port_warn(in, "failed to find tunnel\n"); - goto unlock; + goto put_sw; } out = tunnel->dst_port; @@ -1959,6 +1959,8 @@ static void tb_handle_dp_bandwidth_request(struct work_struct *work) tb_recalc_estimated_bandwidth(tb); } +put_sw: + tb_switch_put(sw); unlock: mutex_unlock(&tb->lock); diff --git a/drivers/thunderbolt/tmu.c b/drivers/thunderbolt/tmu.c index 747f88703d5c..11f2aec2a5d3 100644 --- a/drivers/thunderbolt/tmu.c +++ b/drivers/thunderbolt/tmu.c @@ -382,7 +382,7 @@ static int tmu_mode_init(struct tb_switch *sw) } else if (ucap && tb_port_tmu_is_unidirectional(up)) { if (tmu_rates[TB_SWITCH_TMU_MODE_LOWRES] == rate) sw->tmu.mode = TB_SWITCH_TMU_MODE_LOWRES; - else if (tmu_rates[TB_SWITCH_TMU_MODE_LOWRES] == rate) + else if (tmu_rates[TB_SWITCH_TMU_MODE_HIFI_UNI] == rate) sw->tmu.mode = TB_SWITCH_TMU_MODE_HIFI_UNI; } else if (rate) { sw->tmu.mode = TB_SWITCH_TMU_MODE_HIFI_BI; diff --git a/drivers/thunderbolt/xdomain.c b/drivers/thunderbolt/xdomain.c index 5b5566862318..9803f0bbf20d 100644 --- a/drivers/thunderbolt/xdomain.c +++ b/drivers/thunderbolt/xdomain.c @@ -703,6 +703,27 @@ out_unlock: mutex_unlock(&xdomain_lock); } +static void start_handshake(struct tb_xdomain *xd) +{ + xd->state = XDOMAIN_STATE_INIT; + queue_delayed_work(xd->tb->wq, &xd->state_work, + msecs_to_jiffies(XDOMAIN_SHORT_TIMEOUT)); +} + +/* Can be called from state_work */ +static void __stop_handshake(struct tb_xdomain *xd) +{ + cancel_delayed_work_sync(&xd->properties_changed_work); + xd->properties_changed_retries = 0; + xd->state_retries = 0; +} + +static void stop_handshake(struct tb_xdomain *xd) +{ + cancel_delayed_work_sync(&xd->state_work); + __stop_handshake(xd); +} + static void tb_xdp_handle_request(struct work_struct *work) { struct xdomain_request_work *xw = container_of(work, typeof(*xw), work); @@ -765,6 +786,15 @@ static void tb_xdp_handle_request(struct work_struct *work) case UUID_REQUEST: tb_dbg(tb, "%llx: received XDomain UUID request\n", route); ret = tb_xdp_uuid_response(ctl, route, sequence, uuid); + /* + * If we've stopped the discovery with an error such as + * timing out, we will restart the handshake now that we + * received UUID request from the remote host. + */ + if (!ret && xd && xd->state == XDOMAIN_STATE_ERROR) { + dev_dbg(&xd->dev, "restarting handshake\n"); + start_handshake(xd); + } break; case LINK_STATE_STATUS_REQUEST: @@ -1521,6 +1551,13 @@ static void tb_xdomain_queue_properties_changed(struct tb_xdomain *xd) msecs_to_jiffies(XDOMAIN_SHORT_TIMEOUT)); } +static void tb_xdomain_failed(struct tb_xdomain *xd) +{ + xd->state = XDOMAIN_STATE_ERROR; + queue_delayed_work(xd->tb->wq, &xd->state_work, + msecs_to_jiffies(XDOMAIN_DEFAULT_TIMEOUT)); +} + static void tb_xdomain_state_work(struct work_struct *work) { struct tb_xdomain *xd = container_of(work, typeof(*xd), state_work.work); @@ -1547,7 +1584,7 @@ static void tb_xdomain_state_work(struct work_struct *work) if (ret) { if (ret == -EAGAIN) goto retry_state; - xd->state = XDOMAIN_STATE_ERROR; + tb_xdomain_failed(xd); } else { tb_xdomain_queue_properties_changed(xd); if (xd->bonding_possible) @@ -1612,7 +1649,7 @@ static void tb_xdomain_state_work(struct work_struct *work) if (ret) { if (ret == -EAGAIN) goto retry_state; - xd->state = XDOMAIN_STATE_ERROR; + tb_xdomain_failed(xd); } else { xd->state = XDOMAIN_STATE_ENUMERATED; } @@ -1623,6 +1660,8 @@ static void tb_xdomain_state_work(struct work_struct *work) break; case XDOMAIN_STATE_ERROR: + dev_dbg(&xd->dev, "discovery failed, stopping handshake\n"); + __stop_handshake(xd); break; default: @@ -1833,21 +1872,6 @@ static void tb_xdomain_release(struct device *dev) kfree(xd); } -static void start_handshake(struct tb_xdomain *xd) -{ - xd->state = XDOMAIN_STATE_INIT; - queue_delayed_work(xd->tb->wq, &xd->state_work, - msecs_to_jiffies(XDOMAIN_SHORT_TIMEOUT)); -} - -static void stop_handshake(struct tb_xdomain *xd) -{ - cancel_delayed_work_sync(&xd->properties_changed_work); - cancel_delayed_work_sync(&xd->state_work); - xd->properties_changed_retries = 0; - xd->state_retries = 0; -} - static int __maybe_unused tb_xdomain_suspend(struct device *dev) { stop_handshake(tb_to_xdomain(dev)); diff --git a/drivers/tty/n_gsm.c b/drivers/tty/n_gsm.c index b3550ff9c494..1f3aba607cd5 100644 --- a/drivers/tty/n_gsm.c +++ b/drivers/tty/n_gsm.c @@ -3097,10 +3097,8 @@ static void gsm_cleanup_mux(struct gsm_mux *gsm, bool disc) gsm->has_devices = false; } for (i = NUM_DLCI - 1; i >= 0; i--) - if (gsm->dlci[i]) { + if (gsm->dlci[i]) gsm_dlci_release(gsm->dlci[i]); - gsm->dlci[i] = NULL; - } mutex_unlock(&gsm->mutex); /* Now wipe the queues */ tty_ldisc_flush(gsm->tty); diff --git a/drivers/tty/serial/8250/8250_omap.c b/drivers/tty/serial/8250/8250_omap.c index 26dd089d8e82..ca972fd37725 100644 --- a/drivers/tty/serial/8250/8250_omap.c +++ b/drivers/tty/serial/8250/8250_omap.c @@ -1617,7 +1617,7 @@ static int omap8250_suspend(struct device *dev) { struct omap8250_priv *priv = dev_get_drvdata(dev); struct uart_8250_port *up = serial8250_get_port(priv->line); - int err; + int err = 0; serial8250_suspend_port(priv->line); @@ -1627,7 +1627,8 @@ static int omap8250_suspend(struct device *dev) if (!device_may_wakeup(dev)) priv->wer = 0; serial_out(up, UART_OMAP_WER, priv->wer); - err = pm_runtime_force_suspend(dev); + if (uart_console(&up->port) && console_suspend_enabled) + err = pm_runtime_force_suspend(dev); flush_work(&priv->qos_work); return err; @@ -1636,11 +1637,15 @@ static int omap8250_suspend(struct device *dev) static int omap8250_resume(struct device *dev) { struct omap8250_priv *priv = dev_get_drvdata(dev); + struct uart_8250_port *up = serial8250_get_port(priv->line); int err; - err = pm_runtime_force_resume(dev); - if (err) - return err; + if (uart_console(&up->port) && console_suspend_enabled) { + err = pm_runtime_force_resume(dev); + if (err) + return err; + } + serial8250_resume_port(priv->line); /* Paired with pm_runtime_resume_and_get() in omap8250_suspend() */ pm_runtime_mark_last_busy(dev); @@ -1717,16 +1722,6 @@ static int omap8250_runtime_suspend(struct device *dev) if (priv->line >= 0) up = serial8250_get_port(priv->line); - /* - * When using 'no_console_suspend', the console UART must not be - * suspended. Since driver suspend is managed by runtime suspend, - * preventing runtime suspend (by returning error) will keep device - * active during suspend. - */ - if (priv->is_suspending && !console_suspend_enabled) { - if (up && uart_console(&up->port)) - return -EBUSY; - } if (priv->habit & UART_ERRATA_CLOCK_DISABLE) { int ret; diff --git a/drivers/tty/serial/8250/8250_port.c b/drivers/tty/serial/8250/8250_port.c index fb891b67968f..141627370aab 100644 --- a/drivers/tty/serial/8250/8250_port.c +++ b/drivers/tty/serial/8250/8250_port.c @@ -1936,7 +1936,10 @@ int serial8250_handle_irq(struct uart_port *port, unsigned int iir) skip_rx = true; if (status & (UART_LSR_DR | UART_LSR_BI) && !skip_rx) { - if (irqd_is_wakeup_set(irq_get_irq_data(port->irq))) + struct irq_data *d; + + d = irq_get_irq_data(port->irq); + if (d && irqd_is_wakeup_set(d)) pm_wakeup_event(tport->tty->dev, 0); if (!up->dma || handle_rx_dma(up, iir)) status = serial8250_rx_chars(up, status); diff --git a/drivers/tty/serial/serial_core.c b/drivers/tty/serial/serial_core.c index 7bdc21d5e13b..d5ba6e90bd95 100644 --- a/drivers/tty/serial/serial_core.c +++ b/drivers/tty/serial/serial_core.c @@ -156,7 +156,7 @@ static void __uart_start(struct uart_state *state) * enabled, serial_port_runtime_resume() calls start_tx() again * after enabling the device. */ - if (pm_runtime_active(&port_dev->dev)) + if (!pm_runtime_enabled(port->dev) || pm_runtime_active(port->dev)) port->ops->start_tx(port); pm_runtime_mark_last_busy(&port_dev->dev); pm_runtime_put_autosuspend(&port_dev->dev); @@ -1404,12 +1404,18 @@ static void uart_set_rs485_termination(struct uart_port *port, static int uart_rs485_config(struct uart_port *port) { struct serial_rs485 *rs485 = &port->rs485; + unsigned long flags; int ret; + if (!(rs485->flags & SER_RS485_ENABLED)) + return 0; + uart_sanitize_serial_rs485(port, rs485); uart_set_rs485_termination(port, rs485); + spin_lock_irqsave(&port->lock, flags); ret = port->rs485_config(port, NULL, rs485); + spin_unlock_irqrestore(&port->lock, flags); if (ret) memset(rs485, 0, sizeof(*rs485)); @@ -2474,11 +2480,10 @@ int uart_resume_port(struct uart_driver *drv, struct uart_port *uport) if (ret == 0) { if (tty) uart_change_line_settings(tty, state, NULL); + uart_rs485_config(uport); spin_lock_irq(&uport->lock); if (!(uport->rs485.flags & SER_RS485_ENABLED)) ops->set_mctrl(uport, uport->mctrl); - else - uart_rs485_config(uport); ops->start_tx(uport); spin_unlock_irq(&uport->lock); tty_port_set_initialized(port, true); @@ -2587,10 +2592,10 @@ uart_configure_port(struct uart_driver *drv, struct uart_state *state, port->mctrl &= TIOCM_DTR; if (!(port->rs485.flags & SER_RS485_ENABLED)) port->ops->set_mctrl(port, port->mctrl); - else - uart_rs485_config(port); spin_unlock_irqrestore(&port->lock, flags); + uart_rs485_config(port); + /* * If this driver supports console, and it hasn't been * successfully registered yet, try to re-register it. diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c index 93417518c04d..8382e8cfa414 100644 --- a/drivers/ufs/core/ufshcd.c +++ b/drivers/ufs/core/ufshcd.c @@ -22,6 +22,7 @@ #include #include #include +#include #include #include #include @@ -2299,7 +2300,11 @@ static inline int ufshcd_hba_capabilities(struct ufs_hba *hba) */ static inline bool ufshcd_ready_for_uic_cmd(struct ufs_hba *hba) { - return ufshcd_readl(hba, REG_CONTROLLER_STATUS) & UIC_COMMAND_READY; + u32 val; + int ret = read_poll_timeout(ufshcd_readl, val, val & UIC_COMMAND_READY, + 500, UIC_CMD_TIMEOUT * 1000, false, hba, + REG_CONTROLLER_STATUS); + return ret == 0 ? true : false; } /** @@ -2392,7 +2397,6 @@ __ufshcd_send_uic_cmd(struct ufs_hba *hba, struct uic_command *uic_cmd, bool completion) { lockdep_assert_held(&hba->uic_cmd_mutex); - lockdep_assert_held(hba->host->host_lock); if (!ufshcd_ready_for_uic_cmd(hba)) { dev_err(hba->dev, @@ -2419,7 +2423,6 @@ __ufshcd_send_uic_cmd(struct ufs_hba *hba, struct uic_command *uic_cmd, int ufshcd_send_uic_cmd(struct ufs_hba *hba, struct uic_command *uic_cmd) { int ret; - unsigned long flags; if (hba->quirks & UFSHCD_QUIRK_BROKEN_UIC_CMD) return 0; @@ -2428,9 +2431,7 @@ int ufshcd_send_uic_cmd(struct ufs_hba *hba, struct uic_command *uic_cmd) mutex_lock(&hba->uic_cmd_mutex); ufshcd_add_delay_before_dme_cmd(hba); - spin_lock_irqsave(hba->host->host_lock, flags); ret = __ufshcd_send_uic_cmd(hba, uic_cmd, true); - spin_unlock_irqrestore(hba->host->host_lock, flags); if (!ret) ret = ufshcd_wait_for_uic_cmd(hba, uic_cmd); @@ -4133,8 +4134,8 @@ static int ufshcd_uic_pwr_ctrl(struct ufs_hba *hba, struct uic_command *cmd) wmb(); reenable_intr = true; } - ret = __ufshcd_send_uic_cmd(hba, cmd, false); spin_unlock_irqrestore(hba->host->host_lock, flags); + ret = __ufshcd_send_uic_cmd(hba, cmd, false); if (ret) { dev_err(hba->dev, "pwr ctrl cmd 0x%x with mode 0x%x uic error %d\n", @@ -6894,7 +6895,7 @@ static int ufshcd_clear_tm_cmd(struct ufs_hba *hba, int tag) mask, 0, 1000, 1000); dev_err(hba->dev, "Clearing task management function with tag %d %s\n", - tag, err ? "succeeded" : "failed"); + tag, err < 0 ? "failed" : "succeeded"); out: return err; diff --git a/drivers/usb/cdns3/cdnsp-gadget.c b/drivers/usb/cdns3/cdnsp-gadget.c index fff9ec9c391f..4b67749edb99 100644 --- a/drivers/usb/cdns3/cdnsp-gadget.c +++ b/drivers/usb/cdns3/cdnsp-gadget.c @@ -1125,6 +1125,9 @@ static int cdnsp_gadget_ep_dequeue(struct usb_ep *ep, unsigned long flags; int ret; + if (request->status != -EINPROGRESS) + return 0; + if (!pep->endpoint.desc) { dev_err(pdev->dev, "%s: can't dequeue to disabled endpoint\n", diff --git a/drivers/usb/cdns3/core.h b/drivers/usb/cdns3/core.h index 4a4dbc2c1561..81a9c9d6be08 100644 --- a/drivers/usb/cdns3/core.h +++ b/drivers/usb/cdns3/core.h @@ -131,8 +131,7 @@ void cdns_set_active(struct cdns *cdns, u8 set_active); #else /* CONFIG_PM_SLEEP */ static inline int cdns_resume(struct cdns *cdns) { return 0; } -static inline int cdns_set_active(struct cdns *cdns, u8 set_active) -{ return 0; } +static inline void cdns_set_active(struct cdns *cdns, u8 set_active) { } static inline int cdns_suspend(struct cdns *cdns) { return 0; } #endif /* CONFIG_PM_SLEEP */ diff --git a/drivers/usb/core/hub.c b/drivers/usb/core/hub.c index 3c54b218301c..0ff47eeffb49 100644 --- a/drivers/usb/core/hub.c +++ b/drivers/usb/core/hub.c @@ -151,6 +151,10 @@ int usb_device_supports_lpm(struct usb_device *udev) if (udev->quirks & USB_QUIRK_NO_LPM) return 0; + /* Skip if the device BOS descriptor couldn't be read */ + if (!udev->bos) + return 0; + /* USB 2.1 (and greater) devices indicate LPM support through * their USB 2.0 Extended Capabilities BOS descriptor. */ @@ -327,6 +331,10 @@ static void usb_set_lpm_parameters(struct usb_device *udev) if (!udev->lpm_capable || udev->speed < USB_SPEED_SUPER) return; + /* Skip if the device BOS descriptor couldn't be read */ + if (!udev->bos) + return; + hub = usb_hub_to_struct_hub(udev->parent); /* It doesn't take time to transition the roothub into U0, since it * doesn't have an upstream link. @@ -2704,13 +2712,17 @@ out_authorized: static enum usb_ssp_rate get_port_ssp_rate(struct usb_device *hdev, u32 ext_portstatus) { - struct usb_ssp_cap_descriptor *ssp_cap = hdev->bos->ssp_cap; + struct usb_ssp_cap_descriptor *ssp_cap; u32 attr; u8 speed_id; u8 ssac; u8 lanes; int i; + if (!hdev->bos) + goto out; + + ssp_cap = hdev->bos->ssp_cap; if (!ssp_cap) goto out; @@ -4215,8 +4227,15 @@ static void usb_enable_link_state(struct usb_hcd *hcd, struct usb_device *udev, enum usb3_link_state state) { int timeout; - __u8 u1_mel = udev->bos->ss_cap->bU1devExitLat; - __le16 u2_mel = udev->bos->ss_cap->bU2DevExitLat; + __u8 u1_mel; + __le16 u2_mel; + + /* Skip if the device BOS descriptor couldn't be read */ + if (!udev->bos) + return; + + u1_mel = udev->bos->ss_cap->bU1devExitLat; + u2_mel = udev->bos->ss_cap->bU2DevExitLat; /* If the device says it doesn't have *any* exit latency to come out of * U1 or U2, it's probably lying. Assume it doesn't implement that link diff --git a/drivers/usb/core/hub.h b/drivers/usb/core/hub.h index 37897afd1b64..d44dd7f6623e 100644 --- a/drivers/usb/core/hub.h +++ b/drivers/usb/core/hub.h @@ -153,7 +153,7 @@ static inline int hub_is_superspeedplus(struct usb_device *hdev) { return (hdev->descriptor.bDeviceProtocol == USB_HUB_PR_SS && le16_to_cpu(hdev->descriptor.bcdUSB) >= 0x0310 && - hdev->bos->ssp_cap); + hdev->bos && hdev->bos->ssp_cap); } static inline unsigned hub_power_on_good_delay(struct usb_hub *hub) diff --git a/drivers/usb/dwc3/core.c b/drivers/usb/dwc3/core.c index 9c6bf054f15d..343d2570189f 100644 --- a/drivers/usb/dwc3/core.c +++ b/drivers/usb/dwc3/core.c @@ -279,9 +279,46 @@ int dwc3_core_soft_reset(struct dwc3 *dwc) * XHCI driver will reset the host block. If dwc3 was configured for * host-only mode or current role is host, then we can return early. */ - if (dwc->dr_mode == USB_DR_MODE_HOST || dwc->current_dr_role == DWC3_GCTL_PRTCAP_HOST) + if (dwc->current_dr_role == DWC3_GCTL_PRTCAP_HOST) return 0; + /* + * If the dr_mode is host and the dwc->current_dr_role is not the + * corresponding DWC3_GCTL_PRTCAP_HOST, then the dwc3_core_init_mode + * isn't executed yet. Ensure the phy is ready before the controller + * updates the GCTL.PRTCAPDIR or other settings by soft-resetting + * the phy. + * + * Note: GUSB3PIPECTL[n] and GUSB2PHYCFG[n] are port settings where n + * is port index. If this is a multiport host, then we need to reset + * all active ports. + */ + if (dwc->dr_mode == USB_DR_MODE_HOST) { + u32 usb3_port; + u32 usb2_port; + + usb3_port = dwc3_readl(dwc->regs, DWC3_GUSB3PIPECTL(0)); + usb3_port |= DWC3_GUSB3PIPECTL_PHYSOFTRST; + dwc3_writel(dwc->regs, DWC3_GUSB3PIPECTL(0), usb3_port); + + usb2_port = dwc3_readl(dwc->regs, DWC3_GUSB2PHYCFG(0)); + usb2_port |= DWC3_GUSB2PHYCFG_PHYSOFTRST; + dwc3_writel(dwc->regs, DWC3_GUSB2PHYCFG(0), usb2_port); + + /* Small delay for phy reset assertion */ + usleep_range(1000, 2000); + + usb3_port &= ~DWC3_GUSB3PIPECTL_PHYSOFTRST; + dwc3_writel(dwc->regs, DWC3_GUSB3PIPECTL(0), usb3_port); + + usb2_port &= ~DWC3_GUSB2PHYCFG_PHYSOFTRST; + dwc3_writel(dwc->regs, DWC3_GUSB2PHYCFG(0), usb2_port); + + /* Wait for clock synchronization */ + msleep(50); + return 0; + } + reg = dwc3_readl(dwc->regs, DWC3_DCTL); reg |= DWC3_DCTL_CSFTRST; reg &= ~DWC3_DCTL_RUN_STOP; diff --git a/drivers/usb/gadget/function/f_ncm.c b/drivers/usb/gadget/function/f_ncm.c index feccf4c8cc4f..e6ab8cc225ff 100644 --- a/drivers/usb/gadget/function/f_ncm.c +++ b/drivers/usb/gadget/function/f_ncm.c @@ -1156,7 +1156,8 @@ static int ncm_unwrap_ntb(struct gether *port, struct sk_buff_head *list) { struct f_ncm *ncm = func_to_ncm(&port->func); - __le16 *tmp = (void *) skb->data; + unsigned char *ntb_ptr = skb->data; + __le16 *tmp; unsigned index, index2; int ndp_index; unsigned dg_len, dg_len2; @@ -1169,6 +1170,10 @@ static int ncm_unwrap_ntb(struct gether *port, const struct ndp_parser_opts *opts = ncm->parser_opts; unsigned crc_len = ncm->is_crc ? sizeof(uint32_t) : 0; int dgram_counter; + int to_process = skb->len; + +parse_ntb: + tmp = (__le16 *)ntb_ptr; /* dwSignature */ if (get_unaligned_le32(tmp) != opts->nth_sign) { @@ -1215,7 +1220,7 @@ static int ncm_unwrap_ntb(struct gether *port, * walk through NDP * dwSignature */ - tmp = (void *)(skb->data + ndp_index); + tmp = (__le16 *)(ntb_ptr + ndp_index); if (get_unaligned_le32(tmp) != ncm->ndp_sign) { INFO(port->func.config->cdev, "Wrong NDP SIGN\n"); goto err; @@ -1272,11 +1277,11 @@ static int ncm_unwrap_ntb(struct gether *port, if (ncm->is_crc) { uint32_t crc, crc2; - crc = get_unaligned_le32(skb->data + + crc = get_unaligned_le32(ntb_ptr + index + dg_len - crc_len); crc2 = ~crc32_le(~0, - skb->data + index, + ntb_ptr + index, dg_len - crc_len); if (crc != crc2) { INFO(port->func.config->cdev, @@ -1303,7 +1308,7 @@ static int ncm_unwrap_ntb(struct gether *port, dg_len - crc_len); if (skb2 == NULL) goto err; - skb_put_data(skb2, skb->data + index, + skb_put_data(skb2, ntb_ptr + index, dg_len - crc_len); skb_queue_tail(list, skb2); @@ -1316,10 +1321,17 @@ static int ncm_unwrap_ntb(struct gether *port, } while (ndp_len > 2 * (opts->dgram_item_len * 2)); } while (ndp_index); - dev_consume_skb_any(skb); - VDBG(port->func.config->cdev, "Parsed NTB with %d frames\n", dgram_counter); + + to_process -= block_len; + if (to_process != 0) { + ntb_ptr = (unsigned char *)(ntb_ptr + block_len); + goto parse_ntb; + } + + dev_consume_skb_any(skb); + return 0; err: skb_queue_purge(list); diff --git a/drivers/usb/gadget/udc/udc-xilinx.c b/drivers/usb/gadget/udc/udc-xilinx.c index 56b8286a8009..74590f93ea61 100644 --- a/drivers/usb/gadget/udc/udc-xilinx.c +++ b/drivers/usb/gadget/udc/udc-xilinx.c @@ -497,11 +497,13 @@ static int xudc_eptxrx(struct xusb_ep *ep, struct xusb_req *req, /* Get the Buffer address and copy the transmit data.*/ eprambase = (u32 __force *)(udc->addr + ep->rambase); if (ep->is_in) { - memcpy(eprambase, bufferptr, bytestosend); + memcpy_toio((void __iomem *)eprambase, bufferptr, + bytestosend); udc->write_fn(udc->addr, ep->offset + XUSB_EP_BUF0COUNT_OFFSET, bufferlen); } else { - memcpy(bufferptr, eprambase, bytestosend); + memcpy_toio((void __iomem *)bufferptr, eprambase, + bytestosend); } /* * Enable the buffer for transmission. @@ -515,11 +517,13 @@ static int xudc_eptxrx(struct xusb_ep *ep, struct xusb_req *req, eprambase = (u32 __force *)(udc->addr + ep->rambase + ep->ep_usb.maxpacket); if (ep->is_in) { - memcpy(eprambase, bufferptr, bytestosend); + memcpy_toio((void __iomem *)eprambase, bufferptr, + bytestosend); udc->write_fn(udc->addr, ep->offset + XUSB_EP_BUF1COUNT_OFFSET, bufferlen); } else { - memcpy(bufferptr, eprambase, bytestosend); + memcpy_toio((void __iomem *)bufferptr, eprambase, + bytestosend); } /* * Enable the buffer for transmission. @@ -1021,7 +1025,7 @@ static int __xudc_ep0_queue(struct xusb_ep *ep0, struct xusb_req *req) udc->addr); length = req->usb_req.actual = min_t(u32, length, EP0_MAX_PACKET); - memcpy(corebuf, req->usb_req.buf, length); + memcpy_toio((void __iomem *)corebuf, req->usb_req.buf, length); udc->write_fn(udc->addr, XUSB_EP_BUF0COUNT_OFFSET, length); udc->write_fn(udc->addr, XUSB_BUFFREADY_OFFSET, 1); } else { @@ -1752,7 +1756,7 @@ static void xudc_handle_setup(struct xusb_udc *udc) /* Load up the chapter 9 command buffer.*/ ep0rambase = (u32 __force *) (udc->addr + XUSB_SETUP_PKT_ADDR_OFFSET); - memcpy(&setup, ep0rambase, 8); + memcpy_toio((void __iomem *)&setup, ep0rambase, 8); udc->setup = setup; udc->setup.wValue = cpu_to_le16((u16 __force)setup.wValue); @@ -1839,7 +1843,7 @@ static void xudc_ep0_out(struct xusb_udc *udc) (ep0->rambase << 2)); buffer = req->usb_req.buf + req->usb_req.actual; req->usb_req.actual = req->usb_req.actual + bytes_to_rx; - memcpy(buffer, ep0rambase, bytes_to_rx); + memcpy_toio((void __iomem *)buffer, ep0rambase, bytes_to_rx); if (req->usb_req.length == req->usb_req.actual) { /* Data transfer completed get ready for Status stage */ @@ -1915,7 +1919,7 @@ static void xudc_ep0_in(struct xusb_udc *udc) (ep0->rambase << 2)); buffer = req->usb_req.buf + req->usb_req.actual; req->usb_req.actual = req->usb_req.actual + length; - memcpy(ep0rambase, buffer, length); + memcpy_toio((void __iomem *)ep0rambase, buffer, length); } udc->write_fn(udc->addr, XUSB_EP_BUF0COUNT_OFFSET, count); udc->write_fn(udc->addr, XUSB_BUFFREADY_OFFSET, 1); diff --git a/drivers/usb/host/xhci-hub.c b/drivers/usb/host/xhci-hub.c index 0054d02239e2..0df5d807a77e 100644 --- a/drivers/usb/host/xhci-hub.c +++ b/drivers/usb/host/xhci-hub.c @@ -1062,19 +1062,19 @@ static void xhci_get_usb3_port_status(struct xhci_port *port, u32 *status, *status |= USB_PORT_STAT_C_CONFIG_ERROR << 16; /* USB3 specific wPortStatus bits */ - if (portsc & PORT_POWER) { + if (portsc & PORT_POWER) *status |= USB_SS_PORT_STAT_POWER; - /* link state handling */ - if (link_state == XDEV_U0) - bus_state->suspended_ports &= ~(1 << portnum); - } - /* remote wake resume signaling complete */ - if (bus_state->port_remote_wakeup & (1 << portnum) && + /* no longer suspended or resuming */ + if (link_state != XDEV_U3 && link_state != XDEV_RESUME && link_state != XDEV_RECOVERY) { - bus_state->port_remote_wakeup &= ~(1 << portnum); - usb_hcd_end_port_resume(&hcd->self, portnum); + /* remote wake resume signaling complete */ + if (bus_state->port_remote_wakeup & (1 << portnum)) { + bus_state->port_remote_wakeup &= ~(1 << portnum); + usb_hcd_end_port_resume(&hcd->self, portnum); + } + bus_state->suspended_ports &= ~(1 << portnum); } xhci_hub_report_usb3_link_state(xhci, status, portsc); @@ -1131,6 +1131,7 @@ static void xhci_get_usb2_port_status(struct xhci_port *port, u32 *status, usb_hcd_end_port_resume(&port->rhub->hcd->self, portnum); } port->rexit_active = 0; + bus_state->suspended_ports &= ~(1 << portnum); } } diff --git a/drivers/usb/host/xhci-mem.c b/drivers/usb/host/xhci-mem.c index 8714ab5bf04d..0a37f0d511cf 100644 --- a/drivers/usb/host/xhci-mem.c +++ b/drivers/usb/host/xhci-mem.c @@ -2285,8 +2285,8 @@ xhci_add_interrupter(struct xhci_hcd *xhci, struct xhci_interrupter *ir, writel(erst_size, &ir->ir_set->erst_size); erst_base = xhci_read_64(xhci, &ir->ir_set->erst_base); - erst_base &= ERST_PTR_MASK; - erst_base |= (ir->erst.erst_dma_addr & (u64) ~ERST_PTR_MASK); + erst_base &= ERST_BASE_RSVDP; + erst_base |= ir->erst.erst_dma_addr & ~ERST_BASE_RSVDP; xhci_write_64(xhci, erst_base, &ir->ir_set->erst_base); /* Set the event ring dequeue address of this interrupter */ diff --git a/drivers/usb/host/xhci-ring.c b/drivers/usb/host/xhci-ring.c index 1dde53f6eb31..3e5dc0723a8f 100644 --- a/drivers/usb/host/xhci-ring.c +++ b/drivers/usb/host/xhci-ring.c @@ -798,7 +798,7 @@ static void xhci_giveback_urb_in_irq(struct xhci_hcd *xhci, static void xhci_unmap_td_bounce_buffer(struct xhci_hcd *xhci, struct xhci_ring *ring, struct xhci_td *td) { - struct device *dev = xhci_to_hcd(xhci)->self.controller; + struct device *dev = xhci_to_hcd(xhci)->self.sysdev; struct xhci_segment *seg = td->bounce_seg; struct urb *urb = td->urb; size_t len; @@ -2996,7 +2996,8 @@ static int xhci_handle_event(struct xhci_hcd *xhci, struct xhci_interrupter *ir) */ static void xhci_update_erst_dequeue(struct xhci_hcd *xhci, struct xhci_interrupter *ir, - union xhci_trb *event_ring_deq) + union xhci_trb *event_ring_deq, + bool clear_ehb) { u64 temp_64; dma_addr_t deq; @@ -3017,12 +3018,13 @@ static void xhci_update_erst_dequeue(struct xhci_hcd *xhci, return; /* Update HC event ring dequeue pointer */ - temp_64 &= ERST_PTR_MASK; + temp_64 &= ERST_DESI_MASK; temp_64 |= ((u64) deq & (u64) ~ERST_PTR_MASK); } /* Clear the event handler busy flag (RW1C) */ - temp_64 |= ERST_EHB; + if (clear_ehb) + temp_64 |= ERST_EHB; xhci_write_64(xhci, temp_64, &ir->ir_set->erst_dequeue); } @@ -3103,7 +3105,7 @@ irqreturn_t xhci_irq(struct usb_hcd *hcd) while (xhci_handle_event(xhci, ir) > 0) { if (event_loop++ < TRBS_PER_SEGMENT / 2) continue; - xhci_update_erst_dequeue(xhci, ir, event_ring_deq); + xhci_update_erst_dequeue(xhci, ir, event_ring_deq, false); event_ring_deq = ir->event_ring->dequeue; /* ring is half-full, force isoc trbs to interrupt more often */ @@ -3113,7 +3115,7 @@ irqreturn_t xhci_irq(struct usb_hcd *hcd) event_loop = 0; } - xhci_update_erst_dequeue(xhci, ir, event_ring_deq); + xhci_update_erst_dequeue(xhci, ir, event_ring_deq, true); ret = IRQ_HANDLED; out: @@ -3469,7 +3471,7 @@ static u32 xhci_td_remainder(struct xhci_hcd *xhci, int transferred, static int xhci_align_td(struct xhci_hcd *xhci, struct urb *urb, u32 enqd_len, u32 *trb_buff_len, struct xhci_segment *seg) { - struct device *dev = xhci_to_hcd(xhci)->self.controller; + struct device *dev = xhci_to_hcd(xhci)->self.sysdev; unsigned int unalign; unsigned int max_pkt; u32 new_buff_len; diff --git a/drivers/usb/host/xhci.h b/drivers/usb/host/xhci.h index 7e282b4522c0..5df370482521 100644 --- a/drivers/usb/host/xhci.h +++ b/drivers/usb/host/xhci.h @@ -514,7 +514,7 @@ struct xhci_intr_reg { #define ERST_SIZE_MASK (0xffff << 16) /* erst_base bitmasks */ -#define ERST_BASE_RSVDP (0x3f) +#define ERST_BASE_RSVDP (GENMASK_ULL(5, 0)) /* erst_dequeue bitmasks */ /* Dequeue ERST Segment Index (DESI) - Segment number (or alias) diff --git a/drivers/usb/misc/onboard_usb_hub.c b/drivers/usb/misc/onboard_usb_hub.c index 3da1a4659c5f..57bbe1309094 100644 --- a/drivers/usb/misc/onboard_usb_hub.c +++ b/drivers/usb/misc/onboard_usb_hub.c @@ -434,6 +434,7 @@ static const struct usb_device_id onboard_hub_id_table[] = { { USB_DEVICE(VENDOR_ID_GENESYS, 0x0608) }, /* Genesys Logic GL850G USB 2.0 */ { USB_DEVICE(VENDOR_ID_GENESYS, 0x0610) }, /* Genesys Logic GL852G USB 2.0 */ { USB_DEVICE(VENDOR_ID_GENESYS, 0x0620) }, /* Genesys Logic GL3523 USB 3.1 */ + { USB_DEVICE(VENDOR_ID_MICROCHIP, 0x2412) }, /* USB2412 USB 2.0 */ { USB_DEVICE(VENDOR_ID_MICROCHIP, 0x2514) }, /* USB2514B USB 2.0 */ { USB_DEVICE(VENDOR_ID_MICROCHIP, 0x2517) }, /* USB2517 USB 2.0 */ { USB_DEVICE(VENDOR_ID_REALTEK, 0x0411) }, /* RTS5411 USB 3.1 */ diff --git a/drivers/usb/misc/onboard_usb_hub.h b/drivers/usb/misc/onboard_usb_hub.h index 4026ba64c592..2a4ab5ac0ebe 100644 --- a/drivers/usb/misc/onboard_usb_hub.h +++ b/drivers/usb/misc/onboard_usb_hub.h @@ -47,6 +47,7 @@ static const struct onboard_hub_pdata vialab_vl817_data = { }; static const struct of_device_id onboard_hub_match[] = { + { .compatible = "usb424,2412", .data = µchip_usb424_data, }, { .compatible = "usb424,2514", .data = µchip_usb424_data, }, { .compatible = "usb424,2517", .data = µchip_usb424_data, }, { .compatible = "usb451,8140", .data = &ti_tusb8041_data, }, diff --git a/drivers/usb/musb/musb_debugfs.c b/drivers/usb/musb/musb_debugfs.c index 78c726a71b17..2d623284edf6 100644 --- a/drivers/usb/musb/musb_debugfs.c +++ b/drivers/usb/musb/musb_debugfs.c @@ -39,7 +39,7 @@ static const struct musb_register_map musb_regmap[] = { { "IntrUsbE", MUSB_INTRUSBE, 8 }, { "DevCtl", MUSB_DEVCTL, 8 }, { "VControl", 0x68, 32 }, - { "HWVers", 0x69, 16 }, + { "HWVers", MUSB_HWVERS, 16 }, { "LinkInfo", MUSB_LINKINFO, 8 }, { "VPLen", MUSB_VPLEN, 8 }, { "HS_EOF1", MUSB_HS_EOF1, 8 }, diff --git a/drivers/usb/musb/musb_host.c b/drivers/usb/musb/musb_host.c index a02c29216955..bc4507781167 100644 --- a/drivers/usb/musb/musb_host.c +++ b/drivers/usb/musb/musb_host.c @@ -321,10 +321,16 @@ static void musb_advance_schedule(struct musb *musb, struct urb *urb, musb_giveback(musb, urb, status); qh->is_ready = ready; + /* + * musb->lock had been unlocked in musb_giveback, so qh may + * be freed, need to get it again + */ + qh = musb_ep_get_qh(hw_ep, is_in); + /* reclaim resources (and bandwidth) ASAP; deschedule it, and * invalidate qh as soon as list_empty(&hep->urb_list) */ - if (list_empty(&qh->hep->urb_list)) { + if (qh && list_empty(&qh->hep->urb_list)) { struct list_head *head; struct dma_controller *dma = musb->dma_controller; @@ -2398,6 +2404,7 @@ static int musb_urb_dequeue(struct usb_hcd *hcd, struct urb *urb, int status) * and its URB list has emptied, recycle this qh. */ if (ready && list_empty(&qh->hep->urb_list)) { + musb_ep_set_qh(qh->hw_ep, is_in, NULL); qh->hep->hcpriv = NULL; list_del(&qh->ring); kfree(qh); diff --git a/drivers/usb/serial/option.c b/drivers/usb/serial/option.c index 7994a4549a6c..45dcfaadaf98 100644 --- a/drivers/usb/serial/option.c +++ b/drivers/usb/serial/option.c @@ -203,6 +203,9 @@ static void option_instat_callback(struct urb *urb); #define DELL_PRODUCT_5829E_ESIM 0x81e4 #define DELL_PRODUCT_5829E 0x81e6 +#define DELL_PRODUCT_FM101R 0x8213 +#define DELL_PRODUCT_FM101R_ESIM 0x8215 + #define KYOCERA_VENDOR_ID 0x0c88 #define KYOCERA_PRODUCT_KPC650 0x17da #define KYOCERA_PRODUCT_KPC680 0x180a @@ -1108,6 +1111,8 @@ static const struct usb_device_id option_ids[] = { .driver_info = RSVD(0) | RSVD(6) }, { USB_DEVICE(DELL_VENDOR_ID, DELL_PRODUCT_5829E_ESIM), .driver_info = RSVD(0) | RSVD(6) }, + { USB_DEVICE_INTERFACE_CLASS(DELL_VENDOR_ID, DELL_PRODUCT_FM101R, 0xff) }, + { USB_DEVICE_INTERFACE_CLASS(DELL_VENDOR_ID, DELL_PRODUCT_FM101R_ESIM, 0xff) }, { USB_DEVICE(ANYDATA_VENDOR_ID, ANYDATA_PRODUCT_ADU_E100A) }, /* ADU-E100, ADU-310 */ { USB_DEVICE(ANYDATA_VENDOR_ID, ANYDATA_PRODUCT_ADU_500A) }, { USB_DEVICE(ANYDATA_VENDOR_ID, ANYDATA_PRODUCT_ADU_620UW) }, @@ -1290,6 +1295,7 @@ static const struct usb_device_id option_ids[] = { .driver_info = NCTRL(0) | RSVD(3) }, { USB_DEVICE_INTERFACE_CLASS(TELIT_VENDOR_ID, 0x1033, 0xff), /* Telit LE910C1-EUX (ECM) */ .driver_info = NCTRL(0) }, + { USB_DEVICE_INTERFACE_CLASS(TELIT_VENDOR_ID, 0x1035, 0xff) }, /* Telit LE910C4-WWX (ECM) */ { USB_DEVICE(TELIT_VENDOR_ID, TELIT_PRODUCT_LE922_USBCFG0), .driver_info = RSVD(0) | RSVD(1) | NCTRL(2) | RSVD(3) }, { USB_DEVICE(TELIT_VENDOR_ID, TELIT_PRODUCT_LE922_USBCFG1), @@ -2262,6 +2268,7 @@ static const struct usb_device_id option_ids[] = { { USB_DEVICE_INTERFACE_CLASS(0x305a, 0x1406, 0xff) }, /* GosunCn GM500 ECM/NCM */ { USB_DEVICE_AND_INTERFACE_INFO(OPPO_VENDOR_ID, OPPO_PRODUCT_R11, 0xff, 0xff, 0x30) }, { USB_DEVICE_AND_INTERFACE_INFO(SIERRA_VENDOR_ID, SIERRA_PRODUCT_EM9191, 0xff, 0xff, 0x30) }, + { USB_DEVICE_AND_INTERFACE_INFO(SIERRA_VENDOR_ID, SIERRA_PRODUCT_EM9191, 0xff, 0xff, 0x40) }, { USB_DEVICE_AND_INTERFACE_INFO(SIERRA_VENDOR_ID, SIERRA_PRODUCT_EM9191, 0xff, 0, 0) }, { USB_DEVICE_AND_INTERFACE_INFO(UNISOC_VENDOR_ID, TOZED_PRODUCT_LT70C, 0xff, 0, 0) }, { } /* Terminating entry */ diff --git a/drivers/usb/typec/altmodes/displayport.c b/drivers/usb/typec/altmodes/displayport.c index 426c88a516e5..59e0218a8bc5 100644 --- a/drivers/usb/typec/altmodes/displayport.c +++ b/drivers/usb/typec/altmodes/displayport.c @@ -304,6 +304,11 @@ static int dp_altmode_vdm(struct typec_altmode *alt, typec_altmode_update_active(alt, false); dp->data.status = 0; dp->data.conf = 0; + if (dp->hpd) { + drm_connector_oob_hotplug_event(dp->connector_fwnode); + dp->hpd = false; + sysfs_notify(&dp->alt->dev.kobj, "displayport", "hpd"); + } break; case DP_CMD_STATUS_UPDATE: dp->data.status = *vdo; diff --git a/drivers/usb/typec/tcpm/qcom/qcom_pmic_typec_pdphy.c b/drivers/usb/typec/tcpm/qcom/qcom_pmic_typec_pdphy.c index bb0b8479d80f..52c81378e36e 100644 --- a/drivers/usb/typec/tcpm/qcom/qcom_pmic_typec_pdphy.c +++ b/drivers/usb/typec/tcpm/qcom/qcom_pmic_typec_pdphy.c @@ -381,10 +381,6 @@ static int qcom_pmic_typec_pdphy_enable(struct pmic_typec_pdphy *pmic_typec_pdph struct device *dev = pmic_typec_pdphy->dev; int ret; - ret = regulator_enable(pmic_typec_pdphy->vdd_pdphy); - if (ret) - return ret; - /* PD 2.0, DR=TYPEC_DEVICE, PR=TYPEC_SINK */ ret = regmap_update_bits(pmic_typec_pdphy->regmap, pmic_typec_pdphy->base + USB_PDPHY_MSG_CONFIG_REG, @@ -422,8 +418,6 @@ static int qcom_pmic_typec_pdphy_disable(struct pmic_typec_pdphy *pmic_typec_pdp ret = regmap_write(pmic_typec_pdphy->regmap, pmic_typec_pdphy->base + USB_PDPHY_EN_CONTROL_REG, 0); - regulator_disable(pmic_typec_pdphy->vdd_pdphy); - return ret; } @@ -447,6 +441,10 @@ int qcom_pmic_typec_pdphy_start(struct pmic_typec_pdphy *pmic_typec_pdphy, int i; int ret; + ret = regulator_enable(pmic_typec_pdphy->vdd_pdphy); + if (ret) + return ret; + pmic_typec_pdphy->tcpm_port = tcpm_port; ret = pmic_typec_pdphy_reset(pmic_typec_pdphy); @@ -467,6 +465,8 @@ void qcom_pmic_typec_pdphy_stop(struct pmic_typec_pdphy *pmic_typec_pdphy) disable_irq(pmic_typec_pdphy->irq_data[i].irq); qcom_pmic_typec_pdphy_reset_on(pmic_typec_pdphy); + + regulator_disable(pmic_typec_pdphy->vdd_pdphy); } struct pmic_typec_pdphy *qcom_pmic_typec_pdphy_alloc(struct device *dev) diff --git a/drivers/usb/typec/ucsi/debugfs.c b/drivers/usb/typec/ucsi/debugfs.c index 0c7bf88d4a7f..f67733cecfdf 100644 --- a/drivers/usb/typec/ucsi/debugfs.c +++ b/drivers/usb/typec/ucsi/debugfs.c @@ -84,6 +84,9 @@ void ucsi_debugfs_register(struct ucsi *ucsi) void ucsi_debugfs_unregister(struct ucsi *ucsi) { + if (IS_ERR_OR_NULL(ucsi) || !ucsi->debugfs) + return; + debugfs_remove_recursive(ucsi->debugfs->dentry); kfree(ucsi->debugfs); } diff --git a/drivers/usb/typec/ucsi/psy.c b/drivers/usb/typec/ucsi/psy.c index 384b42267f1f..b35c6e07911e 100644 --- a/drivers/usb/typec/ucsi/psy.c +++ b/drivers/usb/typec/ucsi/psy.c @@ -37,6 +37,15 @@ static int ucsi_psy_get_scope(struct ucsi_connector *con, struct device *dev = con->ucsi->dev; device_property_read_u8(dev, "scope", &scope); + if (scope == POWER_SUPPLY_SCOPE_UNKNOWN) { + u32 mask = UCSI_CAP_ATTR_POWER_AC_SUPPLY | + UCSI_CAP_ATTR_BATTERY_CHARGING; + + if (con->ucsi->cap.attributes & mask) + scope = POWER_SUPPLY_SCOPE_SYSTEM; + else + scope = POWER_SUPPLY_SCOPE_DEVICE; + } val->intval = scope; return 0; } diff --git a/drivers/usb/typec/ucsi/ucsi.c b/drivers/usb/typec/ucsi/ucsi.c index c6dfe3dff346..61b64558f96c 100644 --- a/drivers/usb/typec/ucsi/ucsi.c +++ b/drivers/usb/typec/ucsi/ucsi.c @@ -787,6 +787,7 @@ static void ucsi_unregister_partner(struct ucsi_connector *con) typec_set_mode(con->port, TYPEC_STATE_SAFE); + typec_partner_set_usb_power_delivery(con->partner, NULL); ucsi_unregister_partner_pdos(con); ucsi_unregister_altmodes(con, UCSI_RECIPIENT_SOP); typec_unregister_partner(con->partner); @@ -884,6 +885,7 @@ static void ucsi_handle_connector_change(struct work_struct *work) if (ret < 0) { dev_err(ucsi->dev, "%s: GET_CONNECTOR_STATUS failed (%d)\n", __func__, ret); + clear_bit(EVENT_PENDING, &con->ucsi->flags); goto out_unlock; } diff --git a/drivers/vfio/mdev/mdev_sysfs.c b/drivers/vfio/mdev/mdev_sysfs.c index e4490639d383..9d2738e10c0b 100644 --- a/drivers/vfio/mdev/mdev_sysfs.c +++ b/drivers/vfio/mdev/mdev_sysfs.c @@ -233,7 +233,8 @@ int parent_create_sysfs_files(struct mdev_parent *parent) out_err: while (--i >= 0) mdev_type_remove(parent->types[i]); - return 0; + kset_unregister(parent->mdev_types_kset); + return ret; } static ssize_t remove_store(struct device *dev, struct device_attribute *attr, diff --git a/drivers/vfio/pci/pds/Kconfig b/drivers/vfio/pci/pds/Kconfig index 407b3fd32733..6eceef7b028a 100644 --- a/drivers/vfio/pci/pds/Kconfig +++ b/drivers/vfio/pci/pds/Kconfig @@ -3,7 +3,7 @@ config PDS_VFIO_PCI tristate "VFIO support for PDS PCI devices" - depends on PDS_CORE + depends on PDS_CORE && PCI_IOV select VFIO_PCI_CORE help This provides generic PCI support for PDS devices using the VFIO diff --git a/drivers/vfio/pci/pds/vfio_dev.c b/drivers/vfio/pci/pds/vfio_dev.c index b46174f5eb09..649b18ee394b 100644 --- a/drivers/vfio/pci/pds/vfio_dev.c +++ b/drivers/vfio/pci/pds/vfio_dev.c @@ -162,7 +162,7 @@ static int pds_vfio_init_device(struct vfio_device *vdev) pci_id = PCI_DEVID(pdev->bus->number, pdev->devfn); dev_dbg(&pdev->dev, "%s: PF %#04x VF %#04x vf_id %d domain %d pds_vfio %p\n", - __func__, pci_dev_id(pdev->physfn), pci_id, vf_id, + __func__, pci_dev_id(pci_physfn(pdev)), pci_id, vf_id, pci_domain_nr(pdev->bus), pds_vfio); return 0; diff --git a/drivers/vhost/vringh.c b/drivers/vhost/vringh.c index 955d938eb663..7b8fd977f71c 100644 --- a/drivers/vhost/vringh.c +++ b/drivers/vhost/vringh.c @@ -123,8 +123,18 @@ static inline ssize_t vringh_iov_xfer(struct vringh *vrh, done += partlen; len -= partlen; ptr += partlen; + iov->consumed += partlen; + iov->iov[iov->i].iov_len -= partlen; + iov->iov[iov->i].iov_base += partlen; - vringh_kiov_advance(iov, partlen); + if (!iov->iov[iov->i].iov_len) { + /* Fix up old iov element then increment. */ + iov->iov[iov->i].iov_len = iov->consumed; + iov->iov[iov->i].iov_base -= iov->consumed; + + iov->consumed = 0; + iov->i++; + } } return done; } diff --git a/drivers/video/console/Kconfig b/drivers/video/console/Kconfig index 1b5a319971ed..30577b1d3de5 100644 --- a/drivers/video/console/Kconfig +++ b/drivers/video/console/Kconfig @@ -73,6 +73,7 @@ config DUMMY_CONSOLE_ROWS config FRAMEBUFFER_CONSOLE bool "Framebuffer Console support" depends on FB_CORE && !UML + default DRM_FBDEV_EMULATION select VT_HW_CONSOLE_BINDING select CRC32 select FONT_SUPPORT diff --git a/drivers/video/fbdev/Kconfig b/drivers/video/fbdev/Kconfig index eac0ba39581e..c29754b65c0e 100644 --- a/drivers/video/fbdev/Kconfig +++ b/drivers/video/fbdev/Kconfig @@ -1762,7 +1762,7 @@ config FB_COBALT config FB_SH7760 bool "SH7760/SH7763/SH7720/SH7721 LCDC support" - depends on FB && (CPU_SUBTYPE_SH7760 || CPU_SUBTYPE_SH7763 \ + depends on FB=y && (CPU_SUBTYPE_SH7760 || CPU_SUBTYPE_SH7763 \ || CPU_SUBTYPE_SH7720 || CPU_SUBTYPE_SH7721) select FB_IOMEM_HELPERS help diff --git a/drivers/video/fbdev/aty/atyfb_base.c b/drivers/video/fbdev/aty/atyfb_base.c index 5c87817a4f4c..3dcf83f5e7b4 100644 --- a/drivers/video/fbdev/aty/atyfb_base.c +++ b/drivers/video/fbdev/aty/atyfb_base.c @@ -3440,11 +3440,15 @@ static int atyfb_setup_generic(struct pci_dev *pdev, struct fb_info *info, } info->fix.mmio_start = raddr; +#if defined(__i386__) || defined(__ia64__) /* * By using strong UC we force the MTRR to never have an * effect on the MMIO region on both non-PAT and PAT systems. */ par->ati_regbase = ioremap_uc(info->fix.mmio_start, 0x1000); +#else + par->ati_regbase = ioremap(info->fix.mmio_start, 0x1000); +#endif if (par->ati_regbase == NULL) return -ENOMEM; diff --git a/drivers/video/fbdev/core/Kconfig b/drivers/video/fbdev/core/Kconfig index baf7e852c75b..5ac1b0637531 100644 --- a/drivers/video/fbdev/core/Kconfig +++ b/drivers/video/fbdev/core/Kconfig @@ -28,7 +28,7 @@ config FIRMWARE_EDID config FB_DEVICE bool "Provide legacy /dev/fb* device" depends on FB_CORE - default y + default FB help Say Y here if you want the legacy /dev/fb* device file and interfaces within sysfs anc procfs. It is only required if you diff --git a/drivers/video/fbdev/core/cfbcopyarea.c b/drivers/video/fbdev/core/cfbcopyarea.c index 6d4bfeecee35..5b80bf3dae50 100644 --- a/drivers/video/fbdev/core/cfbcopyarea.c +++ b/drivers/video/fbdev/core/cfbcopyarea.c @@ -382,7 +382,7 @@ void cfb_copyarea(struct fb_info *p, const struct fb_copyarea *area) { u32 dx = area->dx, dy = area->dy, sx = area->sx, sy = area->sy; u32 height = area->height, width = area->width; - unsigned long const bits_per_line = p->fix.line_length*8u; + unsigned int const bits_per_line = p->fix.line_length * 8u; unsigned long __iomem *base = NULL; int bits = BITS_PER_LONG, bytes = bits >> 3; unsigned dst_idx = 0, src_idx = 0, rev_copy = 0; diff --git a/drivers/video/fbdev/core/syscopyarea.c b/drivers/video/fbdev/core/syscopyarea.c index c1eda3190968..7b8bd3a2bedc 100644 --- a/drivers/video/fbdev/core/syscopyarea.c +++ b/drivers/video/fbdev/core/syscopyarea.c @@ -316,7 +316,7 @@ void sys_copyarea(struct fb_info *p, const struct fb_copyarea *area) { u32 dx = area->dx, dy = area->dy, sx = area->sx, sy = area->sy; u32 height = area->height, width = area->width; - unsigned long const bits_per_line = p->fix.line_length*8u; + unsigned int const bits_per_line = p->fix.line_length * 8u; unsigned long *base = NULL; int bits = BITS_PER_LONG, bytes = bits >> 3; unsigned dst_idx = 0, src_idx = 0, rev_copy = 0; diff --git a/drivers/video/fbdev/mmp/hw/mmp_ctrl.h b/drivers/video/fbdev/mmp/hw/mmp_ctrl.h index 167585a889d3..719b99a9bc77 100644 --- a/drivers/video/fbdev/mmp/hw/mmp_ctrl.h +++ b/drivers/video/fbdev/mmp/hw/mmp_ctrl.h @@ -1406,7 +1406,7 @@ struct mmphw_ctrl { /*pathes*/ int path_num; - struct mmphw_path_plat path_plats[]; + struct mmphw_path_plat path_plats[] __counted_by(path_num); }; static inline int overlay_is_vid(struct mmp_overlay *overlay) diff --git a/drivers/video/fbdev/omap/omapfb_main.c b/drivers/video/fbdev/omap/omapfb_main.c index f28cb90947a3..42c96f1cfc93 100644 --- a/drivers/video/fbdev/omap/omapfb_main.c +++ b/drivers/video/fbdev/omap/omapfb_main.c @@ -1645,13 +1645,13 @@ static int omapfb_do_probe(struct platform_device *pdev, } fbdev->int_irq = platform_get_irq(pdev, 0); if (fbdev->int_irq < 0) { - r = ENXIO; + r = -ENXIO; goto cleanup; } fbdev->ext_irq = platform_get_irq(pdev, 1); if (fbdev->ext_irq < 0) { - r = ENXIO; + r = -ENXIO; goto cleanup; } diff --git a/drivers/video/fbdev/sa1100fb.c b/drivers/video/fbdev/sa1100fb.c index 3d76ce111488..cf0f706762b4 100644 --- a/drivers/video/fbdev/sa1100fb.c +++ b/drivers/video/fbdev/sa1100fb.c @@ -1214,7 +1214,7 @@ static struct platform_driver sa1100fb_driver = { }, }; -int __init sa1100fb_init(void) +static int __init sa1100fb_init(void) { if (fb_get_options("sa1100fb", NULL)) return -ENODEV; diff --git a/drivers/video/fbdev/uvesafb.c b/drivers/video/fbdev/uvesafb.c index a1a67830fbbc..e1f421e91b4f 100644 --- a/drivers/video/fbdev/uvesafb.c +++ b/drivers/video/fbdev/uvesafb.c @@ -1928,10 +1928,10 @@ static void uvesafb_exit(void) } } - cn_del_callback(&uvesafb_cn_id); driver_remove_file(&uvesafb_driver.driver, &driver_attr_v86d); platform_device_unregister(uvesafb_device); platform_driver_unregister(&uvesafb_driver); + cn_del_callback(&uvesafb_cn_id); } module_exit(uvesafb_exit); diff --git a/drivers/w1/masters/ds2482.c b/drivers/w1/masters/ds2482.c index c1de8a92e144..b2d76c1784bd 100644 --- a/drivers/w1/masters/ds2482.c +++ b/drivers/w1/masters/ds2482.c @@ -551,7 +551,7 @@ static struct i2c_driver ds2482_driver = { .driver = { .name = "ds2482", }, - .probe_new = ds2482_probe, + .probe = ds2482_probe, .remove = ds2482_remove, .id_table = ds2482_id, }; diff --git a/drivers/xen/events/events_base.c b/drivers/xen/events/events_base.c index 3bdd5b59661d..1b2136fe0fa5 100644 --- a/drivers/xen/events/events_base.c +++ b/drivers/xen/events/events_base.c @@ -33,6 +33,7 @@ #include #include #include +#include #include #include #include @@ -96,6 +97,7 @@ enum xen_irq_type { struct irq_info { struct list_head list; struct list_head eoi_list; + struct rcu_work rwork; short refcnt; u8 spurious_cnt; u8 is_accounted; @@ -146,23 +148,13 @@ const struct evtchn_ops *evtchn_ops; */ static DEFINE_MUTEX(irq_mapping_update_lock); -/* - * Lock protecting event handling loop against removing event channels. - * Adding of event channels is no issue as the associated IRQ becomes active - * only after everything is setup (before request_[threaded_]irq() the handler - * can't be entered for an event, as the event channel will be unmasked only - * then). - */ -static DEFINE_RWLOCK(evtchn_rwlock); - /* * Lock hierarchy: * * irq_mapping_update_lock - * evtchn_rwlock - * IRQ-desc lock - * percpu eoi_list_lock - * irq_info->lock + * IRQ-desc lock + * percpu eoi_list_lock + * irq_info->lock */ static LIST_HEAD(xen_irq_list_head); @@ -306,6 +298,22 @@ static void channels_on_cpu_inc(struct irq_info *info) info->is_accounted = 1; } +static void delayed_free_irq(struct work_struct *work) +{ + struct irq_info *info = container_of(to_rcu_work(work), struct irq_info, + rwork); + unsigned int irq = info->irq; + + /* Remove the info pointer only now, with no potential users left. */ + set_info_for_irq(irq, NULL); + + kfree(info); + + /* Legacy IRQ descriptors are managed by the arch. */ + if (irq >= nr_legacy_irqs()) + irq_free_desc(irq); +} + /* Constructors for packed IRQ information. */ static int xen_irq_info_common_setup(struct irq_info *info, unsigned irq, @@ -668,33 +676,36 @@ static void xen_irq_lateeoi_worker(struct work_struct *work) eoi = container_of(to_delayed_work(work), struct lateeoi_work, delayed); - read_lock_irqsave(&evtchn_rwlock, flags); + rcu_read_lock(); while (true) { - spin_lock(&eoi->eoi_list_lock); + spin_lock_irqsave(&eoi->eoi_list_lock, flags); info = list_first_entry_or_null(&eoi->eoi_list, struct irq_info, eoi_list); - if (info == NULL || now < info->eoi_time) { - spin_unlock(&eoi->eoi_list_lock); + if (info == NULL) + break; + + if (now < info->eoi_time) { + mod_delayed_work_on(info->eoi_cpu, system_wq, + &eoi->delayed, + info->eoi_time - now); break; } list_del_init(&info->eoi_list); - spin_unlock(&eoi->eoi_list_lock); + spin_unlock_irqrestore(&eoi->eoi_list_lock, flags); info->eoi_time = 0; xen_irq_lateeoi_locked(info, false); } - if (info) - mod_delayed_work_on(info->eoi_cpu, system_wq, - &eoi->delayed, info->eoi_time - now); + spin_unlock_irqrestore(&eoi->eoi_list_lock, flags); - read_unlock_irqrestore(&evtchn_rwlock, flags); + rcu_read_unlock(); } static void xen_cpu_init_eoi(unsigned int cpu) @@ -709,16 +720,15 @@ static void xen_cpu_init_eoi(unsigned int cpu) void xen_irq_lateeoi(unsigned int irq, unsigned int eoi_flags) { struct irq_info *info; - unsigned long flags; - read_lock_irqsave(&evtchn_rwlock, flags); + rcu_read_lock(); info = info_for_irq(irq); if (info) xen_irq_lateeoi_locked(info, eoi_flags & XEN_EOI_FLAG_SPURIOUS); - read_unlock_irqrestore(&evtchn_rwlock, flags); + rcu_read_unlock(); } EXPORT_SYMBOL_GPL(xen_irq_lateeoi); @@ -732,6 +742,7 @@ static void xen_irq_init(unsigned irq) info->type = IRQT_UNBOUND; info->refcnt = -1; + INIT_RCU_WORK(&info->rwork, delayed_free_irq); set_info_for_irq(irq, info); /* @@ -789,31 +800,18 @@ static int __must_check xen_allocate_irq_gsi(unsigned gsi) static void xen_free_irq(unsigned irq) { struct irq_info *info = info_for_irq(irq); - unsigned long flags; if (WARN_ON(!info)) return; - write_lock_irqsave(&evtchn_rwlock, flags); - if (!list_empty(&info->eoi_list)) lateeoi_list_del(info); list_del(&info->list); - set_info_for_irq(irq, NULL); - WARN_ON(info->refcnt > 0); - write_unlock_irqrestore(&evtchn_rwlock, flags); - - kfree(info); - - /* Legacy IRQ descriptors are managed by the arch. */ - if (irq < nr_legacy_irqs()) - return; - - irq_free_desc(irq); + queue_rcu_work(system_wq, &info->rwork); } /* Not called for lateeoi events. */ @@ -1704,14 +1702,21 @@ void handle_irq_for_port(evtchn_port_t port, struct evtchn_loop_ctrl *ctrl) generic_handle_irq(irq); } -static int __xen_evtchn_do_upcall(void) +int xen_evtchn_do_upcall(void) { struct vcpu_info *vcpu_info = __this_cpu_read(xen_vcpu); int ret = vcpu_info->evtchn_upcall_pending ? IRQ_HANDLED : IRQ_NONE; int cpu = smp_processor_id(); struct evtchn_loop_ctrl ctrl = { 0 }; - read_lock(&evtchn_rwlock); + /* + * When closing an event channel the associated IRQ must not be freed + * until all cpus have left the event handling loop. This is ensured + * by taking the rcu_read_lock() while handling events, as freeing of + * the IRQ is handled via queue_rcu_work() _after_ closing the event + * channel. + */ + rcu_read_lock(); do { vcpu_info->evtchn_upcall_pending = 0; @@ -1724,7 +1729,7 @@ static int __xen_evtchn_do_upcall(void) } while (vcpu_info->evtchn_upcall_pending); - read_unlock(&evtchn_rwlock); + rcu_read_unlock(); /* * Increment irq_epoch only now to defer EOIs only for @@ -1735,24 +1740,7 @@ static int __xen_evtchn_do_upcall(void) return ret; } - -void xen_evtchn_do_upcall(struct pt_regs *regs) -{ - struct pt_regs *old_regs = set_irq_regs(regs); - - irq_enter(); - - __xen_evtchn_do_upcall(); - - irq_exit(); - set_irq_regs(old_regs); -} - -int xen_hvm_evtchn_do_upcall(void) -{ - return __xen_evtchn_do_upcall(); -} -EXPORT_SYMBOL_GPL(xen_hvm_evtchn_do_upcall); +EXPORT_SYMBOL_GPL(xen_evtchn_do_upcall); /* Rebind a new event channel to an existing irq. */ void rebind_evtchn_irq(evtchn_port_t evtchn, int irq) diff --git a/drivers/xen/platform-pci.c b/drivers/xen/platform-pci.c index fcc819131572..544d3f9010b9 100644 --- a/drivers/xen/platform-pci.c +++ b/drivers/xen/platform-pci.c @@ -64,7 +64,7 @@ static uint64_t get_callback_via(struct pci_dev *pdev) static irqreturn_t do_hvm_evtchn_intr(int irq, void *dev_id) { - return xen_hvm_evtchn_do_upcall(); + return xen_evtchn_do_upcall(); } static int xen_allocate_irq(struct pci_dev *pdev) diff --git a/fs/aio.c b/fs/aio.c index a4c2a6bac72c..f8589caef9c1 100644 --- a/fs/aio.c +++ b/fs/aio.c @@ -80,7 +80,7 @@ struct aio_ring { struct kioctx_table { struct rcu_head rcu; unsigned nr; - struct kioctx __rcu *table[]; + struct kioctx __rcu *table[] __counted_by(nr); }; struct kioctx_cpu { diff --git a/fs/binfmt_elf_fdpic.c b/fs/binfmt_elf_fdpic.c index 43b2a2851ba3..206812ce544a 100644 --- a/fs/binfmt_elf_fdpic.c +++ b/fs/binfmt_elf_fdpic.c @@ -345,10 +345,9 @@ static int load_elf_fdpic_binary(struct linux_binprm *bprm) /* there's now no turning back... the old userspace image is dead, * defunct, deceased, etc. */ + SET_PERSONALITY(exec_params.hdr); if (elf_check_fdpic(&exec_params.hdr)) - set_personality(PER_LINUX_FDPIC); - else - set_personality(PER_LINUX); + current->personality |= PER_LINUX_FDPIC; if (elf_read_implies_exec(&exec_params.hdr, executable_stack)) current->personality |= READ_IMPLIES_EXEC; diff --git a/fs/btrfs/Kconfig b/fs/btrfs/Kconfig index 3282adc84d52..a25c9910d90b 100644 --- a/fs/btrfs/Kconfig +++ b/fs/btrfs/Kconfig @@ -31,7 +31,7 @@ config BTRFS_FS continue to be mountable and usable by newer kernels. For more information, please see the web pages at - http://btrfs.wiki.kernel.org. + https://btrfs.readthedocs.io To compile this file system support as a module, choose M here. The module will be called btrfs. diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c index 0cb1dee965a0..b2e5107b7cec 100644 --- a/fs/btrfs/block-group.c +++ b/fs/btrfs/block-group.c @@ -3028,8 +3028,16 @@ static int update_block_group_item(struct btrfs_trans_handle *trans, btrfs_mark_buffer_dirty(leaf); fail: btrfs_release_path(path); - /* We didn't update the block group item, need to revert @commit_used. */ - if (ret < 0) { + /* + * We didn't update the block group item, need to revert commit_used + * unless the block group item didn't exist yet - this is to prevent a + * race with a concurrent insertion of the block group item, with + * insert_block_group_item(), that happened just after we attempted to + * update. In that case we would reset commit_used to 0 just after the + * insertion set it to a value greater than 0 - if the block group later + * becomes with 0 used bytes, we would incorrectly skip its update. + */ + if (ret < 0 && ret != -ENOENT) { spin_lock(&cache->lock); cache->commit_used = old_commit_used; spin_unlock(&cache->lock); diff --git a/fs/btrfs/ctree.c b/fs/btrfs/ctree.c index a4cb4b642987..da519c1b6ad0 100644 --- a/fs/btrfs/ctree.c +++ b/fs/btrfs/ctree.c @@ -682,18 +682,30 @@ noinline int btrfs_cow_block(struct btrfs_trans_handle *trans, u64 search_start; int ret; - if (test_bit(BTRFS_ROOT_DELETING, &root->state)) - btrfs_err(fs_info, - "COW'ing blocks on a fs root that's being dropped"); - - if (trans->transaction != fs_info->running_transaction) - WARN(1, KERN_CRIT "trans %llu running %llu\n", - trans->transid, - fs_info->running_transaction->transid); + if (unlikely(test_bit(BTRFS_ROOT_DELETING, &root->state))) { + btrfs_abort_transaction(trans, -EUCLEAN); + btrfs_crit(fs_info, + "attempt to COW block %llu on root %llu that is being deleted", + buf->start, btrfs_root_id(root)); + return -EUCLEAN; + } - if (trans->transid != fs_info->generation) - WARN(1, KERN_CRIT "trans %llu running %llu\n", - trans->transid, fs_info->generation); + /* + * COWing must happen through a running transaction, which always + * matches the current fs generation (it's a transaction with a state + * less than TRANS_STATE_UNBLOCKED). If it doesn't, then turn the fs + * into error state to prevent the commit of any transaction. + */ + if (unlikely(trans->transaction != fs_info->running_transaction || + trans->transid != fs_info->generation)) { + btrfs_abort_transaction(trans, -EUCLEAN); + btrfs_crit(fs_info, +"unexpected transaction when attempting to COW block %llu on root %llu, transaction %llu running transaction %llu fs generation %llu", + buf->start, btrfs_root_id(root), trans->transid, + fs_info->running_transaction->transid, + fs_info->generation); + return -EUCLEAN; + } if (!should_cow_block(trans, root, buf)) { *cow_ret = buf; @@ -805,8 +817,22 @@ int btrfs_realloc_node(struct btrfs_trans_handle *trans, int progress_passed = 0; struct btrfs_disk_key disk_key; - WARN_ON(trans->transaction != fs_info->running_transaction); - WARN_ON(trans->transid != fs_info->generation); + /* + * COWing must happen through a running transaction, which always + * matches the current fs generation (it's a transaction with a state + * less than TRANS_STATE_UNBLOCKED). If it doesn't, then turn the fs + * into error state to prevent the commit of any transaction. + */ + if (unlikely(trans->transaction != fs_info->running_transaction || + trans->transid != fs_info->generation)) { + btrfs_abort_transaction(trans, -EUCLEAN); + btrfs_crit(fs_info, +"unexpected transaction when attempting to reallocate parent %llu for root %llu, transaction %llu running transaction %llu fs generation %llu", + parent->start, btrfs_root_id(root), trans->transid, + fs_info->running_transaction->transid, + fs_info->generation); + return -EUCLEAN; + } parent_nritems = btrfs_header_nritems(parent); blocksize = fs_info->nodesize; diff --git a/fs/btrfs/delayed-inode.c b/fs/btrfs/delayed-inode.c index 53c1211dd60b..90aaedce1548 100644 --- a/fs/btrfs/delayed-inode.c +++ b/fs/btrfs/delayed-inode.c @@ -313,7 +313,7 @@ static struct btrfs_delayed_item *btrfs_alloc_delayed_item(u16 data_len, { struct btrfs_delayed_item *item; - item = kmalloc(sizeof(*item) + data_len, GFP_NOFS); + item = kmalloc(struct_size(item, data, data_len), GFP_NOFS); if (item) { item->data_len = data_len; item->type = type; @@ -412,6 +412,7 @@ static void finish_one_item(struct btrfs_delayed_root *delayed_root) static void __btrfs_remove_delayed_item(struct btrfs_delayed_item *delayed_item) { + struct btrfs_delayed_node *delayed_node = delayed_item->delayed_node; struct rb_root_cached *root; struct btrfs_delayed_root *delayed_root; @@ -419,18 +420,21 @@ static void __btrfs_remove_delayed_item(struct btrfs_delayed_item *delayed_item) if (RB_EMPTY_NODE(&delayed_item->rb_node)) return; - delayed_root = delayed_item->delayed_node->root->fs_info->delayed_root; + /* If it's in a rbtree, then we need to have delayed node locked. */ + lockdep_assert_held(&delayed_node->mutex); + + delayed_root = delayed_node->root->fs_info->delayed_root; BUG_ON(!delayed_root); if (delayed_item->type == BTRFS_DELAYED_INSERTION_ITEM) - root = &delayed_item->delayed_node->ins_root; + root = &delayed_node->ins_root; else - root = &delayed_item->delayed_node->del_root; + root = &delayed_node->del_root; rb_erase_cached(&delayed_item->rb_node, root); RB_CLEAR_NODE(&delayed_item->rb_node); - delayed_item->delayed_node->count--; + delayed_node->count--; finish_one_item(delayed_root); } @@ -1153,20 +1157,33 @@ static int __btrfs_run_delayed_items(struct btrfs_trans_handle *trans, int nr) ret = __btrfs_commit_inode_delayed_items(trans, path, curr_node); if (ret) { - btrfs_release_delayed_node(curr_node); - curr_node = NULL; btrfs_abort_transaction(trans, ret); break; } prev_node = curr_node; curr_node = btrfs_next_delayed_node(curr_node); + /* + * See the comment below about releasing path before releasing + * node. If the commit of delayed items was successful the path + * should always be released, but in case of an error, it may + * point to locked extent buffers (a leaf at the very least). + */ + ASSERT(path->nodes[0] == NULL); btrfs_release_delayed_node(prev_node); } + /* + * Release the path to avoid a potential deadlock and lockdep splat when + * releasing the delayed node, as that requires taking the delayed node's + * mutex. If another task starts running delayed items before we take + * the mutex, it will first lock the mutex and then it may try to lock + * the same btree path (leaf). + */ + btrfs_free_path(path); + if (curr_node) btrfs_release_delayed_node(curr_node); - btrfs_free_path(path); trans->block_rsv = block_rsv; return ret; @@ -1413,7 +1430,29 @@ void btrfs_balance_delayed_items(struct btrfs_fs_info *fs_info) btrfs_wq_run_delayed_node(delayed_root, fs_info, BTRFS_DELAYED_BATCH); } -/* Will return 0 or -ENOMEM */ +static void btrfs_release_dir_index_item_space(struct btrfs_trans_handle *trans) +{ + struct btrfs_fs_info *fs_info = trans->fs_info; + const u64 bytes = btrfs_calc_insert_metadata_size(fs_info, 1); + + if (test_bit(BTRFS_FS_LOG_RECOVERING, &fs_info->flags)) + return; + + /* + * Adding the new dir index item does not require touching another + * leaf, so we can release 1 unit of metadata that was previously + * reserved when starting the transaction. This applies only to + * the case where we had a transaction start and excludes the + * transaction join case (when replaying log trees). + */ + trace_btrfs_space_reservation(fs_info, "transaction", + trans->transid, bytes, 0); + btrfs_block_rsv_release(fs_info, trans->block_rsv, bytes, NULL); + ASSERT(trans->bytes_reserved >= bytes); + trans->bytes_reserved -= bytes; +} + +/* Will return 0, -ENOMEM or -EEXIST (index number collision, unexpected). */ int btrfs_insert_delayed_dir_index(struct btrfs_trans_handle *trans, const char *name, int name_len, struct btrfs_inode *dir, @@ -1455,6 +1494,27 @@ int btrfs_insert_delayed_dir_index(struct btrfs_trans_handle *trans, mutex_lock(&delayed_node->mutex); + /* + * First attempt to insert the delayed item. This is to make the error + * handling path simpler in case we fail (-EEXIST). There's no risk of + * any other task coming in and running the delayed item before we do + * the metadata space reservation below, because we are holding the + * delayed node's mutex and that mutex must also be locked before the + * node's delayed items can be run. + */ + ret = __btrfs_add_delayed_item(delayed_node, delayed_item); + if (unlikely(ret)) { + btrfs_err(trans->fs_info, +"error adding delayed dir index item, name: %.*s, index: %llu, root: %llu, dir: %llu, dir->index_cnt: %llu, delayed_node->index_cnt: %llu, error: %d", + name_len, name, index, btrfs_root_id(delayed_node->root), + delayed_node->inode_id, dir->index_cnt, + delayed_node->index_cnt, ret); + btrfs_release_delayed_item(delayed_item); + btrfs_release_dir_index_item_space(trans); + mutex_unlock(&delayed_node->mutex); + goto release_node; + } + if (delayed_node->index_item_leaves == 0 || delayed_node->curr_index_batch_size + data_len > leaf_data_size) { delayed_node->curr_index_batch_size = data_len; @@ -1472,36 +1532,14 @@ int btrfs_insert_delayed_dir_index(struct btrfs_trans_handle *trans, * impossible. */ if (WARN_ON(ret)) { - mutex_unlock(&delayed_node->mutex); btrfs_release_delayed_item(delayed_item); + mutex_unlock(&delayed_node->mutex); goto release_node; } delayed_node->index_item_leaves++; - } else if (!test_bit(BTRFS_FS_LOG_RECOVERING, &fs_info->flags)) { - const u64 bytes = btrfs_calc_insert_metadata_size(fs_info, 1); - - /* - * Adding the new dir index item does not require touching another - * leaf, so we can release 1 unit of metadata that was previously - * reserved when starting the transaction. This applies only to - * the case where we had a transaction start and excludes the - * transaction join case (when replaying log trees). - */ - trace_btrfs_space_reservation(fs_info, "transaction", - trans->transid, bytes, 0); - btrfs_block_rsv_release(fs_info, trans->block_rsv, bytes, NULL); - ASSERT(trans->bytes_reserved >= bytes); - trans->bytes_reserved -= bytes; - } - - ret = __btrfs_add_delayed_item(delayed_node, delayed_item); - if (unlikely(ret)) { - btrfs_err(trans->fs_info, - "err add delayed dir index item(name: %.*s) into the insertion tree of the delayed node(root id: %llu, inode id: %llu, errno: %d)", - name_len, name, delayed_node->root->root_key.objectid, - delayed_node->inode_id, ret); - BUG(); + } else { + btrfs_release_dir_index_item_space(trans); } mutex_unlock(&delayed_node->mutex); diff --git a/fs/btrfs/delayed-inode.h b/fs/btrfs/delayed-inode.h index dc1085b2a397..1da213197f55 100644 --- a/fs/btrfs/delayed-inode.h +++ b/fs/btrfs/delayed-inode.h @@ -95,7 +95,7 @@ struct btrfs_delayed_item { bool logged; /* The maximum leaf size is 64K, so u16 is more than enough. */ u16 data_len; - char data[]; + char data[] __counted_by(data_len); }; static inline void btrfs_init_delayed_root( diff --git a/fs/btrfs/delayed-ref.c b/fs/btrfs/delayed-ref.c index 6a13cf00218b..9fe4ccca50a0 100644 --- a/fs/btrfs/delayed-ref.c +++ b/fs/btrfs/delayed-ref.c @@ -103,24 +103,17 @@ void btrfs_update_delayed_refs_rsv(struct btrfs_trans_handle *trans) * Transfer bytes to our delayed refs rsv. * * @fs_info: the filesystem - * @src: source block rsv to transfer from * @num_bytes: number of bytes to transfer * - * This transfers up to the num_bytes amount from the src rsv to the + * This transfers up to the num_bytes amount, previously reserved, to the * delayed_refs_rsv. Any extra bytes are returned to the space info. */ void btrfs_migrate_to_delayed_refs_rsv(struct btrfs_fs_info *fs_info, - struct btrfs_block_rsv *src, u64 num_bytes) { struct btrfs_block_rsv *delayed_refs_rsv = &fs_info->delayed_refs_rsv; u64 to_free = 0; - spin_lock(&src->lock); - src->reserved -= num_bytes; - src->size -= num_bytes; - spin_unlock(&src->lock); - spin_lock(&delayed_refs_rsv->lock); if (delayed_refs_rsv->size > delayed_refs_rsv->reserved) { u64 delta = delayed_refs_rsv->size - @@ -163,6 +156,8 @@ int btrfs_delayed_refs_rsv_refill(struct btrfs_fs_info *fs_info, struct btrfs_block_rsv *block_rsv = &fs_info->delayed_refs_rsv; u64 limit = btrfs_calc_delayed_ref_bytes(fs_info, 1); u64 num_bytes = 0; + u64 refilled_bytes; + u64 to_free; int ret = -ENOSPC; spin_lock(&block_rsv->lock); @@ -178,9 +173,38 @@ int btrfs_delayed_refs_rsv_refill(struct btrfs_fs_info *fs_info, ret = btrfs_reserve_metadata_bytes(fs_info, block_rsv, num_bytes, flush); if (ret) return ret; - btrfs_block_rsv_add_bytes(block_rsv, num_bytes, false); - trace_btrfs_space_reservation(fs_info, "delayed_refs_rsv", - 0, num_bytes, 1); + + /* + * We may have raced with someone else, so check again if we the block + * reserve is still not full and release any excess space. + */ + spin_lock(&block_rsv->lock); + if (block_rsv->reserved < block_rsv->size) { + u64 needed = block_rsv->size - block_rsv->reserved; + + if (num_bytes >= needed) { + block_rsv->reserved += needed; + block_rsv->full = true; + to_free = num_bytes - needed; + refilled_bytes = needed; + } else { + block_rsv->reserved += num_bytes; + to_free = 0; + refilled_bytes = num_bytes; + } + } else { + to_free = num_bytes; + refilled_bytes = 0; + } + spin_unlock(&block_rsv->lock); + + if (to_free > 0) + btrfs_space_info_free_bytes_may_use(fs_info, block_rsv->space_info, + to_free); + + if (refilled_bytes > 0) + trace_btrfs_space_reservation(fs_info, "delayed_refs_rsv", 0, + refilled_bytes, 1); return 0; } diff --git a/fs/btrfs/delayed-ref.h b/fs/btrfs/delayed-ref.h index b8e14b0ba5f1..fd9bf2b709c0 100644 --- a/fs/btrfs/delayed-ref.h +++ b/fs/btrfs/delayed-ref.h @@ -407,7 +407,6 @@ void btrfs_update_delayed_refs_rsv(struct btrfs_trans_handle *trans); int btrfs_delayed_refs_rsv_refill(struct btrfs_fs_info *fs_info, enum btrfs_reserve_flush_enum flush); void btrfs_migrate_to_delayed_refs_rsv(struct btrfs_fs_info *fs_info, - struct btrfs_block_rsv *src, u64 num_bytes); bool btrfs_check_space_for_delayed_refs(struct btrfs_fs_info *fs_info); diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 0a96ea8c1d3a..68f60d50e1fd 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -520,6 +520,7 @@ static bool btree_dirty_folio(struct address_space *mapping, struct folio *folio) { struct btrfs_fs_info *fs_info = btrfs_sb(mapping->host->i_sb); + struct btrfs_subpage_info *spi = fs_info->subpage_info; struct btrfs_subpage *subpage; struct extent_buffer *eb; int cur_bit = 0; @@ -533,18 +534,19 @@ static bool btree_dirty_folio(struct address_space *mapping, btrfs_assert_tree_write_locked(eb); return filemap_dirty_folio(mapping, folio); } + + ASSERT(spi); subpage = folio_get_private(folio); - ASSERT(subpage->dirty_bitmap); - while (cur_bit < BTRFS_SUBPAGE_BITMAP_SIZE) { + for (cur_bit = spi->dirty_offset; + cur_bit < spi->dirty_offset + spi->bitmap_nr_bits; + cur_bit++) { unsigned long flags; u64 cur; - u16 tmp = (1 << cur_bit); spin_lock_irqsave(&subpage->lock, flags); - if (!(tmp & subpage->dirty_bitmap)) { + if (!test_bit(cur_bit, subpage->bitmaps)) { spin_unlock_irqrestore(&subpage->lock, flags); - cur_bit++; continue; } spin_unlock_irqrestore(&subpage->lock, flags); @@ -557,7 +559,7 @@ static bool btree_dirty_folio(struct address_space *mapping, btrfs_assert_tree_write_locked(eb); free_extent_buffer(eb); - cur_bit += (fs_info->nodesize >> fs_info->sectorsize_bits); + cur_bit += (fs_info->nodesize >> fs_info->sectorsize_bits) - 1; } return filemap_dirty_folio(mapping, folio); } @@ -1547,7 +1549,7 @@ static int transaction_kthread(void *arg) delta = ktime_get_seconds() - cur->start_time; if (!test_and_clear_bit(BTRFS_FS_COMMIT_TRANS, &fs_info->flags) && - cur->state < TRANS_STATE_COMMIT_START && + cur->state < TRANS_STATE_COMMIT_PREP && delta < fs_info->commit_interval) { spin_unlock(&fs_info->trans_lock); delay -= msecs_to_jiffies((delta - 1) * 1000); @@ -2682,8 +2684,8 @@ void btrfs_init_fs_info(struct btrfs_fs_info *fs_info) btrfs_lockdep_init_map(fs_info, btrfs_trans_num_extwriters); btrfs_lockdep_init_map(fs_info, btrfs_trans_pending_ordered); btrfs_lockdep_init_map(fs_info, btrfs_ordered_extent); - btrfs_state_lockdep_init_map(fs_info, btrfs_trans_commit_start, - BTRFS_LOCKDEP_TRANS_COMMIT_START); + btrfs_state_lockdep_init_map(fs_info, btrfs_trans_commit_prep, + BTRFS_LOCKDEP_TRANS_COMMIT_PREP); btrfs_state_lockdep_init_map(fs_info, btrfs_trans_unblocked, BTRFS_LOCKDEP_TRANS_UNBLOCKED); btrfs_state_lockdep_init_map(fs_info, btrfs_trans_super_committed, @@ -4870,7 +4872,7 @@ static int btrfs_cleanup_transaction(struct btrfs_fs_info *fs_info) while (!list_empty(&fs_info->trans_list)) { t = list_first_entry(&fs_info->trans_list, struct btrfs_transaction, list); - if (t->state >= TRANS_STATE_COMMIT_START) { + if (t->state >= TRANS_STATE_COMMIT_PREP) { refcount_inc(&t->use_count); spin_unlock(&fs_info->trans_lock); btrfs_wait_for_commit(fs_info, t->transid); diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index f356f08b55cb..fc313fce5bbd 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -1514,15 +1514,14 @@ static int __btrfs_inc_extent_ref(struct btrfs_trans_handle *trans, btrfs_release_path(path); /* now insert the actual backref */ - if (owner < BTRFS_FIRST_FREE_OBJECTID) { - BUG_ON(refs_to_add != 1); + if (owner < BTRFS_FIRST_FREE_OBJECTID) ret = insert_tree_block_ref(trans, path, bytenr, parent, root_objectid); - } else { + else ret = insert_extent_data_ref(trans, path, bytenr, parent, root_objectid, owner, offset, refs_to_add); - } + if (ret) btrfs_abort_transaction(trans, ret); out: @@ -1656,7 +1655,10 @@ again: goto again; } } else { - err = -EIO; + err = -EUCLEAN; + btrfs_err(fs_info, + "missing extent item for extent %llu num_bytes %llu level %d", + head->bytenr, head->num_bytes, extent_op->level); goto out; } } @@ -1699,12 +1701,12 @@ static int run_delayed_tree_ref(struct btrfs_trans_handle *trans, parent = ref->parent; ref_root = ref->root; - if (node->ref_mod != 1) { + if (unlikely(node->ref_mod != 1)) { btrfs_err(trans->fs_info, - "btree block(%llu) has %d references rather than 1: action %d ref_root %llu parent %llu", + "btree block %llu has %d references rather than 1: action %d ref_root %llu parent %llu", node->bytenr, node->ref_mod, node->action, ref_root, parent); - return -EIO; + return -EUCLEAN; } if (node->action == BTRFS_ADD_DELAYED_REF && insert_reserved) { BUG_ON(!extent_op || !extent_op->update_flags); diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index ac3fca5a5e41..caccd0376342 100644 --- a/fs/btrfs/extent_io.c +++ b/fs/btrfs/extent_io.c @@ -484,10 +484,8 @@ static void end_bio_extent_writepage(struct btrfs_bio *bbio) bvec->bv_offset, bvec->bv_len); btrfs_finish_ordered_extent(bbio->ordered, page, start, len, !error); - if (error) { - btrfs_page_clear_uptodate(fs_info, page, start, len); + if (error) mapping_set_error(page->mapping, error); - } btrfs_page_clear_writeback(fs_info, page, start, len); } @@ -1456,8 +1454,6 @@ done: if (ret) { btrfs_mark_ordered_io_finished(BTRFS_I(inode), page, page_start, PAGE_SIZE, !ret); - btrfs_page_clear_uptodate(btrfs_sb(inode->i_sb), page, - page_start, PAGE_SIZE); mapping_set_error(page->mapping, ret); } unlock_page(page); @@ -1624,8 +1620,6 @@ static void extent_buffer_write_end_io(struct btrfs_bio *bbio) struct page *page = bvec->bv_page; u32 len = bvec->bv_len; - if (!uptodate) - btrfs_page_clear_uptodate(fs_info, page, start, len); btrfs_page_clear_writeback(fs_info, page, start, len); bio_offset += len; } @@ -2201,7 +2195,6 @@ void extent_write_locked_range(struct inode *inode, struct page *locked_page, if (ret) { btrfs_mark_ordered_io_finished(BTRFS_I(inode), page, cur, cur_len, !ret); - btrfs_page_clear_uptodate(fs_info, page, cur, cur_len); mapping_set_error(page->mapping, ret); } btrfs_page_unlock_writer(fs_info, page, cur, cur_len); @@ -4002,8 +3995,14 @@ void read_extent_buffer(const struct extent_buffer *eb, void *dstv, char *dst = (char *)dstv; unsigned long i = get_eb_page_index(start); - if (check_eb_range(eb, start, len)) + if (check_eb_range(eb, start, len)) { + /* + * Invalid range hit, reset the memory, so callers won't get + * some random garbage for their uninitialzed memory. + */ + memset(dstv, 0, len); return; + } offset = get_eb_offset_in_page(eb, start); diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c index ca46a529d56b..361535c71c0f 100644 --- a/fs/btrfs/file.c +++ b/fs/btrfs/file.c @@ -1106,6 +1106,25 @@ void btrfs_check_nocow_unlock(struct btrfs_inode *inode) btrfs_drew_write_unlock(&inode->root->snapshot_lock); } +static void update_time_for_write(struct inode *inode) +{ + struct timespec64 now, ctime; + + if (IS_NOCMTIME(inode)) + return; + + now = current_time(inode); + if (!timespec64_equal(&inode->i_mtime, &now)) + inode->i_mtime = now; + + ctime = inode_get_ctime(inode); + if (!timespec64_equal(&ctime, &now)) + inode_set_ctime_to_ts(inode, now); + + if (IS_I_VERSION(inode)) + inode_inc_iversion(inode); +} + static int btrfs_write_check(struct kiocb *iocb, struct iov_iter *from, size_t count) { @@ -1137,10 +1156,7 @@ static int btrfs_write_check(struct kiocb *iocb, struct iov_iter *from, * need to start yet another transaction to update the inode as we will * update the inode when we finish writing whatever data we write. */ - if (!IS_NOCMTIME(inode)) { - inode->i_mtime = inode_set_ctime_current(inode); - inode_inc_iversion(inode); - } + update_time_for_write(inode); start_pos = round_down(pos, fs_info->sectorsize); oldsize = i_size_read(inode); @@ -1451,8 +1467,13 @@ static ssize_t btrfs_direct_write(struct kiocb *iocb, struct iov_iter *from) if (iocb->ki_flags & IOCB_NOWAIT) ilock_flags |= BTRFS_ILOCK_TRY; - /* If the write DIO is within EOF, use a shared lock */ - if (iocb->ki_pos + iov_iter_count(from) <= i_size_read(inode)) + /* + * If the write DIO is within EOF, use a shared lock and also only if + * security bits will likely not be dropped by file_remove_privs() called + * from btrfs_write_check(). Either will need to be rechecked after the + * lock was acquired. + */ + if (iocb->ki_pos + iov_iter_count(from) <= i_size_read(inode) && IS_NOSEC(inode)) ilock_flags |= BTRFS_ILOCK_SHARED; relock: @@ -1460,6 +1481,13 @@ relock: if (err < 0) return err; + /* Shared lock cannot be used with security bits set. */ + if ((ilock_flags & BTRFS_ILOCK_SHARED) && !IS_NOSEC(inode)) { + btrfs_inode_unlock(BTRFS_I(inode), ilock_flags); + ilock_flags &= ~BTRFS_ILOCK_SHARED; + goto relock; + } + err = generic_write_checks(iocb, from); if (err <= 0) { btrfs_inode_unlock(BTRFS_I(inode), ilock_flags); diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index f09fbdc43f0f..7814b9d654ce 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -1085,9 +1085,6 @@ static void submit_uncompressed_range(struct btrfs_inode *inode, btrfs_mark_ordered_io_finished(inode, locked_page, page_start, PAGE_SIZE, !ret); - btrfs_page_clear_uptodate(inode->root->fs_info, - locked_page, page_start, - PAGE_SIZE); mapping_set_error(locked_page->mapping, ret); unlock_page(locked_page); } @@ -2791,7 +2788,6 @@ out_page: mapping_set_error(page->mapping, ret); btrfs_mark_ordered_io_finished(inode, page, page_start, PAGE_SIZE, !ret); - btrfs_page_clear_uptodate(fs_info, page, page_start, PAGE_SIZE); clear_page_dirty_for_io(page); } btrfs_page_clear_checked(fs_info, page, page_start, PAGE_SIZE); @@ -5769,20 +5765,24 @@ out: static int btrfs_get_dir_last_index(struct btrfs_inode *dir, u64 *index) { - if (dir->index_cnt == (u64)-1) { - int ret; + int ret = 0; + btrfs_inode_lock(dir, 0); + if (dir->index_cnt == (u64)-1) { ret = btrfs_inode_delayed_dir_index_count(dir); if (ret) { ret = btrfs_set_inode_index_count(dir); if (ret) - return ret; + goto out; } } - *index = dir->index_cnt; + /* index_cnt is the index number of next new entry, so decrement it. */ + *index = dir->index_cnt - 1; +out: + btrfs_inode_unlock(dir, 0); - return 0; + return ret; } /* @@ -5817,6 +5817,19 @@ static int btrfs_opendir(struct inode *inode, struct file *file) return 0; } +static loff_t btrfs_dir_llseek(struct file *file, loff_t offset, int whence) +{ + struct btrfs_file_private *private = file->private_data; + int ret; + + ret = btrfs_get_dir_last_index(BTRFS_I(file_inode(file)), + &private->last_index); + if (ret) + return ret; + + return generic_file_llseek(file, offset, whence); +} + struct dir_entry { u64 ino; u64 offset; @@ -10868,7 +10881,7 @@ static const struct inode_operations btrfs_dir_inode_operations = { }; static const struct file_operations btrfs_dir_file_operations = { - .llseek = generic_file_llseek, + .llseek = btrfs_dir_llseek, .read = generic_read_dir, .iterate_shared = btrfs_real_readdir, .open = btrfs_opendir, diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c index a18ee7b5a166..8e7d03bc1b56 100644 --- a/fs/btrfs/ioctl.c +++ b/fs/btrfs/ioctl.c @@ -1958,6 +1958,13 @@ static int btrfs_search_path_in_tree_user(struct mnt_idmap *idmap, goto out_put; } + /* + * We don't need the path anymore, so release it and + * avoid deadlocks and lockdep warnings in case + * btrfs_iget() needs to lookup the inode from its root + * btree and lock the same leaf. + */ + btrfs_release_path(path); temp_inode = btrfs_iget(sb, key2.objectid, root); if (IS_ERR(temp_inode)) { ret = PTR_ERR(temp_inode); @@ -1978,7 +1985,6 @@ static int btrfs_search_path_in_tree_user(struct mnt_idmap *idmap, goto out_put; } - btrfs_release_path(path); key.objectid = key.offset; key.offset = (u64)-1; dirid = key.objectid; @@ -2972,7 +2978,7 @@ static void get_block_group_info(struct list_head *groups_list, static long btrfs_ioctl_space_info(struct btrfs_fs_info *fs_info, void __user *arg) { - struct btrfs_ioctl_space_args space_args; + struct btrfs_ioctl_space_args space_args = { 0 }; struct btrfs_ioctl_space_info space; struct btrfs_ioctl_space_info *dest; struct btrfs_ioctl_space_info *dest_orig; @@ -4332,7 +4338,7 @@ static int _btrfs_ioctl_send(struct inode *inode, void __user *argp, bool compat if (compat) { #if defined(CONFIG_64BIT) && defined(CONFIG_COMPAT) - struct btrfs_ioctl_send_args_32 args32; + struct btrfs_ioctl_send_args_32 args32 = { 0 }; ret = copy_from_user(&args32, argp, sizeof(args32)); if (ret) diff --git a/fs/btrfs/locking.h b/fs/btrfs/locking.h index edb9b4a0dba1..7d6ee1e609bf 100644 --- a/fs/btrfs/locking.h +++ b/fs/btrfs/locking.h @@ -79,7 +79,7 @@ enum btrfs_lock_nesting { }; enum btrfs_lockdep_trans_states { - BTRFS_LOCKDEP_TRANS_COMMIT_START, + BTRFS_LOCKDEP_TRANS_COMMIT_PREP, BTRFS_LOCKDEP_TRANS_UNBLOCKED, BTRFS_LOCKDEP_TRANS_SUPER_COMMITTED, BTRFS_LOCKDEP_TRANS_COMPLETED, diff --git a/fs/btrfs/ordered-data.c b/fs/btrfs/ordered-data.c index b46ab348e8e5..345c449d588c 100644 --- a/fs/btrfs/ordered-data.c +++ b/fs/btrfs/ordered-data.c @@ -639,7 +639,7 @@ void btrfs_remove_ordered_extent(struct btrfs_inode *btrfs_inode, refcount_inc(&trans->use_count); spin_unlock(&fs_info->trans_lock); - ASSERT(trans); + ASSERT(trans || BTRFS_FS_ERROR(fs_info)); if (trans) { if (atomic_dec_and_test(&trans->pending_ordered)) wake_up(&trans->pending_wait); diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c index 09bfe68d2ea3..1a093ec0f7e3 100644 --- a/fs/btrfs/super.c +++ b/fs/btrfs/super.c @@ -2117,7 +2117,7 @@ static int btrfs_statfs(struct dentry *dentry, struct kstatfs *buf) * calculated f_bavail. */ if (!mixed && block_rsv->space_info->full && - total_free_meta - thresh < block_rsv->size) + (total_free_meta < thresh || total_free_meta - thresh < block_rsv->size)) buf->f_bavail = 0; buf->f_type = BTRFS_SUPER_MAGIC; @@ -2150,7 +2150,7 @@ static struct file_system_type btrfs_fs_type = { .name = "btrfs", .mount = btrfs_mount, .kill_sb = btrfs_kill_super, - .fs_flags = FS_REQUIRES_DEV | FS_BINARY_MOUNTDATA | FS_MGTIME, + .fs_flags = FS_REQUIRES_DEV | FS_BINARY_MOUNTDATA, }; static struct file_system_type btrfs_root_fs_type = { @@ -2158,8 +2158,7 @@ static struct file_system_type btrfs_root_fs_type = { .name = "btrfs", .mount = btrfs_mount_root, .kill_sb = btrfs_kill_super, - .fs_flags = FS_REQUIRES_DEV | FS_BINARY_MOUNTDATA | - FS_ALLOW_IDMAP | FS_MGTIME, + .fs_flags = FS_REQUIRES_DEV | FS_BINARY_MOUNTDATA | FS_ALLOW_IDMAP, }; MODULE_ALIAS_FS("btrfs"); diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c index 874e4394df86..c780d3729463 100644 --- a/fs/btrfs/transaction.c +++ b/fs/btrfs/transaction.c @@ -56,12 +56,17 @@ static struct kmem_cache *btrfs_trans_handle_cachep; * | Call btrfs_commit_transaction() on any trans handle attached to * | transaction N * V - * Transaction N [[TRANS_STATE_COMMIT_START]] + * Transaction N [[TRANS_STATE_COMMIT_PREP]] + * | + * | If there are simultaneous calls to btrfs_commit_transaction() one will win + * | the race and the rest will wait for the winner to commit the transaction. + * | + * | The winner will wait for previous running transaction to completely finish + * | if there is one. * | - * | Will wait for previous running transaction to completely finish if there - * | is one + * Transaction N [[TRANS_STATE_COMMIT_START]] * | - * | Then one of the following happes: + * | Then one of the following happens: * | - Wait for all other trans handle holders to release. * | The btrfs_commit_transaction() caller will do the commit work. * | - Wait for current transaction to be committed by others. @@ -112,6 +117,7 @@ static struct kmem_cache *btrfs_trans_handle_cachep; */ static const unsigned int btrfs_blocked_trans_types[TRANS_STATE_MAX] = { [TRANS_STATE_RUNNING] = 0U, + [TRANS_STATE_COMMIT_PREP] = 0U, [TRANS_STATE_COMMIT_START] = (__TRANS_START | __TRANS_ATTACH), [TRANS_STATE_COMMIT_DOING] = (__TRANS_START | __TRANS_ATTACH | @@ -625,14 +631,14 @@ start_transaction(struct btrfs_root *root, unsigned int num_items, reloc_reserved = true; } - ret = btrfs_block_rsv_add(fs_info, rsv, num_bytes, flush); + ret = btrfs_reserve_metadata_bytes(fs_info, rsv, num_bytes, flush); if (ret) goto reserve_fail; if (delayed_refs_bytes) { - btrfs_migrate_to_delayed_refs_rsv(fs_info, rsv, - delayed_refs_bytes); + btrfs_migrate_to_delayed_refs_rsv(fs_info, delayed_refs_bytes); num_bytes -= delayed_refs_bytes; } + btrfs_block_rsv_add_bytes(rsv, num_bytes, true); if (rsv->space_info->force_alloc) do_chunk_alloc = true; @@ -1982,7 +1988,7 @@ void btrfs_commit_transaction_async(struct btrfs_trans_handle *trans) * Wait for the current transaction commit to start and block * subsequent transaction joins */ - btrfs_might_wait_for_state(fs_info, BTRFS_LOCKDEP_TRANS_COMMIT_START); + btrfs_might_wait_for_state(fs_info, BTRFS_LOCKDEP_TRANS_COMMIT_PREP); wait_event(fs_info->transaction_blocked_wait, cur_trans->state >= TRANS_STATE_COMMIT_START || TRANS_ABORTED(cur_trans)); @@ -2129,7 +2135,7 @@ static void add_pending_snapshot(struct btrfs_trans_handle *trans) return; lockdep_assert_held(&trans->fs_info->trans_lock); - ASSERT(cur_trans->state >= TRANS_STATE_COMMIT_START); + ASSERT(cur_trans->state >= TRANS_STATE_COMMIT_PREP); list_add(&trans->pending_snapshot->list, &cur_trans->pending_snapshots); } @@ -2153,7 +2159,7 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans) ktime_t interval; ASSERT(refcount_read(&trans->use_count) == 1); - btrfs_trans_state_lockdep_acquire(fs_info, BTRFS_LOCKDEP_TRANS_COMMIT_START); + btrfs_trans_state_lockdep_acquire(fs_info, BTRFS_LOCKDEP_TRANS_COMMIT_PREP); clear_bit(BTRFS_FS_NEED_TRANS_COMMIT, &fs_info->flags); @@ -2213,7 +2219,7 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans) } spin_lock(&fs_info->trans_lock); - if (cur_trans->state >= TRANS_STATE_COMMIT_START) { + if (cur_trans->state >= TRANS_STATE_COMMIT_PREP) { enum btrfs_trans_state want_state = TRANS_STATE_COMPLETED; add_pending_snapshot(trans); @@ -2225,7 +2231,7 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans) want_state = TRANS_STATE_SUPER_COMMITTED; btrfs_trans_state_lockdep_release(fs_info, - BTRFS_LOCKDEP_TRANS_COMMIT_START); + BTRFS_LOCKDEP_TRANS_COMMIT_PREP); ret = btrfs_end_transaction(trans); wait_for_commit(cur_trans, want_state); @@ -2237,9 +2243,9 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans) return ret; } - cur_trans->state = TRANS_STATE_COMMIT_START; + cur_trans->state = TRANS_STATE_COMMIT_PREP; wake_up(&fs_info->transaction_blocked_wait); - btrfs_trans_state_lockdep_release(fs_info, BTRFS_LOCKDEP_TRANS_COMMIT_START); + btrfs_trans_state_lockdep_release(fs_info, BTRFS_LOCKDEP_TRANS_COMMIT_PREP); if (cur_trans->list.prev != &fs_info->trans_list) { enum btrfs_trans_state want_state = TRANS_STATE_COMPLETED; @@ -2260,11 +2266,9 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans) btrfs_put_transaction(prev_trans); if (ret) goto lockdep_release; - } else { - spin_unlock(&fs_info->trans_lock); + spin_lock(&fs_info->trans_lock); } } else { - spin_unlock(&fs_info->trans_lock); /* * The previous transaction was aborted and was already removed * from the list of transactions at fs_info->trans_list. So we @@ -2272,11 +2276,16 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans) * corrupt state (pointing to trees with unwritten nodes/leafs). */ if (BTRFS_FS_ERROR(fs_info)) { + spin_unlock(&fs_info->trans_lock); ret = -EROFS; goto lockdep_release; } } + cur_trans->state = TRANS_STATE_COMMIT_START; + wake_up(&fs_info->transaction_blocked_wait); + spin_unlock(&fs_info->trans_lock); + /* * Get the time spent on the work done by the commit thread and not * the time spent waiting on a previous commit @@ -2586,7 +2595,7 @@ lockdep_release: goto cleanup_transaction; lockdep_trans_commit_start_release: - btrfs_trans_state_lockdep_release(fs_info, BTRFS_LOCKDEP_TRANS_COMMIT_START); + btrfs_trans_state_lockdep_release(fs_info, BTRFS_LOCKDEP_TRANS_COMMIT_PREP); btrfs_end_transaction(trans); return ret; } diff --git a/fs/btrfs/transaction.h b/fs/btrfs/transaction.h index 8e9fa23bd7fe..93869cda6af9 100644 --- a/fs/btrfs/transaction.h +++ b/fs/btrfs/transaction.h @@ -14,6 +14,7 @@ enum btrfs_trans_state { TRANS_STATE_RUNNING, + TRANS_STATE_COMMIT_PREP, TRANS_STATE_COMMIT_START, TRANS_STATE_COMMIT_DOING, TRANS_STATE_UNBLOCKED, @@ -218,8 +219,8 @@ do { \ (errno))) { \ /* Stack trace printed. */ \ } else { \ - btrfs_debug((trans)->fs_info, \ - "Transaction aborted (error %d)", \ + btrfs_err((trans)->fs_info, \ + "Transaction aborted (error %d)", \ (errno)); \ } \ } \ diff --git a/fs/btrfs/tree-log.c b/fs/btrfs/tree-log.c index d1e46b839519..cbb17b542131 100644 --- a/fs/btrfs/tree-log.c +++ b/fs/btrfs/tree-log.c @@ -4722,7 +4722,7 @@ static int btrfs_log_prealloc_extents(struct btrfs_trans_handle *trans, struct extent_buffer *leaf; int slot; int ins_nr = 0; - int start_slot; + int start_slot = 0; int ret; if (!(inode->flags & BTRFS_INODE_PREALLOC)) diff --git a/fs/btrfs/verity.c b/fs/btrfs/verity.c index c5ff16f9e9fa..744f4f4d4c68 100644 --- a/fs/btrfs/verity.c +++ b/fs/btrfs/verity.c @@ -715,7 +715,7 @@ static struct page *btrfs_read_merkle_tree_page(struct inode *inode, pgoff_t index, unsigned long num_ra_pages) { - struct page *page; + struct folio *folio; u64 off = (u64)index << PAGE_SHIFT; loff_t merkle_pos = merkle_file_pos(inode); int ret; @@ -726,29 +726,36 @@ static struct page *btrfs_read_merkle_tree_page(struct inode *inode, return ERR_PTR(-EFBIG); index += merkle_pos >> PAGE_SHIFT; again: - page = find_get_page_flags(inode->i_mapping, index, FGP_ACCESSED); - if (page) { - if (PageUptodate(page)) - return page; + folio = __filemap_get_folio(inode->i_mapping, index, FGP_ACCESSED, 0); + if (!IS_ERR(folio)) { + if (folio_test_uptodate(folio)) + goto out; - lock_page(page); - /* - * We only insert uptodate pages, so !Uptodate has to be - * an error - */ - if (!PageUptodate(page)) { - unlock_page(page); - put_page(page); + folio_lock(folio); + /* If it's not uptodate after we have the lock, we got a read error. */ + if (!folio_test_uptodate(folio)) { + folio_unlock(folio); + folio_put(folio); return ERR_PTR(-EIO); } - unlock_page(page); - return page; + folio_unlock(folio); + goto out; } - page = __page_cache_alloc(mapping_gfp_constraint(inode->i_mapping, ~__GFP_FS)); - if (!page) + folio = filemap_alloc_folio(mapping_gfp_constraint(inode->i_mapping, ~__GFP_FS), + 0); + if (!folio) return ERR_PTR(-ENOMEM); + ret = filemap_add_folio(inode->i_mapping, folio, index, GFP_NOFS); + if (ret) { + folio_put(folio); + /* Did someone else insert a folio here? */ + if (ret == -EEXIST) + goto again; + return ERR_PTR(ret); + } + /* * Merkle item keys are indexed from byte 0 in the merkle tree. * They have the form: @@ -756,28 +763,19 @@ again: * [ inode objectid, BTRFS_MERKLE_ITEM_KEY, offset in bytes ] */ ret = read_key_bytes(BTRFS_I(inode), BTRFS_VERITY_MERKLE_ITEM_KEY, off, - page_address(page), PAGE_SIZE, page); + folio_address(folio), PAGE_SIZE, &folio->page); if (ret < 0) { - put_page(page); + folio_put(folio); return ERR_PTR(ret); } if (ret < PAGE_SIZE) - memzero_page(page, ret, PAGE_SIZE - ret); + folio_zero_segment(folio, ret, PAGE_SIZE); - SetPageUptodate(page); - ret = add_to_page_cache_lru(page, inode->i_mapping, index, GFP_NOFS); + folio_mark_uptodate(folio); + folio_unlock(folio); - if (!ret) { - /* Inserted and ready for fsverity */ - unlock_page(page); - } else { - put_page(page); - /* Did someone race us into inserting this page? */ - if (ret == -EEXIST) - goto again; - page = ERR_PTR(ret); - } - return page; +out: + return folio_file_page(folio, index); } /* diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c index 9621455edebc..b9ef6f54635c 100644 --- a/fs/btrfs/volumes.c +++ b/fs/btrfs/volumes.c @@ -1594,7 +1594,7 @@ static int find_free_dev_extent(struct btrfs_device *device, u64 num_bytes, u64 search_start; u64 hole_size; u64 max_hole_start; - u64 max_hole_size; + u64 max_hole_size = 0; u64 extent_end; u64 search_end = device->total_bytes; int ret; @@ -1602,17 +1602,16 @@ static int find_free_dev_extent(struct btrfs_device *device, u64 num_bytes, struct extent_buffer *l; search_start = dev_extent_search_start(device); + max_hole_start = search_start; WARN_ON(device->zone_info && !IS_ALIGNED(num_bytes, device->zone_info->zone_size)); path = btrfs_alloc_path(); - if (!path) - return -ENOMEM; - - max_hole_start = search_start; - max_hole_size = 0; - + if (!path) { + ret = -ENOMEM; + goto out; + } again: if (search_start >= search_end || test_bit(BTRFS_DEV_STATE_REPLACE_TGT, &device->dev_state)) { @@ -5110,7 +5109,7 @@ static void init_alloc_chunk_ctl_policy_regular( ASSERT(space_info); ctl->max_chunk_size = READ_ONCE(space_info->chunk_size); - ctl->max_stripe_size = ctl->max_chunk_size; + ctl->max_stripe_size = min_t(u64, ctl->max_chunk_size, SZ_1G); if (ctl->type & BTRFS_BLOCK_GROUP_SYSTEM) ctl->devs_max = min_t(int, ctl->devs_max, BTRFS_MAX_DEVS_SYS_CHUNK); diff --git a/fs/buffer.c b/fs/buffer.c index 2379564e5aea..12e9a71c693d 100644 --- a/fs/buffer.c +++ b/fs/buffer.c @@ -2011,7 +2011,7 @@ void folio_zero_new_buffers(struct folio *folio, size_t from, size_t to) } EXPORT_SYMBOL(folio_zero_new_buffers); -static void +static int iomap_to_bh(struct inode *inode, sector_t block, struct buffer_head *bh, const struct iomap *iomap) { @@ -2025,7 +2025,8 @@ iomap_to_bh(struct inode *inode, sector_t block, struct buffer_head *bh, * current block, then do not map the buffer and let the caller * handle it. */ - BUG_ON(offset >= iomap->offset + iomap->length); + if (offset >= iomap->offset + iomap->length) + return -EIO; switch (iomap->type) { case IOMAP_HOLE: @@ -2037,7 +2038,7 @@ iomap_to_bh(struct inode *inode, sector_t block, struct buffer_head *bh, if (!buffer_uptodate(bh) || (offset >= i_size_read(inode))) set_buffer_new(bh); - break; + return 0; case IOMAP_DELALLOC: if (!buffer_uptodate(bh) || (offset >= i_size_read(inode))) @@ -2045,7 +2046,7 @@ iomap_to_bh(struct inode *inode, sector_t block, struct buffer_head *bh, set_buffer_uptodate(bh); set_buffer_mapped(bh); set_buffer_delay(bh); - break; + return 0; case IOMAP_UNWRITTEN: /* * For unwritten regions, we always need to ensure that regions @@ -2057,12 +2058,24 @@ iomap_to_bh(struct inode *inode, sector_t block, struct buffer_head *bh, fallthrough; case IOMAP_MAPPED: if ((iomap->flags & IOMAP_F_NEW) || - offset >= i_size_read(inode)) + offset >= i_size_read(inode)) { + /* + * This can happen if truncating the block device races + * with the check in the caller as i_size updates on + * block devices aren't synchronized by i_rwsem for + * block devices. + */ + if (S_ISBLK(inode->i_mode)) + return -EIO; set_buffer_new(bh); + } bh->b_blocknr = (iomap->addr + offset - iomap->offset) >> inode->i_blkbits; set_buffer_mapped(bh); - break; + return 0; + default: + WARN_ON_ONCE(1); + return -EIO; } } @@ -2103,13 +2116,12 @@ int __block_write_begin_int(struct folio *folio, loff_t pos, unsigned len, clear_buffer_new(bh); if (!buffer_mapped(bh)) { WARN_ON(bh->b_size != blocksize); - if (get_block) { + if (get_block) err = get_block(inode, block, bh, 1); - if (err) - break; - } else { - iomap_to_bh(inode, block, bh, iomap); - } + else + err = iomap_to_bh(inode, block, bh, iomap); + if (err) + break; if (buffer_new(bh)) { clean_bdev_bh_alias(bh); diff --git a/fs/ceph/crypto.c b/fs/ceph/crypto.c index e4d5cd56a80b..5b5112c78462 100644 --- a/fs/ceph/crypto.c +++ b/fs/ceph/crypto.c @@ -249,11 +249,9 @@ static struct inode *parse_longname(const struct inode *parent, if (!dir) { /* This can happen if we're not mounting cephfs on the root */ dir = ceph_get_inode(parent->i_sb, vino, NULL); - if (!dir) - dir = ERR_PTR(-ENOENT); + if (IS_ERR(dir)) + dout("Can't find inode %s (%s)\n", inode_number, name); } - if (IS_ERR(dir)) - dout("Can't find inode %s (%s)\n", inode_number, name); out: kfree(inode_number); @@ -462,7 +460,7 @@ int ceph_fname_to_usr(const struct ceph_fname *fname, struct fscrypt_str *tname, out: fscrypt_fname_free_buffer(&_tname); out_inode: - if ((dir != fname->dir) && !IS_ERR(dir)) { + if (dir != fname->dir) { if ((dir->i_state & I_NEW)) discard_new_inode(dir); else diff --git a/fs/ceph/file.c b/fs/ceph/file.c index b1da02f5dbe3..b5f8038065d7 100644 --- a/fs/ceph/file.c +++ b/fs/ceph/file.c @@ -2969,7 +2969,7 @@ static ssize_t __ceph_copy_file_range(struct file *src_file, loff_t src_off, ret = do_splice_direct(src_file, &src_off, dst_file, &dst_off, src_objlen, flags); /* Abort on short copies or on error */ - if (ret < src_objlen) { + if (ret < (long)src_objlen) { dout("Failed partial copy (%zd)\n", ret); goto out; } diff --git a/fs/ceph/inode.c b/fs/ceph/inode.c index 800ab7920513..b79100f720b3 100644 --- a/fs/ceph/inode.c +++ b/fs/ceph/inode.c @@ -769,9 +769,7 @@ int ceph_fill_file_size(struct inode *inode, int issued, ci->i_truncate_seq = truncate_seq; /* the MDS should have revoked these caps */ - WARN_ON_ONCE(issued & (CEPH_CAP_FILE_EXCL | - CEPH_CAP_FILE_RD | - CEPH_CAP_FILE_WR | + WARN_ON_ONCE(issued & (CEPH_CAP_FILE_RD | CEPH_CAP_FILE_LAZYIO)); /* * If we hold relevant caps, or in the case where we're diff --git a/fs/efivarfs/super.c b/fs/efivarfs/super.c index e028fafa04f3..996271473609 100644 --- a/fs/efivarfs/super.c +++ b/fs/efivarfs/super.c @@ -32,10 +32,16 @@ static int efivarfs_statfs(struct dentry *dentry, struct kstatfs *buf) u64 storage_space, remaining_space, max_variable_size; efi_status_t status; - status = efivar_query_variable_info(attr, &storage_space, &remaining_space, - &max_variable_size); - if (status != EFI_SUCCESS) - return efi_status_to_err(status); + /* Some UEFI firmware does not implement QueryVariableInfo() */ + storage_space = remaining_space = 0; + if (efi_rt_services_supported(EFI_RT_SUPPORTED_QUERY_VARIABLE_INFO)) { + status = efivar_query_variable_info(attr, &storage_space, + &remaining_space, + &max_variable_size); + if (status != EFI_SUCCESS && status != EFI_UNSUPPORTED) + pr_warn_ratelimited("query_variable_info() failed: 0x%lx\n", + status); + } /* * This is not a normal filesystem, so no point in pretending it has a block diff --git a/fs/erofs/decompressor_lzma.c b/fs/erofs/decompressor_lzma.c index 73091fbe3ea4..dee10d22ada9 100644 --- a/fs/erofs/decompressor_lzma.c +++ b/fs/erofs/decompressor_lzma.c @@ -217,9 +217,12 @@ again: strm->buf.out_size = min_t(u32, outlen, PAGE_SIZE - pageofs); outlen -= strm->buf.out_size; - if (!rq->out[no] && rq->fillgaps) /* deduped */ + if (!rq->out[no] && rq->fillgaps) { /* deduped */ rq->out[no] = erofs_allocpage(pagepool, GFP_KERNEL | __GFP_NOFAIL); + set_page_private(rq->out[no], + Z_EROFS_SHORTLIVED_PAGE); + } if (rq->out[no]) strm->buf.out = kmap(rq->out[no]) + pageofs; pageofs = 0; diff --git a/fs/erofs/super.c b/fs/erofs/super.c index 44a24d573f1f..3700af9ee173 100644 --- a/fs/erofs/super.c +++ b/fs/erofs/super.c @@ -235,7 +235,7 @@ static int erofs_init_device(struct erofs_buf *buf, struct super_block *sb, return PTR_ERR(ptr); dis = ptr + erofs_blkoff(sb, *pos); - if (!dif->path) { + if (!sbi->devs->flatdev && !dif->path) { if (!dis->tag[0]) { erofs_err(sb, "empty device tag @ pos %llu", *pos); return -EINVAL; diff --git a/fs/ext4/mballoc.c b/fs/ext4/mballoc.c index c91db9f57524..1e599305d85f 100644 --- a/fs/ext4/mballoc.c +++ b/fs/ext4/mballoc.c @@ -16,6 +16,7 @@ #include #include #include +#include #include /* @@ -6906,6 +6907,21 @@ __acquires(bitlock) return ret; } +static ext4_grpblk_t ext4_last_grp_cluster(struct super_block *sb, + ext4_group_t grp) +{ + if (grp < ext4_get_groups_count(sb)) + return EXT4_CLUSTERS_PER_GROUP(sb) - 1; + return (ext4_blocks_count(EXT4_SB(sb)->s_es) - + ext4_group_first_block_no(sb, grp) - 1) >> + EXT4_CLUSTER_BITS(sb); +} + +static bool ext4_trim_interrupted(void) +{ + return fatal_signal_pending(current) || freezing(current); +} + static int ext4_try_to_trim_range(struct super_block *sb, struct ext4_buddy *e4b, ext4_grpblk_t start, ext4_grpblk_t max, ext4_grpblk_t minblocks) @@ -6913,9 +6929,12 @@ __acquires(ext4_group_lock_ptr(sb, e4b->bd_group)) __releases(ext4_group_lock_ptr(sb, e4b->bd_group)) { ext4_grpblk_t next, count, free_count; + bool set_trimmed = false; void *bitmap; bitmap = e4b->bd_bitmap; + if (start == 0 && max >= ext4_last_grp_cluster(sb, e4b->bd_group)) + set_trimmed = true; start = max(e4b->bd_info->bb_first_free, start); count = 0; free_count = 0; @@ -6930,16 +6949,14 @@ __releases(ext4_group_lock_ptr(sb, e4b->bd_group)) int ret = ext4_trim_extent(sb, start, next - start, e4b); if (ret && ret != -EOPNOTSUPP) - break; + return count; count += next - start; } free_count += next - start; start = next + 1; - if (fatal_signal_pending(current)) { - count = -ERESTARTSYS; - break; - } + if (ext4_trim_interrupted()) + return count; if (need_resched()) { ext4_unlock_group(sb, e4b->bd_group); @@ -6951,6 +6968,9 @@ __releases(ext4_group_lock_ptr(sb, e4b->bd_group)) break; } + if (set_trimmed) + EXT4_MB_GRP_SET_TRIMMED(e4b->bd_info); + return count; } @@ -6961,7 +6981,6 @@ __releases(ext4_group_lock_ptr(sb, e4b->bd_group)) * @start: first group block to examine * @max: last group block to examine * @minblocks: minimum extent block count - * @set_trimmed: set the trimmed flag if at least one block is trimmed * * ext4_trim_all_free walks through group's block bitmap searching for free * extents. When the free extent is found, mark it as used in group buddy @@ -6971,7 +6990,7 @@ __releases(ext4_group_lock_ptr(sb, e4b->bd_group)) static ext4_grpblk_t ext4_trim_all_free(struct super_block *sb, ext4_group_t group, ext4_grpblk_t start, ext4_grpblk_t max, - ext4_grpblk_t minblocks, bool set_trimmed) + ext4_grpblk_t minblocks) { struct ext4_buddy e4b; int ret; @@ -6988,13 +7007,10 @@ ext4_trim_all_free(struct super_block *sb, ext4_group_t group, ext4_lock_group(sb, group); if (!EXT4_MB_GRP_WAS_TRIMMED(e4b.bd_info) || - minblocks < EXT4_SB(sb)->s_last_trim_minblks) { + minblocks < EXT4_SB(sb)->s_last_trim_minblks) ret = ext4_try_to_trim_range(sb, &e4b, start, max, minblocks); - if (ret >= 0 && set_trimmed) - EXT4_MB_GRP_SET_TRIMMED(e4b.bd_info); - } else { + else ret = 0; - } ext4_unlock_group(sb, group); ext4_mb_unload_buddy(&e4b); @@ -7027,7 +7043,6 @@ int ext4_trim_fs(struct super_block *sb, struct fstrim_range *range) ext4_fsblk_t first_data_blk = le32_to_cpu(EXT4_SB(sb)->s_es->s_first_data_block); ext4_fsblk_t max_blks = ext4_blocks_count(EXT4_SB(sb)->s_es); - bool whole_group, eof = false; int ret = 0; start = range->start >> sb->s_blocksize_bits; @@ -7046,10 +7061,8 @@ int ext4_trim_fs(struct super_block *sb, struct fstrim_range *range) if (minlen > EXT4_CLUSTERS_PER_GROUP(sb)) goto out; } - if (end >= max_blks - 1) { + if (end >= max_blks - 1) end = max_blks - 1; - eof = true; - } if (end <= first_data_blk) goto out; if (start < first_data_blk) @@ -7063,9 +7076,10 @@ int ext4_trim_fs(struct super_block *sb, struct fstrim_range *range) /* end now represents the last cluster to discard in this group */ end = EXT4_CLUSTERS_PER_GROUP(sb) - 1; - whole_group = true; for (group = first_group; group <= last_group; group++) { + if (ext4_trim_interrupted()) + break; grp = ext4_get_group_info(sb, group); if (!grp) continue; @@ -7082,13 +7096,11 @@ int ext4_trim_fs(struct super_block *sb, struct fstrim_range *range) * change it for the last group, note that last_cluster is * already computed earlier by ext4_get_group_no_and_offset() */ - if (group == last_group) { + if (group == last_group) end = last_cluster; - whole_group = eof ? true : end == EXT4_CLUSTERS_PER_GROUP(sb) - 1; - } if (grp->bb_free >= minlen) { cnt = ext4_trim_all_free(sb, group, first_cluster, - end, minlen, whole_group); + end, minlen); if (cnt < 0) { ret = cnt; break; diff --git a/fs/ext4/namei.c b/fs/ext4/namei.c index 41a6411c600b..bbda587f76b8 100644 --- a/fs/ext4/namei.c +++ b/fs/ext4/namei.c @@ -343,17 +343,17 @@ static struct ext4_dir_entry_tail *get_dirent_tail(struct inode *inode, struct buffer_head *bh) { struct ext4_dir_entry_tail *t; + int blocksize = EXT4_BLOCK_SIZE(inode->i_sb); #ifdef PARANOID struct ext4_dir_entry *d, *top; d = (struct ext4_dir_entry *)bh->b_data; top = (struct ext4_dir_entry *)(bh->b_data + - (EXT4_BLOCK_SIZE(inode->i_sb) - - sizeof(struct ext4_dir_entry_tail))); - while (d < top && d->rec_len) + (blocksize - sizeof(struct ext4_dir_entry_tail))); + while (d < top && ext4_rec_len_from_disk(d->rec_len, blocksize)) d = (struct ext4_dir_entry *)(((void *)d) + - le16_to_cpu(d->rec_len)); + ext4_rec_len_from_disk(d->rec_len, blocksize)); if (d != top) return NULL; @@ -364,7 +364,8 @@ static struct ext4_dir_entry_tail *get_dirent_tail(struct inode *inode, #endif if (t->det_reserved_zero1 || - le16_to_cpu(t->det_rec_len) != sizeof(struct ext4_dir_entry_tail) || + (ext4_rec_len_from_disk(t->det_rec_len, blocksize) != + sizeof(struct ext4_dir_entry_tail)) || t->det_reserved_zero2 || t->det_reserved_ft != EXT4_FT_DIR_CSUM) return NULL; @@ -445,13 +446,14 @@ static struct dx_countlimit *get_dx_countlimit(struct inode *inode, struct ext4_dir_entry *dp; struct dx_root_info *root; int count_offset; + int blocksize = EXT4_BLOCK_SIZE(inode->i_sb); + unsigned int rlen = ext4_rec_len_from_disk(dirent->rec_len, blocksize); - if (le16_to_cpu(dirent->rec_len) == EXT4_BLOCK_SIZE(inode->i_sb)) + if (rlen == blocksize) count_offset = 8; - else if (le16_to_cpu(dirent->rec_len) == 12) { + else if (rlen == 12) { dp = (struct ext4_dir_entry *)(((void *)dirent) + 12); - if (le16_to_cpu(dp->rec_len) != - EXT4_BLOCK_SIZE(inode->i_sb) - 12) + if (ext4_rec_len_from_disk(dp->rec_len, blocksize) != blocksize - 12) return NULL; root = (struct dx_root_info *)(((void *)dp + 12)); if (root->reserved_zero || @@ -1315,6 +1317,7 @@ static int dx_make_map(struct inode *dir, struct buffer_head *bh, unsigned int buflen = bh->b_size; char *base = bh->b_data; struct dx_hash_info h = *hinfo; + int blocksize = EXT4_BLOCK_SIZE(dir->i_sb); if (ext4_has_metadata_csum(dir->i_sb)) buflen -= sizeof(struct ext4_dir_entry_tail); @@ -1335,11 +1338,12 @@ static int dx_make_map(struct inode *dir, struct buffer_head *bh, map_tail--; map_tail->hash = h.hash; map_tail->offs = ((char *) de - base)>>2; - map_tail->size = le16_to_cpu(de->rec_len); + map_tail->size = ext4_rec_len_from_disk(de->rec_len, + blocksize); count++; cond_resched(); } - de = ext4_next_entry(de, dir->i_sb->s_blocksize); + de = ext4_next_entry(de, blocksize); } return count; } diff --git a/fs/ext4/super.c b/fs/ext4/super.c index 38217422f938..dbebd8b3127e 100644 --- a/fs/ext4/super.c +++ b/fs/ext4/super.c @@ -7314,7 +7314,7 @@ static struct file_system_type ext4_fs_type = { .init_fs_context = ext4_init_fs_context, .parameters = ext4_param_specs, .kill_sb = ext4_kill_sb, - .fs_flags = FS_REQUIRES_DEV | FS_ALLOW_IDMAP | FS_MGTIME, + .fs_flags = FS_REQUIRES_DEV | FS_ALLOW_IDMAP, }; MODULE_ALIAS_FS("ext4"); diff --git a/fs/fs-writeback.c b/fs/fs-writeback.c index 969ce991b0b0..c1af01b2c42d 100644 --- a/fs/fs-writeback.c +++ b/fs/fs-writeback.c @@ -1535,10 +1535,15 @@ static void requeue_inode(struct inode *inode, struct bdi_writeback *wb, if (wbc->pages_skipped) { /* - * writeback is not making progress due to locked - * buffers. Skip this inode for now. + * Writeback is not making progress due to locked buffers. + * Skip this inode for now. Although having skipped pages + * is odd for clean inodes, it can happen for some + * filesystems so handle that gracefully. */ - redirty_tail_locked(inode, wb); + if (inode->i_state & I_DIRTY_ALL) + redirty_tail_locked(inode, wb); + else + inode_cgwb_move_to_attached(inode, wb); return; } diff --git a/fs/fs_context.c b/fs/fs_context.c index a0ad7a0c4680..98589aae5208 100644 --- a/fs/fs_context.c +++ b/fs/fs_context.c @@ -192,17 +192,19 @@ int vfs_parse_fs_string(struct fs_context *fc, const char *key, EXPORT_SYMBOL(vfs_parse_fs_string); /** - * generic_parse_monolithic - Parse key[=val][,key[=val]]* mount data + * vfs_parse_monolithic_sep - Parse key[=val][,key[=val]]* mount data * @fc: The superblock configuration to fill in. * @data: The data to parse + * @sep: callback for separating next option * - * Parse a blob of data that's in key[=val][,key[=val]]* form. This can be - * called from the ->monolithic_mount_data() fs_context operation. + * Parse a blob of data that's in key[=val][,key[=val]]* form with a custom + * option separator callback. * * Returns 0 on success or the error returned by the ->parse_option() fs_context * operation on failure. */ -int generic_parse_monolithic(struct fs_context *fc, void *data) +int vfs_parse_monolithic_sep(struct fs_context *fc, void *data, + char *(*sep)(char **)) { char *options = data, *key; int ret = 0; @@ -214,7 +216,7 @@ int generic_parse_monolithic(struct fs_context *fc, void *data) if (ret) return ret; - while ((key = strsep(&options, ",")) != NULL) { + while ((key = sep(&options)) != NULL) { if (*key) { size_t v_len = 0; char *value = strchr(key, '='); @@ -233,6 +235,28 @@ int generic_parse_monolithic(struct fs_context *fc, void *data) return ret; } +EXPORT_SYMBOL(vfs_parse_monolithic_sep); + +static char *vfs_parse_comma_sep(char **s) +{ + return strsep(s, ","); +} + +/** + * generic_parse_monolithic - Parse key[=val][,key[=val]]* mount data + * @fc: The superblock configuration to fill in. + * @data: The data to parse + * + * Parse a blob of data that's in key[=val][,key[=val]]* form. This can be + * called from the ->monolithic_mount_data() fs_context operation. + * + * Returns 0 on success or the error returned by the ->parse_option() fs_context + * operation on failure. + */ +int generic_parse_monolithic(struct fs_context *fc, void *data) +{ + return vfs_parse_monolithic_sep(fc, data, vfs_parse_comma_sep); +} EXPORT_SYMBOL(generic_parse_monolithic); /** diff --git a/fs/gfs2/glock.c b/fs/gfs2/glock.c index 9cbf8d98489a..4a280be229a6 100644 --- a/fs/gfs2/glock.c +++ b/fs/gfs2/glock.c @@ -2010,7 +2010,9 @@ static long gfs2_scan_glock_lru(int nr) if (!test_bit(GLF_LOCK, &gl->gl_flags)) { if (!spin_trylock(&gl->gl_lockref.lock)) continue; - if (!gl->gl_lockref.count) { + if (gl->gl_lockref.count <= 1 && + (gl->gl_state == LM_ST_UNLOCKED || + demote_ok(gl))) { list_move(&gl->gl_lru, &dispose); atomic_dec(&lru_count); freed++; diff --git a/fs/gfs2/glops.c b/fs/gfs2/glops.c index d26759a98b10..f41ca89d216b 100644 --- a/fs/gfs2/glops.c +++ b/fs/gfs2/glops.c @@ -567,15 +567,16 @@ static void freeze_go_callback(struct gfs2_glock *gl, bool remote) struct super_block *sb = sdp->sd_vfs; if (!remote || - gl->gl_state != LM_ST_SHARED || + (gl->gl_state != LM_ST_SHARED && + gl->gl_state != LM_ST_UNLOCKED) || gl->gl_demote_state != LM_ST_UNLOCKED) return; /* * Try to get an active super block reference to prevent racing with - * unmount (see trylock_super()). But note that unmount isn't the only - * place where a write lock on s_umount is taken, and we can fail here - * because of things like remount as well. + * unmount (see super_trylock_shared()). But note that unmount isn't + * the only place where a write lock on s_umount is taken, and we can + * fail here because of things like remount as well. */ if (down_read_trylock(&sb->s_umount)) { atomic_inc(&sb->s_active); diff --git a/fs/gfs2/quota.h b/fs/gfs2/quota.h index 21ada332d555..1429945215a0 100644 --- a/fs/gfs2/quota.h +++ b/fs/gfs2/quota.h @@ -50,7 +50,8 @@ static inline int gfs2_quota_lock_check(struct gfs2_inode *ip, ret = gfs2_quota_lock(ip, NO_UID_QUOTA_CHANGE, NO_GID_QUOTA_CHANGE); if (ret) return ret; - if (sdp->sd_args.ar_quota != GFS2_QUOTA_ON) + if (sdp->sd_args.ar_quota != GFS2_QUOTA_ON && + sdp->sd_args.ar_quota != GFS2_QUOTA_QUIET) return 0; ret = gfs2_quota_check(ip, ip->i_inode.i_uid, ip->i_inode.i_gid, ap); if (ret) diff --git a/fs/inode.c b/fs/inode.c index 35fd688168c5..84bc3c76e5cc 100644 --- a/fs/inode.c +++ b/fs/inode.c @@ -2102,52 +2102,10 @@ int file_remove_privs(struct file *file) } EXPORT_SYMBOL(file_remove_privs); -/** - * current_mgtime - Return FS time (possibly fine-grained) - * @inode: inode. - * - * Return the current time truncated to the time granularity supported by - * the fs, as suitable for a ctime/mtime change. If the ctime is flagged - * as having been QUERIED, get a fine-grained timestamp. - */ -struct timespec64 current_mgtime(struct inode *inode) -{ - struct timespec64 now, ctime; - atomic_long_t *pnsec = (atomic_long_t *)&inode->__i_ctime.tv_nsec; - long nsec = atomic_long_read(pnsec); - - if (nsec & I_CTIME_QUERIED) { - ktime_get_real_ts64(&now); - return timestamp_truncate(now, inode); - } - - ktime_get_coarse_real_ts64(&now); - now = timestamp_truncate(now, inode); - - /* - * If we've recently fetched a fine-grained timestamp - * then the coarse-grained one may still be earlier than the - * existing ctime. Just keep the existing value if so. - */ - ctime = inode_get_ctime(inode); - if (timespec64_compare(&ctime, &now) > 0) - now = ctime; - - return now; -} -EXPORT_SYMBOL(current_mgtime); - -static struct timespec64 current_ctime(struct inode *inode) -{ - if (is_mgtime(inode)) - return current_mgtime(inode); - return current_time(inode); -} - static int inode_needs_update_time(struct inode *inode) { int sync_it = 0; - struct timespec64 now = current_ctime(inode); + struct timespec64 now = current_time(inode); struct timespec64 ctime; /* First try to exhaust all avenues to not sync */ @@ -2578,43 +2536,9 @@ EXPORT_SYMBOL(current_time); */ struct timespec64 inode_set_ctime_current(struct inode *inode) { - struct timespec64 now; - struct timespec64 ctime; - - ctime.tv_nsec = READ_ONCE(inode->__i_ctime.tv_nsec); - if (!(ctime.tv_nsec & I_CTIME_QUERIED)) { - now = current_time(inode); + struct timespec64 now = current_time(inode); - /* Just copy it into place if it's not multigrain */ - if (!is_mgtime(inode)) { - inode_set_ctime_to_ts(inode, now); - return now; - } - - /* - * If we've recently updated with a fine-grained timestamp, - * then the coarse-grained one may still be earlier than the - * existing ctime. Just keep the existing value if so. - */ - ctime.tv_sec = inode->__i_ctime.tv_sec; - if (timespec64_compare(&ctime, &now) > 0) - return ctime; - - /* - * Ctime updates are usually protected by the inode_lock, but - * we can still race with someone setting the QUERIED flag. - * Try to swap the new nsec value into place. If it's changed - * in the interim, then just go with a fine-grained timestamp. - */ - if (cmpxchg(&inode->__i_ctime.tv_nsec, ctime.tv_nsec, - now.tv_nsec) != ctime.tv_nsec) - goto fine_grained; - inode->__i_ctime.tv_sec = now.tv_sec; - return now; - } -fine_grained: - ktime_get_real_ts64(&now); - inode_set_ctime_to_ts(inode, timestamp_truncate(now, inode)); + inode_set_ctime(inode, now.tv_sec, now.tv_nsec); return now; } EXPORT_SYMBOL(inode_set_ctime_current); diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c index ae8673ce08b1..2bc0aa23fde3 100644 --- a/fs/iomap/buffered-io.c +++ b/fs/iomap/buffered-io.c @@ -640,11 +640,13 @@ static int __iomap_write_begin(const struct iomap_iter *iter, loff_t pos, size_t poff, plen; /* - * If the write completely overlaps the current folio, then + * If the write or zeroing completely overlaps the current folio, then * entire folio will be dirtied so there is no need for * per-block state tracking structures to be attached to this folio. + * For the unshare case, we must read in the ondisk contents because we + * are not changing pagecache contents. */ - if (pos <= folio_pos(folio) && + if (!(iter->flags & IOMAP_UNSHARE) && pos <= folio_pos(folio) && pos + len >= folio_pos(folio) + folio_size(folio)) return 0; @@ -879,8 +881,10 @@ static loff_t iomap_write_iter(struct iomap_iter *iter, struct iov_iter *i) size_t bytes; /* Bytes to write to folio */ size_t copied; /* Bytes copied from user */ + bytes = iov_iter_count(i); +retry: offset = pos & (chunk - 1); - bytes = min(chunk - offset, iov_iter_count(i)); + bytes = min(chunk - offset, bytes); status = balance_dirty_pages_ratelimited_flags(mapping, bdp_flags); if (unlikely(status)) @@ -931,10 +935,12 @@ static loff_t iomap_write_iter(struct iomap_iter *iter, struct iov_iter *i) * halfway through, might be a race with munmap, * might be severe memory pressure. */ - if (copied) - bytes = copied; if (chunk > PAGE_SIZE) chunk /= 2; + if (copied) { + bytes = copied; + goto retry; + } } else { pos += status; written += status; @@ -1047,7 +1053,7 @@ static int iomap_write_delalloc_punch(struct inode *inode, struct folio *folio, /* * Scan the data range passed to us for dirty page cache folios. If we find a - * dirty folio, punch out the preceeding range and update the offset from which + * dirty folio, punch out the preceding range and update the offset from which * the next punch will start from. * * We can punch out storage reservations under clean pages because they either @@ -1261,7 +1267,6 @@ static loff_t iomap_unshare_iter(struct iomap_iter *iter) const struct iomap *srcmap = iomap_iter_srcmap(iter); loff_t pos = iter->pos; loff_t length = iomap_length(iter); - long status = 0; loff_t written = 0; /* don't bother with blocks that are not shared to start with */ @@ -1272,28 +1277,33 @@ static loff_t iomap_unshare_iter(struct iomap_iter *iter) return length; do { - unsigned long offset = offset_in_page(pos); - unsigned long bytes = min_t(loff_t, PAGE_SIZE - offset, length); struct folio *folio; + int status; + size_t offset; + size_t bytes = min_t(u64, SIZE_MAX, length); status = iomap_write_begin(iter, pos, bytes, &folio); if (unlikely(status)) return status; - if (iter->iomap.flags & IOMAP_F_STALE) + if (iomap->flags & IOMAP_F_STALE) break; - status = iomap_write_end(iter, pos, bytes, bytes, folio); - if (WARN_ON_ONCE(status == 0)) + offset = offset_in_folio(folio, pos); + if (bytes > folio_size(folio) - offset) + bytes = folio_size(folio) - offset; + + bytes = iomap_write_end(iter, pos, bytes, bytes, folio); + if (WARN_ON_ONCE(bytes == 0)) return -EIO; cond_resched(); - pos += status; - written += status; - length -= status; + pos += bytes; + written += bytes; + length -= bytes; balance_dirty_pages_ratelimited(iter->inode->i_mapping); - } while (length); + } while (length > 0); return written; } diff --git a/fs/jbd2/commit.c b/fs/jbd2/commit.c index 1073259902a6..8d6f934c3d95 100644 --- a/fs/jbd2/commit.c +++ b/fs/jbd2/commit.c @@ -298,14 +298,12 @@ static int journal_finish_inode_data_buffers(journal_t *journal, static __u32 jbd2_checksum_data(__u32 crc32_sum, struct buffer_head *bh) { - struct page *page = bh->b_page; char *addr; __u32 checksum; - addr = kmap_atomic(page); - checksum = crc32_be(crc32_sum, - (void *)(addr + offset_in_page(bh->b_data)), bh->b_size); - kunmap_atomic(addr); + addr = kmap_local_folio(bh->b_folio, bh_offset(bh)); + checksum = crc32_be(crc32_sum, addr, bh->b_size); + kunmap_local(addr); return checksum; } @@ -322,7 +320,6 @@ static void jbd2_block_tag_csum_set(journal_t *j, journal_block_tag_t *tag, struct buffer_head *bh, __u32 sequence) { journal_block_tag3_t *tag3 = (journal_block_tag3_t *)tag; - struct page *page = bh->b_page; __u8 *addr; __u32 csum32; __be32 seq; @@ -331,11 +328,10 @@ static void jbd2_block_tag_csum_set(journal_t *j, journal_block_tag_t *tag, return; seq = cpu_to_be32(sequence); - addr = kmap_atomic(page); + addr = kmap_local_folio(bh->b_folio, bh_offset(bh)); csum32 = jbd2_chksum(j, j->j_csum_seed, (__u8 *)&seq, sizeof(seq)); - csum32 = jbd2_chksum(j, csum32, addr + offset_in_page(bh->b_data), - bh->b_size); - kunmap_atomic(addr); + csum32 = jbd2_chksum(j, csum32, addr, bh->b_size); + kunmap_local(addr); if (jbd2_has_feature_csum3(j)) tag3->t_checksum = cpu_to_be32(csum32); diff --git a/fs/jbd2/journal.c b/fs/jbd2/journal.c index 768fa05bcbed..30dec2bd2ecc 100644 --- a/fs/jbd2/journal.c +++ b/fs/jbd2/journal.c @@ -1601,6 +1601,8 @@ static journal_t *journal_init_common(struct block_device *bdev, err_cleanup: percpu_counter_destroy(&journal->j_checkpoint_jh_count); + if (journal->j_chksum_driver) + crypto_free_shash(journal->j_chksum_driver); kfree(journal->j_wbuf); jbd2_journal_destroy_revoke(journal); journal_fail_superblock(journal); diff --git a/fs/jbd2/transaction.c b/fs/jbd2/transaction.c index 4d1fda1f7143..5f08b5fd105a 100644 --- a/fs/jbd2/transaction.c +++ b/fs/jbd2/transaction.c @@ -935,19 +935,15 @@ static void warn_dirty_buffer(struct buffer_head *bh) /* Call t_frozen trigger and copy buffer data into jh->b_frozen_data. */ static void jbd2_freeze_jh_data(struct journal_head *jh) { - struct page *page; - int offset; char *source; struct buffer_head *bh = jh2bh(jh); J_EXPECT_JH(jh, buffer_uptodate(bh), "Possible IO failure.\n"); - page = bh->b_page; - offset = offset_in_page(bh->b_data); - source = kmap_atomic(page); + source = kmap_local_folio(bh->b_folio, bh_offset(bh)); /* Fire data frozen trigger just before we copy the data */ - jbd2_buffer_frozen_trigger(jh, source + offset, jh->b_triggers); - memcpy(jh->b_frozen_data, source + offset, bh->b_size); - kunmap_atomic(source); + jbd2_buffer_frozen_trigger(jh, source, jh->b_triggers); + memcpy(jh->b_frozen_data, source, bh->b_size); + kunmap_local(source); /* * Now that the frozen data is saved off, we need to store any matching diff --git a/fs/libfs.c b/fs/libfs.c index a4eb12757886..37f2d34ee090 100644 --- a/fs/libfs.c +++ b/fs/libfs.c @@ -1903,6 +1903,7 @@ ssize_t direct_write_fallback(struct kiocb *iocb, struct iov_iter *iter, * We don't know how much we wrote, so just return the number of * bytes which were direct-written */ + iocb->ki_pos -= buffered_written; if (direct_written) return direct_written; return err; diff --git a/fs/namei.c b/fs/namei.c index 567ee547492b..94565bd7e73f 100644 --- a/fs/namei.c +++ b/fs/namei.c @@ -188,7 +188,7 @@ getname_flags(const char __user *filename, int flags, int *empty) } } - result->refcnt = 1; + atomic_set(&result->refcnt, 1); /* The empty path is special. */ if (unlikely(!len)) { if (empty) @@ -249,7 +249,7 @@ getname_kernel(const char * filename) memcpy((char *)result->name, filename, len); result->uptr = NULL; result->aname = NULL; - result->refcnt = 1; + atomic_set(&result->refcnt, 1); audit_getname(result); return result; @@ -261,9 +261,10 @@ void putname(struct filename *name) if (IS_ERR(name)) return; - BUG_ON(name->refcnt <= 0); + if (WARN_ON_ONCE(!atomic_read(&name->refcnt))) + return; - if (--name->refcnt > 0) + if (!atomic_dec_and_test(&name->refcnt)) return; if (name->name != name->iname) { diff --git a/fs/netfs/buffered_read.c b/fs/netfs/buffered_read.c index 3404707ddbe7..2cd3ccf4c439 100644 --- a/fs/netfs/buffered_read.c +++ b/fs/netfs/buffered_read.c @@ -47,12 +47,14 @@ void netfs_rreq_unlock_folios(struct netfs_io_request *rreq) xas_for_each(&xas, folio, last_page) { loff_t pg_end; bool pg_failed = false; + bool folio_started; if (xas_retry(&xas, folio)) continue; pg_end = folio_pos(folio) + folio_size(folio) - 1; + folio_started = false; for (;;) { loff_t sreq_end; @@ -60,8 +62,10 @@ void netfs_rreq_unlock_folios(struct netfs_io_request *rreq) pg_failed = true; break; } - if (test_bit(NETFS_SREQ_COPY_TO_CACHE, &subreq->flags)) + if (!folio_started && test_bit(NETFS_SREQ_COPY_TO_CACHE, &subreq->flags)) { folio_start_fscache(folio); + folio_started = true; + } pg_failed |= subreq_failed; sreq_end = subreq->start + subreq->len - 1; if (pg_end < sreq_end) diff --git a/fs/nfs/direct.c b/fs/nfs/direct.c index 47d892a1d363..f6c74f424691 100644 --- a/fs/nfs/direct.c +++ b/fs/nfs/direct.c @@ -93,12 +93,10 @@ nfs_direct_handle_truncated(struct nfs_direct_req *dreq, dreq->max_count = dreq_len; if (dreq->count > dreq_len) dreq->count = dreq_len; - - if (test_bit(NFS_IOHDR_ERROR, &hdr->flags)) - dreq->error = hdr->error; - else /* Clear outstanding error if this is EOF */ - dreq->error = 0; } + + if (test_bit(NFS_IOHDR_ERROR, &hdr->flags) && !dreq->error) + dreq->error = hdr->error; } static void @@ -120,6 +118,18 @@ nfs_direct_count_bytes(struct nfs_direct_req *dreq, dreq->count = dreq_len; } +static void nfs_direct_truncate_request(struct nfs_direct_req *dreq, + struct nfs_page *req) +{ + loff_t offs = req_offset(req); + size_t req_start = (size_t)(offs - dreq->io_start); + + if (req_start < dreq->max_count) + dreq->max_count = req_start; + if (req_start < dreq->count) + dreq->count = req_start; +} + /** * nfs_swap_rw - NFS address space operation for swap I/O * @iocb: target I/O control block @@ -488,7 +498,9 @@ static void nfs_direct_add_page_head(struct list_head *list, kref_get(&head->wb_kref); } -static void nfs_direct_join_group(struct list_head *list, struct inode *inode) +static void nfs_direct_join_group(struct list_head *list, + struct nfs_commit_info *cinfo, + struct inode *inode) { struct nfs_page *req, *subreq; @@ -510,7 +522,7 @@ static void nfs_direct_join_group(struct list_head *list, struct inode *inode) nfs_release_request(subreq); } } while ((subreq = subreq->wb_this_page) != req); - nfs_join_page_group(req, inode); + nfs_join_page_group(req, cinfo, inode); } } @@ -528,20 +540,15 @@ nfs_direct_write_scan_commit_list(struct inode *inode, static void nfs_direct_write_reschedule(struct nfs_direct_req *dreq) { struct nfs_pageio_descriptor desc; - struct nfs_page *req, *tmp; + struct nfs_page *req; LIST_HEAD(reqs); struct nfs_commit_info cinfo; - LIST_HEAD(failed); nfs_init_cinfo_from_dreq(&cinfo, dreq); nfs_direct_write_scan_commit_list(dreq->inode, &reqs, &cinfo); - nfs_direct_join_group(&reqs, dreq->inode); + nfs_direct_join_group(&reqs, &cinfo, dreq->inode); - dreq->count = 0; - dreq->max_count = 0; - list_for_each_entry(req, &reqs, wb_list) - dreq->max_count += req->wb_bytes; nfs_clear_pnfs_ds_commit_verifiers(&dreq->ds_cinfo); get_dreq(dreq); @@ -549,27 +556,40 @@ static void nfs_direct_write_reschedule(struct nfs_direct_req *dreq) &nfs_direct_write_completion_ops); desc.pg_dreq = dreq; - list_for_each_entry_safe(req, tmp, &reqs, wb_list) { + while (!list_empty(&reqs)) { + req = nfs_list_entry(reqs.next); /* Bump the transmission count */ req->wb_nio++; if (!nfs_pageio_add_request(&desc, req)) { - nfs_list_move_request(req, &failed); - spin_lock(&cinfo.inode->i_lock); - dreq->flags = 0; - if (desc.pg_error < 0) + spin_lock(&dreq->lock); + if (dreq->error < 0) { + desc.pg_error = dreq->error; + } else if (desc.pg_error != -EAGAIN) { + dreq->flags = 0; + if (!desc.pg_error) + desc.pg_error = -EIO; dreq->error = desc.pg_error; - else - dreq->error = -EIO; - spin_unlock(&cinfo.inode->i_lock); + } else + dreq->flags = NFS_ODIRECT_RESCHED_WRITES; + spin_unlock(&dreq->lock); + break; } nfs_release_request(req); } nfs_pageio_complete(&desc); - while (!list_empty(&failed)) { - req = nfs_list_entry(failed.next); + while (!list_empty(&reqs)) { + req = nfs_list_entry(reqs.next); nfs_list_remove_request(req); nfs_unlock_and_release_request(req); + if (desc.pg_error == -EAGAIN) { + nfs_mark_request_commit(req, NULL, &cinfo, 0); + } else { + spin_lock(&dreq->lock); + nfs_direct_truncate_request(dreq, req); + spin_unlock(&dreq->lock); + nfs_release_request(req); + } } if (put_dreq(dreq)) @@ -589,8 +609,6 @@ static void nfs_direct_commit_complete(struct nfs_commit_data *data) if (status < 0) { /* Errors in commit are fatal */ dreq->error = status; - dreq->max_count = 0; - dreq->count = 0; dreq->flags = NFS_ODIRECT_DONE; } else { status = dreq->error; @@ -601,7 +619,12 @@ static void nfs_direct_commit_complete(struct nfs_commit_data *data) while (!list_empty(&data->pages)) { req = nfs_list_entry(data->pages.next); nfs_list_remove_request(req); - if (status >= 0 && !nfs_write_match_verf(verf, req)) { + if (status < 0) { + spin_lock(&dreq->lock); + nfs_direct_truncate_request(dreq, req); + spin_unlock(&dreq->lock); + nfs_release_request(req); + } else if (!nfs_write_match_verf(verf, req)) { dreq->flags = NFS_ODIRECT_RESCHED_WRITES; /* * Despite the reboot, the write was successful, @@ -609,7 +632,7 @@ static void nfs_direct_commit_complete(struct nfs_commit_data *data) */ req->wb_nio = 0; nfs_mark_request_commit(req, NULL, &cinfo, 0); - } else /* Error or match */ + } else nfs_release_request(req); nfs_unlock_and_release_request(req); } @@ -662,6 +685,7 @@ static void nfs_direct_write_clear_reqs(struct nfs_direct_req *dreq) while (!list_empty(&reqs)) { req = nfs_list_entry(reqs.next); nfs_list_remove_request(req); + nfs_direct_truncate_request(dreq, req); nfs_release_request(req); nfs_unlock_and_release_request(req); } @@ -711,7 +735,8 @@ static void nfs_direct_write_completion(struct nfs_pgio_header *hdr) } nfs_direct_count_bytes(dreq, hdr); - if (test_bit(NFS_IOHDR_UNSTABLE_WRITES, &hdr->flags)) { + if (test_bit(NFS_IOHDR_UNSTABLE_WRITES, &hdr->flags) && + !test_bit(NFS_IOHDR_ERROR, &hdr->flags)) { if (!dreq->flags) dreq->flags = NFS_ODIRECT_DO_COMMIT; flags = dreq->flags; @@ -755,18 +780,23 @@ static void nfs_write_sync_pgio_error(struct list_head *head, int error) static void nfs_direct_write_reschedule_io(struct nfs_pgio_header *hdr) { struct nfs_direct_req *dreq = hdr->dreq; + struct nfs_page *req; + struct nfs_commit_info cinfo; trace_nfs_direct_write_reschedule_io(dreq); + nfs_init_cinfo_from_dreq(&cinfo, dreq); spin_lock(&dreq->lock); - if (dreq->error == 0) { + if (dreq->error == 0) dreq->flags = NFS_ODIRECT_RESCHED_WRITES; - /* fake unstable write to let common nfs resend pages */ - hdr->verf.committed = NFS_UNSTABLE; - hdr->good_bytes = hdr->args.offset + hdr->args.count - - hdr->io_start; - } + set_bit(NFS_IOHDR_REDO, &hdr->flags); spin_unlock(&dreq->lock); + while (!list_empty(&hdr->pages)) { + req = nfs_list_entry(hdr->pages.next); + nfs_list_remove_request(req); + nfs_unlock_request(req); + nfs_mark_request_commit(req, NULL, &cinfo, 0); + } } static const struct nfs_pgio_completion_ops nfs_direct_write_completion_ops = { @@ -794,9 +824,11 @@ static ssize_t nfs_direct_write_schedule_iovec(struct nfs_direct_req *dreq, { struct nfs_pageio_descriptor desc; struct inode *inode = dreq->inode; + struct nfs_commit_info cinfo; ssize_t result = 0; size_t requested_bytes = 0; size_t wsize = max_t(size_t, NFS_SERVER(inode)->wsize, PAGE_SIZE); + bool defer = false; trace_nfs_direct_write_schedule_iovec(dreq); @@ -837,17 +869,37 @@ static ssize_t nfs_direct_write_schedule_iovec(struct nfs_direct_req *dreq, break; } - nfs_lock_request(req); - if (!nfs_pageio_add_request(&desc, req)) { - result = desc.pg_error; - nfs_unlock_and_release_request(req); - break; - } pgbase = 0; bytes -= req_len; requested_bytes += req_len; pos += req_len; dreq->bytes_left -= req_len; + + if (defer) { + nfs_mark_request_commit(req, NULL, &cinfo, 0); + continue; + } + + nfs_lock_request(req); + if (nfs_pageio_add_request(&desc, req)) + continue; + + /* Exit on hard errors */ + if (desc.pg_error < 0 && desc.pg_error != -EAGAIN) { + result = desc.pg_error; + nfs_unlock_and_release_request(req); + break; + } + + /* If the error is soft, defer remaining requests */ + nfs_init_cinfo_from_dreq(&cinfo, dreq); + spin_lock(&dreq->lock); + dreq->flags = NFS_ODIRECT_RESCHED_WRITES; + spin_unlock(&dreq->lock); + nfs_unlock_request(req); + nfs_mark_request_commit(req, NULL, &cinfo, 0); + desc.pg_error = 0; + defer = true; } nfs_direct_release_pages(pagevec, npages); kvfree(pagevec); diff --git a/fs/nfs/flexfilelayout/flexfilelayout.c b/fs/nfs/flexfilelayout/flexfilelayout.c index 7deb3cd76abe..ef817a0475ff 100644 --- a/fs/nfs/flexfilelayout/flexfilelayout.c +++ b/fs/nfs/flexfilelayout/flexfilelayout.c @@ -1235,6 +1235,7 @@ static void ff_layout_io_track_ds_error(struct pnfs_layout_segment *lseg, case -EPFNOSUPPORT: case -EPROTONOSUPPORT: case -EOPNOTSUPP: + case -EINVAL: case -ECONNREFUSED: case -ECONNRESET: case -EHOSTDOWN: @@ -2519,9 +2520,9 @@ ff_layout_mirror_prepare_stats(struct pnfs_layout_hdr *lo, return i; } -static int -ff_layout_prepare_layoutstats(struct nfs42_layoutstat_args *args) +static int ff_layout_prepare_layoutstats(struct nfs42_layoutstat_args *args) { + struct pnfs_layout_hdr *lo; struct nfs4_flexfile_layout *ff_layout; const int dev_count = PNFS_LAYOUTSTATS_MAXDEV; @@ -2532,11 +2533,14 @@ ff_layout_prepare_layoutstats(struct nfs42_layoutstat_args *args) return -ENOMEM; spin_lock(&args->inode->i_lock); - ff_layout = FF_LAYOUT_FROM_HDR(NFS_I(args->inode)->layout); - args->num_dev = ff_layout_mirror_prepare_stats(&ff_layout->generic_hdr, - &args->devinfo[0], - dev_count, - NFS4_FF_OP_LAYOUTSTATS); + lo = NFS_I(args->inode)->layout; + if (lo && pnfs_layout_is_valid(lo)) { + ff_layout = FF_LAYOUT_FROM_HDR(lo); + args->num_dev = ff_layout_mirror_prepare_stats( + &ff_layout->generic_hdr, &args->devinfo[0], dev_count, + NFS4_FF_OP_LAYOUTSTATS); + } else + args->num_dev = 0; spin_unlock(&args->inode->i_lock); if (!args->num_dev) { kfree(args->devinfo); diff --git a/fs/nfs/nfs42proc.c b/fs/nfs/nfs42proc.c index 063e00aff87e..28704f924612 100644 --- a/fs/nfs/nfs42proc.c +++ b/fs/nfs/nfs42proc.c @@ -81,7 +81,8 @@ static int _nfs42_proc_fallocate(struct rpc_message *msg, struct file *filep, if (status == 0) { if (nfs_should_remove_suid(inode)) { spin_lock(&inode->i_lock); - nfs_set_cache_invalid(inode, NFS_INO_INVALID_MODE); + nfs_set_cache_invalid(inode, + NFS_INO_REVAL_FORCED | NFS_INO_INVALID_MODE); spin_unlock(&inode->i_lock); } status = nfs_post_op_update_inode_force_wcc(inode, diff --git a/fs/nfs/nfs4client.c b/fs/nfs/nfs4client.c index 27fb25567ce7..11e3a285594c 100644 --- a/fs/nfs/nfs4client.c +++ b/fs/nfs/nfs4client.c @@ -417,6 +417,8 @@ static void nfs4_add_trunk(struct nfs_client *clp, struct nfs_client *old) .net = old->cl_net, .servername = old->cl_hostname, }; + int max_connect = test_bit(NFS_CS_PNFS, &clp->cl_flags) ? + clp->cl_max_connect : old->cl_max_connect; if (clp->cl_proto != old->cl_proto) return; @@ -430,7 +432,7 @@ static void nfs4_add_trunk(struct nfs_client *clp, struct nfs_client *old) xprt_args.addrlen = clp_salen; rpc_clnt_add_xprt(old->cl_rpcclient, &xprt_args, - rpc_clnt_test_and_add_xprt, NULL); + rpc_clnt_test_and_add_xprt, &max_connect); } /** @@ -1010,6 +1012,8 @@ struct nfs_client *nfs4_set_ds_client(struct nfs_server *mds_srv, __set_bit(NFS_CS_NORESVPORT, &cl_init.init_flags); __set_bit(NFS_CS_DS, &cl_init.init_flags); + __set_bit(NFS_CS_PNFS, &cl_init.init_flags); + cl_init.max_connect = NFS_MAX_TRANSPORTS; /* * Set an authflavor equual to the MDS value. Use the MDS nfs_client * cl_ipaddr so as to use the same EXCHANGE_ID co_ownerid as the MDS diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c index 794343790ea8..5ee283eb9660 100644 --- a/fs/nfs/nfs4proc.c +++ b/fs/nfs/nfs4proc.c @@ -2703,8 +2703,12 @@ static int _nfs4_proc_open(struct nfs4_opendata *data, return status; } if (!(o_res->f_attr->valid & NFS_ATTR_FATTR)) { + struct nfs_fh *fh = &o_res->fh; + nfs4_sequence_free_slot(&o_res->seq_res); - nfs4_proc_getattr(server, &o_res->fh, o_res->f_attr, NULL); + if (o_arg->claim == NFS4_OPEN_CLAIM_FH) + fh = NFS_FH(d_inode(data->dentry)); + nfs4_proc_getattr(server, fh, o_res->f_attr, NULL); } return 0; } @@ -8866,8 +8870,6 @@ static int _nfs4_proc_exchange_id(struct nfs_client *clp, const struct cred *cre /* Save the EXCHANGE_ID verifier session trunk tests */ memcpy(clp->cl_confirm.data, argp->verifier.data, sizeof(clp->cl_confirm.data)); - if (resp->flags & EXCHGID4_FLAG_USE_PNFS_DS) - set_bit(NFS_CS_DS, &clp->cl_flags); out: trace_nfs4_exchange_id(clp, status); rpc_put_task(task); @@ -10618,7 +10620,9 @@ static void nfs4_disable_swap(struct inode *inode) */ struct nfs_client *clp = NFS_SERVER(inode)->nfs_client; - nfs4_schedule_state_manager(clp); + set_bit(NFS4CLNT_RUN_MANAGER, &clp->cl_state); + clear_bit(NFS4CLNT_MANAGER_AVAILABLE, &clp->cl_state); + wake_up_var(&clp->cl_state); } static const struct inode_operations nfs4_dir_inode_operations = { diff --git a/fs/nfs/nfs4state.c b/fs/nfs/nfs4state.c index e079987af4a3..9a5d911a7edc 100644 --- a/fs/nfs/nfs4state.c +++ b/fs/nfs/nfs4state.c @@ -1209,16 +1209,26 @@ void nfs4_schedule_state_manager(struct nfs_client *clp) { struct task_struct *task; char buf[INET6_ADDRSTRLEN + sizeof("-manager") + 1]; + struct rpc_clnt *clnt = clp->cl_rpcclient; + bool swapon = false; - if (clp->cl_rpcclient->cl_shutdown) + if (clnt->cl_shutdown) return; set_bit(NFS4CLNT_RUN_MANAGER, &clp->cl_state); - if (test_and_set_bit(NFS4CLNT_MANAGER_AVAILABLE, &clp->cl_state) != 0) { - wake_up_var(&clp->cl_state); - return; + + if (atomic_read(&clnt->cl_swapper)) { + swapon = !test_and_set_bit(NFS4CLNT_MANAGER_AVAILABLE, + &clp->cl_state); + if (!swapon) { + wake_up_var(&clp->cl_state); + return; + } } - set_bit(NFS4CLNT_MANAGER_RUNNING, &clp->cl_state); + + if (test_and_set_bit(NFS4CLNT_MANAGER_RUNNING, &clp->cl_state) != 0) + return; + __module_get(THIS_MODULE); refcount_inc(&clp->cl_count); @@ -1235,8 +1245,9 @@ void nfs4_schedule_state_manager(struct nfs_client *clp) __func__, PTR_ERR(task)); if (!nfs_client_init_is_complete(clp)) nfs_mark_client_ready(clp, PTR_ERR(task)); + if (swapon) + clear_bit(NFS4CLNT_MANAGER_AVAILABLE, &clp->cl_state); nfs4_clear_state_manager_bit(clp); - clear_bit(NFS4CLNT_MANAGER_AVAILABLE, &clp->cl_state); nfs_put_client(clp); module_put(THIS_MODULE); } @@ -2703,6 +2714,13 @@ static void nfs4_state_manager(struct nfs_client *clp) nfs4_end_drain_session(clp); nfs4_clear_state_manager_bit(clp); + if (test_bit(NFS4CLNT_RUN_MANAGER, &clp->cl_state) && + !test_and_set_bit(NFS4CLNT_MANAGER_RUNNING, + &clp->cl_state)) { + memflags = memalloc_nofs_save(); + continue; + } + if (!test_and_set_bit(NFS4CLNT_RECALL_RUNNING, &clp->cl_state)) { if (test_and_clear_bit(NFS4CLNT_DELEGRETURN, &clp->cl_state)) { nfs_client_return_marked_delegations(clp); @@ -2741,22 +2759,25 @@ static int nfs4_run_state_manager(void *ptr) allow_signal(SIGKILL); again: - set_bit(NFS4CLNT_MANAGER_RUNNING, &clp->cl_state); nfs4_state_manager(clp); - if (atomic_read(&cl->cl_swapper)) { + + if (test_bit(NFS4CLNT_MANAGER_AVAILABLE, &clp->cl_state) && + !test_bit(NFS4CLNT_MANAGER_RUNNING, &clp->cl_state)) { wait_var_event_interruptible(&clp->cl_state, test_bit(NFS4CLNT_RUN_MANAGER, &clp->cl_state)); - if (atomic_read(&cl->cl_swapper) && - test_bit(NFS4CLNT_RUN_MANAGER, &clp->cl_state)) + if (!atomic_read(&cl->cl_swapper)) + clear_bit(NFS4CLNT_MANAGER_AVAILABLE, &clp->cl_state); + if (refcount_read(&clp->cl_count) > 1 && !signalled() && + !test_and_set_bit(NFS4CLNT_MANAGER_RUNNING, &clp->cl_state)) goto again; /* Either no longer a swapper, or were signalled */ + clear_bit(NFS4CLNT_MANAGER_AVAILABLE, &clp->cl_state); } - clear_bit(NFS4CLNT_MANAGER_AVAILABLE, &clp->cl_state); if (refcount_read(&clp->cl_count) > 1 && !signalled() && test_bit(NFS4CLNT_RUN_MANAGER, &clp->cl_state) && - !test_and_set_bit(NFS4CLNT_MANAGER_AVAILABLE, &clp->cl_state)) + !test_and_set_bit(NFS4CLNT_MANAGER_RUNNING, &clp->cl_state)) goto again; nfs_put_client(clp); diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c index 306cba0b9e69..84343aefbbd6 100644 --- a/fs/nfs/pnfs.c +++ b/fs/nfs/pnfs.c @@ -2634,31 +2634,44 @@ pnfs_should_return_unused_layout(struct pnfs_layout_hdr *lo, return mode == 0; } -static int -pnfs_layout_return_unused_byserver(struct nfs_server *server, void *data) +static int pnfs_layout_return_unused_byserver(struct nfs_server *server, + void *data) { const struct pnfs_layout_range *range = data; + const struct cred *cred; struct pnfs_layout_hdr *lo; struct inode *inode; + nfs4_stateid stateid; + enum pnfs_iomode iomode; + restart: rcu_read_lock(); list_for_each_entry_rcu(lo, &server->layouts, plh_layouts) { - if (!pnfs_layout_can_be_returned(lo) || + inode = lo->plh_inode; + if (!inode || !pnfs_layout_can_be_returned(lo) || test_bit(NFS_LAYOUT_RETURN_REQUESTED, &lo->plh_flags)) continue; - inode = lo->plh_inode; spin_lock(&inode->i_lock); - if (!pnfs_should_return_unused_layout(lo, range)) { + if (!lo->plh_inode || + !pnfs_should_return_unused_layout(lo, range)) { spin_unlock(&inode->i_lock); continue; } + pnfs_get_layout_hdr(lo); + pnfs_set_plh_return_info(lo, range->iomode, 0); + if (pnfs_mark_matching_lsegs_return(lo, &lo->plh_return_segs, + range, 0) != 0 || + !pnfs_prepare_layoutreturn(lo, &stateid, &cred, &iomode)) { + spin_unlock(&inode->i_lock); + rcu_read_unlock(); + pnfs_put_layout_hdr(lo); + cond_resched(); + goto restart; + } spin_unlock(&inode->i_lock); - inode = pnfs_grab_inode_layout_hdr(lo); - if (!inode) - continue; rcu_read_unlock(); - pnfs_mark_layout_for_return(inode, range); - iput(inode); + pnfs_send_layoutreturn(lo, &stateid, &cred, iomode, false); + pnfs_put_layout_hdr(lo); cond_resched(); goto restart; } diff --git a/fs/nfs/write.c b/fs/nfs/write.c index f4cca8f00c0c..9d82d50ce0b1 100644 --- a/fs/nfs/write.c +++ b/fs/nfs/write.c @@ -59,7 +59,8 @@ static const struct nfs_pgio_completion_ops nfs_async_write_completion_ops; static const struct nfs_commit_completion_ops nfs_commit_completion_ops; static const struct nfs_rw_ops nfs_rw_write_ops; static void nfs_inode_remove_request(struct nfs_page *req); -static void nfs_clear_request_commit(struct nfs_page *req); +static void nfs_clear_request_commit(struct nfs_commit_info *cinfo, + struct nfs_page *req); static void nfs_init_cinfo_from_inode(struct nfs_commit_info *cinfo, struct inode *inode); static struct nfs_page * @@ -502,8 +503,8 @@ nfs_destroy_unlinked_subrequests(struct nfs_page *destroy_list, * the (former) group. All subrequests are removed from any write or commit * lists, unlinked from the group and destroyed. */ -void -nfs_join_page_group(struct nfs_page *head, struct inode *inode) +void nfs_join_page_group(struct nfs_page *head, struct nfs_commit_info *cinfo, + struct inode *inode) { struct nfs_page *subreq; struct nfs_page *destroy_list = NULL; @@ -533,7 +534,7 @@ nfs_join_page_group(struct nfs_page *head, struct inode *inode) * Commit list removal accounting is done after locks are dropped */ subreq = head; do { - nfs_clear_request_commit(subreq); + nfs_clear_request_commit(cinfo, subreq); subreq = subreq->wb_this_page; } while (subreq != head); @@ -566,8 +567,10 @@ static struct nfs_page *nfs_lock_and_join_requests(struct folio *folio) { struct inode *inode = folio_file_mapping(folio)->host; struct nfs_page *head; + struct nfs_commit_info cinfo; int ret; + nfs_init_cinfo_from_inode(&cinfo, inode); /* * A reference is taken only on the head request which acts as a * reference to the whole page group - the group will not be destroyed @@ -584,7 +587,7 @@ static struct nfs_page *nfs_lock_and_join_requests(struct folio *folio) return ERR_PTR(ret); } - nfs_join_page_group(head, inode); + nfs_join_page_group(head, &cinfo, inode); return head; } @@ -785,6 +788,8 @@ static void nfs_inode_add_request(struct nfs_page *req) */ static void nfs_inode_remove_request(struct nfs_page *req) { + struct nfs_inode *nfsi = NFS_I(nfs_page_to_inode(req)); + if (nfs_page_group_sync_on_bit(req, PG_REMOVE)) { struct folio *folio = nfs_page_to_folio(req->wb_head); struct address_space *mapping = folio_file_mapping(folio); @@ -799,8 +804,8 @@ static void nfs_inode_remove_request(struct nfs_page *req) } if (test_and_clear_bit(PG_INODE_REF, &req->wb_flags)) { + atomic_long_dec(&nfsi->nrequests); nfs_release_request(req); - atomic_long_dec(&NFS_I(nfs_page_to_inode(req))->nrequests); } } @@ -955,18 +960,16 @@ static void nfs_folio_clear_commit(struct folio *folio) } /* Called holding the request lock on @req */ -static void -nfs_clear_request_commit(struct nfs_page *req) +static void nfs_clear_request_commit(struct nfs_commit_info *cinfo, + struct nfs_page *req) { if (test_bit(PG_CLEAN, &req->wb_flags)) { struct nfs_open_context *ctx = nfs_req_openctx(req); struct inode *inode = d_inode(ctx->dentry); - struct nfs_commit_info cinfo; - nfs_init_cinfo_from_inode(&cinfo, inode); mutex_lock(&NFS_I(inode)->commit_mutex); - if (!pnfs_clear_request_commit(req, &cinfo)) { - nfs_request_remove_commit_list(req, &cinfo); + if (!pnfs_clear_request_commit(req, cinfo)) { + nfs_request_remove_commit_list(req, cinfo); } mutex_unlock(&NFS_I(inode)->commit_mutex); nfs_folio_clear_commit(nfs_page_to_folio(req)); diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c index 5ca748309c26..4199ede0583c 100644 --- a/fs/nfsd/nfs4proc.c +++ b/fs/nfsd/nfs4proc.c @@ -1058,8 +1058,8 @@ nfsd4_rename(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, rename->rn_tname, rename->rn_tnamelen); if (status) return status; - set_change_info(&rename->rn_sinfo, &cstate->current_fh); - set_change_info(&rename->rn_tinfo, &cstate->save_fh); + set_change_info(&rename->rn_sinfo, &cstate->save_fh); + set_change_info(&rename->rn_tinfo, &cstate->current_fh); return nfs_ok; } diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c index 2e40c74d2f72..92c7dde148a4 100644 --- a/fs/nfsd/nfs4xdr.c +++ b/fs/nfsd/nfs4xdr.c @@ -4113,6 +4113,7 @@ static __be32 nfsd4_encode_readv(struct nfsd4_compoundres *resp, struct file *file, unsigned long maxcount) { struct xdr_stream *xdr = resp->xdr; + unsigned int base = xdr->buf->page_len & ~PAGE_MASK; unsigned int starting_len = xdr->buf->len; __be32 zero = xdr_zero; __be32 nfserr; @@ -4121,8 +4122,7 @@ static __be32 nfsd4_encode_readv(struct nfsd4_compoundres *resp, return nfserr_resource; nfserr = nfsd_iter_read(resp->rqstp, read->rd_fhp, file, - read->rd_offset, &maxcount, - xdr->buf->page_len & ~PAGE_MASK, + read->rd_offset, &maxcount, base, &read->rd_eof); read->rd_length = maxcount; if (nfserr) diff --git a/fs/nfsd/nfssvc.c b/fs/nfsd/nfssvc.c index 1582af33e204..c7af1095f6b5 100644 --- a/fs/nfsd/nfssvc.c +++ b/fs/nfsd/nfssvc.c @@ -1082,11 +1082,12 @@ int nfsd_pool_stats_open(struct inode *inode, struct file *file) int nfsd_pool_stats_release(struct inode *inode, struct file *file) { + struct seq_file *seq = file->private_data; + struct svc_serv *serv = seq->private; int ret = seq_release(inode, file); - struct net *net = inode->i_sb->s_fs_info; mutex_lock(&nfsd_mutex); - nfsd_put(net); + svc_put(serv); mutex_unlock(&nfsd_mutex); return ret; } diff --git a/fs/nilfs2/gcinode.c b/fs/nilfs2/gcinode.c index 48fe71d309cb..8beb2730929d 100644 --- a/fs/nilfs2/gcinode.c +++ b/fs/nilfs2/gcinode.c @@ -73,10 +73,8 @@ int nilfs_gccache_submit_read_data(struct inode *inode, sector_t blkoff, struct the_nilfs *nilfs = inode->i_sb->s_fs_info; err = nilfs_dat_translate(nilfs->ns_dat, vbn, &pbn); - if (unlikely(err)) { /* -EIO, -ENOMEM, -ENOENT */ - brelse(bh); + if (unlikely(err)) /* -EIO, -ENOMEM, -ENOENT */ goto failed; - } } lock_buffer(bh); @@ -102,6 +100,8 @@ int nilfs_gccache_submit_read_data(struct inode *inode, sector_t blkoff, failed: unlock_page(bh->b_page); put_page(bh->b_page); + if (unlikely(err)) + brelse(bh); return err; } diff --git a/fs/notify/fanotify/fanotify_user.c b/fs/notify/fanotify/fanotify_user.c index f69c451018e3..62fe0b679e58 100644 --- a/fs/notify/fanotify/fanotify_user.c +++ b/fs/notify/fanotify/fanotify_user.c @@ -1585,16 +1585,25 @@ static int fanotify_test_fsid(struct dentry *dentry, __kernel_fsid_t *fsid) } /* Check if filesystem can encode a unique fid */ -static int fanotify_test_fid(struct dentry *dentry) +static int fanotify_test_fid(struct dentry *dentry, unsigned int flags) { + unsigned int mark_type = flags & FANOTIFY_MARK_TYPE_BITS; + const struct export_operations *nop = dentry->d_sb->s_export_op; + + /* + * We need to make sure that the filesystem supports encoding of + * file handles so user can use name_to_handle_at() to compare fids + * reported with events to the file handle of watched objects. + */ + if (!nop) + return -EOPNOTSUPP; + /* - * We need to make sure that the file system supports at least - * encoding a file handle so user can use name_to_handle_at() to - * compare fid returned with event to the file handle of watched - * objects. However, even the relaxed AT_HANDLE_FID flag requires - * at least empty export_operations for ecoding unique file ids. + * For sb/mount mark, we also need to make sure that the filesystem + * supports decoding file handles, so user has a way to map back the + * reported fids to filesystem objects. */ - if (!dentry->d_sb->s_export_op) + if (mark_type != FAN_MARK_INODE && !nop->fh_to_dentry) return -EOPNOTSUPP; return 0; @@ -1812,7 +1821,7 @@ static int do_fanotify_mark(int fanotify_fd, unsigned int flags, __u64 mask, if (ret) goto path_put_and_out; - ret = fanotify_test_fid(path.dentry); + ret = fanotify_test_fid(path.dentry, flags); if (ret) goto path_put_and_out; diff --git a/fs/ntfs3/attrib.c b/fs/ntfs3/attrib.c index a9d82bbb4729..63f70259edc0 100644 --- a/fs/ntfs3/attrib.c +++ b/fs/ntfs3/attrib.c @@ -1106,10 +1106,10 @@ repack: } } - /* + /* * The code below may require additional cluster (to extend attribute list) - * and / or one MFT record - * It is too complex to undo operations if -ENOSPC occurs deep inside + * and / or one MFT record + * It is too complex to undo operations if -ENOSPC occurs deep inside * in 'ni_insert_nonresident'. * Return in advance -ENOSPC here if there are no free cluster and no free MFT. */ @@ -1736,10 +1736,8 @@ repack: le_b = NULL; attr_b = ni_find_attr(ni, NULL, &le_b, ATTR_DATA, NULL, 0, NULL, &mi_b); - if (!attr_b) { - err = -ENOENT; - goto out; - } + if (!attr_b) + return -ENOENT; attr = attr_b; le = le_b; diff --git a/fs/ntfs3/attrlist.c b/fs/ntfs3/attrlist.c index 42631b31adf1..7c01735d1219 100644 --- a/fs/ntfs3/attrlist.c +++ b/fs/ntfs3/attrlist.c @@ -52,7 +52,8 @@ int ntfs_load_attr_list(struct ntfs_inode *ni, struct ATTRIB *attr) if (!attr->non_res) { lsize = le32_to_cpu(attr->res.data_size); - le = kmalloc(al_aligned(lsize), GFP_NOFS | __GFP_NOWARN); + /* attr is resident: lsize < record_size (1K or 4K) */ + le = kvmalloc(al_aligned(lsize), GFP_KERNEL); if (!le) { err = -ENOMEM; goto out; @@ -80,7 +81,17 @@ int ntfs_load_attr_list(struct ntfs_inode *ni, struct ATTRIB *attr) if (err < 0) goto out; - le = kmalloc(al_aligned(lsize), GFP_NOFS | __GFP_NOWARN); + /* attr is nonresident. + * The worst case: + * 1T (2^40) extremely fragmented file. + * cluster = 4K (2^12) => 2^28 fragments + * 2^9 fragments per one record => 2^19 records + * 2^5 bytes of ATTR_LIST_ENTRY per one record => 2^24 bytes. + * + * the result is 16M bytes per attribute list. + * Use kvmalloc to allocate in range [several Kbytes - dozen Mbytes] + */ + le = kvmalloc(al_aligned(lsize), GFP_KERNEL); if (!le) { err = -ENOMEM; goto out; diff --git a/fs/ntfs3/bitmap.c b/fs/ntfs3/bitmap.c index 107e808e06ea..63f14a0232f6 100644 --- a/fs/ntfs3/bitmap.c +++ b/fs/ntfs3/bitmap.c @@ -125,6 +125,7 @@ void wnd_close(struct wnd_bitmap *wnd) struct rb_node *node, *next; kfree(wnd->free_bits); + wnd->free_bits = NULL; run_close(&wnd->run); node = rb_first(&wnd->start_tree); @@ -659,7 +660,8 @@ int wnd_init(struct wnd_bitmap *wnd, struct super_block *sb, size_t nbits) wnd->bits_last = wbits; wnd->free_bits = - kcalloc(wnd->nwnd, sizeof(u16), GFP_NOFS | __GFP_NOWARN); + kvmalloc_array(wnd->nwnd, sizeof(u16), GFP_KERNEL | __GFP_ZERO); + if (!wnd->free_bits) return -ENOMEM; diff --git a/fs/ntfs3/dir.c b/fs/ntfs3/dir.c index 063a6654199b..ec0566b322d5 100644 --- a/fs/ntfs3/dir.c +++ b/fs/ntfs3/dir.c @@ -309,7 +309,11 @@ static inline int ntfs_filldir(struct ntfs_sb_info *sbi, struct ntfs_inode *ni, return 0; } - dt_type = (fname->dup.fa & FILE_ATTRIBUTE_DIRECTORY) ? DT_DIR : DT_REG; + /* NTFS: symlinks are "dir + reparse" or "file + reparse" */ + if (fname->dup.fa & FILE_ATTRIBUTE_REPARSE_POINT) + dt_type = DT_LNK; + else + dt_type = (fname->dup.fa & FILE_ATTRIBUTE_DIRECTORY) ? DT_DIR : DT_REG; return !dir_emit(ctx, (s8 *)name, name_len, ino, dt_type); } diff --git a/fs/ntfs3/file.c b/fs/ntfs3/file.c index 962f12ce6c0a..1f7a194983c5 100644 --- a/fs/ntfs3/file.c +++ b/fs/ntfs3/file.c @@ -745,8 +745,8 @@ static ssize_t ntfs_file_read_iter(struct kiocb *iocb, struct iov_iter *iter) } static ssize_t ntfs_file_splice_read(struct file *in, loff_t *ppos, - struct pipe_inode_info *pipe, - size_t len, unsigned int flags) + struct pipe_inode_info *pipe, size_t len, + unsigned int flags) { struct inode *inode = in->f_mapping->host; struct ntfs_inode *ni = ntfs_i(inode); diff --git a/fs/ntfs3/frecord.c b/fs/ntfs3/frecord.c index 2b85cb10f0be..dad976a68985 100644 --- a/fs/ntfs3/frecord.c +++ b/fs/ntfs3/frecord.c @@ -2148,7 +2148,7 @@ out1: for (i = 0; i < pages_per_frame; i++) { pg = pages[i]; - if (i == idx) + if (i == idx || !pg) continue; unlock_page(pg); put_page(pg); @@ -3208,6 +3208,12 @@ static bool ni_update_parent(struct ntfs_inode *ni, struct NTFS_DUP_INFO *dup, if (!fname || !memcmp(&fname->dup, dup, sizeof(fname->dup))) continue; + /* Check simple case when parent inode equals current inode. */ + if (ino_get(&fname->home) == ni->vfs_inode.i_ino) { + ntfs_set_state(sbi, NTFS_DIRTY_ERROR); + continue; + } + /* ntfs_iget5 may sleep. */ dir = ntfs_iget5(sb, &fname->home, NULL); if (IS_ERR(dir)) { diff --git a/fs/ntfs3/fslog.c b/fs/ntfs3/fslog.c index 12f28cdf5c83..98ccb6650858 100644 --- a/fs/ntfs3/fslog.c +++ b/fs/ntfs3/fslog.c @@ -2168,8 +2168,10 @@ file_is_valid: if (!page) { page = kmalloc(log->page_size, GFP_NOFS); - if (!page) - return -ENOMEM; + if (!page) { + err = -ENOMEM; + goto out; + } } /* diff --git a/fs/ntfs3/fsntfs.c b/fs/ntfs3/fsntfs.c index 33afee0f5559..fbfe21dbb425 100644 --- a/fs/ntfs3/fsntfs.c +++ b/fs/ntfs3/fsntfs.c @@ -983,18 +983,11 @@ out: if (err) return err; - mark_inode_dirty(&ni->vfs_inode); + mark_inode_dirty_sync(&ni->vfs_inode); /* verify(!ntfs_update_mftmirr()); */ - /* - * If we used wait=1, sync_inode_metadata waits for the io for the - * inode to finish. It hangs when media is removed. - * So wait=0 is sent down to sync_inode_metadata - * and filemap_fdatawrite is used for the data blocks. - */ - err = sync_inode_metadata(&ni->vfs_inode, 0); - if (!err) - err = filemap_fdatawrite(ni->vfs_inode.i_mapping); + /* write mft record on disk. */ + err = _ni_write_inode(&ni->vfs_inode, 1); return err; } @@ -2461,10 +2454,12 @@ void mark_as_free_ex(struct ntfs_sb_info *sbi, CLST lcn, CLST len, bool trim) { CLST end, i, zone_len, zlen; struct wnd_bitmap *wnd = &sbi->used.bitmap; + bool dirty = false; down_write_nested(&wnd->rw_lock, BITMAP_MUTEX_CLUSTERS); if (!wnd_is_used(wnd, lcn, len)) { - ntfs_set_state(sbi, NTFS_DIRTY_ERROR); + /* mark volume as dirty out of wnd->rw_lock */ + dirty = true; end = lcn + len; len = 0; @@ -2518,6 +2513,8 @@ void mark_as_free_ex(struct ntfs_sb_info *sbi, CLST lcn, CLST len, bool trim) out: up_write(&wnd->rw_lock); + if (dirty) + ntfs_set_state(sbi, NTFS_DIRTY_ERROR); } /* diff --git a/fs/ntfs3/index.c b/fs/ntfs3/index.c index 124c6e822623..cf92b2433f7a 100644 --- a/fs/ntfs3/index.c +++ b/fs/ntfs3/index.c @@ -729,6 +729,9 @@ static struct NTFS_DE *hdr_find_e(const struct ntfs_index *indx, u32 total = le32_to_cpu(hdr->total); u16 offs[128]; + if (unlikely(!cmp)) + return NULL; + fill_table: if (end > total) return NULL; diff --git a/fs/ntfs3/inode.c b/fs/ntfs3/inode.c index eb2ed0701495..d6d021e19aaa 100644 --- a/fs/ntfs3/inode.c +++ b/fs/ntfs3/inode.c @@ -170,8 +170,8 @@ next_attr: nt2kernel(std5->cr_time, &ni->i_crtime); #endif nt2kernel(std5->a_time, &inode->i_atime); - ctime = inode_get_ctime(inode); nt2kernel(std5->c_time, &ctime); + inode_set_ctime_to_ts(inode, ctime); nt2kernel(std5->m_time, &inode->i_mtime); ni->std_fa = std5->fa; @@ -1660,7 +1660,8 @@ struct inode *ntfs_create_inode(struct mnt_idmap *idmap, struct inode *dir, d_instantiate(dentry, inode); /* Set original time. inode times (i_ctime) may be changed in ntfs_init_acl. */ - inode->i_atime = inode->i_mtime = inode_set_ctime_to_ts(inode, ni->i_crtime); + inode->i_atime = inode->i_mtime = + inode_set_ctime_to_ts(inode, ni->i_crtime); dir->i_mtime = inode_set_ctime_to_ts(dir, ni->i_crtime); mark_inode_dirty(dir); diff --git a/fs/ntfs3/namei.c b/fs/ntfs3/namei.c index ad430d50bd79..eedacf94edd8 100644 --- a/fs/ntfs3/namei.c +++ b/fs/ntfs3/namei.c @@ -156,8 +156,8 @@ static int ntfs_link(struct dentry *ode, struct inode *dir, struct dentry *de) err = ntfs_link_inode(inode, de); if (!err) { - dir->i_mtime = inode_set_ctime_to_ts(inode, - inode_set_ctime_current(dir)); + dir->i_mtime = inode_set_ctime_to_ts( + inode, inode_set_ctime_current(dir)); mark_inode_dirty(inode); mark_inode_dirty(dir); d_instantiate(de, inode); @@ -373,7 +373,7 @@ static int ntfs_atomic_open(struct inode *dir, struct dentry *dentry, #ifdef CONFIG_NTFS3_FS_POSIX_ACL if (IS_POSIXACL(dir)) { - /* + /* * Load in cache current acl to avoid ni_lock(dir): * ntfs_create_inode -> ntfs_init_acl -> posix_acl_create -> * ntfs_get_acl -> ntfs_get_acl_ex -> ni_lock diff --git a/fs/ntfs3/ntfs.h b/fs/ntfs3/ntfs.h index 98b76d1b09e7..86aecbb01a92 100644 --- a/fs/ntfs3/ntfs.h +++ b/fs/ntfs3/ntfs.h @@ -847,7 +847,7 @@ struct OBJECT_ID { // Birth Volume Id is the Object Id of the Volume on. // which the Object Id was allocated. It never changes. struct GUID BirthVolumeId; //0x10: - + // Birth Object Id is the first Object Id that was // ever assigned to this MFT Record. I.e. If the Object Id // is changed for some reason, this field will reflect the diff --git a/fs/ntfs3/ntfs_fs.h b/fs/ntfs3/ntfs_fs.h index 629403ede6e5..0e6a2777870c 100644 --- a/fs/ntfs3/ntfs_fs.h +++ b/fs/ntfs3/ntfs_fs.h @@ -42,9 +42,11 @@ enum utf16_endian; #define MINUS_ONE_T ((size_t)(-1)) /* Biggest MFT / smallest cluster */ #define MAXIMUM_BYTES_PER_MFT 4096 +#define MAXIMUM_SHIFT_BYTES_PER_MFT 12 #define NTFS_BLOCKS_PER_MFT_RECORD (MAXIMUM_BYTES_PER_MFT / 512) #define MAXIMUM_BYTES_PER_INDEX 4096 +#define MAXIMUM_SHIFT_BYTES_PER_INDEX 12 #define NTFS_BLOCKS_PER_INODE (MAXIMUM_BYTES_PER_INDEX / 512) /* NTFS specific error code when fixup failed. */ @@ -495,8 +497,6 @@ int ntfs_getattr(struct mnt_idmap *idmap, const struct path *path, struct kstat *stat, u32 request_mask, u32 flags); int ntfs3_setattr(struct mnt_idmap *idmap, struct dentry *dentry, struct iattr *attr); -void ntfs_sparse_cluster(struct inode *inode, struct page *page0, CLST vcn, - CLST len); int ntfs_file_open(struct inode *inode, struct file *file); int ntfs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo, __u64 start, __u64 len); diff --git a/fs/ntfs3/record.c b/fs/ntfs3/record.c index c12ebffc94da..53629b1f65e9 100644 --- a/fs/ntfs3/record.c +++ b/fs/ntfs3/record.c @@ -189,12 +189,19 @@ out: return err; } +/* + * mi_enum_attr - start/continue attributes enumeration in record. + * + * NOTE: mi->mrec - memory of size sbi->record_size + * here we sure that mi->mrec->total == sbi->record_size (see mi_read) + */ struct ATTRIB *mi_enum_attr(struct mft_inode *mi, struct ATTRIB *attr) { const struct MFT_REC *rec = mi->mrec; u32 used = le32_to_cpu(rec->used); - u32 t32, off, asize; + u32 t32, off, asize, prev_type; u16 t16; + u64 data_size, alloc_size, tot_size; if (!attr) { u32 total = le32_to_cpu(rec->total); @@ -213,6 +220,7 @@ struct ATTRIB *mi_enum_attr(struct mft_inode *mi, struct ATTRIB *attr) if (!is_rec_inuse(rec)) return NULL; + prev_type = 0; attr = Add2Ptr(rec, off); } else { /* Check if input attr inside record. */ @@ -226,11 +234,11 @@ struct ATTRIB *mi_enum_attr(struct mft_inode *mi, struct ATTRIB *attr) return NULL; } - if (off + asize < off) { - /* Overflow check. */ + /* Overflow check. */ + if (off + asize < off) return NULL; - } + prev_type = le32_to_cpu(attr->type); attr = Add2Ptr(attr, asize); off += asize; } @@ -250,7 +258,11 @@ struct ATTRIB *mi_enum_attr(struct mft_inode *mi, struct ATTRIB *attr) /* 0x100 is last known attribute for now. */ t32 = le32_to_cpu(attr->type); - if ((t32 & 0xf) || (t32 > 0x100)) + if (!t32 || (t32 & 0xf) || (t32 > 0x100)) + return NULL; + + /* attributes in record must be ordered by type */ + if (t32 < prev_type) return NULL; /* Check overflow and boundary. */ @@ -259,16 +271,15 @@ struct ATTRIB *mi_enum_attr(struct mft_inode *mi, struct ATTRIB *attr) /* Check size of attribute. */ if (!attr->non_res) { + /* Check resident fields. */ if (asize < SIZEOF_RESIDENT) return NULL; t16 = le16_to_cpu(attr->res.data_off); - if (t16 > asize) return NULL; - t32 = le32_to_cpu(attr->res.data_size); - if (t16 + t32 > asize) + if (t16 + le32_to_cpu(attr->res.data_size) > asize) return NULL; t32 = sizeof(short) * attr->name_len; @@ -278,21 +289,52 @@ struct ATTRIB *mi_enum_attr(struct mft_inode *mi, struct ATTRIB *attr) return attr; } - /* Check some nonresident fields. */ - if (attr->name_len && - le16_to_cpu(attr->name_off) + sizeof(short) * attr->name_len > - le16_to_cpu(attr->nres.run_off)) { + /* Check nonresident fields. */ + if (attr->non_res != 1) + return NULL; + + t16 = le16_to_cpu(attr->nres.run_off); + if (t16 > asize) + return NULL; + + t32 = sizeof(short) * attr->name_len; + if (t32 && le16_to_cpu(attr->name_off) + t32 > t16) + return NULL; + + /* Check start/end vcn. */ + if (le64_to_cpu(attr->nres.svcn) > le64_to_cpu(attr->nres.evcn) + 1) return NULL; - } - if (attr->nres.svcn || !is_attr_ext(attr)) { + data_size = le64_to_cpu(attr->nres.data_size); + if (le64_to_cpu(attr->nres.valid_size) > data_size) + return NULL; + + alloc_size = le64_to_cpu(attr->nres.alloc_size); + if (data_size > alloc_size) + return NULL; + + t32 = mi->sbi->cluster_mask; + if (alloc_size & t32) + return NULL; + + if (!attr->nres.svcn && is_attr_ext(attr)) { + /* First segment of sparse/compressed attribute */ + if (asize + 8 < SIZEOF_NONRESIDENT_EX) + return NULL; + + tot_size = le64_to_cpu(attr->nres.total_size); + if (tot_size & t32) + return NULL; + + if (tot_size > alloc_size) + return NULL; + } else { if (asize + 8 < SIZEOF_NONRESIDENT) return NULL; if (attr->nres.c_unit) return NULL; - } else if (asize + 8 < SIZEOF_NONRESIDENT_EX) - return NULL; + } return attr; } diff --git a/fs/ntfs3/super.c b/fs/ntfs3/super.c index cfec5e0c7f66..f763e3256ccc 100644 --- a/fs/ntfs3/super.c +++ b/fs/ntfs3/super.c @@ -453,15 +453,23 @@ static struct proc_dir_entry *proc_info_root; * ntfs3.1 * cluster size * number of clusters + * total number of mft records + * number of used mft records ~= number of files + folders + * real state of ntfs "dirty"/"clean" + * current state of ntfs "dirty"/"clean" */ static int ntfs3_volinfo(struct seq_file *m, void *o) { struct super_block *sb = m->private; struct ntfs_sb_info *sbi = sb->s_fs_info; - seq_printf(m, "ntfs%d.%d\n%u\n%zu\n", sbi->volume.major_ver, - sbi->volume.minor_ver, sbi->cluster_size, - sbi->used.bitmap.nbits); + seq_printf(m, "ntfs%d.%d\n%u\n%zu\n\%zu\n%zu\n%s\n%s\n", + sbi->volume.major_ver, sbi->volume.minor_ver, + sbi->cluster_size, sbi->used.bitmap.nbits, + sbi->mft.bitmap.nbits, + sbi->mft.bitmap.nbits - wnd_zeroes(&sbi->mft.bitmap), + sbi->volume.real_dirty ? "dirty" : "clean", + (sbi->volume.flags & VOLUME_FLAG_DIRTY) ? "dirty" : "clean"); return 0; } @@ -488,9 +496,13 @@ static ssize_t ntfs3_label_write(struct file *file, const char __user *buffer, { int err; struct super_block *sb = pde_data(file_inode(file)); - struct ntfs_sb_info *sbi = sb->s_fs_info; ssize_t ret = count; - u8 *label = kmalloc(count, GFP_NOFS); + u8 *label; + + if (sb_rdonly(sb)) + return -EROFS; + + label = kmalloc(count, GFP_NOFS); if (!label) return -ENOMEM; @@ -502,7 +514,7 @@ static ssize_t ntfs3_label_write(struct file *file, const char __user *buffer, while (ret > 0 && label[ret - 1] == '\n') ret -= 1; - err = ntfs_set_label(sbi, label, ret); + err = ntfs_set_label(sb->s_fs_info, label, ret); if (err < 0) { ntfs_err(sb, "failed (%d) to write label", err); @@ -576,20 +588,30 @@ static noinline void ntfs3_put_sbi(struct ntfs_sb_info *sbi) wnd_close(&sbi->mft.bitmap); wnd_close(&sbi->used.bitmap); - if (sbi->mft.ni) + if (sbi->mft.ni) { iput(&sbi->mft.ni->vfs_inode); + sbi->mft.ni = NULL; + } - if (sbi->security.ni) + if (sbi->security.ni) { iput(&sbi->security.ni->vfs_inode); + sbi->security.ni = NULL; + } - if (sbi->reparse.ni) + if (sbi->reparse.ni) { iput(&sbi->reparse.ni->vfs_inode); + sbi->reparse.ni = NULL; + } - if (sbi->objid.ni) + if (sbi->objid.ni) { iput(&sbi->objid.ni->vfs_inode); + sbi->objid.ni = NULL; + } - if (sbi->volume.ni) + if (sbi->volume.ni) { iput(&sbi->volume.ni->vfs_inode); + sbi->volume.ni = NULL; + } ntfs_update_mftmirr(sbi, 0); @@ -836,7 +858,7 @@ static int ntfs_init_from_boot(struct super_block *sb, u32 sector_size, struct ntfs_sb_info *sbi = sb->s_fs_info; int err; u32 mb, gb, boot_sector_size, sct_per_clst, record_size; - u64 sectors, clusters, mlcn, mlcn2; + u64 sectors, clusters, mlcn, mlcn2, dev_size0; struct NTFS_BOOT *boot; struct buffer_head *bh; struct MFT_REC *rec; @@ -845,6 +867,9 @@ static int ntfs_init_from_boot(struct super_block *sb, u32 sector_size, u32 boot_off = 0; const char *hint = "Primary boot"; + /* Save original dev_size. Used with alternative boot. */ + dev_size0 = dev_size; + sbi->volume.blocks = dev_size >> PAGE_SHIFT; bh = ntfs_bread(sb, 0); @@ -853,6 +878,11 @@ static int ntfs_init_from_boot(struct super_block *sb, u32 sector_size, check_boot: err = -EINVAL; + + /* Corrupted image; do not read OOB */ + if (bh->b_size - sizeof(*boot) < boot_off) + goto out; + boot = (struct NTFS_BOOT *)Add2Ptr(bh->b_data, boot_off); if (memcmp(boot->system_id, "NTFS ", sizeof("NTFS ") - 1)) { @@ -899,9 +929,17 @@ check_boot: goto out; } - sbi->record_size = record_size = - boot->record_size < 0 ? 1 << (-boot->record_size) : - (u32)boot->record_size << cluster_bits; + if (boot->record_size >= 0) { + record_size = (u32)boot->record_size << cluster_bits; + } else if (-boot->record_size <= MAXIMUM_SHIFT_BYTES_PER_MFT) { + record_size = 1u << (-boot->record_size); + } else { + ntfs_err(sb, "%s: invalid record size %d.", hint, + boot->record_size); + goto out; + } + + sbi->record_size = record_size; sbi->record_bits = blksize_bits(record_size); sbi->attr_size_tr = (5 * record_size >> 4); // ~320 bytes @@ -918,9 +956,15 @@ check_boot: goto out; } - sbi->index_size = boot->index_size < 0 ? - 1u << (-boot->index_size) : - (u32)boot->index_size << cluster_bits; + if (boot->index_size >= 0) { + sbi->index_size = (u32)boot->index_size << cluster_bits; + } else if (-boot->index_size <= MAXIMUM_SHIFT_BYTES_PER_INDEX) { + sbi->index_size = 1u << (-boot->index_size); + } else { + ntfs_err(sb, "%s: invalid index size %d.", hint, + boot->index_size); + goto out; + } /* Check index record size. */ if (sbi->index_size < SECTOR_SIZE || !is_power_of_2(sbi->index_size)) { @@ -1055,17 +1099,17 @@ check_boot: if (bh->b_blocknr && !sb_rdonly(sb)) { /* - * Alternative boot is ok but primary is not ok. - * Do not update primary boot here 'cause it may be faked boot. - * Let ntfs to be mounted and update boot later. - */ + * Alternative boot is ok but primary is not ok. + * Do not update primary boot here 'cause it may be faked boot. + * Let ntfs to be mounted and update boot later. + */ *boot2 = kmemdup(boot, sizeof(*boot), GFP_NOFS | __GFP_NOWARN); } out: - if (err == -EINVAL && !bh->b_blocknr && dev_size > PAGE_SHIFT) { + if (err == -EINVAL && !bh->b_blocknr && dev_size0 > PAGE_SHIFT) { u32 block_size = min_t(u32, sector_size, PAGE_SIZE); - u64 lbo = dev_size - sizeof(*boot); + u64 lbo = dev_size0 - sizeof(*boot); /* * Try alternative boot (last sector) @@ -1079,6 +1123,7 @@ out: boot_off = lbo & (block_size - 1); hint = "Alternative boot"; + dev_size = dev_size0; /* restore original size. */ goto check_boot; } brelse(bh); @@ -1367,7 +1412,7 @@ static int ntfs_fill_super(struct super_block *sb, struct fs_context *fc) } bytes = inode->i_size; - sbi->def_table = t = kmalloc(bytes, GFP_NOFS | __GFP_NOWARN); + sbi->def_table = t = kvmalloc(bytes, GFP_KERNEL); if (!t) { err = -ENOMEM; goto put_inode_out; @@ -1521,9 +1566,9 @@ load_root: if (boot2) { /* - * Alternative boot is ok but primary is not ok. - * Volume is recognized as NTFS. Update primary boot. - */ + * Alternative boot is ok but primary is not ok. + * Volume is recognized as NTFS. Update primary boot. + */ struct buffer_head *bh0 = sb_getblk(sb, 0); if (bh0) { if (buffer_locked(bh0)) @@ -1562,7 +1607,9 @@ load_root: put_inode_out: iput(inode); out: + ntfs3_put_sbi(sbi); kfree(boot2); + ntfs3_put_sbi(sbi); return err; } @@ -1756,7 +1803,6 @@ static int __init init_ntfs_fs(void) if (IS_ENABLED(CONFIG_NTFS3_LZX_XPRESS)) pr_info("ntfs3: Read-only LZX/Xpress compression included\n"); - #ifdef CONFIG_PROC_FS /* Create "/proc/fs/ntfs3" */ proc_info_root = proc_mkdir("fs/ntfs3", NULL); @@ -1798,7 +1844,6 @@ static void __exit exit_ntfs_fs(void) if (proc_info_root) remove_proc_entry("fs/ntfs3", NULL); #endif - } MODULE_LICENSE("GPL"); diff --git a/fs/ntfs3/xattr.c b/fs/ntfs3/xattr.c index 29fd391899e5..4920548192a0 100644 --- a/fs/ntfs3/xattr.c +++ b/fs/ntfs3/xattr.c @@ -211,7 +211,8 @@ static ssize_t ntfs_list_ea(struct ntfs_inode *ni, char *buffer, size = le32_to_cpu(info->size); /* Enumerate all xattrs. */ - for (ret = 0, off = 0; off < size; off += ea_size) { + ret = 0; + for (off = 0; off + sizeof(struct EA_FULL) < size; off += ea_size) { ea = Add2Ptr(ea_all, off); ea_size = unpacked_ea_size(ea); @@ -219,6 +220,10 @@ static ssize_t ntfs_list_ea(struct ntfs_inode *ni, char *buffer, break; if (buffer) { + /* Check if we can use field ea->name */ + if (off + ea_size > size) + break; + if (ret + ea->name_len + 1 > bytes_per_buffer) { err = -ERANGE; goto out; diff --git a/fs/overlayfs/copy_up.c b/fs/overlayfs/copy_up.c index bae404a1bad4..ada3fcc9c6d5 100644 --- a/fs/overlayfs/copy_up.c +++ b/fs/overlayfs/copy_up.c @@ -337,7 +337,7 @@ static int ovl_set_timestamps(struct ovl_fs *ofs, struct dentry *upperdentry, { struct iattr attr = { .ia_valid = - ATTR_ATIME | ATTR_MTIME | ATTR_ATIME_SET | ATTR_MTIME_SET, + ATTR_ATIME | ATTR_MTIME | ATTR_ATIME_SET | ATTR_MTIME_SET | ATTR_CTIME, .ia_atime = stat->atime, .ia_mtime = stat->mtime, }; @@ -618,7 +618,8 @@ static int ovl_copy_up_metadata(struct ovl_copy_up_ctx *c, struct dentry *temp) if (err) return err; - if (inode->i_flags & OVL_COPY_I_FLAGS_MASK) { + if (inode->i_flags & OVL_COPY_I_FLAGS_MASK && + (S_ISREG(c->stat.mode) || S_ISDIR(c->stat.mode))) { /* * Copy the fileattr inode flags that are the source of already * copied i_flags diff --git a/fs/overlayfs/export.c b/fs/overlayfs/export.c index c8c8588bd98c..26b782c53910 100644 --- a/fs/overlayfs/export.c +++ b/fs/overlayfs/export.c @@ -188,7 +188,7 @@ static int ovl_check_encode_origin(struct dentry *dentry) /* Lower file handle for non-upper non-decodable */ if (!ovl_dentry_upper(dentry) && !decodable) - return 0; + return 1; /* Upper file handle for pure upper */ if (!ovl_dentry_lower(dentry)) diff --git a/fs/overlayfs/file.c b/fs/overlayfs/file.c index 3b4cc633d763..8be4dc050d1e 100644 --- a/fs/overlayfs/file.c +++ b/fs/overlayfs/file.c @@ -19,7 +19,6 @@ struct ovl_aio_req { struct kiocb iocb; refcount_t ref; struct kiocb *orig_iocb; - struct fd fd; }; static struct kmem_cache *ovl_aio_request_cachep; @@ -280,7 +279,7 @@ static rwf_t ovl_iocb_to_rwf(int ifl) static inline void ovl_aio_put(struct ovl_aio_req *aio_req) { if (refcount_dec_and_test(&aio_req->ref)) { - fdput(aio_req->fd); + fput(aio_req->iocb.ki_filp); kmem_cache_free(ovl_aio_request_cachep, aio_req); } } @@ -342,10 +341,8 @@ static ssize_t ovl_read_iter(struct kiocb *iocb, struct iov_iter *iter) if (!aio_req) goto out; - aio_req->fd = real; - real.flags = 0; aio_req->orig_iocb = iocb; - kiocb_clone(&aio_req->iocb, iocb, real.file); + kiocb_clone(&aio_req->iocb, iocb, get_file(real.file)); aio_req->iocb.ki_complete = ovl_aio_rw_complete; refcount_set(&aio_req->ref, 2); ret = vfs_iocb_iter_read(real.file, &aio_req->iocb, iter); @@ -393,6 +390,12 @@ static ssize_t ovl_write_iter(struct kiocb *iocb, struct iov_iter *iter) if (!ovl_should_sync(OVL_FS(inode->i_sb))) ifl &= ~(IOCB_DSYNC | IOCB_SYNC); + /* + * Overlayfs doesn't support deferred completions, don't copy + * this property in case it is set by the issuer. + */ + ifl &= ~IOCB_DIO_CALLER_COMP; + old_cred = ovl_override_creds(file_inode(file)->i_sb); if (is_sync_kiocb(iocb)) { file_start_write(real.file); @@ -409,10 +412,8 @@ static ssize_t ovl_write_iter(struct kiocb *iocb, struct iov_iter *iter) if (!aio_req) goto out; - aio_req->fd = real; - real.flags = 0; aio_req->orig_iocb = iocb; - kiocb_clone(&aio_req->iocb, iocb, real.file); + kiocb_clone(&aio_req->iocb, iocb, get_file(real.file)); aio_req->iocb.ki_flags = ifl; aio_req->iocb.ki_complete = ovl_aio_rw_complete; refcount_set(&aio_req->ref, 2); diff --git a/fs/overlayfs/ovl_entry.h b/fs/overlayfs/ovl_entry.h index e9539f98e86a..d82d2a043da2 100644 --- a/fs/overlayfs/ovl_entry.h +++ b/fs/overlayfs/ovl_entry.h @@ -8,6 +8,7 @@ struct ovl_config { char *upperdir; char *workdir; + char **lowerdirs; bool default_permissions; int redirect_mode; int verity_mode; @@ -39,17 +40,8 @@ struct ovl_layer { int idx; /* One fsid per unique underlying sb (upper fsid == 0) */ int fsid; - char *name; }; -/* - * ovl_free_fs() relies on @mnt being the first member when unmounting - * the private mounts created for each layer. Let's check both the - * offset and type. - */ -static_assert(offsetof(struct ovl_layer, mnt) == 0); -static_assert(__same_type(typeof_member(struct ovl_layer, mnt), struct vfsmount *)); - struct ovl_path { const struct ovl_layer *layer; struct dentry *dentry; diff --git a/fs/overlayfs/params.c b/fs/overlayfs/params.c index b9355bb6d75a..f6ff23fd101c 100644 --- a/fs/overlayfs/params.c +++ b/fs/overlayfs/params.c @@ -157,6 +157,34 @@ const struct fs_parameter_spec ovl_parameter_spec[] = { {} }; +static char *ovl_next_opt(char **s) +{ + char *sbegin = *s; + char *p; + + if (sbegin == NULL) + return NULL; + + for (p = sbegin; *p; p++) { + if (*p == '\\') { + p++; + if (!*p) + break; + } else if (*p == ',') { + *p = '\0'; + *s = p + 1; + return sbegin; + } + } + *s = NULL; + return sbegin; +} + +static int ovl_parse_monolithic(struct fs_context *fc, void *data) +{ + return vfs_parse_monolithic_sep(fc, data, ovl_next_opt); +} + static ssize_t ovl_parse_param_split_lowerdirs(char *str) { ssize_t nr_layers = 1, nr_colons = 0; @@ -164,7 +192,8 @@ static ssize_t ovl_parse_param_split_lowerdirs(char *str) for (s = d = str;; s++, d++) { if (*s == '\\') { - s++; + /* keep esc chars in split lowerdir */ + *d++ = *s++; } else if (*s == ':') { bool next_colon = (*(s + 1) == ':'); @@ -239,7 +268,7 @@ static void ovl_unescape(char *s) } } -static int ovl_mount_dir(const char *name, struct path *path) +static int ovl_mount_dir(const char *name, struct path *path, bool upper) { int err = -ENOMEM; char *tmp = kstrdup(name, GFP_KERNEL); @@ -248,7 +277,7 @@ static int ovl_mount_dir(const char *name, struct path *path) ovl_unescape(tmp); err = ovl_mount_dir_noesc(tmp, path); - if (!err && path->dentry->d_flags & DCACHE_OP_REAL) { + if (!err && upper && path->dentry->d_flags & DCACHE_OP_REAL) { pr_err("filesystem on '%s' not supported as upperdir\n", tmp); path_put_init(path); @@ -269,7 +298,7 @@ static int ovl_parse_param_upperdir(const char *name, struct fs_context *fc, struct path path; char *dup; - err = ovl_mount_dir(name, &path); + err = ovl_mount_dir(name, &path, true); if (err) return err; @@ -321,12 +350,6 @@ static void ovl_parse_param_drop_lowerdir(struct ovl_fs_context *ctx) * Set "/lower1", "/lower2", and "/lower3" as lower layers and * "/data1" and "/data2" as data lower layers. Any existing lower * layers are replaced. - * (2) lowerdir=:/lower4 - * Append "/lower4" to current stack of lower layers. This requires - * that there already is at least one lower layer configured. - * (3) lowerdir=::/lower5 - * Append data "/lower5" as data lower layer. This requires that - * there's at least one regular lower layer present. */ static int ovl_parse_param_lowerdir(const char *name, struct fs_context *fc) { @@ -348,49 +371,9 @@ static int ovl_parse_param_lowerdir(const char *name, struct fs_context *fc) return 0; } - if (strncmp(name, "::", 2) == 0) { - /* - * This is a data layer. - * There must be at least one regular lower layer - * specified. - */ - if (ctx->nr == 0) { - pr_err("data lower layers without regular lower layers not allowed"); - return -EINVAL; - } - - /* Skip the leading "::". */ - name += 2; - data_layer = true; - /* - * A data layer is automatically an append as there - * must've been at least one regular lower layer. - */ - append = true; - } else if (*name == ':') { - /* - * This is a regular lower layer. - * If users want to append a layer enforce that they - * have already specified a first layer before. It's - * better to be strict. - */ - if (ctx->nr == 0) { - pr_err("cannot append layer if no previous layer has been specified"); - return -EINVAL; - } - - /* - * Once a sequence of data layers has started regular - * lower layers are forbidden. - */ - if (ctx->nr_data > 0) { - pr_err("regular lower layers cannot follow data lower layers"); - return -EINVAL; - } - - /* Skip the leading ":". */ - name++; - append = true; + if (*name == ':') { + pr_err("cannot append lower layer"); + return -EINVAL; } dup = kstrdup(name, GFP_KERNEL); @@ -472,7 +455,7 @@ static int ovl_parse_param_lowerdir(const char *name, struct fs_context *fc) l = &ctx->lower[nr]; memset(l, 0, sizeof(*l)); - err = ovl_mount_dir_noesc(dup_iter, &l->path); + err = ovl_mount_dir(dup_iter, &l->path, false); if (err) goto out_put; @@ -682,6 +665,7 @@ static int ovl_reconfigure(struct fs_context *fc) } static const struct fs_context_operations ovl_context_ops = { + .parse_monolithic = ovl_parse_monolithic, .parse_param = ovl_parse_param, .get_tree = ovl_get_tree, .reconfigure = ovl_reconfigure, @@ -752,12 +736,12 @@ void ovl_free_fs(struct ovl_fs *ofs) if (ofs->upperdir_locked) ovl_inuse_unlock(ovl_upper_mnt(ofs)->mnt_root); - /* Hack! Reuse ofs->layers as a vfsmount array before freeing it */ - mounts = (struct vfsmount **) ofs->layers; + /* Reuse ofs->config.lowerdirs as a vfsmount array before freeing it */ + mounts = (struct vfsmount **) ofs->config.lowerdirs; for (i = 0; i < ofs->numlayer; i++) { iput(ofs->layers[i].trap); + kfree(ofs->config.lowerdirs[i]); mounts[i] = ofs->layers[i].mnt; - kfree(ofs->layers[i].name); } kern_unmount_array(mounts, ofs->numlayer); kfree(ofs->layers); @@ -765,6 +749,7 @@ void ovl_free_fs(struct ovl_fs *ofs) free_anon_bdev(ofs->fs[i].pseudo_dev); kfree(ofs->fs); + kfree(ofs->config.lowerdirs); kfree(ofs->config.upperdir); kfree(ofs->config.workdir); if (ofs->creator_cred) @@ -949,16 +934,23 @@ int ovl_show_options(struct seq_file *m, struct dentry *dentry) struct super_block *sb = dentry->d_sb; struct ovl_fs *ofs = OVL_FS(sb); size_t nr, nr_merged_lower = ofs->numlayer - ofs->numdatalayer; - const struct ovl_layer *data_layers = &ofs->layers[nr_merged_lower]; - - /* ofs->layers[0] is the upper layer */ - seq_printf(m, ",lowerdir=%s", ofs->layers[1].name); - /* dump regular lower layers */ - for (nr = 2; nr < nr_merged_lower; nr++) - seq_printf(m, ":%s", ofs->layers[nr].name); - /* dump data lower layers */ - for (nr = 0; nr < ofs->numdatalayer; nr++) - seq_printf(m, "::%s", data_layers[nr].name); + + /* + * lowerdirs[] starts from offset 1, then + * >= 0 regular lower layers prefixed with : and + * >= 0 data-only lower layers prefixed with :: + * + * we need to escase comma and space like seq_show_option() does and + * we also need to escape the colon separator from lowerdir paths. + */ + seq_puts(m, ",lowerdir="); + for (nr = 1; nr < ofs->numlayer; nr++) { + if (nr > 1) + seq_putc(m, ':'); + if (nr >= nr_merged_lower) + seq_putc(m, ':'); + seq_escape(m, ofs->config.lowerdirs[nr], ":, \t\n\\"); + } if (ofs->config.upperdir) { seq_show_option(m, "upperdir", ofs->config.upperdir); seq_show_option(m, "workdir", ofs->config.workdir); diff --git a/fs/overlayfs/super.c b/fs/overlayfs/super.c index def266b5e2a3..3fa2416264a4 100644 --- a/fs/overlayfs/super.c +++ b/fs/overlayfs/super.c @@ -104,8 +104,8 @@ static int ovl_revalidate_real(struct dentry *d, unsigned int flags, bool weak) static int ovl_dentry_revalidate_common(struct dentry *dentry, unsigned int flags, bool weak) { - struct ovl_entry *oe = OVL_E(dentry); - struct ovl_path *lowerstack = ovl_lowerstack(oe); + struct ovl_entry *oe; + struct ovl_path *lowerstack; struct inode *inode = d_inode_rcu(dentry); struct dentry *upper; unsigned int i; @@ -115,6 +115,8 @@ static int ovl_dentry_revalidate_common(struct dentry *dentry, if (!inode) return -ECHILD; + oe = OVL_I_E(inode); + lowerstack = ovl_lowerstack(oe); upper = ovl_i_dentry_upper(inode); if (upper) ret = ovl_revalidate_real(upper, flags, weak); @@ -167,6 +169,7 @@ static void ovl_free_inode(struct inode *inode) struct ovl_inode *oi = OVL_I(inode); kfree(oi->redirect); + kfree(oi->oe); mutex_destroy(&oi->lock); kmem_cache_free(ovl_inode_cachep, oi); } @@ -176,7 +179,7 @@ static void ovl_destroy_inode(struct inode *inode) struct ovl_inode *oi = OVL_I(inode); dput(oi->__upperdentry); - ovl_free_entry(oi->oe); + ovl_stack_put(ovl_lowerstack(oi->oe), ovl_numlower(oi->oe)); if (S_ISDIR(inode->i_mode)) ovl_dir_cache_free(inode); else @@ -569,11 +572,6 @@ static int ovl_get_upper(struct super_block *sb, struct ovl_fs *ofs, upper_layer->idx = 0; upper_layer->fsid = 0; - err = -ENOMEM; - upper_layer->name = kstrdup(ofs->config.upperdir, GFP_KERNEL); - if (!upper_layer->name) - goto out; - /* * Inherit SB_NOSEC flag from upperdir. * @@ -1122,7 +1120,8 @@ static int ovl_get_layers(struct super_block *sb, struct ovl_fs *ofs, layers[ofs->numlayer].idx = ofs->numlayer; layers[ofs->numlayer].fsid = fsid; layers[ofs->numlayer].fs = &ofs->fs[fsid]; - layers[ofs->numlayer].name = l->name; + /* Store for printing lowerdir=... in ovl_show_options() */ + ofs->config.lowerdirs[ofs->numlayer] = l->name; l->name = NULL; ofs->numlayer++; ofs->fs[fsid].is_lower = true; @@ -1367,8 +1366,16 @@ int ovl_fill_super(struct super_block *sb, struct fs_context *fc) if (!layers) goto out_err; + ofs->config.lowerdirs = kcalloc(ctx->nr + 1, sizeof(char *), GFP_KERNEL); + if (!ofs->config.lowerdirs) { + kfree(layers); + goto out_err; + } ofs->layers = layers; - /* Layer 0 is reserved for upper even if there's no upper */ + /* + * Layer 0 is reserved for upper even if there's no upper. + * For consistency, config.lowerdirs[0] is NULL. + */ ofs->numlayer = 1; sb->s_stack_depth = 0; diff --git a/fs/pipe.c b/fs/pipe.c index 6c1a9b1db907..139190165a1c 100644 --- a/fs/pipe.c +++ b/fs/pipe.c @@ -537,7 +537,6 @@ pipe_write(struct kiocb *iocb, struct iov_iter *from) break; } ret += copied; - buf->offset = 0; buf->len = copied; if (!iov_iter_count(from)) diff --git a/fs/proc/internal.h b/fs/proc/internal.h index 9dda7e54b2d0..9a8f32f21ff5 100644 --- a/fs/proc/internal.h +++ b/fs/proc/internal.h @@ -289,9 +289,7 @@ struct proc_maps_private { struct inode *inode; struct task_struct *task; struct mm_struct *mm; -#ifdef CONFIG_MMU struct vma_iterator iter; -#endif #ifdef CONFIG_NUMA struct mempolicy *task_mempolicy; #endif diff --git a/fs/proc/task_nommu.c b/fs/proc/task_nommu.c index a8ac0dd8041e..7cebd397cc26 100644 --- a/fs/proc/task_nommu.c +++ b/fs/proc/task_nommu.c @@ -175,15 +175,28 @@ static int show_map(struct seq_file *m, void *_p) return nommu_vma_show(m, _p); } -static void *m_start(struct seq_file *m, loff_t *pos) +static struct vm_area_struct *proc_get_vma(struct proc_maps_private *priv, + loff_t *ppos) +{ + struct vm_area_struct *vma = vma_next(&priv->iter); + + if (vma) { + *ppos = vma->vm_start; + } else { + *ppos = -1UL; + } + + return vma; +} + +static void *m_start(struct seq_file *m, loff_t *ppos) { struct proc_maps_private *priv = m->private; + unsigned long last_addr = *ppos; struct mm_struct *mm; - struct vm_area_struct *vma; - unsigned long addr = *pos; - /* See m_next(). Zero at the start or after lseek. */ - if (addr == -1UL) + /* See proc_get_vma(). Zero at the start or after lseek. */ + if (last_addr == -1UL) return NULL; /* pin the task and mm whilst we play with them */ @@ -192,44 +205,41 @@ static void *m_start(struct seq_file *m, loff_t *pos) return ERR_PTR(-ESRCH); mm = priv->mm; - if (!mm || !mmget_not_zero(mm)) + if (!mm || !mmget_not_zero(mm)) { + put_task_struct(priv->task); + priv->task = NULL; return NULL; + } if (mmap_read_lock_killable(mm)) { mmput(mm); + put_task_struct(priv->task); + priv->task = NULL; return ERR_PTR(-EINTR); } - /* start the next element from addr */ - vma = find_vma(mm, addr); - if (vma) - return vma; + vma_iter_init(&priv->iter, mm, last_addr); - mmap_read_unlock(mm); - mmput(mm); - return NULL; + return proc_get_vma(priv, ppos); } -static void m_stop(struct seq_file *m, void *_vml) +static void m_stop(struct seq_file *m, void *v) { struct proc_maps_private *priv = m->private; + struct mm_struct *mm = priv->mm; - if (!IS_ERR_OR_NULL(_vml)) { - mmap_read_unlock(priv->mm); - mmput(priv->mm); - } - if (priv->task) { - put_task_struct(priv->task); - priv->task = NULL; - } + if (!priv->task) + return; + + mmap_read_unlock(mm); + mmput(mm); + put_task_struct(priv->task); + priv->task = NULL; } -static void *m_next(struct seq_file *m, void *_p, loff_t *pos) +static void *m_next(struct seq_file *m, void *_p, loff_t *ppos) { - struct vm_area_struct *vma = _p; - - *pos = vma->vm_end; - return find_vma(vma->vm_mm, vma->vm_end); + return proc_get_vma(m->private, ppos); } static const struct seq_operations proc_pid_maps_ops = { diff --git a/fs/quota/dquot.c b/fs/quota/dquot.c index 9e72bfe8bbad..31e897ad5e6a 100644 --- a/fs/quota/dquot.c +++ b/fs/quota/dquot.c @@ -233,19 +233,18 @@ static void put_quota_format(struct quota_format_type *fmt) * All dquots are placed to the end of inuse_list when first created, and this * list is used for invalidate operation, which must look at every dquot. * - * When the last reference of a dquot will be dropped, the dquot will be - * added to releasing_dquots. We'd then queue work item which would call + * When the last reference of a dquot is dropped, the dquot is added to + * releasing_dquots. We'll then queue work item which will call * synchronize_srcu() and after that perform the final cleanup of all the - * dquots on the list. Both releasing_dquots and free_dquots use the - * dq_free list_head in the dquot struct. When a dquot is removed from - * releasing_dquots, a reference count is always subtracted, and if - * dq_count == 0 at that point, the dquot will be added to the free_dquots. + * dquots on the list. Each cleaned up dquot is moved to free_dquots list. + * Both releasing_dquots and free_dquots use the dq_free list_head in the dquot + * struct. * - * Unused dquots (dq_count == 0) are added to the free_dquots list when freed, - * and this list is searched whenever we need an available dquot. Dquots are - * removed from the list as soon as they are used again, and - * dqstats.free_dquots gives the number of dquots on the list. When - * dquot is invalidated it's completely released from memory. + * Unused and cleaned up dquots are in the free_dquots list and this list is + * searched whenever we need an available dquot. Dquots are removed from the + * list as soon as they are used again and dqstats.free_dquots gives the number + * of dquots on the list. When dquot is invalidated it's completely released + * from memory. * * Dirty dquots are added to the dqi_dirty_list of quota_info when mark * dirtied, and this list is searched when writing dirty dquots back to @@ -321,6 +320,7 @@ static inline void put_dquot_last(struct dquot *dquot) static inline void put_releasing_dquots(struct dquot *dquot) { list_add_tail(&dquot->dq_free, &releasing_dquots); + set_bit(DQ_RELEASING_B, &dquot->dq_flags); } static inline void remove_free_dquot(struct dquot *dquot) @@ -328,8 +328,10 @@ static inline void remove_free_dquot(struct dquot *dquot) if (list_empty(&dquot->dq_free)) return; list_del_init(&dquot->dq_free); - if (!atomic_read(&dquot->dq_count)) + if (!test_bit(DQ_RELEASING_B, &dquot->dq_flags)) dqstats_dec(DQST_FREE_DQUOTS); + else + clear_bit(DQ_RELEASING_B, &dquot->dq_flags); } static inline void put_inuse(struct dquot *dquot) @@ -581,12 +583,6 @@ restart: continue; /* Wait for dquot users */ if (atomic_read(&dquot->dq_count)) { - /* dquot in releasing_dquots, flush and retry */ - if (!list_empty(&dquot->dq_free)) { - spin_unlock(&dq_list_lock); - goto restart; - } - atomic_inc(&dquot->dq_count); spin_unlock(&dq_list_lock); /* @@ -605,6 +601,15 @@ restart: * restart. */ goto restart; } + /* + * The last user already dropped its reference but dquot didn't + * get fully cleaned up yet. Restart the scan which flushes the + * work cleaning up released dquots. + */ + if (test_bit(DQ_RELEASING_B, &dquot->dq_flags)) { + spin_unlock(&dq_list_lock); + goto restart; + } /* * Quota now has no users and it has been written on last * dqput() @@ -696,6 +701,13 @@ int dquot_writeback_dquots(struct super_block *sb, int type) dq_dirty); WARN_ON(!dquot_active(dquot)); + /* If the dquot is releasing we should not touch it */ + if (test_bit(DQ_RELEASING_B, &dquot->dq_flags)) { + spin_unlock(&dq_list_lock); + flush_delayed_work("a_release_work); + spin_lock(&dq_list_lock); + continue; + } /* Now we have active dquot from which someone is * holding reference so we can safely just increase @@ -809,18 +821,18 @@ static void quota_release_workfn(struct work_struct *work) /* Exchange the list head to avoid livelock. */ list_replace_init(&releasing_dquots, &rls_head); spin_unlock(&dq_list_lock); + synchronize_srcu(&dquot_srcu); restart: - synchronize_srcu(&dquot_srcu); spin_lock(&dq_list_lock); while (!list_empty(&rls_head)) { dquot = list_first_entry(&rls_head, struct dquot, dq_free); - /* Dquot got used again? */ - if (atomic_read(&dquot->dq_count) > 1) { - remove_free_dquot(dquot); - atomic_dec(&dquot->dq_count); - continue; - } + WARN_ON_ONCE(atomic_read(&dquot->dq_count)); + /* + * Note that DQ_RELEASING_B protects us from racing with + * invalidate_dquots() calls so we are safe to work with the + * dquot even after we drop dq_list_lock. + */ if (dquot_dirty(dquot)) { spin_unlock(&dq_list_lock); /* Commit dquot before releasing */ @@ -834,7 +846,6 @@ restart: } /* Dquot is inactive and clean, now move it to free list */ remove_free_dquot(dquot); - atomic_dec(&dquot->dq_count); put_dquot_last(dquot); } spin_unlock(&dq_list_lock); @@ -875,6 +886,7 @@ void dqput(struct dquot *dquot) BUG_ON(!list_empty(&dquot->dq_free)); #endif put_releasing_dquots(dquot); + atomic_dec(&dquot->dq_count); spin_unlock(&dq_list_lock); queue_delayed_work(system_unbound_wq, "a_release_work, 1); } @@ -963,7 +975,7 @@ we_slept: dqstats_inc(DQST_LOOKUPS); } /* Wait for dq_lock - after this we know that either dquot_release() is - * already finished or it will be canceled due to dq_count > 1 test */ + * already finished or it will be canceled due to dq_count > 0 test */ wait_on_dquot(dquot); /* Read the dquot / allocate space in quota file */ if (!dquot_active(dquot)) { diff --git a/fs/reiserfs/reiserfs.h b/fs/reiserfs/reiserfs.h index b81749492ef9..7d12b8c5b2fa 100644 --- a/fs/reiserfs/reiserfs.h +++ b/fs/reiserfs/reiserfs.h @@ -2699,7 +2699,7 @@ struct reiserfs_iget_args { #define get_journal_desc_magic(bh) (bh->b_data + bh->b_size - 12) #define journal_trans_half(blocksize) \ - ((blocksize - sizeof (struct reiserfs_journal_desc) + sizeof (__u32) - 12) / sizeof (__u32)) + ((blocksize - sizeof(struct reiserfs_journal_desc) - 12) / sizeof(__u32)) /* journal.c see journal.c for all the comments here */ @@ -2711,7 +2711,7 @@ struct reiserfs_journal_desc { __le32 j_len; __le32 j_mount_id; /* mount id of this trans */ - __le32 j_realblock[1]; /* real locations for each block */ + __le32 j_realblock[]; /* real locations for each block */ }; #define get_desc_trans_id(d) le32_to_cpu((d)->j_trans_id) @@ -2726,7 +2726,7 @@ struct reiserfs_journal_desc { struct reiserfs_journal_commit { __le32 j_trans_id; /* must match j_trans_id from the desc block */ __le32 j_len; /* ditto */ - __le32 j_realblock[1]; /* real locations for each block */ + __le32 j_realblock[]; /* real locations for each block */ }; #define get_commit_trans_id(c) le32_to_cpu((c)->j_trans_id) diff --git a/fs/smb/client/cached_dir.c b/fs/smb/client/cached_dir.c index b17f067e4ada..fe1bf5b6e0cb 100644 --- a/fs/smb/client/cached_dir.c +++ b/fs/smb/client/cached_dir.c @@ -15,6 +15,7 @@ static struct cached_fid *init_cached_dir(const char *path); static void free_cached_dir(struct cached_fid *cfid); static void smb2_close_cached_fid(struct kref *ref); +static void cfids_laundromat_worker(struct work_struct *work); static struct cached_fid *find_or_create_cached_dir(struct cached_fids *cfids, const char *path, @@ -169,15 +170,18 @@ int open_cached_dir(unsigned int xid, struct cifs_tcon *tcon, return -ENOENT; } /* - * At this point we either have a lease already and we can just - * return it. If not we are guaranteed to be the only thread accessing - * this cfid. + * Return cached fid if it has a lease. Otherwise, it is either a new + * entry or laundromat worker removed it from @cfids->entries. Caller + * will put last reference if the latter. */ + spin_lock(&cfids->cfid_list_lock); if (cfid->has_lease) { + spin_unlock(&cfids->cfid_list_lock); *ret_cfid = cfid; kfree(utf16_path); return 0; } + spin_unlock(&cfids->cfid_list_lock); /* * Skip any prefix paths in @path as lookup_positive_unlocked() ends up @@ -294,9 +298,11 @@ int open_cached_dir(unsigned int xid, struct cifs_tcon *tcon, goto oshr_free; } } + spin_lock(&cfids->cfid_list_lock); cfid->dentry = dentry; cfid->time = jiffies; cfid->has_lease = true; + spin_unlock(&cfids->cfid_list_lock); oshr_free: kfree(utf16_path); @@ -305,24 +311,28 @@ oshr_free: free_rsp_buf(resp_buftype[0], rsp_iov[0].iov_base); free_rsp_buf(resp_buftype[1], rsp_iov[1].iov_base); spin_lock(&cfids->cfid_list_lock); - if (rc && !cfid->has_lease) { - if (cfid->on_list) { - list_del(&cfid->entry); - cfid->on_list = false; - cfids->num_entries--; + if (!cfid->has_lease) { + if (rc) { + if (cfid->on_list) { + list_del(&cfid->entry); + cfid->on_list = false; + cfids->num_entries--; + } + rc = -ENOENT; + } else { + /* + * We are guaranteed to have two references at this + * point. One for the caller and one for a potential + * lease. Release the Lease-ref so that the directory + * will be closed when the caller closes the cached + * handle. + */ + spin_unlock(&cfids->cfid_list_lock); + kref_put(&cfid->refcount, smb2_close_cached_fid); + goto out; } - rc = -ENOENT; } spin_unlock(&cfids->cfid_list_lock); - if (!rc && !cfid->has_lease) { - /* - * We are guaranteed to have two references at this point. - * One for the caller and one for a potential lease. - * Release the Lease-ref so that the directory will be closed - * when the caller closes the cached handle. - */ - kref_put(&cfid->refcount, smb2_close_cached_fid); - } if (rc) { if (cfid->is_open) SMB2_close(0, cfid->tcon, cfid->fid.persistent_fid, @@ -330,7 +340,7 @@ oshr_free: free_cached_dir(cfid); cfid = NULL; } - +out: if (rc == 0) { *ret_cfid = cfid; atomic_inc(&tcon->num_remote_opens); @@ -452,6 +462,9 @@ void invalidate_all_cached_dirs(struct cifs_tcon *tcon) struct cached_fid *cfid, *q; LIST_HEAD(entry); + if (cfids == NULL) + return; + spin_lock(&cfids->cfid_list_lock); list_for_each_entry_safe(cfid, q, &cfids->entries, entry) { list_move(&cfid->entry, &entry); @@ -569,53 +582,51 @@ static void free_cached_dir(struct cached_fid *cfid) kfree(cfid); } -static int -cifs_cfids_laundromat_thread(void *p) +static void cfids_laundromat_worker(struct work_struct *work) { - struct cached_fids *cfids = p; + struct cached_fids *cfids; struct cached_fid *cfid, *q; - struct list_head entry; + LIST_HEAD(entry); - while (!kthread_should_stop()) { - ssleep(1); - INIT_LIST_HEAD(&entry); - if (kthread_should_stop()) - return 0; - spin_lock(&cfids->cfid_list_lock); - list_for_each_entry_safe(cfid, q, &cfids->entries, entry) { - if (time_after(jiffies, cfid->time + HZ * dir_cache_timeout)) { - list_del(&cfid->entry); - list_add(&cfid->entry, &entry); - cfids->num_entries--; - } - } - spin_unlock(&cfids->cfid_list_lock); + cfids = container_of(work, struct cached_fids, laundromat_work.work); - list_for_each_entry_safe(cfid, q, &entry, entry) { + spin_lock(&cfids->cfid_list_lock); + list_for_each_entry_safe(cfid, q, &cfids->entries, entry) { + if (cfid->time && + time_after(jiffies, cfid->time + HZ * dir_cache_timeout)) { cfid->on_list = false; - list_del(&cfid->entry); + list_move(&cfid->entry, &entry); + cfids->num_entries--; + /* To prevent race with smb2_cached_lease_break() */ + kref_get(&cfid->refcount); + } + } + spin_unlock(&cfids->cfid_list_lock); + + list_for_each_entry_safe(cfid, q, &entry, entry) { + list_del(&cfid->entry); + /* + * Cancel and wait for the work to finish in case we are racing + * with it. + */ + cancel_work_sync(&cfid->lease_break); + if (cfid->has_lease) { /* - * Cancel, and wait for the work to finish in - * case we are racing with it. + * Our lease has not yet been cancelled from the server + * so we need to drop the reference. */ - cancel_work_sync(&cfid->lease_break); - if (cfid->has_lease) { - /* - * We lease has not yet been cancelled from - * the server so we need to drop the reference. - */ - spin_lock(&cfids->cfid_list_lock); - cfid->has_lease = false; - spin_unlock(&cfids->cfid_list_lock); - kref_put(&cfid->refcount, smb2_close_cached_fid); - } + spin_lock(&cfids->cfid_list_lock); + cfid->has_lease = false; + spin_unlock(&cfids->cfid_list_lock); + kref_put(&cfid->refcount, smb2_close_cached_fid); } + /* Drop the extra reference opened above */ + kref_put(&cfid->refcount, smb2_close_cached_fid); } - - return 0; + queue_delayed_work(cifsiod_wq, &cfids->laundromat_work, + dir_cache_timeout * HZ); } - struct cached_fids *init_cached_dirs(void) { struct cached_fids *cfids; @@ -626,19 +637,10 @@ struct cached_fids *init_cached_dirs(void) spin_lock_init(&cfids->cfid_list_lock); INIT_LIST_HEAD(&cfids->entries); - /* - * since we're in a cifs function already, we know that - * this will succeed. No need for try_module_get(). - */ - __module_get(THIS_MODULE); - cfids->laundromat = kthread_run(cifs_cfids_laundromat_thread, - cfids, "cifsd-cfid-laundromat"); - if (IS_ERR(cfids->laundromat)) { - cifs_dbg(VFS, "Failed to start cfids laundromat thread.\n"); - kfree(cfids); - module_put(THIS_MODULE); - return NULL; - } + INIT_DELAYED_WORK(&cfids->laundromat_work, cfids_laundromat_worker); + queue_delayed_work(cifsiod_wq, &cfids->laundromat_work, + dir_cache_timeout * HZ); + return cfids; } @@ -651,11 +653,10 @@ void free_cached_dirs(struct cached_fids *cfids) struct cached_fid *cfid, *q; LIST_HEAD(entry); - if (cfids->laundromat) { - kthread_stop(cfids->laundromat); - cfids->laundromat = NULL; - module_put(THIS_MODULE); - } + if (cfids == NULL) + return; + + cancel_delayed_work_sync(&cfids->laundromat_work); spin_lock(&cfids->cfid_list_lock); list_for_each_entry_safe(cfid, q, &cfids->entries, entry) { diff --git a/fs/smb/client/cached_dir.h b/fs/smb/client/cached_dir.h index a82ff2cea789..81ba0fd5cc16 100644 --- a/fs/smb/client/cached_dir.h +++ b/fs/smb/client/cached_dir.h @@ -57,7 +57,7 @@ struct cached_fids { spinlock_t cfid_list_lock; int num_entries; struct list_head entries; - struct task_struct *laundromat; + struct delayed_work laundromat_work; }; extern struct cached_fids *init_cached_dirs(void); diff --git a/fs/smb/client/cifsglob.h b/fs/smb/client/cifsglob.h index 032d8716f671..02082621d8e0 100644 --- a/fs/smb/client/cifsglob.h +++ b/fs/smb/client/cifsglob.h @@ -1807,6 +1807,7 @@ static inline bool is_retryable_error(int error) #define MID_RETRY_NEEDED 8 /* session closed while this request out */ #define MID_RESPONSE_MALFORMED 0x10 #define MID_SHUTDOWN 0x20 +#define MID_RESPONSE_READY 0x40 /* ready for other process handle the rsp */ /* Flags */ #define MID_WAIT_CANCELLED 1 /* Cancelled while waiting for response */ @@ -1943,7 +1944,7 @@ require use of the stronger protocol */ * cifsInodeInfo->lock_sem cifsInodeInfo->llist cifs_init_once * ->can_cache_brlcks * cifsInodeInfo->deferred_lock cifsInodeInfo->deferred_closes cifsInodeInfo_alloc - * cached_fid->fid_mutex cifs_tcon->crfid tconInfoAlloc + * cached_fid->fid_mutex cifs_tcon->crfid tcon_info_alloc * cifsFileInfo->fh_mutex cifsFileInfo cifs_new_fileinfo * cifsFileInfo->file_info_lock cifsFileInfo->count cifs_new_fileinfo * ->invalidHandle initiate_cifs_search diff --git a/fs/smb/client/cifsproto.h b/fs/smb/client/cifsproto.h index 7d8035846680..0c37eefa18a5 100644 --- a/fs/smb/client/cifsproto.h +++ b/fs/smb/client/cifsproto.h @@ -512,7 +512,7 @@ extern int CIFSSMBLogoff(const unsigned int xid, struct cifs_ses *ses); extern struct cifs_ses *sesInfoAlloc(void); extern void sesInfoFree(struct cifs_ses *); -extern struct cifs_tcon *tconInfoAlloc(void); +extern struct cifs_tcon *tcon_info_alloc(bool dir_leases_enabled); extern void tconInfoFree(struct cifs_tcon *); extern int cifs_sign_rqst(struct smb_rqst *rqst, struct TCP_Server_Info *server, diff --git a/fs/smb/client/connect.c b/fs/smb/client/connect.c index 687754791bf0..7b923e36501b 100644 --- a/fs/smb/client/connect.c +++ b/fs/smb/client/connect.c @@ -1882,7 +1882,8 @@ cifs_setup_ipc(struct cifs_ses *ses, struct smb3_fs_context *ctx) } } - tcon = tconInfoAlloc(); + /* no need to setup directory caching on IPC share, so pass in false */ + tcon = tcon_info_alloc(false); if (tcon == NULL) return -ENOMEM; @@ -2473,8 +2474,9 @@ cifs_put_tcon(struct cifs_tcon *tcon) static struct cifs_tcon * cifs_get_tcon(struct cifs_ses *ses, struct smb3_fs_context *ctx) { - int rc, xid; struct cifs_tcon *tcon; + bool nohandlecache; + int rc, xid; tcon = cifs_find_tcon(ses, ctx); if (tcon) { @@ -2492,11 +2494,17 @@ cifs_get_tcon(struct cifs_ses *ses, struct smb3_fs_context *ctx) goto out_fail; } - tcon = tconInfoAlloc(); + if (ses->server->dialect >= SMB20_PROT_ID && + (ses->server->capabilities & SMB2_GLOBAL_CAP_DIRECTORY_LEASING)) + nohandlecache = ctx->nohandlecache; + else + nohandlecache = true; + tcon = tcon_info_alloc(!nohandlecache); if (tcon == NULL) { rc = -ENOMEM; goto out_fail; } + tcon->nohandlecache = nohandlecache; if (ctx->snapshot_time) { if (ses->server->vals->protocol_id == 0) { @@ -2658,10 +2666,6 @@ cifs_get_tcon(struct cifs_ses *ses, struct smb3_fs_context *ctx) tcon->nocase = ctx->nocase; tcon->broken_sparse_sup = ctx->no_sparse; tcon->max_cached_dirs = ctx->max_cached_dirs; - if (ses->server->capabilities & SMB2_GLOBAL_CAP_DIRECTORY_LEASING) - tcon->nohandlecache = ctx->nohandlecache; - else - tcon->nohandlecache = true; tcon->nodelete = ctx->nodelete; tcon->local_lease = ctx->local_lease; INIT_LIST_HEAD(&tcon->pending_opens); @@ -2891,9 +2895,9 @@ bind_socket(struct TCP_Server_Info *server) if (server->srcaddr.ss_family != AF_UNSPEC) { /* Bind to the specified local IP address */ struct socket *socket = server->ssocket; - rc = socket->ops->bind(socket, - (struct sockaddr *) &server->srcaddr, - sizeof(server->srcaddr)); + rc = kernel_bind(socket, + (struct sockaddr *) &server->srcaddr, + sizeof(server->srcaddr)); if (rc < 0) { struct sockaddr_in *saddr4; struct sockaddr_in6 *saddr6; @@ -3042,8 +3046,8 @@ generic_ip_connect(struct TCP_Server_Info *server) socket->sk->sk_sndbuf, socket->sk->sk_rcvbuf, socket->sk->sk_rcvtimeo); - rc = socket->ops->connect(socket, saddr, slen, - server->noblockcnt ? O_NONBLOCK : 0); + rc = kernel_connect(socket, saddr, slen, + server->noblockcnt ? O_NONBLOCK : 0); /* * When mounting SMB root file systems, we do not want to block in * connect. Otherwise bail out and then let cifs_reconnect() perform diff --git a/fs/smb/client/fs_context.c b/fs/smb/client/fs_context.c index e45ce31bbda7..a3493da12ad1 100644 --- a/fs/smb/client/fs_context.c +++ b/fs/smb/client/fs_context.c @@ -1541,6 +1541,7 @@ static int smb3_fs_context_parse_param(struct fs_context *fc, cifs_parse_mount_err: kfree_sensitive(ctx->password); + ctx->password = NULL; return -EINVAL; } diff --git a/fs/smb/client/inode.c b/fs/smb/client/inode.c index de2dfbaae821..d7c302442c1e 100644 --- a/fs/smb/client/inode.c +++ b/fs/smb/client/inode.c @@ -2680,7 +2680,7 @@ int cifs_fiemap(struct inode *inode, struct fiemap_extent_info *fei, u64 start, } cifsFileInfo_put(cfile); - return -ENOTSUPP; + return -EOPNOTSUPP; } int cifs_truncate_page(struct address_space *mapping, loff_t from) diff --git a/fs/smb/client/misc.c b/fs/smb/client/misc.c index 366b755ca913..35b176457bbe 100644 --- a/fs/smb/client/misc.c +++ b/fs/smb/client/misc.c @@ -113,18 +113,22 @@ sesInfoFree(struct cifs_ses *buf_to_free) } struct cifs_tcon * -tconInfoAlloc(void) +tcon_info_alloc(bool dir_leases_enabled) { struct cifs_tcon *ret_buf; ret_buf = kzalloc(sizeof(*ret_buf), GFP_KERNEL); if (!ret_buf) return NULL; - ret_buf->cfids = init_cached_dirs(); - if (!ret_buf->cfids) { - kfree(ret_buf); - return NULL; + + if (dir_leases_enabled == true) { + ret_buf->cfids = init_cached_dirs(); + if (!ret_buf->cfids) { + kfree(ret_buf); + return NULL; + } } + /* else ret_buf->cfids is already set to NULL above */ atomic_inc(&tconInfoAllocCount); ret_buf->status = TID_NEW; diff --git a/fs/smb/client/smb2inode.c b/fs/smb/client/smb2inode.c index b41e2e872b22..0b89f7008ac0 100644 --- a/fs/smb/client/smb2inode.c +++ b/fs/smb/client/smb2inode.c @@ -539,6 +539,9 @@ static int parse_create_response(struct cifs_open_info_data *data, int rc = 0; switch (rsp->hdr.Status) { + case STATUS_IO_REPARSE_TAG_NOT_HANDLED: + reparse_point = true; + break; case STATUS_STOPPED_ON_SYMLINK: rc = smb2_parse_symlink_response(cifs_sb, iov, &data->symlink_target); diff --git a/fs/smb/client/smb2maperror.c b/fs/smb/client/smb2maperror.c index 194799ddd382..1a90dd78b238 100644 --- a/fs/smb/client/smb2maperror.c +++ b/fs/smb/client/smb2maperror.c @@ -877,8 +877,6 @@ static const struct status_to_posix_error smb2_error_map_table[] = { "STATUS_IO_REPARSE_TAG_MISMATCH"}, {STATUS_IO_REPARSE_DATA_INVALID, -EIO, "STATUS_IO_REPARSE_DATA_INVALID"}, - {STATUS_IO_REPARSE_TAG_NOT_HANDLED, -EIO, - "STATUS_IO_REPARSE_TAG_NOT_HANDLED"}, {STATUS_REPARSE_POINT_NOT_RESOLVED, -EIO, "STATUS_REPARSE_POINT_NOT_RESOLVED"}, {STATUS_DIRECTORY_IS_A_REPARSE_POINT, -EIO, diff --git a/fs/smb/client/smb2ops.c b/fs/smb/client/smb2ops.c index d9eda2e958b4..9aeecee6b91b 100644 --- a/fs/smb/client/smb2ops.c +++ b/fs/smb/client/smb2ops.c @@ -297,7 +297,7 @@ smb2_adjust_credits(struct TCP_Server_Info *server, cifs_server_dbg(VFS, "request has less credits (%d) than required (%d)", credits->value, new_val); - return -ENOTSUPP; + return -EOPNOTSUPP; } spin_lock(&server->req_lock); @@ -1161,7 +1161,7 @@ smb2_set_ea(const unsigned int xid, struct cifs_tcon *tcon, /* Use a fudge factor of 256 bytes in case we collide * with a different set_EAs command. */ - if(CIFSMaxBufSize - MAX_SMB2_CREATE_RESPONSE_SIZE - + if (CIFSMaxBufSize - MAX_SMB2_CREATE_RESPONSE_SIZE - MAX_SMB2_CLOSE_RESPONSE_SIZE - 256 < used_len + ea_name_len + ea_value_len + 1) { rc = -ENOSPC; @@ -4591,7 +4591,7 @@ handle_read_data(struct TCP_Server_Info *server, struct mid_q_entry *mid, if (shdr->Command != SMB2_READ) { cifs_server_dbg(VFS, "only big read responses are supported\n"); - return -ENOTSUPP; + return -EOPNOTSUPP; } if (server->ops->is_session_expired && diff --git a/fs/smb/client/smb2pdu.c b/fs/smb/client/smb2pdu.c index 092b0087c9dc..c75a80bb6d9e 100644 --- a/fs/smb/client/smb2pdu.c +++ b/fs/smb/client/smb2pdu.c @@ -89,20 +89,26 @@ smb2_hdr_assemble(struct smb2_hdr *shdr, __le16 smb2_cmd, struct TCP_Server_Info *server) { struct smb3_hdr_req *smb3_hdr; + shdr->ProtocolId = SMB2_PROTO_NUMBER; shdr->StructureSize = cpu_to_le16(64); shdr->Command = smb2_cmd; - if (server->dialect >= SMB30_PROT_ID) { - /* After reconnect SMB3 must set ChannelSequence on subsequent reqs */ - smb3_hdr = (struct smb3_hdr_req *)shdr; - /* if primary channel is not set yet, use default channel for chan sequence num */ - if (SERVER_IS_CHAN(server)) - smb3_hdr->ChannelSequence = - cpu_to_le16(server->primary_server->channel_sequence_num); - else - smb3_hdr->ChannelSequence = cpu_to_le16(server->channel_sequence_num); - } + if (server) { + /* After reconnect SMB3 must set ChannelSequence on subsequent reqs */ + if (server->dialect >= SMB30_PROT_ID) { + smb3_hdr = (struct smb3_hdr_req *)shdr; + /* + * if primary channel is not set yet, use default + * channel for chan sequence num + */ + if (SERVER_IS_CHAN(server)) + smb3_hdr->ChannelSequence = + cpu_to_le16(server->primary_server->channel_sequence_num); + else + smb3_hdr->ChannelSequence = + cpu_to_le16(server->channel_sequence_num); + } spin_lock(&server->req_lock); /* Request up to 10 credits but don't go over the limit. */ if (server->credits >= server->max_credits) @@ -842,7 +848,7 @@ add_posix_context(struct kvec *iov, unsigned int *num_iovec, umode_t mode) iov[num].iov_base = create_posix_buf(mode); if (mode == ACL_NO_MODE) - cifs_dbg(FYI, "Invalid mode\n"); + cifs_dbg(FYI, "%s: no mode\n", __func__); if (iov[num].iov_base == NULL) return -ENOMEM; iov[num].iov_len = sizeof(struct create_posix); @@ -2234,7 +2240,7 @@ create_durable_v2_buf(struct cifs_open_parms *oparms) * (most servers default to 120 seconds) and most clients default to 0. * This can be overridden at mount ("handletimeout=") if the user wants * a different persistent (or resilient) handle timeout for all opens - * opens on a particular SMB3 mount. + * on a particular SMB3 mount. */ buf->dcontext.Timeout = cpu_to_le32(oparms->tcon->handle_timeout); buf->dcontext.Flags = cpu_to_le32(SMB2_DHANDLE_FLAG_PERSISTENT); @@ -2379,7 +2385,7 @@ add_twarp_context(struct kvec *iov, unsigned int *num_iovec, __u64 timewarp) return 0; } -/* See See http://technet.microsoft.com/en-us/library/hh509017(v=ws.10).aspx */ +/* See http://technet.microsoft.com/en-us/library/hh509017(v=ws.10).aspx */ static void setup_owner_group_sids(char *buf) { struct owner_group_sids *sids = (struct owner_group_sids *)buf; @@ -3124,6 +3130,7 @@ void SMB2_ioctl_free(struct smb_rqst *rqst) { int i; + if (rqst && rqst->rq_iov) { cifs_small_buf_release(rqst->rq_iov[0].iov_base); /* request */ for (i = 1; i < rqst->rq_nvec; i++) @@ -3871,7 +3878,7 @@ void smb2_reconnect_server(struct work_struct *work) goto done; /* allocate a dummy tcon struct used for reconnect */ - tcon = tconInfoAlloc(); + tcon = tcon_info_alloc(false); if (!tcon) { resched = true; list_for_each_entry_safe(ses, ses2, &tmp_ses_list, rlist) { diff --git a/fs/smb/client/smbdirect.c b/fs/smb/client/smbdirect.c index 2a2aec8c6112..94df9eec3d8d 100644 --- a/fs/smb/client/smbdirect.c +++ b/fs/smb/client/smbdirect.c @@ -1401,10 +1401,13 @@ create_conn: server->smbd_conn = smbd_get_connection( server, (struct sockaddr *) &server->dstaddr); - if (server->smbd_conn) + if (server->smbd_conn) { cifs_dbg(VFS, "RDMA transport re-established\n"); - - return server->smbd_conn ? 0 : -ENOENT; + trace_smb3_smbd_connect_done(server->hostname, server->conn_id, &server->dstaddr); + return 0; + } + trace_smb3_smbd_connect_err(server->hostname, server->conn_id, &server->dstaddr); + return -ENOENT; } static void destroy_caches_and_workqueue(struct smbd_connection *info) diff --git a/fs/smb/client/trace.h b/fs/smb/client/trace.h index a7e4755bed0f..de199ec9f726 100644 --- a/fs/smb/client/trace.h +++ b/fs/smb/client/trace.h @@ -935,6 +935,8 @@ DEFINE_EVENT(smb3_connect_class, smb3_##name, \ TP_ARGS(hostname, conn_id, addr)) DEFINE_SMB3_CONNECT_EVENT(connect_done); +DEFINE_SMB3_CONNECT_EVENT(smbd_connect_done); +DEFINE_SMB3_CONNECT_EVENT(smbd_connect_err); DECLARE_EVENT_CLASS(smb3_connect_err_class, TP_PROTO(char *hostname, __u64 conn_id, diff --git a/fs/smb/client/transport.c b/fs/smb/client/transport.c index 1b5d9794ed5b..14710afdc2a3 100644 --- a/fs/smb/client/transport.c +++ b/fs/smb/client/transport.c @@ -18,7 +18,7 @@ #include #include #include -#include +#include #include #include #include @@ -35,6 +35,8 @@ void cifs_wake_up_task(struct mid_q_entry *mid) { + if (mid->mid_state == MID_RESPONSE_RECEIVED) + mid->mid_state = MID_RESPONSE_READY; wake_up_process(mid->callback_data); } @@ -87,7 +89,8 @@ static void __release_mid(struct kref *refcount) struct TCP_Server_Info *server = midEntry->server; if (midEntry->resp_buf && (midEntry->mid_flags & MID_WAIT_CANCELLED) && - midEntry->mid_state == MID_RESPONSE_RECEIVED && + (midEntry->mid_state == MID_RESPONSE_RECEIVED || + midEntry->mid_state == MID_RESPONSE_READY) && server->ops->handle_cancelled_mid) server->ops->handle_cancelled_mid(midEntry, server); @@ -737,7 +740,8 @@ wait_for_response(struct TCP_Server_Info *server, struct mid_q_entry *midQ) int error; error = wait_event_state(server->response_q, - midQ->mid_state != MID_REQUEST_SUBMITTED, + midQ->mid_state != MID_REQUEST_SUBMITTED && + midQ->mid_state != MID_RESPONSE_RECEIVED, (TASK_KILLABLE|TASK_FREEZABLE_UNSAFE)); if (error < 0) return -ERESTARTSYS; @@ -890,7 +894,7 @@ cifs_sync_mid_result(struct mid_q_entry *mid, struct TCP_Server_Info *server) spin_lock(&server->mid_lock); switch (mid->mid_state) { - case MID_RESPONSE_RECEIVED: + case MID_RESPONSE_READY: spin_unlock(&server->mid_lock); return rc; case MID_RETRY_NEEDED: @@ -989,6 +993,9 @@ cifs_compound_callback(struct mid_q_entry *mid) credits.instance = server->reconnect_instance; add_credits(server, &credits, mid->optype); + + if (mid->mid_state == MID_RESPONSE_RECEIVED) + mid->mid_state = MID_RESPONSE_READY; } static void @@ -1209,7 +1216,8 @@ compound_send_recv(const unsigned int xid, struct cifs_ses *ses, send_cancel(server, &rqst[i], midQ[i]); spin_lock(&server->mid_lock); midQ[i]->mid_flags |= MID_WAIT_CANCELLED; - if (midQ[i]->mid_state == MID_REQUEST_SUBMITTED) { + if (midQ[i]->mid_state == MID_REQUEST_SUBMITTED || + midQ[i]->mid_state == MID_RESPONSE_RECEIVED) { midQ[i]->callback = cifs_cancelled_callback; cancelled_mid[i] = true; credits[i].value = 0; @@ -1230,7 +1238,7 @@ compound_send_recv(const unsigned int xid, struct cifs_ses *ses, } if (!midQ[i]->resp_buf || - midQ[i]->mid_state != MID_RESPONSE_RECEIVED) { + midQ[i]->mid_state != MID_RESPONSE_READY) { rc = -EIO; cifs_dbg(FYI, "Bad MID state?\n"); goto out; @@ -1417,7 +1425,8 @@ SendReceive(const unsigned int xid, struct cifs_ses *ses, if (rc != 0) { send_cancel(server, &rqst, midQ); spin_lock(&server->mid_lock); - if (midQ->mid_state == MID_REQUEST_SUBMITTED) { + if (midQ->mid_state == MID_REQUEST_SUBMITTED || + midQ->mid_state == MID_RESPONSE_RECEIVED) { /* no longer considered to be "in-flight" */ midQ->callback = release_mid; spin_unlock(&server->mid_lock); @@ -1434,7 +1443,7 @@ SendReceive(const unsigned int xid, struct cifs_ses *ses, } if (!midQ->resp_buf || !out_buf || - midQ->mid_state != MID_RESPONSE_RECEIVED) { + midQ->mid_state != MID_RESPONSE_READY) { rc = -EIO; cifs_server_dbg(VFS, "Bad MID state?\n"); goto out; @@ -1558,14 +1567,16 @@ SendReceiveBlockingLock(const unsigned int xid, struct cifs_tcon *tcon, /* Wait for a reply - allow signals to interrupt. */ rc = wait_event_interruptible(server->response_q, - (!(midQ->mid_state == MID_REQUEST_SUBMITTED)) || + (!(midQ->mid_state == MID_REQUEST_SUBMITTED || + midQ->mid_state == MID_RESPONSE_RECEIVED)) || ((server->tcpStatus != CifsGood) && (server->tcpStatus != CifsNew))); /* Were we interrupted by a signal ? */ spin_lock(&server->srv_lock); if ((rc == -ERESTARTSYS) && - (midQ->mid_state == MID_REQUEST_SUBMITTED) && + (midQ->mid_state == MID_REQUEST_SUBMITTED || + midQ->mid_state == MID_RESPONSE_RECEIVED) && ((server->tcpStatus == CifsGood) || (server->tcpStatus == CifsNew))) { spin_unlock(&server->srv_lock); @@ -1596,7 +1607,8 @@ SendReceiveBlockingLock(const unsigned int xid, struct cifs_tcon *tcon, if (rc) { send_cancel(server, &rqst, midQ); spin_lock(&server->mid_lock); - if (midQ->mid_state == MID_REQUEST_SUBMITTED) { + if (midQ->mid_state == MID_REQUEST_SUBMITTED || + midQ->mid_state == MID_RESPONSE_RECEIVED) { /* no longer considered to be "in-flight" */ midQ->callback = release_mid; spin_unlock(&server->mid_lock); @@ -1616,7 +1628,7 @@ SendReceiveBlockingLock(const unsigned int xid, struct cifs_tcon *tcon, return rc; /* rcvd frame is ok */ - if (out_buf == NULL || midQ->mid_state != MID_RESPONSE_RECEIVED) { + if (out_buf == NULL || midQ->mid_state != MID_RESPONSE_READY) { rc = -EIO; cifs_tcon_dbg(VFS, "Bad MID state?\n"); goto out; diff --git a/fs/smb/server/connection.c b/fs/smb/server/connection.c index 0d990c2f33cd..4b38c3a285f6 100644 --- a/fs/smb/server/connection.c +++ b/fs/smb/server/connection.c @@ -84,6 +84,8 @@ struct ksmbd_conn *ksmbd_conn_alloc(void) spin_lock_init(&conn->llist_lock); INIT_LIST_HEAD(&conn->lock_list); + init_rwsem(&conn->session_lock); + down_write(&conn_list_lock); list_add(&conn->conns_list, &conn_list); up_write(&conn_list_lock); @@ -197,6 +199,9 @@ int ksmbd_conn_write(struct ksmbd_work *work) if (work->send_no_response) return 0; + if (!work->iov_idx) + return -EINVAL; + ksmbd_conn_lock(conn); sent = conn->transport->ops->writev(conn->transport, work->iov, work->iov_cnt, diff --git a/fs/smb/server/connection.h b/fs/smb/server/connection.h index ab2583f030ce..3c005246a32e 100644 --- a/fs/smb/server/connection.h +++ b/fs/smb/server/connection.h @@ -50,6 +50,7 @@ struct ksmbd_conn { struct nls_table *local_nls; struct unicode_map *um; struct list_head conns_list; + struct rw_semaphore session_lock; /* smb session 1 per user */ struct xarray sessions; unsigned long last_active; diff --git a/fs/smb/server/mgmt/tree_connect.c b/fs/smb/server/mgmt/tree_connect.c index 408cddf2f094..d2c81a8a11dd 100644 --- a/fs/smb/server/mgmt/tree_connect.c +++ b/fs/smb/server/mgmt/tree_connect.c @@ -73,7 +73,10 @@ ksmbd_tree_conn_connect(struct ksmbd_conn *conn, struct ksmbd_session *sess, tree_conn->user = sess->user; tree_conn->share_conf = sc; + tree_conn->t_state = TREE_NEW; status.tree_conn = tree_conn; + atomic_set(&tree_conn->refcount, 1); + init_waitqueue_head(&tree_conn->refcount_q); ret = xa_err(xa_store(&sess->tree_conns, tree_conn->id, tree_conn, GFP_KERNEL)); @@ -93,14 +96,33 @@ out_error: return status; } +void ksmbd_tree_connect_put(struct ksmbd_tree_connect *tcon) +{ + /* + * Checking waitqueue to releasing tree connect on + * tree disconnect. waitqueue_active is safe because it + * uses atomic operation for condition. + */ + if (!atomic_dec_return(&tcon->refcount) && + waitqueue_active(&tcon->refcount_q)) + wake_up(&tcon->refcount_q); +} + int ksmbd_tree_conn_disconnect(struct ksmbd_session *sess, struct ksmbd_tree_connect *tree_conn) { int ret; + write_lock(&sess->tree_conns_lock); + xa_erase(&sess->tree_conns, tree_conn->id); + write_unlock(&sess->tree_conns_lock); + + if (!atomic_dec_and_test(&tree_conn->refcount)) + wait_event(tree_conn->refcount_q, + atomic_read(&tree_conn->refcount) == 0); + ret = ksmbd_ipc_tree_disconnect_request(sess->id, tree_conn->id); ksmbd_release_tree_conn_id(sess, tree_conn->id); - xa_erase(&sess->tree_conns, tree_conn->id); ksmbd_share_config_put(tree_conn->share_conf); kfree(tree_conn); return ret; @@ -111,11 +133,15 @@ struct ksmbd_tree_connect *ksmbd_tree_conn_lookup(struct ksmbd_session *sess, { struct ksmbd_tree_connect *tcon; + read_lock(&sess->tree_conns_lock); tcon = xa_load(&sess->tree_conns, id); if (tcon) { - if (test_bit(TREE_CONN_EXPIRE, &tcon->status)) + if (tcon->t_state != TREE_CONNECTED) + tcon = NULL; + else if (!atomic_inc_not_zero(&tcon->refcount)) tcon = NULL; } + read_unlock(&sess->tree_conns_lock); return tcon; } @@ -129,8 +155,18 @@ int ksmbd_tree_conn_session_logoff(struct ksmbd_session *sess) if (!sess) return -EINVAL; - xa_for_each(&sess->tree_conns, id, tc) + xa_for_each(&sess->tree_conns, id, tc) { + write_lock(&sess->tree_conns_lock); + if (tc->t_state == TREE_DISCONNECTED) { + write_unlock(&sess->tree_conns_lock); + ret = -ENOENT; + continue; + } + tc->t_state = TREE_DISCONNECTED; + write_unlock(&sess->tree_conns_lock); + ret |= ksmbd_tree_conn_disconnect(sess, tc); + } xa_destroy(&sess->tree_conns); return ret; } diff --git a/fs/smb/server/mgmt/tree_connect.h b/fs/smb/server/mgmt/tree_connect.h index 562d647ad9fa..6377a70b811c 100644 --- a/fs/smb/server/mgmt/tree_connect.h +++ b/fs/smb/server/mgmt/tree_connect.h @@ -14,7 +14,11 @@ struct ksmbd_share_config; struct ksmbd_user; struct ksmbd_conn; -#define TREE_CONN_EXPIRE 1 +enum { + TREE_NEW = 0, + TREE_CONNECTED, + TREE_DISCONNECTED +}; struct ksmbd_tree_connect { int id; @@ -27,7 +31,9 @@ struct ksmbd_tree_connect { int maximal_access; bool posix_extensions; - unsigned long status; + atomic_t refcount; + wait_queue_head_t refcount_q; + unsigned int t_state; }; struct ksmbd_tree_conn_status { @@ -46,6 +52,7 @@ struct ksmbd_session; struct ksmbd_tree_conn_status ksmbd_tree_conn_connect(struct ksmbd_conn *conn, struct ksmbd_session *sess, const char *share_name); +void ksmbd_tree_connect_put(struct ksmbd_tree_connect *tcon); int ksmbd_tree_conn_disconnect(struct ksmbd_session *sess, struct ksmbd_tree_connect *tree_conn); diff --git a/fs/smb/server/mgmt/user_session.c b/fs/smb/server/mgmt/user_session.c index 8a5dcab05614..15f68ee05089 100644 --- a/fs/smb/server/mgmt/user_session.c +++ b/fs/smb/server/mgmt/user_session.c @@ -174,7 +174,7 @@ static void ksmbd_expire_session(struct ksmbd_conn *conn) unsigned long id; struct ksmbd_session *sess; - down_write(&sessions_table_lock); + down_write(&conn->session_lock); xa_for_each(&conn->sessions, id, sess) { if (sess->state != SMB2_SESSION_VALID || time_after(jiffies, @@ -185,7 +185,7 @@ static void ksmbd_expire_session(struct ksmbd_conn *conn) continue; } } - up_write(&sessions_table_lock); + up_write(&conn->session_lock); } int ksmbd_session_register(struct ksmbd_conn *conn, @@ -227,7 +227,9 @@ void ksmbd_sessions_deregister(struct ksmbd_conn *conn) } } } + up_write(&sessions_table_lock); + down_write(&conn->session_lock); xa_for_each(&conn->sessions, id, sess) { unsigned long chann_id; struct channel *chann; @@ -244,7 +246,7 @@ void ksmbd_sessions_deregister(struct ksmbd_conn *conn) ksmbd_session_destroy(sess); } } - up_write(&sessions_table_lock); + up_write(&conn->session_lock); } struct ksmbd_session *ksmbd_session_lookup(struct ksmbd_conn *conn, @@ -252,9 +254,11 @@ struct ksmbd_session *ksmbd_session_lookup(struct ksmbd_conn *conn, { struct ksmbd_session *sess; + down_read(&conn->session_lock); sess = xa_load(&conn->sessions, id); if (sess) sess->last_active = jiffies; + up_read(&conn->session_lock); return sess; } @@ -351,6 +355,7 @@ static struct ksmbd_session *__session_create(int protocol) xa_init(&sess->ksmbd_chann_list); xa_init(&sess->rpc_handle_list); sess->sequence_number = 1; + rwlock_init(&sess->tree_conns_lock); ret = __init_smb2_session(sess); if (ret) diff --git a/fs/smb/server/mgmt/user_session.h b/fs/smb/server/mgmt/user_session.h index f99d475b28db..63cb08fffde8 100644 --- a/fs/smb/server/mgmt/user_session.h +++ b/fs/smb/server/mgmt/user_session.h @@ -60,6 +60,7 @@ struct ksmbd_session { struct ksmbd_file_table file_table; unsigned long last_active; + rwlock_t tree_conns_lock; }; static inline int test_session_flag(struct ksmbd_session *sess, int bit) diff --git a/fs/smb/server/server.c b/fs/smb/server/server.c index 5ab2f52f9b35..3079e607c5fe 100644 --- a/fs/smb/server/server.c +++ b/fs/smb/server/server.c @@ -115,8 +115,10 @@ static int __process_request(struct ksmbd_work *work, struct ksmbd_conn *conn, if (check_conn_state(work)) return SERVER_HANDLER_CONTINUE; - if (ksmbd_verify_smb_message(work)) + if (ksmbd_verify_smb_message(work)) { + conn->ops->set_rsp_status(work, STATUS_INVALID_PARAMETER); return SERVER_HANDLER_ABORT; + } command = conn->ops->get_cmd_val(work); *cmd = command; @@ -239,6 +241,8 @@ static void __handle_ksmbd_work(struct ksmbd_work *work, } while (is_chained == true); send: + if (work->tcon) + ksmbd_tree_connect_put(work->tcon); smb3_preauth_hash_rsp(work); if (work->sess && work->sess->enc && work->encrypted && conn->ops->encrypt_resp) { diff --git a/fs/smb/server/smb2misc.c b/fs/smb/server/smb2misc.c index e881df1d10cb..23bd3d1209df 100644 --- a/fs/smb/server/smb2misc.c +++ b/fs/smb/server/smb2misc.c @@ -440,10 +440,8 @@ int ksmbd_smb2_check_message(struct ksmbd_work *work) validate_credit: if ((work->conn->vals->capabilities & SMB2_GLOBAL_CAP_LARGE_MTU) && - smb2_validate_credit_charge(work->conn, hdr)) { - work->conn->ops->set_rsp_status(work, STATUS_INVALID_PARAMETER); + smb2_validate_credit_charge(work->conn, hdr)) return 1; - } return 0; } diff --git a/fs/smb/server/smb2pdu.c b/fs/smb/server/smb2pdu.c index 749660110878..93262ca3f58a 100644 --- a/fs/smb/server/smb2pdu.c +++ b/fs/smb/server/smb2pdu.c @@ -231,11 +231,12 @@ void set_smb2_rsp_status(struct ksmbd_work *work, __le32 err) { struct smb2_hdr *rsp_hdr; - if (work->next_smb2_rcv_hdr_off) - rsp_hdr = ksmbd_resp_buf_next(work); - else - rsp_hdr = smb2_get_msg(work->response_buf); + rsp_hdr = smb2_get_msg(work->response_buf); rsp_hdr->Status = err; + + work->iov_idx = 0; + work->iov_cnt = 0; + work->next_smb2_rcv_hdr_off = 0; smb2_set_err_rsp(work); } @@ -1993,6 +1994,9 @@ int smb2_tree_connect(struct ksmbd_work *work) if (conn->posix_ext_supported) status.tree_conn->posix_extensions = true; + write_lock(&sess->tree_conns_lock); + status.tree_conn->t_state = TREE_CONNECTED; + write_unlock(&sess->tree_conns_lock); rsp->StructureSize = cpu_to_le16(16); out_err1: rsp->Capabilities = 0; @@ -2122,27 +2126,50 @@ int smb2_tree_disconnect(struct ksmbd_work *work) ksmbd_debug(SMB, "request\n"); + if (!tcon) { + ksmbd_debug(SMB, "Invalid tid %d\n", req->hdr.Id.SyncId.TreeId); + + rsp->hdr.Status = STATUS_NETWORK_NAME_DELETED; + err = -ENOENT; + goto err_out; + } + + ksmbd_close_tree_conn_fds(work); + + write_lock(&sess->tree_conns_lock); + if (tcon->t_state == TREE_DISCONNECTED) { + write_unlock(&sess->tree_conns_lock); + rsp->hdr.Status = STATUS_NETWORK_NAME_DELETED; + err = -ENOENT; + goto err_out; + } + + WARN_ON_ONCE(atomic_dec_and_test(&tcon->refcount)); + tcon->t_state = TREE_DISCONNECTED; + write_unlock(&sess->tree_conns_lock); + + err = ksmbd_tree_conn_disconnect(sess, tcon); + if (err) { + rsp->hdr.Status = STATUS_NETWORK_NAME_DELETED; + goto err_out; + } + + work->tcon = NULL; + rsp->StructureSize = cpu_to_le16(4); err = ksmbd_iov_pin_rsp(work, rsp, sizeof(struct smb2_tree_disconnect_rsp)); if (err) { rsp->hdr.Status = STATUS_INSUFFICIENT_RESOURCES; - smb2_set_err_rsp(work); - return err; + goto err_out; } - if (!tcon || test_and_set_bit(TREE_CONN_EXPIRE, &tcon->status)) { - ksmbd_debug(SMB, "Invalid tid %d\n", req->hdr.Id.SyncId.TreeId); + return 0; - rsp->hdr.Status = STATUS_NETWORK_NAME_DELETED; - smb2_set_err_rsp(work); - return -ENOENT; - } +err_out: + smb2_set_err_rsp(work); + return err; - ksmbd_close_tree_conn_fds(work); - ksmbd_tree_conn_disconnect(sess, tcon); - work->tcon = NULL; - return 0; } /** @@ -2164,17 +2191,17 @@ int smb2_session_logoff(struct ksmbd_work *work) ksmbd_debug(SMB, "request\n"); - sess_id = le64_to_cpu(req->hdr.SessionId); - - rsp->StructureSize = cpu_to_le16(4); - err = ksmbd_iov_pin_rsp(work, rsp, sizeof(struct smb2_logoff_rsp)); - if (err) { - rsp->hdr.Status = STATUS_INSUFFICIENT_RESOURCES; + ksmbd_conn_lock(conn); + if (!ksmbd_conn_good(conn)) { + ksmbd_conn_unlock(conn); + rsp->hdr.Status = STATUS_NETWORK_NAME_DELETED; smb2_set_err_rsp(work); - return err; + return -ENOENT; } - + sess_id = le64_to_cpu(req->hdr.SessionId); ksmbd_all_conn_set_status(sess_id, KSMBD_SESS_NEED_RECONNECT); + ksmbd_conn_unlock(conn); + ksmbd_close_session_fds(work); ksmbd_conn_wait_idle(conn, sess_id); @@ -2196,6 +2223,14 @@ int smb2_session_logoff(struct ksmbd_work *work) ksmbd_free_user(sess->user); sess->user = NULL; ksmbd_all_conn_set_status(sess_id, KSMBD_SESS_NEED_NEGOTIATE); + + rsp->StructureSize = cpu_to_le16(4); + err = ksmbd_iov_pin_rsp(work, rsp, sizeof(struct smb2_logoff_rsp)); + if (err) { + rsp->hdr.Status = STATUS_INSUFFICIENT_RESOURCES; + smb2_set_err_rsp(work); + return err; + } return 0; } @@ -3370,8 +3405,10 @@ err_out: } ksmbd_revert_fsids(work); err_out1: - if (!rc) + if (!rc) { + ksmbd_update_fstate(&work->sess->file_table, fp, FP_INITED); rc = ksmbd_iov_pin_rsp(work, (void *)rsp, iov_len); + } if (rc) { if (rc == -EINVAL) rsp->hdr.Status = STATUS_INVALID_PARAMETER; @@ -6115,12 +6152,12 @@ static noinline int smb2_read_pipe(struct ksmbd_work *work) memcpy(aux_payload_buf, rpc_resp->payload, rpc_resp->payload_sz); nbytes = rpc_resp->payload_sz; - kvfree(rpc_resp); err = ksmbd_iov_pin_rsp_read(work, (void *)rsp, offsetof(struct smb2_read_rsp, Buffer), aux_payload_buf, nbytes); if (err) goto out; + kvfree(rpc_resp); } else { err = ksmbd_iov_pin_rsp(work, (void *)rsp, offsetof(struct smb2_read_rsp, Buffer)); @@ -6312,7 +6349,7 @@ int smb2_read(struct ksmbd_work *work) aux_payload_buf, nbytes); kvfree(aux_payload_buf); - + aux_payload_buf = NULL; nbytes = 0; if (remain_bytes < 0) { err = (int)remain_bytes; @@ -7028,10 +7065,6 @@ skip: ksmbd_debug(SMB, "would have to wait for getting lock\n"); - spin_lock(&work->conn->llist_lock); - list_add_tail(&smb_lock->clist, - &work->conn->lock_list); - spin_unlock(&work->conn->llist_lock); list_add(&smb_lock->llist, &rollback_list); argv = kmalloc(sizeof(void *), GFP_KERNEL); @@ -7062,9 +7095,6 @@ skip: if (work->state != KSMBD_WORK_ACTIVE) { list_del(&smb_lock->llist); - spin_lock(&work->conn->llist_lock); - list_del(&smb_lock->clist); - spin_unlock(&work->conn->llist_lock); locks_free_lock(flock); if (work->state == KSMBD_WORK_CANCELLED) { @@ -7084,19 +7114,16 @@ skip: } list_del(&smb_lock->llist); - spin_lock(&work->conn->llist_lock); - list_del(&smb_lock->clist); - spin_unlock(&work->conn->llist_lock); release_async_work(work); goto retry; } else if (!rc) { + list_add(&smb_lock->llist, &rollback_list); spin_lock(&work->conn->llist_lock); list_add_tail(&smb_lock->clist, &work->conn->lock_list); list_add_tail(&smb_lock->flist, &fp->lock_list); spin_unlock(&work->conn->llist_lock); - list_add(&smb_lock->llist, &rollback_list); ksmbd_debug(SMB, "successful in taking lock\n"); } else { goto out; @@ -8036,10 +8063,10 @@ static void smb20_oplock_break_ack(struct ksmbd_work *work) goto err_out; } - opinfo_put(opinfo); - ksmbd_fd_put(work, fp); opinfo->op_state = OPLOCK_STATE_NONE; wake_up_interruptible_all(&opinfo->oplock_q); + opinfo_put(opinfo); + ksmbd_fd_put(work, fp); rsp->StructureSize = cpu_to_le16(24); rsp->OplockLevel = rsp_oplevel; diff --git a/fs/smb/server/smbacl.c b/fs/smb/server/smbacl.c index e5e438bf5499..6c0305be895e 100644 --- a/fs/smb/server/smbacl.c +++ b/fs/smb/server/smbacl.c @@ -1420,7 +1420,6 @@ int set_info_sec(struct ksmbd_conn *conn, struct ksmbd_tree_connect *tcon, out: posix_acl_release(fattr.cf_acls); posix_acl_release(fattr.cf_dacls); - mark_inode_dirty(inode); return rc; } diff --git a/fs/smb/server/vfs_cache.c b/fs/smb/server/vfs_cache.c index f41f8d6108ce..c91eac6514dd 100644 --- a/fs/smb/server/vfs_cache.c +++ b/fs/smb/server/vfs_cache.c @@ -106,7 +106,7 @@ int ksmbd_query_inode_status(struct inode *inode) ci = __ksmbd_inode_lookup(inode); if (ci) { ret = KSMBD_INODE_STATUS_OK; - if (ci->m_flags & S_DEL_PENDING) + if (ci->m_flags & (S_DEL_PENDING | S_DEL_ON_CLS)) ret = KSMBD_INODE_STATUS_PENDING_DELETE; atomic_dec(&ci->m_count); } @@ -116,7 +116,7 @@ int ksmbd_query_inode_status(struct inode *inode) bool ksmbd_inode_pending_delete(struct ksmbd_file *fp) { - return (fp->f_ci->m_flags & S_DEL_PENDING); + return (fp->f_ci->m_flags & (S_DEL_PENDING | S_DEL_ON_CLS)); } void ksmbd_set_inode_pending_delete(struct ksmbd_file *fp) @@ -333,6 +333,9 @@ static void __ksmbd_close_fd(struct ksmbd_file_table *ft, struct ksmbd_file *fp) static struct ksmbd_file *ksmbd_fp_get(struct ksmbd_file *fp) { + if (fp->f_state != FP_INITED) + return NULL; + if (!atomic_inc_not_zero(&fp->refcount)) return NULL; return fp; @@ -382,15 +385,20 @@ int ksmbd_close_fd(struct ksmbd_work *work, u64 id) return 0; ft = &work->sess->file_table; - read_lock(&ft->lock); + write_lock(&ft->lock); fp = idr_find(ft->idr, id); if (fp) { set_close_state_blocked_works(fp); - if (!atomic_dec_and_test(&fp->refcount)) + if (fp->f_state != FP_INITED) fp = NULL; + else { + fp->f_state = FP_CLOSED; + if (!atomic_dec_and_test(&fp->refcount)) + fp = NULL; + } } - read_unlock(&ft->lock); + write_unlock(&ft->lock); if (!fp) return -EINVAL; @@ -570,6 +578,7 @@ struct ksmbd_file *ksmbd_open_fd(struct ksmbd_work *work, struct file *filp) fp->tcon = work->tcon; fp->volatile_id = KSMBD_NO_FID; fp->persistent_id = KSMBD_NO_FID; + fp->f_state = FP_NEW; fp->f_ci = ksmbd_inode_get(fp); if (!fp->f_ci) { @@ -591,6 +600,17 @@ err_out: return ERR_PTR(ret); } +void ksmbd_update_fstate(struct ksmbd_file_table *ft, struct ksmbd_file *fp, + unsigned int state) +{ + if (!fp) + return; + + write_lock(&ft->lock); + fp->f_state = state; + write_unlock(&ft->lock); +} + static int __close_file_table_ids(struct ksmbd_file_table *ft, struct ksmbd_tree_connect *tcon, diff --git a/fs/smb/server/vfs_cache.h b/fs/smb/server/vfs_cache.h index fcb13413fa8d..03d0bf941216 100644 --- a/fs/smb/server/vfs_cache.h +++ b/fs/smb/server/vfs_cache.h @@ -60,6 +60,12 @@ struct ksmbd_inode { __le32 m_fattr; }; +enum { + FP_NEW = 0, + FP_INITED, + FP_CLOSED +}; + struct ksmbd_file { struct file *filp; u64 persistent_id; @@ -98,6 +104,7 @@ struct ksmbd_file { /* if ls is happening on directory, below is valid*/ struct ksmbd_readdir_data readdir_data; int dot_dotdot[2]; + unsigned int f_state; }; static inline void set_ctx_actor(struct dir_context *ctx, @@ -142,6 +149,8 @@ int ksmbd_close_inode_fds(struct ksmbd_work *work, struct inode *inode); int ksmbd_init_global_file_table(void); void ksmbd_free_global_file_table(void); void ksmbd_set_fd_limit(unsigned long limit); +void ksmbd_update_fstate(struct ksmbd_file_table *ft, struct ksmbd_file *fp, + unsigned int state); /* * INODE hash diff --git a/fs/stat.c b/fs/stat.c index 6822ac77aec2..d43a5cc1bfa4 100644 --- a/fs/stat.c +++ b/fs/stat.c @@ -26,37 +26,6 @@ #include "internal.h" #include "mount.h" -/** - * fill_mg_cmtime - Fill in the mtime and ctime and flag ctime as QUERIED - * @stat: where to store the resulting values - * @request_mask: STATX_* values requested - * @inode: inode from which to grab the c/mtime - * - * Given @inode, grab the ctime and mtime out if it and store the result - * in @stat. When fetching the value, flag it as queried so the next write - * will use a fine-grained timestamp. - */ -void fill_mg_cmtime(struct kstat *stat, u32 request_mask, struct inode *inode) -{ - atomic_long_t *pnsec = (atomic_long_t *)&inode->__i_ctime.tv_nsec; - - /* If neither time was requested, then don't report them */ - if (!(request_mask & (STATX_CTIME|STATX_MTIME))) { - stat->result_mask &= ~(STATX_CTIME|STATX_MTIME); - return; - } - - stat->mtime = inode->i_mtime; - stat->ctime.tv_sec = inode->__i_ctime.tv_sec; - /* - * Atomically set the QUERIED flag and fetch the new value with - * the flag masked off. - */ - stat->ctime.tv_nsec = atomic_long_fetch_or(I_CTIME_QUERIED, pnsec) & - ~I_CTIME_QUERIED; -} -EXPORT_SYMBOL(fill_mg_cmtime); - /** * generic_fillattr - Fill in the basic attributes from the inode struct * @idmap: idmap of the mount the inode was found from @@ -89,14 +58,8 @@ void generic_fillattr(struct mnt_idmap *idmap, u32 request_mask, stat->rdev = inode->i_rdev; stat->size = i_size_read(inode); stat->atime = inode->i_atime; - - if (is_mgtime(inode)) { - fill_mg_cmtime(stat, request_mask, inode); - } else { - stat->mtime = inode->i_mtime; - stat->ctime = inode_get_ctime(inode); - } - + stat->mtime = inode->i_mtime; + stat->ctime = inode_get_ctime(inode); stat->blksize = i_blocksize(inode); stat->blocks = inode->i_blocks; @@ -419,12 +382,6 @@ SYSCALL_DEFINE2(fstat, unsigned int, fd, struct __old_kernel_stat __user *, stat #ifdef __ARCH_WANT_NEW_STAT -#if BITS_PER_LONG == 32 -# define choose_32_64(a,b) a -#else -# define choose_32_64(a,b) b -#endif - #ifndef INIT_STRUCT_STAT_PADDING # define INIT_STRUCT_STAT_PADDING(st) memset(&st, 0, sizeof(st)) #endif diff --git a/fs/tracefs/event_inode.c b/fs/tracefs/event_inode.c index 237c6f370ad9..8c8d64e76103 100644 --- a/fs/tracefs/event_inode.c +++ b/fs/tracefs/event_inode.c @@ -70,6 +70,7 @@ static struct dentry *eventfs_root_lookup(struct inode *dir, struct dentry *dentry, unsigned int flags); static int dcache_dir_open_wrapper(struct inode *inode, struct file *file); +static int dcache_readdir_wrapper(struct file *file, struct dir_context *ctx); static int eventfs_release(struct inode *inode, struct file *file); static const struct inode_operations eventfs_root_dir_inode_operations = { @@ -79,7 +80,7 @@ static const struct inode_operations eventfs_root_dir_inode_operations = { static const struct file_operations eventfs_file_operations = { .open = dcache_dir_open_wrapper, .read = generic_read_dir, - .iterate_shared = dcache_readdir, + .iterate_shared = dcache_readdir_wrapper, .llseek = generic_file_llseek, .release = eventfs_release, }; @@ -185,17 +186,49 @@ static struct dentry *create_dir(const char *name, struct dentry *parent, void * /** * eventfs_set_ef_status_free - set the ef->status to free + * @ti: the tracefs_inode of the dentry * @dentry: dentry who's status to be freed * * eventfs_set_ef_status_free will be called if no more * references remain */ -void eventfs_set_ef_status_free(struct dentry *dentry) +void eventfs_set_ef_status_free(struct tracefs_inode *ti, struct dentry *dentry) { struct tracefs_inode *ti_parent; - struct eventfs_file *ef; + struct eventfs_inode *ei; + struct eventfs_file *ef, *tmp; + + /* The top level events directory may be freed by this */ + if (unlikely(ti->flags & TRACEFS_EVENT_TOP_INODE)) { + LIST_HEAD(ef_del_list); + + mutex_lock(&eventfs_mutex); + + ei = ti->private; + + /* Record all the top level files */ + list_for_each_entry_srcu(ef, &ei->e_top_files, list, + lockdep_is_held(&eventfs_mutex)) { + list_add_tail(&ef->del_list, &ef_del_list); + } + + /* Nothing should access this, but just in case! */ + ti->private = NULL; + + mutex_unlock(&eventfs_mutex); + + /* Now safely free the top level files and their children */ + list_for_each_entry_safe(ef, tmp, &ef_del_list, del_list) { + list_del(&ef->del_list); + eventfs_remove(ef); + } + + kfree(ei); + return; + } mutex_lock(&eventfs_mutex); + ti_parent = get_tracefs(dentry->d_parent->d_inode); if (!ti_parent || !(ti_parent->flags & TRACEFS_EVENT_INODE)) goto out; @@ -364,6 +397,11 @@ static struct dentry *eventfs_root_lookup(struct inode *dir, return ret; } +struct dentry_list { + void *cursor; + struct dentry **dentries; +}; + /** * eventfs_release - called to release eventfs file/dir * @inode: inode to be released @@ -372,26 +410,25 @@ static struct dentry *eventfs_root_lookup(struct inode *dir, static int eventfs_release(struct inode *inode, struct file *file) { struct tracefs_inode *ti; - struct eventfs_inode *ei; - struct eventfs_file *ef; - struct dentry *dentry; - int idx; + struct dentry_list *dlist = file->private_data; + void *cursor; + int i; ti = get_tracefs(inode); if (!(ti->flags & TRACEFS_EVENT_INODE)) return -EINVAL; - ei = ti->private; - idx = srcu_read_lock(&eventfs_srcu); - list_for_each_entry_srcu(ef, &ei->e_top_files, list, - srcu_read_lock_held(&eventfs_srcu)) { - mutex_lock(&eventfs_mutex); - dentry = ef->dentry; - mutex_unlock(&eventfs_mutex); - if (dentry) - dput(dentry); + if (WARN_ON_ONCE(!dlist)) + return -EINVAL; + + for (i = 0; dlist->dentries && dlist->dentries[i]; i++) { + dput(dlist->dentries[i]); } - srcu_read_unlock(&eventfs_srcu, idx); + + cursor = dlist->cursor; + kfree(dlist->dentries); + kfree(dlist); + file->private_data = cursor; return dcache_dir_close(inode, file); } @@ -410,21 +447,70 @@ static int dcache_dir_open_wrapper(struct inode *inode, struct file *file) struct tracefs_inode *ti; struct eventfs_inode *ei; struct eventfs_file *ef; + struct dentry_list *dlist; + struct dentry **dentries = NULL; struct dentry *dentry = file_dentry(file); + struct dentry *d; struct inode *f_inode = file_inode(file); + int cnt = 0; int idx; + int ret; ti = get_tracefs(f_inode); if (!(ti->flags & TRACEFS_EVENT_INODE)) return -EINVAL; + if (WARN_ON_ONCE(file->private_data)) + return -EINVAL; + + dlist = kmalloc(sizeof(*dlist), GFP_KERNEL); + if (!dlist) + return -ENOMEM; + ei = ti->private; idx = srcu_read_lock(&eventfs_srcu); - list_for_each_entry_rcu(ef, &ei->e_top_files, list) { - create_dentry(ef, dentry, false); + list_for_each_entry_srcu(ef, &ei->e_top_files, list, + srcu_read_lock_held(&eventfs_srcu)) { + d = create_dentry(ef, dentry, false); + if (d) { + struct dentry **tmp; + + tmp = krealloc(dentries, sizeof(d) * (cnt + 2), GFP_KERNEL); + if (!tmp) + break; + tmp[cnt] = d; + tmp[cnt + 1] = NULL; + cnt++; + dentries = tmp; + } } srcu_read_unlock(&eventfs_srcu, idx); - return dcache_dir_open(inode, file); + ret = dcache_dir_open(inode, file); + + /* + * dcache_dir_open() sets file->private_data to a dentry cursor. + * Need to save that but also save all the dentries that were + * opened by this function. + */ + dlist->cursor = file->private_data; + dlist->dentries = dentries; + file->private_data = dlist; + return ret; +} + +/* + * This just sets the file->private_data back to the cursor and back. + */ +static int dcache_readdir_wrapper(struct file *file, struct dir_context *ctx) +{ + struct dentry_list *dlist = file->private_data; + int ret; + + file->private_data = dlist->cursor; + ret = dcache_readdir(file, ctx); + dlist->cursor = file->private_data; + file->private_data = dlist; + return ret; } /** @@ -491,6 +577,9 @@ struct dentry *eventfs_create_events_dir(const char *name, struct tracefs_inode *ti; struct inode *inode; + if (security_locked_down(LOCKDOWN_TRACEFS)) + return NULL; + if (IS_ERR(dentry)) return dentry; @@ -507,7 +596,7 @@ struct dentry *eventfs_create_events_dir(const char *name, INIT_LIST_HEAD(&ei->e_top_files); ti = get_tracefs(inode); - ti->flags |= TRACEFS_EVENT_INODE; + ti->flags |= TRACEFS_EVENT_INODE | TRACEFS_EVENT_TOP_INODE; ti->private = ei; inode->i_mode = S_IFDIR | S_IRWXU | S_IRUGO | S_IXUGO; @@ -538,6 +627,9 @@ struct eventfs_file *eventfs_add_subsystem_dir(const char *name, struct eventfs_inode *ei_parent; struct eventfs_file *ef; + if (security_locked_down(LOCKDOWN_TRACEFS)) + return NULL; + if (!parent) return ERR_PTR(-EINVAL); @@ -569,6 +661,9 @@ struct eventfs_file *eventfs_add_dir(const char *name, { struct eventfs_file *ef; + if (security_locked_down(LOCKDOWN_TRACEFS)) + return NULL; + if (!ef_parent) return ERR_PTR(-EINVAL); @@ -606,6 +701,9 @@ int eventfs_add_events_file(const char *name, umode_t mode, struct eventfs_inode *ei; struct eventfs_file *ef; + if (security_locked_down(LOCKDOWN_TRACEFS)) + return -ENODEV; + if (!parent) return -EINVAL; @@ -654,6 +752,9 @@ int eventfs_add_file(const char *name, umode_t mode, { struct eventfs_file *ef; + if (security_locked_down(LOCKDOWN_TRACEFS)) + return -ENODEV; + if (!ef_parent) return -EINVAL; @@ -791,7 +892,6 @@ void eventfs_remove(struct eventfs_file *ef) void eventfs_remove_events_dir(struct dentry *dentry) { struct tracefs_inode *ti; - struct eventfs_inode *ei; if (!dentry || !dentry->d_inode) return; @@ -800,8 +900,6 @@ void eventfs_remove_events_dir(struct dentry *dentry) if (!ti || !(ti->flags & TRACEFS_EVENT_INODE)) return; - ei = ti->private; d_invalidate(dentry); dput(dentry); - kfree(ei); } diff --git a/fs/tracefs/inode.c b/fs/tracefs/inode.c index de5b72216b1a..891653ba9cf3 100644 --- a/fs/tracefs/inode.c +++ b/fs/tracefs/inode.c @@ -385,7 +385,7 @@ static void tracefs_dentry_iput(struct dentry *dentry, struct inode *inode) ti = get_tracefs(inode); if (ti && ti->flags & TRACEFS_EVENT_INODE) - eventfs_set_ef_status_free(dentry); + eventfs_set_ef_status_free(ti, dentry); iput(inode); } @@ -673,6 +673,9 @@ static struct dentry *__create_dir(const char *name, struct dentry *parent, */ struct dentry *tracefs_create_dir(const char *name, struct dentry *parent) { + if (security_locked_down(LOCKDOWN_TRACEFS)) + return NULL; + return __create_dir(name, parent, &simple_dir_inode_operations); } diff --git a/fs/tracefs/internal.h b/fs/tracefs/internal.h index 69c2b1d87c46..4f2e49e2197b 100644 --- a/fs/tracefs/internal.h +++ b/fs/tracefs/internal.h @@ -3,7 +3,8 @@ #define _TRACEFS_INTERNAL_H enum { - TRACEFS_EVENT_INODE = BIT(1), + TRACEFS_EVENT_INODE = BIT(1), + TRACEFS_EVENT_TOP_INODE = BIT(2), }; struct tracefs_inode { @@ -24,6 +25,6 @@ struct inode *tracefs_get_inode(struct super_block *sb); struct dentry *eventfs_start_creating(const char *name, struct dentry *parent); struct dentry *eventfs_failed_creating(struct dentry *dentry); struct dentry *eventfs_end_creating(struct dentry *dentry); -void eventfs_set_ef_status_free(struct dentry *dentry); +void eventfs_set_ef_status_free(struct tracefs_inode *ti, struct dentry *dentry); #endif /* _TRACEFS_INTERNAL_H */ diff --git a/fs/xfs/Kconfig b/fs/xfs/Kconfig index c9d653168ad0..ed0bc8cbc703 100644 --- a/fs/xfs/Kconfig +++ b/fs/xfs/Kconfig @@ -147,7 +147,7 @@ config XFS_ONLINE_SCRUB_STATS bool "XFS online metadata check usage data collection" default y depends on XFS_ONLINE_SCRUB - select FS_DEBUG + select XFS_DEBUG help If you say Y here, the kernel will gather usage data about the online metadata check subsystem. This includes the number diff --git a/fs/xfs/libxfs/xfs_ag.c b/fs/xfs/libxfs/xfs_ag.c index e9cc481b4ddf..f9f4d694640d 100644 --- a/fs/xfs/libxfs/xfs_ag.c +++ b/fs/xfs/libxfs/xfs_ag.c @@ -1001,6 +1001,12 @@ xfs_ag_shrink_space( error = -ENOSPC; goto resv_init_out; } + + /* Update perag geometry */ + pag->block_count -= delta; + __xfs_agino_range(pag->pag_mount, pag->block_count, &pag->agino_min, + &pag->agino_max); + xfs_ialloc_log_agi(*tpp, agibp, XFS_AGI_LENGTH); xfs_alloc_log_agf(*tpp, agfbp, XFS_AGF_LENGTH); return 0; diff --git a/fs/xfs/libxfs/xfs_log_recover.h b/fs/xfs/libxfs/xfs_log_recover.h index 2420865f3007..a5100a11faf9 100644 --- a/fs/xfs/libxfs/xfs_log_recover.h +++ b/fs/xfs/libxfs/xfs_log_recover.h @@ -131,4 +131,26 @@ void xlog_check_buf_cancel_table(struct xlog *log); #define xlog_check_buf_cancel_table(log) do { } while (0) #endif +/* + * Transform a regular reservation into one suitable for recovery of a log + * intent item. + * + * Intent recovery only runs a single step of the transaction chain and defers + * the rest to a separate transaction. Therefore, we reduce logcount to 1 here + * to avoid livelocks if the log grant space is nearly exhausted due to the + * recovered intent pinning the tail. Keep the same logflags to avoid tripping + * asserts elsewhere. Struct copies abound below. + */ +static inline struct xfs_trans_res +xlog_recover_resv(const struct xfs_trans_res *r) +{ + struct xfs_trans_res ret = { + .tr_logres = r->tr_logres, + .tr_logcount = 1, + .tr_logflags = r->tr_logflags, + }; + + return ret; +} + #endif /* __XFS_LOG_RECOVER_H__ */ diff --git a/fs/xfs/libxfs/xfs_sb.c b/fs/xfs/libxfs/xfs_sb.c index 5e174685a77c..6264daaab37b 100644 --- a/fs/xfs/libxfs/xfs_sb.c +++ b/fs/xfs/libxfs/xfs_sb.c @@ -266,7 +266,8 @@ xfs_validate_sb_write( return -EFSCORRUPTED; } - if (xfs_sb_has_ro_compat_feature(sbp, XFS_SB_FEAT_RO_COMPAT_UNKNOWN)) { + if (!xfs_is_readonly(mp) && + xfs_sb_has_ro_compat_feature(sbp, XFS_SB_FEAT_RO_COMPAT_UNKNOWN)) { xfs_alert(mp, "Corruption detected in superblock read-only compatible features (0x%x)!", (sbp->sb_features_ro_compat & diff --git a/fs/xfs/libxfs/xfs_trans_inode.c b/fs/xfs/libxfs/xfs_trans_inode.c index ad22656376d3..6b2296ff248a 100644 --- a/fs/xfs/libxfs/xfs_trans_inode.c +++ b/fs/xfs/libxfs/xfs_trans_inode.c @@ -62,12 +62,12 @@ xfs_trans_ichgtime( ASSERT(tp); ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); - /* If the mtime changes, then ctime must also change */ - ASSERT(flags & XFS_ICHGTIME_CHG); + tv = current_time(inode); - tv = inode_set_ctime_current(inode); if (flags & XFS_ICHGTIME_MOD) inode->i_mtime = tv; + if (flags & XFS_ICHGTIME_CHG) + inode_set_ctime_to_ts(inode, tv); if (flags & XFS_ICHGTIME_CREATE) ip->i_crtime = tv; } diff --git a/fs/xfs/scrub/scrub.c b/fs/xfs/scrub/scrub.c index 7d3aa14d81b5..4849efcaa33a 100644 --- a/fs/xfs/scrub/scrub.c +++ b/fs/xfs/scrub/scrub.c @@ -588,6 +588,8 @@ out_nofix: out_teardown: error = xchk_teardown(sc, error); out_sc: + if (error != -ENOENT) + xchk_stats_merge(mp, sm, &run); kfree(sc); out: trace_xchk_done(XFS_I(file_inode(file)), sm, error); @@ -595,8 +597,6 @@ out: sm->sm_flags |= XFS_SCRUB_OFLAG_CORRUPT; error = 0; } - if (error != -ENOENT) - xchk_stats_merge(mp, sm, &run); return error; need_drain: error = xchk_teardown(sc, 0); diff --git a/fs/xfs/scrub/stats.c b/fs/xfs/scrub/stats.c index aeb92624176b..cd91db4a5548 100644 --- a/fs/xfs/scrub/stats.c +++ b/fs/xfs/scrub/stats.c @@ -185,7 +185,10 @@ xchk_stats_merge_one( { struct xchk_scrub_stats *css; - ASSERT(sm->sm_type < XFS_SCRUB_TYPE_NR); + if (sm->sm_type >= XFS_SCRUB_TYPE_NR) { + ASSERT(sm->sm_type < XFS_SCRUB_TYPE_NR); + return; + } css = &cs->cs_stats[sm->sm_type]; spin_lock(&css->css_lock); diff --git a/fs/xfs/scrub/xfile.c b/fs/xfs/scrub/xfile.c index d98e8e77c684..090c3ead43fd 100644 --- a/fs/xfs/scrub/xfile.c +++ b/fs/xfs/scrub/xfile.c @@ -10,7 +10,6 @@ #include "xfs_log_format.h" #include "xfs_trans_resv.h" #include "xfs_mount.h" -#include "xfs_format.h" #include "scrub/xfile.h" #include "scrub/xfarray.h" #include "scrub/scrub.h" diff --git a/fs/xfs/xfs_attr_inactive.c b/fs/xfs/xfs_attr_inactive.c index 5db87b34fb6e..89c7a9f4f930 100644 --- a/fs/xfs/xfs_attr_inactive.c +++ b/fs/xfs/xfs_attr_inactive.c @@ -333,7 +333,6 @@ xfs_attr_inactive( int error = 0; mp = dp->i_mount; - ASSERT(! XFS_NOT_DQATTACHED(mp, dp)); xfs_ilock(dp, lock_mode); if (!xfs_inode_has_attr_fork(dp)) diff --git a/fs/xfs/xfs_attr_item.c b/fs/xfs/xfs_attr_item.c index 2788a6f2edcd..36fe2abb16e6 100644 --- a/fs/xfs/xfs_attr_item.c +++ b/fs/xfs/xfs_attr_item.c @@ -547,7 +547,7 @@ xfs_attri_item_recover( struct xfs_inode *ip; struct xfs_da_args *args; struct xfs_trans *tp; - struct xfs_trans_res tres; + struct xfs_trans_res resv; struct xfs_attri_log_format *attrp; struct xfs_attri_log_nameval *nv = attrip->attri_nameval; int error; @@ -618,8 +618,9 @@ xfs_attri_item_recover( goto out; } - xfs_init_attr_trans(args, &tres, &total); - error = xfs_trans_alloc(mp, &tres, total, 0, XFS_TRANS_RESERVE, &tp); + xfs_init_attr_trans(args, &resv, &total); + resv = xlog_recover_resv(&resv); + error = xfs_trans_alloc(mp, &resv, total, 0, XFS_TRANS_RESERVE, &tp); if (error) goto out; diff --git a/fs/xfs/xfs_bmap_item.c b/fs/xfs/xfs_bmap_item.c index 7551c3ec4ea5..e736a0844c89 100644 --- a/fs/xfs/xfs_bmap_item.c +++ b/fs/xfs/xfs_bmap_item.c @@ -490,6 +490,7 @@ xfs_bui_item_recover( struct list_head *capture_list) { struct xfs_bmap_intent fake = { }; + struct xfs_trans_res resv; struct xfs_bui_log_item *buip = BUI_ITEM(lip); struct xfs_trans *tp; struct xfs_inode *ip = NULL; @@ -515,7 +516,8 @@ xfs_bui_item_recover( return error; /* Allocate transaction and do the work. */ - error = xfs_trans_alloc(mp, &M_RES(mp)->tr_itruncate, + resv = xlog_recover_resv(&M_RES(mp)->tr_itruncate); + error = xfs_trans_alloc(mp, &resv, XFS_EXTENTADD_SPACE_RES(mp, XFS_DATA_FORK), 0, 0, &tp); if (error) goto err_rele; diff --git a/fs/xfs/xfs_discard.c b/fs/xfs/xfs_discard.c index afc4c78b9eed..d5787991bb5b 100644 --- a/fs/xfs/xfs_discard.c +++ b/fs/xfs/xfs_discard.c @@ -1,6 +1,6 @@ // SPDX-License-Identifier: GPL-2.0 /* - * Copyright (C) 2010 Red Hat, Inc. + * Copyright (C) 2010, 2023 Red Hat, Inc. * All Rights Reserved. */ #include "xfs.h" @@ -19,21 +19,147 @@ #include "xfs_log.h" #include "xfs_ag.h" -STATIC int -xfs_trim_extents( +/* + * Notes on an efficient, low latency fstrim algorithm + * + * We need to walk the filesystem free space and issue discards on the free + * space that meet the search criteria (size and location). We cannot issue + * discards on extents that might be in use, or are so recently in use they are + * still marked as busy. To serialise against extent state changes whilst we are + * gathering extents to trim, we must hold the AGF lock to lock out other + * allocations and extent free operations that might change extent state. + * + * However, we cannot just hold the AGF for the entire AG free space walk whilst + * we issue discards on each free space that is found. Storage devices can have + * extremely slow discard implementations (e.g. ceph RBD) and so walking a + * couple of million free extents and issuing synchronous discards on each + * extent can take a *long* time. Whilst we are doing this walk, nothing else + * can access the AGF, and we can stall transactions and hence the log whilst + * modifications wait for the AGF lock to be released. This can lead hung tasks + * kicking the hung task timer and rebooting the system. This is bad. + * + * Hence we need to take a leaf from the bulkstat playbook. It takes the AGI + * lock, gathers a range of inode cluster buffers that are allocated, drops the + * AGI lock and then reads all the inode cluster buffers and processes them. It + * loops doing this, using a cursor to keep track of where it is up to in the AG + * for each iteration to restart the INOBT lookup from. + * + * We can't do this exactly with free space - once we drop the AGF lock, the + * state of the free extent is out of our control and we cannot run a discard + * safely on it in this situation. Unless, of course, we've marked the free + * extent as busy and undergoing a discard operation whilst we held the AGF + * locked. + * + * This is exactly how online discard works - free extents are marked busy when + * they are freed, and once the extent free has been committed to the journal, + * the busy extent record is marked as "undergoing discard" and the discard is + * then issued on the free extent. Once the discard completes, the busy extent + * record is removed and the extent is able to be allocated again. + * + * In the context of fstrim, if we find a free extent we need to discard, we + * don't have to discard it immediately. All we need to do it record that free + * extent as being busy and under discard, and all the allocation routines will + * now avoid trying to allocate it. Hence if we mark the extent as busy under + * the AGF lock, we can safely discard it without holding the AGF lock because + * nothing will attempt to allocate that free space until the discard completes. + * + * This also allows us to issue discards asynchronously like we do with online + * discard, and so for fast devices fstrim will run much faster as we can have + * multiple discard operations in flight at once, as well as pipeline the free + * extent search so that it overlaps in flight discard IO. + */ + +struct workqueue_struct *xfs_discard_wq; + +static void +xfs_discard_endio_work( + struct work_struct *work) +{ + struct xfs_busy_extents *extents = + container_of(work, struct xfs_busy_extents, endio_work); + + xfs_extent_busy_clear(extents->mount, &extents->extent_list, false); + kmem_free(extents->owner); +} + +/* + * Queue up the actual completion to a thread to avoid IRQ-safe locking for + * pagb_lock. + */ +static void +xfs_discard_endio( + struct bio *bio) +{ + struct xfs_busy_extents *extents = bio->bi_private; + + INIT_WORK(&extents->endio_work, xfs_discard_endio_work); + queue_work(xfs_discard_wq, &extents->endio_work); + bio_put(bio); +} + +/* + * Walk the discard list and issue discards on all the busy extents in the + * list. We plug and chain the bios so that we only need a single completion + * call to clear all the busy extents once the discards are complete. + */ +int +xfs_discard_extents( + struct xfs_mount *mp, + struct xfs_busy_extents *extents) +{ + struct xfs_extent_busy *busyp; + struct bio *bio = NULL; + struct blk_plug plug; + int error = 0; + + blk_start_plug(&plug); + list_for_each_entry(busyp, &extents->extent_list, list) { + trace_xfs_discard_extent(mp, busyp->agno, busyp->bno, + busyp->length); + + error = __blkdev_issue_discard(mp->m_ddev_targp->bt_bdev, + XFS_AGB_TO_DADDR(mp, busyp->agno, busyp->bno), + XFS_FSB_TO_BB(mp, busyp->length), + GFP_NOFS, &bio); + if (error && error != -EOPNOTSUPP) { + xfs_info(mp, + "discard failed for extent [0x%llx,%u], error %d", + (unsigned long long)busyp->bno, + busyp->length, + error); + break; + } + } + + if (bio) { + bio->bi_private = extents; + bio->bi_end_io = xfs_discard_endio; + submit_bio(bio); + } else { + xfs_discard_endio_work(&extents->endio_work); + } + blk_finish_plug(&plug); + + return error; +} + + +static int +xfs_trim_gather_extents( struct xfs_perag *pag, xfs_daddr_t start, xfs_daddr_t end, xfs_daddr_t minlen, + struct xfs_alloc_rec_incore *tcur, + struct xfs_busy_extents *extents, uint64_t *blocks_trimmed) { struct xfs_mount *mp = pag->pag_mount; - struct block_device *bdev = mp->m_ddev_targp->bt_bdev; struct xfs_btree_cur *cur; struct xfs_buf *agbp; - struct xfs_agf *agf; int error; int i; + int batch = 100; /* * Force out the log. This means any transactions that might have freed @@ -45,20 +171,28 @@ xfs_trim_extents( error = xfs_alloc_read_agf(pag, NULL, 0, &agbp); if (error) return error; - agf = agbp->b_addr; cur = xfs_allocbt_init_cursor(mp, NULL, agbp, pag, XFS_BTNUM_CNT); /* - * Look up the longest btree in the AGF and start with it. + * Look up the extent length requested in the AGF and start with it. */ - error = xfs_alloc_lookup_ge(cur, 0, be32_to_cpu(agf->agf_longest), &i); + if (tcur->ar_startblock == NULLAGBLOCK) + error = xfs_alloc_lookup_ge(cur, 0, tcur->ar_blockcount, &i); + else + error = xfs_alloc_lookup_le(cur, tcur->ar_startblock, + tcur->ar_blockcount, &i); if (error) goto out_del_cursor; + if (i == 0) { + /* nothing of that length left in the AG, we are done */ + tcur->ar_blockcount = 0; + goto out_del_cursor; + } /* * Loop until we are done with all extents that are large - * enough to be worth discarding. + * enough to be worth discarding or we hit batch limits. */ while (i) { xfs_agblock_t fbno; @@ -73,7 +207,16 @@ xfs_trim_extents( error = -EFSCORRUPTED; break; } - ASSERT(flen <= be32_to_cpu(agf->agf_longest)); + + if (--batch <= 0) { + /* + * Update the cursor to point at this extent so we + * restart the next batch from this extent. + */ + tcur->ar_startblock = fbno; + tcur->ar_blockcount = flen; + break; + } /* * use daddr format for all range/len calculations as that is @@ -88,6 +231,7 @@ xfs_trim_extents( */ if (dlen < minlen) { trace_xfs_discard_toosmall(mp, pag->pag_agno, fbno, flen); + tcur->ar_blockcount = 0; break; } @@ -110,29 +254,103 @@ xfs_trim_extents( goto next_extent; } - trace_xfs_discard_extent(mp, pag->pag_agno, fbno, flen); - error = blkdev_issue_discard(bdev, dbno, dlen, GFP_NOFS); - if (error) - break; + xfs_extent_busy_insert_discard(pag, fbno, flen, + &extents->extent_list); *blocks_trimmed += flen; - next_extent: error = xfs_btree_decrement(cur, 0, &i); if (error) break; - if (fatal_signal_pending(current)) { - error = -ERESTARTSYS; - break; - } + /* + * If there's no more records in the tree, we are done. Set the + * cursor block count to 0 to indicate to the caller that there + * is no more extents to search. + */ + if (i == 0) + tcur->ar_blockcount = 0; } + /* + * If there was an error, release all the gathered busy extents because + * we aren't going to issue a discard on them any more. + */ + if (error) + xfs_extent_busy_clear(mp, &extents->extent_list, false); out_del_cursor: xfs_btree_del_cursor(cur, error); xfs_buf_relse(agbp); return error; } +static bool +xfs_trim_should_stop(void) +{ + return fatal_signal_pending(current) || freezing(current); +} + +/* + * Iterate the free list gathering extents and discarding them. We need a cursor + * for the repeated iteration of gather/discard loop, so use the longest extent + * we found in the last batch as the key to start the next. + */ +static int +xfs_trim_extents( + struct xfs_perag *pag, + xfs_daddr_t start, + xfs_daddr_t end, + xfs_daddr_t minlen, + uint64_t *blocks_trimmed) +{ + struct xfs_alloc_rec_incore tcur = { + .ar_blockcount = pag->pagf_longest, + .ar_startblock = NULLAGBLOCK, + }; + int error = 0; + + do { + struct xfs_busy_extents *extents; + + extents = kzalloc(sizeof(*extents), GFP_KERNEL); + if (!extents) { + error = -ENOMEM; + break; + } + + extents->mount = pag->pag_mount; + extents->owner = extents; + INIT_LIST_HEAD(&extents->extent_list); + + error = xfs_trim_gather_extents(pag, start, end, minlen, + &tcur, extents, blocks_trimmed); + if (error) { + kfree(extents); + break; + } + + /* + * We hand the extent list to the discard function here so the + * discarded extents can be removed from the busy extent list. + * This allows the discards to run asynchronously with gathering + * the next round of extents to discard. + * + * However, we must ensure that we do not reference the extent + * list after this function call, as it may have been freed by + * the time control returns to us. + */ + error = xfs_discard_extents(pag->pag_mount, extents); + if (error) + break; + + if (xfs_trim_should_stop()) + break; + + } while (tcur.ar_blockcount != 0); + + return error; + +} + /* * trim a range of the filesystem. * @@ -195,12 +413,12 @@ xfs_ioc_trim( for_each_perag_range(mp, agno, xfs_daddr_to_agno(mp, end), pag) { error = xfs_trim_extents(pag, start, end, minlen, &blocks_trimmed); - if (error) { + if (error) last_error = error; - if (error == -ERESTARTSYS) { - xfs_perag_rele(pag); - break; - } + + if (xfs_trim_should_stop()) { + xfs_perag_rele(pag); + break; } } diff --git a/fs/xfs/xfs_discard.h b/fs/xfs/xfs_discard.h index de92d9cc958f..2b1a85223a56 100644 --- a/fs/xfs/xfs_discard.h +++ b/fs/xfs/xfs_discard.h @@ -3,8 +3,10 @@ #define XFS_DISCARD_H 1 struct fstrim_range; -struct list_head; +struct xfs_mount; +struct xfs_busy_extents; -extern int xfs_ioc_trim(struct xfs_mount *, struct fstrim_range __user *); +int xfs_discard_extents(struct xfs_mount *mp, struct xfs_busy_extents *busy); +int xfs_ioc_trim(struct xfs_mount *mp, struct fstrim_range __user *fstrim); #endif /* XFS_DISCARD_H */ diff --git a/fs/xfs/xfs_export.c b/fs/xfs/xfs_export.c index 1064c2342876..7cd09c3a82cb 100644 --- a/fs/xfs/xfs_export.c +++ b/fs/xfs/xfs_export.c @@ -146,6 +146,20 @@ xfs_nfs_get_inode( return ERR_PTR(error); } + /* + * Reload the incore unlinked list to avoid failure in inodegc. + * Use an unlocked check here because unrecovered unlinked inodes + * should be somewhat rare. + */ + if (xfs_inode_unlinked_incomplete(ip)) { + error = xfs_inode_reload_unlinked(ip); + if (error) { + xfs_force_shutdown(mp, SHUTDOWN_CORRUPT_INCORE); + xfs_irele(ip); + return ERR_PTR(error); + } + } + if (VFS_I(ip)->i_generation != generation) { xfs_irele(ip); return ERR_PTR(-ESTALE); diff --git a/fs/xfs/xfs_extent_busy.c b/fs/xfs/xfs_extent_busy.c index 7c2fdc71e42d..9ecfdcdc752f 100644 --- a/fs/xfs/xfs_extent_busy.c +++ b/fs/xfs/xfs_extent_busy.c @@ -19,13 +19,13 @@ #include "xfs_log.h" #include "xfs_ag.h" -void -xfs_extent_busy_insert( - struct xfs_trans *tp, +static void +xfs_extent_busy_insert_list( struct xfs_perag *pag, xfs_agblock_t bno, xfs_extlen_t len, - unsigned int flags) + unsigned int flags, + struct list_head *busy_list) { struct xfs_extent_busy *new; struct xfs_extent_busy *busyp; @@ -40,7 +40,7 @@ xfs_extent_busy_insert( new->flags = flags; /* trace before insert to be able to see failed inserts */ - trace_xfs_extent_busy(tp->t_mountp, pag->pag_agno, bno, len); + trace_xfs_extent_busy(pag->pag_mount, pag->pag_agno, bno, len); spin_lock(&pag->pagb_lock); rbp = &pag->pagb_tree.rb_node; @@ -62,10 +62,33 @@ xfs_extent_busy_insert( rb_link_node(&new->rb_node, parent, rbp); rb_insert_color(&new->rb_node, &pag->pagb_tree); - list_add(&new->list, &tp->t_busy); + /* always process discard lists in fifo order */ + list_add_tail(&new->list, busy_list); spin_unlock(&pag->pagb_lock); } +void +xfs_extent_busy_insert( + struct xfs_trans *tp, + struct xfs_perag *pag, + xfs_agblock_t bno, + xfs_extlen_t len, + unsigned int flags) +{ + xfs_extent_busy_insert_list(pag, bno, len, flags, &tp->t_busy); +} + +void +xfs_extent_busy_insert_discard( + struct xfs_perag *pag, + xfs_agblock_t bno, + xfs_extlen_t len, + struct list_head *busy_list) +{ + xfs_extent_busy_insert_list(pag, bno, len, XFS_EXTENT_BUSY_DISCARDED, + busy_list); +} + /* * Search for a busy extent within the range of the extent we are about to * allocate. You need to be holding the busy extent tree lock when calling diff --git a/fs/xfs/xfs_extent_busy.h b/fs/xfs/xfs_extent_busy.h index c37bf87e6781..0639aab336f3 100644 --- a/fs/xfs/xfs_extent_busy.h +++ b/fs/xfs/xfs_extent_busy.h @@ -16,9 +16,6 @@ struct xfs_alloc_arg; /* * Busy block/extent entry. Indexed by a rbtree in perag to mark blocks that * have been freed but whose transactions aren't committed to disk yet. - * - * Note that we use the transaction ID to record the transaction, not the - * transaction structure itself. See xfs_extent_busy_insert() for details. */ struct xfs_extent_busy { struct rb_node rb_node; /* ag by-bno indexed search tree */ @@ -31,10 +28,31 @@ struct xfs_extent_busy { #define XFS_EXTENT_BUSY_SKIP_DISCARD 0x02 /* do not discard */ }; +/* + * List used to track groups of related busy extents all the way through + * to discard completion. + */ +struct xfs_busy_extents { + struct xfs_mount *mount; + struct list_head extent_list; + struct work_struct endio_work; + + /* + * Owner is the object containing the struct xfs_busy_extents to free + * once the busy extents have been processed. If only the + * xfs_busy_extents object needs freeing, then point this at itself. + */ + void *owner; +}; + void xfs_extent_busy_insert(struct xfs_trans *tp, struct xfs_perag *pag, xfs_agblock_t bno, xfs_extlen_t len, unsigned int flags); +void +xfs_extent_busy_insert_discard(struct xfs_perag *pag, xfs_agblock_t bno, + xfs_extlen_t len, struct list_head *busy_list); + void xfs_extent_busy_clear(struct xfs_mount *mp, struct list_head *list, bool do_discard); diff --git a/fs/xfs/xfs_extfree_item.c b/fs/xfs/xfs_extfree_item.c index f1a5ecf099aa..3fa8789820ad 100644 --- a/fs/xfs/xfs_extfree_item.c +++ b/fs/xfs/xfs_extfree_item.c @@ -660,6 +660,7 @@ xfs_efi_item_recover( struct xfs_log_item *lip, struct list_head *capture_list) { + struct xfs_trans_res resv; struct xfs_efi_log_item *efip = EFI_ITEM(lip); struct xfs_mount *mp = lip->li_log->l_mp; struct xfs_efd_log_item *efdp; @@ -683,7 +684,8 @@ xfs_efi_item_recover( } } - error = xfs_trans_alloc(mp, &M_RES(mp)->tr_itruncate, 0, 0, 0, &tp); + resv = xlog_recover_resv(&M_RES(mp)->tr_itruncate); + error = xfs_trans_alloc(mp, &resv, 0, 0, 0, &tp); if (error) return error; efdp = xfs_trans_get_efd(tp, efip, efip->efi_format.efi_nextents); diff --git a/fs/xfs/xfs_fsmap.c b/fs/xfs/xfs_fsmap.c index 10403ba9b58f..736e5545f584 100644 --- a/fs/xfs/xfs_fsmap.c +++ b/fs/xfs/xfs_fsmap.c @@ -565,6 +565,19 @@ err: } #endif /* CONFIG_XFS_RT */ +static inline bool +rmap_not_shareable(struct xfs_mount *mp, const struct xfs_rmap_irec *r) +{ + if (!xfs_has_reflink(mp)) + return true; + if (XFS_RMAP_NON_INODE_OWNER(r->rm_owner)) + return true; + if (r->rm_flags & (XFS_RMAP_ATTR_FORK | XFS_RMAP_BMBT_BLOCK | + XFS_RMAP_UNWRITTEN)) + return true; + return false; +} + /* Execute a getfsmap query against the regular data device. */ STATIC int __xfs_getfsmap_datadev( @@ -598,7 +611,6 @@ __xfs_getfsmap_datadev( * low to the fsmap low key and max out the high key to the end * of the AG. */ - info->low.rm_startblock = XFS_FSB_TO_AGBNO(mp, start_fsb); info->low.rm_offset = XFS_BB_TO_FSBT(mp, keys[0].fmr_offset); error = xfs_fsmap_owner_to_rmap(&info->low, &keys[0]); if (error) @@ -608,12 +620,9 @@ __xfs_getfsmap_datadev( /* Adjust the low key if we are continuing from where we left off. */ if (info->low.rm_blockcount == 0) { - /* empty */ - } else if (XFS_RMAP_NON_INODE_OWNER(info->low.rm_owner) || - (info->low.rm_flags & (XFS_RMAP_ATTR_FORK | - XFS_RMAP_BMBT_BLOCK | - XFS_RMAP_UNWRITTEN))) { - info->low.rm_startblock += info->low.rm_blockcount; + /* No previous record from which to continue */ + } else if (rmap_not_shareable(mp, &info->low)) { + /* Last record seen was an unshareable extent */ info->low.rm_owner = 0; info->low.rm_offset = 0; @@ -621,8 +630,10 @@ __xfs_getfsmap_datadev( if (XFS_FSB_TO_DADDR(mp, start_fsb) >= eofs) return 0; } else { + /* Last record seen was a shareable file data extent */ info->low.rm_offset += info->low.rm_blockcount; } + info->low.rm_startblock = XFS_FSB_TO_AGBNO(mp, start_fsb); info->high.rm_startblock = -1U; info->high.rm_owner = ULLONG_MAX; diff --git a/fs/xfs/xfs_icache.c b/fs/xfs/xfs_icache.c index e541f5c0bc25..3c210ac83713 100644 --- a/fs/xfs/xfs_icache.c +++ b/fs/xfs/xfs_icache.c @@ -113,7 +113,7 @@ xfs_inode_alloc( INIT_LIST_HEAD(&ip->i_ioend_list); spin_lock_init(&ip->i_ioend_lock); ip->i_next_unlinked = NULLAGINO; - ip->i_prev_unlinked = NULLAGINO; + ip->i_prev_unlinked = 0; return ip; } @@ -443,7 +443,7 @@ xfs_inodegc_queue_all( int cpu; bool ret = false; - for_each_online_cpu(cpu) { + for_each_cpu(cpu, &mp->m_inodegc_cpumask) { gc = per_cpu_ptr(mp->m_inodegc, cpu); if (!llist_empty(&gc->list)) { mod_delayed_work_on(cpu, mp->m_inodegc_wq, &gc->work, 0); @@ -463,7 +463,7 @@ xfs_inodegc_wait_all( int error = 0; flush_workqueue(mp->m_inodegc_wq); - for_each_online_cpu(cpu) { + for_each_cpu(cpu, &mp->m_inodegc_cpumask) { struct xfs_inodegc *gc; gc = per_cpu_ptr(mp->m_inodegc, cpu); @@ -1845,9 +1845,17 @@ xfs_inodegc_worker( struct xfs_inodegc, work); struct llist_node *node = llist_del_all(&gc->list); struct xfs_inode *ip, *n; + struct xfs_mount *mp = gc->mp; unsigned int nofs_flag; - ASSERT(gc->cpu == smp_processor_id()); + /* + * Clear the cpu mask bit and ensure that we have seen the latest + * update of the gc structure associated with this CPU. This matches + * with the release semantics used when setting the cpumask bit in + * xfs_inodegc_queue. + */ + cpumask_clear_cpu(gc->cpu, &mp->m_inodegc_cpumask); + smp_mb__after_atomic(); WRITE_ONCE(gc->items, 0); @@ -1862,7 +1870,7 @@ xfs_inodegc_worker( nofs_flag = memalloc_nofs_save(); ip = llist_entry(node, struct xfs_inode, i_gclist); - trace_xfs_inodegc_worker(ip->i_mount, READ_ONCE(gc->shrinker_hits)); + trace_xfs_inodegc_worker(mp, READ_ONCE(gc->shrinker_hits)); WRITE_ONCE(gc->shrinker_hits, 0); llist_for_each_entry_safe(ip, n, node, i_gclist) { @@ -2057,6 +2065,7 @@ xfs_inodegc_queue( struct xfs_inodegc *gc; int items; unsigned int shrinker_hits; + unsigned int cpu_nr; unsigned long queue_delay = 1; trace_xfs_inode_set_need_inactive(ip); @@ -2064,18 +2073,28 @@ xfs_inodegc_queue( ip->i_flags |= XFS_NEED_INACTIVE; spin_unlock(&ip->i_flags_lock); - gc = get_cpu_ptr(mp->m_inodegc); + cpu_nr = get_cpu(); + gc = this_cpu_ptr(mp->m_inodegc); llist_add(&ip->i_gclist, &gc->list); items = READ_ONCE(gc->items); WRITE_ONCE(gc->items, items + 1); shrinker_hits = READ_ONCE(gc->shrinker_hits); + /* + * Ensure the list add is always seen by anyone who finds the cpumask + * bit set. This effectively gives the cpumask bit set operation + * release ordering semantics. + */ + smp_mb__before_atomic(); + if (!cpumask_test_cpu(cpu_nr, &mp->m_inodegc_cpumask)) + cpumask_test_and_set_cpu(cpu_nr, &mp->m_inodegc_cpumask); + /* * We queue the work while holding the current CPU so that the work * is scheduled to run on this CPU. */ if (!xfs_is_inodegc_enabled(mp)) { - put_cpu_ptr(gc); + put_cpu(); return; } @@ -2085,7 +2104,7 @@ xfs_inodegc_queue( trace_xfs_inodegc_queue(mp, __return_address); mod_delayed_work_on(current_cpu(), mp->m_inodegc_wq, &gc->work, queue_delay); - put_cpu_ptr(gc); + put_cpu(); if (xfs_inodegc_want_flush_work(ip, items, shrinker_hits)) { trace_xfs_inodegc_throttle(mp, __return_address); @@ -2093,47 +2112,6 @@ xfs_inodegc_queue( } } -/* - * Fold the dead CPU inodegc queue into the current CPUs queue. - */ -void -xfs_inodegc_cpu_dead( - struct xfs_mount *mp, - unsigned int dead_cpu) -{ - struct xfs_inodegc *dead_gc, *gc; - struct llist_node *first, *last; - unsigned int count = 0; - - dead_gc = per_cpu_ptr(mp->m_inodegc, dead_cpu); - cancel_delayed_work_sync(&dead_gc->work); - - if (llist_empty(&dead_gc->list)) - return; - - first = dead_gc->list.first; - last = first; - while (last->next) { - last = last->next; - count++; - } - dead_gc->list.first = NULL; - dead_gc->items = 0; - - /* Add pending work to current CPU */ - gc = get_cpu_ptr(mp->m_inodegc); - llist_add_batch(first, last, &gc->list); - count += READ_ONCE(gc->items); - WRITE_ONCE(gc->items, count); - - if (xfs_is_inodegc_enabled(mp)) { - trace_xfs_inodegc_queue(mp, __return_address); - mod_delayed_work_on(current_cpu(), mp->m_inodegc_wq, &gc->work, - 0); - } - put_cpu_ptr(gc); -} - /* * We set the inode flag atomically with the radix tree tag. Once we get tag * lookups on the radix tree, this inode flag can go away. @@ -2195,7 +2173,7 @@ xfs_inodegc_shrinker_count( if (!xfs_is_inodegc_enabled(mp)) return 0; - for_each_online_cpu(cpu) { + for_each_cpu(cpu, &mp->m_inodegc_cpumask) { gc = per_cpu_ptr(mp->m_inodegc, cpu); if (!llist_empty(&gc->list)) return XFS_INODEGC_SHRINKER_COUNT; @@ -2220,7 +2198,7 @@ xfs_inodegc_shrinker_scan( trace_xfs_inodegc_shrinker_scan(mp, sc, __return_address); - for_each_online_cpu(cpu) { + for_each_cpu(cpu, &mp->m_inodegc_cpumask) { gc = per_cpu_ptr(mp->m_inodegc, cpu); if (!llist_empty(&gc->list)) { unsigned int h = READ_ONCE(gc->shrinker_hits); diff --git a/fs/xfs/xfs_icache.h b/fs/xfs/xfs_icache.h index 2fa6f2e09d07..905944dafbe5 100644 --- a/fs/xfs/xfs_icache.h +++ b/fs/xfs/xfs_icache.h @@ -79,7 +79,6 @@ void xfs_inodegc_push(struct xfs_mount *mp); int xfs_inodegc_flush(struct xfs_mount *mp); void xfs_inodegc_stop(struct xfs_mount *mp); void xfs_inodegc_start(struct xfs_mount *mp); -void xfs_inodegc_cpu_dead(struct xfs_mount *mp, unsigned int cpu); int xfs_inodegc_register_shrinker(struct xfs_mount *mp); #endif diff --git a/fs/xfs/xfs_inode.c b/fs/xfs/xfs_inode.c index 360fe83a334f..4d55f58d99b7 100644 --- a/fs/xfs/xfs_inode.c +++ b/fs/xfs/xfs_inode.c @@ -1642,8 +1642,11 @@ xfs_inode_needs_inactive( if (VFS_I(ip)->i_mode == 0) return false; - /* If this is a read-only mount, don't do this (would generate I/O) */ - if (xfs_is_readonly(mp)) + /* + * If this is a read-only mount, don't do this (would generate I/O) + * unless we're in log recovery and cleaning the iunlinked list. + */ + if (xfs_is_readonly(mp) && !xlog_recovery_needed(mp->m_log)) return false; /* If the log isn't running, push inodes straight to reclaim. */ @@ -1703,8 +1706,11 @@ xfs_inactive( mp = ip->i_mount; ASSERT(!xfs_iflags_test(ip, XFS_IRECOVERY)); - /* If this is a read-only mount, don't do this (would generate I/O) */ - if (xfs_is_readonly(mp)) + /* + * If this is a read-only mount, don't do this (would generate I/O) + * unless we're in log recovery and cleaning the iunlinked list. + */ + if (xfs_is_readonly(mp) && !xlog_recovery_needed(mp->m_log)) goto out; /* Metadata inodes require explicit resource cleanup. */ @@ -1736,9 +1742,21 @@ xfs_inactive( ip->i_df.if_nextents > 0 || ip->i_delayed_blks > 0)) truncate = 1; - error = xfs_qm_dqattach(ip); - if (error) - goto out; + if (xfs_iflags_test(ip, XFS_IQUOTAUNCHECKED)) { + /* + * If this inode is being inactivated during a quotacheck and + * has not yet been scanned by quotacheck, we /must/ remove + * the dquots from the inode before inactivation changes the + * block and inode counts. Most probably this is a result of + * reloading the incore iunlinked list to purge unrecovered + * unlinked inodes. + */ + xfs_qm_dqdetach(ip); + } else { + error = xfs_qm_dqattach(ip); + if (error) + goto out; + } if (S_ISLNK(VFS_I(ip)->i_mode)) error = xfs_inactive_symlink(ip); @@ -1822,12 +1840,17 @@ xfs_iunlink_lookup( rcu_read_lock(); ip = radix_tree_lookup(&pag->pag_ici_root, agino); + if (!ip) { + /* Caller can handle inode not being in memory. */ + rcu_read_unlock(); + return NULL; + } /* - * Inode not in memory or in RCU freeing limbo should not happen. - * Warn about this and let the caller handle the failure. + * Inode in RCU freeing limbo should not happen. Warn about this and + * let the caller handle the failure. */ - if (WARN_ON_ONCE(!ip || !ip->i_ino)) { + if (WARN_ON_ONCE(!ip->i_ino)) { rcu_read_unlock(); return NULL; } @@ -1836,7 +1859,10 @@ xfs_iunlink_lookup( return ip; } -/* Update the prev pointer of the next agino. */ +/* + * Update the prev pointer of the next agino. Returns -ENOLINK if the inode + * is not in cache. + */ static int xfs_iunlink_update_backref( struct xfs_perag *pag, @@ -1851,7 +1877,8 @@ xfs_iunlink_update_backref( ip = xfs_iunlink_lookup(pag, next_agino); if (!ip) - return -EFSCORRUPTED; + return -ENOLINK; + ip->i_prev_unlinked = prev_agino; return 0; } @@ -1895,6 +1922,64 @@ xfs_iunlink_update_bucket( return 0; } +/* + * Load the inode @next_agino into the cache and set its prev_unlinked pointer + * to @prev_agino. Caller must hold the AGI to synchronize with other changes + * to the unlinked list. + */ +STATIC int +xfs_iunlink_reload_next( + struct xfs_trans *tp, + struct xfs_buf *agibp, + xfs_agino_t prev_agino, + xfs_agino_t next_agino) +{ + struct xfs_perag *pag = agibp->b_pag; + struct xfs_mount *mp = pag->pag_mount; + struct xfs_inode *next_ip = NULL; + xfs_ino_t ino; + int error; + + ASSERT(next_agino != NULLAGINO); + +#ifdef DEBUG + rcu_read_lock(); + next_ip = radix_tree_lookup(&pag->pag_ici_root, next_agino); + ASSERT(next_ip == NULL); + rcu_read_unlock(); +#endif + + xfs_info_ratelimited(mp, + "Found unrecovered unlinked inode 0x%x in AG 0x%x. Initiating recovery.", + next_agino, pag->pag_agno); + + /* + * Use an untrusted lookup just to be cautious in case the AGI has been + * corrupted and now points at a free inode. That shouldn't happen, + * but we'd rather shut down now since we're already running in a weird + * situation. + */ + ino = XFS_AGINO_TO_INO(mp, pag->pag_agno, next_agino); + error = xfs_iget(mp, tp, ino, XFS_IGET_UNTRUSTED, 0, &next_ip); + if (error) + return error; + + /* If this is not an unlinked inode, something is very wrong. */ + if (VFS_I(next_ip)->i_nlink != 0) { + error = -EFSCORRUPTED; + goto rele; + } + + next_ip->i_prev_unlinked = prev_agino; + trace_xfs_iunlink_reload_next(next_ip); +rele: + ASSERT(!(VFS_I(next_ip)->i_state & I_DONTCACHE)); + if (xfs_is_quotacheck_running(mp) && next_ip) + xfs_iflags_set(next_ip, XFS_IQUOTAUNCHECKED); + xfs_irele(next_ip); + return error; +} + static int xfs_iunlink_insert_inode( struct xfs_trans *tp, @@ -1926,6 +2011,8 @@ xfs_iunlink_insert_inode( * inode. */ error = xfs_iunlink_update_backref(pag, agino, next_agino); + if (error == -ENOLINK) + error = xfs_iunlink_reload_next(tp, agibp, agino, next_agino); if (error) return error; @@ -1941,6 +2028,7 @@ xfs_iunlink_insert_inode( } /* Point the head of the list to point to this inode. */ + ip->i_prev_unlinked = NULLAGINO; return xfs_iunlink_update_bucket(tp, pag, agibp, bucket_index, agino); } @@ -2020,6 +2108,9 @@ xfs_iunlink_remove_inode( */ error = xfs_iunlink_update_backref(pag, ip->i_prev_unlinked, ip->i_next_unlinked); + if (error == -ENOLINK) + error = xfs_iunlink_reload_next(tp, agibp, ip->i_prev_unlinked, + ip->i_next_unlinked); if (error) return error; @@ -2040,7 +2131,7 @@ xfs_iunlink_remove_inode( } ip->i_next_unlinked = NULLAGINO; - ip->i_prev_unlinked = NULLAGINO; + ip->i_prev_unlinked = 0; return error; } @@ -3529,3 +3620,117 @@ xfs_iunlock2_io_mmap( if (ip1 != ip2) inode_unlock(VFS_I(ip1)); } + +/* + * Reload the incore inode list for this inode. Caller should ensure that + * the link count cannot change, either by taking ILOCK_SHARED or otherwise + * preventing other threads from executing. + */ +int +xfs_inode_reload_unlinked_bucket( + struct xfs_trans *tp, + struct xfs_inode *ip) +{ + struct xfs_mount *mp = tp->t_mountp; + struct xfs_buf *agibp; + struct xfs_agi *agi; + struct xfs_perag *pag; + xfs_agnumber_t agno = XFS_INO_TO_AGNO(mp, ip->i_ino); + xfs_agino_t agino = XFS_INO_TO_AGINO(mp, ip->i_ino); + xfs_agino_t prev_agino, next_agino; + unsigned int bucket; + bool foundit = false; + int error; + + /* Grab the first inode in the list */ + pag = xfs_perag_get(mp, agno); + error = xfs_ialloc_read_agi(pag, tp, &agibp); + xfs_perag_put(pag); + if (error) + return error; + + /* + * We've taken ILOCK_SHARED and the AGI buffer lock to stabilize the + * incore unlinked list pointers for this inode. Check once more to + * see if we raced with anyone else to reload the unlinked list. + */ + if (!xfs_inode_unlinked_incomplete(ip)) { + foundit = true; + goto out_agibp; + } + + bucket = agino % XFS_AGI_UNLINKED_BUCKETS; + agi = agibp->b_addr; + + trace_xfs_inode_reload_unlinked_bucket(ip); + + xfs_info_ratelimited(mp, + "Found unrecovered unlinked inode 0x%x in AG 0x%x. Initiating list recovery.", + agino, agno); + + prev_agino = NULLAGINO; + next_agino = be32_to_cpu(agi->agi_unlinked[bucket]); + while (next_agino != NULLAGINO) { + struct xfs_inode *next_ip = NULL; + + /* Found this caller's inode, set its backlink. */ + if (next_agino == agino) { + next_ip = ip; + next_ip->i_prev_unlinked = prev_agino; + foundit = true; + goto next_inode; + } + + /* Try in-memory lookup first. */ + next_ip = xfs_iunlink_lookup(pag, next_agino); + if (next_ip) + goto next_inode; + + /* Inode not in memory, try reloading it. */ + error = xfs_iunlink_reload_next(tp, agibp, prev_agino, + next_agino); + if (error) + break; + + /* Grab the reloaded inode. */ + next_ip = xfs_iunlink_lookup(pag, next_agino); + if (!next_ip) { + /* No incore inode at all? We reloaded it... */ + ASSERT(next_ip != NULL); + error = -EFSCORRUPTED; + break; + } + +next_inode: + prev_agino = next_agino; + next_agino = next_ip->i_next_unlinked; + } + +out_agibp: + xfs_trans_brelse(tp, agibp); + /* Should have found this inode somewhere in the iunlinked bucket. */ + if (!error && !foundit) + error = -EFSCORRUPTED; + return error; +} + +/* Decide if this inode is missing its unlinked list and reload it. */ +int +xfs_inode_reload_unlinked( + struct xfs_inode *ip) +{ + struct xfs_trans *tp; + int error; + + error = xfs_trans_alloc_empty(ip->i_mount, &tp); + if (error) + return error; + + xfs_ilock(ip, XFS_ILOCK_SHARED); + if (xfs_inode_unlinked_incomplete(ip)) + error = xfs_inode_reload_unlinked_bucket(tp, ip); + xfs_iunlock(ip, XFS_ILOCK_SHARED); + xfs_trans_cancel(tp); + + return error; +} diff --git a/fs/xfs/xfs_inode.h b/fs/xfs/xfs_inode.h index 7547caf2f2ab..0c5bdb91152e 100644 --- a/fs/xfs/xfs_inode.h +++ b/fs/xfs/xfs_inode.h @@ -68,8 +68,21 @@ typedef struct xfs_inode { uint64_t i_diflags2; /* XFS_DIFLAG2_... */ struct timespec64 i_crtime; /* time created */ - /* unlinked list pointers */ + /* + * Unlinked list pointers. These point to the next and previous inodes + * in the AGI unlinked bucket list, respectively. These fields can + * only be updated with the AGI locked. + * + * i_next_unlinked caches di_next_unlinked. + */ xfs_agino_t i_next_unlinked; + + /* + * If the inode is not on an unlinked list, this field is zero. If the + * inode is the first element in an unlinked list, this field is + * NULLAGINO. Otherwise, i_prev_unlinked points to the previous inode + * in the unlinked list. + */ xfs_agino_t i_prev_unlinked; /* VFS inode */ @@ -81,6 +94,11 @@ typedef struct xfs_inode { struct list_head i_ioend_list; } xfs_inode_t; +static inline bool xfs_inode_on_unlinked_list(const struct xfs_inode *ip) +{ + return ip->i_prev_unlinked != 0; +} + static inline bool xfs_inode_has_attr_fork(struct xfs_inode *ip) { return ip->i_forkoff > 0; @@ -326,6 +344,9 @@ static inline bool xfs_inode_has_large_extent_counts(struct xfs_inode *ip) */ #define XFS_INACTIVATING (1 << 13) +/* Quotacheck is running but inode has not been added to quota counts. */ +#define XFS_IQUOTAUNCHECKED (1 << 14) + /* All inode state flags related to inode reclaim. */ #define XFS_ALL_IRECLAIM_FLAGS (XFS_IRECLAIMABLE | \ XFS_IRECLAIM | \ @@ -340,7 +361,7 @@ static inline bool xfs_inode_has_large_extent_counts(struct xfs_inode *ip) #define XFS_IRECLAIM_RESET_FLAGS \ (XFS_IRECLAIMABLE | XFS_IRECLAIM | \ XFS_IDIRTY_RELEASE | XFS_ITRUNCATED | XFS_NEED_INACTIVE | \ - XFS_INACTIVATING) + XFS_INACTIVATING | XFS_IQUOTAUNCHECKED) /* * Flags for inode locking. @@ -575,4 +596,13 @@ void xfs_end_io(struct work_struct *work); int xfs_ilock2_io_mmap(struct xfs_inode *ip1, struct xfs_inode *ip2); void xfs_iunlock2_io_mmap(struct xfs_inode *ip1, struct xfs_inode *ip2); +static inline bool +xfs_inode_unlinked_incomplete( + struct xfs_inode *ip) +{ + return VFS_I(ip)->i_nlink == 0 && !xfs_inode_on_unlinked_list(ip); +} +int xfs_inode_reload_unlinked_bucket(struct xfs_trans *tp, struct xfs_inode *ip); +int xfs_inode_reload_unlinked(struct xfs_inode *ip); + #endif /* __XFS_INODE_H__ */ diff --git a/fs/xfs/xfs_iops.c b/fs/xfs/xfs_iops.c index 2ededd3f6b8c..2b3b05c28e9e 100644 --- a/fs/xfs/xfs_iops.c +++ b/fs/xfs/xfs_iops.c @@ -573,10 +573,10 @@ xfs_vn_getattr( stat->gid = vfsgid_into_kgid(vfsgid); stat->ino = ip->i_ino; stat->atime = inode->i_atime; + stat->mtime = inode->i_mtime; + stat->ctime = inode_get_ctime(inode); stat->blocks = XFS_FSB_TO_BB(mp, ip->i_nblocks + ip->i_delayed_blks); - fill_mg_cmtime(stat, request_mask, inode); - if (xfs_has_v3inodes(mp)) { if (request_mask & STATX_BTIME) { stat->result_mask |= STATX_BTIME; @@ -584,6 +584,11 @@ xfs_vn_getattr( } } + if ((request_mask & STATX_CHANGE_COOKIE) && IS_I_VERSION(inode)) { + stat->change_cookie = inode_query_iversion(inode); + stat->result_mask |= STATX_CHANGE_COOKIE; + } + /* * Note: If you add another clause to set an attribute flag, please * update attributes_mask below. @@ -917,7 +922,7 @@ xfs_setattr_size( if (newsize != oldsize && !(iattr->ia_valid & (ATTR_CTIME | ATTR_MTIME))) { iattr->ia_ctime = iattr->ia_mtime = - current_mgtime(inode); + current_time(inode); iattr->ia_valid |= ATTR_CTIME | ATTR_MTIME; } diff --git a/fs/xfs/xfs_itable.c b/fs/xfs/xfs_itable.c index c2093cb56092..f5377ba5967a 100644 --- a/fs/xfs/xfs_itable.c +++ b/fs/xfs/xfs_itable.c @@ -80,6 +80,17 @@ xfs_bulkstat_one_int( if (error) goto out; + /* Reload the incore unlinked list to avoid failure in inodegc. */ + if (xfs_inode_unlinked_incomplete(ip)) { + error = xfs_inode_reload_unlinked_bucket(tp, ip); + if (error) { + xfs_iunlock(ip, XFS_ILOCK_SHARED); + xfs_force_shutdown(mp, SHUTDOWN_CORRUPT_INCORE); + xfs_irele(ip); + return error; + } + } + ASSERT(ip != NULL); ASSERT(ip->i_imap.im_blkno != 0); inode = VFS_I(ip); diff --git a/fs/xfs/xfs_log.c b/fs/xfs/xfs_log.c index 79004d193e54..51c100c86177 100644 --- a/fs/xfs/xfs_log.c +++ b/fs/xfs/xfs_log.c @@ -715,15 +715,7 @@ xfs_log_mount( * just worked. */ if (!xfs_has_norecovery(mp)) { - /* - * log recovery ignores readonly state and so we need to clear - * mount-based read only state so it can write to disk. - */ - bool readonly = test_and_clear_bit(XFS_OPSTATE_READONLY, - &mp->m_opstate); error = xlog_recover(log); - if (readonly) - set_bit(XFS_OPSTATE_READONLY, &mp->m_opstate); if (error) { xfs_warn(mp, "log mount/recovery failed: error %d", error); @@ -772,7 +764,6 @@ xfs_log_mount_finish( struct xfs_mount *mp) { struct xlog *log = mp->m_log; - bool readonly; int error = 0; if (xfs_has_norecovery(mp)) { @@ -780,12 +771,6 @@ xfs_log_mount_finish( return 0; } - /* - * log recovery ignores readonly state and so we need to clear - * mount-based read only state so it can write to disk. - */ - readonly = test_and_clear_bit(XFS_OPSTATE_READONLY, &mp->m_opstate); - /* * During the second phase of log recovery, we need iget and * iput to behave like they do for an active filesystem. @@ -835,8 +820,6 @@ xfs_log_mount_finish( xfs_buftarg_drain(mp->m_ddev_targp); clear_bit(XLOG_RECOVERY_NEEDED, &log->l_opstate); - if (readonly) - set_bit(XFS_OPSTATE_READONLY, &mp->m_opstate); /* Make sure the log is dead if we're returning failure. */ ASSERT(!error || xlog_is_shutdown(log)); diff --git a/fs/xfs/xfs_log_cil.c b/fs/xfs/xfs_log_cil.c index eccbfb99e894..67a99d94701e 100644 --- a/fs/xfs/xfs_log_cil.c +++ b/fs/xfs/xfs_log_cil.c @@ -16,8 +16,7 @@ #include "xfs_log.h" #include "xfs_log_priv.h" #include "xfs_trace.h" - -struct workqueue_struct *xfs_discard_wq; +#include "xfs_discard.h" /* * Allocate a new ticket. Failing to get a new ticket makes it really hard to @@ -103,7 +102,7 @@ xlog_cil_ctx_alloc(void) ctx = kmem_zalloc(sizeof(*ctx), KM_NOFS); INIT_LIST_HEAD(&ctx->committing); - INIT_LIST_HEAD(&ctx->busy_extents); + INIT_LIST_HEAD(&ctx->busy_extents.extent_list); INIT_LIST_HEAD(&ctx->log_items); INIT_LIST_HEAD(&ctx->lv_chain); INIT_WORK(&ctx->push_work, xlog_cil_push_work); @@ -124,7 +123,7 @@ xlog_cil_push_pcp_aggregate( struct xlog_cil_pcp *cilpcp; int cpu; - for_each_online_cpu(cpu) { + for_each_cpu(cpu, &ctx->cil_pcpmask) { cilpcp = per_cpu_ptr(cil->xc_pcp, cpu); ctx->ticket->t_curr_res += cilpcp->space_reserved; @@ -132,7 +131,7 @@ xlog_cil_push_pcp_aggregate( if (!list_empty(&cilpcp->busy_extents)) { list_splice_init(&cilpcp->busy_extents, - &ctx->busy_extents); + &ctx->busy_extents.extent_list); } if (!list_empty(&cilpcp->log_items)) list_splice_init(&cilpcp->log_items, &ctx->log_items); @@ -165,7 +164,13 @@ xlog_cil_insert_pcp_aggregate( if (!test_and_clear_bit(XLOG_CIL_PCP_SPACE, &cil->xc_flags)) return; - for_each_online_cpu(cpu) { + /* + * We can race with other cpus setting cil_pcpmask. However, we've + * atomically cleared PCP_SPACE which forces other threads to add to + * the global space used count. cil_pcpmask is a superset of cilpcp + * structures that could have a nonzero space_used. + */ + for_each_cpu(cpu, &ctx->cil_pcpmask) { int old, prev; cilpcp = per_cpu_ptr(cil->xc_pcp, cpu); @@ -554,6 +559,7 @@ xlog_cil_insert_items( int iovhdr_res = 0, split_res = 0, ctx_res = 0; int space_used; int order; + unsigned int cpu_nr; struct xlog_cil_pcp *cilpcp; ASSERT(tp); @@ -577,7 +583,12 @@ xlog_cil_insert_items( * can't be scheduled away between split sample/update operations that * are done without outside locking to serialise them. */ - cilpcp = get_cpu_ptr(cil->xc_pcp); + cpu_nr = get_cpu(); + cilpcp = this_cpu_ptr(cil->xc_pcp); + + /* Tell the future push that there was work added by this CPU. */ + if (!cpumask_test_cpu(cpu_nr, &ctx->cil_pcpmask)) + cpumask_test_and_set_cpu(cpu_nr, &ctx->cil_pcpmask); /* * We need to take the CIL checkpoint unit reservation on the first @@ -663,7 +674,7 @@ xlog_cil_insert_items( continue; list_add_tail(&lip->li_cil, &cilpcp->log_items); } - put_cpu_ptr(cilpcp); + put_cpu(); /* * If we've overrun the reservation, dump the tx details before we move @@ -696,76 +707,6 @@ xlog_cil_free_logvec( } } -static void -xlog_discard_endio_work( - struct work_struct *work) -{ - struct xfs_cil_ctx *ctx = - container_of(work, struct xfs_cil_ctx, discard_endio_work); - struct xfs_mount *mp = ctx->cil->xc_log->l_mp; - - xfs_extent_busy_clear(mp, &ctx->busy_extents, false); - kmem_free(ctx); -} - -/* - * Queue up the actual completion to a thread to avoid IRQ-safe locking for - * pagb_lock. Note that we need a unbounded workqueue, otherwise we might - * get the execution delayed up to 30 seconds for weird reasons. - */ -static void -xlog_discard_endio( - struct bio *bio) -{ - struct xfs_cil_ctx *ctx = bio->bi_private; - - INIT_WORK(&ctx->discard_endio_work, xlog_discard_endio_work); - queue_work(xfs_discard_wq, &ctx->discard_endio_work); - bio_put(bio); -} - -static void -xlog_discard_busy_extents( - struct xfs_mount *mp, - struct xfs_cil_ctx *ctx) -{ - struct list_head *list = &ctx->busy_extents; - struct xfs_extent_busy *busyp; - struct bio *bio = NULL; - struct blk_plug plug; - int error = 0; - - ASSERT(xfs_has_discard(mp)); - - blk_start_plug(&plug); - list_for_each_entry(busyp, list, list) { - trace_xfs_discard_extent(mp, busyp->agno, busyp->bno, - busyp->length); - - error = __blkdev_issue_discard(mp->m_ddev_targp->bt_bdev, - XFS_AGB_TO_DADDR(mp, busyp->agno, busyp->bno), - XFS_FSB_TO_BB(mp, busyp->length), - GFP_NOFS, &bio); - if (error && error != -EOPNOTSUPP) { - xfs_info(mp, - "discard failed for extent [0x%llx,%u], error %d", - (unsigned long long)busyp->bno, - busyp->length, - error); - break; - } - } - - if (bio) { - bio->bi_private = ctx; - bio->bi_end_io = xlog_discard_endio; - submit_bio(bio); - } else { - xlog_discard_endio_work(&ctx->discard_endio_work); - } - blk_finish_plug(&plug); -} - /* * Mark all items committed and clear busy extents. We free the log vector * chains in a separate pass so that we unpin the log items as quickly as @@ -795,8 +736,8 @@ xlog_cil_committed( xfs_trans_committed_bulk(ctx->cil->xc_log->l_ailp, &ctx->lv_chain, ctx->start_lsn, abort); - xfs_extent_busy_sort(&ctx->busy_extents); - xfs_extent_busy_clear(mp, &ctx->busy_extents, + xfs_extent_busy_sort(&ctx->busy_extents.extent_list); + xfs_extent_busy_clear(mp, &ctx->busy_extents.extent_list, xfs_has_discard(mp) && !abort); spin_lock(&ctx->cil->xc_push_lock); @@ -805,10 +746,14 @@ xlog_cil_committed( xlog_cil_free_logvec(&ctx->lv_chain); - if (!list_empty(&ctx->busy_extents)) - xlog_discard_busy_extents(mp, ctx); - else - kmem_free(ctx); + if (!list_empty(&ctx->busy_extents.extent_list)) { + ctx->busy_extents.mount = mp; + ctx->busy_extents.owner = ctx; + xfs_discard_extents(mp, &ctx->busy_extents); + return; + } + + kmem_free(ctx); } void @@ -1790,38 +1735,6 @@ out_shutdown: return 0; } -/* - * Move dead percpu state to the relevant CIL context structures. - * - * We have to lock the CIL context here to ensure that nothing is modifying - * the percpu state, either addition or removal. Both of these are done under - * the CIL context lock, so grabbing that exclusively here will ensure we can - * safely drain the cilpcp for the CPU that is dying. - */ -void -xlog_cil_pcp_dead( - struct xlog *log, - unsigned int cpu) -{ - struct xfs_cil *cil = log->l_cilp; - struct xlog_cil_pcp *cilpcp = per_cpu_ptr(cil->xc_pcp, cpu); - struct xfs_cil_ctx *ctx; - - down_write(&cil->xc_ctx_lock); - ctx = cil->xc_ctx; - if (ctx->ticket) - ctx->ticket->t_curr_res += cilpcp->space_reserved; - cilpcp->space_reserved = 0; - - if (!list_empty(&cilpcp->log_items)) - list_splice_init(&cilpcp->log_items, &ctx->log_items); - if (!list_empty(&cilpcp->busy_extents)) - list_splice_init(&cilpcp->busy_extents, &ctx->busy_extents); - atomic_add(cilpcp->space_used, &ctx->space_used); - cilpcp->space_used = 0; - up_write(&cil->xc_ctx_lock); -} - /* * Perform initial CIL structure initialisation. */ diff --git a/fs/xfs/xfs_log_priv.h b/fs/xfs/xfs_log_priv.h index 1bd2963e8fbd..fa3ad1d7b31c 100644 --- a/fs/xfs/xfs_log_priv.h +++ b/fs/xfs/xfs_log_priv.h @@ -6,6 +6,8 @@ #ifndef __XFS_LOG_PRIV_H__ #define __XFS_LOG_PRIV_H__ +#include "xfs_extent_busy.h" /* for struct xfs_busy_extents */ + struct xfs_buf; struct xlog; struct xlog_ticket; @@ -223,14 +225,19 @@ struct xfs_cil_ctx { struct xlog_in_core *commit_iclog; struct xlog_ticket *ticket; /* chkpt ticket */ atomic_t space_used; /* aggregate size of regions */ - struct list_head busy_extents; /* busy extents in chkpt */ + struct xfs_busy_extents busy_extents; struct list_head log_items; /* log items in chkpt */ struct list_head lv_chain; /* logvecs being pushed */ struct list_head iclog_entry; struct list_head committing; /* ctx committing list */ - struct work_struct discard_endio_work; struct work_struct push_work; atomic_t order_id; + + /* + * CPUs that could have added items to the percpu CIL data. Access is + * coordinated with xc_ctx_lock. + */ + struct cpumask cil_pcpmask; }; /* @@ -278,9 +285,6 @@ struct xfs_cil { wait_queue_head_t xc_push_wait; /* background push throttle */ void __percpu *xc_pcp; /* percpu CIL structures */ -#ifdef CONFIG_HOTPLUG_CPU - struct list_head xc_pcp_list; -#endif } ____cacheline_aligned_in_smp; /* xc_flags bit values */ @@ -705,9 +709,4 @@ xlog_kvmalloc( return p; } -/* - * CIL CPU dead notifier - */ -void xlog_cil_pcp_dead(struct xlog *log, unsigned int cpu); - #endif /* __XFS_LOG_PRIV_H__ */ diff --git a/fs/xfs/xfs_log_recover.c b/fs/xfs/xfs_log_recover.c index 82c81d20459d..13b94d2e605b 100644 --- a/fs/xfs/xfs_log_recover.c +++ b/fs/xfs/xfs_log_recover.c @@ -329,7 +329,7 @@ xlog_find_verify_cycle( * try a smaller size. We need to be able to read at least * a log sector, or we're out of luck. */ - bufblks = 1 << ffs(nbblks); + bufblks = roundup_pow_of_two(nbblks); while (bufblks > log->l_logBBsize) bufblks >>= 1; while (!(buffer = xlog_alloc_buffer(log, bufblks))) { @@ -1528,7 +1528,7 @@ xlog_write_log_records( * a smaller size. We need to be able to write at least a * log sector, or we're out of luck. */ - bufblks = 1 << ffs(blocks); + bufblks = roundup_pow_of_two(blocks); while (bufblks > log->l_logBBsize) bufblks >>= 1; while (!(buffer = xlog_alloc_buffer(log, bufblks))) { diff --git a/fs/xfs/xfs_mount.h b/fs/xfs/xfs_mount.h index a25eece3be2b..d19cca099bc3 100644 --- a/fs/xfs/xfs_mount.h +++ b/fs/xfs/xfs_mount.h @@ -60,6 +60,7 @@ struct xfs_error_cfg { * Per-cpu deferred inode inactivation GC lists. */ struct xfs_inodegc { + struct xfs_mount *mp; struct llist_head list; struct delayed_work work; int error; @@ -67,9 +68,7 @@ struct xfs_inodegc { /* approximate count of inodes in the list */ unsigned int items; unsigned int shrinker_hits; -#if defined(DEBUG) || defined(XFS_WARN) unsigned int cpu; -#endif }; /* @@ -98,7 +97,6 @@ typedef struct xfs_mount { xfs_buftarg_t *m_ddev_targp; /* saves taking the address */ xfs_buftarg_t *m_logdev_targp;/* ptr to log device */ xfs_buftarg_t *m_rtdev_targp; /* ptr to rt device */ - struct list_head m_mount_list; /* global mount list */ void __percpu *m_inodegc; /* percpu inodegc structures */ /* @@ -249,6 +247,9 @@ typedef struct xfs_mount { unsigned int *m_errortag; struct xfs_kobj m_errortag_kobj; #endif + + /* cpus that have inodes queued for inactivation */ + struct cpumask m_inodegc_cpumask; } xfs_mount_t; #define M_IGEO(mp) (&(mp)->m_ino_geo) @@ -404,6 +405,8 @@ __XFS_HAS_FEAT(nouuid, NOUUID) #define XFS_OPSTATE_WARNED_SHRINK 8 /* Kernel has logged a warning about logged xattr updates being used. */ #define XFS_OPSTATE_WARNED_LARP 9 +/* Mount time quotacheck is running */ +#define XFS_OPSTATE_QUOTACHECK_RUNNING 10 #define __XFS_IS_OPSTATE(name, NAME) \ static inline bool xfs_is_ ## name (struct xfs_mount *mp) \ @@ -426,6 +429,11 @@ __XFS_IS_OPSTATE(inode32, INODE32) __XFS_IS_OPSTATE(readonly, READONLY) __XFS_IS_OPSTATE(inodegc_enabled, INODEGC_ENABLED) __XFS_IS_OPSTATE(blockgc_enabled, BLOCKGC_ENABLED) +#ifdef CONFIG_XFS_QUOTA +__XFS_IS_OPSTATE(quotacheck_running, QUOTACHECK_RUNNING) +#else +# define xfs_is_quotacheck_running(mp) (false) +#endif static inline bool xfs_should_warn(struct xfs_mount *mp, long nr) @@ -443,7 +451,8 @@ xfs_should_warn(struct xfs_mount *mp, long nr) { (1UL << XFS_OPSTATE_BLOCKGC_ENABLED), "blockgc" }, \ { (1UL << XFS_OPSTATE_WARNED_SCRUB), "wscrub" }, \ { (1UL << XFS_OPSTATE_WARNED_SHRINK), "wshrink" }, \ - { (1UL << XFS_OPSTATE_WARNED_LARP), "wlarp" } + { (1UL << XFS_OPSTATE_WARNED_LARP), "wlarp" }, \ + { (1UL << XFS_OPSTATE_QUOTACHECK_RUNNING), "quotacheck" } /* * Max and min values for mount-option defined I/O diff --git a/fs/xfs/xfs_notify_failure.c b/fs/xfs/xfs_notify_failure.c index 4a9bbd3fe120..a7daa522e00f 100644 --- a/fs/xfs/xfs_notify_failure.c +++ b/fs/xfs/xfs_notify_failure.c @@ -126,8 +126,8 @@ xfs_dax_notify_ddev_failure( struct xfs_rmap_irec ri_low = { }; struct xfs_rmap_irec ri_high; struct xfs_agf *agf; - xfs_agblock_t agend; struct xfs_perag *pag; + xfs_agblock_t range_agend; pag = xfs_perag_get(mp, agno); error = xfs_alloc_read_agf(pag, tp, 0, &agf_bp); @@ -148,10 +148,10 @@ xfs_dax_notify_ddev_failure( ri_high.rm_startblock = XFS_FSB_TO_AGBNO(mp, end_fsbno); agf = agf_bp->b_addr; - agend = min(be32_to_cpu(agf->agf_length), + range_agend = min(be32_to_cpu(agf->agf_length) - 1, ri_high.rm_startblock); notify.startblock = ri_low.rm_startblock; - notify.blockcount = agend - ri_low.rm_startblock; + notify.blockcount = range_agend + 1 - ri_low.rm_startblock; error = xfs_rmap_query_range(cur, &ri_low, &ri_high, xfs_dax_failure_fn, ¬ify); diff --git a/fs/xfs/xfs_qm.c b/fs/xfs/xfs_qm.c index 6abcc34fafd8..086e78a6143a 100644 --- a/fs/xfs/xfs_qm.c +++ b/fs/xfs/xfs_qm.c @@ -1160,6 +1160,19 @@ xfs_qm_dqusage_adjust( if (error) return error; + /* + * Reload the incore unlinked list to avoid failure in inodegc. + * Use an unlocked check here because unrecovered unlinked inodes + * should be somewhat rare. + */ + if (xfs_inode_unlinked_incomplete(ip)) { + error = xfs_inode_reload_unlinked(ip); + if (error) { + xfs_force_shutdown(mp, SHUTDOWN_CORRUPT_INCORE); + goto error0; + } + } + ASSERT(ip->i_delayed_blks == 0); if (XFS_IS_REALTIME_INODE(ip)) { @@ -1173,6 +1186,7 @@ xfs_qm_dqusage_adjust( } nblks = (xfs_qcnt_t)ip->i_nblocks - rtblks; + xfs_iflags_clear(ip, XFS_IQUOTAUNCHECKED); /* * Add the (disk blocks and inode) resources occupied by this @@ -1319,8 +1333,10 @@ xfs_qm_quotacheck( flags |= XFS_PQUOTA_CHKD; } + xfs_set_quotacheck_running(mp); error = xfs_iwalk_threaded(mp, 0, 0, xfs_qm_dqusage_adjust, 0, true, NULL); + xfs_clear_quotacheck_running(mp); /* * On error, the inode walk may have partially populated the dquot diff --git a/fs/xfs/xfs_refcount_item.c b/fs/xfs/xfs_refcount_item.c index edd8587658d5..2d4444d61e98 100644 --- a/fs/xfs/xfs_refcount_item.c +++ b/fs/xfs/xfs_refcount_item.c @@ -477,6 +477,7 @@ xfs_cui_item_recover( struct xfs_log_item *lip, struct list_head *capture_list) { + struct xfs_trans_res resv; struct xfs_cui_log_item *cuip = CUI_ITEM(lip); struct xfs_cud_log_item *cudp; struct xfs_trans *tp; @@ -514,8 +515,9 @@ xfs_cui_item_recover( * doesn't fit. We need to reserve enough blocks to handle a * full btree split on either end of the refcount range. */ - error = xfs_trans_alloc(mp, &M_RES(mp)->tr_itruncate, - mp->m_refc_maxlevels * 2, 0, XFS_TRANS_RESERVE, &tp); + resv = xlog_recover_resv(&M_RES(mp)->tr_itruncate); + error = xfs_trans_alloc(mp, &resv, mp->m_refc_maxlevels * 2, 0, + XFS_TRANS_RESERVE, &tp); if (error) return error; diff --git a/fs/xfs/xfs_rmap_item.c b/fs/xfs/xfs_rmap_item.c index 520c7ebdfed8..0e0e747028da 100644 --- a/fs/xfs/xfs_rmap_item.c +++ b/fs/xfs/xfs_rmap_item.c @@ -507,6 +507,7 @@ xfs_rui_item_recover( struct xfs_log_item *lip, struct list_head *capture_list) { + struct xfs_trans_res resv; struct xfs_rui_log_item *ruip = RUI_ITEM(lip); struct xfs_rud_log_item *rudp; struct xfs_trans *tp; @@ -530,8 +531,9 @@ xfs_rui_item_recover( } } - error = xfs_trans_alloc(mp, &M_RES(mp)->tr_itruncate, - mp->m_rmap_maxlevels, 0, XFS_TRANS_RESERVE, &tp); + resv = xlog_recover_resv(&M_RES(mp)->tr_itruncate); + error = xfs_trans_alloc(mp, &resv, mp->m_rmap_maxlevels, 0, + XFS_TRANS_RESERVE, &tp); if (error) return error; rudp = xfs_trans_get_rud(tp, ruip); diff --git a/fs/xfs/xfs_super.c b/fs/xfs/xfs_super.c index 1f77014c6e1a..819a3568b28f 100644 --- a/fs/xfs/xfs_super.c +++ b/fs/xfs/xfs_super.c @@ -56,28 +56,6 @@ static struct kset *xfs_kset; /* top-level xfs sysfs dir */ static struct xfs_kobj xfs_dbg_kobj; /* global debug sysfs attrs */ #endif -#ifdef CONFIG_HOTPLUG_CPU -static LIST_HEAD(xfs_mount_list); -static DEFINE_SPINLOCK(xfs_mount_list_lock); - -static inline void xfs_mount_list_add(struct xfs_mount *mp) -{ - spin_lock(&xfs_mount_list_lock); - list_add(&mp->m_mount_list, &xfs_mount_list); - spin_unlock(&xfs_mount_list_lock); -} - -static inline void xfs_mount_list_del(struct xfs_mount *mp) -{ - spin_lock(&xfs_mount_list_lock); - list_del(&mp->m_mount_list); - spin_unlock(&xfs_mount_list_lock); -} -#else /* !CONFIG_HOTPLUG_CPU */ -static inline void xfs_mount_list_add(struct xfs_mount *mp) {} -static inline void xfs_mount_list_del(struct xfs_mount *mp) {} -#endif - enum xfs_dax_mode { XFS_DAX_INODE = 0, XFS_DAX_ALWAYS = 1, @@ -1135,9 +1113,8 @@ xfs_inodegc_init_percpu( for_each_possible_cpu(cpu) { gc = per_cpu_ptr(mp->m_inodegc, cpu); -#if defined(DEBUG) || defined(XFS_WARN) gc->cpu = cpu; -#endif + gc->mp = mp; init_llist_head(&gc->list); gc->items = 0; gc->error = 0; @@ -1168,7 +1145,6 @@ xfs_fs_put_super( xfs_freesb(mp); xchk_mount_stats_free(mp); free_percpu(mp->m_stats.xs_stats); - xfs_mount_list_del(mp); xfs_inodegc_free_percpu(mp); xfs_destroy_percpu_counters(mp); xfs_destroy_mount_workqueues(mp); @@ -1577,13 +1553,6 @@ xfs_fs_fill_super( if (error) goto out_destroy_counters; - /* - * All percpu data structures requiring cleanup when a cpu goes offline - * must be allocated before adding this @mp to the cpu-dead handler's - * mount list. - */ - xfs_mount_list_add(mp); - /* Allocate stats memory before we do operations that might use it */ mp->m_stats.xs_stats = alloc_percpu(struct xfsstats); if (!mp->m_stats.xs_stats) { @@ -1781,7 +1750,6 @@ xfs_fs_fill_super( out_free_stats: free_percpu(mp->m_stats.xs_stats); out_destroy_inodegc: - xfs_mount_list_del(mp); xfs_inodegc_free_percpu(mp); out_destroy_counters: xfs_destroy_percpu_counters(mp); @@ -2065,7 +2033,7 @@ static struct file_system_type xfs_fs_type = { .init_fs_context = xfs_init_fs_context, .parameters = xfs_fs_parameters, .kill_sb = xfs_kill_sb, - .fs_flags = FS_REQUIRES_DEV | FS_ALLOW_IDMAP | FS_MGTIME, + .fs_flags = FS_REQUIRES_DEV | FS_ALLOW_IDMAP, }; MODULE_ALIAS_FS("xfs"); @@ -2326,49 +2294,6 @@ xfs_destroy_workqueues(void) destroy_workqueue(xfs_alloc_wq); } -#ifdef CONFIG_HOTPLUG_CPU -static int -xfs_cpu_dead( - unsigned int cpu) -{ - struct xfs_mount *mp, *n; - - spin_lock(&xfs_mount_list_lock); - list_for_each_entry_safe(mp, n, &xfs_mount_list, m_mount_list) { - spin_unlock(&xfs_mount_list_lock); - xfs_inodegc_cpu_dead(mp, cpu); - xlog_cil_pcp_dead(mp->m_log, cpu); - spin_lock(&xfs_mount_list_lock); - } - spin_unlock(&xfs_mount_list_lock); - return 0; -} - -static int __init -xfs_cpu_hotplug_init(void) -{ - int error; - - error = cpuhp_setup_state_nocalls(CPUHP_XFS_DEAD, "xfs:dead", NULL, - xfs_cpu_dead); - if (error < 0) - xfs_alert(NULL, -"Failed to initialise CPU hotplug, error %d. XFS is non-functional.", - error); - return error; -} - -static void -xfs_cpu_hotplug_destroy(void) -{ - cpuhp_remove_state_nocalls(CPUHP_XFS_DEAD); -} - -#else /* !CONFIG_HOTPLUG_CPU */ -static inline int xfs_cpu_hotplug_init(void) { return 0; } -static inline void xfs_cpu_hotplug_destroy(void) {} -#endif - STATIC int __init init_xfs_fs(void) { @@ -2385,13 +2310,9 @@ init_xfs_fs(void) xfs_dir_startup(); - error = xfs_cpu_hotplug_init(); - if (error) - goto out; - error = xfs_init_caches(); if (error) - goto out_destroy_hp; + goto out; error = xfs_init_workqueues(); if (error) @@ -2475,8 +2396,6 @@ init_xfs_fs(void) xfs_destroy_workqueues(); out_destroy_caches: xfs_destroy_caches(); - out_destroy_hp: - xfs_cpu_hotplug_destroy(); out: return error; } @@ -2500,7 +2419,6 @@ exit_xfs_fs(void) xfs_destroy_workqueues(); xfs_destroy_caches(); xfs_uuid_table_free(); - xfs_cpu_hotplug_destroy(); } module_init(init_xfs_fs); diff --git a/fs/xfs/xfs_trace.h b/fs/xfs/xfs_trace.h index 902c7f67a117..3926cf7f2a6e 100644 --- a/fs/xfs/xfs_trace.h +++ b/fs/xfs/xfs_trace.h @@ -3824,6 +3824,51 @@ TRACE_EVENT(xfs_iunlink_update_dinode, __entry->new_ptr) ); +TRACE_EVENT(xfs_iunlink_reload_next, + TP_PROTO(struct xfs_inode *ip), + TP_ARGS(ip), + TP_STRUCT__entry( + __field(dev_t, dev) + __field(xfs_agnumber_t, agno) + __field(xfs_agino_t, agino) + __field(xfs_agino_t, prev_agino) + __field(xfs_agino_t, next_agino) + ), + TP_fast_assign( + __entry->dev = ip->i_mount->m_super->s_dev; + __entry->agno = XFS_INO_TO_AGNO(ip->i_mount, ip->i_ino); + __entry->agino = XFS_INO_TO_AGINO(ip->i_mount, ip->i_ino); + __entry->prev_agino = ip->i_prev_unlinked; + __entry->next_agino = ip->i_next_unlinked; + ), + TP_printk("dev %d:%d agno 0x%x agino 0x%x prev_unlinked 0x%x next_unlinked 0x%x", + MAJOR(__entry->dev), MINOR(__entry->dev), + __entry->agno, + __entry->agino, + __entry->prev_agino, + __entry->next_agino) +); + +TRACE_EVENT(xfs_inode_reload_unlinked_bucket, + TP_PROTO(struct xfs_inode *ip), + TP_ARGS(ip), + TP_STRUCT__entry( + __field(dev_t, dev) + __field(xfs_agnumber_t, agno) + __field(xfs_agino_t, agino) + ), + TP_fast_assign( + __entry->dev = ip->i_mount->m_super->s_dev; + __entry->agno = XFS_INO_TO_AGNO(ip->i_mount, ip->i_ino); + __entry->agino = XFS_INO_TO_AGINO(ip->i_mount, ip->i_ino); + ), + TP_printk("dev %d:%d agno 0x%x agino 0x%x bucket %u", + MAJOR(__entry->dev), MINOR(__entry->dev), + __entry->agno, + __entry->agino, + __entry->agino % XFS_AGI_UNLINKED_BUCKETS) +); + DECLARE_EVENT_CLASS(xfs_ag_inode_class, TP_PROTO(struct xfs_inode *ip), TP_ARGS(ip), diff --git a/fs/xfs/xfs_xattr.c b/fs/xfs/xfs_xattr.c index 43e5c219aaed..a3975f325f4e 100644 --- a/fs/xfs/xfs_xattr.c +++ b/fs/xfs/xfs_xattr.c @@ -46,6 +46,17 @@ xfs_attr_grab_log_assist( if (xfs_sb_version_haslogxattrs(&mp->m_sb)) return 0; + /* + * Check if the filesystem featureset is new enough to set this log + * incompat feature bit. Strictly speaking, the minimum requirement is + * a V5 filesystem for the superblock field, but we'll require rmap + * or reflink to avoid having to deal with really old kernels. + */ + if (!xfs_has_reflink(mp) && !xfs_has_rmapbt(mp)) { + error = -EOPNOTSUPP; + goto drop_incompat; + } + /* Enable log-assisted xattrs. */ error = xfs_add_incompat_log_feature(mp, XFS_SB_FEAT_INCOMPAT_LOG_XATTRS); diff --git a/include/acpi/processor.h b/include/acpi/processor.h index 94181fe9780a..3f34ebb27525 100644 --- a/include/acpi/processor.h +++ b/include/acpi/processor.h @@ -465,9 +465,4 @@ extern int acpi_processor_ffh_lpi_probe(unsigned int cpu); extern int acpi_processor_ffh_lpi_enter(struct acpi_lpi_state *lpi); #endif -#ifdef CONFIG_ACPI_HOTPLUG_CPU -extern int arch_register_cpu(int cpu); -extern void arch_unregister_cpu(int cpu); -#endif - #endif diff --git a/include/asm-generic/hugetlb.h b/include/asm-generic/hugetlb.h index 4da02798a00b..6dcf4d576970 100644 --- a/include/asm-generic/hugetlb.h +++ b/include/asm-generic/hugetlb.h @@ -76,7 +76,7 @@ static inline void hugetlb_free_pgd_range(struct mmu_gather *tlb, #ifndef __HAVE_ARCH_HUGE_SET_HUGE_PTE_AT static inline void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, pte_t pte) + pte_t *ptep, pte_t pte, unsigned long sz) { set_pte_at(mm, addr, ptep, pte); } diff --git a/include/asm-generic/mshyperv.h b/include/asm-generic/mshyperv.h index cecd2b7bd033..430f0ae0dde2 100644 --- a/include/asm-generic/mshyperv.h +++ b/include/asm-generic/mshyperv.h @@ -36,6 +36,7 @@ struct ms_hyperv_info { u32 nested_features; u32 max_vp_index; u32 max_lp_index; + u8 vtl; union { u32 isolation_config_a; struct { @@ -54,7 +55,6 @@ struct ms_hyperv_info { }; }; u64 shared_gpa_boundary; - u8 vtl; }; extern struct ms_hyperv_info ms_hyperv; extern bool hv_nested; diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h index 9c59409104f6..67d8dd2f1bde 100644 --- a/include/asm-generic/vmlinux.lds.h +++ b/include/asm-generic/vmlinux.lds.h @@ -138,13 +138,6 @@ * are handled as text/data or they can be discarded (which * often happens at runtime) */ -#ifdef CONFIG_HOTPLUG_CPU -#define CPU_KEEP(sec) *(.cpu##sec) -#define CPU_DISCARD(sec) -#else -#define CPU_KEEP(sec) -#define CPU_DISCARD(sec) *(.cpu##sec) -#endif #if defined(CONFIG_MEMORY_HOTPLUG) #define MEM_KEEP(sec) *(.mem##sec) diff --git a/include/drm/drm_exec.h b/include/drm/drm_exec.h index e0462361adf9..b5bf0b6da791 100644 --- a/include/drm/drm_exec.h +++ b/include/drm/drm_exec.h @@ -51,6 +51,20 @@ struct drm_exec { struct drm_gem_object *prelocked; }; +/** + * drm_exec_obj() - Return the object for a give drm_exec index + * @exec: Pointer to the drm_exec context + * @index: The index. + * + * Return: Pointer to the locked object corresponding to @index if + * index is within the number of locked objects. NULL otherwise. + */ +static inline struct drm_gem_object * +drm_exec_obj(struct drm_exec *exec, unsigned long index) +{ + return index < exec->num_objects ? exec->objects[index] : NULL; +} + /** * drm_exec_for_each_locked_object - iterate over all the locked objects * @exec: drm_exec object @@ -59,10 +73,23 @@ struct drm_exec { * * Iterate over all the locked GEM objects inside the drm_exec object. */ -#define drm_exec_for_each_locked_object(exec, index, obj) \ - for (index = 0, obj = (exec)->objects[0]; \ - index < (exec)->num_objects; \ - ++index, obj = (exec)->objects[index]) +#define drm_exec_for_each_locked_object(exec, index, obj) \ + for ((index) = 0; ((obj) = drm_exec_obj(exec, index)); ++(index)) + +/** + * drm_exec_for_each_locked_object_reverse - iterate over all the locked + * objects in reverse locking order + * @exec: drm_exec object + * @index: unsigned long index for the iteration + * @obj: the current GEM object + * + * Iterate over all the locked GEM objects inside the drm_exec object in + * reverse locking order. Note that @index may go below zero and wrap, + * but that will be caught by drm_exec_obj(), returning a NULL object. + */ +#define drm_exec_for_each_locked_object_reverse(exec, index, obj) \ + for ((index) = (exec)->num_objects - 1; \ + ((obj) = drm_exec_obj(exec, index)); --(index)) /** * drm_exec_until_all_locked - loop until all GEM objects are locked diff --git a/include/drm/drm_kunit_helpers.h b/include/drm/drm_kunit_helpers.h index 514c8a7a32f0..ba483c87f0e7 100644 --- a/include/drm/drm_kunit_helpers.h +++ b/include/drm/drm_kunit_helpers.h @@ -3,6 +3,8 @@ #ifndef DRM_KUNIT_HELPERS_H_ #define DRM_KUNIT_HELPERS_H_ +#include + #include struct drm_device; @@ -51,7 +53,7 @@ __drm_kunit_helper_alloc_drm_device(struct kunit *test, { struct drm_driver *driver; - driver = kunit_kzalloc(test, sizeof(*driver), GFP_KERNEL); + driver = devm_kzalloc(dev, sizeof(*driver), GFP_KERNEL); KUNIT_ASSERT_NOT_NULL(test, driver); driver->driver_features = features; diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h index f9544d9b670d..ac65f0626cfc 100644 --- a/include/drm/gpu_scheduler.h +++ b/include/drm/gpu_scheduler.h @@ -68,8 +68,7 @@ enum drm_sched_priority { DRM_SCHED_PRIORITY_HIGH, DRM_SCHED_PRIORITY_KERNEL, - DRM_SCHED_PRIORITY_COUNT, - DRM_SCHED_PRIORITY_UNSET = -2 + DRM_SCHED_PRIORITY_COUNT }; /* Used to chose between FIFO and RR jobs scheduling */ diff --git a/include/kvm/arm_arch_timer.h b/include/kvm/arm_arch_timer.h index bb3cb005873e..e748bc957d83 100644 --- a/include/kvm/arm_arch_timer.h +++ b/include/kvm/arm_arch_timer.h @@ -82,6 +82,8 @@ struct timer_map { struct arch_timer_context *emul_ptimer; }; +void get_timer_map(struct kvm_vcpu *vcpu, struct timer_map *map); + struct arch_timer_cpu { struct arch_timer_context timers[NR_KVM_TIMERS]; @@ -145,4 +147,9 @@ u64 timer_get_cval(struct arch_timer_context *ctxt); void kvm_timer_cpu_up(void); void kvm_timer_cpu_down(void); +static inline bool has_cntpoff(void) +{ + return (has_vhe() && cpus_have_final_cap(ARM64_HAS_ECV_CNTPOFF)); +} + #endif diff --git a/include/linux/acpi.h b/include/linux/acpi.h index a73246c3c35e..afd94c9b8b8a 100644 --- a/include/linux/acpi.h +++ b/include/linux/acpi.h @@ -1480,6 +1480,15 @@ static inline int lpit_read_residency_count_address(u64 *address) } #endif +#ifdef CONFIG_ACPI_PROCESSOR_IDLE +#ifndef arch_get_idle_state_flags +static inline unsigned int arch_get_idle_state_flags(u32 arch_flags) +{ + return 0; +} +#endif +#endif /* CONFIG_ACPI_PROCESSOR_IDLE */ + #ifdef CONFIG_ACPI_PPTT int acpi_pptt_cpu_is_thread(unsigned int cpu); int find_acpi_cpu_topology(unsigned int cpu, int level); diff --git a/include/linux/aer.h b/include/linux/aer.h index 2dd175f5debd..29cc10220952 100644 --- a/include/linux/aer.h +++ b/include/linux/aer.h @@ -42,11 +42,13 @@ struct aer_capability_regs { #if defined(CONFIG_PCIEAER) int pci_aer_clear_nonfatal_status(struct pci_dev *dev); +int pcie_aer_is_native(struct pci_dev *dev); #else static inline int pci_aer_clear_nonfatal_status(struct pci_dev *dev) { return -EINVAL; } +static inline int pcie_aer_is_native(struct pci_dev *dev) { return 0; } #endif void cper_print_aer(struct pci_dev *dev, int aer_severity, diff --git a/include/linux/arm-smccc.h b/include/linux/arm-smccc.h index 7c67c17321d4..083f85653716 100644 --- a/include/linux/arm-smccc.h +++ b/include/linux/arm-smccc.h @@ -67,6 +67,8 @@ #define ARM_SMCCC_VERSION_1_3 0x10003 #define ARM_SMCCC_1_3_SVE_HINT 0x10000 +#define ARM_SMCCC_CALL_HINTS ARM_SMCCC_1_3_SVE_HINT + #define ARM_SMCCC_VERSION_FUNC_ID \ ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, \ diff --git a/include/linux/atomic/atomic-arch-fallback.h b/include/linux/atomic/atomic-arch-fallback.h index 18f5744dfb5d..b83ef19da13d 100644 --- a/include/linux/atomic/atomic-arch-fallback.h +++ b/include/linux/atomic/atomic-arch-fallback.h @@ -459,8 +459,6 @@ raw_atomic_read_acquire(const atomic_t *v) { #if defined(arch_atomic_read_acquire) return arch_atomic_read_acquire(v); -#elif defined(arch_atomic_read) - return arch_atomic_read(v); #else int ret; @@ -508,8 +506,6 @@ raw_atomic_set_release(atomic_t *v, int i) { #if defined(arch_atomic_set_release) arch_atomic_set_release(v, i); -#elif defined(arch_atomic_set) - arch_atomic_set(v, i); #else if (__native_word(atomic_t)) { smp_store_release(&(v)->counter, i); @@ -2575,8 +2571,6 @@ raw_atomic64_read_acquire(const atomic64_t *v) { #if defined(arch_atomic64_read_acquire) return arch_atomic64_read_acquire(v); -#elif defined(arch_atomic64_read) - return arch_atomic64_read(v); #else s64 ret; @@ -2624,8 +2618,6 @@ raw_atomic64_set_release(atomic64_t *v, s64 i) { #if defined(arch_atomic64_set_release) arch_atomic64_set_release(v, i); -#elif defined(arch_atomic64_set) - arch_atomic64_set(v, i); #else if (__native_word(atomic64_t)) { smp_store_release(&(v)->counter, i); @@ -4657,4 +4649,4 @@ raw_atomic64_dec_if_positive(atomic64_t *v) } #endif /* _LINUX_ATOMIC_FALLBACK_H */ -// 202b45c7db600ce36198eb1f1fc2c2d5268ace2d +// 2fdd6702823fa842f9cea57a002e6e4476ae780c diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 024e8b28c34b..49f8b691496c 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -1307,7 +1307,7 @@ static inline int bpf_trampoline_unlink_prog(struct bpf_tramp_link *link, static inline struct bpf_trampoline *bpf_trampoline_get(u64 key, struct bpf_attach_target_info *tgt_info) { - return ERR_PTR(-EOPNOTSUPP); + return NULL; } static inline void bpf_trampoline_put(struct bpf_trampoline *tr) {} #define DEFINE_BPF_DISPATCHER(name) diff --git a/include/linux/btf_ids.h b/include/linux/btf_ids.h index a3462a9b8e18..a9cb10b0e2e9 100644 --- a/include/linux/btf_ids.h +++ b/include/linux/btf_ids.h @@ -49,7 +49,7 @@ word \ ____BTF_ID(symbol, word) #define __ID(prefix) \ - __PASTE(prefix, __COUNTER__) + __PASTE(__PASTE(prefix, __COUNTER__), __LINE__) /* * The BTF_ID defines unique symbol for each ID pointing diff --git a/include/linux/buffer_head.h b/include/linux/buffer_head.h index 4ede47649a81..44e9de51eedf 100644 --- a/include/linux/buffer_head.h +++ b/include/linux/buffer_head.h @@ -171,7 +171,10 @@ static __always_inline int buffer_uptodate(const struct buffer_head *bh) return test_bit_acquire(BH_Uptodate, &bh->b_state); } -#define bh_offset(bh) ((unsigned long)(bh)->b_data & ~PAGE_MASK) +static inline unsigned long bh_offset(const struct buffer_head *bh) +{ + return (unsigned long)(bh)->b_data & (page_size(bh->b_page) - 1); +} /* If we *know* page->private refers to buffer_heads */ #define page_buffers(page) \ diff --git a/include/linux/ceph/ceph_fs.h b/include/linux/ceph/ceph_fs.h index 5f2301ee88bc..f3b3593254b9 100644 --- a/include/linux/ceph/ceph_fs.h +++ b/include/linux/ceph/ceph_fs.h @@ -467,19 +467,17 @@ union ceph_mds_request_args { } __attribute__ ((packed)); union ceph_mds_request_args_ext { - union { - union ceph_mds_request_args old; - struct { - __le32 mode; - __le32 uid; - __le32 gid; - struct ceph_timespec mtime; - struct ceph_timespec atime; - __le64 size, old_size; /* old_size needed by truncate */ - __le32 mask; /* CEPH_SETATTR_* */ - struct ceph_timespec btime; - } __attribute__ ((packed)) setattr_ext; - }; + union ceph_mds_request_args old; + struct { + __le32 mode; + __le32 uid; + __le32 gid; + struct ceph_timespec mtime; + struct ceph_timespec atime; + __le64 size, old_size; /* old_size needed by truncate */ + __le32 mask; /* CEPH_SETATTR_* */ + struct ceph_timespec btime; + } __attribute__ ((packed)) setattr_ext; }; #define CEPH_MDS_FLAG_REPLAY 1 /* this is a replayed op */ diff --git a/include/linux/cgroup-defs.h b/include/linux/cgroup-defs.h index f1b3151ac30b..265da00a1a8b 100644 --- a/include/linux/cgroup-defs.h +++ b/include/linux/cgroup-defs.h @@ -238,7 +238,7 @@ struct css_set { * Lists running through all tasks using this cgroup group. * mg_tasks lists tasks which belong to this cset but are in the * process of being migrated out or in. Protected by - * css_set_rwsem, but, during migration, once tasks are moved to + * css_set_lock, but, during migration, once tasks are moved to * mg_tasks, it can be read safely while holding cgroup_mutex. */ struct list_head tasks; diff --git a/include/linux/cpu.h b/include/linux/cpu.h index 0abd60a7987b..eb768a866fe3 100644 --- a/include/linux/cpu.h +++ b/include/linux/cpu.h @@ -80,6 +80,8 @@ extern __printf(4, 5) struct device *cpu_device_create(struct device *parent, void *drvdata, const struct attribute_group **groups, const char *fmt, ...); +extern int arch_register_cpu(int cpu); +extern void arch_unregister_cpu(int cpu); #ifdef CONFIG_HOTPLUG_CPU extern void unregister_cpu(struct cpu *cpu); extern ssize_t arch_cpu_probe(const char *, size_t); diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h index 06dda85f0424..068f7738be22 100644 --- a/include/linux/cpuhotplug.h +++ b/include/linux/cpuhotplug.h @@ -90,7 +90,6 @@ enum cpuhp_state { CPUHP_FS_BUFF_DEAD, CPUHP_PRINTK_DEAD, CPUHP_MM_MEMCQ_DEAD, - CPUHP_XFS_DEAD, CPUHP_PERCPU_CNT_DEAD, CPUHP_RADIX_DEAD, CPUHP_PAGE_ALLOC, diff --git a/include/linux/dma-fence.h b/include/linux/dma-fence.h index 0d678e9a7b24..ebe78bd3d121 100644 --- a/include/linux/dma-fence.h +++ b/include/linux/dma-fence.h @@ -568,6 +568,25 @@ static inline void dma_fence_set_error(struct dma_fence *fence, fence->error = error; } +/** + * dma_fence_timestamp - helper to get the completion timestamp of a fence + * @fence: fence to get the timestamp from. + * + * After a fence is signaled the timestamp is updated with the signaling time, + * but setting the timestamp can race with tasks waiting for the signaling. This + * helper busy waits for the correct timestamp to appear. + */ +static inline ktime_t dma_fence_timestamp(struct dma_fence *fence) +{ + if (WARN_ON(!test_bit(DMA_FENCE_FLAG_SIGNALED_BIT, &fence->flags))) + return ktime_get(); + + while (!test_bit(DMA_FENCE_FLAG_TIMESTAMP_BIT, &fence->flags)) + cpu_relax(); + + return fence->timestamp; +} + signed long dma_fence_wait_timeout(struct dma_fence *, bool intr, signed long timeout); signed long dma_fence_wait_any_timeout(struct dma_fence **fences, diff --git a/include/linux/export-internal.h b/include/linux/export-internal.h index 1c849db953a5..45fca09b2319 100644 --- a/include/linux/export-internal.h +++ b/include/linux/export-internal.h @@ -52,6 +52,8 @@ #ifdef CONFIG_IA64 #define KSYM_FUNC(name) @fptr(name) +#elif defined(CONFIG_PARISC) && defined(CONFIG_64BIT) +#define KSYM_FUNC(name) P%name #else #define KSYM_FUNC(name) name #endif diff --git a/include/linux/fs.h b/include/linux/fs.h index 4aeb3fa11927..4a40823c3c67 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -1508,47 +1508,18 @@ static inline bool fsuidgid_has_mapping(struct super_block *sb, kgid_has_mapping(fs_userns, kgid); } -struct timespec64 current_mgtime(struct inode *inode); struct timespec64 current_time(struct inode *inode); struct timespec64 inode_set_ctime_current(struct inode *inode); -/* - * Multigrain timestamps - * - * Conditionally use fine-grained ctime and mtime timestamps when there - * are users actively observing them via getattr. The primary use-case - * for this is NFS clients that use the ctime to distinguish between - * different states of the file, and that are often fooled by multiple - * operations that occur in the same coarse-grained timer tick. - * - * The kernel always keeps normalized struct timespec64 values in the ctime, - * which means that only the first 30 bits of the value are used. Use the - * 31st bit of the ctime's tv_nsec field as a flag to indicate that the value - * has been queried since it was last updated. - */ -#define I_CTIME_QUERIED (1L<<30) - /** * inode_get_ctime - fetch the current ctime from the inode * @inode: inode from which to fetch ctime * - * Grab the current ctime tv_nsec field from the inode, mask off the - * I_CTIME_QUERIED flag and return it. This is mostly intended for use by - * internal consumers of the ctime that aren't concerned with ensuring a - * fine-grained update on the next change (e.g. when preparing to store - * the value in the backing store for later retrieval). - * - * This is safe to call regardless of whether the underlying filesystem - * is using multigrain timestamps. + * Grab the current ctime from the inode and return it. */ static inline struct timespec64 inode_get_ctime(const struct inode *inode) { - struct timespec64 ctime; - - ctime.tv_sec = inode->__i_ctime.tv_sec; - ctime.tv_nsec = inode->__i_ctime.tv_nsec & ~I_CTIME_QUERIED; - - return ctime; + return inode->__i_ctime; } /** @@ -2334,7 +2305,6 @@ struct file_system_type { #define FS_USERNS_MOUNT 8 /* Can be mounted by userns root */ #define FS_DISALLOW_NOTIFY_PERM 16 /* Disable fanotify permission events */ #define FS_ALLOW_IDMAP 32 /* FS has been updated to handle vfs idmappings. */ -#define FS_MGTIME 64 /* FS uses multigrain timestamps */ #define FS_RENAME_DOES_D_MOVE 32768 /* FS will handle d_move() during rename() internally. */ int (*init_fs_context)(struct fs_context *); const struct fs_parameter_spec *parameters; @@ -2358,17 +2328,6 @@ struct file_system_type { #define MODULE_ALIAS_FS(NAME) MODULE_ALIAS("fs-" NAME) -/** - * is_mgtime: is this inode using multigrain timestamps - * @inode: inode to test for multigrain timestamps - * - * Return true if the inode uses multigrain timestamps, false otherwise. - */ -static inline bool is_mgtime(const struct inode *inode) -{ - return inode->i_sb->s_type->fs_flags & FS_MGTIME; -} - extern struct dentry *mount_bdev(struct file_system_type *fs_type, int flags, const char *dev_name, void *data, int (*fill_super)(struct super_block *, void *, int)); @@ -2444,7 +2403,7 @@ struct audit_names; struct filename { const char *name; /* pointer to actual string */ const __user char *uptr; /* original userland pointer */ - int refcnt; + atomic_t refcnt; struct audit_names *aname; const char iname[]; }; @@ -3054,7 +3013,6 @@ extern void page_put_link(void *); extern int page_symlink(struct inode *inode, const char *symname, int len); extern const struct inode_operations page_symlink_inode_operations; extern void kfree_link(void *); -void fill_mg_cmtime(struct kstat *stat, u32 request_mask, struct inode *inode); void generic_fillattr(struct mnt_idmap *, u32, struct inode *, struct kstat *); void generic_fill_statx_attr(struct inode *inode, struct kstat *stat); extern int vfs_getattr_nosec(const struct path *, struct kstat *, u32, unsigned int); diff --git a/include/linux/fs_context.h b/include/linux/fs_context.h index 96332db693d5..c13e99cbbf81 100644 --- a/include/linux/fs_context.h +++ b/include/linux/fs_context.h @@ -136,6 +136,8 @@ extern struct fs_context *vfs_dup_fs_context(struct fs_context *fc); extern int vfs_parse_fs_param(struct fs_context *fc, struct fs_parameter *param); extern int vfs_parse_fs_string(struct fs_context *fc, const char *key, const char *value, size_t v_size); +int vfs_parse_monolithic_sep(struct fs_context *fc, void *data, + char *(*sep)(char **)); extern int generic_parse_monolithic(struct fs_context *fc, void *data); extern int vfs_get_tree(struct fs_context *fc); extern void put_fs_context(struct fs_context *fc); diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 5b2626063f4f..a30686e649f7 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -984,7 +984,9 @@ static inline void huge_ptep_modify_prot_commit(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, pte_t old_pte, pte_t pte) { - set_huge_pte_at(vma->vm_mm, addr, ptep, pte); + unsigned long psize = huge_page_size(hstate_vma(vma)); + + set_huge_pte_at(vma->vm_mm, addr, ptep, pte, psize); } #endif @@ -1173,7 +1175,7 @@ static inline pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, } static inline void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, pte_t pte) + pte_t *ptep, pte_t pte, unsigned long sz) { } diff --git a/include/linux/i2c.h b/include/linux/i2c.h index 3430cc2b05a6..0dae9db27538 100644 --- a/include/linux/i2c.h +++ b/include/linux/i2c.h @@ -237,7 +237,6 @@ enum i2c_driver_flags { * struct i2c_driver - represent an I2C device driver * @class: What kind of i2c device we instantiate (for detect) * @probe: Callback for device binding - * @probe_new: Transitional callback for device binding - do not use * @remove: Callback for device unbinding * @shutdown: Callback for device shutdown * @alert: Alert callback, for example for the SMBus alert protocol @@ -272,16 +271,8 @@ enum i2c_driver_flags { struct i2c_driver { unsigned int class; - union { /* Standard driver model interfaces */ - int (*probe)(struct i2c_client *client); - /* - * Legacy callback that was part of a conversion of .probe(). - * Today it has the same semantic as .probe(). Don't use for new - * code. - */ - int (*probe_new)(struct i2c_client *client); - }; + int (*probe)(struct i2c_client *client); void (*remove)(struct i2c_client *client); diff --git a/include/linux/if_team.h b/include/linux/if_team.h index 1b9b15a492fa..cdc684e04a2f 100644 --- a/include/linux/if_team.h +++ b/include/linux/if_team.h @@ -189,6 +189,8 @@ struct team { struct net_device *dev; /* associated netdevice */ struct team_pcpu_stats __percpu *pcpu_stats; + const struct header_ops *header_ops_cache; + struct mutex lock; /* used for overall locking, e.g. port lists write */ /* diff --git a/include/linux/instruction_pointer.h b/include/linux/instruction_pointer.h index cda1f706eaeb..aa0b3ffea935 100644 --- a/include/linux/instruction_pointer.h +++ b/include/linux/instruction_pointer.h @@ -2,7 +2,12 @@ #ifndef _LINUX_INSTRUCTION_POINTER_H #define _LINUX_INSTRUCTION_POINTER_H +#include + #define _RET_IP_ (unsigned long)__builtin_return_address(0) + +#ifndef _THIS_IP_ #define _THIS_IP_ ({ __label__ __here; __here: (unsigned long)&&__here; }) +#endif #endif /* _LINUX_INSTRUCTION_POINTER_H */ diff --git a/include/linux/interrupt.h b/include/linux/interrupt.h index a92bce40b04b..4a1dc88ddbff 100644 --- a/include/linux/interrupt.h +++ b/include/linux/interrupt.h @@ -569,8 +569,12 @@ enum * 2) rcu_report_dead() reports the final quiescent states. * * _ IRQ_POLL: irq_poll_cpu_dead() migrates the queue + * + * _ (HR)TIMER_SOFTIRQ: (hr)timers_dead_cpu() migrates the queue */ -#define SOFTIRQ_HOTPLUG_SAFE_MASK (BIT(RCU_SOFTIRQ) | BIT(IRQ_POLL_SOFTIRQ)) +#define SOFTIRQ_HOTPLUG_SAFE_MASK (BIT(TIMER_SOFTIRQ) | BIT(IRQ_POLL_SOFTIRQ) |\ + BIT(HRTIMER_SOFTIRQ) | BIT(RCU_SOFTIRQ)) + /* map softirq index to softirq name. update 'softirq_to_name' in * kernel/softirq.c when adding a new softirq. diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 3df5499f7936..842623d708c2 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -54,7 +54,7 @@ extern p4d_t kasan_early_shadow_p4d[MAX_PTRS_PER_P4D]; int kasan_populate_early_shadow(const void *shadow_start, const void *shadow_end); -#ifndef __HAVE_ARCH_SHADOW_MAP +#ifndef kasan_mem_to_shadow static inline void *kasan_mem_to_shadow(const void *addr) { return (void *)((unsigned long)addr >> KASAN_SHADOW_SCALE_SHIFT) diff --git a/include/linux/libata.h b/include/linux/libata.h index 52d58b13e5ee..2a7d2af0ed80 100644 --- a/include/linux/libata.h +++ b/include/linux/libata.h @@ -192,6 +192,7 @@ enum { ATA_PFLAG_UNLOADING = (1 << 9), /* driver is being unloaded */ ATA_PFLAG_UNLOADED = (1 << 10), /* driver is unloaded */ + ATA_PFLAG_RESUMING = (1 << 16), /* port is being resumed */ ATA_PFLAG_SUSPENDED = (1 << 17), /* port is suspended (power) */ ATA_PFLAG_PM_PENDING = (1 << 18), /* PM operation pending */ ATA_PFLAG_INIT_GTM_VALID = (1 << 19), /* initial gtm data valid */ @@ -222,6 +223,10 @@ enum { ATA_HOST_PARALLEL_SCAN = (1 << 2), /* Ports on this host can be scanned in parallel */ ATA_HOST_IGNORE_ATA = (1 << 3), /* Ignore ATA devices on this host. */ + ATA_HOST_NO_PART = (1 << 4), /* Host does not support partial */ + ATA_HOST_NO_SSC = (1 << 5), /* Host does not support slumber */ + ATA_HOST_NO_DEVSLP = (1 << 6), /* Host does not support devslp */ + /* bits 24:31 of host->flags are reserved for LLD specific flags */ /* various lengths of time */ @@ -255,7 +260,7 @@ enum { * advised to wait only for the following duration before * doing SRST. */ - ATA_TMOUT_PMP_SRST_WAIT = 5000, + ATA_TMOUT_PMP_SRST_WAIT = 10000, /* When the LPM policy is set to ATA_LPM_MAX_POWER, there might * be a spurious PHY event, so ignore the first PHY event that @@ -314,9 +319,10 @@ enum { ATA_EH_ENABLE_LINK = (1 << 3), ATA_EH_PARK = (1 << 5), /* unload heads and stop I/O */ ATA_EH_GET_SUCCESS_SENSE = (1 << 6), /* Get sense data for successful cmd */ + ATA_EH_SET_ACTIVE = (1 << 7), /* Set a device to active power mode */ ATA_EH_PERDEV_MASK = ATA_EH_REVALIDATE | ATA_EH_PARK | - ATA_EH_GET_SUCCESS_SENSE, + ATA_EH_GET_SUCCESS_SENSE | ATA_EH_SET_ACTIVE, ATA_EH_ALL_ACTIONS = ATA_EH_REVALIDATE | ATA_EH_RESET | ATA_EH_ENABLE_LINK, @@ -353,7 +359,7 @@ enum { /* This should match the actual table size of * ata_eh_cmd_timeout_table in libata-eh.c. */ - ATA_EH_CMD_TIMEOUT_TABLE_SIZE = 7, + ATA_EH_CMD_TIMEOUT_TABLE_SIZE = 8, /* Horkage types. May be set by libata or controller on drives (some horkage may be drive/controller pair dependent */ @@ -1144,6 +1150,7 @@ extern int ata_std_bios_param(struct scsi_device *sdev, struct block_device *bdev, sector_t capacity, int geom[]); extern void ata_scsi_unlock_native_capacity(struct scsi_device *sdev); +extern int ata_scsi_slave_alloc(struct scsi_device *sdev); extern int ata_scsi_slave_config(struct scsi_device *sdev); extern void ata_scsi_slave_destroy(struct scsi_device *sdev); extern int ata_scsi_change_queue_depth(struct scsi_device *sdev, @@ -1392,6 +1399,7 @@ extern const struct attribute_group *ata_common_sdev_groups[]; .this_id = ATA_SHT_THIS_ID, \ .emulated = ATA_SHT_EMULATED, \ .proc_name = drv_name, \ + .slave_alloc = ata_scsi_slave_alloc, \ .slave_destroy = ata_scsi_slave_destroy, \ .bios_param = ata_std_bios_param, \ .unlock_native_capacity = ata_scsi_unlock_native_capacity,\ diff --git a/include/linux/maple_tree.h b/include/linux/maple_tree.h index e41c70ac7744..d01e850b570f 100644 --- a/include/linux/maple_tree.h +++ b/include/linux/maple_tree.h @@ -428,6 +428,8 @@ struct ma_wr_state { #define MAS_ROOT ((struct maple_enode *)5UL) #define MAS_NONE ((struct maple_enode *)9UL) #define MAS_PAUSE ((struct maple_enode *)17UL) +#define MAS_OVERFLOW ((struct maple_enode *)33UL) +#define MAS_UNDERFLOW ((struct maple_enode *)65UL) #define MA_ERROR(err) \ ((struct maple_enode *)(((unsigned long)err << 2) | 2UL)) @@ -511,6 +513,15 @@ static inline bool mas_is_paused(const struct ma_state *mas) return mas->node == MAS_PAUSE; } +/* Check if the mas is pointing to a node or not */ +static inline bool mas_is_active(struct ma_state *mas) +{ + if ((unsigned long)mas->node >= MAPLE_RESERVED_RANGE) + return true; + + return false; +} + /** * mas_reset() - Reset a Maple Tree operation state. * @mas: Maple Tree operation state. diff --git a/include/linux/mcb.h b/include/linux/mcb.h index 1e5893138afe..0b971b24a804 100644 --- a/include/linux/mcb.h +++ b/include/linux/mcb.h @@ -63,7 +63,6 @@ static inline struct mcb_bus *to_mcb_bus(struct device *dev) struct mcb_device { struct device dev; struct mcb_bus *bus; - bool is_added; struct mcb_driver *driver; u16 id; int inst; diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index ab94ad4597d0..e4e24da16d2c 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -920,7 +920,7 @@ unsigned long mem_cgroup_get_zone_lru_size(struct lruvec *lruvec, return READ_ONCE(mz->lru_zone_size[zone_idx][lru]); } -void mem_cgroup_handle_over_high(void); +void mem_cgroup_handle_over_high(gfp_t gfp_mask); unsigned long mem_cgroup_get_max(struct mem_cgroup *memcg); @@ -1458,7 +1458,7 @@ static inline void mem_cgroup_unlock_pages(void) rcu_read_unlock(); } -static inline void mem_cgroup_handle_over_high(void) +static inline void mem_cgroup_handle_over_high(gfp_t gfp_mask) { } diff --git a/include/linux/mtd/jedec.h b/include/linux/mtd/jedec.h index 0b6b59f7cfbd..56047a4e54c9 100644 --- a/include/linux/mtd/jedec.h +++ b/include/linux/mtd/jedec.h @@ -21,6 +21,9 @@ struct jedec_ecc_info { /* JEDEC features */ #define JEDEC_FEATURE_16_BIT_BUS (1 << 0) +/* JEDEC Optional Commands */ +#define JEDEC_OPT_CMD_READ_CACHE BIT(1) + struct nand_jedec_params { /* rev info and features block */ /* 'J' 'E' 'S' 'D' */ diff --git a/include/linux/mtd/onfi.h b/include/linux/mtd/onfi.h index a7376f9beddf..55ab2e4d62f9 100644 --- a/include/linux/mtd/onfi.h +++ b/include/linux/mtd/onfi.h @@ -55,6 +55,7 @@ #define ONFI_SUBFEATURE_PARAM_LEN 4 /* ONFI optional commands SET/GET FEATURES supported? */ +#define ONFI_OPT_CMD_READ_CACHE BIT(1) #define ONFI_OPT_CMD_SET_GET_FEATURES BIT(2) struct nand_onfi_params { diff --git a/include/linux/mtd/rawnand.h b/include/linux/mtd/rawnand.h index 90a141ba2a5a..c29ace15a053 100644 --- a/include/linux/mtd/rawnand.h +++ b/include/linux/mtd/rawnand.h @@ -225,6 +225,7 @@ struct gpio_desc; * struct nand_parameters - NAND generic parameters from the parameter page * @model: Model name * @supports_set_get_features: The NAND chip supports setting/getting features + * @supports_read_cache: The NAND chip supports read cache operations * @set_feature_list: Bitmap of features that can be set * @get_feature_list: Bitmap of features that can be get * @onfi: ONFI specific parameters @@ -233,6 +234,7 @@ struct nand_parameters { /* Generic parameters */ const char *model; bool supports_set_get_features; + bool supports_read_cache; DECLARE_BITMAP(set_feature_list, ONFI_FEATURE_NUMBER); DECLARE_BITMAP(get_feature_list, ONFI_FEATURE_NUMBER); diff --git a/include/linux/netfilter/nf_conntrack_sctp.h b/include/linux/netfilter/nf_conntrack_sctp.h index 625f491b95de..fb31312825ae 100644 --- a/include/linux/netfilter/nf_conntrack_sctp.h +++ b/include/linux/netfilter/nf_conntrack_sctp.h @@ -9,6 +9,7 @@ struct ip_ct_sctp { enum sctp_conntrack state; __be32 vtag[IP_CT_DIR_MAX]; + u8 init[IP_CT_DIR_MAX]; u8 last_dir; u8 flags; }; diff --git a/include/linux/nfs_fs_sb.h b/include/linux/nfs_fs_sb.h index 20eeba8b009d..cd628c4b011e 100644 --- a/include/linux/nfs_fs_sb.h +++ b/include/linux/nfs_fs_sb.h @@ -48,6 +48,7 @@ struct nfs_client { #define NFS_CS_NOPING 6 /* - don't ping on connect */ #define NFS_CS_DS 7 /* - Server is a DS */ #define NFS_CS_REUSEPORT 8 /* - reuse src port on reconnect */ +#define NFS_CS_PNFS 9 /* - Server used for pnfs */ struct sockaddr_storage cl_addr; /* server identifier */ size_t cl_addrlen; char * cl_hostname; /* hostname of server */ diff --git a/include/linux/nfs_page.h b/include/linux/nfs_page.h index aa9f4c6ebe26..1c315f854ea8 100644 --- a/include/linux/nfs_page.h +++ b/include/linux/nfs_page.h @@ -157,7 +157,9 @@ extern void nfs_unlock_request(struct nfs_page *req); extern void nfs_unlock_and_release_request(struct nfs_page *); extern struct nfs_page *nfs_page_group_lock_head(struct nfs_page *req); extern int nfs_page_group_lock_subrequests(struct nfs_page *head); -extern void nfs_join_page_group(struct nfs_page *head, struct inode *inode); +extern void nfs_join_page_group(struct nfs_page *head, + struct nfs_commit_info *cinfo, + struct inode *inode); extern int nfs_page_group_lock(struct nfs_page *); extern void nfs_page_group_unlock(struct nfs_page *); extern bool nfs_page_group_sync_on_bit(struct nfs_page *, unsigned int); diff --git a/include/linux/nvme-fc-driver.h b/include/linux/nvme-fc-driver.h index f6ef8cf5d774..4109f1bd6128 100644 --- a/include/linux/nvme-fc-driver.h +++ b/include/linux/nvme-fc-driver.h @@ -53,10 +53,10 @@ struct nvmefc_ls_req { void *rqstaddr; dma_addr_t rqstdma; - __le32 rqstlen; + u32 rqstlen; void *rspaddr; dma_addr_t rspdma; - __le32 rsplen; + u32 rsplen; u32 timeout; void *private; @@ -120,7 +120,7 @@ struct nvmefc_ls_req { struct nvmefc_ls_rsp { void *rspbuf; dma_addr_t rspdma; - __le32 rsplen; + u16 rsplen; void (*done)(struct nvmefc_ls_rsp *rsp); void *nvme_fc_private; /* LLDD is not to access !! */ diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h index e85cd1c0eaf3..7b5406e3288d 100644 --- a/include/linux/perf_event.h +++ b/include/linux/perf_event.h @@ -704,6 +704,7 @@ struct perf_event { /* The cumulative AND of all event_caps for events in this group. */ int group_caps; + unsigned int group_generation; struct perf_event *group_leader; /* * event->pmu will always point to pmu in which this event belongs. diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h index 1fba072b3dac..af7639c3b0a3 100644 --- a/include/linux/pgtable.h +++ b/include/linux/pgtable.h @@ -206,6 +206,14 @@ static inline int pmd_young(pmd_t pmd) #endif #ifndef set_ptes + +#ifndef pte_next_pfn +static inline pte_t pte_next_pfn(pte_t pte) +{ + return __pte(pte_val(pte) + (1UL << PFN_PTE_SHIFT)); +} +#endif + /** * set_ptes - Map consecutive pages to a contiguous range of addresses. * @mm: Address space to map the pages into. @@ -231,7 +239,7 @@ static inline void set_ptes(struct mm_struct *mm, unsigned long addr, if (--nr == 0) break; ptep++; - pte = __pte(pte_val(pte) + (1UL << PFN_PTE_SHIFT)); + pte = pte_next_pfn(pte); } arch_leave_lazy_mmu_mode(); } diff --git a/include/linux/quota.h b/include/linux/quota.h index fd692b4a41d5..07071e64abf3 100644 --- a/include/linux/quota.h +++ b/include/linux/quota.h @@ -285,7 +285,9 @@ static inline void dqstats_dec(unsigned int type) #define DQ_FAKE_B 3 /* no limits only usage */ #define DQ_READ_B 4 /* dquot was read into memory */ #define DQ_ACTIVE_B 5 /* dquot is active (dquot_release not called) */ -#define DQ_LASTSET_B 6 /* Following 6 bits (see QIF_) are reserved\ +#define DQ_RELEASING_B 6 /* dquot is in releasing_dquots list waiting + * to be cleaned up */ +#define DQ_LASTSET_B 7 /* Following 6 bits (see QIF_) are reserved\ * for the mask of entries set via SETQUOTA\ * quotactl. They are set under dq_data_lock\ * and the quota format handling dquot can\ diff --git a/include/linux/quotaops.h b/include/linux/quotaops.h index 11a4becff3a9..4fa4ef0a173a 100644 --- a/include/linux/quotaops.h +++ b/include/linux/quotaops.h @@ -57,7 +57,7 @@ static inline bool dquot_is_busy(struct dquot *dquot) { if (test_bit(DQ_MOD_B, &dquot->dq_flags)) return true; - if (atomic_read(&dquot->dq_count) > 1) + if (atomic_read(&dquot->dq_count) > 0) return true; return false; } diff --git a/include/linux/resume_user_mode.h b/include/linux/resume_user_mode.h index 285189454449..f8f3e958e9cf 100644 --- a/include/linux/resume_user_mode.h +++ b/include/linux/resume_user_mode.h @@ -55,7 +55,7 @@ static inline void resume_user_mode_work(struct pt_regs *regs) } #endif - mem_cgroup_handle_over_high(); + mem_cgroup_handle_over_high(GFP_KERNEL); blkcg_maybe_throttle_current(); rseq_handle_notify_resume(NULL, regs); diff --git a/include/linux/seqlock.h b/include/linux/seqlock.h index 987a59d977c5..e9bd2f65d7f4 100644 --- a/include/linux/seqlock.h +++ b/include/linux/seqlock.h @@ -512,8 +512,8 @@ do { \ static inline void do_write_seqcount_begin_nested(seqcount_t *s, int subclass) { - do_raw_write_seqcount_begin(s); seqcount_acquire(&s->dep_map, subclass, 0, _RET_IP_); + do_raw_write_seqcount_begin(s); } /** diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index 4174c4b82d13..97bfef071255 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -1309,7 +1309,7 @@ struct sk_buff_fclones { * * Returns true if skb is a fast clone, and its clone is not freed. * Some drivers call skb_orphan() in their ndo_start_xmit(), - * so we also check that this didnt happen. + * so we also check that didn't happen. */ static inline bool skb_fclone_busy(const struct sock *sk, const struct sk_buff *skb) @@ -2016,7 +2016,7 @@ static inline struct sk_buff *skb_share_check(struct sk_buff *skb, gfp_t pri) * Copy shared buffers into a new sk_buff. We effectively do COW on * packets to handle cases where we have a local reader and forward * and a couple of other messy ones. The normal one is tcpdumping - * a packet thats being forwarded. + * a packet that's being forwarded. */ /** diff --git a/include/linux/sunrpc/xdr.h b/include/linux/sunrpc/xdr.h index 5b4fb3c791bc..2f8dc47f1eb0 100644 --- a/include/linux/sunrpc/xdr.h +++ b/include/linux/sunrpc/xdr.h @@ -779,7 +779,9 @@ xdr_stream_decode_uint32_array(struct xdr_stream *xdr, if (unlikely(xdr_stream_decode_u32(xdr, &len) < 0)) return -EBADMSG; - p = xdr_inline_decode(xdr, size_mul(len, sizeof(*p))); + if (U32_MAX >= SIZE_MAX / sizeof(*p) && len > SIZE_MAX / sizeof(*p)) + return -EBADMSG; + p = xdr_inline_decode(xdr, len * sizeof(*p)); if (unlikely(!p)) return -EBADMSG; if (array == NULL) diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h index b4536626f8ff..ecde0312dd52 100644 --- a/include/linux/swiotlb.h +++ b/include/linux/swiotlb.h @@ -172,14 +172,23 @@ static inline bool is_swiotlb_buffer(struct device *dev, phys_addr_t paddr) if (!mem) return false; - if (IS_ENABLED(CONFIG_SWIOTLB_DYNAMIC)) { - /* Pairs with smp_wmb() in swiotlb_find_slots() and - * swiotlb_dyn_alloc(), which modify the RCU lists. - */ - smp_rmb(); - return swiotlb_find_pool(dev, paddr); - } +#ifdef CONFIG_SWIOTLB_DYNAMIC + /* + * All SWIOTLB buffer addresses must have been returned by + * swiotlb_tbl_map_single() and passed to a device driver. + * If a SWIOTLB address is checked on another CPU, then it was + * presumably loaded by the device driver from an unspecified private + * data structure. Make sure that this load is ordered before reading + * dev->dma_uses_io_tlb here and mem->pools in swiotlb_find_pool(). + * + * This barrier pairs with smp_mb() in swiotlb_find_slots(). + */ + smp_rmb(); + return READ_ONCE(dev->dma_uses_io_tlb) && + swiotlb_find_pool(dev, paddr); +#else return paddr >= mem->defpool.start && paddr < mem->defpool.end; +#endif } static inline bool is_swiotlb_force_bounce(struct device *dev) diff --git a/include/linux/thermal.h b/include/linux/thermal.h index c99440aac1a1..a5ae4af955ff 100644 --- a/include/linux/thermal.h +++ b/include/linux/thermal.h @@ -80,8 +80,8 @@ struct thermal_zone_device_ops { int (*set_trip_hyst) (struct thermal_zone_device *, int, int); int (*get_crit_temp) (struct thermal_zone_device *, int *); int (*set_emul_temp) (struct thermal_zone_device *, int); - int (*get_trend) (struct thermal_zone_device *, struct thermal_trip *, - enum thermal_trend *); + int (*get_trend) (struct thermal_zone_device *, + const struct thermal_trip *, enum thermal_trend *); void (*hot)(struct thermal_zone_device *); void (*critical)(struct thermal_zone_device *); }; diff --git a/include/linux/trace_events.h b/include/linux/trace_events.h index eb5c3add939b..21ae37e49319 100644 --- a/include/linux/trace_events.h +++ b/include/linux/trace_events.h @@ -62,13 +62,13 @@ void trace_event_printf(struct trace_iterator *iter, const char *fmt, ...); /* Used to find the offset and length of dynamic fields in trace events */ struct trace_dynamic_info { #ifdef CONFIG_CPU_BIG_ENDIAN - u16 offset; u16 len; + u16 offset; #else - u16 len; u16 offset; + u16 len; #endif -}; +} __packed; /* * The trace entry - the most basic unit of tracing. This is what @@ -650,7 +650,6 @@ struct trace_event_file { struct trace_event_call *event_call; struct event_filter __rcu *filter; struct eventfs_file *ef; - struct dentry *dir; struct trace_array *tr; struct trace_subsystem_dir *system; struct list_head triggers; diff --git a/include/linux/virtio_net.h b/include/linux/virtio_net.h index 7b4dd69555e4..27cc1d464321 100644 --- a/include/linux/virtio_net.h +++ b/include/linux/virtio_net.h @@ -3,8 +3,8 @@ #define _LINUX_VIRTIO_NET_H #include +#include #include -#include #include static inline bool virtio_net_hdr_match_proto(__be16 protocol, __u8 gso_type) @@ -151,9 +151,22 @@ retry: unsigned int nh_off = p_off; struct skb_shared_info *shinfo = skb_shinfo(skb); - /* UFO may not include transport header in gso_size. */ - if (gso_type & SKB_GSO_UDP) + switch (gso_type & ~SKB_GSO_TCP_ECN) { + case SKB_GSO_UDP: + /* UFO may not include transport header in gso_size. */ nh_off -= thlen; + break; + case SKB_GSO_UDP_L4: + if (!(hdr->flags & VIRTIO_NET_HDR_F_NEEDS_CSUM)) + return -EINVAL; + if (skb->csum_offset != offsetof(struct udphdr, check)) + return -EINVAL; + if (skb->len - p_off > gso_size * UDP_MAX_SEGMENTS) + return -EINVAL; + if (gso_type != SKB_GSO_UDP_L4) + return -EINVAL; + break; + } /* Kernel has a special handling for GSO_BY_FRAGS. */ if (gso_size == GSO_BY_FRAGS) diff --git a/include/net/bluetooth/hci_core.h b/include/net/bluetooth/hci_core.h index e6359f7346f1..c33348ba1657 100644 --- a/include/net/bluetooth/hci_core.h +++ b/include/net/bluetooth/hci_core.h @@ -350,7 +350,7 @@ struct hci_dev { struct list_head list; struct mutex lock; - char name[8]; + const char *name; unsigned long flags; __u16 id; __u8 bus; diff --git a/include/net/bluetooth/hci_mon.h b/include/net/bluetooth/hci_mon.h index 2d5fcda1bcd0..082f89531b88 100644 --- a/include/net/bluetooth/hci_mon.h +++ b/include/net/bluetooth/hci_mon.h @@ -56,7 +56,7 @@ struct hci_mon_new_index { __u8 type; __u8 bus; bdaddr_t bdaddr; - char name[8]; + char name[8] __nonstring; } __packed; #define HCI_MON_NEW_INDEX_SIZE 16 diff --git a/include/net/cfg80211.h b/include/net/cfg80211.h index 3a4b684f89bf..7192346e4a22 100644 --- a/include/net/cfg80211.h +++ b/include/net/cfg80211.h @@ -5941,6 +5941,7 @@ void wiphy_delayed_work_cancel(struct wiphy *wiphy, * @event_lock: (private) lock for event list * @owner_nlportid: (private) owner socket port ID * @nl_owner_dead: (private) owner socket went away + * @cqm_rssi_work: (private) CQM RSSI reporting work * @cqm_config: (private) nl80211 RSSI monitor state * @pmsr_list: (private) peer measurement requests * @pmsr_lock: (private) peer measurements requests/results lock @@ -6013,7 +6014,8 @@ struct wireless_dev { } wext; #endif - struct cfg80211_cqm_config *cqm_config; + struct wiphy_work cqm_rssi_work; + struct cfg80211_cqm_config __rcu *cqm_config; struct list_head pmsr_list; spinlock_t pmsr_lock; @@ -7231,7 +7233,7 @@ struct cfg80211_rx_assoc_resp { int uapsd_queues; const u8 *ap_mld_addr; struct { - const u8 *addr; + u8 addr[ETH_ALEN] __aligned(2); struct cfg80211_bss *bss; u16 status; } links[IEEE80211_MLD_MAX_NUM_LINKS]; diff --git a/include/net/ip_fib.h b/include/net/ip_fib.h index f0c13864180e..15de07d36540 100644 --- a/include/net/ip_fib.h +++ b/include/net/ip_fib.h @@ -154,6 +154,7 @@ struct fib_info { int fib_nhs; bool fib_nh_is_v6; bool nh_updated; + bool pfsrc_removed; struct nexthop *nh; struct rcu_head rcu; struct fib_nh fib_nh[]; diff --git a/include/net/ipv6.h b/include/net/ipv6.h index 0675be0f3fa0..c6932d1a3fa8 100644 --- a/include/net/ipv6.h +++ b/include/net/ipv6.h @@ -784,6 +784,11 @@ static inline bool ipv6_addr_v4mapped(const struct in6_addr *a) cpu_to_be32(0x0000ffff))) == 0UL; } +static inline bool ipv6_addr_v4mapped_any(const struct in6_addr *a) +{ + return ipv6_addr_v4mapped(a) && ipv4_is_zeronet(a->s6_addr32[3]); +} + static inline bool ipv6_addr_v4mapped_loopback(const struct in6_addr *a) { return ipv6_addr_v4mapped(a) && ipv4_is_loopback(a->s6_addr32[3]); @@ -1360,7 +1365,7 @@ static inline int __ip6_sock_set_addr_preferences(struct sock *sk, int val) return 0; } -static inline int ip6_sock_set_addr_preferences(struct sock *sk, bool val) +static inline int ip6_sock_set_addr_preferences(struct sock *sk, int val) { int ret; diff --git a/include/net/macsec.h b/include/net/macsec.h index 75a6f4863c83..ebf9bc54036a 100644 --- a/include/net/macsec.h +++ b/include/net/macsec.h @@ -258,6 +258,7 @@ struct macsec_context { struct macsec_secy *secy; struct macsec_rx_sc *rx_sc; struct { + bool update_pn; unsigned char assoc_num; u8 key[MACSEC_MAX_KEY_LEN]; union { diff --git a/include/net/mana/mana.h b/include/net/mana/mana.h index 9f70b4332238..4d43adf18606 100644 --- a/include/net/mana/mana.h +++ b/include/net/mana/mana.h @@ -103,9 +103,10 @@ struct mana_txq { /* skb data and frags dma mappings */ struct mana_skb_head { - dma_addr_t dma_handle[MAX_SKB_FRAGS + 1]; + /* GSO pkts may have 2 SGEs for the linear part*/ + dma_addr_t dma_handle[MAX_SKB_FRAGS + 2]; - u32 size[MAX_SKB_FRAGS + 1]; + u32 size[MAX_SKB_FRAGS + 2]; }; #define MANA_HEADROOM sizeof(struct mana_skb_head) diff --git a/include/net/neighbour.h b/include/net/neighbour.h index 6da68886fabb..07022bb0d44d 100644 --- a/include/net/neighbour.h +++ b/include/net/neighbour.h @@ -539,7 +539,7 @@ static inline int neigh_output(struct neighbour *n, struct sk_buff *skb, READ_ONCE(hh->hh_len)) return neigh_hh_output(hh, skb); - return n->output(n, skb); + return READ_ONCE(n->output)(n, skb); } static inline struct neighbour * diff --git a/include/net/netfilter/nf_tables.h b/include/net/netfilter/nf_tables.h index dd40c75011d2..7c816359d5a9 100644 --- a/include/net/netfilter/nf_tables.h +++ b/include/net/netfilter/nf_tables.h @@ -1682,7 +1682,7 @@ struct nft_trans_gc { struct net *net; struct nft_set *set; u32 seq; - u8 count; + u16 count; void *priv[NFT_TRANS_GC_BATCHCOUNT]; struct rcu_head rcu; }; @@ -1700,8 +1700,9 @@ void nft_trans_gc_queue_sync_done(struct nft_trans_gc *trans); void nft_trans_gc_elem_add(struct nft_trans_gc *gc, void *priv); -struct nft_trans_gc *nft_trans_gc_catchall(struct nft_trans_gc *gc, - unsigned int gc_seq); +struct nft_trans_gc *nft_trans_gc_catchall_async(struct nft_trans_gc *gc, + unsigned int gc_seq); +struct nft_trans_gc *nft_trans_gc_catchall_sync(struct nft_trans_gc *gc); void nft_setelem_data_deactivate(const struct net *net, const struct nft_set *set, diff --git a/include/net/netns/xfrm.h b/include/net/netns/xfrm.h index bd7c3be4af5d..423b52eca908 100644 --- a/include/net/netns/xfrm.h +++ b/include/net/netns/xfrm.h @@ -50,6 +50,7 @@ struct netns_xfrm { struct list_head policy_all; struct hlist_head *policy_byidx; unsigned int policy_idx_hmask; + unsigned int idx_generator; struct hlist_head policy_inexact[XFRM_POLICY_MAX]; struct xfrm_policy_hash policy_bydst[XFRM_POLICY_MAX]; unsigned int policy_count[XFRM_POLICY_MAX * 2]; diff --git a/include/net/page_pool/helpers.h b/include/net/page_pool/helpers.h index 94231533a369..8e7751464ff5 100644 --- a/include/net/page_pool/helpers.h +++ b/include/net/page_pool/helpers.h @@ -16,13 +16,13 @@ * page_pool_alloc_pages() call. Drivers should use * page_pool_dev_alloc_pages() replacing dev_alloc_pages(). * - * API keeps track of in-flight pages, in order to let API user know + * The API keeps track of in-flight pages, in order to let API users know * when it is safe to free a page_pool object. Thus, API users * must call page_pool_put_page() to free the page, or attach - * the page to a page_pool-aware objects like skbs marked with + * the page to a page_pool-aware object like skbs marked with * skb_mark_for_recycle(). * - * API user must call page_pool_put_page() once on a page, as it + * API users must call page_pool_put_page() once on a page, as it * will either recycle the page, or in case of refcnt > 1, it will * release the DMA mapping and in-flight state accounting. */ diff --git a/include/net/sock.h b/include/net/sock.h index b770261fbdaf..92f7ea62a915 100644 --- a/include/net/sock.h +++ b/include/net/sock.h @@ -336,7 +336,7 @@ struct sk_filter; * @sk_cgrp_data: cgroup data for this cgroup * @sk_memcg: this socket's memory cgroup association * @sk_write_pending: a write to stream socket waits to start - * @sk_wait_pending: number of threads blocked on this socket + * @sk_disconnects: number of disconnect operations performed on this sock * @sk_state_change: callback to indicate change in the state of the sock * @sk_data_ready: callback to indicate there is data to be processed * @sk_write_space: callback to indicate there is bf sending space available @@ -429,7 +429,7 @@ struct sock { unsigned int sk_napi_id; #endif int sk_rcvbuf; - int sk_wait_pending; + int sk_disconnects; struct sk_filter __rcu *sk_filter; union { @@ -1189,8 +1189,7 @@ static inline void sock_rps_reset_rxhash(struct sock *sk) } #define sk_wait_event(__sk, __timeo, __condition, __wait) \ - ({ int __rc; \ - __sk->sk_wait_pending++; \ + ({ int __rc, __dis = __sk->sk_disconnects; \ release_sock(__sk); \ __rc = __condition; \ if (!__rc) { \ @@ -1200,8 +1199,7 @@ static inline void sock_rps_reset_rxhash(struct sock *sk) } \ sched_annotate_sleep(); \ lock_sock(__sk); \ - __sk->sk_wait_pending--; \ - __rc = __condition; \ + __rc = __dis == __sk->sk_disconnects ? __condition : -EPIPE; \ __rc; \ }) diff --git a/include/net/tcp.h b/include/net/tcp.h index 91688d0dadcd..4b03ca7cb8a5 100644 --- a/include/net/tcp.h +++ b/include/net/tcp.h @@ -141,6 +141,9 @@ void tcp_time_wait(struct sock *sk, int state, int timeo); #define TCP_RTO_MAX ((unsigned)(120*HZ)) #define TCP_RTO_MIN ((unsigned)(HZ/5)) #define TCP_TIMEOUT_MIN (2U) /* Min timeout for TCP timers in jiffies */ + +#define TCP_TIMEOUT_MIN_US (2*USEC_PER_MSEC) /* Min TCP timeout in microsecs */ + #define TCP_TIMEOUT_INIT ((unsigned)(1*HZ)) /* RFC6298 2.1 initial RTO value */ #define TCP_TIMEOUT_FALLBACK ((unsigned)(3*HZ)) /* RFC 1122 initial RTO value, now * used as a fallback RTO for the @@ -348,12 +351,14 @@ ssize_t tcp_splice_read(struct socket *sk, loff_t *ppos, struct sk_buff *tcp_stream_alloc_skb(struct sock *sk, gfp_t gfp, bool force_schedule); -static inline void tcp_dec_quickack_mode(struct sock *sk, - const unsigned int pkts) +static inline void tcp_dec_quickack_mode(struct sock *sk) { struct inet_connection_sock *icsk = inet_csk(sk); if (icsk->icsk_ack.quick) { + /* How many ACKs S/ACKing new data have we sent? */ + const unsigned int pkts = inet_csk_ack_scheduled(sk) ? 1 : 0; + if (pkts >= icsk->icsk_ack.quick) { icsk->icsk_ack.quick = 0; /* Leaving quickack mode we deflate ATO. */ diff --git a/include/scsi/scsi.h b/include/scsi/scsi.h index ec093594ba53..4498f845b112 100644 --- a/include/scsi/scsi.h +++ b/include/scsi/scsi.h @@ -157,6 +157,9 @@ enum scsi_disposition { #define SCSI_3 4 /* SPC */ #define SCSI_SPC_2 5 #define SCSI_SPC_3 6 +#define SCSI_SPC_4 7 +#define SCSI_SPC_5 8 +#define SCSI_SPC_6 14 /* * INQ PERIPHERAL QUALIFIERS diff --git a/include/scsi/scsi_device.h b/include/scsi/scsi_device.h index b9230b6add04..fd41fdac0a8e 100644 --- a/include/scsi/scsi_device.h +++ b/include/scsi/scsi_device.h @@ -161,6 +161,10 @@ struct scsi_device { * pass settings from slave_alloc to scsi * core. */ unsigned int eh_timeout; /* Error handling timeout */ + + bool manage_system_start_stop; /* Let HLD (sd) manage system start/stop */ + bool manage_runtime_start_stop; /* Let HLD (sd) manage runtime start/stop */ + unsigned removable:1; unsigned changed:1; /* Data invalid due to media change */ unsigned busy:1; /* Used to prevent races */ @@ -193,7 +197,6 @@ struct scsi_device { unsigned use_192_bytes_for_3f:1; /* ask for 192 bytes from page 0x3f */ unsigned no_start_on_add:1; /* do not issue start on add */ unsigned allow_restart:1; /* issue START_UNIT in error handler */ - unsigned manage_start_stop:1; /* Let HLD (sd) manage start/stop */ unsigned no_start_on_resume:1; /* Do not issue START_STOP_UNIT on resume */ unsigned start_stop_pwr_cond:1; /* Set power cond. in START_STOP_UNIT */ unsigned no_uld_attach:1; /* disable connecting to upper level drivers */ diff --git a/include/scsi/scsi_host.h b/include/scsi/scsi_host.h index 49f768d0ff37..4c2dc8150c6d 100644 --- a/include/scsi/scsi_host.h +++ b/include/scsi/scsi_host.h @@ -764,7 +764,7 @@ scsi_template_proc_dir(const struct scsi_host_template *sht); #define scsi_template_proc_dir(sht) NULL #endif extern void scsi_scan_host(struct Scsi_Host *); -extern void scsi_rescan_device(struct scsi_device *); +extern int scsi_rescan_device(struct scsi_device *sdev); extern void scsi_remove_host(struct Scsi_Host *); extern struct Scsi_Host *scsi_host_get(struct Scsi_Host *); extern int scsi_host_busy(struct Scsi_Host *shost); diff --git a/include/trace/events/neigh.h b/include/trace/events/neigh.h index 5eaa1fa99171..833143d0992e 100644 --- a/include/trace/events/neigh.h +++ b/include/trace/events/neigh.h @@ -39,7 +39,6 @@ TRACE_EVENT(neigh_create, ), TP_fast_assign( - struct in6_addr *pin6; __be32 *p32; __entry->family = tbl->family; @@ -47,7 +46,6 @@ TRACE_EVENT(neigh_create, __entry->entries = atomic_read(&tbl->gc_entries); __entry->created = n != NULL; __entry->gc_exempt = exempt_from_gc; - pin6 = (struct in6_addr *)__entry->primary_key6; p32 = (__be32 *)__entry->primary_key4; if (tbl->family == AF_INET) @@ -57,6 +55,8 @@ TRACE_EVENT(neigh_create, #if IS_ENABLED(CONFIG_IPV6) if (tbl->family == AF_INET6) { + struct in6_addr *pin6; + pin6 = (struct in6_addr *)__entry->primary_key6; *pin6 = *(struct in6_addr *)pkey; } diff --git a/include/trace/events/xen.h b/include/trace/events/xen.h index 44a3f565264d..0577f0cdd231 100644 --- a/include/trace/events/xen.h +++ b/include/trace/events/xen.h @@ -6,26 +6,26 @@ #define _TRACE_XEN_H #include -#include +#include #include struct multicall_entry; /* Multicalls */ DECLARE_EVENT_CLASS(xen_mc__batch, - TP_PROTO(enum paravirt_lazy_mode mode), + TP_PROTO(enum xen_lazy_mode mode), TP_ARGS(mode), TP_STRUCT__entry( - __field(enum paravirt_lazy_mode, mode) + __field(enum xen_lazy_mode, mode) ), TP_fast_assign(__entry->mode = mode), TP_printk("start batch LAZY_%s", - (__entry->mode == PARAVIRT_LAZY_MMU) ? "MMU" : - (__entry->mode == PARAVIRT_LAZY_CPU) ? "CPU" : "NONE") + (__entry->mode == XEN_LAZY_MMU) ? "MMU" : + (__entry->mode == XEN_LAZY_CPU) ? "CPU" : "NONE") ); #define DEFINE_XEN_MC_BATCH(name) \ DEFINE_EVENT(xen_mc__batch, name, \ - TP_PROTO(enum paravirt_lazy_mode mode), \ + TP_PROTO(enum xen_lazy_mode mode), \ TP_ARGS(mode)) DEFINE_XEN_MC_BATCH(xen_mc_batch); diff --git a/include/uapi/drm/nouveau_drm.h b/include/uapi/drm/nouveau_drm.h index 8d7402c13e56..0bade1592f34 100644 --- a/include/uapi/drm/nouveau_drm.h +++ b/include/uapi/drm/nouveau_drm.h @@ -44,6 +44,16 @@ extern "C" { #define NOUVEAU_GETPARAM_PTIMER_TIME 14 #define NOUVEAU_GETPARAM_HAS_BO_USAGE 15 #define NOUVEAU_GETPARAM_HAS_PAGEFLIP 16 + +/* + * NOUVEAU_GETPARAM_EXEC_PUSH_MAX - query max pushes through getparam + * + * Query the maximum amount of IBs that can be pushed through a single + * &drm_nouveau_exec structure and hence a single &DRM_IOCTL_NOUVEAU_EXEC + * ioctl(). + */ +#define NOUVEAU_GETPARAM_EXEC_PUSH_MAX 17 + struct drm_nouveau_getparam { __u64 param; __u64 value; diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 8790b3962e4b..0448700890f7 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -1962,7 +1962,9 @@ union bpf_attr { * performed again, if the helper is used in combination with * direct packet access. * Return - * 0 on success, or a negative error in case of failure. + * 0 on success, or a negative error in case of failure. Positive + * error indicates a potential drop or congestion in the target + * device. The particular positive error codes are not defined. * * u64 bpf_get_current_pid_tgid(void) * Description diff --git a/include/uapi/linux/if_packet.h b/include/uapi/linux/if_packet.h index 4d0ad22f83b5..9efc42382fdb 100644 --- a/include/uapi/linux/if_packet.h +++ b/include/uapi/linux/if_packet.h @@ -18,11 +18,7 @@ struct sockaddr_ll { unsigned short sll_hatype; unsigned char sll_pkttype; unsigned char sll_halen; - union { - unsigned char sll_addr[8]; - /* Actual length is in sll_halen. */ - __DECLARE_FLEX_ARRAY(unsigned char, sll_addr_flex); - }; + unsigned char sll_addr[8]; }; /* Packet types */ diff --git a/include/uapi/linux/stddef.h b/include/uapi/linux/stddef.h index 7c3fc3980881..5c6c4269f7ef 100644 --- a/include/uapi/linux/stddef.h +++ b/include/uapi/linux/stddef.h @@ -29,6 +29,11 @@ struct TAG { MEMBERS } ATTRS NAME; \ } +#ifdef __cplusplus +/* sizeof(struct{}) is 1 in C++, not 0, can't use C version of the macro. */ +#define __DECLARE_FLEX_ARRAY(T, member) \ + T member[0] +#else /** * __DECLARE_FLEX_ARRAY() - Declare a flexible array usable in a union * @@ -49,3 +54,5 @@ #ifndef __counted_by #define __counted_by(m) #endif + +#endif /* _UAPI_LINUX_STDDEF_H */ diff --git a/include/video/mmp_disp.h b/include/video/mmp_disp.h index 77252cb46361..a722dcbf5073 100644 --- a/include/video/mmp_disp.h +++ b/include/video/mmp_disp.h @@ -231,7 +231,7 @@ struct mmp_path { /* layers */ int overlay_num; - struct mmp_overlay overlays[]; + struct mmp_overlay overlays[] __counted_by(overlay_num); }; extern struct mmp_path *mmp_get_path(const char *name); diff --git a/include/video/uvesafb.h b/include/video/uvesafb.h index 8d2a3bfc8dac..47d96e75e8ef 100644 --- a/include/video/uvesafb.h +++ b/include/video/uvesafb.h @@ -109,8 +109,6 @@ struct uvesafb_ktask { u32 ack; }; -static int uvesafb_exec(struct uvesafb_ktask *tsk); - #define UVESAFB_EXACT_RES 1 #define UVESAFB_EXACT_DEPTH 2 diff --git a/include/xen/arm/hypervisor.h b/include/xen/arm/hypervisor.h index 43ef24dd030e..9995695204f5 100644 --- a/include/xen/arm/hypervisor.h +++ b/include/xen/arm/hypervisor.h @@ -7,18 +7,6 @@ extern struct shared_info *HYPERVISOR_shared_info; extern struct start_info *xen_start_info; -/* Lazy mode for batching updates / context switch */ -enum paravirt_lazy_mode { - PARAVIRT_LAZY_NONE, - PARAVIRT_LAZY_MMU, - PARAVIRT_LAZY_CPU, -}; - -static inline enum paravirt_lazy_mode paravirt_get_lazy_mode(void) -{ - return PARAVIRT_LAZY_NONE; -} - #ifdef CONFIG_XEN void __init xen_early_init(void); #else diff --git a/include/xen/events.h b/include/xen/events.h index 95d5e28de324..23932b0673dc 100644 --- a/include/xen/events.h +++ b/include/xen/events.h @@ -105,8 +105,7 @@ int irq_from_virq(unsigned int cpu, unsigned int virq); evtchn_port_t evtchn_from_irq(unsigned irq); int xen_set_callback_via(uint64_t via); -void xen_evtchn_do_upcall(struct pt_regs *regs); -int xen_hvm_evtchn_do_upcall(void); +int xen_evtchn_do_upcall(void); /* Bind a pirq for a physical interrupt to an irq. */ int xen_bind_pirq_gsi_to_irq(unsigned gsi, diff --git a/io_uring/fs.c b/io_uring/fs.c index f6a69a549fd4..08e3b175469c 100644 --- a/io_uring/fs.c +++ b/io_uring/fs.c @@ -243,7 +243,7 @@ int io_linkat_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe) struct io_link *lnk = io_kiocb_to_cmd(req, struct io_link); const char __user *oldf, *newf; - if (sqe->rw_flags || sqe->buf_index || sqe->splice_fd_in) + if (sqe->buf_index || sqe->splice_fd_in) return -EINVAL; if (unlikely(req->flags & REQ_F_FIXED_FILE)) return -EBADF; diff --git a/io_uring/io-wq.c b/io_uring/io-wq.c index 1ecc8c748768..522196dfb0ff 100644 --- a/io_uring/io-wq.c +++ b/io_uring/io-wq.c @@ -1151,9 +1151,6 @@ struct io_wq *io_wq_create(unsigned bounded, struct io_wq_data *data) wq = kzalloc(sizeof(struct io_wq), GFP_KERNEL); if (!wq) return ERR_PTR(-ENOMEM); - ret = cpuhp_state_add_instance_nocalls(io_wq_online, &wq->cpuhp_node); - if (ret) - goto err_wq; refcount_inc(&data->hash->refs); wq->hash = data->hash; @@ -1186,13 +1183,14 @@ struct io_wq *io_wq_create(unsigned bounded, struct io_wq_data *data) wq->task = get_task_struct(data->task); atomic_set(&wq->worker_refs, 1); init_completion(&wq->worker_done); + ret = cpuhp_state_add_instance_nocalls(io_wq_online, &wq->cpuhp_node); + if (ret) + goto err; + return wq; err: io_wq_put_hash(data->hash); - cpuhp_state_remove_instance_nocalls(io_wq_online, &wq->cpuhp_node); - free_cpumask_var(wq->cpu_mask); -err_wq: kfree(wq); return ERR_PTR(ret); } diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c index 783ed0fff71b..8d1bc6cdfe71 100644 --- a/io_uring/io_uring.c +++ b/io_uring/io_uring.c @@ -2674,7 +2674,11 @@ static void io_pages_free(struct page ***pages, int npages) if (!pages) return; + page_array = *pages; + if (!page_array) + return; + for (i = 0; i < npages; i++) unpin_user_page(page_array[i]); kvfree(page_array); @@ -2686,7 +2690,7 @@ static void *__io_uaddr_map(struct page ***pages, unsigned short *npages, { struct page **page_array; unsigned int nr_pages; - int ret; + int ret, i; *npages = 0; @@ -2716,6 +2720,20 @@ err: */ if (page_array[0] != page_array[ret - 1]) goto err; + + /* + * Can't support mapping user allocated ring memory on 32-bit archs + * where it could potentially reside in highmem. Just fail those with + * -EINVAL, just like we did on kernels that didn't support this + * feature. + */ + for (i = 0; i < nr_pages; i++) { + if (PageHighMem(page_array[i])) { + ret = -EINVAL; + goto err; + } + } + *pages = page_array; *npages = nr_pages; return page_to_virt(page_array[0]); @@ -2744,7 +2762,9 @@ static void io_rings_free(struct io_ring_ctx *ctx) ctx->sq_sqes = NULL; } else { io_pages_free(&ctx->ring_pages, ctx->n_ring_pages); + ctx->n_ring_pages = 0; io_pages_free(&ctx->sqe_pages, ctx->n_sqe_pages); + ctx->n_sqe_pages = 0; } } diff --git a/io_uring/io_uring.h b/io_uring/io_uring.h index 547c30582fb8..0bc145614a6e 100644 --- a/io_uring/io_uring.h +++ b/io_uring/io_uring.h @@ -86,20 +86,33 @@ bool __io_alloc_req_refill(struct io_ring_ctx *ctx); bool io_match_task_safe(struct io_kiocb *head, struct task_struct *task, bool cancel_all); -#define io_lockdep_assert_cq_locked(ctx) \ - do { \ - lockdep_assert(in_task()); \ - \ - if (ctx->flags & IORING_SETUP_IOPOLL) { \ - lockdep_assert_held(&ctx->uring_lock); \ - } else if (!ctx->task_complete) { \ - lockdep_assert_held(&ctx->completion_lock); \ - } else if (ctx->submitter_task->flags & PF_EXITING) { \ - lockdep_assert(current_work()); \ - } else { \ - lockdep_assert(current == ctx->submitter_task); \ - } \ - } while (0) +#if defined(CONFIG_PROVE_LOCKING) +static inline void io_lockdep_assert_cq_locked(struct io_ring_ctx *ctx) +{ + lockdep_assert(in_task()); + + if (ctx->flags & IORING_SETUP_IOPOLL) { + lockdep_assert_held(&ctx->uring_lock); + } else if (!ctx->task_complete) { + lockdep_assert_held(&ctx->completion_lock); + } else if (ctx->submitter_task) { + /* + * ->submitter_task may be NULL and we can still post a CQE, + * if the ring has been setup with IORING_SETUP_R_DISABLED. + * Not from an SQE, as those cannot be submitted, but via + * updating tagged resources. + */ + if (ctx->submitter_task->flags & PF_EXITING) + lockdep_assert(current_work()); + else + lockdep_assert(current == ctx->submitter_task); + } +} +#else +static inline void io_lockdep_assert_cq_locked(struct io_ring_ctx *ctx) +{ +} +#endif static inline void io_req_task_work_add(struct io_kiocb *req) { diff --git a/io_uring/kbuf.c b/io_uring/kbuf.c index 556f4df25b0f..9123138aa9f4 100644 --- a/io_uring/kbuf.c +++ b/io_uring/kbuf.c @@ -477,7 +477,7 @@ static int io_pin_pbuf_ring(struct io_uring_buf_reg *reg, { struct io_uring_buf_ring *br; struct page **pages; - int nr_pages; + int i, nr_pages; pages = io_pin_pages(reg->ring_addr, flex_array_size(br, bufs, reg->ring_entries), @@ -485,6 +485,17 @@ static int io_pin_pbuf_ring(struct io_uring_buf_reg *reg, if (IS_ERR(pages)) return PTR_ERR(pages); + /* + * Apparently some 32-bit boxes (ARM) will return highmem pages, + * which then need to be mapped. We could support that, but it'd + * complicate the code and slowdown the common cases quite a bit. + * So just error out, returning -EINVAL just like we did on kernels + * that didn't support mapped buffer rings. + */ + for (i = 0; i < nr_pages; i++) + if (PageHighMem(pages[i])) + goto error_unpin; + br = page_address(pages[0]); #ifdef SHM_COLOUR /* @@ -496,13 +507,8 @@ static int io_pin_pbuf_ring(struct io_uring_buf_reg *reg, * should use IOU_PBUF_RING_MMAP instead, and liburing will handle * this transparently. */ - if ((reg->ring_addr | (unsigned long) br) & (SHM_COLOUR - 1)) { - int i; - - for (i = 0; i < nr_pages; i++) - unpin_user_page(pages[i]); - return -EINVAL; - } + if ((reg->ring_addr | (unsigned long) br) & (SHM_COLOUR - 1)) + goto error_unpin; #endif bl->buf_pages = pages; bl->buf_nr_pages = nr_pages; @@ -510,6 +516,11 @@ static int io_pin_pbuf_ring(struct io_uring_buf_reg *reg, bl->is_mapped = 1; bl->is_mmap = 0; return 0; +error_unpin: + for (i = 0; i < nr_pages; i++) + unpin_user_page(pages[i]); + kvfree(pages); + return -EINVAL; } static int io_alloc_pbuf_ring(struct io_uring_buf_reg *reg, diff --git a/io_uring/net.c b/io_uring/net.c index 3d07bf79c1e0..7a8e298af81b 100644 --- a/io_uring/net.c +++ b/io_uring/net.c @@ -183,6 +183,10 @@ static int io_setup_async_msg(struct io_kiocb *req, memcpy(async_msg, kmsg, sizeof(*kmsg)); if (async_msg->msg.msg_name) async_msg->msg.msg_name = &async_msg->addr; + + if ((req->flags & REQ_F_BUFFER_SELECT) && !async_msg->msg.msg_iter.nr_segs) + return -EAGAIN; + /* if were using fast_iov, set it to the new one */ if (iter_is_iovec(&kmsg->msg.msg_iter) && !kmsg->free_iov) { size_t fast_idx = iter_iov(&kmsg->msg.msg_iter) - kmsg->fast_iov; @@ -542,6 +546,7 @@ static int io_recvmsg_copy_hdr(struct io_kiocb *req, struct io_async_msghdr *iomsg) { iomsg->msg.msg_name = &iomsg->addr; + iomsg->msg.msg_iter.nr_segs = 0; #ifdef CONFIG_COMPAT if (req->ctx->compat) diff --git a/kernel/auditsc.c b/kernel/auditsc.c index 21d2fa815e78..6f0d6fb6523f 100644 --- a/kernel/auditsc.c +++ b/kernel/auditsc.c @@ -2212,7 +2212,7 @@ __audit_reusename(const __user char *uptr) if (!n->name) continue; if (n->name->uptr == uptr) { - n->name->refcnt++; + atomic_inc(&n->name->refcnt); return n->name; } } @@ -2241,7 +2241,7 @@ void __audit_getname(struct filename *name) n->name = name; n->name_len = AUDIT_NAME_FULL; name->aname = n; - name->refcnt++; + atomic_inc(&name->refcnt); } static inline int audit_copy_fcaps(struct audit_names *name, @@ -2373,7 +2373,7 @@ out_alloc: return; if (name) { n->name = name; - name->refcnt++; + atomic_inc(&name->refcnt); } out: @@ -2500,7 +2500,7 @@ void __audit_inode_child(struct inode *parent, if (found_parent) { found_child->name = found_parent->name; found_child->name_len = AUDIT_NAME_FULL; - found_child->name->refcnt++; + atomic_inc(&found_child->name->refcnt); } } diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index 1095bbe29859..8090d7fb11ef 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -8501,7 +8501,7 @@ bool btf_nested_type_is_trusted(struct bpf_verifier_log *log, tname = btf_name_by_offset(btf, walk_type->name_off); ret = snprintf(safe_tname, sizeof(safe_tname), "%s%s", tname, suffix); - if (ret < 0) + if (ret >= sizeof(safe_tname)) return false; safe_id = btf_find_by_name_kind(btf, safe_tname, BTF_INFO_KIND(walk_type->info)); diff --git a/kernel/bpf/cgroup.c b/kernel/bpf/cgroup.c index 5b2741aa0d9b..03b3d4492980 100644 --- a/kernel/bpf/cgroup.c +++ b/kernel/bpf/cgroup.c @@ -785,7 +785,8 @@ found: * to descendants * @cgrp: The cgroup which descendants to traverse * @link: A link for which to replace BPF program - * @type: Type of attach operation + * @new_prog: &struct bpf_prog for the target BPF program with its refcnt + * incremented * * Must be called with cgroup_mutex held. */ @@ -1334,7 +1335,7 @@ int cgroup_bpf_prog_query(const union bpf_attr *attr, * __cgroup_bpf_run_filter_skb() - Run a program for packet filtering * @sk: The socket sending or receiving traffic * @skb: The skb that is being sent or received - * @type: The type of program to be executed + * @atype: The type of program to be executed * * If no socket is passed, or the socket is not of type INET or INET6, * this function does nothing and returns 0. @@ -1424,7 +1425,7 @@ EXPORT_SYMBOL(__cgroup_bpf_run_filter_skb); /** * __cgroup_bpf_run_filter_sk() - Run a program on a sock * @sk: sock structure to manipulate - * @type: The type of program to be executed + * @atype: The type of program to be executed * * socket is passed is expected to be of type INET or INET6. * @@ -1449,7 +1450,7 @@ EXPORT_SYMBOL(__cgroup_bpf_run_filter_sk); * provided by user sockaddr * @sk: sock struct that will use sockaddr * @uaddr: sockaddr struct provided by user - * @type: The type of program to be executed + * @atype: The type of program to be executed * @t_ctx: Pointer to attach type specific context * @flags: Pointer to u32 which contains higher bits of BPF program * return value (OR'ed together). @@ -1496,7 +1497,7 @@ EXPORT_SYMBOL(__cgroup_bpf_run_filter_sock_addr); * @sock_ops: bpf_sock_ops_kern struct to pass to program. Contains * sk with connection information (IP addresses, etc.) May not contain * cgroup info if it is a req sock. - * @type: The type of program to be executed + * @atype: The type of program to be executed * * socket passed is expected to be of type INET or INET6. * @@ -1670,7 +1671,7 @@ const struct bpf_verifier_ops cg_dev_verifier_ops = { * @ppos: value-result argument: value is position at which read from or write * to sysctl is happening, result is new position if program overrode it, * initial value otherwise - * @type: type of program to be executed + * @atype: type of program to be executed * * Program is run when sysctl is being accessed, either read or written, and * can allow or deny such access. diff --git a/kernel/bpf/memalloc.c b/kernel/bpf/memalloc.c index 9c49ae53deaf..d93ddac283d4 100644 --- a/kernel/bpf/memalloc.c +++ b/kernel/bpf/memalloc.c @@ -459,8 +459,7 @@ static void notrace irq_work_raise(struct bpf_mem_cache *c) * Typical case will be between 11K and 116K closer to 11K. * bpf progs can and should share bpf_mem_cache when possible. */ - -static void prefill_mem_cache(struct bpf_mem_cache *c, int cpu) +static void init_refill_work(struct bpf_mem_cache *c) { init_irq_work(&c->refill_work, bpf_mem_refill); if (c->unit_size <= 256) { @@ -476,7 +475,10 @@ static void prefill_mem_cache(struct bpf_mem_cache *c, int cpu) c->high_watermark = max(96 * 256 / c->unit_size, 3); } c->batch = max((c->high_watermark - c->low_watermark) / 4 * 3, 1); +} +static void prefill_mem_cache(struct bpf_mem_cache *c, int cpu) +{ /* To avoid consuming memory assume that 1st run of bpf * prog won't be doing more than 4 map_update_elem from * irq disabled region @@ -484,6 +486,31 @@ static void prefill_mem_cache(struct bpf_mem_cache *c, int cpu) alloc_bulk(c, c->unit_size <= 256 ? 4 : 1, cpu_to_node(cpu), false); } +static int check_obj_size(struct bpf_mem_cache *c, unsigned int idx) +{ + struct llist_node *first; + unsigned int obj_size; + + /* For per-cpu allocator, the size of free objects in free list doesn't + * match with unit_size and now there is no way to get the size of + * per-cpu pointer saved in free object, so just skip the checking. + */ + if (c->percpu_size) + return 0; + + first = c->free_llist.first; + if (!first) + return 0; + + obj_size = ksize(first); + if (obj_size != c->unit_size) { + WARN_ONCE(1, "bpf_mem_cache[%u]: unexpected object size %u, expect %u\n", + idx, obj_size, c->unit_size); + return -EINVAL; + } + return 0; +} + /* When size != 0 bpf_mem_cache for each cpu. * This is typical bpf hash map use case when all elements have equal size. * @@ -494,10 +521,10 @@ static void prefill_mem_cache(struct bpf_mem_cache *c, int cpu) int bpf_mem_alloc_init(struct bpf_mem_alloc *ma, int size, bool percpu) { static u16 sizes[NUM_CACHES] = {96, 192, 16, 32, 64, 128, 256, 512, 1024, 2048, 4096}; + int cpu, i, err, unit_size, percpu_size = 0; struct bpf_mem_caches *cc, __percpu *pcc; struct bpf_mem_cache *c, __percpu *pc; struct obj_cgroup *objcg = NULL; - int cpu, i, unit_size, percpu_size = 0; if (size) { pc = __alloc_percpu_gfp(sizeof(*pc), 8, GFP_KERNEL); @@ -521,6 +548,7 @@ int bpf_mem_alloc_init(struct bpf_mem_alloc *ma, int size, bool percpu) c->objcg = objcg; c->percpu_size = percpu_size; c->tgt = c; + init_refill_work(c); prefill_mem_cache(c, cpu); } ma->cache = pc; @@ -534,6 +562,7 @@ int bpf_mem_alloc_init(struct bpf_mem_alloc *ma, int size, bool percpu) pcc = __alloc_percpu_gfp(sizeof(*cc), 8, GFP_KERNEL); if (!pcc) return -ENOMEM; + err = 0; #ifdef CONFIG_MEMCG_KMEM objcg = get_obj_cgroup_from_current(); #endif @@ -544,11 +573,30 @@ int bpf_mem_alloc_init(struct bpf_mem_alloc *ma, int size, bool percpu) c->unit_size = sizes[i]; c->objcg = objcg; c->tgt = c; + + init_refill_work(c); + /* Another bpf_mem_cache will be used when allocating + * c->unit_size in bpf_mem_alloc(), so doesn't prefill + * for the bpf_mem_cache because these free objects will + * never be used. + */ + if (i != bpf_mem_cache_idx(c->unit_size)) + continue; prefill_mem_cache(c, cpu); + err = check_obj_size(c, i); + if (err) + goto out; } } + +out: ma->caches = pcc; - return 0; + /* refill_work is either zeroed or initialized, so it is safe to + * call irq_work_sync(). + */ + if (err) + bpf_mem_alloc_destroy(ma); + return err; } static void drain_mem_cache(struct bpf_mem_cache *c) @@ -916,3 +964,35 @@ void notrace *bpf_mem_cache_alloc_flags(struct bpf_mem_alloc *ma, gfp_t flags) return !ret ? NULL : ret + LLIST_NODE_SZ; } + +static __init int bpf_mem_cache_adjust_size(void) +{ + unsigned int size; + + /* Adjusting the indexes in size_index() according to the object_size + * of underlying slab cache, so bpf_mem_alloc() will select a + * bpf_mem_cache with unit_size equal to the object_size of + * the underlying slab cache. + * + * The maximal value of KMALLOC_MIN_SIZE and __kmalloc_minalign() is + * 256-bytes, so only do adjustment for [8-bytes, 192-bytes]. + */ + for (size = 192; size >= 8; size -= 8) { + unsigned int kmalloc_size, index; + + kmalloc_size = kmalloc_size_roundup(size); + if (kmalloc_size == size) + continue; + + if (kmalloc_size <= 192) + index = size_index[(kmalloc_size - 1) / 8]; + else + index = fls(kmalloc_size - 1) - 1; + /* Only overwrite if necessary */ + if (size_index[(size - 1) / 8] != index) + size_index[(size - 1) / 8] = index; + } + + return 0; +} +subsys_initcall(bpf_mem_cache_adjust_size); diff --git a/kernel/bpf/mprog.c b/kernel/bpf/mprog.c index 32d2c4829eb8..1394168062e8 100644 --- a/kernel/bpf/mprog.c +++ b/kernel/bpf/mprog.c @@ -253,6 +253,9 @@ int bpf_mprog_attach(struct bpf_mprog_entry *entry, goto out; } idx = tidx; + } else if (bpf_mprog_total(entry) == bpf_mprog_max()) { + ret = -ERANGE; + goto out; } if (flags & BPF_F_BEFORE) { tidx = bpf_mprog_pos_before(entry, &rtuple); @@ -398,14 +401,16 @@ int bpf_mprog_query(const union bpf_attr *attr, union bpf_attr __user *uattr, struct bpf_mprog_cp *cp; struct bpf_prog *prog; const u32 flags = 0; + u32 id, count = 0; + u64 revision = 1; int i, ret = 0; - u32 id, count; - u64 revision; if (attr->query.query_flags || attr->query.attach_flags) return -EINVAL; - revision = bpf_mprog_revision(entry); - count = bpf_mprog_total(entry); + if (entry) { + revision = bpf_mprog_revision(entry); + count = bpf_mprog_total(entry); + } if (copy_to_user(&uattr->query.attach_flags, &flags, sizeof(flags))) return -EFAULT; if (copy_to_user(&uattr->query.revision, &revision, sizeof(revision))) diff --git a/kernel/bpf/offload.c b/kernel/bpf/offload.c index 3e4f2ec1af06..87d6693d8233 100644 --- a/kernel/bpf/offload.c +++ b/kernel/bpf/offload.c @@ -199,12 +199,14 @@ static int __bpf_prog_dev_bound_init(struct bpf_prog *prog, struct net_device *n offload->netdev = netdev; ondev = bpf_offload_find_netdev(offload->netdev); + /* When program is offloaded require presence of "true" + * bpf_offload_netdev, avoid the one created for !ondev case below. + */ + if (bpf_prog_is_offloaded(prog->aux) && (!ondev || !ondev->offdev)) { + err = -EINVAL; + goto err_free; + } if (!ondev) { - if (bpf_prog_is_offloaded(prog->aux)) { - err = -EINVAL; - goto err_free; - } - /* When only binding to the device, explicitly * create an entry in the hashtable. */ diff --git a/kernel/bpf/queue_stack_maps.c b/kernel/bpf/queue_stack_maps.c index 8d2ddcb7566b..d869f51ea93a 100644 --- a/kernel/bpf/queue_stack_maps.c +++ b/kernel/bpf/queue_stack_maps.c @@ -98,7 +98,12 @@ static long __queue_map_get(struct bpf_map *map, void *value, bool delete) int err = 0; void *ptr; - raw_spin_lock_irqsave(&qs->lock, flags); + if (in_nmi()) { + if (!raw_spin_trylock_irqsave(&qs->lock, flags)) + return -EBUSY; + } else { + raw_spin_lock_irqsave(&qs->lock, flags); + } if (queue_stack_map_is_empty(qs)) { memset(value, 0, qs->map.value_size); @@ -128,7 +133,12 @@ static long __stack_map_get(struct bpf_map *map, void *value, bool delete) void *ptr; u32 index; - raw_spin_lock_irqsave(&qs->lock, flags); + if (in_nmi()) { + if (!raw_spin_trylock_irqsave(&qs->lock, flags)) + return -EBUSY; + } else { + raw_spin_lock_irqsave(&qs->lock, flags); + } if (queue_stack_map_is_empty(qs)) { memset(value, 0, qs->map.value_size); @@ -193,7 +203,12 @@ static long queue_stack_map_push_elem(struct bpf_map *map, void *value, if (flags & BPF_NOEXIST || flags > BPF_EXIST) return -EINVAL; - raw_spin_lock_irqsave(&qs->lock, irq_flags); + if (in_nmi()) { + if (!raw_spin_trylock_irqsave(&qs->lock, irq_flags)) + return -EBUSY; + } else { + raw_spin_lock_irqsave(&qs->lock, irq_flags); + } if (queue_stack_map_is_full(qs)) { if (!replace) { diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index eb01c31ed591..d77b2f8b9364 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -3796,7 +3796,6 @@ static int bpf_prog_attach(const union bpf_attr *attr) { enum bpf_prog_type ptype; struct bpf_prog *prog; - u32 mask; int ret; if (CHECK_ATTR(BPF_PROG_ATTACH)) @@ -3805,10 +3804,16 @@ static int bpf_prog_attach(const union bpf_attr *attr) ptype = attach_type_to_prog_type(attr->attach_type); if (ptype == BPF_PROG_TYPE_UNSPEC) return -EINVAL; - mask = bpf_mprog_supported(ptype) ? - BPF_F_ATTACH_MASK_MPROG : BPF_F_ATTACH_MASK_BASE; - if (attr->attach_flags & ~mask) - return -EINVAL; + if (bpf_mprog_supported(ptype)) { + if (attr->attach_flags & ~BPF_F_ATTACH_MASK_MPROG) + return -EINVAL; + } else { + if (attr->attach_flags & ~BPF_F_ATTACH_MASK_BASE) + return -EINVAL; + if (attr->relative_fd || + attr->expected_revision) + return -EINVAL; + } prog = bpf_prog_get_type(attr->attach_bpf_fd, ptype); if (IS_ERR(prog)) @@ -3878,6 +3883,10 @@ static int bpf_prog_detach(const union bpf_attr *attr) if (IS_ERR(prog)) return PTR_ERR(prog); } + } else if (attr->attach_flags || + attr->relative_fd || + attr->expected_revision) { + return -EINVAL; } switch (ptype) { @@ -3913,7 +3922,7 @@ static int bpf_prog_detach(const union bpf_attr *attr) return ret; } -#define BPF_PROG_QUERY_LAST_FIELD query.link_attach_flags +#define BPF_PROG_QUERY_LAST_FIELD query.revision static int bpf_prog_query(const union bpf_attr *attr, union bpf_attr __user *uattr) diff --git a/kernel/bpf/tcx.c b/kernel/bpf/tcx.c index 13f0b5dc8262..1338a13a8b64 100644 --- a/kernel/bpf/tcx.c +++ b/kernel/bpf/tcx.c @@ -123,7 +123,6 @@ int tcx_prog_query(const union bpf_attr *attr, union bpf_attr __user *uattr) { bool ingress = attr->query.attach_type == BPF_TCX_INGRESS; struct net *net = current->nsproxy->net_ns; - struct bpf_mprog_entry *entry; struct net_device *dev; int ret; @@ -133,12 +132,7 @@ int tcx_prog_query(const union bpf_attr *attr, union bpf_attr __user *uattr) ret = -ENODEV; goto out; } - entry = tcx_entry_fetch(dev, ingress); - if (!entry) { - ret = -ENOENT; - goto out; - } - ret = bpf_mprog_query(attr, uattr, entry); + ret = bpf_mprog_query(attr, uattr, tcx_entry_fetch(dev, ingress)); out: rtnl_unlock(); return ret; diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index bb78212fa5b2..873ade146f3d 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -4047,11 +4047,9 @@ static int __mark_chain_precision(struct bpf_verifier_env *env, int regno) bitmap_from_u64(mask, bt_reg_mask(bt)); for_each_set_bit(i, mask, 32) { reg = &st->frame[0]->regs[i]; - if (reg->type != SCALAR_VALUE) { - bt_clear_reg(bt, i); - continue; - } - reg->precise = true; + bt_clear_reg(bt, i); + if (reg->type == SCALAR_VALUE) + reg->precise = true; } return 0; } @@ -14481,7 +14479,7 @@ static int check_return_code(struct bpf_verifier_env *env) struct tnum enforce_attach_type_range = tnum_unknown; const struct bpf_prog *prog = env->prog; struct bpf_reg_state *reg; - struct tnum range = tnum_range(0, 1); + struct tnum range = tnum_range(0, 1), const_0 = tnum_const(0); enum bpf_prog_type prog_type = resolve_prog_type(env->prog); int err; struct bpf_func_state *frame = env->cur_state->frame[0]; @@ -14529,8 +14527,8 @@ static int check_return_code(struct bpf_verifier_env *env) return -EINVAL; } - if (!tnum_in(tnum_const(0), reg->var_off)) { - verbose_invalid_scalar(env, reg, &range, "async callback", "R0"); + if (!tnum_in(const_0, reg->var_off)) { + verbose_invalid_scalar(env, reg, &const_0, "async callback", "R0"); return -EINVAL; } return 0; diff --git a/kernel/cgroup/cgroup-v1.c b/kernel/cgroup/cgroup-v1.c index c487ffef6652..76db6c67e39a 100644 --- a/kernel/cgroup/cgroup-v1.c +++ b/kernel/cgroup/cgroup-v1.c @@ -360,10 +360,9 @@ static int pidlist_array_load(struct cgroup *cgrp, enum cgroup_filetype type, } css_task_iter_end(&it); length = n; - /* now sort & (if procs) strip out duplicates */ + /* now sort & strip out duplicates (tgids or recycled thread PIDs) */ sort(array, length, sizeof(pid_t), cmppid, NULL); - if (type == CGROUP_FILE_PROCS) - length = pidlist_uniq(array, length); + length = pidlist_uniq(array, length); l = cgroup_pidlist_find_create(cgrp, type); if (!l) { diff --git a/kernel/crash_core.c b/kernel/crash_core.c index 03a7932cde0a..2f675ef045d4 100644 --- a/kernel/crash_core.c +++ b/kernel/crash_core.c @@ -739,6 +739,17 @@ subsys_initcall(crash_notes_memory_init); #undef pr_fmt #define pr_fmt(fmt) "crash hp: " fmt +/* + * Different than kexec/kdump loading/unloading/jumping/shrinking which + * usually rarely happen, there will be many crash hotplug events notified + * during one short period, e.g one memory board is hot added and memory + * regions are online. So mutex lock __crash_hotplug_lock is used to + * serialize the crash hotplug handling specifically. + */ +DEFINE_MUTEX(__crash_hotplug_lock); +#define crash_hotplug_lock() mutex_lock(&__crash_hotplug_lock) +#define crash_hotplug_unlock() mutex_unlock(&__crash_hotplug_lock) + /* * This routine utilized when the crash_hotplug sysfs node is read. * It reflects the kernel's ability/permission to update the crash @@ -748,9 +759,11 @@ int crash_check_update_elfcorehdr(void) { int rc = 0; + crash_hotplug_lock(); /* Obtain lock while reading crash information */ if (!kexec_trylock()) { pr_info("kexec_trylock() failed, elfcorehdr may be inaccurate\n"); + crash_hotplug_unlock(); return 0; } if (kexec_crash_image) { @@ -761,6 +774,7 @@ int crash_check_update_elfcorehdr(void) } /* Release lock now that update complete */ kexec_unlock(); + crash_hotplug_unlock(); return rc; } @@ -783,9 +797,11 @@ static void crash_handle_hotplug_event(unsigned int hp_action, unsigned int cpu) { struct kimage *image; + crash_hotplug_lock(); /* Obtain lock while changing crash information */ if (!kexec_trylock()) { pr_info("kexec_trylock() failed, elfcorehdr may be inaccurate\n"); + crash_hotplug_unlock(); return; } @@ -852,6 +868,7 @@ static void crash_handle_hotplug_event(unsigned int hp_action, unsigned int cpu) out: /* Release lock now that update complete */ kexec_unlock(); + crash_hotplug_unlock(); } static int crash_memhp_notifier(struct notifier_block *nb, unsigned long val, void *v) diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c index 394494a6b1f3..01637677736f 100644 --- a/kernel/dma/swiotlb.c +++ b/kernel/dma/swiotlb.c @@ -399,14 +399,13 @@ void __init swiotlb_init_remap(bool addressing_limit, unsigned int flags, } mem->areas = memblock_alloc(array_size(sizeof(struct io_tlb_area), - default_nareas), SMP_CACHE_BYTES); + nareas), SMP_CACHE_BYTES); if (!mem->areas) { pr_warn("%s: Failed to allocate mem->areas.\n", __func__); return; } - swiotlb_init_io_tlb_pool(mem, __pa(tlb), nslabs, false, - default_nareas); + swiotlb_init_io_tlb_pool(mem, __pa(tlb), nslabs, false, nareas); add_mem_pool(&io_tlb_default_mem, mem); if (flags & SWIOTLB_VERBOSE) @@ -729,9 +728,6 @@ static void swiotlb_dyn_alloc(struct work_struct *work) } add_mem_pool(mem, pool); - - /* Pairs with smp_rmb() in is_swiotlb_buffer(). */ - smp_wmb(); } /** @@ -1152,9 +1148,26 @@ static int swiotlb_find_slots(struct device *dev, phys_addr_t orig_addr, spin_unlock_irqrestore(&dev->dma_io_tlb_lock, flags); found: - dev->dma_uses_io_tlb = true; - /* Pairs with smp_rmb() in is_swiotlb_buffer() */ - smp_wmb(); + WRITE_ONCE(dev->dma_uses_io_tlb, true); + + /* + * The general barrier orders reads and writes against a presumed store + * of the SWIOTLB buffer address by a device driver (to a driver private + * data structure). It serves two purposes. + * + * First, the store to dev->dma_uses_io_tlb must be ordered before the + * presumed store. This guarantees that the returned buffer address + * cannot be passed to another CPU before updating dev->dma_uses_io_tlb. + * + * Second, the load from mem->pools must be ordered before the same + * presumed store. This guarantees that the returned buffer address + * cannot be observed by another CPU before an update of the RCU list + * that was made by swiotlb_dyn_alloc() on a third CPU (cf. multicopy + * atomicity). + * + * See also the comment in is_swiotlb_buffer(). + */ + smp_mb(); *retpool = pool; return index; diff --git a/kernel/events/core.c b/kernel/events/core.c index 4c72a41f11af..d0663b9324e7 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -1954,6 +1954,7 @@ static void perf_group_attach(struct perf_event *event) list_add_tail(&event->sibling_list, &group_leader->sibling_list); group_leader->nr_siblings++; + group_leader->group_generation++; perf_event__header_size(group_leader); @@ -2144,6 +2145,7 @@ static void perf_group_detach(struct perf_event *event) if (leader != event) { list_del_init(&event->sibling_list); event->group_leader->nr_siblings--; + event->group_leader->group_generation++; goto out; } @@ -5440,7 +5442,7 @@ static int __perf_read_group_add(struct perf_event *leader, u64 read_format, u64 *values) { struct perf_event_context *ctx = leader->ctx; - struct perf_event *sub; + struct perf_event *sub, *parent; unsigned long flags; int n = 1; /* skip @nr */ int ret; @@ -5450,6 +5452,33 @@ static int __perf_read_group_add(struct perf_event *leader, return ret; raw_spin_lock_irqsave(&ctx->lock, flags); + /* + * Verify the grouping between the parent and child (inherited) + * events is still in tact. + * + * Specifically: + * - leader->ctx->lock pins leader->sibling_list + * - parent->child_mutex pins parent->child_list + * - parent->ctx->mutex pins parent->sibling_list + * + * Because parent->ctx != leader->ctx (and child_list nests inside + * ctx->mutex), group destruction is not atomic between children, also + * see perf_event_release_kernel(). Additionally, parent can grow the + * group. + * + * Therefore it is possible to have parent and child groups in a + * different configuration and summing over such a beast makes no sense + * what so ever. + * + * Reject this. + */ + parent = leader->parent; + if (parent && + (parent->group_generation != leader->group_generation || + parent->nr_siblings != leader->nr_siblings)) { + ret = -ECHILD; + goto unlock; + } /* * Since we co-schedule groups, {enabled,running} times of siblings @@ -5483,8 +5512,9 @@ static int __perf_read_group_add(struct perf_event *leader, values[n++] = atomic64_read(&sub->lost_samples); } +unlock: raw_spin_unlock_irqrestore(&ctx->lock, flags); - return 0; + return ret; } static int perf_read_group(struct perf_event *event, @@ -5503,10 +5533,6 @@ static int perf_read_group(struct perf_event *event, values[0] = 1 + leader->nr_siblings; - /* - * By locking the child_mutex of the leader we effectively - * lock the child list of all siblings.. XXX explain how. - */ mutex_lock(&leader->child_mutex); ret = __perf_read_group_add(leader, read_format, values); @@ -13346,6 +13372,7 @@ static int inherit_group(struct perf_event *parent_event, !perf_get_aux_event(child_ctr, leader)) return -EINVAL; } + leader->group_generation = parent_event->group_generation; return 0; } diff --git a/kernel/panic.c b/kernel/panic.c index 07239d4ad81e..ffa037fa777d 100644 --- a/kernel/panic.c +++ b/kernel/panic.c @@ -697,6 +697,7 @@ void warn_slowpath_fmt(const char *file, int line, unsigned taint, if (!fmt) { __warn(file, line, __builtin_return_address(0), taint, NULL, NULL); + warn_rcu_exit(rcu); return; } diff --git a/kernel/pid.c b/kernel/pid.c index fee14a4486a3..6500ef956f2f 100644 --- a/kernel/pid.c +++ b/kernel/pid.c @@ -609,7 +609,7 @@ int pidfd_create(struct pid *pid, unsigned int flags) } /** - * pidfd_open() - Open new pid file descriptor. + * sys_pidfd_open() - Open new pid file descriptor. * * @pid: pid for which to retrieve a pidfd * @flags: flags to pass diff --git a/kernel/power/hibernate.c b/kernel/power/hibernate.c index 2b4a946a6ff5..8d35b9f9aaa3 100644 --- a/kernel/power/hibernate.c +++ b/kernel/power/hibernate.c @@ -786,9 +786,9 @@ int hibernate(void) unlock_device_hotplug(); if (snapshot_test) { pm_pr_dbg("Checking hibernation image\n"); - error = swsusp_check(snapshot_test); + error = swsusp_check(false); if (!error) - error = load_image_and_restore(snapshot_test); + error = load_image_and_restore(false); } thaw_processes(); @@ -945,14 +945,14 @@ static int software_resume(void) pm_pr_dbg("Looking for hibernation image.\n"); mutex_lock(&system_transition_mutex); - error = swsusp_check(false); + error = swsusp_check(true); if (error) goto Unlock; /* The snapshot device should not be opened while we're running */ if (!hibernate_acquire()) { error = -EBUSY; - swsusp_close(false); + swsusp_close(true); goto Unlock; } @@ -973,7 +973,7 @@ static int software_resume(void) goto Close_Finish; } - error = load_image_and_restore(false); + error = load_image_and_restore(true); thaw_processes(); Finish: pm_notifier_call_chain(PM_POST_RESTORE); @@ -987,7 +987,7 @@ static int software_resume(void) pm_pr_dbg("Hibernation image not present or could not be loaded.\n"); return error; Close_Finish: - swsusp_close(false); + swsusp_close(true); goto Finish; } diff --git a/kernel/power/power.h b/kernel/power/power.h index 46eb14dc50c3..a98f95e309a3 100644 --- a/kernel/power/power.h +++ b/kernel/power/power.h @@ -168,11 +168,11 @@ extern int swsusp_swap_in_use(void); #define SF_HW_SIG 8 /* kernel/power/hibernate.c */ -int swsusp_check(bool snapshot_test); +int swsusp_check(bool exclusive); extern void swsusp_free(void); extern int swsusp_read(unsigned int *flags_p); extern int swsusp_write(unsigned int flags); -void swsusp_close(bool snapshot_test); +void swsusp_close(bool exclusive); #ifdef CONFIG_SUSPEND extern int swsusp_unmark(void); #endif diff --git a/kernel/power/snapshot.c b/kernel/power/snapshot.c index 87e9f7e2bdc0..0f12e0a97e43 100644 --- a/kernel/power/snapshot.c +++ b/kernel/power/snapshot.c @@ -2647,7 +2647,7 @@ static int prepare_image(struct memory_bitmap *new_bm, struct memory_bitmap *bm, memory_bm_free(bm, PG_UNSAFE_KEEP); /* Make a copy of zero_bm so it can be created in safe pages */ - error = memory_bm_create(&tmp, GFP_ATOMIC, PG_ANY); + error = memory_bm_create(&tmp, GFP_ATOMIC, PG_SAFE); if (error) goto Free; @@ -2660,7 +2660,7 @@ static int prepare_image(struct memory_bitmap *new_bm, struct memory_bitmap *bm, goto Free; duplicate_memory_bitmap(zero_bm, &tmp); - memory_bm_free(&tmp, PG_UNSAFE_KEEP); + memory_bm_free(&tmp, PG_UNSAFE_CLEAR); /* At this point zero_bm is in safe pages and it can be used for restoring. */ if (nr_highmem > 0) { diff --git a/kernel/power/swap.c b/kernel/power/swap.c index f6ebcd00c410..74edbce2320b 100644 --- a/kernel/power/swap.c +++ b/kernel/power/swap.c @@ -1513,12 +1513,13 @@ end: static void *swsusp_holder; /** - * swsusp_check - Check for swsusp signature in the resume device + * swsusp_check - Check for swsusp signature in the resume device + * @exclusive: Open the resume device exclusively. */ -int swsusp_check(bool snapshot_test) +int swsusp_check(bool exclusive) { - void *holder = snapshot_test ? &swsusp_holder : NULL; + void *holder = exclusive ? &swsusp_holder : NULL; int error; hib_resume_bdev = blkdev_get_by_dev(swsusp_resume_device, BLK_OPEN_READ, @@ -1563,17 +1564,18 @@ put: } /** - * swsusp_close - close swap device. + * swsusp_close - close swap device. + * @exclusive: Close the resume device which is exclusively opened. */ -void swsusp_close(bool snapshot_test) +void swsusp_close(bool exclusive) { if (IS_ERR(hib_resume_bdev)) { pr_debug("Image device not initialised\n"); return; } - blkdev_put(hib_resume_bdev, snapshot_test ? &swsusp_holder : NULL); + blkdev_put(hib_resume_bdev, exclusive ? &swsusp_holder : NULL); } /** diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c index 7e0b4dd02398..0b3af1529778 100644 --- a/kernel/printk/printk.c +++ b/kernel/printk/printk.c @@ -3740,12 +3740,18 @@ static bool __pr_flush(struct console *con, int timeout_ms, bool reset_on_progre seq = prb_next_seq(prb); + /* Flush the consoles so that records up to @seq are printed. */ + console_lock(); + console_unlock(); + for (;;) { diff = 0; /* * Hold the console_lock to guarantee safe access to - * console->seq. + * console->seq. Releasing console_lock flushes more + * records in case @seq is still not printed on all + * usable consoles. */ console_lock(); diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 2299a5cfbfb9..802551e0009b 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -9269,7 +9269,7 @@ void __init init_idle(struct task_struct *idle, int cpu) * PF_KTHREAD should already be set at this point; regardless, make it * look like a proper per-CPU kthread. */ - idle->flags |= PF_IDLE | PF_KTHREAD | PF_NO_SETAFFINITY; + idle->flags |= PF_KTHREAD | PF_NO_SETAFFINITY; kthread_set_per_cpu(idle, cpu); #ifdef CONFIG_SMP diff --git a/kernel/sched/cpufreq_schedutil.c b/kernel/sched/cpufreq_schedutil.c index 4492608b7d7f..458d359f5991 100644 --- a/kernel/sched/cpufreq_schedutil.c +++ b/kernel/sched/cpufreq_schedutil.c @@ -350,7 +350,8 @@ static void sugov_update_single_freq(struct update_util_data *hook, u64 time, * Except when the rq is capped by uclamp_max. */ if (!uclamp_rq_is_capped(cpu_rq(sg_cpu->cpu)) && - sugov_cpu_is_busy(sg_cpu) && next_f < sg_policy->next_freq) { + sugov_cpu_is_busy(sg_cpu) && next_f < sg_policy->next_freq && + !sg_policy->need_freq_update) { next_f = sg_policy->next_freq; /* Restore cached freq as next_freq has changed */ diff --git a/kernel/sched/cpupri.c b/kernel/sched/cpupri.c index a286e726eb4b..42c40cfdf836 100644 --- a/kernel/sched/cpupri.c +++ b/kernel/sched/cpupri.c @@ -101,6 +101,7 @@ static inline int __cpupri_find(struct cpupri *cp, struct task_struct *p, if (lowest_mask) { cpumask_and(lowest_mask, &p->cpus_mask, vec->mask); + cpumask_and(lowest_mask, lowest_mask, cpu_active_mask); /* * We have to ensure that we have at least one bit diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 8dbff6e7ad4f..df348aa55d3c 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -664,6 +664,10 @@ void avg_vruntime_update(struct cfs_rq *cfs_rq, s64 delta) cfs_rq->avg_vruntime -= cfs_rq->avg_load * delta; } +/* + * Specifically: avg_runtime() + 0 must result in entity_eligible() := true + * For this to be so, the result of this function must have a left bias. + */ u64 avg_vruntime(struct cfs_rq *cfs_rq) { struct sched_entity *curr = cfs_rq->curr; @@ -677,8 +681,12 @@ u64 avg_vruntime(struct cfs_rq *cfs_rq) load += weight; } - if (load) + if (load) { + /* sign flips effective floor / ceil */ + if (avg < 0) + avg -= (load - 1); avg = div_s64(avg, load); + } return cfs_rq->min_vruntime + avg; } @@ -864,14 +872,16 @@ struct sched_entity *__pick_first_entity(struct cfs_rq *cfs_rq) * * Which allows an EDF like search on (sub)trees. */ -static struct sched_entity *pick_eevdf(struct cfs_rq *cfs_rq) +static struct sched_entity *__pick_eevdf(struct cfs_rq *cfs_rq) { struct rb_node *node = cfs_rq->tasks_timeline.rb_root.rb_node; struct sched_entity *curr = cfs_rq->curr; struct sched_entity *best = NULL; + struct sched_entity *best_left = NULL; if (curr && (!curr->on_rq || !entity_eligible(cfs_rq, curr))) curr = NULL; + best = curr; /* * Once selected, run a task until it either becomes non-eligible or @@ -892,33 +902,75 @@ static struct sched_entity *pick_eevdf(struct cfs_rq *cfs_rq) } /* - * If this entity has an earlier deadline than the previous - * best, take this one. If it also has the earliest deadline - * of its subtree, we're done. + * Now we heap search eligible trees for the best (min_)deadline */ - if (!best || deadline_gt(deadline, best, se)) { + if (!best || deadline_gt(deadline, best, se)) best = se; - if (best->deadline == best->min_deadline) - break; - } /* - * If the earlest deadline in this subtree is in the fully - * eligible left half of our space, go there. + * Every se in a left branch is eligible, keep track of the + * branch with the best min_deadline */ + if (node->rb_left) { + struct sched_entity *left = __node_2_se(node->rb_left); + + if (!best_left || deadline_gt(min_deadline, best_left, left)) + best_left = left; + + /* + * min_deadline is in the left branch. rb_left and all + * descendants are eligible, so immediately switch to the second + * loop. + */ + if (left->min_deadline == se->min_deadline) + break; + } + + /* min_deadline is at this node, no need to look right */ + if (se->deadline == se->min_deadline) + break; + + /* else min_deadline is in the right branch. */ + node = node->rb_right; + } + + /* + * We ran into an eligible node which is itself the best. + * (Or nr_running == 0 and both are NULL) + */ + if (!best_left || (s64)(best_left->min_deadline - best->deadline) > 0) + return best; + + /* + * Now best_left and all of its children are eligible, and we are just + * looking for deadline == min_deadline + */ + node = &best_left->run_node; + while (node) { + struct sched_entity *se = __node_2_se(node); + + /* min_deadline is the current node */ + if (se->deadline == se->min_deadline) + return se; + + /* min_deadline is in the left branch */ if (node->rb_left && __node_2_se(node->rb_left)->min_deadline == se->min_deadline) { node = node->rb_left; continue; } + /* else min_deadline is in the right branch */ node = node->rb_right; } + return NULL; +} - if (!best || (curr && deadline_gt(deadline, best, curr))) - best = curr; +static struct sched_entity *pick_eevdf(struct cfs_rq *cfs_rq) +{ + struct sched_entity *se = __pick_eevdf(cfs_rq); - if (unlikely(!best)) { + if (!se) { struct sched_entity *left = __pick_first_entity(cfs_rq); if (left) { pr_err("EEVDF scheduling fail, picking leftmost\n"); @@ -926,7 +978,7 @@ static struct sched_entity *pick_eevdf(struct cfs_rq *cfs_rq) } } - return best; + return se; } #ifdef CONFIG_SCHED_DEBUG @@ -3605,6 +3657,8 @@ static void reweight_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, */ deadline = div_s64(deadline * old_weight, weight); se->deadline = se->vruntime + deadline; + if (se != cfs_rq->curr) + min_deadline_cb_propagate(&se->run_node, NULL); } #ifdef CONFIG_SMP @@ -4919,10 +4973,12 @@ static inline void update_misfit_status(struct task_struct *p, struct rq *rq) {} static void place_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int flags) { - u64 vslice = calc_delta_fair(se->slice, se); - u64 vruntime = avg_vruntime(cfs_rq); + u64 vslice, vruntime = avg_vruntime(cfs_rq); s64 lag = 0; + se->slice = sysctl_sched_base_slice; + vslice = calc_delta_fair(se->slice, se); + /* * Due to how V is constructed as the weighted average of entities, * adding tasks with positive lag, or removing tasks with negative lag @@ -6619,6 +6675,7 @@ dequeue_throttle: /* Working cpumask for: load_balance, load_balance_newidle. */ static DEFINE_PER_CPU(cpumask_var_t, load_balance_mask); static DEFINE_PER_CPU(cpumask_var_t, select_rq_mask); +static DEFINE_PER_CPU(cpumask_var_t, should_we_balance_tmpmask); #ifdef CONFIG_NO_HZ_COMMON @@ -9579,7 +9636,7 @@ static inline long sibling_imbalance(struct lb_env *env, imbalance /= ncores_local + ncores_busiest; /* Take advantage of resource in an empty sched group */ - if (imbalance == 0 && local->sum_nr_running == 0 && + if (imbalance <= 1 && local->sum_nr_running == 0 && busiest->sum_nr_running > 1) imbalance = 2; @@ -9767,6 +9824,15 @@ static bool update_sd_pick_busiest(struct lb_env *env, break; case group_smt_balance: + /* + * Check if we have spare CPUs on either SMT group to + * choose has spare or fully busy handling. + */ + if (sgs->idle_cpus != 0 || busiest->idle_cpus != 0) + goto has_spare; + + fallthrough; + case group_fully_busy: /* * Select the fully busy group with highest avg_load. In @@ -9806,6 +9872,7 @@ static bool update_sd_pick_busiest(struct lb_env *env, else return true; } +has_spare: /* * Select not overloaded group with lowest number of idle cpus @@ -10917,6 +10984,7 @@ static int active_load_balance_cpu_stop(void *data); static int should_we_balance(struct lb_env *env) { + struct cpumask *swb_cpus = this_cpu_cpumask_var_ptr(should_we_balance_tmpmask); struct sched_group *sg = env->sd->groups; int cpu, idle_smt = -1; @@ -10940,8 +11008,9 @@ static int should_we_balance(struct lb_env *env) return 1; } + cpumask_copy(swb_cpus, group_balance_mask(sg)); /* Try to find first idle CPU */ - for_each_cpu_and(cpu, group_balance_mask(sg), env->cpus) { + for_each_cpu_and(cpu, swb_cpus, env->cpus) { if (!idle_cpu(cpu)) continue; @@ -10953,6 +11022,14 @@ static int should_we_balance(struct lb_env *env) if (!(env->sd->flags & SD_SHARE_CPUCAPACITY) && !is_core_idle(cpu)) { if (idle_smt == -1) idle_smt = cpu; + /* + * If the core is not idle, and first SMT sibling which is + * idle has been found, then its not needed to check other + * SMT siblings for idleness: + */ +#ifdef CONFIG_SCHED_SMT + cpumask_andnot(swb_cpus, swb_cpus, cpu_smt_mask(cpu)); +#endif continue; } @@ -12918,6 +12995,8 @@ __init void init_sched_fair_class(void) for_each_possible_cpu(i) { zalloc_cpumask_var_node(&per_cpu(load_balance_mask, i), GFP_KERNEL, cpu_to_node(i)); zalloc_cpumask_var_node(&per_cpu(select_rq_mask, i), GFP_KERNEL, cpu_to_node(i)); + zalloc_cpumask_var_node(&per_cpu(should_we_balance_tmpmask, i), + GFP_KERNEL, cpu_to_node(i)); #ifdef CONFIG_CFS_BANDWIDTH INIT_CSD(&cpu_rq(i)->cfsb_csd, __cfsb_csd_unthrottle, cpu_rq(i)); diff --git a/kernel/sched/idle.c b/kernel/sched/idle.c index 342f58a329f5..5007b25c5bc6 100644 --- a/kernel/sched/idle.c +++ b/kernel/sched/idle.c @@ -373,6 +373,7 @@ EXPORT_SYMBOL_GPL(play_idle_precise); void cpu_startup_entry(enum cpuhp_state state) { + current->flags |= PF_IDLE; arch_cpu_idle_prepare(); cpuhp_online_idle(state); while (1) diff --git a/kernel/task_work.c b/kernel/task_work.c index 065e1ef8fc8d..95a7e1b7f1da 100644 --- a/kernel/task_work.c +++ b/kernel/task_work.c @@ -78,6 +78,7 @@ int task_work_add(struct task_struct *task, struct callback_head *work, * task_work_cancel_match - cancel a pending work added by task_work_add() * @task: the task which should execute the work * @match: match function to call + * @data: data to be passed in to match function * * RETURNS: * The found work or NULL if not found. diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index a7264b2c17ad..868008f56fec 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -2853,6 +2853,17 @@ static int get_modules_for_addrs(struct module ***mods, unsigned long *addrs, u3 return arr.mods_cnt; } +static int addrs_check_error_injection_list(unsigned long *addrs, u32 cnt) +{ + u32 i; + + for (i = 0; i < cnt; i++) { + if (!within_error_injection_list(addrs[i])) + return -EINVAL; + } + return 0; +} + int bpf_kprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *prog) { struct bpf_kprobe_multi_link *link = NULL; @@ -2930,6 +2941,11 @@ int bpf_kprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr goto error; } + if (prog->kprobe_override && addrs_check_error_injection_list(addrs, cnt)) { + err = -EINVAL; + goto error; + } + link = kzalloc(sizeof(*link), GFP_KERNEL); if (!link) { err = -ENOMEM; @@ -3207,8 +3223,10 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr rcu_read_lock(); task = get_pid_task(find_vpid(pid), PIDTYPE_PID); rcu_read_unlock(); - if (!task) + if (!task) { + err = -ESRCH; goto error_path_put; + } } err = -ENOMEM; diff --git a/kernel/trace/fprobe.c b/kernel/trace/fprobe.c index 3b21f4063258..881f90f0cbcf 100644 --- a/kernel/trace/fprobe.c +++ b/kernel/trace/fprobe.c @@ -189,7 +189,7 @@ static int fprobe_init_rethook(struct fprobe *fp, int num) { int i, size; - if (num < 0) + if (num <= 0) return -EINVAL; if (!fp->exit_handler) { @@ -202,8 +202,8 @@ static int fprobe_init_rethook(struct fprobe *fp, int num) size = fp->nr_maxactive; else size = num * num_possible_cpus() * 2; - if (size < 0) - return -E2BIG; + if (size <= 0) + return -EINVAL; fp->rethook = rethook_alloc((void *)fp, fprobe_exit_handler); if (!fp->rethook) diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index 78502d4c7214..515cafdb18d9 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -354,6 +354,11 @@ static void rb_init_page(struct buffer_data_page *bpage) local_set(&bpage->commit, 0); } +static __always_inline unsigned int rb_page_commit(struct buffer_page *bpage) +{ + return local_read(&bpage->page->commit); +} + static void free_buffer_page(struct buffer_page *bpage) { free_page((unsigned long)bpage->page); @@ -1132,6 +1137,9 @@ __poll_t ring_buffer_poll_wait(struct trace_buffer *buffer, int cpu, if (full) { poll_wait(filp, &work->full_waiters, poll_table); work->full_waiters_pending = true; + if (!cpu_buffer->shortest_full || + cpu_buffer->shortest_full > full) + cpu_buffer->shortest_full = full; } else { poll_wait(filp, &work->waiters, poll_table); work->waiters_pending = true; @@ -2003,7 +2011,7 @@ rb_remove_pages(struct ring_buffer_per_cpu *cpu_buffer, unsigned long nr_pages) * Increment overrun to account for the lost events. */ local_add(page_entries, &cpu_buffer->overrun); - local_sub(BUF_PAGE_SIZE, &cpu_buffer->entries_bytes); + local_sub(rb_page_commit(to_remove_page), &cpu_buffer->entries_bytes); local_inc(&cpu_buffer->pages_lost); } @@ -2198,6 +2206,8 @@ int ring_buffer_resize(struct trace_buffer *buffer, unsigned long size, err = -ENOMEM; goto out_err; } + + cond_resched(); } cpus_read_lock(); @@ -2365,11 +2375,6 @@ rb_reader_event(struct ring_buffer_per_cpu *cpu_buffer) cpu_buffer->reader_page->read); } -static __always_inline unsigned rb_page_commit(struct buffer_page *bpage) -{ - return local_read(&bpage->page->commit); -} - static struct ring_buffer_event * rb_iter_head_event(struct ring_buffer_iter *iter) { @@ -2388,6 +2393,11 @@ rb_iter_head_event(struct ring_buffer_iter *iter) */ commit = rb_page_commit(iter_head_page); smp_rmb(); + + /* An event needs to be at least 8 bytes in size */ + if (iter->head > commit - 8) + goto reset; + event = __rb_page_index(iter_head_page, iter->head); length = rb_event_length(event); @@ -2510,7 +2520,7 @@ rb_handle_head_page(struct ring_buffer_per_cpu *cpu_buffer, * the counters. */ local_add(entries, &cpu_buffer->overrun); - local_sub(BUF_PAGE_SIZE, &cpu_buffer->entries_bytes); + local_sub(rb_page_commit(next_page), &cpu_buffer->entries_bytes); local_inc(&cpu_buffer->pages_lost); /* @@ -2653,9 +2663,6 @@ rb_reset_tail(struct ring_buffer_per_cpu *cpu_buffer, event = __rb_page_index(tail_page, tail); - /* account for padding bytes */ - local_add(BUF_PAGE_SIZE - tail, &cpu_buffer->entries_bytes); - /* * Save the original length to the meta data. * This will be used by the reader to add lost event @@ -2669,7 +2676,8 @@ rb_reset_tail(struct ring_buffer_per_cpu *cpu_buffer, * write counter enough to allow another writer to slip * in on this page. * We put in a discarded commit instead, to make sure - * that this space is not used again. + * that this space is not used again, and this space will + * not be accounted into 'entries_bytes'. * * If we are less than the minimum size, we don't need to * worry about it. @@ -2694,6 +2702,9 @@ rb_reset_tail(struct ring_buffer_per_cpu *cpu_buffer, /* time delta must be non zero */ event->time_delta = 1; + /* account for padding bytes */ + local_add(BUF_PAGE_SIZE - tail, &cpu_buffer->entries_bytes); + /* Make sure the padding is visible before the tail_page->write update */ smp_wmb(); @@ -4208,7 +4219,7 @@ u64 ring_buffer_oldest_event_ts(struct trace_buffer *buffer, int cpu) EXPORT_SYMBOL_GPL(ring_buffer_oldest_event_ts); /** - * ring_buffer_bytes_cpu - get the number of bytes consumed in a cpu buffer + * ring_buffer_bytes_cpu - get the number of bytes unconsumed in a cpu buffer * @buffer: The ring buffer * @cpu: The per CPU buffer to read from. */ @@ -4716,6 +4727,7 @@ static void rb_advance_reader(struct ring_buffer_per_cpu *cpu_buffer) length = rb_event_length(event); cpu_buffer->reader_page->read += length; + cpu_buffer->read_bytes += length; } static void rb_advance_iter(struct ring_buffer_iter *iter) @@ -5809,7 +5821,7 @@ int ring_buffer_read_page(struct trace_buffer *buffer, } else { /* update the entry counter */ cpu_buffer->read += rb_page_entries(reader); - cpu_buffer->read_bytes += BUF_PAGE_SIZE; + cpu_buffer->read_bytes += rb_page_commit(reader); /* swap the pages */ rb_init_page(bpage); diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c index 2b4ded753367..abaaf516fcae 100644 --- a/kernel/trace/trace.c +++ b/kernel/trace/trace.c @@ -1772,7 +1772,7 @@ static void trace_create_maxlat_file(struct trace_array *tr, init_irq_work(&tr->fsnotify_irqwork, latency_fsnotify_workfn_irq); tr->d_max_latency = trace_create_file("tracing_max_latency", TRACE_MODE_WRITE, - d_tracer, &tr->max_latency, + d_tracer, tr, &tracing_max_lat_fops); } @@ -1805,7 +1805,7 @@ void latency_fsnotify(struct trace_array *tr) #define trace_create_maxlat_file(tr, d_tracer) \ trace_create_file("tracing_max_latency", TRACE_MODE_WRITE, \ - d_tracer, &tr->max_latency, &tracing_max_lat_fops) + d_tracer, tr, &tracing_max_lat_fops) #endif @@ -4973,6 +4973,33 @@ int tracing_open_generic_tr(struct inode *inode, struct file *filp) return 0; } +/* + * The private pointer of the inode is the trace_event_file. + * Update the tr ref count associated to it. + */ +int tracing_open_file_tr(struct inode *inode, struct file *filp) +{ + struct trace_event_file *file = inode->i_private; + int ret; + + ret = tracing_check_open_get_tr(file->tr); + if (ret) + return ret; + + filp->private_data = inode->i_private; + + return 0; +} + +int tracing_release_file_tr(struct inode *inode, struct file *filp) +{ + struct trace_event_file *file = inode->i_private; + + trace_array_put(file->tr); + + return 0; +} + static int tracing_mark_open(struct inode *inode, struct file *filp) { stream_open(inode, filp); @@ -6691,14 +6718,18 @@ static ssize_t tracing_max_lat_read(struct file *filp, char __user *ubuf, size_t cnt, loff_t *ppos) { - return tracing_nsecs_read(filp->private_data, ubuf, cnt, ppos); + struct trace_array *tr = filp->private_data; + + return tracing_nsecs_read(&tr->max_latency, ubuf, cnt, ppos); } static ssize_t tracing_max_lat_write(struct file *filp, const char __user *ubuf, size_t cnt, loff_t *ppos) { - return tracing_nsecs_write(filp->private_data, ubuf, cnt, ppos); + struct trace_array *tr = filp->private_data; + + return tracing_nsecs_write(&tr->max_latency, ubuf, cnt, ppos); } #endif @@ -7752,18 +7783,20 @@ static const struct file_operations tracing_thresh_fops = { #ifdef CONFIG_TRACER_MAX_TRACE static const struct file_operations tracing_max_lat_fops = { - .open = tracing_open_generic, + .open = tracing_open_generic_tr, .read = tracing_max_lat_read, .write = tracing_max_lat_write, .llseek = generic_file_llseek, + .release = tracing_release_generic_tr, }; #endif static const struct file_operations set_tracer_fops = { - .open = tracing_open_generic, + .open = tracing_open_generic_tr, .read = tracing_set_trace_read, .write = tracing_set_trace_write, .llseek = generic_file_llseek, + .release = tracing_release_generic_tr, }; static const struct file_operations tracing_pipe_fops = { @@ -8956,12 +8989,33 @@ trace_options_write(struct file *filp, const char __user *ubuf, size_t cnt, return cnt; } +static int tracing_open_options(struct inode *inode, struct file *filp) +{ + struct trace_option_dentry *topt = inode->i_private; + int ret; + + ret = tracing_check_open_get_tr(topt->tr); + if (ret) + return ret; + + filp->private_data = inode->i_private; + return 0; +} + +static int tracing_release_options(struct inode *inode, struct file *file) +{ + struct trace_option_dentry *topt = file->private_data; + + trace_array_put(topt->tr); + return 0; +} static const struct file_operations trace_options_fops = { - .open = tracing_open_generic, + .open = tracing_open_options, .read = trace_options_read, .write = trace_options_write, .llseek = generic_file_llseek, + .release = tracing_release_options, }; /* @@ -9739,8 +9793,8 @@ init_tracer_tracefs(struct trace_array *tr, struct dentry *d_tracer) tr, &tracing_mark_fops); file = __find_event_file(tr, "ftrace", "print"); - if (file && file->dir) - trace_create_file("trigger", TRACE_MODE_WRITE, file->dir, + if (file && file->ef) + eventfs_add_file("trigger", TRACE_MODE_WRITE, file->ef, file, &event_trigger_fops); tr->trace_marker_file = file; diff --git a/kernel/trace/trace.h b/kernel/trace/trace.h index 5669dd1f90d9..77debe53f07c 100644 --- a/kernel/trace/trace.h +++ b/kernel/trace/trace.h @@ -610,6 +610,8 @@ void tracing_reset_all_online_cpus(void); void tracing_reset_all_online_cpus_unlocked(void); int tracing_open_generic(struct inode *inode, struct file *filp); int tracing_open_generic_tr(struct inode *inode, struct file *filp); +int tracing_open_file_tr(struct inode *inode, struct file *filp); +int tracing_release_file_tr(struct inode *inode, struct file *filp); bool tracing_is_disabled(void); bool tracer_tracing_is_on(struct trace_array *tr); void tracer_tracing_on(struct trace_array *tr); diff --git a/kernel/trace/trace_events.c b/kernel/trace/trace_events.c index ed367d713be0..f49d6ddb6342 100644 --- a/kernel/trace/trace_events.c +++ b/kernel/trace/trace_events.c @@ -992,19 +992,6 @@ static void remove_subsystem(struct trace_subsystem_dir *dir) static void remove_event_file_dir(struct trace_event_file *file) { - struct dentry *dir = file->dir; - struct dentry *child; - - if (dir) { - spin_lock(&dir->d_lock); /* probably unneeded */ - list_for_each_entry(child, &dir->d_subdirs, d_child) { - if (d_really_is_positive(child)) /* probably unneeded */ - d_inode(child)->i_private = NULL; - } - spin_unlock(&dir->d_lock); - - tracefs_remove(dir); - } eventfs_remove(file->ef); list_del(&file->list); remove_subsystem(file->system); @@ -2103,9 +2090,10 @@ static const struct file_operations ftrace_set_event_notrace_pid_fops = { }; static const struct file_operations ftrace_enable_fops = { - .open = tracing_open_generic, + .open = tracing_open_file_tr, .read = event_enable_read, .write = event_enable_write, + .release = tracing_release_file_tr, .llseek = default_llseek, }; @@ -2122,9 +2110,10 @@ static const struct file_operations ftrace_event_id_fops = { }; static const struct file_operations ftrace_event_filter_fops = { - .open = tracing_open_generic, + .open = tracing_open_file_tr, .read = event_filter_read, .write = event_filter_write, + .release = tracing_release_file_tr, .llseek = default_llseek, }; @@ -2297,6 +2286,7 @@ event_subsystem_dir(struct trace_array *tr, const char *name, { struct event_subsystem *system, *iter; struct trace_subsystem_dir *dir; + struct eventfs_file *ef; int res; /* First see if we did not already create this dir */ @@ -2329,13 +2319,14 @@ event_subsystem_dir(struct trace_array *tr, const char *name, } else __get_system(system); - dir->ef = eventfs_add_subsystem_dir(name, parent); - if (IS_ERR(dir->ef)) { + ef = eventfs_add_subsystem_dir(name, parent); + if (IS_ERR(ef)) { pr_warn("Failed to create system directory %s\n", name); __put_system(system); goto out_free; } + dir->ef = ef; dir->tr = tr; dir->ref_count = 1; dir->nr_events = 1; @@ -2415,6 +2406,7 @@ event_create_dir(struct dentry *parent, struct trace_event_file *file) struct trace_event_call *call = file->event_call; struct eventfs_file *ef_subsystem = NULL; struct trace_array *tr = file->tr; + struct eventfs_file *ef; const char *name; int ret; @@ -2431,12 +2423,14 @@ event_create_dir(struct dentry *parent, struct trace_event_file *file) return -ENOMEM; name = trace_event_name(call); - file->ef = eventfs_add_dir(name, ef_subsystem); - if (IS_ERR(file->ef)) { + ef = eventfs_add_dir(name, ef_subsystem); + if (IS_ERR(ef)) { pr_warn("Could not create tracefs '%s' directory\n", name); return -1; } + file->ef = ef; + if (call->class->reg && !(call->flags & TRACE_EVENT_FL_IGNORE_ENABLE)) eventfs_add_file("enable", TRACE_MODE_WRITE, file->ef, file, &ftrace_enable_fops); @@ -2776,6 +2770,7 @@ void trace_event_eval_update(struct trace_eval_map **map, int len) update_event_fields(call, map[i]); } } + cond_resched(); } up_write(&trace_event_sem); } diff --git a/kernel/trace/trace_events_inject.c b/kernel/trace/trace_events_inject.c index abe805d471eb..8650562bdaa9 100644 --- a/kernel/trace/trace_events_inject.c +++ b/kernel/trace/trace_events_inject.c @@ -328,7 +328,8 @@ event_inject_read(struct file *file, char __user *buf, size_t size, } const struct file_operations event_inject_fops = { - .open = tracing_open_generic, + .open = tracing_open_file_tr, .read = event_inject_read, .write = event_inject_write, + .release = tracing_release_file_tr, }; diff --git a/kernel/trace/trace_events_synth.c b/kernel/trace/trace_events_synth.c index 9897d0bfcab7..14cb275a0bab 100644 --- a/kernel/trace/trace_events_synth.c +++ b/kernel/trace/trace_events_synth.c @@ -337,7 +337,7 @@ static void print_synth_event_num_val(struct trace_seq *s, break; default: - trace_seq_printf(s, print_fmt, name, val, space); + trace_seq_printf(s, print_fmt, name, val->as_u64, space); break; } } diff --git a/kernel/trace/trace_events_user.c b/kernel/trace/trace_events_user.c index 6f046650e527..b87f41187c6a 100644 --- a/kernel/trace/trace_events_user.c +++ b/kernel/trace/trace_events_user.c @@ -127,8 +127,13 @@ struct user_event_enabler { /* Bit 7 is for freeing status of enablement */ #define ENABLE_VAL_FREEING_BIT 7 -/* Only duplicate the bit value */ -#define ENABLE_VAL_DUP_MASK ENABLE_VAL_BIT_MASK +/* Bit 8 is for marking 32-bit on 64-bit */ +#define ENABLE_VAL_32_ON_64_BIT 8 + +#define ENABLE_VAL_COMPAT_MASK (1 << ENABLE_VAL_32_ON_64_BIT) + +/* Only duplicate the bit and compat values */ +#define ENABLE_VAL_DUP_MASK (ENABLE_VAL_BIT_MASK | ENABLE_VAL_COMPAT_MASK) #define ENABLE_BITOPS(e) (&(e)->values) @@ -174,6 +179,30 @@ struct user_event_validator { int flags; }; +static inline void align_addr_bit(unsigned long *addr, int *bit, + unsigned long *flags) +{ + if (IS_ALIGNED(*addr, sizeof(long))) { +#ifdef __BIG_ENDIAN + /* 32 bit on BE 64 bit requires a 32 bit offset when aligned. */ + if (test_bit(ENABLE_VAL_32_ON_64_BIT, flags)) + *bit += 32; +#endif + return; + } + + *addr = ALIGN_DOWN(*addr, sizeof(long)); + + /* + * We only support 32 and 64 bit values. The only time we need + * to align is a 32 bit value on a 64 bit kernel, which on LE + * is always 32 bits, and on BE requires no change when unaligned. + */ +#ifdef __LITTLE_ENDIAN + *bit += 32; +#endif +} + typedef void (*user_event_func_t) (struct user_event *user, struct iov_iter *i, void *tpdata, bool *faulted); @@ -482,6 +511,7 @@ static int user_event_enabler_write(struct user_event_mm *mm, unsigned long *ptr; struct page *page; void *kaddr; + int bit = ENABLE_BIT(enabler); int ret; lockdep_assert_held(&event_mutex); @@ -497,6 +527,8 @@ static int user_event_enabler_write(struct user_event_mm *mm, test_bit(ENABLE_VAL_FREEING_BIT, ENABLE_BITOPS(enabler)))) return -EBUSY; + align_addr_bit(&uaddr, &bit, ENABLE_BITOPS(enabler)); + ret = pin_user_pages_remote(mm->mm, uaddr, 1, FOLL_WRITE | FOLL_NOFAULT, &page, NULL); @@ -515,9 +547,9 @@ static int user_event_enabler_write(struct user_event_mm *mm, /* Update bit atomically, user tracers must be atomic as well */ if (enabler->event && enabler->event->status) - set_bit(ENABLE_BIT(enabler), ptr); + set_bit(bit, ptr); else - clear_bit(ENABLE_BIT(enabler), ptr); + clear_bit(bit, ptr); kunmap_local(kaddr); unpin_user_pages_dirty_lock(&page, 1, true); @@ -849,6 +881,12 @@ static struct user_event_enabler enabler->event = user; enabler->addr = uaddr; enabler->values = reg->enable_bit; + +#if BITS_PER_LONG >= 64 + if (reg->enable_size == 4) + set_bit(ENABLE_VAL_32_ON_64_BIT, ENABLE_BITOPS(enabler)); +#endif + retry: /* Prevents state changes from racing with new enablers */ mutex_lock(&event_mutex); @@ -2377,7 +2415,8 @@ static long user_unreg_get(struct user_unreg __user *ureg, } static int user_event_mm_clear_bit(struct user_event_mm *user_mm, - unsigned long uaddr, unsigned char bit) + unsigned long uaddr, unsigned char bit, + unsigned long flags) { struct user_event_enabler enabler; int result; @@ -2385,7 +2424,7 @@ static int user_event_mm_clear_bit(struct user_event_mm *user_mm, memset(&enabler, 0, sizeof(enabler)); enabler.addr = uaddr; - enabler.values = bit; + enabler.values = bit | flags; retry: /* Prevents state changes from racing with new enablers */ mutex_lock(&event_mutex); @@ -2415,6 +2454,7 @@ static long user_events_ioctl_unreg(unsigned long uarg) struct user_event_mm *mm = current->user_event_mm; struct user_event_enabler *enabler, *next; struct user_unreg reg; + unsigned long flags; long ret; ret = user_unreg_get(ureg, ®); @@ -2425,6 +2465,7 @@ static long user_events_ioctl_unreg(unsigned long uarg) if (!mm) return -ENOENT; + flags = 0; ret = -ENOENT; /* @@ -2441,6 +2482,9 @@ static long user_events_ioctl_unreg(unsigned long uarg) ENABLE_BIT(enabler) == reg.disable_bit) { set_bit(ENABLE_VAL_FREEING_BIT, ENABLE_BITOPS(enabler)); + /* We must keep compat flags for the clear */ + flags |= enabler->values & ENABLE_VAL_COMPAT_MASK; + if (!test_bit(ENABLE_VAL_FAULTING_BIT, ENABLE_BITOPS(enabler))) user_event_enabler_destroy(enabler, true); @@ -2454,7 +2498,7 @@ static long user_events_ioctl_unreg(unsigned long uarg) /* Ensure bit is now cleared for user, regardless of event status */ if (!ret) ret = user_event_mm_clear_bit(mm, reg.disable_addr, - reg.disable_bit); + reg.disable_bit, flags); return ret; } diff --git a/kernel/trace/trace_kprobe.c b/kernel/trace/trace_kprobe.c index 3d7a180a8427..a8fef6ab0872 100644 --- a/kernel/trace/trace_kprobe.c +++ b/kernel/trace/trace_kprobe.c @@ -705,6 +705,25 @@ static struct notifier_block trace_kprobe_module_nb = { .priority = 1 /* Invoked after kprobe module callback */ }; +static int count_symbols(void *data, unsigned long unused) +{ + unsigned int *count = data; + + (*count)++; + + return 0; +} + +static unsigned int number_of_same_symbols(char *func_name) +{ + unsigned int count; + + count = 0; + kallsyms_on_each_match_symbol(count_symbols, func_name, &count); + + return count; +} + static int __trace_kprobe_create(int argc, const char *argv[]) { /* @@ -836,6 +855,31 @@ static int __trace_kprobe_create(int argc, const char *argv[]) } } + if (symbol && !strchr(symbol, ':')) { + unsigned int count; + + count = number_of_same_symbols(symbol); + if (count > 1) { + /* + * Users should use ADDR to remove the ambiguity of + * using KSYM only. + */ + trace_probe_log_err(0, NON_UNIQ_SYMBOL); + ret = -EADDRNOTAVAIL; + + goto error; + } else if (count == 0) { + /* + * We can return ENOENT earlier than when register the + * kprobe. + */ + trace_probe_log_err(0, BAD_PROBE_ADDR); + ret = -ENOENT; + + goto error; + } + } + trace_probe_log_set_index(0); if (event) { ret = traceprobe_parse_event_name(&event, &group, gbuf, @@ -1695,6 +1739,7 @@ static int unregister_kprobe_event(struct trace_kprobe *tk) } #ifdef CONFIG_PERF_EVENTS + /* create a trace_kprobe, but don't add it to global lists */ struct trace_event_call * create_local_trace_kprobe(char *func, void *addr, unsigned long offs, @@ -1705,6 +1750,24 @@ create_local_trace_kprobe(char *func, void *addr, unsigned long offs, int ret; char *event; + if (func) { + unsigned int count; + + count = number_of_same_symbols(func); + if (count > 1) + /* + * Users should use addr to remove the ambiguity of + * using func only. + */ + return ERR_PTR(-EADDRNOTAVAIL); + else if (count == 0) + /* + * We can return ENOENT earlier than when register the + * kprobe. + */ + return ERR_PTR(-ENOENT); + } + /* * local trace_kprobes are not added to dyn_event, so they are never * searched in find_trace_kprobe(). Therefore, there is no concern of diff --git a/kernel/trace/trace_probe.h b/kernel/trace/trace_probe.h index 02b432ae7513..850d9ecb6765 100644 --- a/kernel/trace/trace_probe.h +++ b/kernel/trace/trace_probe.h @@ -450,6 +450,7 @@ extern int traceprobe_define_arg_fields(struct trace_event_call *event_call, C(BAD_MAXACT, "Invalid maxactive number"), \ C(MAXACT_TOO_BIG, "Maxactive is too big"), \ C(BAD_PROBE_ADDR, "Invalid probed address or symbol"), \ + C(NON_UNIQ_SYMBOL, "The symbol is not unique"), \ C(BAD_RETPROBE, "Retprobe address must be an function entry"), \ C(NO_TRACEPOINT, "Tracepoint is not found"), \ C(BAD_ADDR_SUFFIX, "Invalid probed address suffix"), \ diff --git a/kernel/workqueue.c b/kernel/workqueue.c index c85825e17df8..a3522b70218d 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -2166,7 +2166,7 @@ static struct worker *create_worker(struct worker_pool *pool) { struct worker *worker; int id; - char id_buf[16]; + char id_buf[23]; /* ID is needed to determine kthread name */ id = ida_alloc(&pool->worker_ida, GFP_KERNEL); @@ -4600,12 +4600,22 @@ static int alloc_and_link_pwqs(struct workqueue_struct *wq) } cpus_read_unlock(); + /* for unbound pwq, flush the pwq_release_worker ensures that the + * pwq_release_workfn() completes before calling kfree(wq). + */ + if (ret) + kthread_flush_worker(pwq_release_worker); + return ret; enomem: if (wq->cpu_pwq) { - for_each_possible_cpu(cpu) - kfree(*per_cpu_ptr(wq->cpu_pwq, cpu)); + for_each_possible_cpu(cpu) { + struct pool_workqueue *pwq = *per_cpu_ptr(wq->cpu_pwq, cpu); + + if (pwq) + kmem_cache_free(pwq_cache, pwq); + } free_percpu(wq->cpu_pwq); wq->cpu_pwq = NULL; } @@ -5782,9 +5792,13 @@ static int workqueue_apply_unbound_cpumask(const cpumask_var_t unbound_cpumask) list_for_each_entry(wq, &workqueues, list) { if (!(wq->flags & WQ_UNBOUND)) continue; + /* creating multiple pwqs breaks ordering guarantee */ - if (wq->flags & __WQ_ORDERED) - continue; + if (!list_empty(&wq->pwqs)) { + if (wq->flags & __WQ_ORDERED_EXPLICIT) + continue; + wq->flags &= ~__WQ_ORDERED; + } ctx = apply_wqattrs_prepare(wq, wq->unbound_attrs, unbound_cpumask); if (IS_ERR(ctx)) { @@ -6535,9 +6549,6 @@ void __init workqueue_init_early(void) BUG_ON(!zalloc_cpumask_var_node(&pt->pod_cpus[0], GFP_KERNEL, NUMA_NO_NODE)); - wq_update_pod_attrs_buf = alloc_workqueue_attrs(); - BUG_ON(!wq_update_pod_attrs_buf); - pt->nr_pods = 1; cpumask_copy(pt->pod_cpus[0], cpu_possible_mask); pt->pod_node[0] = NUMA_NO_NODE; @@ -6605,13 +6616,13 @@ static void __init wq_cpu_intensive_thresh_init(void) unsigned long thresh; unsigned long bogo; + pwq_release_worker = kthread_create_worker(0, "pool_workqueue_release"); + BUG_ON(IS_ERR(pwq_release_worker)); + /* if the user set it to a specific value, keep it */ if (wq_cpu_intensive_thresh_us != ULONG_MAX) return; - pwq_release_worker = kthread_create_worker(0, "pool_workqueue_release"); - BUG_ON(IS_ERR(pwq_release_worker)); - /* * The default of 10ms is derived from the fact that most modern (as of * 2023) processors can do a lot in 10ms and that it's just below what diff --git a/lib/argv_split.c b/lib/argv_split.c index 1a19a0a93dc1..e28db8e3b58c 100644 --- a/lib/argv_split.c +++ b/lib/argv_split.c @@ -28,7 +28,7 @@ static int count_argc(const char *str) /** * argv_free - free an argv - * @argv - the argument vector to be freed + * @argv: the argument vector to be freed * * Frees an argv and the strings it points to. */ @@ -46,7 +46,7 @@ EXPORT_SYMBOL(argv_free); * @str: the string to be split * @argcp: returned argument count * - * Returns an array of pointers to strings which are split out from + * Returns: an array of pointers to strings which are split out from * @str. This is performed by strictly splitting on white-space; no * quote processing is performed. Multiple whitespace characters are * considered to be a single argument separator. The returned array diff --git a/lib/kunit/executor.c b/lib/kunit/executor.c index 5181aa2e760b..a6348489d45f 100644 --- a/lib/kunit/executor.c +++ b/lib/kunit/executor.c @@ -65,7 +65,7 @@ struct kunit_glob_filter { }; /* Split "suite_glob.test_glob" into two. Assumes filter_glob is not empty. */ -static void kunit_parse_glob_filter(struct kunit_glob_filter *parsed, +static int kunit_parse_glob_filter(struct kunit_glob_filter *parsed, const char *filter_glob) { const int len = strlen(filter_glob); @@ -73,16 +73,28 @@ static void kunit_parse_glob_filter(struct kunit_glob_filter *parsed, if (!period) { parsed->suite_glob = kzalloc(len + 1, GFP_KERNEL); + if (!parsed->suite_glob) + return -ENOMEM; + parsed->test_glob = NULL; strcpy(parsed->suite_glob, filter_glob); - return; + return 0; } parsed->suite_glob = kzalloc(period - filter_glob + 1, GFP_KERNEL); + if (!parsed->suite_glob) + return -ENOMEM; + parsed->test_glob = kzalloc(len - (period - filter_glob) + 1, GFP_KERNEL); + if (!parsed->test_glob) { + kfree(parsed->suite_glob); + return -ENOMEM; + } strncpy(parsed->suite_glob, filter_glob, period - filter_glob); strncpy(parsed->test_glob, period + 1, len - (period - filter_glob)); + + return 0; } /* Create a copy of suite with only tests that match test_glob. */ @@ -152,21 +164,24 @@ kunit_filter_suites(const struct kunit_suite_set *suite_set, } copy_start = copy; - if (filter_glob) - kunit_parse_glob_filter(&parsed_glob, filter_glob); + if (filter_glob) { + *err = kunit_parse_glob_filter(&parsed_glob, filter_glob); + if (*err) + goto free_copy; + } /* Parse attribute filters */ if (filters) { filter_count = kunit_get_filter_count(filters); parsed_filters = kcalloc(filter_count, sizeof(*parsed_filters), GFP_KERNEL); if (!parsed_filters) { - kfree(copy); - return filtered; + *err = -ENOMEM; + goto free_parsed_glob; } for (j = 0; j < filter_count; j++) parsed_filters[j] = kunit_next_attr_filter(&filters, err); if (*err) - goto err; + goto free_parsed_filters; } for (i = 0; &suite_set->start[i] != suite_set->end; i++) { @@ -178,7 +193,7 @@ kunit_filter_suites(const struct kunit_suite_set *suite_set, parsed_glob.test_glob); if (IS_ERR(filtered_suite)) { *err = PTR_ERR(filtered_suite); - goto err; + goto free_parsed_filters; } } if (filter_count > 0 && parsed_filters != NULL) { @@ -195,10 +210,11 @@ kunit_filter_suites(const struct kunit_suite_set *suite_set, filtered_suite = new_filtered_suite; if (*err) - goto err; + goto free_parsed_filters; + if (IS_ERR(filtered_suite)) { *err = PTR_ERR(filtered_suite); - goto err; + goto free_parsed_filters; } if (!filtered_suite) break; @@ -213,17 +229,19 @@ kunit_filter_suites(const struct kunit_suite_set *suite_set, filtered.start = copy_start; filtered.end = copy; -err: - if (*err) - kfree(copy); +free_parsed_filters: + if (filter_count) + kfree(parsed_filters); +free_parsed_glob: if (filter_glob) { kfree(parsed_glob.suite_glob); kfree(parsed_glob.test_glob); } - if (filter_count) - kfree(parsed_filters); +free_copy: + if (*err) + kfree(copy); return filtered; } diff --git a/lib/kunit/executor_test.c b/lib/kunit/executor_test.c index 4084071d0eb5..b4f6f96b2844 100644 --- a/lib/kunit/executor_test.c +++ b/lib/kunit/executor_test.c @@ -119,7 +119,7 @@ static void parse_filter_attr_test(struct kunit *test) { int j, filter_count; struct kunit_attr_filter *parsed_filters; - char *filters = "speed>slow, module!=example"; + char filters[] = "speed>slow, module!=example", *filter = filters; int err = 0; filter_count = kunit_get_filter_count(filters); @@ -128,7 +128,7 @@ static void parse_filter_attr_test(struct kunit *test) parsed_filters = kunit_kcalloc(test, filter_count, sizeof(*parsed_filters), GFP_KERNEL); for (j = 0; j < filter_count; j++) { - parsed_filters[j] = kunit_next_attr_filter(&filters, &err); + parsed_filters[j] = kunit_next_attr_filter(&filter, &err); KUNIT_ASSERT_EQ_MSG(test, err, 0, "failed to parse filter '%s'", filters[j]); } @@ -154,6 +154,7 @@ static void filter_attr_test(struct kunit *test) .start = subsuite, .end = &subsuite[2], }; struct kunit_suite_set got; + char filter[] = "speed>slow"; int err = 0; subsuite[0] = alloc_fake_suite(test, "normal_suite", dummy_attr_test_cases); @@ -168,7 +169,7 @@ static void filter_attr_test(struct kunit *test) * attribute is unset and thus, the filtering is based on the parent attribute * of slow. */ - got = kunit_filter_suites(&suite_set, NULL, "speed>slow", NULL, &err); + got = kunit_filter_suites(&suite_set, NULL, filter, NULL, &err); KUNIT_ASSERT_NOT_ERR_OR_NULL(test, got.start); KUNIT_ASSERT_EQ(test, err, 0); kfree_at_end(test, got.start); @@ -191,12 +192,13 @@ static void filter_attr_empty_test(struct kunit *test) .start = subsuite, .end = &subsuite[2], }; struct kunit_suite_set got; + char filter[] = "module!=dummy"; int err = 0; subsuite[0] = alloc_fake_suite(test, "suite1", dummy_attr_test_cases); subsuite[1] = alloc_fake_suite(test, "suite2", dummy_attr_test_cases); - got = kunit_filter_suites(&suite_set, NULL, "module!=dummy", NULL, &err); + got = kunit_filter_suites(&suite_set, NULL, filter, NULL, &err); KUNIT_ASSERT_EQ(test, err, 0); kfree_at_end(test, got.start); /* just in case */ @@ -211,12 +213,13 @@ static void filter_attr_skip_test(struct kunit *test) .start = subsuite, .end = &subsuite[1], }; struct kunit_suite_set got; + char filter[] = "speed>slow"; int err = 0; subsuite[0] = alloc_fake_suite(test, "suite", dummy_attr_test_cases); /* Want: suite(slow, normal), NULL -> suite(slow with SKIP, normal), NULL */ - got = kunit_filter_suites(&suite_set, NULL, "speed>slow", "skip", &err); + got = kunit_filter_suites(&suite_set, NULL, filter, "skip", &err); KUNIT_ASSERT_NOT_ERR_OR_NULL(test, got.start); KUNIT_ASSERT_EQ(test, err, 0); kfree_at_end(test, got.start); diff --git a/lib/kunit/test.c b/lib/kunit/test.c index 49698a168437..421f13981412 100644 --- a/lib/kunit/test.c +++ b/lib/kunit/test.c @@ -784,12 +784,13 @@ static int kunit_module_notify(struct notifier_block *nb, unsigned long val, switch (val) { case MODULE_STATE_LIVE: - kunit_module_init(mod); break; case MODULE_STATE_GOING: kunit_module_exit(mod); break; case MODULE_STATE_COMING: + kunit_module_init(mod); + break; case MODULE_STATE_UNFORMED: break; } diff --git a/lib/maple_tree.c b/lib/maple_tree.c index ee1ff0c59fd7..0e00a84e8e8f 100644 --- a/lib/maple_tree.c +++ b/lib/maple_tree.c @@ -256,6 +256,22 @@ bool mas_is_err(struct ma_state *mas) return xa_is_err(mas->node); } +static __always_inline bool mas_is_overflow(struct ma_state *mas) +{ + if (unlikely(mas->node == MAS_OVERFLOW)) + return true; + + return false; +} + +static __always_inline bool mas_is_underflow(struct ma_state *mas) +{ + if (unlikely(mas->node == MAS_UNDERFLOW)) + return true; + + return false; +} + static inline bool mas_searchable(struct ma_state *mas) { if (mas_is_none(mas)) @@ -4415,10 +4431,13 @@ no_entry: * * @mas: The maple state * @max: The minimum starting range + * @empty: Can be empty + * @set_underflow: Set the @mas->node to underflow state on limit. * * Return: The entry in the previous slot which is possibly NULL */ -static void *mas_prev_slot(struct ma_state *mas, unsigned long min, bool empty) +static void *mas_prev_slot(struct ma_state *mas, unsigned long min, bool empty, + bool set_underflow) { void *entry; void __rcu **slots; @@ -4435,7 +4454,6 @@ retry: if (unlikely(mas_rewalk_if_dead(mas, node, save_point))) goto retry; -again: if (mas->min <= min) { pivot = mas_safe_min(mas, pivots, mas->offset); @@ -4443,9 +4461,10 @@ again: goto retry; if (pivot <= min) - return NULL; + goto underflow; } +again: if (likely(mas->offset)) { mas->offset--; mas->last = mas->index - 1; @@ -4457,7 +4476,7 @@ again: } if (mas_is_none(mas)) - return NULL; + goto underflow; mas->last = mas->max; node = mas_mn(mas); @@ -4474,10 +4493,19 @@ again: if (likely(entry)) return entry; - if (!empty) + if (!empty) { + if (mas->index <= min) + goto underflow; + goto again; + } return entry; + +underflow: + if (set_underflow) + mas->node = MAS_UNDERFLOW; + return NULL; } /* @@ -4567,10 +4595,13 @@ no_entry: * @mas: The maple state * @max: The maximum starting range * @empty: Can be empty + * @set_overflow: Should @mas->node be set to overflow when the limit is + * reached. * * Return: The entry in the next slot which is possibly NULL */ -static void *mas_next_slot(struct ma_state *mas, unsigned long max, bool empty) +static void *mas_next_slot(struct ma_state *mas, unsigned long max, bool empty, + bool set_overflow) { void __rcu **slots; unsigned long *pivots; @@ -4589,22 +4620,22 @@ retry: if (unlikely(mas_rewalk_if_dead(mas, node, save_point))) goto retry; -again: if (mas->max >= max) { if (likely(mas->offset < data_end)) pivot = pivots[mas->offset]; else - return NULL; /* must be mas->max */ + goto overflow; if (unlikely(mas_rewalk_if_dead(mas, node, save_point))) goto retry; if (pivot >= max) - return NULL; + goto overflow; } if (likely(mas->offset < data_end)) { mas->index = pivots[mas->offset] + 1; +again: mas->offset++; if (likely(mas->offset < data_end)) mas->last = pivots[mas->offset]; @@ -4616,8 +4647,11 @@ again: goto retry; } - if (mas_is_none(mas)) + if (WARN_ON_ONCE(mas_is_none(mas))) { + mas->node = MAS_OVERFLOW; return NULL; + goto overflow; + } mas->offset = 0; mas->index = mas->min; @@ -4636,12 +4670,20 @@ again: return entry; if (!empty) { - if (!mas->offset) - data_end = 2; + if (mas->last >= max) + goto overflow; + + mas->index = mas->last + 1; + /* Node cannot end on NULL, so it's safe to short-cut here */ goto again; } return entry; + +overflow: + if (set_overflow) + mas->node = MAS_OVERFLOW; + return NULL; } /* @@ -4651,17 +4693,20 @@ again: * * Set the @mas->node to the next entry and the range_start to * the beginning value for the entry. Does not check beyond @limit. - * Sets @mas->index and @mas->last to the limit if it is hit. + * Sets @mas->index and @mas->last to the range, Does not update @mas->index and + * @mas->last on overflow. * Restarts on dead nodes. * * Return: the next entry or %NULL. */ static inline void *mas_next_entry(struct ma_state *mas, unsigned long limit) { - if (mas->last >= limit) + if (mas->last >= limit) { + mas->node = MAS_OVERFLOW; return NULL; + } - return mas_next_slot(mas, limit, false); + return mas_next_slot(mas, limit, false, true); } /* @@ -4837,7 +4882,7 @@ void *mas_walk(struct ma_state *mas) { void *entry; - if (mas_is_none(mas) || mas_is_paused(mas) || mas_is_ptr(mas)) + if (!mas_is_active(mas) || !mas_is_start(mas)) mas->node = MAS_START; retry: entry = mas_state_walk(mas); @@ -5294,14 +5339,22 @@ static inline void mte_destroy_walk(struct maple_enode *enode, static void mas_wr_store_setup(struct ma_wr_state *wr_mas) { - if (mas_is_start(wr_mas->mas)) - return; + if (!mas_is_active(wr_mas->mas)) { + if (mas_is_start(wr_mas->mas)) + return; - if (unlikely(mas_is_paused(wr_mas->mas))) - goto reset; + if (unlikely(mas_is_paused(wr_mas->mas))) + goto reset; - if (unlikely(mas_is_none(wr_mas->mas))) - goto reset; + if (unlikely(mas_is_none(wr_mas->mas))) + goto reset; + + if (unlikely(mas_is_overflow(wr_mas->mas))) + goto reset; + + if (unlikely(mas_is_underflow(wr_mas->mas))) + goto reset; + } /* * A less strict version of mas_is_span_wr() where we allow spanning @@ -5595,8 +5648,25 @@ static inline bool mas_next_setup(struct ma_state *mas, unsigned long max, { bool was_none = mas_is_none(mas); - if (mas_is_none(mas) || mas_is_paused(mas)) + if (unlikely(mas->last >= max)) { + mas->node = MAS_OVERFLOW; + return true; + } + + if (mas_is_active(mas)) + return false; + + if (mas_is_none(mas) || mas_is_paused(mas)) { + mas->node = MAS_START; + } else if (mas_is_overflow(mas)) { + /* Overflowed before, but the max changed */ mas->node = MAS_START; + } else if (mas_is_underflow(mas)) { + mas->node = MAS_START; + *entry = mas_walk(mas); + if (*entry) + return true; + } if (mas_is_start(mas)) *entry = mas_walk(mas); /* Retries on dead nodes handled by mas_walk */ @@ -5615,6 +5685,7 @@ static inline bool mas_next_setup(struct ma_state *mas, unsigned long max, if (mas_is_none(mas)) return true; + return false; } @@ -5637,7 +5708,7 @@ void *mas_next(struct ma_state *mas, unsigned long max) return entry; /* Retries on dead nodes handled by mas_next_slot */ - return mas_next_slot(mas, max, false); + return mas_next_slot(mas, max, false, true); } EXPORT_SYMBOL_GPL(mas_next); @@ -5660,7 +5731,7 @@ void *mas_next_range(struct ma_state *mas, unsigned long max) return entry; /* Retries on dead nodes handled by mas_next_slot */ - return mas_next_slot(mas, max, true); + return mas_next_slot(mas, max, true, true); } EXPORT_SYMBOL_GPL(mas_next_range); @@ -5691,18 +5762,31 @@ EXPORT_SYMBOL_GPL(mt_next); static inline bool mas_prev_setup(struct ma_state *mas, unsigned long min, void **entry) { - if (mas->index <= min) - goto none; + if (unlikely(mas->index <= min)) { + mas->node = MAS_UNDERFLOW; + return true; + } - if (mas_is_none(mas) || mas_is_paused(mas)) + if (mas_is_active(mas)) + return false; + + if (mas_is_overflow(mas)) { mas->node = MAS_START; + *entry = mas_walk(mas); + if (*entry) + return true; + } - if (mas_is_start(mas)) { - mas_walk(mas); - if (!mas->index) - goto none; + if (mas_is_none(mas) || mas_is_paused(mas)) { + mas->node = MAS_START; + } else if (mas_is_underflow(mas)) { + /* underflowed before but the min changed */ + mas->node = MAS_START; } + if (mas_is_start(mas)) + mas_walk(mas); + if (unlikely(mas_is_ptr(mas))) { if (!mas->index) goto none; @@ -5747,7 +5831,7 @@ void *mas_prev(struct ma_state *mas, unsigned long min) if (mas_prev_setup(mas, min, &entry)) return entry; - return mas_prev_slot(mas, min, false); + return mas_prev_slot(mas, min, false, true); } EXPORT_SYMBOL_GPL(mas_prev); @@ -5770,7 +5854,7 @@ void *mas_prev_range(struct ma_state *mas, unsigned long min) if (mas_prev_setup(mas, min, &entry)) return entry; - return mas_prev_slot(mas, min, true); + return mas_prev_slot(mas, min, true, true); } EXPORT_SYMBOL_GPL(mas_prev_range); @@ -5828,24 +5912,35 @@ EXPORT_SYMBOL_GPL(mas_pause); static inline bool mas_find_setup(struct ma_state *mas, unsigned long max, void **entry) { - *entry = NULL; + if (mas_is_active(mas)) { + if (mas->last < max) + return false; - if (unlikely(mas_is_none(mas))) { + return true; + } + + if (mas_is_paused(mas)) { if (unlikely(mas->last >= max)) return true; - mas->index = mas->last; + mas->index = ++mas->last; mas->node = MAS_START; - } else if (unlikely(mas_is_paused(mas))) { + } else if (mas_is_none(mas)) { if (unlikely(mas->last >= max)) return true; + mas->index = mas->last; mas->node = MAS_START; - mas->index = ++mas->last; - } else if (unlikely(mas_is_ptr(mas))) - goto ptr_out_of_range; + } else if (mas_is_overflow(mas) || mas_is_underflow(mas)) { + if (mas->index > max) { + mas->node = MAS_OVERFLOW; + return true; + } + + mas->node = MAS_START; + } - if (unlikely(mas_is_start(mas))) { + if (mas_is_start(mas)) { /* First run or continue */ if (mas->index > max) return true; @@ -5895,7 +5990,7 @@ void *mas_find(struct ma_state *mas, unsigned long max) return entry; /* Retries on dead nodes handled by mas_next_slot */ - return mas_next_slot(mas, max, false); + return mas_next_slot(mas, max, false, false); } EXPORT_SYMBOL_GPL(mas_find); @@ -5913,13 +6008,13 @@ EXPORT_SYMBOL_GPL(mas_find); */ void *mas_find_range(struct ma_state *mas, unsigned long max) { - void *entry; + void *entry = NULL; if (mas_find_setup(mas, max, &entry)) return entry; /* Retries on dead nodes handled by mas_next_slot */ - return mas_next_slot(mas, max, true); + return mas_next_slot(mas, max, true, false); } EXPORT_SYMBOL_GPL(mas_find_range); @@ -5934,26 +6029,36 @@ EXPORT_SYMBOL_GPL(mas_find_range); static inline bool mas_find_rev_setup(struct ma_state *mas, unsigned long min, void **entry) { - *entry = NULL; - - if (unlikely(mas_is_none(mas))) { - if (mas->index <= min) - goto none; + if (mas_is_active(mas)) { + if (mas->index > min) + return false; - mas->last = mas->index; - mas->node = MAS_START; + return true; } - if (unlikely(mas_is_paused(mas))) { + if (mas_is_paused(mas)) { if (unlikely(mas->index <= min)) { mas->node = MAS_NONE; return true; } mas->node = MAS_START; mas->last = --mas->index; + } else if (mas_is_none(mas)) { + if (mas->index <= min) + goto none; + + mas->last = mas->index; + mas->node = MAS_START; + } else if (mas_is_underflow(mas) || mas_is_overflow(mas)) { + if (mas->last <= min) { + mas->node = MAS_UNDERFLOW; + return true; + } + + mas->node = MAS_START; } - if (unlikely(mas_is_start(mas))) { + if (mas_is_start(mas)) { /* First run or continue */ if (mas->index < min) return true; @@ -6004,13 +6109,13 @@ none: */ void *mas_find_rev(struct ma_state *mas, unsigned long min) { - void *entry; + void *entry = NULL; if (mas_find_rev_setup(mas, min, &entry)) return entry; /* Retries on dead nodes handled by mas_prev_slot */ - return mas_prev_slot(mas, min, false); + return mas_prev_slot(mas, min, false, false); } EXPORT_SYMBOL_GPL(mas_find_rev); @@ -6030,13 +6135,13 @@ EXPORT_SYMBOL_GPL(mas_find_rev); */ void *mas_find_range_rev(struct ma_state *mas, unsigned long min) { - void *entry; + void *entry = NULL; if (mas_find_rev_setup(mas, min, &entry)) return entry; /* Retries on dead nodes handled by mas_prev_slot */ - return mas_prev_slot(mas, min, true); + return mas_prev_slot(mas, min, true, false); } EXPORT_SYMBOL_GPL(mas_find_range_rev); diff --git a/lib/scatterlist.c b/lib/scatterlist.c index c65566b4dc66..68b45c82c37a 100644 --- a/lib/scatterlist.c +++ b/lib/scatterlist.c @@ -265,7 +265,8 @@ EXPORT_SYMBOL(sg_free_table); * @table: The sg table header to use * @nents: Number of entries in sg list * @max_ents: The maximum number of entries the allocator returns per call - * @nents_first_chunk: Number of entries int the (preallocated) first + * @first_chunk: first SGL if preallocated (may be %NULL) + * @nents_first_chunk: Number of entries in the (preallocated) first * scatterlist chunk, 0 means no such preallocated chunk provided by user * @gfp_mask: GFP allocation mask * @alloc_fn: Allocator to use @@ -788,6 +789,7 @@ EXPORT_SYMBOL(__sg_page_iter_dma_next); * @miter: sg mapping iter to be started * @sgl: sg list to iterate over * @nents: number of sg entries + * @flags: sg iterator flags * * Description: * Starts mapping iterator @miter. diff --git a/lib/test_maple_tree.c b/lib/test_maple_tree.c index 0674aebd4423..06959165e2f9 100644 --- a/lib/test_maple_tree.c +++ b/lib/test_maple_tree.c @@ -2166,7 +2166,7 @@ static noinline void __init next_prev_test(struct maple_tree *mt) MT_BUG_ON(mt, val != NULL); MT_BUG_ON(mt, mas.index != 0); MT_BUG_ON(mt, mas.last != 5); - MT_BUG_ON(mt, mas.node != MAS_NONE); + MT_BUG_ON(mt, mas.node != MAS_UNDERFLOW); mas.index = 0; mas.last = 5; @@ -2917,6 +2917,7 @@ static noinline void __init check_empty_area_fill(struct maple_tree *mt) * exists MAS_NONE active range * exists active active range * DNE active active set to last range + * ERANGE active MAS_OVERFLOW last range * * Function ENTRY Start Result index & last * mas_prev() @@ -2945,6 +2946,7 @@ static noinline void __init check_empty_area_fill(struct maple_tree *mt) * any MAS_ROOT MAS_NONE 0 * exists active active range * DNE active active last range + * ERANGE active MAS_UNDERFLOW last range * * Function ENTRY Start Result index & last * mas_find() @@ -2955,7 +2957,7 @@ static noinline void __init check_empty_area_fill(struct maple_tree *mt) * DNE MAS_START MAS_NONE 0 * DNE MAS_PAUSE MAS_NONE 0 * DNE MAS_ROOT MAS_NONE 0 - * DNE MAS_NONE MAS_NONE 0 + * DNE MAS_NONE MAS_NONE 1 * if index == 0 * exists MAS_START MAS_ROOT 0 * exists MAS_PAUSE MAS_ROOT 0 @@ -2967,7 +2969,7 @@ static noinline void __init check_empty_area_fill(struct maple_tree *mt) * DNE MAS_START active set to max * exists MAS_PAUSE active range * DNE MAS_PAUSE active set to max - * exists MAS_NONE active range + * exists MAS_NONE active range (start at last) * exists active active range * DNE active active last range (max < last) * @@ -2992,7 +2994,7 @@ static noinline void __init check_empty_area_fill(struct maple_tree *mt) * DNE MAS_START active set to min * exists MAS_PAUSE active range * DNE MAS_PAUSE active set to min - * exists MAS_NONE active range + * exists MAS_NONE active range (start at index) * exists active active range * DNE active active last range (min > index) * @@ -3039,10 +3041,10 @@ static noinline void __init check_state_handling(struct maple_tree *mt) mtree_store_range(mt, 0, 0, ptr, GFP_KERNEL); mas_lock(&mas); - /* prev: Start -> none */ + /* prev: Start -> underflow*/ entry = mas_prev(&mas, 0); MT_BUG_ON(mt, entry != NULL); - MT_BUG_ON(mt, mas.node != MAS_NONE); + MT_BUG_ON(mt, mas.node != MAS_UNDERFLOW); /* prev: Start -> root */ mas_set(&mas, 10); @@ -3069,7 +3071,7 @@ static noinline void __init check_state_handling(struct maple_tree *mt) MT_BUG_ON(mt, entry != NULL); MT_BUG_ON(mt, mas.node != MAS_NONE); - /* next: start -> none */ + /* next: start -> none*/ mas_set(&mas, 10); entry = mas_next(&mas, ULONG_MAX); MT_BUG_ON(mt, mas.index != 1); @@ -3268,25 +3270,46 @@ static noinline void __init check_state_handling(struct maple_tree *mt) MT_BUG_ON(mt, mas.last != 0x2500); MT_BUG_ON(mt, !mas_active(mas)); - /* next:active -> active out of range*/ + /* next:active -> active beyond data */ entry = mas_next(&mas, 0x2999); MT_BUG_ON(mt, entry != NULL); MT_BUG_ON(mt, mas.index != 0x2501); MT_BUG_ON(mt, mas.last != 0x2fff); MT_BUG_ON(mt, !mas_active(mas)); - /* Continue after out of range*/ + /* Continue after last range ends after max */ entry = mas_next(&mas, ULONG_MAX); MT_BUG_ON(mt, entry != ptr3); MT_BUG_ON(mt, mas.index != 0x3000); MT_BUG_ON(mt, mas.last != 0x3500); MT_BUG_ON(mt, !mas_active(mas)); - /* next:active -> active out of range*/ + /* next:active -> active continued */ + entry = mas_next(&mas, ULONG_MAX); + MT_BUG_ON(mt, entry != NULL); + MT_BUG_ON(mt, mas.index != 0x3501); + MT_BUG_ON(mt, mas.last != ULONG_MAX); + MT_BUG_ON(mt, !mas_active(mas)); + + /* next:active -> overflow */ entry = mas_next(&mas, ULONG_MAX); MT_BUG_ON(mt, entry != NULL); MT_BUG_ON(mt, mas.index != 0x3501); MT_BUG_ON(mt, mas.last != ULONG_MAX); + MT_BUG_ON(mt, mas.node != MAS_OVERFLOW); + + /* next:overflow -> overflow */ + entry = mas_next(&mas, ULONG_MAX); + MT_BUG_ON(mt, entry != NULL); + MT_BUG_ON(mt, mas.index != 0x3501); + MT_BUG_ON(mt, mas.last != ULONG_MAX); + MT_BUG_ON(mt, mas.node != MAS_OVERFLOW); + + /* prev:overflow -> active */ + entry = mas_prev(&mas, 0); + MT_BUG_ON(mt, entry != ptr3); + MT_BUG_ON(mt, mas.index != 0x3000); + MT_BUG_ON(mt, mas.last != 0x3500); MT_BUG_ON(mt, !mas_active(mas)); /* next: none -> active, skip value at location */ @@ -3307,11 +3330,46 @@ static noinline void __init check_state_handling(struct maple_tree *mt) MT_BUG_ON(mt, mas.last != 0x1500); MT_BUG_ON(mt, !mas_active(mas)); - /* prev:active -> active out of range*/ + /* prev:active -> active spanning end range */ + entry = mas_prev(&mas, 0x0100); + MT_BUG_ON(mt, entry != NULL); + MT_BUG_ON(mt, mas.index != 0); + MT_BUG_ON(mt, mas.last != 0x0FFF); + MT_BUG_ON(mt, !mas_active(mas)); + + /* prev:active -> underflow */ + entry = mas_prev(&mas, 0); + MT_BUG_ON(mt, entry != NULL); + MT_BUG_ON(mt, mas.index != 0); + MT_BUG_ON(mt, mas.last != 0x0FFF); + MT_BUG_ON(mt, mas.node != MAS_UNDERFLOW); + + /* prev:underflow -> underflow */ entry = mas_prev(&mas, 0); MT_BUG_ON(mt, entry != NULL); MT_BUG_ON(mt, mas.index != 0); MT_BUG_ON(mt, mas.last != 0x0FFF); + MT_BUG_ON(mt, mas.node != MAS_UNDERFLOW); + + /* next:underflow -> active */ + entry = mas_next(&mas, ULONG_MAX); + MT_BUG_ON(mt, entry != ptr); + MT_BUG_ON(mt, mas.index != 0x1000); + MT_BUG_ON(mt, mas.last != 0x1500); + MT_BUG_ON(mt, !mas_active(mas)); + + /* prev:first value -> underflow */ + entry = mas_prev(&mas, 0x1000); + MT_BUG_ON(mt, entry != NULL); + MT_BUG_ON(mt, mas.index != 0x1000); + MT_BUG_ON(mt, mas.last != 0x1500); + MT_BUG_ON(mt, mas.node != MAS_UNDERFLOW); + + /* find:underflow -> first value */ + entry = mas_find(&mas, ULONG_MAX); + MT_BUG_ON(mt, entry != ptr); + MT_BUG_ON(mt, mas.index != 0x1000); + MT_BUG_ON(mt, mas.last != 0x1500); MT_BUG_ON(mt, !mas_active(mas)); /* prev: pause ->active */ @@ -3325,14 +3383,14 @@ static noinline void __init check_state_handling(struct maple_tree *mt) MT_BUG_ON(mt, mas.last != 0x2500); MT_BUG_ON(mt, !mas_active(mas)); - /* prev:active -> active out of range*/ + /* prev:active -> active spanning min */ entry = mas_prev(&mas, 0x1600); MT_BUG_ON(mt, entry != NULL); MT_BUG_ON(mt, mas.index != 0x1501); MT_BUG_ON(mt, mas.last != 0x1FFF); MT_BUG_ON(mt, !mas_active(mas)); - /* prev: active ->active, continue*/ + /* prev: active ->active, continue */ entry = mas_prev(&mas, 0); MT_BUG_ON(mt, entry != ptr); MT_BUG_ON(mt, mas.index != 0x1000); @@ -3379,7 +3437,7 @@ static noinline void __init check_state_handling(struct maple_tree *mt) MT_BUG_ON(mt, mas.last != 0x2FFF); MT_BUG_ON(mt, !mas_active(mas)); - /* find: none ->active */ + /* find: overflow ->active */ entry = mas_find(&mas, 0x5000); MT_BUG_ON(mt, entry != ptr3); MT_BUG_ON(mt, mas.index != 0x3000); @@ -3778,7 +3836,6 @@ static int __init maple_tree_seed(void) check_empty_area_fill(&tree); mtree_destroy(&tree); - mt_init_flags(&tree, MT_FLAGS_ALLOC_RANGE); check_state_handling(&tree); mtree_destroy(&tree); diff --git a/mm/damon/vaddr-test.h b/mm/damon/vaddr-test.h index c4b455b5ee30..dcf1ca6b31cc 100644 --- a/mm/damon/vaddr-test.h +++ b/mm/damon/vaddr-test.h @@ -148,6 +148,8 @@ static void damon_do_test_apply_three_regions(struct kunit *test, KUNIT_EXPECT_EQ(test, r->ar.start, expected[i * 2]); KUNIT_EXPECT_EQ(test, r->ar.end, expected[i * 2 + 1]); } + + damon_destroy_target(t); } /* diff --git a/mm/damon/vaddr.c b/mm/damon/vaddr.c index 4c81a9dbd044..cf8a9fc5c9d1 100644 --- a/mm/damon/vaddr.c +++ b/mm/damon/vaddr.c @@ -341,13 +341,14 @@ static void damon_hugetlb_mkold(pte_t *pte, struct mm_struct *mm, bool referenced = false; pte_t entry = huge_ptep_get(pte); struct folio *folio = pfn_folio(pte_pfn(entry)); + unsigned long psize = huge_page_size(hstate_vma(vma)); folio_get(folio); if (pte_young(entry)) { referenced = true; entry = pte_mkold(entry); - set_huge_pte_at(mm, addr, pte, entry); + set_huge_pte_at(mm, addr, pte, entry, psize); } #ifdef CONFIG_MMU_NOTIFIER diff --git a/mm/filemap.c b/mm/filemap.c index 582f5317ff71..f0a15ce1bd1b 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -3475,13 +3475,11 @@ skip: */ static vm_fault_t filemap_map_folio_range(struct vm_fault *vmf, struct folio *folio, unsigned long start, - unsigned long addr, unsigned int nr_pages) + unsigned long addr, unsigned int nr_pages, + unsigned int *mmap_miss) { vm_fault_t ret = 0; - struct vm_area_struct *vma = vmf->vma; - struct file *file = vma->vm_file; struct page *page = folio_page(folio, start); - unsigned int mmap_miss = READ_ONCE(file->f_ra.mmap_miss); unsigned int count = 0; pte_t *old_ptep = vmf->pte; @@ -3489,8 +3487,7 @@ static vm_fault_t filemap_map_folio_range(struct vm_fault *vmf, if (PageHWPoison(page + count)) goto skip; - if (mmap_miss > 0) - mmap_miss--; + (*mmap_miss)++; /* * NOTE: If there're PTE markers, we'll leave them to be @@ -3506,7 +3503,7 @@ skip: if (count) { set_pte_range(vmf, folio, page, count, addr); folio_ref_add(folio, count); - if (in_range(vmf->address, addr, count)) + if (in_range(vmf->address, addr, count * PAGE_SIZE)) ret = VM_FAULT_NOPAGE; } @@ -3520,12 +3517,40 @@ skip: if (count) { set_pte_range(vmf, folio, page, count, addr); folio_ref_add(folio, count); - if (in_range(vmf->address, addr, count)) + if (in_range(vmf->address, addr, count * PAGE_SIZE)) ret = VM_FAULT_NOPAGE; } vmf->pte = old_ptep; - WRITE_ONCE(file->f_ra.mmap_miss, mmap_miss); + + return ret; +} + +static vm_fault_t filemap_map_order0_folio(struct vm_fault *vmf, + struct folio *folio, unsigned long addr, + unsigned int *mmap_miss) +{ + vm_fault_t ret = 0; + struct page *page = &folio->page; + + if (PageHWPoison(page)) + return ret; + + (*mmap_miss)++; + + /* + * NOTE: If there're PTE markers, we'll leave them to be + * handled in the specific fault path, and it'll prohibit + * the fault-around logic. + */ + if (!pte_none(ptep_get(vmf->pte))) + return ret; + + if (vmf->address == addr) + ret = VM_FAULT_NOPAGE; + + set_pte_range(vmf, folio, page, 1, addr); + folio_ref_inc(folio); return ret; } @@ -3541,7 +3566,7 @@ vm_fault_t filemap_map_pages(struct vm_fault *vmf, XA_STATE(xas, &mapping->i_pages, start_pgoff); struct folio *folio; vm_fault_t ret = 0; - int nr_pages = 0; + unsigned int nr_pages = 0, mmap_miss = 0, mmap_miss_saved; rcu_read_lock(); folio = next_uptodate_folio(&xas, mapping, end_pgoff); @@ -3569,25 +3594,27 @@ vm_fault_t filemap_map_pages(struct vm_fault *vmf, end = folio->index + folio_nr_pages(folio) - 1; nr_pages = min(end, end_pgoff) - xas.xa_index + 1; - /* - * NOTE: If there're PTE markers, we'll leave them to be - * handled in the specific fault path, and it'll prohibit the - * fault-around logic. - */ - if (!pte_none(ptep_get(vmf->pte))) - goto unlock; - - ret |= filemap_map_folio_range(vmf, folio, - xas.xa_index - folio->index, addr, nr_pages); + if (!folio_test_large(folio)) + ret |= filemap_map_order0_folio(vmf, + folio, addr, &mmap_miss); + else + ret |= filemap_map_folio_range(vmf, folio, + xas.xa_index - folio->index, addr, + nr_pages, &mmap_miss); -unlock: folio_unlock(folio); folio_put(folio); - folio = next_uptodate_folio(&xas, mapping, end_pgoff); - } while (folio); + } while ((folio = next_uptodate_folio(&xas, mapping, end_pgoff)) != NULL); pte_unmap_unlock(vmf->pte, vmf->ptl); out: rcu_read_unlock(); + + mmap_miss_saved = READ_ONCE(file->f_ra.mmap_miss); + if (mmap_miss >= mmap_miss_saved) + WRITE_ONCE(file->f_ra.mmap_miss, 0); + else + WRITE_ONCE(file->f_ra.mmap_miss, mmap_miss_saved - mmap_miss); + return ret; } EXPORT_SYMBOL(filemap_map_pages); diff --git a/mm/hugetlb.c b/mm/hugetlb.c index ba6d39b71cb1..52d26072dfda 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -4980,7 +4980,7 @@ static bool is_hugetlb_entry_hwpoisoned(pte_t pte) static void hugetlb_install_folio(struct vm_area_struct *vma, pte_t *ptep, unsigned long addr, - struct folio *new_folio, pte_t old) + struct folio *new_folio, pte_t old, unsigned long sz) { pte_t newpte = make_huge_pte(vma, &new_folio->page, 1); @@ -4988,7 +4988,7 @@ hugetlb_install_folio(struct vm_area_struct *vma, pte_t *ptep, unsigned long add hugepage_add_new_anon_rmap(new_folio, vma, addr); if (userfaultfd_wp(vma) && huge_pte_uffd_wp(old)) newpte = huge_pte_mkuffd_wp(newpte); - set_huge_pte_at(vma->vm_mm, addr, ptep, newpte); + set_huge_pte_at(vma->vm_mm, addr, ptep, newpte, sz); hugetlb_count_add(pages_per_huge_page(hstate_vma(vma)), vma->vm_mm); folio_set_hugetlb_migratable(new_folio); } @@ -5065,7 +5065,7 @@ again: } else if (unlikely(is_hugetlb_entry_hwpoisoned(entry))) { if (!userfaultfd_wp(dst_vma)) entry = huge_pte_clear_uffd_wp(entry); - set_huge_pte_at(dst, addr, dst_pte, entry); + set_huge_pte_at(dst, addr, dst_pte, entry, sz); } else if (unlikely(is_hugetlb_entry_migration(entry))) { swp_entry_t swp_entry = pte_to_swp_entry(entry); bool uffd_wp = pte_swp_uffd_wp(entry); @@ -5080,18 +5080,18 @@ again: entry = swp_entry_to_pte(swp_entry); if (userfaultfd_wp(src_vma) && uffd_wp) entry = pte_swp_mkuffd_wp(entry); - set_huge_pte_at(src, addr, src_pte, entry); + set_huge_pte_at(src, addr, src_pte, entry, sz); } if (!userfaultfd_wp(dst_vma)) entry = huge_pte_clear_uffd_wp(entry); - set_huge_pte_at(dst, addr, dst_pte, entry); + set_huge_pte_at(dst, addr, dst_pte, entry, sz); } else if (unlikely(is_pte_marker(entry))) { pte_marker marker = copy_pte_marker( pte_to_swp_entry(entry), dst_vma); if (marker) set_huge_pte_at(dst, addr, dst_pte, - make_pte_marker(marker)); + make_pte_marker(marker), sz); } else { entry = huge_ptep_get(src_pte); pte_folio = page_folio(pte_page(entry)); @@ -5145,7 +5145,7 @@ again: goto again; } hugetlb_install_folio(dst_vma, dst_pte, addr, - new_folio, src_pte_old); + new_folio, src_pte_old, sz); spin_unlock(src_ptl); spin_unlock(dst_ptl); continue; @@ -5166,7 +5166,7 @@ again: if (!userfaultfd_wp(dst_vma)) entry = huge_pte_clear_uffd_wp(entry); - set_huge_pte_at(dst, addr, dst_pte, entry); + set_huge_pte_at(dst, addr, dst_pte, entry, sz); hugetlb_count_add(npages, dst); } spin_unlock(src_ptl); @@ -5184,7 +5184,8 @@ again: } static void move_huge_pte(struct vm_area_struct *vma, unsigned long old_addr, - unsigned long new_addr, pte_t *src_pte, pte_t *dst_pte) + unsigned long new_addr, pte_t *src_pte, pte_t *dst_pte, + unsigned long sz) { struct hstate *h = hstate_vma(vma); struct mm_struct *mm = vma->vm_mm; @@ -5202,7 +5203,7 @@ static void move_huge_pte(struct vm_area_struct *vma, unsigned long old_addr, spin_lock_nested(src_ptl, SINGLE_DEPTH_NESTING); pte = huge_ptep_get_and_clear(mm, old_addr, src_pte); - set_huge_pte_at(mm, new_addr, dst_pte, pte); + set_huge_pte_at(mm, new_addr, dst_pte, pte, sz); if (src_ptl != dst_ptl) spin_unlock(src_ptl); @@ -5259,7 +5260,7 @@ int move_hugetlb_page_tables(struct vm_area_struct *vma, if (!dst_pte) break; - move_huge_pte(vma, old_addr, new_addr, src_pte, dst_pte); + move_huge_pte(vma, old_addr, new_addr, src_pte, dst_pte, sz); } if (shared_pmd) @@ -5337,7 +5338,8 @@ static void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct if (pte_swp_uffd_wp_any(pte) && !(zap_flags & ZAP_FLAG_DROP_MARKER)) set_huge_pte_at(mm, address, ptep, - make_pte_marker(PTE_MARKER_UFFD_WP)); + make_pte_marker(PTE_MARKER_UFFD_WP), + sz); else huge_pte_clear(mm, address, ptep, sz); spin_unlock(ptl); @@ -5371,7 +5373,8 @@ static void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct if (huge_pte_uffd_wp(pte) && !(zap_flags & ZAP_FLAG_DROP_MARKER)) set_huge_pte_at(mm, address, ptep, - make_pte_marker(PTE_MARKER_UFFD_WP)); + make_pte_marker(PTE_MARKER_UFFD_WP), + sz); hugetlb_count_sub(pages_per_huge_page(h), mm); page_remove_rmap(page, vma, true); @@ -5676,7 +5679,7 @@ retry_avoidcopy: hugepage_add_new_anon_rmap(new_folio, vma, haddr); if (huge_pte_uffd_wp(pte)) newpte = huge_pte_mkuffd_wp(newpte); - set_huge_pte_at(mm, haddr, ptep, newpte); + set_huge_pte_at(mm, haddr, ptep, newpte, huge_page_size(h)); folio_set_hugetlb_migratable(new_folio); /* Make the old page be freed below */ new_folio = old_folio; @@ -5972,7 +5975,7 @@ static vm_fault_t hugetlb_no_page(struct mm_struct *mm, */ if (unlikely(pte_marker_uffd_wp(old_pte))) new_pte = huge_pte_mkuffd_wp(new_pte); - set_huge_pte_at(mm, haddr, ptep, new_pte); + set_huge_pte_at(mm, haddr, ptep, new_pte, huge_page_size(h)); hugetlb_count_add(pages_per_huge_page(h), mm); if ((flags & FAULT_FLAG_WRITE) && !(vma->vm_flags & VM_SHARED)) { @@ -6261,7 +6264,8 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_pte, } _dst_pte = make_pte_marker(PTE_MARKER_POISONED); - set_huge_pte_at(dst_mm, dst_addr, dst_pte, _dst_pte); + set_huge_pte_at(dst_mm, dst_addr, dst_pte, _dst_pte, + huge_page_size(h)); /* No need to invalidate - it was non-present before */ update_mmu_cache(dst_vma, dst_addr, dst_pte); @@ -6412,7 +6416,7 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_pte, if (wp_enabled) _dst_pte = huge_pte_mkuffd_wp(_dst_pte); - set_huge_pte_at(dst_mm, dst_addr, dst_pte, _dst_pte); + set_huge_pte_at(dst_mm, dst_addr, dst_pte, _dst_pte, huge_page_size(h)); hugetlb_count_add(pages_per_huge_page(h), dst_mm); @@ -6598,7 +6602,7 @@ long hugetlb_change_protection(struct vm_area_struct *vma, else if (uffd_wp_resolve) newpte = pte_swp_clear_uffd_wp(newpte); if (!pte_same(pte, newpte)) - set_huge_pte_at(mm, address, ptep, newpte); + set_huge_pte_at(mm, address, ptep, newpte, psize); } else if (unlikely(is_pte_marker(pte))) { /* No other markers apply for now. */ WARN_ON_ONCE(!pte_marker_uffd_wp(pte)); @@ -6623,7 +6627,8 @@ long hugetlb_change_protection(struct vm_area_struct *vma, if (unlikely(uffd_wp)) /* Safe to modify directly (none->non-present). */ set_huge_pte_at(mm, address, ptep, - make_pte_marker(PTE_MARKER_UFFD_WP)); + make_pte_marker(PTE_MARKER_UFFD_WP), + psize); } spin_unlock(ptl); } diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index f70e3d7a602e..d37831b8511c 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -291,7 +291,7 @@ struct kasan_stack_ring { #if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) -#ifndef __HAVE_ARCH_SHADOW_MAP +#ifndef kasan_shadow_to_mem static inline const void *kasan_shadow_to_mem(const void *shadow_addr) { return (void *)(((unsigned long)shadow_addr - KASAN_SHADOW_OFFSET) @@ -299,15 +299,13 @@ static inline const void *kasan_shadow_to_mem(const void *shadow_addr) } #endif +#ifndef addr_has_metadata static __always_inline bool addr_has_metadata(const void *addr) { -#ifdef __HAVE_ARCH_SHADOW_MAP - return (kasan_mem_to_shadow((void *)addr) != NULL); -#else return (kasan_reset_tag(addr) >= kasan_shadow_to_mem((void *)KASAN_SHADOW_START)); -#endif } +#endif /** * kasan_check_range - Check memory region, and report if invalid access. diff --git a/mm/memcontrol.c b/mm/memcontrol.c index a4d3282493b6..5b009b233ab8 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -2555,7 +2555,7 @@ static unsigned long calculate_high_delay(struct mem_cgroup *memcg, * Scheduled by try_charge() to be executed from the userland return path * and reclaims memory over the high limit. */ -void mem_cgroup_handle_over_high(void) +void mem_cgroup_handle_over_high(gfp_t gfp_mask) { unsigned long penalty_jiffies; unsigned long pflags; @@ -2583,7 +2583,7 @@ retry_reclaim: */ nr_reclaimed = reclaim_high(memcg, in_retry ? SWAP_CLUSTER_MAX : nr_pages, - GFP_KERNEL); + gfp_mask); /* * memory.high is breached and reclaim is unable to keep up. Throttle @@ -2819,7 +2819,7 @@ done_restock: if (current->memcg_nr_pages_over_high > MEMCG_CHARGE_BATCH && !(current->flags & PF_MEMALLOC) && gfpflags_allow_blocking(gfp_mask)) { - mem_cgroup_handle_over_high(); + mem_cgroup_handle_over_high(gfp_mask); } return 0; } @@ -3867,6 +3867,13 @@ static ssize_t mem_cgroup_write(struct kernfs_open_file *of, case _MEMSWAP: ret = mem_cgroup_resize_max(memcg, nr_pages, true); break; + case _KMEM: + pr_warn_once("kmem.limit_in_bytes is deprecated and will be removed. " + "Writing any value to this file has no effect. " + "Please report your usecase to linux-mm@kvack.org if you " + "depend on this functionality.\n"); + ret = 0; + break; case _TCP: ret = memcg_update_tcp_max(memcg, nr_pages); break; @@ -5077,6 +5084,12 @@ static struct cftype mem_cgroup_legacy_files[] = { .seq_show = memcg_numa_stat_show, }, #endif + { + .name = "kmem.limit_in_bytes", + .private = MEMFILE_PRIVATE(_KMEM, RES_LIMIT), + .write = mem_cgroup_write, + .read_u64 = mem_cgroup_read_u64, + }, { .name = "kmem.usage_in_bytes", .private = MEMFILE_PRIVATE(_KMEM, RES_USAGE), diff --git a/mm/mempolicy.c b/mm/mempolicy.c index 42b5567e3773..f1b00d6ac7ee 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -426,6 +426,7 @@ struct queue_pages { unsigned long start; unsigned long end; struct vm_area_struct *first; + bool has_unmovable; }; /* @@ -446,9 +447,8 @@ static inline bool queue_folio_required(struct folio *folio, /* * queue_folios_pmd() has three possible return values: * 0 - folios are placed on the right node or queued successfully, or - * special page is met, i.e. huge zero page. - * 1 - there is unmovable folio, and MPOL_MF_MOVE* & MPOL_MF_STRICT were - * specified. + * special page is met, i.e. zero page, or unmovable page is found + * but continue walking (indicated by queue_pages.has_unmovable). * -EIO - is migration entry or only MPOL_MF_STRICT was specified and an * existing folio was already on a node that does not follow the * policy. @@ -479,7 +479,7 @@ static int queue_folios_pmd(pmd_t *pmd, spinlock_t *ptl, unsigned long addr, if (flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) { if (!vma_migratable(walk->vma) || migrate_folio_add(folio, qp->pagelist, flags)) { - ret = 1; + qp->has_unmovable = true; goto unlock; } } else @@ -495,9 +495,8 @@ unlock: * * queue_folios_pte_range() has three possible return values: * 0 - folios are placed on the right node or queued successfully, or - * special page is met, i.e. zero page. - * 1 - there is unmovable folio, and MPOL_MF_MOVE* & MPOL_MF_STRICT were - * specified. + * special page is met, i.e. zero page, or unmovable page is found + * but continue walking (indicated by queue_pages.has_unmovable). * -EIO - only MPOL_MF_STRICT was specified and an existing folio was already * on a node that does not follow the policy. */ @@ -508,7 +507,6 @@ static int queue_folios_pte_range(pmd_t *pmd, unsigned long addr, struct folio *folio; struct queue_pages *qp = walk->private; unsigned long flags = qp->flags; - bool has_unmovable = false; pte_t *pte, *mapped_pte; pte_t ptent; spinlock_t *ptl; @@ -538,11 +536,12 @@ static int queue_folios_pte_range(pmd_t *pmd, unsigned long addr, if (!queue_folio_required(folio, qp)) continue; if (flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) { - /* MPOL_MF_STRICT must be specified if we get here */ - if (!vma_migratable(vma)) { - has_unmovable = true; - break; - } + /* + * MPOL_MF_STRICT must be specified if we get here. + * Continue walking vmas due to MPOL_MF_MOVE* flags. + */ + if (!vma_migratable(vma)) + qp->has_unmovable = true; /* * Do not abort immediately since there may be @@ -550,16 +549,13 @@ static int queue_folios_pte_range(pmd_t *pmd, unsigned long addr, * need migrate other LRU pages. */ if (migrate_folio_add(folio, qp->pagelist, flags)) - has_unmovable = true; + qp->has_unmovable = true; } else break; } pte_unmap_unlock(mapped_pte, ptl); cond_resched(); - if (has_unmovable) - return 1; - return addr != end ? -EIO : 0; } @@ -599,7 +595,7 @@ static int queue_folios_hugetlb(pte_t *pte, unsigned long hmask, * Detecting misplaced folio but allow migrating folios which * have been queued. */ - ret = 1; + qp->has_unmovable = true; goto unlock; } @@ -620,7 +616,7 @@ static int queue_folios_hugetlb(pte_t *pte, unsigned long hmask, * Failed to isolate folio but allow migrating pages * which have been queued. */ - ret = 1; + qp->has_unmovable = true; } unlock: spin_unlock(ptl); @@ -756,12 +752,15 @@ queue_pages_range(struct mm_struct *mm, unsigned long start, unsigned long end, .start = start, .end = end, .first = NULL, + .has_unmovable = false, }; const struct mm_walk_ops *ops = lock_vma ? &queue_pages_lock_vma_walk_ops : &queue_pages_walk_ops; err = walk_page_range(mm, start, end, ops, &qp); + if (qp.has_unmovable) + err = 1; if (!qp.first) /* whole range in hole */ err = -EFAULT; @@ -1358,7 +1357,7 @@ static long do_mbind(unsigned long start, unsigned long len, putback_movable_pages(&pagelist); } - if ((ret > 0) || (nr_failed && (flags & MPOL_MF_STRICT))) + if (((ret > 0) || nr_failed) && (flags & MPOL_MF_STRICT)) err = -EIO; } else { up_out: diff --git a/mm/migrate.c b/mm/migrate.c index b7fa020003f3..2053b54556ca 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -243,7 +243,9 @@ static bool remove_migration_pte(struct folio *folio, #ifdef CONFIG_HUGETLB_PAGE if (folio_test_hugetlb(folio)) { - unsigned int shift = huge_page_shift(hstate_vma(vma)); + struct hstate *h = hstate_vma(vma); + unsigned int shift = huge_page_shift(h); + unsigned long psize = huge_page_size(h); pte = arch_make_huge_pte(pte, shift, vma->vm_flags); if (folio_test_anon(folio)) @@ -251,7 +253,8 @@ static bool remove_migration_pte(struct folio *folio, rmap_flags); else page_dup_file_rmap(new, true); - set_huge_pte_at(vma->vm_mm, pvmw.address, pvmw.pte, pte); + set_huge_pte_at(vma->vm_mm, pvmw.address, pvmw.pte, pte, + psize); } else #endif { diff --git a/mm/mremap.c b/mm/mremap.c index 056478c106ee..382e81c33fc4 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -715,7 +715,7 @@ static unsigned long move_vma(struct vm_area_struct *vma, } vma_iter_init(&vmi, mm, old_addr); - if (!do_vmi_munmap(&vmi, mm, old_addr, old_len, uf_unmap, false)) { + if (do_vmi_munmap(&vmi, mm, old_addr, old_len, uf_unmap, false) < 0) { /* OOM: unable to split vma, just get accounts right */ if (vm_flags & VM_ACCOUNT && !(flags & MREMAP_DONTUNMAP)) vm_acct_memory(old_len >> PAGE_SHIFT); diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 0c5be12f9336..95546f376302 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -2400,7 +2400,7 @@ void free_unref_page(struct page *page, unsigned int order) struct per_cpu_pages *pcp; struct zone *zone; unsigned long pfn = page_to_pfn(page); - int migratetype; + int migratetype, pcpmigratetype; if (!free_unref_page_prepare(page, pfn, order)) return; @@ -2408,24 +2408,24 @@ void free_unref_page(struct page *page, unsigned int order) /* * We only track unmovable, reclaimable and movable on pcp lists. * Place ISOLATE pages on the isolated list because they are being - * offlined but treat HIGHATOMIC as movable pages so we can get those - * areas back if necessary. Otherwise, we may have to free + * offlined but treat HIGHATOMIC and CMA as movable pages so we can + * get those areas back if necessary. Otherwise, we may have to free * excessively into the page allocator */ - migratetype = get_pcppage_migratetype(page); + migratetype = pcpmigratetype = get_pcppage_migratetype(page); if (unlikely(migratetype >= MIGRATE_PCPTYPES)) { if (unlikely(is_migrate_isolate(migratetype))) { free_one_page(page_zone(page), page, pfn, order, migratetype, FPI_NONE); return; } - migratetype = MIGRATE_MOVABLE; + pcpmigratetype = MIGRATE_MOVABLE; } zone = page_zone(page); pcp_trylock_prepare(UP_flags); pcp = pcp_spin_trylock(zone->per_cpu_pageset); if (pcp) { - free_unref_page_commit(zone, pcp, page, migratetype, order); + free_unref_page_commit(zone, pcp, page, pcpmigratetype, order); pcp_spin_unlock(pcp); } else { free_one_page(zone, page, pfn, order, migratetype, FPI_NONE); diff --git a/mm/rmap.c b/mm/rmap.c index ec7f8e6c9e48..9f795b93cf40 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1480,6 +1480,7 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma, struct mmu_notifier_range range; enum ttu_flags flags = (enum ttu_flags)(long)arg; unsigned long pfn; + unsigned long hsz = 0; /* * When racing against e.g. zap_pte_range() on another cpu, @@ -1511,6 +1512,9 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma, */ adjust_range_if_pmd_sharing_possible(vma, &range.start, &range.end); + + /* We need the huge page size for set_huge_pte_at() */ + hsz = huge_page_size(hstate_vma(vma)); } mmu_notifier_invalidate_range_start(&range); @@ -1628,7 +1632,8 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma, pteval = swp_entry_to_pte(make_hwpoison_entry(subpage)); if (folio_test_hugetlb(folio)) { hugetlb_count_sub(folio_nr_pages(folio), mm); - set_huge_pte_at(mm, address, pvmw.pte, pteval); + set_huge_pte_at(mm, address, pvmw.pte, pteval, + hsz); } else { dec_mm_counter(mm, mm_counter(&folio->page)); set_pte_at(mm, address, pvmw.pte, pteval); @@ -1820,6 +1825,7 @@ static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma, struct mmu_notifier_range range; enum ttu_flags flags = (enum ttu_flags)(long)arg; unsigned long pfn; + unsigned long hsz = 0; /* * When racing against e.g. zap_pte_range() on another cpu, @@ -1855,6 +1861,9 @@ static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma, */ adjust_range_if_pmd_sharing_possible(vma, &range.start, &range.end); + + /* We need the huge page size for set_huge_pte_at() */ + hsz = huge_page_size(hstate_vma(vma)); } mmu_notifier_invalidate_range_start(&range); @@ -2020,7 +2029,8 @@ static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma, pteval = swp_entry_to_pte(make_hwpoison_entry(subpage)); if (folio_test_hugetlb(folio)) { hugetlb_count_sub(folio_nr_pages(folio), mm); - set_huge_pte_at(mm, address, pvmw.pte, pteval); + set_huge_pte_at(mm, address, pvmw.pte, pteval, + hsz); } else { dec_mm_counter(mm, mm_counter(&folio->page)); set_pte_at(mm, address, pvmw.pte, pteval); @@ -2044,7 +2054,8 @@ static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma, if (arch_unmap_one(mm, vma, address, pteval) < 0) { if (folio_test_hugetlb(folio)) - set_huge_pte_at(mm, address, pvmw.pte, pteval); + set_huge_pte_at(mm, address, pvmw.pte, + pteval, hsz); else set_pte_at(mm, address, pvmw.pte, pteval); ret = false; @@ -2058,7 +2069,8 @@ static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma, if (anon_exclusive && page_try_share_anon_rmap(subpage)) { if (folio_test_hugetlb(folio)) - set_huge_pte_at(mm, address, pvmw.pte, pteval); + set_huge_pte_at(mm, address, pvmw.pte, + pteval, hsz); else set_pte_at(mm, address, pvmw.pte, pteval); ret = false; @@ -2090,7 +2102,8 @@ static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma, if (pte_uffd_wp(pteval)) swp_pte = pte_swp_mkuffd_wp(swp_pte); if (folio_test_hugetlb(folio)) - set_huge_pte_at(mm, address, pvmw.pte, swp_pte); + set_huge_pte_at(mm, address, pvmw.pte, swp_pte, + hsz); else set_pte_at(mm, address, pvmw.pte, swp_pte); trace_set_migration_pte(address, pte_val(swp_pte), diff --git a/mm/shmem.c b/mm/shmem.c index 02e62fccc80d..69595d341882 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -4586,7 +4586,7 @@ static struct file_system_type shmem_fs_type = { #endif .kill_sb = kill_litter_super, #ifdef CONFIG_SHMEM - .fs_flags = FS_USERNS_MOUNT | FS_ALLOW_IDMAP | FS_MGTIME, + .fs_flags = FS_USERNS_MOUNT | FS_ALLOW_IDMAP, #else .fs_flags = FS_USERNS_MOUNT, #endif diff --git a/mm/slab_common.c b/mm/slab_common.c index cd71f9581e67..9bbffe82d65a 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -479,7 +479,7 @@ void slab_kmem_cache_release(struct kmem_cache *s) void kmem_cache_destroy(struct kmem_cache *s) { - int refcnt; + int err = -EBUSY; bool rcu_set; if (unlikely(!s) || !kasan_check_byte(s)) @@ -490,17 +490,17 @@ void kmem_cache_destroy(struct kmem_cache *s) rcu_set = s->flags & SLAB_TYPESAFE_BY_RCU; - refcnt = --s->refcount; - if (refcnt) + s->refcount--; + if (s->refcount) goto out_unlock; - WARN(shutdown_cache(s), - "%s %s: Slab cache still has objects when called from %pS", + err = shutdown_cache(s); + WARN(err, "%s %s: Slab cache still has objects when called from %pS", __func__, s->name, (void *)_RET_IP_); out_unlock: mutex_unlock(&slab_mutex); cpus_read_unlock(); - if (!refcnt && !rcu_set) + if (!err && !rcu_set) kmem_cache_release(s); } EXPORT_SYMBOL(kmem_cache_destroy); @@ -745,24 +745,24 @@ struct kmem_cache *kmalloc_slab(size_t size, gfp_t flags, unsigned long caller) size_t kmalloc_size_roundup(size_t size) { - struct kmem_cache *c; + if (size && size <= KMALLOC_MAX_CACHE_SIZE) { + /* + * The flags don't matter since size_index is common to all. + * Neither does the caller for just getting ->object_size. + */ + return kmalloc_slab(size, GFP_KERNEL, 0)->object_size; + } - /* Short-circuit the 0 size case. */ - if (unlikely(size == 0)) - return 0; - /* Short-circuit saturated "too-large" case. */ - if (unlikely(size == SIZE_MAX)) - return SIZE_MAX; /* Above the smaller buckets, size is a multiple of page size. */ - if (size > KMALLOC_MAX_CACHE_SIZE) + if (size && size <= KMALLOC_MAX_SIZE) return PAGE_SIZE << get_order(size); /* - * The flags don't matter since size_index is common to all. - * Neither does the caller for just getting ->object_size. + * Return 'size' for 0 - kmalloc() returns ZERO_SIZE_PTR + * and very large size - kmalloc() may fail. */ - c = kmalloc_slab(size, GFP_KERNEL, 0); - return c ? c->object_size : 0; + return size; + } EXPORT_SYMBOL(kmalloc_size_roundup); @@ -895,10 +895,13 @@ void __init setup_kmalloc_cache_index_table(void) static unsigned int __kmalloc_minalign(void) { + unsigned int minalign = dma_get_cache_alignment(); + if (IS_ENABLED(CONFIG_DMA_BOUNCE_UNALIGNED_KMALLOC) && is_swiotlb_allocated()) - return ARCH_KMALLOC_MINALIGN; - return dma_get_cache_alignment(); + minalign = ARCH_KMALLOC_MINALIGN; + + return max(minalign, arch_slab_minalign()); } void __init diff --git a/mm/vmalloc.c b/mm/vmalloc.c index ef8599d394fd..a3fedb3ee0db 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -111,7 +111,7 @@ static int vmap_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end, pte_t entry = pfn_pte(pfn, prot); entry = arch_make_huge_pte(entry, ilog2(size), 0); - set_huge_pte_at(&init_mm, addr, pte, entry); + set_huge_pte_at(&init_mm, addr, pte, entry, size); pfn += PFN_DOWN(size); continue; } diff --git a/mm/zswap.c b/mm/zswap.c index 412b1409a0d7..083c693602b8 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -1218,6 +1218,19 @@ bool zswap_store(struct folio *folio) if (!zswap_enabled || !tree) return false; + /* + * If this is a duplicate, it must be removed before attempting to store + * it, otherwise, if the store fails the old page won't be removed from + * the tree, and it might be written back overriding the new data. + */ + spin_lock(&tree->lock); + dupentry = zswap_rb_search(&tree->rbroot, offset); + if (dupentry) { + zswap_duplicate_entry++; + zswap_invalidate_entry(tree, dupentry); + } + spin_unlock(&tree->lock); + /* * XXX: zswap reclaim does not work with cgroups yet. Without a * cgroup-aware entry LRU, we will push out entries system-wide based on @@ -1333,7 +1346,14 @@ insert_entry: /* map */ spin_lock(&tree->lock); + /* + * A duplicate entry should have been removed at the beginning of this + * function. Since the swap entry should be pinned, if a duplicate is + * found again here it means that something went wrong in the swap + * cache. + */ while (zswap_rb_insert(&tree->rbroot, entry, &dupentry) == -EEXIST) { + WARN_ON(1); zswap_duplicate_entry++; zswap_invalidate_entry(tree, dupentry); } diff --git a/net/ax25/Kconfig b/net/ax25/Kconfig index d3a9843a043d..fdb666607f10 100644 --- a/net/ax25/Kconfig +++ b/net/ax25/Kconfig @@ -10,7 +10,7 @@ menuconfig HAMRADIO If you want to connect your Linux box to an amateur radio, answer Y here. You want to read and more specifically about AX.25 on Linux - . + . Note that the answer to this question won't directly affect the kernel: saying N will just cause the configurator to skip all @@ -61,7 +61,7 @@ config AX25_DAMA_SLAVE configuration. Linux cannot yet act as a DAMA server. This option only compiles DAMA slave support into the kernel. It still needs to be enabled at runtime. For more about DAMA see - . If unsure, say Y. + . If unsure, say Y. # placeholder until implemented config AX25_DAMA_MASTER @@ -87,9 +87,9 @@ config NETROM A comprehensive listing of all the software for Linux amateur radio users as well as information about how to configure an AX.25 port is contained in the Linux Ham Wiki, available from - . You also might want to check out the - file . More information about - digital amateur radio in general is on the WWW at + . You also might want to check out + the file . More information + about digital amateur radio in general is on the WWW at . To compile this driver as a module, choose M here: the @@ -106,9 +106,9 @@ config ROSE A comprehensive listing of all the software for Linux amateur radio users as well as information about how to configure an AX.25 port is contained in the Linux Ham Wiki, available from - . You also might want to check out the - file . More information about - digital amateur radio in general is on the WWW at + . You also might want to check out + the file . More information + about digital amateur radio in general is on the WWW at . To compile this driver as a module, choose M here: the diff --git a/net/bluetooth/hci_conn.c b/net/bluetooth/hci_conn.c index 9d5057cef30a..73470cc3518a 100644 --- a/net/bluetooth/hci_conn.c +++ b/net/bluetooth/hci_conn.c @@ -1627,6 +1627,15 @@ struct hci_conn *hci_connect_acl(struct hci_dev *hdev, bdaddr_t *dst, return ERR_PTR(-EOPNOTSUPP); } + /* Reject outgoing connection to device with same BD ADDR against + * CVE-2020-26555 + */ + if (!bacmp(&hdev->bdaddr, dst)) { + bt_dev_dbg(hdev, "Reject connection with same BD_ADDR %pMR\n", + dst); + return ERR_PTR(-ECONNREFUSED); + } + acl = hci_conn_hash_lookup_ba(hdev, ACL_LINK, dst); if (!acl) { acl = hci_conn_add(hdev, ACL_LINK, dst, HCI_ROLE_MASTER); @@ -2413,34 +2422,41 @@ int hci_conn_security(struct hci_conn *conn, __u8 sec_level, __u8 auth_type, if (!test_bit(HCI_CONN_AUTH, &conn->flags)) goto auth; - /* An authenticated FIPS approved combination key has sufficient - * security for security level 4. */ - if (conn->key_type == HCI_LK_AUTH_COMBINATION_P256 && - sec_level == BT_SECURITY_FIPS) - goto encrypt; - - /* An authenticated combination key has sufficient security for - security level 3. */ - if ((conn->key_type == HCI_LK_AUTH_COMBINATION_P192 || - conn->key_type == HCI_LK_AUTH_COMBINATION_P256) && - sec_level == BT_SECURITY_HIGH) - goto encrypt; - - /* An unauthenticated combination key has sufficient security for - security level 1 and 2. */ - if ((conn->key_type == HCI_LK_UNAUTH_COMBINATION_P192 || - conn->key_type == HCI_LK_UNAUTH_COMBINATION_P256) && - (sec_level == BT_SECURITY_MEDIUM || sec_level == BT_SECURITY_LOW)) - goto encrypt; - - /* A combination key has always sufficient security for the security - levels 1 or 2. High security level requires the combination key - is generated using maximum PIN code length (16). - For pre 2.1 units. */ - if (conn->key_type == HCI_LK_COMBINATION && - (sec_level == BT_SECURITY_MEDIUM || sec_level == BT_SECURITY_LOW || - conn->pin_length == 16)) - goto encrypt; + switch (conn->key_type) { + case HCI_LK_AUTH_COMBINATION_P256: + /* An authenticated FIPS approved combination key has + * sufficient security for security level 4 or lower. + */ + if (sec_level <= BT_SECURITY_FIPS) + goto encrypt; + break; + case HCI_LK_AUTH_COMBINATION_P192: + /* An authenticated combination key has sufficient security for + * security level 3 or lower. + */ + if (sec_level <= BT_SECURITY_HIGH) + goto encrypt; + break; + case HCI_LK_UNAUTH_COMBINATION_P192: + case HCI_LK_UNAUTH_COMBINATION_P256: + /* An unauthenticated combination key has sufficient security + * for security level 2 or lower. + */ + if (sec_level <= BT_SECURITY_MEDIUM) + goto encrypt; + break; + case HCI_LK_COMBINATION: + /* A combination key has always sufficient security for the + * security levels 2 or lower. High security level requires the + * combination key is generated using maximum PIN code length + * (16). For pre 2.1 units. + */ + if (sec_level <= BT_SECURITY_MEDIUM || conn->pin_length == 16) + goto encrypt; + break; + default: + break; + } auth: if (test_bit(HCI_CONN_ENCRYPT_PEND, &conn->flags)) diff --git a/net/bluetooth/hci_core.c b/net/bluetooth/hci_core.c index a5992f1b3c9b..195aea2198a9 100644 --- a/net/bluetooth/hci_core.c +++ b/net/bluetooth/hci_core.c @@ -2617,7 +2617,11 @@ int hci_register_dev(struct hci_dev *hdev) if (id < 0) return id; - snprintf(hdev->name, sizeof(hdev->name), "hci%d", id); + error = dev_set_name(&hdev->dev, "hci%u", id); + if (error) + return error; + + hdev->name = dev_name(&hdev->dev); hdev->id = id; BT_DBG("%p name %s bus %d", hdev, hdev->name, hdev->bus); @@ -2639,8 +2643,6 @@ int hci_register_dev(struct hci_dev *hdev) if (!IS_ERR_OR_NULL(bt_debugfs)) hdev->debugfs = debugfs_create_dir(hdev->name, bt_debugfs); - dev_set_name(&hdev->dev, "%s", hdev->name); - error = device_add(&hdev->dev); if (error < 0) goto err_wqueue; @@ -2784,6 +2786,7 @@ void hci_release_dev(struct hci_dev *hdev) hci_conn_params_clear_all(hdev); hci_discovery_filter_clear(hdev); hci_blocked_keys_clear(hdev); + hci_codec_list_clear(&hdev->local_codecs); hci_dev_unlock(hdev); ida_simple_remove(&hci_index_ida, hdev->id); @@ -3418,7 +3421,12 @@ static void hci_link_tx_to(struct hci_dev *hdev, __u8 type) if (c->type == type && c->sent) { bt_dev_err(hdev, "killing stalled connection %pMR", &c->dst); + /* hci_disconnect might sleep, so, we have to release + * the RCU read lock before calling it. + */ + rcu_read_unlock(); hci_disconnect(c, HCI_ERROR_REMOTE_USER_TERM); + rcu_read_lock(); } } diff --git a/net/bluetooth/hci_event.c b/net/bluetooth/hci_event.c index 35f251041eeb..1e1c9147356c 100644 --- a/net/bluetooth/hci_event.c +++ b/net/bluetooth/hci_event.c @@ -26,6 +26,8 @@ /* Bluetooth HCI event handling. */ #include +#include +#include #include #include @@ -33,6 +35,7 @@ #include "hci_request.h" #include "hci_debugfs.h" +#include "hci_codec.h" #include "a2mp.h" #include "amp.h" #include "smp.h" @@ -3267,6 +3270,16 @@ static void hci_conn_request_evt(struct hci_dev *hdev, void *data, bt_dev_dbg(hdev, "bdaddr %pMR type 0x%x", &ev->bdaddr, ev->link_type); + /* Reject incoming connection from device with same BD ADDR against + * CVE-2020-26555 + */ + if (hdev && !bacmp(&hdev->bdaddr, &ev->bdaddr)) { + bt_dev_dbg(hdev, "Reject connection with same BD_ADDR %pMR\n", + &ev->bdaddr); + hci_reject_conn(hdev, &ev->bdaddr); + return; + } + mask |= hci_proto_connect_ind(hdev, &ev->bdaddr, ev->link_type, &flags); @@ -4741,6 +4754,15 @@ static void hci_link_key_notify_evt(struct hci_dev *hdev, void *data, if (!conn) goto unlock; + /* Ignore NULL link key against CVE-2020-26555 */ + if (!crypto_memneq(ev->link_key, ZERO_KEY, HCI_LINK_KEY_SIZE)) { + bt_dev_dbg(hdev, "Ignore NULL link key (ZERO KEY) for %pMR", + &ev->bdaddr); + hci_disconnect(conn, HCI_ERROR_AUTH_FAILURE); + hci_conn_drop(conn); + goto unlock; + } + hci_conn_hold(conn); conn->disc_timeout = HCI_DISCONN_TIMEOUT; hci_conn_drop(conn); @@ -5273,8 +5295,8 @@ static u8 bredr_oob_data_present(struct hci_conn *conn) * available, then do not declare that OOB data is * present. */ - if (!memcmp(data->rand256, ZERO_KEY, 16) || - !memcmp(data->hash256, ZERO_KEY, 16)) + if (!crypto_memneq(data->rand256, ZERO_KEY, 16) || + !crypto_memneq(data->hash256, ZERO_KEY, 16)) return 0x00; return 0x02; @@ -5284,8 +5306,8 @@ static u8 bredr_oob_data_present(struct hci_conn *conn) * not supported by the hardware, then check that if * P-192 data values are present. */ - if (!memcmp(data->rand192, ZERO_KEY, 16) || - !memcmp(data->hash192, ZERO_KEY, 16)) + if (!crypto_memneq(data->rand192, ZERO_KEY, 16) || + !crypto_memneq(data->hash192, ZERO_KEY, 16)) return 0x00; return 0x01; @@ -5302,7 +5324,7 @@ static void hci_io_capa_request_evt(struct hci_dev *hdev, void *data, hci_dev_lock(hdev); conn = hci_conn_hash_lookup_ba(hdev, ACL_LINK, &ev->bdaddr); - if (!conn) + if (!conn || !hci_conn_ssp_enabled(conn)) goto unlock; hci_conn_hold(conn); @@ -5549,7 +5571,7 @@ static void hci_simple_pair_complete_evt(struct hci_dev *hdev, void *data, hci_dev_lock(hdev); conn = hci_conn_hash_lookup_ba(hdev, ACL_LINK, &ev->bdaddr); - if (!conn) + if (!conn || !hci_conn_ssp_enabled(conn)) goto unlock; /* Reset the authentication requirement to unknown */ @@ -7020,6 +7042,14 @@ unlock: hci_dev_unlock(hdev); } +static int hci_iso_term_big_sync(struct hci_dev *hdev, void *data) +{ + u8 handle = PTR_UINT(data); + + return hci_le_terminate_big_sync(hdev, handle, + HCI_ERROR_LOCAL_HOST_TERM); +} + static void hci_le_create_big_complete_evt(struct hci_dev *hdev, void *data, struct sk_buff *skb) { @@ -7064,16 +7094,17 @@ static void hci_le_create_big_complete_evt(struct hci_dev *hdev, void *data, rcu_read_lock(); } + rcu_read_unlock(); + if (!ev->status && !i) /* If no BISes have been connected for the BIG, * terminate. This is in case all bound connections * have been closed before the BIG creation * has completed. */ - hci_le_terminate_big_sync(hdev, ev->handle, - HCI_ERROR_LOCAL_HOST_TERM); + hci_cmd_sync_queue(hdev, hci_iso_term_big_sync, + UINT_PTR(ev->handle), NULL); - rcu_read_unlock(); hci_dev_unlock(hdev); } diff --git a/net/bluetooth/hci_request.h b/net/bluetooth/hci_request.h index b9c5a9823837..0be75cf0efed 100644 --- a/net/bluetooth/hci_request.h +++ b/net/bluetooth/hci_request.h @@ -71,7 +71,5 @@ struct sk_buff *hci_prepare_cmd(struct hci_dev *hdev, u16 opcode, u32 plen, void hci_req_add_le_scan_disable(struct hci_request *req, bool rpa_le_conn); void hci_req_add_le_passive_scan(struct hci_request *req); -void hci_req_prepare_suspend(struct hci_dev *hdev, enum suspended_state next); - void hci_request_setup(struct hci_dev *hdev); void hci_request_cancel_all(struct hci_dev *hdev); diff --git a/net/bluetooth/hci_sock.c b/net/bluetooth/hci_sock.c index 5e4f718073b7..3e7cd330d731 100644 --- a/net/bluetooth/hci_sock.c +++ b/net/bluetooth/hci_sock.c @@ -488,7 +488,8 @@ static struct sk_buff *create_monitor_event(struct hci_dev *hdev, int event) ni->type = hdev->dev_type; ni->bus = hdev->bus; bacpy(&ni->bdaddr, &hdev->bdaddr); - memcpy(ni->name, hdev->name, 8); + memcpy_and_pad(ni->name, sizeof(ni->name), hdev->name, + strnlen(hdev->name, sizeof(ni->name)), '\0'); opcode = cpu_to_le16(HCI_MON_NEW_INDEX); break; diff --git a/net/bluetooth/hci_sync.c b/net/bluetooth/hci_sync.c index 9b93653c6197..a15ab0b874a9 100644 --- a/net/bluetooth/hci_sync.c +++ b/net/bluetooth/hci_sync.c @@ -413,11 +413,6 @@ static int hci_le_scan_restart_sync(struct hci_dev *hdev) LE_SCAN_FILTER_DUP_ENABLE); } -static int le_scan_restart_sync(struct hci_dev *hdev, void *data) -{ - return hci_le_scan_restart_sync(hdev); -} - static void le_scan_restart(struct work_struct *work) { struct hci_dev *hdev = container_of(work, struct hci_dev, @@ -427,15 +422,15 @@ static void le_scan_restart(struct work_struct *work) bt_dev_dbg(hdev, ""); - hci_dev_lock(hdev); - - status = hci_cmd_sync_queue(hdev, le_scan_restart_sync, NULL, NULL); + status = hci_le_scan_restart_sync(hdev); if (status) { bt_dev_err(hdev, "failed to restart LE scan: status %d", status); - goto unlock; + return; } + hci_dev_lock(hdev); + if (!test_bit(HCI_QUIRK_STRICT_DUPLICATE_FILTER, &hdev->quirks) || !hdev->discovery.scan_start) goto unlock; @@ -5079,6 +5074,7 @@ int hci_dev_close_sync(struct hci_dev *hdev) memset(hdev->eir, 0, sizeof(hdev->eir)); memset(hdev->dev_class, 0, sizeof(hdev->dev_class)); bacpy(&hdev->random_addr, BDADDR_ANY); + hci_codec_list_clear(&hdev->local_codecs); hci_dev_put(hdev); return err; @@ -5373,6 +5369,7 @@ int hci_abort_conn_sync(struct hci_dev *hdev, struct hci_conn *conn, u8 reason) { int err = 0; u16 handle = conn->handle; + bool disconnect = false; struct hci_conn *c; switch (conn->state) { @@ -5403,24 +5400,15 @@ int hci_abort_conn_sync(struct hci_dev *hdev, struct hci_conn *conn, u8 reason) hci_dev_unlock(hdev); return 0; case BT_BOUND: - hci_dev_lock(hdev); - hci_conn_failed(conn, reason); - hci_dev_unlock(hdev); - return 0; + break; default: - hci_dev_lock(hdev); - conn->state = BT_CLOSED; - hci_disconn_cfm(conn, reason); - hci_conn_del(conn); - hci_dev_unlock(hdev); - return 0; + disconnect = true; + break; } hci_dev_lock(hdev); - /* Check if the connection hasn't been cleanup while waiting - * commands to complete. - */ + /* Check if the connection has been cleaned up concurrently */ c = hci_conn_hash_lookup_handle(hdev, handle); if (!c || c != conn) { err = 0; @@ -5432,7 +5420,13 @@ int hci_abort_conn_sync(struct hci_dev *hdev, struct hci_conn *conn, u8 reason) * or in case of LE it was still scanning so it can be cleanup * safely. */ - hci_conn_failed(conn, reason); + if (disconnect) { + conn->state = BT_CLOSED; + hci_disconn_cfm(conn, reason); + hci_conn_del(conn); + } else { + hci_conn_failed(conn, reason); + } unlock: hci_dev_unlock(hdev); diff --git a/net/bluetooth/iso.c b/net/bluetooth/iso.c index 16da946f5881..71248163ce9a 100644 --- a/net/bluetooth/iso.c +++ b/net/bluetooth/iso.c @@ -502,7 +502,7 @@ drop: } /* -------- Socket interface ---------- */ -static struct sock *__iso_get_sock_listen_by_addr(bdaddr_t *ba) +static struct sock *__iso_get_sock_listen_by_addr(bdaddr_t *src, bdaddr_t *dst) { struct sock *sk; @@ -510,7 +510,10 @@ static struct sock *__iso_get_sock_listen_by_addr(bdaddr_t *ba) if (sk->sk_state != BT_LISTEN) continue; - if (!bacmp(&iso_pi(sk)->src, ba)) + if (bacmp(&iso_pi(sk)->dst, dst)) + continue; + + if (!bacmp(&iso_pi(sk)->src, src)) return sk; } @@ -952,7 +955,7 @@ static int iso_listen_cis(struct sock *sk) write_lock(&iso_sk_list.lock); - if (__iso_get_sock_listen_by_addr(&iso_pi(sk)->src)) + if (__iso_get_sock_listen_by_addr(&iso_pi(sk)->src, &iso_pi(sk)->dst)) err = -EADDRINUSE; write_unlock(&iso_sk_list.lock); diff --git a/net/bridge/br_forward.c b/net/bridge/br_forward.c index 9d7bc8b96b53..7431f89e897b 100644 --- a/net/bridge/br_forward.c +++ b/net/bridge/br_forward.c @@ -124,7 +124,7 @@ static int deliver_clone(const struct net_bridge_port *prev, skb = skb_clone(skb, GFP_ATOMIC); if (!skb) { - dev->stats.tx_dropped++; + DEV_STATS_INC(dev, tx_dropped); return -ENOMEM; } @@ -268,7 +268,7 @@ static void maybe_deliver_addr(struct net_bridge_port *p, struct sk_buff *skb, skb = skb_copy(skb, GFP_ATOMIC); if (!skb) { - dev->stats.tx_dropped++; + DEV_STATS_INC(dev, tx_dropped); return; } diff --git a/net/bridge/br_input.c b/net/bridge/br_input.c index c34a0b0901b0..c729528b5e85 100644 --- a/net/bridge/br_input.c +++ b/net/bridge/br_input.c @@ -181,12 +181,12 @@ int br_handle_frame_finish(struct net *net, struct sock *sk, struct sk_buff *skb if ((mdst && mdst->host_joined) || br_multicast_is_router(brmctx, skb)) { local_rcv = true; - br->dev->stats.multicast++; + DEV_STATS_INC(br->dev, multicast); } mcast_hit = true; } else { local_rcv = true; - br->dev->stats.multicast++; + DEV_STATS_INC(br->dev, multicast); } break; case BR_PKT_UNICAST: diff --git a/net/bridge/br_netfilter_hooks.c b/net/bridge/br_netfilter_hooks.c index 15186247b59a..033034d68f1f 100644 --- a/net/bridge/br_netfilter_hooks.c +++ b/net/bridge/br_netfilter_hooks.c @@ -294,7 +294,7 @@ int br_nf_pre_routing_finish_bridge(struct net *net, struct sock *sk, struct sk_ /* tell br_dev_xmit to continue with forwarding */ nf_bridge->bridged_dnat = 1; /* FIXME Need to refragment */ - ret = neigh->output(neigh, skb); + ret = READ_ONCE(neigh->output)(neigh, skb); } neigh_release(neigh); return ret; diff --git a/net/can/isotp.c b/net/can/isotp.c index f02b5d3e4733..d1c6f206f429 100644 --- a/net/can/isotp.c +++ b/net/can/isotp.c @@ -948,21 +948,18 @@ static int isotp_sendmsg(struct socket *sock, struct msghdr *msg, size_t size) if (!so->bound || so->tx.state == ISOTP_SHUTDOWN) return -EADDRNOTAVAIL; -wait_free_buffer: - /* we do not support multiple buffers - for now */ - if (wq_has_sleeper(&so->wait) && (msg->msg_flags & MSG_DONTWAIT)) - return -EAGAIN; + while (cmpxchg(&so->tx.state, ISOTP_IDLE, ISOTP_SENDING) != ISOTP_IDLE) { + /* we do not support multiple buffers - for now */ + if (msg->msg_flags & MSG_DONTWAIT) + return -EAGAIN; - /* wait for complete transmission of current pdu */ - err = wait_event_interruptible(so->wait, so->tx.state == ISOTP_IDLE); - if (err) - goto err_event_drop; - - if (cmpxchg(&so->tx.state, ISOTP_IDLE, ISOTP_SENDING) != ISOTP_IDLE) { if (so->tx.state == ISOTP_SHUTDOWN) return -EADDRNOTAVAIL; - goto wait_free_buffer; + /* wait for complete transmission of current pdu */ + err = wait_event_interruptible(so->wait, so->tx.state == ISOTP_IDLE); + if (err) + goto err_event_drop; } /* PDU size > default => try max_pdu_size */ diff --git a/net/ceph/messenger.c b/net/ceph/messenger.c index 10a41cd9c523..3c8b78d9c4d1 100644 --- a/net/ceph/messenger.c +++ b/net/ceph/messenger.c @@ -459,8 +459,8 @@ int ceph_tcp_connect(struct ceph_connection *con) set_sock_callbacks(sock, con); con_sock_state_connecting(con); - ret = sock->ops->connect(sock, (struct sockaddr *)&ss, sizeof(ss), - O_NONBLOCK); + ret = kernel_connect(sock, (struct sockaddr *)&ss, sizeof(ss), + O_NONBLOCK); if (ret == -EINPROGRESS) { dout("connect %s EINPROGRESS sk_state = %u\n", ceph_pr_addr(&con->peer_addr), diff --git a/net/core/dev.c b/net/core/dev.c index ccff2b6ef958..9f3f8930c691 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -69,7 +69,7 @@ */ #include -#include +#include #include #include #include @@ -345,7 +345,6 @@ int netdev_name_node_alt_create(struct net_device *dev, const char *name) static void __netdev_name_node_alt_destroy(struct netdev_name_node *name_node) { list_del(&name_node->list); - netdev_name_node_del(name_node); kfree(name_node->name); netdev_name_node_free(name_node); } @@ -364,6 +363,8 @@ int netdev_name_node_alt_destroy(struct net_device *dev, const char *name) if (name_node == dev->name_node || name_node->dev != dev) return -EINVAL; + netdev_name_node_del(name_node); + synchronize_rcu(); __netdev_name_node_alt_destroy(name_node); return 0; @@ -380,6 +381,7 @@ static void netdev_name_node_alt_flush(struct net_device *dev) /* Device list insertion */ static void list_netdevice(struct net_device *dev) { + struct netdev_name_node *name_node; struct net *net = dev_net(dev); ASSERT_RTNL(); @@ -390,6 +392,10 @@ static void list_netdevice(struct net_device *dev) hlist_add_head_rcu(&dev->index_hlist, dev_index_hash(net, dev->ifindex)); write_unlock(&dev_base_lock); + + netdev_for_each_altname(dev, name_node) + netdev_name_node_add(net, name_node); + /* We reserved the ifindex, this can't fail */ WARN_ON(xa_store(&net->dev_by_index, dev->ifindex, dev, GFP_KERNEL)); @@ -401,12 +407,16 @@ static void list_netdevice(struct net_device *dev) */ static void unlist_netdevice(struct net_device *dev, bool lock) { + struct netdev_name_node *name_node; struct net *net = dev_net(dev); ASSERT_RTNL(); xa_erase(&net->dev_by_index, dev->ifindex); + netdev_for_each_altname(dev, name_node) + netdev_name_node_del(name_node); + /* Unlink dev from the device chain */ if (lock) write_lock(&dev_base_lock); @@ -1080,13 +1090,14 @@ static int __dev_alloc_name(struct net *net, const char *name, char *buf) return -EINVAL; /* Use one page as a bit array of possible slots */ - inuse = (unsigned long *) get_zeroed_page(GFP_ATOMIC); + inuse = bitmap_zalloc(max_netdevices, GFP_ATOMIC); if (!inuse) return -ENOMEM; for_each_netdev(net, d) { struct netdev_name_node *name_node; - list_for_each_entry(name_node, &d->name_node->list, list) { + + netdev_for_each_altname(d, name_node) { if (!sscanf(name_node->name, name, &i)) continue; if (i < 0 || i >= max_netdevices) @@ -1109,7 +1120,7 @@ static int __dev_alloc_name(struct net *net, const char *name, char *buf) } i = find_first_zero_bit(inuse, max_netdevices); - free_page((unsigned long) inuse); + bitmap_free(inuse); } snprintf(buf, IFNAMSIZ, name, i); @@ -1123,6 +1134,26 @@ static int __dev_alloc_name(struct net *net, const char *name, char *buf) return -ENFILE; } +static int dev_prep_valid_name(struct net *net, struct net_device *dev, + const char *want_name, char *out_name) +{ + int ret; + + if (!dev_valid_name(want_name)) + return -EINVAL; + + if (strchr(want_name, '%')) { + ret = __dev_alloc_name(net, want_name, out_name); + return ret < 0 ? ret : 0; + } else if (netdev_name_in_use(net, want_name)) { + return -EEXIST; + } else if (out_name != want_name) { + strscpy(out_name, want_name, IFNAMSIZ); + } + + return 0; +} + static int dev_alloc_name_ns(struct net *net, struct net_device *dev, const char *name) @@ -1160,19 +1191,13 @@ EXPORT_SYMBOL(dev_alloc_name); static int dev_get_valid_name(struct net *net, struct net_device *dev, const char *name) { - BUG_ON(!net); - - if (!dev_valid_name(name)) - return -EINVAL; - - if (strchr(name, '%')) - return dev_alloc_name_ns(net, dev, name); - else if (netdev_name_in_use(net, name)) - return -EEXIST; - else if (dev->name != name) - strscpy(dev->name, name, IFNAMSIZ); + char buf[IFNAMSIZ]; + int ret; - return 0; + ret = dev_prep_valid_name(net, dev, name, buf); + if (ret >= 0) + strscpy(dev->name, buf, IFNAMSIZ); + return ret; } /** @@ -3292,15 +3317,19 @@ int skb_checksum_help(struct sk_buff *skb) offset = skb_checksum_start_offset(skb); ret = -EINVAL; - if (WARN_ON_ONCE(offset >= skb_headlen(skb))) { + if (unlikely(offset >= skb_headlen(skb))) { DO_ONCE_LITE(skb_dump, KERN_ERR, skb, false); + WARN_ONCE(true, "offset (%d) >= skb_headlen() (%u)\n", + offset, skb_headlen(skb)); goto out; } csum = skb_checksum(skb, offset, skb->len - offset, 0); offset += skb->csum_offset; - if (WARN_ON_ONCE(offset + sizeof(__sum16) > skb_headlen(skb))) { + if (unlikely(offset + sizeof(__sum16) > skb_headlen(skb))) { DO_ONCE_LITE(skb_dump, KERN_ERR, skb, false); + WARN_ONCE(true, "offset+2 (%zu) > skb_headlen() (%u)\n", + offset + sizeof(__sum16), skb_headlen(skb)); goto out; } ret = skb_ensure_writable(skb, offset + sizeof(__sum16)); @@ -11033,7 +11062,9 @@ EXPORT_SYMBOL(unregister_netdev); int __dev_change_net_namespace(struct net_device *dev, struct net *net, const char *pat, int new_ifindex) { + struct netdev_name_node *name_node; struct net *net_old = dev_net(dev); + char new_name[IFNAMSIZ] = {}; int err, new_nsid; ASSERT_RTNL(); @@ -11060,10 +11091,15 @@ int __dev_change_net_namespace(struct net_device *dev, struct net *net, /* We get here if we can't use the current device name */ if (!pat) goto out; - err = dev_get_valid_name(net, dev, pat); + err = dev_prep_valid_name(net, dev, pat, new_name); if (err < 0) goto out; } + /* Check that none of the altnames conflicts. */ + err = -EEXIST; + netdev_for_each_altname(dev, name_node) + if (netdev_name_in_use(net, name_node->name)) + goto out; /* Check that new_ifindex isn't used yet. */ if (new_ifindex) { @@ -11131,6 +11167,9 @@ int __dev_change_net_namespace(struct net_device *dev, struct net *net, kobject_uevent(&dev->dev.kobj, KOBJ_ADD); netdev_adjacent_add_links(dev); + if (new_name[0]) /* Rename the netdev to prepared name */ + strscpy(dev->name, new_name, IFNAMSIZ); + /* Fixup kobjects */ err = device_rename(&dev->dev, dev->name); WARN_ON(err); diff --git a/net/core/dev.h b/net/core/dev.h index e075e198092c..fa2e9c5c4122 100644 --- a/net/core/dev.h +++ b/net/core/dev.h @@ -62,6 +62,9 @@ struct netdev_name_node { int netdev_get_name(struct net *net, char *name, int ifindex); int dev_change_name(struct net_device *dev, const char *newname); +#define netdev_for_each_altname(dev, namenode) \ + list_for_each_entry((namenode), &(dev)->name_node->list, list) + int netdev_name_node_alt_create(struct net_device *dev, const char *name); int netdev_name_node_alt_destroy(struct net_device *dev, const char *name); diff --git a/net/core/flow_dissector.c b/net/core/flow_dissector.c index b3b3af0e7844..272f09251343 100644 --- a/net/core/flow_dissector.c +++ b/net/core/flow_dissector.c @@ -1446,7 +1446,7 @@ proto_again: break; } - nhoff += ntohs(hdr->message_length); + nhoff += sizeof(struct ptp_header); fdret = FLOW_DISSECT_RET_OUT_GOOD; break; } diff --git a/net/core/neighbour.c b/net/core/neighbour.c index 6b76cd103195..9c09f091cbff 100644 --- a/net/core/neighbour.c +++ b/net/core/neighbour.c @@ -410,7 +410,7 @@ static void neigh_flush_dev(struct neigh_table *tbl, struct net_device *dev, */ __skb_queue_purge(&n->arp_queue); n->arp_queue_len_bytes = 0; - n->output = neigh_blackhole; + WRITE_ONCE(n->output, neigh_blackhole); if (n->nud_state & NUD_VALID) n->nud_state = NUD_NOARP; else @@ -920,7 +920,7 @@ static void neigh_suspect(struct neighbour *neigh) { neigh_dbg(2, "neigh %p is suspected\n", neigh); - neigh->output = neigh->ops->output; + WRITE_ONCE(neigh->output, neigh->ops->output); } /* Neighbour state is OK; @@ -932,7 +932,7 @@ static void neigh_connect(struct neighbour *neigh) { neigh_dbg(2, "neigh %p is connected\n", neigh); - neigh->output = neigh->ops->connected_output; + WRITE_ONCE(neigh->output, neigh->ops->connected_output); } static void neigh_periodic_work(struct work_struct *work) @@ -988,7 +988,9 @@ static void neigh_periodic_work(struct work_struct *work) (state == NUD_FAILED || !time_in_range_open(jiffies, n->used, n->used + NEIGH_VAR(n->parms, GC_STALETIME)))) { - *np = n->next; + rcu_assign_pointer(*np, + rcu_dereference_protected(n->next, + lockdep_is_held(&tbl->lock))); neigh_mark_dead(n); write_unlock(&n->lock); neigh_cleanup_and_release(n); @@ -1447,7 +1449,7 @@ static int __neigh_update(struct neighbour *neigh, const u8 *lladdr, if (n2) n1 = n2; } - n1->output(n1, skb); + READ_ONCE(n1->output)(n1, skb); if (n2) neigh_release(n2); rcu_read_unlock(); @@ -3153,7 +3155,7 @@ int neigh_xmit(int index, struct net_device *dev, rcu_read_unlock(); goto out_kfree_skb; } - err = neigh->output(neigh, skb); + err = READ_ONCE(neigh->output)(neigh, skb); rcu_read_unlock(); } else if (index == NEIGH_LINK_TABLE) { diff --git a/net/core/pktgen.c b/net/core/pktgen.c index f56b8d697014..4d1696677c48 100644 --- a/net/core/pktgen.c +++ b/net/core/pktgen.c @@ -669,19 +669,19 @@ static int pktgen_if_show(struct seq_file *seq, void *v) seq_puts(seq, " Flags: "); for (i = 0; i < NR_PKT_FLAGS; i++) { - if (i == F_FLOW_SEQ) + if (i == FLOW_SEQ_SHIFT) if (!pkt_dev->cflows) continue; - if (pkt_dev->flags & (1 << i)) + if (pkt_dev->flags & (1 << i)) { seq_printf(seq, "%s ", pkt_flag_names[i]); - else if (i == F_FLOW_SEQ) - seq_puts(seq, "FLOW_RND "); - #ifdef CONFIG_XFRM - if (i == F_IPSEC && pkt_dev->spi) - seq_printf(seq, "spi:%u", pkt_dev->spi); + if (i == IPSEC_SHIFT && pkt_dev->spi) + seq_printf(seq, "spi:%u ", pkt_dev->spi); #endif + } else if (i == FLOW_SEQ_SHIFT) { + seq_puts(seq, "FLOW_RND "); + } } seq_puts(seq, "\n"); diff --git a/net/core/rtnetlink.c b/net/core/rtnetlink.c index 4a2ec33bfb51..53c377d054f0 100644 --- a/net/core/rtnetlink.c +++ b/net/core/rtnetlink.c @@ -5503,13 +5503,11 @@ static unsigned int rtnl_offload_xstats_get_size_hw_s_info_one(const struct net_device *dev, enum netdev_offload_xstats_type type) { - bool enabled = netdev_offload_xstats_enabled(dev, type); - return nla_total_size(0) + /* IFLA_OFFLOAD_XSTATS_HW_S_INFO_REQUEST */ nla_total_size(sizeof(u8)) + /* IFLA_OFFLOAD_XSTATS_HW_S_INFO_USED */ - (enabled ? nla_total_size(sizeof(u8)) : 0) + + nla_total_size(sizeof(u8)) + 0; } diff --git a/net/core/sock_map.c b/net/core/sock_map.c index cb11750b1df5..4292c2ed1828 100644 --- a/net/core/sock_map.c +++ b/net/core/sock_map.c @@ -668,6 +668,8 @@ BPF_CALL_4(bpf_msg_redirect_map, struct sk_msg *, msg, sk = __sock_map_lookup_elem(map, key); if (unlikely(!sk || !sock_map_redirect_allowed(sk))) return SK_DROP; + if (!(flags & BPF_F_INGRESS) && !sk_is_tcp(sk)) + return SK_DROP; msg->flags = flags; msg->sk_redir = sk; @@ -1267,6 +1269,8 @@ BPF_CALL_4(bpf_msg_redirect_hash, struct sk_msg *, msg, sk = __sock_hash_lookup_elem(map, key); if (unlikely(!sk || !sock_map_redirect_allowed(sk))) return SK_DROP; + if (!(flags & BPF_F_INGRESS) && !sk_is_tcp(sk)) + return SK_DROP; msg->flags = flags; msg->sk_redir = sk; diff --git a/net/core/stream.c b/net/core/stream.c index f5c4e47df165..96fbcb9bbb30 100644 --- a/net/core/stream.c +++ b/net/core/stream.c @@ -117,7 +117,7 @@ EXPORT_SYMBOL(sk_stream_wait_close); */ int sk_stream_wait_memory(struct sock *sk, long *timeo_p) { - int err = 0; + int ret, err = 0; long vm_wait = 0; long current_timeo = *timeo_p; DEFINE_WAIT_FUNC(wait, woken_wake_function); @@ -142,11 +142,13 @@ int sk_stream_wait_memory(struct sock *sk, long *timeo_p) set_bit(SOCK_NOSPACE, &sk->sk_socket->flags); sk->sk_write_pending++; - sk_wait_event(sk, ¤t_timeo, READ_ONCE(sk->sk_err) || - (READ_ONCE(sk->sk_shutdown) & SEND_SHUTDOWN) || - (sk_stream_memory_free(sk) && - !vm_wait), &wait); + ret = sk_wait_event(sk, ¤t_timeo, READ_ONCE(sk->sk_err) || + (READ_ONCE(sk->sk_shutdown) & SEND_SHUTDOWN) || + (sk_stream_memory_free(sk) && !vm_wait), + &wait); sk->sk_write_pending--; + if (ret < 0) + goto do_error; if (vm_wait) { vm_wait -= current_timeo; diff --git a/net/dccp/ipv4.c b/net/dccp/ipv4.c index 8f56e8723c73..69453b936bd5 100644 --- a/net/dccp/ipv4.c +++ b/net/dccp/ipv4.c @@ -254,13 +254,8 @@ static int dccp_v4_err(struct sk_buff *skb, u32 info) int err; struct net *net = dev_net(skb->dev); - /* For the first __dccp_basic_hdr_len() check, we only need dh->dccph_x, - * which is in byte 7 of the dccp header. - * Our caller (icmp_socket_deliver()) already pulled 8 bytes for us. - * - * Later on, we want to access the sequence number fields, which are - * beyond 8 bytes, so we have to pskb_may_pull() ourselves. - */ + if (!pskb_may_pull(skb, offset + sizeof(*dh))) + return -EINVAL; dh = (struct dccp_hdr *)(skb->data + offset); if (!pskb_may_pull(skb, offset + __dccp_basic_hdr_len(dh))) return -EINVAL; diff --git a/net/dccp/ipv6.c b/net/dccp/ipv6.c index 33f6ccf6ba77..c693a570682f 100644 --- a/net/dccp/ipv6.c +++ b/net/dccp/ipv6.c @@ -83,13 +83,8 @@ static int dccp_v6_err(struct sk_buff *skb, struct inet6_skb_parm *opt, __u64 seq; struct net *net = dev_net(skb->dev); - /* For the first __dccp_basic_hdr_len() check, we only need dh->dccph_x, - * which is in byte 7 of the dccp header. - * Our caller (icmpv6_notify()) already pulled 8 bytes for us. - * - * Later on, we want to access the sequence number fields, which are - * beyond 8 bytes, so we have to pskb_may_pull() ourselves. - */ + if (!pskb_may_pull(skb, offset + sizeof(*dh))) + return -EINVAL; dh = (struct dccp_hdr *)(skb->data + offset); if (!pskb_may_pull(skb, offset + __dccp_basic_hdr_len(dh))) return -EINVAL; diff --git a/net/devlink/health.c b/net/devlink/health.c index 638cad8d5c65..51e6e81e31bb 100644 --- a/net/devlink/health.c +++ b/net/devlink/health.c @@ -58,7 +58,6 @@ struct devlink_health_reporter { struct devlink *devlink; struct devlink_port *devlink_port; struct devlink_fmsg *dump_fmsg; - struct mutex dump_lock; /* lock parallel read/write from dump buffers */ u64 graceful_period; bool auto_recover; bool auto_dump; @@ -125,7 +124,6 @@ __devlink_health_reporter_create(struct devlink *devlink, reporter->graceful_period = graceful_period; reporter->auto_recover = !!ops->recover; reporter->auto_dump = !!ops->dump; - mutex_init(&reporter->dump_lock); return reporter; } @@ -226,7 +224,6 @@ EXPORT_SYMBOL_GPL(devlink_health_reporter_create); static void devlink_health_reporter_free(struct devlink_health_reporter *reporter) { - mutex_destroy(&reporter->dump_lock); if (reporter->dump_fmsg) devlink_fmsg_free(reporter->dump_fmsg); kfree(reporter); @@ -625,10 +622,10 @@ int devlink_health_report(struct devlink_health_reporter *reporter, } if (reporter->auto_dump) { - mutex_lock(&reporter->dump_lock); + devl_lock(devlink); /* store current dump of current error, for later analysis */ devlink_health_do_dump(reporter, priv_ctx, NULL); - mutex_unlock(&reporter->dump_lock); + devl_unlock(devlink); } if (!reporter->auto_recover) @@ -1262,7 +1259,7 @@ out: } static struct devlink_health_reporter * -devlink_health_reporter_get_from_cb(struct netlink_callback *cb) +devlink_health_reporter_get_from_cb_lock(struct netlink_callback *cb) { const struct genl_info *info = genl_info_dump(cb); struct devlink_health_reporter *reporter; @@ -1272,10 +1269,12 @@ devlink_health_reporter_get_from_cb(struct netlink_callback *cb) devlink = devlink_get_from_attrs_lock(sock_net(cb->skb->sk), attrs); if (IS_ERR(devlink)) return NULL; - devl_unlock(devlink); reporter = devlink_health_reporter_get_from_attrs(devlink, attrs); - devlink_put(devlink); + if (!reporter) { + devl_unlock(devlink); + devlink_put(devlink); + } return reporter; } @@ -1284,16 +1283,20 @@ int devlink_nl_cmd_health_reporter_dump_get_dumpit(struct sk_buff *skb, { struct devlink_nl_dump_state *state = devlink_dump_state(cb); struct devlink_health_reporter *reporter; + struct devlink *devlink; int err; - reporter = devlink_health_reporter_get_from_cb(cb); + reporter = devlink_health_reporter_get_from_cb_lock(cb); if (!reporter) return -EINVAL; - if (!reporter->ops->dump) + devlink = reporter->devlink; + if (!reporter->ops->dump) { + devl_unlock(devlink); + devlink_put(devlink); return -EOPNOTSUPP; + } - mutex_lock(&reporter->dump_lock); if (!state->idx) { err = devlink_health_do_dump(reporter, NULL, cb->extack); if (err) @@ -1309,7 +1312,8 @@ int devlink_nl_cmd_health_reporter_dump_get_dumpit(struct sk_buff *skb, err = devlink_fmsg_dumpit(reporter->dump_fmsg, skb, cb, DEVLINK_CMD_HEALTH_REPORTER_DUMP_GET); unlock: - mutex_unlock(&reporter->dump_lock); + devl_unlock(devlink); + devlink_put(devlink); return err; } @@ -1326,9 +1330,7 @@ int devlink_nl_cmd_health_reporter_dump_clear_doit(struct sk_buff *skb, if (!reporter->ops->dump) return -EOPNOTSUPP; - mutex_lock(&reporter->dump_lock); devlink_health_dump_clear(reporter); - mutex_unlock(&reporter->dump_lock); return 0; } diff --git a/net/ethtool/plca.c b/net/ethtool/plca.c index b238a1afe9ae..b1e2e3b5027f 100644 --- a/net/ethtool/plca.c +++ b/net/ethtool/plca.c @@ -21,16 +21,6 @@ struct plca_reply_data { #define PLCA_REPDATA(__reply_base) \ container_of(__reply_base, struct plca_reply_data, base) -static void plca_update_sint(int *dst, const struct nlattr *attr, - bool *mod) -{ - if (!attr) - return; - - *dst = nla_get_u32(attr); - *mod = true; -} - // PLCA get configuration message ------------------------------------------- // const struct nla_policy ethnl_plca_get_cfg_policy[] = { @@ -38,6 +28,29 @@ const struct nla_policy ethnl_plca_get_cfg_policy[] = { NLA_POLICY_NESTED(ethnl_header_policy), }; +static void plca_update_sint(int *dst, struct nlattr **tb, u32 attrid, + bool *mod) +{ + const struct nlattr *attr = tb[attrid]; + + if (!attr || + WARN_ON_ONCE(attrid >= ARRAY_SIZE(ethnl_plca_set_cfg_policy))) + return; + + switch (ethnl_plca_set_cfg_policy[attrid].type) { + case NLA_U8: + *dst = nla_get_u8(attr); + break; + case NLA_U32: + *dst = nla_get_u32(attr); + break; + default: + WARN_ON_ONCE(1); + } + + *mod = true; +} + static int plca_get_cfg_prepare_data(const struct ethnl_req_info *req_base, struct ethnl_reply_data *reply_base, const struct genl_info *info) @@ -144,13 +157,13 @@ ethnl_set_plca(struct ethnl_req_info *req_info, struct genl_info *info) return -EOPNOTSUPP; memset(&plca_cfg, 0xff, sizeof(plca_cfg)); - plca_update_sint(&plca_cfg.enabled, tb[ETHTOOL_A_PLCA_ENABLED], &mod); - plca_update_sint(&plca_cfg.node_id, tb[ETHTOOL_A_PLCA_NODE_ID], &mod); - plca_update_sint(&plca_cfg.node_cnt, tb[ETHTOOL_A_PLCA_NODE_CNT], &mod); - plca_update_sint(&plca_cfg.to_tmr, tb[ETHTOOL_A_PLCA_TO_TMR], &mod); - plca_update_sint(&plca_cfg.burst_cnt, tb[ETHTOOL_A_PLCA_BURST_CNT], + plca_update_sint(&plca_cfg.enabled, tb, ETHTOOL_A_PLCA_ENABLED, &mod); + plca_update_sint(&plca_cfg.node_id, tb, ETHTOOL_A_PLCA_NODE_ID, &mod); + plca_update_sint(&plca_cfg.node_cnt, tb, ETHTOOL_A_PLCA_NODE_CNT, &mod); + plca_update_sint(&plca_cfg.to_tmr, tb, ETHTOOL_A_PLCA_TO_TMR, &mod); + plca_update_sint(&plca_cfg.burst_cnt, tb, ETHTOOL_A_PLCA_BURST_CNT, &mod); - plca_update_sint(&plca_cfg.burst_tmr, tb[ETHTOOL_A_PLCA_BURST_TMR], + plca_update_sint(&plca_cfg.burst_tmr, tb, ETHTOOL_A_PLCA_BURST_TMR, &mod); if (!mod) return 0; diff --git a/net/handshake/handshake-test.c b/net/handshake/handshake-test.c index 6d37bab35c8f..16ed7bfd29e4 100644 --- a/net/handshake/handshake-test.c +++ b/net/handshake/handshake-test.c @@ -235,7 +235,7 @@ static void handshake_req_submit_test4(struct kunit *test) KUNIT_EXPECT_PTR_EQ(test, req, result); handshake_req_cancel(sock->sk); - sock_release(sock); + fput(filp); } static void handshake_req_submit_test5(struct kunit *test) @@ -272,7 +272,7 @@ static void handshake_req_submit_test5(struct kunit *test) /* Assert */ KUNIT_EXPECT_EQ(test, err, -EAGAIN); - sock_release(sock); + fput(filp); hn->hn_pending = saved; } @@ -306,7 +306,7 @@ static void handshake_req_submit_test6(struct kunit *test) KUNIT_EXPECT_EQ(test, err, -EBUSY); handshake_req_cancel(sock->sk); - sock_release(sock); + fput(filp); } static void handshake_req_cancel_test1(struct kunit *test) @@ -340,7 +340,7 @@ static void handshake_req_cancel_test1(struct kunit *test) /* Assert */ KUNIT_EXPECT_TRUE(test, result); - sock_release(sock); + fput(filp); } static void handshake_req_cancel_test2(struct kunit *test) @@ -382,7 +382,7 @@ static void handshake_req_cancel_test2(struct kunit *test) /* Assert */ KUNIT_EXPECT_TRUE(test, result); - sock_release(sock); + fput(filp); } static void handshake_req_cancel_test3(struct kunit *test) @@ -427,7 +427,7 @@ static void handshake_req_cancel_test3(struct kunit *test) /* Assert */ KUNIT_EXPECT_FALSE(test, result); - sock_release(sock); + fput(filp); } static struct handshake_req *handshake_req_destroy_test; @@ -471,7 +471,7 @@ static void handshake_req_destroy_test1(struct kunit *test) handshake_req_cancel(sock->sk); /* Act */ - sock_release(sock); + fput(filp); /* Assert */ KUNIT_EXPECT_PTR_EQ(test, handshake_req_destroy_test, req); diff --git a/net/hsr/hsr_forward.c b/net/hsr/hsr_forward.c index 629daacc9607..b71dab630a87 100644 --- a/net/hsr/hsr_forward.c +++ b/net/hsr/hsr_forward.c @@ -594,6 +594,7 @@ static int fill_frame_info(struct hsr_frame_info *frame, proto = vlan_hdr->vlanhdr.h_vlan_encapsulated_proto; /* FIXME: */ netdev_warn_once(skb->dev, "VLAN not yet supported"); + return -EINVAL; } frame->is_from_san = false; diff --git a/net/hsr/hsr_framereg.c b/net/hsr/hsr_framereg.c index b77f1189d19d..6d14d935ee82 100644 --- a/net/hsr/hsr_framereg.c +++ b/net/hsr/hsr_framereg.c @@ -288,13 +288,13 @@ void hsr_handle_sup_frame(struct hsr_frame_info *frame) /* And leave the HSR tag. */ if (ethhdr->h_proto == htons(ETH_P_HSR)) { - pull_size = sizeof(struct ethhdr); + pull_size = sizeof(struct hsr_tag); skb_pull(skb, pull_size); total_pull_size += pull_size; } /* And leave the HSR sup tag. */ - pull_size = sizeof(struct hsr_tag); + pull_size = sizeof(struct hsr_sup_tag); skb_pull(skb, pull_size); total_pull_size += pull_size; diff --git a/net/hsr/hsr_main.h b/net/hsr/hsr_main.h index 6851e33df7d1..18e01791ad79 100644 --- a/net/hsr/hsr_main.h +++ b/net/hsr/hsr_main.h @@ -83,7 +83,7 @@ struct hsr_vlan_ethhdr { struct hsr_sup_tlv { u8 HSR_TLV_type; u8 HSR_TLV_length; -}; +} __packed; /* HSR/PRP Supervision Frame data types. * Field names as defined in the IEC:2010 standard for HSR. diff --git a/net/ipv4/af_inet.c b/net/ipv4/af_inet.c index 3d2e30e20473..2713c9b06c4c 100644 --- a/net/ipv4/af_inet.c +++ b/net/ipv4/af_inet.c @@ -597,7 +597,6 @@ static long inet_wait_for_connect(struct sock *sk, long timeo, int writebias) add_wait_queue(sk_sleep(sk), &wait); sk->sk_write_pending += writebias; - sk->sk_wait_pending++; /* Basic assumption: if someone sets sk->sk_err, he _must_ * change state of the socket from TCP_SYN_*. @@ -613,7 +612,6 @@ static long inet_wait_for_connect(struct sock *sk, long timeo, int writebias) } remove_wait_queue(sk_sleep(sk), &wait); sk->sk_write_pending -= writebias; - sk->sk_wait_pending--; return timeo; } @@ -642,6 +640,7 @@ int __inet_stream_connect(struct socket *sock, struct sockaddr *uaddr, return -EINVAL; if (uaddr->sa_family == AF_UNSPEC) { + sk->sk_disconnects++; err = sk->sk_prot->disconnect(sk, flags); sock->state = err ? SS_DISCONNECTING : SS_UNCONNECTED; goto out; @@ -696,6 +695,7 @@ int __inet_stream_connect(struct socket *sock, struct sockaddr *uaddr, int writebias = (sk->sk_protocol == IPPROTO_TCP) && tcp_sk(sk)->fastopen_req && tcp_sk(sk)->fastopen_req->data ? 1 : 0; + int dis = sk->sk_disconnects; /* Error code is set above */ if (!timeo || !inet_wait_for_connect(sk, timeo, writebias)) @@ -704,6 +704,11 @@ int __inet_stream_connect(struct socket *sock, struct sockaddr *uaddr, err = sock_intr_errno(timeo); if (signal_pending(current)) goto out; + + if (dis != sk->sk_disconnects) { + err = -EPIPE; + goto out; + } } /* Connection was closed by RST, timeout, ICMP error @@ -725,6 +730,7 @@ out: sock_error: err = sock_error(sk) ? : -ECONNABORTED; sock->state = SS_UNCONNECTED; + sk->sk_disconnects++; if (sk->sk_prot->disconnect(sk, flags)) sock->state = SS_DISCONNECTING; goto out; diff --git a/net/ipv4/devinet.c b/net/ipv4/devinet.c index 9cf64ee47dd2..ca0ff15dc8fa 100644 --- a/net/ipv4/devinet.c +++ b/net/ipv4/devinet.c @@ -355,14 +355,14 @@ static void __inet_del_ifa(struct in_device *in_dev, { struct in_ifaddr *promote = NULL; struct in_ifaddr *ifa, *ifa1; - struct in_ifaddr *last_prim; + struct in_ifaddr __rcu **last_prim; struct in_ifaddr *prev_prom = NULL; int do_promote = IN_DEV_PROMOTE_SECONDARIES(in_dev); ASSERT_RTNL(); ifa1 = rtnl_dereference(*ifap); - last_prim = rtnl_dereference(in_dev->ifa_list); + last_prim = ifap; if (in_dev->dead) goto no_promotions; @@ -376,7 +376,7 @@ static void __inet_del_ifa(struct in_device *in_dev, while ((ifa = rtnl_dereference(*ifap1)) != NULL) { if (!(ifa->ifa_flags & IFA_F_SECONDARY) && ifa1->ifa_scope <= ifa->ifa_scope) - last_prim = ifa; + last_prim = &ifa->ifa_next; if (!(ifa->ifa_flags & IFA_F_SECONDARY) || ifa1->ifa_mask != ifa->ifa_mask || @@ -440,9 +440,9 @@ no_promotions: rcu_assign_pointer(prev_prom->ifa_next, next_sec); - last_sec = rtnl_dereference(last_prim->ifa_next); + last_sec = rtnl_dereference(*last_prim); rcu_assign_pointer(promote->ifa_next, last_sec); - rcu_assign_pointer(last_prim->ifa_next, promote); + rcu_assign_pointer(*last_prim, promote); } promote->ifa_flags &= ~IFA_F_SECONDARY; diff --git a/net/ipv4/esp4.c b/net/ipv4/esp4.c index 2be2d4922557..d18f0f092fe7 100644 --- a/net/ipv4/esp4.c +++ b/net/ipv4/esp4.c @@ -732,7 +732,9 @@ static inline int esp_remove_trailer(struct sk_buff *skb) skb->csum = csum_block_sub(skb->csum, csumdiff, skb->len - trimlen); } - pskb_trim(skb, skb->len - trimlen); + ret = pskb_trim(skb, skb->len - trimlen); + if (unlikely(ret)) + return ret; ret = nexthdr[1]; diff --git a/net/ipv4/fib_semantics.c b/net/ipv4/fib_semantics.c index eafa4a033515..5eb1b8d302bb 100644 --- a/net/ipv4/fib_semantics.c +++ b/net/ipv4/fib_semantics.c @@ -1325,15 +1325,18 @@ __be32 fib_info_update_nhc_saddr(struct net *net, struct fib_nh_common *nhc, unsigned char scope) { struct fib_nh *nh; + __be32 saddr; if (nhc->nhc_family != AF_INET) return inet_select_addr(nhc->nhc_dev, 0, scope); nh = container_of(nhc, struct fib_nh, nh_common); - nh->nh_saddr = inet_select_addr(nh->fib_nh_dev, nh->fib_nh_gw4, scope); - nh->nh_saddr_genid = atomic_read(&net->ipv4.dev_addr_genid); + saddr = inet_select_addr(nh->fib_nh_dev, nh->fib_nh_gw4, scope); - return nh->nh_saddr; + WRITE_ONCE(nh->nh_saddr, saddr); + WRITE_ONCE(nh->nh_saddr_genid, atomic_read(&net->ipv4.dev_addr_genid)); + + return saddr; } __be32 fib_result_prefsrc(struct net *net, struct fib_result *res) @@ -1347,8 +1350,9 @@ __be32 fib_result_prefsrc(struct net *net, struct fib_result *res) struct fib_nh *nh; nh = container_of(nhc, struct fib_nh, nh_common); - if (nh->nh_saddr_genid == atomic_read(&net->ipv4.dev_addr_genid)) - return nh->nh_saddr; + if (READ_ONCE(nh->nh_saddr_genid) == + atomic_read(&net->ipv4.dev_addr_genid)) + return READ_ONCE(nh->nh_saddr); } return fib_info_update_nhc_saddr(net, nhc, res->fi->fib_scope); @@ -1887,6 +1891,7 @@ int fib_sync_down_addr(struct net_device *dev, __be32 local) continue; if (fi->fib_prefsrc == local) { fi->fib_flags |= RTNH_F_DEAD; + fi->pfsrc_removed = true; ret++; } } diff --git a/net/ipv4/fib_trie.c b/net/ipv4/fib_trie.c index d13fb9e76b97..9bdfdab906fe 100644 --- a/net/ipv4/fib_trie.c +++ b/net/ipv4/fib_trie.c @@ -2027,6 +2027,7 @@ void fib_table_flush_external(struct fib_table *tb) int fib_table_flush(struct net *net, struct fib_table *tb, bool flush_all) { struct trie *t = (struct trie *)tb->tb_data; + struct nl_info info = { .nl_net = net }; struct key_vector *pn = t->kv; unsigned long cindex = 1; struct hlist_node *tmp; @@ -2089,6 +2090,9 @@ int fib_table_flush(struct net *net, struct fib_table *tb, bool flush_all) fib_notify_alias_delete(net, n->key, &n->leaf, fa, NULL); + if (fi->pfsrc_removed) + rtmsg_fib(RTM_DELROUTE, htonl(n->key), fa, + KEYLENGTH - fa->fa_slen, tb->tb_id, &info, 0); hlist_del_rcu(&fa->fa_list); fib_release_info(fa->fa_info); alias_free_mem_rcu(fa); diff --git a/net/ipv4/inet_connection_sock.c b/net/ipv4/inet_connection_sock.c index aeebe8816689..394a498c2823 100644 --- a/net/ipv4/inet_connection_sock.c +++ b/net/ipv4/inet_connection_sock.c @@ -1145,7 +1145,6 @@ struct sock *inet_csk_clone_lock(const struct sock *sk, if (newsk) { struct inet_connection_sock *newicsk = inet_csk(newsk); - newsk->sk_wait_pending = 0; inet_sk_set_state(newsk, TCP_SYN_RECV); newicsk->icsk_bind_hash = NULL; newicsk->icsk_bind2_hash = NULL; diff --git a/net/ipv4/inet_hashtables.c b/net/ipv4/inet_hashtables.c index 7876b7d703cb..598c1b114d2c 100644 --- a/net/ipv4/inet_hashtables.c +++ b/net/ipv4/inet_hashtables.c @@ -149,8 +149,14 @@ static bool inet_bind2_bucket_addr_match(const struct inet_bind2_bucket *tb2, const struct sock *sk) { #if IS_ENABLED(CONFIG_IPV6) - if (sk->sk_family != tb2->family) - return false; + if (sk->sk_family != tb2->family) { + if (sk->sk_family == AF_INET) + return ipv6_addr_v4mapped(&tb2->v6_rcv_saddr) && + tb2->v6_rcv_saddr.s6_addr32[3] == sk->sk_rcv_saddr; + + return ipv6_addr_v4mapped(&sk->sk_v6_rcv_saddr) && + sk->sk_v6_rcv_saddr.s6_addr32[3] == tb2->rcv_saddr; + } if (sk->sk_family == AF_INET6) return ipv6_addr_equal(&tb2->v6_rcv_saddr, @@ -815,41 +821,33 @@ static bool inet_bind2_bucket_match(const struct inet_bind2_bucket *tb, const struct net *net, unsigned short port, int l3mdev, const struct sock *sk) { -#if IS_ENABLED(CONFIG_IPV6) - if (sk->sk_family != tb->family) + if (!net_eq(ib2_net(tb), net) || tb->port != port || + tb->l3mdev != l3mdev) return false; - if (sk->sk_family == AF_INET6) - return net_eq(ib2_net(tb), net) && tb->port == port && - tb->l3mdev == l3mdev && - ipv6_addr_equal(&tb->v6_rcv_saddr, &sk->sk_v6_rcv_saddr); - else -#endif - return net_eq(ib2_net(tb), net) && tb->port == port && - tb->l3mdev == l3mdev && tb->rcv_saddr == sk->sk_rcv_saddr; + return inet_bind2_bucket_addr_match(tb, sk); } bool inet_bind2_bucket_match_addr_any(const struct inet_bind2_bucket *tb, const struct net *net, unsigned short port, int l3mdev, const struct sock *sk) { + if (!net_eq(ib2_net(tb), net) || tb->port != port || + tb->l3mdev != l3mdev) + return false; + #if IS_ENABLED(CONFIG_IPV6) if (sk->sk_family != tb->family) { if (sk->sk_family == AF_INET) - return net_eq(ib2_net(tb), net) && tb->port == port && - tb->l3mdev == l3mdev && - ipv6_addr_any(&tb->v6_rcv_saddr); + return ipv6_addr_any(&tb->v6_rcv_saddr) || + ipv6_addr_v4mapped_any(&tb->v6_rcv_saddr); return false; } if (sk->sk_family == AF_INET6) - return net_eq(ib2_net(tb), net) && tb->port == port && - tb->l3mdev == l3mdev && - ipv6_addr_any(&tb->v6_rcv_saddr); - else + return ipv6_addr_any(&tb->v6_rcv_saddr); #endif - return net_eq(ib2_net(tb), net) && tb->port == port && - tb->l3mdev == l3mdev && tb->rcv_saddr == 0; + return tb->rcv_saddr == 0; } /* The socket's bhash2 hashbucket spinlock must be held when this is called */ diff --git a/net/ipv4/route.c b/net/ipv4/route.c index 66f419e7f9a7..b214b5a2e045 100644 --- a/net/ipv4/route.c +++ b/net/ipv4/route.c @@ -1213,6 +1213,7 @@ EXPORT_INDIRECT_CALLABLE(ipv4_dst_check); static void ipv4_send_dest_unreach(struct sk_buff *skb) { + struct net_device *dev; struct ip_options opt; int res; @@ -1230,7 +1231,8 @@ static void ipv4_send_dest_unreach(struct sk_buff *skb) opt.optlen = ip_hdr(skb)->ihl * 4 - sizeof(struct iphdr); rcu_read_lock(); - res = __ip_options_compile(dev_net(skb->dev), &opt, skb, NULL); + dev = skb->dev ? skb->dev : skb_rtable(skb)->dst.dev; + res = __ip_options_compile(dev_net(dev), &opt, skb, NULL); rcu_read_unlock(); if (res) @@ -3415,6 +3417,8 @@ static int inet_rtm_getroute(struct sk_buff *in_skb, struct nlmsghdr *nlh, fa->fa_type == fri.type) { fri.offload = READ_ONCE(fa->offload); fri.trap = READ_ONCE(fa->trap); + fri.offload_failed = + READ_ONCE(fa->offload_failed); break; } } diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c index 0c3040a63ebd..d3456cf840de 100644 --- a/net/ipv4/tcp.c +++ b/net/ipv4/tcp.c @@ -831,7 +831,9 @@ ssize_t tcp_splice_read(struct socket *sock, loff_t *ppos, */ if (!skb_queue_empty(&sk->sk_receive_queue)) break; - sk_wait_data(sk, &timeo, NULL); + ret = sk_wait_data(sk, &timeo, NULL); + if (ret < 0) + break; if (signal_pending(current)) { ret = sock_intr_errno(timeo); break; @@ -1621,16 +1623,13 @@ EXPORT_SYMBOL(tcp_read_sock); int tcp_read_skb(struct sock *sk, skb_read_actor_t recv_actor) { - struct tcp_sock *tp = tcp_sk(sk); - u32 seq = tp->copied_seq; struct sk_buff *skb; int copied = 0; - u32 offset; if (sk->sk_state == TCP_LISTEN) return -ENOTCONN; - while ((skb = tcp_recv_skb(sk, seq, &offset)) != NULL) { + while ((skb = skb_peek(&sk->sk_receive_queue)) != NULL) { u8 tcp_flags; int used; @@ -1643,13 +1642,10 @@ int tcp_read_skb(struct sock *sk, skb_read_actor_t recv_actor) copied = used; break; } - seq += used; copied += used; - if (tcp_flags & TCPHDR_FIN) { - ++seq; + if (tcp_flags & TCPHDR_FIN) break; - } } return copied; } @@ -2448,7 +2444,11 @@ static int tcp_recvmsg_locked(struct sock *sk, struct msghdr *msg, size_t len, __sk_flush_backlog(sk); } else { tcp_cleanup_rbuf(sk, copied); - sk_wait_data(sk, &timeo, last); + err = sk_wait_data(sk, &timeo, last); + if (err < 0) { + err = copied ? : err; + goto out; + } } if ((flags & MSG_PEEK) && @@ -2972,12 +2972,6 @@ int tcp_disconnect(struct sock *sk, int flags) int old_state = sk->sk_state; u32 seq; - /* Deny disconnect if other threads are blocked in sk_wait_event() - * or inet_wait_for_connect(). - */ - if (sk->sk_wait_pending) - return -EBUSY; - if (old_state != TCP_CLOSE) tcp_set_state(sk, TCP_CLOSE); diff --git a/net/ipv4/tcp_bpf.c b/net/ipv4/tcp_bpf.c index 81f0dff69e0b..53b0d62fd2c2 100644 --- a/net/ipv4/tcp_bpf.c +++ b/net/ipv4/tcp_bpf.c @@ -222,6 +222,7 @@ static int tcp_bpf_recvmsg_parser(struct sock *sk, int *addr_len) { struct tcp_sock *tcp = tcp_sk(sk); + int peek = flags & MSG_PEEK; u32 seq = tcp->copied_seq; struct sk_psock *psock; int copied = 0; @@ -306,15 +307,22 @@ msg_bytes_ready: } data = tcp_msg_wait_data(sk, psock, timeo); + if (data < 0) { + copied = data; + goto unlock; + } if (data && !sk_psock_queue_empty(psock)) goto msg_bytes_ready; copied = -EAGAIN; } out: - WRITE_ONCE(tcp->copied_seq, seq); + if (!peek) + WRITE_ONCE(tcp->copied_seq, seq); tcp_rcv_space_adjust(sk); if (copied > 0) __tcp_cleanup_rbuf(sk, copied); + +unlock: release_sock(sk); sk_psock_put(sk, psock); return copied; @@ -349,6 +357,10 @@ msg_bytes_ready: timeo = sock_rcvtimeo(sk, flags & MSG_DONTWAIT); data = tcp_msg_wait_data(sk, psock, timeo); + if (data < 0) { + ret = data; + goto unlock; + } if (data) { if (!sk_psock_queue_empty(psock)) goto msg_bytes_ready; @@ -359,6 +371,8 @@ msg_bytes_ready: copied = -EAGAIN; } ret = copied; + +unlock: release_sock(sk); sk_psock_put(sk, psock); return ret; diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c index 06fe1cf645d5..8afb0950a697 100644 --- a/net/ipv4/tcp_input.c +++ b/net/ipv4/tcp_input.c @@ -253,6 +253,19 @@ static void tcp_measure_rcv_mss(struct sock *sk, const struct sk_buff *skb) if (unlikely(len > icsk->icsk_ack.rcv_mss + MAX_TCP_OPTION_SPACE)) tcp_gro_dev_warn(sk, skb, len); + /* If the skb has a len of exactly 1*MSS and has the PSH bit + * set then it is likely the end of an application write. So + * more data may not be arriving soon, and yet the data sender + * may be waiting for an ACK if cwnd-bound or using TX zero + * copy. So we set ICSK_ACK_PUSHED here so that + * tcp_cleanup_rbuf() will send an ACK immediately if the app + * reads all of the data and is not ping-pong. If len > MSS + * then this logic does not matter (and does not hurt) because + * tcp_cleanup_rbuf() will always ACK immediately if the app + * reads data and there is more than an MSS of unACKed data. + */ + if (TCP_SKB_CB(skb)->tcp_flags & TCPHDR_PSH) + icsk->icsk_ack.pending |= ICSK_ACK_PUSHED; } else { /* Otherwise, we make more careful check taking into account, * that SACKs block is variable. diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c index 27140e5cdc06..4167e8a48b60 100644 --- a/net/ipv4/tcp_ipv4.c +++ b/net/ipv4/tcp_ipv4.c @@ -1869,6 +1869,7 @@ bool tcp_add_backlog(struct sock *sk, struct sk_buff *skb, #ifdef CONFIG_TLS_DEVICE tail->decrypted != skb->decrypted || #endif + !mptcp_skb_can_collapse(tail, skb) || thtail->doff != th->doff || memcmp(thtail + 1, th + 1, hdrlen - sizeof(*th))) goto no_coalesce; diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c index ccfc8bbf7455..f0723460753c 100644 --- a/net/ipv4/tcp_output.c +++ b/net/ipv4/tcp_output.c @@ -177,8 +177,7 @@ static void tcp_event_data_sent(struct tcp_sock *tp, } /* Account for an ACK we sent. */ -static inline void tcp_event_ack_sent(struct sock *sk, unsigned int pkts, - u32 rcv_nxt) +static inline void tcp_event_ack_sent(struct sock *sk, u32 rcv_nxt) { struct tcp_sock *tp = tcp_sk(sk); @@ -192,7 +191,7 @@ static inline void tcp_event_ack_sent(struct sock *sk, unsigned int pkts, if (unlikely(rcv_nxt != tp->rcv_nxt)) return; /* Special ACK sent by DCTCP to reflect ECN */ - tcp_dec_quickack_mode(sk, pkts); + tcp_dec_quickack_mode(sk); inet_csk_clear_xmit_timer(sk, ICSK_TIME_DACK); } @@ -1387,7 +1386,7 @@ static int __tcp_transmit_skb(struct sock *sk, struct sk_buff *skb, sk, skb); if (likely(tcb->tcp_flags & TCPHDR_ACK)) - tcp_event_ack_sent(sk, tcp_skb_pcount(skb), rcv_nxt); + tcp_event_ack_sent(sk, rcv_nxt); if (skb->len != tcp_header_size) { tcp_event_data_sent(tp, sk); @@ -2457,6 +2456,7 @@ static int tcp_mtu_probe(struct sock *sk) /* build the payload, and be prepared to abort if this fails. */ if (tcp_clone_payload(sk, nskb, probe_size)) { + tcp_skb_tsorted_anchor_cleanup(nskb); consume_skb(nskb); return -1; } @@ -2542,6 +2542,18 @@ static bool tcp_pacing_check(struct sock *sk) return true; } +static bool tcp_rtx_queue_empty_or_single_skb(const struct sock *sk) +{ + const struct rb_node *node = sk->tcp_rtx_queue.rb_node; + + /* No skb in the rtx queue. */ + if (!node) + return true; + + /* Only one skb in rtx queue. */ + return !node->rb_left && !node->rb_right; +} + /* TCP Small Queues : * Control number of packets in qdisc/devices to two packets / or ~1 ms. * (These limits are doubled for retransmits) @@ -2579,12 +2591,12 @@ static bool tcp_small_queue_check(struct sock *sk, const struct sk_buff *skb, limit += extra_bytes; } if (refcount_read(&sk->sk_wmem_alloc) > limit) { - /* Always send skb if rtx queue is empty. + /* Always send skb if rtx queue is empty or has one skb. * No need to wait for TX completion to call us back, * after softirq/tasklet schedule. * This helps when TX completions are delayed too much. */ - if (tcp_rtx_queue_empty(sk)) + if (tcp_rtx_queue_empty_or_single_skb(sk)) return false; set_bit(TSQ_THROTTLED, &sk->sk_tsq_flags); @@ -2788,7 +2800,7 @@ bool tcp_schedule_loss_probe(struct sock *sk, bool advancing_rto) { struct inet_connection_sock *icsk = inet_csk(sk); struct tcp_sock *tp = tcp_sk(sk); - u32 timeout, rto_delta_us; + u32 timeout, timeout_us, rto_delta_us; int early_retrans; /* Don't do any loss probe on a Fast Open connection before 3WHS @@ -2812,11 +2824,12 @@ bool tcp_schedule_loss_probe(struct sock *sk, bool advancing_rto) * sample is available then probe after TCP_TIMEOUT_INIT. */ if (tp->srtt_us) { - timeout = usecs_to_jiffies(tp->srtt_us >> 2); + timeout_us = tp->srtt_us >> 2; if (tp->packets_out == 1) - timeout += TCP_RTO_MIN; + timeout_us += tcp_rto_min_us(sk); else - timeout += TCP_TIMEOUT_MIN; + timeout_us += TCP_TIMEOUT_MIN_US; + timeout = usecs_to_jiffies(timeout_us); } else { timeout = TCP_TIMEOUT_INIT; } diff --git a/net/ipv4/tcp_recovery.c b/net/ipv4/tcp_recovery.c index acf4869c5d3b..bba10110fbbc 100644 --- a/net/ipv4/tcp_recovery.c +++ b/net/ipv4/tcp_recovery.c @@ -104,7 +104,7 @@ bool tcp_rack_mark_lost(struct sock *sk) tp->rack.advanced = 0; tcp_rack_detect_loss(sk, &timeout); if (timeout) { - timeout = usecs_to_jiffies(timeout) + TCP_TIMEOUT_MIN; + timeout = usecs_to_jiffies(timeout + TCP_TIMEOUT_MIN_US); inet_csk_reset_xmit_timer(sk, ICSK_TIME_REO_TIMEOUT, timeout, inet_csk(sk)->icsk_rto); } diff --git a/net/ipv6/esp6.c b/net/ipv6/esp6.c index fddd0cbdede1..e023d29e919c 100644 --- a/net/ipv6/esp6.c +++ b/net/ipv6/esp6.c @@ -770,7 +770,9 @@ static inline int esp_remove_trailer(struct sk_buff *skb) skb->csum = csum_block_sub(skb->csum, csumdiff, skb->len - trimlen); } - pskb_trim(skb, skb->len - trimlen); + ret = pskb_trim(skb, skb->len - trimlen); + if (unlikely(ret)) + return ret; ret = nexthdr[1]; diff --git a/net/ipv6/tcp_ipv6.c b/net/ipv6/tcp_ipv6.c index 3a88545a265d..44b6949d72b2 100644 --- a/net/ipv6/tcp_ipv6.c +++ b/net/ipv6/tcp_ipv6.c @@ -1640,9 +1640,12 @@ process: struct sock *nsk; sk = req->rsk_listener; - drop_reason = tcp_inbound_md5_hash(sk, skb, - &hdr->saddr, &hdr->daddr, - AF_INET6, dif, sdif); + if (!xfrm6_policy_check(sk, XFRM_POLICY_IN, skb)) + drop_reason = SKB_DROP_REASON_XFRM_POLICY; + else + drop_reason = tcp_inbound_md5_hash(sk, skb, + &hdr->saddr, &hdr->daddr, + AF_INET6, dif, sdif); if (drop_reason) { sk_drops_add(sk, skb); reqsk_put(req); @@ -1689,6 +1692,7 @@ process: } goto discard_and_relse; } + nf_reset_ct(skb); if (nsk == sk) { reqsk_put(req); tcp_v6_restore_cb(skb); diff --git a/net/ipv6/xfrm6_policy.c b/net/ipv6/xfrm6_policy.c index 41a680c76d2e..42fb6996b077 100644 --- a/net/ipv6/xfrm6_policy.c +++ b/net/ipv6/xfrm6_policy.c @@ -117,10 +117,10 @@ static void xfrm6_dst_destroy(struct dst_entry *dst) { struct xfrm_dst *xdst = (struct xfrm_dst *)dst; - if (likely(xdst->u.rt6.rt6i_idev)) - in6_dev_put(xdst->u.rt6.rt6i_idev); dst_destroy_metrics_generic(dst); rt6_uncached_list_del(&xdst->u.rt6); + if (likely(xdst->u.rt6.rt6i_idev)) + in6_dev_put(xdst->u.rt6.rt6i_idev); xfrm_dst_destroy(xdst); } diff --git a/net/kcm/kcmsock.c b/net/kcm/kcmsock.c index 4580f61426bb..dd1d8ffd5f59 100644 --- a/net/kcm/kcmsock.c +++ b/net/kcm/kcmsock.c @@ -930,15 +930,18 @@ partial_message: out_error: kcm_push(kcm); - if (copied && sock->type == SOCK_SEQPACKET) { + if (sock->type == SOCK_SEQPACKET) { /* Wrote some bytes before encountering an * error, return partial success. */ - goto partial_message; - } - - if (head != kcm->seq_skb) + if (copied) + goto partial_message; + if (head != kcm->seq_skb) + kfree_skb(head); + } else { kfree_skb(head); + kcm->seq_skb = NULL; + } err = sk_stream_error(sk, msg->msg_flags, err); diff --git a/net/l2tp/l2tp_ip6.c b/net/l2tp/l2tp_ip6.c index ed8ebb6f5909..11f3d375cec0 100644 --- a/net/l2tp/l2tp_ip6.c +++ b/net/l2tp/l2tp_ip6.c @@ -507,7 +507,6 @@ static int l2tp_ip6_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) */ if (len > INT_MAX - transhdrlen) return -EMSGSIZE; - ulen = len + transhdrlen; /* Mirror BSD error message compatibility */ if (msg->msg_flags & MSG_OOB) @@ -628,6 +627,7 @@ static int l2tp_ip6_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) back_from_confirm: lock_sock(sk); + ulen = len + skb_queue_empty(&sk->sk_write_queue) ? transhdrlen : 0; err = ip6_append_data(sk, ip_generic_getfrag, msg, ulen, transhdrlen, &ipc6, &fl6, (struct rt6_info *)dst, diff --git a/net/mac80211/cfg.c b/net/mac80211/cfg.c index 45e7a5d9c7d9..0e3a1753a51c 100644 --- a/net/mac80211/cfg.c +++ b/net/mac80211/cfg.c @@ -566,6 +566,9 @@ static int ieee80211_add_key(struct wiphy *wiphy, struct net_device *dev, } err = ieee80211_key_link(key, link, sta); + /* KRACK protection, shouldn't happen but just silently accept key */ + if (err == -EALREADY) + err = 0; out_unlock: mutex_unlock(&local->sta_mtx); @@ -1857,7 +1860,8 @@ static int sta_link_apply_parameters(struct ieee80211_local *local, /* VHT can override some HT caps such as the A-MSDU max length */ if (params->vht_capa) ieee80211_vht_cap_ie_to_sta_vht_cap(sdata, sband, - params->vht_capa, link_sta); + params->vht_capa, NULL, + link_sta); if (params->he_capa) ieee80211_he_cap_ie_to_sta_he_cap(sdata, sband, diff --git a/net/mac80211/ibss.c b/net/mac80211/ibss.c index e1900077bc4b..5542c93edfba 100644 --- a/net/mac80211/ibss.c +++ b/net/mac80211/ibss.c @@ -1072,7 +1072,7 @@ static void ieee80211_update_sta_info(struct ieee80211_sub_if_data *sdata, &chandef); memcpy(&cap_ie, elems->vht_cap_elem, sizeof(cap_ie)); ieee80211_vht_cap_ie_to_sta_vht_cap(sdata, sband, - &cap_ie, + &cap_ie, NULL, &sta->deflink); if (memcmp(&cap, &sta->sta.deflink.vht_cap, sizeof(cap))) rates_updated |= true; diff --git a/net/mac80211/ieee80211_i.h b/net/mac80211/ieee80211_i.h index 06bd406846d2..98ef1fe1226e 100644 --- a/net/mac80211/ieee80211_i.h +++ b/net/mac80211/ieee80211_i.h @@ -676,7 +676,7 @@ struct ieee80211_if_mesh { struct timer_list mesh_path_root_timer; unsigned long wrkq_flags; - unsigned long mbss_changed; + unsigned long mbss_changed[64 / BITS_PER_LONG]; bool userspace_handles_dfs; @@ -2141,6 +2141,7 @@ void ieee80211_vht_cap_ie_to_sta_vht_cap(struct ieee80211_sub_if_data *sdata, struct ieee80211_supported_band *sband, const struct ieee80211_vht_cap *vht_cap_ie, + const struct ieee80211_vht_cap *vht_cap_ie2, struct link_sta_info *link_sta); enum ieee80211_sta_rx_bandwidth ieee80211_sta_cap_rx_bw(struct link_sta_info *link_sta); diff --git a/net/mac80211/key.c b/net/mac80211/key.c index 13050dc9321f..a2db0585dce0 100644 --- a/net/mac80211/key.c +++ b/net/mac80211/key.c @@ -802,6 +802,9 @@ static void ieee80211_key_destroy(struct ieee80211_key *key, void ieee80211_key_free_unused(struct ieee80211_key *key) { + if (!key) + return; + WARN_ON(key->sdata || key->local); ieee80211_key_free_common(key); } @@ -854,7 +857,7 @@ int ieee80211_key_link(struct ieee80211_key *key, * can cause warnings to appear. */ bool delay_tailroom = sdata->vif.type == NL80211_IFTYPE_STATION; - int ret = -EOPNOTSUPP; + int ret; mutex_lock(&sdata->local->key_mtx); @@ -868,8 +871,10 @@ int ieee80211_key_link(struct ieee80211_key *key, * the same cipher. Enforce the assumption for pairwise keys. */ if ((alt_key && alt_key->conf.cipher != key->conf.cipher) || - (old_key && old_key->conf.cipher != key->conf.cipher)) + (old_key && old_key->conf.cipher != key->conf.cipher)) { + ret = -EOPNOTSUPP; goto out; + } } else if (sta) { struct link_sta_info *link_sta = &sta->deflink; int link_id = key->conf.link_id; @@ -895,8 +900,10 @@ int ieee80211_key_link(struct ieee80211_key *key, /* Non-pairwise keys must also not switch the cipher on rekey */ if (!pairwise) { - if (old_key && old_key->conf.cipher != key->conf.cipher) + if (old_key && old_key->conf.cipher != key->conf.cipher) { + ret = -EOPNOTSUPP; goto out; + } } /* @@ -904,8 +911,7 @@ int ieee80211_key_link(struct ieee80211_key *key, * new version of the key to avoid nonce reuse or replay issues. */ if (ieee80211_key_identical(sdata, old_key, key)) { - ieee80211_key_free_unused(key); - ret = 0; + ret = -EALREADY; goto out; } @@ -930,7 +936,10 @@ int ieee80211_key_link(struct ieee80211_key *key, ieee80211_key_free(key, delay_tailroom); } + key = NULL; + out: + ieee80211_key_free_unused(key); mutex_unlock(&sdata->local->key_mtx); return ret; diff --git a/net/mac80211/mesh.c b/net/mac80211/mesh.c index af8c5fc2db14..e31c312c124a 100644 --- a/net/mac80211/mesh.c +++ b/net/mac80211/mesh.c @@ -1175,7 +1175,7 @@ void ieee80211_mbss_info_change_notify(struct ieee80211_sub_if_data *sdata, /* if we race with running work, worst case this work becomes a noop */ for_each_set_bit(bit, &bits, sizeof(changed) * BITS_PER_BYTE) - set_bit(bit, &ifmsh->mbss_changed); + set_bit(bit, ifmsh->mbss_changed); set_bit(MESH_WORK_MBSS_CHANGED, &ifmsh->wrkq_flags); wiphy_work_queue(sdata->local->hw.wiphy, &sdata->work); } @@ -1257,7 +1257,7 @@ void ieee80211_stop_mesh(struct ieee80211_sub_if_data *sdata) /* clear any mesh work (for next join) we may have accrued */ ifmsh->wrkq_flags = 0; - ifmsh->mbss_changed = 0; + memset(ifmsh->mbss_changed, 0, sizeof(ifmsh->mbss_changed)); local->fif_other_bss--; atomic_dec(&local->iff_allmultis); @@ -1724,9 +1724,9 @@ static void mesh_bss_info_changed(struct ieee80211_sub_if_data *sdata) u32 bit; u64 changed = 0; - for_each_set_bit(bit, &ifmsh->mbss_changed, + for_each_set_bit(bit, ifmsh->mbss_changed, sizeof(changed) * BITS_PER_BYTE) { - clear_bit(bit, &ifmsh->mbss_changed); + clear_bit(bit, ifmsh->mbss_changed); changed |= BIT(bit); } diff --git a/net/mac80211/mesh_plink.c b/net/mac80211/mesh_plink.c index f3d5bb0a59f1..a1e526419e9d 100644 --- a/net/mac80211/mesh_plink.c +++ b/net/mac80211/mesh_plink.c @@ -451,7 +451,7 @@ static void mesh_sta_info_init(struct ieee80211_sub_if_data *sdata, changed |= IEEE80211_RC_BW_CHANGED; ieee80211_vht_cap_ie_to_sta_vht_cap(sdata, sband, - elems->vht_cap_elem, + elems->vht_cap_elem, NULL, &sta->deflink); ieee80211_he_cap_ie_to_sta_he_cap(sdata, sband, elems->he_cap, diff --git a/net/mac80211/mlme.c b/net/mac80211/mlme.c index f93eb38ae0b8..0c9198997482 100644 --- a/net/mac80211/mlme.c +++ b/net/mac80211/mlme.c @@ -4202,10 +4202,33 @@ static bool ieee80211_assoc_config_link(struct ieee80211_link_data *link, elems->ht_cap_elem, link_sta); - if (elems->vht_cap_elem && !(link->u.mgd.conn_flags & IEEE80211_CONN_DISABLE_VHT)) + if (elems->vht_cap_elem && + !(link->u.mgd.conn_flags & IEEE80211_CONN_DISABLE_VHT)) { + const struct ieee80211_vht_cap *bss_vht_cap = NULL; + const struct cfg80211_bss_ies *ies; + + /* + * Cisco AP module 9115 with FW 17.3 has a bug and sends a + * too large maximum MPDU length in the association response + * (indicating 12k) that it cannot actually process ... + * Work around that. + */ + rcu_read_lock(); + ies = rcu_dereference(cbss->ies); + if (ies) { + const struct element *elem; + + elem = cfg80211_find_elem(WLAN_EID_VHT_CAPABILITY, + ies->data, ies->len); + if (elem && elem->datalen >= sizeof(*bss_vht_cap)) + bss_vht_cap = (const void *)elem->data; + } + ieee80211_vht_cap_ie_to_sta_vht_cap(sdata, sband, elems->vht_cap_elem, - link_sta); + bss_vht_cap, link_sta); + rcu_read_unlock(); + } if (elems->he_operation && !(link->u.mgd.conn_flags & IEEE80211_CONN_DISABLE_HE) && elems->he_cap) { @@ -5107,9 +5130,10 @@ static bool ieee80211_assoc_success(struct ieee80211_sub_if_data *sdata, continue; valid_links |= BIT(link_id); - if (assoc_data->link[link_id].disabled) { + if (assoc_data->link[link_id].disabled) dormant_links |= BIT(link_id); - } else if (link_id != assoc_data->assoc_link_id) { + + if (link_id != assoc_data->assoc_link_id) { err = ieee80211_sta_allocate_link(sta, link_id); if (err) goto out_err; @@ -5124,7 +5148,7 @@ static bool ieee80211_assoc_success(struct ieee80211_sub_if_data *sdata, struct ieee80211_link_data *link; struct link_sta_info *link_sta; - if (!cbss || assoc_data->link[link_id].disabled) + if (!cbss) continue; link = sdata_dereference(sdata->link[link_id], sdata); @@ -5429,17 +5453,18 @@ static void ieee80211_rx_mgmt_assoc_resp(struct ieee80211_sub_if_data *sdata, for (link_id = 0; link_id < IEEE80211_MLD_MAX_NUM_LINKS; link_id++) { struct ieee80211_link_data *link; - link = sdata_dereference(sdata->link[link_id], sdata); - if (!link) - continue; - if (!assoc_data->link[link_id].bss) continue; resp.links[link_id].bss = assoc_data->link[link_id].bss; - resp.links[link_id].addr = link->conf->addr; + ether_addr_copy(resp.links[link_id].addr, + assoc_data->link[link_id].addr); resp.links[link_id].status = assoc_data->link[link_id].status; + link = sdata_dereference(sdata->link[link_id], sdata); + if (!link) + continue; + /* get uapsd queues configuration - same for all links */ resp.uapsd_queues = 0; for (ac = 0; ac < IEEE80211_NUM_ACS; ac++) diff --git a/net/mac80211/tx.c b/net/mac80211/tx.c index 7fe7280e8437..d45d4be63dd8 100644 --- a/net/mac80211/tx.c +++ b/net/mac80211/tx.c @@ -665,7 +665,8 @@ ieee80211_tx_h_select_key(struct ieee80211_tx_data *tx) } if (unlikely(tx->key && tx->key->flags & KEY_FLAG_TAINTED && - !ieee80211_is_deauth(hdr->frame_control))) + !ieee80211_is_deauth(hdr->frame_control)) && + tx->skb->protocol != tx->sdata->control_port_protocol) return TX_DROP; if (!skip_hw && tx->key && diff --git a/net/mac80211/vht.c b/net/mac80211/vht.c index c1250aa47808..b3a5c3e96a72 100644 --- a/net/mac80211/vht.c +++ b/net/mac80211/vht.c @@ -4,7 +4,7 @@ * * Portions of this file * Copyright(c) 2015 - 2016 Intel Deutschland GmbH - * Copyright (C) 2018 - 2022 Intel Corporation + * Copyright (C) 2018 - 2023 Intel Corporation */ #include @@ -116,12 +116,14 @@ void ieee80211_vht_cap_ie_to_sta_vht_cap(struct ieee80211_sub_if_data *sdata, struct ieee80211_supported_band *sband, const struct ieee80211_vht_cap *vht_cap_ie, + const struct ieee80211_vht_cap *vht_cap_ie2, struct link_sta_info *link_sta) { struct ieee80211_sta_vht_cap *vht_cap = &link_sta->pub->vht_cap; struct ieee80211_sta_vht_cap own_cap; u32 cap_info, i; bool have_80mhz; + u32 mpdu_len; memset(vht_cap, 0, sizeof(*vht_cap)); @@ -317,11 +319,21 @@ ieee80211_vht_cap_ie_to_sta_vht_cap(struct ieee80211_sub_if_data *sdata, link_sta->pub->bandwidth = ieee80211_sta_cur_vht_bw(link_sta); + /* + * Work around the Cisco 9115 FW 17.3 bug by taking the min of + * both reported MPDU lengths. + */ + mpdu_len = vht_cap->cap & IEEE80211_VHT_CAP_MAX_MPDU_MASK; + if (vht_cap_ie2) + mpdu_len = min_t(u32, mpdu_len, + le32_get_bits(vht_cap_ie2->vht_cap_info, + IEEE80211_VHT_CAP_MAX_MPDU_MASK)); + /* * FIXME - should the amsdu len be per link? store per link * and maintain a minimum? */ - switch (vht_cap->cap & IEEE80211_VHT_CAP_MAX_MPDU_MASK) { + switch (mpdu_len) { case IEEE80211_VHT_CAP_MAX_MPDU_LENGTH_11454: link_sta->pub->agg.max_amsdu_len = IEEE80211_MAX_MPDU_LEN_VHT_11454; break; diff --git a/net/mctp/route.c b/net/mctp/route.c index ab62fe447038..7a47a58aa54b 100644 --- a/net/mctp/route.c +++ b/net/mctp/route.c @@ -737,6 +737,8 @@ struct mctp_route *mctp_route_lookup(struct net *net, unsigned int dnet, { struct mctp_route *tmp, *rt = NULL; + rcu_read_lock(); + list_for_each_entry_rcu(tmp, &net->mctp.routes, list) { /* TODO: add metrics */ if (mctp_rt_match_eid(tmp, dnet, daddr)) { @@ -747,21 +749,29 @@ struct mctp_route *mctp_route_lookup(struct net *net, unsigned int dnet, } } + rcu_read_unlock(); + return rt; } static struct mctp_route *mctp_route_lookup_null(struct net *net, struct net_device *dev) { - struct mctp_route *rt; + struct mctp_route *tmp, *rt = NULL; - list_for_each_entry_rcu(rt, &net->mctp.routes, list) { - if (rt->dev->dev == dev && rt->type == RTN_LOCAL && - refcount_inc_not_zero(&rt->refs)) - return rt; + rcu_read_lock(); + + list_for_each_entry_rcu(tmp, &net->mctp.routes, list) { + if (tmp->dev->dev == dev && tmp->type == RTN_LOCAL && + refcount_inc_not_zero(&tmp->refs)) { + rt = tmp; + break; + } } - return NULL; + rcu_read_unlock(); + + return rt; } static int mctp_do_fragment_route(struct mctp_route *rt, struct sk_buff *skb, diff --git a/net/mptcp/options.c b/net/mptcp/options.c index c254accb14de..cd15ec73073e 100644 --- a/net/mptcp/options.c +++ b/net/mptcp/options.c @@ -1269,12 +1269,13 @@ static void mptcp_set_rwin(struct tcp_sock *tp, struct tcphdr *th) if (rcv_wnd == rcv_wnd_old) break; - if (before64(rcv_wnd_new, rcv_wnd)) { + + rcv_wnd_old = rcv_wnd; + if (before64(rcv_wnd_new, rcv_wnd_old)) { MPTCP_INC_STATS(sock_net(ssk), MPTCP_MIB_RCVWNDCONFLICTUPDATE); goto raise_win; } MPTCP_INC_STATS(sock_net(ssk), MPTCP_MIB_RCVWNDCONFLICT); - rcv_wnd_old = rcv_wnd; } return; } diff --git a/net/mptcp/pm_userspace.c b/net/mptcp/pm_userspace.c index b5a8aa4c1ebd..d042d32beb4d 100644 --- a/net/mptcp/pm_userspace.c +++ b/net/mptcp/pm_userspace.c @@ -307,12 +307,6 @@ int mptcp_nl_cmd_sf_create(struct sk_buff *skb, struct genl_info *info) goto create_err; } - if (addr_l.id == 0) { - NL_SET_ERR_MSG_ATTR(info->extack, laddr, "missing local addr id"); - err = -EINVAL; - goto create_err; - } - err = mptcp_pm_parse_addr(raddr, info, &addr_r); if (err < 0) { NL_SET_ERR_MSG_ATTR(info->extack, raddr, "error parsing remote addr"); diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c index a7fc16f5175d..886ab689a8ae 100644 --- a/net/mptcp/protocol.c +++ b/net/mptcp/protocol.c @@ -405,7 +405,7 @@ drop: return false; } -static void mptcp_stop_timer(struct sock *sk) +static void mptcp_stop_rtx_timer(struct sock *sk) { struct inet_connection_sock *icsk = inet_csk(sk); @@ -770,6 +770,46 @@ static bool __mptcp_ofo_queue(struct mptcp_sock *msk) return moved; } +static bool __mptcp_subflow_error_report(struct sock *sk, struct sock *ssk) +{ + int err = sock_error(ssk); + int ssk_state; + + if (!err) + return false; + + /* only propagate errors on fallen-back sockets or + * on MPC connect + */ + if (sk->sk_state != TCP_SYN_SENT && !__mptcp_check_fallback(mptcp_sk(sk))) + return false; + + /* We need to propagate only transition to CLOSE state. + * Orphaned socket will see such state change via + * subflow_sched_work_if_closed() and that path will properly + * destroy the msk as needed. + */ + ssk_state = inet_sk_state_load(ssk); + if (ssk_state == TCP_CLOSE && !sock_flag(sk, SOCK_DEAD)) + inet_sk_state_store(sk, ssk_state); + WRITE_ONCE(sk->sk_err, -err); + + /* This barrier is coupled with smp_rmb() in mptcp_poll() */ + smp_wmb(); + sk_error_report(sk); + return true; +} + +void __mptcp_error_report(struct sock *sk) +{ + struct mptcp_subflow_context *subflow; + struct mptcp_sock *msk = mptcp_sk(sk); + + mptcp_for_each_subflow(msk, subflow) + if (__mptcp_subflow_error_report(sk, mptcp_subflow_tcp_sock(subflow))) + break; +} + /* In most cases we will be able to lock the mptcp socket. If its already * owned, we need to defer to the work queue to avoid ABBA deadlock. */ @@ -852,6 +892,7 @@ static bool __mptcp_finish_join(struct mptcp_sock *msk, struct sock *ssk) mptcp_subflow_ctx(ssk)->subflow_id = msk->subflow_id++; mptcp_sockopt_sync_locked(msk, ssk); mptcp_subflow_joined(msk, ssk); + mptcp_stop_tout_timer(sk); return true; } @@ -871,12 +912,12 @@ static void __mptcp_flush_join_list(struct sock *sk, struct list_head *join_list } } -static bool mptcp_timer_pending(struct sock *sk) +static bool mptcp_rtx_timer_pending(struct sock *sk) { return timer_pending(&inet_csk(sk)->icsk_retransmit_timer); } -static void mptcp_reset_timer(struct sock *sk) +static void mptcp_reset_rtx_timer(struct sock *sk) { struct inet_connection_sock *icsk = inet_csk(sk); unsigned long tout; @@ -1010,10 +1051,10 @@ static void __mptcp_clean_una(struct sock *sk) out: if (snd_una == READ_ONCE(msk->snd_nxt) && snd_una == READ_ONCE(msk->write_seq)) { - if (mptcp_timer_pending(sk) && !mptcp_data_fin_enabled(msk)) - mptcp_stop_timer(sk); + if (mptcp_rtx_timer_pending(sk) && !mptcp_data_fin_enabled(msk)) + mptcp_stop_rtx_timer(sk); } else { - mptcp_reset_timer(sk); + mptcp_reset_rtx_timer(sk); } } @@ -1257,7 +1298,7 @@ alloc_skb: if (copy == 0) { u64 snd_una = READ_ONCE(msk->snd_una); - if (snd_una != msk->snd_nxt) { + if (snd_una != msk->snd_nxt || tcp_write_queue_tail(ssk)) { tcp_remove_empty_skb(ssk); return 0; } @@ -1265,11 +1306,6 @@ alloc_skb: zero_window_probe = true; data_seq = snd_una - 1; copy = 1; - - /* all mptcp-level data is acked, no skbs should be present into the - * ssk write queue - */ - WARN_ON_ONCE(reuse_skb); } copy = min_t(size_t, copy, info->limit - info->sent); @@ -1298,7 +1334,6 @@ alloc_skb: if (reuse_skb) { TCP_SKB_CB(skb)->tcp_flags &= ~TCPHDR_PSH; mpext->data_len += copy; - WARN_ON_ONCE(zero_window_probe); goto out; } @@ -1586,8 +1621,8 @@ void __mptcp_push_pending(struct sock *sk, unsigned int flags) mptcp_push_release(ssk, &info); /* ensure the rtx timer is running */ - if (!mptcp_timer_pending(sk)) - mptcp_reset_timer(sk); + if (!mptcp_rtx_timer_pending(sk)) + mptcp_reset_rtx_timer(sk); if (do_check_data_fin) mptcp_check_send_data_fin(sk); } @@ -1650,8 +1685,8 @@ out: if (copied) { tcp_push(ssk, 0, info.mss_now, tcp_sk(ssk)->nonagle, info.size_goal); - if (!mptcp_timer_pending(sk)) - mptcp_reset_timer(sk); + if (!mptcp_rtx_timer_pending(sk)) + mptcp_reset_rtx_timer(sk); if (msk->snd_data_fin_enable && msk->snd_nxt + 1 == msk->write_seq) @@ -2220,7 +2255,7 @@ static void mptcp_retransmit_timer(struct timer_list *t) sock_put(sk); } -static void mptcp_timeout_timer(struct timer_list *t) +static void mptcp_tout_timer(struct timer_list *t) { struct sock *sk = from_timer(sk, t, sk_timer); @@ -2313,6 +2348,26 @@ bool __mptcp_retransmit_pending_data(struct sock *sk) #define MPTCP_CF_PUSH BIT(1) #define MPTCP_CF_FASTCLOSE BIT(2) +/* be sure to send a reset only if the caller asked for it, also + * clean completely the subflow status when the subflow reaches + * TCP_CLOSE state + */ +static void __mptcp_subflow_disconnect(struct sock *ssk, + struct mptcp_subflow_context *subflow, + unsigned int flags) +{ + if (((1 << ssk->sk_state) & (TCPF_CLOSE | TCPF_LISTEN)) || + (flags & MPTCP_CF_FASTCLOSE)) { + /* The MPTCP code never wait on the subflow sockets, TCP-level + * disconnect should never fail + */ + WARN_ON_ONCE(tcp_disconnect(ssk, 0)); + mptcp_subflow_ctx_reset(subflow); + } else { + tcp_shutdown(ssk, SEND_SHUTDOWN); + } +} + /* subflow sockets can be either outgoing (connect) or incoming * (accept). * @@ -2329,18 +2384,14 @@ static void __mptcp_close_ssk(struct sock *sk, struct sock *ssk, bool dispose_it, need_push = false; /* If the first subflow moved to a close state before accept, e.g. due - * to an incoming reset, mptcp either: - * - if either the subflow or the msk are dead, destroy the context - * (the subflow socket is deleted by inet_child_forget) and the msk - * - otherwise do nothing at the moment and take action at accept and/or - * listener shutdown - user-space must be able to accept() the closed - * socket. + * to an incoming reset or listener shutdown, the subflow socket is + * already deleted by inet_child_forget() and the mptcp socket can't + * survive too. */ - if (msk->in_accept_queue && msk->first == ssk) { - if (!sock_flag(sk, SOCK_DEAD) && !sock_flag(ssk, SOCK_DEAD)) - return; - + if (msk->in_accept_queue && msk->first == ssk && + (sock_flag(sk, SOCK_DEAD) || sock_flag(ssk, SOCK_DEAD))) { /* ensure later check in mptcp_worker() will dispose the msk */ + mptcp_set_close_tout(sk, tcp_jiffies32 - (TCP_TIMEWAIT_LEN + 1)); sock_set_flag(sk, SOCK_DEAD); lock_sock_nested(ssk, SINGLE_DEPTH_NESTING); mptcp_subflow_drop_ctx(ssk); @@ -2354,7 +2405,7 @@ static void __mptcp_close_ssk(struct sock *sk, struct sock *ssk, lock_sock_nested(ssk, SINGLE_DEPTH_NESTING); if ((flags & MPTCP_CF_FASTCLOSE) && !__mptcp_check_fallback(msk)) { - /* be sure to force the tcp_disconnect() path, + /* be sure to force the tcp_close path * to generate the egress reset */ ssk->sk_lingertime = 0; @@ -2364,11 +2415,7 @@ static void __mptcp_close_ssk(struct sock *sk, struct sock *ssk, need_push = (flags & MPTCP_CF_PUSH) && __mptcp_retransmit_pending_data(sk); if (!dispose_it) { - /* The MPTCP code never wait on the subflow sockets, TCP-level - * disconnect should never fail - */ - WARN_ON_ONCE(tcp_disconnect(ssk, 0)); - mptcp_subflow_ctx_reset(subflow); + __mptcp_subflow_disconnect(ssk, subflow, flags); release_sock(ssk); goto out; @@ -2392,6 +2439,7 @@ static void __mptcp_close_ssk(struct sock *sk, struct sock *ssk, } out_release: + __mptcp_subflow_error_report(sk, ssk); release_sock(ssk); sock_put(ssk); @@ -2402,6 +2450,22 @@ out_release: out: if (need_push) __mptcp_push_pending(sk, 0); + + /* Catch every 'all subflows closed' scenario, including peers silently + * closing them, e.g. due to timeout. + * For established sockets, allow an additional timeout before closing, + * as the protocol can still create more subflows. + */ + if (list_is_singular(&msk->conn_list) && msk->first && + inet_sk_state_load(msk->first) == TCP_CLOSE) { + if (sk->sk_state != TCP_ESTABLISHED || + msk->in_accept_queue || sock_flag(sk, SOCK_DEAD)) { + inet_sk_state_store(sk, TCP_CLOSE); + mptcp_close_wake_up(sk); + } else { + mptcp_start_tout_timer(sk); + } + } } void mptcp_close_ssk(struct sock *sk, struct sock *ssk, @@ -2445,23 +2509,14 @@ static void __mptcp_close_subflow(struct sock *sk) } -static bool mptcp_should_close(const struct sock *sk) +static bool mptcp_close_tout_expired(const struct sock *sk) { - s32 delta = tcp_jiffies32 - inet_csk(sk)->icsk_mtup.probe_timestamp; - struct mptcp_subflow_context *subflow; - - if (delta >= TCP_TIMEWAIT_LEN || mptcp_sk(sk)->in_accept_queue) - return true; + if (!inet_csk(sk)->icsk_mtup.probe_timestamp || + sk->sk_state == TCP_CLOSE) + return false; - /* if all subflows are in closed status don't bother with additional - * timeout - */ - mptcp_for_each_subflow(mptcp_sk(sk), subflow) { - if (inet_sk_state_load(mptcp_subflow_tcp_sock(subflow)) != - TCP_CLOSE) - return false; - } - return true; + return time_after32(tcp_jiffies32, + inet_csk(sk)->icsk_mtup.probe_timestamp + TCP_TIMEWAIT_LEN); } static void mptcp_check_fastclose(struct mptcp_sock *msk) @@ -2588,27 +2643,28 @@ static void __mptcp_retrans(struct sock *sk) reset_timer: mptcp_check_and_set_pending(sk); - if (!mptcp_timer_pending(sk)) - mptcp_reset_timer(sk); + if (!mptcp_rtx_timer_pending(sk)) + mptcp_reset_rtx_timer(sk); } /* schedule the timeout timer for the relevant event: either close timeout * or mp_fail timeout. The close timeout takes precedence on the mp_fail one */ -void mptcp_reset_timeout(struct mptcp_sock *msk, unsigned long fail_tout) +void mptcp_reset_tout_timer(struct mptcp_sock *msk, unsigned long fail_tout) { struct sock *sk = (struct sock *)msk; unsigned long timeout, close_timeout; - if (!fail_tout && !sock_flag(sk, SOCK_DEAD)) + if (!fail_tout && !inet_csk(sk)->icsk_mtup.probe_timestamp) return; - close_timeout = inet_csk(sk)->icsk_mtup.probe_timestamp - tcp_jiffies32 + jiffies + TCP_TIMEWAIT_LEN; + close_timeout = inet_csk(sk)->icsk_mtup.probe_timestamp - tcp_jiffies32 + jiffies + + TCP_TIMEWAIT_LEN; /* the close timeout takes precedence on the fail one, and here at least one of * them is active */ - timeout = sock_flag(sk, SOCK_DEAD) ? close_timeout : fail_tout; + timeout = inet_csk(sk)->icsk_mtup.probe_timestamp ? close_timeout : fail_tout; sk_reset_timer(sk, &sk->sk_timer, timeout); } @@ -2627,8 +2683,6 @@ static void mptcp_mp_fail_no_response(struct mptcp_sock *msk) mptcp_subflow_reset(ssk); WRITE_ONCE(mptcp_subflow_ctx(ssk)->fail_tout, 0); unlock_sock_fast(ssk, slow); - - mptcp_reset_timeout(msk, 0); } static void mptcp_do_fastclose(struct sock *sk) @@ -2665,18 +2719,14 @@ static void mptcp_worker(struct work_struct *work) if (test_and_clear_bit(MPTCP_WORK_CLOSE_SUBFLOW, &msk->flags)) __mptcp_close_subflow(sk); - /* There is no point in keeping around an orphaned sk timedout or - * closed, but we need the msk around to reply to incoming DATA_FIN, - * even if it is orphaned and in FIN_WAIT2 state - */ - if (sock_flag(sk, SOCK_DEAD)) { - if (mptcp_should_close(sk)) - mptcp_do_fastclose(sk); + if (mptcp_close_tout_expired(sk)) { + mptcp_do_fastclose(sk); + mptcp_close_wake_up(sk); + } - if (sk->sk_state == TCP_CLOSE) { - __mptcp_destroy_sock(sk); - goto unlock; - } + if (sock_flag(sk, SOCK_DEAD) && sk->sk_state == TCP_CLOSE) { + __mptcp_destroy_sock(sk); + goto unlock; } if (test_and_clear_bit(MPTCP_WORK_RTX, &msk->flags)) @@ -2717,7 +2767,7 @@ static void __mptcp_init_sock(struct sock *sk) /* re-use the csk retrans timer for MPTCP-level retrans */ timer_setup(&msk->sk.icsk_retransmit_timer, mptcp_retransmit_timer, 0); - timer_setup(&sk->sk_timer, mptcp_timeout_timer, 0); + timer_setup(&sk->sk_timer, mptcp_tout_timer, 0); } static void mptcp_ca_reset(struct sock *sk) @@ -2808,8 +2858,8 @@ void mptcp_subflow_shutdown(struct sock *sk, struct sock *ssk, int how) } else { pr_debug("Sending DATA_FIN on subflow %p", ssk); tcp_send_ack(ssk); - if (!mptcp_timer_pending(sk)) - mptcp_reset_timer(sk); + if (!mptcp_rtx_timer_pending(sk)) + mptcp_reset_rtx_timer(sk); } break; } @@ -2892,7 +2942,7 @@ static void __mptcp_destroy_sock(struct sock *sk) might_sleep(); - mptcp_stop_timer(sk); + mptcp_stop_rtx_timer(sk); sk_stop_timer(sk, &sk->sk_timer); msk->pm.status = 0; mptcp_release_sched(msk); @@ -2975,7 +3025,6 @@ bool __mptcp_close(struct sock *sk, long timeout) cleanup: /* orphan all the subflows */ - inet_csk(sk)->icsk_mtup.probe_timestamp = tcp_jiffies32; mptcp_for_each_subflow(msk, subflow) { struct sock *ssk = mptcp_subflow_tcp_sock(subflow); bool slow = lock_sock_fast_nested(ssk); @@ -3012,7 +3061,7 @@ cleanup: __mptcp_destroy_sock(sk); do_cancel_work = true; } else { - mptcp_reset_timeout(msk, 0); + mptcp_start_tout_timer(sk); } return do_cancel_work; @@ -3059,12 +3108,6 @@ static int mptcp_disconnect(struct sock *sk, int flags) { struct mptcp_sock *msk = mptcp_sk(sk); - /* Deny disconnect if other threads are blocked in sk_wait_event() - * or inet_wait_for_connect(). - */ - if (sk->sk_wait_pending) - return -EBUSY; - /* We are on the fastopen error path. We can't call straight into the * subflows cleanup code due to lock nesting (we are already under * msk->firstsocket lock). @@ -3075,8 +3118,8 @@ static int mptcp_disconnect(struct sock *sk, int flags) mptcp_check_listen_stop(sk); inet_sk_state_store(sk, TCP_CLOSE); - mptcp_stop_timer(sk); - sk_stop_timer(sk, &sk->sk_timer); + mptcp_stop_rtx_timer(sk); + mptcp_stop_tout_timer(sk); if (msk->token) mptcp_event(MPTCP_EVENT_CLOSED, msk, NULL, GFP_KERNEL); @@ -3134,7 +3177,6 @@ struct sock *mptcp_sk_clone_init(const struct sock *sk, inet_sk(nsk)->pinet6 = mptcp_inet6_sk(nsk); #endif - nsk->sk_wait_pending = 0; __mptcp_init_sock(nsk); msk = mptcp_sk(nsk); @@ -3386,24 +3428,21 @@ static void schedule_3rdack_retransmission(struct sock *ssk) sk_reset_timer(ssk, &icsk->icsk_delack_timer, timeout); } -void mptcp_subflow_process_delegated(struct sock *ssk) +void mptcp_subflow_process_delegated(struct sock *ssk, long status) { struct mptcp_subflow_context *subflow = mptcp_subflow_ctx(ssk); struct sock *sk = subflow->conn; - if (test_bit(MPTCP_DELEGATE_SEND, &subflow->delegated_status)) { + if (status & BIT(MPTCP_DELEGATE_SEND)) { mptcp_data_lock(sk); if (!sock_owned_by_user(sk)) __mptcp_subflow_push_pending(sk, ssk, true); else __set_bit(MPTCP_PUSH_PENDING, &mptcp_sk(sk)->cb_flags); mptcp_data_unlock(sk); - mptcp_subflow_delegated_done(subflow, MPTCP_DELEGATE_SEND); } - if (test_bit(MPTCP_DELEGATE_ACK, &subflow->delegated_status)) { + if (status & BIT(MPTCP_DELEGATE_ACK)) schedule_3rdack_retransmission(ssk); - mptcp_subflow_delegated_done(subflow, MPTCP_DELEGATE_ACK); - } } static int mptcp_hash(struct sock *sk) @@ -3929,14 +3968,17 @@ static int mptcp_napi_poll(struct napi_struct *napi, int budget) struct sock *ssk = mptcp_subflow_tcp_sock(subflow); bh_lock_sock_nested(ssk); - if (!sock_owned_by_user(ssk) && - mptcp_subflow_has_delegated_action(subflow)) - mptcp_subflow_process_delegated(ssk); - /* ... elsewhere tcp_release_cb_override already processed - * the action or will do at next release_sock(). - * In both case must dequeue the subflow here - on the same - * CPU that scheduled it. - */ + if (!sock_owned_by_user(ssk)) { + mptcp_subflow_process_delegated(ssk, xchg(&subflow->delegated_status, 0)); + } else { + /* tcp_release_cb_override already processed + * the action or will do at next release_sock(). + * In both case must dequeue the subflow here - on the same + * CPU that scheduled it. + */ + smp_wmb(); + clear_bit(MPTCP_DELEGATE_SCHEDULED, &subflow->delegated_status); + } bh_unlock_sock(ssk); sock_put(ssk); diff --git a/net/mptcp/protocol.h b/net/mptcp/protocol.h index 7254b3562575..3612545fa62e 100644 --- a/net/mptcp/protocol.h +++ b/net/mptcp/protocol.h @@ -444,9 +444,11 @@ struct mptcp_delegated_action { DECLARE_PER_CPU(struct mptcp_delegated_action, mptcp_delegated_actions); -#define MPTCP_DELEGATE_SEND 0 -#define MPTCP_DELEGATE_ACK 1 +#define MPTCP_DELEGATE_SCHEDULED 0 +#define MPTCP_DELEGATE_SEND 1 +#define MPTCP_DELEGATE_ACK 2 +#define MPTCP_DELEGATE_ACTIONS_MASK (~BIT(MPTCP_DELEGATE_SCHEDULED)) /* MPTCP subflow context */ struct mptcp_subflow_context { struct list_head node;/* conn_list of subflows */ @@ -564,23 +566,24 @@ mptcp_subflow_get_mapped_dsn(const struct mptcp_subflow_context *subflow) return subflow->map_seq + mptcp_subflow_get_map_offset(subflow); } -void mptcp_subflow_process_delegated(struct sock *ssk); +void mptcp_subflow_process_delegated(struct sock *ssk, long actions); static inline void mptcp_subflow_delegate(struct mptcp_subflow_context *subflow, int action) { + long old, set_bits = BIT(MPTCP_DELEGATE_SCHEDULED) | BIT(action); struct mptcp_delegated_action *delegated; bool schedule; /* the caller held the subflow bh socket lock */ lockdep_assert_in_softirq(); - /* The implied barrier pairs with mptcp_subflow_delegated_done(), and - * ensures the below list check sees list updates done prior to status - * bit changes + /* The implied barrier pairs with tcp_release_cb_override() + * mptcp_napi_poll(), and ensures the below list check sees list + * updates done prior to delegated status bits changes */ - if (!test_and_set_bit(action, &subflow->delegated_status)) { - /* still on delegated list from previous scheduling */ - if (!list_empty(&subflow->delegated_node)) + old = set_mask_bits(&subflow->delegated_status, 0, set_bits); + if (!(old & BIT(MPTCP_DELEGATE_SCHEDULED))) { + if (WARN_ON_ONCE(!list_empty(&subflow->delegated_node))) return; delegated = this_cpu_ptr(&mptcp_delegated_actions); @@ -605,20 +608,6 @@ mptcp_subflow_delegated_next(struct mptcp_delegated_action *delegated) return ret; } -static inline bool mptcp_subflow_has_delegated_action(const struct mptcp_subflow_context *subflow) -{ - return !!READ_ONCE(subflow->delegated_status); -} - -static inline void mptcp_subflow_delegated_done(struct mptcp_subflow_context *subflow, int action) -{ - /* pairs with mptcp_subflow_delegate, ensures delegate_node is updated before - * touching the status bit - */ - smp_wmb(); - clear_bit(action, &subflow->delegated_status); -} - int mptcp_is_enabled(const struct net *net); unsigned int mptcp_get_add_addr_timeout(const struct net *net); int mptcp_is_checksum_enabled(const struct net *net); @@ -718,7 +707,29 @@ void mptcp_get_options(const struct sk_buff *skb, void mptcp_finish_connect(struct sock *sk); void __mptcp_set_connected(struct sock *sk); -void mptcp_reset_timeout(struct mptcp_sock *msk, unsigned long fail_tout); +void mptcp_reset_tout_timer(struct mptcp_sock *msk, unsigned long fail_tout); + +static inline void mptcp_stop_tout_timer(struct sock *sk) +{ + if (!inet_csk(sk)->icsk_mtup.probe_timestamp) + return; + + sk_stop_timer(sk, &sk->sk_timer); + inet_csk(sk)->icsk_mtup.probe_timestamp = 0; +} + +static inline void mptcp_set_close_tout(struct sock *sk, unsigned long tout) +{ + /* avoid 0 timestamp, as that means no close timeout */ + inet_csk(sk)->icsk_mtup.probe_timestamp = tout ? : 1; +} + +static inline void mptcp_start_tout_timer(struct sock *sk) +{ + mptcp_set_close_tout(sk, tcp_jiffies32); + mptcp_reset_tout_timer(mptcp_sk(sk), 0); +} + static inline bool mptcp_is_fully_established(struct sock *sk) { return inet_sk_state_load(sk) == TCP_ESTABLISHED && diff --git a/net/mptcp/subflow.c b/net/mptcp/subflow.c index 9bf3c7bc1762..9c1f8d1d63d2 100644 --- a/net/mptcp/subflow.c +++ b/net/mptcp/subflow.c @@ -1226,7 +1226,7 @@ static void mptcp_subflow_fail(struct mptcp_sock *msk, struct sock *ssk) WRITE_ONCE(subflow->fail_tout, fail_tout); tcp_send_ack(ssk); - mptcp_reset_timeout(msk, subflow->fail_tout); + mptcp_reset_tout_timer(msk, subflow->fail_tout); } static bool subflow_check_data_avail(struct sock *ssk) @@ -1362,42 +1362,6 @@ void mptcp_space(const struct sock *ssk, int *space, int *full_space) *full_space = mptcp_win_from_space(sk, READ_ONCE(sk->sk_rcvbuf)); } -void __mptcp_error_report(struct sock *sk) -{ - struct mptcp_subflow_context *subflow; - struct mptcp_sock *msk = mptcp_sk(sk); - - mptcp_for_each_subflow(msk, subflow) { - struct sock *ssk = mptcp_subflow_tcp_sock(subflow); - int err = sock_error(ssk); - int ssk_state; - - if (!err) - continue; - - /* only propagate errors on fallen-back sockets or - * on MPC connect - */ - if (sk->sk_state != TCP_SYN_SENT && !__mptcp_check_fallback(msk)) - continue; - - /* We need to propagate only transition to CLOSE state. - * Orphaned socket will see such state change via - * subflow_sched_work_if_closed() and that path will properly - * destroy the msk as needed. - */ - ssk_state = inet_sk_state_load(ssk); - if (ssk_state == TCP_CLOSE && !sock_flag(sk, SOCK_DEAD)) - inet_sk_state_store(sk, ssk_state); - WRITE_ONCE(sk->sk_err, -err); - - /* This barrier is coupled with smp_rmb() in mptcp_poll() */ - smp_wmb(); - sk_error_report(sk); - break; - } -} - static void subflow_error_report(struct sock *ssk) { struct sock *sk = mptcp_subflow_ctx(ssk)->conn; @@ -1588,6 +1552,7 @@ int __mptcp_subflow_connect(struct sock *sk, const struct mptcp_addr_info *loc, mptcp_sock_graft(ssk, sk->sk_socket); iput(SOCK_INODE(sf)); WRITE_ONCE(msk->allow_infinite_fallback, false); + mptcp_stop_tout_timer(sk); return 0; failed_unlink: @@ -1991,9 +1956,15 @@ static void subflow_ulp_clone(const struct request_sock *req, static void tcp_release_cb_override(struct sock *ssk) { struct mptcp_subflow_context *subflow = mptcp_subflow_ctx(ssk); + long status; - if (mptcp_subflow_has_delegated_action(subflow)) - mptcp_subflow_process_delegated(ssk); + /* process and clear all the pending actions, but leave the subflow into + * the napi queue. To respect locking, only the same CPU that originated + * the action can touch the list. mptcp_napi_poll will take care of it. + */ + status = set_mask_bits(&subflow->delegated_status, MPTCP_DELEGATE_ACTIONS_MASK, 0); + if (status) + mptcp_subflow_process_delegated(ssk, status); tcp_release_cb(ssk); } diff --git a/net/ncsi/ncsi-aen.c b/net/ncsi/ncsi-aen.c index 62fb1031763d..f8854bff286c 100644 --- a/net/ncsi/ncsi-aen.c +++ b/net/ncsi/ncsi-aen.c @@ -89,6 +89,11 @@ static int ncsi_aen_handler_lsc(struct ncsi_dev_priv *ndp, if ((had_link == has_link) || chained) return 0; + if (had_link) + netif_carrier_off(ndp->ndev.dev); + else + netif_carrier_on(ndp->ndev.dev); + if (!ndp->multi_package && !nc->package->multi_channel) { if (had_link) { ndp->flags |= NCSI_DEV_RESHUFFLE; diff --git a/net/netfilter/ipset/ip_set_core.c b/net/netfilter/ipset/ip_set_core.c index e564b5174261..35d2f9c9ada0 100644 --- a/net/netfilter/ipset/ip_set_core.c +++ b/net/netfilter/ipset/ip_set_core.c @@ -682,6 +682,14 @@ __ip_set_put(struct ip_set *set) /* set->ref can be swapped out by ip_set_swap, netlink events (like dump) need * a separate reference counter */ +static void +__ip_set_get_netlink(struct ip_set *set) +{ + write_lock_bh(&ip_set_ref_lock); + set->ref_netlink++; + write_unlock_bh(&ip_set_ref_lock); +} + static void __ip_set_put_netlink(struct ip_set *set) { @@ -1693,11 +1701,11 @@ call_ad(struct net *net, struct sock *ctnl, struct sk_buff *skb, do { if (retried) { - __ip_set_get(set); + __ip_set_get_netlink(set); nfnl_unlock(NFNL_SUBSYS_IPSET); cond_resched(); nfnl_lock(NFNL_SUBSYS_IPSET); - __ip_set_put(set); + __ip_set_put_netlink(set); } ip_set_lock(set); diff --git a/net/netfilter/ipvs/ip_vs_sync.c b/net/netfilter/ipvs/ip_vs_sync.c index da5af28ff57b..4174076c66fa 100644 --- a/net/netfilter/ipvs/ip_vs_sync.c +++ b/net/netfilter/ipvs/ip_vs_sync.c @@ -1439,7 +1439,7 @@ static int bind_mcastif_addr(struct socket *sock, struct net_device *dev) sin.sin_addr.s_addr = addr; sin.sin_port = 0; - return sock->ops->bind(sock, (struct sockaddr*)&sin, sizeof(sin)); + return kernel_bind(sock, (struct sockaddr *)&sin, sizeof(sin)); } static void get_mcast_sockaddr(union ipvs_sockaddr *sa, int *salen, @@ -1505,8 +1505,8 @@ static int make_send_sock(struct netns_ipvs *ipvs, int id, } get_mcast_sockaddr(&mcast_addr, &salen, &ipvs->mcfg, id); - result = sock->ops->connect(sock, (struct sockaddr *) &mcast_addr, - salen, 0); + result = kernel_connect(sock, (struct sockaddr *)&mcast_addr, + salen, 0); if (result < 0) { pr_err("Error connecting to the multicast addr\n"); goto error; @@ -1546,7 +1546,7 @@ static int make_receive_sock(struct netns_ipvs *ipvs, int id, get_mcast_sockaddr(&mcast_addr, &salen, &ipvs->bcfg, id); sock->sk->sk_bound_dev_if = dev->ifindex; - result = sock->ops->bind(sock, (struct sockaddr *)&mcast_addr, salen); + result = kernel_bind(sock, (struct sockaddr *)&mcast_addr, salen); if (result < 0) { pr_err("Error binding to the multicast addr\n"); goto error; diff --git a/net/netfilter/nf_conntrack_bpf.c b/net/netfilter/nf_conntrack_bpf.c index c7a6114091ae..b21799d468d2 100644 --- a/net/netfilter/nf_conntrack_bpf.c +++ b/net/netfilter/nf_conntrack_bpf.c @@ -381,6 +381,8 @@ __bpf_kfunc struct nf_conn *bpf_ct_insert_entry(struct nf_conn___init *nfct_i) struct nf_conn *nfct = (struct nf_conn *)nfct_i; int err; + if (!nf_ct_is_confirmed(nfct)) + nfct->timeout += nfct_time_stamp; nfct->status |= IPS_CONFIRMED; err = nf_conntrack_hash_check_insert(nfct); if (err < 0) { diff --git a/net/netfilter/nf_conntrack_extend.c b/net/netfilter/nf_conntrack_extend.c index 0b513f7bf9f3..dd62cc12e775 100644 --- a/net/netfilter/nf_conntrack_extend.c +++ b/net/netfilter/nf_conntrack_extend.c @@ -40,10 +40,10 @@ static const u8 nf_ct_ext_type_len[NF_CT_EXT_NUM] = { [NF_CT_EXT_ECACHE] = sizeof(struct nf_conntrack_ecache), #endif #ifdef CONFIG_NF_CONNTRACK_TIMESTAMP - [NF_CT_EXT_TSTAMP] = sizeof(struct nf_conn_acct), + [NF_CT_EXT_TSTAMP] = sizeof(struct nf_conn_tstamp), #endif #ifdef CONFIG_NF_CONNTRACK_TIMEOUT - [NF_CT_EXT_TIMEOUT] = sizeof(struct nf_conn_tstamp), + [NF_CT_EXT_TIMEOUT] = sizeof(struct nf_conn_timeout), #endif #ifdef CONFIG_NF_CONNTRACK_LABELS [NF_CT_EXT_LABELS] = sizeof(struct nf_conn_labels), diff --git a/net/netfilter/nf_conntrack_proto_sctp.c b/net/netfilter/nf_conntrack_proto_sctp.c index b6bcc8f2f46b..c6bd533983c1 100644 --- a/net/netfilter/nf_conntrack_proto_sctp.c +++ b/net/netfilter/nf_conntrack_proto_sctp.c @@ -112,7 +112,7 @@ static const u8 sctp_conntracks[2][11][SCTP_CONNTRACK_MAX] = { /* shutdown_ack */ {sSA, sCL, sCW, sCE, sES, sSA, sSA, sSA, sSA}, /* error */ {sCL, sCL, sCW, sCE, sES, sSS, sSR, sSA, sCL},/* Can't have Stale cookie*/ /* cookie_echo */ {sCL, sCL, sCE, sCE, sES, sSS, sSR, sSA, sCL},/* 5.2.4 - Big TODO */ -/* cookie_ack */ {sCL, sCL, sCW, sCE, sES, sSS, sSR, sSA, sCL},/* Can't come in orig dir */ +/* cookie_ack */ {sCL, sCL, sCW, sES, sES, sSS, sSR, sSA, sCL},/* Can't come in orig dir */ /* shutdown_comp*/ {sCL, sCL, sCW, sCE, sES, sSS, sSR, sCL, sCL}, /* heartbeat */ {sHS, sCL, sCW, sCE, sES, sSS, sSR, sSA, sHS}, /* heartbeat_ack*/ {sCL, sCL, sCW, sCE, sES, sSS, sSR, sSA, sHS}, @@ -126,7 +126,7 @@ static const u8 sctp_conntracks[2][11][SCTP_CONNTRACK_MAX] = { /* shutdown */ {sIV, sCL, sCW, sCE, sSR, sSS, sSR, sSA, sIV}, /* shutdown_ack */ {sIV, sCL, sCW, sCE, sES, sSA, sSA, sSA, sIV}, /* error */ {sIV, sCL, sCW, sCL, sES, sSS, sSR, sSA, sIV}, -/* cookie_echo */ {sIV, sCL, sCW, sCE, sES, sSS, sSR, sSA, sIV},/* Can't come in reply dir */ +/* cookie_echo */ {sIV, sCL, sCE, sCE, sES, sSS, sSR, sSA, sIV},/* Can't come in reply dir */ /* cookie_ack */ {sIV, sCL, sCW, sES, sES, sSS, sSR, sSA, sIV}, /* shutdown_comp*/ {sIV, sCL, sCW, sCE, sES, sSS, sSR, sCL, sIV}, /* heartbeat */ {sIV, sCL, sCW, sCE, sES, sSS, sSR, sSA, sHS}, @@ -412,6 +412,9 @@ int nf_conntrack_sctp_packet(struct nf_conn *ct, /* (D) vtag must be same as init_vtag as found in INIT_ACK */ if (sh->vtag != ct->proto.sctp.vtag[dir]) goto out_unlock; + } else if (sch->type == SCTP_CID_COOKIE_ACK) { + ct->proto.sctp.init[dir] = 0; + ct->proto.sctp.init[!dir] = 0; } else if (sch->type == SCTP_CID_HEARTBEAT) { if (ct->proto.sctp.vtag[dir] == 0) { pr_debug("Setting %d vtag %x for dir %d\n", sch->type, sh->vtag, dir); @@ -461,16 +464,18 @@ int nf_conntrack_sctp_packet(struct nf_conn *ct, } /* If it is an INIT or an INIT ACK note down the vtag */ - if (sch->type == SCTP_CID_INIT || - sch->type == SCTP_CID_INIT_ACK) { - struct sctp_inithdr _inithdr, *ih; + if (sch->type == SCTP_CID_INIT) { + struct sctp_inithdr _ih, *ih; - ih = skb_header_pointer(skb, offset + sizeof(_sch), - sizeof(_inithdr), &_inithdr); - if (ih == NULL) + ih = skb_header_pointer(skb, offset + sizeof(_sch), sizeof(*ih), &_ih); + if (!ih) goto out_unlock; - pr_debug("Setting vtag %x for dir %d\n", - ih->init_tag, !dir); + + if (ct->proto.sctp.init[dir] && ct->proto.sctp.init[!dir]) + ct->proto.sctp.init[!dir] = 0; + ct->proto.sctp.init[dir] = 1; + + pr_debug("Setting vtag %x for dir %d\n", ih->init_tag, !dir); ct->proto.sctp.vtag[!dir] = ih->init_tag; /* don't renew timeout on init retransmit so @@ -481,6 +486,24 @@ int nf_conntrack_sctp_packet(struct nf_conn *ct, old_state == SCTP_CONNTRACK_CLOSED && nf_ct_is_confirmed(ct)) ignore = true; + } else if (sch->type == SCTP_CID_INIT_ACK) { + struct sctp_inithdr _ih, *ih; + __be32 vtag; + + ih = skb_header_pointer(skb, offset + sizeof(_sch), sizeof(*ih), &_ih); + if (!ih) + goto out_unlock; + + vtag = ct->proto.sctp.vtag[!dir]; + if (!ct->proto.sctp.init[!dir] && vtag && vtag != ih->init_tag) + goto out_unlock; + /* collision */ + if (ct->proto.sctp.init[dir] && ct->proto.sctp.init[!dir] && + vtag != ih->init_tag) + goto out_unlock; + + pr_debug("Setting vtag %x for dir %d\n", ih->init_tag, !dir); + ct->proto.sctp.vtag[!dir] = ih->init_tag; } ct->proto.sctp.state = new_state; diff --git a/net/netfilter/nf_tables_api.c b/net/netfilter/nf_tables_api.c index e429ebba74b3..29c651804cb2 100644 --- a/net/netfilter/nf_tables_api.c +++ b/net/netfilter/nf_tables_api.c @@ -1219,6 +1219,10 @@ static int nf_tables_updtable(struct nft_ctx *ctx) flags & NFT_TABLE_F_OWNER)) return -EOPNOTSUPP; + /* No dormant off/on/off/on games in single transaction */ + if (ctx->table->flags & __NFT_TABLE_F_UPDATE) + return -EINVAL; + trans = nft_trans_alloc(ctx, NFT_MSG_NEWTABLE, sizeof(struct nft_trans_table)); if (trans == NULL) @@ -1432,7 +1436,7 @@ static int nft_flush_table(struct nft_ctx *ctx) if (!nft_is_active_next(ctx->net, chain)) continue; - if (nft_chain_is_bound(chain)) + if (nft_chain_binding(chain)) continue; ctx->chain = chain; @@ -1446,8 +1450,7 @@ static int nft_flush_table(struct nft_ctx *ctx) if (!nft_is_active_next(ctx->net, set)) continue; - if (nft_set_is_anonymous(set) && - !list_empty(&set->bindings)) + if (nft_set_is_anonymous(set)) continue; err = nft_delset(ctx, set); @@ -1477,7 +1480,7 @@ static int nft_flush_table(struct nft_ctx *ctx) if (!nft_is_active_next(ctx->net, chain)) continue; - if (nft_chain_is_bound(chain)) + if (nft_chain_binding(chain)) continue; ctx->chain = chain; @@ -2910,6 +2913,9 @@ static int nf_tables_delchain(struct sk_buff *skb, const struct nfnl_info *info, return PTR_ERR(chain); } + if (nft_chain_binding(chain)) + return -EOPNOTSUPP; + nft_ctx_init(&ctx, net, skb, info->nlh, family, table, chain, nla); if (nla[NFTA_CHAIN_HOOK]) { @@ -3160,7 +3166,7 @@ int nft_expr_inner_parse(const struct nft_ctx *ctx, const struct nlattr *nla, if (err < 0) return err; - if (!tb[NFTA_EXPR_DATA]) + if (!tb[NFTA_EXPR_DATA] || !tb[NFTA_EXPR_NAME]) return -EINVAL; type = __nft_expr_type_get(ctx->family, tb[NFTA_EXPR_NAME]); @@ -3449,6 +3455,8 @@ static int __nf_tables_dump_rules(struct sk_buff *skb, struct net *net = sock_net(skb->sk); const struct nft_rule *rule, *prule; unsigned int s_idx = cb->args[0]; + unsigned int entries = 0; + int ret = 0; u64 handle; prule = NULL; @@ -3471,9 +3479,11 @@ static int __nf_tables_dump_rules(struct sk_buff *skb, NFT_MSG_NEWRULE, NLM_F_MULTI | NLM_F_APPEND, table->family, - table, chain, rule, handle, reset) < 0) - return 1; - + table, chain, rule, handle, reset) < 0) { + ret = 1; + break; + } + entries++; nl_dump_check_consistent(cb, nlmsg_hdr(skb)); cont: prule = rule; @@ -3481,10 +3491,10 @@ cont_skip: (*idx)++; } - if (reset && *idx) - audit_log_rule_reset(table, cb->seq, *idx); + if (reset && entries) + audit_log_rule_reset(table, cb->seq, entries); - return 0; + return ret; } static int nf_tables_dump_rules(struct sk_buff *skb, @@ -3971,6 +3981,11 @@ static int nf_tables_newrule(struct sk_buff *skb, const struct nfnl_info *info, } if (info->nlh->nlmsg_flags & NLM_F_REPLACE) { + if (nft_chain_binding(chain)) { + err = -EOPNOTSUPP; + goto err_destroy_flow_rule; + } + err = nft_delrule(&ctx, old_rule); if (err < 0) goto err_destroy_flow_rule; @@ -4078,7 +4093,7 @@ static int nf_tables_delrule(struct sk_buff *skb, const struct nfnl_info *info, NL_SET_BAD_ATTR(extack, nla[NFTA_RULE_CHAIN]); return PTR_ERR(chain); } - if (nft_chain_is_bound(chain)) + if (nft_chain_binding(chain)) return -EOPNOTSUPP; } @@ -4112,7 +4127,7 @@ static int nf_tables_delrule(struct sk_buff *skb, const struct nfnl_info *info, list_for_each_entry(chain, &table->chains, list) { if (!nft_is_active_next(net, chain)) continue; - if (nft_chain_is_bound(chain)) + if (nft_chain_binding(chain)) continue; ctx.chain = chain; @@ -5541,7 +5556,6 @@ static int nf_tables_fill_setelem(struct sk_buff *skb, const struct nft_set_ext *ext = nft_set_elem_ext(set, elem->priv); unsigned char *b = skb_tail_pointer(skb); struct nlattr *nest; - u64 timeout = 0; nest = nla_nest_start_noflag(skb, NFTA_LIST_ELEM); if (nest == NULL) @@ -5577,15 +5591,11 @@ static int nf_tables_fill_setelem(struct sk_buff *skb, htonl(*nft_set_ext_flags(ext)))) goto nla_put_failure; - if (nft_set_ext_exists(ext, NFT_SET_EXT_TIMEOUT)) { - timeout = *nft_set_ext_timeout(ext); - if (nla_put_be64(skb, NFTA_SET_ELEM_TIMEOUT, - nf_jiffies64_to_msecs(timeout), - NFTA_SET_ELEM_PAD)) - goto nla_put_failure; - } else if (set->flags & NFT_SET_TIMEOUT) { - timeout = READ_ONCE(set->timeout); - } + if (nft_set_ext_exists(ext, NFT_SET_EXT_TIMEOUT) && + nla_put_be64(skb, NFTA_SET_ELEM_TIMEOUT, + nf_jiffies64_to_msecs(*nft_set_ext_timeout(ext)), + NFTA_SET_ELEM_PAD)) + goto nla_put_failure; if (nft_set_ext_exists(ext, NFT_SET_EXT_EXPIRATION)) { u64 expires, now = get_jiffies_64(); @@ -5600,9 +5610,6 @@ static int nf_tables_fill_setelem(struct sk_buff *skb, nf_jiffies64_to_msecs(expires), NFTA_SET_ELEM_PAD)) goto nla_put_failure; - - if (reset) - *nft_set_ext_expiration(ext) = now + timeout; } if (nft_set_ext_exists(ext, NFT_SET_EXT_USERDATA)) { @@ -7183,8 +7190,10 @@ static int nf_tables_delsetelem(struct sk_buff *skb, if (IS_ERR(set)) return PTR_ERR(set); - if (!list_empty(&set->bindings) && - (set->flags & (NFT_SET_CONSTANT | NFT_SET_ANONYMOUS))) + if (nft_set_is_anonymous(set)) + return -EOPNOTSUPP; + + if (!list_empty(&set->bindings) && (set->flags & NFT_SET_CONSTANT)) return -EBUSY; nft_ctx_init(&ctx, net, skb, info->nlh, family, table, NULL, nla); @@ -7598,6 +7607,16 @@ nla_put_failure: return -1; } +static void audit_log_obj_reset(const struct nft_table *table, + unsigned int base_seq, unsigned int nentries) +{ + char *buf = kasprintf(GFP_ATOMIC, "%s:%u", table->name, base_seq); + + audit_log_nfcfg(buf, table->family, nentries, + AUDIT_NFT_OP_OBJ_RESET, GFP_ATOMIC); + kfree(buf); +} + struct nft_obj_filter { char *table; u32 type; @@ -7612,8 +7631,10 @@ static int nf_tables_dump_obj(struct sk_buff *skb, struct netlink_callback *cb) struct net *net = sock_net(skb->sk); int family = nfmsg->nfgen_family; struct nftables_pernet *nft_net; + unsigned int entries = 0; struct nft_object *obj; bool reset = false; + int rc = 0; if (NFNL_MSG_TYPE(cb->nlh->nlmsg_type) == NFT_MSG_GETOBJ_RESET) reset = true; @@ -7626,6 +7647,7 @@ static int nf_tables_dump_obj(struct sk_buff *skb, struct netlink_callback *cb) if (family != NFPROTO_UNSPEC && family != table->family) continue; + entries = 0; list_for_each_entry_rcu(obj, &table->objects, list) { if (!nft_is_active(net, obj)) goto cont; @@ -7641,34 +7663,27 @@ static int nf_tables_dump_obj(struct sk_buff *skb, struct netlink_callback *cb) filter->type != NFT_OBJECT_UNSPEC && obj->ops->type->type != filter->type) goto cont; - if (reset) { - char *buf = kasprintf(GFP_ATOMIC, - "%s:%u", - table->name, - nft_net->base_seq); - - audit_log_nfcfg(buf, - family, - obj->handle, - AUDIT_NFT_OP_OBJ_RESET, - GFP_ATOMIC); - kfree(buf); - } - if (nf_tables_fill_obj_info(skb, net, NETLINK_CB(cb->skb).portid, - cb->nlh->nlmsg_seq, - NFT_MSG_NEWOBJ, - NLM_F_MULTI | NLM_F_APPEND, - table->family, table, - obj, reset) < 0) - goto done; + rc = nf_tables_fill_obj_info(skb, net, + NETLINK_CB(cb->skb).portid, + cb->nlh->nlmsg_seq, + NFT_MSG_NEWOBJ, + NLM_F_MULTI | NLM_F_APPEND, + table->family, table, + obj, reset); + if (rc < 0) + break; + entries++; nl_dump_check_consistent(cb, nlmsg_hdr(skb)); cont: idx++; } + if (reset && entries) + audit_log_obj_reset(table, nft_net->base_seq, entries); + if (rc < 0) + break; } -done: rcu_read_unlock(); cb->args[0] = idx; @@ -7773,7 +7788,7 @@ static int nf_tables_getobj(struct sk_buff *skb, const struct nfnl_info *info, audit_log_nfcfg(buf, family, - obj->handle, + 1, AUDIT_NFT_OP_OBJ_RESET, GFP_ATOMIC); kfree(buf); @@ -7854,24 +7869,14 @@ static int nf_tables_delobj(struct sk_buff *skb, const struct nfnl_info *info, return nft_delobj(&ctx, obj); } -void nft_obj_notify(struct net *net, const struct nft_table *table, - struct nft_object *obj, u32 portid, u32 seq, int event, - u16 flags, int family, int report, gfp_t gfp) +static void +__nft_obj_notify(struct net *net, const struct nft_table *table, + struct nft_object *obj, u32 portid, u32 seq, int event, + u16 flags, int family, int report, gfp_t gfp) { struct nftables_pernet *nft_net = nft_pernet(net); struct sk_buff *skb; int err; - char *buf = kasprintf(gfp, "%s:%u", - table->name, nft_net->base_seq); - - audit_log_nfcfg(buf, - family, - obj->handle, - event == NFT_MSG_NEWOBJ ? - AUDIT_NFT_OP_OBJ_REGISTER : - AUDIT_NFT_OP_OBJ_UNREGISTER, - gfp); - kfree(buf); if (!report && !nfnetlink_has_listeners(net, NFNLGRP_NFTABLES)) @@ -7894,13 +7899,35 @@ void nft_obj_notify(struct net *net, const struct nft_table *table, err: nfnetlink_set_err(net, portid, NFNLGRP_NFTABLES, -ENOBUFS); } + +void nft_obj_notify(struct net *net, const struct nft_table *table, + struct nft_object *obj, u32 portid, u32 seq, int event, + u16 flags, int family, int report, gfp_t gfp) +{ + struct nftables_pernet *nft_net = nft_pernet(net); + char *buf = kasprintf(gfp, "%s:%u", + table->name, nft_net->base_seq); + + audit_log_nfcfg(buf, + family, + obj->handle, + event == NFT_MSG_NEWOBJ ? + AUDIT_NFT_OP_OBJ_REGISTER : + AUDIT_NFT_OP_OBJ_UNREGISTER, + gfp); + kfree(buf); + + __nft_obj_notify(net, table, obj, portid, seq, event, + flags, family, report, gfp); +} EXPORT_SYMBOL_GPL(nft_obj_notify); static void nf_tables_obj_notify(const struct nft_ctx *ctx, struct nft_object *obj, int event) { - nft_obj_notify(ctx->net, ctx->table, obj, ctx->portid, ctx->seq, event, - ctx->flags, ctx->family, ctx->report, GFP_KERNEL); + __nft_obj_notify(ctx->net, ctx->table, obj, ctx->portid, + ctx->seq, event, ctx->flags, ctx->family, + ctx->report, GFP_KERNEL); } /* @@ -9562,12 +9589,15 @@ static int nft_trans_gc_space(struct nft_trans_gc *trans) struct nft_trans_gc *nft_trans_gc_queue_async(struct nft_trans_gc *gc, unsigned int gc_seq, gfp_t gfp) { + struct nft_set *set; + if (nft_trans_gc_space(gc)) return gc; + set = gc->set; nft_trans_gc_queue_work(gc); - return nft_trans_gc_alloc(gc->set, gc_seq, gfp); + return nft_trans_gc_alloc(set, gc_seq, gfp); } void nft_trans_gc_queue_async_done(struct nft_trans_gc *trans) @@ -9582,15 +9612,18 @@ void nft_trans_gc_queue_async_done(struct nft_trans_gc *trans) struct nft_trans_gc *nft_trans_gc_queue_sync(struct nft_trans_gc *gc, gfp_t gfp) { + struct nft_set *set; + if (WARN_ON_ONCE(!lockdep_commit_lock_is_held(gc->net))) return NULL; if (nft_trans_gc_space(gc)) return gc; + set = gc->set; call_rcu(&gc->rcu, nft_trans_gc_trans_free); - return nft_trans_gc_alloc(gc->set, 0, gfp); + return nft_trans_gc_alloc(set, 0, gfp); } void nft_trans_gc_queue_sync_done(struct nft_trans_gc *trans) @@ -9605,8 +9638,9 @@ void nft_trans_gc_queue_sync_done(struct nft_trans_gc *trans) call_rcu(&trans->rcu, nft_trans_gc_trans_free); } -struct nft_trans_gc *nft_trans_gc_catchall(struct nft_trans_gc *gc, - unsigned int gc_seq) +static struct nft_trans_gc *nft_trans_gc_catchall(struct nft_trans_gc *gc, + unsigned int gc_seq, + bool sync) { struct nft_set_elem_catchall *catchall; const struct nft_set *set = gc->set; @@ -9622,7 +9656,11 @@ struct nft_trans_gc *nft_trans_gc_catchall(struct nft_trans_gc *gc, nft_set_elem_dead(ext); dead_elem: - gc = nft_trans_gc_queue_async(gc, gc_seq, GFP_ATOMIC); + if (sync) + gc = nft_trans_gc_queue_sync(gc, GFP_ATOMIC); + else + gc = nft_trans_gc_queue_async(gc, gc_seq, GFP_ATOMIC); + if (!gc) return NULL; @@ -9632,6 +9670,17 @@ dead_elem: return gc; } +struct nft_trans_gc *nft_trans_gc_catchall_async(struct nft_trans_gc *gc, + unsigned int gc_seq) +{ + return nft_trans_gc_catchall(gc, gc_seq, false); +} + +struct nft_trans_gc *nft_trans_gc_catchall_sync(struct nft_trans_gc *gc) +{ + return nft_trans_gc_catchall(gc, 0, true); +} + static void nf_tables_module_autoload_cleanup(struct net *net) { struct nftables_pernet *nft_net = nft_pernet(net); @@ -11054,7 +11103,7 @@ static void __nft_release_table(struct net *net, struct nft_table *table) ctx.family = table->family; ctx.table = table; list_for_each_entry(chain, &table->chains, list) { - if (nft_chain_is_bound(chain)) + if (nft_chain_binding(chain)) continue; ctx.chain = chain; diff --git a/net/netfilter/nfnetlink_log.c b/net/netfilter/nfnetlink_log.c index 53c9e76473ba..f03f4d4d7d88 100644 --- a/net/netfilter/nfnetlink_log.c +++ b/net/netfilter/nfnetlink_log.c @@ -698,8 +698,8 @@ nfulnl_log_packet(struct net *net, unsigned int plen = 0; struct nfnl_log_net *log = nfnl_log_pernet(net); const struct nfnl_ct_hook *nfnl_ct = NULL; + enum ip_conntrack_info ctinfo = 0; struct nf_conn *ct = NULL; - enum ip_conntrack_info ctinfo; if (li_user && li_user->type == NF_LOG_TYPE_ULOG) li = li_user; diff --git a/net/netfilter/nft_inner.c b/net/netfilter/nft_inner.c index 28e2873ba24e..928312d01eb1 100644 --- a/net/netfilter/nft_inner.c +++ b/net/netfilter/nft_inner.c @@ -298,6 +298,7 @@ static int nft_inner_init(const struct nft_ctx *ctx, int err; if (!tb[NFTA_INNER_FLAGS] || + !tb[NFTA_INNER_NUM] || !tb[NFTA_INNER_HDRSIZE] || !tb[NFTA_INNER_TYPE] || !tb[NFTA_INNER_EXPR]) diff --git a/net/netfilter/nft_payload.c b/net/netfilter/nft_payload.c index 8cb800989947..0a689c8e0295 100644 --- a/net/netfilter/nft_payload.c +++ b/net/netfilter/nft_payload.c @@ -154,6 +154,17 @@ int nft_payload_inner_offset(const struct nft_pktinfo *pkt) return pkt->inneroff; } +static bool nft_payload_need_vlan_copy(const struct nft_payload *priv) +{ + unsigned int len = priv->offset + priv->len; + + /* data past ether src/dst requested, copy needed */ + if (len > offsetof(struct ethhdr, h_proto)) + return true; + + return false; +} + void nft_payload_eval(const struct nft_expr *expr, struct nft_regs *regs, const struct nft_pktinfo *pkt) @@ -168,11 +179,11 @@ void nft_payload_eval(const struct nft_expr *expr, switch (priv->base) { case NFT_PAYLOAD_LL_HEADER: - if (!skb_mac_header_was_set(skb)) + if (!skb_mac_header_was_set(skb) || skb_mac_header_len(skb) == 0) goto err; if (skb_vlan_tag_present(skb) && - priv->offset >= offsetof(struct ethhdr, h_proto)) { + nft_payload_need_vlan_copy(priv)) { if (!nft_payload_copy_vlan(dest, skb, priv->offset, priv->len)) goto err; diff --git a/net/netfilter/nft_set_hash.c b/net/netfilter/nft_set_hash.c index 524763659f25..2013de934cef 100644 --- a/net/netfilter/nft_set_hash.c +++ b/net/netfilter/nft_set_hash.c @@ -338,12 +338,9 @@ static void nft_rhash_gc(struct work_struct *work) while ((he = rhashtable_walk_next(&hti))) { if (IS_ERR(he)) { - if (PTR_ERR(he) != -EAGAIN) { - nft_trans_gc_destroy(gc); - gc = NULL; - goto try_later; - } - continue; + nft_trans_gc_destroy(gc); + gc = NULL; + goto try_later; } /* Ruleset has been updated, try later. */ @@ -372,7 +369,7 @@ dead_elem: nft_trans_gc_elem_add(gc, he); } - gc = nft_trans_gc_catchall(gc, gc_seq); + gc = nft_trans_gc_catchall_async(gc, gc_seq); try_later: /* catchall list iteration requires rcu read side lock. */ diff --git a/net/netfilter/nft_set_pipapo.c b/net/netfilter/nft_set_pipapo.c index 6af9c9ed4b5c..c0dcc40de358 100644 --- a/net/netfilter/nft_set_pipapo.c +++ b/net/netfilter/nft_set_pipapo.c @@ -1596,7 +1596,7 @@ static void pipapo_gc(const struct nft_set *_set, struct nft_pipapo_match *m) gc = nft_trans_gc_queue_sync(gc, GFP_ATOMIC); if (!gc) - break; + return; nft_pipapo_gc_deactivate(net, set, e); pipapo_drop(m, rulemap); @@ -1610,7 +1610,7 @@ static void pipapo_gc(const struct nft_set *_set, struct nft_pipapo_match *m) } } - gc = nft_trans_gc_catchall(gc, 0); + gc = nft_trans_gc_catchall_sync(gc); if (gc) { nft_trans_gc_queue_sync_done(gc); priv->last_gc = jiffies; diff --git a/net/netfilter/nft_set_pipapo.h b/net/netfilter/nft_set_pipapo.h index 25a75591583e..2e164a319945 100644 --- a/net/netfilter/nft_set_pipapo.h +++ b/net/netfilter/nft_set_pipapo.h @@ -147,7 +147,7 @@ struct nft_pipapo_match { unsigned long * __percpu *scratch; size_t bsize_max; struct rcu_head rcu; - struct nft_pipapo_field f[]; + struct nft_pipapo_field f[] __counted_by(field_count); }; /** diff --git a/net/netfilter/nft_set_rbtree.c b/net/netfilter/nft_set_rbtree.c index f250b5399344..e34662f4a71e 100644 --- a/net/netfilter/nft_set_rbtree.c +++ b/net/netfilter/nft_set_rbtree.c @@ -233,10 +233,9 @@ static void nft_rbtree_gc_remove(struct net *net, struct nft_set *set, rb_erase(&rbe->node, &priv->root); } -static int nft_rbtree_gc_elem(const struct nft_set *__set, - struct nft_rbtree *priv, - struct nft_rbtree_elem *rbe, - u8 genmask) +static const struct nft_rbtree_elem * +nft_rbtree_gc_elem(const struct nft_set *__set, struct nft_rbtree *priv, + struct nft_rbtree_elem *rbe, u8 genmask) { struct nft_set *set = (struct nft_set *)__set; struct rb_node *prev = rb_prev(&rbe->node); @@ -246,7 +245,7 @@ static int nft_rbtree_gc_elem(const struct nft_set *__set, gc = nft_trans_gc_alloc(set, 0, GFP_ATOMIC); if (!gc) - return -ENOMEM; + return ERR_PTR(-ENOMEM); /* search for end interval coming before this element. * end intervals don't carry a timeout extension, they @@ -261,6 +260,7 @@ static int nft_rbtree_gc_elem(const struct nft_set *__set, prev = rb_prev(prev); } + rbe_prev = NULL; if (prev) { rbe_prev = rb_entry(prev, struct nft_rbtree_elem, node); nft_rbtree_gc_remove(net, set, priv, rbe_prev); @@ -272,7 +272,7 @@ static int nft_rbtree_gc_elem(const struct nft_set *__set, */ gc = nft_trans_gc_queue_sync(gc, GFP_ATOMIC); if (WARN_ON_ONCE(!gc)) - return -ENOMEM; + return ERR_PTR(-ENOMEM); nft_trans_gc_elem_add(gc, rbe_prev); } @@ -280,13 +280,13 @@ static int nft_rbtree_gc_elem(const struct nft_set *__set, nft_rbtree_gc_remove(net, set, priv, rbe); gc = nft_trans_gc_queue_sync(gc, GFP_ATOMIC); if (WARN_ON_ONCE(!gc)) - return -ENOMEM; + return ERR_PTR(-ENOMEM); nft_trans_gc_elem_add(gc, rbe); nft_trans_gc_queue_sync_done(gc); - return 0; + return rbe_prev; } static bool nft_rbtree_update_first(const struct nft_set *set, @@ -314,7 +314,7 @@ static int __nft_rbtree_insert(const struct net *net, const struct nft_set *set, struct nft_rbtree *priv = nft_set_priv(set); u8 cur_genmask = nft_genmask_cur(net); u8 genmask = nft_genmask_next(net); - int d, err; + int d; /* Descend the tree to search for an existing element greater than the * key value to insert that is greater than the new element. This is the @@ -363,9 +363,14 @@ static int __nft_rbtree_insert(const struct net *net, const struct nft_set *set, */ if (nft_set_elem_expired(&rbe->ext) && nft_set_elem_active(&rbe->ext, cur_genmask)) { - err = nft_rbtree_gc_elem(set, priv, rbe, genmask); - if (err < 0) - return err; + const struct nft_rbtree_elem *removed_end; + + removed_end = nft_rbtree_gc_elem(set, priv, rbe, genmask); + if (IS_ERR(removed_end)) + return PTR_ERR(removed_end); + + if (removed_end == rbe_le || removed_end == rbe_ge) + return -EAGAIN; continue; } @@ -486,11 +491,18 @@ static int nft_rbtree_insert(const struct net *net, const struct nft_set *set, struct nft_rbtree_elem *rbe = elem->priv; int err; - write_lock_bh(&priv->lock); - write_seqcount_begin(&priv->count); - err = __nft_rbtree_insert(net, set, rbe, ext); - write_seqcount_end(&priv->count); - write_unlock_bh(&priv->lock); + do { + if (fatal_signal_pending(current)) + return -EINTR; + + cond_resched(); + + write_lock_bh(&priv->lock); + write_seqcount_begin(&priv->count); + err = __nft_rbtree_insert(net, set, rbe, ext); + write_seqcount_end(&priv->count); + write_unlock_bh(&priv->lock); + } while (err == -EAGAIN); return err; } @@ -556,6 +568,8 @@ static void *nft_rbtree_deactivate(const struct net *net, nft_rbtree_interval_end(this)) { parent = parent->rb_right; continue; + } else if (nft_set_elem_expired(&rbe->ext)) { + break; } else if (!nft_set_elem_active(&rbe->ext, genmask)) { parent = parent->rb_left; continue; @@ -622,8 +636,7 @@ static void nft_rbtree_gc(struct work_struct *work) if (!gc) goto done; - write_lock_bh(&priv->lock); - write_seqcount_begin(&priv->count); + read_lock_bh(&priv->lock); for (node = rb_first(&priv->root); node != NULL; node = rb_next(node)) { /* Ruleset has been updated, try later. */ @@ -670,11 +683,10 @@ dead_elem: nft_trans_gc_elem_add(gc, rbe); } - gc = nft_trans_gc_catchall(gc, gc_seq); + gc = nft_trans_gc_catchall_async(gc, gc_seq); try_later: - write_seqcount_end(&priv->count); - write_unlock_bh(&priv->lock); + read_unlock_bh(&priv->lock); if (gc) nft_trans_gc_queue_async_done(gc); diff --git a/net/netlink/af_netlink.c b/net/netlink/af_netlink.c index 642b9d382fb4..eb086b06d60d 100644 --- a/net/netlink/af_netlink.c +++ b/net/netlink/af_netlink.c @@ -352,7 +352,7 @@ static void netlink_overrun(struct sock *sk) if (!nlk_test_bit(RECV_NO_ENOBUFS, sk)) { if (!test_and_set_bit(NETLINK_S_CONGESTED, &nlk_sk(sk)->state)) { - sk->sk_err = ENOBUFS; + WRITE_ONCE(sk->sk_err, ENOBUFS); sk_error_report(sk); } } @@ -1605,7 +1605,7 @@ static int do_one_set_err(struct sock *sk, struct netlink_set_err_data *p) goto out; } - sk->sk_err = p->code; + WRITE_ONCE(sk->sk_err, p->code); sk_error_report(sk); out: return ret; @@ -1991,7 +1991,7 @@ static int netlink_recvmsg(struct socket *sock, struct msghdr *msg, size_t len, atomic_read(&sk->sk_rmem_alloc) <= sk->sk_rcvbuf / 2) { ret = netlink_dump(sk); if (ret) { - sk->sk_err = -ret; + WRITE_ONCE(sk->sk_err, -ret); sk_error_report(sk); } } @@ -2511,7 +2511,7 @@ void netlink_ack(struct sk_buff *in_skb, struct nlmsghdr *nlh, int err, err_bad_put: nlmsg_free(skb); err_skb: - NETLINK_CB(in_skb).sk->sk_err = ENOBUFS; + WRITE_ONCE(NETLINK_CB(in_skb).sk->sk_err, ENOBUFS); sk_error_report(NETLINK_CB(in_skb).sk); } EXPORT_SYMBOL(netlink_ack); diff --git a/net/nfc/llcp_core.c b/net/nfc/llcp_core.c index f60e424e0607..1dac28136e6a 100644 --- a/net/nfc/llcp_core.c +++ b/net/nfc/llcp_core.c @@ -203,17 +203,13 @@ static struct nfc_llcp_sock *nfc_llcp_sock_get(struct nfc_llcp_local *local, if (tmp_sock->ssap == ssap && tmp_sock->dsap == dsap) { llcp_sock = tmp_sock; + sock_hold(&llcp_sock->sk); break; } } read_unlock(&local->sockets.lock); - if (llcp_sock == NULL) - return NULL; - - sock_hold(&llcp_sock->sk); - return llcp_sock; } @@ -346,7 +342,8 @@ static int nfc_llcp_wks_sap(const char *service_name, size_t service_name_len) static struct nfc_llcp_sock *nfc_llcp_sock_from_sn(struct nfc_llcp_local *local, - const u8 *sn, size_t sn_len) + const u8 *sn, size_t sn_len, + bool needref) { struct sock *sk; struct nfc_llcp_sock *llcp_sock, *tmp_sock; @@ -382,6 +379,8 @@ struct nfc_llcp_sock *nfc_llcp_sock_from_sn(struct nfc_llcp_local *local, if (memcmp(sn, tmp_sock->service_name, sn_len) == 0) { llcp_sock = tmp_sock; + if (needref) + sock_hold(&llcp_sock->sk); break; } } @@ -423,7 +422,8 @@ u8 nfc_llcp_get_sdp_ssap(struct nfc_llcp_local *local, * to this service name. */ if (nfc_llcp_sock_from_sn(local, sock->service_name, - sock->service_name_len) != NULL) { + sock->service_name_len, + false) != NULL) { mutex_unlock(&local->sdp_lock); return LLCP_SAP_MAX; @@ -824,16 +824,7 @@ out: static struct nfc_llcp_sock *nfc_llcp_sock_get_sn(struct nfc_llcp_local *local, const u8 *sn, size_t sn_len) { - struct nfc_llcp_sock *llcp_sock; - - llcp_sock = nfc_llcp_sock_from_sn(local, sn, sn_len); - - if (llcp_sock == NULL) - return NULL; - - sock_hold(&llcp_sock->sk); - - return llcp_sock; + return nfc_llcp_sock_from_sn(local, sn, sn_len, true); } static const u8 *nfc_llcp_connect_sn(const struct sk_buff *skb, size_t *sn_len) @@ -1298,7 +1289,8 @@ static void nfc_llcp_recv_snl(struct nfc_llcp_local *local, } llcp_sock = nfc_llcp_sock_from_sn(local, service_name, - service_name_len); + service_name_len, + true); if (!llcp_sock) { sap = 0; goto add_snl; @@ -1318,6 +1310,7 @@ static void nfc_llcp_recv_snl(struct nfc_llcp_local *local, if (sap == LLCP_SAP_MAX) { sap = 0; + nfc_llcp_sock_put(llcp_sock); goto add_snl; } @@ -1335,6 +1328,7 @@ static void nfc_llcp_recv_snl(struct nfc_llcp_local *local, pr_debug("%p %d\n", llcp_sock, sap); + nfc_llcp_sock_put(llcp_sock); add_snl: sdp = nfc_llcp_build_sdres_tlv(tid, sap); if (sdp == NULL) @@ -1636,7 +1630,9 @@ int nfc_llcp_register_device(struct nfc_dev *ndev) timer_setup(&local->sdreq_timer, nfc_llcp_sdreq_timer, 0); INIT_WORK(&local->sdreq_timeout_work, nfc_llcp_sdreq_timeout_work); + spin_lock(&llcp_devices_lock); list_add(&local->list, &llcp_devices); + spin_unlock(&llcp_devices_lock); return 0; } diff --git a/net/nfc/nci/core.c b/net/nfc/nci/core.c index fff755dde30d..6c9592d05120 100644 --- a/net/nfc/nci/core.c +++ b/net/nfc/nci/core.c @@ -909,6 +909,11 @@ static int nci_activate_target(struct nfc_dev *nfc_dev, return -EINVAL; } + if (protocol >= NFC_PROTO_MAX) { + pr_err("the requested nfc protocol is invalid\n"); + return -EINVAL; + } + if (!(nci_target->supported_protocols & (1 << protocol))) { pr_err("target does not support the requested protocol 0x%x\n", protocol); diff --git a/net/nfc/nci/spi.c b/net/nfc/nci/spi.c index 0935527d1d12..b68150c971d0 100644 --- a/net/nfc/nci/spi.c +++ b/net/nfc/nci/spi.c @@ -151,6 +151,8 @@ static int send_acknowledge(struct nci_spi *nspi, u8 acknowledge) int ret; skb = nci_skb_alloc(nspi->ndev, 0, GFP_KERNEL); + if (!skb) + return -ENOMEM; /* add the NCI SPI header to the start of the buffer */ hdr = skb_push(skb, NCI_SPI_HDR_LEN); diff --git a/net/packet/af_packet.c b/net/packet/af_packet.c index 8f97648d652f..a84e00b5904b 100644 --- a/net/packet/af_packet.c +++ b/net/packet/af_packet.c @@ -3607,7 +3607,12 @@ static int packet_getname(struct socket *sock, struct sockaddr *uaddr, if (dev) { sll->sll_hatype = dev->type; sll->sll_halen = dev->addr_len; - memcpy(sll->sll_addr_flex, dev->dev_addr, dev->addr_len); + + /* Let __fortify_memcpy_chk() know the actual buffer size. */ + memcpy(((struct sockaddr_storage *)sll)->__data + + offsetof(struct sockaddr_ll, sll_addr) - + offsetofend(struct sockaddr_ll, sll_family), + dev->dev_addr, dev->addr_len); } else { sll->sll_hatype = 0; /* Bad: we have no ARPHRD_UNSPEC */ sll->sll_halen = 0; diff --git a/net/rds/rdma_transport.c b/net/rds/rdma_transport.c index d36f3f6b4351..b15cf316b23a 100644 --- a/net/rds/rdma_transport.c +++ b/net/rds/rdma_transport.c @@ -86,11 +86,13 @@ static int rds_rdma_cm_event_handler_cmn(struct rdma_cm_id *cm_id, break; case RDMA_CM_EVENT_ADDR_RESOLVED: - rdma_set_service_type(cm_id, conn->c_tos); - rdma_set_min_rnr_timer(cm_id, IB_RNR_TIMER_000_32); - /* XXX do we need to clean up if this fails? */ - ret = rdma_resolve_route(cm_id, - RDS_RDMA_RESOLVE_TIMEOUT_MS); + if (conn) { + rdma_set_service_type(cm_id, conn->c_tos); + rdma_set_min_rnr_timer(cm_id, IB_RNR_TIMER_000_32); + /* XXX do we need to clean up if this fails? */ + ret = rdma_resolve_route(cm_id, + RDS_RDMA_RESOLVE_TIMEOUT_MS); + } break; case RDMA_CM_EVENT_ROUTE_RESOLVED: diff --git a/net/rds/tcp_connect.c b/net/rds/tcp_connect.c index f0c477c5d1db..a0046e99d6df 100644 --- a/net/rds/tcp_connect.c +++ b/net/rds/tcp_connect.c @@ -145,7 +145,7 @@ int rds_tcp_conn_path_connect(struct rds_conn_path *cp) addrlen = sizeof(sin); } - ret = sock->ops->bind(sock, addr, addrlen); + ret = kernel_bind(sock, addr, addrlen); if (ret) { rdsdebug("bind failed with %d at address %pI6c\n", ret, &conn->c_laddr); @@ -173,7 +173,7 @@ int rds_tcp_conn_path_connect(struct rds_conn_path *cp) * own the socket */ rds_tcp_set_callbacks(sock, cp); - ret = sock->ops->connect(sock, addr, addrlen, O_NONBLOCK); + ret = kernel_connect(sock, addr, addrlen, O_NONBLOCK); rdsdebug("connect to address %pI6c returned %d\n", &conn->c_faddr, ret); if (ret == -EINPROGRESS) diff --git a/net/rds/tcp_listen.c b/net/rds/tcp_listen.c index 014fa24418c1..53b3535a1e4a 100644 --- a/net/rds/tcp_listen.c +++ b/net/rds/tcp_listen.c @@ -306,7 +306,7 @@ struct socket *rds_tcp_listen_init(struct net *net, bool isv6) addr_len = sizeof(*sin); } - ret = sock->ops->bind(sock, (struct sockaddr *)&ss, addr_len); + ret = kernel_bind(sock, (struct sockaddr *)&ss, addr_len); if (ret < 0) { rdsdebug("could not bind %s listener socket: %d\n", isv6 ? "IPv6" : "IPv4", ret); diff --git a/net/rfkill/core.c b/net/rfkill/core.c index 01fca7a10b4b..14cc8fe8584b 100644 --- a/net/rfkill/core.c +++ b/net/rfkill/core.c @@ -48,6 +48,7 @@ struct rfkill { bool persistent; bool polling_paused; bool suspended; + bool need_sync; const struct rfkill_ops *ops; void *data; @@ -368,6 +369,17 @@ static void rfkill_set_block(struct rfkill *rfkill, bool blocked) rfkill_event(rfkill); } +static void rfkill_sync(struct rfkill *rfkill) +{ + lockdep_assert_held(&rfkill_global_mutex); + + if (!rfkill->need_sync) + return; + + rfkill_set_block(rfkill, rfkill_global_states[rfkill->type].cur); + rfkill->need_sync = false; +} + static void rfkill_update_global_state(enum rfkill_type type, bool blocked) { int i; @@ -730,6 +742,10 @@ static ssize_t soft_show(struct device *dev, struct device_attribute *attr, { struct rfkill *rfkill = to_rfkill(dev); + mutex_lock(&rfkill_global_mutex); + rfkill_sync(rfkill); + mutex_unlock(&rfkill_global_mutex); + return sysfs_emit(buf, "%d\n", (rfkill->state & RFKILL_BLOCK_SW) ? 1 : 0); } @@ -751,6 +767,7 @@ static ssize_t soft_store(struct device *dev, struct device_attribute *attr, return -EINVAL; mutex_lock(&rfkill_global_mutex); + rfkill_sync(rfkill); rfkill_set_block(rfkill, state); mutex_unlock(&rfkill_global_mutex); @@ -783,6 +800,10 @@ static ssize_t state_show(struct device *dev, struct device_attribute *attr, { struct rfkill *rfkill = to_rfkill(dev); + mutex_lock(&rfkill_global_mutex); + rfkill_sync(rfkill); + mutex_unlock(&rfkill_global_mutex); + return sysfs_emit(buf, "%d\n", user_state_from_blocked(rfkill->state)); } @@ -805,6 +826,7 @@ static ssize_t state_store(struct device *dev, struct device_attribute *attr, return -EINVAL; mutex_lock(&rfkill_global_mutex); + rfkill_sync(rfkill); rfkill_set_block(rfkill, state == RFKILL_USER_STATE_SOFT_BLOCKED); mutex_unlock(&rfkill_global_mutex); @@ -1032,14 +1054,10 @@ static void rfkill_uevent_work(struct work_struct *work) static void rfkill_sync_work(struct work_struct *work) { - struct rfkill *rfkill; - bool cur; - - rfkill = container_of(work, struct rfkill, sync_work); + struct rfkill *rfkill = container_of(work, struct rfkill, sync_work); mutex_lock(&rfkill_global_mutex); - cur = rfkill_global_states[rfkill->type].cur; - rfkill_set_block(rfkill, cur); + rfkill_sync(rfkill); mutex_unlock(&rfkill_global_mutex); } @@ -1087,6 +1105,7 @@ int __must_check rfkill_register(struct rfkill *rfkill) round_jiffies_relative(POLL_INTERVAL)); if (!rfkill->persistent || rfkill_epo_lock_active) { + rfkill->need_sync = true; schedule_work(&rfkill->sync_work); } else { #ifdef CONFIG_RFKILL_INPUT @@ -1161,7 +1180,6 @@ static int rfkill_fop_open(struct inode *inode, struct file *file) init_waitqueue_head(&data->read_wait); mutex_lock(&rfkill_global_mutex); - mutex_lock(&data->mtx); /* * start getting events from elsewhere but hold mtx to get * startup events added first @@ -1171,11 +1189,13 @@ static int rfkill_fop_open(struct inode *inode, struct file *file) ev = kzalloc(sizeof(*ev), GFP_KERNEL); if (!ev) goto free; + rfkill_sync(rfkill); rfkill_fill_event(&ev->ev, rfkill, RFKILL_OP_ADD); + mutex_lock(&data->mtx); list_add_tail(&ev->list, &data->events); + mutex_unlock(&data->mtx); } list_add(&data->list, &rfkill_fds); - mutex_unlock(&data->mtx); mutex_unlock(&rfkill_global_mutex); file->private_data = data; @@ -1183,7 +1203,6 @@ static int rfkill_fop_open(struct inode *inode, struct file *file) return stream_open(inode, file); free: - mutex_unlock(&data->mtx); mutex_unlock(&rfkill_global_mutex); mutex_destroy(&data->mtx); list_for_each_entry_safe(ev, tmp, &data->events, list) diff --git a/net/rfkill/rfkill-gpio.c b/net/rfkill/rfkill-gpio.c index e9d1b2f2ff0a..5a81505fba9a 100644 --- a/net/rfkill/rfkill-gpio.c +++ b/net/rfkill/rfkill-gpio.c @@ -108,13 +108,13 @@ static int rfkill_gpio_probe(struct platform_device *pdev) rfkill->clk = devm_clk_get(&pdev->dev, NULL); - gpio = devm_gpiod_get_optional(&pdev->dev, "reset", GPIOD_OUT_LOW); + gpio = devm_gpiod_get_optional(&pdev->dev, "reset", GPIOD_ASIS); if (IS_ERR(gpio)) return PTR_ERR(gpio); rfkill->reset_gpio = gpio; - gpio = devm_gpiod_get_optional(&pdev->dev, "shutdown", GPIOD_OUT_LOW); + gpio = devm_gpiod_get_optional(&pdev->dev, "shutdown", GPIOD_ASIS); if (IS_ERR(gpio)) return PTR_ERR(gpio); diff --git a/net/sched/cls_u32.c b/net/sched/cls_u32.c index da4c179a4d41..6663e971a13e 100644 --- a/net/sched/cls_u32.c +++ b/net/sched/cls_u32.c @@ -366,7 +366,7 @@ static int u32_init(struct tcf_proto *tp) idr_init(&root_ht->handle_idr); if (tp_c == NULL) { - tp_c = kzalloc(struct_size(tp_c, hlist->ht, 1), GFP_KERNEL); + tp_c = kzalloc(sizeof(*tp_c), GFP_KERNEL); if (tp_c == NULL) { kfree(root_ht); return -ENOBUFS; diff --git a/net/sched/sch_hfsc.c b/net/sched/sch_hfsc.c index 3554085bc2be..880c5f16b29c 100644 --- a/net/sched/sch_hfsc.c +++ b/net/sched/sch_hfsc.c @@ -902,6 +902,14 @@ hfsc_change_usc(struct hfsc_class *cl, struct tc_service_curve *usc, cl->cl_flags |= HFSC_USC; } +static void +hfsc_upgrade_rt(struct hfsc_class *cl) +{ + cl->cl_fsc = cl->cl_rsc; + rtsc_init(&cl->cl_virtual, &cl->cl_fsc, cl->cl_vt, cl->cl_total); + cl->cl_flags |= HFSC_FSC; +} + static const struct nla_policy hfsc_policy[TCA_HFSC_MAX + 1] = { [TCA_HFSC_RSC] = { .len = sizeof(struct tc_service_curve) }, [TCA_HFSC_FSC] = { .len = sizeof(struct tc_service_curve) }, @@ -1011,10 +1019,6 @@ hfsc_change_class(struct Qdisc *sch, u32 classid, u32 parentid, if (parent == NULL) return -ENOENT; } - if (!(parent->cl_flags & HFSC_FSC) && parent != &q->root) { - NL_SET_ERR_MSG(extack, "Invalid parent - parent class must have FSC"); - return -EINVAL; - } if (classid == 0 || TC_H_MAJ(classid ^ sch->handle) != 0) return -EINVAL; @@ -1065,6 +1069,12 @@ hfsc_change_class(struct Qdisc *sch, u32 classid, u32 parentid, cl->cf_tree = RB_ROOT; sch_tree_lock(sch); + /* Check if the inner class is a misconfigured 'rt' */ + if (!(parent->cl_flags & HFSC_FSC) && parent != &q->root) { + NL_SET_ERR_MSG(extack, + "Forced curve change on parent 'rt' to 'sc'"); + hfsc_upgrade_rt(parent); + } qdisc_class_hash_insert(&q->clhash, &cl->cl_common); list_add_tail(&cl->siblings, &parent->children); if (parent->level == 0) diff --git a/net/sctp/associola.c b/net/sctp/associola.c index 796529167e8d..c45c192b7878 100644 --- a/net/sctp/associola.c +++ b/net/sctp/associola.c @@ -1159,8 +1159,7 @@ int sctp_assoc_update(struct sctp_association *asoc, /* Add any peer addresses from the new association. */ list_for_each_entry(trans, &new->peer.transport_addr_list, transports) - if (!sctp_assoc_lookup_paddr(asoc, &trans->ipaddr) && - !sctp_assoc_add_peer(asoc, &trans->ipaddr, + if (!sctp_assoc_add_peer(asoc, &trans->ipaddr, GFP_ATOMIC, trans->state)) return -ENOMEM; diff --git a/net/sctp/socket.c b/net/sctp/socket.c index ab943e8fb1db..7f89e43154c0 100644 --- a/net/sctp/socket.c +++ b/net/sctp/socket.c @@ -2450,6 +2450,7 @@ static int sctp_apply_peer_addr_params(struct sctp_paddrparams *params, if (trans) { trans->hbinterval = msecs_to_jiffies(params->spp_hbinterval); + sctp_transport_reset_hb_timer(trans); } else if (asoc) { asoc->hbinterval = msecs_to_jiffies(params->spp_hbinterval); diff --git a/net/smc/Kconfig b/net/smc/Kconfig index 1ab3c5a2c5ad..746be3996768 100644 --- a/net/smc/Kconfig +++ b/net/smc/Kconfig @@ -2,6 +2,7 @@ config SMC tristate "SMC socket protocol family" depends on INET && INFINIBAND + depends on m || ISM != m help SMC-R provides a "sockets over RDMA" solution making use of RDMA over Converged Ethernet (RoCE) technology to upgrade diff --git a/net/smc/af_smc.c b/net/smc/af_smc.c index bacdd971615e..35ddebae8894 100644 --- a/net/smc/af_smc.c +++ b/net/smc/af_smc.c @@ -1201,6 +1201,7 @@ static int smc_connect_rdma_v2_prepare(struct smc_sock *smc, (struct smc_clc_msg_accept_confirm_v2 *)aclc; struct smc_clc_first_contact_ext *fce = smc_get_clc_first_contact_ext(clc_v2, false); + struct net *net = sock_net(&smc->sk); int rc; if (!ini->first_contact_peer || aclc->hdr.version == SMC_V1) @@ -1210,7 +1211,7 @@ static int smc_connect_rdma_v2_prepare(struct smc_sock *smc, memcpy(ini->smcrv2.nexthop_mac, &aclc->r0.lcl.mac, ETH_ALEN); ini->smcrv2.uses_gateway = false; } else { - if (smc_ib_find_route(smc->clcsock->sk->sk_rcv_saddr, + if (smc_ib_find_route(net, smc->clcsock->sk->sk_rcv_saddr, smc_ib_gid_to_ipv4(aclc->r0.lcl.gid), ini->smcrv2.nexthop_mac, &ini->smcrv2.uses_gateway)) @@ -2361,7 +2362,7 @@ static int smc_listen_find_device(struct smc_sock *new_smc, smc_find_ism_store_rc(rc, ini); return (!rc) ? 0 : ini->rc; } - return SMC_CLC_DECL_NOSMCDEV; + return prfx_rc; } /* listen worker: finish RDMA setup */ diff --git a/net/smc/smc_core.c b/net/smc/smc_core.c index bd01dd31e4bd..d520ee62c8ec 100644 --- a/net/smc/smc_core.c +++ b/net/smc/smc_core.c @@ -1662,6 +1662,7 @@ void smcr_port_add(struct smc_ib_device *smcibdev, u8 ibport) { struct smc_link_group *lgr, *n; + spin_lock_bh(&smc_lgr_list.lock); list_for_each_entry_safe(lgr, n, &smc_lgr_list.list, list) { struct smc_link *link; @@ -1680,6 +1681,7 @@ void smcr_port_add(struct smc_ib_device *smcibdev, u8 ibport) if (link) smc_llc_add_link_local(link); } + spin_unlock_bh(&smc_lgr_list.lock); } /* link is down - switch connections to alternate link, diff --git a/net/smc/smc_ib.c b/net/smc/smc_ib.c index 9b66d6aeeb1a..89981dbe46c9 100644 --- a/net/smc/smc_ib.c +++ b/net/smc/smc_ib.c @@ -193,7 +193,7 @@ bool smc_ib_port_active(struct smc_ib_device *smcibdev, u8 ibport) return smcibdev->pattr[ibport - 1].state == IB_PORT_ACTIVE; } -int smc_ib_find_route(__be32 saddr, __be32 daddr, +int smc_ib_find_route(struct net *net, __be32 saddr, __be32 daddr, u8 nexthop_mac[], u8 *uses_gateway) { struct neighbour *neigh = NULL; @@ -205,7 +205,7 @@ int smc_ib_find_route(__be32 saddr, __be32 daddr, if (daddr == cpu_to_be32(INADDR_NONE)) goto out; - rt = ip_route_output_flow(&init_net, &fl4, NULL); + rt = ip_route_output_flow(net, &fl4, NULL); if (IS_ERR(rt)) goto out; if (rt->rt_uses_gateway && rt->rt_gw_family != AF_INET) @@ -235,6 +235,7 @@ static int smc_ib_determine_gid_rcu(const struct net_device *ndev, if (smcrv2 && attr->gid_type == IB_GID_TYPE_ROCE_UDP_ENCAP && smc_ib_gid_to_ipv4((u8 *)&attr->gid) != cpu_to_be32(INADDR_NONE)) { struct in_device *in_dev = __in_dev_get_rcu(ndev); + struct net *net = dev_net(ndev); const struct in_ifaddr *ifa; bool subnet_match = false; @@ -248,7 +249,7 @@ static int smc_ib_determine_gid_rcu(const struct net_device *ndev, } if (!subnet_match) goto out; - if (smcrv2->daddr && smc_ib_find_route(smcrv2->saddr, + if (smcrv2->daddr && smc_ib_find_route(net, smcrv2->saddr, smcrv2->daddr, smcrv2->nexthop_mac, &smcrv2->uses_gateway)) diff --git a/net/smc/smc_ib.h b/net/smc/smc_ib.h index 4df5f8c8a0a1..ef8ac2b7546d 100644 --- a/net/smc/smc_ib.h +++ b/net/smc/smc_ib.h @@ -112,7 +112,7 @@ void smc_ib_sync_sg_for_device(struct smc_link *lnk, int smc_ib_determine_gid(struct smc_ib_device *smcibdev, u8 ibport, unsigned short vlan_id, u8 gid[], u8 *sgid_index, struct smc_init_info_smcrv2 *smcrv2); -int smc_ib_find_route(__be32 saddr, __be32 daddr, +int smc_ib_find_route(struct net *net, __be32 saddr, __be32 daddr, u8 nexthop_mac[], u8 *uses_gateway); bool smc_ib_is_valid_local_systemid(void); int smcr_nl_get_device(struct sk_buff *skb, struct netlink_callback *cb); diff --git a/net/smc/smc_stats.h b/net/smc/smc_stats.h index b60fe1eb37ab..9d32058db2b5 100644 --- a/net/smc/smc_stats.h +++ b/net/smc/smc_stats.h @@ -92,13 +92,14 @@ do { \ typeof(_smc_stats) stats = (_smc_stats); \ typeof(_tech) t = (_tech); \ typeof(_len) l = (_len); \ - int _pos = fls64((l) >> 13); \ + int _pos; \ typeof(_rc) r = (_rc); \ int m = SMC_BUF_MAX - 1; \ this_cpu_inc((*stats).smc[t].key ## _cnt); \ - if (r <= 0) \ + if (r <= 0 || l <= 0) \ break; \ - _pos = (_pos < m) ? ((l == 1 << (_pos + 12)) ? _pos - 1 : _pos) : m; \ + _pos = fls64((l - 1) >> 13); \ + _pos = (_pos <= m) ? _pos : m; \ this_cpu_inc((*stats).smc[t].key ## _pd.buf[_pos]); \ this_cpu_add((*stats).smc[t].key ## _bytes, r); \ } \ @@ -138,9 +139,12 @@ while (0) do { \ typeof(_len) _l = (_len); \ typeof(_tech) t = (_tech); \ - int _pos = fls((_l) >> 13); \ + int _pos; \ int m = SMC_BUF_MAX - 1; \ - _pos = (_pos < m) ? ((_l == 1 << (_pos + 12)) ? _pos - 1 : _pos) : m; \ + if (_l <= 0) \ + break; \ + _pos = fls((_l - 1) >> 13); \ + _pos = (_pos <= m) ? _pos : m; \ this_cpu_inc((*(_smc_stats)).smc[t].k ## _rmbsize.buf[_pos]); \ } \ while (0) @@ -243,8 +247,9 @@ while (0) #define SMC_STAT_SERV_SUCC_INC(net, _ini) \ do { \ typeof(_ini) i = (_ini); \ - bool is_v2 = (i->smcd_version & SMC_V2); \ bool is_smcd = (i->is_smcd); \ + u8 version = is_smcd ? i->smcd_version : i->smcr_version; \ + bool is_v2 = (version & SMC_V2); \ typeof(net->smc.smc_stats) smc_stats = (net)->smc.smc_stats; \ if (is_v2 && is_smcd) \ this_cpu_inc(smc_stats->smc[SMC_TYPE_D].srv_v2_succ_cnt); \ diff --git a/net/socket.c b/net/socket.c index c8b08b32f097..c4a6f5532955 100644 --- a/net/socket.c +++ b/net/socket.c @@ -737,6 +737,14 @@ static inline int sock_sendmsg_nosec(struct socket *sock, struct msghdr *msg) return ret; } +static int __sock_sendmsg(struct socket *sock, struct msghdr *msg) +{ + int err = security_socket_sendmsg(sock, msg, + msg_data_left(msg)); + + return err ?: sock_sendmsg_nosec(sock, msg); +} + /** * sock_sendmsg - send a message through @sock * @sock: socket @@ -747,10 +755,19 @@ static inline int sock_sendmsg_nosec(struct socket *sock, struct msghdr *msg) */ int sock_sendmsg(struct socket *sock, struct msghdr *msg) { - int err = security_socket_sendmsg(sock, msg, - msg_data_left(msg)); + struct sockaddr_storage *save_addr = (struct sockaddr_storage *)msg->msg_name; + struct sockaddr_storage address; + int ret; - return err ?: sock_sendmsg_nosec(sock, msg); + if (msg->msg_name) { + memcpy(&address, msg->msg_name, msg->msg_namelen); + msg->msg_name = &address; + } + + ret = __sock_sendmsg(sock, msg); + msg->msg_name = save_addr; + + return ret; } EXPORT_SYMBOL(sock_sendmsg); @@ -1138,7 +1155,7 @@ static ssize_t sock_write_iter(struct kiocb *iocb, struct iov_iter *from) if (sock->type == SOCK_SEQPACKET) msg.msg_flags |= MSG_EOR; - res = sock_sendmsg(sock, &msg); + res = __sock_sendmsg(sock, &msg); *from = msg.msg_iter; return res; } @@ -2174,7 +2191,7 @@ int __sys_sendto(int fd, void __user *buff, size_t len, unsigned int flags, if (sock->file->f_flags & O_NONBLOCK) flags |= MSG_DONTWAIT; msg.msg_flags = flags; - err = sock_sendmsg(sock, &msg); + err = __sock_sendmsg(sock, &msg); out_put: fput_light(sock->file, fput_needed); @@ -2538,7 +2555,7 @@ static int ____sys_sendmsg(struct socket *sock, struct msghdr *msg_sys, err = sock_sendmsg_nosec(sock, msg_sys); goto out_freectl; } - err = sock_sendmsg(sock, msg_sys); + err = __sock_sendmsg(sock, msg_sys); /* * If this is sendmmsg() and sending to current destination address was * successful, remember it. @@ -3499,7 +3516,12 @@ static long compat_sock_ioctl(struct file *file, unsigned int cmd, int kernel_bind(struct socket *sock, struct sockaddr *addr, int addrlen) { - return READ_ONCE(sock->ops)->bind(sock, addr, addrlen); + struct sockaddr_storage address; + + memcpy(&address, addr, addrlen); + + return READ_ONCE(sock->ops)->bind(sock, (struct sockaddr *)&address, + addrlen); } EXPORT_SYMBOL(kernel_bind); diff --git a/net/sunrpc/auth.c b/net/sunrpc/auth.c index 2f16f9d17966..814b0169f972 100644 --- a/net/sunrpc/auth.c +++ b/net/sunrpc/auth.c @@ -769,9 +769,14 @@ int rpcauth_wrap_req(struct rpc_task *task, struct xdr_stream *xdr) * @task: controlling RPC task * @xdr: xdr_stream containing RPC Reply header * - * On success, @xdr is updated to point past the verifier and - * zero is returned. Otherwise, @xdr is in an undefined state - * and a negative errno is returned. + * Return values: + * %0: Verifier is valid. @xdr now points past the verifier. + * %-EIO: Verifier is corrupted or message ended early. + * %-EACCES: Verifier is intact but not valid. + * %-EPROTONOSUPPORT: Server does not support the requested auth type. + * + * When a negative errno is returned, @xdr is left in an undefined + * state. */ int rpcauth_checkverf(struct rpc_task *task, struct xdr_stream *xdr) diff --git a/net/sunrpc/auth_tls.c b/net/sunrpc/auth_tls.c index de7678f8a23d..87f570fd3b00 100644 --- a/net/sunrpc/auth_tls.c +++ b/net/sunrpc/auth_tls.c @@ -129,9 +129,9 @@ static int tls_validate(struct rpc_task *task, struct xdr_stream *xdr) if (*p != rpc_auth_null) return -EIO; if (xdr_stream_decode_opaque_inline(xdr, &str, starttls_len) != starttls_len) - return -EIO; + return -EPROTONOSUPPORT; if (memcmp(str, starttls_token, starttls_len)) - return -EIO; + return -EPROTONOSUPPORT; return 0; } diff --git a/net/sunrpc/clnt.c b/net/sunrpc/clnt.c index 8d75290f1a31..9c210273d06b 100644 --- a/net/sunrpc/clnt.c +++ b/net/sunrpc/clnt.c @@ -2476,8 +2476,7 @@ call_status(struct rpc_task *task) goto out_exit; } task->tk_action = call_encode; - if (status != -ECONNRESET && status != -ECONNABORTED) - rpc_check_timeout(task); + rpc_check_timeout(task); return; out_exit: rpc_call_rpcerror(task, status); @@ -2725,7 +2724,15 @@ out_unparsable: out_verifier: trace_rpc_bad_verifier(task); - goto out_err; + switch (error) { + case -EPROTONOSUPPORT: + goto out_err; + case -EACCES: + /* Re-encode with a fresh cred */ + fallthrough; + default: + goto out_garbage; + } out_msg_denied: error = -EACCES; @@ -2751,6 +2758,7 @@ out_msg_denied: case rpc_autherr_rejectedverf: case rpcsec_gsserr_credproblem: case rpcsec_gsserr_ctxproblem: + rpcauth_invalcred(task); if (!task->tk_cred_retry) break; task->tk_cred_retry--; @@ -2907,19 +2915,22 @@ static const struct rpc_call_ops rpc_cb_add_xprt_call_ops = { * @clnt: pointer to struct rpc_clnt * @xps: pointer to struct rpc_xprt_switch, * @xprt: pointer struct rpc_xprt - * @dummy: unused + * @in_max_connect: pointer to the max_connect value for the passed in xprt transport */ int rpc_clnt_test_and_add_xprt(struct rpc_clnt *clnt, struct rpc_xprt_switch *xps, struct rpc_xprt *xprt, - void *dummy) + void *in_max_connect) { struct rpc_cb_add_xprt_calldata *data; struct rpc_task *task; + int max_connect = clnt->cl_max_connect; - if (xps->xps_nunique_destaddr_xprts + 1 > clnt->cl_max_connect) { + if (in_max_connect) + max_connect = *(int *)in_max_connect; + if (xps->xps_nunique_destaddr_xprts + 1 > max_connect) { rcu_read_lock(); pr_warn("SUNRPC: reached max allowed number (%d) did not add " - "transport to server: %s\n", clnt->cl_max_connect, + "transport to server: %s\n", max_connect, rpc_peeraddr2str(clnt, RPC_DISPLAY_ADDR)); rcu_read_unlock(); return -EINVAL; diff --git a/net/sunrpc/xprtsock.c b/net/sunrpc/xprtsock.c index 71cd916e384f..a15bf2ede89b 100644 --- a/net/sunrpc/xprtsock.c +++ b/net/sunrpc/xprtsock.c @@ -2672,6 +2672,10 @@ static void xs_tcp_tls_setup_socket(struct work_struct *work) rcu_read_lock(); lower_xprt = rcu_dereference(lower_clnt->cl_xprt); rcu_read_unlock(); + + if (wait_on_bit_lock(&lower_xprt->state, XPRT_LOCKED, TASK_KILLABLE)) + goto out_unlock; + status = xs_tls_handshake_sync(lower_xprt, &upper_xprt->xprtsec); if (status) { trace_rpc_tls_not_started(upper_clnt, upper_xprt); @@ -2681,6 +2685,7 @@ static void xs_tcp_tls_setup_socket(struct work_struct *work) status = xs_tcp_tls_finish_connecting(lower_xprt, upper_transport); if (status) goto out_close; + xprt_release_write(lower_xprt, NULL); trace_rpc_socket_connect(upper_xprt, upper_transport->sock, 0); if (!xprt_test_and_set_connected(upper_xprt)) { @@ -2702,6 +2707,7 @@ out_unlock: return; out_close: + xprt_release_write(lower_xprt, NULL); rpc_shutdown_client(lower_clnt); /* xprt_force_disconnect() wakes tasks with a fixed tk_status code. diff --git a/net/tipc/crypto.c b/net/tipc/crypto.c index 302fd749c424..43c3f1c971b8 100644 --- a/net/tipc/crypto.c +++ b/net/tipc/crypto.c @@ -1441,14 +1441,14 @@ static int tipc_crypto_key_revoke(struct net *net, u8 tx_key) struct tipc_crypto *tx = tipc_net(net)->crypto_tx; struct tipc_key key; - spin_lock(&tx->lock); + spin_lock_bh(&tx->lock); key = tx->key; WARN_ON(!key.active || tx_key != key.active); /* Free the active key */ tipc_crypto_key_set_state(tx, key.passive, 0, key.pending); tipc_crypto_key_detach(tx->aead[key.active], &tx->lock); - spin_unlock(&tx->lock); + spin_unlock_bh(&tx->lock); pr_warn("%s: key is revoked\n", tx->name); return -EKEYREVOKED; diff --git a/net/tls/tls_main.c b/net/tls/tls_main.c index 02f583ff9239..002483e60c19 100644 --- a/net/tls/tls_main.c +++ b/net/tls/tls_main.c @@ -139,8 +139,8 @@ void update_sk_prot(struct sock *sk, struct tls_context *ctx) int wait_on_pending_writer(struct sock *sk, long *timeo) { - int rc = 0; DEFINE_WAIT_FUNC(wait, woken_wake_function); + int ret, rc = 0; add_wait_queue(sk_sleep(sk), &wait); while (1) { @@ -154,9 +154,13 @@ int wait_on_pending_writer(struct sock *sk, long *timeo) break; } - if (sk_wait_event(sk, timeo, - !READ_ONCE(sk->sk_write_pending), &wait)) + ret = sk_wait_event(sk, timeo, + !READ_ONCE(sk->sk_write_pending), &wait); + if (ret) { + if (ret < 0) + rc = ret; break; + } } remove_wait_queue(sk_sleep(sk), &wait); return rc; diff --git a/net/tls/tls_sw.c b/net/tls/tls_sw.c index 1ed4a611631f..e9d1e83a859d 100644 --- a/net/tls/tls_sw.c +++ b/net/tls/tls_sw.c @@ -817,7 +817,7 @@ static int bpf_exec_tx_verdict(struct sk_msg *msg, struct sock *sk, psock = sk_psock_get(sk); if (!psock || !policy) { err = tls_push_record(sk, flags, record_type); - if (err && sk->sk_err == EBADMSG) { + if (err && err != -EINPROGRESS && sk->sk_err == EBADMSG) { *copied -= sk_msg_free(sk, msg); tls_free_open_rec(sk); err = -sk->sk_err; @@ -846,7 +846,7 @@ more_data: switch (psock->eval) { case __SK_PASS: err = tls_push_record(sk, flags, record_type); - if (err && sk->sk_err == EBADMSG) { + if (err && err != -EINPROGRESS && sk->sk_err == EBADMSG) { *copied -= sk_msg_free(sk, msg); tls_free_open_rec(sk); err = -sk->sk_err; @@ -1291,6 +1291,7 @@ tls_rx_rec_wait(struct sock *sk, struct sk_psock *psock, bool nonblock, struct tls_context *tls_ctx = tls_get_ctx(sk); struct tls_sw_context_rx *ctx = tls_sw_ctx_rx(tls_ctx); DEFINE_WAIT_FUNC(wait, woken_wake_function); + int ret = 0; long timeo; timeo = sock_rcvtimeo(sk, nonblock); @@ -1302,6 +1303,9 @@ tls_rx_rec_wait(struct sock *sk, struct sk_psock *psock, bool nonblock, if (sk->sk_err) return sock_error(sk); + if (ret < 0) + return ret; + if (!skb_queue_empty(&sk->sk_receive_queue)) { tls_strp_check_rcv(&ctx->strp); if (tls_strp_msg_ready(ctx)) @@ -1320,10 +1324,10 @@ tls_rx_rec_wait(struct sock *sk, struct sk_psock *psock, bool nonblock, released = true; add_wait_queue(sk_sleep(sk), &wait); sk_set_bit(SOCKWQ_ASYNC_WAITDATA, sk); - sk_wait_event(sk, &timeo, - tls_strp_msg_ready(ctx) || - !sk_psock_queue_empty(psock), - &wait); + ret = sk_wait_event(sk, &timeo, + tls_strp_msg_ready(ctx) || + !sk_psock_queue_empty(psock), + &wait); sk_clear_bit(SOCKWQ_ASYNC_WAITDATA, sk); remove_wait_queue(sk_sleep(sk), &wait); @@ -1852,6 +1856,7 @@ static int tls_rx_reader_acquire(struct sock *sk, struct tls_sw_context_rx *ctx, bool nonblock) { long timeo; + int ret; timeo = sock_rcvtimeo(sk, nonblock); @@ -1861,14 +1866,16 @@ static int tls_rx_reader_acquire(struct sock *sk, struct tls_sw_context_rx *ctx, ctx->reader_contended = 1; add_wait_queue(&ctx->wq, &wait); - sk_wait_event(sk, &timeo, - !READ_ONCE(ctx->reader_present), &wait); + ret = sk_wait_event(sk, &timeo, + !READ_ONCE(ctx->reader_present), &wait); remove_wait_queue(&ctx->wq, &wait); if (timeo <= 0) return -EAGAIN; if (signal_pending(current)) return sock_intr_errno(timeo); + if (ret < 0) + return ret; } WRITE_ONCE(ctx->reader_present, 1); diff --git a/net/wireless/core.c b/net/wireless/core.c index 25bc2e50a061..acec41c1809a 100644 --- a/net/wireless/core.c +++ b/net/wireless/core.c @@ -1181,16 +1181,11 @@ void wiphy_rfkill_set_hw_state_reason(struct wiphy *wiphy, bool blocked, } EXPORT_SYMBOL(wiphy_rfkill_set_hw_state_reason); -void cfg80211_cqm_config_free(struct wireless_dev *wdev) -{ - kfree(wdev->cqm_config); - wdev->cqm_config = NULL; -} - static void _cfg80211_unregister_wdev(struct wireless_dev *wdev, bool unregister_netdev) { struct cfg80211_registered_device *rdev = wiphy_to_rdev(wdev->wiphy); + struct cfg80211_cqm_config *cqm_config; unsigned int link_id; ASSERT_RTNL(); @@ -1227,7 +1222,10 @@ static void _cfg80211_unregister_wdev(struct wireless_dev *wdev, kfree_sensitive(wdev->wext.keys); wdev->wext.keys = NULL; #endif - cfg80211_cqm_config_free(wdev); + wiphy_work_cancel(wdev->wiphy, &wdev->cqm_rssi_work); + /* deleted from the list, so can't be found from nl80211 any more */ + cqm_config = rcu_access_pointer(wdev->cqm_config); + kfree_rcu(cqm_config, rcu_head); /* * Ensure that all events have been processed and @@ -1379,6 +1377,8 @@ void cfg80211_init_wdev(struct wireless_dev *wdev) wdev->wext.connect.auth_type = NL80211_AUTHTYPE_AUTOMATIC; #endif + wiphy_work_init(&wdev->cqm_rssi_work, cfg80211_cqm_rssi_notify_work); + if (wdev->wiphy->flags & WIPHY_FLAG_PS_ON_BY_DEFAULT) wdev->ps = true; else @@ -1622,7 +1622,7 @@ void wiphy_work_queue(struct wiphy *wiphy, struct wiphy_work *work) list_add_tail(&work->entry, &rdev->wiphy_work_list); spin_unlock_irqrestore(&rdev->wiphy_work_lock, flags); - schedule_work(&rdev->wiphy_work); + queue_work(system_unbound_wq, &rdev->wiphy_work); } EXPORT_SYMBOL_GPL(wiphy_work_queue); diff --git a/net/wireless/core.h b/net/wireless/core.h index 507d184b8b40..ba9c7170afa4 100644 --- a/net/wireless/core.h +++ b/net/wireless/core.h @@ -295,12 +295,17 @@ struct cfg80211_beacon_registration { }; struct cfg80211_cqm_config { + struct rcu_head rcu_head; u32 rssi_hyst; s32 last_rssi_event_value; + enum nl80211_cqm_rssi_threshold_event last_rssi_event_type; int n_rssi_thresholds; s32 rssi_thresholds[] __counted_by(n_rssi_thresholds); }; +void cfg80211_cqm_rssi_notify_work(struct wiphy *wiphy, + struct wiphy_work *work); + void cfg80211_destroy_ifaces(struct cfg80211_registered_device *rdev); /* free object */ @@ -566,8 +571,6 @@ cfg80211_bss_update(struct cfg80211_registered_device *rdev, #define CFG80211_DEV_WARN_ON(cond) ({bool __r = (cond); __r; }) #endif -void cfg80211_cqm_config_free(struct wireless_dev *wdev); - void cfg80211_release_pmsr(struct wireless_dev *wdev, u32 portid); void cfg80211_pmsr_wdev_down(struct wireless_dev *wdev); void cfg80211_pmsr_free_wk(struct work_struct *work); diff --git a/net/wireless/mlme.c b/net/wireless/mlme.c index 775cac4d6100..3e2c398abddc 100644 --- a/net/wireless/mlme.c +++ b/net/wireless/mlme.c @@ -52,7 +52,8 @@ void cfg80211_rx_assoc_resp(struct net_device *dev, cr.links[link_id].bssid = data->links[link_id].bss->bssid; cr.links[link_id].addr = data->links[link_id].addr; /* need to have local link addresses for MLO connections */ - WARN_ON(cr.ap_mld_addr && !cr.links[link_id].addr); + WARN_ON(cr.ap_mld_addr && + !is_valid_ether_addr(cr.links[link_id].addr)); BUG_ON(!cr.links[link_id].bss->channel); diff --git a/net/wireless/nl80211.c b/net/wireless/nl80211.c index de47838aca4f..931a03f4549c 100644 --- a/net/wireless/nl80211.c +++ b/net/wireless/nl80211.c @@ -5909,6 +5909,21 @@ out: nlmsg_free(msg); } +static int nl80211_validate_ap_phy_operation(struct cfg80211_ap_settings *params) +{ + struct ieee80211_channel *channel = params->chandef.chan; + + if ((params->he_cap || params->he_oper) && + (channel->flags & IEEE80211_CHAN_NO_HE)) + return -EOPNOTSUPP; + + if ((params->eht_cap || params->eht_oper) && + (channel->flags & IEEE80211_CHAN_NO_EHT)) + return -EOPNOTSUPP; + + return 0; +} + static int nl80211_start_ap(struct sk_buff *skb, struct genl_info *info) { struct cfg80211_registered_device *rdev = info->user_ptr[0]; @@ -6178,6 +6193,10 @@ static int nl80211_start_ap(struct sk_buff *skb, struct genl_info *info) if (err) goto out_unlock; + err = nl80211_validate_ap_phy_operation(params); + if (err) + goto out_unlock; + if (info->attrs[NL80211_ATTR_AP_SETTINGS_FLAGS]) params->flags = nla_get_u32( info->attrs[NL80211_ATTR_AP_SETTINGS_FLAGS]); @@ -8482,7 +8501,7 @@ static int nl80211_update_mesh_config(struct sk_buff *skb, struct cfg80211_registered_device *rdev = info->user_ptr[0]; struct net_device *dev = info->user_ptr[1]; struct wireless_dev *wdev = dev->ieee80211_ptr; - struct mesh_config cfg; + struct mesh_config cfg = {}; u32 mask; int err; @@ -12796,7 +12815,8 @@ static int nl80211_set_cqm_txe(struct genl_info *info, } static int cfg80211_cqm_rssi_update(struct cfg80211_registered_device *rdev, - struct net_device *dev) + struct net_device *dev, + struct cfg80211_cqm_config *cqm_config) { struct wireless_dev *wdev = dev->ieee80211_ptr; s32 last, low, high; @@ -12805,7 +12825,7 @@ static int cfg80211_cqm_rssi_update(struct cfg80211_registered_device *rdev, int err; /* RSSI reporting disabled? */ - if (!wdev->cqm_config) + if (!cqm_config) return rdev_set_cqm_rssi_range_config(rdev, dev, 0, 0); /* @@ -12814,7 +12834,7 @@ static int cfg80211_cqm_rssi_update(struct cfg80211_registered_device *rdev, * connection is established and enough beacons received to calculate * the average. */ - if (!wdev->cqm_config->last_rssi_event_value && + if (!cqm_config->last_rssi_event_value && wdev->links[0].client.current_bss && rdev->ops->get_station) { struct station_info sinfo = {}; @@ -12828,30 +12848,30 @@ static int cfg80211_cqm_rssi_update(struct cfg80211_registered_device *rdev, cfg80211_sinfo_release_content(&sinfo); if (sinfo.filled & BIT_ULL(NL80211_STA_INFO_BEACON_SIGNAL_AVG)) - wdev->cqm_config->last_rssi_event_value = + cqm_config->last_rssi_event_value = (s8) sinfo.rx_beacon_signal_avg; } - last = wdev->cqm_config->last_rssi_event_value; - hyst = wdev->cqm_config->rssi_hyst; - n = wdev->cqm_config->n_rssi_thresholds; + last = cqm_config->last_rssi_event_value; + hyst = cqm_config->rssi_hyst; + n = cqm_config->n_rssi_thresholds; for (i = 0; i < n; i++) { i = array_index_nospec(i, n); - if (last < wdev->cqm_config->rssi_thresholds[i]) + if (last < cqm_config->rssi_thresholds[i]) break; } low_index = i - 1; if (low_index >= 0) { low_index = array_index_nospec(low_index, n); - low = wdev->cqm_config->rssi_thresholds[low_index] - hyst; + low = cqm_config->rssi_thresholds[low_index] - hyst; } else { low = S32_MIN; } if (i < n) { i = array_index_nospec(i, n); - high = wdev->cqm_config->rssi_thresholds[i] + hyst - 1; + high = cqm_config->rssi_thresholds[i] + hyst - 1; } else { high = S32_MAX; } @@ -12864,6 +12884,7 @@ static int nl80211_set_cqm_rssi(struct genl_info *info, u32 hysteresis) { struct cfg80211_registered_device *rdev = info->user_ptr[0]; + struct cfg80211_cqm_config *cqm_config = NULL, *old; struct net_device *dev = info->user_ptr[1]; struct wireless_dev *wdev = dev->ieee80211_ptr; int i, err; @@ -12881,10 +12902,6 @@ static int nl80211_set_cqm_rssi(struct genl_info *info, wdev->iftype != NL80211_IFTYPE_P2P_CLIENT) return -EOPNOTSUPP; - wdev_lock(wdev); - cfg80211_cqm_config_free(wdev); - wdev_unlock(wdev); - if (n_thresholds <= 1 && rdev->ops->set_cqm_rssi_config) { if (n_thresholds == 0 || thresholds[0] == 0) /* Disabling */ return rdev_set_cqm_rssi_config(rdev, dev, 0, 0); @@ -12901,9 +12918,10 @@ static int nl80211_set_cqm_rssi(struct genl_info *info, n_thresholds = 0; wdev_lock(wdev); - if (n_thresholds) { - struct cfg80211_cqm_config *cqm_config; + old = rcu_dereference_protected(wdev->cqm_config, + lockdep_is_held(&wdev->mtx)); + if (n_thresholds) { cqm_config = kzalloc(struct_size(cqm_config, rssi_thresholds, n_thresholds), GFP_KERNEL); @@ -12918,11 +12936,18 @@ static int nl80211_set_cqm_rssi(struct genl_info *info, flex_array_size(cqm_config, rssi_thresholds, n_thresholds)); - wdev->cqm_config = cqm_config; + rcu_assign_pointer(wdev->cqm_config, cqm_config); + } else { + RCU_INIT_POINTER(wdev->cqm_config, NULL); } - err = cfg80211_cqm_rssi_update(rdev, dev); - + err = cfg80211_cqm_rssi_update(rdev, dev, cqm_config); + if (err) { + rcu_assign_pointer(wdev->cqm_config, old); + kfree_rcu(cqm_config, rcu_head); + } else { + kfree_rcu(old, rcu_head); + } unlock: wdev_unlock(wdev); @@ -19073,9 +19098,8 @@ void cfg80211_cqm_rssi_notify(struct net_device *dev, enum nl80211_cqm_rssi_threshold_event rssi_event, s32 rssi_level, gfp_t gfp) { - struct sk_buff *msg; struct wireless_dev *wdev = dev->ieee80211_ptr; - struct cfg80211_registered_device *rdev = wiphy_to_rdev(wdev->wiphy); + struct cfg80211_cqm_config *cqm_config; trace_cfg80211_cqm_rssi_notify(dev, rssi_event, rssi_level); @@ -19083,18 +19107,41 @@ void cfg80211_cqm_rssi_notify(struct net_device *dev, rssi_event != NL80211_CQM_RSSI_THRESHOLD_EVENT_HIGH)) return; - if (wdev->cqm_config) { - wdev->cqm_config->last_rssi_event_value = rssi_level; + rcu_read_lock(); + cqm_config = rcu_dereference(wdev->cqm_config); + if (cqm_config) { + cqm_config->last_rssi_event_value = rssi_level; + cqm_config->last_rssi_event_type = rssi_event; + wiphy_work_queue(wdev->wiphy, &wdev->cqm_rssi_work); + } + rcu_read_unlock(); +} +EXPORT_SYMBOL(cfg80211_cqm_rssi_notify); - cfg80211_cqm_rssi_update(rdev, dev); +void cfg80211_cqm_rssi_notify_work(struct wiphy *wiphy, struct wiphy_work *work) +{ + struct wireless_dev *wdev = container_of(work, struct wireless_dev, + cqm_rssi_work); + struct cfg80211_registered_device *rdev = wiphy_to_rdev(wiphy); + enum nl80211_cqm_rssi_threshold_event rssi_event; + struct cfg80211_cqm_config *cqm_config; + struct sk_buff *msg; + s32 rssi_level; - if (rssi_level == 0) - rssi_level = wdev->cqm_config->last_rssi_event_value; - } + wdev_lock(wdev); + cqm_config = rcu_dereference_protected(wdev->cqm_config, + lockdep_is_held(&wdev->mtx)); + if (!wdev->cqm_config) + goto unlock; - msg = cfg80211_prepare_cqm(dev, NULL, gfp); + cfg80211_cqm_rssi_update(rdev, wdev->netdev, cqm_config); + + rssi_level = cqm_config->last_rssi_event_value; + rssi_event = cqm_config->last_rssi_event_type; + + msg = cfg80211_prepare_cqm(wdev->netdev, NULL, GFP_KERNEL); if (!msg) - return; + goto unlock; if (nla_put_u32(msg, NL80211_ATTR_CQM_RSSI_THRESHOLD_EVENT, rssi_event)) @@ -19104,14 +19151,15 @@ void cfg80211_cqm_rssi_notify(struct net_device *dev, rssi_level)) goto nla_put_failure; - cfg80211_send_cqm(msg, gfp); + cfg80211_send_cqm(msg, GFP_KERNEL); - return; + goto unlock; nla_put_failure: nlmsg_free(msg); + unlock: + wdev_unlock(wdev); } -EXPORT_SYMBOL(cfg80211_cqm_rssi_notify); void cfg80211_cqm_txe_notify(struct net_device *dev, const u8 *peer, u32 num_packets, diff --git a/net/wireless/scan.c b/net/wireless/scan.c index 0cf1ce7b6934..939deecf0bbe 100644 --- a/net/wireless/scan.c +++ b/net/wireless/scan.c @@ -908,6 +908,10 @@ static int cfg80211_scan_6ghz(struct cfg80211_registered_device *rdev) !cfg80211_find_ssid_match(ap, request)) continue; + if (!is_broadcast_ether_addr(request->bssid) && + !ether_addr_equal(request->bssid, ap->bssid)) + continue; + if (!request->n_ssids && ap->multi_bss && !ap->transmitted_bssid) continue; diff --git a/net/xdp/xsk_queue.c b/net/xdp/xsk_queue.c index f8905400ee07..d2c264030017 100644 --- a/net/xdp/xsk_queue.c +++ b/net/xdp/xsk_queue.c @@ -34,6 +34,16 @@ struct xsk_queue *xskq_create(u32 nentries, bool umem_queue) q->ring_mask = nentries - 1; size = xskq_get_ring_size(q, umem_queue); + + /* size which is overflowing or close to SIZE_MAX will become 0 in + * PAGE_ALIGN(), checking SIZE_MAX is enough due to the previous + * is_power_of_2(), the rest will be handled by vmalloc_user() + */ + if (unlikely(size == SIZE_MAX)) { + kfree(q); + return NULL; + } + size = PAGE_ALIGN(size); q->ring = vmalloc_user(size); diff --git a/net/xfrm/xfrm_interface_core.c b/net/xfrm/xfrm_interface_core.c index b86474084690..e21cc71095bb 100644 --- a/net/xfrm/xfrm_interface_core.c +++ b/net/xfrm/xfrm_interface_core.c @@ -380,8 +380,8 @@ static int xfrmi_rcv_cb(struct sk_buff *skb, int err) skb->dev = dev; if (err) { - dev->stats.rx_errors++; - dev->stats.rx_dropped++; + DEV_STATS_INC(dev, rx_errors); + DEV_STATS_INC(dev, rx_dropped); return 0; } @@ -426,7 +426,6 @@ static int xfrmi_xmit2(struct sk_buff *skb, struct net_device *dev, struct flowi *fl) { struct xfrm_if *xi = netdev_priv(dev); - struct net_device_stats *stats = &xi->dev->stats; struct dst_entry *dst = skb_dst(skb); unsigned int length = skb->len; struct net_device *tdev; @@ -473,7 +472,7 @@ xfrmi_xmit2(struct sk_buff *skb, struct net_device *dev, struct flowi *fl) tdev = dst->dev; if (tdev == dev) { - stats->collisions++; + DEV_STATS_INC(dev, collisions); net_warn_ratelimited("%s: Local routing loop detected!\n", dev->name); goto tx_err_dst_release; @@ -512,13 +511,13 @@ xmit: if (net_xmit_eval(err) == 0) { dev_sw_netstats_tx_add(dev, 1, length); } else { - stats->tx_errors++; - stats->tx_aborted_errors++; + DEV_STATS_INC(dev, tx_errors); + DEV_STATS_INC(dev, tx_aborted_errors); } return 0; tx_err_link_failure: - stats->tx_carrier_errors++; + DEV_STATS_INC(dev, tx_carrier_errors); dst_link_failure(skb); tx_err_dst_release: dst_release(dst); @@ -528,7 +527,6 @@ tx_err_dst_release: static netdev_tx_t xfrmi_xmit(struct sk_buff *skb, struct net_device *dev) { struct xfrm_if *xi = netdev_priv(dev); - struct net_device_stats *stats = &xi->dev->stats; struct dst_entry *dst = skb_dst(skb); struct flowi fl; int ret; @@ -545,7 +543,7 @@ static netdev_tx_t xfrmi_xmit(struct sk_buff *skb, struct net_device *dev) dst = ip6_route_output(dev_net(dev), NULL, &fl.u.ip6); if (dst->error) { dst_release(dst); - stats->tx_carrier_errors++; + DEV_STATS_INC(dev, tx_carrier_errors); goto tx_err; } skb_dst_set(skb, dst); @@ -561,7 +559,7 @@ static netdev_tx_t xfrmi_xmit(struct sk_buff *skb, struct net_device *dev) fl.u.ip4.flowi4_flags |= FLOWI_FLAG_ANYSRC; rt = __ip_route_output_key(dev_net(dev), &fl.u.ip4); if (IS_ERR(rt)) { - stats->tx_carrier_errors++; + DEV_STATS_INC(dev, tx_carrier_errors); goto tx_err; } skb_dst_set(skb, &rt->dst); @@ -580,8 +578,8 @@ static netdev_tx_t xfrmi_xmit(struct sk_buff *skb, struct net_device *dev) return NETDEV_TX_OK; tx_err: - stats->tx_errors++; - stats->tx_dropped++; + DEV_STATS_INC(dev, tx_errors); + DEV_STATS_INC(dev, tx_dropped); kfree_skb(skb); return NETDEV_TX_OK; } diff --git a/net/xfrm/xfrm_policy.c b/net/xfrm/xfrm_policy.c index d6b405782b63..d24b4d4f620e 100644 --- a/net/xfrm/xfrm_policy.c +++ b/net/xfrm/xfrm_policy.c @@ -851,7 +851,7 @@ static void xfrm_policy_inexact_list_reinsert(struct net *net, struct hlist_node *newpos = NULL; bool matches_s, matches_d; - if (!policy->bydst_reinsert) + if (policy->walk.dead || !policy->bydst_reinsert) continue; WARN_ON_ONCE(policy->family != family); @@ -1256,8 +1256,11 @@ static void xfrm_hash_rebuild(struct work_struct *work) struct xfrm_pol_inexact_bin *bin; u8 dbits, sbits; + if (policy->walk.dead) + continue; + dir = xfrm_policy_id2dir(policy->index); - if (policy->walk.dead || dir >= XFRM_POLICY_MAX) + if (dir >= XFRM_POLICY_MAX) continue; if ((dir & XFRM_POLICY_MASK) == XFRM_POLICY_OUT) { @@ -1372,8 +1375,6 @@ EXPORT_SYMBOL(xfrm_policy_hash_rebuild); * of an absolute inpredictability of ordering of rules. This will not pass. */ static u32 xfrm_gen_index(struct net *net, int dir, u32 index) { - static u32 idx_generator; - for (;;) { struct hlist_head *list; struct xfrm_policy *p; @@ -1381,8 +1382,8 @@ static u32 xfrm_gen_index(struct net *net, int dir, u32 index) int found; if (!index) { - idx = (idx_generator | dir); - idx_generator += 8; + idx = (net->xfrm.idx_generator | dir); + net->xfrm.idx_generator += 8; } else { idx = index; index = 0; @@ -1823,9 +1824,11 @@ int xfrm_policy_flush(struct net *net, u8 type, bool task_valid) again: list_for_each_entry(pol, &net->xfrm.policy_all, walk.all) { + if (pol->walk.dead) + continue; + dir = xfrm_policy_id2dir(pol->index); - if (pol->walk.dead || - dir >= XFRM_POLICY_MAX || + if (dir >= XFRM_POLICY_MAX || pol->type != type) continue; @@ -1862,9 +1865,11 @@ int xfrm_dev_policy_flush(struct net *net, struct net_device *dev, again: list_for_each_entry(pol, &net->xfrm.policy_all, walk.all) { + if (pol->walk.dead) + continue; + dir = xfrm_policy_id2dir(pol->index); - if (pol->walk.dead || - dir >= XFRM_POLICY_MAX || + if (dir >= XFRM_POLICY_MAX || pol->xdo.dev != dev) continue; @@ -3215,7 +3220,7 @@ no_transform: } for (i = 0; i < num_pols; i++) - pols[i]->curlft.use_time = ktime_get_real_seconds(); + WRITE_ONCE(pols[i]->curlft.use_time, ktime_get_real_seconds()); if (num_xfrms < 0) { /* Prohibit the flow */ diff --git a/rust/Makefile b/rust/Makefile index 87958e864be0..7dbf9abe0d01 100644 --- a/rust/Makefile +++ b/rust/Makefile @@ -93,15 +93,14 @@ quiet_cmd_rustdoc = RUSTDOC $(if $(rustdoc_host),H, ) $< # and then retouch the generated files. rustdoc: rustdoc-core rustdoc-macros rustdoc-compiler_builtins \ rustdoc-alloc rustdoc-kernel - $(Q)cp $(srctree)/Documentation/images/logo.svg $(rustdoc_output) - $(Q)cp $(srctree)/Documentation/images/COPYING-logo $(rustdoc_output) + $(Q)cp $(srctree)/Documentation/images/logo.svg $(rustdoc_output)/static.files/ + $(Q)cp $(srctree)/Documentation/images/COPYING-logo $(rustdoc_output)/static.files/ $(Q)find $(rustdoc_output) -name '*.html' -type f -print0 | xargs -0 sed -Ei \ - -e 's:rust-logo\.svg:logo.svg:g' \ - -e 's:rust-logo\.png:logo.svg:g' \ - -e 's:favicon\.svg:logo.svg:g' \ - -e 's:::g' - $(Q)echo '.logo-container > img { object-fit: contain; }' \ - >> $(rustdoc_output)/rustdoc.css + -e 's:rust-logo-[0-9a-f]+\.svg:logo.svg:g' \ + -e 's:favicon-[0-9a-f]+\.svg:logo.svg:g' \ + -e 's:::g' + $(Q)for f in $(rustdoc_output)/static.files/rustdoc-*.css; do \ + echo ".logo-container > img { object-fit: contain; }" >> $$f; done rustdoc-macros: private rustdoc_host = yes rustdoc-macros: private rustc_target_flags = --crate-type proc-macro \ @@ -290,6 +289,7 @@ bindgen_skip_c_flags := -mno-fp-ret-in-387 -mpreferred-stack-boundary=% \ -fno-reorder-blocks -fno-allow-store-data-races -fasan-shadow-offset=% \ -fzero-call-used-regs=% -fno-stack-clash-protection \ -fno-inline-functions-called-once -fsanitize=bounds-strict \ + -fstrict-flex-arrays=% \ --param=% --param asan-% # Derived from `scripts/Makefile.clang`. diff --git a/rust/kernel/error.rs b/rust/kernel/error.rs index 05fcab6abfe6..032b64543953 100644 --- a/rust/kernel/error.rs +++ b/rust/kernel/error.rs @@ -37,7 +37,7 @@ pub mod code { declare_err!(E2BIG, "Argument list too long."); declare_err!(ENOEXEC, "Exec format error."); declare_err!(EBADF, "Bad file number."); - declare_err!(ECHILD, "Exec format error."); + declare_err!(ECHILD, "No child processes."); declare_err!(EAGAIN, "Try again."); declare_err!(ENOMEM, "Out of memory."); declare_err!(EACCES, "Permission denied."); @@ -133,7 +133,7 @@ impl Error { /// Returns the error encoded as a pointer. #[allow(dead_code)] pub(crate) fn to_ptr(self) -> *mut T { - // SAFETY: self.0 is a valid error due to its invariant. + // SAFETY: `self.0` is a valid error due to its invariant. unsafe { bindings::ERR_PTR(self.0.into()) as *mut _ } } diff --git a/scripts/Makefile.modinst b/scripts/Makefile.modinst index c59cc57286ba..0afd75472679 100644 --- a/scripts/Makefile.modinst +++ b/scripts/Makefile.modinst @@ -113,7 +113,7 @@ quiet_cmd_sign := endif # Create necessary directories -$(shell mkdir -p $(sort $(dir $(install-y)))) +$(foreach dir, $(sort $(dir $(install-y))), $(shell mkdir -p $(dir))) $(dst)/%.ko: $(extmod_prefix)%.ko FORCE $(call cmd,install) @@ -144,7 +144,7 @@ endif quiet_cmd_gzip = GZIP $@ cmd_gzip = $(KGZIP) -n -f $< quiet_cmd_xz = XZ $@ - cmd_xz = $(XZ) --lzma2=dict=2MiB -f $< + cmd_xz = $(XZ) --check=crc32 --lzma2=dict=1MiB -f $< quiet_cmd_zstd = ZSTD $@ cmd_zstd = $(ZSTD) -T0 --rm -f -q $< diff --git a/scripts/atomic/gen-atomic-fallback.sh b/scripts/atomic/gen-atomic-fallback.sh index c0c8a85d7c81..a45154cefa48 100755 --- a/scripts/atomic/gen-atomic-fallback.sh +++ b/scripts/atomic/gen-atomic-fallback.sh @@ -102,7 +102,7 @@ gen_proto_order_variant() fi # Allow ACQUIRE/RELEASE/RELAXED ops to be defined in terms of FULL ops - if [ ! -z "${order}" ]; then + if [ ! -z "${order}" ] && ! meta_is_implicitly_relaxed "${meta}"; then printf "#elif defined(arch_${basename})\n" printf "\t${retstmt}arch_${basename}(${args});\n" fi diff --git a/scripts/gdb/linux/symbols.py b/scripts/gdb/linux/symbols.py index 5179edd1b627..c8047f4441e6 100644 --- a/scripts/gdb/linux/symbols.py +++ b/scripts/gdb/linux/symbols.py @@ -111,12 +111,11 @@ lx-symbols command.""" return "{textaddr} {sections}".format( textaddr=textaddr, sections="".join(args)) - def load_module_symbols(self, module, module_file=None): + def load_module_symbols(self, module): module_name = module['name'].string() module_addr = str(module['mem'][constants.LX_MOD_TEXT]['base']).split()[0] - if not module_file: - module_file = self._get_module_file(module_name) + module_file = self._get_module_file(module_name) if not module_file and not self.module_files_updated: self._update_module_files() module_file = self._get_module_file(module_name) @@ -139,19 +138,6 @@ lx-symbols command.""" else: gdb.write("no module object found for '{0}'\n".format(module_name)) - def load_ko_symbols(self, mod_path): - self.loaded_modules = [] - module_list = modules.module_list() - - for module in module_list: - module_name = module['name'].string() - module_pattern = ".*/{0}\.ko(?:.debug)?$".format( - module_name.replace("_", r"[_\-]")) - if re.match(module_pattern, mod_path) and os.path.exists(mod_path): - self.load_module_symbols(module, mod_path) - return - raise gdb.GdbError("%s is not a valid .ko\n" % mod_path) - def load_all_symbols(self): gdb.write("loading vmlinux\n") @@ -190,11 +176,6 @@ lx-symbols command.""" self.module_files = [] self.module_files_updated = False - argv = gdb.string_to_argv(arg) - if len(argv) == 1: - self.load_ko_symbols(argv[0]) - return - self.load_all_symbols() if hasattr(gdb, 'Breakpoint'): diff --git a/scripts/mod/file2alias.c b/scripts/mod/file2alias.c index 38120f932b0d..7056751c29b1 100644 --- a/scripts/mod/file2alias.c +++ b/scripts/mod/file2alias.c @@ -1604,7 +1604,7 @@ void handle_moddevtable(struct module *mod, struct elf_info *info, /* First handle the "special" cases */ if (sym_is(name, namelen, "usb")) do_usb_table(symval, sym->st_size, mod); - if (sym_is(name, namelen, "of")) + else if (sym_is(name, namelen, "of")) do_of_table(symval, sym->st_size, mod); else if (sym_is(name, namelen, "pnp")) do_pnp_device_entry(symval, sym->st_size, mod); diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c index 34a5386d444a..b3dee80497cb 100644 --- a/scripts/mod/modpost.c +++ b/scripts/mod/modpost.c @@ -1015,9 +1015,20 @@ static int secref_whitelist(const char *fromsec, const char *fromsym, "*_console"))) return 0; - /* symbols in data sections that may refer to meminit/exit sections */ + /* symbols in data sections that may refer to meminit sections */ if (match(fromsec, PATTERNS(DATA_SECTIONS)) && - match(tosec, PATTERNS(ALL_XXXINIT_SECTIONS, ALL_EXIT_SECTIONS)) && + match(tosec, PATTERNS(ALL_XXXINIT_SECTIONS, ALL_XXXEXIT_SECTIONS)) && + match(fromsym, PATTERNS("*driver"))) + return 0; + + /* + * symbols in data sections must not refer to .exit.*, but there are + * quite a few offenders, so hide these unless for W=1 builds until + * these are fixed. + */ + if (!extra_warn && + match(fromsec, PATTERNS(DATA_SECTIONS)) && + match(tosec, PATTERNS(EXIT_SECTIONS)) && match(fromsym, PATTERNS("*driver"))) return 0; @@ -1228,6 +1239,15 @@ static void check_export_symbol(struct module *mod, struct elf_info *elf, */ s->is_func = (ELF_ST_TYPE(sym->st_info) == STT_FUNC); + /* + * For parisc64, symbols prefixed $$ from the library have the symbol type + * STT_LOPROC. They should be handled as functions too. + */ + if (elf->hdr->e_ident[EI_CLASS] == ELFCLASS64 && + elf->hdr->e_machine == EM_PARISC && + ELF_ST_TYPE(sym->st_info) == STT_LOPROC) + s->is_func = true; + if (match(secname, PATTERNS(INIT_SECTIONS))) warn("%s: %s: EXPORT_SYMBOL used for init symbol. Remove __init or EXPORT_SYMBOL.\n", mod->name, name); diff --git a/scripts/package/builddeb b/scripts/package/builddeb index bf3f8561aa68..d7dd0d04c70c 100755 --- a/scripts/package/builddeb +++ b/scripts/package/builddeb @@ -64,7 +64,6 @@ install_linux_image () { ${MAKE} -f ${srctree}/Makefile INSTALL_MOD_PATH="${pdir}" modules_install rm -f "${pdir}/lib/modules/${KERNELRELEASE}/build" - rm -f "${pdir}/lib/modules/${KERNELRELEASE}/source" # Install the kernel if [ "${ARCH}" = um ] ; then diff --git a/scripts/package/install-extmod-build b/scripts/package/install-extmod-build index af7fe9f5b1e4..8a7051fad087 100755 --- a/scripts/package/install-extmod-build +++ b/scripts/package/install-extmod-build @@ -20,7 +20,7 @@ mkdir -p "${destdir}" find "arch/${SRCARCH}" -maxdepth 1 -name 'Makefile*' find include scripts -type f -o -type l find "arch/${SRCARCH}" -name Kbuild.platforms -o -name Platform - find "$(find "arch/${SRCARCH}" -name include -o -name scripts -type d)" -type f + find "arch/${SRCARCH}" -name include -o -name scripts -type d ) | tar -c -f - -C "${srctree}" -T - | tar -xf - -C "${destdir}" { diff --git a/scripts/package/kernel.spec b/scripts/package/kernel.spec index ac3f2ee6d7a0..3eee0143e0c5 100644 --- a/scripts/package/kernel.spec +++ b/scripts/package/kernel.spec @@ -68,7 +68,6 @@ cp $(%{make} %{makeflags} -s image_name) %{buildroot}/boot/vmlinuz-%{KERNELRELEA cp System.map %{buildroot}/boot/System.map-%{KERNELRELEASE} cp .config %{buildroot}/boot/config-%{KERNELRELEASE} ln -fns /usr/src/kernels/%{KERNELRELEASE} %{buildroot}/lib/modules/%{KERNELRELEASE}/build -ln -fns /usr/src/kernels/%{KERNELRELEASE} %{buildroot}/lib/modules/%{KERNELRELEASE}/source %if %{with_devel} %{make} %{makeflags} run-command KBUILD_RUN_COMMAND='${srctree}/scripts/package/install-extmod-build %{buildroot}/usr/src/kernels/%{KERNELRELEASE}' %endif @@ -101,7 +100,6 @@ fi %defattr (-, root, root) /lib/modules/%{KERNELRELEASE} %exclude /lib/modules/%{KERNELRELEASE}/build -%exclude /lib/modules/%{KERNELRELEASE}/source /boot/* %files headers @@ -113,5 +111,4 @@ fi %defattr (-, root, root) /usr/src/kernels/%{KERNELRELEASE} /lib/modules/%{KERNELRELEASE}/build -/lib/modules/%{KERNELRELEASE}/source %endif diff --git a/security/integrity/ima/Kconfig b/security/integrity/ima/Kconfig index ecddc807c536..a6bd817efc1a 100644 --- a/security/integrity/ima/Kconfig +++ b/security/integrity/ima/Kconfig @@ -29,9 +29,11 @@ config IMA to learn more about IMA. If unsure, say N. +if IMA + config IMA_KEXEC bool "Enable carrying the IMA measurement list across a soft boot" - depends on IMA && TCG_TPM && HAVE_IMA_KEXEC + depends on TCG_TPM && HAVE_IMA_KEXEC default n help TPM PCRs are only reset on a hard reboot. In order to validate @@ -43,7 +45,6 @@ config IMA_KEXEC config IMA_MEASURE_PCR_IDX int - depends on IMA range 8 14 default 10 help @@ -53,7 +54,7 @@ config IMA_MEASURE_PCR_IDX config IMA_LSM_RULES bool - depends on IMA && AUDIT && (SECURITY_SELINUX || SECURITY_SMACK || SECURITY_APPARMOR) + depends on AUDIT && (SECURITY_SELINUX || SECURITY_SMACK || SECURITY_APPARMOR) default y help Disabling this option will disregard LSM based policy rules. @@ -61,7 +62,6 @@ config IMA_LSM_RULES choice prompt "Default template" default IMA_NG_TEMPLATE - depends on IMA help Select the default IMA measurement template. @@ -80,14 +80,12 @@ endchoice config IMA_DEFAULT_TEMPLATE string - depends on IMA default "ima-ng" if IMA_NG_TEMPLATE default "ima-sig" if IMA_SIG_TEMPLATE choice prompt "Default integrity hash algorithm" default IMA_DEFAULT_HASH_SHA1 - depends on IMA help Select the default hash algorithm used for the measurement list, integrity appraisal and audit log. The compiled default @@ -117,7 +115,6 @@ endchoice config IMA_DEFAULT_HASH string - depends on IMA default "sha1" if IMA_DEFAULT_HASH_SHA1 default "sha256" if IMA_DEFAULT_HASH_SHA256 default "sha512" if IMA_DEFAULT_HASH_SHA512 @@ -126,7 +123,6 @@ config IMA_DEFAULT_HASH config IMA_WRITE_POLICY bool "Enable multiple writes to the IMA policy" - depends on IMA default n help IMA policy can now be updated multiple times. The new rules get @@ -137,7 +133,6 @@ config IMA_WRITE_POLICY config IMA_READ_POLICY bool "Enable reading back the current IMA policy" - depends on IMA default y if IMA_WRITE_POLICY default n if !IMA_WRITE_POLICY help @@ -147,7 +142,6 @@ config IMA_READ_POLICY config IMA_APPRAISE bool "Appraise integrity measurements" - depends on IMA default n help This option enables local measurement integrity appraisal. @@ -269,7 +263,7 @@ config IMA_KEYRINGS_PERMIT_SIGNED_BY_BUILTIN_OR_SECONDARY config IMA_BLACKLIST_KEYRING bool "Create IMA machine owner blacklist keyrings (EXPERIMENTAL)" depends on SYSTEM_TRUSTED_KEYRING - depends on IMA_TRUSTED_KEYRING + depends on INTEGRITY_TRUSTED_KEYRING default n help This option creates an IMA blacklist keyring, which contains all @@ -279,7 +273,7 @@ config IMA_BLACKLIST_KEYRING config IMA_LOAD_X509 bool "Load X509 certificate onto the '.ima' trusted keyring" - depends on IMA_TRUSTED_KEYRING + depends on INTEGRITY_TRUSTED_KEYRING default n help File signature verification is based on the public keys @@ -304,7 +298,6 @@ config IMA_APPRAISE_SIGNED_INIT config IMA_MEASURE_ASYMMETRIC_KEYS bool - depends on IMA depends on ASYMMETRIC_PUBLIC_KEY_SUBTYPE=y default y @@ -323,7 +316,8 @@ config IMA_SECURE_AND_OR_TRUSTED_BOOT config IMA_DISABLE_HTABLE bool "Disable htable to allow measurement of duplicate records" - depends on IMA default n help This option disables htable to allow measurement of duplicate records. + +endif diff --git a/security/keys/trusted-keys/trusted_core.c b/security/keys/trusted-keys/trusted_core.c index c6fc50d67214..85fb5c22529a 100644 --- a/security/keys/trusted-keys/trusted_core.c +++ b/security/keys/trusted-keys/trusted_core.c @@ -44,13 +44,12 @@ static const struct trusted_key_source trusted_key_sources[] = { #endif }; -DEFINE_STATIC_CALL_NULL(trusted_key_init, *trusted_key_sources[0].ops->init); DEFINE_STATIC_CALL_NULL(trusted_key_seal, *trusted_key_sources[0].ops->seal); DEFINE_STATIC_CALL_NULL(trusted_key_unseal, *trusted_key_sources[0].ops->unseal); DEFINE_STATIC_CALL_NULL(trusted_key_get_random, *trusted_key_sources[0].ops->get_random); -DEFINE_STATIC_CALL_NULL(trusted_key_exit, *trusted_key_sources[0].ops->exit); +static void (*trusted_key_exit)(void); static unsigned char migratable; enum { @@ -359,19 +358,16 @@ static int __init init_trusted(void) if (!get_random) get_random = kernel_get_random; - static_call_update(trusted_key_init, - trusted_key_sources[i].ops->init); static_call_update(trusted_key_seal, trusted_key_sources[i].ops->seal); static_call_update(trusted_key_unseal, trusted_key_sources[i].ops->unseal); static_call_update(trusted_key_get_random, get_random); - static_call_update(trusted_key_exit, - trusted_key_sources[i].ops->exit); + trusted_key_exit = trusted_key_sources[i].ops->exit; migratable = trusted_key_sources[i].ops->migratable; - ret = static_call(trusted_key_init)(); + ret = trusted_key_sources[i].ops->init(); if (!ret) break; } @@ -388,7 +384,8 @@ static int __init init_trusted(void) static void __exit cleanup_trusted(void) { - static_call_cond(trusted_key_exit)(); + if (trusted_key_exit) + (*trusted_key_exit)(); } late_initcall(init_trusted); diff --git a/security/selinux/hooks.c b/security/selinux/hooks.c index 10350534de6d..2aa0e219d721 100644 --- a/security/selinux/hooks.c +++ b/security/selinux/hooks.c @@ -2775,14 +2775,20 @@ static int selinux_umount(struct vfsmount *mnt, int flags) static int selinux_fs_context_submount(struct fs_context *fc, struct super_block *reference) { - const struct superblock_security_struct *sbsec; + const struct superblock_security_struct *sbsec = selinux_superblock(reference); struct selinux_mnt_opts *opts; + /* + * Ensure that fc->security remains NULL when no options are set + * as expected by selinux_set_mnt_opts(). + */ + if (!(sbsec->flags & (FSCONTEXT_MNT|CONTEXT_MNT|DEFCONTEXT_MNT))) + return 0; + opts = kzalloc(sizeof(*opts), GFP_KERNEL); if (!opts) return -ENOMEM; - sbsec = selinux_superblock(reference); if (sbsec->flags & FSCONTEXT_MNT) opts->fscontext_sid = sbsec->sid; if (sbsec->flags & CONTEXT_MNT) diff --git a/sound/core/init.c b/sound/core/init.c index d61bde1225f2..22c0d217b860 100644 --- a/sound/core/init.c +++ b/sound/core/init.c @@ -278,9 +278,6 @@ static int snd_card_init(struct snd_card *card, struct device *parent, size_t extra_size) { int err; -#ifdef CONFIG_SND_DEBUG - char name[8]; -#endif if (extra_size > 0) card->private_data = (char *)card + sizeof(struct snd_card); @@ -364,8 +361,8 @@ static int snd_card_init(struct snd_card *card, struct device *parent, } #ifdef CONFIG_SND_DEBUG - sprintf(name, "card%d", idx); - card->debugfs_root = debugfs_create_dir(name, sound_debugfs_root); + card->debugfs_root = debugfs_create_dir(dev_name(&card->card_dev), + sound_debugfs_root); #endif return 0; diff --git a/sound/core/rawmidi.c b/sound/core/rawmidi.c index ba06484ac4aa..1431cb997808 100644 --- a/sound/core/rawmidi.c +++ b/sound/core/rawmidi.c @@ -1770,7 +1770,7 @@ static void snd_rawmidi_proc_info_read(struct snd_info_entry *entry, if (IS_ENABLED(CONFIG_SND_UMP)) snd_iprintf(buffer, "Type: %s\n", rawmidi_is_ump(rmidi) ? "UMP" : "Legacy"); - if (rmidi->ops->proc_read) + if (rmidi->ops && rmidi->ops->proc_read) rmidi->ops->proc_read(entry, buffer); mutex_lock(&rmidi->open_mutex); if (rmidi->info_flags & SNDRV_RAWMIDI_INFO_OUTPUT) { diff --git a/sound/core/seq/seq_midi.c b/sound/core/seq/seq_midi.c index 44302d98950e..18320a248aa7 100644 --- a/sound/core/seq/seq_midi.c +++ b/sound/core/seq/seq_midi.c @@ -349,9 +349,9 @@ snd_seq_midisynth_probe(struct device *_dev) if (! port->name[0]) { if (info->name[0]) { if (ports > 1) - snprintf(port->name, sizeof(port->name), "%s-%u", info->name, p); + scnprintf(port->name, sizeof(port->name), "%s-%u", info->name, p); else - snprintf(port->name, sizeof(port->name), "%s", info->name); + scnprintf(port->name, sizeof(port->name), "%s", info->name); } else { /* last resort */ if (ports > 1) diff --git a/sound/core/seq/seq_ump_client.c b/sound/core/seq/seq_ump_client.c index f26a1812dfa7..2db371d79930 100644 --- a/sound/core/seq/seq_ump_client.c +++ b/sound/core/seq/seq_ump_client.c @@ -207,7 +207,7 @@ static void fill_port_info(struct snd_seq_port_info *port, SNDRV_SEQ_PORT_TYPE_PORT; port->midi_channels = 16; if (*group->name) - snprintf(port->name, sizeof(port->name), "Group %d (%s)", + snprintf(port->name, sizeof(port->name), "Group %d (%.53s)", group->group + 1, group->name); else sprintf(port->name, "Group %d", group->group + 1); @@ -416,6 +416,25 @@ static void setup_client_midi_version(struct seq_ump_client *client) snd_seq_kernel_client_put(cptr); } +/* set up client's group_filter bitmap */ +static void setup_client_group_filter(struct seq_ump_client *client) +{ + struct snd_seq_client *cptr; + unsigned int filter; + int p; + + cptr = snd_seq_kernel_client_get(client->seq_client); + if (!cptr) + return; + filter = ~(1U << 0); /* always allow groupless messages */ + for (p = 0; p < SNDRV_UMP_MAX_GROUPS; p++) { + if (client->groups[p].active) + filter &= ~(1U << (p + 1)); + } + cptr->group_filter = filter; + snd_seq_kernel_client_put(cptr); +} + /* UMP group change notification */ static void handle_group_notify(struct work_struct *work) { @@ -424,6 +443,7 @@ static void handle_group_notify(struct work_struct *work) update_group_attrs(client); update_port_infos(client); + setup_client_group_filter(client); } /* UMP FB change notification */ @@ -492,6 +512,8 @@ static int snd_seq_ump_probe(struct device *_dev) goto error; } + setup_client_group_filter(client); + err = create_ump_endpoint_port(client); if (err < 0) goto error; diff --git a/sound/core/seq/seq_ump_convert.c b/sound/core/seq/seq_ump_convert.c index 7cc84e137999..b141024830ec 100644 --- a/sound/core/seq/seq_ump_convert.c +++ b/sound/core/seq/seq_ump_convert.c @@ -1197,6 +1197,8 @@ int snd_seq_deliver_to_ump(struct snd_seq_client *source, struct snd_seq_event *event, int atomic, int hop) { + if (dest->group_filter & (1U << dest_port->ump_group)) + return 0; /* group filtered - skip the event */ if (event->type == SNDRV_SEQ_EVENT_SYSEX) return cvt_sysex_to_ump(dest, dest_port, event, atomic, hop); else if (snd_seq_client_is_midi2(dest)) diff --git a/sound/firewire/bebob/bebob_midi.c b/sound/firewire/bebob/bebob_midi.c index 6f597d03e7c1..b1425bf98c3b 100644 --- a/sound/firewire/bebob/bebob_midi.c +++ b/sound/firewire/bebob/bebob_midi.c @@ -84,9 +84,9 @@ static void set_midi_substream_names(struct snd_bebob *bebob, struct snd_rawmidi_substream *subs; list_for_each_entry(subs, &str->substreams, list) { - snprintf(subs->name, sizeof(subs->name), - "%s MIDI %d", - bebob->card->shortname, subs->number + 1); + scnprintf(subs->name, sizeof(subs->name), + "%s MIDI %d", + bebob->card->shortname, subs->number + 1); } } diff --git a/sound/firewire/dice/dice-midi.c b/sound/firewire/dice/dice-midi.c index 4c2998034313..78988e44b8bc 100644 --- a/sound/firewire/dice/dice-midi.c +++ b/sound/firewire/dice/dice-midi.c @@ -88,8 +88,8 @@ static void set_midi_substream_names(struct snd_dice *dice, struct snd_rawmidi_substream *subs; list_for_each_entry(subs, &str->substreams, list) { - snprintf(subs->name, sizeof(subs->name), - "%s MIDI %d", dice->card->shortname, subs->number + 1); + scnprintf(subs->name, sizeof(subs->name), + "%s MIDI %d", dice->card->shortname, subs->number + 1); } } diff --git a/sound/firewire/digi00x/digi00x-midi.c b/sound/firewire/digi00x/digi00x-midi.c index 68eb8c39afa6..8f4bace16050 100644 --- a/sound/firewire/digi00x/digi00x-midi.c +++ b/sound/firewire/digi00x/digi00x-midi.c @@ -100,14 +100,14 @@ static void set_substream_names(struct snd_dg00x *dg00x, list_for_each_entry(subs, &str->substreams, list) { if (!is_console) { - snprintf(subs->name, sizeof(subs->name), - "%s MIDI %d", - dg00x->card->shortname, - subs->number + 1); + scnprintf(subs->name, sizeof(subs->name), + "%s MIDI %d", + dg00x->card->shortname, + subs->number + 1); } else { - snprintf(subs->name, sizeof(subs->name), - "%s control", - dg00x->card->shortname); + scnprintf(subs->name, sizeof(subs->name), + "%s control", + dg00x->card->shortname); } } } diff --git a/sound/firewire/fireface/ff-midi.c b/sound/firewire/fireface/ff-midi.c index 25821d186b87..da3054fdcc7d 100644 --- a/sound/firewire/fireface/ff-midi.c +++ b/sound/firewire/fireface/ff-midi.c @@ -79,8 +79,8 @@ static void set_midi_substream_names(struct snd_rawmidi_str *stream, struct snd_rawmidi_substream *substream; list_for_each_entry(substream, &stream->substreams, list) { - snprintf(substream->name, sizeof(substream->name), - "%s MIDI %d", name, substream->number + 1); + scnprintf(substream->name, sizeof(substream->name), + "%s MIDI %d", name, substream->number + 1); } } diff --git a/sound/firewire/fireworks/fireworks.c b/sound/firewire/fireworks/fireworks.c index dd4298876ac0..e3ed4e094ccd 100644 --- a/sound/firewire/fireworks/fireworks.c +++ b/sound/firewire/fireworks/fireworks.c @@ -93,11 +93,11 @@ get_hardware_info(struct snd_efw *efw) strcpy(efw->card->driver, "Fireworks"); strcpy(efw->card->shortname, hwinfo->model_name); strcpy(efw->card->mixername, hwinfo->model_name); - snprintf(efw->card->longname, sizeof(efw->card->longname), - "%s %s v%s, GUID %08x%08x at %s, S%d", - hwinfo->vendor_name, hwinfo->model_name, version, - hwinfo->guid_hi, hwinfo->guid_lo, - dev_name(&efw->unit->device), 100 << fw_dev->max_speed); + scnprintf(efw->card->longname, sizeof(efw->card->longname), + "%s %s v%s, GUID %08x%08x at %s, S%d", + hwinfo->vendor_name, hwinfo->model_name, version, + hwinfo->guid_hi, hwinfo->guid_lo, + dev_name(&efw->unit->device), 100 << fw_dev->max_speed); if (hwinfo->flags & BIT(FLAG_RESP_ADDR_CHANGABLE)) efw->resp_addr_changable = true; diff --git a/sound/firewire/fireworks/fireworks_midi.c b/sound/firewire/fireworks/fireworks_midi.c index 84621e356848..350bf4d299c2 100644 --- a/sound/firewire/fireworks/fireworks_midi.c +++ b/sound/firewire/fireworks/fireworks_midi.c @@ -84,8 +84,8 @@ static void set_midi_substream_names(struct snd_efw *efw, struct snd_rawmidi_substream *subs; list_for_each_entry(subs, &str->substreams, list) { - snprintf(subs->name, sizeof(subs->name), - "%s MIDI %d", efw->card->shortname, subs->number + 1); + scnprintf(subs->name, sizeof(subs->name), + "%s MIDI %d", efw->card->shortname, subs->number + 1); } } diff --git a/sound/firewire/motu/motu-midi.c b/sound/firewire/motu/motu-midi.c index 2365f7dfde26..eebc7e790ee2 100644 --- a/sound/firewire/motu/motu-midi.c +++ b/sound/firewire/motu/motu-midi.c @@ -88,8 +88,8 @@ static void set_midi_substream_names(struct snd_motu *motu, struct snd_rawmidi_substream *subs; list_for_each_entry(subs, &str->substreams, list) { - snprintf(subs->name, sizeof(subs->name), - "%s MIDI %d", motu->card->shortname, subs->number + 1); + scnprintf(subs->name, sizeof(subs->name), + "%s MIDI %d", motu->card->shortname, subs->number + 1); } } diff --git a/sound/firewire/oxfw/oxfw-midi.c b/sound/firewire/oxfw/oxfw-midi.c index 775cba3f1f02..c215fa6f7a03 100644 --- a/sound/firewire/oxfw/oxfw-midi.c +++ b/sound/firewire/oxfw/oxfw-midi.c @@ -129,9 +129,9 @@ static void set_midi_substream_names(struct snd_oxfw *oxfw, struct snd_rawmidi_substream *subs; list_for_each_entry(subs, &str->substreams, list) { - snprintf(subs->name, sizeof(subs->name), - "%s MIDI %d", - oxfw->card->shortname, subs->number + 1); + scnprintf(subs->name, sizeof(subs->name), + "%s MIDI %d", + oxfw->card->shortname, subs->number + 1); } } diff --git a/sound/firewire/oxfw/oxfw.c b/sound/firewire/oxfw/oxfw.c index 63d40f1a914f..241a697ce26b 100644 --- a/sound/firewire/oxfw/oxfw.c +++ b/sound/firewire/oxfw/oxfw.c @@ -108,11 +108,11 @@ static int name_card(struct snd_oxfw *oxfw, const struct ieee1394_device_id *ent strcpy(oxfw->card->mixername, m); strcpy(oxfw->card->shortname, m); - snprintf(oxfw->card->longname, sizeof(oxfw->card->longname), - "%s %s (OXFW%x %04x), GUID %08x%08x at %s, S%d", - v, m, firmware >> 20, firmware & 0xffff, - fw_dev->config_rom[3], fw_dev->config_rom[4], - dev_name(&oxfw->unit->device), 100 << fw_dev->max_speed); + scnprintf(oxfw->card->longname, sizeof(oxfw->card->longname), + "%s %s (OXFW%x %04x), GUID %08x%08x at %s, S%d", + v, m, firmware >> 20, firmware & 0xffff, + fw_dev->config_rom[3], fw_dev->config_rom[4], + dev_name(&oxfw->unit->device), 100 << fw_dev->max_speed); end: return err; } diff --git a/sound/firewire/tascam/tascam-midi.c b/sound/firewire/tascam/tascam-midi.c index 02eed2dce435..c57fac4f1968 100644 --- a/sound/firewire/tascam/tascam-midi.c +++ b/sound/firewire/tascam/tascam-midi.c @@ -108,9 +108,9 @@ int snd_tscm_create_midi_devices(struct snd_tscm *tscm) /* TODO: support virtual MIDI ports. */ if (subs->number < tscm->spec->midi_capture_ports) { /* Hardware MIDI ports. */ - snprintf(subs->name, sizeof(subs->name), - "%s MIDI %d", - tscm->card->shortname, subs->number + 1); + scnprintf(subs->name, sizeof(subs->name), + "%s MIDI %d", + tscm->card->shortname, subs->number + 1); } } @@ -123,9 +123,9 @@ int snd_tscm_create_midi_devices(struct snd_tscm *tscm) list_for_each_entry(subs, &stream->substreams, list) { if (subs->number < tscm->spec->midi_playback_ports) { /* Hardware MIDI ports only. */ - snprintf(subs->name, sizeof(subs->name), - "%s MIDI %d", - tscm->card->shortname, subs->number + 1); + scnprintf(subs->name, sizeof(subs->name), + "%s MIDI %d", + tscm->card->shortname, subs->number + 1); } } diff --git a/sound/hda/intel-sdw-acpi.c b/sound/hda/intel-sdw-acpi.c index 5cb92f7ccbca..b57d72ea4503 100644 --- a/sound/hda/intel-sdw-acpi.c +++ b/sound/hda/intel-sdw-acpi.c @@ -23,7 +23,7 @@ static int ctrl_link_mask; module_param_named(sdw_link_mask, ctrl_link_mask, int, 0444); MODULE_PARM_DESC(sdw_link_mask, "Intel link mask (one bit per link)"); -static bool is_link_enabled(struct fwnode_handle *fw_node, int i) +static bool is_link_enabled(struct fwnode_handle *fw_node, u8 idx) { struct fwnode_handle *link; char name[32]; @@ -31,7 +31,7 @@ static bool is_link_enabled(struct fwnode_handle *fw_node, int i) /* Find master handle */ snprintf(name, sizeof(name), - "mipi-sdw-link-%d-subproperties", i); + "mipi-sdw-link-%hhu-subproperties", idx); link = fwnode_get_named_child_node(fw_node, name); if (!link) @@ -51,8 +51,8 @@ static int sdw_intel_scan_controller(struct sdw_intel_acpi_info *info) { struct acpi_device *adev = acpi_fetch_acpi_dev(info->handle); - int ret, i; - u8 count; + u8 count, i; + int ret; if (!adev) return -EINVAL; diff --git a/sound/isa/ad1848/ad1848.c b/sound/isa/ad1848/ad1848.c index c471ac2aa450..401d8df28d87 100644 --- a/sound/isa/ad1848/ad1848.c +++ b/sound/isa/ad1848/ad1848.c @@ -96,13 +96,13 @@ static int snd_ad1848_probe(struct device *dev, unsigned int n) strscpy(card->shortname, chip->pcm->name, sizeof(card->shortname)); if (!thinkpad[n]) - snprintf(card->longname, sizeof(card->longname), - "%s at 0x%lx, irq %d, dma %d", - chip->pcm->name, chip->port, irq[n], dma1[n]); + scnprintf(card->longname, sizeof(card->longname), + "%s at 0x%lx, irq %d, dma %d", + chip->pcm->name, chip->port, irq[n], dma1[n]); else - snprintf(card->longname, sizeof(card->longname), - "%s at 0x%lx, irq %d, dma %d [Thinkpad]", - chip->pcm->name, chip->port, irq[n], dma1[n]); + scnprintf(card->longname, sizeof(card->longname), + "%s at 0x%lx, irq %d, dma %d [Thinkpad]", + chip->pcm->name, chip->port, irq[n], dma1[n]); error = snd_card_register(card); if (error < 0) diff --git a/sound/isa/cs423x/cs4231.c b/sound/isa/cs423x/cs4231.c index 1e8923385366..c87be4be6df1 100644 --- a/sound/isa/cs423x/cs4231.c +++ b/sound/isa/cs423x/cs4231.c @@ -98,13 +98,13 @@ static int snd_cs4231_probe(struct device *dev, unsigned int n) strscpy(card->shortname, chip->pcm->name, sizeof(card->shortname)); if (dma2[n] < 0) - snprintf(card->longname, sizeof(card->longname), - "%s at 0x%lx, irq %d, dma %d", - chip->pcm->name, chip->port, irq[n], dma1[n]); + scnprintf(card->longname, sizeof(card->longname), + "%s at 0x%lx, irq %d, dma %d", + chip->pcm->name, chip->port, irq[n], dma1[n]); else - snprintf(card->longname, sizeof(card->longname), - "%s at 0x%lx, irq %d, dma %d&%d", - chip->pcm->name, chip->port, irq[n], dma1[n], dma2[n]); + scnprintf(card->longname, sizeof(card->longname), + "%s at 0x%lx, irq %d, dma %d&%d", + chip->pcm->name, chip->port, irq[n], dma1[n], dma2[n]); error = snd_wss_mixer(chip); if (error < 0) diff --git a/sound/isa/cs423x/cs4236.c b/sound/isa/cs423x/cs4236.c index 10112e1bb25d..7226cbf2d7de 100644 --- a/sound/isa/cs423x/cs4236.c +++ b/sound/isa/cs423x/cs4236.c @@ -367,14 +367,14 @@ static int snd_cs423x_probe(struct snd_card *card, int dev) strscpy(card->driver, chip->pcm->name, sizeof(card->driver)); strscpy(card->shortname, chip->pcm->name, sizeof(card->shortname)); if (dma2[dev] < 0) - snprintf(card->longname, sizeof(card->longname), - "%s at 0x%lx, irq %i, dma %i", - chip->pcm->name, chip->port, irq[dev], dma1[dev]); + scnprintf(card->longname, sizeof(card->longname), + "%s at 0x%lx, irq %i, dma %i", + chip->pcm->name, chip->port, irq[dev], dma1[dev]); else - snprintf(card->longname, sizeof(card->longname), - "%s at 0x%lx, irq %i, dma %i&%d", - chip->pcm->name, chip->port, irq[dev], dma1[dev], - dma2[dev]); + scnprintf(card->longname, sizeof(card->longname), + "%s at 0x%lx, irq %i, dma %i&%d", + chip->pcm->name, chip->port, irq[dev], dma1[dev], + dma2[dev]); err = snd_wss_timer(chip, 0); if (err < 0) diff --git a/sound/isa/es1688/es1688.c b/sound/isa/es1688/es1688.c index f935b56eeec7..97728bf45474 100644 --- a/sound/isa/es1688/es1688.c +++ b/sound/isa/es1688/es1688.c @@ -130,9 +130,9 @@ static int snd_es1688_probe(struct snd_card *card, unsigned int n) strscpy(card->driver, "ES1688", sizeof(card->driver)); strscpy(card->shortname, chip->pcm->name, sizeof(card->shortname)); - snprintf(card->longname, sizeof(card->longname), - "%s at 0x%lx, irq %i, dma %i", chip->pcm->name, chip->port, - chip->irq, chip->dma8); + scnprintf(card->longname, sizeof(card->longname), + "%s at 0x%lx, irq %i, dma %i", chip->pcm->name, chip->port, + chip->irq, chip->dma8); if (fm_port[n] == SNDRV_AUTO_PORT) fm_port[n] = port[n]; /* share the same port */ diff --git a/sound/isa/opti9xx/miro.c b/sound/isa/opti9xx/miro.c index 59242baed576..59792f2fada1 100644 --- a/sound/isa/opti9xx/miro.c +++ b/sound/isa/opti9xx/miro.c @@ -1344,10 +1344,10 @@ static int snd_miro_probe(struct snd_card *card) } strcpy(card->driver, "miro"); - snprintf(card->longname, sizeof(card->longname), - "%s: OPTi%s, %s at 0x%lx, irq %d, dma %d&%d", - card->shortname, miro->name, codec->pcm->name, - miro->wss_base + 4, miro->irq, miro->dma1, miro->dma2); + scnprintf(card->longname, sizeof(card->longname), + "%s: OPTi%s, %s at 0x%lx, irq %d, dma %d&%d", + card->shortname, miro->name, codec->pcm->name, + miro->wss_base + 4, miro->irq, miro->dma1, miro->dma2); if (mpu_port <= 0 || mpu_port == SNDRV_AUTO_PORT) rmidi = NULL; diff --git a/sound/isa/opti9xx/opti92x-ad1848.c b/sound/isa/opti9xx/opti92x-ad1848.c index 4beeb32fe2a7..c33f67dd5133 100644 --- a/sound/isa/opti9xx/opti92x-ad1848.c +++ b/sound/isa/opti9xx/opti92x-ad1848.c @@ -859,15 +859,15 @@ static int snd_opti9xx_probe(struct snd_card *card) strcpy(card->driver, chip->name); sprintf(card->shortname, "OPTi %s", card->driver); #if defined(CS4231) || defined(OPTi93X) - snprintf(card->longname, sizeof(card->longname), - "%s, %s at 0x%lx, irq %d, dma %d&%d", - card->shortname, codec->pcm->name, - chip->wss_base + 4, irq, dma1, xdma2); + scnprintf(card->longname, sizeof(card->longname), + "%s, %s at 0x%lx, irq %d, dma %d&%d", + card->shortname, codec->pcm->name, + chip->wss_base + 4, irq, dma1, xdma2); #else - snprintf(card->longname, sizeof(card->longname), - "%s, %s at 0x%lx, irq %d, dma %d", - card->shortname, codec->pcm->name, chip->wss_base + 4, irq, - dma1); + scnprintf(card->longname, sizeof(card->longname), + "%s, %s at 0x%lx, irq %d, dma %d", + card->shortname, codec->pcm->name, chip->wss_base + 4, irq, + dma1); #endif /* CS4231 || OPTi93X */ if (mpu_port <= 0 || mpu_port == SNDRV_AUTO_PORT) diff --git a/sound/isa/sscape.c b/sound/isa/sscape.c index 0bc0025f7c19..cc56fafd27b1 100644 --- a/sound/isa/sscape.c +++ b/sound/isa/sscape.c @@ -557,7 +557,7 @@ static int sscape_upload_microcode(struct snd_card *card, int version) char name[14]; int err; - snprintf(name, sizeof(name), "sndscape.co%d", version); + scnprintf(name, sizeof(name), "sndscape.co%d", version); err = request_firmware(&init_fw, name, card->dev); if (err < 0) { diff --git a/sound/pci/cmipci.c b/sound/pci/cmipci.c index 1415baac9c36..08e34b184780 100644 --- a/sound/pci/cmipci.c +++ b/sound/pci/cmipci.c @@ -3102,11 +3102,13 @@ static int snd_cmipci_create(struct snd_card *card, struct pci_dev *pci, } sprintf(card->shortname, "C-Media CMI%d", val); if (cm->chip_version < 68) - sprintf(modelstr, " (model %d)", cm->chip_version); + scnprintf(modelstr, sizeof(modelstr), + " (model %d)", cm->chip_version); else modelstr[0] = '\0'; - sprintf(card->longname, "%s%s at %#lx, irq %i", - card->shortname, modelstr, cm->iobase, cm->irq); + scnprintf(card->longname, sizeof(card->longname), + "%s%s at %#lx, irq %i", + card->shortname, modelstr, cm->iobase, cm->irq); if (cm->chip_version >= 39) { val = snd_cmipci_read_b(cm, CM_REG_MPU_PCI + 1); diff --git a/sound/pci/hda/cs35l41_hda.c b/sound/pci/hda/cs35l41_hda.c index c74faa2ff46c..b24ab7dfd46a 100644 --- a/sound/pci/hda/cs35l41_hda.c +++ b/sound/pci/hda/cs35l41_hda.c @@ -185,10 +185,14 @@ static int cs35l41_request_firmware_files_spkid(struct cs35l41_hda *cs35l41, cs35l41->speaker_id, "wmfw"); if (!ret) { /* try cirrus/part-dspN-fwtype-sub<-spkidN><-ampname>.bin */ - return cs35l41_request_firmware_file(cs35l41, coeff_firmware, coeff_filename, - CS35L41_FIRMWARE_ROOT, - cs35l41->acpi_subsystem_id, cs35l41->amp_name, - cs35l41->speaker_id, "bin"); + ret = cs35l41_request_firmware_file(cs35l41, coeff_firmware, coeff_filename, + CS35L41_FIRMWARE_ROOT, + cs35l41->acpi_subsystem_id, cs35l41->amp_name, + cs35l41->speaker_id, "bin"); + if (ret) + goto coeff_err; + + return 0; } /* try cirrus/part-dspN-fwtype-sub<-ampname>.wmfw */ @@ -197,10 +201,14 @@ static int cs35l41_request_firmware_files_spkid(struct cs35l41_hda *cs35l41, cs35l41->amp_name, -1, "wmfw"); if (!ret) { /* try cirrus/part-dspN-fwtype-sub<-spkidN><-ampname>.bin */ - return cs35l41_request_firmware_file(cs35l41, coeff_firmware, coeff_filename, - CS35L41_FIRMWARE_ROOT, - cs35l41->acpi_subsystem_id, cs35l41->amp_name, - cs35l41->speaker_id, "bin"); + ret = cs35l41_request_firmware_file(cs35l41, coeff_firmware, coeff_filename, + CS35L41_FIRMWARE_ROOT, + cs35l41->acpi_subsystem_id, cs35l41->amp_name, + cs35l41->speaker_id, "bin"); + if (ret) + goto coeff_err; + + return 0; } /* try cirrus/part-dspN-fwtype-sub<-spkidN>.wmfw */ @@ -215,10 +223,14 @@ static int cs35l41_request_firmware_files_spkid(struct cs35l41_hda *cs35l41, cs35l41->amp_name, cs35l41->speaker_id, "bin"); if (ret) /* try cirrus/part-dspN-fwtype-sub<-spkidN>.bin */ - return cs35l41_request_firmware_file(cs35l41, coeff_firmware, - coeff_filename, CS35L41_FIRMWARE_ROOT, - cs35l41->acpi_subsystem_id, NULL, - cs35l41->speaker_id, "bin"); + ret = cs35l41_request_firmware_file(cs35l41, coeff_firmware, + coeff_filename, CS35L41_FIRMWARE_ROOT, + cs35l41->acpi_subsystem_id, NULL, + cs35l41->speaker_id, "bin"); + if (ret) + goto coeff_err; + + return 0; } /* try cirrus/part-dspN-fwtype-sub.wmfw */ @@ -233,12 +245,50 @@ static int cs35l41_request_firmware_files_spkid(struct cs35l41_hda *cs35l41, cs35l41->speaker_id, "bin"); if (ret) /* try cirrus/part-dspN-fwtype-sub<-spkidN>.bin */ - return cs35l41_request_firmware_file(cs35l41, coeff_firmware, - coeff_filename, CS35L41_FIRMWARE_ROOT, - cs35l41->acpi_subsystem_id, NULL, - cs35l41->speaker_id, "bin"); + ret = cs35l41_request_firmware_file(cs35l41, coeff_firmware, + coeff_filename, CS35L41_FIRMWARE_ROOT, + cs35l41->acpi_subsystem_id, NULL, + cs35l41->speaker_id, "bin"); + if (ret) + goto coeff_err; + } + + return ret; +coeff_err: + release_firmware(*wmfw_firmware); + kfree(*wmfw_filename); + return ret; +} + +static int cs35l41_fallback_firmware_file(struct cs35l41_hda *cs35l41, + const struct firmware **wmfw_firmware, + char **wmfw_filename, + const struct firmware **coeff_firmware, + char **coeff_filename) +{ + int ret; + + /* Handle fallback */ + dev_warn(cs35l41->dev, "Falling back to default firmware.\n"); + + /* fallback try cirrus/part-dspN-fwtype.wmfw */ + ret = cs35l41_request_firmware_file(cs35l41, wmfw_firmware, wmfw_filename, + CS35L41_FIRMWARE_ROOT, NULL, NULL, -1, "wmfw"); + if (ret) + goto err; + + /* fallback try cirrus/part-dspN-fwtype.bin */ + ret = cs35l41_request_firmware_file(cs35l41, coeff_firmware, coeff_filename, + CS35L41_FIRMWARE_ROOT, NULL, NULL, -1, "bin"); + if (ret) { + release_firmware(*wmfw_firmware); + kfree(*wmfw_filename); + goto err; } + return 0; +err: + dev_warn(cs35l41->dev, "Unable to find firmware and tuning\n"); return ret; } @@ -254,7 +304,6 @@ static int cs35l41_request_firmware_files(struct cs35l41_hda *cs35l41, ret = cs35l41_request_firmware_files_spkid(cs35l41, wmfw_firmware, wmfw_filename, coeff_firmware, coeff_filename); goto out; - } /* try cirrus/part-dspN-fwtype-sub<-ampname>.wmfw */ @@ -267,6 +316,9 @@ static int cs35l41_request_firmware_files(struct cs35l41_hda *cs35l41, CS35L41_FIRMWARE_ROOT, cs35l41->acpi_subsystem_id, cs35l41->amp_name, -1, "bin"); + if (ret) + goto coeff_err; + goto out; } @@ -286,32 +338,23 @@ static int cs35l41_request_firmware_files(struct cs35l41_hda *cs35l41, CS35L41_FIRMWARE_ROOT, cs35l41->acpi_subsystem_id, NULL, -1, "bin"); + if (ret) + goto coeff_err; } out: - if (!ret) - return 0; + if (ret) + /* if all attempts at finding firmware fail, try fallback */ + goto fallback; - /* Handle fallback */ - dev_warn(cs35l41->dev, "Falling back to default firmware.\n"); + return 0; +coeff_err: release_firmware(*wmfw_firmware); kfree(*wmfw_filename); - - /* fallback try cirrus/part-dspN-fwtype.wmfw */ - ret = cs35l41_request_firmware_file(cs35l41, wmfw_firmware, wmfw_filename, - CS35L41_FIRMWARE_ROOT, NULL, NULL, -1, "wmfw"); - if (!ret) - /* fallback try cirrus/part-dspN-fwtype.bin */ - ret = cs35l41_request_firmware_file(cs35l41, coeff_firmware, coeff_filename, - CS35L41_FIRMWARE_ROOT, NULL, NULL, -1, "bin"); - - if (ret) { - release_firmware(*wmfw_firmware); - kfree(*wmfw_filename); - dev_warn(cs35l41->dev, "Unable to find firmware and tuning\n"); - } - return ret; +fallback: + return cs35l41_fallback_firmware_file(cs35l41, wmfw_firmware, wmfw_filename, + coeff_firmware, coeff_filename); } #if IS_ENABLED(CONFIG_EFI) diff --git a/sound/pci/hda/cs35l56_hda.c b/sound/pci/hda/cs35l56_hda.c index 76b9c685560b..7adc1d373d65 100644 --- a/sound/pci/hda/cs35l56_hda.c +++ b/sound/pci/hda/cs35l56_hda.c @@ -105,7 +105,7 @@ static void cs35l56_hda_playback_hook(struct device *dev, int action) } } -static int __maybe_unused cs35l56_hda_runtime_suspend(struct device *dev) +static int cs35l56_hda_runtime_suspend(struct device *dev) { struct cs35l56_hda *cs35l56 = dev_get_drvdata(dev); @@ -115,7 +115,7 @@ static int __maybe_unused cs35l56_hda_runtime_suspend(struct device *dev) return cs35l56_runtime_suspend_common(&cs35l56->base); } -static int __maybe_unused cs35l56_hda_runtime_resume(struct device *dev) +static int cs35l56_hda_runtime_resume(struct device *dev) { struct cs35l56_hda *cs35l56 = dev_get_drvdata(dev); int ret; @@ -218,7 +218,7 @@ static int cs35l56_hda_posture_get(struct snd_kcontrol *kcontrol, ucontrol->value.integer.value[0] = pos; - return ret; + return 0; } static int cs35l56_hda_posture_put(struct snd_kcontrol *kcontrol, @@ -865,15 +865,13 @@ static int cs35l56_hda_read_acpi(struct cs35l56_hda *cs35l56, int id) sub = acpi_get_subsystem_id(ACPI_HANDLE(cs35l56->base.dev)); if (IS_ERR(sub)) { - /* If no ACPI SUB, return 0 and fallback to legacy firmware path, otherwise fail */ - if (PTR_ERR(sub) == -ENODATA) - return 0; - else - return PTR_ERR(sub); + dev_info(cs35l56->base.dev, + "Read ACPI _SUB failed(%ld): fallback to generic firmware\n", + PTR_ERR(sub)); + } else { + cs35l56->system_name = sub; } - cs35l56->system_name = sub; - cs35l56->base.reset_gpio = devm_gpiod_get_index_optional(cs35l56->base.dev, "reset", cs35l56->index, @@ -1003,6 +1001,7 @@ void cs35l56_hda_remove(struct device *dev) { struct cs35l56_hda *cs35l56 = dev_get_drvdata(dev); + pm_runtime_dont_use_autosuspend(cs35l56->base.dev); pm_runtime_get_sync(cs35l56->base.dev); pm_runtime_disable(cs35l56->base.dev); @@ -1016,7 +1015,7 @@ void cs35l56_hda_remove(struct device *dev) EXPORT_SYMBOL_NS_GPL(cs35l56_hda_remove, SND_HDA_SCODEC_CS35L56); const struct dev_pm_ops cs35l56_hda_pm_ops = { - SET_RUNTIME_PM_OPS(cs35l56_hda_runtime_suspend, cs35l56_hda_runtime_resume, NULL) + RUNTIME_PM_OPS(cs35l56_hda_runtime_suspend, cs35l56_hda_runtime_resume, NULL) SYSTEM_SLEEP_PM_OPS(cs35l56_hda_system_suspend, cs35l56_hda_system_resume) LATE_SYSTEM_SLEEP_PM_OPS(cs35l56_hda_system_suspend_late, cs35l56_hda_system_resume_early) diff --git a/sound/pci/hda/cs35l56_hda_i2c.c b/sound/pci/hda/cs35l56_hda_i2c.c index 83e4acdd89ac..757a4d193e0f 100644 --- a/sound/pci/hda/cs35l56_hda_i2c.c +++ b/sound/pci/hda/cs35l56_hda_i2c.c @@ -21,7 +21,6 @@ static int cs35l56_hda_i2c_probe(struct i2c_client *clt) return -ENOMEM; cs35l56->base.dev = &clt->dev; - cs35l56->base.can_hibernate = true; cs35l56->base.regmap = devm_regmap_init_i2c(clt, &cs35l56_regmap_i2c); if (IS_ERR(cs35l56->base.regmap)) { ret = PTR_ERR(cs35l56->base.regmap); diff --git a/sound/pci/hda/hda_generic.c b/sound/pci/hda/hda_generic.c index dbf7aa88e0e3..bf685d01259d 100644 --- a/sound/pci/hda/hda_generic.c +++ b/sound/pci/hda/hda_generic.c @@ -998,7 +998,11 @@ static int add_control_with_pfx(struct hda_gen_spec *spec, int type, const char *sfx, int cidx, unsigned long val) { char name[SNDRV_CTL_ELEM_ID_NAME_MAXLEN]; - snprintf(name, sizeof(name), "%s %s %s", pfx, dir, sfx); + int len; + + len = snprintf(name, sizeof(name), "%s %s %s", pfx, dir, sfx); + if (snd_BUG_ON(len >= sizeof(name))) + return -EINVAL; if (!add_control(spec, type, name, cidx, val)) return -ENOMEM; return 0; diff --git a/sound/pci/hda/hda_intel.c b/sound/pci/hda/hda_intel.c index 765d95e79861..ca765ac4765f 100644 --- a/sound/pci/hda/hda_intel.c +++ b/sound/pci/hda/hda_intel.c @@ -2211,6 +2211,7 @@ static const struct snd_pci_quirk power_save_denylist[] = { SND_PCI_QUIRK(0x8086, 0x2068, "Intel NUC7i3BNB", 0), /* https://bugzilla.kernel.org/show_bug.cgi?id=198611 */ SND_PCI_QUIRK(0x17aa, 0x2227, "Lenovo X1 Carbon 3rd Gen", 0), + SND_PCI_QUIRK(0x17aa, 0x316e, "Lenovo ThinkCentre M70q", 0), /* https://bugzilla.redhat.com/show_bug.cgi?id=1689623 */ SND_PCI_QUIRK(0x17aa, 0x367b, "Lenovo IdeaCentre B550", 0), /* https://bugzilla.redhat.com/show_bug.cgi?id=1572975 */ diff --git a/sound/pci/hda/patch_realtek.c b/sound/pci/hda/patch_realtek.c index b7e78bfcffd8..9677c09cf7a9 100644 --- a/sound/pci/hda/patch_realtek.c +++ b/sound/pci/hda/patch_realtek.c @@ -7073,8 +7073,28 @@ static void alc287_fixup_bind_dacs(struct hda_codec *codec, snd_hda_override_conn_list(codec, 0x17, ARRAY_SIZE(conn), conn); spec->gen.preferred_dacs = preferred_pairs; spec->gen.auto_mute_via_amp = 1; - snd_hda_codec_write_cache(codec, 0x14, 0, AC_VERB_SET_PIN_WIDGET_CONTROL, - 0x0); /* Make sure 0x14 was disable */ + if (spec->gen.autocfg.speaker_pins[0] != 0x14) { + snd_hda_codec_write_cache(codec, 0x14, 0, AC_VERB_SET_PIN_WIDGET_CONTROL, + 0x0); /* Make sure 0x14 was disable */ + } +} +/* Fix none verb table of Headset Mic pin */ +static void alc_fixup_headset_mic(struct hda_codec *codec, + const struct hda_fixup *fix, int action) +{ + struct alc_spec *spec = codec->spec; + static const struct hda_pintbl pincfgs[] = { + { 0x19, 0x03a1103c }, + { } + }; + + switch (action) { + case HDA_FIXUP_ACT_PRE_PROBE: + snd_hda_apply_pincfgs(codec, pincfgs); + alc_update_coef_idx(codec, 0x45, 0xf<<12 | 1<<10, 5<<12); + spec->parse_flags |= HDA_PINCFG_HEADSET_MIC; + break; + } } @@ -7341,6 +7361,8 @@ enum { ALC245_FIXUP_HP_MUTE_LED_COEFBIT, ALC245_FIXUP_HP_X360_MUTE_LEDS, ALC287_FIXUP_THINKPAD_I2S_SPK, + ALC287_FIXUP_MG_RTKC_CSAMP_CS35L41_I2C_THINKPAD, + ALC2XX_FIXUP_HEADSET_MIC, }; /* A special fixup for Lenovo C940 and Yoga Duet 7; @@ -9439,6 +9461,16 @@ static const struct hda_fixup alc269_fixups[] = { .type = HDA_FIXUP_FUNC, .v.func = alc287_fixup_bind_dacs, }, + [ALC287_FIXUP_MG_RTKC_CSAMP_CS35L41_I2C_THINKPAD] = { + .type = HDA_FIXUP_FUNC, + .v.func = alc287_fixup_bind_dacs, + .chained = true, + .chain_id = ALC287_FIXUP_CS35L41_I2C_2_THINKPAD_ACPI, + }, + [ALC2XX_FIXUP_HEADSET_MIC] = { + .type = HDA_FIXUP_FUNC, + .v.func = alc_fixup_headset_mic, + }, }; static const struct snd_pci_quirk alc269_fixup_tbl[] = { @@ -9713,6 +9745,7 @@ static const struct snd_pci_quirk alc269_fixup_tbl[] = { SND_PCI_QUIRK(0x103c, 0x89c6, "Zbook Fury 17 G9", ALC245_FIXUP_CS35L41_SPI_2_HP_GPIO_LED), SND_PCI_QUIRK(0x103c, 0x89ca, "HP", ALC236_FIXUP_HP_MUTE_LED_MICMUTE_VREF), SND_PCI_QUIRK(0x103c, 0x89d3, "HP EliteBook 645 G9 (MB 89D2)", ALC236_FIXUP_HP_MUTE_LED_MICMUTE_VREF), + SND_PCI_QUIRK(0x103c, 0x8a20, "HP Laptop 15s-fq5xxx", ALC236_FIXUP_HP_MUTE_LED_COEFBIT2), SND_PCI_QUIRK(0x103c, 0x8a25, "HP Victus 16-d1xxx (MB 8A25)", ALC245_FIXUP_HP_MUTE_LED_COEFBIT), SND_PCI_QUIRK(0x103c, 0x8a78, "HP Dev One", ALC285_FIXUP_HP_LIMIT_INT_MIC_BOOST), SND_PCI_QUIRK(0x103c, 0x8aa0, "HP ProBook 440 G9 (MB 8A9E)", ALC236_FIXUP_HP_GPIO_LED), @@ -9782,6 +9815,7 @@ static const struct snd_pci_quirk alc269_fixup_tbl[] = { SND_PCI_QUIRK(0x1043, 0x1517, "Asus Zenbook UX31A", ALC269VB_FIXUP_ASUS_ZENBOOK_UX31A), SND_PCI_QUIRK(0x1043, 0x1573, "ASUS GZ301V", ALC285_FIXUP_ASUS_HEADSET_MIC), SND_PCI_QUIRK(0x1043, 0x1662, "ASUS GV301QH", ALC294_FIXUP_ASUS_DUAL_SPK), + SND_PCI_QUIRK(0x1043, 0x1663, "ASUS GU603ZV", ALC285_FIXUP_ASUS_HEADSET_MIC), SND_PCI_QUIRK(0x1043, 0x1683, "ASUS UM3402YAR", ALC287_FIXUP_CS35L41_I2C_2), SND_PCI_QUIRK(0x1043, 0x16b2, "ASUS GU603", ALC289_FIXUP_ASUS_GA401), SND_PCI_QUIRK(0x1043, 0x16e3, "ASUS UX50", ALC269_FIXUP_STEREO_DMIC), @@ -9812,7 +9846,8 @@ static const struct snd_pci_quirk alc269_fixup_tbl[] = { SND_PCI_QUIRK(0x1043, 0x1d1f, "ASUS ROG Strix G17 2023 (G713PV)", ALC287_FIXUP_CS35L41_I2C_2), SND_PCI_QUIRK(0x1043, 0x1d42, "ASUS Zephyrus G14 2022", ALC289_FIXUP_ASUS_GA401), SND_PCI_QUIRK(0x1043, 0x1d4e, "ASUS TM420", ALC256_FIXUP_ASUS_HPE), - SND_PCI_QUIRK(0x1043, 0x1e02, "ASUS UX3402", ALC245_FIXUP_CS35L41_SPI_2), + SND_PCI_QUIRK(0x1043, 0x1e02, "ASUS UX3402ZA", ALC245_FIXUP_CS35L41_SPI_2), + SND_PCI_QUIRK(0x1043, 0x16a3, "ASUS UX3402VA", ALC245_FIXUP_CS35L41_SPI_2), SND_PCI_QUIRK(0x1043, 0x1e11, "ASUS Zephyrus G15", ALC289_FIXUP_ASUS_GA502), SND_PCI_QUIRK(0x1043, 0x1e12, "ASUS UM3402", ALC287_FIXUP_CS35L41_I2C_2), SND_PCI_QUIRK(0x1043, 0x1e51, "ASUS Zephyrus M15", ALC294_FIXUP_ASUS_GU502_PINS), @@ -9851,7 +9886,7 @@ static const struct snd_pci_quirk alc269_fixup_tbl[] = { SND_PCI_QUIRK(0x10ec, 0x124c, "Intel Reference board", ALC295_FIXUP_CHROME_BOOK), SND_PCI_QUIRK(0x10ec, 0x1252, "Intel Reference board", ALC295_FIXUP_CHROME_BOOK), SND_PCI_QUIRK(0x10ec, 0x1254, "Intel Reference board", ALC295_FIXUP_CHROME_BOOK), - SND_PCI_QUIRK(0x10ec, 0x12cc, "Intel Reference board", ALC225_FIXUP_HEADSET_JACK), + SND_PCI_QUIRK(0x10ec, 0x12cc, "Intel Reference board", ALC295_FIXUP_CHROME_BOOK), SND_PCI_QUIRK(0x10f7, 0x8338, "Panasonic CF-SZ6", ALC269_FIXUP_HEADSET_MODE), SND_PCI_QUIRK(0x144d, 0xc109, "Samsung Ativ book 9 (NP900X3G)", ALC269_FIXUP_INV_DMIC), SND_PCI_QUIRK(0x144d, 0xc169, "Samsung Notebook 9 Pen (NP930SBE-K01US)", ALC298_FIXUP_SAMSUNG_AMP), @@ -9985,14 +10020,14 @@ static const struct snd_pci_quirk alc269_fixup_tbl[] = { SND_PCI_QUIRK(0x17aa, 0x22be, "Thinkpad X1 Carbon 8th", ALC285_FIXUP_THINKPAD_HEADSET_JACK), SND_PCI_QUIRK(0x17aa, 0x22c1, "Thinkpad P1 Gen 3", ALC285_FIXUP_THINKPAD_NO_BASS_SPK_HEADSET_JACK), SND_PCI_QUIRK(0x17aa, 0x22c2, "Thinkpad X1 Extreme Gen 3", ALC285_FIXUP_THINKPAD_NO_BASS_SPK_HEADSET_JACK), - SND_PCI_QUIRK(0x17aa, 0x22f1, "Thinkpad", ALC287_FIXUP_CS35L41_I2C_2_THINKPAD_ACPI), - SND_PCI_QUIRK(0x17aa, 0x22f2, "Thinkpad", ALC287_FIXUP_CS35L41_I2C_2_THINKPAD_ACPI), - SND_PCI_QUIRK(0x17aa, 0x22f3, "Thinkpad", ALC287_FIXUP_CS35L41_I2C_2_THINKPAD_ACPI), - SND_PCI_QUIRK(0x17aa, 0x2316, "Thinkpad P1 Gen 6", ALC287_FIXUP_CS35L41_I2C_2_THINKPAD_ACPI), - SND_PCI_QUIRK(0x17aa, 0x2317, "Thinkpad P1 Gen 6", ALC287_FIXUP_CS35L41_I2C_2_THINKPAD_ACPI), - SND_PCI_QUIRK(0x17aa, 0x2318, "Thinkpad Z13 Gen2", ALC287_FIXUP_CS35L41_I2C_2_THINKPAD_ACPI), - SND_PCI_QUIRK(0x17aa, 0x2319, "Thinkpad Z16 Gen2", ALC287_FIXUP_CS35L41_I2C_2_THINKPAD_ACPI), - SND_PCI_QUIRK(0x17aa, 0x231a, "Thinkpad Z16 Gen2", ALC287_FIXUP_CS35L41_I2C_2_THINKPAD_ACPI), + SND_PCI_QUIRK(0x17aa, 0x22f1, "Thinkpad", ALC287_FIXUP_MG_RTKC_CSAMP_CS35L41_I2C_THINKPAD), + SND_PCI_QUIRK(0x17aa, 0x22f2, "Thinkpad", ALC287_FIXUP_MG_RTKC_CSAMP_CS35L41_I2C_THINKPAD), + SND_PCI_QUIRK(0x17aa, 0x22f3, "Thinkpad", ALC287_FIXUP_MG_RTKC_CSAMP_CS35L41_I2C_THINKPAD), + SND_PCI_QUIRK(0x17aa, 0x2316, "Thinkpad P1 Gen 6", ALC287_FIXUP_MG_RTKC_CSAMP_CS35L41_I2C_THINKPAD), + SND_PCI_QUIRK(0x17aa, 0x2317, "Thinkpad P1 Gen 6", ALC287_FIXUP_MG_RTKC_CSAMP_CS35L41_I2C_THINKPAD), + SND_PCI_QUIRK(0x17aa, 0x2318, "Thinkpad Z13 Gen2", ALC287_FIXUP_MG_RTKC_CSAMP_CS35L41_I2C_THINKPAD), + SND_PCI_QUIRK(0x17aa, 0x2319, "Thinkpad Z16 Gen2", ALC287_FIXUP_MG_RTKC_CSAMP_CS35L41_I2C_THINKPAD), + SND_PCI_QUIRK(0x17aa, 0x231a, "Thinkpad Z16 Gen2", ALC287_FIXUP_MG_RTKC_CSAMP_CS35L41_I2C_THINKPAD), SND_PCI_QUIRK(0x17aa, 0x30bb, "ThinkCentre AIO", ALC233_FIXUP_LENOVO_LINE2_MIC_HOTKEY), SND_PCI_QUIRK(0x17aa, 0x30e2, "ThinkCentre AIO", ALC233_FIXUP_LENOVO_LINE2_MIC_HOTKEY), SND_PCI_QUIRK(0x17aa, 0x310c, "ThinkCentre Station", ALC294_FIXUP_LENOVO_MIC_LOCATION), @@ -10088,7 +10123,7 @@ static const struct snd_pci_quirk alc269_fixup_tbl[] = { SND_PCI_QUIRK(0x8086, 0x2074, "Intel NUC 8", ALC233_FIXUP_INTEL_NUC8_DMIC), SND_PCI_QUIRK(0x8086, 0x2080, "Intel NUC 8 Rugged", ALC256_FIXUP_INTEL_NUC8_RUGGED), SND_PCI_QUIRK(0x8086, 0x2081, "Intel NUC 10", ALC256_FIXUP_INTEL_NUC10), - SND_PCI_QUIRK(0x8086, 0x3038, "Intel NUC 13", ALC225_FIXUP_HEADSET_JACK), + SND_PCI_QUIRK(0x8086, 0x3038, "Intel NUC 13", ALC295_FIXUP_CHROME_BOOK), SND_PCI_QUIRK(0xf111, 0x0001, "Framework Laptop", ALC295_FIXUP_FRAMEWORK_LAPTOP_MIC_NO_PRESENCE), #if 0 @@ -10574,6 +10609,10 @@ static const struct snd_hda_pin_quirk alc269_pin_fixup_tbl[] = { {0x17, 0x90170110}, {0x19, 0x03a11030}, {0x21, 0x03211020}), + SND_HDA_PIN_QUIRK(0x10ec0287, 0x17aa, "Lenovo", ALC287_FIXUP_THINKPAD_I2S_SPK, + {0x17, 0x90170110}, /* 0x231f with RTK I2S AMP */ + {0x19, 0x04a11040}, + {0x21, 0x04211020}), SND_HDA_PIN_QUIRK(0x10ec0286, 0x1025, "Acer", ALC286_FIXUP_ACER_AIO_MIC_NO_PRESENCE, {0x12, 0x90a60130}, {0x17, 0x90170110}, @@ -10736,6 +10775,8 @@ static const struct snd_hda_pin_quirk alc269_fallback_pin_fixup_tbl[] = { SND_HDA_PIN_QUIRK(0x10ec0274, 0x1028, "Dell", ALC274_FIXUP_DELL_AIO_LINEOUT_VERB, {0x19, 0x40000000}, {0x1a, 0x40000000}), + SND_HDA_PIN_QUIRK(0x10ec0256, 0x1043, "ASUS", ALC2XX_FIXUP_HEADSET_MIC, + {0x19, 0x40000000}), {} }; diff --git a/sound/pci/riptide/riptide.c b/sound/pci/riptide/riptide.c index b37c877c2c16..9dee0345f22c 100644 --- a/sound/pci/riptide/riptide.c +++ b/sound/pci/riptide/riptide.c @@ -2105,15 +2105,15 @@ __snd_card_riptide_probe(struct pci_dev *pci, const struct pci_device_id *pci_id strcpy(card->driver, "RIPTIDE"); strcpy(card->shortname, "Riptide"); #ifdef SUPPORT_JOYSTICK - snprintf(card->longname, sizeof(card->longname), - "%s at 0x%lx, irq %i mpu 0x%x opl3 0x%x gameport 0x%x", - card->shortname, chip->port, chip->irq, chip->mpuaddr, - chip->opladdr, chip->gameaddr); + scnprintf(card->longname, sizeof(card->longname), + "%s at 0x%lx, irq %i mpu 0x%x opl3 0x%x gameport 0x%x", + card->shortname, chip->port, chip->irq, chip->mpuaddr, + chip->opladdr, chip->gameaddr); #else - snprintf(card->longname, sizeof(card->longname), - "%s at 0x%lx, irq %i mpu 0x%x opl3 0x%x", - card->shortname, chip->port, chip->irq, chip->mpuaddr, - chip->opladdr); + scnprintf(card->longname, sizeof(card->longname), + "%s at 0x%lx, irq %i mpu 0x%x opl3 0x%x", + card->shortname, chip->port, chip->irq, chip->mpuaddr, + chip->opladdr); #endif snd_riptide_proc_init(chip); err = snd_card_register(card); diff --git a/sound/soc/codecs/cs42l42-sdw.c b/sound/soc/codecs/cs42l42-sdw.c index 974bae4abfad..94a66a325303 100644 --- a/sound/soc/codecs/cs42l42-sdw.c +++ b/sound/soc/codecs/cs42l42-sdw.c @@ -6,6 +6,7 @@ #include #include +#include #include #include #include diff --git a/sound/soc/codecs/da7219-aad.c b/sound/soc/codecs/da7219-aad.c index 581b334a6631..3bbe85091649 100644 --- a/sound/soc/codecs/da7219-aad.c +++ b/sound/soc/codecs/da7219-aad.c @@ -59,9 +59,6 @@ static void da7219_aad_btn_det_work(struct work_struct *work) bool micbias_up = false; int retries = 0; - /* Disable ground switch */ - snd_soc_component_update_bits(component, 0xFB, 0x01, 0x00); - /* Drive headphones/lineout */ snd_soc_component_update_bits(component, DA7219_HP_L_CTRL, DA7219_HP_L_AMP_OE_MASK, @@ -155,9 +152,6 @@ static void da7219_aad_hptest_work(struct work_struct *work) tonegen_freq_hptest = cpu_to_le16(DA7219_AAD_HPTEST_RAMP_FREQ_INT_OSC); } - /* Disable ground switch */ - snd_soc_component_update_bits(component, 0xFB, 0x01, 0x00); - /* Ensure gain ramping at fastest rate */ gain_ramp_ctrl = snd_soc_component_read(component, DA7219_GAIN_RAMP_CTRL); snd_soc_component_write(component, DA7219_GAIN_RAMP_CTRL, DA7219_GAIN_RAMP_RATE_X8); @@ -421,6 +415,11 @@ static irqreturn_t da7219_aad_irq_thread(int irq, void *data) * handle a removal, and we can check at the end of * hptest if we have a valid result or not. */ + + cancel_delayed_work_sync(&da7219_aad->jack_det_work); + /* Disable ground switch */ + snd_soc_component_update_bits(component, 0xFB, 0x01, 0x00); + if (statusa & DA7219_JACK_TYPE_STS_MASK) { report |= SND_JACK_HEADSET; mask |= SND_JACK_HEADSET | SND_JACK_LINEOUT; diff --git a/sound/soc/codecs/rt5645.c b/sound/soc/codecs/rt5645.c index 1a137ca3f496..7938b52d741d 100644 --- a/sound/soc/codecs/rt5645.c +++ b/sound/soc/codecs/rt5645.c @@ -3257,6 +3257,8 @@ int rt5645_set_jack_detect(struct snd_soc_component *component, RT5645_GP1_PIN_IRQ, RT5645_GP1_PIN_IRQ); regmap_update_bits(rt5645->regmap, RT5645_GEN_CTRL1, RT5645_DIG_GATE_CTRL, RT5645_DIG_GATE_CTRL); + regmap_update_bits(rt5645->regmap, RT5645_DEPOP_M1, + RT5645_HP_CB_MASK, RT5645_HP_CB_PU); } rt5645_irq(0, rt5645); diff --git a/sound/soc/codecs/tas2780.c b/sound/soc/codecs/tas2780.c index 86bd6c18a944..41076be23854 100644 --- a/sound/soc/codecs/tas2780.c +++ b/sound/soc/codecs/tas2780.c @@ -39,7 +39,7 @@ static void tas2780_reset(struct tas2780_priv *tas2780) usleep_range(2000, 2050); } - snd_soc_component_write(tas2780->component, TAS2780_SW_RST, + ret = snd_soc_component_write(tas2780->component, TAS2780_SW_RST, TAS2780_RST); if (ret) dev_err(tas2780->dev, "%s:errCode:0x%x Reset error!\n", diff --git a/sound/soc/dwc/dwc-i2s.c b/sound/soc/dwc/dwc-i2s.c index 2ff619a29655..c04466f5492e 100644 --- a/sound/soc/dwc/dwc-i2s.c +++ b/sound/soc/dwc/dwc-i2s.c @@ -917,7 +917,7 @@ static int jh7110_i2stx0_clk_cfg(struct i2s_clk_config_data *config) static int dw_i2s_probe(struct platform_device *pdev) { - const struct i2s_platform_data *pdata = of_device_get_match_data(&pdev->dev); + const struct i2s_platform_data *pdata = pdev->dev.platform_data; struct dw_i2s_dev *dev; struct resource *res; int ret, irq; diff --git a/sound/soc/ti/ams-delta.c b/sound/soc/ti/ams-delta.c index 687c1d969285..9c2cd993763a 100644 --- a/sound/soc/ti/ams-delta.c +++ b/sound/soc/ti/ams-delta.c @@ -336,8 +336,8 @@ static void cx81801_hangup(struct tty_struct *tty) } /* Line discipline .receive_buf() */ -static void cx81801_receive(struct tty_struct *tty, const u8 *cp, - const char *fp, int count) +static void cx81801_receive(struct tty_struct *tty, const u8 *cp, const u8 *fp, + size_t count) { struct snd_soc_component *component = tty->disc_data; const unsigned char *c; diff --git a/sound/usb/caiaq/device.c b/sound/usb/caiaq/device.c index 49f63f878e6f..b5cbf1f195c4 100644 --- a/sound/usb/caiaq/device.c +++ b/sound/usb/caiaq/device.c @@ -485,7 +485,7 @@ static int init_card(struct snd_usb_caiaqdev *cdev) } usb_make_path(usb_dev, usbpath, sizeof(usbpath)); - snprintf(card->longname, sizeof(card->longname), "%s %s (%s)", + scnprintf(card->longname, sizeof(card->longname), "%s %s (%s)", cdev->vendor_name, cdev->product_name, usbpath); setup_card(cdev); diff --git a/sound/usb/mixer.c b/sound/usb/mixer.c index 9105ec623120..409fc1164694 100644 --- a/sound/usb/mixer.c +++ b/sound/usb/mixer.c @@ -1204,6 +1204,13 @@ static void volume_control_quirks(struct usb_mixer_elem_info *cval, cval->res = 16; } break; + case USB_ID(0x1bcf, 0x2283): /* NexiGo N930AF FHD Webcam */ + if (!strcmp(kctl->id.name, "Mic Capture Volume")) { + usb_audio_info(chip, + "set resolution quirk: cval->res = 16\n"); + cval->res = 16; + } + break; } } @@ -1929,7 +1936,6 @@ static int parse_clock_source_unit(struct mixer_build *state, int unitid, struct uac_clock_source_descriptor *hdr = _ftr; struct usb_mixer_elem_info *cval; struct snd_kcontrol *kctl; - char name[SNDRV_CTL_ELEM_ID_NAME_MAXLEN]; int ret; if (state->mixer->protocol != UAC_VERSION_2) @@ -1966,10 +1972,9 @@ static int parse_clock_source_unit(struct mixer_build *state, int unitid, kctl->private_free = snd_usb_mixer_elem_free; ret = snd_usb_copy_string_desc(state->chip, hdr->iClockSource, - name, sizeof(name)); + kctl->id.name, sizeof(kctl->id.name)); if (ret > 0) - snprintf(kctl->id.name, sizeof(kctl->id.name), - "%s Validity", name); + append_ctl_name(kctl, " Validity"); else snprintf(kctl->id.name, sizeof(kctl->id.name), "Clock Source %d Validity", hdr->bClockID); diff --git a/sound/usb/mixer_scarlett_gen2.c b/sound/usb/mixer_scarlett_gen2.c index 9d11bb08667e..d260be8cb6bc 100644 --- a/sound/usb/mixer_scarlett_gen2.c +++ b/sound/usb/mixer_scarlett_gen2.c @@ -3205,8 +3205,8 @@ static int scarlett2_add_line_in_ctls(struct usb_mixer_interface *mixer) /* Add input phantom controls */ if (info->inputs_per_phantom == 1) { for (i = 0; i < info->phantom_count; i++) { - snprintf(s, sizeof(s), fmt, i + 1, - "Phantom Power", "Switch"); + scnprintf(s, sizeof(s), fmt, i + 1, + "Phantom Power", "Switch"); err = scarlett2_add_new_ctl( mixer, &scarlett2_phantom_ctl, i, 1, s, &private->phantom_ctls[i]); @@ -3218,8 +3218,8 @@ static int scarlett2_add_line_in_ctls(struct usb_mixer_interface *mixer) int from = i * info->inputs_per_phantom + 1; int to = (i + 1) * info->inputs_per_phantom; - snprintf(s, sizeof(s), fmt2, from, to, - "Phantom Power", "Switch"); + scnprintf(s, sizeof(s), fmt2, from, to, + "Phantom Power", "Switch"); err = scarlett2_add_new_ctl( mixer, &scarlett2_phantom_ctl, i, 1, s, &private->phantom_ctls[i]); diff --git a/sound/usb/quirks.c b/sound/usb/quirks.c index 598659d761cc..4e64842245e1 100644 --- a/sound/usb/quirks.c +++ b/sound/usb/quirks.c @@ -1994,7 +1994,11 @@ void snd_usb_audioformat_attributes_quirk(struct snd_usb_audio *chip, /* mic works only when ep packet size is set to wMaxPacketSize */ fp->attributes |= UAC_EP_CS_ATTR_FILL_MAX; break; - + case USB_ID(0x3511, 0x2b1e): /* Opencomm2 UC USB Bluetooth dongle */ + /* mic works only when ep pitch control is not set */ + if (stream == SNDRV_PCM_STREAM_CAPTURE) + fp->attributes &= ~UAC_EP_CS_ATTR_PITCH_CONTROL; + break; } } @@ -2173,6 +2177,8 @@ static const struct usb_audio_quirk_flags_table quirk_flags_table[] = { QUIRK_FLAG_FIXED_RATE), DEVICE_FLG(0x0ecb, 0x2069, /* JBL Quantum810 Wireless */ QUIRK_FLAG_FIXED_RATE), + DEVICE_FLG(0x1bcf, 0x2283, /* NexiGo N930AF FHD Webcam */ + QUIRK_FLAG_GET_SAMPLE_RATE), /* Vendor matches */ VENDOR_FLG(0x045e, /* MS Lifecam */ diff --git a/sound/xen/xen_snd_front_cfg.c b/sound/xen/xen_snd_front_cfg.c index 63b0398c3276..55ecf766ca67 100644 --- a/sound/xen/xen_snd_front_cfg.c +++ b/sound/xen/xen_snd_front_cfg.c @@ -483,7 +483,7 @@ int xen_snd_front_cfg_card(struct xen_snd_front_info *front_info, *stream_cnt = 0; num_devices = 0; do { - snprintf(node, sizeof(node), "%d", num_devices); + scnprintf(node, sizeof(node), "%d", num_devices); if (!xenbus_exists(XBT_NIL, xb_dev->nodename, node)) break; diff --git a/tools/arch/x86/include/asm/msr-index.h b/tools/arch/x86/include/asm/msr-index.h index a00a53e15ab7..1d111350197f 100644 --- a/tools/arch/x86/include/asm/msr-index.h +++ b/tools/arch/x86/include/asm/msr-index.h @@ -57,6 +57,7 @@ #define MSR_IA32_PRED_CMD 0x00000049 /* Prediction Command */ #define PRED_CMD_IBPB BIT(0) /* Indirect Branch Prediction Barrier */ +#define PRED_CMD_SBPB BIT(7) /* Selective Branch Prediction Barrier */ #define MSR_PPIN_CTL 0x0000004e #define MSR_PPIN 0x0000004f @@ -155,6 +156,15 @@ * Not susceptible to Post-Barrier * Return Stack Buffer Predictions. */ +#define ARCH_CAP_GDS_CTRL BIT(25) /* + * CPU is vulnerable to Gather + * Data Sampling (GDS) and + * has controls for mitigation. + */ +#define ARCH_CAP_GDS_NO BIT(26) /* + * CPU is not vulnerable to Gather + * Data Sampling (GDS). + */ #define ARCH_CAP_XAPIC_DISABLE BIT(21) /* * IA32_XAPIC_DISABLE_STATUS MSR @@ -178,6 +188,8 @@ #define RNGDS_MITG_DIS BIT(0) /* SRBDS support */ #define RTM_ALLOW BIT(1) /* TSX development mode */ #define FB_CLEAR_DIS BIT(3) /* CPU Fill buffer clear disable */ +#define GDS_MITG_DIS BIT(4) /* Disable GDS mitigation */ +#define GDS_MITG_LOCKED BIT(5) /* GDS mitigation locked */ #define MSR_IA32_SYSENTER_CS 0x00000174 #define MSR_IA32_SYSENTER_ESP 0x00000175 diff --git a/tools/arch/x86/include/uapi/asm/unistd_32.h b/tools/arch/x86/include/uapi/asm/unistd_32.h index 4798f9d18fe8..9de35df1afc3 100644 --- a/tools/arch/x86/include/uapi/asm/unistd_32.h +++ b/tools/arch/x86/include/uapi/asm/unistd_32.h @@ -26,6 +26,6 @@ #ifndef __NR_setns #define __NR_setns 346 #endif -#ifdef __NR_seccomp +#ifndef __NR_seccomp #define __NR_seccomp 354 #endif diff --git a/tools/build/feature/test-llvm.cpp b/tools/build/feature/test-llvm.cpp new file mode 100644 index 000000000000..88a3d1bdd9f6 --- /dev/null +++ b/tools/build/feature/test-llvm.cpp @@ -0,0 +1,14 @@ +// SPDX-License-Identifier: GPL-2.0 +#include "llvm/Support/ManagedStatic.h" +#include "llvm/Support/raw_ostream.h" +#define NUM_VERSION (((LLVM_VERSION_MAJOR) << 16) + (LLVM_VERSION_MINOR << 8) + LLVM_VERSION_PATCH) + +#if NUM_VERSION < 0x030900 +# error "LLVM version too low" +#endif +int main() +{ + llvm::errs() << "Hello World!\n"; + llvm::llvm_shutdown(); + return 0; +} diff --git a/tools/hv/hv_kvp_daemon.c b/tools/hv/hv_kvp_daemon.c index 27f5e7dfc2f7..264eeb9c46a9 100644 --- a/tools/hv/hv_kvp_daemon.c +++ b/tools/hv/hv_kvp_daemon.c @@ -1171,12 +1171,79 @@ static int process_ip_string(FILE *f, char *ip_string, int type) return 0; } +/* + * Only IPv4 subnet strings needs to be converted to plen + * For IPv6 the subnet is already privided in plen format + */ +static int kvp_subnet_to_plen(char *subnet_addr_str) +{ + int plen = 0; + struct in_addr subnet_addr4; + + /* + * Convert subnet address to binary representation + */ + if (inet_pton(AF_INET, subnet_addr_str, &subnet_addr4) == 1) { + uint32_t subnet_mask = ntohl(subnet_addr4.s_addr); + + while (subnet_mask & 0x80000000) { + plen++; + subnet_mask <<= 1; + } + } else { + return -1; + } + + return plen; +} + +static int process_ip_string_nm(FILE *f, char *ip_string, char *subnet, + int is_ipv6) +{ + char addr[INET6_ADDRSTRLEN]; + char subnet_addr[INET6_ADDRSTRLEN]; + int error, i = 0; + int ip_offset = 0, subnet_offset = 0; + int plen; + + memset(addr, 0, sizeof(addr)); + memset(subnet_addr, 0, sizeof(subnet_addr)); + + while (parse_ip_val_buffer(ip_string, &ip_offset, addr, + (MAX_IP_ADDR_SIZE * 2)) && + parse_ip_val_buffer(subnet, + &subnet_offset, + subnet_addr, + (MAX_IP_ADDR_SIZE * + 2))) { + if (!is_ipv6) + plen = kvp_subnet_to_plen((char *)subnet_addr); + else + plen = atoi(subnet_addr); + + if (plen < 0) + return plen; + + error = fprintf(f, "address%d=%s/%d\n", ++i, (char *)addr, + plen); + if (error < 0) + return error; + + memset(addr, 0, sizeof(addr)); + memset(subnet_addr, 0, sizeof(subnet_addr)); + } + + return 0; +} + static int kvp_set_ip_info(char *if_name, struct hv_kvp_ipaddr_value *new_val) { int error = 0; - char if_file[PATH_MAX]; - FILE *file; + char if_filename[PATH_MAX]; + char nm_filename[PATH_MAX]; + FILE *ifcfg_file, *nmfile; char cmd[PATH_MAX]; + int is_ipv6 = 0; char *mac_addr; int str_len; @@ -1197,7 +1264,7 @@ static int kvp_set_ip_info(char *if_name, struct hv_kvp_ipaddr_value *new_val) * in a given distro to configure the interface and so are free * ignore information that may not be relevant. * - * Here is the format of the ip configuration file: + * Here is the ifcfg format of the ip configuration file: * * HWADDR=macaddr * DEVICE=interface name @@ -1220,6 +1287,32 @@ static int kvp_set_ip_info(char *if_name, struct hv_kvp_ipaddr_value *new_val) * tagged as IPV6_DEFAULTGW and IPV6 NETMASK will be tagged as * IPV6NETMASK. * + * Here is the keyfile format of the ip configuration file: + * + * [ethernet] + * mac-address=macaddr + * [connection] + * interface-name=interface name + * + * [ipv4] + * method= (where is "auto" if DHCP is configured + * or "manual" if no boot-time protocol should be used) + * + * address1=ipaddr1/plen + * address2=ipaddr2/plen + * + * gateway=gateway1;gateway2 + * + * dns=dns1;dns2 + * + * [ipv6] + * address1=ipaddr1/plen + * address2=ipaddr2/plen + * + * gateway=gateway1;gateway2 + * + * dns=dns1;dns2 + * * The host can specify multiple ipv4 and ipv6 addresses to be * configured for the interface. Furthermore, the configuration * needs to be persistent. A subsequent GET call on the interface @@ -1227,14 +1320,29 @@ static int kvp_set_ip_info(char *if_name, struct hv_kvp_ipaddr_value *new_val) * call. */ - snprintf(if_file, sizeof(if_file), "%s%s%s", KVP_CONFIG_LOC, - "/ifcfg-", if_name); + /* + * We are populating both ifcfg and nmconnection files + */ + snprintf(if_filename, sizeof(if_filename), "%s%s%s", KVP_CONFIG_LOC, + "/ifcfg-", if_name); - file = fopen(if_file, "w"); + ifcfg_file = fopen(if_filename, "w"); - if (file == NULL) { + if (!ifcfg_file) { syslog(LOG_ERR, "Failed to open config file; error: %d %s", - errno, strerror(errno)); + errno, strerror(errno)); + return HV_E_FAIL; + } + + snprintf(nm_filename, sizeof(nm_filename), "%s%s%s%s", KVP_CONFIG_LOC, + "/", if_name, ".nmconnection"); + + nmfile = fopen(nm_filename, "w"); + + if (!nmfile) { + syslog(LOG_ERR, "Failed to open config file; error: %d %s", + errno, strerror(errno)); + fclose(ifcfg_file); return HV_E_FAIL; } @@ -1248,14 +1356,31 @@ static int kvp_set_ip_info(char *if_name, struct hv_kvp_ipaddr_value *new_val) goto setval_error; } - error = kvp_write_file(file, "HWADDR", "", mac_addr); - free(mac_addr); + error = kvp_write_file(ifcfg_file, "HWADDR", "", mac_addr); + if (error < 0) + goto setmac_error; + + error = kvp_write_file(ifcfg_file, "DEVICE", "", if_name); + if (error < 0) + goto setmac_error; + + error = fprintf(nmfile, "\n[connection]\n"); + if (error < 0) + goto setmac_error; + + error = kvp_write_file(nmfile, "interface-name", "", if_name); if (error) - goto setval_error; + goto setmac_error; - error = kvp_write_file(file, "DEVICE", "", if_name); + error = fprintf(nmfile, "\n[ethernet]\n"); + if (error < 0) + goto setmac_error; + + error = kvp_write_file(nmfile, "mac-address", "", mac_addr); if (error) - goto setval_error; + goto setmac_error; + + free(mac_addr); /* * The dhcp_enabled flag is only for IPv4. In the case the host only @@ -1263,47 +1388,91 @@ static int kvp_set_ip_info(char *if_name, struct hv_kvp_ipaddr_value *new_val) * proceed to parse and pass the IPv6 information to the * disto-specific script hv_set_ifconfig. */ + + /* + * First populate the ifcfg file format + */ if (new_val->dhcp_enabled) { - error = kvp_write_file(file, "BOOTPROTO", "", "dhcp"); + error = kvp_write_file(ifcfg_file, "BOOTPROTO", "", "dhcp"); if (error) goto setval_error; - } else { - error = kvp_write_file(file, "BOOTPROTO", "", "none"); + error = kvp_write_file(ifcfg_file, "BOOTPROTO", "", "none"); if (error) goto setval_error; } - /* - * Write the configuration for ipaddress, netmask, gateway and - * name servers. - */ - - error = process_ip_string(file, (char *)new_val->ip_addr, IPADDR); + error = process_ip_string(ifcfg_file, (char *)new_val->ip_addr, + IPADDR); if (error) goto setval_error; - error = process_ip_string(file, (char *)new_val->sub_net, NETMASK); + error = process_ip_string(ifcfg_file, (char *)new_val->sub_net, + NETMASK); if (error) goto setval_error; - error = process_ip_string(file, (char *)new_val->gate_way, GATEWAY); + error = process_ip_string(ifcfg_file, (char *)new_val->gate_way, + GATEWAY); if (error) goto setval_error; - error = process_ip_string(file, (char *)new_val->dns_addr, DNS); + error = process_ip_string(ifcfg_file, (char *)new_val->dns_addr, DNS); if (error) goto setval_error; - fclose(file); + if (new_val->addr_family == ADDR_FAMILY_IPV6) { + error = fprintf(nmfile, "\n[ipv6]\n"); + if (error < 0) + goto setval_error; + is_ipv6 = 1; + } else { + error = fprintf(nmfile, "\n[ipv4]\n"); + if (error < 0) + goto setval_error; + } + + /* + * Now we populate the keyfile format + */ + + if (new_val->dhcp_enabled) { + error = kvp_write_file(nmfile, "method", "", "auto"); + if (error < 0) + goto setval_error; + } else { + error = kvp_write_file(nmfile, "method", "", "manual"); + if (error < 0) + goto setval_error; + } + + /* + * Write the configuration for ipaddress, netmask, gateway and + * name services + */ + error = process_ip_string_nm(nmfile, (char *)new_val->ip_addr, + (char *)new_val->sub_net, is_ipv6); + if (error < 0) + goto setval_error; + + error = fprintf(nmfile, "gateway=%s\n", (char *)new_val->gate_way); + if (error < 0) + goto setval_error; + + error = fprintf(nmfile, "dns=%s\n", (char *)new_val->dns_addr); + if (error < 0) + goto setval_error; + + fclose(nmfile); + fclose(ifcfg_file); /* * Now that we have populated the configuration file, * invoke the external script to do its magic. */ - str_len = snprintf(cmd, sizeof(cmd), KVP_SCRIPTS_PATH "%s %s", - "hv_set_ifconfig", if_file); + str_len = snprintf(cmd, sizeof(cmd), KVP_SCRIPTS_PATH "%s %s %s", + "hv_set_ifconfig", if_filename, nm_filename); /* * This is a little overcautious, but it's necessary to suppress some * false warnings from gcc 8.0.1. @@ -1316,14 +1485,16 @@ static int kvp_set_ip_info(char *if_name, struct hv_kvp_ipaddr_value *new_val) if (system(cmd)) { syslog(LOG_ERR, "Failed to execute cmd '%s'; error: %d %s", - cmd, errno, strerror(errno)); + cmd, errno, strerror(errno)); return HV_E_FAIL; } return 0; - +setmac_error: + free(mac_addr); setval_error: syslog(LOG_ERR, "Failed to write config file"); - fclose(file); + fclose(ifcfg_file); + fclose(nmfile); return error; } diff --git a/tools/hv/hv_set_ifconfig.sh b/tools/hv/hv_set_ifconfig.sh index d10fe35b7f25..ae5a7a8249a2 100755 --- a/tools/hv/hv_set_ifconfig.sh +++ b/tools/hv/hv_set_ifconfig.sh @@ -18,12 +18,12 @@ # # This example script is based on a RHEL environment. # -# Here is the format of the ip configuration file: +# Here is the ifcfg format of the ip configuration file: # # HWADDR=macaddr # DEVICE=interface name # BOOTPROTO= (where is "dhcp" if DHCP is configured -# or "none" if no boot-time protocol should be used) +# or "none" if no boot-time protocol should be used) # # IPADDR0=ipaddr1 # IPADDR1=ipaddr2 @@ -41,6 +41,32 @@ # tagged as IPV6_DEFAULTGW and IPV6 NETMASK will be tagged as # IPV6NETMASK. # +# Here is the keyfile format of the ip configuration file: +# +# [ethernet] +# mac-address=macaddr +# [connection] +# interface-name=interface name +# +# [ipv4] +# method= (where is "auto" if DHCP is configured +# or "manual" if no boot-time protocol should be used) +# +# address1=ipaddr1/plen +# address=ipaddr2/plen +# +# gateway=gateway1;gateway2 +# +# dns=dns1; +# +# [ipv6] +# address1=ipaddr1/plen +# address2=ipaddr1/plen +# +# gateway=gateway1;gateway2 +# +# dns=dns1;dns2 +# # The host can specify multiple ipv4 and ipv6 addresses to be # configured for the interface. Furthermore, the configuration # needs to be persistent. A subsequent GET call on the interface @@ -48,18 +74,19 @@ # call. # - - echo "IPV6INIT=yes" >> $1 echo "NM_CONTROLLED=no" >> $1 echo "PEERDNS=yes" >> $1 echo "ONBOOT=yes" >> $1 - cp $1 /etc/sysconfig/network-scripts/ +chmod 600 $2 +interface=$(echo $2 | awk -F - '{ print $2 }') +filename="${2##*/}" + +sed '/\[connection\]/a autoconnect=true' $2 > /etc/NetworkManager/system-connections/${filename} -interface=$(echo $1 | awk -F - '{ print $2 }') /sbin/ifdown $interface 2>/dev/null /sbin/ifup $interface 2>/dev/null diff --git a/tools/include/linux/btf_ids.h b/tools/include/linux/btf_ids.h index 71e54b1e3796..2f882d5cb30f 100644 --- a/tools/include/linux/btf_ids.h +++ b/tools/include/linux/btf_ids.h @@ -38,7 +38,7 @@ asm( \ ____BTF_ID(symbol) #define __ID(prefix) \ - __PASTE(prefix, __COUNTER__) + __PASTE(__PASTE(prefix, __COUNTER__), __LINE__) /* * The BTF_ID defines unique symbol for each ID pointing diff --git a/tools/include/linux/mm.h b/tools/include/linux/mm.h index a03d9bba5151..f3c82ab5b14c 100644 --- a/tools/include/linux/mm.h +++ b/tools/include/linux/mm.h @@ -11,8 +11,6 @@ #define PHYS_ADDR_MAX (~(phys_addr_t)0) -#define __ALIGN_KERNEL(x, a) __ALIGN_KERNEL_MASK(x, (typeof(x))(a) - 1) -#define __ALIGN_KERNEL_MASK(x, mask) (((x) + (mask)) & ~(mask)) #define ALIGN(x, a) __ALIGN_KERNEL((x), (a)) #define ALIGN_DOWN(x, a) __ALIGN_KERNEL((x) - ((a) - 1), (a)) @@ -29,7 +27,7 @@ static inline void *phys_to_virt(unsigned long address) return __va(address); } -void reserve_bootmem_region(phys_addr_t start, phys_addr_t end); +void reserve_bootmem_region(phys_addr_t start, phys_addr_t end, int nid); static inline void totalram_pages_inc(void) { diff --git a/tools/include/linux/seq_file.h b/tools/include/linux/seq_file.h index 102fd9217f1f..f6bc226af0c1 100644 --- a/tools/include/linux/seq_file.h +++ b/tools/include/linux/seq_file.h @@ -1,4 +1,6 @@ #ifndef _TOOLS_INCLUDE_LINUX_SEQ_FILE_H #define _TOOLS_INCLUDE_LINUX_SEQ_FILE_H +struct seq_file; + #endif /* _TOOLS_INCLUDE_LINUX_SEQ_FILE_H */ diff --git a/tools/include/uapi/asm-generic/unistd.h b/tools/include/uapi/asm-generic/unistd.h index fd6c1cb585db..abe087c53b4b 100644 --- a/tools/include/uapi/asm-generic/unistd.h +++ b/tools/include/uapi/asm-generic/unistd.h @@ -820,8 +820,11 @@ __SYSCALL(__NR_set_mempolicy_home_node, sys_set_mempolicy_home_node) #define __NR_cachestat 451 __SYSCALL(__NR_cachestat, sys_cachestat) +#define __NR_fchmodat2 452 +__SYSCALL(__NR_fchmodat2, sys_fchmodat2) + #undef __NR_syscalls -#define __NR_syscalls 452 +#define __NR_syscalls 453 /* * 32 bit systems traditionally used different diff --git a/tools/include/uapi/drm/drm.h b/tools/include/uapi/drm/drm.h index a87bbbbca2d4..794c1d857677 100644 --- a/tools/include/uapi/drm/drm.h +++ b/tools/include/uapi/drm/drm.h @@ -673,8 +673,11 @@ struct drm_gem_open { * Bitfield of supported PRIME sharing capabilities. See &DRM_PRIME_CAP_IMPORT * and &DRM_PRIME_CAP_EXPORT. * - * PRIME buffers are exposed as dma-buf file descriptors. See - * Documentation/gpu/drm-mm.rst, section "PRIME Buffer Sharing". + * Starting from kernel version 6.6, both &DRM_PRIME_CAP_IMPORT and + * &DRM_PRIME_CAP_EXPORT are always advertised. + * + * PRIME buffers are exposed as dma-buf file descriptors. + * See :ref:`prime_buffer_sharing`. */ #define DRM_CAP_PRIME 0x5 /** @@ -682,6 +685,8 @@ struct drm_gem_open { * * If this bit is set in &DRM_CAP_PRIME, the driver supports importing PRIME * buffers via the &DRM_IOCTL_PRIME_FD_TO_HANDLE ioctl. + * + * Starting from kernel version 6.6, this bit is always set in &DRM_CAP_PRIME. */ #define DRM_PRIME_CAP_IMPORT 0x1 /** @@ -689,6 +694,8 @@ struct drm_gem_open { * * If this bit is set in &DRM_CAP_PRIME, the driver supports exporting PRIME * buffers via the &DRM_IOCTL_PRIME_HANDLE_TO_FD ioctl. + * + * Starting from kernel version 6.6, this bit is always set in &DRM_CAP_PRIME. */ #define DRM_PRIME_CAP_EXPORT 0x2 /** @@ -756,15 +763,14 @@ struct drm_gem_open { /** * DRM_CAP_SYNCOBJ * - * If set to 1, the driver supports sync objects. See - * Documentation/gpu/drm-mm.rst, section "DRM Sync Objects". + * If set to 1, the driver supports sync objects. See :ref:`drm_sync_objects`. */ #define DRM_CAP_SYNCOBJ 0x13 /** * DRM_CAP_SYNCOBJ_TIMELINE * * If set to 1, the driver supports timeline operations on sync objects. See - * Documentation/gpu/drm-mm.rst, section "DRM Sync Objects". + * :ref:`drm_sync_objects`. */ #define DRM_CAP_SYNCOBJ_TIMELINE 0x14 @@ -909,6 +915,27 @@ struct drm_syncobj_timeline_wait { __u32 pad; }; +/** + * struct drm_syncobj_eventfd + * @handle: syncobj handle. + * @flags: Zero to wait for the point to be signalled, or + * &DRM_SYNCOBJ_WAIT_FLAGS_WAIT_AVAILABLE to wait for a fence to be + * available for the point. + * @point: syncobj timeline point (set to zero for binary syncobjs). + * @fd: Existing eventfd to sent events to. + * @pad: Must be zero. + * + * Register an eventfd to be signalled by a syncobj. The eventfd counter will + * be incremented by one. + */ +struct drm_syncobj_eventfd { + __u32 handle; + __u32 flags; + __u64 point; + __s32 fd; + __u32 pad; +}; + struct drm_syncobj_array { __u64 handles; @@ -1169,6 +1196,8 @@ extern "C" { */ #define DRM_IOCTL_MODE_GETFB2 DRM_IOWR(0xCE, struct drm_mode_fb_cmd2) +#define DRM_IOCTL_SYNCOBJ_EVENTFD DRM_IOWR(0xCF, struct drm_syncobj_eventfd) + /* * Device specific ioctls should only be in their respective headers * The device specific ioctl range is from 0x40 to 0x9f. @@ -1180,25 +1209,50 @@ extern "C" { #define DRM_COMMAND_BASE 0x40 #define DRM_COMMAND_END 0xA0 -/* - * Header for events written back to userspace on the drm fd. The - * type defines the type of event, the length specifies the total - * length of the event (including the header), and user_data is - * typically a 64 bit value passed with the ioctl that triggered the - * event. A read on the drm fd will always only return complete - * events, that is, if for example the read buffer is 100 bytes, and - * there are two 64 byte events pending, only one will be returned. +/** + * struct drm_event - Header for DRM events + * @type: event type. + * @length: total number of payload bytes (including header). * - * Event types 0 - 0x7fffffff are generic drm events, 0x80000000 and - * up are chipset specific. + * This struct is a header for events written back to user-space on the DRM FD. + * A read on the DRM FD will always only return complete events: e.g. if the + * read buffer is 100 bytes large and there are two 64 byte events pending, + * only one will be returned. + * + * Event types 0 - 0x7fffffff are generic DRM events, 0x80000000 and + * up are chipset specific. Generic DRM events include &DRM_EVENT_VBLANK, + * &DRM_EVENT_FLIP_COMPLETE and &DRM_EVENT_CRTC_SEQUENCE. */ struct drm_event { __u32 type; __u32 length; }; +/** + * DRM_EVENT_VBLANK - vertical blanking event + * + * This event is sent in response to &DRM_IOCTL_WAIT_VBLANK with the + * &_DRM_VBLANK_EVENT flag set. + * + * The event payload is a struct drm_event_vblank. + */ #define DRM_EVENT_VBLANK 0x01 +/** + * DRM_EVENT_FLIP_COMPLETE - page-flip completion event + * + * This event is sent in response to an atomic commit or legacy page-flip with + * the &DRM_MODE_PAGE_FLIP_EVENT flag set. + * + * The event payload is a struct drm_event_vblank. + */ #define DRM_EVENT_FLIP_COMPLETE 0x02 +/** + * DRM_EVENT_CRTC_SEQUENCE - CRTC sequence event + * + * This event is sent in response to &DRM_IOCTL_CRTC_QUEUE_SEQUENCE. + * + * The event payload is a struct drm_event_crtc_sequence. + */ #define DRM_EVENT_CRTC_SEQUENCE 0x03 struct drm_event_vblank { diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index 8790b3962e4b..0448700890f7 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -1962,7 +1962,9 @@ union bpf_attr { * performed again, if the helper is used in combination with * direct packet access. * Return - * 0 on success, or a negative error in case of failure. + * 0 on success, or a negative error in case of failure. Positive + * error indicates a potential drop or congestion in the target + * device. The particular positive error codes are not defined. * * u64 bpf_get_current_pid_tgid(void) * Description diff --git a/tools/include/uapi/linux/seccomp.h b/tools/include/uapi/linux/seccomp.h new file mode 100644 index 000000000000..dbfc9b37fcae --- /dev/null +++ b/tools/include/uapi/linux/seccomp.h @@ -0,0 +1,157 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +#ifndef _UAPI_LINUX_SECCOMP_H +#define _UAPI_LINUX_SECCOMP_H + +#include +#include + + +/* Valid values for seccomp.mode and prctl(PR_SET_SECCOMP, ) */ +#define SECCOMP_MODE_DISABLED 0 /* seccomp is not in use. */ +#define SECCOMP_MODE_STRICT 1 /* uses hard-coded filter. */ +#define SECCOMP_MODE_FILTER 2 /* uses user-supplied filter. */ + +/* Valid operations for seccomp syscall. */ +#define SECCOMP_SET_MODE_STRICT 0 +#define SECCOMP_SET_MODE_FILTER 1 +#define SECCOMP_GET_ACTION_AVAIL 2 +#define SECCOMP_GET_NOTIF_SIZES 3 + +/* Valid flags for SECCOMP_SET_MODE_FILTER */ +#define SECCOMP_FILTER_FLAG_TSYNC (1UL << 0) +#define SECCOMP_FILTER_FLAG_LOG (1UL << 1) +#define SECCOMP_FILTER_FLAG_SPEC_ALLOW (1UL << 2) +#define SECCOMP_FILTER_FLAG_NEW_LISTENER (1UL << 3) +#define SECCOMP_FILTER_FLAG_TSYNC_ESRCH (1UL << 4) +/* Received notifications wait in killable state (only respond to fatal signals) */ +#define SECCOMP_FILTER_FLAG_WAIT_KILLABLE_RECV (1UL << 5) + +/* + * All BPF programs must return a 32-bit value. + * The bottom 16-bits are for optional return data. + * The upper 16-bits are ordered from least permissive values to most, + * as a signed value (so 0x8000000 is negative). + * + * The ordering ensures that a min_t() over composed return values always + * selects the least permissive choice. + */ +#define SECCOMP_RET_KILL_PROCESS 0x80000000U /* kill the process */ +#define SECCOMP_RET_KILL_THREAD 0x00000000U /* kill the thread */ +#define SECCOMP_RET_KILL SECCOMP_RET_KILL_THREAD +#define SECCOMP_RET_TRAP 0x00030000U /* disallow and force a SIGSYS */ +#define SECCOMP_RET_ERRNO 0x00050000U /* returns an errno */ +#define SECCOMP_RET_USER_NOTIF 0x7fc00000U /* notifies userspace */ +#define SECCOMP_RET_TRACE 0x7ff00000U /* pass to a tracer or disallow */ +#define SECCOMP_RET_LOG 0x7ffc0000U /* allow after logging */ +#define SECCOMP_RET_ALLOW 0x7fff0000U /* allow */ + +/* Masks for the return value sections. */ +#define SECCOMP_RET_ACTION_FULL 0xffff0000U +#define SECCOMP_RET_ACTION 0x7fff0000U +#define SECCOMP_RET_DATA 0x0000ffffU + +/** + * struct seccomp_data - the format the BPF program executes over. + * @nr: the system call number + * @arch: indicates system call convention as an AUDIT_ARCH_* value + * as defined in . + * @instruction_pointer: at the time of the system call. + * @args: up to 6 system call arguments always stored as 64-bit values + * regardless of the architecture. + */ +struct seccomp_data { + int nr; + __u32 arch; + __u64 instruction_pointer; + __u64 args[6]; +}; + +struct seccomp_notif_sizes { + __u16 seccomp_notif; + __u16 seccomp_notif_resp; + __u16 seccomp_data; +}; + +struct seccomp_notif { + __u64 id; + __u32 pid; + __u32 flags; + struct seccomp_data data; +}; + +/* + * Valid flags for struct seccomp_notif_resp + * + * Note, the SECCOMP_USER_NOTIF_FLAG_CONTINUE flag must be used with caution! + * If set by the process supervising the syscalls of another process the + * syscall will continue. This is problematic because of an inherent TOCTOU. + * An attacker can exploit the time while the supervised process is waiting on + * a response from the supervising process to rewrite syscall arguments which + * are passed as pointers of the intercepted syscall. + * It should be absolutely clear that this means that the seccomp notifier + * _cannot_ be used to implement a security policy! It should only ever be used + * in scenarios where a more privileged process supervises the syscalls of a + * lesser privileged process to get around kernel-enforced security + * restrictions when the privileged process deems this safe. In other words, + * in order to continue a syscall the supervising process should be sure that + * another security mechanism or the kernel itself will sufficiently block + * syscalls if arguments are rewritten to something unsafe. + * + * Similar precautions should be applied when stacking SECCOMP_RET_USER_NOTIF + * or SECCOMP_RET_TRACE. For SECCOMP_RET_USER_NOTIF filters acting on the + * same syscall, the most recently added filter takes precedence. This means + * that the new SECCOMP_RET_USER_NOTIF filter can override any + * SECCOMP_IOCTL_NOTIF_SEND from earlier filters, essentially allowing all + * such filtered syscalls to be executed by sending the response + * SECCOMP_USER_NOTIF_FLAG_CONTINUE. Note that SECCOMP_RET_TRACE can equally + * be overriden by SECCOMP_USER_NOTIF_FLAG_CONTINUE. + */ +#define SECCOMP_USER_NOTIF_FLAG_CONTINUE (1UL << 0) + +struct seccomp_notif_resp { + __u64 id; + __s64 val; + __s32 error; + __u32 flags; +}; + +#define SECCOMP_USER_NOTIF_FD_SYNC_WAKE_UP (1UL << 0) + +/* valid flags for seccomp_notif_addfd */ +#define SECCOMP_ADDFD_FLAG_SETFD (1UL << 0) /* Specify remote fd */ +#define SECCOMP_ADDFD_FLAG_SEND (1UL << 1) /* Addfd and return it, atomically */ + +/** + * struct seccomp_notif_addfd + * @id: The ID of the seccomp notification + * @flags: SECCOMP_ADDFD_FLAG_* + * @srcfd: The local fd number + * @newfd: Optional remote FD number if SETFD option is set, otherwise 0. + * @newfd_flags: The O_* flags the remote FD should have applied + */ +struct seccomp_notif_addfd { + __u64 id; + __u32 flags; + __u32 srcfd; + __u32 newfd; + __u32 newfd_flags; +}; + +#define SECCOMP_IOC_MAGIC '!' +#define SECCOMP_IO(nr) _IO(SECCOMP_IOC_MAGIC, nr) +#define SECCOMP_IOR(nr, type) _IOR(SECCOMP_IOC_MAGIC, nr, type) +#define SECCOMP_IOW(nr, type) _IOW(SECCOMP_IOC_MAGIC, nr, type) +#define SECCOMP_IOWR(nr, type) _IOWR(SECCOMP_IOC_MAGIC, nr, type) + +/* Flags for seccomp notification fd ioctl. */ +#define SECCOMP_IOCTL_NOTIF_RECV SECCOMP_IOWR(0, struct seccomp_notif) +#define SECCOMP_IOCTL_NOTIF_SEND SECCOMP_IOWR(1, \ + struct seccomp_notif_resp) +#define SECCOMP_IOCTL_NOTIF_ID_VALID SECCOMP_IOW(2, __u64) +/* On success, the return value is the remote process's added fd number */ +#define SECCOMP_IOCTL_NOTIF_ADDFD SECCOMP_IOW(3, \ + struct seccomp_notif_addfd) + +#define SECCOMP_IOCTL_NOTIF_SET_FLAGS SECCOMP_IOW(4, __u64) + +#endif /* _UAPI_LINUX_SECCOMP_H */ diff --git a/tools/net/ynl/generated/devlink-user.c b/tools/net/ynl/generated/devlink-user.c index 3a8d8499fab6..2cb2518500cb 100644 --- a/tools/net/ynl/generated/devlink-user.c +++ b/tools/net/ynl/generated/devlink-user.c @@ -16,19 +16,19 @@ static const char * const devlink_op_strmap[] = { [3] = "get", [7] = "port-get", - [DEVLINK_CMD_SB_GET] = "sb-get", - [DEVLINK_CMD_SB_POOL_GET] = "sb-pool-get", - [DEVLINK_CMD_SB_PORT_POOL_GET] = "sb-port-pool-get", - [DEVLINK_CMD_SB_TC_POOL_BIND_GET] = "sb-tc-pool-bind-get", + [13] = "sb-get", + [17] = "sb-pool-get", + [21] = "sb-port-pool-get", + [25] = "sb-tc-pool-bind-get", [DEVLINK_CMD_PARAM_GET] = "param-get", [DEVLINK_CMD_REGION_GET] = "region-get", [DEVLINK_CMD_INFO_GET] = "info-get", [DEVLINK_CMD_HEALTH_REPORTER_GET] = "health-reporter-get", - [DEVLINK_CMD_TRAP_GET] = "trap-get", - [DEVLINK_CMD_TRAP_GROUP_GET] = "trap-group-get", - [DEVLINK_CMD_TRAP_POLICER_GET] = "trap-policer-get", - [DEVLINK_CMD_RATE_GET] = "rate-get", - [DEVLINK_CMD_LINECARD_GET] = "linecard-get", + [63] = "trap-get", + [67] = "trap-group-get", + [71] = "trap-policer-get", + [76] = "rate-get", + [80] = "linecard-get", [DEVLINK_CMD_SELFTESTS_GET] = "selftests-get", }; @@ -838,7 +838,7 @@ devlink_sb_get(struct ynl_sock *ys, struct devlink_sb_get_req *req) rsp = calloc(1, sizeof(*rsp)); yrs.yarg.data = rsp; yrs.cb = devlink_sb_get_rsp_parse; - yrs.rsp_cmd = DEVLINK_CMD_SB_GET; + yrs.rsp_cmd = 13; err = ynl_exec(ys, nlh, &yrs); if (err < 0) @@ -876,7 +876,7 @@ devlink_sb_get_dump(struct ynl_sock *ys, struct devlink_sb_get_req_dump *req) yds.ys = ys; yds.alloc_sz = sizeof(struct devlink_sb_get_list); yds.cb = devlink_sb_get_rsp_parse; - yds.rsp_cmd = DEVLINK_CMD_SB_GET; + yds.rsp_cmd = 13; yds.rsp_policy = &devlink_nest; nlh = ynl_gemsg_start_dump(ys, ys->family_id, DEVLINK_CMD_SB_GET, 1); @@ -987,7 +987,7 @@ devlink_sb_pool_get(struct ynl_sock *ys, struct devlink_sb_pool_get_req *req) rsp = calloc(1, sizeof(*rsp)); yrs.yarg.data = rsp; yrs.cb = devlink_sb_pool_get_rsp_parse; - yrs.rsp_cmd = DEVLINK_CMD_SB_POOL_GET; + yrs.rsp_cmd = 17; err = ynl_exec(ys, nlh, &yrs); if (err < 0) @@ -1026,7 +1026,7 @@ devlink_sb_pool_get_dump(struct ynl_sock *ys, yds.ys = ys; yds.alloc_sz = sizeof(struct devlink_sb_pool_get_list); yds.cb = devlink_sb_pool_get_rsp_parse; - yds.rsp_cmd = DEVLINK_CMD_SB_POOL_GET; + yds.rsp_cmd = 17; yds.rsp_policy = &devlink_nest; nlh = ynl_gemsg_start_dump(ys, ys->family_id, DEVLINK_CMD_SB_POOL_GET, 1); @@ -1147,7 +1147,7 @@ devlink_sb_port_pool_get(struct ynl_sock *ys, rsp = calloc(1, sizeof(*rsp)); yrs.yarg.data = rsp; yrs.cb = devlink_sb_port_pool_get_rsp_parse; - yrs.rsp_cmd = DEVLINK_CMD_SB_PORT_POOL_GET; + yrs.rsp_cmd = 21; err = ynl_exec(ys, nlh, &yrs); if (err < 0) @@ -1187,7 +1187,7 @@ devlink_sb_port_pool_get_dump(struct ynl_sock *ys, yds.ys = ys; yds.alloc_sz = sizeof(struct devlink_sb_port_pool_get_list); yds.cb = devlink_sb_port_pool_get_rsp_parse; - yds.rsp_cmd = DEVLINK_CMD_SB_PORT_POOL_GET; + yds.rsp_cmd = 21; yds.rsp_policy = &devlink_nest; nlh = ynl_gemsg_start_dump(ys, ys->family_id, DEVLINK_CMD_SB_PORT_POOL_GET, 1); @@ -1316,7 +1316,7 @@ devlink_sb_tc_pool_bind_get(struct ynl_sock *ys, rsp = calloc(1, sizeof(*rsp)); yrs.yarg.data = rsp; yrs.cb = devlink_sb_tc_pool_bind_get_rsp_parse; - yrs.rsp_cmd = DEVLINK_CMD_SB_TC_POOL_BIND_GET; + yrs.rsp_cmd = 25; err = ynl_exec(ys, nlh, &yrs); if (err < 0) @@ -1356,7 +1356,7 @@ devlink_sb_tc_pool_bind_get_dump(struct ynl_sock *ys, yds.ys = ys; yds.alloc_sz = sizeof(struct devlink_sb_tc_pool_bind_get_list); yds.cb = devlink_sb_tc_pool_bind_get_rsp_parse; - yds.rsp_cmd = DEVLINK_CMD_SB_TC_POOL_BIND_GET; + yds.rsp_cmd = 25; yds.rsp_policy = &devlink_nest; nlh = ynl_gemsg_start_dump(ys, ys->family_id, DEVLINK_CMD_SB_TC_POOL_BIND_GET, 1); @@ -2183,7 +2183,7 @@ devlink_trap_get(struct ynl_sock *ys, struct devlink_trap_get_req *req) rsp = calloc(1, sizeof(*rsp)); yrs.yarg.data = rsp; yrs.cb = devlink_trap_get_rsp_parse; - yrs.rsp_cmd = DEVLINK_CMD_TRAP_GET; + yrs.rsp_cmd = 63; err = ynl_exec(ys, nlh, &yrs); if (err < 0) @@ -2223,7 +2223,7 @@ devlink_trap_get_dump(struct ynl_sock *ys, yds.ys = ys; yds.alloc_sz = sizeof(struct devlink_trap_get_list); yds.cb = devlink_trap_get_rsp_parse; - yds.rsp_cmd = DEVLINK_CMD_TRAP_GET; + yds.rsp_cmd = 63; yds.rsp_policy = &devlink_nest; nlh = ynl_gemsg_start_dump(ys, ys->family_id, DEVLINK_CMD_TRAP_GET, 1); @@ -2336,7 +2336,7 @@ devlink_trap_group_get(struct ynl_sock *ys, rsp = calloc(1, sizeof(*rsp)); yrs.yarg.data = rsp; yrs.cb = devlink_trap_group_get_rsp_parse; - yrs.rsp_cmd = DEVLINK_CMD_TRAP_GROUP_GET; + yrs.rsp_cmd = 67; err = ynl_exec(ys, nlh, &yrs); if (err < 0) @@ -2376,7 +2376,7 @@ devlink_trap_group_get_dump(struct ynl_sock *ys, yds.ys = ys; yds.alloc_sz = sizeof(struct devlink_trap_group_get_list); yds.cb = devlink_trap_group_get_rsp_parse; - yds.rsp_cmd = DEVLINK_CMD_TRAP_GROUP_GET; + yds.rsp_cmd = 67; yds.rsp_policy = &devlink_nest; nlh = ynl_gemsg_start_dump(ys, ys->family_id, DEVLINK_CMD_TRAP_GROUP_GET, 1); @@ -2483,7 +2483,7 @@ devlink_trap_policer_get(struct ynl_sock *ys, rsp = calloc(1, sizeof(*rsp)); yrs.yarg.data = rsp; yrs.cb = devlink_trap_policer_get_rsp_parse; - yrs.rsp_cmd = DEVLINK_CMD_TRAP_POLICER_GET; + yrs.rsp_cmd = 71; err = ynl_exec(ys, nlh, &yrs); if (err < 0) @@ -2523,7 +2523,7 @@ devlink_trap_policer_get_dump(struct ynl_sock *ys, yds.ys = ys; yds.alloc_sz = sizeof(struct devlink_trap_policer_get_list); yds.cb = devlink_trap_policer_get_rsp_parse; - yds.rsp_cmd = DEVLINK_CMD_TRAP_POLICER_GET; + yds.rsp_cmd = 71; yds.rsp_policy = &devlink_nest; nlh = ynl_gemsg_start_dump(ys, ys->family_id, DEVLINK_CMD_TRAP_POLICER_GET, 1); @@ -2642,7 +2642,7 @@ devlink_rate_get(struct ynl_sock *ys, struct devlink_rate_get_req *req) rsp = calloc(1, sizeof(*rsp)); yrs.yarg.data = rsp; yrs.cb = devlink_rate_get_rsp_parse; - yrs.rsp_cmd = DEVLINK_CMD_RATE_GET; + yrs.rsp_cmd = 76; err = ynl_exec(ys, nlh, &yrs); if (err < 0) @@ -2682,7 +2682,7 @@ devlink_rate_get_dump(struct ynl_sock *ys, yds.ys = ys; yds.alloc_sz = sizeof(struct devlink_rate_get_list); yds.cb = devlink_rate_get_rsp_parse; - yds.rsp_cmd = DEVLINK_CMD_RATE_GET; + yds.rsp_cmd = 76; yds.rsp_policy = &devlink_nest; nlh = ynl_gemsg_start_dump(ys, ys->family_id, DEVLINK_CMD_RATE_GET, 1); @@ -2786,7 +2786,7 @@ devlink_linecard_get(struct ynl_sock *ys, struct devlink_linecard_get_req *req) rsp = calloc(1, sizeof(*rsp)); yrs.yarg.data = rsp; yrs.cb = devlink_linecard_get_rsp_parse; - yrs.rsp_cmd = DEVLINK_CMD_LINECARD_GET; + yrs.rsp_cmd = 80; err = ynl_exec(ys, nlh, &yrs); if (err < 0) @@ -2825,7 +2825,7 @@ devlink_linecard_get_dump(struct ynl_sock *ys, yds.ys = ys; yds.alloc_sz = sizeof(struct devlink_linecard_get_list); yds.cb = devlink_linecard_get_rsp_parse; - yds.rsp_cmd = DEVLINK_CMD_LINECARD_GET; + yds.rsp_cmd = 80; yds.rsp_policy = &devlink_nest; nlh = ynl_gemsg_start_dump(ys, ys->family_id, DEVLINK_CMD_LINECARD_GET, 1); diff --git a/tools/objtool/check.c b/tools/objtool/check.c index 1384090530db..e308d1ba664e 100644 --- a/tools/objtool/check.c +++ b/tools/objtool/check.c @@ -4333,7 +4333,8 @@ static int validate_ibt_insn(struct objtool_file *file, struct instruction *insn continue; } - if (insn_func(dest) && insn_func(dest) == insn_func(insn)) { + if (insn_func(dest) && insn_func(insn) && + insn_func(dest)->pfunc == insn_func(insn)->pfunc) { /* * Anything from->to self is either _THIS_IP_ or * IRET-to-self. diff --git a/tools/perf/arch/mips/entry/syscalls/syscall_n64.tbl b/tools/perf/arch/mips/entry/syscalls/syscall_n64.tbl index cfda2511badf..cb5e757f6621 100644 --- a/tools/perf/arch/mips/entry/syscalls/syscall_n64.tbl +++ b/tools/perf/arch/mips/entry/syscalls/syscall_n64.tbl @@ -366,3 +366,4 @@ 449 n64 futex_waitv sys_futex_waitv 450 common set_mempolicy_home_node sys_set_mempolicy_home_node 451 n64 cachestat sys_cachestat +452 n64 fchmodat2 sys_fchmodat2 diff --git a/tools/perf/arch/powerpc/entry/syscalls/syscall.tbl b/tools/perf/arch/powerpc/entry/syscalls/syscall.tbl index 8c0b08b7a80e..20e50586e8a2 100644 --- a/tools/perf/arch/powerpc/entry/syscalls/syscall.tbl +++ b/tools/perf/arch/powerpc/entry/syscalls/syscall.tbl @@ -538,3 +538,4 @@ 449 common futex_waitv sys_futex_waitv 450 nospu set_mempolicy_home_node sys_set_mempolicy_home_node 451 common cachestat sys_cachestat +452 common fchmodat2 sys_fchmodat2 diff --git a/tools/perf/arch/s390/entry/syscalls/syscall.tbl b/tools/perf/arch/s390/entry/syscalls/syscall.tbl index a6935af2235c..0122cc156952 100644 --- a/tools/perf/arch/s390/entry/syscalls/syscall.tbl +++ b/tools/perf/arch/s390/entry/syscalls/syscall.tbl @@ -454,3 +454,4 @@ 449 common futex_waitv sys_futex_waitv sys_futex_waitv 450 common set_mempolicy_home_node sys_set_mempolicy_home_node sys_set_mempolicy_home_node 451 common cachestat sys_cachestat sys_cachestat +452 common fchmodat2 sys_fchmodat2 sys_fchmodat2 diff --git a/tools/perf/arch/x86/entry/syscalls/syscall_64.tbl b/tools/perf/arch/x86/entry/syscalls/syscall_64.tbl index 227538b0ce80..1d6eee30eceb 100644 --- a/tools/perf/arch/x86/entry/syscalls/syscall_64.tbl +++ b/tools/perf/arch/x86/entry/syscalls/syscall_64.tbl @@ -373,6 +373,8 @@ 449 common futex_waitv sys_futex_waitv 450 common set_mempolicy_home_node sys_set_mempolicy_home_node 451 common cachestat sys_cachestat +452 common fchmodat2 sys_fchmodat2 +453 64 map_shadow_stack sys_map_shadow_stack # # Due to a historical design error, certain syscalls are numbered differently diff --git a/tools/perf/bench/sched-seccomp-notify.c b/tools/perf/bench/sched-seccomp-notify.c index b04ebcde4036..a01c40131493 100644 --- a/tools/perf/bench/sched-seccomp-notify.c +++ b/tools/perf/bench/sched-seccomp-notify.c @@ -9,7 +9,7 @@ #include #include #include -#include +#include #include #include diff --git a/tools/perf/check-headers.sh b/tools/perf/check-headers.sh index 4314c9197850..e21caadda7c1 100755 --- a/tools/perf/check-headers.sh +++ b/tools/perf/check-headers.sh @@ -21,6 +21,7 @@ FILES=( "include/uapi/linux/perf_event.h" "include/uapi/linux/prctl.h" "include/uapi/linux/sched.h" + "include/uapi/linux/seccomp.h" "include/uapi/linux/stat.h" "include/uapi/linux/usbdevice_fs.h" "include/uapi/linux/vhost.h" diff --git a/tools/perf/dlfilters/dlfilter-test-api-v0.c b/tools/perf/dlfilters/dlfilter-test-api-v0.c index 72f263d49121..4083b1abeaab 100644 --- a/tools/perf/dlfilters/dlfilter-test-api-v0.c +++ b/tools/perf/dlfilters/dlfilter-test-api-v0.c @@ -289,6 +289,15 @@ static int check_attr(void *ctx) return 0; } +static int check_object_code(void *ctx, const struct perf_dlfilter_sample *sample) +{ + __u8 buf[15]; + + CHECK(perf_dlfilter_fns.object_code(ctx, sample->ip, buf, sizeof(buf)) > 0); + + return 0; +} + static int do_checks(void *data, const struct perf_dlfilter_sample *sample, void *ctx, bool early) { struct filter_data *d = data; @@ -314,7 +323,8 @@ static int do_checks(void *data, const struct perf_dlfilter_sample *sample, void if (early && !d->do_early) return 0; - if (check_al(ctx) || check_addr_al(ctx) || check_address_al(ctx, sample)) + if (check_al(ctx) || check_addr_al(ctx) || check_address_al(ctx, sample) || + check_object_code(ctx, sample)) return -1; if (early) diff --git a/tools/perf/dlfilters/dlfilter-test-api-v2.c b/tools/perf/dlfilters/dlfilter-test-api-v2.c index 38e593d92920..32ff619e881c 100644 --- a/tools/perf/dlfilters/dlfilter-test-api-v2.c +++ b/tools/perf/dlfilters/dlfilter-test-api-v2.c @@ -308,6 +308,15 @@ static int check_attr(void *ctx) return 0; } +static int check_object_code(void *ctx, const struct perf_dlfilter_sample *sample) +{ + __u8 buf[15]; + + CHECK(perf_dlfilter_fns.object_code(ctx, sample->ip, buf, sizeof(buf)) > 0); + + return 0; +} + static int do_checks(void *data, const struct perf_dlfilter_sample *sample, void *ctx, bool early) { struct filter_data *d = data; @@ -333,7 +342,8 @@ static int do_checks(void *data, const struct perf_dlfilter_sample *sample, void if (early && !d->do_early) return 0; - if (check_al(ctx) || check_addr_al(ctx) || check_address_al(ctx, sample)) + if (check_al(ctx) || check_addr_al(ctx) || check_address_al(ctx, sample) || + check_object_code(ctx, sample)) return -1; if (early) diff --git a/tools/perf/pmu-events/jevents.py b/tools/perf/pmu-events/jevents.py index a7e88332276d..72ba4a9239c6 100755 --- a/tools/perf/pmu-events/jevents.py +++ b/tools/perf/pmu-events/jevents.py @@ -991,7 +991,7 @@ const struct pmu_events_table *perf_pmu__find_events_table(struct perf_pmu *pmu) } } free(cpuid); - if (!pmu) + if (!pmu || !table) return table; for (i = 0; i < table->num_pmus; i++) { diff --git a/tools/perf/pmu-events/metric.py b/tools/perf/pmu-events/metric.py index 0e9ec65d92ae..3e673f25d5fd 100644 --- a/tools/perf/pmu-events/metric.py +++ b/tools/perf/pmu-events/metric.py @@ -413,10 +413,10 @@ def has_event(event: Event) -> Function: # pylint: disable=invalid-name return Function('has_event', event) -def strcmp_cpuid_str(event: str) -> Function: +def strcmp_cpuid_str(cpuid: Event) -> Function: # pylint: disable=redefined-builtin # pylint: disable=invalid-name - return Function('strcmp_cpuid_str', event) + return Function('strcmp_cpuid_str', cpuid) class Metric: """An individual metric that will specifiable on the perf command line.""" diff --git a/tools/perf/util/bpf-prologue.c b/tools/perf/util/bpf-prologue.c deleted file mode 100644 index 9887ae09242d..000000000000 --- a/tools/perf/util/bpf-prologue.c +++ /dev/null @@ -1,508 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * bpf-prologue.c - * - * Copyright (C) 2015 He Kuang - * Copyright (C) 2015 Wang Nan - * Copyright (C) 2015 Huawei Inc. - */ - -#include -#include "debug.h" -#include "bpf-loader.h" -#include "bpf-prologue.h" -#include "probe-finder.h" -#include -#include -#include -#include - -#define BPF_REG_SIZE 8 - -#define JMP_TO_ERROR_CODE -1 -#define JMP_TO_SUCCESS_CODE -2 -#define JMP_TO_USER_CODE -3 - -struct bpf_insn_pos { - struct bpf_insn *begin; - struct bpf_insn *end; - struct bpf_insn *pos; -}; - -static inline int -pos_get_cnt(struct bpf_insn_pos *pos) -{ - return pos->pos - pos->begin; -} - -static int -append_insn(struct bpf_insn new_insn, struct bpf_insn_pos *pos) -{ - if (!pos->pos) - return -BPF_LOADER_ERRNO__PROLOGUE2BIG; - - if (pos->pos + 1 >= pos->end) { - pr_err("bpf prologue: prologue too long\n"); - pos->pos = NULL; - return -BPF_LOADER_ERRNO__PROLOGUE2BIG; - } - - *(pos->pos)++ = new_insn; - return 0; -} - -static int -check_pos(struct bpf_insn_pos *pos) -{ - if (!pos->pos || pos->pos >= pos->end) - return -BPF_LOADER_ERRNO__PROLOGUE2BIG; - return 0; -} - -/* - * Convert type string (u8/u16/u32/u64/s8/s16/s32/s64 ..., see - * Documentation/trace/kprobetrace.rst) to size field of BPF_LDX_MEM - * instruction (BPF_{B,H,W,DW}). - */ -static int -argtype_to_ldx_size(const char *type) -{ - int arg_size = type ? atoi(&type[1]) : 64; - - switch (arg_size) { - case 8: - return BPF_B; - case 16: - return BPF_H; - case 32: - return BPF_W; - case 64: - default: - return BPF_DW; - } -} - -static const char * -insn_sz_to_str(int insn_sz) -{ - switch (insn_sz) { - case BPF_B: - return "BPF_B"; - case BPF_H: - return "BPF_H"; - case BPF_W: - return "BPF_W"; - case BPF_DW: - return "BPF_DW"; - default: - return "UNKNOWN"; - } -} - -/* Give it a shorter name */ -#define ins(i, p) append_insn((i), (p)) - -/* - * Give a register name (in 'reg'), generate instruction to - * load register into an eBPF register rd: - * 'ldd target_reg, offset(ctx_reg)', where: - * ctx_reg is pre initialized to pointer of 'struct pt_regs'. - */ -static int -gen_ldx_reg_from_ctx(struct bpf_insn_pos *pos, int ctx_reg, - const char *reg, int target_reg) -{ - int offset = regs_query_register_offset(reg); - - if (offset < 0) { - pr_err("bpf: prologue: failed to get register %s\n", - reg); - return offset; - } - ins(BPF_LDX_MEM(BPF_DW, target_reg, ctx_reg, offset), pos); - - return check_pos(pos); -} - -/* - * Generate a BPF_FUNC_probe_read function call. - * - * src_base_addr_reg is a register holding base address, - * dst_addr_reg is a register holding dest address (on stack), - * result is: - * - * *[dst_addr_reg] = *([src_base_addr_reg] + offset) - * - * Arguments of BPF_FUNC_probe_read: - * ARG1: ptr to stack (dest) - * ARG2: size (8) - * ARG3: unsafe ptr (src) - */ -static int -gen_read_mem(struct bpf_insn_pos *pos, - int src_base_addr_reg, - int dst_addr_reg, - long offset, - int probeid) -{ - /* mov arg3, src_base_addr_reg */ - if (src_base_addr_reg != BPF_REG_ARG3) - ins(BPF_MOV64_REG(BPF_REG_ARG3, src_base_addr_reg), pos); - /* add arg3, #offset */ - if (offset) - ins(BPF_ALU64_IMM(BPF_ADD, BPF_REG_ARG3, offset), pos); - - /* mov arg2, #reg_size */ - ins(BPF_ALU64_IMM(BPF_MOV, BPF_REG_ARG2, BPF_REG_SIZE), pos); - - /* mov arg1, dst_addr_reg */ - if (dst_addr_reg != BPF_REG_ARG1) - ins(BPF_MOV64_REG(BPF_REG_ARG1, dst_addr_reg), pos); - - /* Call probe_read */ - ins(BPF_EMIT_CALL(probeid), pos); - /* - * Error processing: if read fail, goto error code, - * will be relocated. Target should be the start of - * error processing code. - */ - ins(BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, JMP_TO_ERROR_CODE), - pos); - - return check_pos(pos); -} - -/* - * Each arg should be bare register. Fetch and save them into argument - * registers (r3 - r5). - * - * BPF_REG_1 should have been initialized with pointer to - * 'struct pt_regs'. - */ -static int -gen_prologue_fastpath(struct bpf_insn_pos *pos, - struct probe_trace_arg *args, int nargs) -{ - int i, err = 0; - - for (i = 0; i < nargs; i++) { - err = gen_ldx_reg_from_ctx(pos, BPF_REG_1, args[i].value, - BPF_PROLOGUE_START_ARG_REG + i); - if (err) - goto errout; - } - - return check_pos(pos); -errout: - return err; -} - -/* - * Slow path: - * At least one argument has the form of 'offset($rx)'. - * - * Following code first stores them into stack, then loads all of then - * to r2 - r5. - * Before final loading, the final result should be: - * - * low address - * BPF_REG_FP - 24 ARG3 - * BPF_REG_FP - 16 ARG2 - * BPF_REG_FP - 8 ARG1 - * BPF_REG_FP - * high address - * - * For each argument (described as: offn(...off2(off1(reg)))), - * generates following code: - * - * r7 <- fp - * r7 <- r7 - stack_offset // Ideal code should initialize r7 using - * // fp before generating args. However, - * // eBPF won't regard r7 as stack pointer - * // if it is generated by minus 8 from - * // another stack pointer except fp. - * // This is why we have to set r7 - * // to fp for each variable. - * r3 <- value of 'reg'-> generated using gen_ldx_reg_from_ctx() - * (r7) <- r3 // skip following instructions for bare reg - * r3 <- r3 + off1 . // skip if off1 == 0 - * r2 <- 8 \ - * r1 <- r7 |-> generated by gen_read_mem() - * call probe_read / - * jnei r0, 0, err ./ - * r3 <- (r7) - * r3 <- r3 + off2 . // skip if off2 == 0 - * r2 <- 8 \ // r2 may be broken by probe_read, so set again - * r1 <- r7 |-> generated by gen_read_mem() - * call probe_read / - * jnei r0, 0, err ./ - * ... - */ -static int -gen_prologue_slowpath(struct bpf_insn_pos *pos, - struct probe_trace_arg *args, int nargs) -{ - int err, i, probeid; - - for (i = 0; i < nargs; i++) { - struct probe_trace_arg *arg = &args[i]; - const char *reg = arg->value; - struct probe_trace_arg_ref *ref = NULL; - int stack_offset = (i + 1) * -8; - - pr_debug("prologue: fetch arg %d, base reg is %s\n", - i, reg); - - /* value of base register is stored into ARG3 */ - err = gen_ldx_reg_from_ctx(pos, BPF_REG_CTX, reg, - BPF_REG_ARG3); - if (err) { - pr_err("prologue: failed to get offset of register %s\n", - reg); - goto errout; - } - - /* Make r7 the stack pointer. */ - ins(BPF_MOV64_REG(BPF_REG_7, BPF_REG_FP), pos); - /* r7 += -8 */ - ins(BPF_ALU64_IMM(BPF_ADD, BPF_REG_7, stack_offset), pos); - /* - * Store r3 (base register) onto stack - * Ensure fp[offset] is set. - * fp is the only valid base register when storing - * into stack. We are not allowed to use r7 as base - * register here. - */ - ins(BPF_STX_MEM(BPF_DW, BPF_REG_FP, BPF_REG_ARG3, - stack_offset), pos); - - ref = arg->ref; - probeid = BPF_FUNC_probe_read_kernel; - while (ref) { - pr_debug("prologue: arg %d: offset %ld\n", - i, ref->offset); - - if (ref->user_access) - probeid = BPF_FUNC_probe_read_user; - - err = gen_read_mem(pos, BPF_REG_3, BPF_REG_7, - ref->offset, probeid); - if (err) { - pr_err("prologue: failed to generate probe_read function call\n"); - goto errout; - } - - ref = ref->next; - /* - * Load previous result into ARG3. Use - * BPF_REG_FP instead of r7 because verifier - * allows FP based addressing only. - */ - if (ref) - ins(BPF_LDX_MEM(BPF_DW, BPF_REG_ARG3, - BPF_REG_FP, stack_offset), pos); - } - } - - /* Final pass: read to registers */ - for (i = 0; i < nargs; i++) { - int insn_sz = (args[i].ref) ? argtype_to_ldx_size(args[i].type) : BPF_DW; - - pr_debug("prologue: load arg %d, insn_sz is %s\n", - i, insn_sz_to_str(insn_sz)); - ins(BPF_LDX_MEM(insn_sz, BPF_PROLOGUE_START_ARG_REG + i, - BPF_REG_FP, -BPF_REG_SIZE * (i + 1)), pos); - } - - ins(BPF_JMP_IMM(BPF_JA, BPF_REG_0, 0, JMP_TO_SUCCESS_CODE), pos); - - return check_pos(pos); -errout: - return err; -} - -static int -prologue_relocate(struct bpf_insn_pos *pos, struct bpf_insn *error_code, - struct bpf_insn *success_code, struct bpf_insn *user_code) -{ - struct bpf_insn *insn; - - if (check_pos(pos)) - return -BPF_LOADER_ERRNO__PROLOGUE2BIG; - - for (insn = pos->begin; insn < pos->pos; insn++) { - struct bpf_insn *target; - u8 class = BPF_CLASS(insn->code); - u8 opcode; - - if (class != BPF_JMP) - continue; - opcode = BPF_OP(insn->code); - if (opcode == BPF_CALL) - continue; - - switch (insn->off) { - case JMP_TO_ERROR_CODE: - target = error_code; - break; - case JMP_TO_SUCCESS_CODE: - target = success_code; - break; - case JMP_TO_USER_CODE: - target = user_code; - break; - default: - pr_err("bpf prologue: internal error: relocation failed\n"); - return -BPF_LOADER_ERRNO__PROLOGUE; - } - - insn->off = target - (insn + 1); - } - return 0; -} - -int bpf__gen_prologue(struct probe_trace_arg *args, int nargs, - struct bpf_insn *new_prog, size_t *new_cnt, - size_t cnt_space) -{ - struct bpf_insn *success_code = NULL; - struct bpf_insn *error_code = NULL; - struct bpf_insn *user_code = NULL; - struct bpf_insn_pos pos; - bool fastpath = true; - int err = 0, i; - - if (!new_prog || !new_cnt) - return -EINVAL; - - if (cnt_space > BPF_MAXINSNS) - cnt_space = BPF_MAXINSNS; - - pos.begin = new_prog; - pos.end = new_prog + cnt_space; - pos.pos = new_prog; - - if (!nargs) { - ins(BPF_ALU64_IMM(BPF_MOV, BPF_PROLOGUE_FETCH_RESULT_REG, 0), - &pos); - - if (check_pos(&pos)) - goto errout; - - *new_cnt = pos_get_cnt(&pos); - return 0; - } - - if (nargs > BPF_PROLOGUE_MAX_ARGS) { - pr_warning("bpf: prologue: %d arguments are dropped\n", - nargs - BPF_PROLOGUE_MAX_ARGS); - nargs = BPF_PROLOGUE_MAX_ARGS; - } - - /* First pass: validation */ - for (i = 0; i < nargs; i++) { - struct probe_trace_arg_ref *ref = args[i].ref; - - if (args[i].value[0] == '@') { - /* TODO: fetch global variable */ - pr_err("bpf: prologue: global %s%+ld not support\n", - args[i].value, ref ? ref->offset : 0); - return -ENOTSUP; - } - - while (ref) { - /* fastpath is true if all args has ref == NULL */ - fastpath = false; - - /* - * Instruction encodes immediate value using - * s32, ref->offset is long. On systems which - * can't fill long in s32, refuse to process if - * ref->offset too large (or small). - */ -#ifdef __LP64__ -#define OFFSET_MAX ((1LL << 31) - 1) -#define OFFSET_MIN ((1LL << 31) * -1) - if (ref->offset > OFFSET_MAX || - ref->offset < OFFSET_MIN) { - pr_err("bpf: prologue: offset out of bound: %ld\n", - ref->offset); - return -BPF_LOADER_ERRNO__PROLOGUEOOB; - } -#endif - ref = ref->next; - } - } - pr_debug("prologue: pass validation\n"); - - if (fastpath) { - /* If all variables are registers... */ - pr_debug("prologue: fast path\n"); - err = gen_prologue_fastpath(&pos, args, nargs); - if (err) - goto errout; - } else { - pr_debug("prologue: slow path\n"); - - /* Initialization: move ctx to a callee saved register. */ - ins(BPF_MOV64_REG(BPF_REG_CTX, BPF_REG_ARG1), &pos); - - err = gen_prologue_slowpath(&pos, args, nargs); - if (err) - goto errout; - /* - * start of ERROR_CODE (only slow pass needs error code) - * mov r2 <- 1 // r2 is error number - * mov r3 <- 0 // r3, r4... should be touched or - * // verifier would complain - * mov r4 <- 0 - * ... - * goto usercode - */ - error_code = pos.pos; - ins(BPF_ALU64_IMM(BPF_MOV, BPF_PROLOGUE_FETCH_RESULT_REG, 1), - &pos); - - for (i = 0; i < nargs; i++) - ins(BPF_ALU64_IMM(BPF_MOV, - BPF_PROLOGUE_START_ARG_REG + i, - 0), - &pos); - ins(BPF_JMP_IMM(BPF_JA, BPF_REG_0, 0, JMP_TO_USER_CODE), - &pos); - } - - /* - * start of SUCCESS_CODE: - * mov r2 <- 0 - * goto usercode // skip - */ - success_code = pos.pos; - ins(BPF_ALU64_IMM(BPF_MOV, BPF_PROLOGUE_FETCH_RESULT_REG, 0), &pos); - - /* - * start of USER_CODE: - * Restore ctx to r1 - */ - user_code = pos.pos; - if (!fastpath) { - /* - * Only slow path needs restoring of ctx. In fast path, - * register are loaded directly from r1. - */ - ins(BPF_MOV64_REG(BPF_REG_ARG1, BPF_REG_CTX), &pos); - err = prologue_relocate(&pos, error_code, success_code, - user_code); - if (err) - goto errout; - } - - err = check_pos(&pos); - if (err) - goto errout; - - *new_cnt = pos_get_cnt(&pos); - return 0; -errout: - return err; -} diff --git a/tools/perf/util/bpf_skel/augmented_raw_syscalls.bpf.c b/tools/perf/util/bpf_skel/augmented_raw_syscalls.bpf.c index 90ce22f9c1a9..939ec769bf4a 100644 --- a/tools/perf/util/bpf_skel/augmented_raw_syscalls.bpf.c +++ b/tools/perf/util/bpf_skel/augmented_raw_syscalls.bpf.c @@ -23,7 +23,9 @@ #define MAX_CPUS 4096 // FIXME: These should come from system headers +#ifndef bool typedef char bool; +#endif typedef int pid_t; typedef long long int __s64; typedef __s64 time64_t; diff --git a/tools/perf/util/dlfilter.c b/tools/perf/util/dlfilter.c index 1dbf27822ee2..4a1dc21b0450 100644 --- a/tools/perf/util/dlfilter.c +++ b/tools/perf/util/dlfilter.c @@ -282,13 +282,21 @@ static struct perf_event_attr *dlfilter__attr(void *ctx) return &d->evsel->core.attr; } +static __s32 code_read(__u64 ip, struct map *map, struct machine *machine, void *buf, __u32 len) +{ + u64 offset = map__map_ip(map, ip); + + if (ip + len >= map__end(map)) + len = map__end(map) - ip; + + return dso__data_read_offset(map__dso(map), machine, offset, buf, len); +} + static __s32 dlfilter__object_code(void *ctx, __u64 ip, void *buf, __u32 len) { struct dlfilter *d = (struct dlfilter *)ctx; struct addr_location *al; struct addr_location a; - struct map *map; - u64 offset; __s32 ret; if (!d->ctx_valid) @@ -298,27 +306,17 @@ static __s32 dlfilter__object_code(void *ctx, __u64 ip, void *buf, __u32 len) if (!al) return -1; - map = al->map; - - if (map && ip >= map__start(map) && ip < map__end(map) && + if (al->map && ip >= map__start(al->map) && ip < map__end(al->map) && machine__kernel_ip(d->machine, ip) == machine__kernel_ip(d->machine, d->sample->ip)) - goto have_map; + return code_read(ip, al->map, d->machine, buf, len); addr_location__init(&a); + thread__find_map_fb(al->thread, d->sample->cpumode, ip, &a); - if (!a.map) { - ret = -1; - goto out; - } + ret = a.map ? code_read(ip, a.map, d->machine, buf, len) : -1; - map = a.map; -have_map: - offset = map__map_ip(map, ip); - if (ip + len >= map__end(map)) - len = map__end(map) - ip; - ret = dso__data_read_offset(map__dso(map), d->machine, offset, buf, len); -out: addr_location__exit(&a); + return ret; } diff --git a/tools/perf/util/hashmap.h b/tools/perf/util/hashmap.h index 0a5bf1937a7c..c12f8320e668 100644 --- a/tools/perf/util/hashmap.h +++ b/tools/perf/util/hashmap.h @@ -80,16 +80,6 @@ struct hashmap { size_t sz; }; -#define HASHMAP_INIT(hash_fn, equal_fn, ctx) { \ - .hash_fn = (hash_fn), \ - .equal_fn = (equal_fn), \ - .ctx = (ctx), \ - .buckets = NULL, \ - .cap = 0, \ - .cap_bits = 0, \ - .sz = 0, \ -} - void hashmap__init(struct hashmap *map, hashmap_hash_fn hash_fn, hashmap_equal_fn equal_fn, void *ctx); struct hashmap *hashmap__new(hashmap_hash_fn hash_fn, diff --git a/tools/perf/util/pmu.c b/tools/perf/util/pmu.c index d85602aa4b9f..d515ba8a0e16 100644 --- a/tools/perf/util/pmu.c +++ b/tools/perf/util/pmu.c @@ -295,7 +295,7 @@ static int perf_pmu__parse_scale(struct perf_pmu *pmu, struct perf_pmu_alias *al len = perf_pmu__event_source_devices_scnprintf(path, sizeof(path)); if (!len) return 0; - scnprintf(path + len, sizeof(path) - len, "%s/%s.scale", pmu->name, alias->name); + scnprintf(path + len, sizeof(path) - len, "%s/events/%s.scale", pmu->name, alias->name); fd = open(path, O_RDONLY); if (fd == -1) @@ -330,7 +330,7 @@ static int perf_pmu__parse_unit(struct perf_pmu *pmu, struct perf_pmu_alias *ali len = perf_pmu__event_source_devices_scnprintf(path, sizeof(path)); if (!len) return 0; - scnprintf(path + len, sizeof(path) - len, "%s/%s.unit", pmu->name, alias->name); + scnprintf(path + len, sizeof(path) - len, "%s/events/%s.unit", pmu->name, alias->name); fd = open(path, O_RDONLY); if (fd == -1) @@ -364,7 +364,7 @@ perf_pmu__parse_per_pkg(struct perf_pmu *pmu, struct perf_pmu_alias *alias) len = perf_pmu__event_source_devices_scnprintf(path, sizeof(path)); if (!len) return 0; - scnprintf(path + len, sizeof(path) - len, "%s/%s.per-pkg", pmu->name, alias->name); + scnprintf(path + len, sizeof(path) - len, "%s/events/%s.per-pkg", pmu->name, alias->name); fd = open(path, O_RDONLY); if (fd == -1) @@ -385,7 +385,7 @@ static int perf_pmu__parse_snapshot(struct perf_pmu *pmu, struct perf_pmu_alias len = perf_pmu__event_source_devices_scnprintf(path, sizeof(path)); if (!len) return 0; - scnprintf(path + len, sizeof(path) - len, "%s/%s.snapshot", pmu->name, alias->name); + scnprintf(path + len, sizeof(path) - len, "%s/events/%s.snapshot", pmu->name, alias->name); fd = open(path, O_RDONLY); if (fd == -1) @@ -520,7 +520,7 @@ static int perf_pmu__new_alias(struct perf_pmu *pmu, const char *name, pmu_name = pe->pmu; } - alias = malloc(sizeof(*alias)); + alias = zalloc(sizeof(*alias)); if (!alias) return -ENOMEM; diff --git a/tools/testing/memblock/internal.h b/tools/testing/memblock/internal.h index fdb7f5db7308..f6c6e5474c3a 100644 --- a/tools/testing/memblock/internal.h +++ b/tools/testing/memblock/internal.h @@ -20,4 +20,8 @@ void memblock_free_pages(struct page *page, unsigned long pfn, { } +static inline void accept_memory(phys_addr_t start, phys_addr_t end) +{ +} + #endif diff --git a/tools/testing/memblock/mmzone.c b/tools/testing/memblock/mmzone.c index 7b0909e8b759..d3d58851864e 100644 --- a/tools/testing/memblock/mmzone.c +++ b/tools/testing/memblock/mmzone.c @@ -11,7 +11,7 @@ struct pglist_data *next_online_pgdat(struct pglist_data *pgdat) return NULL; } -void reserve_bootmem_region(phys_addr_t start, phys_addr_t end) +void reserve_bootmem_region(phys_addr_t start, phys_addr_t end, int nid) { } diff --git a/tools/testing/memblock/tests/basic_api.c b/tools/testing/memblock/tests/basic_api.c index 411647094cc3..57bf2688edfd 100644 --- a/tools/testing/memblock/tests/basic_api.c +++ b/tools/testing/memblock/tests/basic_api.c @@ -1,7 +1,7 @@ // SPDX-License-Identifier: GPL-2.0-or-later +#include "basic_api.h" #include #include -#include "basic_api.h" #define EXPECTED_MEMBLOCK_REGIONS 128 #define FUNC_ADD "memblock_add" diff --git a/tools/testing/memblock/tests/common.h b/tools/testing/memblock/tests/common.h index 4f23302ee677..b5ec59aa62d7 100644 --- a/tools/testing/memblock/tests/common.h +++ b/tools/testing/memblock/tests/common.h @@ -5,6 +5,7 @@ #include #include #include +#include #include #include #include diff --git a/tools/testing/selftests/Makefile b/tools/testing/selftests/Makefile index 42806add0114..1a21d6beebc6 100644 --- a/tools/testing/selftests/Makefile +++ b/tools/testing/selftests/Makefile @@ -92,7 +92,7 @@ endif TARGETS += tmpfs TARGETS += tpm2 TARGETS += tty -TARGETS += uevents +TARGETS += uevent TARGETS += user TARGETS += user_events TARGETS += vDSO diff --git a/tools/testing/selftests/alsa/conf.c b/tools/testing/selftests/alsa/conf.c index d7aafe5a1993..2f1685a3eae1 100644 --- a/tools/testing/selftests/alsa/conf.c +++ b/tools/testing/selftests/alsa/conf.c @@ -431,7 +431,6 @@ long conf_get_long(snd_config_t *root, const char *key1, const char *key2, long int conf_get_bool(snd_config_t *root, const char *key1, const char *key2, int def) { snd_config_t *cfg; - long l; int ret; if (!root) diff --git a/tools/testing/selftests/alsa/mixer-test.c b/tools/testing/selftests/alsa/mixer-test.c index c95d63e553f4..21e482b23f50 100644 --- a/tools/testing/selftests/alsa/mixer-test.c +++ b/tools/testing/selftests/alsa/mixer-test.c @@ -188,7 +188,7 @@ static int wait_for_event(struct ctl_data *ctl, int timeout) { unsigned short revents; snd_ctl_event_t *event; - int count, err; + int err; unsigned int mask = 0; unsigned int ev_id; @@ -430,7 +430,6 @@ static bool strend(const char *haystack, const char *needle) static void test_ctl_name(struct ctl_data *ctl) { bool name_ok = true; - bool check; ksft_print_msg("%d.%d %s\n", ctl->card->card, ctl->elem, ctl->name); @@ -863,7 +862,6 @@ static bool test_ctl_write_invalid_value(struct ctl_data *ctl, snd_ctl_elem_value_t *val) { int err; - long val_read; /* Ideally this will fail... */ err = snd_ctl_elem_write(ctl->card->handle, val); @@ -883,8 +881,7 @@ static bool test_ctl_write_invalid_value(struct ctl_data *ctl, static bool test_ctl_write_invalid_boolean(struct ctl_data *ctl) { - int err, i; - long val_read; + int i; bool fail = false; snd_ctl_elem_value_t *val; snd_ctl_elem_value_alloca(&val); @@ -994,8 +991,7 @@ static bool test_ctl_write_invalid_integer64(struct ctl_data *ctl) static bool test_ctl_write_invalid_enumerated(struct ctl_data *ctl) { - int err, i; - unsigned int val_read; + int i; bool fail = false; snd_ctl_elem_value_t *val; snd_ctl_elem_value_alloca(&val); @@ -1027,7 +1023,6 @@ static bool test_ctl_write_invalid_enumerated(struct ctl_data *ctl) static void test_ctl_write_invalid(struct ctl_data *ctl) { bool pass; - int err; /* If the control is turned off let's be polite */ if (snd_ctl_elem_info_is_inactive(ctl->info)) { diff --git a/tools/testing/selftests/alsa/pcm-test.c b/tools/testing/selftests/alsa/pcm-test.c index 2f5e3c462194..c0a39818c5a4 100644 --- a/tools/testing/selftests/alsa/pcm-test.c +++ b/tools/testing/selftests/alsa/pcm-test.c @@ -257,7 +257,7 @@ static void find_pcms(void) static void test_pcm_time(struct pcm_data *data, enum test_class class, const char *test_name, snd_config_t *pcm_cfg) { - char name[64], key[128], msg[256]; + char name[64], msg[256]; const int duration_s = 2, margin_ms = 100; const int duration_ms = duration_s * 1000; const char *cs; @@ -567,7 +567,7 @@ int main(void) { struct card_data *card; struct pcm_data *pcm; - snd_config_t *global_config, *cfg, *pcm_cfg; + snd_config_t *global_config, *cfg; int num_pcm_tests = 0, num_tests, num_std_pcm_tests; int ret; void *thread_ret; diff --git a/tools/testing/selftests/alsa/test-pcmtest-driver.c b/tools/testing/selftests/alsa/test-pcmtest-driver.c index 357adc722cba..a52ecd43dbe3 100644 --- a/tools/testing/selftests/alsa/test-pcmtest-driver.c +++ b/tools/testing/selftests/alsa/test-pcmtest-driver.c @@ -313,7 +313,6 @@ TEST_F(pcmtest, ni_playback) { */ TEST_F(pcmtest, reset_ioctl) { snd_pcm_t *handle; - unsigned char *it; int test_res; struct pcmtest_test_params *params = &self->params; diff --git a/tools/testing/selftests/bpf/DENYLIST.aarch64 b/tools/testing/selftests/bpf/DENYLIST.aarch64 index 7f768d335698..3babaf3eee5c 100644 --- a/tools/testing/selftests/bpf/DENYLIST.aarch64 +++ b/tools/testing/selftests/bpf/DENYLIST.aarch64 @@ -1,14 +1,8 @@ bpf_cookie/multi_kprobe_attach_api # kprobe_multi_link_api_subtest:FAIL:fentry_raw_skel_load unexpected error: -3 bpf_cookie/multi_kprobe_link_api # kprobe_multi_link_api_subtest:FAIL:fentry_raw_skel_load unexpected error: -3 fexit_sleep # The test never returns. The remaining tests cannot start. -kprobe_multi_bench_attach # bpf_program__attach_kprobe_multi_opts unexpected error: -95 -kprobe_multi_test/attach_api_addrs # bpf_program__attach_kprobe_multi_opts unexpected error: -95 -kprobe_multi_test/attach_api_pattern # bpf_program__attach_kprobe_multi_opts unexpected error: -95 -kprobe_multi_test/attach_api_syms # bpf_program__attach_kprobe_multi_opts unexpected error: -95 -kprobe_multi_test/bench_attach # bpf_program__attach_kprobe_multi_opts unexpected error: -95 -kprobe_multi_test/link_api_addrs # link_fd unexpected link_fd: actual -95 < expected 0 -kprobe_multi_test/link_api_syms # link_fd unexpected link_fd: actual -95 < expected 0 -kprobe_multi_test/skel_api # libbpf: failed to load BPF skeleton 'kprobe_multi': -3 +kprobe_multi_bench_attach # needs CONFIG_FPROBE +kprobe_multi_test # needs CONFIG_FPROBE module_attach # prog 'kprobe_multi': failed to auto-attach: -95 fentry_test/fentry_many_args # fentry_many_args:FAIL:fentry_many_args_attach unexpected error: -524 fexit_test/fexit_many_args # fexit_many_args:FAIL:fexit_many_args_attach unexpected error: -524 diff --git a/tools/testing/selftests/bpf/config b/tools/testing/selftests/bpf/config index 1c7584e8dd9e..e41eb33b2704 100644 --- a/tools/testing/selftests/bpf/config +++ b/tools/testing/selftests/bpf/config @@ -4,6 +4,7 @@ CONFIG_BOOTPARAM_SOFTLOCKUP_PANIC=y CONFIG_BPF=y CONFIG_BPF_EVENTS=y CONFIG_BPF_JIT=y +CONFIG_BPF_KPROBE_OVERRIDE=y CONFIG_BPF_LIRC_MODE2=y CONFIG_BPF_LSM=y CONFIG_BPF_STREAM_PARSER=y diff --git a/tools/testing/selftests/bpf/config.x86_64 b/tools/testing/selftests/bpf/config.x86_64 index b650b2e617b8..2e70a6048278 100644 --- a/tools/testing/selftests/bpf/config.x86_64 +++ b/tools/testing/selftests/bpf/config.x86_64 @@ -20,7 +20,6 @@ CONFIG_BLK_DEV_THROTTLING=y CONFIG_BONDING=y CONFIG_BOOTTIME_TRACING=y CONFIG_BPF_JIT_ALWAYS_ON=y -CONFIG_BPF_KPROBE_OVERRIDE=y CONFIG_BPF_PRELOAD=y CONFIG_BPF_PRELOAD_UMD=y CONFIG_BPFILTER=y diff --git a/tools/testing/selftests/bpf/prog_tests/bpf_tcp_ca.c b/tools/testing/selftests/bpf/prog_tests/bpf_tcp_ca.c index a53c254c6058..4aabeaa525d4 100644 --- a/tools/testing/selftests/bpf/prog_tests/bpf_tcp_ca.c +++ b/tools/testing/selftests/bpf/prog_tests/bpf_tcp_ca.c @@ -185,6 +185,8 @@ static void test_cubic(void) do_test("bpf_cubic", NULL); + ASSERT_EQ(cubic_skel->bss->bpf_cubic_acked_called, 1, "pkts_acked called"); + bpf_link__destroy(link); bpf_cubic__destroy(cubic_skel); } diff --git a/tools/testing/selftests/bpf/prog_tests/empty_skb.c b/tools/testing/selftests/bpf/prog_tests/empty_skb.c index 3b77d8a422db..261228eb68e8 100644 --- a/tools/testing/selftests/bpf/prog_tests/empty_skb.c +++ b/tools/testing/selftests/bpf/prog_tests/empty_skb.c @@ -24,6 +24,7 @@ void test_empty_skb(void) int *ifindex; int err; int ret; + int lwt_egress_ret; /* expected retval at lwt/egress */ bool success_on_tc; } tests[] = { /* Empty packets are always rejected. */ @@ -57,6 +58,7 @@ void test_empty_skb(void) .data_size_in = sizeof(eth_hlen), .ifindex = &veth_ifindex, .ret = -ERANGE, + .lwt_egress_ret = -ERANGE, .success_on_tc = true, }, { @@ -70,6 +72,7 @@ void test_empty_skb(void) .data_size_in = sizeof(eth_hlen), .ifindex = &ipip_ifindex, .ret = -ERANGE, + .lwt_egress_ret = -ERANGE, }, /* ETH_HLEN+1-sized packet should be redirected. */ @@ -79,6 +82,7 @@ void test_empty_skb(void) .data_in = eth_hlen_pp, .data_size_in = sizeof(eth_hlen_pp), .ifindex = &veth_ifindex, + .lwt_egress_ret = 1, /* veth_xmit NET_XMIT_DROP */ }, { .msg = "ipip ETH_HLEN+1 packet ingress", @@ -108,8 +112,12 @@ void test_empty_skb(void) for (i = 0; i < ARRAY_SIZE(tests); i++) { bpf_object__for_each_program(prog, bpf_obj->obj) { - char buf[128]; + bool at_egress = strstr(bpf_program__name(prog), "egress") != NULL; bool at_tc = !strncmp(bpf_program__section_name(prog), "tc", 2); + int expected_ret; + char buf[128]; + + expected_ret = at_egress && !at_tc ? tests[i].lwt_egress_ret : tests[i].ret; tattr.data_in = tests[i].data_in; tattr.data_size_in = tests[i].data_size_in; @@ -128,7 +136,7 @@ void test_empty_skb(void) if (at_tc && tests[i].success_on_tc) ASSERT_GE(bpf_obj->bss->ret, 0, buf); else - ASSERT_EQ(bpf_obj->bss->ret, tests[i].ret, buf); + ASSERT_EQ(bpf_obj->bss->ret, expected_ret, buf); } } diff --git a/tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c b/tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c index 179fe300534f..4041cfa670eb 100644 --- a/tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c +++ b/tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c @@ -3,6 +3,7 @@ #include "kprobe_multi.skel.h" #include "trace_helpers.h" #include "kprobe_multi_empty.skel.h" +#include "kprobe_multi_override.skel.h" #include "bpf/libbpf_internal.h" #include "bpf/hashmap.h" @@ -453,6 +454,40 @@ cleanup: } } +static void test_attach_override(void) +{ + struct kprobe_multi_override *skel = NULL; + struct bpf_link *link = NULL; + + skel = kprobe_multi_override__open_and_load(); + if (!ASSERT_OK_PTR(skel, "kprobe_multi_empty__open_and_load")) + goto cleanup; + + /* The test_override calls bpf_override_return so it should fail + * to attach to bpf_fentry_test1 function, which is not on error + * injection list. + */ + link = bpf_program__attach_kprobe_multi_opts(skel->progs.test_override, + "bpf_fentry_test1", NULL); + if (!ASSERT_ERR_PTR(link, "override_attached_bpf_fentry_test1")) { + bpf_link__destroy(link); + goto cleanup; + } + + /* The should_fail_bio function is on error injection list, + * attach should succeed. + */ + link = bpf_program__attach_kprobe_multi_opts(skel->progs.test_override, + "should_fail_bio", NULL); + if (!ASSERT_OK_PTR(link, "override_attached_should_fail_bio")) + goto cleanup; + + bpf_link__destroy(link); + +cleanup: + kprobe_multi_override__destroy(skel); +} + void serial_test_kprobe_multi_bench_attach(void) { if (test__start_subtest("kernel")) @@ -480,4 +515,6 @@ void test_kprobe_multi_test(void) test_attach_api_syms(); if (test__start_subtest("attach_api_fails")) test_attach_api_fails(); + if (test__start_subtest("attach_override")) + test_attach_override(); } diff --git a/tools/testing/selftests/bpf/prog_tests/sockmap_basic.c b/tools/testing/selftests/bpf/prog_tests/sockmap_basic.c index 064cc5e8d9ad..dda7060e86a0 100644 --- a/tools/testing/selftests/bpf/prog_tests/sockmap_basic.c +++ b/tools/testing/selftests/bpf/prog_tests/sockmap_basic.c @@ -475,6 +475,55 @@ out: test_sockmap_drop_prog__destroy(drop); } +static void test_sockmap_skb_verdict_peek(void) +{ + int err, map, verdict, s, c1, p1, zero = 0, sent, recvd, avail; + struct test_sockmap_pass_prog *pass; + char snd[256] = "0123456789"; + char rcv[256] = "0"; + + pass = test_sockmap_pass_prog__open_and_load(); + if (!ASSERT_OK_PTR(pass, "open_and_load")) + return; + verdict = bpf_program__fd(pass->progs.prog_skb_verdict); + map = bpf_map__fd(pass->maps.sock_map_rx); + + err = bpf_prog_attach(verdict, map, BPF_SK_SKB_STREAM_VERDICT, 0); + if (!ASSERT_OK(err, "bpf_prog_attach")) + goto out; + + s = socket_loopback(AF_INET, SOCK_STREAM); + if (!ASSERT_GT(s, -1, "socket_loopback(s)")) + goto out; + + err = create_pair(s, AF_INET, SOCK_STREAM, &c1, &p1); + if (!ASSERT_OK(err, "create_pairs(s)")) + goto out; + + err = bpf_map_update_elem(map, &zero, &c1, BPF_NOEXIST); + if (!ASSERT_OK(err, "bpf_map_update_elem(c1)")) + goto out_close; + + sent = xsend(p1, snd, sizeof(snd), 0); + ASSERT_EQ(sent, sizeof(snd), "xsend(p1)"); + recvd = recv(c1, rcv, sizeof(rcv), MSG_PEEK); + ASSERT_EQ(recvd, sizeof(rcv), "recv(c1)"); + err = ioctl(c1, FIONREAD, &avail); + ASSERT_OK(err, "ioctl(FIONREAD) error"); + ASSERT_EQ(avail, sizeof(snd), "after peek ioctl(FIONREAD)"); + recvd = recv(c1, rcv, sizeof(rcv), 0); + ASSERT_EQ(recvd, sizeof(rcv), "recv(p0)"); + err = ioctl(c1, FIONREAD, &avail); + ASSERT_OK(err, "ioctl(FIONREAD) error"); + ASSERT_EQ(avail, 0, "after read ioctl(FIONREAD)"); + +out_close: + close(c1); + close(p1); +out: + test_sockmap_pass_prog__destroy(pass); +} + void test_sockmap_basic(void) { if (test__start_subtest("sockmap create_update_free")) @@ -515,4 +564,6 @@ void test_sockmap_basic(void) test_sockmap_skb_verdict_fionread(true); if (test__start_subtest("sockmap skb_verdict fionread on drop")) test_sockmap_skb_verdict_fionread(false); + if (test__start_subtest("sockmap skb_verdict msg_f_peek")) + test_sockmap_skb_verdict_peek(); } diff --git a/tools/testing/selftests/bpf/prog_tests/tc_helpers.h b/tools/testing/selftests/bpf/prog_tests/tc_helpers.h index 6c93215be8a3..67f985f7d215 100644 --- a/tools/testing/selftests/bpf/prog_tests/tc_helpers.h +++ b/tools/testing/selftests/bpf/prog_tests/tc_helpers.h @@ -45,7 +45,7 @@ static inline __u32 ifindex_from_link_fd(int fd) return link_info.tcx.ifindex; } -static inline void __assert_mprog_count(int target, int expected, bool miniq, int ifindex) +static inline void __assert_mprog_count(int target, int expected, int ifindex) { __u32 count = 0, attach_flags = 0; int err; @@ -53,20 +53,22 @@ static inline void __assert_mprog_count(int target, int expected, bool miniq, in err = bpf_prog_query(ifindex, target, 0, &attach_flags, NULL, &count); ASSERT_EQ(count, expected, "count"); - if (!expected && !miniq) - ASSERT_EQ(err, -ENOENT, "prog_query"); - else - ASSERT_EQ(err, 0, "prog_query"); + ASSERT_EQ(err, 0, "prog_query"); } static inline void assert_mprog_count(int target, int expected) { - __assert_mprog_count(target, expected, false, loopback); + __assert_mprog_count(target, expected, loopback); } static inline void assert_mprog_count_ifindex(int ifindex, int target, int expected) { - __assert_mprog_count(target, expected, false, ifindex); + __assert_mprog_count(target, expected, ifindex); +} + +static inline void tc_skel_reset_all_seen(struct test_tc_link *skel) +{ + memset(skel->bss, 0, sizeof(*skel->bss)); } #endif /* TC_HELPERS */ diff --git a/tools/testing/selftests/bpf/prog_tests/tc_links.c b/tools/testing/selftests/bpf/prog_tests/tc_links.c index 74fc1fe9ee26..bc9841144685 100644 --- a/tools/testing/selftests/bpf/prog_tests/tc_links.c +++ b/tools/testing/selftests/bpf/prog_tests/tc_links.c @@ -65,6 +65,7 @@ void serial_test_tc_links_basic(void) ASSERT_EQ(optq.prog_ids[1], 0, "prog_ids[1]"); ASSERT_EQ(optq.link_ids[1], 0, "link_ids[1]"); + tc_skel_reset_all_seen(skel); ASSERT_OK(system(ping_cmd), ping_cmd); ASSERT_EQ(skel->bss->seen_tc1, true, "seen_tc1"); @@ -97,6 +98,7 @@ void serial_test_tc_links_basic(void) ASSERT_EQ(optq.prog_ids[1], 0, "prog_ids[1]"); ASSERT_EQ(optq.link_ids[1], 0, "link_ids[1]"); + tc_skel_reset_all_seen(skel); ASSERT_OK(system(ping_cmd), ping_cmd); ASSERT_EQ(skel->bss->seen_tc1, true, "seen_tc1"); @@ -187,6 +189,7 @@ static void test_tc_links_before_target(int target) ASSERT_EQ(optq.prog_ids[2], 0, "prog_ids[2]"); ASSERT_EQ(optq.link_ids[2], 0, "link_ids[2]"); + tc_skel_reset_all_seen(skel); ASSERT_OK(system(ping_cmd), ping_cmd); ASSERT_EQ(skel->bss->seen_tc1, true, "seen_tc1"); @@ -194,9 +197,6 @@ static void test_tc_links_before_target(int target) ASSERT_EQ(skel->bss->seen_tc3, false, "seen_tc3"); ASSERT_EQ(skel->bss->seen_tc4, false, "seen_tc4"); - skel->bss->seen_tc1 = false; - skel->bss->seen_tc2 = false; - LIBBPF_OPTS_RESET(optl, .flags = BPF_F_BEFORE, .relative_fd = bpf_program__fd(skel->progs.tc2), @@ -246,6 +246,7 @@ static void test_tc_links_before_target(int target) ASSERT_EQ(optq.prog_ids[4], 0, "prog_ids[4]"); ASSERT_EQ(optq.link_ids[4], 0, "link_ids[4]"); + tc_skel_reset_all_seen(skel); ASSERT_OK(system(ping_cmd), ping_cmd); ASSERT_EQ(skel->bss->seen_tc1, true, "seen_tc1"); @@ -342,6 +343,7 @@ static void test_tc_links_after_target(int target) ASSERT_EQ(optq.prog_ids[2], 0, "prog_ids[2]"); ASSERT_EQ(optq.link_ids[2], 0, "link_ids[2]"); + tc_skel_reset_all_seen(skel); ASSERT_OK(system(ping_cmd), ping_cmd); ASSERT_EQ(skel->bss->seen_tc1, true, "seen_tc1"); @@ -349,9 +351,6 @@ static void test_tc_links_after_target(int target) ASSERT_EQ(skel->bss->seen_tc3, false, "seen_tc3"); ASSERT_EQ(skel->bss->seen_tc4, false, "seen_tc4"); - skel->bss->seen_tc1 = false; - skel->bss->seen_tc2 = false; - LIBBPF_OPTS_RESET(optl, .flags = BPF_F_AFTER, .relative_fd = bpf_program__fd(skel->progs.tc1), @@ -401,6 +400,7 @@ static void test_tc_links_after_target(int target) ASSERT_EQ(optq.prog_ids[4], 0, "prog_ids[4]"); ASSERT_EQ(optq.link_ids[4], 0, "link_ids[4]"); + tc_skel_reset_all_seen(skel); ASSERT_OK(system(ping_cmd), ping_cmd); ASSERT_EQ(skel->bss->seen_tc1, true, "seen_tc1"); @@ -502,6 +502,7 @@ static void test_tc_links_revision_target(int target) ASSERT_EQ(optq.prog_ids[2], 0, "prog_ids[2]"); ASSERT_EQ(optq.link_ids[2], 0, "prog_ids[2]"); + tc_skel_reset_all_seen(skel); ASSERT_OK(system(ping_cmd), ping_cmd); ASSERT_EQ(skel->bss->seen_tc1, true, "seen_tc1"); @@ -581,22 +582,20 @@ static void test_tc_chain_classic(int target, bool chain_tc_old) assert_mprog_count(target, 2); + tc_skel_reset_all_seen(skel); ASSERT_OK(system(ping_cmd), ping_cmd); ASSERT_EQ(skel->bss->seen_tc1, true, "seen_tc1"); ASSERT_EQ(skel->bss->seen_tc2, true, "seen_tc2"); ASSERT_EQ(skel->bss->seen_tc3, chain_tc_old, "seen_tc3"); - skel->bss->seen_tc1 = false; - skel->bss->seen_tc2 = false; - skel->bss->seen_tc3 = false; - err = bpf_link__detach(skel->links.tc2); if (!ASSERT_OK(err, "prog_detach")) goto cleanup; assert_mprog_count(target, 1); + tc_skel_reset_all_seen(skel); ASSERT_OK(system(ping_cmd), ping_cmd); ASSERT_EQ(skel->bss->seen_tc1, true, "seen_tc1"); @@ -707,16 +706,13 @@ static void test_tc_links_replace_target(int target) ASSERT_EQ(optq.prog_ids[2], 0, "prog_ids[2]"); ASSERT_EQ(optq.link_ids[2], 0, "link_ids[2]"); + tc_skel_reset_all_seen(skel); ASSERT_OK(system(ping_cmd), ping_cmd); ASSERT_EQ(skel->bss->seen_tc1, true, "seen_tc1"); ASSERT_EQ(skel->bss->seen_tc2, true, "seen_tc2"); ASSERT_EQ(skel->bss->seen_tc3, false, "seen_tc3"); - skel->bss->seen_tc1 = false; - skel->bss->seen_tc2 = false; - skel->bss->seen_tc3 = false; - LIBBPF_OPTS_RESET(optl, .flags = BPF_F_REPLACE, .relative_fd = bpf_program__fd(skel->progs.tc2), @@ -781,16 +777,13 @@ static void test_tc_links_replace_target(int target) ASSERT_EQ(optq.prog_ids[2], 0, "prog_ids[2]"); ASSERT_EQ(optq.link_ids[2], 0, "link_ids[2]"); + tc_skel_reset_all_seen(skel); ASSERT_OK(system(ping_cmd), ping_cmd); ASSERT_EQ(skel->bss->seen_tc1, true, "seen_tc1"); ASSERT_EQ(skel->bss->seen_tc2, false, "seen_tc2"); ASSERT_EQ(skel->bss->seen_tc3, true, "seen_tc3"); - skel->bss->seen_tc1 = false; - skel->bss->seen_tc2 = false; - skel->bss->seen_tc3 = false; - err = bpf_link__detach(skel->links.tc2); if (!ASSERT_OK(err, "link_detach")) goto cleanup; @@ -812,16 +805,13 @@ static void test_tc_links_replace_target(int target) ASSERT_EQ(optq.prog_ids[1], 0, "prog_ids[1]"); ASSERT_EQ(optq.link_ids[1], 0, "link_ids[1]"); + tc_skel_reset_all_seen(skel); ASSERT_OK(system(ping_cmd), ping_cmd); ASSERT_EQ(skel->bss->seen_tc1, true, "seen_tc1"); ASSERT_EQ(skel->bss->seen_tc2, false, "seen_tc2"); ASSERT_EQ(skel->bss->seen_tc3, false, "seen_tc3"); - skel->bss->seen_tc1 = false; - skel->bss->seen_tc2 = false; - skel->bss->seen_tc3 = false; - err = bpf_link__update_program(skel->links.tc1, skel->progs.tc1); if (!ASSERT_OK(err, "link_update_self")) goto cleanup; @@ -843,6 +833,7 @@ static void test_tc_links_replace_target(int target) ASSERT_EQ(optq.prog_ids[1], 0, "prog_ids[1]"); ASSERT_EQ(optq.link_ids[1], 0, "link_ids[1]"); + tc_skel_reset_all_seen(skel); ASSERT_OK(system(ping_cmd), ping_cmd); ASSERT_EQ(skel->bss->seen_tc1, true, "seen_tc1"); @@ -1254,6 +1245,7 @@ static void test_tc_links_prepend_target(int target) ASSERT_EQ(optq.prog_ids[2], 0, "prog_ids[2]"); ASSERT_EQ(optq.link_ids[2], 0, "link_ids[2]"); + tc_skel_reset_all_seen(skel); ASSERT_OK(system(ping_cmd), ping_cmd); ASSERT_EQ(skel->bss->seen_tc1, true, "seen_tc1"); @@ -1261,9 +1253,6 @@ static void test_tc_links_prepend_target(int target) ASSERT_EQ(skel->bss->seen_tc3, false, "seen_tc3"); ASSERT_EQ(skel->bss->seen_tc4, false, "seen_tc4"); - skel->bss->seen_tc1 = false; - skel->bss->seen_tc2 = false; - LIBBPF_OPTS_RESET(optl, .flags = BPF_F_BEFORE, ); @@ -1311,6 +1300,7 @@ static void test_tc_links_prepend_target(int target) ASSERT_EQ(optq.prog_ids[4], 0, "prog_ids[4]"); ASSERT_EQ(optq.link_ids[4], 0, "link_ids[4]"); + tc_skel_reset_all_seen(skel); ASSERT_OK(system(ping_cmd), ping_cmd); ASSERT_EQ(skel->bss->seen_tc1, true, "seen_tc1"); @@ -1411,6 +1401,7 @@ static void test_tc_links_append_target(int target) ASSERT_EQ(optq.prog_ids[2], 0, "prog_ids[2]"); ASSERT_EQ(optq.link_ids[2], 0, "link_ids[2]"); + tc_skel_reset_all_seen(skel); ASSERT_OK(system(ping_cmd), ping_cmd); ASSERT_EQ(skel->bss->seen_tc1, true, "seen_tc1"); @@ -1418,9 +1409,6 @@ static void test_tc_links_append_target(int target) ASSERT_EQ(skel->bss->seen_tc3, false, "seen_tc3"); ASSERT_EQ(skel->bss->seen_tc4, false, "seen_tc4"); - skel->bss->seen_tc1 = false; - skel->bss->seen_tc2 = false; - LIBBPF_OPTS_RESET(optl, .flags = BPF_F_AFTER, ); @@ -1468,6 +1456,7 @@ static void test_tc_links_append_target(int target) ASSERT_EQ(optq.prog_ids[4], 0, "prog_ids[4]"); ASSERT_EQ(optq.link_ids[4], 0, "link_ids[4]"); + tc_skel_reset_all_seen(skel); ASSERT_OK(system(ping_cmd), ping_cmd); ASSERT_EQ(skel->bss->seen_tc1, true, "seen_tc1"); @@ -1637,38 +1626,33 @@ static void test_tc_chain_mixed(int target) assert_mprog_count(target, 1); + tc_skel_reset_all_seen(skel); ASSERT_OK(system(ping_cmd), ping_cmd); ASSERT_EQ(skel->bss->seen_tc4, false, "seen_tc4"); ASSERT_EQ(skel->bss->seen_tc5, false, "seen_tc5"); ASSERT_EQ(skel->bss->seen_tc6, true, "seen_tc6"); - skel->bss->seen_tc4 = false; - skel->bss->seen_tc5 = false; - skel->bss->seen_tc6 = false; - err = bpf_link__update_program(skel->links.tc6, skel->progs.tc4); if (!ASSERT_OK(err, "link_update")) goto cleanup; assert_mprog_count(target, 1); + tc_skel_reset_all_seen(skel); ASSERT_OK(system(ping_cmd), ping_cmd); ASSERT_EQ(skel->bss->seen_tc4, true, "seen_tc4"); ASSERT_EQ(skel->bss->seen_tc5, true, "seen_tc5"); ASSERT_EQ(skel->bss->seen_tc6, false, "seen_tc6"); - skel->bss->seen_tc4 = false; - skel->bss->seen_tc5 = false; - skel->bss->seen_tc6 = false; - err = bpf_link__detach(skel->links.tc6); if (!ASSERT_OK(err, "prog_detach")) goto cleanup; - __assert_mprog_count(target, 0, true, loopback); + assert_mprog_count(target, 0); + tc_skel_reset_all_seen(skel); ASSERT_OK(system(ping_cmd), ping_cmd); ASSERT_EQ(skel->bss->seen_tc4, false, "seen_tc4"); @@ -1758,22 +1742,20 @@ static void test_tc_links_ingress(int target, bool chain_tc_old, assert_mprog_count(target, 2); + tc_skel_reset_all_seen(skel); ASSERT_OK(system(ping_cmd), ping_cmd); ASSERT_EQ(skel->bss->seen_tc1, true, "seen_tc1"); ASSERT_EQ(skel->bss->seen_tc2, true, "seen_tc2"); ASSERT_EQ(skel->bss->seen_tc3, chain_tc_old, "seen_tc3"); - skel->bss->seen_tc1 = false; - skel->bss->seen_tc2 = false; - skel->bss->seen_tc3 = false; - err = bpf_link__detach(skel->links.tc2); if (!ASSERT_OK(err, "prog_detach")) goto cleanup; assert_mprog_count(target, 1); + tc_skel_reset_all_seen(skel); ASSERT_OK(system(ping_cmd), ping_cmd); ASSERT_EQ(skel->bss->seen_tc1, true, "seen_tc1"); diff --git a/tools/testing/selftests/bpf/prog_tests/tc_opts.c b/tools/testing/selftests/bpf/prog_tests/tc_opts.c index 7a2ecd4eca5d..ca506d2fcf58 100644 --- a/tools/testing/selftests/bpf/prog_tests/tc_opts.c +++ b/tools/testing/selftests/bpf/prog_tests/tc_opts.c @@ -59,6 +59,7 @@ void serial_test_tc_opts_basic(void) ASSERT_EQ(optq.prog_ids[0], id1, "prog_ids[0]"); ASSERT_EQ(optq.prog_ids[1], 0, "prog_ids[1]"); + tc_skel_reset_all_seen(skel); ASSERT_OK(system(ping_cmd), ping_cmd); ASSERT_EQ(skel->bss->seen_tc1, true, "seen_tc1"); @@ -83,6 +84,7 @@ void serial_test_tc_opts_basic(void) ASSERT_EQ(optq.prog_ids[0], id2, "prog_ids[0]"); ASSERT_EQ(optq.prog_ids[1], 0, "prog_ids[1]"); + tc_skel_reset_all_seen(skel); ASSERT_OK(system(ping_cmd), ping_cmd); ASSERT_EQ(skel->bss->seen_tc1, true, "seen_tc1"); @@ -163,6 +165,7 @@ static void test_tc_opts_before_target(int target) ASSERT_EQ(optq.prog_ids[1], id2, "prog_ids[1]"); ASSERT_EQ(optq.prog_ids[2], 0, "prog_ids[2]"); + tc_skel_reset_all_seen(skel); ASSERT_OK(system(ping_cmd), ping_cmd); ASSERT_EQ(skel->bss->seen_tc1, true, "seen_tc1"); @@ -219,6 +222,7 @@ static void test_tc_opts_before_target(int target) ASSERT_EQ(optq.prog_ids[3], id2, "prog_ids[3]"); ASSERT_EQ(optq.prog_ids[4], 0, "prog_ids[4]"); + tc_skel_reset_all_seen(skel); ASSERT_OK(system(ping_cmd), ping_cmd); ASSERT_EQ(skel->bss->seen_tc1, true, "seen_tc1"); @@ -313,6 +317,7 @@ static void test_tc_opts_after_target(int target) ASSERT_EQ(optq.prog_ids[1], id2, "prog_ids[1]"); ASSERT_EQ(optq.prog_ids[2], 0, "prog_ids[2]"); + tc_skel_reset_all_seen(skel); ASSERT_OK(system(ping_cmd), ping_cmd); ASSERT_EQ(skel->bss->seen_tc1, true, "seen_tc1"); @@ -369,6 +374,7 @@ static void test_tc_opts_after_target(int target) ASSERT_EQ(optq.prog_ids[3], id4, "prog_ids[3]"); ASSERT_EQ(optq.prog_ids[4], 0, "prog_ids[4]"); + tc_skel_reset_all_seen(skel); ASSERT_OK(system(ping_cmd), ping_cmd); ASSERT_EQ(skel->bss->seen_tc1, true, "seen_tc1"); @@ -514,6 +520,7 @@ static void test_tc_opts_revision_target(int target) ASSERT_EQ(optq.prog_ids[1], id2, "prog_ids[1]"); ASSERT_EQ(optq.prog_ids[2], 0, "prog_ids[2]"); + tc_skel_reset_all_seen(skel); ASSERT_OK(system(ping_cmd), ping_cmd); ASSERT_EQ(skel->bss->seen_tc1, true, "seen_tc1"); @@ -608,22 +615,20 @@ static void test_tc_chain_classic(int target, bool chain_tc_old) assert_mprog_count(target, 2); + tc_skel_reset_all_seen(skel); ASSERT_OK(system(ping_cmd), ping_cmd); ASSERT_EQ(skel->bss->seen_tc1, true, "seen_tc1"); ASSERT_EQ(skel->bss->seen_tc2, true, "seen_tc2"); ASSERT_EQ(skel->bss->seen_tc3, chain_tc_old, "seen_tc3"); - skel->bss->seen_tc1 = false; - skel->bss->seen_tc2 = false; - skel->bss->seen_tc3 = false; - err = bpf_prog_detach_opts(fd2, loopback, target, &optd); if (!ASSERT_OK(err, "prog_detach")) goto cleanup_detach; assert_mprog_count(target, 1); + tc_skel_reset_all_seen(skel); ASSERT_OK(system(ping_cmd), ping_cmd); ASSERT_EQ(skel->bss->seen_tc1, true, "seen_tc1"); @@ -635,7 +640,7 @@ cleanup_detach: if (!ASSERT_OK(err, "prog_detach")) goto cleanup; - __assert_mprog_count(target, 0, chain_tc_old, loopback); + assert_mprog_count(target, 0); cleanup: if (tc_attached) { tc_opts.flags = tc_opts.prog_fd = tc_opts.prog_id = 0; @@ -730,16 +735,13 @@ static void test_tc_opts_replace_target(int target) ASSERT_EQ(optq.prog_attach_flags[1], 0, "prog_flags[1]"); ASSERT_EQ(optq.prog_attach_flags[2], 0, "prog_flags[2]"); + tc_skel_reset_all_seen(skel); ASSERT_OK(system(ping_cmd), ping_cmd); ASSERT_EQ(skel->bss->seen_tc1, true, "seen_tc1"); ASSERT_EQ(skel->bss->seen_tc2, true, "seen_tc2"); ASSERT_EQ(skel->bss->seen_tc3, false, "seen_tc3"); - skel->bss->seen_tc1 = false; - skel->bss->seen_tc2 = false; - skel->bss->seen_tc3 = false; - LIBBPF_OPTS_RESET(opta, .flags = BPF_F_REPLACE, .replace_prog_fd = fd2, @@ -767,16 +769,13 @@ static void test_tc_opts_replace_target(int target) ASSERT_EQ(optq.prog_ids[1], id1, "prog_ids[1]"); ASSERT_EQ(optq.prog_ids[2], 0, "prog_ids[2]"); + tc_skel_reset_all_seen(skel); ASSERT_OK(system(ping_cmd), ping_cmd); ASSERT_EQ(skel->bss->seen_tc1, true, "seen_tc1"); ASSERT_EQ(skel->bss->seen_tc2, false, "seen_tc2"); ASSERT_EQ(skel->bss->seen_tc3, true, "seen_tc3"); - skel->bss->seen_tc1 = false; - skel->bss->seen_tc2 = false; - skel->bss->seen_tc3 = false; - LIBBPF_OPTS_RESET(opta, .flags = BPF_F_REPLACE | BPF_F_BEFORE, .replace_prog_fd = fd3, @@ -805,6 +804,7 @@ static void test_tc_opts_replace_target(int target) ASSERT_EQ(optq.prog_ids[1], id1, "prog_ids[1]"); ASSERT_EQ(optq.prog_ids[2], 0, "prog_ids[2]"); + tc_skel_reset_all_seen(skel); ASSERT_OK(system(ping_cmd), ping_cmd); ASSERT_EQ(skel->bss->seen_tc1, true, "seen_tc1"); @@ -1084,6 +1084,7 @@ static void test_tc_opts_prepend_target(int target) ASSERT_EQ(optq.prog_ids[1], id1, "prog_ids[1]"); ASSERT_EQ(optq.prog_ids[2], 0, "prog_ids[2]"); + tc_skel_reset_all_seen(skel); ASSERT_OK(system(ping_cmd), ping_cmd); ASSERT_EQ(skel->bss->seen_tc1, true, "seen_tc1"); @@ -1124,6 +1125,7 @@ static void test_tc_opts_prepend_target(int target) ASSERT_EQ(optq.prog_ids[3], id1, "prog_ids[3]"); ASSERT_EQ(optq.prog_ids[4], 0, "prog_ids[4]"); + tc_skel_reset_all_seen(skel); ASSERT_OK(system(ping_cmd), ping_cmd); ASSERT_EQ(skel->bss->seen_tc1, true, "seen_tc1"); @@ -1222,6 +1224,7 @@ static void test_tc_opts_append_target(int target) ASSERT_EQ(optq.prog_ids[1], id2, "prog_ids[1]"); ASSERT_EQ(optq.prog_ids[2], 0, "prog_ids[2]"); + tc_skel_reset_all_seen(skel); ASSERT_OK(system(ping_cmd), ping_cmd); ASSERT_EQ(skel->bss->seen_tc1, true, "seen_tc1"); @@ -1262,6 +1265,7 @@ static void test_tc_opts_append_target(int target) ASSERT_EQ(optq.prog_ids[3], id4, "prog_ids[3]"); ASSERT_EQ(optq.prog_ids[4], 0, "prog_ids[4]"); + tc_skel_reset_all_seen(skel); ASSERT_OK(system(ping_cmd), ping_cmd); ASSERT_EQ(skel->bss->seen_tc1, true, "seen_tc1"); @@ -2250,7 +2254,7 @@ static void test_tc_opts_delete_empty(int target, bool chain_tc_old) BPF_TC_INGRESS : BPF_TC_EGRESS; err = bpf_tc_hook_create(&tc_hook); ASSERT_OK(err, "bpf_tc_hook_create"); - __assert_mprog_count(target, 0, true, loopback); + assert_mprog_count(target, 0); } err = bpf_prog_detach_opts(0, loopback, target, &optd); ASSERT_EQ(err, -ENOENT, "prog_detach"); @@ -2316,16 +2320,13 @@ static void test_tc_chain_mixed(int target) assert_mprog_count(target, 1); + tc_skel_reset_all_seen(skel); ASSERT_OK(system(ping_cmd), ping_cmd); ASSERT_EQ(skel->bss->seen_tc4, false, "seen_tc4"); ASSERT_EQ(skel->bss->seen_tc5, false, "seen_tc5"); ASSERT_EQ(skel->bss->seen_tc6, true, "seen_tc6"); - skel->bss->seen_tc4 = false; - skel->bss->seen_tc5 = false; - skel->bss->seen_tc6 = false; - LIBBPF_OPTS_RESET(opta, .flags = BPF_F_REPLACE, .replace_prog_fd = fd3, @@ -2339,21 +2340,19 @@ static void test_tc_chain_mixed(int target) assert_mprog_count(target, 1); + tc_skel_reset_all_seen(skel); ASSERT_OK(system(ping_cmd), ping_cmd); ASSERT_EQ(skel->bss->seen_tc4, true, "seen_tc4"); ASSERT_EQ(skel->bss->seen_tc5, true, "seen_tc5"); ASSERT_EQ(skel->bss->seen_tc6, false, "seen_tc6"); - skel->bss->seen_tc4 = false; - skel->bss->seen_tc5 = false; - skel->bss->seen_tc6 = false; - cleanup_opts: err = bpf_prog_detach_opts(detach_fd, loopback, target, &optd); ASSERT_OK(err, "prog_detach"); - __assert_mprog_count(target, 0, true, loopback); + assert_mprog_count(target, 0); + tc_skel_reset_all_seen(skel); ASSERT_OK(system(ping_cmd), ping_cmd); ASSERT_EQ(skel->bss->seen_tc4, false, "seen_tc4"); @@ -2378,3 +2377,313 @@ void serial_test_tc_opts_chain_mixed(void) test_tc_chain_mixed(BPF_TCX_INGRESS); test_tc_chain_mixed(BPF_TCX_EGRESS); } + +static int generate_dummy_prog(void) +{ + const struct bpf_insn prog_insns[] = { + BPF_MOV64_IMM(BPF_REG_0, 0), + BPF_EXIT_INSN(), + }; + const size_t prog_insn_cnt = sizeof(prog_insns) / sizeof(struct bpf_insn); + LIBBPF_OPTS(bpf_prog_load_opts, opts); + const size_t log_buf_sz = 256; + char *log_buf; + int fd = -1; + + log_buf = malloc(log_buf_sz); + if (!ASSERT_OK_PTR(log_buf, "log_buf_alloc")) + return fd; + opts.log_buf = log_buf; + opts.log_size = log_buf_sz; + + log_buf[0] = '\0'; + opts.log_level = 0; + fd = bpf_prog_load(BPF_PROG_TYPE_SCHED_CLS, "tcx_prog", "GPL", + prog_insns, prog_insn_cnt, &opts); + ASSERT_STREQ(log_buf, "", "log_0"); + ASSERT_GE(fd, 0, "prog_fd"); + free(log_buf); + return fd; +} + +static void test_tc_opts_max_target(int target, int flags, bool relative) +{ + int err, ifindex, i, prog_fd, last_fd = -1; + LIBBPF_OPTS(bpf_prog_attach_opts, opta); + const int max_progs = 63; + + ASSERT_OK(system("ip link add dev tcx_opts1 type veth peer name tcx_opts2"), "add veth"); + ifindex = if_nametoindex("tcx_opts1"); + ASSERT_NEQ(ifindex, 0, "non_zero_ifindex"); + + assert_mprog_count_ifindex(ifindex, target, 0); + + for (i = 0; i < max_progs; i++) { + prog_fd = generate_dummy_prog(); + if (!ASSERT_GE(prog_fd, 0, "dummy_prog")) + goto cleanup; + err = bpf_prog_attach_opts(prog_fd, ifindex, target, &opta); + if (!ASSERT_EQ(err, 0, "prog_attach")) + goto cleanup; + assert_mprog_count_ifindex(ifindex, target, i + 1); + if (i == max_progs - 1 && relative) + last_fd = prog_fd; + else + close(prog_fd); + } + + prog_fd = generate_dummy_prog(); + if (!ASSERT_GE(prog_fd, 0, "dummy_prog")) + goto cleanup; + opta.flags = flags; + if (last_fd > 0) + opta.relative_fd = last_fd; + err = bpf_prog_attach_opts(prog_fd, ifindex, target, &opta); + ASSERT_EQ(err, -ERANGE, "prog_64_attach"); + assert_mprog_count_ifindex(ifindex, target, max_progs); + close(prog_fd); +cleanup: + if (last_fd > 0) + close(last_fd); + ASSERT_OK(system("ip link del dev tcx_opts1"), "del veth"); + ASSERT_EQ(if_nametoindex("tcx_opts1"), 0, "dev1_removed"); + ASSERT_EQ(if_nametoindex("tcx_opts2"), 0, "dev2_removed"); +} + +void serial_test_tc_opts_max(void) +{ + test_tc_opts_max_target(BPF_TCX_INGRESS, 0, false); + test_tc_opts_max_target(BPF_TCX_EGRESS, 0, false); + + test_tc_opts_max_target(BPF_TCX_INGRESS, BPF_F_BEFORE, false); + test_tc_opts_max_target(BPF_TCX_EGRESS, BPF_F_BEFORE, true); + + test_tc_opts_max_target(BPF_TCX_INGRESS, BPF_F_AFTER, true); + test_tc_opts_max_target(BPF_TCX_EGRESS, BPF_F_AFTER, false); +} + +static void test_tc_opts_query_target(int target) +{ + const size_t attr_size = offsetofend(union bpf_attr, query); + LIBBPF_OPTS(bpf_prog_attach_opts, opta); + LIBBPF_OPTS(bpf_prog_detach_opts, optd); + LIBBPF_OPTS(bpf_prog_query_opts, optq); + __u32 fd1, fd2, fd3, fd4, id1, id2, id3, id4; + struct test_tc_link *skel; + union bpf_attr attr; + __u32 prog_ids[5]; + int err; + + skel = test_tc_link__open_and_load(); + if (!ASSERT_OK_PTR(skel, "skel_load")) + goto cleanup; + + fd1 = bpf_program__fd(skel->progs.tc1); + fd2 = bpf_program__fd(skel->progs.tc2); + fd3 = bpf_program__fd(skel->progs.tc3); + fd4 = bpf_program__fd(skel->progs.tc4); + + id1 = id_from_prog_fd(fd1); + id2 = id_from_prog_fd(fd2); + id3 = id_from_prog_fd(fd3); + id4 = id_from_prog_fd(fd4); + + assert_mprog_count(target, 0); + + LIBBPF_OPTS_RESET(opta, + .expected_revision = 1, + ); + + err = bpf_prog_attach_opts(fd1, loopback, target, &opta); + if (!ASSERT_EQ(err, 0, "prog_attach")) + goto cleanup; + + assert_mprog_count(target, 1); + + LIBBPF_OPTS_RESET(opta, + .expected_revision = 2, + ); + + err = bpf_prog_attach_opts(fd2, loopback, target, &opta); + if (!ASSERT_EQ(err, 0, "prog_attach")) + goto cleanup1; + + assert_mprog_count(target, 2); + + LIBBPF_OPTS_RESET(opta, + .expected_revision = 3, + ); + + err = bpf_prog_attach_opts(fd3, loopback, target, &opta); + if (!ASSERT_EQ(err, 0, "prog_attach")) + goto cleanup2; + + assert_mprog_count(target, 3); + + LIBBPF_OPTS_RESET(opta, + .expected_revision = 4, + ); + + err = bpf_prog_attach_opts(fd4, loopback, target, &opta); + if (!ASSERT_EQ(err, 0, "prog_attach")) + goto cleanup3; + + assert_mprog_count(target, 4); + + /* Test 1: Double query via libbpf API */ + err = bpf_prog_query_opts(loopback, target, &optq); + if (!ASSERT_OK(err, "prog_query")) + goto cleanup4; + + ASSERT_EQ(optq.count, 4, "count"); + ASSERT_EQ(optq.revision, 5, "revision"); + ASSERT_EQ(optq.prog_ids, NULL, "prog_ids"); + ASSERT_EQ(optq.link_ids, NULL, "link_ids"); + + memset(prog_ids, 0, sizeof(prog_ids)); + optq.prog_ids = prog_ids; + + err = bpf_prog_query_opts(loopback, target, &optq); + if (!ASSERT_OK(err, "prog_query")) + goto cleanup4; + + ASSERT_EQ(optq.count, 4, "count"); + ASSERT_EQ(optq.revision, 5, "revision"); + ASSERT_EQ(optq.prog_ids[0], id1, "prog_ids[0]"); + ASSERT_EQ(optq.prog_ids[1], id2, "prog_ids[1]"); + ASSERT_EQ(optq.prog_ids[2], id3, "prog_ids[2]"); + ASSERT_EQ(optq.prog_ids[3], id4, "prog_ids[3]"); + ASSERT_EQ(optq.prog_ids[4], 0, "prog_ids[4]"); + ASSERT_EQ(optq.link_ids, NULL, "link_ids"); + + /* Test 2: Double query via bpf_attr & bpf(2) directly */ + memset(&attr, 0, attr_size); + attr.query.target_ifindex = loopback; + attr.query.attach_type = target; + + err = syscall(__NR_bpf, BPF_PROG_QUERY, &attr, attr_size); + if (!ASSERT_OK(err, "prog_query")) + goto cleanup4; + + ASSERT_EQ(attr.query.count, 4, "count"); + ASSERT_EQ(attr.query.revision, 5, "revision"); + ASSERT_EQ(attr.query.query_flags, 0, "query_flags"); + ASSERT_EQ(attr.query.attach_flags, 0, "attach_flags"); + ASSERT_EQ(attr.query.target_ifindex, loopback, "target_ifindex"); + ASSERT_EQ(attr.query.attach_type, target, "attach_type"); + ASSERT_EQ(attr.query.prog_ids, 0, "prog_ids"); + ASSERT_EQ(attr.query.prog_attach_flags, 0, "prog_attach_flags"); + ASSERT_EQ(attr.query.link_ids, 0, "link_ids"); + ASSERT_EQ(attr.query.link_attach_flags, 0, "link_attach_flags"); + + memset(prog_ids, 0, sizeof(prog_ids)); + attr.query.prog_ids = ptr_to_u64(prog_ids); + + err = syscall(__NR_bpf, BPF_PROG_QUERY, &attr, attr_size); + if (!ASSERT_OK(err, "prog_query")) + goto cleanup4; + + ASSERT_EQ(attr.query.count, 4, "count"); + ASSERT_EQ(attr.query.revision, 5, "revision"); + ASSERT_EQ(attr.query.query_flags, 0, "query_flags"); + ASSERT_EQ(attr.query.attach_flags, 0, "attach_flags"); + ASSERT_EQ(attr.query.target_ifindex, loopback, "target_ifindex"); + ASSERT_EQ(attr.query.attach_type, target, "attach_type"); + ASSERT_EQ(attr.query.prog_ids, ptr_to_u64(prog_ids), "prog_ids"); + ASSERT_EQ(prog_ids[0], id1, "prog_ids[0]"); + ASSERT_EQ(prog_ids[1], id2, "prog_ids[1]"); + ASSERT_EQ(prog_ids[2], id3, "prog_ids[2]"); + ASSERT_EQ(prog_ids[3], id4, "prog_ids[3]"); + ASSERT_EQ(prog_ids[4], 0, "prog_ids[4]"); + ASSERT_EQ(attr.query.prog_attach_flags, 0, "prog_attach_flags"); + ASSERT_EQ(attr.query.link_ids, 0, "link_ids"); + ASSERT_EQ(attr.query.link_attach_flags, 0, "link_attach_flags"); + +cleanup4: + err = bpf_prog_detach_opts(fd4, loopback, target, &optd); + ASSERT_OK(err, "prog_detach"); + assert_mprog_count(target, 3); + +cleanup3: + err = bpf_prog_detach_opts(fd3, loopback, target, &optd); + ASSERT_OK(err, "prog_detach"); + assert_mprog_count(target, 2); + +cleanup2: + err = bpf_prog_detach_opts(fd2, loopback, target, &optd); + ASSERT_OK(err, "prog_detach"); + assert_mprog_count(target, 1); + +cleanup1: + err = bpf_prog_detach_opts(fd1, loopback, target, &optd); + ASSERT_OK(err, "prog_detach"); + assert_mprog_count(target, 0); + +cleanup: + test_tc_link__destroy(skel); +} + +void serial_test_tc_opts_query(void) +{ + test_tc_opts_query_target(BPF_TCX_INGRESS); + test_tc_opts_query_target(BPF_TCX_EGRESS); +} + +static void test_tc_opts_query_attach_target(int target) +{ + LIBBPF_OPTS(bpf_prog_attach_opts, opta); + LIBBPF_OPTS(bpf_prog_detach_opts, optd); + LIBBPF_OPTS(bpf_prog_query_opts, optq); + struct test_tc_link *skel; + __u32 prog_ids[2]; + __u32 fd1, id1; + int err; + + skel = test_tc_link__open_and_load(); + if (!ASSERT_OK_PTR(skel, "skel_load")) + goto cleanup; + + fd1 = bpf_program__fd(skel->progs.tc1); + id1 = id_from_prog_fd(fd1); + + err = bpf_prog_query_opts(loopback, target, &optq); + if (!ASSERT_OK(err, "prog_query")) + goto cleanup; + + ASSERT_EQ(optq.count, 0, "count"); + ASSERT_EQ(optq.revision, 1, "revision"); + + LIBBPF_OPTS_RESET(opta, + .expected_revision = optq.revision, + ); + + err = bpf_prog_attach_opts(fd1, loopback, target, &opta); + if (!ASSERT_EQ(err, 0, "prog_attach")) + goto cleanup; + + memset(prog_ids, 0, sizeof(prog_ids)); + optq.prog_ids = prog_ids; + optq.count = ARRAY_SIZE(prog_ids); + + err = bpf_prog_query_opts(loopback, target, &optq); + if (!ASSERT_OK(err, "prog_query")) + goto cleanup1; + + ASSERT_EQ(optq.count, 1, "count"); + ASSERT_EQ(optq.revision, 2, "revision"); + ASSERT_EQ(optq.prog_ids[0], id1, "prog_ids[0]"); + ASSERT_EQ(optq.prog_ids[1], 0, "prog_ids[1]"); + +cleanup1: + err = bpf_prog_detach_opts(fd1, loopback, target, &optd); + ASSERT_OK(err, "prog_detach"); + assert_mprog_count(target, 0); +cleanup: + test_tc_link__destroy(skel); +} + +void serial_test_tc_opts_query_attach(void) +{ + test_tc_opts_query_attach_target(BPF_TCX_INGRESS); + test_tc_opts_query_attach_target(BPF_TCX_EGRESS); +} diff --git a/tools/testing/selftests/bpf/prog_tests/test_bpf_ma.c b/tools/testing/selftests/bpf/prog_tests/test_bpf_ma.c new file mode 100644 index 000000000000..0cca4e8ae38e --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/test_bpf_ma.c @@ -0,0 +1,50 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (C) 2023. Huawei Technologies Co., Ltd */ +#define _GNU_SOURCE +#include +#include +#include +#include +#include + +#include "test_bpf_ma.skel.h" + +void test_test_bpf_ma(void) +{ + struct test_bpf_ma *skel; + struct btf *btf; + int i, err; + + skel = test_bpf_ma__open(); + if (!ASSERT_OK_PTR(skel, "open")) + return; + + btf = bpf_object__btf(skel->obj); + if (!ASSERT_OK_PTR(btf, "btf")) + goto out; + + for (i = 0; i < ARRAY_SIZE(skel->rodata->data_sizes); i++) { + char name[32]; + int id; + + snprintf(name, sizeof(name), "bin_data_%u", skel->rodata->data_sizes[i]); + id = btf__find_by_name_kind(btf, name, BTF_KIND_STRUCT); + if (!ASSERT_GT(id, 0, "bin_data")) + goto out; + skel->rodata->data_btf_ids[i] = id; + } + + err = test_bpf_ma__load(skel); + if (!ASSERT_OK(err, "load")) + goto out; + + err = test_bpf_ma__attach(skel); + if (!ASSERT_OK(err, "attach")) + goto out; + + skel->bss->pid = getpid(); + usleep(1); + ASSERT_OK(skel->bss->err, "test error"); +out: + test_bpf_ma__destroy(skel); +} diff --git a/tools/testing/selftests/bpf/prog_tests/timer.c b/tools/testing/selftests/bpf/prog_tests/timer.c index 290c21dbe65a..ce2c61d62fc6 100644 --- a/tools/testing/selftests/bpf/prog_tests/timer.c +++ b/tools/testing/selftests/bpf/prog_tests/timer.c @@ -2,6 +2,7 @@ /* Copyright (c) 2021 Facebook */ #include #include "timer.skel.h" +#include "timer_failure.skel.h" static int timer(struct timer *timer_skel) { @@ -49,10 +50,11 @@ void serial_test_timer(void) timer_skel = timer__open_and_load(); if (!ASSERT_OK_PTR(timer_skel, "timer_skel_load")) - goto cleanup; + return; err = timer(timer_skel); ASSERT_OK(err, "timer"); -cleanup: timer__destroy(timer_skel); + + RUN_TESTS(timer_failure); } diff --git a/tools/testing/selftests/bpf/prog_tests/xdp_dev_bound_only.c b/tools/testing/selftests/bpf/prog_tests/xdp_dev_bound_only.c new file mode 100644 index 000000000000..7dd18c6d06c6 --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/xdp_dev_bound_only.c @@ -0,0 +1,61 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include + +#define LOCAL_NETNS "xdp_dev_bound_only_netns" + +static int load_dummy_prog(char *name, __u32 ifindex, __u32 flags) +{ + struct bpf_insn insns[] = { BPF_MOV64_IMM(BPF_REG_0, 0), BPF_EXIT_INSN() }; + LIBBPF_OPTS(bpf_prog_load_opts, opts); + + opts.prog_flags = flags; + opts.prog_ifindex = ifindex; + return bpf_prog_load(BPF_PROG_TYPE_XDP, name, "GPL", insns, ARRAY_SIZE(insns), &opts); +} + +/* A test case for bpf_offload_netdev->offload handling bug: + * - create a veth device (does not support offload); + * - create a device bound XDP program with BPF_F_XDP_DEV_BOUND_ONLY flag + * (such programs are not offloaded); + * - create a device bound XDP program without flags (such programs are offloaded). + * This might lead to 'BUG: kernel NULL pointer dereference'. + */ +void test_xdp_dev_bound_only_offdev(void) +{ + struct nstoken *tok = NULL; + __u32 ifindex; + int fd1 = -1; + int fd2 = -1; + + SYS(out, "ip netns add " LOCAL_NETNS); + tok = open_netns(LOCAL_NETNS); + if (!ASSERT_OK_PTR(tok, "open_netns")) + goto out; + SYS(out, "ip link add eth42 type veth"); + ifindex = if_nametoindex("eth42"); + if (!ASSERT_NEQ(ifindex, 0, "if_nametoindex")) { + perror("if_nametoindex"); + goto out; + } + fd1 = load_dummy_prog("dummy1", ifindex, BPF_F_XDP_DEV_BOUND_ONLY); + if (!ASSERT_GE(fd1, 0, "load_dummy_prog #1")) { + perror("load_dummy_prog #1"); + goto out; + } + /* Program with ifindex is considered offloaded, however veth + * does not support offload => error should be reported. + */ + fd2 = load_dummy_prog("dummy2", ifindex, 0); + ASSERT_EQ(fd2, -EINVAL, "load_dummy_prog #2 (offloaded)"); + +out: + close(fd1); + close(fd2); + close_netns(tok); + /* eth42 was added inside netns, removing the netns will + * also remove eth42 veth pair. + */ + SYS_NOFAIL("ip netns del " LOCAL_NETNS); +} diff --git a/tools/testing/selftests/bpf/progs/bpf_cubic.c b/tools/testing/selftests/bpf/progs/bpf_cubic.c index d9660e7200e2..c997e3e3d3fb 100644 --- a/tools/testing/selftests/bpf/progs/bpf_cubic.c +++ b/tools/testing/selftests/bpf/progs/bpf_cubic.c @@ -490,6 +490,8 @@ static __always_inline void hystart_update(struct sock *sk, __u32 delay) } } +int bpf_cubic_acked_called = 0; + void BPF_STRUCT_OPS(bpf_cubic_acked, struct sock *sk, const struct ack_sample *sample) { @@ -497,6 +499,7 @@ void BPF_STRUCT_OPS(bpf_cubic_acked, struct sock *sk, struct bictcp *ca = inet_csk_ca(sk); __u32 delay; + bpf_cubic_acked_called = 1; /* Some calls are for duplicates without timetamps */ if (sample->rtt_us < 0) return; diff --git a/tools/testing/selftests/bpf/progs/kprobe_multi_override.c b/tools/testing/selftests/bpf/progs/kprobe_multi_override.c new file mode 100644 index 000000000000..28f8487c9059 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/kprobe_multi_override.c @@ -0,0 +1,13 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include + +char _license[] SEC("license") = "GPL"; + +SEC("kprobe.multi") +int test_override(struct pt_regs *ctx) +{ + bpf_override_return(ctx, 123); + return 0; +} diff --git a/tools/testing/selftests/bpf/progs/test_bpf_ma.c b/tools/testing/selftests/bpf/progs/test_bpf_ma.c new file mode 100644 index 000000000000..ecde41ae0fc8 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/test_bpf_ma.c @@ -0,0 +1,123 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (C) 2023. Huawei Technologies Co., Ltd */ +#include +#include +#include + +#include "bpf_experimental.h" +#include "bpf_misc.h" + +#ifndef ARRAY_SIZE +#define ARRAY_SIZE(x) (sizeof(x) / sizeof((x)[0])) +#endif + +struct generic_map_value { + void *data; +}; + +char _license[] SEC("license") = "GPL"; + +const unsigned int data_sizes[] = {8, 16, 32, 64, 96, 128, 192, 256, 512, 1024, 2048, 4096}; +const volatile unsigned int data_btf_ids[ARRAY_SIZE(data_sizes)] = {}; + +int err = 0; +int pid = 0; + +#define DEFINE_ARRAY_WITH_KPTR(_size) \ + struct bin_data_##_size { \ + char data[_size - sizeof(void *)]; \ + }; \ + struct map_value_##_size { \ + struct bin_data_##_size __kptr * data; \ + /* To emit BTF info for bin_data_xx */ \ + struct bin_data_##_size not_used; \ + }; \ + struct { \ + __uint(type, BPF_MAP_TYPE_ARRAY); \ + __type(key, int); \ + __type(value, struct map_value_##_size); \ + __uint(max_entries, 128); \ + } array_##_size SEC(".maps"); + +static __always_inline void batch_alloc_free(struct bpf_map *map, unsigned int batch, + unsigned int idx) +{ + struct generic_map_value *value; + unsigned int i, key; + void *old, *new; + + for (i = 0; i < batch; i++) { + key = i; + value = bpf_map_lookup_elem(map, &key); + if (!value) { + err = 1; + return; + } + new = bpf_obj_new_impl(data_btf_ids[idx], NULL); + if (!new) { + err = 2; + return; + } + old = bpf_kptr_xchg(&value->data, new); + if (old) { + bpf_obj_drop(old); + err = 3; + return; + } + } + for (i = 0; i < batch; i++) { + key = i; + value = bpf_map_lookup_elem(map, &key); + if (!value) { + err = 4; + return; + } + old = bpf_kptr_xchg(&value->data, NULL); + if (!old) { + err = 5; + return; + } + bpf_obj_drop(old); + } +} + +#define CALL_BATCH_ALLOC_FREE(size, batch, idx) \ + batch_alloc_free((struct bpf_map *)(&array_##size), batch, idx) + +DEFINE_ARRAY_WITH_KPTR(8); +DEFINE_ARRAY_WITH_KPTR(16); +DEFINE_ARRAY_WITH_KPTR(32); +DEFINE_ARRAY_WITH_KPTR(64); +DEFINE_ARRAY_WITH_KPTR(96); +DEFINE_ARRAY_WITH_KPTR(128); +DEFINE_ARRAY_WITH_KPTR(192); +DEFINE_ARRAY_WITH_KPTR(256); +DEFINE_ARRAY_WITH_KPTR(512); +DEFINE_ARRAY_WITH_KPTR(1024); +DEFINE_ARRAY_WITH_KPTR(2048); +DEFINE_ARRAY_WITH_KPTR(4096); + +SEC("fentry/" SYS_PREFIX "sys_nanosleep") +int test_bpf_mem_alloc_free(void *ctx) +{ + if ((u32)bpf_get_current_pid_tgid() != pid) + return 0; + + /* Alloc 128 8-bytes objects in batch to trigger refilling, + * then free 128 8-bytes objects in batch to trigger freeing. + */ + CALL_BATCH_ALLOC_FREE(8, 128, 0); + CALL_BATCH_ALLOC_FREE(16, 128, 1); + CALL_BATCH_ALLOC_FREE(32, 128, 2); + CALL_BATCH_ALLOC_FREE(64, 128, 3); + CALL_BATCH_ALLOC_FREE(96, 128, 4); + CALL_BATCH_ALLOC_FREE(128, 128, 5); + CALL_BATCH_ALLOC_FREE(192, 128, 6); + CALL_BATCH_ALLOC_FREE(256, 128, 7); + CALL_BATCH_ALLOC_FREE(512, 64, 8); + CALL_BATCH_ALLOC_FREE(1024, 32, 9); + CALL_BATCH_ALLOC_FREE(2048, 16, 10); + CALL_BATCH_ALLOC_FREE(4096, 8, 11); + + return 0; +} diff --git a/tools/testing/selftests/bpf/progs/timer_failure.c b/tools/testing/selftests/bpf/progs/timer_failure.c new file mode 100644 index 000000000000..226d33b5a05c --- /dev/null +++ b/tools/testing/selftests/bpf/progs/timer_failure.c @@ -0,0 +1,47 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2023 Meta Platforms, Inc. and affiliates. */ + +#include +#include +#include +#include +#include "bpf_misc.h" +#include "bpf_tcp_helpers.h" + +char _license[] SEC("license") = "GPL"; + +struct elem { + struct bpf_timer t; +}; + +struct { + __uint(type, BPF_MAP_TYPE_ARRAY); + __uint(max_entries, 1); + __type(key, int); + __type(value, struct elem); +} timer_map SEC(".maps"); + +static int timer_cb_ret1(void *map, int *key, struct bpf_timer *timer) +{ + if (bpf_get_smp_processor_id() % 2) + return 1; + else + return 0; +} + +SEC("fentry/bpf_fentry_test1") +__failure __msg("should have been in (0x0; 0x0)") +int BPF_PROG2(test_ret_1, int, a) +{ + int key = 0; + struct bpf_timer *timer; + + timer = bpf_map_lookup_elem(&timer_map, &key); + if (timer) { + bpf_timer_init(timer, &timer_map, CLOCK_BOOTTIME); + bpf_timer_set_callback(timer, timer_cb_ret1); + bpf_timer_start(timer, 1000, 0); + } + + return 0; +} diff --git a/tools/testing/selftests/bpf/test_verifier.c b/tools/testing/selftests/bpf/test_verifier.c index 31f1c935cd07..98107e0452d3 100644 --- a/tools/testing/selftests/bpf/test_verifier.c +++ b/tools/testing/selftests/bpf/test_verifier.c @@ -1880,7 +1880,7 @@ int main(int argc, char **argv) } } - get_unpriv_disabled(); + unpriv_disabled = get_unpriv_disabled(); if (unpriv && unpriv_disabled) { printf("Cannot run as unprivileged user with sysctl %s.\n", UNPRIV_SYSCTL); diff --git a/tools/testing/selftests/fchmodat2/Makefile b/tools/testing/selftests/fchmodat2/Makefile index 20839f8e43f2..71ec34bf1501 100644 --- a/tools/testing/selftests/fchmodat2/Makefile +++ b/tools/testing/selftests/fchmodat2/Makefile @@ -1,6 +1,6 @@ # SPDX-License-Identifier: GPL-2.0-or-later -CFLAGS += -Wall -O2 -g -fsanitize=address -fsanitize=undefined $(KHDR_INCLUDES) +CFLAGS += -Wall -O2 -g -fsanitize=address -fsanitize=undefined -static-libasan $(KHDR_INCLUDES) TEST_GEN_PROGS := fchmodat2_test include ../lib.mk diff --git a/tools/testing/selftests/ftrace/ftracetest b/tools/testing/selftests/ftrace/ftracetest index cb5f18c06593..c778d4dcc17e 100755 --- a/tools/testing/selftests/ftrace/ftracetest +++ b/tools/testing/selftests/ftrace/ftracetest @@ -31,6 +31,9 @@ err_ret=1 # kselftest skip code is 4 err_skip=4 +# umount required +UMOUNT_DIR="" + # cgroup RT scheduling prevents chrt commands from succeeding, which # induces failures in test wakeup tests. Disable for the duration of # the tests. @@ -45,6 +48,9 @@ setup() { cleanup() { echo $sched_rt_runtime_orig > $sched_rt_runtime + if [ -n "${UMOUNT_DIR}" ]; then + umount ${UMOUNT_DIR} ||: + fi } errexit() { # message @@ -124,6 +130,7 @@ parse_opts() { # opts ;; --logdir|-l) LOG_DIR=$2 + LINK_PTR= shift 2 ;; *.tc) @@ -160,11 +167,13 @@ if [ -z "$TRACING_DIR" ]; then mount -t tracefs nodev /sys/kernel/tracing || errexit "Failed to mount /sys/kernel/tracing" TRACING_DIR="/sys/kernel/tracing" + UMOUNT_DIR=${TRACING_DIR} # If debugfs exists, then so does /sys/kernel/debug elif [ -d "/sys/kernel/debug" ]; then mount -t debugfs nodev /sys/kernel/debug || errexit "Failed to mount /sys/kernel/debug" TRACING_DIR="/sys/kernel/debug/tracing" + UMOUNT_DIR=${TRACING_DIR} else err_ret=$err_skip errexit "debugfs and tracefs are not configured in this kernel" @@ -181,7 +190,10 @@ fi TOP_DIR=`absdir $0` TEST_DIR=$TOP_DIR/test.d TEST_CASES=`find_testcases $TEST_DIR` -LOG_DIR=$TOP_DIR/logs/`date +%Y%m%d-%H%M%S`/ +LOG_TOP_DIR=$TOP_DIR/logs +LOG_DATE=`date +%Y%m%d-%H%M%S` +LOG_DIR=$LOG_TOP_DIR/$LOG_DATE/ +LINK_PTR=$LOG_TOP_DIR/latest KEEP_LOG=0 KTAP=0 DEBUG=0 @@ -207,6 +219,10 @@ else LOG_FILE=$LOG_DIR/ftracetest.log mkdir -p $LOG_DIR || errexit "Failed to make a log directory: $LOG_DIR" date > $LOG_FILE + if [ "x-$LINK_PTR" != "x-" ]; then + unlink $LINK_PTR + ln -fs $LOG_DATE $LINK_PTR + fi fi # Define text colors diff --git a/tools/testing/selftests/ftrace/test.d/instances/instance-event.tc b/tools/testing/selftests/ftrace/test.d/instances/instance-event.tc index 0eb47fbb3f44..42422e425107 100644 --- a/tools/testing/selftests/ftrace/test.d/instances/instance-event.tc +++ b/tools/testing/selftests/ftrace/test.d/instances/instance-event.tc @@ -39,7 +39,7 @@ instance_read() { instance_set() { while :; do - echo 1 > foo/events/sched/sched_switch + echo 1 > foo/events/sched/sched_switch/enable done 2> /dev/null } diff --git a/tools/testing/selftests/ftrace/test.d/kprobe/kprobe_non_uniq_symbol.tc b/tools/testing/selftests/ftrace/test.d/kprobe/kprobe_non_uniq_symbol.tc new file mode 100644 index 000000000000..bc9514428dba --- /dev/null +++ b/tools/testing/selftests/ftrace/test.d/kprobe/kprobe_non_uniq_symbol.tc @@ -0,0 +1,13 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0 +# description: Test failure of registering kprobe on non unique symbol +# requires: kprobe_events + +SYMBOL='name_show' + +# We skip this test on kernel where SYMBOL is unique or does not exist. +if [ "$(grep -c -E "[[:alnum:]]+ t ${SYMBOL}" /proc/kallsyms)" -le '1' ]; then + exit_unsupported +fi + +! echo "p:test_non_unique ${SYMBOL}" > kprobe_events diff --git a/tools/testing/selftests/ftrace/test.d/trigger/inter-event/trigger-synthetic-event-dynstring.tc b/tools/testing/selftests/ftrace/test.d/trigger/inter-event/trigger-synthetic-event-dynstring.tc index 213d890ed188..174376ddbc6c 100644 --- a/tools/testing/selftests/ftrace/test.d/trigger/inter-event/trigger-synthetic-event-dynstring.tc +++ b/tools/testing/selftests/ftrace/test.d/trigger/inter-event/trigger-synthetic-event-dynstring.tc @@ -1,7 +1,7 @@ #!/bin/sh # SPDX-License-Identifier: GPL-2.0 # description: event trigger - test inter-event histogram trigger trace action with dynamic string param -# requires: set_event synthetic_events events/sched/sched_process_exec/hist "char name[]' >> synthetic_events":README ping:program +# requires: set_event synthetic_events events/sched/sched_process_exec/hist "' >> synthetic_events":README ping:program fail() { #msg echo $1 diff --git a/tools/testing/selftests/ftrace/test.d/trigger/inter-event/trigger-synthetic_event_syntax_errors.tc b/tools/testing/selftests/ftrace/test.d/trigger/inter-event/trigger-synthetic_event_syntax_errors.tc index 955e3ceea44b..b927ee54c02d 100644 --- a/tools/testing/selftests/ftrace/test.d/trigger/inter-event/trigger-synthetic_event_syntax_errors.tc +++ b/tools/testing/selftests/ftrace/test.d/trigger/inter-event/trigger-synthetic_event_syntax_errors.tc @@ -1,7 +1,7 @@ #!/bin/sh # SPDX-License-Identifier: GPL-2.0 # description: event trigger - test synthetic_events syntax parser errors -# requires: synthetic_events error_log "char name[]' >> synthetic_events":README +# requires: synthetic_events error_log "' >> synthetic_events":README check_error() { # command-with-error-pos-by-^ ftrace_errlog_check 'synthetic_events' "$1" 'synthetic_events' diff --git a/tools/testing/selftests/kselftest/runner.sh b/tools/testing/selftests/kselftest/runner.sh index 261c73cab41b..cd2fb43eea61 100644 --- a/tools/testing/selftests/kselftest/runner.sh +++ b/tools/testing/selftests/kselftest/runner.sh @@ -36,7 +36,8 @@ tap_timeout() { # Make sure tests will time out if utility is available. if [ -x /usr/bin/timeout ] ; then - /usr/bin/timeout --foreground "$kselftest_timeout" $1 + /usr/bin/timeout --foreground "$kselftest_timeout" \ + /usr/bin/timeout "$kselftest_timeout" $1 else $1 fi diff --git a/tools/testing/selftests/kselftest_deps.sh b/tools/testing/selftests/kselftest_deps.sh index 4bc14d9e8ff1..de59cc8f03c3 100755 --- a/tools/testing/selftests/kselftest_deps.sh +++ b/tools/testing/selftests/kselftest_deps.sh @@ -46,11 +46,11 @@ fi print_targets=0 while getopts "p" arg; do - case $arg in - p) + case $arg in + p) print_targets=1 shift;; - esac + esac done if [ $# -eq 0 ] @@ -92,6 +92,10 @@ pass_cnt=0 # Get all TARGETS from selftests Makefile targets=$(grep -E "^TARGETS +|^TARGETS =" Makefile | cut -d "=" -f2) +# Initially, in LDLIBS related lines, the dep checker needs +# to ignore lines containing the following strings: +filter="\$(VAR_LDLIBS)\|pkg-config\|PKG_CONFIG\|IOURING_EXTRA_LIBS" + # Single test case if [ $# -eq 2 ] then @@ -100,6 +104,8 @@ then l1_test $test l2_test $test l3_test $test + l4_test $test + l5_test $test print_results $1 $2 exit $? @@ -113,7 +119,7 @@ fi # Append space at the end of the list to append more tests. l1_tests=$(grep -r --include=Makefile "^LDLIBS" | \ - grep -v "VAR_LDLIBS" | awk -F: '{print $1}') + grep -v "$filter" | awk -F: '{print $1}' | uniq) # Level 2: LDLIBS set dynamically. # @@ -126,7 +132,7 @@ l1_tests=$(grep -r --include=Makefile "^LDLIBS" | \ # Append space at the end of the list to append more tests. l2_tests=$(grep -r --include=Makefile ": LDLIBS" | \ - grep -v "VAR_LDLIBS" | awk -F: '{print $1}') + grep -v "$filter" | awk -F: '{print $1}' | uniq) # Level 3 # memfd and others use pkg-config to find mount and fuse libs @@ -138,11 +144,32 @@ l2_tests=$(grep -r --include=Makefile ": LDLIBS" | \ # VAR_LDLIBS := $(shell pkg-config fuse --libs 2>/dev/null) l3_tests=$(grep -r --include=Makefile "^VAR_LDLIBS" | \ - grep -v "pkg-config" | awk -F: '{print $1}') + grep -v "pkg-config\|PKG_CONFIG" | awk -F: '{print $1}' | uniq) -#echo $l1_tests -#echo $l2_1_tests -#echo $l3_tests +# Level 4 +# some tests may fall back to default using `|| echo -l` +# if pkg-config doesn't find the libs, instead of using VAR_LDLIBS +# as per level 3 checks. +# e.g: +# netfilter/Makefile +# LDLIBS += $(shell $(HOSTPKG_CONFIG) --libs libmnl 2>/dev/null || echo -lmnl) +l4_tests=$(grep -r --include=Makefile "^LDLIBS" | \ + grep "pkg-config\|PKG_CONFIG" | awk -F: '{print $1}' | uniq) + +# Level 5 +# some tests may use IOURING_EXTRA_LIBS to add extra libs to LDLIBS, +# which in turn may be defined in a sub-Makefile +# e.g.: +# mm/Makefile +# $(OUTPUT)/gup_longterm: LDLIBS += $(IOURING_EXTRA_LIBS) +l5_tests=$(grep -r --include=Makefile "LDLIBS +=.*\$(IOURING_EXTRA_LIBS)" | \ + awk -F: '{print $1}' | uniq) + +#echo l1_tests $l1_tests +#echo l2_tests $l2_tests +#echo l3_tests $l3_tests +#echo l4_tests $l4_tests +#echo l5_tests $l5_tests all_tests print_results $1 $2 @@ -164,24 +191,32 @@ all_tests() for test in $l3_tests; do l3_test $test done + + for test in $l4_tests; do + l4_test $test + done + + for test in $l5_tests; do + l5_test $test + done } # Use same parsing used for l1_tests and pick libraries this time. l1_test() { test_libs=$(grep --include=Makefile "^LDLIBS" $test | \ - grep -v "VAR_LDLIBS" | \ + grep -v "$filter" | \ sed -e 's/\:/ /' | \ sed -e 's/+/ /' | cut -d "=" -f 2) check_libs $test $test_libs } -# Use same parsing used for l2__tests and pick libraries this time. +# Use same parsing used for l2_tests and pick libraries this time. l2_test() { test_libs=$(grep --include=Makefile ": LDLIBS" $test | \ - grep -v "VAR_LDLIBS" | \ + grep -v "$filter" | \ sed -e 's/\:/ /' | sed -e 's/+/ /' | \ cut -d "=" -f 2) @@ -197,6 +232,24 @@ l3_test() check_libs $test $test_libs } +l4_test() +{ + test_libs=$(grep --include=Makefile "^VAR_LDLIBS\|^LDLIBS" $test | \ + grep "\(pkg-config\|PKG_CONFIG\).*|| echo " | \ + sed -e 's/.*|| echo //' | sed -e 's/)$//') + + check_libs $test $test_libs +} + +l5_test() +{ + tests=$(find $(dirname "$test") -type f -name "*.mk") + test_libs=$(grep "^IOURING_EXTRA_LIBS +\?=" $tests | \ + cut -d "=" -f 2) + + check_libs $test $test_libs +} + check_libs() { diff --git a/tools/testing/selftests/kvm/include/ucall_common.h b/tools/testing/selftests/kvm/include/ucall_common.h index 112bc1da732a..ce33d306c2cb 100644 --- a/tools/testing/selftests/kvm/include/ucall_common.h +++ b/tools/testing/selftests/kvm/include/ucall_common.h @@ -1,7 +1,5 @@ /* SPDX-License-Identifier: GPL-2.0-only */ /* - * tools/testing/selftests/kvm/include/kvm_util.h - * * Copyright (C) 2018, Google LLC. */ #ifndef SELFTEST_KVM_UCALL_COMMON_H diff --git a/tools/testing/selftests/kvm/include/x86_64/processor.h b/tools/testing/selftests/kvm/include/x86_64/processor.h index 4fd042112526..25bc61dac5fb 100644 --- a/tools/testing/selftests/kvm/include/x86_64/processor.h +++ b/tools/testing/selftests/kvm/include/x86_64/processor.h @@ -68,6 +68,12 @@ struct xstate { #define XFEATURE_MASK_OPMASK BIT_ULL(5) #define XFEATURE_MASK_ZMM_Hi256 BIT_ULL(6) #define XFEATURE_MASK_Hi16_ZMM BIT_ULL(7) +#define XFEATURE_MASK_PT BIT_ULL(8) +#define XFEATURE_MASK_PKRU BIT_ULL(9) +#define XFEATURE_MASK_PASID BIT_ULL(10) +#define XFEATURE_MASK_CET_USER BIT_ULL(11) +#define XFEATURE_MASK_CET_KERNEL BIT_ULL(12) +#define XFEATURE_MASK_LBR BIT_ULL(15) #define XFEATURE_MASK_XTILE_CFG BIT_ULL(17) #define XFEATURE_MASK_XTILE_DATA BIT_ULL(18) @@ -147,6 +153,7 @@ struct kvm_x86_cpu_feature { #define X86_FEATURE_CLWB KVM_X86_CPU_FEATURE(0x7, 0, EBX, 24) #define X86_FEATURE_UMIP KVM_X86_CPU_FEATURE(0x7, 0, ECX, 2) #define X86_FEATURE_PKU KVM_X86_CPU_FEATURE(0x7, 0, ECX, 3) +#define X86_FEATURE_OSPKE KVM_X86_CPU_FEATURE(0x7, 0, ECX, 4) #define X86_FEATURE_LA57 KVM_X86_CPU_FEATURE(0x7, 0, ECX, 16) #define X86_FEATURE_RDPID KVM_X86_CPU_FEATURE(0x7, 0, ECX, 22) #define X86_FEATURE_SGX_LC KVM_X86_CPU_FEATURE(0x7, 0, ECX, 30) @@ -553,6 +560,13 @@ static inline void xsetbv(u32 index, u64 value) __asm__ __volatile__("xsetbv" :: "a" (eax), "d" (edx), "c" (index)); } +static inline void wrpkru(u32 pkru) +{ + /* Note, ECX and EDX are architecturally required to be '0'. */ + asm volatile(".byte 0x0f,0x01,0xef\n\t" + : : "a" (pkru), "c"(0), "d"(0)); +} + static inline struct desc_ptr get_gdt(void) { struct desc_ptr gdt; @@ -908,6 +922,15 @@ static inline bool kvm_pmu_has(struct kvm_x86_pmu_feature feature) !kvm_cpu_has(feature.anti_feature); } +static __always_inline uint64_t kvm_cpu_supported_xcr0(void) +{ + if (!kvm_cpu_has_p(X86_PROPERTY_SUPPORTED_XCR0_LO)) + return 0; + + return kvm_cpu_property(X86_PROPERTY_SUPPORTED_XCR0_LO) | + ((uint64_t)kvm_cpu_property(X86_PROPERTY_SUPPORTED_XCR0_HI) << 32); +} + static inline size_t kvm_cpuid2_size(int nr_entries) { return sizeof(struct kvm_cpuid2) + diff --git a/tools/testing/selftests/kvm/lib/guest_sprintf.c b/tools/testing/selftests/kvm/lib/guest_sprintf.c index c4a69d8aeb68..74627514c4d4 100644 --- a/tools/testing/selftests/kvm/lib/guest_sprintf.c +++ b/tools/testing/selftests/kvm/lib/guest_sprintf.c @@ -200,6 +200,13 @@ repeat: ++fmt; } + /* + * Play nice with %llu, %llx, etc. KVM selftests only support + * 64-bit builds, so just treat %ll* the same as %l*. + */ + if (qualifier == 'l' && *fmt == 'l') + ++fmt; + /* default base */ base = 10; diff --git a/tools/testing/selftests/kvm/lib/test_util.c b/tools/testing/selftests/kvm/lib/test_util.c index 3e36019eeb4a..5d7f28b02d73 100644 --- a/tools/testing/selftests/kvm/lib/test_util.c +++ b/tools/testing/selftests/kvm/lib/test_util.c @@ -387,7 +387,7 @@ char *strdup_printf(const char *fmt, ...) char *str; va_start(ap, fmt); - vasprintf(&str, fmt, ap); + TEST_ASSERT(vasprintf(&str, fmt, ap) >= 0, "vasprintf() failed"); va_end(ap); return str; diff --git a/tools/testing/selftests/kvm/lib/x86_64/apic.c b/tools/testing/selftests/kvm/lib/x86_64/apic.c index 7168e25c194e..89153a333e83 100644 --- a/tools/testing/selftests/kvm/lib/x86_64/apic.c +++ b/tools/testing/selftests/kvm/lib/x86_64/apic.c @@ -1,7 +1,5 @@ // SPDX-License-Identifier: GPL-2.0-only /* - * tools/testing/selftests/kvm/lib/x86_64/processor.c - * * Copyright (C) 2021, Google LLC. */ diff --git a/tools/testing/selftests/kvm/memslot_perf_test.c b/tools/testing/selftests/kvm/memslot_perf_test.c index 20eb2e730800..8698d1ab60d0 100644 --- a/tools/testing/selftests/kvm/memslot_perf_test.c +++ b/tools/testing/selftests/kvm/memslot_perf_test.c @@ -1033,9 +1033,8 @@ static bool test_loop(const struct test_data *data, struct test_result *rbestruntime) { uint64_t maxslots; - struct test_result result; + struct test_result result = {}; - result.nloops = 0; if (!test_execute(targs->nslots, &maxslots, targs->seconds, data, &result.nloops, &result.slot_runtime, &result.guest_runtime)) { @@ -1089,7 +1088,7 @@ int main(int argc, char *argv[]) .seconds = 5, .runs = 1, }; - struct test_result rbestslottime; + struct test_result rbestslottime = {}; int tctr; if (!check_memory_sizes()) @@ -1098,11 +1097,10 @@ int main(int argc, char *argv[]) if (!parse_args(argc, argv, &targs)) return -1; - rbestslottime.slottimens = 0; for (tctr = targs.tfirst; tctr <= targs.tlast; tctr++) { const struct test_data *data = &tests[tctr]; unsigned int runctr; - struct test_result rbestruntime; + struct test_result rbestruntime = {}; if (tctr > targs.tfirst) pr_info("\n"); @@ -1110,7 +1108,6 @@ int main(int argc, char *argv[]) pr_info("Testing %s performance with %i runs, %d seconds each\n", data->name, targs.runs, targs.seconds); - rbestruntime.runtimens = 0; for (runctr = 0; runctr < targs.runs; runctr++) if (!test_loop(data, &targs, &rbestslottime, &rbestruntime)) diff --git a/tools/testing/selftests/kvm/riscv/get-reg-list.c b/tools/testing/selftests/kvm/riscv/get-reg-list.c index d8ecacd03ecf..9f99ea42f45f 100644 --- a/tools/testing/selftests/kvm/riscv/get-reg-list.c +++ b/tools/testing/selftests/kvm/riscv/get-reg-list.c @@ -12,19 +12,37 @@ #define REG_MASK (KVM_REG_ARCH_MASK | KVM_REG_SIZE_MASK) +static bool isa_ext_cant_disable[KVM_RISCV_ISA_EXT_MAX]; + bool filter_reg(__u64 reg) { + switch (reg & ~REG_MASK) { /* - * Some ISA extensions are optional and not present on all host, - * but they can't be disabled through ISA_EXT registers when present. - * So, to make life easy, just filtering out these kind of registers. + * Same set of ISA_EXT registers are not present on all host because + * ISA_EXT registers are visible to the KVM user space based on the + * ISA extensions available on the host. Also, disabling an ISA + * extension using corresponding ISA_EXT register does not affect + * the visibility of the ISA_EXT register itself. + * + * Based on above, we should filter-out all ISA_EXT registers. */ - switch (reg & ~REG_MASK) { + case KVM_REG_RISCV_ISA_EXT | KVM_RISCV_ISA_EXT_A: + case KVM_REG_RISCV_ISA_EXT | KVM_RISCV_ISA_EXT_C: + case KVM_REG_RISCV_ISA_EXT | KVM_RISCV_ISA_EXT_D: + case KVM_REG_RISCV_ISA_EXT | KVM_RISCV_ISA_EXT_F: + case KVM_REG_RISCV_ISA_EXT | KVM_RISCV_ISA_EXT_H: + case KVM_REG_RISCV_ISA_EXT | KVM_RISCV_ISA_EXT_I: + case KVM_REG_RISCV_ISA_EXT | KVM_RISCV_ISA_EXT_M: + case KVM_REG_RISCV_ISA_EXT | KVM_RISCV_ISA_EXT_SVPBMT: case KVM_REG_RISCV_ISA_EXT | KVM_RISCV_ISA_EXT_SSTC: case KVM_REG_RISCV_ISA_EXT | KVM_RISCV_ISA_EXT_SVINVAL: case KVM_REG_RISCV_ISA_EXT | KVM_RISCV_ISA_EXT_ZIHINTPAUSE: + case KVM_REG_RISCV_ISA_EXT | KVM_RISCV_ISA_EXT_ZICBOM: + case KVM_REG_RISCV_ISA_EXT | KVM_RISCV_ISA_EXT_ZICBOZ: case KVM_REG_RISCV_ISA_EXT | KVM_RISCV_ISA_EXT_ZBB: case KVM_REG_RISCV_ISA_EXT | KVM_RISCV_ISA_EXT_SSAIA: + case KVM_REG_RISCV_ISA_EXT | KVM_RISCV_ISA_EXT_V: + case KVM_REG_RISCV_ISA_EXT | KVM_RISCV_ISA_EXT_SVNAPOT: case KVM_REG_RISCV_ISA_EXT | KVM_RISCV_ISA_EXT_ZBA: case KVM_REG_RISCV_ISA_EXT | KVM_RISCV_ISA_EXT_ZBS: case KVM_REG_RISCV_ISA_EXT | KVM_RISCV_ISA_EXT_ZICNTR: @@ -32,6 +50,15 @@ bool filter_reg(__u64 reg) case KVM_REG_RISCV_ISA_EXT | KVM_RISCV_ISA_EXT_ZIFENCEI: case KVM_REG_RISCV_ISA_EXT | KVM_RISCV_ISA_EXT_ZIHPM: return true; + /* AIA registers are always available when Ssaia can't be disabled */ + case KVM_REG_RISCV_CSR | KVM_REG_RISCV_CSR_AIA | KVM_REG_RISCV_CSR_AIA_REG(siselect): + case KVM_REG_RISCV_CSR | KVM_REG_RISCV_CSR_AIA | KVM_REG_RISCV_CSR_AIA_REG(iprio1): + case KVM_REG_RISCV_CSR | KVM_REG_RISCV_CSR_AIA | KVM_REG_RISCV_CSR_AIA_REG(iprio2): + case KVM_REG_RISCV_CSR | KVM_REG_RISCV_CSR_AIA | KVM_REG_RISCV_CSR_AIA_REG(sieh): + case KVM_REG_RISCV_CSR | KVM_REG_RISCV_CSR_AIA | KVM_REG_RISCV_CSR_AIA_REG(siph): + case KVM_REG_RISCV_CSR | KVM_REG_RISCV_CSR_AIA | KVM_REG_RISCV_CSR_AIA_REG(iprio1h): + case KVM_REG_RISCV_CSR | KVM_REG_RISCV_CSR_AIA | KVM_REG_RISCV_CSR_AIA_REG(iprio2h): + return isa_ext_cant_disable[KVM_RISCV_ISA_EXT_SSAIA]; default: break; } @@ -50,24 +77,27 @@ static inline bool vcpu_has_ext(struct kvm_vcpu *vcpu, int ext) unsigned long value; ret = __vcpu_get_reg(vcpu, RISCV_ISA_EXT_REG(ext), &value); - if (ret) { - printf("Failed to get ext %d", ext); - return false; - } - - return !!value; + return (ret) ? false : !!value; } void finalize_vcpu(struct kvm_vcpu *vcpu, struct vcpu_reg_list *c) { + unsigned long isa_ext_state[KVM_RISCV_ISA_EXT_MAX] = { 0 }; struct vcpu_reg_sublist *s; + int rc; + + for (int i = 0; i < KVM_RISCV_ISA_EXT_MAX; i++) + __vcpu_get_reg(vcpu, RISCV_ISA_EXT_REG(i), &isa_ext_state[i]); /* * Disable all extensions which were enabled by default * if they were available in the risc-v host. */ - for (int i = 0; i < KVM_RISCV_ISA_EXT_MAX; i++) - __vcpu_set_reg(vcpu, RISCV_ISA_EXT_REG(i), 0); + for (int i = 0; i < KVM_RISCV_ISA_EXT_MAX; i++) { + rc = __vcpu_set_reg(vcpu, RISCV_ISA_EXT_REG(i), 0); + if (rc && isa_ext_state[i]) + isa_ext_cant_disable[i] = true; + } for_each_sublist(c, s) { if (!s->feature) @@ -506,10 +536,6 @@ static __u64 base_regs[] = { KVM_REG_RISCV | KVM_REG_SIZE_U64 | KVM_REG_RISCV_TIMER | KVM_REG_RISCV_TIMER_REG(time), KVM_REG_RISCV | KVM_REG_SIZE_U64 | KVM_REG_RISCV_TIMER | KVM_REG_RISCV_TIMER_REG(compare), KVM_REG_RISCV | KVM_REG_SIZE_U64 | KVM_REG_RISCV_TIMER | KVM_REG_RISCV_TIMER_REG(state), - KVM_REG_RISCV | KVM_REG_SIZE_ULONG | KVM_REG_RISCV_ISA_EXT | KVM_RISCV_ISA_EXT_A, - KVM_REG_RISCV | KVM_REG_SIZE_ULONG | KVM_REG_RISCV_ISA_EXT | KVM_RISCV_ISA_EXT_C, - KVM_REG_RISCV | KVM_REG_SIZE_ULONG | KVM_REG_RISCV_ISA_EXT | KVM_RISCV_ISA_EXT_I, - KVM_REG_RISCV | KVM_REG_SIZE_ULONG | KVM_REG_RISCV_ISA_EXT | KVM_RISCV_ISA_EXT_M, KVM_REG_RISCV | KVM_REG_SIZE_ULONG | KVM_REG_RISCV_SBI_EXT | KVM_REG_RISCV_SBI_SINGLE | KVM_RISCV_SBI_EXT_V01, KVM_REG_RISCV | KVM_REG_SIZE_ULONG | KVM_REG_RISCV_SBI_EXT | KVM_REG_RISCV_SBI_SINGLE | KVM_RISCV_SBI_EXT_TIME, KVM_REG_RISCV | KVM_REG_SIZE_ULONG | KVM_REG_RISCV_SBI_EXT | KVM_REG_RISCV_SBI_SINGLE | KVM_RISCV_SBI_EXT_IPI, diff --git a/tools/testing/selftests/kvm/x86_64/hyperv_svm_test.c b/tools/testing/selftests/kvm/x86_64/hyperv_svm_test.c index e446d76d1c0c..6c1278562090 100644 --- a/tools/testing/selftests/kvm/x86_64/hyperv_svm_test.c +++ b/tools/testing/selftests/kvm/x86_64/hyperv_svm_test.c @@ -1,7 +1,5 @@ // SPDX-License-Identifier: GPL-2.0-only /* - * KVM_GET/SET_* tests - * * Copyright (C) 2022, Red Hat, Inc. * * Tests for Hyper-V extensions to SVM. diff --git a/tools/testing/selftests/kvm/x86_64/nx_huge_pages_test.c b/tools/testing/selftests/kvm/x86_64/nx_huge_pages_test.c index 7f36c32fa760..18ac5c1952a3 100644 --- a/tools/testing/selftests/kvm/x86_64/nx_huge_pages_test.c +++ b/tools/testing/selftests/kvm/x86_64/nx_huge_pages_test.c @@ -1,7 +1,5 @@ // SPDX-License-Identifier: GPL-2.0-only /* - * tools/testing/selftests/kvm/nx_huge_page_test.c - * * Usage: to be run via nx_huge_page_test.sh, which does the necessary * environment setup and teardown * diff --git a/tools/testing/selftests/kvm/x86_64/nx_huge_pages_test.sh b/tools/testing/selftests/kvm/x86_64/nx_huge_pages_test.sh index 0560149e66ed..7cbb409801ee 100755 --- a/tools/testing/selftests/kvm/x86_64/nx_huge_pages_test.sh +++ b/tools/testing/selftests/kvm/x86_64/nx_huge_pages_test.sh @@ -4,7 +4,6 @@ # Wrapper script which performs setup and cleanup for nx_huge_pages_test. # Makes use of root privileges to set up huge pages and KVM module parameters. # -# tools/testing/selftests/kvm/nx_huge_page_test.sh # Copyright (C) 2022, Google LLC. set -e diff --git a/tools/testing/selftests/kvm/x86_64/state_test.c b/tools/testing/selftests/kvm/x86_64/state_test.c index 4c4925a8ab45..88b58aab7207 100644 --- a/tools/testing/selftests/kvm/x86_64/state_test.c +++ b/tools/testing/selftests/kvm/x86_64/state_test.c @@ -139,6 +139,83 @@ static void vmx_l1_guest_code(struct vmx_pages *vmx_pages) static void __attribute__((__flatten__)) guest_code(void *arg) { GUEST_SYNC(1); + + if (this_cpu_has(X86_FEATURE_XSAVE)) { + uint64_t supported_xcr0 = this_cpu_supported_xcr0(); + uint8_t buffer[4096]; + + memset(buffer, 0xcc, sizeof(buffer)); + + set_cr4(get_cr4() | X86_CR4_OSXSAVE); + GUEST_ASSERT(this_cpu_has(X86_FEATURE_OSXSAVE)); + + xsetbv(0, xgetbv(0) | supported_xcr0); + + /* + * Modify state for all supported xfeatures to take them out of + * their "init" state, i.e. to make them show up in XSTATE_BV. + * + * Note off-by-default features, e.g. AMX, are out of scope for + * this particular testcase as they have a different ABI. + */ + GUEST_ASSERT(supported_xcr0 & XFEATURE_MASK_FP); + asm volatile ("fincstp"); + + GUEST_ASSERT(supported_xcr0 & XFEATURE_MASK_SSE); + asm volatile ("vmovdqu %0, %%xmm0" :: "m" (buffer)); + + if (supported_xcr0 & XFEATURE_MASK_YMM) + asm volatile ("vmovdqu %0, %%ymm0" :: "m" (buffer)); + + if (supported_xcr0 & XFEATURE_MASK_AVX512) { + asm volatile ("kmovq %0, %%k1" :: "r" (-1ull)); + asm volatile ("vmovupd %0, %%zmm0" :: "m" (buffer)); + asm volatile ("vmovupd %0, %%zmm16" :: "m" (buffer)); + } + + if (this_cpu_has(X86_FEATURE_MPX)) { + uint64_t bounds[2] = { 10, 0xffffffffull }; + uint64_t output[2] = { }; + + GUEST_ASSERT(supported_xcr0 & XFEATURE_MASK_BNDREGS); + GUEST_ASSERT(supported_xcr0 & XFEATURE_MASK_BNDCSR); + + /* + * Don't bother trying to get BNDCSR into the INUSE + * state. MSR_IA32_BNDCFGS doesn't count as it isn't + * managed via XSAVE/XRSTOR, and BNDCFGU can only be + * modified by XRSTOR. Stuffing XSTATE_BV in the host + * is simpler than doing XRSTOR here in the guest. + * + * However, temporarily enable MPX in BNDCFGS so that + * BNDMOV actually loads BND1. If MPX isn't *fully* + * enabled, all MPX instructions are treated as NOPs. + * + * Hand encode "bndmov (%rax),%bnd1" as support for MPX + * mnemonics/registers has been removed from gcc and + * clang (and was never fully supported by clang). + */ + wrmsr(MSR_IA32_BNDCFGS, BIT_ULL(0)); + asm volatile (".byte 0x66,0x0f,0x1a,0x08" :: "a" (bounds)); + /* + * Hand encode "bndmov %bnd1, (%rax)" to sanity check + * that BND1 actually got loaded. + */ + asm volatile (".byte 0x66,0x0f,0x1b,0x08" :: "a" (output)); + wrmsr(MSR_IA32_BNDCFGS, 0); + + GUEST_ASSERT_EQ(bounds[0], output[0]); + GUEST_ASSERT_EQ(bounds[1], output[1]); + } + if (this_cpu_has(X86_FEATURE_PKU)) { + GUEST_ASSERT(supported_xcr0 & XFEATURE_MASK_PKRU); + set_cr4(get_cr4() | X86_CR4_PKE); + GUEST_ASSERT(this_cpu_has(X86_FEATURE_OSPKE)); + + wrpkru(-1u); + } + } + GUEST_SYNC(2); if (arg) { @@ -153,10 +230,11 @@ static void __attribute__((__flatten__)) guest_code(void *arg) int main(int argc, char *argv[]) { + uint64_t *xstate_bv, saved_xstate_bv; vm_vaddr_t nested_gva = 0; - + struct kvm_cpuid2 empty_cpuid = {}; struct kvm_regs regs1, regs2; - struct kvm_vcpu *vcpu; + struct kvm_vcpu *vcpu, *vcpuN; struct kvm_vm *vm; struct kvm_x86_state *state; struct ucall uc; @@ -209,6 +287,34 @@ int main(int argc, char *argv[]) /* Restore state in a new VM. */ vcpu = vm_recreate_with_one_vcpu(vm); vcpu_load_state(vcpu, state); + + /* + * Restore XSAVE state in a dummy vCPU, first without doing + * KVM_SET_CPUID2, and then with an empty guest CPUID. Except + * for off-by-default xfeatures, e.g. AMX, KVM is supposed to + * allow KVM_SET_XSAVE regardless of guest CPUID. Manually + * load only XSAVE state, MSRs in particular have a much more + * convoluted ABI. + * + * Load two versions of XSAVE state: one with the actual guest + * XSAVE state, and one with all supported features forced "on" + * in xstate_bv, e.g. to ensure that KVM allows loading all + * supported features, even if something goes awry in saving + * the original snapshot. + */ + xstate_bv = (void *)&((uint8_t *)state->xsave->region)[512]; + saved_xstate_bv = *xstate_bv; + + vcpuN = __vm_vcpu_add(vm, vcpu->id + 1); + vcpu_xsave_set(vcpuN, state->xsave); + *xstate_bv = kvm_cpu_supported_xcr0(); + vcpu_xsave_set(vcpuN, state->xsave); + + vcpu_init_cpuid(vcpuN, &empty_cpuid); + vcpu_xsave_set(vcpuN, state->xsave); + *xstate_bv = saved_xstate_bv; + vcpu_xsave_set(vcpuN, state->xsave); + kvm_x86_state_cleanup(state); memset(®s2, 0, sizeof(regs2)); diff --git a/tools/testing/selftests/kvm/x86_64/tsc_scaling_sync.c b/tools/testing/selftests/kvm/x86_64/tsc_scaling_sync.c index 5b669818e39a..59c7304f805e 100644 --- a/tools/testing/selftests/kvm/x86_64/tsc_scaling_sync.c +++ b/tools/testing/selftests/kvm/x86_64/tsc_scaling_sync.c @@ -1,10 +1,6 @@ // SPDX-License-Identifier: GPL-2.0-only /* - * svm_vmcall_test - * * Copyright © 2021 Amazon.com, Inc. or its affiliates. - * - * Xen shared_info / pvclock testing */ #include "test_util.h" diff --git a/tools/testing/selftests/kvm/x86_64/xen_shinfo_test.c b/tools/testing/selftests/kvm/x86_64/xen_shinfo_test.c index 05898ad9f4d9..9ec9ab60b63e 100644 --- a/tools/testing/selftests/kvm/x86_64/xen_shinfo_test.c +++ b/tools/testing/selftests/kvm/x86_64/xen_shinfo_test.c @@ -1,10 +1,6 @@ // SPDX-License-Identifier: GPL-2.0-only /* - * svm_vmcall_test - * * Copyright © 2021 Amazon.com, Inc. or its affiliates. - * - * Xen shared_info / pvclock testing */ #include "test_util.h" diff --git a/tools/testing/selftests/lib.mk b/tools/testing/selftests/lib.mk index d17854285f2b..118e0964bda9 100644 --- a/tools/testing/selftests/lib.mk +++ b/tools/testing/selftests/lib.mk @@ -106,7 +106,7 @@ endef run_tests: all ifdef building_out_of_srctree @if [ "X$(TEST_PROGS)$(TEST_PROGS_EXTENDED)$(TEST_FILES)" != "X" ]; then \ - rsync -aLq $(TEST_PROGS) $(TEST_PROGS_EXTENDED) $(TEST_FILES) $(OUTPUT); \ + rsync -aq --copy-unsafe-links $(TEST_PROGS) $(TEST_PROGS_EXTENDED) $(TEST_FILES) $(OUTPUT); \ fi @if [ "X$(TEST_PROGS)" != "X" ]; then \ $(call RUN_TESTS, $(TEST_GEN_PROGS) $(TEST_CUSTOM_PROGS) \ @@ -120,7 +120,7 @@ endif define INSTALL_SINGLE_RULE $(if $(INSTALL_LIST),@mkdir -p $(INSTALL_PATH)) - $(if $(INSTALL_LIST),rsync -aL $(INSTALL_LIST) $(INSTALL_PATH)/) + $(if $(INSTALL_LIST),rsync -a --copy-unsafe-links $(INSTALL_LIST) $(INSTALL_PATH)/) endef define INSTALL_RULE diff --git a/tools/testing/selftests/mm/charge_reserved_hugetlb.sh b/tools/testing/selftests/mm/charge_reserved_hugetlb.sh index a5cb4b09a46c..0899019a7fcb 100755 --- a/tools/testing/selftests/mm/charge_reserved_hugetlb.sh +++ b/tools/testing/selftests/mm/charge_reserved_hugetlb.sh @@ -25,7 +25,7 @@ if [[ "$1" == "-cgroup-v2" ]]; then fi if [[ $cgroup2 ]]; then - cgroup_path=$(mount -t cgroup2 | head -1 | awk -e '{print $3}') + cgroup_path=$(mount -t cgroup2 | head -1 | awk '{print $3}') if [[ -z "$cgroup_path" ]]; then cgroup_path=/dev/cgroup/memory mount -t cgroup2 none $cgroup_path @@ -33,7 +33,7 @@ if [[ $cgroup2 ]]; then fi echo "+hugetlb" >$cgroup_path/cgroup.subtree_control else - cgroup_path=$(mount -t cgroup | grep ",hugetlb" | awk -e '{print $3}') + cgroup_path=$(mount -t cgroup | grep ",hugetlb" | awk '{print $3}') if [[ -z "$cgroup_path" ]]; then cgroup_path=/dev/cgroup/memory mount -t cgroup memory,hugetlb $cgroup_path diff --git a/tools/testing/selftests/mm/hugetlb_reparenting_test.sh b/tools/testing/selftests/mm/hugetlb_reparenting_test.sh index bf2d2a684edf..14d26075c863 100755 --- a/tools/testing/selftests/mm/hugetlb_reparenting_test.sh +++ b/tools/testing/selftests/mm/hugetlb_reparenting_test.sh @@ -20,7 +20,7 @@ fi if [[ $cgroup2 ]]; then - CGROUP_ROOT=$(mount -t cgroup2 | head -1 | awk -e '{print $3}') + CGROUP_ROOT=$(mount -t cgroup2 | head -1 | awk '{print $3}') if [[ -z "$CGROUP_ROOT" ]]; then CGROUP_ROOT=/dev/cgroup/memory mount -t cgroup2 none $CGROUP_ROOT @@ -28,7 +28,7 @@ if [[ $cgroup2 ]]; then fi echo "+hugetlb +memory" >$CGROUP_ROOT/cgroup.subtree_control else - CGROUP_ROOT=$(mount -t cgroup | grep ",hugetlb" | awk -e '{print $3}') + CGROUP_ROOT=$(mount -t cgroup | grep ",hugetlb" | awk '{print $3}') if [[ -z "$CGROUP_ROOT" ]]; then CGROUP_ROOT=/dev/cgroup/memory mount -t cgroup memory,hugetlb $CGROUP_ROOT diff --git a/tools/testing/selftests/net/Makefile b/tools/testing/selftests/net/Makefile index 8b017070960d..4a2881d43989 100644 --- a/tools/testing/selftests/net/Makefile +++ b/tools/testing/selftests/net/Makefile @@ -34,6 +34,7 @@ TEST_PROGS += gro.sh TEST_PROGS += gre_gso.sh TEST_PROGS += cmsg_so_mark.sh TEST_PROGS += cmsg_time.sh cmsg_ipv6.sh +TEST_PROGS += netns-name.sh TEST_PROGS += srv6_end_dt46_l3vpn_test.sh TEST_PROGS += srv6_end_dt4_l3vpn_test.sh TEST_PROGS += srv6_end_dt6_l3vpn_test.sh diff --git a/tools/testing/selftests/net/bind_bhash.sh b/tools/testing/selftests/net/bind_bhash.sh index ca0292d4b441..a28563bdaae0 100755 --- a/tools/testing/selftests/net/bind_bhash.sh +++ b/tools/testing/selftests/net/bind_bhash.sh @@ -2,7 +2,7 @@ # SPDX-License-Identifier: GPL-2.0 NR_FILES=32768 -SAVED_NR_FILES=$(ulimit -n) +readonly NETNS="ns-$(mktemp -u XXXXXX)" # default values port=443 @@ -36,21 +36,21 @@ while getopts "ha:p:64" opt; do done setup() { + ip netns add "${NETNS}" + ip -netns "${NETNS}" link add veth0 type veth peer name veth1 + ip -netns "${NETNS}" link set lo up + ip -netns "${NETNS}" link set veth0 up + ip -netns "${NETNS}" link set veth1 up + if [[ "$use_v6" == true ]]; then - ip addr add $addr_v6 nodad dev eth0 + ip -netns "${NETNS}" addr add $addr_v6 nodad dev veth0 else - ip addr add $addr_v4 dev lo + ip -netns "${NETNS}" addr add $addr_v4 dev lo fi - ulimit -n $NR_FILES } cleanup() { - if [[ "$use_v6" == true ]]; then - ip addr del $addr_v6 dev eth0 - else - ip addr del $addr_v4/32 dev lo - fi - ulimit -n $SAVED_NR_FILES + ip netns del "${NETNS}" } if [[ "$addr" != "" ]]; then @@ -59,8 +59,10 @@ if [[ "$addr" != "" ]]; then fi setup if [[ "$use_v6" == true ]] ; then - ./bind_bhash $port "ipv6" $addr_v6 + ip netns exec "${NETNS}" sh -c \ + "ulimit -n ${NR_FILES};./bind_bhash ${port} ipv6 ${addr_v6}" else - ./bind_bhash $port "ipv4" $addr_v4 + ip netns exec "${NETNS}" sh -c \ + "ulimit -n ${NR_FILES};./bind_bhash ${port} ipv4 ${addr_v4}" fi cleanup diff --git a/tools/testing/selftests/net/bind_wildcard.c b/tools/testing/selftests/net/bind_wildcard.c index 58edfc15d28b..a2662348cdb1 100644 --- a/tools/testing/selftests/net/bind_wildcard.c +++ b/tools/testing/selftests/net/bind_wildcard.c @@ -6,41 +6,91 @@ #include "../kselftest_harness.h" +struct in6_addr in6addr_v4mapped_any = { + .s6_addr = { + 0, 0, 0, 0, + 0, 0, 0, 0, + 0, 0, 255, 255, + 0, 0, 0, 0 + } +}; + +struct in6_addr in6addr_v4mapped_loopback = { + .s6_addr = { + 0, 0, 0, 0, + 0, 0, 0, 0, + 0, 0, 255, 255, + 127, 0, 0, 1 + } +}; + FIXTURE(bind_wildcard) { struct sockaddr_in addr4; struct sockaddr_in6 addr6; - int expected_errno; }; FIXTURE_VARIANT(bind_wildcard) { const __u32 addr4_const; const struct in6_addr *addr6_const; + int expected_errno; }; FIXTURE_VARIANT_ADD(bind_wildcard, v4_any_v6_any) { .addr4_const = INADDR_ANY, .addr6_const = &in6addr_any, + .expected_errno = EADDRINUSE, }; FIXTURE_VARIANT_ADD(bind_wildcard, v4_any_v6_local) { .addr4_const = INADDR_ANY, .addr6_const = &in6addr_loopback, + .expected_errno = 0, +}; + +FIXTURE_VARIANT_ADD(bind_wildcard, v4_any_v6_v4mapped_any) +{ + .addr4_const = INADDR_ANY, + .addr6_const = &in6addr_v4mapped_any, + .expected_errno = EADDRINUSE, +}; + +FIXTURE_VARIANT_ADD(bind_wildcard, v4_any_v6_v4mapped_local) +{ + .addr4_const = INADDR_ANY, + .addr6_const = &in6addr_v4mapped_loopback, + .expected_errno = EADDRINUSE, }; FIXTURE_VARIANT_ADD(bind_wildcard, v4_local_v6_any) { .addr4_const = INADDR_LOOPBACK, .addr6_const = &in6addr_any, + .expected_errno = EADDRINUSE, }; FIXTURE_VARIANT_ADD(bind_wildcard, v4_local_v6_local) { .addr4_const = INADDR_LOOPBACK, .addr6_const = &in6addr_loopback, + .expected_errno = 0, +}; + +FIXTURE_VARIANT_ADD(bind_wildcard, v4_local_v6_v4mapped_any) +{ + .addr4_const = INADDR_LOOPBACK, + .addr6_const = &in6addr_v4mapped_any, + .expected_errno = EADDRINUSE, +}; + +FIXTURE_VARIANT_ADD(bind_wildcard, v4_local_v6_v4mapped_local) +{ + .addr4_const = INADDR_LOOPBACK, + .addr6_const = &in6addr_v4mapped_loopback, + .expected_errno = EADDRINUSE, }; FIXTURE_SETUP(bind_wildcard) @@ -52,11 +102,6 @@ FIXTURE_SETUP(bind_wildcard) self->addr6.sin6_family = AF_INET6; self->addr6.sin6_port = htons(0); self->addr6.sin6_addr = *variant->addr6_const; - - if (variant->addr6_const == &in6addr_any) - self->expected_errno = EADDRINUSE; - else - self->expected_errno = 0; } FIXTURE_TEARDOWN(bind_wildcard) @@ -65,6 +110,7 @@ FIXTURE_TEARDOWN(bind_wildcard) void bind_sockets(struct __test_metadata *_metadata, FIXTURE_DATA(bind_wildcard) *self, + int expected_errno, struct sockaddr *addr1, socklen_t addrlen1, struct sockaddr *addr2, socklen_t addrlen2) { @@ -86,9 +132,9 @@ void bind_sockets(struct __test_metadata *_metadata, ASSERT_GT(fd[1], 0); ret = bind(fd[1], addr2, addrlen2); - if (self->expected_errno) { + if (expected_errno) { ASSERT_EQ(ret, -1); - ASSERT_EQ(errno, self->expected_errno); + ASSERT_EQ(errno, expected_errno); } else { ASSERT_EQ(ret, 0); } @@ -99,14 +145,14 @@ void bind_sockets(struct __test_metadata *_metadata, TEST_F(bind_wildcard, v4_v6) { - bind_sockets(_metadata, self, - (struct sockaddr *)&self->addr4, sizeof(self->addr6), + bind_sockets(_metadata, self, variant->expected_errno, + (struct sockaddr *)&self->addr4, sizeof(self->addr4), (struct sockaddr *)&self->addr6, sizeof(self->addr6)); } TEST_F(bind_wildcard, v6_v4) { - bind_sockets(_metadata, self, + bind_sockets(_metadata, self, variant->expected_errno, (struct sockaddr *)&self->addr6, sizeof(self->addr6), (struct sockaddr *)&self->addr4, sizeof(self->addr4)); } diff --git a/tools/testing/selftests/net/fib_tests.sh b/tools/testing/selftests/net/fib_tests.sh index e7d2a530618a..66d0db7a2614 100755 --- a/tools/testing/selftests/net/fib_tests.sh +++ b/tools/testing/selftests/net/fib_tests.sh @@ -2437,6 +2437,9 @@ ipv4_mpath_list_test() run_cmd "ip -n ns2 route add 203.0.113.0/24 nexthop via 172.16.201.2 nexthop via 172.16.202.2" run_cmd "ip netns exec ns2 sysctl -qw net.ipv4.fib_multipath_hash_policy=1" + run_cmd "ip netns exec ns2 sysctl -qw net.ipv4.conf.veth2.rp_filter=0" + run_cmd "ip netns exec ns2 sysctl -qw net.ipv4.conf.all.rp_filter=0" + run_cmd "ip netns exec ns2 sysctl -qw net.ipv4.conf.default.rp_filter=0" set +e local dmac=$(ip -n ns2 -j link show dev veth2 | jq -r '.[]["address"]') @@ -2449,7 +2452,7 @@ ipv4_mpath_list_test() # words, the FIB lookup tracepoint needs to be triggered for every # packet. local t0_rx_pkts=$(link_stats_get ns2 veth2 rx packets) - run_cmd "perf stat -e fib:fib_table_lookup --filter 'err == 0' -j -o $tmp_file -- $cmd" + run_cmd "perf stat -a -e fib:fib_table_lookup --filter 'err == 0' -j -o $tmp_file -- $cmd" local t1_rx_pkts=$(link_stats_get ns2 veth2 rx packets) local diff=$(echo $t1_rx_pkts - $t0_rx_pkts | bc -l) list_rcv_eval $tmp_file $diff @@ -2494,7 +2497,7 @@ ipv6_mpath_list_test() # words, the FIB lookup tracepoint needs to be triggered for every # packet. local t0_rx_pkts=$(link_stats_get ns2 veth2 rx packets) - run_cmd "perf stat -e fib6:fib6_table_lookup --filter 'err == 0' -j -o $tmp_file -- $cmd" + run_cmd "perf stat -a -e fib6:fib6_table_lookup --filter 'err == 0' -j -o $tmp_file -- $cmd" local t1_rx_pkts=$(link_stats_get ns2 veth2 rx packets) local diff=$(echo $t1_rx_pkts - $t0_rx_pkts | bc -l) list_rcv_eval $tmp_file $diff diff --git a/tools/testing/selftests/net/hsr/hsr_ping.sh b/tools/testing/selftests/net/hsr/hsr_ping.sh index df9143538708..1c6457e54625 100755 --- a/tools/testing/selftests/net/hsr/hsr_ping.sh +++ b/tools/testing/selftests/net/hsr/hsr_ping.sh @@ -41,61 +41,6 @@ cleanup() done } -ip -Version > /dev/null 2>&1 -if [ $? -ne 0 ];then - echo "SKIP: Could not run test without ip tool" - exit $ksft_skip -fi - -trap cleanup EXIT - -for i in "$ns1" "$ns2" "$ns3" ;do - ip netns add $i || exit $ksft_skip - ip -net $i link set lo up -done - -echo "INFO: preparing interfaces." -# Three HSR nodes. Each node has one link to each of its neighbour, two links in total. -# -# ns1eth1 ----- ns2eth1 -# hsr1 hsr2 -# ns1eth2 ns2eth2 -# | | -# ns3eth1 ns3eth2 -# \ / -# hsr3 -# -# Interfaces -ip link add ns1eth1 netns "$ns1" type veth peer name ns2eth1 netns "$ns2" -ip link add ns1eth2 netns "$ns1" type veth peer name ns3eth1 netns "$ns3" -ip link add ns3eth2 netns "$ns3" type veth peer name ns2eth2 netns "$ns2" - -# HSRv0. -ip -net "$ns1" link add name hsr1 type hsr slave1 ns1eth1 slave2 ns1eth2 supervision 45 version 0 proto 0 -ip -net "$ns2" link add name hsr2 type hsr slave1 ns2eth1 slave2 ns2eth2 supervision 45 version 0 proto 0 -ip -net "$ns3" link add name hsr3 type hsr slave1 ns3eth1 slave2 ns3eth2 supervision 45 version 0 proto 0 - -# IP for HSR -ip -net "$ns1" addr add 100.64.0.1/24 dev hsr1 -ip -net "$ns1" addr add dead:beef:1::1/64 dev hsr1 nodad -ip -net "$ns2" addr add 100.64.0.2/24 dev hsr2 -ip -net "$ns2" addr add dead:beef:1::2/64 dev hsr2 nodad -ip -net "$ns3" addr add 100.64.0.3/24 dev hsr3 -ip -net "$ns3" addr add dead:beef:1::3/64 dev hsr3 nodad - -# All Links up -ip -net "$ns1" link set ns1eth1 up -ip -net "$ns1" link set ns1eth2 up -ip -net "$ns1" link set hsr1 up - -ip -net "$ns2" link set ns2eth1 up -ip -net "$ns2" link set ns2eth2 up -ip -net "$ns2" link set hsr2 up - -ip -net "$ns3" link set ns3eth1 up -ip -net "$ns3" link set ns3eth2 up -ip -net "$ns3" link set hsr3 up - # $1: IP address is_v6() { @@ -164,93 +109,168 @@ stop_if_error() fi } - -echo "INFO: Initial validation ping." -# Each node has to be able each one. -do_ping "$ns1" 100.64.0.2 -do_ping "$ns2" 100.64.0.1 -do_ping "$ns3" 100.64.0.1 -stop_if_error "Initial validation failed." - -do_ping "$ns1" 100.64.0.3 -do_ping "$ns2" 100.64.0.3 -do_ping "$ns3" 100.64.0.2 - -do_ping "$ns1" dead:beef:1::2 -do_ping "$ns1" dead:beef:1::3 -do_ping "$ns2" dead:beef:1::1 -do_ping "$ns2" dead:beef:1::2 -do_ping "$ns3" dead:beef:1::1 -do_ping "$ns3" dead:beef:1::2 - -stop_if_error "Initial validation failed." +do_complete_ping_test() +{ + echo "INFO: Initial validation ping." + # Each node has to be able each one. + do_ping "$ns1" 100.64.0.2 + do_ping "$ns2" 100.64.0.1 + do_ping "$ns3" 100.64.0.1 + stop_if_error "Initial validation failed." + + do_ping "$ns1" 100.64.0.3 + do_ping "$ns2" 100.64.0.3 + do_ping "$ns3" 100.64.0.2 + + do_ping "$ns1" dead:beef:1::2 + do_ping "$ns1" dead:beef:1::3 + do_ping "$ns2" dead:beef:1::1 + do_ping "$ns2" dead:beef:1::2 + do_ping "$ns3" dead:beef:1::1 + do_ping "$ns3" dead:beef:1::2 + + stop_if_error "Initial validation failed." # Wait until supervisor all supervision frames have been processed and the node # entries have been merged. Otherwise duplicate frames will be observed which is # valid at this stage. -WAIT=5 -while [ ${WAIT} -gt 0 ] -do - grep 00:00:00:00:00:00 /sys/kernel/debug/hsr/hsr*/node_table - if [ $? -ne 0 ] - then - break - fi - sleep 1 - let WAIT = WAIT - 1 -done + WAIT=5 + while [ ${WAIT} -gt 0 ] + do + grep 00:00:00:00:00:00 /sys/kernel/debug/hsr/hsr*/node_table + if [ $? -ne 0 ] + then + break + fi + sleep 1 + let "WAIT = WAIT - 1" + done # Just a safety delay in case the above check didn't handle it. -sleep 1 + sleep 1 + + echo "INFO: Longer ping test." + do_ping_long "$ns1" 100.64.0.2 + do_ping_long "$ns1" dead:beef:1::2 + do_ping_long "$ns1" 100.64.0.3 + do_ping_long "$ns1" dead:beef:1::3 -echo "INFO: Longer ping test." -do_ping_long "$ns1" 100.64.0.2 -do_ping_long "$ns1" dead:beef:1::2 -do_ping_long "$ns1" 100.64.0.3 -do_ping_long "$ns1" dead:beef:1::3 + stop_if_error "Longer ping test failed." -stop_if_error "Longer ping test failed." + do_ping_long "$ns2" 100.64.0.1 + do_ping_long "$ns2" dead:beef:1::1 + do_ping_long "$ns2" 100.64.0.3 + do_ping_long "$ns2" dead:beef:1::2 + stop_if_error "Longer ping test failed." -do_ping_long "$ns2" 100.64.0.1 -do_ping_long "$ns2" dead:beef:1::1 -do_ping_long "$ns2" 100.64.0.3 -do_ping_long "$ns2" dead:beef:1::2 -stop_if_error "Longer ping test failed." + do_ping_long "$ns3" 100.64.0.1 + do_ping_long "$ns3" dead:beef:1::1 + do_ping_long "$ns3" 100.64.0.2 + do_ping_long "$ns3" dead:beef:1::2 + stop_if_error "Longer ping test failed." -do_ping_long "$ns3" 100.64.0.1 -do_ping_long "$ns3" dead:beef:1::1 -do_ping_long "$ns3" 100.64.0.2 -do_ping_long "$ns3" dead:beef:1::2 -stop_if_error "Longer ping test failed." + echo "INFO: Cutting one link." + do_ping_long "$ns1" 100.64.0.3 & -echo "INFO: Cutting one link." -do_ping_long "$ns1" 100.64.0.3 & + sleep 3 + ip -net "$ns3" link set ns3eth1 down + wait -sleep 3 -ip -net "$ns3" link set ns3eth1 down -wait + ip -net "$ns3" link set ns3eth1 up -ip -net "$ns3" link set ns3eth1 up + stop_if_error "Failed with one link down." -stop_if_error "Failed with one link down." + echo "INFO: Delay the link and drop a few packages." + tc -net "$ns3" qdisc add dev ns3eth1 root netem delay 50ms + tc -net "$ns2" qdisc add dev ns2eth1 root netem delay 5ms loss 25% -echo "INFO: Delay the link and drop a few packages." -tc -net "$ns3" qdisc add dev ns3eth1 root netem delay 50ms -tc -net "$ns2" qdisc add dev ns2eth1 root netem delay 5ms loss 25% + do_ping_long "$ns1" 100.64.0.2 + do_ping_long "$ns1" 100.64.0.3 -do_ping_long "$ns1" 100.64.0.2 -do_ping_long "$ns1" 100.64.0.3 + stop_if_error "Failed with delay and packetloss." -stop_if_error "Failed with delay and packetloss." + do_ping_long "$ns2" 100.64.0.1 + do_ping_long "$ns2" 100.64.0.3 -do_ping_long "$ns2" 100.64.0.1 -do_ping_long "$ns2" 100.64.0.3 + stop_if_error "Failed with delay and packetloss." -stop_if_error "Failed with delay and packetloss." + do_ping_long "$ns3" 100.64.0.1 + do_ping_long "$ns3" 100.64.0.2 + stop_if_error "Failed with delay and packetloss." + + echo "INFO: All good." +} + +setup_hsr_interfaces() +{ + local HSRv="$1" + + echo "INFO: preparing interfaces for HSRv${HSRv}." +# Three HSR nodes. Each node has one link to each of its neighbour, two links in total. +# +# ns1eth1 ----- ns2eth1 +# hsr1 hsr2 +# ns1eth2 ns2eth2 +# | | +# ns3eth1 ns3eth2 +# \ / +# hsr3 +# + # Interfaces + ip link add ns1eth1 netns "$ns1" type veth peer name ns2eth1 netns "$ns2" + ip link add ns1eth2 netns "$ns1" type veth peer name ns3eth1 netns "$ns3" + ip link add ns3eth2 netns "$ns3" type veth peer name ns2eth2 netns "$ns2" + + # HSRv0/1 + ip -net "$ns1" link add name hsr1 type hsr slave1 ns1eth1 slave2 ns1eth2 supervision 45 version $HSRv proto 0 + ip -net "$ns2" link add name hsr2 type hsr slave1 ns2eth1 slave2 ns2eth2 supervision 45 version $HSRv proto 0 + ip -net "$ns3" link add name hsr3 type hsr slave1 ns3eth1 slave2 ns3eth2 supervision 45 version $HSRv proto 0 + + # IP for HSR + ip -net "$ns1" addr add 100.64.0.1/24 dev hsr1 + ip -net "$ns1" addr add dead:beef:1::1/64 dev hsr1 nodad + ip -net "$ns2" addr add 100.64.0.2/24 dev hsr2 + ip -net "$ns2" addr add dead:beef:1::2/64 dev hsr2 nodad + ip -net "$ns3" addr add 100.64.0.3/24 dev hsr3 + ip -net "$ns3" addr add dead:beef:1::3/64 dev hsr3 nodad + + # All Links up + ip -net "$ns1" link set ns1eth1 up + ip -net "$ns1" link set ns1eth2 up + ip -net "$ns1" link set hsr1 up + + ip -net "$ns2" link set ns2eth1 up + ip -net "$ns2" link set ns2eth2 up + ip -net "$ns2" link set hsr2 up + + ip -net "$ns3" link set ns3eth1 up + ip -net "$ns3" link set ns3eth2 up + ip -net "$ns3" link set hsr3 up +} + +ip -Version > /dev/null 2>&1 +if [ $? -ne 0 ];then + echo "SKIP: Could not run test without ip tool" + exit $ksft_skip +fi + +trap cleanup EXIT + +for i in "$ns1" "$ns2" "$ns3" ;do + ip netns add $i || exit $ksft_skip + ip -net $i link set lo up +done + +setup_hsr_interfaces 0 +do_complete_ping_test +cleanup + +for i in "$ns1" "$ns2" "$ns3" ;do + ip netns add $i || exit $ksft_skip + ip -net $i link set lo up +done -do_ping_long "$ns3" 100.64.0.1 -do_ping_long "$ns3" 100.64.0.2 -stop_if_error "Failed with delay and packetloss." +setup_hsr_interfaces 1 +do_complete_ping_test -echo "INFO: All good." exit $ret diff --git a/tools/testing/selftests/net/mptcp/mptcp_join.sh b/tools/testing/selftests/net/mptcp/mptcp_join.sh index ee1f89a872b3..dc895b7b94e1 100755 --- a/tools/testing/selftests/net/mptcp/mptcp_join.sh +++ b/tools/testing/selftests/net/mptcp/mptcp_join.sh @@ -1432,7 +1432,9 @@ chk_rst_nr() count=$(get_counter ${ns_tx} "MPTcpExtMPRstTx") if [ -z "$count" ]; then print_skip - elif [ $count -lt $rst_tx ]; then + # accept more rst than expected except if we don't expect any + elif { [ $rst_tx -ne 0 ] && [ $count -lt $rst_tx ]; } || + { [ $rst_tx -eq 0 ] && [ $count -ne 0 ]; }; then fail_test "got $count MP_RST[s] TX expected $rst_tx" else print_ok @@ -1442,7 +1444,9 @@ chk_rst_nr() count=$(get_counter ${ns_rx} "MPTcpExtMPRstRx") if [ -z "$count" ]; then print_skip - elif [ "$count" -lt "$rst_rx" ]; then + # accept more rst than expected except if we don't expect any + elif { [ $rst_rx -ne 0 ] && [ $count -lt $rst_rx ]; } || + { [ $rst_rx -eq 0 ] && [ $count -ne 0 ]; }; then fail_test "got $count MP_RST[s] RX expected $rst_rx" else print_ok @@ -2305,6 +2309,7 @@ remove_tests() chk_join_nr 1 1 1 chk_rm_tx_nr 1 chk_rm_nr 1 1 + chk_rst_nr 0 0 fi # multiple subflows, remove @@ -2317,6 +2322,7 @@ remove_tests() run_tests $ns1 $ns2 10.0.1.1 chk_join_nr 2 2 2 chk_rm_nr 2 2 + chk_rst_nr 0 0 fi # single address, remove @@ -2329,6 +2335,7 @@ remove_tests() chk_join_nr 1 1 1 chk_add_nr 1 1 chk_rm_nr 1 1 invert + chk_rst_nr 0 0 fi # subflow and signal, remove @@ -2342,6 +2349,7 @@ remove_tests() chk_join_nr 2 2 2 chk_add_nr 1 1 chk_rm_nr 1 1 + chk_rst_nr 0 0 fi # subflows and signal, remove @@ -2356,6 +2364,7 @@ remove_tests() chk_join_nr 3 3 3 chk_add_nr 1 1 chk_rm_nr 2 2 + chk_rst_nr 0 0 fi # addresses remove @@ -2370,6 +2379,7 @@ remove_tests() chk_join_nr 3 3 3 chk_add_nr 3 3 chk_rm_nr 3 3 invert + chk_rst_nr 0 0 fi # invalid addresses remove @@ -2384,6 +2394,7 @@ remove_tests() chk_join_nr 1 1 1 chk_add_nr 3 3 chk_rm_nr 3 1 invert + chk_rst_nr 0 0 fi # subflows and signal, flush @@ -2398,6 +2409,7 @@ remove_tests() chk_join_nr 3 3 3 chk_add_nr 1 1 chk_rm_nr 1 3 invert simult + chk_rst_nr 0 0 fi # subflows flush @@ -2417,6 +2429,7 @@ remove_tests() else chk_rm_nr 3 3 fi + chk_rst_nr 0 0 fi # addresses flush @@ -2431,6 +2444,7 @@ remove_tests() chk_join_nr 3 3 3 chk_add_nr 3 3 chk_rm_nr 3 3 invert simult + chk_rst_nr 0 0 fi # invalid addresses flush @@ -2445,6 +2459,7 @@ remove_tests() chk_join_nr 1 1 1 chk_add_nr 3 3 chk_rm_nr 3 1 invert + chk_rst_nr 0 0 fi # remove id 0 subflow @@ -2456,6 +2471,7 @@ remove_tests() run_tests $ns1 $ns2 10.0.1.1 chk_join_nr 1 1 1 chk_rm_nr 1 1 + chk_rst_nr 0 0 fi # remove id 0 address @@ -2468,6 +2484,7 @@ remove_tests() chk_join_nr 1 1 1 chk_add_nr 1 1 chk_rm_nr 1 1 invert + chk_rst_nr 0 0 invert fi } diff --git a/tools/testing/selftests/net/netns-name.sh b/tools/testing/selftests/net/netns-name.sh new file mode 100755 index 000000000000..7d3d3fc99461 --- /dev/null +++ b/tools/testing/selftests/net/netns-name.sh @@ -0,0 +1,87 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 + +set -o pipefail + +NS=netns-name-test +DEV=dummy-dev0 +DEV2=dummy-dev1 +ALT_NAME=some-alt-name + +RET_CODE=0 + +cleanup() { + ip netns del $NS +} + +trap cleanup EXIT + +fail() { + echo "ERROR: ${1:-unexpected return code} (ret: $_)" >&2 + RET_CODE=1 +} + +ip netns add $NS + +# +# Test basic move without a rename +# +ip -netns $NS link add name $DEV type dummy || fail +ip -netns $NS link set dev $DEV netns 1 || + fail "Can't perform a netns move" +ip link show dev $DEV >> /dev/null || fail "Device not found after move" +ip link del $DEV || fail + +# +# Test move with a conflict +# +ip link add name $DEV type dummy +ip -netns $NS link add name $DEV type dummy || fail +ip -netns $NS link set dev $DEV netns 1 2> /dev/null && + fail "Performed a netns move with a name conflict" +ip link show dev $DEV >> /dev/null || fail "Device not found after move" +ip -netns $NS link del $DEV || fail +ip link del $DEV || fail + +# +# Test move with a conflict and rename +# +ip link add name $DEV type dummy +ip -netns $NS link add name $DEV type dummy || fail +ip -netns $NS link set dev $DEV netns 1 name $DEV2 || + fail "Can't perform a netns move with rename" +ip link del $DEV2 || fail +ip link del $DEV || fail + +# +# Test dup alt-name with netns move +# +ip link add name $DEV type dummy || fail +ip link property add dev $DEV altname $ALT_NAME || fail +ip -netns $NS link add name $DEV2 type dummy || fail +ip -netns $NS link property add dev $DEV2 altname $ALT_NAME || fail + +ip -netns $NS link set dev $DEV2 netns 1 2> /dev/null && + fail "Moved with alt-name dup" + +ip link del $DEV || fail +ip -netns $NS link del $DEV2 || fail + +# +# Test creating alt-name in one net-ns and using in another +# +ip -netns $NS link add name $DEV type dummy || fail +ip -netns $NS link property add dev $DEV altname $ALT_NAME || fail +ip -netns $NS link set dev $DEV netns 1 || fail +ip link show dev $ALT_NAME >> /dev/null || fail "Can't find alt-name after move" +ip -netns $NS link show dev $ALT_NAME 2> /dev/null && + fail "Can still find alt-name after move" +ip link del $DEV || fail + +echo -ne "$(basename $0) \t\t\t\t" +if [ $RET_CODE -eq 0 ]; then + echo "[ OK ]" +else + echo "[ FAIL ]" +fi +exit $RET_CODE diff --git a/tools/testing/selftests/net/openvswitch/openvswitch.sh b/tools/testing/selftests/net/openvswitch/openvswitch.sh index 9c2012d70b08..f8499d4c87f3 100755 --- a/tools/testing/selftests/net/openvswitch/openvswitch.sh +++ b/tools/testing/selftests/net/openvswitch/openvswitch.sh @@ -3,6 +3,8 @@ # # OVS kernel module self tests +trap ovs_exit_sig EXIT TERM INT ERR + # Kselftest framework requirement - SKIP code is 4. ksft_skip=4 @@ -142,6 +144,12 @@ ovs_add_flow () { return 0 } +ovs_del_flows () { + info "Deleting all flows from DP: sbx:$1 br:$2" + ovs_sbx "$1" python3 $ovs_base/ovs-dpctl.py del-flows "$2" + return 0 +} + ovs_drop_record_and_run () { local sbx=$1 shift @@ -198,6 +206,17 @@ test_drop_reason() { ip netns exec server ip addr add 172.31.110.20/24 dev s1 ip netns exec server ip link set s1 up + # Check if drop reasons can be sent + ovs_add_flow "test_drop_reason" dropreason \ + 'in_port(1),eth(),eth_type(0x0806),arp()' 'drop(10)' 2>/dev/null + if [ $? == 1 ]; then + info "no support for drop reasons - skipping" + ovs_exit_sig + return $ksft_skip + fi + + ovs_del_flows "test_drop_reason" dropreason + # Allow ARP ovs_add_flow "test_drop_reason" dropreason \ 'in_port(1),eth(),eth_type(0x0806),arp()' '2' || return 1 @@ -525,7 +544,7 @@ run_test() { fi if python3 ovs-dpctl.py -h 2>&1 | \ - grep "Need to install the python" >/dev/null 2>&1; then + grep -E "Need to (install|upgrade) the python" >/dev/null 2>&1; then stdbuf -o0 printf "TEST: %-60s [PYLIB]\n" "${tdesc}" return $ksft_skip fi diff --git a/tools/testing/selftests/net/openvswitch/ovs-dpctl.py b/tools/testing/selftests/net/openvswitch/ovs-dpctl.py index 912dc8c49085..b97e621face9 100644 --- a/tools/testing/selftests/net/openvswitch/ovs-dpctl.py +++ b/tools/testing/selftests/net/openvswitch/ovs-dpctl.py @@ -28,8 +28,10 @@ try: from pyroute2.netlink import nlmsg_atoms from pyroute2.netlink.exceptions import NetlinkError from pyroute2.netlink.generic import GenericNetlinkSocket + import pyroute2 + except ModuleNotFoundError: - print("Need to install the python pyroute2 package.") + print("Need to install the python pyroute2 package >= 0.6.") sys.exit(0) @@ -1117,12 +1119,14 @@ class ovskey(nla): "src", lambda x: str(ipaddress.IPv4Address(x)), int, + convert_ipv4, ), ( "dst", "dst", - lambda x: str(ipaddress.IPv6Address(x)), + lambda x: str(ipaddress.IPv4Address(x)), int, + convert_ipv4, ), ("tp_src", "tp_src", "%d", int), ("tp_dst", "tp_dst", "%d", int), @@ -1904,6 +1908,32 @@ class OvsFlow(GenericNetlinkSocket): raise ne return reply + def del_flows(self, dpifindex): + """ + Send a del message to the kernel that will drop all flows. + + dpifindex should be a valid datapath obtained by calling + into the OvsDatapath lookup + """ + + flowmsg = OvsFlow.ovs_flow_msg() + flowmsg["cmd"] = OVS_FLOW_CMD_DEL + flowmsg["version"] = OVS_DATAPATH_VERSION + flowmsg["reserved"] = 0 + flowmsg["dpifindex"] = dpifindex + + try: + reply = self.nlm_request( + flowmsg, + msg_type=self.prid, + msg_flags=NLM_F_REQUEST | NLM_F_ACK, + ) + reply = reply[0] + except NetlinkError as ne: + print(flowmsg) + raise ne + return reply + def dump(self, dpifindex, flowspec=None): """ Returns a list of messages containing flows. @@ -1998,6 +2028,12 @@ def main(argv): nlmsg_atoms.ovskey = ovskey nlmsg_atoms.ovsactions = ovsactions + # version check for pyroute2 + prverscheck = pyroute2.__version__.split(".") + if int(prverscheck[0]) == 0 and int(prverscheck[1]) < 6: + print("Need to upgrade the python pyroute2 package to >= 0.6.") + sys.exit(0) + parser = argparse.ArgumentParser() parser.add_argument( "-v", @@ -2060,6 +2096,9 @@ def main(argv): addflcmd.add_argument("flow", help="Flow specification") addflcmd.add_argument("acts", help="Flow actions") + delfscmd = subparsers.add_parser("del-flows") + delfscmd.add_argument("flsbr", help="Datapath name") + args = parser.parse_args() if args.verbose > 0: @@ -2143,6 +2182,11 @@ def main(argv): flow = OvsFlow.ovs_flow_msg() flow.parse(args.flow, args.acts, rep["dpifindex"]) ovsflow.add_flow(rep["dpifindex"], flow) + elif hasattr(args, "flsbr"): + rep = ovsdp.info(args.flsbr, 0) + if rep is None: + print("DP '%s' not found." % args.flsbr) + ovsflow.del_flows(rep["dpifindex"]) return 0 diff --git a/tools/testing/selftests/net/tls.c b/tools/testing/selftests/net/tls.c index 297d972558fb..464853a7f982 100644 --- a/tools/testing/selftests/net/tls.c +++ b/tools/testing/selftests/net/tls.c @@ -613,11 +613,11 @@ TEST_F(tls, sendmsg_large) msg.msg_iov = &vec; msg.msg_iovlen = 1; - EXPECT_EQ(sendmsg(self->cfd, &msg, 0), send_len); + EXPECT_EQ(sendmsg(self->fd, &msg, 0), send_len); } while (recvs++ < sends) { - EXPECT_NE(recv(self->fd, mem, send_len, 0), -1); + EXPECT_NE(recv(self->cfd, mem, send_len, 0), -1); } free(mem); @@ -646,9 +646,9 @@ TEST_F(tls, sendmsg_multiple) msg.msg_iov = vec; msg.msg_iovlen = iov_len; - EXPECT_EQ(sendmsg(self->cfd, &msg, 0), total_len); + EXPECT_EQ(sendmsg(self->fd, &msg, 0), total_len); buf = malloc(total_len); - EXPECT_NE(recv(self->fd, buf, total_len, 0), -1); + EXPECT_NE(recv(self->cfd, buf, total_len, 0), -1); for (i = 0; i < iov_len; i++) { EXPECT_EQ(memcmp(test_strs[i], buf + len_cmp, strlen(test_strs[i])), diff --git a/tools/testing/selftests/netfilter/.gitignore b/tools/testing/selftests/netfilter/.gitignore index 4cb887b57413..4b2928e1c19d 100644 --- a/tools/testing/selftests/netfilter/.gitignore +++ b/tools/testing/selftests/netfilter/.gitignore @@ -1,3 +1,4 @@ # SPDX-License-Identifier: GPL-2.0-only nf-queue connect_close +audit_logread diff --git a/tools/testing/selftests/netfilter/Makefile b/tools/testing/selftests/netfilter/Makefile index 3686bfa6c58d..ef90aca4cc96 100644 --- a/tools/testing/selftests/netfilter/Makefile +++ b/tools/testing/selftests/netfilter/Makefile @@ -6,13 +6,14 @@ TEST_PROGS := nft_trans_stress.sh nft_fib.sh nft_nat.sh bridge_brouter.sh \ nft_concat_range.sh nft_conntrack_helper.sh \ nft_queue.sh nft_meta.sh nf_nat_edemux.sh \ ipip-conntrack-mtu.sh conntrack_tcp_unreplied.sh \ - conntrack_vrf.sh nft_synproxy.sh rpath.sh + conntrack_vrf.sh nft_synproxy.sh rpath.sh nft_audit.sh \ + conntrack_sctp_collision.sh HOSTPKG_CONFIG := pkg-config CFLAGS += $(shell $(HOSTPKG_CONFIG) --cflags libmnl 2>/dev/null) LDLIBS += $(shell $(HOSTPKG_CONFIG) --libs libmnl 2>/dev/null || echo -lmnl) -TEST_GEN_FILES = nf-queue connect_close +TEST_GEN_FILES = nf-queue connect_close audit_logread sctp_collision include ../lib.mk diff --git a/tools/testing/selftests/netfilter/audit_logread.c b/tools/testing/selftests/netfilter/audit_logread.c new file mode 100644 index 000000000000..a0a880fc2d9d --- /dev/null +++ b/tools/testing/selftests/netfilter/audit_logread.c @@ -0,0 +1,165 @@ +// SPDX-License-Identifier: GPL-2.0 + +#define _GNU_SOURCE +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +static int fd; + +#define MAX_AUDIT_MESSAGE_LENGTH 8970 +struct audit_message { + struct nlmsghdr nlh; + union { + struct audit_status s; + char data[MAX_AUDIT_MESSAGE_LENGTH]; + } u; +}; + +int audit_recv(int fd, struct audit_message *rep) +{ + struct sockaddr_nl addr; + socklen_t addrlen = sizeof(addr); + int ret; + + do { + ret = recvfrom(fd, rep, sizeof(*rep), 0, + (struct sockaddr *)&addr, &addrlen); + } while (ret < 0 && errno == EINTR); + + if (ret < 0 || + addrlen != sizeof(addr) || + addr.nl_pid != 0 || + rep->nlh.nlmsg_type == NLMSG_ERROR) /* short-cut for now */ + return -1; + + return ret; +} + +int audit_send(int fd, uint16_t type, uint32_t key, uint32_t val) +{ + static int seq = 0; + struct audit_message msg = { + .nlh = { + .nlmsg_len = NLMSG_SPACE(sizeof(msg.u.s)), + .nlmsg_type = type, + .nlmsg_flags = NLM_F_REQUEST | NLM_F_ACK, + .nlmsg_seq = ++seq, + }, + .u.s = { + .mask = key, + .enabled = key == AUDIT_STATUS_ENABLED ? val : 0, + .pid = key == AUDIT_STATUS_PID ? val : 0, + } + }; + struct sockaddr_nl addr = { + .nl_family = AF_NETLINK, + }; + int ret; + + do { + ret = sendto(fd, &msg, msg.nlh.nlmsg_len, 0, + (struct sockaddr *)&addr, sizeof(addr)); + } while (ret < 0 && errno == EINTR); + + if (ret != (int)msg.nlh.nlmsg_len) + return -1; + return 0; +} + +int audit_set(int fd, uint32_t key, uint32_t val) +{ + struct audit_message rep = { 0 }; + int ret; + + ret = audit_send(fd, AUDIT_SET, key, val); + if (ret) + return ret; + + ret = audit_recv(fd, &rep); + if (ret < 0) + return ret; + return 0; +} + +int readlog(int fd) +{ + struct audit_message rep = { 0 }; + int ret = audit_recv(fd, &rep); + const char *sep = ""; + char *k, *v; + + if (ret < 0) + return ret; + + if (rep.nlh.nlmsg_type != AUDIT_NETFILTER_CFG) + return 0; + + /* skip the initial "audit(...): " part */ + strtok(rep.u.data, " "); + + while ((k = strtok(NULL, "="))) { + v = strtok(NULL, " "); + + /* these vary and/or are uninteresting, ignore */ + if (!strcmp(k, "pid") || + !strcmp(k, "comm") || + !strcmp(k, "subj")) + continue; + + /* strip the varying sequence number */ + if (!strcmp(k, "table")) + *strchrnul(v, ':') = '\0'; + + printf("%s%s=%s", sep, k, v); + sep = " "; + } + if (*sep) { + printf("\n"); + fflush(stdout); + } + return 0; +} + +void cleanup(int sig) +{ + audit_set(fd, AUDIT_STATUS_ENABLED, 0); + close(fd); + if (sig) + exit(0); +} + +int main(int argc, char **argv) +{ + struct sigaction act = { + .sa_handler = cleanup, + }; + + fd = socket(PF_NETLINK, SOCK_RAW, NETLINK_AUDIT); + if (fd < 0) { + perror("Can't open netlink socket"); + return -1; + } + + if (sigaction(SIGTERM, &act, NULL) < 0 || + sigaction(SIGINT, &act, NULL) < 0) { + perror("Can't set signal handler"); + close(fd); + return -1; + } + + audit_set(fd, AUDIT_STATUS_ENABLED, 1); + audit_set(fd, AUDIT_STATUS_PID, getpid()); + + while (1) + readlog(fd); +} diff --git a/tools/testing/selftests/netfilter/config b/tools/testing/selftests/netfilter/config index 4faf2ce021d9..7c42b1b2c69b 100644 --- a/tools/testing/selftests/netfilter/config +++ b/tools/testing/selftests/netfilter/config @@ -6,3 +6,4 @@ CONFIG_NFT_REDIR=m CONFIG_NFT_MASQ=m CONFIG_NFT_FLOW_OFFLOAD=m CONFIG_NF_CT_NETLINK=m +CONFIG_AUDIT=y diff --git a/tools/testing/selftests/netfilter/conntrack_sctp_collision.sh b/tools/testing/selftests/netfilter/conntrack_sctp_collision.sh new file mode 100755 index 000000000000..a924e595cfd8 --- /dev/null +++ b/tools/testing/selftests/netfilter/conntrack_sctp_collision.sh @@ -0,0 +1,89 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +# +# Testing For SCTP COLLISION SCENARIO as Below: +# +# 14:35:47.655279 IP CLIENT_IP.PORT > SERVER_IP.PORT: sctp (1) [INIT] [init tag: 2017837359] +# 14:35:48.353250 IP SERVER_IP.PORT > CLIENT_IP.PORT: sctp (1) [INIT] [init tag: 1187206187] +# 14:35:48.353275 IP CLIENT_IP.PORT > SERVER_IP.PORT: sctp (1) [INIT ACK] [init tag: 2017837359] +# 14:35:48.353283 IP SERVER_IP.PORT > CLIENT_IP.PORT: sctp (1) [COOKIE ECHO] +# 14:35:48.353977 IP CLIENT_IP.PORT > SERVER_IP.PORT: sctp (1) [COOKIE ACK] +# 14:35:48.855335 IP SERVER_IP.PORT > CLIENT_IP.PORT: sctp (1) [INIT ACK] [init tag: 164579970] +# +# TOPO: SERVER_NS (link0)<--->(link1) ROUTER_NS (link2)<--->(link3) CLIENT_NS + +CLIENT_NS=$(mktemp -u client-XXXXXXXX) +CLIENT_IP="198.51.200.1" +CLIENT_PORT=1234 + +SERVER_NS=$(mktemp -u server-XXXXXXXX) +SERVER_IP="198.51.100.1" +SERVER_PORT=1234 + +ROUTER_NS=$(mktemp -u router-XXXXXXXX) +CLIENT_GW="198.51.200.2" +SERVER_GW="198.51.100.2" + +# setup the topo +setup() { + ip net add $CLIENT_NS + ip net add $SERVER_NS + ip net add $ROUTER_NS + ip -n $SERVER_NS link add link0 type veth peer name link1 netns $ROUTER_NS + ip -n $CLIENT_NS link add link3 type veth peer name link2 netns $ROUTER_NS + + ip -n $SERVER_NS link set link0 up + ip -n $SERVER_NS addr add $SERVER_IP/24 dev link0 + ip -n $SERVER_NS route add $CLIENT_IP dev link0 via $SERVER_GW + + ip -n $ROUTER_NS link set link1 up + ip -n $ROUTER_NS link set link2 up + ip -n $ROUTER_NS addr add $SERVER_GW/24 dev link1 + ip -n $ROUTER_NS addr add $CLIENT_GW/24 dev link2 + ip net exec $ROUTER_NS sysctl -wq net.ipv4.ip_forward=1 + + ip -n $CLIENT_NS link set link3 up + ip -n $CLIENT_NS addr add $CLIENT_IP/24 dev link3 + ip -n $CLIENT_NS route add $SERVER_IP dev link3 via $CLIENT_GW + + # simulate the delay on OVS upcall by setting up a delay for INIT_ACK with + # tc on $SERVER_NS side + tc -n $SERVER_NS qdisc add dev link0 root handle 1: htb + tc -n $SERVER_NS class add dev link0 parent 1: classid 1:1 htb rate 100mbit + tc -n $SERVER_NS filter add dev link0 parent 1: protocol ip u32 match ip protocol 132 \ + 0xff match u8 2 0xff at 32 flowid 1:1 + tc -n $SERVER_NS qdisc add dev link0 parent 1:1 handle 10: netem delay 1200ms + + # simulate the ctstate check on OVS nf_conntrack + ip net exec $ROUTER_NS iptables -A FORWARD -m state --state INVALID,UNTRACKED -j DROP + ip net exec $ROUTER_NS iptables -A INPUT -p sctp -j DROP + + # use a smaller number for assoc's max_retrans to reproduce the issue + modprobe sctp + ip net exec $CLIENT_NS sysctl -wq net.sctp.association_max_retrans=3 +} + +cleanup() { + ip net exec $CLIENT_NS pkill sctp_collision 2>&1 >/dev/null + ip net exec $SERVER_NS pkill sctp_collision 2>&1 >/dev/null + ip net del "$CLIENT_NS" + ip net del "$SERVER_NS" + ip net del "$ROUTER_NS" +} + +do_test() { + ip net exec $SERVER_NS ./sctp_collision server \ + $SERVER_IP $SERVER_PORT $CLIENT_IP $CLIENT_PORT & + ip net exec $CLIENT_NS ./sctp_collision client \ + $CLIENT_IP $CLIENT_PORT $SERVER_IP $SERVER_PORT +} + +# NOTE: one way to work around the issue is set a smaller hb_interval +# ip net exec $CLIENT_NS sysctl -wq net.sctp.hb_interval=3500 + +# run the test case +trap cleanup EXIT +setup && \ +echo "Test for SCTP Collision in nf_conntrack:" && \ +do_test && echo "PASS!" +exit $? diff --git a/tools/testing/selftests/netfilter/nft_audit.sh b/tools/testing/selftests/netfilter/nft_audit.sh new file mode 100755 index 000000000000..99ed5bd6e840 --- /dev/null +++ b/tools/testing/selftests/netfilter/nft_audit.sh @@ -0,0 +1,245 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +# +# Check that audit logs generated for nft commands are as expected. + +SKIP_RC=4 +RC=0 + +nft --version >/dev/null 2>&1 || { + echo "SKIP: missing nft tool" + exit $SKIP_RC +} + +# Run everything in a separate network namespace +[ "${1}" != "run" ] && { unshare -n "${0}" run; exit $?; } + +# give other scripts a chance to finish - audit_logread sees all activity +sleep 1 + +logfile=$(mktemp) +rulefile=$(mktemp) +echo "logging into $logfile" +./audit_logread >"$logfile" & +logread_pid=$! +trap 'kill $logread_pid; rm -f $logfile $rulefile' EXIT +exec 3<"$logfile" + +do_test() { # (cmd, log) + echo -n "testing for cmd: $1 ... " + cat <&3 >/dev/null + $1 >/dev/null || exit 1 + sleep 0.1 + res=$(diff -a -u <(echo "$2") - <&3) + [ $? -eq 0 ] && { echo "OK"; return; } + echo "FAIL" + grep -v '^\(---\|+++\|@@\)' <<< "$res" + ((RC--)) +} + +nft flush ruleset + +# adding tables, chains and rules + +for table in t1 t2; do + do_test "nft add table $table" \ + "table=$table family=2 entries=1 op=nft_register_table" + + do_test "nft add chain $table c1" \ + "table=$table family=2 entries=1 op=nft_register_chain" + + do_test "nft add chain $table c2; add chain $table c3" \ + "table=$table family=2 entries=2 op=nft_register_chain" + + cmd="add rule $table c1 counter" + + do_test "nft $cmd" \ + "table=$table family=2 entries=1 op=nft_register_rule" + + do_test "nft $cmd; $cmd" \ + "table=$table family=2 entries=2 op=nft_register_rule" + + cmd="" + sep="" + for chain in c2 c3; do + for i in {1..3}; do + cmd+="$sep add rule $table $chain counter" + sep=";" + done + done + do_test "nft $cmd" \ + "table=$table family=2 entries=6 op=nft_register_rule" +done + +for ((i = 0; i < 500; i++)); do + echo "add rule t2 c3 counter accept comment \"rule $i\"" +done >$rulefile +do_test "nft -f $rulefile" \ +'table=t2 family=2 entries=500 op=nft_register_rule' + +# adding sets and elements + +settype='type inet_service; counter' +setelem='{ 22, 80, 443 }' +setblock="{ $settype; elements = $setelem; }" +do_test "nft add set t1 s $setblock" \ +"table=t1 family=2 entries=4 op=nft_register_set" + +do_test "nft add set t1 s2 $setblock; add set t1 s3 { $settype; }" \ +"table=t1 family=2 entries=5 op=nft_register_set" + +do_test "nft add element t1 s3 $setelem" \ +"table=t1 family=2 entries=3 op=nft_register_setelem" + +# adding counters + +do_test 'nft add counter t1 c1' \ +'table=t1 family=2 entries=1 op=nft_register_obj' + +do_test 'nft add counter t2 c1; add counter t2 c2' \ +'table=t2 family=2 entries=2 op=nft_register_obj' + +for ((i = 3; i <= 500; i++)); do + echo "add counter t2 c$i" +done >$rulefile +do_test "nft -f $rulefile" \ +'table=t2 family=2 entries=498 op=nft_register_obj' + +# adding/updating quotas + +do_test 'nft add quota t1 q1 { 10 bytes }' \ +'table=t1 family=2 entries=1 op=nft_register_obj' + +do_test 'nft add quota t2 q1 { 10 bytes }; add quota t2 q2 { 10 bytes }' \ +'table=t2 family=2 entries=2 op=nft_register_obj' + +for ((i = 3; i <= 500; i++)); do + echo "add quota t2 q$i { 10 bytes }" +done >$rulefile +do_test "nft -f $rulefile" \ +'table=t2 family=2 entries=498 op=nft_register_obj' + +# changing the quota value triggers obj update path +do_test 'nft add quota t1 q1 { 20 bytes }' \ +'table=t1 family=2 entries=1 op=nft_register_obj' + +# resetting rules + +do_test 'nft reset rules t1 c2' \ +'table=t1 family=2 entries=3 op=nft_reset_rule' + +do_test 'nft reset rules table t1' \ +'table=t1 family=2 entries=3 op=nft_reset_rule +table=t1 family=2 entries=3 op=nft_reset_rule +table=t1 family=2 entries=3 op=nft_reset_rule' + +do_test 'nft reset rules t2 c3' \ +'table=t2 family=2 entries=189 op=nft_reset_rule +table=t2 family=2 entries=188 op=nft_reset_rule +table=t2 family=2 entries=126 op=nft_reset_rule' + +do_test 'nft reset rules t2' \ +'table=t2 family=2 entries=3 op=nft_reset_rule +table=t2 family=2 entries=3 op=nft_reset_rule +table=t2 family=2 entries=186 op=nft_reset_rule +table=t2 family=2 entries=188 op=nft_reset_rule +table=t2 family=2 entries=129 op=nft_reset_rule' + +do_test 'nft reset rules' \ +'table=t1 family=2 entries=3 op=nft_reset_rule +table=t1 family=2 entries=3 op=nft_reset_rule +table=t1 family=2 entries=3 op=nft_reset_rule +table=t2 family=2 entries=3 op=nft_reset_rule +table=t2 family=2 entries=3 op=nft_reset_rule +table=t2 family=2 entries=180 op=nft_reset_rule +table=t2 family=2 entries=188 op=nft_reset_rule +table=t2 family=2 entries=135 op=nft_reset_rule' + +# resetting sets and elements + +elem=(22 ,80 ,443) +relem="" +for i in {1..3}; do + relem+="${elem[((i - 1))]}" + do_test "nft reset element t1 s { $relem }" \ + "table=t1 family=2 entries=$i op=nft_reset_setelem" +done + +do_test 'nft reset set t1 s' \ +'table=t1 family=2 entries=3 op=nft_reset_setelem' + +# resetting counters + +do_test 'nft reset counter t1 c1' \ +'table=t1 family=2 entries=1 op=nft_reset_obj' + +do_test 'nft reset counters t1' \ +'table=t1 family=2 entries=1 op=nft_reset_obj' + +do_test 'nft reset counters t2' \ +'table=t2 family=2 entries=342 op=nft_reset_obj +table=t2 family=2 entries=158 op=nft_reset_obj' + +do_test 'nft reset counters' \ +'table=t1 family=2 entries=1 op=nft_reset_obj +table=t2 family=2 entries=341 op=nft_reset_obj +table=t2 family=2 entries=159 op=nft_reset_obj' + +# resetting quotas + +do_test 'nft reset quota t1 q1' \ +'table=t1 family=2 entries=1 op=nft_reset_obj' + +do_test 'nft reset quotas t1' \ +'table=t1 family=2 entries=1 op=nft_reset_obj' + +do_test 'nft reset quotas t2' \ +'table=t2 family=2 entries=315 op=nft_reset_obj +table=t2 family=2 entries=185 op=nft_reset_obj' + +do_test 'nft reset quotas' \ +'table=t1 family=2 entries=1 op=nft_reset_obj +table=t2 family=2 entries=314 op=nft_reset_obj +table=t2 family=2 entries=186 op=nft_reset_obj' + +# deleting rules + +readarray -t handles < <(nft -a list chain t1 c1 | \ + sed -n 's/.*counter.* handle \(.*\)$/\1/p') + +do_test "nft delete rule t1 c1 handle ${handles[0]}" \ +'table=t1 family=2 entries=1 op=nft_unregister_rule' + +cmd='delete rule t1 c1 handle' +do_test "nft $cmd ${handles[1]}; $cmd ${handles[2]}" \ +'table=t1 family=2 entries=2 op=nft_unregister_rule' + +do_test 'nft flush chain t1 c2' \ +'table=t1 family=2 entries=3 op=nft_unregister_rule' + +do_test 'nft flush table t2' \ +'table=t2 family=2 entries=509 op=nft_unregister_rule' + +# deleting chains + +do_test 'nft delete chain t2 c2' \ +'table=t2 family=2 entries=1 op=nft_unregister_chain' + +# deleting sets and elements + +do_test 'nft delete element t1 s { 22 }' \ +'table=t1 family=2 entries=1 op=nft_unregister_setelem' + +do_test 'nft delete element t1 s { 80, 443 }' \ +'table=t1 family=2 entries=2 op=nft_unregister_setelem' + +do_test 'nft flush set t1 s2' \ +'table=t1 family=2 entries=3 op=nft_unregister_setelem' + +do_test 'nft delete set t1 s2' \ +'table=t1 family=2 entries=1 op=nft_unregister_set' + +do_test 'nft delete set t1 s3' \ +'table=t1 family=2 entries=1 op=nft_unregister_set' + +exit $RC diff --git a/tools/testing/selftests/netfilter/sctp_collision.c b/tools/testing/selftests/netfilter/sctp_collision.c new file mode 100644 index 000000000000..21bb1cfd8a85 --- /dev/null +++ b/tools/testing/selftests/netfilter/sctp_collision.c @@ -0,0 +1,99 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include +#include +#include +#include +#include + +int main(int argc, char *argv[]) +{ + struct sockaddr_in saddr = {}, daddr = {}; + int sd, ret, len = sizeof(daddr); + struct timeval tv = {25, 0}; + char buf[] = "hello"; + + if (argc != 6 || (strcmp(argv[1], "server") && strcmp(argv[1], "client"))) { + printf("%s \n", + argv[0]); + return -1; + } + + sd = socket(AF_INET, SOCK_SEQPACKET, IPPROTO_SCTP); + if (sd < 0) { + printf("Failed to create sd\n"); + return -1; + } + + saddr.sin_family = AF_INET; + saddr.sin_addr.s_addr = inet_addr(argv[2]); + saddr.sin_port = htons(atoi(argv[3])); + + ret = bind(sd, (struct sockaddr *)&saddr, sizeof(saddr)); + if (ret < 0) { + printf("Failed to bind to address\n"); + goto out; + } + + ret = listen(sd, 5); + if (ret < 0) { + printf("Failed to listen on port\n"); + goto out; + } + + daddr.sin_family = AF_INET; + daddr.sin_addr.s_addr = inet_addr(argv[4]); + daddr.sin_port = htons(atoi(argv[5])); + + /* make test shorter than 25s */ + ret = setsockopt(sd, SOL_SOCKET, SO_RCVTIMEO, &tv, sizeof(tv)); + if (ret < 0) { + printf("Failed to setsockopt SO_RCVTIMEO\n"); + goto out; + } + + if (!strcmp(argv[1], "server")) { + sleep(1); /* wait a bit for client's INIT */ + ret = connect(sd, (struct sockaddr *)&daddr, len); + if (ret < 0) { + printf("Failed to connect to peer\n"); + goto out; + } + ret = recvfrom(sd, buf, sizeof(buf), 0, (struct sockaddr *)&daddr, &len); + if (ret < 0) { + printf("Failed to recv msg %d\n", ret); + goto out; + } + ret = sendto(sd, buf, strlen(buf) + 1, 0, (struct sockaddr *)&daddr, len); + if (ret < 0) { + printf("Failed to send msg %d\n", ret); + goto out; + } + printf("Server: sent! %d\n", ret); + } + + if (!strcmp(argv[1], "client")) { + usleep(300000); /* wait a bit for server's listening */ + ret = connect(sd, (struct sockaddr *)&daddr, len); + if (ret < 0) { + printf("Failed to connect to peer\n"); + goto out; + } + sleep(1); /* wait a bit for server's delayed INIT_ACK to reproduce the issue */ + ret = sendto(sd, buf, strlen(buf) + 1, 0, (struct sockaddr *)&daddr, len); + if (ret < 0) { + printf("Failed to send msg %d\n", ret); + goto out; + } + ret = recvfrom(sd, buf, sizeof(buf), 0, (struct sockaddr *)&daddr, &len); + if (ret < 0) { + printf("Failed to recv msg %d\n", ret); + goto out; + } + printf("Client: rcvd! %d\n", ret); + } + ret = 0; +out: + close(sd); + return ret; +} diff --git a/tools/testing/selftests/openat2/Makefile b/tools/testing/selftests/openat2/Makefile index 843ba56d8e49..254d676a2689 100644 --- a/tools/testing/selftests/openat2/Makefile +++ b/tools/testing/selftests/openat2/Makefile @@ -1,6 +1,6 @@ # SPDX-License-Identifier: GPL-2.0-or-later -CFLAGS += -Wall -O2 -g -fsanitize=address -fsanitize=undefined +CFLAGS += -Wall -O2 -g -fsanitize=address -fsanitize=undefined -static-libasan TEST_GEN_PROGS := openat2_test resolve_test rename_attack_test include ../lib.mk diff --git a/tools/testing/selftests/powerpc/Makefile b/tools/testing/selftests/powerpc/Makefile index 49f2ad1793fd..7ea42fa02eab 100644 --- a/tools/testing/selftests/powerpc/Makefile +++ b/tools/testing/selftests/powerpc/Makefile @@ -59,12 +59,11 @@ override define INSTALL_RULE done; endef -override define EMIT_TESTS +emit_tests: +@for TARGET in $(SUB_DIRS); do \ BUILD_TARGET=$(OUTPUT)/$$TARGET; \ - $(MAKE) OUTPUT=$$BUILD_TARGET -s -C $$TARGET emit_tests;\ + $(MAKE) OUTPUT=$$BUILD_TARGET -s -C $$TARGET $@;\ done; -endef override define CLEAN +@for TARGET in $(SUB_DIRS); do \ @@ -77,4 +76,4 @@ endef tags: find . -name '*.c' -o -name '*.h' | xargs ctags -.PHONY: tags $(SUB_DIRS) +.PHONY: tags $(SUB_DIRS) emit_tests diff --git a/tools/testing/selftests/powerpc/pmu/Makefile b/tools/testing/selftests/powerpc/pmu/Makefile index 2b95e44d20ff..a284fa874a9f 100644 --- a/tools/testing/selftests/powerpc/pmu/Makefile +++ b/tools/testing/selftests/powerpc/pmu/Makefile @@ -30,13 +30,14 @@ override define RUN_TESTS +TARGET=event_code_tests; BUILD_TARGET=$$OUTPUT/$$TARGET; $(MAKE) OUTPUT=$$BUILD_TARGET -C $$TARGET run_tests endef -DEFAULT_EMIT_TESTS := $(EMIT_TESTS) -override define EMIT_TESTS - $(DEFAULT_EMIT_TESTS) +emit_tests: + for TEST in $(TEST_GEN_PROGS); do \ + BASENAME_TEST=`basename $$TEST`; \ + echo "$(COLLECTION):$$BASENAME_TEST"; \ + done +TARGET=ebb; BUILD_TARGET=$$OUTPUT/$$TARGET; $(MAKE) OUTPUT=$$BUILD_TARGET -s -C $$TARGET emit_tests +TARGET=sampling_tests; BUILD_TARGET=$$OUTPUT/$$TARGET; $(MAKE) OUTPUT=$$BUILD_TARGET -s -C $$TARGET emit_tests +TARGET=event_code_tests; BUILD_TARGET=$$OUTPUT/$$TARGET; $(MAKE) OUTPUT=$$BUILD_TARGET -s -C $$TARGET emit_tests -endef DEFAULT_INSTALL_RULE := $(INSTALL_RULE) override define INSTALL_RULE @@ -64,4 +65,4 @@ sampling_tests: event_code_tests: TARGET=$@; BUILD_TARGET=$$OUTPUT/$$TARGET; mkdir -p $$BUILD_TARGET; $(MAKE) OUTPUT=$$BUILD_TARGET -k -C $$TARGET all -.PHONY: all run_tests ebb sampling_tests event_code_tests +.PHONY: all run_tests ebb sampling_tests event_code_tests emit_tests diff --git a/tools/testing/selftests/proc/proc-empty-vm.c b/tools/testing/selftests/proc/proc-empty-vm.c index b16c13688b88..ee71ce52cb6a 100644 --- a/tools/testing/selftests/proc/proc-empty-vm.c +++ b/tools/testing/selftests/proc/proc-empty-vm.c @@ -267,6 +267,7 @@ static const char g_smaps_rollup[] = "Private_Dirty: 0 kB\n" "Referenced: 0 kB\n" "Anonymous: 0 kB\n" +"KSM: 0 kB\n" "LazyFree: 0 kB\n" "AnonHugePages: 0 kB\n" "ShmemPmdMapped: 0 kB\n" diff --git a/tools/testing/selftests/riscv/mm/Makefile b/tools/testing/selftests/riscv/mm/Makefile index 11e0f0568923..c333263f2b27 100644 --- a/tools/testing/selftests/riscv/mm/Makefile +++ b/tools/testing/selftests/riscv/mm/Makefile @@ -5,11 +5,11 @@ # Additional include paths needed by kselftest.h and local headers CFLAGS += -D_GNU_SOURCE -std=gnu99 -I. -TEST_GEN_FILES := testcases/mmap_default testcases/mmap_bottomup +TEST_GEN_FILES := mmap_default mmap_bottomup -TEST_PROGS := testcases/run_mmap.sh +TEST_PROGS := run_mmap.sh include ../../lib.mk -$(OUTPUT)/mm: testcases/mmap_default.c testcases/mmap_bottomup.c testcases/mmap_tests.h +$(OUTPUT)/mm: mmap_default.c mmap_bottomup.c mmap_tests.h $(CC) -o$@ $(CFLAGS) $(LDFLAGS) $^ diff --git a/tools/testing/selftests/riscv/mm/mmap_bottomup.c b/tools/testing/selftests/riscv/mm/mmap_bottomup.c new file mode 100644 index 000000000000..1757d19ca89b --- /dev/null +++ b/tools/testing/selftests/riscv/mm/mmap_bottomup.c @@ -0,0 +1,35 @@ +// SPDX-License-Identifier: GPL-2.0-only +#include +#include + +#include "../../kselftest_harness.h" + +TEST(infinite_rlimit) +{ +// Only works on 64 bit +#if __riscv_xlen == 64 + struct addresses mmap_addresses; + + EXPECT_EQ(BOTTOM_UP, memory_layout()); + + do_mmaps(&mmap_addresses); + + EXPECT_NE(MAP_FAILED, mmap_addresses.no_hint); + EXPECT_NE(MAP_FAILED, mmap_addresses.on_37_addr); + EXPECT_NE(MAP_FAILED, mmap_addresses.on_38_addr); + EXPECT_NE(MAP_FAILED, mmap_addresses.on_46_addr); + EXPECT_NE(MAP_FAILED, mmap_addresses.on_47_addr); + EXPECT_NE(MAP_FAILED, mmap_addresses.on_55_addr); + EXPECT_NE(MAP_FAILED, mmap_addresses.on_56_addr); + + EXPECT_GT(1UL << 47, (unsigned long)mmap_addresses.no_hint); + EXPECT_GT(1UL << 38, (unsigned long)mmap_addresses.on_37_addr); + EXPECT_GT(1UL << 38, (unsigned long)mmap_addresses.on_38_addr); + EXPECT_GT(1UL << 38, (unsigned long)mmap_addresses.on_46_addr); + EXPECT_GT(1UL << 47, (unsigned long)mmap_addresses.on_47_addr); + EXPECT_GT(1UL << 47, (unsigned long)mmap_addresses.on_55_addr); + EXPECT_GT(1UL << 56, (unsigned long)mmap_addresses.on_56_addr); +#endif +} + +TEST_HARNESS_MAIN diff --git a/tools/testing/selftests/riscv/mm/mmap_default.c b/tools/testing/selftests/riscv/mm/mmap_default.c new file mode 100644 index 000000000000..c63c60b9397e --- /dev/null +++ b/tools/testing/selftests/riscv/mm/mmap_default.c @@ -0,0 +1,35 @@ +// SPDX-License-Identifier: GPL-2.0-only +#include +#include + +#include "../../kselftest_harness.h" + +TEST(default_rlimit) +{ +// Only works on 64 bit +#if __riscv_xlen == 64 + struct addresses mmap_addresses; + + EXPECT_EQ(TOP_DOWN, memory_layout()); + + do_mmaps(&mmap_addresses); + + EXPECT_NE(MAP_FAILED, mmap_addresses.no_hint); + EXPECT_NE(MAP_FAILED, mmap_addresses.on_37_addr); + EXPECT_NE(MAP_FAILED, mmap_addresses.on_38_addr); + EXPECT_NE(MAP_FAILED, mmap_addresses.on_46_addr); + EXPECT_NE(MAP_FAILED, mmap_addresses.on_47_addr); + EXPECT_NE(MAP_FAILED, mmap_addresses.on_55_addr); + EXPECT_NE(MAP_FAILED, mmap_addresses.on_56_addr); + + EXPECT_GT(1UL << 47, (unsigned long)mmap_addresses.no_hint); + EXPECT_GT(1UL << 38, (unsigned long)mmap_addresses.on_37_addr); + EXPECT_GT(1UL << 38, (unsigned long)mmap_addresses.on_38_addr); + EXPECT_GT(1UL << 38, (unsigned long)mmap_addresses.on_46_addr); + EXPECT_GT(1UL << 47, (unsigned long)mmap_addresses.on_47_addr); + EXPECT_GT(1UL << 47, (unsigned long)mmap_addresses.on_55_addr); + EXPECT_GT(1UL << 56, (unsigned long)mmap_addresses.on_56_addr); +#endif +} + +TEST_HARNESS_MAIN diff --git a/tools/testing/selftests/riscv/mm/mmap_test.h b/tools/testing/selftests/riscv/mm/mmap_test.h new file mode 100644 index 000000000000..9b8434f62f57 --- /dev/null +++ b/tools/testing/selftests/riscv/mm/mmap_test.h @@ -0,0 +1,64 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef _TESTCASES_MMAP_TEST_H +#define _TESTCASES_MMAP_TEST_H +#include +#include +#include + +#define TOP_DOWN 0 +#define BOTTOM_UP 1 + +struct addresses { + int *no_hint; + int *on_37_addr; + int *on_38_addr; + int *on_46_addr; + int *on_47_addr; + int *on_55_addr; + int *on_56_addr; +}; + +static inline void do_mmaps(struct addresses *mmap_addresses) +{ + /* + * Place all of the hint addresses on the boundaries of mmap + * sv39, sv48, sv57 + * User addresses end at 1<<38, 1<<47, 1<<56 respectively + */ + void *on_37_bits = (void *)(1UL << 37); + void *on_38_bits = (void *)(1UL << 38); + void *on_46_bits = (void *)(1UL << 46); + void *on_47_bits = (void *)(1UL << 47); + void *on_55_bits = (void *)(1UL << 55); + void *on_56_bits = (void *)(1UL << 56); + + int prot = PROT_READ | PROT_WRITE; + int flags = MAP_PRIVATE | MAP_ANONYMOUS; + + mmap_addresses->no_hint = + mmap(NULL, 5 * sizeof(int), prot, flags, 0, 0); + mmap_addresses->on_37_addr = + mmap(on_37_bits, 5 * sizeof(int), prot, flags, 0, 0); + mmap_addresses->on_38_addr = + mmap(on_38_bits, 5 * sizeof(int), prot, flags, 0, 0); + mmap_addresses->on_46_addr = + mmap(on_46_bits, 5 * sizeof(int), prot, flags, 0, 0); + mmap_addresses->on_47_addr = + mmap(on_47_bits, 5 * sizeof(int), prot, flags, 0, 0); + mmap_addresses->on_55_addr = + mmap(on_55_bits, 5 * sizeof(int), prot, flags, 0, 0); + mmap_addresses->on_56_addr = + mmap(on_56_bits, 5 * sizeof(int), prot, flags, 0, 0); +} + +static inline int memory_layout(void) +{ + int prot = PROT_READ | PROT_WRITE; + int flags = MAP_PRIVATE | MAP_ANONYMOUS; + + void *value1 = mmap(NULL, sizeof(int), prot, flags, 0, 0); + void *value2 = mmap(NULL, sizeof(int), prot, flags, 0, 0); + + return value2 > value1; +} +#endif /* _TESTCASES_MMAP_TEST_H */ diff --git a/tools/testing/selftests/riscv/mm/run_mmap.sh b/tools/testing/selftests/riscv/mm/run_mmap.sh new file mode 100755 index 000000000000..ca5ad7c48bad --- /dev/null +++ b/tools/testing/selftests/riscv/mm/run_mmap.sh @@ -0,0 +1,12 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0 + +original_stack_limit=$(ulimit -s) + +./mmap_default + +# Force mmap_bottomup to be ran with bottomup memory due to +# the unlimited stack +ulimit -s unlimited +./mmap_bottomup +ulimit -s $original_stack_limit diff --git a/tools/testing/selftests/riscv/mm/testcases/mmap_bottomup.c b/tools/testing/selftests/riscv/mm/testcases/mmap_bottomup.c deleted file mode 100644 index b29379f7e478..000000000000 --- a/tools/testing/selftests/riscv/mm/testcases/mmap_bottomup.c +++ /dev/null @@ -1,35 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0-only -#include -#include - -#include "../../kselftest_harness.h" - -TEST(infinite_rlimit) -{ -// Only works on 64 bit -#if __riscv_xlen == 64 - struct addresses mmap_addresses; - - EXPECT_EQ(BOTTOM_UP, memory_layout()); - - do_mmaps(&mmap_addresses); - - EXPECT_NE(MAP_FAILED, mmap_addresses.no_hint); - EXPECT_NE(MAP_FAILED, mmap_addresses.on_37_addr); - EXPECT_NE(MAP_FAILED, mmap_addresses.on_38_addr); - EXPECT_NE(MAP_FAILED, mmap_addresses.on_46_addr); - EXPECT_NE(MAP_FAILED, mmap_addresses.on_47_addr); - EXPECT_NE(MAP_FAILED, mmap_addresses.on_55_addr); - EXPECT_NE(MAP_FAILED, mmap_addresses.on_56_addr); - - EXPECT_GT(1UL << 47, (unsigned long)mmap_addresses.no_hint); - EXPECT_GT(1UL << 38, (unsigned long)mmap_addresses.on_37_addr); - EXPECT_GT(1UL << 38, (unsigned long)mmap_addresses.on_38_addr); - EXPECT_GT(1UL << 38, (unsigned long)mmap_addresses.on_46_addr); - EXPECT_GT(1UL << 47, (unsigned long)mmap_addresses.on_47_addr); - EXPECT_GT(1UL << 47, (unsigned long)mmap_addresses.on_55_addr); - EXPECT_GT(1UL << 56, (unsigned long)mmap_addresses.on_56_addr); -#endif -} - -TEST_HARNESS_MAIN diff --git a/tools/testing/selftests/riscv/mm/testcases/mmap_default.c b/tools/testing/selftests/riscv/mm/testcases/mmap_default.c deleted file mode 100644 index d1accb91b726..000000000000 --- a/tools/testing/selftests/riscv/mm/testcases/mmap_default.c +++ /dev/null @@ -1,35 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0-only -#include -#include - -#include "../../kselftest_harness.h" - -TEST(default_rlimit) -{ -// Only works on 64 bit -#if __riscv_xlen == 64 - struct addresses mmap_addresses; - - EXPECT_EQ(TOP_DOWN, memory_layout()); - - do_mmaps(&mmap_addresses); - - EXPECT_NE(MAP_FAILED, mmap_addresses.no_hint); - EXPECT_NE(MAP_FAILED, mmap_addresses.on_37_addr); - EXPECT_NE(MAP_FAILED, mmap_addresses.on_38_addr); - EXPECT_NE(MAP_FAILED, mmap_addresses.on_46_addr); - EXPECT_NE(MAP_FAILED, mmap_addresses.on_47_addr); - EXPECT_NE(MAP_FAILED, mmap_addresses.on_55_addr); - EXPECT_NE(MAP_FAILED, mmap_addresses.on_56_addr); - - EXPECT_GT(1UL << 47, (unsigned long)mmap_addresses.no_hint); - EXPECT_GT(1UL << 38, (unsigned long)mmap_addresses.on_37_addr); - EXPECT_GT(1UL << 38, (unsigned long)mmap_addresses.on_38_addr); - EXPECT_GT(1UL << 38, (unsigned long)mmap_addresses.on_46_addr); - EXPECT_GT(1UL << 47, (unsigned long)mmap_addresses.on_47_addr); - EXPECT_GT(1UL << 47, (unsigned long)mmap_addresses.on_55_addr); - EXPECT_GT(1UL << 56, (unsigned long)mmap_addresses.on_56_addr); -#endif -} - -TEST_HARNESS_MAIN diff --git a/tools/testing/selftests/riscv/mm/testcases/mmap_test.h b/tools/testing/selftests/riscv/mm/testcases/mmap_test.h deleted file mode 100644 index 9b8434f62f57..000000000000 --- a/tools/testing/selftests/riscv/mm/testcases/mmap_test.h +++ /dev/null @@ -1,64 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0-only */ -#ifndef _TESTCASES_MMAP_TEST_H -#define _TESTCASES_MMAP_TEST_H -#include -#include -#include - -#define TOP_DOWN 0 -#define BOTTOM_UP 1 - -struct addresses { - int *no_hint; - int *on_37_addr; - int *on_38_addr; - int *on_46_addr; - int *on_47_addr; - int *on_55_addr; - int *on_56_addr; -}; - -static inline void do_mmaps(struct addresses *mmap_addresses) -{ - /* - * Place all of the hint addresses on the boundaries of mmap - * sv39, sv48, sv57 - * User addresses end at 1<<38, 1<<47, 1<<56 respectively - */ - void *on_37_bits = (void *)(1UL << 37); - void *on_38_bits = (void *)(1UL << 38); - void *on_46_bits = (void *)(1UL << 46); - void *on_47_bits = (void *)(1UL << 47); - void *on_55_bits = (void *)(1UL << 55); - void *on_56_bits = (void *)(1UL << 56); - - int prot = PROT_READ | PROT_WRITE; - int flags = MAP_PRIVATE | MAP_ANONYMOUS; - - mmap_addresses->no_hint = - mmap(NULL, 5 * sizeof(int), prot, flags, 0, 0); - mmap_addresses->on_37_addr = - mmap(on_37_bits, 5 * sizeof(int), prot, flags, 0, 0); - mmap_addresses->on_38_addr = - mmap(on_38_bits, 5 * sizeof(int), prot, flags, 0, 0); - mmap_addresses->on_46_addr = - mmap(on_46_bits, 5 * sizeof(int), prot, flags, 0, 0); - mmap_addresses->on_47_addr = - mmap(on_47_bits, 5 * sizeof(int), prot, flags, 0, 0); - mmap_addresses->on_55_addr = - mmap(on_55_bits, 5 * sizeof(int), prot, flags, 0, 0); - mmap_addresses->on_56_addr = - mmap(on_56_bits, 5 * sizeof(int), prot, flags, 0, 0); -} - -static inline int memory_layout(void) -{ - int prot = PROT_READ | PROT_WRITE; - int flags = MAP_PRIVATE | MAP_ANONYMOUS; - - void *value1 = mmap(NULL, sizeof(int), prot, flags, 0, 0); - void *value2 = mmap(NULL, sizeof(int), prot, flags, 0, 0); - - return value2 > value1; -} -#endif /* _TESTCASES_MMAP_TEST_H */ diff --git a/tools/testing/selftests/riscv/mm/testcases/run_mmap.sh b/tools/testing/selftests/riscv/mm/testcases/run_mmap.sh deleted file mode 100755 index ca5ad7c48bad..000000000000 --- a/tools/testing/selftests/riscv/mm/testcases/run_mmap.sh +++ /dev/null @@ -1,12 +0,0 @@ -#!/bin/sh -# SPDX-License-Identifier: GPL-2.0 - -original_stack_limit=$(ulimit -s) - -./mmap_default - -# Force mmap_bottomup to be ran with bottomup memory due to -# the unlimited stack -ulimit -s unlimited -./mmap_bottomup -ulimit -s $original_stack_limit diff --git a/tools/testing/selftests/user_events/abi_test.c b/tools/testing/selftests/user_events/abi_test.c index 5125c42efe65..f5575ef2007c 100644 --- a/tools/testing/selftests/user_events/abi_test.c +++ b/tools/testing/selftests/user_events/abi_test.c @@ -19,6 +19,7 @@ #include #include "../kselftest_harness.h" +#include "user_events_selftests.h" const char *data_file = "/sys/kernel/tracing/user_events_data"; const char *enable_file = "/sys/kernel/tracing/events/user_events/__abi_event/enable"; @@ -46,7 +47,7 @@ static int change_event(bool enable) return ret; } -static int reg_enable(long *enable, int size, int bit) +static int reg_enable(void *enable, int size, int bit) { struct user_reg reg = {0}; int fd = open(data_file, O_RDWR); @@ -68,7 +69,7 @@ static int reg_enable(long *enable, int size, int bit) return ret; } -static int reg_disable(long *enable, int bit) +static int reg_disable(void *enable, int bit) { struct user_unreg reg = {0}; int fd = open(data_file, O_RDWR); @@ -89,15 +90,21 @@ static int reg_disable(long *enable, int bit) } FIXTURE(user) { - long check; + int check; + long check_long; + bool umount; }; FIXTURE_SETUP(user) { + USER_EVENT_FIXTURE_SETUP(return, self->umount); + change_event(false); self->check = 0; + self->check_long = 0; } FIXTURE_TEARDOWN(user) { + USER_EVENT_FIXTURE_TEARDOWN(self->umount); } TEST_F(user, enablement) { @@ -131,9 +138,9 @@ TEST_F(user, bit_sizes) { #if BITS_PER_LONG == 8 /* Allow 0-64 bits for 64-bit */ - ASSERT_EQ(0, reg_enable(&self->check, sizeof(long), 63)); - ASSERT_NE(0, reg_enable(&self->check, sizeof(long), 64)); - ASSERT_EQ(0, reg_disable(&self->check, 63)); + ASSERT_EQ(0, reg_enable(&self->check_long, sizeof(long), 63)); + ASSERT_NE(0, reg_enable(&self->check_long, sizeof(long), 64)); + ASSERT_EQ(0, reg_disable(&self->check_long, 63)); #endif /* Disallowed sizes (everything beside 4 and 8) */ @@ -195,7 +202,7 @@ static int clone_check(void *check) for (i = 0; i < 10; ++i) { usleep(100000); - if (*(long *)check) + if (*(int *)check) return 0; } diff --git a/tools/testing/selftests/user_events/config b/tools/testing/selftests/user_events/config new file mode 100644 index 000000000000..64f7a9a90cec --- /dev/null +++ b/tools/testing/selftests/user_events/config @@ -0,0 +1 @@ +CONFIG_USER_EVENTS=y diff --git a/tools/testing/selftests/user_events/dyn_test.c b/tools/testing/selftests/user_events/dyn_test.c index 91a4444ad42b..a85980190bea 100644 --- a/tools/testing/selftests/user_events/dyn_test.c +++ b/tools/testing/selftests/user_events/dyn_test.c @@ -15,6 +15,7 @@ #include #include "../kselftest_harness.h" +#include "user_events_selftests.h" const char *abi_file = "/sys/kernel/tracing/user_events_data"; const char *enable_file = "/sys/kernel/tracing/events/user_events/__test_event/enable"; @@ -143,12 +144,16 @@ do { \ FIXTURE(user) { int check; + bool umount; }; FIXTURE_SETUP(user) { + USER_EVENT_FIXTURE_SETUP(return, self->umount); } FIXTURE_TEARDOWN(user) { + USER_EVENT_FIXTURE_TEARDOWN(self->umount); + wait_for_delete(); } diff --git a/tools/testing/selftests/user_events/ftrace_test.c b/tools/testing/selftests/user_events/ftrace_test.c index 5beb0aef1d81..dcd7509fe2e0 100644 --- a/tools/testing/selftests/user_events/ftrace_test.c +++ b/tools/testing/selftests/user_events/ftrace_test.c @@ -16,6 +16,7 @@ #include #include "../kselftest_harness.h" +#include "user_events_selftests.h" const char *data_file = "/sys/kernel/tracing/user_events_data"; const char *status_file = "/sys/kernel/tracing/user_events_status"; @@ -203,9 +204,12 @@ FIXTURE(user) { int data_fd; int enable_fd; int check; + bool umount; }; FIXTURE_SETUP(user) { + USER_EVENT_FIXTURE_SETUP(return, self->umount); + self->status_fd = open(status_file, O_RDONLY); ASSERT_NE(-1, self->status_fd); @@ -216,6 +220,8 @@ FIXTURE_SETUP(user) { } FIXTURE_TEARDOWN(user) { + USER_EVENT_FIXTURE_TEARDOWN(self->umount); + close(self->status_fd); close(self->data_fd); diff --git a/tools/testing/selftests/user_events/perf_test.c b/tools/testing/selftests/user_events/perf_test.c index 8b09be566fa2..5288e768b207 100644 --- a/tools/testing/selftests/user_events/perf_test.c +++ b/tools/testing/selftests/user_events/perf_test.c @@ -17,6 +17,7 @@ #include #include "../kselftest_harness.h" +#include "user_events_selftests.h" const char *data_file = "/sys/kernel/tracing/user_events_data"; const char *id_file = "/sys/kernel/tracing/events/user_events/__test_event/id"; @@ -110,14 +111,19 @@ static int clear(int *check) FIXTURE(user) { int data_fd; int check; + bool umount; }; FIXTURE_SETUP(user) { + USER_EVENT_FIXTURE_SETUP(return, self->umount); + self->data_fd = open(data_file, O_RDWR); ASSERT_NE(-1, self->data_fd); } FIXTURE_TEARDOWN(user) { + USER_EVENT_FIXTURE_TEARDOWN(self->umount); + close(self->data_fd); if (clear(&self->check) != 0) diff --git a/tools/testing/selftests/user_events/user_events_selftests.h b/tools/testing/selftests/user_events/user_events_selftests.h new file mode 100644 index 000000000000..e1c3c063c031 --- /dev/null +++ b/tools/testing/selftests/user_events/user_events_selftests.h @@ -0,0 +1,114 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef _USER_EVENTS_SELFTESTS_H +#define _USER_EVENTS_SELFTESTS_H + +#include +#include +#include +#include +#include + +#include "../kselftest.h" + +static inline void tracefs_unmount(void) +{ + umount("/sys/kernel/tracing"); +} + +static inline bool tracefs_enabled(char **message, bool *fail, bool *umount) +{ + struct stat buf; + int ret; + + *message = ""; + *fail = false; + *umount = false; + + /* Ensure tracefs is installed */ + ret = stat("/sys/kernel/tracing", &buf); + + if (ret == -1) { + *message = "Tracefs is not installed"; + return false; + } + + /* Ensure mounted tracefs */ + ret = stat("/sys/kernel/tracing/README", &buf); + + if (ret == -1 && errno == ENOENT) { + if (mount(NULL, "/sys/kernel/tracing", "tracefs", 0, NULL) != 0) { + *message = "Cannot mount tracefs"; + *fail = true; + return false; + } + + *umount = true; + + ret = stat("/sys/kernel/tracing/README", &buf); + } + + if (ret == -1) { + *message = "Cannot access tracefs"; + *fail = true; + return false; + } + + return true; +} + +static inline bool user_events_enabled(char **message, bool *fail, bool *umount) +{ + struct stat buf; + int ret; + + *message = ""; + *fail = false; + *umount = false; + + if (getuid() != 0) { + *message = "Must be run as root"; + *fail = true; + return false; + } + + if (!tracefs_enabled(message, fail, umount)) + return false; + + /* Ensure user_events is installed */ + ret = stat("/sys/kernel/tracing/user_events_data", &buf); + + if (ret == -1) { + switch (errno) { + case ENOENT: + *message = "user_events is not installed"; + return false; + + default: + *message = "Cannot access user_events_data"; + *fail = true; + return false; + } + } + + return true; +} + +#define USER_EVENT_FIXTURE_SETUP(statement, umount) do { \ + char *message; \ + bool fail; \ + if (!user_events_enabled(&message, &fail, &(umount))) { \ + if (fail) { \ + TH_LOG("Setup failed due to: %s", message); \ + ASSERT_FALSE(fail); \ + } \ + SKIP(statement, "Skipping due to: %s", message); \ + } \ +} while (0) + +#define USER_EVENT_FIXTURE_TEARDOWN(umount) do { \ + if ((umount)) \ + tracefs_unmount(); \ +} while (0) + +#endif /* _USER_EVENTS_SELFTESTS_H */ diff --git a/tools/tracing/rtla/src/timerlat_aa.c b/tools/tracing/rtla/src/timerlat_aa.c index e0ffe69c271c..7093fd5333be 100644 --- a/tools/tracing/rtla/src/timerlat_aa.c +++ b/tools/tracing/rtla/src/timerlat_aa.c @@ -159,6 +159,7 @@ static int timerlat_aa_irq_latency(struct timerlat_aa_data *taa_data, taa_data->thread_nmi_sum = 0; taa_data->thread_irq_sum = 0; taa_data->thread_softirq_sum = 0; + taa_data->thread_thread_sum = 0; taa_data->thread_blocking_duration = 0; taa_data->timer_irq_start_time = 0; taa_data->timer_irq_duration = 0; @@ -337,7 +338,23 @@ static int timerlat_aa_irq_handler(struct trace_seq *s, struct tep_record *recor taa_data->timer_irq_start_time = start; taa_data->timer_irq_duration = duration; - taa_data->timer_irq_start_delay = taa_data->timer_irq_start_time - expected_start; + /* + * We are dealing with two different clock sources: the + * external clock source that timerlat uses as a reference + * and the clock used by the tracer. There are also two + * moments: the time reading the clock and the timer in + * which the event is placed in the buffer (the trace + * event timestamp). If the processor is slow or there + * is some hardware noise, the difference between the + * timestamp and the external clock read can be longer + * than the IRQ handler delay, resulting in a negative + * time. If so, set IRQ start delay as 0. In the end, + * it is less relevant than the noise. + */ + if (expected_start < taa_data->timer_irq_start_time) + taa_data->timer_irq_start_delay = taa_data->timer_irq_start_time - expected_start; + else + taa_data->timer_irq_start_delay = 0; /* * not exit from idle. @@ -528,7 +545,7 @@ static int timerlat_aa_kworker_start_handler(struct trace_seq *s, struct tep_rec static void timerlat_thread_analysis(struct timerlat_aa_data *taa_data, int cpu, int irq_thresh, int thread_thresh) { - unsigned long long exp_irq_ts; + long long exp_irq_ts; int total; int irq; @@ -545,12 +562,15 @@ static void timerlat_thread_analysis(struct timerlat_aa_data *taa_data, int cpu, /* * Expected IRQ arrival time using the trace clock as the base. + * + * TODO: Add a list of previous IRQ, and then run the list backwards. */ exp_irq_ts = taa_data->timer_irq_start_time - taa_data->timer_irq_start_delay; - - if (exp_irq_ts < taa_data->prev_irq_timstamp + taa_data->prev_irq_duration) - printf(" Previous IRQ interference: \t\t up to %9.2f us\n", - ns_to_usf(taa_data->prev_irq_duration)); + if (exp_irq_ts < taa_data->prev_irq_timstamp + taa_data->prev_irq_duration) { + if (taa_data->prev_irq_timstamp < taa_data->timer_irq_start_time) + printf(" Previous IRQ interference: \t\t up to %9.2f us\n", + ns_to_usf(taa_data->prev_irq_duration)); + } /* * The delay that the IRQ suffered before starting. diff --git a/tools/tracing/rtla/src/timerlat_u.c b/tools/tracing/rtla/src/timerlat_u.c index 05e310696dd5..01dbf9a6b5a5 100644 --- a/tools/tracing/rtla/src/timerlat_u.c +++ b/tools/tracing/rtla/src/timerlat_u.c @@ -45,7 +45,7 @@ static int timerlat_u_main(int cpu, struct timerlat_u_params *params) retval = sched_setaffinity(gettid(), sizeof(set), &set); if (retval == -1) { - err_msg("Error setting user thread affinity\n"); + debug_msg("Error setting user thread affinity %d, is the CPU online?\n", cpu); exit(1); } @@ -193,7 +193,9 @@ void *timerlat_u_dispatcher(void *data) procs_count--; } } - break; + + if (!procs_count) + break; } sleep(1);