Rudolf Marek <R.Marek@sh.cvut.cz>
Rui Saraiva <rmps@joel.ist.utl.pt>
Sachin P Sant <ssant@in.ibm.com>
+Sarangdhar Joshi <spjoshi@codeaurora.org>
Sam Ravnborg <sam@mars.ravnborg.org>
Santosh Shilimkar <ssantosh@kernel.org>
Santosh Shilimkar <santosh.shilimkar@oracle.org>
Simon Kelley <simon@thekelleys.org.uk>
Stéphane Witzmann <stephane.witzmann@ubpmes.univ-bpclermont.fr>
Stephen Hemminger <shemminger@osdl.org>
+Subash Abhinov Kasiviswanathan <subashab@codeaurora.org>
+Subhash Jadavani <subhashj@codeaurora.org>
Sudeep Holla <sudeep.holla@arm.com> Sudeep KarkadaNagesha <sudeep.karkadanagesha@arm.com>
Sumit Semwal <sumit.semwal@ti.com>
Tejun Heo <htejun@gmail.com>
Thomas Graf <tgraf@suug.ch>
+Thomas Pedersen <twp@codeaurora.org>
Tony Luck <tony.luck@intel.com>
Tsuneo Yoshioka <Tsuneo.Yoshioka@f-secure.com>
Uwe Kleine-König <ukleinek@informatik.uni-freiburg.de>
+++ /dev/null
-What: /sys/devices/.../deferred_probe
-Date: August 2016
-Contact: Ben Hutchings <ben.hutchings@codethink.co.uk>
-Description:
- The /sys/devices/.../deferred_probe attribute is
- present for all devices. If a driver detects during
- probing a device that a related device is not yet
- ready, it may defer probing of the first device. The
- kernel will retry probing the first device after any
- other device is successfully probed. This attribute
- reads as 1 if probing of this device is currently
- deferred, or 0 otherwise.
"osc_clk"
"xclk"
+* TV Encoder output device
+
+Required properties:
+ - compatible: should be "zte,zx296718-tvenc"
+ - reg: Physical base address and length of the TVENC device IO region
+ - zte,tvenc-power-control: the phandle to SYSCTRL block followed by two
+ integer cells. The first cell is the offset of SYSCTRL register used
+ to control TV Encoder DAC power, and the second cell is the bit mask.
+
Example:
vou: vou@1440000 {
<&topcrm HDMI_XCLK>;
clock-names = "osc_cec", "osc_clk", "xclk";
};
+
+ tvenc: tvenc@2000 {
+ compatible = "zte,zx296718-tvenc";
+ reg = <0x2000 0x1000>;
+ zte,tvenc-power-control = <&sysctrl 0x170 0x10>;
+ };
};
"irq" and "wakeup" names are recognized by I2C core, other names are
left to individual drivers.
+- host-notify
+ device uses SMBus host notify protocol instead of interrupt line.
+
- multi-master
states that there is another master active on this bus. The OS can use
this information to adapt power management to keep the arbitration awake
used by the device. I2C core will assign "irq" interrupt (or the very first
interrupt if not using interrupt names) as primary interrupt for the slave.
+Alternatively, devices supporting SMbus Host Notify, and connected to
+adapters that support this feature, may use "host-notify" property. I2C
+core will create a virtual interrupt for Host Notify and assign it as
+primary interrupt for the slave.
+
Also, if device is marked as a wakeup source, I2C core will set up "wakeup"
interrupt for the device. If "wakeup" interrupt name is not present in the
binding, then primary interrupt will be used as wakeup interrupt.
- compatible: "sigma,smp8758-nand"
- reg: address/size of nfc_reg, nfc_mem, and pbus_reg
- dmas: reference to the DMA channel used by the controller
-- dma-names: "nfc_sbox"
+- dma-names: "rxtx"
- clocks: reference to the system clock
- #address-cells: <1>
- #size-cells: <0>
nandc: nand-controller@2c000 {
compatible = "sigma,smp8758-nand";
- reg = <0x2c000 0x30 0x2d000 0x800 0x20000 0x1000>;
+ reg = <0x2c000 0x30>, <0x2d000 0x800>, <0x20000 0x1000>;
dmas = <&dma0 3>;
- dma-names = "nfc_sbox";
+ dma-names = "rxtx";
clocks = <&clkgen SYS_CLK>;
#address-cells = <1>;
#size-cells = <0>;
Required properties:
- reg - The ID number for the phy, usually a small integer
- ti,rx-internal-delay - RGMII Receive Clock Delay - see dt-bindings/net/ti-dp83867.h
- for applicable values
+ for applicable values. Required only if interface type is
+ PHY_INTERFACE_MODE_RGMII_ID or PHY_INTERFACE_MODE_RGMII_RXID
- ti,tx-internal-delay - RGMII Transmit Clock Delay - see dt-bindings/net/ti-dp83867.h
- for applicable values
+ for applicable values. Required only if interface type is
+ PHY_INTERFACE_MODE_RGMII_ID or PHY_INTERFACE_MODE_RGMII_TXID
- ti,fifo-depth - Transmitt FIFO depth- see dt-bindings/net/ti-dp83867.h
for applicable values
Renesas MSIOF spi controller
Required properties:
-- compatible : "renesas,msiof-<soctype>" for SoCs,
- "renesas,sh-msiof" for SuperH, or
- "renesas,sh-mobile-msiof" for SH Mobile series.
- Examples with soctypes are:
- "renesas,msiof-r8a7790" (R-Car H2)
+- compatible : "renesas,msiof-r8a7790" (R-Car H2)
"renesas,msiof-r8a7791" (R-Car M2-W)
"renesas,msiof-r8a7792" (R-Car V2H)
"renesas,msiof-r8a7793" (R-Car M2-N)
"renesas,msiof-r8a7794" (R-Car E2)
"renesas,msiof-r8a7796" (R-Car M3-W)
"renesas,msiof-sh73a0" (SH-Mobile AG5)
+ "renesas,sh-mobile-msiof" (generic SH-Mobile compatibile device)
+ "renesas,rcar-gen2-msiof" (generic R-Car Gen2 compatible device)
+ "renesas,rcar-gen3-msiof" (generic R-Car Gen3 compatible device)
+ "renesas,sh-msiof" (deprecated)
+
+ When compatible with the generic version, nodes
+ must list the SoC-specific version corresponding
+ to the platform first followed by the generic
+ version.
+
- reg : A list of offsets and lengths of the register sets for
the device.
If only one register set is present, it is to be used
Example:
msiof0: spi@e6e20000 {
- compatible = "renesas,msiof-r8a7791";
+ compatible = "renesas,msiof-r8a7791",
+ "renesas,rcar-gen2-msiof";
reg = <0 0xe6e20000 0 0x0064>;
interrupts = <0 156 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp0_clks R8A7791_CLK_MSIOF0>;
snapshot of a moment, you can see /proc/<pid>/smaps file and scan page table.
It's slow but very precise.
-Table 1-2: Contents of the status files (as of 4.1)
+Table 1-2: Contents of the status files (as of 4.8)
..............................................................................
Field Content
Name filename of the executable
+ Umask file mode creation mask
State state (R is running, S is sleeping, D is sleeping
in an uninterruptible wait, Z is zombie,
T is traced or stopped)
TracerPid PID of process tracing this process (0 if not)
Uid Real, effective, saved set, and file system UIDs
Gid Real, effective, saved set, and file system GIDs
- Umask file mode creation mask
FDSize number of file descriptor slots currently allocated
Groups supplementary group list
NStgid descendant namespace thread group ID hierarchy
VmPeak peak virtual memory size
VmSize total program size
VmLck locked memory size
+ VmPin pinned memory size
VmHWM peak resident set size ("high water mark")
VmRSS size of memory portions. It contains the three
following parts (VmRSS = RssAnon + RssFile + RssShmem)
===
To be added
+Pin Initialization
+==================
+Each pin may have several device entries (virtual pins). On Intel platform,
+the device entries number is dynamically changed. If DP MST hub is connected,
+it is in DP MST mode, and the device entries number is 3. Otherwise, the
+device entries number is 1.
+
+To simplify the implementation, all the device entries will be initialized
+when bootup no matter whether it is in DP MST mode or not.
+
+Connection list
+===============
+DP MST reuses connection list code. The code can be reused because
+device entries on the same pin have the same connection list.
+
+This means DP MST gets the device entry connection list without the
+device entry setting.
Jack
====
--- /dev/null
+Page fragments
+--------------
+
+A page fragment is an arbitrary-length arbitrary-offset area of memory
+which resides within a 0 or higher order compound page. Multiple
+fragments within that page are individually refcounted, in the page's
+reference counter.
+
+The page_frag functions, page_frag_alloc and page_frag_free, provide a
+simple allocation framework for page fragments. This is used by the
+network stack and network device drivers to provide a backing region of
+memory for use as either an sk_buff->head, or to be used in the "frags"
+portion of skb_shared_info.
+
+In order to make use of the page fragment APIs a backing page fragment
+cache is needed. This provides a central point for the fragment allocation
+and tracks allows multiple calls to make use of a cached page. The
+advantage to doing this is that multiple calls to get_page can be avoided
+which can be expensive at allocation time. However due to the nature of
+this caching it is required that any calls to the cache be protected by
+either a per-cpu limitation, or a per-cpu limitation and forcing interrupts
+to be disabled when executing the fragment allocation.
+
+The network stack uses two separate caches per CPU to handle fragment
+allocation. The netdev_alloc_cache is used by callers making use of the
+__netdev_alloc_frag and __netdev_alloc_skb calls. The napi_alloc_cache is
+used by callers of the __napi_alloc_frag and __napi_alloc_skb calls. The
+main difference between these two calls is the context in which they may be
+called. The "netdev" prefixed functions are usable in any context as these
+functions will disable interrupts, while the "napi" prefixed functions are
+only usable within the softirq context.
+
+Many network device drivers use a similar methodology for allocating page
+fragments, but the page fragments are cached at the ring or descriptor
+level. In order to enable these cases it is necessary to provide a generic
+way of tearing down a page cache. For this reason __page_frag_cache_drain
+was implemented. It allows for freeing multiple references from a single
+page via a single call. The advantage to doing this is that it allows for
+cleaning up the multiple references that were added to a page in order to
+avoid calling get_page per allocation.
+
+Alexander Duyck, Nov 29, 2016.
Q: Patchwork web based patch tracking system site
T: SCM tree type and location.
Type is one of: git, hg, quilt, stgit, topgit
- B: Bug tracking system location.
S: Status, one of the following:
Supported: Someone is actually paid to look after this.
Maintained: Someone actually looks after it.
L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
W: http://www.armlinux.org.uk/
S: Maintained
+T: git git://git.armlinux.org.uk/~rmk/linux-arm.git
F: arch/arm/
ARM SUB-ARCHITECTURES
M: Russell King <linux@armlinux.org.uk>
L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
S: Maintained
+T: git git://git.armlinux.org.uk/~rmk/linux-arm.git clkdev
F: arch/arm/include/asm/clkdev.h
F: drivers/clk/clkdev.c
R: Javier Martinez Canillas <javier@osg.samsung.com>
L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
L: linux-samsung-soc@vger.kernel.org (moderated for non-subscribers)
+Q: https://patchwork.kernel.org/project/linux-samsung-soc/list/
S: Maintained
F: arch/arm/boot/dts/s3c*
F: arch/arm/boot/dts/s5p*
S: Supported
F: sound/soc/atmel
-ATMEL DMA DRIVER
-M: Nicolas Ferre <nicolas.ferre@atmel.com>
-L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
-S: Supported
-F: drivers/dma/at_hdmac.c
-F: drivers/dma/at_hdmac_regs.h
-F: include/linux/platform_data/dma-atmel.h
-
ATMEL XDMA DRIVER
M: Ludovic Desroches <ludovic.desroches@atmel.com>
L: linux-arm-kernel@lists.infradead.org
DRM DRIVER FOR BOCHS VIRTUAL GPU
M: Gerd Hoffmann <kraxel@redhat.com>
-S: Odd Fixes
+L: virtualization@lists.linux-foundation.org
+T: git git://git.kraxel.org/linux drm-qemu
+S: Maintained
F: drivers/gpu/drm/bochs/
DRM DRIVER FOR QEMU'S CIRRUS DEVICE
M: Dave Airlie <airlied@redhat.com>
-S: Odd Fixes
+M: Gerd Hoffmann <kraxel@redhat.com>
+L: virtualization@lists.linux-foundation.org
+T: git git://git.kraxel.org/linux drm-qemu
+S: Obsolete
+W: https://www.kraxel.org/blog/2014/10/qemu-using-cirrus-considered-harmful/
F: drivers/gpu/drm/cirrus/
RADEON and AMDGPU DRM DRIVERS
M: Alex Deucher <alexander.deucher@amd.com>
M: Christian König <christian.koenig@amd.com>
-L: dri-devel@lists.freedesktop.org
+L: amd-gfx@lists.freedesktop.org
T: git git://people.freedesktop.org/~agd5f/linux
S: Supported
F: drivers/gpu/drm/radeon/
DRM DRIVER FOR QXL VIRTUAL GPU
M: Dave Airlie <airlied@redhat.com>
-S: Odd Fixes
+M: Gerd Hoffmann <kraxel@redhat.com>
+L: virtualization@lists.linux-foundation.org
+T: git git://git.kraxel.org/linux drm-qemu
+S: Maintained
F: drivers/gpu/drm/qxl/
F: include/uapi/drm/qxl_drm.h
F: Documentation/devicetree/bindings/net/dsa/marvell.txt
MARVELL ARMADA DRM SUPPORT
-M: Russell King <rmk+kernel@armlinux.org.uk>
+M: Russell King <linux@armlinux.org.uk>
S: Maintained
+T: git git://git.armlinux.org.uk/~rmk/linux-arm.git drm-armada-devel
+T: git git://git.armlinux.org.uk/~rmk/linux-arm.git drm-armada-fixes
F: drivers/gpu/drm/armada/
F: include/uapi/drm/armada_drm.h
F: Documentation/devicetree/bindings/display/armada/
F: drivers/tty/serial/atmel_serial.c
F: include/linux/atmel_serial.h
+MICROCHIP / ATMEL DMA DRIVER
+M: Ludovic Desroches <ludovic.desroches@microchip.com>
+L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
+L: dmaengine@vger.kernel.org
+S: Supported
+F: drivers/dma/at_hdmac.c
+F: drivers/dma/at_hdmac_regs.h
+F: include/linux/platform_data/dma-atmel.h
+
MICROCHIP / ATMEL ISC DRIVER
M: Songjun Wu <songjun.wu@microchip.com>
L: linux-media@vger.kernel.org
F: drivers/nfc/nxp-nci
NXP TDA998X DRM DRIVER
-M: Russell King <rmk+kernel@armlinux.org.uk>
+M: Russell King <linux@armlinux.org.uk>
S: Supported
+T: git git://git.armlinux.org.uk/~rmk/linux-arm.git drm-tda998x-devel
+T: git git://git.armlinux.org.uk/~rmk/linux-arm.git drm-tda998x-fixes
F: drivers/gpu/drm/i2c/tda998x_drv.c
F: include/drm/i2c/tda998x.h
M: Gerd Hoffmann <kraxel@redhat.com>
L: dri-devel@lists.freedesktop.org
L: virtualization@lists.linux-foundation.org
+T: git git://git.kraxel.org/linux drm-qemu
S: Maintained
F: drivers/gpu/drm/virtio/
F: include/uapi/linux/virtio_gpu.h
X86 PLATFORM DRIVERS
M: Darren Hart <dvhart@infradead.org>
+M: Andy Shevchenko <andy@infradead.org>
L: platform-driver-x86@vger.kernel.org
T: git git://git.infradead.org/users/dvhart/linux-platform-drivers-x86.git
S: Maintained
ZBUD COMPRESSED PAGE ALLOCATOR
M: Seth Jennings <sjenning@redhat.com>
+M: Dan Streetman <ddstreet@ieee.org>
L: linux-mm@kvack.org
S: Maintained
F: mm/zbud.c
ZSWAP COMPRESSED SWAP CACHING
M: Seth Jennings <sjenning@redhat.com>
+M: Dan Streetman <ddstreet@ieee.org>
L: linux-mm@kvack.org
S: Maintained
F: mm/zswap.c
VERSION = 4
PATCHLEVEL = 10
SUBLEVEL = 0
-EXTRAVERSION = -rc3
-NAME = Roaring Lionus
+EXTRAVERSION = -rc5
+NAME = Anniversary Edition
# *DOCUMENTATION*
# To see a list of typical targets execute "make help"
select HAVE_KPROBES
select HAVE_KRETPROBES
select HAVE_MEMBLOCK
- select HAVE_MOD_ARCH_SPECIFIC if ARC_DW2_UNWIND
+ select HAVE_MOD_ARCH_SPECIFIC
select HAVE_OPROFILE
select HAVE_PERF_EVENTS
select HANDLE_DOMAIN_IRQ
#define ARC_REG_IC_PTAG_HI 0x1F
/* Bit val in IC_CTRL */
-#define IC_CTRL_CACHE_DISABLE 0x1
+#define IC_CTRL_DIS 0x1
/* Data cache related Auxiliary registers */
#define ARC_REG_DC_BCR 0x72 /* Build Config reg */
#define ARC_REG_DC_PTAG_HI 0x5F
/* Bit val in DC_CTRL */
-#define DC_CTRL_INV_MODE_FLUSH 0x40
-#define DC_CTRL_FLUSH_STATUS 0x100
+#define DC_CTRL_DIS 0x001
+#define DC_CTRL_INV_MODE_FLUSH 0x040
+#define DC_CTRL_FLUSH_STATUS 0x100
/*System-level cache (L2 cache) related Auxiliary registers */
#define ARC_REG_SLC_CFG 0x901
#define ARC_REG_SLC_RGN_END 0x916
/* Bit val in SLC_CONTROL */
+#define SLC_CTRL_DIS 0x001
#define SLC_CTRL_IM 0x040
-#define SLC_CTRL_DISABLE 0x001
#define SLC_CTRL_BUSY 0x100
#define SLC_CTRL_RGN_OP_INV 0x200
;
; Now manually save: r12, sp, fp, gp, r25
+ PUSH r30
PUSH r12
; Saving pt_regs->sp correctly requires some extra work due to the way
POPAX AUX_USER_SP
1:
POP r12
+ POP r30
.endm
#include <asm-generic/module.h>
-#ifdef CONFIG_ARC_DW2_UNWIND
struct mod_arch_specific {
+#ifdef CONFIG_ARC_DW2_UNWIND
void *unw_info;
int unw_sec_idx;
+#endif
const char *secstr;
};
-#endif
#define MODULE_PROC_FAMILY "ARC700"
unsigned long fp;
unsigned long sp; /* user/kernel sp depending on where we came from */
- unsigned long r12;
+ unsigned long r12, r30;
/*------- Below list auto saved by h/w -----------*/
unsigned long r0, r1, r2, r3, r4, r5, r6, r7, r8, r9, r10, r11;
void setup_processor(void);
void __init setup_arch_memory(void);
+long __init arc_get_mem_sz(void);
/* Helpers used in arc_*_mumbojumbo routines */
#define IS_AVAIL1(v, s) ((v) ? s : "")
static void arcv2_irq_mask(struct irq_data *data)
{
- write_aux_reg(AUX_IRQ_SELECT, data->irq);
+ write_aux_reg(AUX_IRQ_SELECT, data->hwirq);
write_aux_reg(AUX_IRQ_ENABLE, 0);
}
static void arcv2_irq_unmask(struct irq_data *data)
{
- write_aux_reg(AUX_IRQ_SELECT, data->irq);
+ write_aux_reg(AUX_IRQ_SELECT, data->hwirq);
write_aux_reg(AUX_IRQ_ENABLE, 1);
}
void arcv2_irq_enable(struct irq_data *data)
{
/* set default priority */
- write_aux_reg(AUX_IRQ_SELECT, data->irq);
+ write_aux_reg(AUX_IRQ_SELECT, data->hwirq);
write_aux_reg(AUX_IRQ_PRIORITY, ARCV2_IRQ_DEF_PRIO);
/*
unsigned int ienb;
ienb = read_aux_reg(AUX_IENABLE);
- ienb &= ~(1 << data->irq);
+ ienb &= ~(1 << data->hwirq);
write_aux_reg(AUX_IENABLE, ienb);
}
unsigned int ienb;
ienb = read_aux_reg(AUX_IENABLE);
- ienb |= (1 << data->irq);
+ ienb |= (1 << data->hwirq);
write_aux_reg(AUX_IENABLE, ienb);
}
#include <linux/smp.h>
#include <linux/irq.h>
+#include <linux/irqchip/chained_irq.h>
#include <linux/spinlock.h>
#include <soc/arc/mcip.h>
#include <asm/irqflags-arcv2.h>
static void idu_cascade_isr(struct irq_desc *desc)
{
struct irq_domain *idu_domain = irq_desc_get_handler_data(desc);
+ struct irq_chip *core_chip = irq_desc_get_chip(desc);
irq_hw_number_t core_hwirq = irqd_to_hwirq(irq_desc_get_irq_data(desc));
irq_hw_number_t idu_hwirq = core_hwirq - idu_first_hwirq;
+ chained_irq_enter(core_chip, desc);
generic_handle_irq(irq_find_mapping(idu_domain, idu_hwirq));
+ chained_irq_exit(core_chip, desc);
}
static int idu_irq_map(struct irq_domain *d, unsigned int virq, irq_hw_number_t hwirq)
#ifdef CONFIG_ARC_DW2_UNWIND
mod->arch.unw_sec_idx = 0;
mod->arch.unw_info = NULL;
- mod->arch.secstr = secstr;
#endif
+ mod->arch.secstr = secstr;
return 0;
}
}
+#ifdef CONFIG_ARC_DW2_UNWIND
if (strcmp(module->arch.secstr+sechdrs[tgtsec].sh_name, ".eh_frame") == 0)
module->arch.unw_sec_idx = tgtsec;
+#endif
return 0;
static int l2_line_sz;
static int ioc_exists;
-int slc_enable = 1, ioc_enable = 0;
+int slc_enable = 1, ioc_enable = 1;
unsigned long perip_base = ARC_UNCACHED_ADDR_SPACE; /* legacy value for boot */
unsigned long perip_end = 0xFFFFFFFF; /* legacy value */
/*
* For ARC700 MMUv3 I-cache and D-cache flushes
- * Also reused for HS38 aliasing I-cache configuration
+ * - ARC700 programming model requires paddr and vaddr be passed in seperate
+ * AUX registers (*_IV*L and *_PTAG respectively) irrespective of whether the
+ * caches actually alias or not.
+ * - For HS38, only the aliasing I-cache configuration uses the PTAG reg
+ * (non aliasing I-cache version doesn't; while D-cache can't possibly alias)
*/
static inline
void __cache_line_loop_v3(phys_addr_t paddr, unsigned long vaddr,
__after_dc_op(op);
}
+static inline void __dc_disable(void)
+{
+ const int r = ARC_REG_DC_CTRL;
+
+ __dc_entire_op(OP_FLUSH_N_INV);
+ write_aux_reg(r, read_aux_reg(r) | DC_CTRL_DIS);
+}
+
+static void __dc_enable(void)
+{
+ const int r = ARC_REG_DC_CTRL;
+
+ write_aux_reg(r, read_aux_reg(r) & ~DC_CTRL_DIS);
+}
+
/* For kernel mappings cache operation: index is same as paddr */
#define __dc_line_op_k(p, sz, op) __dc_line_op(p, p, sz, op)
#else
#define __dc_entire_op(op)
+#define __dc_disable()
+#define __dc_enable()
#define __dc_line_op(paddr, vaddr, sz, op)
#define __dc_line_op_k(paddr, sz, op)
#endif
}
+noinline static void slc_entire_op(const int op)
+{
+ unsigned int ctrl, r = ARC_REG_SLC_CTRL;
+
+ ctrl = read_aux_reg(r);
+
+ if (!(op & OP_FLUSH)) /* i.e. OP_INV */
+ ctrl &= ~SLC_CTRL_IM; /* clear IM: Disable flush before Inv */
+ else
+ ctrl |= SLC_CTRL_IM;
+
+ write_aux_reg(r, ctrl);
+
+ write_aux_reg(ARC_REG_SLC_INVALIDATE, 1);
+
+ /* Important to wait for flush to complete */
+ while (read_aux_reg(r) & SLC_CTRL_BUSY);
+}
+
+static inline void arc_slc_disable(void)
+{
+ const int r = ARC_REG_SLC_CTRL;
+
+ slc_entire_op(OP_FLUSH_N_INV);
+ write_aux_reg(r, read_aux_reg(r) | SLC_CTRL_DIS);
+}
+
+static inline void arc_slc_enable(void)
+{
+ const int r = ARC_REG_SLC_CTRL;
+
+ write_aux_reg(r, read_aux_reg(r) & ~SLC_CTRL_DIS);
+}
+
/***********************************************************
* Exported APIs
*/
return 0;
}
-void arc_cache_init(void)
+/*
+ * IO-Coherency (IOC) setup rules:
+ *
+ * 1. Needs to be at system level, so only once by Master core
+ * Non-Masters need not be accessing caches at that time
+ * - They are either HALT_ON_RESET and kick started much later or
+ * - if run on reset, need to ensure that arc_platform_smp_wait_to_boot()
+ * doesn't perturb caches or coherency unit
+ *
+ * 2. caches (L1 and SLC) need to be purged (flush+inv) before setting up IOC,
+ * otherwise any straggler data might behave strangely post IOC enabling
+ *
+ * 3. All Caches need to be disabled when setting up IOC to elide any in-flight
+ * Coherency transactions
+ */
+noinline void __init arc_ioc_setup(void)
{
- unsigned int __maybe_unused cpu = smp_processor_id();
- char str[256];
+ unsigned int ap_sz;
- printk(arc_cache_mumbojumbo(0, str, sizeof(str)));
+ /* Flush + invalidate + disable L1 dcache */
+ __dc_disable();
+
+ /* Flush + invalidate SLC */
+ if (read_aux_reg(ARC_REG_SLC_BCR))
+ slc_entire_op(OP_FLUSH_N_INV);
+
+ /* IOC Aperture start: TDB: handle non default CONFIG_LINUX_LINK_BASE */
+ write_aux_reg(ARC_REG_IO_COH_AP0_BASE, 0x80000);
/*
- * Only master CPU needs to execute rest of function:
- * - Assume SMP so all cores will have same cache config so
- * any geomtry checks will be same for all
- * - IOC setup / dma callbacks only need to be setup once
+ * IOC Aperture size:
+ * decoded as 2 ^ (SIZE + 2) KB: so setting 0x11 implies 512M
+ * TBD: fix for PGU + 1GB of low mem
+ * TBD: fix for PAE
*/
- if (cpu)
- return;
+ ap_sz = order_base_2(arc_get_mem_sz()/1024) - 2;
+ write_aux_reg(ARC_REG_IO_COH_AP0_SIZE, ap_sz);
+
+ write_aux_reg(ARC_REG_IO_COH_PARTIAL, 1);
+ write_aux_reg(ARC_REG_IO_COH_ENABLE, 1);
+
+ /* Re-enable L1 dcache */
+ __dc_enable();
+}
+
+void __init arc_cache_init_master(void)
+{
+ unsigned int __maybe_unused cpu = smp_processor_id();
if (IS_ENABLED(CONFIG_ARC_HAS_ICACHE)) {
struct cpuinfo_arc_cache *ic = &cpuinfo_arc700[cpu].icache;
}
}
- if (is_isa_arcv2() && l2_line_sz && !slc_enable) {
-
- /* IM set : flush before invalidate */
- write_aux_reg(ARC_REG_SLC_CTRL,
- read_aux_reg(ARC_REG_SLC_CTRL) | SLC_CTRL_IM);
+ /* Note that SLC disable not formally supported till HS 3.0 */
+ if (is_isa_arcv2() && l2_line_sz && !slc_enable)
+ arc_slc_disable();
- write_aux_reg(ARC_REG_SLC_INVALIDATE, 1);
-
- /* Important to wait for flush to complete */
- while (read_aux_reg(ARC_REG_SLC_CTRL) & SLC_CTRL_BUSY);
- write_aux_reg(ARC_REG_SLC_CTRL,
- read_aux_reg(ARC_REG_SLC_CTRL) | SLC_CTRL_DISABLE);
- }
+ if (is_isa_arcv2() && ioc_enable)
+ arc_ioc_setup();
if (is_isa_arcv2() && ioc_enable) {
- /* IO coherency base - 0x8z */
- write_aux_reg(ARC_REG_IO_COH_AP0_BASE, 0x80000);
- /* IO coherency aperture size - 512Mb: 0x8z-0xAz */
- write_aux_reg(ARC_REG_IO_COH_AP0_SIZE, 0x11);
- /* Enable partial writes */
- write_aux_reg(ARC_REG_IO_COH_PARTIAL, 1);
- /* Enable IO coherency */
- write_aux_reg(ARC_REG_IO_COH_ENABLE, 1);
-
__dma_cache_wback_inv = __dma_cache_wback_inv_ioc;
__dma_cache_inv = __dma_cache_inv_ioc;
__dma_cache_wback = __dma_cache_wback_ioc;
__dma_cache_wback = __dma_cache_wback_l1;
}
}
+
+void __ref arc_cache_init(void)
+{
+ unsigned int __maybe_unused cpu = smp_processor_id();
+ char str[256];
+
+ printk(arc_cache_mumbojumbo(0, str, sizeof(str)));
+
+ /*
+ * Only master CPU needs to execute rest of function:
+ * - Assume SMP so all cores will have same cache config so
+ * any geomtry checks will be same for all
+ * - IOC setup / dma callbacks only need to be setup once
+ */
+ if (!cpu)
+ arc_cache_init_master();
+}
EXPORT_SYMBOL(node_data);
#endif
+long __init arc_get_mem_sz(void)
+{
+ return low_mem_sz;
+}
+
/* User can over-ride above with "mem=nnn[KkMm]" in cmdline */
static int __init setup_mem_sz(char *str)
{
sun8i-a83t-allwinner-h8homlet-v2.dtb \
sun8i-a83t-cubietruck-plus.dtb \
sun8i-h3-bananapi-m2-plus.dtb \
+ sun8i-h3-nanopi-m1.dtb \
sun8i-h3-nanopi-neo.dtb \
sun8i-h3-orangepi-2.dtb \
sun8i-h3-orangepi-lite.dtb \
AM33XX_IOPAD(0x8fc, PIN_INPUT_PULLUP | MUX_MODE0) /* (G16) mmc0_dat0.mmc0_dat0 */
AM33XX_IOPAD(0x900, PIN_INPUT_PULLUP | MUX_MODE0) /* (G17) mmc0_clk.mmc0_clk */
AM33XX_IOPAD(0x904, PIN_INPUT_PULLUP | MUX_MODE0) /* (G18) mmc0_cmd.mmc0_cmd */
- AM33XX_IOPAD(0x960, PIN_INPUT_PULLUP | MUX_MODE5) /* (C15) spi0_cs1.mmc0_sdcd */
>;
};
axi {
compatible = "simple-bus";
- ranges = <0x00000000 0x18000000 0x0011c40a>;
+ ranges = <0x00000000 0x18000000 0x0011c40c>;
#address-cells = <1>;
#size-cells = <1>;
#size-cells = <1>;
compatible = "m25p64";
spi-max-frequency = <30000000>;
+ m25p,fast-read;
reg = <0>;
partition@0 {
label = "U-Boot-SPL";
phy-names = "sata-phy";
clocks = <&sata_ref_clk>;
ti,hwmods = "sata";
+ ports-implemented = <0x1>;
};
rtc: rtc@48838000 {
ti,rx-internal-delay = <DP83867_RGMIIDCTL_2_25_NS>;
ti,tx-internal-delay = <DP83867_RGMIIDCTL_250_PS>;
ti,fifo-depth = <DP83867_PHYCR_FIFO_DEPTH_8_B_NIB>;
- ti,min-output-imepdance;
+ ti,min-output-impedance;
};
};
compatible = "fsl,imx6q-nitrogen6_max-sgtl5000",
"fsl,imx-audio-sgtl5000";
model = "imx6q-nitrogen6_max-sgtl5000";
- pinctrl-names = "default";
- pinctrl-0 = <&pinctrl_sgtl5000>;
ssi-controller = <&ssi1>;
audio-codec = <&codec>;
audio-routing =
codec: sgtl5000@0a {
compatible = "fsl,sgtl5000";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_sgtl5000>;
reg = <0x0a>;
clocks = <&clks IMX6QDL_CLK_CKO>;
VDDA-supply = <®_2p5v>;
compatible = "fsl,imx6q-nitrogen6_som2-sgtl5000",
"fsl,imx-audio-sgtl5000";
model = "imx6q-nitrogen6_som2-sgtl5000";
- pinctrl-names = "default";
- pinctrl-0 = <&pinctrl_sgtl5000>;
ssi-controller = <&ssi1>;
audio-codec = <&codec>;
audio-routing =
codec: sgtl5000@0a {
compatible = "fsl,sgtl5000";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_sgtl5000>;
reg = <0x0a>;
clocks = <&clks IMX6QDL_CLK_CKO>;
VDDA-supply = <®_2p5v>;
&mmc1 {
interrupts-extended = <&intc 83 &omap3_pmx_core 0x11a>;
pinctrl-names = "default";
- pinctrl-0 = <&mmc1_pins &mmc1_cd>;
+ pinctrl-0 = <&mmc1_pins>;
wp-gpios = <&gpio4 30 GPIO_ACTIVE_HIGH>; /* gpio_126 */
cd-gpios = <&gpio4 14 IRQ_TYPE_LEVEL_LOW>; /* gpio_110 */
vmmc-supply = <&vmmc1>;
OMAP3_CORE1_IOPAD(0x214a, PIN_INPUT | MUX_MODE0) /* sdmmc1_dat1.sdmmc1_dat1 */
OMAP3_CORE1_IOPAD(0x214c, PIN_INPUT | MUX_MODE0) /* sdmmc1_dat2.sdmmc1_dat2 */
OMAP3_CORE1_IOPAD(0x214e, PIN_INPUT | MUX_MODE0) /* sdmmc1_dat3.sdmmc1_dat3 */
- OMAP3_CORE1_IOPAD(0x2132, PIN_INPUT_PULLUP | MUX_MODE4) /* cam_strobe.gpio_126 sdmmc1_wp*/
+ OMAP3_CORE1_IOPAD(0x2132, PIN_INPUT_PULLUP | MUX_MODE4) /* cam_strobe.gpio_126 */
+ OMAP3_CORE1_IOPAD(0x212c, PIN_INPUT_PULLUP | MUX_MODE4) /* cam_d11.gpio_110 */
>;
};
OMAP3_WKUP_IOPAD(0x2a16, PIN_OUTPUT | PIN_OFF_OUTPUT_LOW | MUX_MODE4) /* sys_boot6.gpio_8 */
>;
};
-
- mmc1_cd: pinmux_mmc1_cd {
- pinctrl-single,pins = <
- OMAP3_WKUP_IOPAD(0x212c, PIN_INPUT_PULLUP | MUX_MODE4) /* cam_d11.gpio_110 */
- >;
- };
};
phy-names = "sata-phy";
clocks = <&sata_ref_clk>;
ti,hwmods = "sata";
+ ports-implemented = <0x1>;
};
dss: dss@58000000 {
};
amba {
- compatible = "arm,amba-bus";
+ compatible = "simple-bus";
#address-cells = <1>;
#size-cells = <1>;
ranges;
cpu-supply = <®_dcdc3>;
};
+&de {
+ status = "okay";
+};
+
&ehci0 {
status = "okay";
};
de: display-engine {
compatible = "allwinner,sun6i-a31-display-engine";
allwinner,pipelines = <&fe0>;
+ status = "disabled";
};
soc@01c00000 {
};
&pio {
- mmc2_pins_nrst: mmc2@0 {
+ mmc2_pins_nrst: mmc2-rst-pin {
allwinner,pins = "PC16";
allwinner,function = "gpio_out";
allwinner,drive = <SUN4I_PINCTRL_10_MA>;
CONFIG_DW_WATCHDOG=y
CONFIG_DIGICOLOR_WATCHDOG=y
CONFIG_BCM2835_WDT=y
-CONFIG_BCM47XX_WATCHDOG=y
+CONFIG_BCM47XX_WDT=y
CONFIG_BCM7038_WDT=m
CONFIG_BCM_KONA_WDT=y
CONFIG_MFD_ACT8945A=y
CONFIG_RASPBERRYPI_FIRMWARE=y
CONFIG_EFI_VARS=m
CONFIG_EFI_CAPSULE_LOADER=m
-CONFIG_CONFIG_BCM47XX_NVRAM=y
+CONFIG_BCM47XX_NVRAM=y
CONFIG_BCM47XX_SPROM=y
CONFIG_EXT4_FS=y
CONFIG_AUTOFS4_FS=y
CONFIG_NETFILTER=y
CONFIG_NF_CONNTRACK=m
CONFIG_NF_CONNTRACK_EVENTS=y
-CONFIG_NF_CT_PROTO_DCCP=m
-CONFIG_NF_CT_PROTO_SCTP=m
-CONFIG_NF_CT_PROTO_UDPLITE=m
+CONFIG_NF_CT_PROTO_DCCP=y
+CONFIG_NF_CT_PROTO_SCTP=y
+CONFIG_NF_CT_PROTO_UDPLITE=y
CONFIG_NF_CONNTRACK_AMANDA=m
CONFIG_NF_CONNTRACK_FTP=m
CONFIG_NF_CONNTRACK_H323=m
#define ARM_CPU_XSCALE_ARCH_V2 0x4000
#define ARM_CPU_XSCALE_ARCH_V3 0x6000
+/* Qualcomm implemented cores */
+#define ARM_CPU_PART_SCORPION 0x510002d0
+
extern unsigned int processor_id;
#ifdef CONFIG_CPU_CP15
#define ftrace_return_address(n) return_address(n)
+#define ARCH_HAS_SYSCALL_MATCH_SYM_NAME
+
+static inline bool arch_syscall_match_sym_name(const char *sym,
+ const char *name)
+{
+ if (!strcmp(sym, "sys_mmap2"))
+ sym = "sys_mmap_pgoff";
+ else if (!strcmp(sym, "sys_statfs64_wrapper"))
+ sym = "sys_statfs64";
+ else if (!strcmp(sym, "sys_fstatfs64_wrapper"))
+ sym = "sys_fstatfs64";
+ else if (!strcmp(sym, "sys_arm_fadvise64_64"))
+ sym = "sys_fadvise64_64";
+
+ /* Ignore case since sym may start with "SyS" instead of "sys" */
+ return !strcasecmp(sym, name);
+}
+
#endif /* ifndef __ASSEMBLY__ */
#endif /* _ASM_ARM_FTRACE */
+++ /dev/null
-#ifndef _ASM_TYPES_H
-#define _ASM_TYPES_H
-
-#include <asm-generic/int-ll64.h>
-
-/*
- * The C99 types uintXX_t that are usually defined in 'stdint.h' are not as
- * unambiguous on ARM as you would expect. For the types below, there is a
- * difference on ARM between GCC built for bare metal ARM, GCC built for glibc
- * and the kernel itself, which results in build errors if you try to build with
- * -ffreestanding and include 'stdint.h' (such as when you include 'arm_neon.h'
- * in order to use NEON intrinsics)
- *
- * As the typedefs for these types in 'stdint.h' are based on builtin defines
- * supplied by GCC, we can tweak these to align with the kernel's idea of those
- * types, so 'linux/types.h' and 'stdint.h' can be safely included from the same
- * source file (provided that -ffreestanding is used).
- *
- * int32_t uint32_t uintptr_t
- * bare metal GCC long unsigned long unsigned int
- * glibc GCC int unsigned int unsigned int
- * kernel int unsigned int unsigned long
- */
-
-#ifdef __INT32_TYPE__
-#undef __INT32_TYPE__
-#define __INT32_TYPE__ int
-#endif
-
-#ifdef __UINT32_TYPE__
-#undef __UINT32_TYPE__
-#define __UINT32_TYPE__ unsigned int
-#endif
-
-#ifdef __UINTPTR_TYPE__
-#undef __UINTPTR_TYPE__
-#define __UINTPTR_TYPE__ unsigned long
-#endif
-
-#endif /* _ASM_TYPES_H */
return false;
}
+static inline bool has_vhe(void)
+{
+ return false;
+}
+
/* The section containing the hypervisor idmap text */
extern char __hyp_idmap_text_start[];
extern char __hyp_idmap_text_end[];
--- /dev/null
+#ifndef _UAPI_ASM_TYPES_H
+#define _UAPI_ASM_TYPES_H
+
+#include <asm-generic/int-ll64.h>
+
+/*
+ * The C99 types uintXX_t that are usually defined in 'stdint.h' are not as
+ * unambiguous on ARM as you would expect. For the types below, there is a
+ * difference on ARM between GCC built for bare metal ARM, GCC built for glibc
+ * and the kernel itself, which results in build errors if you try to build with
+ * -ffreestanding and include 'stdint.h' (such as when you include 'arm_neon.h'
+ * in order to use NEON intrinsics)
+ *
+ * As the typedefs for these types in 'stdint.h' are based on builtin defines
+ * supplied by GCC, we can tweak these to align with the kernel's idea of those
+ * types, so 'linux/types.h' and 'stdint.h' can be safely included from the same
+ * source file (provided that -ffreestanding is used).
+ *
+ * int32_t uint32_t uintptr_t
+ * bare metal GCC long unsigned long unsigned int
+ * glibc GCC int unsigned int unsigned int
+ * kernel int unsigned int unsigned long
+ */
+
+#ifdef __INT32_TYPE__
+#undef __INT32_TYPE__
+#define __INT32_TYPE__ int
+#endif
+
+#ifdef __UINT32_TYPE__
+#undef __UINT32_TYPE__
+#define __UINT32_TYPE__ unsigned int
+#endif
+
+#ifdef __UINTPTR_TYPE__
+#undef __UINTPTR_TYPE__
+#define __UINTPTR_TYPE__ unsigned long
+#endif
+
+#endif /* _UAPI_ASM_TYPES_H */
return 0;
}
+ /*
+ * Scorpion CPUs (at least those in APQ8060) seem to set DBGPRSR.SPD
+ * whenever a WFI is issued, even if the core is not powered down, in
+ * violation of the architecture. When DBGPRSR.SPD is set, accesses to
+ * breakpoint and watchpoint registers are treated as undefined, so
+ * this results in boot time and runtime failures when these are
+ * accessed and we unexpectedly take a trap.
+ *
+ * It's not clear if/how this can be worked around, so we blacklist
+ * Scorpion CPUs to avoid these issues.
+ */
+ if (read_cpuid_part() == ARM_CPU_PART_SCORPION) {
+ pr_info("Scorpion CPU detected. Hardware breakpoints and watchpoints disabled\n");
+ return 0;
+ }
+
has_ossr = core_has_os_save_restore();
/* Determine how many BRPs/WRPs are available. */
*/
#include <linux/preempt.h>
#include <linux/smp.h>
+#include <linux/uaccess.h>
#include <asm/smp_plat.h>
#include <asm/tlbflush.h>
static inline void ipi_flush_tlb_page(void *arg)
{
struct tlb_args *ta = (struct tlb_args *)arg;
+ unsigned int __ua_flags = uaccess_save_and_enable();
local_flush_tlb_page(ta->ta_vma, ta->ta_start);
+
+ uaccess_restore(__ua_flags);
}
static inline void ipi_flush_tlb_kernel_page(void *arg)
static inline void ipi_flush_tlb_range(void *arg)
{
struct tlb_args *ta = (struct tlb_args *)arg;
+ unsigned int __ua_flags = uaccess_save_and_enable();
local_flush_tlb_range(ta->ta_vma, ta->ta_start, ta->ta_end);
+
+ uaccess_restore(__ua_flags);
}
static inline void ipi_flush_tlb_kernel_range(void *arg)
__cpu_init_hyp_mode(pgd_ptr, hyp_stack_ptr, vector_ptr);
__cpu_init_stage2();
+ if (is_kernel_in_hyp_mode())
+ kvm_timer_init_vhe();
+
kvm_arm_init_debug();
}
#include "soc.h"
#define OMAP1_DMA_BASE (0xfffed800)
-#define OMAP1_LOGICAL_DMA_CH_COUNT 17
static u32 enable_1510_mode;
goto exit_iounmap;
}
- d->lch_count = OMAP1_LOGICAL_DMA_CH_COUNT;
-
/* Valid attributes for omap1 plus processors */
if (cpu_is_omap15xx())
d->dev_caps = ENABLE_1510_MODE;
d->dev_caps |= CLEAR_CSR_ON_READ;
d->dev_caps |= IS_WORD_16;
- if (cpu_is_omap15xx())
- d->chan_count = 9;
- else if (cpu_is_omap16xx() || cpu_is_omap7xx()) {
- if (!(d->dev_caps & ENABLE_1510_MODE))
- d->chan_count = 16;
+ /* available logical channels */
+ if (cpu_is_omap15xx()) {
+ d->lch_count = 9;
+ } else {
+ if (d->dev_caps & ENABLE_1510_MODE)
+ d->lch_count = 9;
else
- d->chan_count = 9;
+ d->lch_count = 16;
}
p = dma_plat_info;
.nshutdown_gpio = 162,
.dev_name = "/dev/ttyO1",
.flow_cntrl = 1,
- .baud_rate = 300000,
+ .baud_rate = 3000000,
};
static struct platform_device wl128x_device = {
*/
bool prcmu_is_cpu_in_wfi(int cpu)
{
- return readl(PRCM_ARM_WFI_STANDBY) & cpu ? PRCM_ARM_WFI_STANDBY_WFI1 :
- PRCM_ARM_WFI_STANDBY_WFI0;
+ return readl(PRCM_ARM_WFI_STANDBY) &
+ (cpu ? PRCM_ARM_WFI_STANDBY_WFI1 : PRCM_ARM_WFI_STANDBY_WFI0);
}
/*
};
};
+&scpi_clocks {
+ status = "disabled";
+};
+
&uart_AO {
status = "okay";
pinctrl-0 = <&uart_ao_a_pins>;
mboxes = <&mailbox 1 &mailbox 2>;
shmem = <&cpu_scp_lpri &cpu_scp_hpri>;
- clocks {
+ scpi_clocks: clocks {
compatible = "arm,scpi-clocks";
scpi_dvfs: scpi_clocks@0 {
};
amba {
- compatible = "arm,amba-bus";
+ compatible = "simple-bus";
#address-cells = <1>;
#size-cells = <1>;
ranges;
stdout-path = "serial0:115200n8";
};
- memory {
+ memory@0 {
device_type = "memory";
reg = <0x0 0x0 0x0 0x40000000>;
};
<1 10 0xf08>;
};
- amba_apu {
+ amba_apu: amba_apu@0 {
compatible = "simple-bus";
#address-cells = <2>;
#size-cells = <1>;
};
i2c0: i2c@ff020000 {
- compatible = "cdns,i2c-r1p10";
+ compatible = "cdns,i2c-r1p14", "cdns,i2c-r1p10";
status = "disabled";
interrupt-parent = <&gic>;
interrupts = <0 17 4>;
};
i2c1: i2c@ff030000 {
- compatible = "cdns,i2c-r1p10";
+ compatible = "cdns,i2c-r1p14", "cdns,i2c-r1p10";
status = "disabled";
interrupt-parent = <&gic>;
interrupts = <0 18 4>;
/*
* Pseudo-ops for PC-relative adr/ldr/str <reg>, <symbol> where
- * <symbol> is within the range +/- 4 GB of the PC.
+ * <symbol> is within the range +/- 4 GB of the PC when running
+ * in core kernel context. In module context, a movz/movk sequence
+ * is used, since modules may be loaded far away from the kernel
+ * when KASLR is in effect.
*/
/*
* @dst: destination register (64 bit wide)
* @sym: name of the symbol
- * @tmp: optional scratch register to be used if <dst> == sp, which
- * is not allowed in an adrp instruction
*/
- .macro adr_l, dst, sym, tmp=
- .ifb \tmp
+ .macro adr_l, dst, sym
+#ifndef MODULE
adrp \dst, \sym
add \dst, \dst, :lo12:\sym
- .else
- adrp \tmp, \sym
- add \dst, \tmp, :lo12:\sym
- .endif
+#else
+ movz \dst, #:abs_g3:\sym
+ movk \dst, #:abs_g2_nc:\sym
+ movk \dst, #:abs_g1_nc:\sym
+ movk \dst, #:abs_g0_nc:\sym
+#endif
.endm
/*
* the address
*/
.macro ldr_l, dst, sym, tmp=
+#ifndef MODULE
.ifb \tmp
adrp \dst, \sym
ldr \dst, [\dst, :lo12:\sym]
adrp \tmp, \sym
ldr \dst, [\tmp, :lo12:\sym]
.endif
+#else
+ .ifb \tmp
+ adr_l \dst, \sym
+ ldr \dst, [\dst]
+ .else
+ adr_l \tmp, \sym
+ ldr \dst, [\tmp]
+ .endif
+#endif
.endm
/*
* while <src> needs to be preserved.
*/
.macro str_l, src, sym, tmp
+#ifndef MODULE
adrp \tmp, \sym
str \src, [\tmp, :lo12:\sym]
+#else
+ adr_l \tmp, \sym
+ str \src, [\tmp]
+#endif
.endm
/*
#define _virt_addr_valid(kaddr) pfn_valid(__pa(kaddr) >> PAGE_SHIFT)
#else
#define __virt_to_pgoff(kaddr) (((u64)(kaddr) & ~PAGE_OFFSET) / PAGE_SIZE * sizeof(struct page))
-#define __page_to_voff(page) (((u64)(page) & ~VMEMMAP_START) * PAGE_SIZE / sizeof(struct page))
+#define __page_to_voff(kaddr) (((u64)(kaddr) & ~VMEMMAP_START) * PAGE_SIZE / sizeof(struct page))
#define page_to_virt(page) ((void *)((__page_to_voff(page)) | PAGE_OFFSET))
#define virt_to_page(vaddr) ((struct page *)((__virt_to_pgoff(vaddr)) | VMEMMAP_START))
#include <asm/ptrace.h>
#include <asm/sections.h>
#include <asm/sysreg.h>
+#include <asm/cpufeature.h>
/*
* __boot_cpu_mode records what mode CPUs were booted in.
return read_sysreg(CurrentEL) == CurrentEL_EL2;
}
+static inline bool has_vhe(void)
+{
+ if (cpus_have_const_cap(ARM64_HAS_VIRT_HOST_EXTN))
+ return true;
+
+ return false;
+}
+
#ifdef CONFIG_ARM64_VHE
extern void verify_cpu_run_el(void);
#else
__uint128_t vregs[32];
__u32 fpsr;
__u32 fpcr;
+ __u32 __reserved[2];
};
struct user_hwdebug_state {
mov x0, sp
mov x1, #BAD_SYNC
mov x2, x25
- bl bad_mode
+ bl bad_el0_sync
b ret_to_user
ENDPROC(el0_sync)
/* (address, ctrl) registers */
limit = regset->n * regset->size;
while (count && offset < limit) {
+ if (count < PTRACE_HBP_ADDR_SZ)
+ return -EINVAL;
ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, &addr,
offset, offset + PTRACE_HBP_ADDR_SZ);
if (ret)
return ret;
offset += PTRACE_HBP_ADDR_SZ;
+ if (!count)
+ break;
ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, &ctrl,
offset, offset + PTRACE_HBP_CTRL_SZ);
if (ret)
const void *kbuf, const void __user *ubuf)
{
int ret;
- struct user_pt_regs newregs;
+ struct user_pt_regs newregs = task_pt_regs(target)->user_regs;
ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, &newregs, 0, -1);
if (ret)
const void *kbuf, const void __user *ubuf)
{
int ret;
- struct user_fpsimd_state newstate;
+ struct user_fpsimd_state newstate =
+ target->thread.fpsimd_state.user_fpsimd;
ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, &newstate, 0, -1);
if (ret)
const void *kbuf, const void __user *ubuf)
{
int ret;
- unsigned long tls;
+ unsigned long tls = target->thread.tp_value;
ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, &tls, 0, -1);
if (ret)
unsigned int pos, unsigned int count,
const void *kbuf, const void __user *ubuf)
{
- int syscallno, ret;
+ int syscallno = task_pt_regs(target)->syscallno;
+ int ret;
ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, &syscallno, 0, -1);
if (ret)
const void __user *ubuf)
{
int ret;
- compat_ulong_t tls;
+ compat_ulong_t tls = target->thread.tp_value;
ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, &tls, 0, -1);
if (ret)
}
/*
- * bad_mode handles the impossible case in the exception vector.
+ * bad_mode handles the impossible case in the exception vector. This is always
+ * fatal.
*/
asmlinkage void bad_mode(struct pt_regs *regs, int reason, unsigned int esr)
{
- siginfo_t info;
- void __user *pc = (void __user *)instruction_pointer(regs);
console_verbose();
pr_crit("Bad mode in %s handler detected on CPU%d, code 0x%08x -- %s\n",
handler[reason], smp_processor_id(), esr,
esr_get_class_string(esr));
+
+ die("Oops - bad mode", regs, 0);
+ local_irq_disable();
+ panic("bad mode");
+}
+
+/*
+ * bad_el0_sync handles unexpected, but potentially recoverable synchronous
+ * exceptions taken from EL0. Unlike bad_mode, this returns.
+ */
+asmlinkage void bad_el0_sync(struct pt_regs *regs, int reason, unsigned int esr)
+{
+ siginfo_t info;
+ void __user *pc = (void __user *)instruction_pointer(regs);
+ console_verbose();
+
+ pr_crit("Bad EL0 synchronous exception detected on CPU%d, code 0x%08x -- %s\n",
+ smp_processor_id(), esr, esr_get_class_string(esr));
__show_regs(regs);
info.si_signo = SIGILL;
info.si_code = ILL_ILLOPC;
info.si_addr = pc;
- arm64_notify_die("Oops - bad mode", regs, &info, 0);
+ current->thread.fault_address = 0;
+ current->thread.fault_code = 0;
+
+ force_sig_info(info.si_signo, &info, current);
}
void __pte_error(const char *file, int line, unsigned long val)
ncontig = find_num_contig(vma->vm_mm, addr, cpte,
*cpte, &pgsize);
for (i = 0; i < ncontig; ++i, ++cpte, addr += pgsize) {
- changed = ptep_set_access_flags(vma, addr, cpte,
+ changed |= ptep_set_access_flags(vma, addr, cpte,
pfn_pte(pfn,
hugeprot),
dirty);
if (swiotlb_force == SWIOTLB_FORCE ||
max_pfn > (arm64_dma_phys_limit >> PAGE_SHIFT))
swiotlb_init(1);
+ else
+ swiotlb_force = SWIOTLB_NO_FORCE;
set_max_mapnr(pfn_to_page(max_pfn) - mem_map);
#define atomic64_sub_and_test(i,v) (atomic64_sub_return((i), (v)) == 0)
#define atomic64_dec_and_test(v) (atomic64_dec_return((v)) == 0)
#define atomic64_inc_and_test(v) (atomic64_inc_return((v)) == 0)
-
+#define atomic64_inc_not_zero(v) atomic64_add_unless((v), 1, 0)
#define atomic_cmpxchg(v, old, new) (cmpxchg(&(v)->counter, old, new))
#define atomic_xchg(v, new) (xchg(&(v)->counter, new))
return c;
}
+static inline int atomic64_add_unless(atomic64_t *v, long long i, long long u)
+{
+ long long c, old;
+
+ c = atomic64_read(v);
+ for (;;) {
+ if (unlikely(c == u))
+ break;
+ old = atomic64_cmpxchg(v, c, c + i);
+ if (likely(old == c))
+ break;
+ c = old;
+ }
+ return c != u;
+}
+
+static inline long long atomic64_dec_if_positive(atomic64_t *v)
+{
+ long long c, old, dec;
+
+ c = atomic64_read(v);
+ for (;;) {
+ dec = c - 1;
+ if (unlikely(dec < 0))
+ break;
+ old = atomic64_cmpxchg((v), c, dec);
+ if (likely(old == c))
+ break;
+ c = old;
+ }
+ return dec;
+}
+
#define ATOMIC_OP(op) \
static inline int atomic_fetch_##op(int i, atomic_t *v) \
{ \
struct task_struct;
struct thread_struct;
-#if !defined(CONFIG_LAZY_SAVE_FPU)
+#if defined(CONFIG_FPU) && !defined(CONFIG_LAZY_SAVE_FPU)
struct fpu_state_struct;
extern asmlinkage void fpu_save(struct fpu_state_struct *);
#define switch_fpu(prev, next) \
#ifdef CONFIG_HUGETLB_PAGE
static inline int hash__hugepd_ok(hugepd_t hpd)
{
+ unsigned long hpdval = hpd_val(hpd);
/*
* if it is not a pte and have hugepd shift mask
* set, then it is a hugepd directory pointer
*/
- if (!(hpd.pd & _PAGE_PTE) &&
- ((hpd.pd & HUGEPD_SHIFT_MASK) != 0))
+ if (!(hpdval & _PAGE_PTE) &&
+ ((hpdval & HUGEPD_SHIFT_MASK) != 0))
return true;
return false;
}
unsigned long phys);
extern void hash__vmemmap_remove_mapping(unsigned long start,
unsigned long page_size);
+
+int hash__create_section_mapping(unsigned long start, unsigned long end);
+int hash__remove_section_mapping(unsigned long start, unsigned long end);
+
#endif /* !__ASSEMBLY__ */
#endif /* __KERNEL__ */
#endif /* _ASM_POWERPC_BOOK3S_64_HASH_H */
* We have only four bits to encode, MMU page size
*/
BUILD_BUG_ON((MMU_PAGE_COUNT - 1) > 0xf);
- return __va(hpd.pd & HUGEPD_ADDR_MASK);
+ return __va(hpd_val(hpd) & HUGEPD_ADDR_MASK);
}
static inline unsigned int hugepd_mmu_psize(hugepd_t hpd)
{
- return (hpd.pd & HUGEPD_SHIFT_MASK) >> 2;
+ return (hpd_val(hpd) & HUGEPD_SHIFT_MASK) >> 2;
}
static inline unsigned int hugepd_shift(hugepd_t hpd)
{
BUG_ON(!hugepd_ok(hpd));
#ifdef CONFIG_PPC_8xx
- return (pte_t *)__va(hpd.pd & ~(_PMD_PAGE_MASK | _PMD_PRESENT_MASK));
+ return (pte_t *)__va(hpd_val(hpd) &
+ ~(_PMD_PAGE_MASK | _PMD_PRESENT_MASK));
#else
- return (pte_t *)((hpd.pd & ~HUGEPD_SHIFT_MASK) | PD_HUGE);
+ return (pte_t *)((hpd_val(hpd) &
+ ~HUGEPD_SHIFT_MASK) | PD_HUGE);
#endif
}
static inline unsigned int hugepd_shift(hugepd_t hpd)
{
#ifdef CONFIG_PPC_8xx
- return ((hpd.pd & _PMD_PAGE_MASK) >> 1) + 17;
+ return ((hpd_val(hpd) & _PMD_PAGE_MASK) >> 1) + 17;
#else
- return hpd.pd & HUGEPD_SHIFT_MASK;
+ return hpd_val(hpd) & HUGEPD_SHIFT_MASK;
#endif
}
static inline int hugepd_ok(hugepd_t hpd)
{
#ifdef CONFIG_PPC_8xx
- return ((hpd.pd & 0x4) != 0);
+ return ((hpd_val(hpd) & 0x4) != 0);
#else
- return (hpd.pd > 0);
+ /* We clear the top bit to indicate hugepd */
+ return ((hpd_val(hpd) & PD_HUGE) == 0);
#endif
}
#include <asm/pgtable-types.h>
#endif
-typedef struct { signed long pd; } hugepd_t;
#ifndef CONFIG_HUGETLB_PAGE
#define is_hugepd(pdep) (0)
#define pgd_huge(pgd) (0)
#endif /* CONFIG_HUGETLB_PAGE */
-#define __hugepd(x) ((hugepd_t) { (x) })
-
struct page;
extern void clear_user_page(void *page, unsigned long vaddr, struct page *pg);
extern void copy_user_page(void *to, void *from, unsigned long vaddr,
#define PPMU_HAS_SSLOT 0x00000020 /* Has sampled slot in MMCRA */
#define PPMU_HAS_SIER 0x00000040 /* Has SIER */
#define PPMU_ARCH_207S 0x00000080 /* PMC is architecture v2.07S */
+#define PPMU_NO_SIAR 0x00000100 /* Do not use SIAR */
/*
* Values for flags to get_alternatives()
return pmd_raw(old) == prev;
}
+typedef struct { __be64 pdbe; } hugepd_t;
+#define __hugepd(x) ((hugepd_t) { cpu_to_be64(x) })
+
+static inline unsigned long hpd_val(hugepd_t x)
+{
+ return be64_to_cpu(x.pdbe);
+}
+
#endif /* _ASM_POWERPC_PGTABLE_BE_TYPES_H */
}
#endif
+typedef struct { unsigned long pd; } hugepd_t;
+#define __hugepd(x) ((hugepd_t) { (x) })
+static inline unsigned long hpd_val(hugepd_t x)
+{
+ return x.pd;
+}
+
#endif /* _ASM_POWERPC_PGTABLE_TYPES_H */
#define PPC_INST_MCRXR 0x7c000400
#define PPC_INST_MCRXR_MASK 0xfc0007fe
#define PPC_INST_MFSPR_PVR 0x7c1f42a6
-#define PPC_INST_MFSPR_PVR_MASK 0xfc1fffff
+#define PPC_INST_MFSPR_PVR_MASK 0xfc1ffffe
#define PPC_INST_MFTMR 0x7c0002dc
#define PPC_INST_MSGSND 0x7c00019c
#define PPC_INST_MSGCLR 0x7c0001dc
#define PPC_INST_RFDI 0x4c00004e
#define PPC_INST_RFMCI 0x4c00004c
#define PPC_INST_MFSPR_DSCR 0x7c1102a6
-#define PPC_INST_MFSPR_DSCR_MASK 0xfc1fffff
+#define PPC_INST_MFSPR_DSCR_MASK 0xfc1ffffe
#define PPC_INST_MTSPR_DSCR 0x7c1103a6
-#define PPC_INST_MTSPR_DSCR_MASK 0xfc1fffff
+#define PPC_INST_MTSPR_DSCR_MASK 0xfc1ffffe
#define PPC_INST_MFSPR_DSCR_USER 0x7c0302a6
-#define PPC_INST_MFSPR_DSCR_USER_MASK 0xfc1fffff
+#define PPC_INST_MFSPR_DSCR_USER_MASK 0xfc1ffffe
#define PPC_INST_MTSPR_DSCR_USER 0x7c0303a6
-#define PPC_INST_MTSPR_DSCR_USER_MASK 0xfc1fffff
+#define PPC_INST_MTSPR_DSCR_USER_MASK 0xfc1ffffe
#define PPC_INST_MFVSRD 0x7c000066
#define PPC_INST_MTVSRD 0x7c000166
#define PPC_INST_SLBFEE 0x7c0007a7
*
* For pHyp, we have to enable IO for log retrieval. Otherwise,
* 0xFF's is always returned from PCI config space.
+ *
+ * When the @severity is EEH_LOG_PERM, the PE is going to be
+ * removed. Prior to that, the drivers for devices included in
+ * the PE will be closed. The drivers rely on working IO path
+ * to bring the devices to quiet state. Otherwise, PCI traffic
+ * from those devices after they are removed is like to cause
+ * another unexpected EEH error.
*/
if (!(pe->type & EEH_PE_PHB)) {
- if (eeh_has_flag(EEH_ENABLE_IO_FOR_LOG))
+ if (eeh_has_flag(EEH_ENABLE_IO_FOR_LOG) ||
+ severity == EEH_LOG_PERM)
eeh_pci_enable(pe, EEH_OPT_THAW_MMIO);
/*
flush_fp_to_thread(target);
+ for (i = 0; i < 32 ; i++)
+ buf[i] = target->thread.TS_FPR(i);
+ buf[32] = target->thread.fp_state.fpscr;
+
/* copy to local buffer then write that out */
i = user_regset_copyin(&pos, &count, &kbuf, &ubuf, buf, 0, -1);
if (i)
flush_altivec_to_thread(target);
flush_vsx_to_thread(target);
+ for (i = 0; i < 32 ; i++)
+ buf[i] = target->thread.fp_state.fpr[i][TS_VSRLOWOFFSET];
+
ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf,
buf, 0, 32 * sizeof(double));
if (!ret)
flush_fp_to_thread(target);
flush_altivec_to_thread(target);
+ for (i = 0; i < 32; i++)
+ buf[i] = target->thread.TS_CKFPR(i);
+ buf[32] = target->thread.ckfp_state.fpscr;
+
/* copy to local buffer then write that out */
i = user_regset_copyin(&pos, &count, &kbuf, &ubuf, buf, 0, -1);
if (i)
flush_altivec_to_thread(target);
flush_vsx_to_thread(target);
+ for (i = 0; i < 32 ; i++)
+ buf[i] = target->thread.ckfp_state.fpr[i][TS_VSRLOWOFFSET];
+
ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf,
buf, 0, 32 * sizeof(double));
if (!ret)
}
#ifdef CONFIG_MEMORY_HOTPLUG
-int create_section_mapping(unsigned long start, unsigned long end)
+int hash__create_section_mapping(unsigned long start, unsigned long end)
{
int rc = htab_bolt_mapping(start, end, __pa(start),
pgprot_val(PAGE_KERNEL), mmu_linear_psize,
return rc;
}
-int remove_section_mapping(unsigned long start, unsigned long end)
+int hash__remove_section_mapping(unsigned long start, unsigned long end)
{
int rc = htab_remove_mapping(start, end, mmu_linear_psize,
mmu_kernel_ssize);
int hugepd_ok(hugepd_t hpd)
{
bool is_hugepd;
+ unsigned long hpdval;
+
+ hpdval = hpd_val(hpd);
/*
* We should not find this format in page directory, warn otherwise.
*/
- is_hugepd = (((hpd.pd & 0x3) == 0x0) && ((hpd.pd & HUGEPD_SHIFT_MASK) != 0));
+ is_hugepd = (((hpdval & 0x3) == 0x0) && ((hpdval & HUGEPD_SHIFT_MASK) != 0));
WARN(is_hugepd, "Found wrong page directory format\n");
return 0;
}
static unsigned nr_gpages;
#endif
-#define hugepd_none(hpd) ((hpd).pd == 0)
+#define hugepd_none(hpd) (hpd_val(hpd) == 0)
pte_t *huge_pte_offset(struct mm_struct *mm, unsigned long addr)
{
for (i = 0; i < num_hugepd; i++, hpdp++) {
if (unlikely(!hugepd_none(*hpdp)))
break;
- else
+ else {
#ifdef CONFIG_PPC_BOOK3S_64
- hpdp->pd = __pa(new) |
- (shift_to_mmu_psize(pshift) << 2);
+ *hpdp = __hugepd(__pa(new) |
+ (shift_to_mmu_psize(pshift) << 2));
#elif defined(CONFIG_PPC_8xx)
- hpdp->pd = __pa(new) |
- (pshift == PAGE_SHIFT_8M ? _PMD_PAGE_8M :
- _PMD_PAGE_512K) |
- _PMD_PRESENT;
+ *hpdp = __hugepd(__pa(new) |
+ (pshift == PAGE_SHIFT_8M ? _PMD_PAGE_8M :
+ _PMD_PAGE_512K) | _PMD_PRESENT);
#else
/* We use the old format for PPC_FSL_BOOK3E */
- hpdp->pd = ((unsigned long)new & ~PD_HUGE) | pshift;
+ *hpdp = __hugepd(((unsigned long)new & ~PD_HUGE) | pshift);
#endif
+ }
}
/* If we bailed from the for loop early, an error occurred, clean up */
if (i < num_hugepd) {
for (i = i - 1 ; i >= 0; i--, hpdp--)
- hpdp->pd = 0;
+ *hpdp = __hugepd(0);
kmem_cache_free(cachep, new);
}
spin_unlock(&mm->page_table_lock);
return;
for (i = 0; i < num_hugepd; i++, hpdp++)
- hpdp->pd = 0;
+ *hpdp = __hugepd(0);
if (shift >= pdshift)
hugepd_free(tlb, hugepte);
* if we have pdshift and shift value same, we don't
* use pgt cache for hugepd.
*/
- if (pdshift > shift) {
+ if (pdshift > shift)
pgtable_cache_add(pdshift - shift, NULL);
- if (!PGT_CACHE(pdshift - shift))
- panic("hugetlbpage_init(): could not create "
- "pgtable cache for %d bit pagesize\n", shift);
- }
#if defined(CONFIG_PPC_FSL_BOOK3E) || defined(CONFIG_PPC_8xx)
else if (!hugepte_cache) {
/*
else if (mmu_psize_defs[MMU_PAGE_2M].shift)
HPAGE_SHIFT = mmu_psize_defs[MMU_PAGE_2M].shift;
#endif
- else
- panic("%s: Unable to set default huge page size\n", __func__);
-
return 0;
}
align = max_t(unsigned long, align, minalign);
name = kasprintf(GFP_KERNEL, "pgtable-2^%d", shift);
new = kmem_cache_create(name, table_size, align, 0, ctor);
+ if (!new)
+ panic("Could not allocate pgtable cache for order %d", shift);
+
kfree(name);
pgtable_cache[shift - 1] = new;
+
pr_debug("Allocated pgtable cache for order %d\n", shift);
}
{
pgtable_cache_add(PGD_INDEX_SIZE, pgd_ctor);
- if (PMD_INDEX_SIZE && !PGT_CACHE(PMD_INDEX_SIZE))
+ if (PMD_CACHE_INDEX && !PGT_CACHE(PMD_CACHE_INDEX))
pgtable_cache_add(PMD_CACHE_INDEX, pmd_ctor);
/*
* In all current configs, when the PUD index exists it's the
*/
if (PUD_INDEX_SIZE && !PGT_CACHE(PUD_INDEX_SIZE))
pgtable_cache_add(PUD_INDEX_SIZE, pud_ctor);
-
- if (!PGT_CACHE(PGD_INDEX_SIZE))
- panic("Couldn't allocate pgd cache");
- if (PMD_INDEX_SIZE && !PGT_CACHE(PMD_INDEX_SIZE))
- panic("Couldn't allocate pmd pgtable caches");
- if (PUD_INDEX_SIZE && !PGT_CACHE(PUD_INDEX_SIZE))
- panic("Couldn't allocate pud pgtable caches");
}
else if (mmu_hash_ops.hpte_clear_all)
mmu_hash_ops.hpte_clear_all();
}
+
+#ifdef CONFIG_MEMORY_HOTPLUG
+int create_section_mapping(unsigned long start, unsigned long end)
+{
+ if (radix_enabled())
+ return -ENODEV;
+
+ return hash__create_section_mapping(start, end);
+}
+
+int remove_section_mapping(unsigned long start, unsigned long end)
+{
+ if (radix_enabled())
+ return -ENODEV;
+
+ return hash__remove_section_mapping(start, end);
+}
+#endif /* CONFIG_MEMORY_HOTPLUG */
*/
if (TRAP(regs) != 0xf00)
use_siar = 0;
+ else if ((ppmu->flags & PPMU_NO_SIAR))
+ use_siar = 0;
else if (marked)
use_siar = 1;
else if ((ppmu->flags & PPMU_NO_CONT_SAMPLING))
EVENT(PM_ICT_NOSLOT_CYC, 0x100f8)
EVENT(PM_CMPLU_STALL, 0x1e054)
EVENT(PM_INST_CMPL, 0x00002)
-EVENT(PM_BRU_CMPL, 0x40060)
+EVENT(PM_BRU_CMPL, 0x10012)
EVENT(PM_BR_MPRED_CMPL, 0x400f6)
/* All L1 D cache load references counted at finish, gated by reject */
.bhrb_filter_map = power9_bhrb_filter_map,
.get_constraint = isa207_get_constraint,
.disable_pmc = isa207_disable_pmc,
- .flags = PPMU_HAS_SIER | PPMU_ARCH_207S,
+ .flags = PPMU_NO_SIAR | PPMU_ARCH_207S,
.n_generic = ARRAY_SIZE(power9_generic_events),
.generic_events = power9_generic_events,
.cache_events = &power9_cache_events,
#include <asm/xics.h>
#include <asm/io.h>
#include <asm/opal.h>
+#include <asm/kvm_ppc.h>
static void icp_opal_teardown_cpu(void)
{
* Should we be flagging idle loop instead?
* Or creating some task to be scheduled?
*/
- opal_int_eoi((0x00 << 24) | XICS_IPI);
+ if (opal_int_eoi((0x00 << 24) | XICS_IPI) > 0)
+ force_external_irq_replay();
+}
+
+static unsigned int icp_opal_get_xirr(void)
+{
+ unsigned int kvm_xirr;
+ __be32 hw_xirr;
+ int64_t rc;
+
+ /* Handle an interrupt latched by KVM first */
+ kvm_xirr = kvmppc_get_xics_latch();
+ if (kvm_xirr)
+ return kvm_xirr;
+
+ /* Then ask OPAL */
+ rc = opal_int_get_xirr(&hw_xirr, false);
+ if (rc < 0)
+ return 0;
+ return be32_to_cpu(hw_xirr);
}
static unsigned int icp_opal_get_irq(void)
unsigned int xirr;
unsigned int vec;
unsigned int irq;
- int64_t rc;
- rc = opal_int_get_xirr(&xirr, false);
- if (rc < 0)
- return 0;
- xirr = be32_to_cpu(xirr);
+ xirr = icp_opal_get_xirr();
vec = xirr & 0x00ffffff;
if (vec == XICS_IRQ_SPURIOUS)
return 0;
xics_mask_unknown_vec(vec);
/* We might learn about it later, so EOI it */
- opal_int_eoi(xirr);
+ if (opal_int_eoi(xirr) > 0)
+ force_external_irq_replay();
return 0;
}
CONFIG_CMA_DEBUG=y
CONFIG_CMA_DEBUGFS=y
CONFIG_MEM_SOFT_DIRTY=y
-CONFIG_ZPOOL=m
+CONFIG_ZSWAP=y
CONFIG_ZBUD=m
CONFIG_ZSMALLOC=m
CONFIG_ZSMALLOC_STAT=y
CONFIG_NF_CONNTRACK_EVENTS=y
CONFIG_NF_CONNTRACK_TIMEOUT=y
CONFIG_NF_CONNTRACK_TIMESTAMP=y
-CONFIG_NF_CT_PROTO_DCCP=m
-CONFIG_NF_CT_PROTO_UDPLITE=m
CONFIG_NF_CONNTRACK_AMANDA=m
CONFIG_NF_CONNTRACK_FTP=m
CONFIG_NF_CONNTRACK_H323=m
CONFIG_NFT_EXTHDR=m
CONFIG_NFT_META=m
CONFIG_NFT_CT=m
-CONFIG_NFT_RBTREE=m
-CONFIG_NFT_HASH=m
CONFIG_NFT_COUNTER=m
CONFIG_NFT_LOG=m
CONFIG_NFT_LIMIT=m
CONFIG_NFT_NAT=m
CONFIG_NFT_COMPAT=m
+CONFIG_NFT_HASH=m
CONFIG_NETFILTER_XT_SET=m
CONFIG_NETFILTER_XT_TARGET_AUDIT=m
CONFIG_NETFILTER_XT_TARGET_CHECKSUM=m
CONFIG_NETFILTER_XT_MATCH_RATEEST=m
CONFIG_NETFILTER_XT_MATCH_REALM=m
CONFIG_NETFILTER_XT_MATCH_RECENT=m
-CONFIG_NETFILTER_XT_MATCH_SOCKET=m
CONFIG_NETFILTER_XT_MATCH_STATE=m
CONFIG_NETFILTER_XT_MATCH_STATISTIC=m
CONFIG_NETFILTER_XT_MATCH_STRING=m
CONFIG_IP_VS_FTP=m
CONFIG_IP_VS_PE_SIP=m
CONFIG_NF_CONNTRACK_IPV4=m
-# CONFIG_NF_CONNTRACK_PROC_COMPAT is not set
CONFIG_NF_TABLES_IPV4=m
CONFIG_NFT_CHAIN_ROUTE_IPV4=m
CONFIG_NF_TABLES_ARP=m
CONFIG_IFB=m
CONFIG_MACVLAN=m
CONFIG_MACVTAP=m
-CONFIG_IPVLAN=m
CONFIG_VXLAN=m
CONFIG_TUN=m
CONFIG_VETH=m
CONFIG_EXT4_FS=y
CONFIG_EXT4_FS_POSIX_ACL=y
CONFIG_EXT4_FS_SECURITY=y
+CONFIG_EXT4_ENCRYPTION=y
CONFIG_JBD2_DEBUG=y
CONFIG_JFS_FS=m
CONFIG_JFS_POSIX_ACL=y
CONFIG_DEBUG_LOCKDEP=y
CONFIG_DEBUG_ATOMIC_SLEEP=y
CONFIG_DEBUG_LOCKING_API_SELFTESTS=y
-CONFIG_DEBUG_LIST=y
CONFIG_DEBUG_SG=y
CONFIG_DEBUG_NOTIFIERS=y
CONFIG_DEBUG_CREDENTIALS=y
CONFIG_RCU_TORTURE_TEST=m
CONFIG_RCU_CPU_STALL_TIMEOUT=300
CONFIG_NOTIFIER_ERROR_INJECTION=m
-CONFIG_CPU_NOTIFIER_ERROR_INJECT=m
CONFIG_PM_NOTIFIER_ERROR_INJECT=m
CONFIG_FAULT_INJECTION=y
CONFIG_FAILSLAB=y
CONFIG_BLK_DEV_IO_TRACE=y
CONFIG_UPROBE_EVENT=y
CONFIG_FUNCTION_PROFILER=y
+CONFIG_HIST_TRIGGERS=y
CONFIG_TRACE_ENUM_MAP_FILE=y
CONFIG_LKDTM=m
CONFIG_TEST_LIST_SORT=y
CONFIG_TEST_KSTRTOX=y
CONFIG_DMA_API_DEBUG=y
CONFIG_TEST_BPF=m
+CONFIG_BUG_ON_DATA_CORRUPTION=y
CONFIG_S390_PTDUMP=y
CONFIG_ENCRYPTED_KEYS=m
CONFIG_SECURITY=y
CONFIG_SECURITY_SELINUX_DISABLE=y
CONFIG_IMA=y
CONFIG_IMA_APPRAISE=y
+CONFIG_CRYPTO_RSA=m
+CONFIG_CRYPTO_DH=m
+CONFIG_CRYPTO_ECDH=m
CONFIG_CRYPTO_USER=m
-# CONFIG_CRYPTO_MANAGER_DISABLE_TESTS is not set
CONFIG_CRYPTO_CRYPTD=m
CONFIG_CRYPTO_TEST=m
CONFIG_CRYPTO_CCM=m
CONFIG_CRYPTO_GCM=m
-CONFIG_CRYPTO_CTS=m
+CONFIG_CRYPTO_CHACHA20POLY1305=m
CONFIG_CRYPTO_LRW=m
CONFIG_CRYPTO_PCBC=m
-CONFIG_CRYPTO_XTS=m
+CONFIG_CRYPTO_KEYWRAP=m
CONFIG_CRYPTO_XCBC=m
CONFIG_CRYPTO_VMAC=m
CONFIG_CRYPTO_CRC32=m
CONFIG_CRYPTO_SERPENT=m
CONFIG_CRYPTO_TEA=m
CONFIG_CRYPTO_TWOFISH=m
-CONFIG_CRYPTO_LZO=m
+CONFIG_CRYPTO_842=m
CONFIG_CRYPTO_LZ4=m
CONFIG_CRYPTO_LZ4HC=m
CONFIG_CRYPTO_USER_API_HASH=m
CONFIG_CRYPTO_USER_API_SKCIPHER=m
+CONFIG_CRYPTO_USER_API_RNG=m
+CONFIG_CRYPTO_USER_API_AEAD=m
CONFIG_ZCRYPT=m
CONFIG_CRYPTO_SHA1_S390=m
CONFIG_CRYPTO_SHA256_S390=m
CONFIG_IKCONFIG=y
CONFIG_IKCONFIG_PROC=y
CONFIG_NUMA_BALANCING=y
+# CONFIG_NUMA_BALANCING_DEFAULT_ENABLED is not set
CONFIG_MEMCG=y
CONFIG_MEMCG_SWAP=y
CONFIG_BLK_CGROUP=y
CONFIG_UNIXWARE_DISKLABEL=y
CONFIG_CFQ_GROUP_IOSCHED=y
CONFIG_DEFAULT_DEADLINE=y
+CONFIG_LIVEPATCH=y
CONFIG_TUNE_ZEC12=y
-CONFIG_NR_CPUS=256
+CONFIG_NR_CPUS=512
CONFIG_NUMA=y
CONFIG_HZ_100=y
CONFIG_MEMORY_HOTPLUG=y
CONFIG_CLEANCACHE=y
CONFIG_FRONTSWAP=y
CONFIG_CMA=y
+CONFIG_MEM_SOFT_DIRTY=y
CONFIG_ZSWAP=y
CONFIG_ZBUD=m
CONFIG_ZSMALLOC=m
CONFIG_NF_CONNTRACK_EVENTS=y
CONFIG_NF_CONNTRACK_TIMEOUT=y
CONFIG_NF_CONNTRACK_TIMESTAMP=y
-CONFIG_NF_CT_PROTO_DCCP=m
-CONFIG_NF_CT_PROTO_UDPLITE=m
CONFIG_NF_CONNTRACK_AMANDA=m
CONFIG_NF_CONNTRACK_FTP=m
CONFIG_NF_CONNTRACK_H323=m
CONFIG_NFT_EXTHDR=m
CONFIG_NFT_META=m
CONFIG_NFT_CT=m
-CONFIG_NFT_RBTREE=m
-CONFIG_NFT_HASH=m
CONFIG_NFT_COUNTER=m
CONFIG_NFT_LOG=m
CONFIG_NFT_LIMIT=m
CONFIG_NFT_NAT=m
CONFIG_NFT_COMPAT=m
+CONFIG_NFT_HASH=m
CONFIG_NETFILTER_XT_SET=m
CONFIG_NETFILTER_XT_TARGET_AUDIT=m
CONFIG_NETFILTER_XT_TARGET_CHECKSUM=m
CONFIG_NETFILTER_XT_MATCH_RATEEST=m
CONFIG_NETFILTER_XT_MATCH_REALM=m
CONFIG_NETFILTER_XT_MATCH_RECENT=m
-CONFIG_NETFILTER_XT_MATCH_SOCKET=m
CONFIG_NETFILTER_XT_MATCH_STATE=m
CONFIG_NETFILTER_XT_MATCH_STATISTIC=m
CONFIG_NETFILTER_XT_MATCH_STRING=m
CONFIG_IP_VS_FTP=m
CONFIG_IP_VS_PE_SIP=m
CONFIG_NF_CONNTRACK_IPV4=m
-# CONFIG_NF_CONNTRACK_PROC_COMPAT is not set
CONFIG_NF_TABLES_IPV4=m
CONFIG_NFT_CHAIN_ROUTE_IPV4=m
CONFIG_NF_TABLES_ARP=m
CONFIG_IFB=m
CONFIG_MACVLAN=m
CONFIG_MACVTAP=m
-CONFIG_IPVLAN=m
CONFIG_VXLAN=m
CONFIG_TUN=m
CONFIG_VETH=m
CONFIG_RAW_DRIVER=m
CONFIG_HANGCHECK_TIMER=m
CONFIG_TN3270_FS=y
+# CONFIG_HWMON is not set
CONFIG_WATCHDOG=y
CONFIG_WATCHDOG_NOWAYOUT=y
CONFIG_SOFT_WATCHDOG=m
CONFIG_EXT4_FS=y
CONFIG_EXT4_FS_POSIX_ACL=y
CONFIG_EXT4_FS_SECURITY=y
+CONFIG_EXT4_ENCRYPTION=y
CONFIG_JBD2_DEBUG=y
CONFIG_JFS_FS=m
CONFIG_JFS_POSIX_ACL=y
CONFIG_FUSE_FS=y
CONFIG_CUSE=m
CONFIG_OVERLAY_FS=m
+CONFIG_OVERLAY_FS_REDIRECT_DIR=y
CONFIG_FSCACHE=m
CONFIG_CACHEFILES=m
CONFIG_ISO9660_FS=y
CONFIG_UNUSED_SYMBOLS=y
CONFIG_MAGIC_SYSRQ=y
CONFIG_DEBUG_MEMORY_INIT=y
-CONFIG_MEMORY_NOTIFIER_ERROR_INJECT=m
CONFIG_PANIC_ON_OOPS=y
CONFIG_TIMER_STATS=y
CONFIG_RCU_TORTURE_TEST=m
CONFIG_RCU_CPU_STALL_TIMEOUT=60
-CONFIG_NOTIFIER_ERROR_INJECTION=m
-CONFIG_CPU_NOTIFIER_ERROR_INJECT=m
-CONFIG_PM_NOTIFIER_ERROR_INJECT=m
CONFIG_LATENCYTOP=y
+CONFIG_SCHED_TRACER=y
+CONFIG_FTRACE_SYSCALLS=y
+CONFIG_STACK_TRACER=y
CONFIG_BLK_DEV_IO_TRACE=y
-# CONFIG_KPROBE_EVENT is not set
+CONFIG_UPROBE_EVENT=y
+CONFIG_FUNCTION_PROFILER=y
+CONFIG_HIST_TRIGGERS=y
CONFIG_TRACE_ENUM_MAP_FILE=y
CONFIG_LKDTM=m
-CONFIG_RBTREE_TEST=m
-CONFIG_INTERVAL_TREE_TEST=m
CONFIG_PERCPU_TEST=m
CONFIG_ATOMIC64_SELFTEST=y
CONFIG_TEST_BPF=m
+CONFIG_BUG_ON_DATA_CORRUPTION=y
CONFIG_S390_PTDUMP=y
+CONFIG_PERSISTENT_KEYRINGS=y
+CONFIG_BIG_KEYS=y
CONFIG_ENCRYPTED_KEYS=m
CONFIG_SECURITY=y
CONFIG_SECURITY_NETWORK=y
CONFIG_SECURITY_SELINUX_BOOTPARAM=y
CONFIG_SECURITY_SELINUX_BOOTPARAM_VALUE=0
CONFIG_SECURITY_SELINUX_DISABLE=y
+CONFIG_INTEGRITY_SIGNATURE=y
+CONFIG_INTEGRITY_ASYMMETRIC_KEYS=y
CONFIG_IMA=y
+CONFIG_IMA_WRITE_POLICY=y
CONFIG_IMA_APPRAISE=y
+CONFIG_CRYPTO_DH=m
+CONFIG_CRYPTO_ECDH=m
CONFIG_CRYPTO_USER=m
# CONFIG_CRYPTO_MANAGER_DISABLE_TESTS is not set
+CONFIG_CRYPTO_PCRYPT=m
CONFIG_CRYPTO_CRYPTD=m
+CONFIG_CRYPTO_MCRYPTD=m
CONFIG_CRYPTO_TEST=m
CONFIG_CRYPTO_CCM=m
CONFIG_CRYPTO_GCM=m
-CONFIG_CRYPTO_CTS=m
+CONFIG_CRYPTO_CHACHA20POLY1305=m
CONFIG_CRYPTO_LRW=m
CONFIG_CRYPTO_PCBC=m
-CONFIG_CRYPTO_XTS=m
+CONFIG_CRYPTO_KEYWRAP=m
CONFIG_CRYPTO_XCBC=m
CONFIG_CRYPTO_VMAC=m
CONFIG_CRYPTO_CRC32=m
CONFIG_CRYPTO_RMD256=m
CONFIG_CRYPTO_RMD320=m
CONFIG_CRYPTO_SHA512=m
+CONFIG_CRYPTO_SHA3=m
CONFIG_CRYPTO_TGR192=m
CONFIG_CRYPTO_WP512=m
CONFIG_CRYPTO_ANUBIS=m
CONFIG_CRYPTO_SERPENT=m
CONFIG_CRYPTO_TEA=m
CONFIG_CRYPTO_TWOFISH=m
+CONFIG_CRYPTO_842=m
CONFIG_CRYPTO_LZ4=m
CONFIG_CRYPTO_LZ4HC=m
CONFIG_CRYPTO_USER_API_HASH=m
CONFIG_CRYPTO_USER_API_SKCIPHER=m
+CONFIG_CRYPTO_USER_API_RNG=m
+CONFIG_CRYPTO_USER_API_AEAD=m
CONFIG_ZCRYPT=m
CONFIG_CRYPTO_SHA1_S390=m
CONFIG_CRYPTO_SHA256_S390=m
CONFIG_CRYPTO_AES_S390=m
CONFIG_CRYPTO_GHASH_S390=m
CONFIG_CRYPTO_CRC32_S390=y
-CONFIG_ASYMMETRIC_KEY_TYPE=y
-CONFIG_ASYMMETRIC_PUBLIC_KEY_SUBTYPE=m
-CONFIG_X509_CERTIFICATE_PARSER=m
CONFIG_CRC7=m
CONFIG_CRC8=m
CONFIG_CORDIC=m
CONFIG_CLEANCACHE=y
CONFIG_FRONTSWAP=y
CONFIG_CMA=y
+CONFIG_MEM_SOFT_DIRTY=y
CONFIG_ZSWAP=y
CONFIG_ZBUD=m
CONFIG_ZSMALLOC=m
CONFIG_NF_CONNTRACK_EVENTS=y
CONFIG_NF_CONNTRACK_TIMEOUT=y
CONFIG_NF_CONNTRACK_TIMESTAMP=y
-CONFIG_NF_CT_PROTO_DCCP=m
-CONFIG_NF_CT_PROTO_UDPLITE=m
CONFIG_NF_CONNTRACK_AMANDA=m
CONFIG_NF_CONNTRACK_FTP=m
CONFIG_NF_CONNTRACK_H323=m
CONFIG_NFT_EXTHDR=m
CONFIG_NFT_META=m
CONFIG_NFT_CT=m
-CONFIG_NFT_RBTREE=m
-CONFIG_NFT_HASH=m
CONFIG_NFT_COUNTER=m
CONFIG_NFT_LOG=m
CONFIG_NFT_LIMIT=m
CONFIG_NFT_NAT=m
CONFIG_NFT_COMPAT=m
+CONFIG_NFT_HASH=m
CONFIG_NETFILTER_XT_SET=m
CONFIG_NETFILTER_XT_TARGET_AUDIT=m
CONFIG_NETFILTER_XT_TARGET_CHECKSUM=m
CONFIG_NETFILTER_XT_MATCH_RATEEST=m
CONFIG_NETFILTER_XT_MATCH_REALM=m
CONFIG_NETFILTER_XT_MATCH_RECENT=m
-CONFIG_NETFILTER_XT_MATCH_SOCKET=m
CONFIG_NETFILTER_XT_MATCH_STATE=m
CONFIG_NETFILTER_XT_MATCH_STATISTIC=m
CONFIG_NETFILTER_XT_MATCH_STRING=m
CONFIG_IP_VS_FTP=m
CONFIG_IP_VS_PE_SIP=m
CONFIG_NF_CONNTRACK_IPV4=m
-# CONFIG_NF_CONNTRACK_PROC_COMPAT is not set
CONFIG_NF_TABLES_IPV4=m
CONFIG_NFT_CHAIN_ROUTE_IPV4=m
CONFIG_NF_TABLES_ARP=m
CONFIG_IFB=m
CONFIG_MACVLAN=m
CONFIG_MACVTAP=m
-CONFIG_IPVLAN=m
CONFIG_VXLAN=m
CONFIG_TUN=m
CONFIG_VETH=m
CONFIG_EXT4_FS=y
CONFIG_EXT4_FS_POSIX_ACL=y
CONFIG_EXT4_FS_SECURITY=y
+CONFIG_EXT4_ENCRYPTION=y
CONFIG_JBD2_DEBUG=y
CONFIG_JFS_FS=m
CONFIG_JFS_POSIX_ACL=y
CONFIG_FUSE_FS=y
CONFIG_CUSE=m
CONFIG_OVERLAY_FS=m
+CONFIG_OVERLAY_FS_REDIRECT_DIR=y
CONFIG_FSCACHE=m
CONFIG_CACHEFILES=m
CONFIG_ISO9660_FS=y
CONFIG_BLK_DEV_IO_TRACE=y
CONFIG_UPROBE_EVENT=y
CONFIG_FUNCTION_PROFILER=y
+CONFIG_HIST_TRIGGERS=y
CONFIG_TRACE_ENUM_MAP_FILE=y
CONFIG_LKDTM=m
CONFIG_PERCPU_TEST=m
CONFIG_ATOMIC64_SELFTEST=y
CONFIG_TEST_BPF=m
+CONFIG_BUG_ON_DATA_CORRUPTION=y
CONFIG_S390_PTDUMP=y
+CONFIG_PERSISTENT_KEYRINGS=y
+CONFIG_BIG_KEYS=y
CONFIG_ENCRYPTED_KEYS=m
CONFIG_SECURITY=y
CONFIG_SECURITY_NETWORK=y
CONFIG_SECURITY_SELINUX_BOOTPARAM=y
CONFIG_SECURITY_SELINUX_BOOTPARAM_VALUE=0
CONFIG_SECURITY_SELINUX_DISABLE=y
+CONFIG_INTEGRITY_SIGNATURE=y
+CONFIG_INTEGRITY_ASYMMETRIC_KEYS=y
CONFIG_IMA=y
+CONFIG_IMA_WRITE_POLICY=y
CONFIG_IMA_APPRAISE=y
+CONFIG_CRYPTO_DH=m
+CONFIG_CRYPTO_ECDH=m
CONFIG_CRYPTO_USER=m
# CONFIG_CRYPTO_MANAGER_DISABLE_TESTS is not set
+CONFIG_CRYPTO_PCRYPT=m
CONFIG_CRYPTO_CRYPTD=m
+CONFIG_CRYPTO_MCRYPTD=m
CONFIG_CRYPTO_TEST=m
CONFIG_CRYPTO_CCM=m
CONFIG_CRYPTO_GCM=m
-CONFIG_CRYPTO_CTS=m
+CONFIG_CRYPTO_CHACHA20POLY1305=m
CONFIG_CRYPTO_LRW=m
CONFIG_CRYPTO_PCBC=m
-CONFIG_CRYPTO_XTS=m
+CONFIG_CRYPTO_KEYWRAP=m
CONFIG_CRYPTO_XCBC=m
CONFIG_CRYPTO_VMAC=m
CONFIG_CRYPTO_CRC32=m
CONFIG_CRYPTO_RMD256=m
CONFIG_CRYPTO_RMD320=m
CONFIG_CRYPTO_SHA512=m
+CONFIG_CRYPTO_SHA3=m
CONFIG_CRYPTO_TGR192=m
CONFIG_CRYPTO_WP512=m
CONFIG_CRYPTO_ANUBIS=m
CONFIG_CRYPTO_SERPENT=m
CONFIG_CRYPTO_TEA=m
CONFIG_CRYPTO_TWOFISH=m
+CONFIG_CRYPTO_842=m
CONFIG_CRYPTO_LZ4=m
CONFIG_CRYPTO_LZ4HC=m
CONFIG_CRYPTO_USER_API_HASH=m
CONFIG_CRYPTO_USER_API_SKCIPHER=m
+CONFIG_CRYPTO_USER_API_RNG=m
+CONFIG_CRYPTO_USER_API_AEAD=m
CONFIG_ZCRYPT=m
CONFIG_CRYPTO_SHA1_S390=m
CONFIG_CRYPTO_SHA256_S390=m
CONFIG_CRYPTO_AES_S390=m
CONFIG_CRYPTO_GHASH_S390=m
CONFIG_CRYPTO_CRC32_S390=y
-CONFIG_ASYMMETRIC_KEY_TYPE=y
-CONFIG_ASYMMETRIC_PUBLIC_KEY_SUBTYPE=m
-CONFIG_X509_CERTIFICATE_PARSER=m
CONFIG_CRC7=m
CONFIG_CRC8=m
CONFIG_CORDIC=m
CONFIG_STATIC_KEYS_SELFTEST=y
CONFIG_MODULES=y
CONFIG_MODULE_UNLOAD=y
-CONFIG_MODVERSIONS=y
CONFIG_BLK_DEV_INTEGRITY=y
CONFIG_PARTITION_ADVANCED=y
CONFIG_IBM_PARTITION=y
CONFIG_EQUALIZER=m
CONFIG_TUN=m
CONFIG_VIRTIO_NET=y
+# CONFIG_NET_VENDOR_ALACRITECH is not set
+# CONFIG_NET_VENDOR_SOLARFLARE is not set
# CONFIG_INPUT is not set
# CONFIG_SERIO is not set
+CONFIG_DEVKMEM=y
CONFIG_RAW_DRIVER=m
CONFIG_VIRTIO_BALLOON=y
CONFIG_EXT4_FS=y
CONFIG_KPROBES_SANITY_TEST=y
CONFIG_S390_PTDUMP=y
CONFIG_CRYPTO_CRYPTD=m
-CONFIG_CRYPTO_AUTHENC=m
CONFIG_CRYPTO_TEST=m
CONFIG_CRYPTO_CCM=m
CONFIG_CRYPTO_GCM=m
BUILD_BUG_ON(sizeof(addrtype) != (high - low + 1) * sizeof(long));\
asm volatile( \
" lctlg %1,%2,%0\n" \
- : : "Q" (*(addrtype *)(&array)), "i" (low), "i" (high));\
+ : \
+ : "Q" (*(addrtype *)(&array)), "i" (low), "i" (high) \
+ : "memory"); \
}
#define __ctl_store(array, low, high) { \
memcpy(&mach->fac_mask, kvm->arch.model.fac_mask,
S390_ARCH_FAC_LIST_SIZE_BYTE);
memcpy((unsigned long *)&mach->fac_list, S390_lowcore.stfle_fac_list,
- S390_ARCH_FAC_LIST_SIZE_BYTE);
+ sizeof(S390_lowcore.stfle_fac_list));
if (copy_to_user((void __user *)attr->addr, mach, sizeof(*mach)))
ret = -EFAULT;
kfree(mach);
/* Populate the facility mask initially. */
memcpy(kvm->arch.model.fac_mask, S390_lowcore.stfle_fac_list,
- S390_ARCH_FAC_LIST_SIZE_BYTE);
+ sizeof(S390_lowcore.stfle_fac_list));
for (i = 0; i < S390_ARCH_FAC_LIST_SIZE_U64; i++) {
if (i < kvm_s390_fac_list_mask_size())
kvm->arch.model.fac_mask[i] &= kvm_s390_fac_list_mask[i];
const void *kbuf, const void __user *ubuf)
{
int ret;
- struct pt_regs regs;
+ struct pt_regs regs = *task_pt_regs(target);
ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, ®s, 0,
sizeof(regs));
#include <linux/types.h>
#include "ctype.h"
+#include "string.h"
int memcmp(const void *s1, const void *s2, size_t len)
{
#define memset(d,c,l) __builtin_memset(d,c,l)
#define memcmp __builtin_memcmp
+extern int strcmp(const char *str1, const char *str2);
+extern int strncmp(const char *cs, const char *ct, size_t count);
+extern size_t strlen(const char *s);
+extern char *strstr(const char *s1, const char *s2);
+extern size_t strnlen(const char *s, size_t maxlen);
+extern unsigned int atou(const char *s);
+extern unsigned long long simple_strtoull(const char *cp, char **endp,
+ unsigned int base);
+
#endif /* BOOT_STRING_H */
const char *basename;
struct simd_skcipher_alg *simd;
} aesni_simd_skciphers2[] = {
-#if IS_ENABLED(CONFIG_CRYPTO_PCBC)
+#if (defined(MODULE) && IS_ENABLED(CONFIG_CRYPTO_PCBC)) || \
+ IS_BUILTIN(CONFIG_CRYPTO_PCBC)
{
.algname = "pcbc(aes)",
.drvname = "pcbc-aes-aesni",
jmp __switch_to
END(__switch_to_asm)
-/*
- * The unwinder expects the last frame on the stack to always be at the same
- * offset from the end of the page, which allows it to validate the stack.
- * Calling schedule_tail() directly would break that convention because its an
- * asmlinkage function so its argument has to be pushed on the stack. This
- * wrapper creates a proper "end of stack" frame header before the call.
- */
-ENTRY(schedule_tail_wrapper)
- FRAME_BEGIN
-
- pushl %eax
- call schedule_tail
- popl %eax
-
- FRAME_END
- ret
-ENDPROC(schedule_tail_wrapper)
/*
* A newly forked process directly context switches into this address.
*
* edi: kernel thread arg
*/
ENTRY(ret_from_fork)
- call schedule_tail_wrapper
+ FRAME_BEGIN /* help unwinder find end of stack */
+
+ /*
+ * schedule_tail() is asmlinkage so we have to put its 'prev' argument
+ * on the stack.
+ */
+ pushl %eax
+ call schedule_tail
+ popl %eax
testl %ebx, %ebx
jnz 1f /* kernel threads are uncommon */
2:
/* When we fork, we trace the syscall return in the child, too. */
- movl %esp, %eax
+ leal FRAME_OFFSET(%esp), %eax
call syscall_return_slowpath
+ FRAME_END
jmp restore_all
/* kernel thread */
#include <asm/smap.h>
#include <asm/pgtable_types.h>
#include <asm/export.h>
+#include <asm/frame.h>
#include <linux/err.h>
.code64
* r12: kernel thread arg
*/
ENTRY(ret_from_fork)
+ FRAME_BEGIN /* help unwinder find end of stack */
movq %rax, %rdi
- call schedule_tail /* rdi: 'prev' task parameter */
+ call schedule_tail /* rdi: 'prev' task parameter */
- testq %rbx, %rbx /* from kernel_thread? */
- jnz 1f /* kernel threads are uncommon */
+ testq %rbx, %rbx /* from kernel_thread? */
+ jnz 1f /* kernel threads are uncommon */
2:
- movq %rsp, %rdi
+ leaq FRAME_OFFSET(%rsp),%rdi /* pt_regs pointer */
call syscall_return_slowpath /* returns with IRQs disabled */
TRACE_IRQS_ON /* user mode is traced as IRQS on */
SWAPGS
+ FRAME_END
jmp restore_regs_and_iret
1:
* all online cpus.
*/
cpuhp_setup_state(CPUHP_AP_PERF_X86_AMD_IBS_STARTING,
- "perf/x86/amd/ibs:STARTING",
+ "perf/x86/amd/ibs:starting",
x86_pmu_amd_ibs_starting_cpu,
x86_pmu_amd_ibs_dying_cpu);
if (event->attr.precise_ip > precise)
return -EOPNOTSUPP;
+
+ /* There's no sense in having PEBS for non sampling events: */
+ if (!is_sampling_event(event))
+ return -EINVAL;
}
/*
* check that PEBS LBR correction does not conflict with
if (x86_pmu.flags & PMU_FL_EXCL_CNTRS) {
for_each_cpu(i, topology_sibling_cpumask(cpu)) {
+ struct cpu_hw_events *sibling;
struct intel_excl_cntrs *c;
- c = per_cpu(cpu_hw_events, i).excl_cntrs;
+ sibling = &per_cpu(cpu_hw_events, i);
+ c = sibling->excl_cntrs;
if (c && c->core_id == core_id) {
cpuc->kfree_on_online[1] = cpuc->excl_cntrs;
cpuc->excl_cntrs = c;
- cpuc->excl_thread_id = 1;
+ if (!sibling->excl_thread_id)
+ cpuc->excl_thread_id = 1;
break;
}
}
x86_pmu.num_counters, INTEL_PMC_MAX_GENERIC);
x86_pmu.num_counters = INTEL_PMC_MAX_GENERIC;
}
- x86_pmu.intel_ctrl = (1 << x86_pmu.num_counters) - 1;
+ x86_pmu.intel_ctrl = (1ULL << x86_pmu.num_counters) - 1;
if (x86_pmu.num_counters_fixed > INTEL_PMC_MAX_FIXED) {
WARN(1, KERN_ERR "hw perf events fixed %d > max(%d), clipping!",
.stop = cstate_pmu_event_stop,
.read = cstate_pmu_event_update,
.capabilities = PERF_PMU_CAP_NO_INTERRUPT,
+ .module = THIS_MODULE,
};
static struct pmu cstate_pkg_pmu = {
.stop = cstate_pmu_event_stop,
.read = cstate_pmu_event_update,
.capabilities = PERF_PMU_CAP_NO_INTERRUPT,
+ .module = THIS_MODULE,
};
static const struct cstate_model nhm_cstates __initconst = {
continue;
/* log dropped samples number */
- if (error[bit])
+ if (error[bit]) {
perf_log_lost_samples(event, error[bit]);
+ if (perf_event_account_interrupt(event))
+ x86_pmu_stop(event, 0);
+ }
+
if (counts[bit]) {
__intel_pmu_pebs_event(event, iregs, base,
top, bit, counts[bit]);
rapl_pmus->pmu.start = rapl_pmu_event_start;
rapl_pmus->pmu.stop = rapl_pmu_event_stop;
rapl_pmus->pmu.read = rapl_pmu_event_read;
+ rapl_pmus->pmu.module = THIS_MODULE;
return 0;
}
.start = uncore_pmu_event_start,
.stop = uncore_pmu_event_stop,
.read = uncore_pmu_event_read,
+ .module = THIS_MODULE,
};
} else {
pmu->pmu = *pmu->type->pmu;
void hswep_uncore_cpu_init(void)
{
- int pkg = topology_phys_to_logical_pkg(0);
+ int pkg = boot_cpu_data.logical_proc_id;
if (hswep_uncore_cbox.num_boxes > boot_cpu_data.x86_max_cores)
hswep_uncore_cbox.num_boxes = boot_cpu_data.x86_max_cores;
#define INTEL_FAM6_ATOM_SILVERMONT2 0x4D /* Avaton/Rangely */
#define INTEL_FAM6_ATOM_AIRMONT 0x4C /* CherryTrail / Braswell */
#define INTEL_FAM6_ATOM_MERRIFIELD 0x4A /* Tangier */
-#define INTEL_FAM6_ATOM_MOOREFIELD 0x5A /* Annidale */
+#define INTEL_FAM6_ATOM_MOOREFIELD 0x5A /* Anniedale */
#define INTEL_FAM6_ATOM_GOLDMONT 0x5C
#define INTEL_FAM6_ATOM_DENVERTON 0x5F /* Goldmont Microserver */
#define exttable_size(et) ((et)->count * EXT_SIGNATURE_SIZE + EXT_HEADER_SIZE)
+static inline u32 intel_get_microcode_revision(void)
+{
+ u32 rev, dummy;
+
+ native_wrmsrl(MSR_IA32_UCODE_REV, 0);
+
+ /* As documented in the SDM: Do a CPUID 1 here */
+ native_cpuid_eax(1);
+
+ /* get the current revision from MSR 0x8B */
+ native_rdmsr(MSR_IA32_UCODE_REV, dummy, rev);
+
+ return rev;
+}
+
#ifdef CONFIG_MICROCODE_INTEL
extern void __init load_ucode_intel_bsp(void);
extern void load_ucode_intel_ap(void);
: "memory");
}
+#define native_cpuid_reg(reg) \
+static inline unsigned int native_cpuid_##reg(unsigned int op) \
+{ \
+ unsigned int eax = op, ebx, ecx = 0, edx; \
+ \
+ native_cpuid(&eax, &ebx, &ecx, &edx); \
+ \
+ return reg; \
+}
+
+/*
+ * Native CPUID functions returning a single datum.
+ */
+native_cpuid_reg(eax)
+native_cpuid_reg(ebx)
+native_cpuid_reg(ecx)
+native_cpuid_reg(edx)
+
static inline void load_cr3(pgd_t *pgdir)
{
write_cr3(__pa(pgdir));
if (task == current)
return __builtin_frame_address(0);
- return (unsigned long *)((struct inactive_task_frame *)task->thread.sp)->bp;
+ return &((struct inactive_task_frame *)task->thread.sp)->bp;
}
#else
static inline unsigned long *
asmlinkage void ret_from_fork(void);
-/* data that is pointed to by thread.sp */
+/*
+ * This is the structure pointed to by thread.sp for an inactive task. The
+ * order of the fields must match the code in __switch_to_asm().
+ */
struct inactive_task_frame {
#ifdef CONFIG_X86_64
unsigned long r15;
unsigned long di;
#endif
unsigned long bx;
+
+ /*
+ * These two fields must be together. They form a stack frame header,
+ * needed by get_frame_pointer().
+ */
unsigned long bp;
unsigned long ret_addr;
};
.irq_ack = irq_chip_ack_parent,
.irq_eoi = ioapic_ack_level,
.irq_set_affinity = ioapic_set_affinity,
+ .irq_retrigger = irq_chip_retrigger_hierarchy,
.flags = IRQCHIP_SKIP_SET_WAKE,
};
.irq_ack = irq_chip_ack_parent,
.irq_eoi = ioapic_ir_ack_level,
.irq_set_affinity = ioapic_set_affinity,
+ .irq_retrigger = irq_chip_retrigger_hierarchy,
.flags = IRQCHIP_SKIP_SET_WAKE,
};
/* get information required for multi-node processors */
if (boot_cpu_has(X86_FEATURE_TOPOEXT)) {
- u32 eax, ebx, ecx, edx;
- cpuid(0x8000001e, &eax, &ebx, &ecx, &edx);
- node_id = ecx & 7;
-
- /* get compute unit information */
- smp_num_siblings = ((ebx >> 8) & 3) + 1;
- c->x86_max_cores /= smp_num_siblings;
- c->cpu_core_id = ebx & 0xff;
+ node_id = cpuid_ecx(0x8000001e) & 7;
/*
* We may have multiple LLCs if L3 caches exist, so check if we
{
int bit;
- if (get_option(&arg, &bit) && bit < NCAPINTS*32)
+ if (get_option(&arg, &bit) && bit >= 0 && bit < NCAPINTS * 32)
setup_clear_cpu_cap(bit);
else
return 0;
#include <asm/bugs.h>
#include <asm/cpu.h>
#include <asm/intel-family.h>
+#include <asm/microcode_intel.h>
#ifdef CONFIG_X86_64
#include <linux/topology.h>
(c->x86 == 0x6 && c->x86_model >= 0x0e))
set_cpu_cap(c, X86_FEATURE_CONSTANT_TSC);
- if (c->x86 >= 6 && !cpu_has(c, X86_FEATURE_IA64)) {
- unsigned lower_word;
-
- wrmsr(MSR_IA32_UCODE_REV, 0, 0);
- /* Required by the SDM */
- sync_core();
- rdmsr(MSR_IA32_UCODE_REV, lower_word, c->microcode);
- }
+ if (c->x86 >= 6 && !cpu_has(c, X86_FEATURE_IA64))
+ c->microcode = intel_get_microcode_revision();
/*
* Atom erratum AAE44/AAF40/AAG38/AAH41:
{
struct ucode_patch *p;
- p = kzalloc(size, GFP_KERNEL);
+ p = kzalloc(sizeof(struct ucode_patch), GFP_KERNEL);
if (!p)
return ERR_PTR(-ENOMEM);
return patch;
}
-static void cpuid_1(void)
-{
- /*
- * According to the Intel SDM, Volume 3, 9.11.7:
- *
- * CPUID returns a value in a model specific register in
- * addition to its usual register return values. The
- * semantics of CPUID cause it to deposit an update ID value
- * in the 64-bit model-specific register at address 08BH
- * (IA32_BIOS_SIGN_ID). If no update is present in the
- * processor, the value in the MSR remains unmodified.
- *
- * Use native_cpuid -- this code runs very early and we don't
- * want to mess with paravirt.
- */
- unsigned int eax = 1, ebx, ecx = 0, edx;
-
- native_cpuid(&eax, &ebx, &ecx, &edx);
-}
-
static int collect_cpu_info_early(struct ucode_cpu_info *uci)
{
unsigned int val[2];
native_rdmsr(MSR_IA32_PLATFORM_ID, val[0], val[1]);
csig.pf = 1 << ((val[1] >> 18) & 7);
}
- native_wrmsrl(MSR_IA32_UCODE_REV, 0);
-
- /* As documented in the SDM: Do a CPUID 1 here */
- cpuid_1();
- /* get the current revision from MSR 0x8B */
- native_rdmsr(MSR_IA32_UCODE_REV, val[0], val[1]);
-
- csig.rev = val[1];
+ csig.rev = intel_get_microcode_revision();
uci->cpu_sig = csig;
uci->valid = 1;
static int apply_microcode_early(struct ucode_cpu_info *uci, bool early)
{
struct microcode_intel *mc;
- unsigned int val[2];
+ u32 rev;
mc = uci->mc;
if (!mc)
/* write microcode via MSR 0x79 */
native_wrmsrl(MSR_IA32_UCODE_WRITE, (unsigned long)mc->bits);
- native_wrmsrl(MSR_IA32_UCODE_REV, 0);
- /* As documented in the SDM: Do a CPUID 1 here */
- cpuid_1();
-
- /* get the current revision from MSR 0x8B */
- native_rdmsr(MSR_IA32_UCODE_REV, val[0], val[1]);
- if (val[1] != mc->hdr.rev)
+ rev = intel_get_microcode_revision();
+ if (rev != mc->hdr.rev)
return -1;
#ifdef CONFIG_X86_64
/* Flush global tlb. This is precaution. */
flush_tlb_early();
#endif
- uci->cpu_sig.rev = val[1];
+ uci->cpu_sig.rev = rev;
if (early)
print_ucode(uci);
struct microcode_intel *mc;
struct ucode_cpu_info *uci;
struct cpuinfo_x86 *c;
- unsigned int val[2];
static int prev_rev;
+ u32 rev;
/* We should bind the task to the CPU */
if (WARN_ON(raw_smp_processor_id() != cpu))
/* write microcode via MSR 0x79 */
wrmsrl(MSR_IA32_UCODE_WRITE, (unsigned long)mc->bits);
- wrmsrl(MSR_IA32_UCODE_REV, 0);
-
- /* As documented in the SDM: Do a CPUID 1 here */
- cpuid_1();
- /* get the current revision from MSR 0x8B */
- rdmsr(MSR_IA32_UCODE_REV, val[0], val[1]);
+ rev = intel_get_microcode_revision();
- if (val[1] != mc->hdr.rev) {
+ if (rev != mc->hdr.rev) {
pr_err("CPU%d update to revision 0x%x failed\n",
cpu, mc->hdr.rev);
return -1;
}
- if (val[1] != prev_rev) {
+ if (rev != prev_rev) {
pr_info("updated to revision 0x%x, date = %04x-%02x-%02x\n",
- val[1],
+ rev,
mc->hdr.date & 0xffff,
mc->hdr.date >> 24,
(mc->hdr.date >> 16) & 0xff);
- prev_rev = val[1];
+ prev_rev = rev;
}
c = &cpu_data(cpu);
- uci->cpu_sig.rev = val[1];
- c->microcode = val[1];
+ uci->cpu_sig.rev = rev;
+ c->microcode = rev;
return 0;
}
u8 *ucode_ptr = data, *new_mc = NULL, *mc = NULL;
int new_rev = uci->cpu_sig.rev;
unsigned int leftover = size;
- unsigned int curr_mc_size = 0;
+ unsigned int curr_mc_size = 0, new_mc_size = 0;
unsigned int csig, cpf;
while (leftover) {
vfree(new_mc);
new_rev = mc_header.rev;
new_mc = mc;
+ new_mc_size = mc_size;
mc = NULL; /* trigger new vmalloc */
}
* permanent memory. So it will be loaded early when a CPU is hot added
* or resumes.
*/
- save_mc_for_early(new_mc, curr_mc_size);
+ save_mc_for_early(new_mc, new_mc_size);
pr_debug("CPU%d found a matching microcode update with version 0x%x (current=0x%x)\n",
cpu, new_rev, uci->cpu_sig.rev);
crystal_khz = 24000; /* 24.0 MHz */
break;
case INTEL_FAM6_SKYLAKE_X:
+ case INTEL_FAM6_ATOM_DENVERTON:
crystal_khz = 25000; /* 25.0 MHz */
break;
case INTEL_FAM6_ATOM_GOLDMONT:
#define FRAME_HEADER_SIZE (sizeof(long) * 2)
+/*
+ * This disables KASAN checking when reading a value from another task's stack,
+ * since the other task could be running on another CPU and could have poisoned
+ * the stack in the meantime.
+ */
+#define READ_ONCE_TASK_STACK(task, x) \
+({ \
+ unsigned long val; \
+ if (task == current) \
+ val = READ_ONCE(x); \
+ else \
+ val = READ_ONCE_NOCHECK(x); \
+ val; \
+})
+
static void unwind_dump(struct unwind_state *state, unsigned long *sp)
{
static bool dumped_before = false;
if (state->regs && user_mode(state->regs))
return 0;
- addr = ftrace_graph_ret_addr(state->task, &state->graph_idx, *addr_p,
+ addr = READ_ONCE_TASK_STACK(state->task, *addr_p);
+ addr = ftrace_graph_ret_addr(state->task, &state->graph_idx, addr,
addr_p);
return __kernel_text_address(addr) ? addr : 0;
if (state->regs)
next_bp = (unsigned long *)state->regs->bp;
else
- next_bp = (unsigned long *)*state->bp;
+ next_bp = (unsigned long *)READ_ONCE_TASK_STACK(state->task,*state->bp);
/* is the next frame pointer an encoded pointer to pt_regs? */
regs = decode_frame_pointer(next_bp);
return true;
bad_address:
+ /*
+ * When unwinding a non-current task, the task might actually be
+ * running on another CPU, in which case it could be modifying its
+ * stack while we're reading it. This is generally not a problem and
+ * can be ignored as long as the caller understands that unwinding
+ * another task will not always succeed.
+ */
+ if (state->task != current)
+ goto the_end;
+
if (state->regs) {
printk_deferred_once(KERN_WARNING
"WARNING: kernel stack regs at %p in %s:%d has bad 'bp' value %p\n",
return ctxt->ops->read_std(ctxt, linear, data, size, &ctxt->exception);
}
+static int segmented_write_std(struct x86_emulate_ctxt *ctxt,
+ struct segmented_address addr,
+ void *data,
+ unsigned int size)
+{
+ int rc;
+ ulong linear;
+
+ rc = linearize(ctxt, addr, size, true, &linear);
+ if (rc != X86EMUL_CONTINUE)
+ return rc;
+ return ctxt->ops->write_std(ctxt, linear, data, size, &ctxt->exception);
+}
+
/*
* Prefetch the remaining bytes of the instruction without crossing page
* boundary if they are not in fetch_cache yet.
&ctxt->exception);
}
-/* Does not support long mode */
static int __load_segment_descriptor(struct x86_emulate_ctxt *ctxt,
u16 selector, int seg, u8 cpl,
enum x86_transfer_type transfer,
rpl = selector & 3;
- /* NULL selector is not valid for TR, CS and SS (except for long mode) */
- if ((seg == VCPU_SREG_CS
- || (seg == VCPU_SREG_SS
- && (ctxt->mode != X86EMUL_MODE_PROT64 || rpl != cpl))
- || seg == VCPU_SREG_TR)
- && null_selector)
- goto exception;
-
/* TR should be in GDT only */
if (seg == VCPU_SREG_TR && (selector & (1 << 2)))
goto exception;
- if (null_selector) /* for NULL selector skip all following checks */
+ /* NULL selector is not valid for TR, CS and (except for long mode) SS */
+ if (null_selector) {
+ if (seg == VCPU_SREG_CS || seg == VCPU_SREG_TR)
+ goto exception;
+
+ if (seg == VCPU_SREG_SS) {
+ if (ctxt->mode != X86EMUL_MODE_PROT64 || rpl != cpl)
+ goto exception;
+
+ /*
+ * ctxt->ops->set_segment expects the CPL to be in
+ * SS.DPL, so fake an expand-up 32-bit data segment.
+ */
+ seg_desc.type = 3;
+ seg_desc.p = 1;
+ seg_desc.s = 1;
+ seg_desc.dpl = cpl;
+ seg_desc.d = 1;
+ seg_desc.g = 1;
+ }
+
+ /* Skip all following checks */
goto load;
+ }
ret = read_segment_descriptor(ctxt, selector, &seg_desc, &desc_addr);
if (ret != X86EMUL_CONTINUE)
u16 selector, int seg)
{
u8 cpl = ctxt->ops->cpl(ctxt);
+
+ /*
+ * None of MOV, POP and LSS can load a NULL selector in CPL=3, but
+ * they can load it at CPL<3 (Intel's manual says only LSS can,
+ * but it's wrong).
+ *
+ * However, the Intel manual says that putting IST=1/DPL=3 in
+ * an interrupt gate will result in SS=3 (the AMD manual instead
+ * says it doesn't), so allow SS=3 in __load_segment_descriptor
+ * and only forbid it here.
+ */
+ if (seg == VCPU_SREG_SS && selector == 3 &&
+ ctxt->mode == X86EMUL_MODE_PROT64)
+ return emulate_exception(ctxt, GP_VECTOR, 0, true);
+
return __load_segment_descriptor(ctxt, selector, seg, cpl,
X86_TRANSFER_NONE, NULL);
}
}
/* Disable writeback. */
ctxt->dst.type = OP_NONE;
- return segmented_write(ctxt, ctxt->dst.addr.mem,
- &desc_ptr, 2 + ctxt->op_bytes);
+ return segmented_write_std(ctxt, ctxt->dst.addr.mem,
+ &desc_ptr, 2 + ctxt->op_bytes);
}
static int em_sgdt(struct x86_emulate_ctxt *ctxt)
else
size = offsetof(struct fxregs_state, xmm_space[0]);
- return segmented_write(ctxt, ctxt->memop.addr.mem, &fx_state, size);
+ return segmented_write_std(ctxt, ctxt->memop.addr.mem, &fx_state, size);
}
static int fxrstor_fixup(struct x86_emulate_ctxt *ctxt,
if (rc != X86EMUL_CONTINUE)
return rc;
- rc = segmented_read(ctxt, ctxt->memop.addr.mem, &fx_state, 512);
+ rc = segmented_read_std(ctxt, ctxt->memop.addr.mem, &fx_state, 512);
if (rc != X86EMUL_CONTINUE)
return rc;
jump_label_rate_limit(&apic_hw_disabled, HZ);
jump_label_rate_limit(&apic_sw_disabled, HZ);
}
+
+void kvm_lapic_exit(void)
+{
+ static_key_deferred_flush(&apic_hw_disabled);
+ static_key_deferred_flush(&apic_sw_disabled);
+}
int kvm_lapic_enable_pv_eoi(struct kvm_vcpu *vcpu, u64 data);
void kvm_lapic_init(void);
+void kvm_lapic_exit(void);
#define VEC_POS(v) ((v) & (32 - 1))
#define REG_POS(v) (((v) >> 5) << 4)
switch (cap->cap) {
case KVM_CAP_HYPERV_SYNIC:
+ if (!irqchip_in_kernel(vcpu->kvm))
+ return -EINVAL;
return kvm_hv_activate_synic(vcpu);
default:
return -EINVAL;
void kvm_arch_exit(void)
{
+ kvm_lapic_exit();
perf_unregister_guest_info_callbacks(&kvm_guest_cbs);
if (!boot_cpu_has(X86_FEATURE_CONSTANT_TSC))
kvm_x86_ops->patch_hypercall(vcpu, instruction);
- return emulator_write_emulated(ctxt, rip, instruction, 3, NULL);
+ return emulator_write_emulated(ctxt, rip, instruction, 3,
+ &ctxt->exception);
}
static int dm_request_for_irq_injection(struct kvm_vcpu *vcpu)
* We were not able to extract an address from the instruction,
* probably because there was something invalid in it.
*/
- if (info->si_addr == (void *)-1) {
+ if (info->si_addr == (void __user *)-1) {
err = -EINVAL;
goto err_out;
}
set_memory_ro((unsigned long)header, header->pages);
prog->bpf_func = (void *)image;
prog->jited = 1;
+ } else {
+ prog = orig_prog;
}
out_addrs:
DMI_MATCH(DMI_BIOS_VERSION, "6JET85WW (1.43 )"),
},
},
+ /* https://bugzilla.kernel.org/show_bug.cgi?id=42606 */
+ {
+ .callback = set_nouse_crs,
+ .ident = "Supermicro X8DTH",
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR, "Supermicro"),
+ DMI_MATCH(DMI_PRODUCT_NAME, "X8DTH-i/6/iF/6F"),
+ DMI_MATCH(DMI_BIOS_VERSION, "2.0a"),
+ },
+ },
/* https://bugzilla.kernel.org/show_bug.cgi?id=15362 */
{
return 0;
}
+#define OVERFLOW_ADDR_SHIFT (64 - EFI_PAGE_SHIFT)
+#define OVERFLOW_ADDR_MASK (U64_MAX << OVERFLOW_ADDR_SHIFT)
+#define U64_HIGH_BIT (~(U64_MAX >> 1))
+
+static bool __init efi_memmap_entry_valid(const efi_memory_desc_t *md, int i)
+{
+ u64 end = (md->num_pages << EFI_PAGE_SHIFT) + md->phys_addr - 1;
+ u64 end_hi = 0;
+ char buf[64];
+
+ if (md->num_pages == 0) {
+ end = 0;
+ } else if (md->num_pages > EFI_PAGES_MAX ||
+ EFI_PAGES_MAX - md->num_pages <
+ (md->phys_addr >> EFI_PAGE_SHIFT)) {
+ end_hi = (md->num_pages & OVERFLOW_ADDR_MASK)
+ >> OVERFLOW_ADDR_SHIFT;
+
+ if ((md->phys_addr & U64_HIGH_BIT) && !(end & U64_HIGH_BIT))
+ end_hi += 1;
+ } else {
+ return true;
+ }
+
+ pr_warn_once(FW_BUG "Invalid EFI memory map entries:\n");
+
+ if (end_hi) {
+ pr_warn("mem%02u: %s range=[0x%016llx-0x%llx%016llx] (invalid)\n",
+ i, efi_md_typeattr_format(buf, sizeof(buf), md),
+ md->phys_addr, end_hi, end);
+ } else {
+ pr_warn("mem%02u: %s range=[0x%016llx-0x%016llx] (invalid)\n",
+ i, efi_md_typeattr_format(buf, sizeof(buf), md),
+ md->phys_addr, end);
+ }
+ return false;
+}
+
+static void __init efi_clean_memmap(void)
+{
+ efi_memory_desc_t *out = efi.memmap.map;
+ const efi_memory_desc_t *in = out;
+ const efi_memory_desc_t *end = efi.memmap.map_end;
+ int i, n_removal;
+
+ for (i = n_removal = 0; in < end; i++) {
+ if (efi_memmap_entry_valid(in, i)) {
+ if (out != in)
+ memcpy(out, in, efi.memmap.desc_size);
+ out = (void *)out + efi.memmap.desc_size;
+ } else {
+ n_removal++;
+ }
+ in = (void *)in + efi.memmap.desc_size;
+ }
+
+ if (n_removal > 0) {
+ u64 size = efi.memmap.nr_map - n_removal;
+
+ pr_warn("Removing %d invalid memory map entries.\n", n_removal);
+ efi_memmap_install(efi.memmap.phys_map, size);
+ }
+}
+
void __init efi_print_memmap(void)
{
efi_memory_desc_t *md;
}
}
+ efi_clean_memmap();
+
if (efi_enabled(EFI_DBG))
efi_print_memmap();
}
new_size = efi.memmap.desc_size * num_entries;
- new_phys = memblock_alloc(new_size, 0);
+ new_phys = efi_memmap_alloc(num_entries);
if (!new_phys) {
pr_err("Could not allocate boot services memmap\n");
return;
}
new_size = efi.memmap.desc_size * num_entries;
- new_phys = memblock_alloc(new_size, 0);
+ new_phys = efi_memmap_alloc(num_entries);
if (!new_phys) {
pr_err("Failed to allocate new EFI memmap\n");
return;
obj-$(subst m,y,$(CONFIG_GPIO_INTEL_PMIC)) += platform_pmic_gpio.o
obj-$(subst m,y,$(CONFIG_INTEL_MFLD_THERMAL)) += platform_msic_thermal.o
# SPI Devices
-obj-$(subst m,y,$(CONFIG_SPI_SPIDEV)) += platform_spidev.o
+obj-$(subst m,y,$(CONFIG_SPI_SPIDEV)) += platform_mrfld_spidev.o
# I2C Devices
obj-$(subst m,y,$(CONFIG_SENSORS_EMC1403)) += platform_emc1403.o
obj-$(subst m,y,$(CONFIG_SENSORS_LIS3LV02D)) += platform_lis331.o
--- /dev/null
+/*
+ * spidev platform data initilization file
+ *
+ * (C) Copyright 2014, 2016 Intel Corporation
+ * Authors: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
+ * Dan O'Donovan <dan@emutex.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; version 2
+ * of the License.
+ */
+
+#include <linux/err.h>
+#include <linux/init.h>
+#include <linux/sfi.h>
+#include <linux/spi/pxa2xx_spi.h>
+#include <linux/spi/spi.h>
+
+#include <asm/intel-mid.h>
+
+#define MRFLD_SPI_DEFAULT_DMA_BURST 8
+#define MRFLD_SPI_DEFAULT_TIMEOUT 500
+
+/* GPIO pin for spidev chipselect */
+#define MRFLD_SPIDEV_GPIO_CS 111
+
+static struct pxa2xx_spi_chip spidev_spi_chip = {
+ .dma_burst_size = MRFLD_SPI_DEFAULT_DMA_BURST,
+ .timeout = MRFLD_SPI_DEFAULT_TIMEOUT,
+ .gpio_cs = MRFLD_SPIDEV_GPIO_CS,
+};
+
+static void __init *spidev_platform_data(void *info)
+{
+ struct spi_board_info *spi_info = info;
+
+ if (intel_mid_identify_cpu() != INTEL_MID_CPU_CHIP_TANGIER)
+ return ERR_PTR(-ENODEV);
+
+ spi_info->mode = SPI_MODE_0;
+ spi_info->controller_data = &spidev_spi_chip;
+
+ return NULL;
+}
+
+static const struct devs_id spidev_dev_id __initconst = {
+ .name = "spidev",
+ .type = SFI_DEV_TYPE_SPI,
+ .delay = 0,
+ .get_platform_data = &spidev_platform_data,
+};
+
+sfi_device(spidev_dev_id);
+++ /dev/null
-/*
- * spidev platform data initilization file
- *
- * (C) Copyright 2014, 2016 Intel Corporation
- * Authors: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
- * Dan O'Donovan <dan@emutex.com>
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; version 2
- * of the License.
- */
-
-#include <linux/init.h>
-#include <linux/sfi.h>
-#include <linux/spi/pxa2xx_spi.h>
-#include <linux/spi/spi.h>
-
-#include <asm/intel-mid.h>
-
-#define MRFLD_SPI_DEFAULT_DMA_BURST 8
-#define MRFLD_SPI_DEFAULT_TIMEOUT 500
-
-/* GPIO pin for spidev chipselect */
-#define MRFLD_SPIDEV_GPIO_CS 111
-
-static struct pxa2xx_spi_chip spidev_spi_chip = {
- .dma_burst_size = MRFLD_SPI_DEFAULT_DMA_BURST,
- .timeout = MRFLD_SPI_DEFAULT_TIMEOUT,
- .gpio_cs = MRFLD_SPIDEV_GPIO_CS,
-};
-
-static void __init *spidev_platform_data(void *info)
-{
- struct spi_board_info *spi_info = info;
-
- spi_info->mode = SPI_MODE_0;
- spi_info->controller_data = &spidev_spi_chip;
-
- return NULL;
-}
-
-static const struct devs_id spidev_dev_id __initconst = {
- .name = "spidev",
- .type = SFI_DEV_TYPE_SPI,
- .delay = 0,
- .get_platform_data = &spidev_platform_data,
-};
-
-sfi_device(spidev_dev_id);
if ((sector | nr_sects) & bs_mask)
return -EINVAL;
- if (discard) {
- ret = __blkdev_issue_discard(bdev, sector, nr_sects, gfp_mask,
- BLKDEV_DISCARD_ZERO, biop);
- if (ret == 0 || (ret && ret != -EOPNOTSUPP))
- goto out;
- }
-
ret = __blkdev_issue_write_zeroes(bdev, sector, nr_sects, gfp_mask,
biop);
if (ret == 0 || (ret && ret != -EOPNOTSUPP))
struct bio *bio = NULL;
struct blk_plug plug;
+ if (discard) {
+ if (!blkdev_issue_discard(bdev, sector, nr_sects, gfp_mask,
+ BLKDEV_DISCARD_ZERO))
+ return 0;
+ }
+
blk_start_plug(&plug);
ret = __blkdev_issue_zeroout(bdev, sector, nr_sects, gfp_mask,
&bio, discard);
static void blk_mq_process_rq_list(struct blk_mq_hw_ctx *hctx)
{
LIST_HEAD(rq_list);
- LIST_HEAD(driver_list);
if (unlikely(blk_mq_hctx_stopped(hctx)))
return;
static inline sector_t blk_zone_start(struct request_queue *q,
sector_t sector)
{
- sector_t zone_mask = blk_queue_zone_size(q) - 1;
+ sector_t zone_mask = blk_queue_zone_sectors(q) - 1;
return sector & ~zone_mask;
}
return -EINVAL;
/* Check alignment (handle eventual smaller last zone) */
- zone_sectors = blk_queue_zone_size(q);
+ zone_sectors = blk_queue_zone_sectors(q);
if (sector & (zone_sectors - 1))
return -EINVAL;
struct block_device *bdev,
sector_t from, sector_t size)
{
- unsigned int zone_size = bdev_zone_size(bdev);
+ unsigned int zone_sectors = bdev_zone_sectors(bdev);
/*
* If this function is called, then the disk is a zoned block device
* regular block devices (no zone operation) and their zone size will
* be reported as 0. Allow this case.
*/
- if (!zone_size)
+ if (!zone_sectors)
return true;
/*
* use it. Check the zone size too: it should be a power of 2 number
* of sectors.
*/
- if (WARN_ON_ONCE(!is_power_of_2(zone_size))) {
+ if (WARN_ON_ONCE(!is_power_of_2(zone_sectors))) {
u32 rem;
- div_u64_rem(from, zone_size, &rem);
+ div_u64_rem(from, zone_sectors, &rem);
if (rem)
return false;
if ((from + size) < get_capacity(disk)) {
- div_u64_rem(size, zone_size, &rem);
+ div_u64_rem(size, zone_sectors, &rem);
if (rem)
return false;
}
} else {
- if (from & (zone_size - 1))
+ if (from & (zone_sectors - 1))
return false;
if ((from + size) < get_capacity(disk) &&
- (size & (zone_size - 1)))
+ (size & (zone_sectors - 1)))
return false;
}
tristate "Holtek Ht16K33 LED controller with keyscan"
depends on FB && OF && I2C && INPUT
select FB_SYS_FOPS
- select FB_CFB_FILLRECT
- select FB_CFB_COPYAREA
- select FB_CFB_IMAGEBLIT
+ select FB_SYS_FILLRECT
+ select FB_SYS_COPYAREA
+ select FB_SYS_IMAGEBLIT
select INPUT_MATRIXKMAP
select FB_BACKLIGHT
help
extern struct kset *devices_kset;
extern void devices_kset_move_last(struct device *dev);
-extern struct device_attribute dev_attr_deferred_probe;
-
#if defined(CONFIG_MODULES) && defined(CONFIG_SYSFS)
extern void module_add_driver(struct module *mod, struct device_driver *drv);
extern void module_remove_driver(struct device_driver *drv);
goto err_remove_dev_groups;
}
- error = device_create_file(dev, &dev_attr_deferred_probe);
- if (error)
- goto err_remove_online;
-
return 0;
- err_remove_online:
- device_remove_file(dev, &dev_attr_online);
err_remove_dev_groups:
device_remove_groups(dev, dev->groups);
err_remove_type_groups:
struct class *class = dev->class;
const struct device_type *type = dev->type;
- device_remove_file(dev, &dev_attr_deferred_probe);
device_remove_file(dev, &dev_attr_online);
device_remove_groups(dev, dev->groups);
static LIST_HEAD(deferred_probe_active_list);
static atomic_t deferred_trigger_count = ATOMIC_INIT(0);
-static ssize_t deferred_probe_show(struct device *dev,
- struct device_attribute *attr, char *buf)
-{
- bool value;
-
- mutex_lock(&deferred_probe_mutex);
- value = !list_empty(&dev->p->deferred_probe);
- mutex_unlock(&deferred_probe_mutex);
-
- return sprintf(buf, "%d\n", value);
-}
-DEVICE_ATTR_RO(deferred_probe);
-
/*
* In some cases, like suspend to RAM or hibernation, It might be reasonable
* to prohibit probing of devices as it could be unsafe.
sprintf(buf, "%s", zone->name);
/* MMOP_ONLINE_KERNEL */
- zone_shift = zone_can_shift(start_pfn, nr_pages, ZONE_NORMAL);
+ zone_can_shift(start_pfn, nr_pages, ZONE_NORMAL, &zone_shift);
if (zone_shift) {
strcat(buf, " ");
strcat(buf, (zone + zone_shift)->name);
}
/* MMOP_ONLINE_MOVABLE */
- zone_shift = zone_can_shift(start_pfn, nr_pages, ZONE_MOVABLE);
+ zone_can_shift(start_pfn, nr_pages, ZONE_MOVABLE, &zone_shift);
if (zone_shift) {
strcat(buf, " ");
strcat(buf, (zone + zone_shift)->name);
static int nbd_send_cmd(struct nbd_device *nbd, struct nbd_cmd *cmd, int index)
{
struct request *req = blk_mq_rq_from_pdu(cmd);
- int result, flags;
+ int result;
struct nbd_request request;
unsigned long size = blk_rq_bytes(req);
struct bio *bio;
if (type != NBD_CMD_WRITE)
return 0;
- flags = 0;
bio = req->bio;
while (bio) {
struct bio *next = bio->bi_next;
bio_for_each_segment(bvec, bio, iter) {
bool is_last = !next && bio_iter_last(bvec, iter);
+ int flags = is_last ? 0 : MSG_MORE;
- if (is_last)
- flags = MSG_MORE;
dev_dbg(nbd_to_dev(nbd), "request %p: sending %d bytes data\n",
cmd, bvec.bv_len);
result = sock_send_bvec(nbd, index, &bvec, flags);
return -ENOMEM;
for (i = 0; i < nbds_max; i++) {
+ struct request_queue *q;
struct gendisk *disk = alloc_disk(1 << part_shift);
if (!disk)
goto out;
* every gendisk to have its very own request_queue struct.
* These structs are big so we dynamically allocate them.
*/
- disk->queue = blk_mq_init_queue(&nbd_dev[i].tag_set);
- if (!disk->queue) {
+ q = blk_mq_init_queue(&nbd_dev[i].tag_set);
+ if (IS_ERR(q)) {
blk_mq_free_tag_set(&nbd_dev[i].tag_set);
put_disk(disk);
goto out;
}
+ disk->queue = q;
/*
* Tell the block layer that we are not a rotational device
struct virtio_blk_outhdr out_hdr;
struct virtio_scsi_inhdr in_hdr;
u8 status;
+ u8 sense[SCSI_SENSE_BUFFERSIZE];
struct scatterlist sg[];
};
}
if (type == cpu_to_virtio32(vq->vdev, VIRTIO_BLK_T_SCSI_CMD)) {
- sg_init_one(&sense, vbr->req->sense, SCSI_SENSE_BUFFERSIZE);
+ memcpy(vbr->sense, vbr->req->sense, SCSI_SENSE_BUFFERSIZE);
+ sg_init_one(&sense, vbr->sense, SCSI_SENSE_BUFFERSIZE);
sgs[num_out + num_in++] = &sense;
sg_init_one(&inhdr, &vbr->in_hdr, sizeof(vbr->in_hdr));
sgs[num_out + num_in++] = &inhdr;
if (err)
goto out_put_disk;
- q = vblk->disk->queue = blk_mq_init_queue(&vblk->tag_set);
+ q = blk_mq_init_queue(&vblk->tag_set);
if (IS_ERR(q)) {
err = -ENOMEM;
goto out_free_tags;
}
+ vblk->disk->queue = q;
q->queuedata = vblk;
#include <linux/genhd.h>
#include <linux/highmem.h>
#include <linux/slab.h>
+#include <linux/backing-dev.h>
#include <linux/string.h>
#include <linux/vmalloc.h>
#include <linux/err.h>
return bvec->bv_len != PAGE_SIZE;
}
+static void zram_revalidate_disk(struct zram *zram)
+{
+ revalidate_disk(zram->disk);
+ /* revalidate_disk reset the BDI_CAP_STABLE_WRITES so set again */
+ zram->disk->queue->backing_dev_info.capabilities |=
+ BDI_CAP_STABLE_WRITES;
+}
+
/*
* Check if request is within bounds and aligned on zram logical blocks.
*/
zram->comp = comp;
zram->disksize = disksize;
set_capacity(zram->disk, zram->disksize >> SECTOR_SHIFT);
+ zram_revalidate_disk(zram);
up_write(&zram->init_lock);
- /*
- * Revalidate disk out of the init_lock to avoid lockdep splat.
- * It's okay because disk's capacity is protected by init_lock
- * so that revalidate_disk always sees up-to-date capacity.
- */
- revalidate_disk(zram->disk);
-
return len;
out_destroy_comp:
/* Make sure all the pending I/O are finished */
fsync_bdev(bdev);
zram_reset_device(zram);
- revalidate_disk(zram->disk);
+ zram_revalidate_disk(zram);
bdput(bdev);
mutex_lock(&bdev->bd_mutex);
char *kbuf; /* k-addr because vread() takes vmlist_lock rwlock */
int err = 0;
- if (!pfn_valid(PFN_DOWN(p)))
- return -EIO;
-
read = 0;
if (p < (unsigned long) high_memory) {
low_count = count;
* by the kernel or data corruption may occur
*/
kbuf = xlate_dev_kmem_ptr((void *)p);
+ if (!virt_addr_valid(kbuf))
+ return -ENXIO;
if (copy_to_user(buf, kbuf, sz))
return -EFAULT;
* corruption may occur.
*/
ptr = xlate_dev_kmem_ptr((void *)p);
+ if (!virt_addr_valid(ptr))
+ return -ENXIO;
copied = copy_from_user(ptr, buf, sz);
if (copied) {
char *kbuf; /* k-addr because vwrite() takes vmlist_lock rwlock */
int err = 0;
- if (!pfn_valid(PFN_DOWN(p)))
- return -EIO;
-
if (p < (unsigned long) high_memory) {
unsigned long to_write = min_t(unsigned long, count,
(unsigned long)high_memory - p);
struct pardevice *pdev = NULL;
char *name;
struct pardev_cb ppdev_cb;
+ int rc = 0;
name = kasprintf(GFP_KERNEL, CHRDEV "%x", minor);
if (name == NULL)
port = parport_find_number(minor);
if (!port) {
pr_warn("%s: no associated port!\n", name);
- kfree(name);
- return -ENXIO;
+ rc = -ENXIO;
+ goto err;
}
memset(&ppdev_cb, 0, sizeof(ppdev_cb));
ppdev_cb.private = pp;
pdev = parport_register_dev_model(port, name, &ppdev_cb, minor);
parport_put_port(port);
- kfree(name);
if (!pdev) {
pr_warn("%s: failed to register device!\n", name);
- return -ENXIO;
+ rc = -ENXIO;
+ goto err;
}
pp->pdev = pdev;
dev_dbg(&pdev->dev, "registered pardevice\n");
- return 0;
+err:
+ kfree(name);
+ return rc;
}
static enum ieee1284_phase init_phase(int mode)
{
struct ports_device *portdev;
- portdev = container_of(work, struct ports_device, control_work);
+ portdev = container_of(work, struct ports_device, config_work);
if (!use_multiport(portdev)) {
struct virtio_device *vdev;
struct port *port;
GATE(CLK_ACLK550_CAM, "aclk550_cam", "mout_user_aclk550_cam",
GATE_BUS_TOP, 24, 0, 0),
GATE(CLK_ACLK432_SCALER, "aclk432_scaler", "mout_user_aclk432_scaler",
- GATE_BUS_TOP, 27, 0, 0),
+ GATE_BUS_TOP, 27, CLK_IS_CRITICAL, 0),
};
static const struct samsung_mux_clock exynos5420_mux_clks[] __initconst = {
GATE(CLK_SMMU_G2D, "smmu_g2d", "aclk333_g2d", GATE_IP_G2D, 7, 0, 0),
GATE(0, "aclk200_fsys", "mout_user_aclk200_fsys",
- GATE_BUS_FSYS0, 9, CLK_IGNORE_UNUSED, 0),
+ GATE_BUS_FSYS0, 9, CLK_IS_CRITICAL, 0),
GATE(0, "aclk200_fsys2", "mout_user_aclk200_fsys2",
GATE_BUS_FSYS0, 10, CLK_IGNORE_UNUSED, 0),
GATE(0, "aclk333_g2d", "mout_user_aclk333_g2d",
GATE_BUS_TOP, 0, CLK_IGNORE_UNUSED, 0),
GATE(0, "aclk266_g2d", "mout_user_aclk266_g2d",
- GATE_BUS_TOP, 1, CLK_IGNORE_UNUSED, 0),
+ GATE_BUS_TOP, 1, CLK_IS_CRITICAL, 0),
GATE(0, "aclk300_jpeg", "mout_user_aclk300_jpeg",
GATE_BUS_TOP, 4, CLK_IGNORE_UNUSED, 0),
GATE(0, "aclk333_432_isp0", "mout_user_aclk333_432_isp0",
GATE_BUS_TOP, 5, 0, 0),
GATE(0, "aclk300_gscl", "mout_user_aclk300_gscl",
- GATE_BUS_TOP, 6, CLK_IGNORE_UNUSED, 0),
+ GATE_BUS_TOP, 6, CLK_IS_CRITICAL, 0),
GATE(0, "aclk333_432_gscl", "mout_user_aclk333_432_gscl",
GATE_BUS_TOP, 7, CLK_IGNORE_UNUSED, 0),
GATE(0, "aclk333_432_isp", "mout_user_aclk333_432_isp",
GATE(0, "aclk166", "mout_user_aclk166",
GATE_BUS_TOP, 14, CLK_IGNORE_UNUSED, 0),
GATE(CLK_ACLK333, "aclk333", "mout_user_aclk333",
- GATE_BUS_TOP, 15, CLK_IGNORE_UNUSED, 0),
+ GATE_BUS_TOP, 15, CLK_IS_CRITICAL, 0),
GATE(0, "aclk400_isp", "mout_user_aclk400_isp",
GATE_BUS_TOP, 16, 0, 0),
GATE(0, "aclk400_mscl", "mout_user_aclk400_mscl",
GATE_BUS_TOP, 17, 0, 0),
GATE(0, "aclk200_disp1", "mout_user_aclk200_disp1",
- GATE_BUS_TOP, 18, 0, 0),
+ GATE_BUS_TOP, 18, CLK_IS_CRITICAL, 0),
GATE(CLK_SCLK_MPHY_IXTAL24, "sclk_mphy_ixtal24", "mphy_refclk_ixtal24",
GATE_BUS_TOP, 28, 0, 0),
GATE(CLK_SCLK_HSIC_12M, "sclk_hsic_12m", "ff_hsic_12m",
GATE_BUS_TOP, 29, 0, 0),
GATE(0, "aclk300_disp1", "mout_user_aclk300_disp1",
- SRC_MASK_TOP2, 24, 0, 0),
+ SRC_MASK_TOP2, 24, CLK_IS_CRITICAL, 0),
GATE(CLK_MAU_EPLL, "mau_epll", "mout_mau_epll_clk",
SRC_MASK_TOP7, 20, 0, 0),
if (mct_int_type == MCT_INT_SPI) {
if (evt->irq != -1)
disable_irq_nosync(evt->irq);
+ exynos4_mct_write(0x1, mevt->base + MCT_L_INT_CSTAT_OFFSET);
} else {
disable_percpu_irq(mct_irqs[MCT_L0_IRQ]);
}
select DW_DMAC_CORE
help
Support the Synopsys DesignWare AHB DMA controller on the
- platfroms that enumerate it as a PCI device. For example,
+ platforms that enumerate it as a PCI device. For example,
Intel Medfield has integrated this GPDMA controller.
#define PCI_DEVICE_ID_INTEL_IOAT_BDX8 0x6f2e
#define PCI_DEVICE_ID_INTEL_IOAT_BDX9 0x6f2f
+#define PCI_DEVICE_ID_INTEL_IOAT_SKX 0x2021
+
#define IOAT_VER_1_2 0x12 /* Version 1.2 */
#define IOAT_VER_2_0 0x20 /* Version 2.0 */
#define IOAT_VER_3_0 0x30 /* Version 3.0 */
{ PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_IOAT_BDX8) },
{ PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_IOAT_BDX9) },
+ { PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_IOAT_SKX) },
+
/* I/OAT v3.3 platforms */
{ PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_IOAT_BWD0) },
{ PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_IOAT_BWD1) },
}
}
+static inline bool is_skx_ioat(struct pci_dev *pdev)
+{
+ return (pdev->device == PCI_DEVICE_ID_INTEL_IOAT_SKX) ? true : false;
+}
+
static bool is_xeon_cb32(struct pci_dev *pdev)
{
return is_jf_ioat(pdev) || is_snb_ioat(pdev) || is_ivb_ioat(pdev) ||
- is_hsw_ioat(pdev) || is_bdx_ioat(pdev);
+ is_hsw_ioat(pdev) || is_bdx_ioat(pdev) || is_skx_ioat(pdev);
}
bool is_bwd_ioat(struct pci_dev *pdev)
/* doing 2 32bit writes to mmio since 1 64b write doesn't work */
ioat_chan->completion =
dma_pool_zalloc(ioat_chan->ioat_dma->completion_pool,
- GFP_KERNEL, &ioat_chan->completion_dma);
+ GFP_NOWAIT, &ioat_chan->completion_dma);
if (!ioat_chan->completion)
return -ENOMEM;
ioat_chan->reg_base + IOAT_CHANCMP_OFFSET_HIGH);
order = IOAT_MAX_ORDER;
- ring = ioat_alloc_ring(c, order, GFP_KERNEL);
+ ring = ioat_alloc_ring(c, order, GFP_NOWAIT);
if (!ring)
return -ENOMEM;
device->version = readb(device->reg_base + IOAT_VER_OFFSET);
if (device->version >= IOAT_VER_3_0) {
+ if (is_skx_ioat(pdev))
+ device->version = IOAT_VER_3_2;
err = ioat3_dma_probe(device, ioat_dca_enabled);
if (device->version >= IOAT_VER_3_3)
d->ccr |= CCR_DST_AMODE_POSTINC;
if (port_window) {
d->ccr |= CCR_SRC_AMODE_DBLIDX;
- d->ei = 1;
- /*
- * One frame covers the port_window and by configure
- * the source frame index to be -1 * (port_window - 1)
- * we instruct the sDMA that after a frame is processed
- * it should move back to the start of the window.
- */
- d->fi = -(port_window_bytes - 1);
if (port_window_bytes >= 64)
- d->csdp = CSDP_SRC_BURST_64 | CSDP_SRC_PACKED;
+ d->csdp |= CSDP_SRC_BURST_64;
else if (port_window_bytes >= 32)
- d->csdp = CSDP_SRC_BURST_32 | CSDP_SRC_PACKED;
+ d->csdp |= CSDP_SRC_BURST_32;
else if (port_window_bytes >= 16)
- d->csdp = CSDP_SRC_BURST_16 | CSDP_SRC_PACKED;
+ d->csdp |= CSDP_SRC_BURST_16;
+
} else {
d->ccr |= CCR_SRC_AMODE_CONSTANT;
}
d->ccr |= CCR_SRC_AMODE_POSTINC;
if (port_window) {
d->ccr |= CCR_DST_AMODE_DBLIDX;
+ d->ei = 1;
+ /*
+ * One frame covers the port_window and by configure
+ * the source frame index to be -1 * (port_window - 1)
+ * we instruct the sDMA that after a frame is processed
+ * it should move back to the start of the window.
+ */
+ d->fi = -(port_window_bytes - 1);
if (port_window_bytes >= 64)
- d->csdp = CSDP_DST_BURST_64 | CSDP_DST_PACKED;
+ d->csdp |= CSDP_DST_BURST_64;
else if (port_window_bytes >= 32)
- d->csdp = CSDP_DST_BURST_32 | CSDP_DST_PACKED;
+ d->csdp |= CSDP_DST_BURST_32;
else if (port_window_bytes >= 16)
- d->csdp = CSDP_DST_BURST_16 | CSDP_DST_PACKED;
+ d->csdp |= CSDP_DST_BURST_16;
} else {
d->ccr |= CCR_DST_AMODE_CONSTANT;
}
osg->addr = sg_dma_address(sgent);
osg->en = en;
osg->fn = sg_dma_len(sgent) / frame_bytes;
- if (port_window && dir == DMA_MEM_TO_DEV) {
+ if (port_window && dir == DMA_DEV_TO_MEM) {
osg->ei = 1;
/*
* One frame covers the port_window and by configure
struct omap_dmadev *od;
struct resource *res;
int rc, i, irq;
+ u32 lch_count;
od = devm_kzalloc(&pdev->dev, sizeof(*od), GFP_KERNEL);
if (!od)
spin_lock_init(&od->lock);
spin_lock_init(&od->irq_lock);
- if (!pdev->dev.of_node) {
- od->dma_requests = od->plat->dma_attr->lch_count;
- if (unlikely(!od->dma_requests))
- od->dma_requests = OMAP_SDMA_REQUESTS;
- } else if (of_property_read_u32(pdev->dev.of_node, "dma-requests",
- &od->dma_requests)) {
+ /* Number of DMA requests */
+ od->dma_requests = OMAP_SDMA_REQUESTS;
+ if (pdev->dev.of_node && of_property_read_u32(pdev->dev.of_node,
+ "dma-requests",
+ &od->dma_requests)) {
dev_info(&pdev->dev,
"Missing dma-requests property, using %u.\n",
OMAP_SDMA_REQUESTS);
- od->dma_requests = OMAP_SDMA_REQUESTS;
}
- od->lch_map = devm_kcalloc(&pdev->dev, od->dma_requests,
- sizeof(*od->lch_map), GFP_KERNEL);
+ /* Number of available logical channels */
+ if (!pdev->dev.of_node) {
+ lch_count = od->plat->dma_attr->lch_count;
+ if (unlikely(!lch_count))
+ lch_count = OMAP_SDMA_CHANNELS;
+ } else if (of_property_read_u32(pdev->dev.of_node, "dma-channels",
+ &lch_count)) {
+ dev_info(&pdev->dev,
+ "Missing dma-channels property, using %u.\n",
+ OMAP_SDMA_CHANNELS);
+ lch_count = OMAP_SDMA_CHANNELS;
+ }
+
+ od->lch_map = devm_kcalloc(&pdev->dev, lch_count, sizeof(*od->lch_map),
+ GFP_KERNEL);
if (!od->lch_map)
return -ENOMEM;
/* for cyclic capability */
bool cyclic;
+
+ /* for runtime pm tracking */
+ bool active;
};
struct pl330_dmac {
_stop(pch->thread);
spin_unlock(&pch->thread->dmac->lock);
power_down = true;
+ pch->active = false;
} else {
/* Make sure the PL330 Channel thread is active */
spin_lock(&pch->thread->dmac->lock);
desc->status = PREP;
list_move_tail(&desc->node, &pch->work_list);
if (power_down) {
+ pch->active = true;
spin_lock(&pch->thread->dmac->lock);
_start(pch->thread);
spin_unlock(&pch->thread->dmac->lock);
unsigned long flags;
struct pl330_dmac *pl330 = pch->dmac;
LIST_HEAD(list);
+ bool power_down = false;
pm_runtime_get_sync(pl330->ddma.dev);
spin_lock_irqsave(&pch->lock, flags);
pch->thread->req[0].desc = NULL;
pch->thread->req[1].desc = NULL;
pch->thread->req_running = -1;
+ power_down = pch->active;
+ pch->active = false;
/* Mark all desc done */
list_for_each_entry(desc, &pch->submitted_list, node) {
list_splice_tail_init(&pch->completed_list, &pl330->desc_pool);
spin_unlock_irqrestore(&pch->lock, flags);
pm_runtime_mark_last_busy(pl330->ddma.dev);
+ if (power_down)
+ pm_runtime_put_autosuspend(pl330->ddma.dev);
pm_runtime_put_autosuspend(pl330->ddma.dev);
return 0;
* updated on work_list emptiness status.
*/
WARN_ON(list_empty(&pch->submitted_list));
+ pch->active = true;
pm_runtime_get_sync(pch->dmac->ddma.dev);
}
list_splice_tail_init(&pch->submitted_list, &pch->work_list);
{
struct rcar_dmac_chan *rchan = to_rcar_dmac_chan(chan);
struct rcar_dmac *dmac = to_rcar_dmac(chan->device);
+ struct rcar_dmac_chan_map *map = &rchan->map;
struct rcar_dmac_desc_page *page, *_page;
struct rcar_dmac_desc *desc;
LIST_HEAD(list);
free_page((unsigned long)page);
}
+ /* Remove slave mapping if present. */
+ if (map->slave.xfer_size) {
+ dma_unmap_resource(chan->device->dev, map->addr,
+ map->slave.xfer_size, map->dir, 0);
+ map->slave.xfer_size = 0;
+ }
+
pm_runtime_put(chan->device->dev);
}
struct virt_dma_desc *vdesc;
enum dma_status status;
unsigned long flags;
- u32 residue;
+ u32 residue = 0;
status = dma_cookie_status(c, cookie, state);
if ((status == DMA_COMPLETE) || (!state))
spin_lock_irqsave(&chan->vchan.lock, flags);
vdesc = vchan_find_desc(&chan->vchan, cookie);
- if (cookie == chan->desc->vdesc.tx.cookie) {
+ if (chan->desc && cookie == chan->desc->vdesc.tx.cookie)
residue = stm32_dma_desc_residue(chan, chan->desc,
chan->next_sg);
- } else if (vdesc) {
+ else if (vdesc)
residue = stm32_dma_desc_residue(chan,
to_stm32_dma_desc(vdesc), 0);
- } else {
- residue = 0;
- }
-
dma_set_residue(state, residue);
spin_unlock_irqrestore(&chan->vchan.lock, flags);
struct stm32_dma_chan *chan;
struct dma_chan *c;
- if (dma_spec->args_count < 3)
+ if (dma_spec->args_count < 4)
return NULL;
cfg.channel_id = dma_spec->args[0];
cfg.request_line = dma_spec->args[1];
cfg.stream_config = dma_spec->args[2];
- cfg.threshold = 0;
+ cfg.threshold = dma_spec->args[3];
if ((cfg.channel_id >= STM32_DMA_MAX_CHANNELS) || (cfg.request_line >=
STM32_DMA_MAX_REQUEST_ID))
return NULL;
- if (dma_spec->args_count > 3)
- cfg.threshold = dma_spec->args[3];
-
chan = &dmadev->chan[cfg.channel_id];
c = dma_get_slave_channel(&chan->vchan.chan);
match = of_match_node(ti_am335x_master_match, dma_node);
if (!match) {
dev_err(&pdev->dev, "DMA master is not supported\n");
+ of_node_put(dma_node);
return -EINVAL;
}
match = of_match_node(ti_dra7_master_match, dma_node);
if (!match) {
dev_err(&pdev->dev, "DMA master is not supported\n");
+ of_node_put(dma_node);
return -EINVAL;
}
dev_err(&edev->dev, "out of memory in extcon_set_state\n");
kobject_uevent(&edev->dev.kobj, KOBJ_CHANGE);
- return 0;
+ return -ENOMEM;
}
length = name_show(&edev->dev, NULL, prop_buf);
}
/* allocate memory for new EFI memmap */
- new_memmap_phy = memblock_alloc(efi.memmap.desc_size * new_nr_map,
- PAGE_SIZE);
+ new_memmap_phy = efi_memmap_alloc(new_nr_map);
if (!new_memmap_phy)
return;
unsigned long get_dram_base(efi_system_table_t *sys_table_arg);
-efi_status_t update_fdt(efi_system_table_t *sys_table, void *orig_fdt,
- unsigned long orig_fdt_size,
- void *fdt, int new_fdt_size, char *cmdline_ptr,
- u64 initrd_addr, u64 initrd_size,
- efi_memory_desc_t *memory_map,
- unsigned long map_size, unsigned long desc_size,
- u32 desc_ver);
-
efi_status_t allocate_new_fdt_and_exit_boot(efi_system_table_t *sys_table,
void *handle,
unsigned long *new_fdt_addr,
#include "efistub.h"
-efi_status_t update_fdt(efi_system_table_t *sys_table, void *orig_fdt,
- unsigned long orig_fdt_size,
- void *fdt, int new_fdt_size, char *cmdline_ptr,
- u64 initrd_addr, u64 initrd_size,
- efi_memory_desc_t *memory_map,
- unsigned long map_size, unsigned long desc_size,
- u32 desc_ver)
+static efi_status_t update_fdt(efi_system_table_t *sys_table, void *orig_fdt,
+ unsigned long orig_fdt_size,
+ void *fdt, int new_fdt_size, char *cmdline_ptr,
+ u64 initrd_addr, u64 initrd_size)
{
int node, num_rsv;
int status;
if (status)
goto fdt_set_fail;
- fdt_val64 = cpu_to_fdt64((u64)(unsigned long)memory_map);
+ fdt_val64 = U64_MAX; /* placeholder */
status = fdt_setprop(fdt, node, "linux,uefi-mmap-start",
&fdt_val64, sizeof(fdt_val64));
if (status)
goto fdt_set_fail;
- fdt_val32 = cpu_to_fdt32(map_size);
+ fdt_val32 = U32_MAX; /* placeholder */
status = fdt_setprop(fdt, node, "linux,uefi-mmap-size",
&fdt_val32, sizeof(fdt_val32));
if (status)
goto fdt_set_fail;
- fdt_val32 = cpu_to_fdt32(desc_size);
status = fdt_setprop(fdt, node, "linux,uefi-mmap-desc-size",
&fdt_val32, sizeof(fdt_val32));
if (status)
goto fdt_set_fail;
- fdt_val32 = cpu_to_fdt32(desc_ver);
status = fdt_setprop(fdt, node, "linux,uefi-mmap-desc-ver",
&fdt_val32, sizeof(fdt_val32));
if (status)
return EFI_LOAD_ERROR;
}
+static efi_status_t update_fdt_memmap(void *fdt, struct efi_boot_memmap *map)
+{
+ int node = fdt_path_offset(fdt, "/chosen");
+ u64 fdt_val64;
+ u32 fdt_val32;
+ int err;
+
+ if (node < 0)
+ return EFI_LOAD_ERROR;
+
+ fdt_val64 = cpu_to_fdt64((unsigned long)*map->map);
+ err = fdt_setprop_inplace(fdt, node, "linux,uefi-mmap-start",
+ &fdt_val64, sizeof(fdt_val64));
+ if (err)
+ return EFI_LOAD_ERROR;
+
+ fdt_val32 = cpu_to_fdt32(*map->map_size);
+ err = fdt_setprop_inplace(fdt, node, "linux,uefi-mmap-size",
+ &fdt_val32, sizeof(fdt_val32));
+ if (err)
+ return EFI_LOAD_ERROR;
+
+ fdt_val32 = cpu_to_fdt32(*map->desc_size);
+ err = fdt_setprop_inplace(fdt, node, "linux,uefi-mmap-desc-size",
+ &fdt_val32, sizeof(fdt_val32));
+ if (err)
+ return EFI_LOAD_ERROR;
+
+ fdt_val32 = cpu_to_fdt32(*map->desc_ver);
+ err = fdt_setprop_inplace(fdt, node, "linux,uefi-mmap-desc-ver",
+ &fdt_val32, sizeof(fdt_val32));
+ if (err)
+ return EFI_LOAD_ERROR;
+
+ return EFI_SUCCESS;
+}
+
#ifndef EFI_FDT_ALIGN
#define EFI_FDT_ALIGN EFI_PAGE_SIZE
#endif
goto fail;
}
- /*
- * Now that we have done our final memory allocation (and free)
- * we can get the memory map key needed for
- * exit_boot_services().
- */
- status = efi_get_memory_map(sys_table, &map);
- if (status != EFI_SUCCESS)
- goto fail_free_new_fdt;
-
status = update_fdt(sys_table,
(void *)fdt_addr, fdt_size,
(void *)*new_fdt_addr, new_fdt_size,
- cmdline_ptr, initrd_addr, initrd_size,
- memory_map, map_size, desc_size, desc_ver);
+ cmdline_ptr, initrd_addr, initrd_size);
/* Succeeding the first time is the expected case. */
if (status == EFI_SUCCESS)
/*
* We need to allocate more space for the new
* device tree, so free existing buffer that is
- * too small. Also free memory map, as we will need
- * to get new one that reflects the free/alloc we do
- * on the device tree buffer.
+ * too small.
*/
efi_free(sys_table, new_fdt_size, *new_fdt_addr);
- sys_table->boottime->free_pool(memory_map);
new_fdt_size += EFI_PAGE_SIZE;
} else {
pr_efi_err(sys_table, "Unable to construct new device tree.\n");
- goto fail_free_mmap;
+ goto fail_free_new_fdt;
}
}
- sys_table->boottime->free_pool(memory_map);
priv.runtime_map = runtime_map;
priv.runtime_entry_count = &runtime_entry_count;
status = efi_exit_boot_services(sys_table, handle, &map, &priv,
if (status == EFI_SUCCESS) {
efi_set_virtual_address_map_t *svam;
+ status = update_fdt_memmap((void *)*new_fdt_addr, &map);
+ if (status != EFI_SUCCESS) {
+ /*
+ * The kernel won't get far without the memory map, but
+ * may still be able to print something meaningful so
+ * return success here.
+ */
+ return EFI_SUCCESS;
+ }
+
/* Install the new virtual address map */
svam = sys_table->runtime->set_virtual_address_map;
status = svam(runtime_entry_count * desc_size, desc_size,
pr_efi_err(sys_table, "Exit boot services failed.\n");
-fail_free_mmap:
- sys_table->boottime->free_pool(memory_map);
-
fail_free_new_fdt:
efi_free(sys_table, new_fdt_size, *new_fdt_addr);
#include <linux/efi.h>
#include <linux/io.h>
#include <asm/early_ioremap.h>
+#include <linux/memblock.h>
+#include <linux/slab.h>
+
+static phys_addr_t __init __efi_memmap_alloc_early(unsigned long size)
+{
+ return memblock_alloc(size, 0);
+}
+
+static phys_addr_t __init __efi_memmap_alloc_late(unsigned long size)
+{
+ unsigned int order = get_order(size);
+ struct page *p = alloc_pages(GFP_KERNEL, order);
+
+ if (!p)
+ return 0;
+
+ return PFN_PHYS(page_to_pfn(p));
+}
+
+/**
+ * efi_memmap_alloc - Allocate memory for the EFI memory map
+ * @num_entries: Number of entries in the allocated map.
+ *
+ * Depending on whether mm_init() has already been invoked or not,
+ * either memblock or "normal" page allocation is used.
+ *
+ * Returns the physical address of the allocated memory map on
+ * success, zero on failure.
+ */
+phys_addr_t __init efi_memmap_alloc(unsigned int num_entries)
+{
+ unsigned long size = num_entries * efi.memmap.desc_size;
+
+ if (slab_is_available())
+ return __efi_memmap_alloc_late(size);
+
+ return __efi_memmap_alloc_early(size);
+}
/**
* __efi_memmap_init - Common code for mapping the EFI memory map
return 0;
}
-static int __init mxs_gpio_init_gc(struct mxs_gpio_port *port, int irq_base)
+static int mxs_gpio_init_gc(struct mxs_gpio_port *port, int irq_base)
{
struct irq_chip_generic *gc;
struct irq_chip_type *ct;
/* FIXME: should the legacy sysfs handling be moved to gpio_device? */
gpiochip_sysfs_unregister(gdev);
+ gpiochip_free_hogs(chip);
/* Numb the device, cancelling all outstanding operations */
gdev->chip = NULL;
gpiochip_irqchip_remove(chip);
acpi_gpiochip_remove(chip);
gpiochip_remove_pin_ranges(chip);
- gpiochip_free_hogs(chip);
of_gpiochip_remove(chip);
/*
* We accept no more calls into the driver from this point, so
}
/**
- * _gpiochip_irqchip_add() - adds an irqchip to a gpiochip
+ * gpiochip_irqchip_add_key() - adds an irqchip to a gpiochip
* @gpiochip: the gpiochip to add the irqchip to
* @irqchip: the irqchip to add to the gpiochip
* @first_irq: if not dynamically assigned, the base (first) IRQ to
* the pins on the gpiochip can generate a unique IRQ. Everything else
* need to be open coded.
*/
-int _gpiochip_irqchip_add(struct gpio_chip *gpiochip,
- struct irq_chip *irqchip,
- unsigned int first_irq,
- irq_flow_handler_t handler,
- unsigned int type,
- bool nested,
- struct lock_class_key *lock_key)
+int gpiochip_irqchip_add_key(struct gpio_chip *gpiochip,
+ struct irq_chip *irqchip,
+ unsigned int first_irq,
+ irq_flow_handler_t handler,
+ unsigned int type,
+ bool nested,
+ struct lock_class_key *lock_key)
{
struct device_node *of_node;
bool irq_base_set = false;
return 0;
}
-EXPORT_SYMBOL_GPL(_gpiochip_irqchip_add);
+EXPORT_SYMBOL_GPL(gpiochip_irqchip_add_key);
#else /* CONFIG_GPIOLIB_IRQCHIP */
else if (type == CGS_UCODE_ID_SMU_SK)
strcpy(fw_name, "amdgpu/polaris10_smc_sk.bin");
break;
+ case CHIP_POLARIS12:
+ strcpy(fw_name, "amdgpu/polaris12_smc.bin");
+ break;
default:
DRM_ERROR("SMC firmware not supported\n");
return -EINVAL;
"STONEY",
"POLARIS10",
"POLARIS11",
+ "POLARIS12",
"LAST",
};
case CHIP_FIJI:
case CHIP_POLARIS11:
case CHIP_POLARIS10:
+ case CHIP_POLARIS12:
case CHIP_CARRIZO:
case CHIP_STONEY:
if (adev->asic_type == CHIP_CARRIZO || adev->asic_type == CHIP_STONEY)
{0x1002, 0x67CA, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_POLARIS10},
{0x1002, 0x67CC, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_POLARIS10},
{0x1002, 0x67CF, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_POLARIS10},
+ /* Polaris12 */
+ {0x1002, 0x6980, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_POLARIS12},
+ {0x1002, 0x6981, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_POLARIS12},
+ {0x1002, 0x6985, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_POLARIS12},
+ {0x1002, 0x6986, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_POLARIS12},
+ {0x1002, 0x6987, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_POLARIS12},
+ {0x1002, 0x699F, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_POLARIS12},
{0, 0, 0}
};
switch (adev->asic_type) {
case CHIP_POLARIS11:
case CHIP_POLARIS10:
+ case CHIP_POLARIS12:
case CHIP_TONGA:
case CHIP_FIJI:
case CHIP_TOPAZ:
#define FIRMWARE_STONEY "amdgpu/stoney_uvd.bin"
#define FIRMWARE_POLARIS10 "amdgpu/polaris10_uvd.bin"
#define FIRMWARE_POLARIS11 "amdgpu/polaris11_uvd.bin"
+#define FIRMWARE_POLARIS12 "amdgpu/polaris12_uvd.bin"
/**
* amdgpu_uvd_cs_ctx - Command submission parser context
MODULE_FIRMWARE(FIRMWARE_STONEY);
MODULE_FIRMWARE(FIRMWARE_POLARIS10);
MODULE_FIRMWARE(FIRMWARE_POLARIS11);
+MODULE_FIRMWARE(FIRMWARE_POLARIS12);
static void amdgpu_uvd_idle_work_handler(struct work_struct *work);
case CHIP_POLARIS11:
fw_name = FIRMWARE_POLARIS11;
break;
+ case CHIP_POLARIS12:
+ fw_name = FIRMWARE_POLARIS12;
+ break;
default:
return -EINVAL;
}
#define FIRMWARE_STONEY "amdgpu/stoney_vce.bin"
#define FIRMWARE_POLARIS10 "amdgpu/polaris10_vce.bin"
#define FIRMWARE_POLARIS11 "amdgpu/polaris11_vce.bin"
+#define FIRMWARE_POLARIS12 "amdgpu/polaris12_vce.bin"
#ifdef CONFIG_DRM_AMDGPU_CIK
MODULE_FIRMWARE(FIRMWARE_BONAIRE);
MODULE_FIRMWARE(FIRMWARE_STONEY);
MODULE_FIRMWARE(FIRMWARE_POLARIS10);
MODULE_FIRMWARE(FIRMWARE_POLARIS11);
+MODULE_FIRMWARE(FIRMWARE_POLARIS12);
static void amdgpu_vce_idle_work_handler(struct work_struct *work);
case CHIP_POLARIS11:
fw_name = FIRMWARE_POLARIS11;
break;
+ case CHIP_POLARIS12:
+ fw_name = FIRMWARE_POLARIS12;
+ break;
default:
return -EINVAL;
WREG32(mmCUR_POSITION + amdgpu_crtc->crtc_offset, (x << 16) | y);
WREG32(mmCUR_HOT_SPOT + amdgpu_crtc->crtc_offset, (xorigin << 16) | yorigin);
+ WREG32(mmCUR_SIZE + amdgpu_crtc->crtc_offset,
+ ((amdgpu_crtc->cursor_width - 1) << 16) | (amdgpu_crtc->cursor_height - 1));
return 0;
}
int32_t hot_y)
{
struct amdgpu_crtc *amdgpu_crtc = to_amdgpu_crtc(crtc);
- struct amdgpu_device *adev = crtc->dev->dev_private;
struct drm_gem_object *obj;
struct amdgpu_bo *aobj;
int ret;
dce_v10_0_lock_cursor(crtc, true);
- if (hot_x != amdgpu_crtc->cursor_hot_x ||
+ if (width != amdgpu_crtc->cursor_width ||
+ height != amdgpu_crtc->cursor_height ||
+ hot_x != amdgpu_crtc->cursor_hot_x ||
hot_y != amdgpu_crtc->cursor_hot_y) {
int x, y;
dce_v10_0_cursor_move_locked(crtc, x, y);
- amdgpu_crtc->cursor_hot_x = hot_x;
- amdgpu_crtc->cursor_hot_y = hot_y;
- }
-
- if (width != amdgpu_crtc->cursor_width ||
- height != amdgpu_crtc->cursor_height) {
- WREG32(mmCUR_SIZE + amdgpu_crtc->crtc_offset,
- (width - 1) << 16 | (height - 1));
amdgpu_crtc->cursor_width = width;
amdgpu_crtc->cursor_height = height;
+ amdgpu_crtc->cursor_hot_x = hot_x;
+ amdgpu_crtc->cursor_hot_y = hot_y;
}
dce_v10_0_show_cursor(crtc);
static void dce_v10_0_cursor_reset(struct drm_crtc *crtc)
{
struct amdgpu_crtc *amdgpu_crtc = to_amdgpu_crtc(crtc);
- struct amdgpu_device *adev = crtc->dev->dev_private;
if (amdgpu_crtc->cursor_bo) {
dce_v10_0_lock_cursor(crtc, true);
dce_v10_0_cursor_move_locked(crtc, amdgpu_crtc->cursor_x,
amdgpu_crtc->cursor_y);
- WREG32(mmCUR_SIZE + amdgpu_crtc->crtc_offset,
- (amdgpu_crtc->cursor_width - 1) << 16 |
- (amdgpu_crtc->cursor_height - 1));
-
dce_v10_0_show_cursor(crtc);
dce_v10_0_lock_cursor(crtc, false);
(const u32)ARRAY_SIZE(stoney_golden_settings_a11));
break;
case CHIP_POLARIS11:
+ case CHIP_POLARIS12:
amdgpu_program_register_sequence(adev,
polaris11_golden_settings_a11,
(const u32)ARRAY_SIZE(polaris11_golden_settings_a11));
num_crtc = 6;
break;
case CHIP_POLARIS11:
+ case CHIP_POLARIS12:
num_crtc = 5;
break;
default:
adev->mode_info.audio.num_pins = 8;
break;
case CHIP_POLARIS11:
+ case CHIP_POLARIS12:
adev->mode_info.audio.num_pins = 6;
break;
default:
int pll;
if ((adev->asic_type == CHIP_POLARIS10) ||
- (adev->asic_type == CHIP_POLARIS11)) {
+ (adev->asic_type == CHIP_POLARIS11) ||
+ (adev->asic_type == CHIP_POLARIS12)) {
struct amdgpu_encoder *amdgpu_encoder =
to_amdgpu_encoder(amdgpu_crtc->encoder);
struct amdgpu_encoder_atom_dig *dig = amdgpu_encoder->enc_priv;
WREG32(mmCUR_POSITION + amdgpu_crtc->crtc_offset, (x << 16) | y);
WREG32(mmCUR_HOT_SPOT + amdgpu_crtc->crtc_offset, (xorigin << 16) | yorigin);
+ WREG32(mmCUR_SIZE + amdgpu_crtc->crtc_offset,
+ ((amdgpu_crtc->cursor_width - 1) << 16) | (amdgpu_crtc->cursor_height - 1));
return 0;
}
int32_t hot_y)
{
struct amdgpu_crtc *amdgpu_crtc = to_amdgpu_crtc(crtc);
- struct amdgpu_device *adev = crtc->dev->dev_private;
struct drm_gem_object *obj;
struct amdgpu_bo *aobj;
int ret;
dce_v11_0_lock_cursor(crtc, true);
- if (hot_x != amdgpu_crtc->cursor_hot_x ||
+ if (width != amdgpu_crtc->cursor_width ||
+ height != amdgpu_crtc->cursor_height ||
+ hot_x != amdgpu_crtc->cursor_hot_x ||
hot_y != amdgpu_crtc->cursor_hot_y) {
int x, y;
dce_v11_0_cursor_move_locked(crtc, x, y);
- amdgpu_crtc->cursor_hot_x = hot_x;
- amdgpu_crtc->cursor_hot_y = hot_y;
- }
-
- if (width != amdgpu_crtc->cursor_width ||
- height != amdgpu_crtc->cursor_height) {
- WREG32(mmCUR_SIZE + amdgpu_crtc->crtc_offset,
- (width - 1) << 16 | (height - 1));
amdgpu_crtc->cursor_width = width;
amdgpu_crtc->cursor_height = height;
+ amdgpu_crtc->cursor_hot_x = hot_x;
+ amdgpu_crtc->cursor_hot_y = hot_y;
}
dce_v11_0_show_cursor(crtc);
static void dce_v11_0_cursor_reset(struct drm_crtc *crtc)
{
struct amdgpu_crtc *amdgpu_crtc = to_amdgpu_crtc(crtc);
- struct amdgpu_device *adev = crtc->dev->dev_private;
if (amdgpu_crtc->cursor_bo) {
dce_v11_0_lock_cursor(crtc, true);
dce_v11_0_cursor_move_locked(crtc, amdgpu_crtc->cursor_x,
amdgpu_crtc->cursor_y);
- WREG32(mmCUR_SIZE + amdgpu_crtc->crtc_offset,
- (amdgpu_crtc->cursor_width - 1) << 16 |
- (amdgpu_crtc->cursor_height - 1));
-
dce_v11_0_show_cursor(crtc);
dce_v11_0_lock_cursor(crtc, false);
return -EINVAL;
if ((adev->asic_type == CHIP_POLARIS10) ||
- (adev->asic_type == CHIP_POLARIS11)) {
+ (adev->asic_type == CHIP_POLARIS11) ||
+ (adev->asic_type == CHIP_POLARIS12)) {
struct amdgpu_encoder *amdgpu_encoder =
to_amdgpu_encoder(amdgpu_crtc->encoder);
int encoder_mode =
adev->mode_info.num_dig = 6;
break;
case CHIP_POLARIS11:
+ case CHIP_POLARIS12:
adev->mode_info.num_hpd = 5;
adev->mode_info.num_dig = 5;
break;
amdgpu_atombios_crtc_powergate_init(adev);
amdgpu_atombios_encoder_init_dig(adev);
if ((adev->asic_type == CHIP_POLARIS10) ||
- (adev->asic_type == CHIP_POLARIS11)) {
+ (adev->asic_type == CHIP_POLARIS11) ||
+ (adev->asic_type == CHIP_POLARIS12)) {
amdgpu_atombios_crtc_set_dce_clock(adev, adev->clock.default_dispclk,
DCE_CLOCK_TYPE_DISPCLK, ATOM_GCK_DFS);
amdgpu_atombios_crtc_set_dce_clock(adev, 0,
struct amdgpu_device *adev = crtc->dev->dev_private;
int xorigin = 0, yorigin = 0;
+ int w = amdgpu_crtc->cursor_width;
+
amdgpu_crtc->cursor_x = x;
amdgpu_crtc->cursor_y = y;
WREG32(mmCUR_POSITION + amdgpu_crtc->crtc_offset, (x << 16) | y);
WREG32(mmCUR_HOT_SPOT + amdgpu_crtc->crtc_offset, (xorigin << 16) | yorigin);
+ WREG32(mmCUR_SIZE + amdgpu_crtc->crtc_offset,
+ ((w - 1) << 16) | (amdgpu_crtc->cursor_height - 1));
return 0;
}
int32_t hot_y)
{
struct amdgpu_crtc *amdgpu_crtc = to_amdgpu_crtc(crtc);
- struct amdgpu_device *adev = crtc->dev->dev_private;
struct drm_gem_object *obj;
struct amdgpu_bo *aobj;
int ret;
dce_v6_0_lock_cursor(crtc, true);
- if (hot_x != amdgpu_crtc->cursor_hot_x ||
+ if (width != amdgpu_crtc->cursor_width ||
+ height != amdgpu_crtc->cursor_height ||
+ hot_x != amdgpu_crtc->cursor_hot_x ||
hot_y != amdgpu_crtc->cursor_hot_y) {
int x, y;
dce_v6_0_cursor_move_locked(crtc, x, y);
- amdgpu_crtc->cursor_hot_x = hot_x;
- amdgpu_crtc->cursor_hot_y = hot_y;
- }
-
- if (width != amdgpu_crtc->cursor_width ||
- height != amdgpu_crtc->cursor_height) {
- WREG32(mmCUR_SIZE + amdgpu_crtc->crtc_offset,
- (width - 1) << 16 | (height - 1));
amdgpu_crtc->cursor_width = width;
amdgpu_crtc->cursor_height = height;
+ amdgpu_crtc->cursor_hot_x = hot_x;
+ amdgpu_crtc->cursor_hot_y = hot_y;
}
dce_v6_0_show_cursor(crtc);
static void dce_v6_0_cursor_reset(struct drm_crtc *crtc)
{
struct amdgpu_crtc *amdgpu_crtc = to_amdgpu_crtc(crtc);
- struct amdgpu_device *adev = crtc->dev->dev_private;
if (amdgpu_crtc->cursor_bo) {
dce_v6_0_lock_cursor(crtc, true);
dce_v6_0_cursor_move_locked(crtc, amdgpu_crtc->cursor_x,
amdgpu_crtc->cursor_y);
- WREG32(mmCUR_SIZE + amdgpu_crtc->crtc_offset,
- (amdgpu_crtc->cursor_width - 1) << 16 |
- (amdgpu_crtc->cursor_height - 1));
-
dce_v6_0_show_cursor(crtc);
dce_v6_0_lock_cursor(crtc, false);
}
WREG32(mmCUR_POSITION + amdgpu_crtc->crtc_offset, (x << 16) | y);
WREG32(mmCUR_HOT_SPOT + amdgpu_crtc->crtc_offset, (xorigin << 16) | yorigin);
+ WREG32(mmCUR_SIZE + amdgpu_crtc->crtc_offset,
+ ((amdgpu_crtc->cursor_width - 1) << 16) | (amdgpu_crtc->cursor_height - 1));
return 0;
}
int32_t hot_y)
{
struct amdgpu_crtc *amdgpu_crtc = to_amdgpu_crtc(crtc);
- struct amdgpu_device *adev = crtc->dev->dev_private;
struct drm_gem_object *obj;
struct amdgpu_bo *aobj;
int ret;
dce_v8_0_lock_cursor(crtc, true);
- if (hot_x != amdgpu_crtc->cursor_hot_x ||
+ if (width != amdgpu_crtc->cursor_width ||
+ height != amdgpu_crtc->cursor_height ||
+ hot_x != amdgpu_crtc->cursor_hot_x ||
hot_y != amdgpu_crtc->cursor_hot_y) {
int x, y;
dce_v8_0_cursor_move_locked(crtc, x, y);
- amdgpu_crtc->cursor_hot_x = hot_x;
- amdgpu_crtc->cursor_hot_y = hot_y;
- }
-
- if (width != amdgpu_crtc->cursor_width ||
- height != amdgpu_crtc->cursor_height) {
- WREG32(mmCUR_SIZE + amdgpu_crtc->crtc_offset,
- (width - 1) << 16 | (height - 1));
amdgpu_crtc->cursor_width = width;
amdgpu_crtc->cursor_height = height;
+ amdgpu_crtc->cursor_hot_x = hot_x;
+ amdgpu_crtc->cursor_hot_y = hot_y;
}
dce_v8_0_show_cursor(crtc);
static void dce_v8_0_cursor_reset(struct drm_crtc *crtc)
{
struct amdgpu_crtc *amdgpu_crtc = to_amdgpu_crtc(crtc);
- struct amdgpu_device *adev = crtc->dev->dev_private;
if (amdgpu_crtc->cursor_bo) {
dce_v8_0_lock_cursor(crtc, true);
dce_v8_0_cursor_move_locked(crtc, amdgpu_crtc->cursor_x,
amdgpu_crtc->cursor_y);
- WREG32(mmCUR_SIZE + amdgpu_crtc->crtc_offset,
- (amdgpu_crtc->cursor_width - 1) << 16 |
- (amdgpu_crtc->cursor_height - 1));
-
dce_v8_0_show_cursor(crtc);
dce_v8_0_lock_cursor(crtc, false);
MODULE_FIRMWARE("amdgpu/polaris10_mec2.bin");
MODULE_FIRMWARE("amdgpu/polaris10_rlc.bin");
+MODULE_FIRMWARE("amdgpu/polaris12_ce.bin");
+MODULE_FIRMWARE("amdgpu/polaris12_pfp.bin");
+MODULE_FIRMWARE("amdgpu/polaris12_me.bin");
+MODULE_FIRMWARE("amdgpu/polaris12_mec.bin");
+MODULE_FIRMWARE("amdgpu/polaris12_mec2.bin");
+MODULE_FIRMWARE("amdgpu/polaris12_rlc.bin");
+
static const struct amdgpu_gds_reg_offset amdgpu_gds_reg_offset[] =
{
{mmGDS_VMID0_BASE, mmGDS_VMID0_SIZE, mmGDS_GWS_VMID0, mmGDS_OA_VMID0},
(const u32)ARRAY_SIZE(tonga_golden_common_all));
break;
case CHIP_POLARIS11:
+ case CHIP_POLARIS12:
amdgpu_program_register_sequence(adev,
golden_settings_polaris11_a11,
(const u32)ARRAY_SIZE(golden_settings_polaris11_a11));
case CHIP_POLARIS10:
chip_name = "polaris10";
break;
+ case CHIP_POLARIS12:
+ chip_name = "polaris12";
+ break;
case CHIP_STONEY:
chip_name = "stoney";
break;
gb_addr_config = TONGA_GB_ADDR_CONFIG_GOLDEN;
break;
case CHIP_POLARIS11:
+ case CHIP_POLARIS12:
ret = amdgpu_atombios_get_gfx_info(adev);
if (ret)
return ret;
break;
case CHIP_POLARIS11:
+ case CHIP_POLARIS12:
modearray[0] = (ARRAY_MODE(ARRAY_2D_TILED_THIN1) |
PIPE_CONFIG(ADDR_SURF_P4_16x16) |
TILE_SPLIT(ADDR_SURF_TILE_SPLIT_64B) |
*rconf1 |= 0x0;
break;
case CHIP_POLARIS11:
+ case CHIP_POLARIS12:
*rconf |= RB_MAP_PKR0(2) | RB_XSEL2(1) | SE_MAP(2) |
SE_XSEL(1) | SE_YSEL(1);
*rconf1 |= 0x0;
cz_enable_cp_power_gating(adev, true);
else
cz_enable_cp_power_gating(adev, false);
- } else if (adev->asic_type == CHIP_POLARIS11) {
+ } else if ((adev->asic_type == CHIP_POLARIS11) ||
+ (adev->asic_type == CHIP_POLARIS12)) {
gfx_v8_0_init_csb(adev);
gfx_v8_0_init_save_restore_list(adev);
gfx_v8_0_enable_save_restore_machine(adev);
RLC_CGCG_CGLS_CTRL__CGLS_EN_MASK);
WREG32(mmRLC_CGCG_CGLS_CTRL, tmp);
if (adev->asic_type == CHIP_POLARIS11 ||
- adev->asic_type == CHIP_POLARIS10) {
+ adev->asic_type == CHIP_POLARIS10 ||
+ adev->asic_type == CHIP_POLARIS12) {
tmp = RREG32(mmRLC_CGCG_CGLS_CTRL_3D);
tmp &= ~0x3;
WREG32(mmRLC_CGCG_CGLS_CTRL_3D, tmp);
amdgpu_ring_write(ring, 0x0000002A);
break;
case CHIP_POLARIS11:
+ case CHIP_POLARIS12:
amdgpu_ring_write(ring, 0x16000012);
amdgpu_ring_write(ring, 0x00000000);
break;
(adev->asic_type == CHIP_FIJI) ||
(adev->asic_type == CHIP_STONEY) ||
(adev->asic_type == CHIP_POLARIS11) ||
- (adev->asic_type == CHIP_POLARIS10)) {
+ (adev->asic_type == CHIP_POLARIS10) ||
+ (adev->asic_type == CHIP_POLARIS12)) {
WREG32(mmCP_MEC_DOORBELL_RANGE_LOWER,
AMDGPU_DOORBELL_KIQ << 2);
WREG32(mmCP_MEC_DOORBELL_RANGE_UPPER,
mqd->cp_hqd_persistent_state = tmp;
if (adev->asic_type == CHIP_STONEY ||
adev->asic_type == CHIP_POLARIS11 ||
- adev->asic_type == CHIP_POLARIS10) {
+ adev->asic_type == CHIP_POLARIS10 ||
+ adev->asic_type == CHIP_POLARIS12) {
tmp = RREG32(mmCP_ME1_PIPE3_INT_CNTL);
tmp = REG_SET_FIELD(tmp, CP_ME1_PIPE3_INT_CNTL, GENERIC2_INT_ENABLE, 1);
WREG32(mmCP_ME1_PIPE3_INT_CNTL, tmp);
static void gfx_v8_0_enable_gfx_static_mg_power_gating(struct amdgpu_device *adev,
bool enable)
{
- if (adev->asic_type == CHIP_POLARIS11)
+ if ((adev->asic_type == CHIP_POLARIS11) ||
+ (adev->asic_type == CHIP_POLARIS12))
/* Send msg to SMU via Powerplay */
amdgpu_set_powergating_state(adev,
AMD_IP_BLOCK_TYPE_SMC,
gfx_v8_0_enable_gfx_dynamic_mg_power_gating(adev, false);
break;
case CHIP_POLARIS11:
+ case CHIP_POLARIS12:
if ((adev->pg_flags & AMD_PG_SUPPORT_GFX_SMG) && enable)
gfx_v8_0_enable_gfx_static_mg_power_gating(adev, true);
else
MODULE_FIRMWARE("radeon/pitcairn_mc.bin");
MODULE_FIRMWARE("radeon/verde_mc.bin");
MODULE_FIRMWARE("radeon/oland_mc.bin");
+MODULE_FIRMWARE("radeon/si58_mc.bin");
#define MC_SEQ_MISC0__MT__MASK 0xf0000000
#define MC_SEQ_MISC0__MT__GDDR1 0x10000000
const char *chip_name;
char fw_name[30];
int err;
+ bool is_58_fw = false;
DRM_DEBUG("\n");
default: BUG();
}
- snprintf(fw_name, sizeof(fw_name), "radeon/%s_mc.bin", chip_name);
+ /* this memory configuration requires special firmware */
+ if (((RREG32(mmMC_SEQ_MISC0) & 0xff000000) >> 24) == 0x58)
+ is_58_fw = true;
+
+ if (is_58_fw)
+ snprintf(fw_name, sizeof(fw_name), "radeon/si58_mc.bin");
+ else
+ snprintf(fw_name, sizeof(fw_name), "radeon/%s_mc.bin", chip_name);
err = request_firmware(&adev->mc.fw, fw_name, adev->dev);
if (err)
goto out;
WREG32(mmVM_CONTEXT1_CNTL,
VM_CONTEXT1_CNTL__ENABLE_CONTEXT_MASK |
(1UL << VM_CONTEXT1_CNTL__PAGE_TABLE_DEPTH__SHIFT) |
- ((amdgpu_vm_block_size - 9) << VM_CONTEXT1_CNTL__PAGE_TABLE_BLOCK_SIZE__SHIFT) |
- VM_CONTEXT1_CNTL__RANGE_PROTECTION_FAULT_ENABLE_INTERRUPT_MASK |
- VM_CONTEXT1_CNTL__RANGE_PROTECTION_FAULT_ENABLE_DEFAULT_MASK |
- VM_CONTEXT1_CNTL__DUMMY_PAGE_PROTECTION_FAULT_ENABLE_INTERRUPT_MASK |
- VM_CONTEXT1_CNTL__DUMMY_PAGE_PROTECTION_FAULT_ENABLE_DEFAULT_MASK |
- VM_CONTEXT1_CNTL__PDE0_PROTECTION_FAULT_ENABLE_INTERRUPT_MASK |
- VM_CONTEXT1_CNTL__PDE0_PROTECTION_FAULT_ENABLE_DEFAULT_MASK |
- VM_CONTEXT1_CNTL__VALID_PROTECTION_FAULT_ENABLE_INTERRUPT_MASK |
- VM_CONTEXT1_CNTL__VALID_PROTECTION_FAULT_ENABLE_DEFAULT_MASK |
- VM_CONTEXT1_CNTL__READ_PROTECTION_FAULT_ENABLE_INTERRUPT_MASK |
- VM_CONTEXT1_CNTL__READ_PROTECTION_FAULT_ENABLE_DEFAULT_MASK |
- VM_CONTEXT1_CNTL__WRITE_PROTECTION_FAULT_ENABLE_INTERRUPT_MASK |
- VM_CONTEXT1_CNTL__WRITE_PROTECTION_FAULT_ENABLE_DEFAULT_MASK);
+ ((amdgpu_vm_block_size - 9) << VM_CONTEXT1_CNTL__PAGE_TABLE_BLOCK_SIZE__SHIFT));
+ if (amdgpu_vm_fault_stop == AMDGPU_VM_FAULT_STOP_ALWAYS)
+ gmc_v6_0_set_fault_enable_default(adev, false);
+ else
+ gmc_v6_0_set_fault_enable_default(adev, true);
gmc_v6_0_gart_flush_gpu_tlb(adev, 0);
dev_info(adev->dev, "PCIE GART of %uM enabled (table at 0x%016llX).\n",
{
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- return amdgpu_irq_get(adev, &adev->mc.vm_fault, 0);
+ if (amdgpu_vm_fault_stop != AMDGPU_VM_FAULT_STOP_ALWAYS)
+ return amdgpu_irq_get(adev, &adev->mc.vm_fault, 0);
+ else
+ return 0;
}
static int gmc_v6_0_sw_init(void *handle)
MODULE_FIRMWARE("amdgpu/tonga_mc.bin");
MODULE_FIRMWARE("amdgpu/polaris11_mc.bin");
MODULE_FIRMWARE("amdgpu/polaris10_mc.bin");
+MODULE_FIRMWARE("amdgpu/polaris12_mc.bin");
static const u32 golden_settings_tonga_a11[] =
{
(const u32)ARRAY_SIZE(golden_settings_tonga_a11));
break;
case CHIP_POLARIS11:
+ case CHIP_POLARIS12:
amdgpu_program_register_sequence(adev,
golden_settings_polaris11_a11,
(const u32)ARRAY_SIZE(golden_settings_polaris11_a11));
case CHIP_POLARIS10:
chip_name = "polaris10";
break;
+ case CHIP_POLARIS12:
+ chip_name = "polaris12";
+ break;
case CHIP_FIJI:
case CHIP_CARRIZO:
case CHIP_STONEY:
MODULE_FIRMWARE("amdgpu/polaris10_sdma1.bin");
MODULE_FIRMWARE("amdgpu/polaris11_sdma.bin");
MODULE_FIRMWARE("amdgpu/polaris11_sdma1.bin");
+MODULE_FIRMWARE("amdgpu/polaris12_sdma.bin");
+MODULE_FIRMWARE("amdgpu/polaris12_sdma1.bin");
static const u32 sdma_offsets[SDMA_MAX_INSTANCE] =
(const u32)ARRAY_SIZE(golden_settings_tonga_a11));
break;
case CHIP_POLARIS11:
+ case CHIP_POLARIS12:
amdgpu_program_register_sequence(adev,
golden_settings_polaris11_a11,
(const u32)ARRAY_SIZE(golden_settings_polaris11_a11));
case CHIP_POLARIS10:
chip_name = "polaris10";
break;
+ case CHIP_POLARIS12:
+ chip_name = "polaris12";
+ break;
case CHIP_CARRIZO:
chip_name = "carrizo";
break;
#define BIOS_SCRATCH_4 0x5cd
MODULE_FIRMWARE("radeon/tahiti_smc.bin");
-MODULE_FIRMWARE("radeon/tahiti_k_smc.bin");
MODULE_FIRMWARE("radeon/pitcairn_smc.bin");
MODULE_FIRMWARE("radeon/pitcairn_k_smc.bin");
MODULE_FIRMWARE("radeon/verde_smc.bin");
MODULE_FIRMWARE("radeon/oland_k_smc.bin");
MODULE_FIRMWARE("radeon/hainan_smc.bin");
MODULE_FIRMWARE("radeon/hainan_k_smc.bin");
+MODULE_FIRMWARE("radeon/banks_k_2_smc.bin");
union power_info {
struct _ATOM_POWERPLAY_INFO info;
(adev->pdev->device == 0x6817) ||
(adev->pdev->device == 0x6806))
max_mclk = 120000;
- } else if (adev->asic_type == CHIP_VERDE) {
- if ((adev->pdev->revision == 0x81) ||
- (adev->pdev->revision == 0x83) ||
- (adev->pdev->revision == 0x87) ||
- (adev->pdev->device == 0x6820) ||
- (adev->pdev->device == 0x6821) ||
- (adev->pdev->device == 0x6822) ||
- (adev->pdev->device == 0x6823) ||
- (adev->pdev->device == 0x682A) ||
- (adev->pdev->device == 0x682B)) {
- max_sclk = 75000;
- max_mclk = 80000;
- }
- } else if (adev->asic_type == CHIP_OLAND) {
- if ((adev->pdev->revision == 0xC7) ||
- (adev->pdev->revision == 0x80) ||
- (adev->pdev->revision == 0x81) ||
- (adev->pdev->revision == 0x83) ||
- (adev->pdev->revision == 0x87) ||
- (adev->pdev->device == 0x6604) ||
- (adev->pdev->device == 0x6605)) {
- max_sclk = 75000;
- max_mclk = 80000;
- }
} else if (adev->asic_type == CHIP_HAINAN) {
if ((adev->pdev->revision == 0x81) ||
(adev->pdev->revision == 0x83) ||
(adev->pdev->device == 0x6665) ||
(adev->pdev->device == 0x6667)) {
max_sclk = 75000;
- max_mclk = 80000;
}
}
/* Apply dpm quirks */
chip_name = "tahiti";
break;
case CHIP_PITCAIRN:
- if ((adev->pdev->revision == 0x81) ||
- (adev->pdev->device == 0x6810) ||
- (adev->pdev->device == 0x6811) ||
- (adev->pdev->device == 0x6816) ||
- (adev->pdev->device == 0x6817) ||
- (adev->pdev->device == 0x6806))
+ if ((adev->pdev->revision == 0x81) &&
+ ((adev->pdev->device == 0x6810) ||
+ (adev->pdev->device == 0x6811)))
chip_name = "pitcairn_k";
else
chip_name = "pitcairn";
break;
case CHIP_VERDE:
- if ((adev->pdev->revision == 0x81) ||
- (adev->pdev->revision == 0x83) ||
- (adev->pdev->revision == 0x87) ||
- (adev->pdev->device == 0x6820) ||
- (adev->pdev->device == 0x6821) ||
- (adev->pdev->device == 0x6822) ||
- (adev->pdev->device == 0x6823) ||
- (adev->pdev->device == 0x682A) ||
- (adev->pdev->device == 0x682B))
+ if (((adev->pdev->device == 0x6820) &&
+ ((adev->pdev->revision == 0x81) ||
+ (adev->pdev->revision == 0x83))) ||
+ ((adev->pdev->device == 0x6821) &&
+ ((adev->pdev->revision == 0x83) ||
+ (adev->pdev->revision == 0x87))) ||
+ ((adev->pdev->revision == 0x87) &&
+ ((adev->pdev->device == 0x6823) ||
+ (adev->pdev->device == 0x682b))))
chip_name = "verde_k";
else
chip_name = "verde";
break;
case CHIP_OLAND:
- if ((adev->pdev->revision == 0xC7) ||
- (adev->pdev->revision == 0x80) ||
- (adev->pdev->revision == 0x81) ||
- (adev->pdev->revision == 0x83) ||
- (adev->pdev->revision == 0x87) ||
- (adev->pdev->device == 0x6604) ||
- (adev->pdev->device == 0x6605))
+ if (((adev->pdev->revision == 0x81) &&
+ ((adev->pdev->device == 0x6600) ||
+ (adev->pdev->device == 0x6604) ||
+ (adev->pdev->device == 0x6605) ||
+ (adev->pdev->device == 0x6610))) ||
+ ((adev->pdev->revision == 0x83) &&
+ (adev->pdev->device == 0x6610)))
chip_name = "oland_k";
else
chip_name = "oland";
break;
case CHIP_HAINAN:
- if ((adev->pdev->revision == 0x81) ||
- (adev->pdev->revision == 0x83) ||
- (adev->pdev->revision == 0xC3) ||
- (adev->pdev->device == 0x6664) ||
- (adev->pdev->device == 0x6665) ||
- (adev->pdev->device == 0x6667))
+ if (((adev->pdev->revision == 0x81) &&
+ (adev->pdev->device == 0x6660)) ||
+ ((adev->pdev->revision == 0x83) &&
+ ((adev->pdev->device == 0x6660) ||
+ (adev->pdev->device == 0x6663) ||
+ (adev->pdev->device == 0x6665) ||
+ (adev->pdev->device == 0x6667))))
chip_name = "hainan_k";
+ else if ((adev->pdev->revision == 0xc3) &&
+ (adev->pdev->device == 0x6665))
+ chip_name = "banks_k_2";
else
chip_name = "hainan";
break;
#include "smu/smu_7_0_1_sh_mask.h"
static void uvd_v4_2_mc_resume(struct amdgpu_device *adev);
-static void uvd_v4_2_init_cg(struct amdgpu_device *adev);
static void uvd_v4_2_set_ring_funcs(struct amdgpu_device *adev);
static void uvd_v4_2_set_irq_funcs(struct amdgpu_device *adev);
static int uvd_v4_2_start(struct amdgpu_device *adev);
static void uvd_v4_2_stop(struct amdgpu_device *adev);
static int uvd_v4_2_set_clockgating_state(void *handle,
enum amd_clockgating_state state);
+static void uvd_v4_2_set_dcm(struct amdgpu_device *adev,
+ bool sw_mode);
/**
* uvd_v4_2_ring_get_rptr - get read pointer
*
return r;
}
-
+static void uvd_v4_2_enable_mgcg(struct amdgpu_device *adev,
+ bool enable);
/**
* uvd_v4_2_hw_init - start and test UVD block
*
uint32_t tmp;
int r;
- uvd_v4_2_init_cg(adev);
- uvd_v4_2_set_clockgating_state(adev, AMD_CG_STATE_GATE);
+ uvd_v4_2_enable_mgcg(adev, true);
amdgpu_asic_set_uvd_clocks(adev, 10000, 10000);
r = uvd_v4_2_start(adev);
if (r)
struct amdgpu_ring *ring = &adev->uvd.ring;
uint32_t rb_bufsz;
int i, j, r;
-
/* disable byte swapping */
u32 lmi_swap_cntl = 0;
u32 mp_swap_cntl = 0;
+ WREG32(mmUVD_CGC_GATE, 0);
+ uvd_v4_2_set_dcm(adev, true);
+
uvd_v4_2_mc_resume(adev);
/* disable interupt */
/* Unstall UMC and register bus */
WREG32_P(mmUVD_LMI_CTRL2, 0, ~(1 << 8));
+
+ uvd_v4_2_set_dcm(adev, false);
}
/**
WREG32_UVD_CTX(ixUVD_CGC_CTRL2, tmp2);
}
-static void uvd_v4_2_init_cg(struct amdgpu_device *adev)
-{
- bool hw_mode = true;
-
- if (hw_mode) {
- uvd_v4_2_set_dcm(adev, false);
- } else {
- u32 tmp = RREG32(mmUVD_CGC_CTRL);
- tmp &= ~UVD_CGC_CTRL__DYN_CLOCK_MODE_MASK;
- WREG32(mmUVD_CGC_CTRL, tmp);
- }
-}
-
static bool uvd_v4_2_is_idle(void *handle)
{
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
static int uvd_v4_2_set_clockgating_state(void *handle,
enum amd_clockgating_state state)
{
- bool gate = false;
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
-
- if (!(adev->cg_flags & AMD_CG_SUPPORT_UVD_MGCG))
- return 0;
-
- if (state == AMD_CG_STATE_GATE)
- gate = true;
-
- uvd_v4_2_enable_mgcg(adev, gate);
-
return 0;
}
*/
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- if (!(adev->pg_flags & AMD_PG_SUPPORT_UVD))
- return 0;
-
if (state == AMD_PG_STATE_GATE) {
uvd_v4_2_stop(adev);
return 0;
{
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
bool enable = (state == AMD_CG_STATE_GATE) ? true : false;
- static int curstate = -1;
if (!(adev->cg_flags & AMD_CG_SUPPORT_UVD_MGCG))
return 0;
- if (curstate == state)
- return 0;
-
- curstate = state;
if (enable) {
/* wait for STATUS to clear */
if (uvd_v5_0_wait_for_idle(handle))
#define GRBM_GFX_INDEX__VCE_INSTANCE__SHIFT 0x04
#define GRBM_GFX_INDEX__VCE_INSTANCE_MASK 0x10
+#define GRBM_GFX_INDEX__VCE_ALL_PIPE 0x07
+
#define mmVCE_LMI_VCPU_CACHE_40BIT_BAR0 0x8616
#define mmVCE_LMI_VCPU_CACHE_40BIT_BAR1 0x8617
#define mmVCE_LMI_VCPU_CACHE_40BIT_BAR2 0x8618
+#define mmGRBM_GFX_INDEX_DEFAULT 0xE0000000
+
#define VCE_STATUS_VCPU_REPORT_FW_LOADED_MASK 0x02
#define VCE_V3_0_FW_SIZE (384 * 1024)
#define FW_52_8_3 ((52 << 24) | (8 << 16) | (3 << 8))
+#define GET_VCE_INSTANCE(i) ((i) << GRBM_GFX_INDEX__VCE_INSTANCE__SHIFT \
+ | GRBM_GFX_INDEX__VCE_ALL_PIPE)
+
static void vce_v3_0_mc_resume(struct amdgpu_device *adev, int idx);
static void vce_v3_0_set_ring_funcs(struct amdgpu_device *adev);
static void vce_v3_0_set_irq_funcs(struct amdgpu_device *adev);
WREG32(mmVCE_UENC_CLOCK_GATING_2, data);
data = RREG32(mmVCE_UENC_REG_CLOCK_GATING);
- data &= ~0xffc00000;
+ data &= ~0x3ff;
WREG32(mmVCE_UENC_REG_CLOCK_GATING, data);
data = RREG32(mmVCE_UENC_DMA_DCLK_CTRL);
if (adev->vce.harvest_config & (1 << idx))
continue;
- WREG32_FIELD(GRBM_GFX_INDEX, VCE_INSTANCE, idx);
+ WREG32(mmGRBM_GFX_INDEX, GET_VCE_INSTANCE(idx));
vce_v3_0_mc_resume(adev, idx);
WREG32_FIELD(VCE_STATUS, JOB_BUSY, 1);
}
}
- WREG32_FIELD(GRBM_GFX_INDEX, VCE_INSTANCE, 0);
+ WREG32(mmGRBM_GFX_INDEX, mmGRBM_GFX_INDEX_DEFAULT);
mutex_unlock(&adev->grbm_idx_mutex);
return 0;
if (adev->vce.harvest_config & (1 << idx))
continue;
- WREG32_FIELD(GRBM_GFX_INDEX, VCE_INSTANCE, idx);
+ WREG32(mmGRBM_GFX_INDEX, GET_VCE_INSTANCE(idx));
if (adev->asic_type >= CHIP_STONEY)
WREG32_P(mmVCE_VCPU_CNTL, 0, ~0x200001);
vce_v3_0_set_vce_sw_clock_gating(adev, false);
}
- WREG32_FIELD(GRBM_GFX_INDEX, VCE_INSTANCE, 0);
+ WREG32(mmGRBM_GFX_INDEX, mmGRBM_GFX_INDEX_DEFAULT);
mutex_unlock(&adev->grbm_idx_mutex);
return 0;
{
u32 tmp;
- /* Fiji, Stoney, Polaris10, Polaris11 are single pipe */
+ /* Fiji, Stoney, Polaris10, Polaris11, Polaris12 are single pipe */
if ((adev->asic_type == CHIP_FIJI) ||
(adev->asic_type == CHIP_STONEY) ||
(adev->asic_type == CHIP_POLARIS10) ||
- (adev->asic_type == CHIP_POLARIS11))
+ (adev->asic_type == CHIP_POLARIS11) ||
+ (adev->asic_type == CHIP_POLARIS12))
return AMDGPU_VCE_HARVEST_VCE1;
/* Tonga and CZ are dual or single pipe */
* VCE team suggest use bit 3--bit 6 for busy status check
*/
mutex_lock(&adev->grbm_idx_mutex);
- WREG32_FIELD(GRBM_GFX_INDEX, INSTANCE_INDEX, 0);
+ WREG32(mmGRBM_GFX_INDEX, GET_VCE_INSTANCE(0));
if (RREG32(mmVCE_STATUS) & AMDGPU_VCE_STATUS_BUSY_MASK) {
srbm_soft_reset = REG_SET_FIELD(srbm_soft_reset, SRBM_SOFT_RESET, SOFT_RESET_VCE0, 1);
srbm_soft_reset = REG_SET_FIELD(srbm_soft_reset, SRBM_SOFT_RESET, SOFT_RESET_VCE1, 1);
}
- WREG32_FIELD(GRBM_GFX_INDEX, INSTANCE_INDEX, 0x10);
+ WREG32(mmGRBM_GFX_INDEX, GET_VCE_INSTANCE(1));
if (RREG32(mmVCE_STATUS) & AMDGPU_VCE_STATUS_BUSY_MASK) {
srbm_soft_reset = REG_SET_FIELD(srbm_soft_reset, SRBM_SOFT_RESET, SOFT_RESET_VCE0, 1);
srbm_soft_reset = REG_SET_FIELD(srbm_soft_reset, SRBM_SOFT_RESET, SOFT_RESET_VCE1, 1);
}
- WREG32_FIELD(GRBM_GFX_INDEX, INSTANCE_INDEX, 0);
+ WREG32(mmGRBM_GFX_INDEX, GET_VCE_INSTANCE(0));
mutex_unlock(&adev->grbm_idx_mutex);
if (srbm_soft_reset) {
if (adev->vce.harvest_config & (1 << i))
continue;
- WREG32_FIELD(GRBM_GFX_INDEX, VCE_INSTANCE, i);
+ WREG32(mmGRBM_GFX_INDEX, GET_VCE_INSTANCE(i));
if (enable) {
/* initialize VCE_CLOCK_GATING_A: Clock ON/OFF delay */
vce_v3_0_set_vce_sw_clock_gating(adev, enable);
}
- WREG32_FIELD(GRBM_GFX_INDEX, VCE_INSTANCE, 0);
+ WREG32(mmGRBM_GFX_INDEX, mmGRBM_GFX_INDEX_DEFAULT);
mutex_unlock(&adev->grbm_idx_mutex);
return 0;
MODULE_FIRMWARE("amdgpu/polaris10_smc_sk.bin");
MODULE_FIRMWARE("amdgpu/polaris11_smc.bin");
MODULE_FIRMWARE("amdgpu/polaris11_smc_sk.bin");
+MODULE_FIRMWARE("amdgpu/polaris12_smc.bin");
/*
* Indirect registers accessor
break;
case CHIP_POLARIS11:
case CHIP_POLARIS10:
+ case CHIP_POLARIS12:
default:
break;
}
case CHIP_TONGA:
case CHIP_POLARIS11:
case CHIP_POLARIS10:
+ case CHIP_POLARIS12:
case CHIP_CARRIZO:
case CHIP_STONEY:
asic_register_table = cz_allowed_read_registers;
adev->pg_flags = 0;
adev->external_rev_id = adev->rev_id + 0x50;
break;
+ case CHIP_POLARIS12:
+ adev->cg_flags = AMD_CG_SUPPORT_UVD_MGCG;
+ adev->pg_flags = 0;
+ adev->external_rev_id = adev->rev_id + 0x64;
+ break;
case CHIP_CARRIZO:
adev->cg_flags = AMD_CG_SUPPORT_UVD_MGCG |
AMD_CG_SUPPORT_GFX_MGCG |
case CHIP_TONGA:
case CHIP_POLARIS10:
case CHIP_POLARIS11:
+ case CHIP_POLARIS12:
vi_common_set_clockgating_state_by_smu(adev, state);
default:
break;
break;
case CHIP_POLARIS11:
case CHIP_POLARIS10:
+ case CHIP_POLARIS12:
amdgpu_ip_block_add(adev, &vi_common_ip_block);
amdgpu_ip_block_add(adev, &gmc_v8_1_ip_block);
amdgpu_ip_block_add(adev, &tonga_ih_ip_block);
return status;
}
-static int kfd_ioctl_dbg_unrgesiter(struct file *filep,
+static int kfd_ioctl_dbg_unregister(struct file *filep,
struct kfd_process *p, void *data)
{
struct kfd_ioctl_dbg_unregister_args *args = data;
return -EINVAL;
if (dev->device_info->asic_family == CHIP_CARRIZO) {
- pr_debug("kfd_ioctl_dbg_unrgesiter not supported on CZ\n");
+ pr_debug("kfd_ioctl_dbg_unregister not supported on CZ\n");
return -EINVAL;
}
kfd_ioctl_dbg_register, 0),
AMDKFD_IOCTL_DEF(AMDKFD_IOC_DBG_UNREGISTER,
- kfd_ioctl_dbg_unrgesiter, 0),
+ kfd_ioctl_dbg_unregister, 0),
AMDKFD_IOCTL_DEF(AMDKFD_IOC_DBG_ADDRESS_WATCH,
kfd_ioctl_dbg_address_watch, 0),
struct kfd_event_data event_data;
if (copy_from_user(&event_data, &events[i],
- sizeof(struct kfd_event_data)))
+ sizeof(struct kfd_event_data))) {
+ ret = -EFAULT;
goto fail;
+ }
ret = init_event_waiter(p, &event_waiters[i],
event_data.event_id, i);
#ifndef __AMD_SHARED_H__
#define __AMD_SHARED_H__
-#define AMD_MAX_USEC_TIMEOUT 100000 /* 100 ms */
+#define AMD_MAX_USEC_TIMEOUT 200000 /* 200 ms */
/*
* Supported ASIC types
CHIP_STONEY,
CHIP_POLARIS10,
CHIP_POLARIS11,
+ CHIP_POLARIS12,
CHIP_LAST,
};
cgs_set_clockgating_state(
hwmgr->device,
AMD_IP_BLOCK_TYPE_VCE,
- AMD_CG_STATE_UNGATE);
+ AMD_CG_STATE_GATE);
cgs_set_powergating_state(
hwmgr->device,
AMD_IP_BLOCK_TYPE_VCE,
cgs_set_clockgating_state(
hwmgr->device,
AMD_IP_BLOCK_TYPE_VCE,
- AMD_PG_STATE_GATE);
+ AMD_PG_STATE_UNGATE);
cz_dpm_update_vce_dpm(hwmgr);
cz_enable_disable_vce_dpm(hwmgr, true);
return 0;
cz_hwmgr->vce_dpm.hard_min_clk,
PPSMC_MSG_SetEclkHardMin));
} else {
- /*EPR# 419220 -HW limitation to to */
- cz_hwmgr->vce_dpm.hard_min_clk = hwmgr->vce_arbiter.ecclk;
- smum_send_msg_to_smc_with_parameter(hwmgr->smumgr,
- PPSMC_MSG_SetEclkHardMin,
- cz_get_eclk_level(hwmgr,
- cz_hwmgr->vce_dpm.hard_min_clk,
- PPSMC_MSG_SetEclkHardMin));
-
+ /*Program HardMin based on the vce_arbiter.ecclk */
+ if (hwmgr->vce_arbiter.ecclk == 0) {
+ smum_send_msg_to_smc_with_parameter(hwmgr->smumgr,
+ PPSMC_MSG_SetEclkHardMin, 0);
+ /* disable ECLK DPM 0. Otherwise VCE could hang if
+ * switching SCLK from DPM 0 to 6/7 */
+ smum_send_msg_to_smc_with_parameter(hwmgr->smumgr,
+ PPSMC_MSG_SetEclkSoftMin, 1);
+ } else {
+ cz_hwmgr->vce_dpm.hard_min_clk = hwmgr->vce_arbiter.ecclk;
+ smum_send_msg_to_smc_with_parameter(hwmgr->smumgr,
+ PPSMC_MSG_SetEclkHardMin,
+ cz_get_eclk_level(hwmgr,
+ cz_hwmgr->vce_dpm.hard_min_clk,
+ PPSMC_MSG_SetEclkHardMin));
+ }
}
return 0;
}
break;
case CHIP_POLARIS11:
case CHIP_POLARIS10:
+ case CHIP_POLARIS12:
polaris_set_asic_special_caps(hwmgr);
hwmgr->feature_mask &= ~(PP_UVD_HANDSHAKE_MASK);
break;
phm_cap_set(hwmgr->platform_descriptor.platformCaps,
PHM_PlatformCaps_TablelessHardwareInterface);
- if (hwmgr->chip_id == CHIP_POLARIS11)
+ if ((hwmgr->chip_id == CHIP_POLARIS11) || (hwmgr->chip_id == CHIP_POLARIS12))
phm_cap_set(hwmgr->platform_descriptor.platformCaps,
PHM_PlatformCaps_SPLLShutdownSupport);
return 0;
PP_ASSERT_WITH_CODE((result == 0), "DIDT Config failed.", return result);
result = smu7_program_pt_config_registers(hwmgr, DIDTConfig_Polaris10);
PP_ASSERT_WITH_CODE((result == 0), "DIDT Config failed.", return result);
- } else if (hwmgr->chip_id == CHIP_POLARIS11) {
+ } else if ((hwmgr->chip_id == CHIP_POLARIS11) || (hwmgr->chip_id == CHIP_POLARIS12)) {
result = smu7_program_pt_config_registers(hwmgr, GCCACConfig_Polaris11);
PP_ASSERT_WITH_CODE((result == 0), "DIDT Config failed.", return result);
result = smu7_program_pt_config_registers(hwmgr, DIDTConfig_Polaris11);
break;
case CHIP_POLARIS11:
case CHIP_POLARIS10:
+ case CHIP_POLARIS12:
polaris10_smum_init(smumgr);
break;
default:
#include <drm/drm_atomic_helper.h>
#include <drm/drm_crtc.h>
#include <drm/drm_crtc_helper.h>
-#include <drm/drm_fb_helper.h>
#include <drm/drm_fb_cma_helper.h>
#include <drm/drm_gem_cma_helper.h>
#include <drm/drm_of.h>
};
MODULE_DEVICE_TABLE(of, malidp_drm_of_match);
+static bool malidp_is_compatible_hw_id(struct malidp_hw_device *hwdev,
+ const struct of_device_id *dev_id)
+{
+ u32 core_id;
+ const char *compatstr_dp500 = "arm,mali-dp500";
+ bool is_dp500;
+ bool dt_is_dp500;
+
+ /*
+ * The DP500 CORE_ID register is in a different location, so check it
+ * first. If the product id field matches, then this is DP500, otherwise
+ * check the DP550/650 CORE_ID register.
+ */
+ core_id = malidp_hw_read(hwdev, MALIDP500_DC_BASE + MALIDP_DE_CORE_ID);
+ /* Offset 0x18 will never read 0x500 on products other than DP500. */
+ is_dp500 = (MALIDP_PRODUCT_ID(core_id) == 0x500);
+ dt_is_dp500 = strnstr(dev_id->compatible, compatstr_dp500,
+ sizeof(dev_id->compatible)) != NULL;
+ if (is_dp500 != dt_is_dp500) {
+ DRM_ERROR("Device-tree expects %s, but hardware %s DP500.\n",
+ dev_id->compatible, is_dp500 ? "is" : "is not");
+ return false;
+ } else if (!dt_is_dp500) {
+ u16 product_id;
+ char buf[32];
+
+ core_id = malidp_hw_read(hwdev,
+ MALIDP550_DC_BASE + MALIDP_DE_CORE_ID);
+ product_id = MALIDP_PRODUCT_ID(core_id);
+ snprintf(buf, sizeof(buf), "arm,mali-dp%X", product_id);
+ if (!strnstr(dev_id->compatible, buf,
+ sizeof(dev_id->compatible))) {
+ DRM_ERROR("Device-tree expects %s, but hardware is DP%03X.\n",
+ dev_id->compatible, product_id);
+ return false;
+ }
+ }
+ return true;
+}
+
+static bool malidp_has_sufficient_address_space(const struct resource *res,
+ const struct of_device_id *dev_id)
+{
+ resource_size_t res_size = resource_size(res);
+ const char *compatstr_dp500 = "arm,mali-dp500";
+
+ if (!strnstr(dev_id->compatible, compatstr_dp500,
+ sizeof(dev_id->compatible)))
+ return res_size >= MALIDP550_ADDR_SPACE_SIZE;
+ else if (res_size < MALIDP500_ADDR_SPACE_SIZE)
+ return false;
+ return true;
+}
+
#define MAX_OUTPUT_CHANNELS 3
static int malidp_bind(struct device *dev)
struct malidp_drm *malidp;
struct malidp_hw_device *hwdev;
struct platform_device *pdev = to_platform_device(dev);
+ struct of_device_id const *dev_id;
/* number of lines for the R, G and B output */
u8 output_width[MAX_OUTPUT_CHANNELS];
int ret = 0, i;
memcpy(hwdev, of_device_get_match_data(dev), sizeof(*hwdev));
malidp->dev = hwdev;
- INIT_LIST_HEAD(&malidp->event_list);
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
hwdev->regs = devm_ioremap_resource(dev, res);
clk_prepare_enable(hwdev->aclk);
clk_prepare_enable(hwdev->mclk);
+ dev_id = of_match_device(malidp_drm_of_match, dev);
+ if (!dev_id) {
+ ret = -EINVAL;
+ goto query_hw_fail;
+ }
+
+ if (!malidp_has_sufficient_address_space(res, dev_id)) {
+ DRM_ERROR("Insufficient address space in device-tree.\n");
+ ret = -EINVAL;
+ goto query_hw_fail;
+ }
+
+ if (!malidp_is_compatible_hw_id(hwdev, dev_id)) {
+ ret = -EINVAL;
+ goto query_hw_fail;
+ }
+
ret = hwdev->query_hw(hwdev);
if (ret) {
DRM_ERROR("Invalid HW configuration\n");
#include <linux/mutex.h>
#include <linux/wait.h>
+#include <drm/drmP.h>
#include "malidp_hw.h"
struct malidp_drm {
struct malidp_hw_device *dev;
struct drm_fbdev_cma *fbdev;
- struct list_head event_list;
struct drm_crtc crtc;
wait_queue_head_t wq;
atomic_t config_valid;
#include "malidp_drv.h"
#include "malidp_hw.h"
-static const struct malidp_input_format malidp500_de_formats[] = {
+static const struct malidp_format_id malidp500_de_formats[] = {
/* fourcc, layers supporting the format, internal id */
{ DRM_FORMAT_ARGB2101010, DE_VIDEO1 | DE_GRAPHICS1 | DE_GRAPHICS2, 0 },
{ DRM_FORMAT_ABGR2101010, DE_VIDEO1 | DE_GRAPHICS1 | DE_GRAPHICS2, 1 },
{ DRM_FORMAT_NV12, DE_VIDEO1 | DE_VIDEO2, MALIDP_ID(5, 6) }, \
{ DRM_FORMAT_YUV420, DE_VIDEO1 | DE_VIDEO2, MALIDP_ID(5, 7) }
-static const struct malidp_input_format malidp550_de_formats[] = {
+static const struct malidp_format_id malidp550_de_formats[] = {
MALIDP_COMMON_FORMATS,
};
static const struct malidp_layer malidp500_layers[] = {
- { DE_VIDEO1, MALIDP500_DE_LV_BASE, MALIDP500_DE_LV_PTR_BASE },
- { DE_GRAPHICS1, MALIDP500_DE_LG1_BASE, MALIDP500_DE_LG1_PTR_BASE },
- { DE_GRAPHICS2, MALIDP500_DE_LG2_BASE, MALIDP500_DE_LG2_PTR_BASE },
+ { DE_VIDEO1, MALIDP500_DE_LV_BASE, MALIDP500_DE_LV_PTR_BASE, MALIDP_DE_LV_STRIDE0 },
+ { DE_GRAPHICS1, MALIDP500_DE_LG1_BASE, MALIDP500_DE_LG1_PTR_BASE, MALIDP_DE_LG_STRIDE },
+ { DE_GRAPHICS2, MALIDP500_DE_LG2_BASE, MALIDP500_DE_LG2_PTR_BASE, MALIDP_DE_LG_STRIDE },
};
static const struct malidp_layer malidp550_layers[] = {
- { DE_VIDEO1, MALIDP550_DE_LV1_BASE, MALIDP550_DE_LV1_PTR_BASE },
- { DE_GRAPHICS1, MALIDP550_DE_LG_BASE, MALIDP550_DE_LG_PTR_BASE },
- { DE_VIDEO2, MALIDP550_DE_LV2_BASE, MALIDP550_DE_LV2_PTR_BASE },
- { DE_SMART, MALIDP550_DE_LS_BASE, MALIDP550_DE_LS_PTR_BASE },
+ { DE_VIDEO1, MALIDP550_DE_LV1_BASE, MALIDP550_DE_LV1_PTR_BASE, MALIDP_DE_LV_STRIDE0 },
+ { DE_GRAPHICS1, MALIDP550_DE_LG_BASE, MALIDP550_DE_LG_PTR_BASE, MALIDP_DE_LG_STRIDE },
+ { DE_VIDEO2, MALIDP550_DE_LV2_BASE, MALIDP550_DE_LV2_PTR_BASE, MALIDP_DE_LV_STRIDE0 },
+ { DE_SMART, MALIDP550_DE_LS_BASE, MALIDP550_DE_LS_PTR_BASE, 0 },
};
#define MALIDP_DE_DEFAULT_PREFETCH_START 5
.irq_mask = MALIDP500_DE_IRQ_CONF_VALID,
.vsync_irq = MALIDP500_DE_IRQ_CONF_VALID,
},
- .input_formats = malidp500_de_formats,
- .n_input_formats = ARRAY_SIZE(malidp500_de_formats),
+ .pixel_formats = malidp500_de_formats,
+ .n_pixel_formats = ARRAY_SIZE(malidp500_de_formats),
.bus_align_bytes = 8,
},
.query_hw = malidp500_query_hw,
.set_config_valid = malidp500_set_config_valid,
.modeset = malidp500_modeset,
.rotmem_required = malidp500_rotmem_required,
+ .features = MALIDP_DEVICE_LV_HAS_3_STRIDES,
},
[MALIDP_550] = {
.map = {
.irq_mask = MALIDP550_DC_IRQ_CONF_VALID,
.vsync_irq = MALIDP550_DC_IRQ_CONF_VALID,
},
- .input_formats = malidp550_de_formats,
- .n_input_formats = ARRAY_SIZE(malidp550_de_formats),
+ .pixel_formats = malidp550_de_formats,
+ .n_pixel_formats = ARRAY_SIZE(malidp550_de_formats),
.bus_align_bytes = 8,
},
.query_hw = malidp550_query_hw,
.set_config_valid = malidp550_set_config_valid,
.modeset = malidp550_modeset,
.rotmem_required = malidp550_rotmem_required,
+ .features = 0,
},
[MALIDP_650] = {
.map = {
.irq_mask = MALIDP550_DC_IRQ_CONF_VALID,
.vsync_irq = MALIDP550_DC_IRQ_CONF_VALID,
},
- .input_formats = malidp550_de_formats,
- .n_input_formats = ARRAY_SIZE(malidp550_de_formats),
+ .pixel_formats = malidp550_de_formats,
+ .n_pixel_formats = ARRAY_SIZE(malidp550_de_formats),
.bus_align_bytes = 16,
},
.query_hw = malidp650_query_hw,
.set_config_valid = malidp550_set_config_valid,
.modeset = malidp550_modeset,
.rotmem_required = malidp550_rotmem_required,
+ .features = 0,
},
};
{
unsigned int i;
- for (i = 0; i < map->n_input_formats; i++) {
- if (((map->input_formats[i].layer & layer_id) == layer_id) &&
- (map->input_formats[i].format == format))
- return map->input_formats[i].id;
+ for (i = 0; i < map->n_pixel_formats; i++) {
+ if (((map->pixel_formats[i].layer & layer_id) == layer_id) &&
+ (map->pixel_formats[i].format == format))
+ return map->pixel_formats[i].id;
}
return MALIDP_INVALID_FORMAT_ID;
DE_SMART = BIT(4),
};
-struct malidp_input_format {
+struct malidp_format_id {
u32 format; /* DRM fourcc */
u8 layer; /* bitmask of layers supporting it */
u8 id; /* used internally */
u16 id; /* layer ID */
u16 base; /* address offset for the register bank */
u16 ptr; /* address offset for the pointer register */
+ u16 stride_offset; /* Offset to the first stride register. */
};
/* regmap features */
const struct malidp_irq_map se_irq_map;
const struct malidp_irq_map dc_irq_map;
- /* list of supported input formats for each layer */
- const struct malidp_input_format *input_formats;
- const u8 n_input_formats;
+ /* list of supported pixel formats for each layer */
+ const struct malidp_format_id *pixel_formats;
+ const u8 n_pixel_formats;
/* pitch alignment requirement in bytes */
const u8 bus_align_bytes;
};
+/* device features */
+/* Unlike DP550/650, DP500 has 3 stride registers in its video layer. */
+#define MALIDP_DEVICE_LV_HAS_3_STRIDES BIT(0)
+
struct malidp_hw_device {
const struct malidp_hw_regmap map;
void __iomem *regs;
*/
#include <drm/drmP.h>
+#include <drm/drm_atomic.h>
#include <drm/drm_atomic_helper.h>
#include <drm/drm_fb_cma_helper.h>
#include <drm/drm_gem_cma_helper.h>
#define LAYER_V_VAL(x) (((x) & 0x1fff) << 16)
#define MALIDP_LAYER_COMP_SIZE 0x010
#define MALIDP_LAYER_OFFSET 0x014
-#define MALIDP_LAYER_STRIDE 0x018
/*
* This 4-entry look-up-table is used to determine the full 8-bit alpha value
return NULL;
state = kmalloc(sizeof(*state), GFP_KERNEL);
- if (state) {
- m_state = to_malidp_plane_state(plane->state);
- __drm_atomic_helper_plane_duplicate_state(plane, &state->base);
- state->rotmem_size = m_state->rotmem_size;
- state->format = m_state->format;
- state->n_planes = m_state->n_planes;
- }
+ if (!state)
+ return NULL;
+
+ m_state = to_malidp_plane_state(plane->state);
+ __drm_atomic_helper_plane_duplicate_state(plane, &state->base);
+ state->rotmem_size = m_state->rotmem_size;
+ state->format = m_state->format;
+ state->n_planes = m_state->n_planes;
return &state->base;
}
{
struct malidp_plane *mp = to_malidp_plane(plane);
struct malidp_plane_state *ms = to_malidp_plane_state(state);
+ struct drm_crtc_state *crtc_state;
struct drm_framebuffer *fb;
- int i;
+ struct drm_rect clip = { 0 };
+ int i, ret;
u32 src_w, src_h;
if (!state->crtc || !state->fb)
if ((state->crtc_w > mp->hwdev->max_line_size) ||
(state->crtc_h > mp->hwdev->max_line_size) ||
(state->crtc_w < mp->hwdev->min_line_size) ||
- (state->crtc_h < mp->hwdev->min_line_size) ||
- (state->crtc_w != src_w) || (state->crtc_h != src_h))
+ (state->crtc_h < mp->hwdev->min_line_size))
+ return -EINVAL;
+
+ /*
+ * DP550/650 video layers can accept 3 plane formats only if
+ * fb->pitches[1] == fb->pitches[2] since they don't have a
+ * third plane stride register.
+ */
+ if (ms->n_planes == 3 &&
+ !(mp->hwdev->features & MALIDP_DEVICE_LV_HAS_3_STRIDES) &&
+ (state->fb->pitches[1] != state->fb->pitches[2]))
return -EINVAL;
/* packed RGB888 / BGR888 can't be rotated or flipped */
fb->format->format == DRM_FORMAT_BGR888))
return -EINVAL;
+ crtc_state = drm_atomic_get_existing_crtc_state(state->state, state->crtc);
+ clip.x2 = crtc_state->adjusted_mode.hdisplay;
+ clip.y2 = crtc_state->adjusted_mode.vdisplay;
+ ret = drm_plane_helper_check_state(state, &clip,
+ DRM_PLANE_HELPER_NO_SCALING,
+ DRM_PLANE_HELPER_NO_SCALING,
+ true, true);
+ if (ret)
+ return ret;
+
ms->rotmem_size = 0;
if (state->rotation & MALIDP_ROTATED_MASK) {
int val;
return 0;
}
+static void malidp_de_set_plane_pitches(struct malidp_plane *mp,
+ int num_planes, unsigned int pitches[3])
+{
+ int i;
+ int num_strides = num_planes;
+
+ if (!mp->layer->stride_offset)
+ return;
+
+ if (num_planes == 3)
+ num_strides = (mp->hwdev->features &
+ MALIDP_DEVICE_LV_HAS_3_STRIDES) ? 3 : 2;
+
+ for (i = 0; i < num_strides; ++i)
+ malidp_hw_write(mp->hwdev, pitches[i],
+ mp->layer->base +
+ mp->layer->stride_offset + i * 4);
+}
+
static void malidp_de_plane_update(struct drm_plane *plane,
struct drm_plane_state *old_state)
{
/* convert src values from Q16 fixed point to integer */
src_w = plane->state->src_w >> 16;
src_h = plane->state->src_h >> 16;
- if (plane->state->rotation & MALIDP_ROTATED_MASK) {
- dest_w = plane->state->crtc_h;
- dest_h = plane->state->crtc_w;
- } else {
- dest_w = plane->state->crtc_w;
- dest_h = plane->state->crtc_h;
- }
+ dest_w = plane->state->crtc_w;
+ dest_h = plane->state->crtc_h;
malidp_hw_write(mp->hwdev, ms->format, mp->layer->base);
ptr = mp->layer->ptr + (i << 4);
obj = drm_fb_cma_get_gem_obj(plane->state->fb, i);
+ obj->paddr += plane->state->fb->offsets[i];
malidp_hw_write(mp->hwdev, lower_32_bits(obj->paddr), ptr);
malidp_hw_write(mp->hwdev, upper_32_bits(obj->paddr), ptr + 4);
- malidp_hw_write(mp->hwdev, plane->state->fb->pitches[i],
- mp->layer->base + MALIDP_LAYER_STRIDE);
}
+ malidp_de_set_plane_pitches(mp, ms->n_planes,
+ plane->state->fb->pitches);
malidp_hw_write(mp->hwdev, LAYER_H_VAL(src_w) | LAYER_V_VAL(src_h),
mp->layer->base + MALIDP_LAYER_SIZE);
/* setup the rotation and axis flip bits */
if (plane->state->rotation & DRM_ROTATE_MASK)
- val = ilog2(plane->state->rotation & DRM_ROTATE_MASK) << LAYER_ROT_OFFSET;
+ val |= ilog2(plane->state->rotation & DRM_ROTATE_MASK) <<
+ LAYER_ROT_OFFSET;
if (plane->state->rotation & DRM_REFLECT_X)
- val |= LAYER_V_FLIP;
- if (plane->state->rotation & DRM_REFLECT_Y)
val |= LAYER_H_FLIP;
+ if (plane->state->rotation & DRM_REFLECT_Y)
+ val |= LAYER_V_FLIP;
/*
* always enable pixel alpha blending until we have a way to change
u32 *formats;
int ret, i, j, n;
- formats = kcalloc(map->n_input_formats, sizeof(*formats), GFP_KERNEL);
+ formats = kcalloc(map->n_pixel_formats, sizeof(*formats), GFP_KERNEL);
if (!formats) {
ret = -ENOMEM;
goto cleanup;
}
/* build the list of DRM supported formats based on the map */
- for (n = 0, j = 0; j < map->n_input_formats; j++) {
- if ((map->input_formats[j].layer & id) == id)
- formats[n++] = map->input_formats[j].format;
+ for (n = 0, j = 0; j < map->n_pixel_formats; j++) {
+ if ((map->pixel_formats[j].layer & id) == id)
+ formats[n++] = map->pixel_formats[j].format;
}
plane_type = (i == 0) ? DRM_PLANE_TYPE_PRIMARY :
#define MALIDP_DE_SYNC_WIDTH 0x8
#define MALIDP_DE_HV_ACTIVE 0xc
+/* Stride register offsets relative to Lx_BASE */
+#define MALIDP_DE_LG_STRIDE 0x18
+#define MALIDP_DE_LV_STRIDE0 0x18
+
/* macros to set values into registers */
#define MALIDP_DE_H_FRONTPORCH(x) (((x) & 0xfff) << 0)
#define MALIDP_DE_H_BACKPORCH(x) (((x) & 0x3ff) << 16)
#define MALIDP_DE_H_ACTIVE(x) (((x) & 0x1fff) << 0)
#define MALIDP_DE_V_ACTIVE(x) (((x) & 0x1fff) << 16)
+#define MALIDP_PRODUCT_ID(__core_id) ((u32)(__core_id) >> 16)
+
/* register offsets and bits specific to DP500 */
+#define MALIDP500_ADDR_SPACE_SIZE 0x01000
#define MALIDP500_DC_BASE 0x00000
#define MALIDP500_DC_CONTROL 0x0000c
#define MALIDP500_DC_CONFIG_REQ (1 << 17)
#define MALIDP500_CONFIG_ID 0x00fd4
/* register offsets and bits specific to DP550/DP650 */
+#define MALIDP550_ADDR_SPACE_SIZE 0x10000
#define MALIDP550_DE_CONTROL 0x00010
#define MALIDP550_DE_LINE_COUNTER 0x00014
#define MALIDP550_DE_AXI_CONTROL 0x00018
{
struct atmel_hlcdc_dc *dc = dev->dev_private;
- if (dc->fbdev) {
+ if (dc->fbdev)
drm_fbdev_cma_hotplug_event(dc->fbdev);
- } else {
- dc->fbdev = drm_fbdev_cma_init(dev, 24,
- dev->mode_config.num_crtc,
- dev->mode_config.num_connector);
- if (IS_ERR(dc->fbdev))
- dc->fbdev = NULL;
- }
}
struct atmel_hlcdc_dc_commit {
platform_set_drvdata(pdev, dev);
- drm_kms_helper_poll_init(dev);
+ dc->fbdev = drm_fbdev_cma_init(dev, 24,
+ dev->mode_config.num_crtc,
+ dev->mode_config.num_connector);
+ if (IS_ERR(dc->fbdev))
+ dc->fbdev = NULL;
- /* force connectors detection */
- drm_helper_hpd_irq_event(dev);
+ drm_kms_helper_poll_init(dev);
return 0;
pm_runtime_enable(dev);
+ pm_runtime_get_sync(dev);
phy_power_on(dp->phy);
analogix_dp_init_dp(dp);
goto err_disable_pm_runtime;
}
+ phy_power_off(dp->phy);
+ pm_runtime_put(dev);
+
return 0;
err_disable_pm_runtime:
+
+ phy_power_off(dp->phy);
+ pm_runtime_put(dev);
pm_runtime_disable(dev);
return ret;
This is a KMS driver for emulated cirrus device in qemu.
It is *NOT* intended for real cirrus devices. This requires
the modesetting userspace X.org driver.
+
+ Cirrus is obsolete, the hardware was designed in the 90ies
+ and can't keep up with todays needs. More background:
+ https://www.kraxel.org/blog/2014/10/qemu-using-cirrus-considered-harmful/
+
+ Better alternatives are:
+ - stdvga (DRM_BOCHS, qemu -vga std, default in qemu 2.2+)
+ - qxl (DRM_QXL, qemu -vga qxl, works best with spice)
+ - virtio (DRM_VIRTIO_GPU), qemu -vga virtio)
if (!nonblock) {
ret = drm_atomic_helper_wait_for_fences(dev, state, true);
- if (ret)
+ if (ret) {
+ drm_atomic_helper_cleanup_planes(dev, state);
return ret;
+ }
}
/*
{
struct drm_device *dev = container_of(ref, struct drm_device, ref);
+ drm_vblank_cleanup(dev);
+
if (drm_core_check_feature(dev, DRIVER_GEM))
drm_gem_destroy(dev);
if (dev->agp)
drm_pci_agp_destroy(dev);
- drm_vblank_cleanup(dev);
-
list_for_each_entry_safe(r_list, list_temp, &dev->maplist, head)
drm_legacy_rmmap(dev, r_list->map);
return NULL;
mode->type |= DRM_MODE_TYPE_USERDEF;
+ /* fix up 1368x768: GFT/CVT can't express 1366 width due to alignment */
+ if (cmd->xres == 1366 && mode->hdisplay == 1368) {
+ mode->hdisplay = 1366;
+ mode->hsync_start--;
+ mode->hsync_end--;
+ drm_mode_set_name(mode);
+ }
drm_mode_set_crtcinfo(mode, CRTC_INTERLACE_HALVE_V);
return mode;
}
#define DRM_OUTPUT_POLL_PERIOD (10*HZ)
/**
- * drm_kms_helper_poll_enable_locked - re-enable output polling.
+ * drm_kms_helper_poll_enable - re-enable output polling.
* @dev: drm_device
*
- * This function re-enables the output polling work without
- * locking the mode_config mutex.
+ * This function re-enables the output polling work, after it has been
+ * temporarily disabled using drm_kms_helper_poll_disable(), for example over
+ * suspend/resume.
*
- * This is like drm_kms_helper_poll_enable() however it is to be
- * called from a context where the mode_config mutex is locked
- * already.
+ * Drivers can call this helper from their device resume implementation. It is
+ * an error to call this when the output polling support has not yet been set
+ * up.
+ *
+ * Note that calls to enable and disable polling must be strictly ordered, which
+ * is automatically the case when they're only call from suspend/resume
+ * callbacks.
*/
-void drm_kms_helper_poll_enable_locked(struct drm_device *dev)
+void drm_kms_helper_poll_enable(struct drm_device *dev)
{
bool poll = false;
struct drm_connector *connector;
struct drm_connector_list_iter conn_iter;
unsigned long delay = DRM_OUTPUT_POLL_PERIOD;
- WARN_ON(!mutex_is_locked(&dev->mode_config.mutex));
-
if (!dev->mode_config.poll_enabled || !drm_kms_helper_poll)
return;
drm_connector_list_iter_put(&conn_iter);
if (dev->mode_config.delayed_event) {
+ /*
+ * FIXME:
+ *
+ * Use short (1s) delay to handle the initial delayed event.
+ * This delay should not be needed, but Optimus/nouveau will
+ * fail in a mysterious way if the delayed event is handled as
+ * soon as possible like it is done in
+ * drm_helper_probe_single_connector_modes() in case the poll
+ * was enabled before.
+ */
poll = true;
- delay = 0;
+ delay = HZ;
}
if (poll)
schedule_delayed_work(&dev->mode_config.output_poll_work, delay);
}
-EXPORT_SYMBOL(drm_kms_helper_poll_enable_locked);
+EXPORT_SYMBOL(drm_kms_helper_poll_enable);
static enum drm_connector_status
drm_connector_detect(struct drm_connector *connector, bool force)
/* Re-enable polling in case the global poll config changed. */
if (drm_kms_helper_poll != dev->mode_config.poll_running)
- drm_kms_helper_poll_enable_locked(dev);
+ drm_kms_helper_poll_enable(dev);
dev->mode_config.poll_running = drm_kms_helper_poll;
* This function disables the output polling work.
*
* Drivers can call this helper from their device suspend implementation. It is
- * not an error to call this even when output polling isn't enabled or arlready
- * disabled.
+ * not an error to call this even when output polling isn't enabled or already
+ * disabled. Polling is re-enabled by calling drm_kms_helper_poll_enable().
+ *
+ * Note that calls to enable and disable polling must be strictly ordered, which
+ * is automatically the case when they're only call from suspend/resume
+ * callbacks.
*/
void drm_kms_helper_poll_disable(struct drm_device *dev)
{
}
EXPORT_SYMBOL(drm_kms_helper_poll_disable);
-/**
- * drm_kms_helper_poll_enable - re-enable output polling.
- * @dev: drm_device
- *
- * This function re-enables the output polling work.
- *
- * Drivers can call this helper from their device resume implementation. It is
- * an error to call this when the output polling support has not yet been set
- * up.
- */
-void drm_kms_helper_poll_enable(struct drm_device *dev)
-{
- mutex_lock(&dev->mode_config.mutex);
- drm_kms_helper_poll_enable_locked(dev);
- mutex_unlock(&dev->mode_config.mutex);
-}
-EXPORT_SYMBOL(drm_kms_helper_poll_enable);
-
/**
* drm_kms_helper_poll_init - initialize and enable output polling
* @dev: drm_device
struct list_head list;
bool found;
+ /*
+ * XXX: The DRM_MM_SEARCH_BELOW is really a hack to trick
+ * drm_mm into giving out a low IOVA after address space
+ * rollover. This needs a proper fix.
+ */
ret = drm_mm_insert_node_in_range(&mmu->mm, node,
size, 0, mmu->last_iova, ~0UL,
- DRM_MM_SEARCH_DEFAULT);
+ mmu->last_iova ? DRM_MM_SEARCH_DEFAULT : DRM_MM_SEARCH_BELOW);
if (ret != -ENOSPC)
break;
BIT_CLKS_ENABLED,
BIT_IRQS_ENABLED,
BIT_WIN_UPDATED,
- BIT_SUSPENDED
+ BIT_SUSPENDED,
+ BIT_REQUEST_UPDATE
};
struct decon_context {
m->crtc_vsync_end = m->crtc_vsync_start + 1;
}
- decon_set_bits(ctx, DECON_VIDCON0, VIDCON0_ENVID, 0);
-
- /* enable clock gate */
- val = CMU_CLKGAGE_MODE_SFR_F | CMU_CLKGAGE_MODE_MEM_F;
- writel(val, ctx->addr + DECON_CMU);
-
if (ctx->out_type & (IFTYPE_I80 | I80_HW_TRG))
decon_setup_trigger(ctx);
/* window enable */
decon_set_bits(ctx, DECON_WINCONx(win), WINCONx_ENWIN_F, ~0);
+ set_bit(BIT_REQUEST_UPDATE, &ctx->flags);
}
static void decon_disable_plane(struct exynos_drm_crtc *crtc,
return;
decon_set_bits(ctx, DECON_WINCONx(win), WINCONx_ENWIN_F, 0);
+ set_bit(BIT_REQUEST_UPDATE, &ctx->flags);
}
static void decon_atomic_flush(struct exynos_drm_crtc *crtc)
for (i = ctx->first_win; i < WINDOWS_NR; i++)
decon_shadow_protect_win(ctx, i, false);
- /* standalone update */
- decon_set_bits(ctx, DECON_UPDATE, STANDALONE_UPDATE_F, ~0);
+ if (test_and_clear_bit(BIT_REQUEST_UPDATE, &ctx->flags))
+ decon_set_bits(ctx, DECON_UPDATE, STANDALONE_UPDATE_F, ~0);
if (ctx->out_type & IFTYPE_I80)
set_bit(BIT_WIN_UPDATED, &ctx->flags);
# general-purpose microcontroller (GuC) support
i915-y += intel_uc.o \
+ intel_guc_log.o \
intel_guc_loader.o \
+ intel_huc.o \
i915_guc_submission.o
# autogenerated null render state
#include "i915_drv.h"
#include "gvt.h"
-#define MB_TO_BYTES(mb) ((mb) << 20ULL)
-#define BYTES_TO_MB(b) ((b) >> 20ULL)
-
-#define HOST_LOW_GM_SIZE MB_TO_BYTES(128)
-#define HOST_HIGH_GM_SIZE MB_TO_BYTES(384)
-#define HOST_FENCE 4
-
static int alloc_gm(struct intel_vgpu *vgpu, bool high_gm)
{
struct intel_gvt *gvt = vgpu->gvt;
struct drm_i915_private *dev_priv = gvt->dev_priv;
- u32 alloc_flag, search_flag;
+ unsigned int flags;
u64 start, end, size;
struct drm_mm_node *node;
- int retried = 0;
int ret;
if (high_gm) {
- search_flag = DRM_MM_SEARCH_BELOW;
- alloc_flag = DRM_MM_CREATE_TOP;
node = &vgpu->gm.high_gm_node;
size = vgpu_hidden_sz(vgpu);
start = gvt_hidden_gmadr_base(gvt);
end = gvt_hidden_gmadr_end(gvt);
+ flags = PIN_HIGH;
} else {
- search_flag = DRM_MM_SEARCH_DEFAULT;
- alloc_flag = DRM_MM_CREATE_DEFAULT;
node = &vgpu->gm.low_gm_node;
size = vgpu_aperture_sz(vgpu);
start = gvt_aperture_gmadr_base(gvt);
end = gvt_aperture_gmadr_end(gvt);
+ flags = PIN_MAPPABLE;
}
mutex_lock(&dev_priv->drm.struct_mutex);
-search_again:
- ret = drm_mm_insert_node_in_range_generic(&dev_priv->ggtt.base.mm,
- node, size, 4096,
- I915_COLOR_UNEVICTABLE,
- start, end, search_flag,
- alloc_flag);
- if (ret) {
- ret = i915_gem_evict_something(&dev_priv->ggtt.base,
- size, 4096,
- I915_COLOR_UNEVICTABLE,
- start, end, 0);
- if (ret == 0 && ++retried < 3)
- goto search_again;
-
- gvt_err("fail to alloc %s gm space from host, retried %d\n",
- high_gm ? "high" : "low", retried);
- }
+ ret = i915_gem_gtt_insert(&dev_priv->ggtt.base, node,
+ size, 4096, I915_COLOR_UNEVICTABLE,
+ start, end, flags);
mutex_unlock(&dev_priv->drm.struct_mutex);
+ if (ret)
+ gvt_err("fail to alloc %s gm space from host\n",
+ high_gm ? "high" : "low");
+
return ret;
}
POSTING_READ(fence_reg_lo);
}
+static void _clear_vgpu_fence(struct intel_vgpu *vgpu)
+{
+ int i;
+
+ for (i = 0; i < vgpu_fence_sz(vgpu); i++)
+ intel_vgpu_write_fence(vgpu, i, 0);
+}
+
static void free_vgpu_fence(struct intel_vgpu *vgpu)
{
struct intel_gvt *gvt = vgpu->gvt;
intel_runtime_pm_get(dev_priv);
mutex_lock(&dev_priv->drm.struct_mutex);
+ _clear_vgpu_fence(vgpu);
for (i = 0; i < vgpu_fence_sz(vgpu); i++) {
reg = vgpu->fence.regs[i];
- intel_vgpu_write_fence(vgpu, i, 0);
list_add_tail(®->link,
&dev_priv->mm.fence_list);
}
continue;
list_del(pos);
vgpu->fence.regs[i] = reg;
- intel_vgpu_write_fence(vgpu, i, 0);
if (++i == vgpu_fence_sz(vgpu))
break;
}
if (i != vgpu_fence_sz(vgpu))
goto out_free_fence;
+ _clear_vgpu_fence(vgpu);
+
mutex_unlock(&dev_priv->drm.struct_mutex);
intel_runtime_pm_put(dev_priv);
return 0;
free_resource(vgpu);
}
+/**
+ * intel_vgpu_reset_resource - reset resource state owned by a vGPU
+ * @vgpu: a vGPU
+ *
+ * This function is used to reset resource state owned by a vGPU.
+ *
+ */
+void intel_vgpu_reset_resource(struct intel_vgpu *vgpu)
+{
+ struct drm_i915_private *dev_priv = vgpu->gvt->dev_priv;
+
+ intel_runtime_pm_get(dev_priv);
+ _clear_vgpu_fence(vgpu);
+ intel_runtime_pm_put(dev_priv);
+}
+
/**
* intel_alloc_vgpu_resource - allocate HW resource for a vGPU
* @vgpu: vGPU
}
return 0;
}
+
+/**
+ * intel_vgpu_init_cfg_space - init vGPU configuration space when create vGPU
+ *
+ * @vgpu: a vGPU
+ * @primary: is the vGPU presented as primary
+ *
+ */
+void intel_vgpu_init_cfg_space(struct intel_vgpu *vgpu,
+ bool primary)
+{
+ struct intel_gvt *gvt = vgpu->gvt;
+ const struct intel_gvt_device_info *info = &gvt->device_info;
+ u16 *gmch_ctl;
+ int i;
+
+ memcpy(vgpu_cfg_space(vgpu), gvt->firmware.cfg_space,
+ info->cfg_space_size);
+
+ if (!primary) {
+ vgpu_cfg_space(vgpu)[PCI_CLASS_DEVICE] =
+ INTEL_GVT_PCI_CLASS_VGA_OTHER;
+ vgpu_cfg_space(vgpu)[PCI_CLASS_PROG] =
+ INTEL_GVT_PCI_CLASS_VGA_OTHER;
+ }
+
+ /* Show guest that there isn't any stolen memory.*/
+ gmch_ctl = (u16 *)(vgpu_cfg_space(vgpu) + INTEL_GVT_PCI_GMCH_CONTROL);
+ *gmch_ctl &= ~(BDW_GMCH_GMS_MASK << BDW_GMCH_GMS_SHIFT);
+
+ intel_vgpu_write_pci_bar(vgpu, PCI_BASE_ADDRESS_2,
+ gvt_aperture_pa_base(gvt), true);
+
+ vgpu_cfg_space(vgpu)[PCI_COMMAND] &= ~(PCI_COMMAND_IO
+ | PCI_COMMAND_MEMORY
+ | PCI_COMMAND_MASTER);
+ /*
+ * Clear the bar upper 32bit and let guest to assign the new value
+ */
+ memset(vgpu_cfg_space(vgpu) + PCI_BASE_ADDRESS_1, 0, 4);
+ memset(vgpu_cfg_space(vgpu) + PCI_BASE_ADDRESS_3, 0, 4);
+ memset(vgpu_cfg_space(vgpu) + INTEL_GVT_PCI_OPREGION, 0, 4);
+
+ for (i = 0; i < INTEL_GVT_MAX_BAR_NUM; i++) {
+ vgpu->cfg_space.bar[i].size = pci_resource_len(
+ gvt->dev_priv->drm.pdev, i * 2);
+ vgpu->cfg_space.bar[i].tracked = false;
+ }
+}
+
+/**
+ * intel_vgpu_reset_cfg_space - reset vGPU configuration space
+ *
+ * @vgpu: a vGPU
+ *
+ */
+void intel_vgpu_reset_cfg_space(struct intel_vgpu *vgpu)
+{
+ u8 cmd = vgpu_cfg_space(vgpu)[PCI_COMMAND];
+ bool primary = vgpu_cfg_space(vgpu)[PCI_CLASS_DEVICE] !=
+ INTEL_GVT_PCI_CLASS_VGA_OTHER;
+
+ if (cmd & PCI_COMMAND_MEMORY) {
+ trap_gttmmio(vgpu, false);
+ map_aperture(vgpu, false);
+ }
+
+ /**
+ * Currently we only do such reset when vGPU is not
+ * owned by any VM, so we simply restore entire cfg
+ * space to default value.
+ */
+ intel_vgpu_init_cfg_space(vgpu, primary);
+}
static u64 read_pte64(struct drm_i915_private *dev_priv, unsigned long index)
{
void __iomem *addr = (gen8_pte_t __iomem *)dev_priv->ggtt.gsm + index;
- u64 pte;
-#ifdef readq
- pte = readq(addr);
-#else
- pte = ioread32(addr);
- pte |= (u64)ioread32(addr + 4) << 32;
-#endif
- return pte;
+ return readq(addr);
}
static void write_pte64(struct drm_i915_private *dev_priv,
{
void __iomem *addr = (gen8_pte_t __iomem *)dev_priv->ggtt.gsm + index;
-#ifdef writeq
writeq(pte, addr);
-#else
- iowrite32((u32)pte, addr);
- iowrite32(pte >> 32, addr + 4);
-#endif
+
I915_WRITE(GFX_FLSH_CNTL_GEN6, GFX_FLSH_CNTL_EN);
POSTING_READ(GFX_FLSH_CNTL_GEN6);
}
info->gtt_entry_size;
mem = kzalloc(mm->has_shadow_page_table ?
mm->page_table_entry_size * 2
- : mm->page_table_entry_size,
- GFP_ATOMIC);
+ : mm->page_table_entry_size, GFP_KERNEL);
if (!mem)
return -ENOMEM;
mm->virtual_page_table = mem;
struct intel_vgpu_mm *mm;
int ret;
- mm = kzalloc(sizeof(*mm), GFP_ATOMIC);
+ mm = kzalloc(sizeof(*mm), GFP_KERNEL);
if (!mm) {
ret = -ENOMEM;
goto fail;
struct intel_gvt_gtt_pte_ops *ops = vgpu->gvt->gtt.pte_ops;
int page_entry_num = GTT_PAGE_SIZE >>
vgpu->gvt->device_info.gtt_entry_size_shift;
- struct page *scratch_pt;
+ void *scratch_pt;
unsigned long mfn;
int i;
- void *p;
if (WARN_ON(type < GTT_TYPE_PPGTT_PTE_PT || type >= GTT_TYPE_MAX))
return -EINVAL;
- scratch_pt = alloc_page(GFP_KERNEL | GFP_ATOMIC | __GFP_ZERO);
+ scratch_pt = (void *)get_zeroed_page(GFP_KERNEL);
if (!scratch_pt) {
gvt_err("fail to allocate scratch page\n");
return -ENOMEM;
}
- p = kmap_atomic(scratch_pt);
- mfn = intel_gvt_hypervisor_virt_to_mfn(p);
+ mfn = intel_gvt_hypervisor_virt_to_mfn(scratch_pt);
if (mfn == INTEL_GVT_INVALID_ADDR) {
- gvt_err("fail to translate vaddr:0x%llx\n", (u64)p);
- kunmap_atomic(p);
- __free_page(scratch_pt);
+ gvt_err("fail to translate vaddr:0x%lx\n", (unsigned long)scratch_pt);
+ free_page((unsigned long)scratch_pt);
return -EFAULT;
}
gtt->scratch_pt[type].page_mfn = mfn;
- gtt->scratch_pt[type].page = scratch_pt;
+ gtt->scratch_pt[type].page = virt_to_page(scratch_pt);
gvt_dbg_mm("vgpu%d create scratch_pt: type %d mfn=0x%lx\n",
vgpu->id, type, mfn);
* scratch_pt[type] indicate the scratch pt/scratch page used by the
* 'type' pt.
* e.g. scratch_pt[GTT_TYPE_PPGTT_PDE_PT] is used by
- * GTT_TYPE_PPGTT_PDE_PT level pt, that means this scatch_pt it self
+ * GTT_TYPE_PPGTT_PDE_PT level pt, that means this scratch_pt it self
* is GTT_TYPE_PPGTT_PTE_PT, and full filled by scratch page mfn.
*/
if (type > GTT_TYPE_PPGTT_PTE_PT && type < GTT_TYPE_MAX) {
se.val64 |= PPAT_CACHED_INDEX;
for (i = 0; i < page_entry_num; i++)
- ops->set_entry(p, &se, i, false, 0, vgpu);
+ ops->set_entry(scratch_pt, &se, i, false, 0, vgpu);
}
- kunmap_atomic(p);
-
return 0;
}
int intel_gvt_init_gtt(struct intel_gvt *gvt)
{
int ret;
- void *page_addr;
+ void *page;
gvt_dbg_core("init gtt\n");
return -ENODEV;
}
- gvt->gtt.scratch_ggtt_page =
- alloc_page(GFP_KERNEL | GFP_ATOMIC | __GFP_ZERO);
- if (!gvt->gtt.scratch_ggtt_page) {
+ page = (void *)get_zeroed_page(GFP_KERNEL);
+ if (!page) {
gvt_err("fail to allocate scratch ggtt page\n");
return -ENOMEM;
}
+ gvt->gtt.scratch_ggtt_page = virt_to_page(page);
- page_addr = page_address(gvt->gtt.scratch_ggtt_page);
-
- gvt->gtt.scratch_ggtt_mfn =
- intel_gvt_hypervisor_virt_to_mfn(page_addr);
+ gvt->gtt.scratch_ggtt_mfn = intel_gvt_hypervisor_virt_to_mfn(page);
if (gvt->gtt.scratch_ggtt_mfn == INTEL_GVT_INVALID_ADDR) {
gvt_err("fail to translate scratch ggtt page\n");
__free_page(gvt->gtt.scratch_ggtt_page);
for (offset = 0; offset < num_entries; offset++)
ops->set_entry(NULL, &e, index + offset, false, 0, vgpu);
}
+
+/**
+ * intel_vgpu_reset_gtt - reset the all GTT related status
+ * @vgpu: a vGPU
+ * @dmlr: true for vGPU Device Model Level Reset, false for GT Reset
+ *
+ * This function is called from vfio core to reset reset all
+ * GTT related status, including GGTT, PPGTT, scratch page.
+ *
+ */
+void intel_vgpu_reset_gtt(struct intel_vgpu *vgpu, bool dmlr)
+{
+ int i;
+
+ ppgtt_free_all_shadow_page(vgpu);
+ if (!dmlr)
+ return;
+
+ intel_vgpu_reset_ggtt(vgpu);
+
+ /* clear scratch page for security */
+ for (i = GTT_TYPE_PPGTT_PTE_PT; i < GTT_TYPE_MAX; i++) {
+ if (vgpu->gtt.scratch_pt[i].page != NULL)
+ memset(page_address(vgpu->gtt.scratch_pt[i].page),
+ 0, PAGE_SIZE);
+ }
+}
void intel_vgpu_reset_ggtt(struct intel_vgpu *vgpu);
extern int intel_gvt_init_gtt(struct intel_gvt *gvt);
+extern void intel_vgpu_reset_gtt(struct intel_vgpu *vgpu, bool dmlr);
extern void intel_gvt_clean_gtt(struct intel_gvt *gvt);
extern struct intel_vgpu_mm *intel_gvt_find_ppgtt_mm(struct intel_vgpu *vgpu,
intel_gvt_hypervisor_host_exit(&dev_priv->drm.pdev->dev, gvt);
intel_gvt_clean_vgpu_types(gvt);
+ idr_destroy(&gvt->vgpu_idr);
+
kfree(dev_priv->gvt);
dev_priv->gvt = NULL;
}
gvt_dbg_core("init gvt device\n");
+ idr_init(&gvt->vgpu_idr);
+
mutex_init(&gvt->lock);
gvt->dev_priv = dev_priv;
ret = intel_gvt_setup_mmio_info(gvt);
if (ret)
- return ret;
+ goto out_clean_idr;
ret = intel_gvt_load_firmware(gvt);
if (ret)
intel_gvt_free_firmware(gvt);
out_clean_mmio_info:
intel_gvt_clean_mmio_info(gvt);
+out_clean_idr:
+ idr_destroy(&gvt->vgpu_idr);
kfree(gvt);
return ret;
}
int intel_vgpu_alloc_resource(struct intel_vgpu *vgpu,
struct intel_vgpu_creation_params *param);
+void intel_vgpu_reset_resource(struct intel_vgpu *vgpu);
void intel_vgpu_free_resource(struct intel_vgpu *vgpu);
void intel_vgpu_write_fence(struct intel_vgpu *vgpu,
u32 fence, u64 value);
struct intel_vgpu *intel_gvt_create_vgpu(struct intel_gvt *gvt,
struct intel_vgpu_type *type);
void intel_gvt_destroy_vgpu(struct intel_vgpu *vgpu);
+void intel_gvt_reset_vgpu_locked(struct intel_vgpu *vgpu, bool dmlr,
+ unsigned int engine_mask);
void intel_gvt_reset_vgpu(struct intel_vgpu *vgpu);
int intel_gvt_ggtt_h2g_index(struct intel_vgpu *vgpu, unsigned long h_index,
unsigned long *g_index);
+void intel_vgpu_init_cfg_space(struct intel_vgpu *vgpu,
+ bool primary);
+void intel_vgpu_reset_cfg_space(struct intel_vgpu *vgpu);
+
int intel_vgpu_emulate_cfg_read(struct intel_vgpu *vgpu, unsigned int offset,
void *p_data, unsigned int bytes);
int intel_vgpu_init_opregion(struct intel_vgpu *vgpu, u32 gpa);
int intel_vgpu_emulate_opregion_request(struct intel_vgpu *vgpu, u32 swsci);
-int setup_vgpu_mmio(struct intel_vgpu *vgpu);
void populate_pvinfo_page(struct intel_vgpu *vgpu);
struct intel_gvt_ops {
static int new_mmio_info(struct intel_gvt *gvt,
u32 offset, u32 flags, u32 size,
u32 addr_mask, u32 ro_mask, u32 device,
- void *read, void *write)
+ int (*read)(struct intel_vgpu *, unsigned int, void *, unsigned int),
+ int (*write)(struct intel_vgpu *, unsigned int, void *, unsigned int))
{
struct intel_gvt_mmio_info *info, *p;
u32 start, end, i;
default:
/*should not hit here*/
gvt_err("invalid forcewake offset 0x%x\n", offset);
- return 1;
+ return -EINVAL;
}
} else {
ack_reg_offset = FORCEWAKE_ACK_HSW_REG;
return 0;
}
-static int handle_device_reset(struct intel_vgpu *vgpu, unsigned int offset,
- void *p_data, unsigned int bytes, unsigned long bitmap)
-{
- struct intel_gvt_workload_scheduler *scheduler =
- &vgpu->gvt->scheduler;
-
- vgpu->resetting = true;
-
- intel_vgpu_stop_schedule(vgpu);
- /*
- * The current_vgpu will set to NULL after stopping the
- * scheduler when the reset is triggered by current vgpu.
- */
- if (scheduler->current_vgpu == NULL) {
- mutex_unlock(&vgpu->gvt->lock);
- intel_gvt_wait_vgpu_idle(vgpu);
- mutex_lock(&vgpu->gvt->lock);
- }
-
- intel_vgpu_reset_execlist(vgpu, bitmap);
-
- /* full GPU reset */
- if (bitmap == 0xff) {
- mutex_unlock(&vgpu->gvt->lock);
- intel_vgpu_clean_gtt(vgpu);
- mutex_lock(&vgpu->gvt->lock);
- setup_vgpu_mmio(vgpu);
- populate_pvinfo_page(vgpu);
- intel_vgpu_init_gtt(vgpu);
- }
-
- vgpu->resetting = false;
-
- return 0;
-}
-
static int gdrst_mmio_write(struct intel_vgpu *vgpu, unsigned int offset,
- void *p_data, unsigned int bytes)
+ void *p_data, unsigned int bytes)
{
+ unsigned int engine_mask = 0;
u32 data;
- u64 bitmap = 0;
write_vreg(vgpu, offset, p_data, bytes);
data = vgpu_vreg(vgpu, offset);
if (data & GEN6_GRDOM_FULL) {
gvt_dbg_mmio("vgpu%d: request full GPU reset\n", vgpu->id);
- bitmap = 0xff;
- }
- if (data & GEN6_GRDOM_RENDER) {
- gvt_dbg_mmio("vgpu%d: request RCS reset\n", vgpu->id);
- bitmap |= (1 << RCS);
- }
- if (data & GEN6_GRDOM_MEDIA) {
- gvt_dbg_mmio("vgpu%d: request VCS reset\n", vgpu->id);
- bitmap |= (1 << VCS);
- }
- if (data & GEN6_GRDOM_BLT) {
- gvt_dbg_mmio("vgpu%d: request BCS Reset\n", vgpu->id);
- bitmap |= (1 << BCS);
- }
- if (data & GEN6_GRDOM_VECS) {
- gvt_dbg_mmio("vgpu%d: request VECS Reset\n", vgpu->id);
- bitmap |= (1 << VECS);
- }
- if (data & GEN8_GRDOM_MEDIA2) {
- gvt_dbg_mmio("vgpu%d: request VCS2 Reset\n", vgpu->id);
- if (HAS_BSD2(vgpu->gvt->dev_priv))
- bitmap |= (1 << VCS2);
+ engine_mask = ALL_ENGINES;
+ } else {
+ if (data & GEN6_GRDOM_RENDER) {
+ gvt_dbg_mmio("vgpu%d: request RCS reset\n", vgpu->id);
+ engine_mask |= (1 << RCS);
+ }
+ if (data & GEN6_GRDOM_MEDIA) {
+ gvt_dbg_mmio("vgpu%d: request VCS reset\n", vgpu->id);
+ engine_mask |= (1 << VCS);
+ }
+ if (data & GEN6_GRDOM_BLT) {
+ gvt_dbg_mmio("vgpu%d: request BCS Reset\n", vgpu->id);
+ engine_mask |= (1 << BCS);
+ }
+ if (data & GEN6_GRDOM_VECS) {
+ gvt_dbg_mmio("vgpu%d: request VECS Reset\n", vgpu->id);
+ engine_mask |= (1 << VECS);
+ }
+ if (data & GEN8_GRDOM_MEDIA2) {
+ gvt_dbg_mmio("vgpu%d: request VCS2 Reset\n", vgpu->id);
+ if (HAS_BSD2(vgpu->gvt->dev_priv))
+ engine_mask |= (1 << VCS2);
+ }
}
- return handle_device_reset(vgpu, offset, p_data, bytes, bitmap);
+
+ intel_gvt_reset_vgpu_locked(vgpu, false, engine_mask);
+
+ return 0;
}
static int gmbus_mmio_read(struct intel_vgpu *vgpu, unsigned int offset,
return 0;
}
-static bool sbi_ctl_mmio_write(struct intel_vgpu *vgpu, unsigned int offset,
+static int sbi_ctl_mmio_write(struct intel_vgpu *vgpu, unsigned int offset,
void *p_data, unsigned int bytes)
{
u32 data;
static int gvt_reg_tlb_control_handler(struct intel_vgpu *vgpu,
unsigned int offset, void *p_data, unsigned int bytes)
{
- int rc = 0;
unsigned int id = 0;
write_vreg(vgpu, offset, p_data, bytes);
id = VECS;
break;
default:
- rc = -EINVAL;
- break;
+ return -EINVAL;
}
set_bit(id, (void *)vgpu->tlb_handle_pending);
- return rc;
+ return 0;
}
static int ring_reset_ctl_write(struct intel_vgpu *vgpu,
struct intel_vgpu_type *type;
struct device *pdev;
void *gvt;
+ int ret;
pdev = mdev_parent_dev(mdev);
gvt = kdev_to_i915(pdev)->gvt;
if (!type) {
gvt_err("failed to find type %s to create\n",
kobject_name(kobj));
- return -EINVAL;
+ ret = -EINVAL;
+ goto out;
}
vgpu = intel_gvt_ops->vgpu_create(gvt, type);
if (IS_ERR_OR_NULL(vgpu)) {
- gvt_err("create intel vgpu failed\n");
- return -EINVAL;
+ ret = vgpu == NULL ? -EFAULT : PTR_ERR(vgpu);
+ gvt_err("failed to create intel vgpu: %d\n", ret);
+ goto out;
}
INIT_WORK(&vgpu->vdev.release_work, intel_vgpu_release_work);
gvt_dbg_core("intel_vgpu_create succeeded for mdev: %s\n",
dev_name(mdev_dev(mdev)));
- return 0;
+ ret = 0;
+
+out:
+ return ret;
}
static int intel_vgpu_remove(struct mdev_device *mdev)
if (WARN_ON(!reg_is_mmio(gvt, offset + bytes - 1)))
goto err;
- mmio = intel_gvt_find_mmio_info(gvt, rounddown(offset, 4));
- if (!mmio && !vgpu->mmio.disable_warn_untrack) {
- gvt_err("vgpu%d: read untracked MMIO %x len %d val %x\n",
- vgpu->id, offset, bytes, *(u32 *)p_data);
-
- if (offset == 0x206c) {
- gvt_err("------------------------------------------\n");
- gvt_err("vgpu%d: likely triggers a gfx reset\n",
- vgpu->id);
- gvt_err("------------------------------------------\n");
- vgpu->mmio.disable_warn_untrack = true;
- }
- }
-
if (!intel_gvt_mmio_is_unalign(gvt, offset)) {
if (WARN_ON(!IS_ALIGNED(offset, bytes)))
goto err;
}
+ mmio = intel_gvt_find_mmio_info(gvt, rounddown(offset, 4));
if (mmio) {
if (!intel_gvt_mmio_is_unalign(gvt, mmio->offset)) {
if (WARN_ON(offset + bytes > mmio->offset + mmio->size))
goto err;
}
ret = mmio->read(vgpu, offset, p_data, bytes);
- } else
+ } else {
ret = intel_vgpu_default_mmio_read(vgpu, offset, p_data, bytes);
+ if (!vgpu->mmio.disable_warn_untrack) {
+ gvt_err("vgpu%d: read untracked MMIO %x(%dB) val %x\n",
+ vgpu->id, offset, bytes, *(u32 *)p_data);
+
+ if (offset == 0x206c) {
+ gvt_err("------------------------------------------\n");
+ gvt_err("vgpu%d: likely triggers a gfx reset\n",
+ vgpu->id);
+ gvt_err("------------------------------------------\n");
+ vgpu->mmio.disable_warn_untrack = true;
+ }
+ }
+ }
+
if (ret)
goto err;
mutex_unlock(&gvt->lock);
return ret;
}
+
+
+/**
+ * intel_vgpu_reset_mmio - reset virtual MMIO space
+ * @vgpu: a vGPU
+ *
+ */
+void intel_vgpu_reset_mmio(struct intel_vgpu *vgpu)
+{
+ struct intel_gvt *gvt = vgpu->gvt;
+ const struct intel_gvt_device_info *info = &gvt->device_info;
+
+ memcpy(vgpu->mmio.vreg, gvt->firmware.mmio, info->mmio_size);
+ memcpy(vgpu->mmio.sreg, gvt->firmware.mmio, info->mmio_size);
+
+ vgpu_vreg(vgpu, GEN6_GT_THREAD_STATUS_REG) = 0;
+
+ /* set the bit 0:2(Core C-State ) to C0 */
+ vgpu_vreg(vgpu, GEN6_GT_CORE_STATUS) = 0;
+}
+
+/**
+ * intel_vgpu_init_mmio - init MMIO space
+ * @vgpu: a vGPU
+ *
+ * Returns:
+ * Zero on success, negative error code if failed
+ */
+int intel_vgpu_init_mmio(struct intel_vgpu *vgpu)
+{
+ const struct intel_gvt_device_info *info = &vgpu->gvt->device_info;
+
+ vgpu->mmio.vreg = vzalloc(info->mmio_size * 2);
+ if (!vgpu->mmio.vreg)
+ return -ENOMEM;
+
+ vgpu->mmio.sreg = vgpu->mmio.vreg + info->mmio_size;
+
+ intel_vgpu_reset_mmio(vgpu);
+
+ return 0;
+}
+
+/**
+ * intel_vgpu_clean_mmio - clean MMIO space
+ * @vgpu: a vGPU
+ *
+ */
+void intel_vgpu_clean_mmio(struct intel_vgpu *vgpu)
+{
+ vfree(vgpu->mmio.vreg);
+ vgpu->mmio.vreg = vgpu->mmio.sreg = NULL;
+}
*offset; \
})
+int intel_vgpu_init_mmio(struct intel_vgpu *vgpu);
+void intel_vgpu_reset_mmio(struct intel_vgpu *vgpu);
+void intel_vgpu_clean_mmio(struct intel_vgpu *vgpu);
+
int intel_vgpu_gpa_to_mmio_offset(struct intel_vgpu *vgpu, u64 gpa);
int intel_vgpu_emulate_mmio_read(struct intel_vgpu *vgpu, u64 pa,
vgpu->id))
return -EINVAL;
- vgpu_opregion(vgpu)->va = (void *)__get_free_pages(GFP_ATOMIC |
- GFP_DMA32 | __GFP_ZERO,
- INTEL_GVT_OPREGION_PORDER);
+ vgpu_opregion(vgpu)->va = (void *)__get_free_pages(GFP_KERNEL |
+ __GFP_ZERO,
+ get_order(INTEL_GVT_OPREGION_SIZE));
if (!vgpu_opregion(vgpu)->va)
return -ENOMEM;
if (intel_gvt_host.hypervisor_type == INTEL_GVT_HYPERVISOR_XEN) {
map_vgpu_opregion(vgpu, false);
free_pages((unsigned long)vgpu_opregion(vgpu)->va,
- INTEL_GVT_OPREGION_PORDER);
+ get_order(INTEL_GVT_OPREGION_SIZE));
vgpu_opregion(vgpu)->va = NULL;
}
#define INTEL_GVT_OPREGION_PARM 0x204
#define INTEL_GVT_OPREGION_PAGES 2
-#define INTEL_GVT_OPREGION_PORDER 1
-#define INTEL_GVT_OPREGION_SIZE (2 * 4096)
+#define INTEL_GVT_OPREGION_SIZE (INTEL_GVT_OPREGION_PAGES * PAGE_SIZE)
#define VGT_SPRSTRIDE(pipe) _PIPE(pipe, _SPRA_STRIDE, _PLANE_STRIDE_2_B)
{
struct intel_gvt_workload_scheduler *scheduler = &gvt->scheduler;
struct intel_vgpu_workload *workload;
+ struct intel_vgpu *vgpu;
int event;
mutex_lock(&gvt->lock);
workload = scheduler->current_workload[ring_id];
+ vgpu = workload->vgpu;
- if (!workload->status && !workload->vgpu->resetting) {
+ if (!workload->status && !vgpu->resetting) {
wait_event(workload->shadow_ctx_status_wq,
!atomic_read(&workload->shadow_ctx_active));
for_each_set_bit(event, workload->pending_events,
INTEL_GVT_EVENT_MAX)
- intel_vgpu_trigger_virtual_event(workload->vgpu,
- event);
+ intel_vgpu_trigger_virtual_event(vgpu, event);
}
gvt_dbg_sched("ring id %d complete workload %p status %d\n",
scheduler->current_workload[ring_id] = NULL;
- atomic_dec(&workload->vgpu->running_workload_num);
-
list_del_init(&workload->list);
workload->complete(workload);
+ atomic_dec(&vgpu->running_workload_num);
wake_up(&scheduler->workload_complete_wq);
mutex_unlock(&gvt->lock);
}
gvt_dbg_sched("will complete workload %p\n, status: %d\n",
workload, workload->status);
- complete_current_workload(gvt, ring_id);
-
if (workload->req)
i915_gem_request_put(fetch_and_zero(&workload->req));
+ complete_current_workload(gvt, ring_id);
+
if (need_force_wake)
intel_uncore_forcewake_put(gvt->dev_priv,
FORCEWAKE_ALL);
#include "gvt.h"
#include "i915_pvinfo.h"
-static void clean_vgpu_mmio(struct intel_vgpu *vgpu)
-{
- vfree(vgpu->mmio.vreg);
- vgpu->mmio.vreg = vgpu->mmio.sreg = NULL;
-}
-
-int setup_vgpu_mmio(struct intel_vgpu *vgpu)
-{
- struct intel_gvt *gvt = vgpu->gvt;
- const struct intel_gvt_device_info *info = &gvt->device_info;
-
- if (vgpu->mmio.vreg)
- memset(vgpu->mmio.vreg, 0, info->mmio_size * 2);
- else {
- vgpu->mmio.vreg = vzalloc(info->mmio_size * 2);
- if (!vgpu->mmio.vreg)
- return -ENOMEM;
- }
-
- vgpu->mmio.sreg = vgpu->mmio.vreg + info->mmio_size;
-
- memcpy(vgpu->mmio.vreg, gvt->firmware.mmio, info->mmio_size);
- memcpy(vgpu->mmio.sreg, gvt->firmware.mmio, info->mmio_size);
-
- vgpu_vreg(vgpu, GEN6_GT_THREAD_STATUS_REG) = 0;
-
- /* set the bit 0:2(Core C-State ) to C0 */
- vgpu_vreg(vgpu, GEN6_GT_CORE_STATUS) = 0;
- return 0;
-}
-
-static void setup_vgpu_cfg_space(struct intel_vgpu *vgpu,
- struct intel_vgpu_creation_params *param)
-{
- struct intel_gvt *gvt = vgpu->gvt;
- const struct intel_gvt_device_info *info = &gvt->device_info;
- u16 *gmch_ctl;
- int i;
-
- memcpy(vgpu_cfg_space(vgpu), gvt->firmware.cfg_space,
- info->cfg_space_size);
-
- if (!param->primary) {
- vgpu_cfg_space(vgpu)[PCI_CLASS_DEVICE] =
- INTEL_GVT_PCI_CLASS_VGA_OTHER;
- vgpu_cfg_space(vgpu)[PCI_CLASS_PROG] =
- INTEL_GVT_PCI_CLASS_VGA_OTHER;
- }
-
- /* Show guest that there isn't any stolen memory.*/
- gmch_ctl = (u16 *)(vgpu_cfg_space(vgpu) + INTEL_GVT_PCI_GMCH_CONTROL);
- *gmch_ctl &= ~(BDW_GMCH_GMS_MASK << BDW_GMCH_GMS_SHIFT);
-
- intel_vgpu_write_pci_bar(vgpu, PCI_BASE_ADDRESS_2,
- gvt_aperture_pa_base(gvt), true);
-
- vgpu_cfg_space(vgpu)[PCI_COMMAND] &= ~(PCI_COMMAND_IO
- | PCI_COMMAND_MEMORY
- | PCI_COMMAND_MASTER);
- /*
- * Clear the bar upper 32bit and let guest to assign the new value
- */
- memset(vgpu_cfg_space(vgpu) + PCI_BASE_ADDRESS_1, 0, 4);
- memset(vgpu_cfg_space(vgpu) + PCI_BASE_ADDRESS_3, 0, 4);
- memset(vgpu_cfg_space(vgpu) + INTEL_GVT_PCI_OPREGION, 0, 4);
-
- for (i = 0; i < INTEL_GVT_MAX_BAR_NUM; i++) {
- vgpu->cfg_space.bar[i].size = pci_resource_len(
- gvt->dev_priv->drm.pdev, i * 2);
- vgpu->cfg_space.bar[i].tracked = false;
- }
-}
-
void populate_pvinfo_page(struct intel_vgpu *vgpu)
{
/* setup the ballooning information */
if (low_avail / min_low == 0)
break;
gvt->types[i].low_gm_size = min_low;
- gvt->types[i].high_gm_size = 3 * gvt->types[i].low_gm_size;
+ gvt->types[i].high_gm_size = max((min_low<<3), MB_TO_BYTES(384U));
gvt->types[i].fence = 4;
gvt->types[i].max_instance = low_avail / min_low;
gvt->types[i].avail_instance = gvt->types[i].max_instance;
*/
low_gm_avail = MB_TO_BYTES(256) - HOST_LOW_GM_SIZE -
gvt->gm.vgpu_allocated_low_gm_size;
- high_gm_avail = MB_TO_BYTES(256) * 3 - HOST_HIGH_GM_SIZE -
+ high_gm_avail = MB_TO_BYTES(256) * 8UL - HOST_HIGH_GM_SIZE -
gvt->gm.vgpu_allocated_high_gm_size;
fence_avail = gvt_fence_sz(gvt) - HOST_FENCE -
gvt->fence.vgpu_allocated_fence_num;
intel_vgpu_clean_gtt(vgpu);
intel_gvt_hypervisor_detach_vgpu(vgpu);
intel_vgpu_free_resource(vgpu);
- clean_vgpu_mmio(vgpu);
+ intel_vgpu_clean_mmio(vgpu);
vfree(vgpu);
intel_gvt_update_vgpu_types(gvt);
vgpu->gvt = gvt;
bitmap_zero(vgpu->tlb_handle_pending, I915_NUM_ENGINES);
- setup_vgpu_cfg_space(vgpu, param);
+ intel_vgpu_init_cfg_space(vgpu, param->primary);
- ret = setup_vgpu_mmio(vgpu);
+ ret = intel_vgpu_init_mmio(vgpu);
if (ret)
- goto out_free_vgpu;
+ goto out_clean_idr;
ret = intel_vgpu_alloc_resource(vgpu, param);
if (ret)
out_clean_vgpu_resource:
intel_vgpu_free_resource(vgpu);
out_clean_vgpu_mmio:
- clean_vgpu_mmio(vgpu);
+ intel_vgpu_clean_mmio(vgpu);
+out_clean_idr:
+ idr_remove(&gvt->vgpu_idr, vgpu->id);
out_free_vgpu:
vfree(vgpu);
mutex_unlock(&gvt->lock);
}
/**
- * intel_gvt_reset_vgpu - reset a virtual GPU
+ * intel_gvt_reset_vgpu_locked - reset a virtual GPU by DMLR or GT reset
+ * @vgpu: virtual GPU
+ * @dmlr: vGPU Device Model Level Reset or GT Reset
+ * @engine_mask: engines to reset for GT reset
+ *
+ * This function is called when user wants to reset a virtual GPU through
+ * device model reset or GT reset. The caller should hold the gvt lock.
+ *
+ * vGPU Device Model Level Reset (DMLR) simulates the PCI level reset to reset
+ * the whole vGPU to default state as when it is created. This vGPU function
+ * is required both for functionary and security concerns.The ultimate goal
+ * of vGPU FLR is that reuse a vGPU instance by virtual machines. When we
+ * assign a vGPU to a virtual machine we must isse such reset first.
+ *
+ * Full GT Reset and Per-Engine GT Reset are soft reset flow for GPU engines
+ * (Render, Blitter, Video, Video Enhancement). It is defined by GPU Spec.
+ * Unlike the FLR, GT reset only reset particular resource of a vGPU per
+ * the reset request. Guest driver can issue a GT reset by programming the
+ * virtual GDRST register to reset specific virtual GPU engine or all
+ * engines.
+ *
+ * The parameter dev_level is to identify if we will do DMLR or GT reset.
+ * The parameter engine_mask is to specific the engines that need to be
+ * resetted. If value ALL_ENGINES is given for engine_mask, it means
+ * the caller requests a full GT reset that we will reset all virtual
+ * GPU engines. For FLR, engine_mask is ignored.
+ */
+void intel_gvt_reset_vgpu_locked(struct intel_vgpu *vgpu, bool dmlr,
+ unsigned int engine_mask)
+{
+ struct intel_gvt *gvt = vgpu->gvt;
+ struct intel_gvt_workload_scheduler *scheduler = &gvt->scheduler;
+
+ gvt_dbg_core("------------------------------------------\n");
+ gvt_dbg_core("resseting vgpu%d, dmlr %d, engine_mask %08x\n",
+ vgpu->id, dmlr, engine_mask);
+ vgpu->resetting = true;
+
+ intel_vgpu_stop_schedule(vgpu);
+ /*
+ * The current_vgpu will set to NULL after stopping the
+ * scheduler when the reset is triggered by current vgpu.
+ */
+ if (scheduler->current_vgpu == NULL) {
+ mutex_unlock(&gvt->lock);
+ intel_gvt_wait_vgpu_idle(vgpu);
+ mutex_lock(&gvt->lock);
+ }
+
+ intel_vgpu_reset_execlist(vgpu, dmlr ? ALL_ENGINES : engine_mask);
+
+ /* full GPU reset or device model level reset */
+ if (engine_mask == ALL_ENGINES || dmlr) {
+ intel_vgpu_reset_gtt(vgpu, dmlr);
+ intel_vgpu_reset_resource(vgpu);
+ intel_vgpu_reset_mmio(vgpu);
+ populate_pvinfo_page(vgpu);
+
+ if (dmlr)
+ intel_vgpu_reset_cfg_space(vgpu);
+ }
+
+ vgpu->resetting = false;
+ gvt_dbg_core("reset vgpu%d done\n", vgpu->id);
+ gvt_dbg_core("------------------------------------------\n");
+}
+
+/**
+ * intel_gvt_reset_vgpu - reset a virtual GPU (Function Level)
* @vgpu: virtual GPU
*
* This function is called when user wants to reset a virtual GPU.
*/
void intel_gvt_reset_vgpu(struct intel_vgpu *vgpu)
{
+ mutex_lock(&vgpu->gvt->lock);
+ intel_gvt_reset_vgpu_locked(vgpu, true, 0);
+ mutex_unlock(&vgpu->gvt->lock);
}
seq_printf(m, " (%sgtt offset: %08llx, size: %08llx",
i915_vma_is_ggtt(vma) ? "g" : "pp",
vma->node.start, vma->node.size);
- if (i915_vma_is_ggtt(vma))
- seq_printf(m, ", type: %u", vma->ggtt_view.type);
+ if (i915_vma_is_ggtt(vma)) {
+ switch (vma->ggtt_view.type) {
+ case I915_GGTT_VIEW_NORMAL:
+ seq_puts(m, ", normal");
+ break;
+
+ case I915_GGTT_VIEW_PARTIAL:
+ seq_printf(m, ", partial [%08llx+%x]",
+ vma->ggtt_view.partial.offset << PAGE_SHIFT,
+ vma->ggtt_view.partial.size << PAGE_SHIFT);
+ break;
+
+ case I915_GGTT_VIEW_ROTATED:
+ seq_printf(m, ", rotated [(%ux%u, stride=%u, offset=%u), (%ux%u, stride=%u, offset=%u)]",
+ vma->ggtt_view.rotated.plane[0].width,
+ vma->ggtt_view.rotated.plane[0].height,
+ vma->ggtt_view.rotated.plane[0].stride,
+ vma->ggtt_view.rotated.plane[0].offset,
+ vma->ggtt_view.rotated.plane[1].width,
+ vma->ggtt_view.rotated.plane[1].height,
+ vma->ggtt_view.rotated.plane[1].stride,
+ vma->ggtt_view.rotated.plane[1].offset);
+ break;
+
+ default:
+ MISSING_CASE(vma->ggtt_view.type);
+ break;
+ }
+ }
if (vma->fence)
seq_printf(m, " , fence: %d%s",
vma->fence->id,
return 0;
}
+static int i915_huc_load_status_info(struct seq_file *m, void *data)
+{
+ struct drm_i915_private *dev_priv = node_to_i915(m->private);
+ struct intel_uc_fw *huc_fw = &dev_priv->huc.fw;
+
+ if (!HAS_HUC_UCODE(dev_priv))
+ return 0;
+
+ seq_puts(m, "HuC firmware status:\n");
+ seq_printf(m, "\tpath: %s\n", huc_fw->path);
+ seq_printf(m, "\tfetch: %s\n",
+ intel_uc_fw_status_repr(huc_fw->fetch_status));
+ seq_printf(m, "\tload: %s\n",
+ intel_uc_fw_status_repr(huc_fw->load_status));
+ seq_printf(m, "\tversion wanted: %d.%d\n",
+ huc_fw->major_ver_wanted, huc_fw->minor_ver_wanted);
+ seq_printf(m, "\tversion found: %d.%d\n",
+ huc_fw->major_ver_found, huc_fw->minor_ver_found);
+ seq_printf(m, "\theader: offset is %d; size = %d\n",
+ huc_fw->header_offset, huc_fw->header_size);
+ seq_printf(m, "\tuCode: offset is %d; size = %d\n",
+ huc_fw->ucode_offset, huc_fw->ucode_size);
+ seq_printf(m, "\tRSA: offset is %d; size = %d\n",
+ huc_fw->rsa_offset, huc_fw->rsa_size);
+
+ seq_printf(m, "\nHuC status 0x%08x:\n", I915_READ(HUC_STATUS2));
+
+ return 0;
+}
+
static int i915_guc_load_status_info(struct seq_file *m, void *data)
{
struct drm_i915_private *dev_priv = node_to_i915(m->private);
- struct intel_guc_fw *guc_fw = &dev_priv->guc.guc_fw;
+ struct intel_uc_fw *guc_fw = &dev_priv->guc.fw;
u32 tmp, i;
if (!HAS_GUC_UCODE(dev_priv))
seq_printf(m, "GuC firmware status:\n");
seq_printf(m, "\tpath: %s\n",
- guc_fw->guc_fw_path);
+ guc_fw->path);
seq_printf(m, "\tfetch: %s\n",
- intel_guc_fw_status_repr(guc_fw->guc_fw_fetch_status));
+ intel_uc_fw_status_repr(guc_fw->fetch_status));
seq_printf(m, "\tload: %s\n",
- intel_guc_fw_status_repr(guc_fw->guc_fw_load_status));
+ intel_uc_fw_status_repr(guc_fw->load_status));
seq_printf(m, "\tversion wanted: %d.%d\n",
- guc_fw->guc_fw_major_wanted, guc_fw->guc_fw_minor_wanted);
+ guc_fw->major_ver_wanted, guc_fw->minor_ver_wanted);
seq_printf(m, "\tversion found: %d.%d\n",
- guc_fw->guc_fw_major_found, guc_fw->guc_fw_minor_found);
+ guc_fw->major_ver_found, guc_fw->minor_ver_found);
seq_printf(m, "\theader: offset is %d; size = %d\n",
guc_fw->header_offset, guc_fw->header_size);
seq_printf(m, "\tuCode: offset is %d; size = %d\n",
i915_guc_log_control_get, i915_guc_log_control_set,
"%lld\n");
+static const char *psr2_live_status(u32 val)
+{
+ static const char * const live_status[] = {
+ "IDLE",
+ "CAPTURE",
+ "CAPTURE_FS",
+ "SLEEP",
+ "BUFON_FW",
+ "ML_UP",
+ "SU_STANDBY",
+ "FAST_SLEEP",
+ "DEEP_SLEEP",
+ "BUF_ON",
+ "TG_ON"
+ };
+
+ val = (val & EDP_PSR2_STATUS_STATE_MASK) >> EDP_PSR2_STATUS_STATE_SHIFT;
+ if (val < ARRAY_SIZE(live_status))
+ return live_status[val];
+
+ return "unknown";
+}
+
static int i915_edp_psr_status(struct seq_file *m, void *data)
{
struct drm_i915_private *dev_priv = node_to_i915(m->private);
seq_printf(m, "Performance_Counter: %u\n", psrperf);
}
+ if (dev_priv->psr.psr2_support) {
+ u32 psr2 = I915_READ(EDP_PSR2_STATUS_CTL);
+
+ seq_printf(m, "EDP_PSR2_STATUS_CTL: %x [%s]\n",
+ psr2, psr2_live_status(psr2));
+ }
mutex_unlock(&dev_priv->psr.lock);
intel_runtime_pm_put(dev_priv);
{"i915_guc_info", i915_guc_info, 0},
{"i915_guc_load_status", i915_guc_load_status_info, 0},
{"i915_guc_log_dump", i915_guc_log_dump, 0},
+ {"i915_huc_load_status", i915_huc_load_status_info, 0},
{"i915_frequency_info", i915_frequency_info, 0},
{"i915_hangcheck_info", i915_hangcheck_info, 0},
{"i915_drpc_info", i915_drpc_info, 0},
#include "i915_trace.h"
#include "i915_vgpu.h"
#include "intel_drv.h"
+#include "intel_uc.h"
static struct drm_driver driver;
case I915_PARAM_MIN_EU_IN_POOL:
value = INTEL_INFO(dev_priv)->sseu.min_eu_in_pool;
break;
+ case I915_PARAM_HUC_STATUS:
+ /* The register is already force-woken. We dont need
+ * any rpm here
+ */
+ value = I915_READ(HUC_STATUS2) & HUC_FW_VERIFIED;
+ break;
case I915_PARAM_MMAP_GTT_VERSION:
/* Though we've started our numbering from 1, and so class all
* earlier versions as 0, in effect their value is undefined as
if (ret)
goto cleanup_irq;
+ intel_huc_init(dev_priv);
intel_guc_init(dev_priv);
ret = i915_gem_init(dev_priv);
i915_gem_fini(dev_priv);
cleanup_irq:
intel_guc_fini(dev_priv);
+ intel_huc_fini(dev_priv);
drm_irq_uninstall(dev);
intel_teardown_gmbus(dev_priv);
cleanup_csr:
/* Reveal our presence to userspace */
if (drm_dev_register(dev, 0) == 0) {
i915_debugfs_register(dev_priv);
- i915_guc_register(dev_priv);
+ i915_guc_log_register(dev_priv);
i915_setup_sysfs(dev_priv);
/* Depends on sysfs having been initialized */
i915_perf_unregister(dev_priv);
i915_teardown_sysfs(dev_priv);
- i915_guc_unregister(dev_priv);
+ i915_guc_log_unregister(dev_priv);
i915_debugfs_unregister(dev_priv);
drm_dev_unregister(&dev_priv->drm);
drain_workqueue(dev_priv->wq);
intel_guc_fini(dev_priv);
+ intel_huc_fini(dev_priv);
i915_gem_fini(dev_priv);
intel_fbc_cleanup_cfb(dev_priv);
intel_display_set_init_power(dev_priv, false);
- fw_csr = !IS_BROXTON(dev_priv) &&
+ fw_csr = !IS_GEN9_LP(dev_priv) &&
suspend_to_idle(dev_priv) && dev_priv->csr.dmc_payload;
/*
* In case of firmware assisted context save/restore don't manually
intel_power_domains_suspend(dev_priv);
ret = 0;
- if (IS_BROXTON(dev_priv))
+ if (IS_GEN9_LP(dev_priv))
bxt_enable_dc9(dev_priv);
else if (IS_HASWELL(dev_priv) || IS_BROADWELL(dev_priv))
hsw_enable_pc8(dev_priv);
intel_uncore_early_sanitize(dev_priv, true);
- if (IS_BROXTON(dev_priv)) {
+ if (IS_GEN9_LP(dev_priv)) {
if (!dev_priv->suspended_to_idle)
gen9_sanitize_dc_state(dev_priv);
bxt_disable_dc9(dev_priv);
intel_uncore_sanitize(dev_priv);
- if (IS_BROXTON(dev_priv) ||
+ if (IS_GEN9_LP(dev_priv) ||
!(dev_priv->suspended_to_idle && dev_priv->csr.dmc_payload))
intel_power_domains_init_hw(dev_priv, true);
return i915_drm_resume(dev);
}
-static void disable_engines_irq(struct drm_i915_private *dev_priv)
-{
- struct intel_engine_cs *engine;
- enum intel_engine_id id;
-
- /* Ensure irq handler finishes, and not run again. */
- disable_irq(dev_priv->drm.irq);
- for_each_engine(engine, dev_priv, id)
- tasklet_kill(&engine->irq_tasklet);
-}
-
-static void enable_engines_irq(struct drm_i915_private *dev_priv)
-{
- enable_irq(dev_priv->drm.irq);
-}
-
/**
* i915_reset - reset chip after a hang
- * @dev: drm device to reset
+ * @dev_priv: device private to reset
*
* Reset the chip. Useful if a hang is detected. Marks the device as wedged
* on failure.
error->reset_count++;
pr_notice("drm/i915: Resetting chip after gpu hang\n");
- i915_gem_reset_prepare(dev_priv);
+ disable_irq(dev_priv->drm.irq);
+ ret = i915_gem_reset_prepare(dev_priv);
+ if (ret) {
+ DRM_ERROR("GPU recovery failed\n");
+ intel_gpu_reset(dev_priv, ALL_ENGINES);
+ goto error;
+ }
- disable_engines_irq(dev_priv);
ret = intel_gpu_reset(dev_priv, ALL_ENGINES);
- enable_engines_irq(dev_priv);
-
if (ret) {
if (ret != -ENODEV)
DRM_ERROR("Failed to reset chip: %i\n", ret);
i915_queue_hangcheck(dev_priv);
wakeup:
+ enable_irq(dev_priv->drm.irq);
wake_up_bit(&error->flags, I915_RESET_IN_PROGRESS);
return;
intel_runtime_pm_disable_interrupts(dev_priv);
ret = 0;
- if (IS_BROXTON(dev_priv)) {
+ if (IS_GEN9_LP(dev_priv)) {
bxt_display_core_uninit(dev_priv);
bxt_enable_dc9(dev_priv);
} else if (IS_HASWELL(dev_priv) || IS_BROADWELL(dev_priv)) {
if (IS_GEN6(dev_priv))
intel_init_pch_refclk(dev_priv);
- if (IS_BROXTON(dev_priv)) {
+ if (IS_GEN9_LP(dev_priv)) {
bxt_disable_dc9(dev_priv);
bxt_display_core_init(dev_priv, true);
if (dev_priv->csr.dmc_payload &&
DRM_IOCTL_DEF_DRV(I915_GEM_MMAP_GTT, i915_gem_mmap_gtt_ioctl, DRM_RENDER_ALLOW),
DRM_IOCTL_DEF_DRV(I915_GEM_SET_DOMAIN, i915_gem_set_domain_ioctl, DRM_RENDER_ALLOW),
DRM_IOCTL_DEF_DRV(I915_GEM_SW_FINISH, i915_gem_sw_finish_ioctl, DRM_RENDER_ALLOW),
- DRM_IOCTL_DEF_DRV(I915_GEM_SET_TILING, i915_gem_set_tiling, DRM_RENDER_ALLOW),
- DRM_IOCTL_DEF_DRV(I915_GEM_GET_TILING, i915_gem_get_tiling, DRM_RENDER_ALLOW),
+ DRM_IOCTL_DEF_DRV(I915_GEM_SET_TILING, i915_gem_set_tiling_ioctl, DRM_RENDER_ALLOW),
+ DRM_IOCTL_DEF_DRV(I915_GEM_GET_TILING, i915_gem_get_tiling_ioctl, DRM_RENDER_ALLOW),
DRM_IOCTL_DEF_DRV(I915_GEM_GET_APERTURE, i915_gem_get_aperture_ioctl, DRM_RENDER_ALLOW),
DRM_IOCTL_DEF_DRV(I915_GET_PIPE_FROM_CRTC_ID, intel_get_pipe_from_crtc_id, 0),
DRM_IOCTL_DEF_DRV(I915_GEM_MADVISE, i915_gem_madvise_ioctl, DRM_RENDER_ALLOW),
#define DRIVER_NAME "i915"
#define DRIVER_DESC "Intel Graphics"
-#define DRIVER_DATE "20170109"
-#define DRIVER_TIMESTAMP 1483953121
+#define DRIVER_DATE "20170123"
+#define DRIVER_TIMESTAMP 1485156432
#undef WARN_ON
/* Many gcc seem to no see through this and fall over :( */
struct work_struct underrun_work;
struct intel_fbc_state_cache {
+ struct i915_vma *vma;
+
struct {
unsigned int mode_flags;
uint32_t hsw_bdw_pixel_rate;
} plane;
struct {
- u64 ilk_ggtt_offset;
const struct drm_format_info *format;
unsigned int stride;
- int fence_reg;
- unsigned int tiling_mode;
} fb;
} state_cache;
struct intel_fbc_reg_params {
+ struct i915_vma *vma;
+
struct {
enum pipe pipe;
enum plane plane;
} crtc;
struct {
- u64 ggtt_offset;
const struct drm_format_info *format;
unsigned int stride;
- int fence_reg;
} fb;
int cfb_size;
bool psr2_support;
bool aux_frame_sync;
bool link_standby;
+ bool y_cord_support;
+ bool colorimetry_support;
+ bool alpm;
};
enum intel_pch {
enum intel_pipe_crc_source source;
int head, tail;
wait_queue_head_t wq;
+ int skipped;
};
struct i915_frontbuffer_tracking {
struct intel_gvt *gvt;
+ struct intel_huc huc;
struct intel_guc guc;
struct intel_csr csr;
#define HAS_GUC(dev_priv) ((dev_priv)->info.has_guc)
#define HAS_GUC_UCODE(dev_priv) (HAS_GUC(dev_priv))
#define HAS_GUC_SCHED(dev_priv) (HAS_GUC(dev_priv))
+#define HAS_HUC_UCODE(dev_priv) (HAS_GUC(dev_priv))
#define HAS_RESOURCE_STREAMER(dev_priv) ((dev_priv)->info.has_resource_streamer)
struct drm_file *file_priv);
int i915_gem_madvise_ioctl(struct drm_device *dev, void *data,
struct drm_file *file_priv);
-int i915_gem_set_tiling(struct drm_device *dev, void *data,
- struct drm_file *file_priv);
-int i915_gem_get_tiling(struct drm_device *dev, void *data,
- struct drm_file *file_priv);
+int i915_gem_set_tiling_ioctl(struct drm_device *dev, void *data,
+ struct drm_file *file_priv);
+int i915_gem_get_tiling_ioctl(struct drm_device *dev, void *data,
+ struct drm_file *file_priv);
void i915_gem_init_userptr(struct drm_i915_private *dev_priv);
int i915_gem_userptr_ioctl(struct drm_device *dev, void *data,
struct drm_file *file);
return READ_ONCE(error->reset_count);
}
-void i915_gem_reset_prepare(struct drm_i915_private *dev_priv);
+int i915_gem_reset_prepare(struct drm_i915_private *dev_priv);
void i915_gem_reset_finish(struct drm_i915_private *dev_priv);
void i915_gem_set_wedged(struct drm_i915_private *dev_priv);
void i915_gem_clflush_object(struct drm_i915_gem_object *obj, bool force);
int i915_gem_open(struct drm_device *dev, struct drm_file *file);
void i915_gem_release(struct drm_device *dev, struct drm_file *file);
-u64 i915_gem_get_ggtt_size(struct drm_i915_private *dev_priv, u64 size,
- int tiling_mode);
-u64 i915_gem_get_ggtt_alignment(struct drm_i915_private *dev_priv, u64 size,
- int tiling_mode, bool fenced);
-
int i915_gem_object_set_cache_level(struct drm_i915_gem_object *obj,
enum i915_cache_level cache_level);
struct dma_buf *i915_gem_prime_export(struct drm_device *dev,
struct drm_gem_object *gem_obj, int flags);
-struct i915_vma *
-i915_gem_obj_to_vma(struct drm_i915_gem_object *obj,
- struct i915_address_space *vm,
- const struct i915_ggtt_view *view);
-
-struct i915_vma *
-i915_gem_obj_lookup_or_create_vma(struct drm_i915_gem_object *obj,
- struct i915_address_space *vm,
- const struct i915_ggtt_view *view);
-
static inline struct i915_hw_ppgtt *
i915_vm_to_ppgtt(struct i915_address_space *vm)
{
return container_of(vm, struct i915_hw_ppgtt, base);
}
-static inline struct i915_vma *
-i915_gem_object_to_ggtt(struct drm_i915_gem_object *obj,
- const struct i915_ggtt_view *view)
-{
- return i915_gem_obj_to_vma(obj, &to_i915(obj->base.dev)->ggtt.base, view);
-}
-
-static inline unsigned long
-i915_gem_object_ggtt_offset(struct drm_i915_gem_object *o,
- const struct i915_ggtt_view *view)
-{
- return i915_ggtt_offset(i915_gem_object_to_ggtt(o, view));
-}
-
/* i915_gem_fence_reg.c */
int __must_check i915_vma_get_fence(struct i915_vma *vma);
int __must_check i915_vma_put_fence(struct i915_vma *vma);
unsigned cache_level,
u64 start, u64 end,
unsigned flags);
-int __must_check i915_gem_evict_for_vma(struct i915_vma *vma,
- unsigned int flags);
+int __must_check i915_gem_evict_for_node(struct i915_address_space *vm,
+ struct drm_mm_node *node,
+ unsigned int flags);
int i915_gem_evict_vm(struct i915_address_space *vm, bool do_idle);
/* belongs in i915_gem_gtt.h */
/* i915_gem_internal.c */
struct drm_i915_gem_object *
i915_gem_object_create_internal(struct drm_i915_private *dev_priv,
- unsigned int size);
+ phys_addr_t size);
/* i915_gem_shrinker.c */
unsigned long i915_gem_shrink(struct drm_i915_private *dev_priv,
i915_gem_object_is_tiled(obj);
}
+u32 i915_gem_fence_size(struct drm_i915_private *dev_priv, u32 size,
+ unsigned int tiling, unsigned int stride);
+u32 i915_gem_fence_alignment(struct drm_i915_private *dev_priv, u32 size,
+ unsigned int tiling, unsigned int stride);
+
/* i915_debugfs.c */
#ifdef CONFIG_DEBUG_FS
int i915_debugfs_register(struct drm_i915_private *dev_priv);
static unsigned int tile_row_pages(struct drm_i915_gem_object *obj)
{
- u64 size;
-
- size = i915_gem_object_get_stride(obj);
- size *= i915_gem_object_get_tiling(obj) == I915_TILING_Y ? 32 : 8;
-
- return size >> PAGE_SHIFT;
+ return i915_gem_object_get_tile_row_size(obj) >> PAGE_SHIFT;
}
/**
return 1;
}
+static inline struct i915_ggtt_view
+compute_partial_view(struct drm_i915_gem_object *obj,
+ pgoff_t page_offset,
+ unsigned int chunk)
+{
+ struct i915_ggtt_view view;
+
+ if (i915_gem_object_is_tiled(obj))
+ chunk = roundup(chunk, tile_row_pages(obj));
+
+ view.type = I915_GGTT_VIEW_PARTIAL;
+ view.partial.offset = rounddown(page_offset, chunk);
+ view.partial.size =
+ min_t(unsigned int, chunk,
+ (obj->base.size >> PAGE_SHIFT) - view.partial.offset);
+
+ /* If the partial covers the entire object, just create a normal VMA. */
+ if (chunk >= obj->base.size >> PAGE_SHIFT)
+ view.type = I915_GGTT_VIEW_NORMAL;
+
+ return view;
+}
+
/**
* i915_gem_fault - fault a page into the GTT
* @area: CPU VMA in question
/* Now pin it into the GTT as needed */
vma = i915_gem_object_ggtt_pin(obj, NULL, 0, 0, flags);
if (IS_ERR(vma)) {
- struct i915_ggtt_view view;
- unsigned int chunk_size;
-
/* Use a partial view if it is bigger than available space */
- chunk_size = MIN_CHUNK_PAGES;
- if (i915_gem_object_is_tiled(obj))
- chunk_size = roundup(chunk_size, tile_row_pages(obj));
-
- memset(&view, 0, sizeof(view));
- view.type = I915_GGTT_VIEW_PARTIAL;
- view.params.partial.offset = rounddown(page_offset, chunk_size);
- view.params.partial.size =
- min_t(unsigned int, chunk_size,
- vma_pages(area) - view.params.partial.offset);
-
- /* If the partial covers the entire object, just create a
- * normal VMA.
- */
- if (chunk_size >= obj->base.size >> PAGE_SHIFT)
- view.type = I915_GGTT_VIEW_NORMAL;
+ struct i915_ggtt_view view =
+ compute_partial_view(obj, page_offset, MIN_CHUNK_PAGES);
/* Userspace is now writing through an untracked VMA, abandon
* all hope that the hardware is able to track future writes.
/* Finally, remap it using the new GTT offset */
ret = remap_io_mapping(area,
- area->vm_start + (vma->ggtt_view.params.partial.offset << PAGE_SHIFT),
+ area->vm_start + (vma->ggtt_view.partial.offset << PAGE_SHIFT),
(ggtt->mappable_base + vma->node.start) >> PAGE_SHIFT,
min_t(u64, vma->size, area->vm_end - area->vm_start),
&ggtt->mappable);
}
}
-/**
- * i915_gem_get_ggtt_size - return required global GTT size for an object
- * @dev_priv: i915 device
- * @size: object size
- * @tiling_mode: tiling mode
- *
- * Return the required global GTT size for an object, taking into account
- * potential fence register mapping.
- */
-u64 i915_gem_get_ggtt_size(struct drm_i915_private *dev_priv,
- u64 size, int tiling_mode)
-{
- u64 ggtt_size;
-
- GEM_BUG_ON(size == 0);
-
- if (INTEL_GEN(dev_priv) >= 4 ||
- tiling_mode == I915_TILING_NONE)
- return size;
-
- /* Previous chips need a power-of-two fence region when tiling */
- if (IS_GEN3(dev_priv))
- ggtt_size = 1024*1024;
- else
- ggtt_size = 512*1024;
-
- while (ggtt_size < size)
- ggtt_size <<= 1;
-
- return ggtt_size;
-}
-
-/**
- * i915_gem_get_ggtt_alignment - return required global GTT alignment
- * @dev_priv: i915 device
- * @size: object size
- * @tiling_mode: tiling mode
- * @fenced: is fenced alignment required or not
- *
- * Return the required global GTT alignment for an object, taking into account
- * potential fence register mapping.
- */
-u64 i915_gem_get_ggtt_alignment(struct drm_i915_private *dev_priv, u64 size,
- int tiling_mode, bool fenced)
-{
- GEM_BUG_ON(size == 0);
-
- /*
- * Minimum alignment is 4k (GTT page size), but might be greater
- * if a fence register is needed for the object.
- */
- if (INTEL_GEN(dev_priv) >= 4 ||
- (!fenced && (IS_G33(dev_priv) || IS_PINEVIEW(dev_priv))) ||
- tiling_mode == I915_TILING_NONE)
- return 4096;
-
- /*
- * Previous chips need to be aligned to the size of the smallest
- * fence register that can contain the object.
- */
- return i915_gem_get_ggtt_size(dev_priv, size, tiling_mode);
-}
-
static int i915_gem_object_create_mmap_offset(struct drm_i915_gem_object *obj)
{
struct drm_i915_private *dev_priv = to_i915(obj->base.dev);
if (__i915_gem_request_completed(request))
continue;
+ GEM_BUG_ON(request->engine != engine);
return request;
}
return NULL;
}
-static void reset_request(struct drm_i915_gem_request *request)
+static bool engine_stalled(struct intel_engine_cs *engine)
+{
+ if (!engine->hangcheck.stalled)
+ return false;
+
+ /* Check for possible seqno movement after hang declaration */
+ if (engine->hangcheck.seqno != intel_engine_get_seqno(engine)) {
+ DRM_DEBUG_DRIVER("%s pardoned\n", engine->name);
+ return false;
+ }
+
+ return true;
+}
+
+int i915_gem_reset_prepare(struct drm_i915_private *dev_priv)
+{
+ struct intel_engine_cs *engine;
+ enum intel_engine_id id;
+ int err = 0;
+
+ /* Ensure irq handler finishes, and not run again. */
+ for_each_engine(engine, dev_priv, id) {
+ struct drm_i915_gem_request *request;
+
+ tasklet_kill(&engine->irq_tasklet);
+
+ if (engine_stalled(engine)) {
+ request = i915_gem_find_active_request(engine);
+ if (request && request->fence.error == -EIO)
+ err = -EIO; /* Previous reset failed! */
+ }
+ }
+
+ i915_gem_revoke_fences(dev_priv);
+
+ return err;
+}
+
+static void skip_request(struct drm_i915_gem_request *request)
{
void *vaddr = request->ring->vaddr;
u32 head;
head = 0;
}
memset(vaddr + head, 0, request->postfix - head);
+
+ dma_fence_set_error(&request->fence, -EIO);
}
-void i915_gem_reset_prepare(struct drm_i915_private *dev_priv)
+static void engine_skip_context(struct drm_i915_gem_request *request)
{
- i915_gem_revoke_fences(dev_priv);
+ struct intel_engine_cs *engine = request->engine;
+ struct i915_gem_context *hung_ctx = request->ctx;
+ struct intel_timeline *timeline;
+ unsigned long flags;
+
+ timeline = i915_gem_context_lookup_timeline(hung_ctx, engine);
+
+ spin_lock_irqsave(&engine->timeline->lock, flags);
+ spin_lock(&timeline->lock);
+
+ list_for_each_entry_continue(request, &engine->timeline->requests, link)
+ if (request->ctx == hung_ctx)
+ skip_request(request);
+
+ list_for_each_entry(request, &timeline->requests, link)
+ skip_request(request);
+
+ spin_unlock(&timeline->lock);
+ spin_unlock_irqrestore(&engine->timeline->lock, flags);
+}
+
+/* Returns true if the request was guilty of hang */
+static bool i915_gem_reset_request(struct drm_i915_gem_request *request)
+{
+ /* Read once and return the resolution */
+ const bool guilty = engine_stalled(request->engine);
+
+ /* The guilty request will get skipped on a hung engine.
+ *
+ * Users of client default contexts do not rely on logical
+ * state preserved between batches so it is safe to execute
+ * queued requests following the hang. Non default contexts
+ * rely on preserved state, so skipping a batch loses the
+ * evolution of the state and it needs to be considered corrupted.
+ * Executing more queued batches on top of corrupted state is
+ * risky. But we take the risk by trying to advance through
+ * the queued requests in order to make the client behaviour
+ * more predictable around resets, by not throwing away random
+ * amount of batches it has prepared for execution. Sophisticated
+ * clients can use gem_reset_stats_ioctl and dma fence status
+ * (exported via sync_file info ioctl on explicit fences) to observe
+ * when it loses the context state and should rebuild accordingly.
+ *
+ * The context ban, and ultimately the client ban, mechanism are safety
+ * valves if client submission ends up resulting in nothing more than
+ * subsequent hangs.
+ */
+
+ if (guilty) {
+ i915_gem_context_mark_guilty(request->ctx);
+ skip_request(request);
+ } else {
+ i915_gem_context_mark_innocent(request->ctx);
+ dma_fence_set_error(&request->fence, -EAGAIN);
+ }
+
+ return guilty;
}
static void i915_gem_reset_engine(struct intel_engine_cs *engine)
{
struct drm_i915_gem_request *request;
- struct i915_gem_context *hung_ctx;
- struct intel_timeline *timeline;
- unsigned long flags;
- bool ring_hung;
if (engine->irq_seqno_barrier)
engine->irq_seqno_barrier(engine);
if (!request)
return;
- hung_ctx = request->ctx;
-
- ring_hung = engine->hangcheck.stalled;
- if (engine->hangcheck.seqno != intel_engine_get_seqno(engine)) {
- DRM_DEBUG_DRIVER("%s pardoned, was guilty? %s\n",
- engine->name,
- yesno(ring_hung));
- ring_hung = false;
- }
-
- if (ring_hung)
- i915_gem_context_mark_guilty(hung_ctx);
- else
- i915_gem_context_mark_innocent(hung_ctx);
-
- if (!ring_hung)
+ if (!i915_gem_reset_request(request))
return;
DRM_DEBUG_DRIVER("resetting %s to restart from tail of request 0x%x\n",
engine->reset_hw(engine, request);
/* If this context is now banned, skip all of its pending requests. */
- if (!i915_gem_context_is_banned(hung_ctx))
- return;
-
- /* Users of the default context do not rely on logical state
- * preserved between batches. They have to emit full state on
- * every batch and so it is safe to execute queued requests following
- * the hang.
- *
- * Other contexts preserve state, now corrupt. We want to skip all
- * queued requests that reference the corrupt context.
- */
- if (i915_gem_context_is_default(hung_ctx))
- return;
-
- timeline = i915_gem_context_lookup_timeline(hung_ctx, engine);
-
- spin_lock_irqsave(&engine->timeline->lock, flags);
- spin_lock(&timeline->lock);
-
- list_for_each_entry_continue(request, &engine->timeline->requests, link)
- if (request->ctx == hung_ctx)
- reset_request(request);
-
- list_for_each_entry(request, &timeline->requests, link)
- reset_request(request);
-
- spin_unlock(&timeline->lock);
- spin_unlock_irqrestore(&engine->timeline->lock, flags);
+ if (i915_gem_context_is_banned(request->ctx))
+ engine_skip_context(request);
}
void i915_gem_reset_finish(struct drm_i915_private *dev_priv)
static void nop_submit_request(struct drm_i915_gem_request *request)
{
+ dma_fence_set_error(&request->fence, -EIO);
i915_gem_request_submit(request);
intel_engine_init_global_seqno(request->engine, request->global_seqno);
}
-static void i915_gem_cleanup_engine(struct intel_engine_cs *engine)
+static void engine_set_wedged(struct intel_engine_cs *engine)
{
+ struct drm_i915_gem_request *request;
+ unsigned long flags;
+
/* We need to be sure that no thread is running the old callback as
* we install the nop handler (otherwise we would submit a request
* to hardware that will never complete). In order to prevent this
*/
engine->submit_request = nop_submit_request;
+ /* Mark all executing requests as skipped */
+ spin_lock_irqsave(&engine->timeline->lock, flags);
+ list_for_each_entry(request, &engine->timeline->requests, link)
+ dma_fence_set_error(&request->fence, -EIO);
+ spin_unlock_irqrestore(&engine->timeline->lock, flags);
+
/* Mark all pending requests as complete so that any concurrent
* (lockless) lookup doesn't try and wait upon the request as we
* reset it.
enum intel_engine_id id;
for_each_engine(engine, i915, id)
- i915_gem_cleanup_engine(engine);
+ engine_set_wedged(engine);
return 0;
}
struct drm_i915_gem_caching *args = data;
struct drm_i915_gem_object *obj;
enum i915_cache_level level;
- int ret;
+ int ret = 0;
switch (args->caching) {
case I915_CACHING_NONE:
return -EINVAL;
}
- ret = i915_mutex_lock_interruptible(dev);
+ obj = i915_gem_object_lookup(file, args->handle);
+ if (!obj)
+ return -ENOENT;
+
+ if (obj->cache_level == level)
+ goto out;
+
+ ret = i915_gem_object_wait(obj,
+ I915_WAIT_INTERRUPTIBLE,
+ MAX_SCHEDULE_TIMEOUT,
+ to_rps_client(file));
if (ret)
- return ret;
+ goto out;
- obj = i915_gem_object_lookup(file, args->handle);
- if (!obj) {
- ret = -ENOENT;
- goto unlock;
- }
+ ret = i915_mutex_lock_interruptible(dev);
+ if (ret)
+ goto out;
ret = i915_gem_object_set_cache_level(obj, level);
- i915_gem_object_put(obj);
-unlock:
mutex_unlock(&dev->struct_mutex);
+
+out:
+ i915_gem_object_put(obj);
return ret;
}
* try to preserve the existing ABI).
*/
vma = ERR_PTR(-ENOSPC);
- if (view->type == I915_GGTT_VIEW_NORMAL)
+ if (!view || view->type == I915_GGTT_VIEW_NORMAL)
vma = i915_gem_object_ggtt_pin(obj, view, 0, alignment,
PIN_MAPPABLE | PIN_NONBLOCK);
if (IS_ERR(vma)) {
return;
if (--vma->obj->pin_display == 0)
- vma->display_alignment = 0;
+ vma->display_alignment = I915_GTT_MIN_ALIGNMENT;
/* Bump the LRU to try and avoid premature eviction whilst flipping */
- if (!i915_vma_is_active(vma))
- list_move_tail(&vma->vm_link, &vma->vm->inactive_list);
+ i915_gem_object_bump_inactive_ggtt(vma->obj);
i915_vma_unpin(vma);
}
lockdep_assert_held(&obj->base.dev->struct_mutex);
- vma = i915_gem_obj_lookup_or_create_vma(obj, vm, view);
- if (IS_ERR(vma))
+ vma = i915_vma_instance(obj, vm, view);
+ if (unlikely(IS_ERR(vma)))
return vma;
if (i915_vma_misplaced(vma, size, alignment, flags)) {
return ERR_PTR(-ENOSPC);
if (flags & PIN_MAPPABLE) {
- u32 fence_size;
-
- fence_size = i915_gem_get_ggtt_size(dev_priv, vma->size,
- i915_gem_object_get_tiling(obj));
/* If the required space is larger than the available
* aperture, we will not able to find a slot for the
* object and unbinding the object now will be in
* the object in and out of the Global GTT and
* waste a lot of cycles under the mutex.
*/
- if (fence_size > dev_priv->ggtt.mappable_end)
+ if (vma->fence_size > dev_priv->ggtt.mappable_end)
return ERR_PTR(-E2BIG);
/* If NONBLOCK is set the caller is optimistically
* we could try to minimise harm to others.
*/
if (flags & PIN_NONBLOCK &&
- fence_size > dev_priv->ggtt.mappable_end / 2)
+ vma->fence_size > dev_priv->ggtt.mappable_end / 2)
return ERR_PTR(-ENOSPC);
}
enum intel_engine_id id;
for_each_engine(engine, dev_priv, id)
- GEM_BUG_ON(!i915_gem_context_is_kernel(engine->last_retired_context));
+ GEM_BUG_ON(engine->last_retired_context &&
+ !i915_gem_context_is_kernel(engine->last_retired_context));
}
int i915_gem_suspend(struct drm_i915_private *dev_priv)
* part. It should be safe to decrease this, but it's more future proof as is.
*/
#define GEN6_CONTEXT_ALIGN (64<<10)
-#define GEN7_CONTEXT_ALIGN 4096
+#define GEN7_CONTEXT_ALIGN I915_GTT_MIN_ALIGNMENT
static size_t get_context_alignment(struct drm_i915_private *dev_priv)
{
return obj;
}
-static void i915_ppgtt_close(struct i915_address_space *vm)
-{
- struct list_head *phases[] = {
- &vm->active_list,
- &vm->inactive_list,
- &vm->unbound_list,
- NULL,
- }, **phase;
-
- GEM_BUG_ON(vm->closed);
- vm->closed = true;
-
- for (phase = phases; *phase; phase++) {
- struct i915_vma *vma, *vn;
-
- list_for_each_entry_safe(vma, vn, *phase, vm_link)
- if (!i915_vma_is_closed(vma))
- i915_vma_close(vma);
- }
-}
-
static void context_close(struct i915_gem_context *ctx)
{
i915_gem_context_set_closed(ctx);
goto err_out;
}
- vma = i915_vma_create(obj, &dev_priv->ggtt.base, NULL);
+ vma = i915_vma_instance(obj, &dev_priv->ggtt.base, NULL);
if (IS_ERR(vma)) {
i915_gem_object_put(obj);
ret = PTR_ERR(vma);
if (HAS_GUC(dev_priv) && i915.enable_guc_loading)
ctx->ggtt_offset_bias = GUC_WOPCM_TOP;
else
- ctx->ggtt_offset_bias = 4096;
+ ctx->ggtt_offset_bias = I915_GTT_PAGE_SIZE;
return ctx;
dev_priv->hw_context_size = 0;
} else if (HAS_HW_CONTEXTS(dev_priv)) {
dev_priv->hw_context_size =
- round_up(get_context_size(dev_priv), 4096);
+ round_up(get_context_size(dev_priv),
+ I915_GTT_PAGE_SIZE);
if (dev_priv->hw_context_size > (1<<20)) {
DRM_DEBUG_DRIVER("Disabling HW Contexts; invalid size %d\n",
dev_priv->hw_context_size);
return do_rcs_switch(req);
}
+static bool engine_has_kernel_context(struct intel_engine_cs *engine)
+{
+ struct i915_gem_timeline *timeline;
+
+ list_for_each_entry(timeline, &engine->i915->gt.timelines, link) {
+ struct intel_timeline *tl;
+
+ if (timeline == &engine->i915->gt.global_timeline)
+ continue;
+
+ tl = &timeline->engine[engine->id];
+ if (i915_gem_active_peek(&tl->last_request,
+ &engine->i915->drm.struct_mutex))
+ return false;
+ }
+
+ return (!engine->last_retired_context ||
+ i915_gem_context_is_kernel(engine->last_retired_context));
+}
+
int i915_gem_switch_to_kernel_context(struct drm_i915_private *dev_priv)
{
struct intel_engine_cs *engine;
lockdep_assert_held(&dev_priv->drm.struct_mutex);
+ i915_gem_retire_requests(dev_priv);
+
for_each_engine(engine, dev_priv, id) {
struct drm_i915_gem_request *req;
int ret;
+ if (engine_has_kernel_context(engine))
+ continue;
+
req = i915_gem_request_alloc(engine, dev_priv->kernel_context);
if (IS_ERR(req))
return PTR_ERR(req);
/**
* i915_gem_evict_for_vma - Evict vmas to make room for binding a new one
- * @target: address space and range to evict for
+ * @vm: address space to evict from
+ * @target: range (and color) to evict for
* @flags: additional flags to control the eviction algorithm
*
* This function will try to evict vmas that overlap the target node.
* To clarify: This is for freeing up virtual address space, not for freeing
* memory in e.g. the shrinker.
*/
-int i915_gem_evict_for_vma(struct i915_vma *target, unsigned int flags)
+int i915_gem_evict_for_node(struct i915_address_space *vm,
+ struct drm_mm_node *target,
+ unsigned int flags)
{
LIST_HEAD(eviction_list);
struct drm_mm_node *node;
- u64 start = target->node.start;
- u64 end = start + target->node.size;
+ u64 start = target->start;
+ u64 end = start + target->size;
struct i915_vma *vma, *next;
bool check_color;
int ret = 0;
- lockdep_assert_held(&target->vm->i915->drm.struct_mutex);
- trace_i915_gem_evict_vma(target, flags);
+ lockdep_assert_held(&vm->i915->drm.struct_mutex);
+ trace_i915_gem_evict_node(vm, target, flags);
/* Retire before we search the active list. Although we have
* reasonable accuracy in our retirement lists, we may have
* retiring.
*/
if (!(flags & PIN_NONBLOCK))
- i915_gem_retire_requests(target->vm->i915);
+ i915_gem_retire_requests(vm->i915);
- check_color = target->vm->mm.color_adjust;
+ check_color = vm->mm.color_adjust;
if (check_color) {
/* Expand search to cover neighbouring guard pages (or lack!) */
- if (start > target->vm->start)
- start -= 4096;
- if (end < target->vm->start + target->vm->total)
- end += 4096;
+ if (start > vm->start)
+ start -= I915_GTT_PAGE_SIZE;
+ if (end < vm->start + vm->total)
+ end += I915_GTT_PAGE_SIZE;
}
- drm_mm_for_each_node_in_range(node, &target->vm->mm, start, end) {
+ drm_mm_for_each_node_in_range(node, &vm->mm, start, end) {
/* If we find any non-objects (!vma), we cannot evict them */
if (node->color == I915_COLOR_UNEVICTABLE) {
ret = -ENOSPC;
* those as well to make room for our guard pages.
*/
if (check_color) {
- if (vma->node.start + vma->node.size == target->node.start) {
- if (vma->node.color == target->node.color)
+ if (vma->node.start + vma->node.size == node->start) {
+ if (vma->node.color == node->color)
continue;
}
- if (vma->node.start == target->node.start + target->node.size) {
- if (vma->node.color == target->node.color)
+ if (vma->node.start == node->start + node->size) {
+ if (vma->node.color == node->color)
continue;
}
}
}
/* Overlap of objects in the same batch? */
- if (i915_vma_is_pinned(vma)) {
+ if (i915_vma_is_pinned(vma) || !list_empty(&vma->exec_list)) {
ret = -ENOSPC;
if (vma->exec_entry &&
vma->exec_entry->flags & EXEC_OBJECT_PINNED)
* from the (obj, vm) we don't run the risk of creating
* duplicated vmas for the same vm.
*/
- vma = i915_gem_obj_lookup_or_create_vma(obj, vm, NULL);
+ vma = i915_vma_instance(obj, vm, NULL);
if (unlikely(IS_ERR(vma))) {
DRM_DEBUG("Failed to lookup VMA\n");
ret = PTR_ERR(vma);
memset(&cache->node, 0, sizeof(cache->node));
ret = drm_mm_insert_node_in_range_generic
(&ggtt->base.mm, &cache->node,
- 4096, 0, I915_COLOR_UNEVICTABLE,
+ PAGE_SIZE, 0, I915_COLOR_UNEVICTABLE,
0, ggtt->mappable_end,
DRM_MM_SEARCH_DEFAULT,
DRM_MM_CREATE_DEFAULT);
WARN_ON(entry->flags & __EXEC_OBJECT_NEEDS_MAP &&
!i915_vma_is_ggtt(vma));
- if (entry->alignment &&
- vma->node.start & (entry->alignment - 1))
+ if (entry->alignment && !IS_ALIGNED(vma->node.start, entry->alignment))
return true;
if (vma->node.size < entry->pad_to_size)
val = 0;
if (vma) {
- unsigned int tiling = i915_gem_object_get_tiling(vma->obj);
- bool is_y_tiled = tiling == I915_TILING_Y;
unsigned int stride = i915_gem_object_get_stride(vma->obj);
- u32 row_size = stride * (is_y_tiled ? 32 : 8);
- u32 size = rounddown((u32)vma->node.size, row_size);
- val = ((vma->node.start + size - 4096) & 0xfffff000) << 32;
- val |= vma->node.start & 0xfffff000;
+ GEM_BUG_ON(!i915_vma_is_map_and_fenceable(vma));
+ GEM_BUG_ON(!IS_ALIGNED(vma->node.start, I965_FENCE_PAGE));
+ GEM_BUG_ON(!IS_ALIGNED(vma->fence_size, I965_FENCE_PAGE));
+ GEM_BUG_ON(!IS_ALIGNED(stride, 128));
+
+ val = (vma->node.start + vma->fence_size - I965_FENCE_PAGE) << 32;
+ val |= vma->node.start;
val |= (u64)((stride / 128) - 1) << fence_pitch_shift;
- if (is_y_tiled)
+ if (i915_gem_object_get_tiling(vma->obj) == I915_TILING_Y)
val |= BIT(I965_FENCE_TILING_Y_SHIFT);
val |= I965_FENCE_REG_VALID;
}
unsigned int tiling = i915_gem_object_get_tiling(vma->obj);
bool is_y_tiled = tiling == I915_TILING_Y;
unsigned int stride = i915_gem_object_get_stride(vma->obj);
- int pitch_val;
- int tile_width;
- WARN((vma->node.start & ~I915_FENCE_START_MASK) ||
- !is_power_of_2(vma->node.size) ||
- (vma->node.start & (vma->node.size - 1)),
- "object 0x%08llx [fenceable? %d] not 1M or pot-size (0x%08llx) aligned\n",
- vma->node.start,
- i915_vma_is_map_and_fenceable(vma),
- vma->node.size);
+ GEM_BUG_ON(!i915_vma_is_map_and_fenceable(vma));
+ GEM_BUG_ON(vma->node.start & ~I915_FENCE_START_MASK);
+ GEM_BUG_ON(!is_power_of_2(vma->fence_size));
+ GEM_BUG_ON(!IS_ALIGNED(vma->node.start, vma->fence_size));
if (is_y_tiled && HAS_128_BYTE_Y_TILING(fence->i915))
- tile_width = 128;
+ stride /= 128;
else
- tile_width = 512;
-
- /* Note: pitch better be a power of two tile widths */
- pitch_val = stride / tile_width;
- pitch_val = ffs(pitch_val) - 1;
+ stride /= 512;
+ GEM_BUG_ON(!is_power_of_2(stride));
val = vma->node.start;
if (is_y_tiled)
val |= BIT(I830_FENCE_TILING_Y_SHIFT);
- val |= I915_FENCE_SIZE_BITS(vma->node.size);
- val |= pitch_val << I830_FENCE_PITCH_SHIFT;
+ val |= I915_FENCE_SIZE_BITS(vma->fence_size);
+ val |= ilog2(stride) << I830_FENCE_PITCH_SHIFT;
+
val |= I830_FENCE_REG_VALID;
}
val = 0;
if (vma) {
- unsigned int tiling = i915_gem_object_get_tiling(vma->obj);
- bool is_y_tiled = tiling == I915_TILING_Y;
unsigned int stride = i915_gem_object_get_stride(vma->obj);
- u32 pitch_val;
- WARN((vma->node.start & ~I830_FENCE_START_MASK) ||
- !is_power_of_2(vma->node.size) ||
- (vma->node.start & (vma->node.size - 1)),
- "object 0x%08llx not 512K or pot-size 0x%08llx aligned\n",
- vma->node.start, vma->node.size);
-
- pitch_val = stride / 128;
- pitch_val = ffs(pitch_val) - 1;
+ GEM_BUG_ON(!i915_vma_is_map_and_fenceable(vma));
+ GEM_BUG_ON(vma->node.start & ~I830_FENCE_START_MASK);
+ GEM_BUG_ON(!is_power_of_2(vma->fence_size));
+ GEM_BUG_ON(!is_power_of_2(stride / 128));
+ GEM_BUG_ON(!IS_ALIGNED(vma->node.start, vma->fence_size));
val = vma->node.start;
- if (is_y_tiled)
+ if (i915_gem_object_get_tiling(vma->obj) == I915_TILING_Y)
val |= BIT(I830_FENCE_TILING_Y_SHIFT);
- val |= I830_FENCE_SIZE_BITS(vma->node.size);
- val |= pitch_val << I830_FENCE_PITCH_SHIFT;
+ val |= I830_FENCE_SIZE_BITS(vma->fence_size);
+ val |= ilog2(stride / 128) << I830_FENCE_PITCH_SHIFT;
val |= I830_FENCE_REG_VALID;
}
struct drm_i915_private;
struct i915_vma;
+#define I965_FENCE_PAGE 4096UL
+
struct drm_i915_fence_reg {
struct list_head link;
struct drm_i915_private *i915;
*
*/
+#include <linux/log2.h>
+#include <linux/random.h>
#include <linux/seq_file.h>
#include <linux/stop_machine.h>
+
#include <drm/drmP.h>
#include <drm/i915_drm.h>
+
#include "i915_drv.h"
#include "i915_vgpu.h"
#include "i915_trace.h"
static int
i915_get_ggtt_vma_pages(struct i915_vma *vma);
-const struct i915_ggtt_view i915_ggtt_view_normal = {
- .type = I915_GGTT_VIEW_NORMAL,
-};
-const struct i915_ggtt_view i915_ggtt_view_rotated = {
- .type = I915_GGTT_VIEW_ROTATED,
-};
+static void gen6_ggtt_invalidate(struct drm_i915_private *dev_priv)
+{
+ /* Note that as an uncached mmio write, this should flush the
+ * WCB of the writes into the GGTT before it triggers the invalidate.
+ */
+ I915_WRITE(GFX_FLSH_CNTL_GEN6, GFX_FLSH_CNTL_EN);
+}
+
+static void guc_ggtt_invalidate(struct drm_i915_private *dev_priv)
+{
+ gen6_ggtt_invalidate(dev_priv);
+ I915_WRITE(GEN8_GTCR, GEN8_GTCR_INVALIDATE);
+}
+
+static void gmch_ggtt_invalidate(struct drm_i915_private *dev_priv)
+{
+ intel_gtt_chipset_flush();
+}
+
+static inline void i915_ggtt_invalidate(struct drm_i915_private *i915)
+{
+ i915->ggtt.invalidate(i915);
+}
int intel_sanitize_enable_ppgtt(struct drm_i915_private *dev_priv,
int enable_ppgtt)
return -ENOMEM;
p->daddr = dma_map_page(kdev,
- p->page, 0, 4096, PCI_DMA_BIDIRECTIONAL);
+ p->page, 0, PAGE_SIZE, PCI_DMA_BIDIRECTIONAL);
if (dma_mapping_error(kdev, p->daddr)) {
__free_page(p->page);
if (WARN_ON(!p->page))
return;
- dma_unmap_page(&pdev->dev, p->daddr, 4096, PCI_DMA_BIDIRECTIONAL);
+ dma_unmap_page(&pdev->dev, p->daddr, PAGE_SIZE, PCI_DMA_BIDIRECTIONAL);
__free_page(p->page);
memset(p, 0, sizeof(*p));
}
}
static void
-gen8_setup_page_directory(struct i915_hw_ppgtt *ppgtt,
- struct i915_page_directory_pointer *pdp,
- struct i915_page_directory *pd,
- int index)
+gen8_setup_pdpe(struct i915_hw_ppgtt *ppgtt,
+ struct i915_page_directory_pointer *pdp,
+ struct i915_page_directory *pd,
+ int index)
{
gen8_ppgtt_pdpe_t *page_directorypo;
}
static void
-gen8_setup_page_directory_pointer(struct i915_hw_ppgtt *ppgtt,
- struct i915_pml4 *pml4,
- struct i915_page_directory_pointer *pdp,
- int index)
+gen8_setup_pml4e(struct i915_hw_ppgtt *ppgtt,
+ struct i915_pml4 *pml4,
+ struct i915_page_directory_pointer *pdp,
+ int index)
{
gen8_ppgtt_pml4e_t *pagemap = kmap_px(pml4);
struct i915_hw_ppgtt *ppgtt = i915_vm_to_ppgtt(vm);
struct i915_page_directory *pd;
uint64_t pdpe;
- gen8_ppgtt_pdpe_t *pdpe_vaddr;
- gen8_ppgtt_pdpe_t scratch_pdpe =
- gen8_pdpe_encode(px_dma(vm->scratch_pd), I915_CACHE_LLC);
gen8_for_each_pdpe(pd, pdp, start, length, pdpe) {
if (WARN_ON(!pdp->page_directory[pdpe]))
if (gen8_ppgtt_clear_pd(vm, pd, start, length)) {
__clear_bit(pdpe, pdp->used_pdpes);
- if (USES_FULL_48BIT_PPGTT(dev_priv)) {
- pdpe_vaddr = kmap_px(pdp);
- pdpe_vaddr[pdpe] = scratch_pdpe;
- kunmap_px(ppgtt, pdpe_vaddr);
- }
+ gen8_setup_pdpe(ppgtt, pdp, vm->scratch_pd, pdpe);
free_pd(vm->i915, pd);
}
}
struct i915_hw_ppgtt *ppgtt = i915_vm_to_ppgtt(vm);
struct i915_page_directory_pointer *pdp;
uint64_t pml4e;
- gen8_ppgtt_pml4e_t *pml4e_vaddr;
- gen8_ppgtt_pml4e_t scratch_pml4e =
- gen8_pml4e_encode(px_dma(vm->scratch_pdp), I915_CACHE_LLC);
GEM_BUG_ON(!USES_FULL_48BIT_PPGTT(vm->i915));
if (gen8_ppgtt_clear_pdp(vm, pdp, start, length)) {
__clear_bit(pml4e, pml4->used_pml4es);
- pml4e_vaddr = kmap_px(pml4);
- pml4e_vaddr[pml4e] = scratch_pml4e;
- kunmap_px(ppgtt, pml4e_vaddr);
+ gen8_setup_pml4e(ppgtt, pml4, vm->scratch_pdp, pml4e);
free_pdp(vm->i915, pdp);
}
}
kunmap_px(ppgtt, page_directory);
__set_bit(pdpe, pdp->used_pdpes);
- gen8_setup_page_directory(ppgtt, pdp, pd, pdpe);
+ gen8_setup_pdpe(ppgtt, pdp, pd, pdpe);
}
free_gen8_temp_bitmaps(new_page_dirs, new_page_tables);
if (ret)
goto err_out;
- gen8_setup_page_directory_pointer(ppgtt, pml4, pdp, pml4e);
+ gen8_setup_pml4e(ppgtt, pml4, pdp, pml4e);
}
bitmap_or(pml4->used_pml4es, new_pdps, pml4->used_pml4es,
struct i915_address_space *vm = &ppgtt->base;
struct drm_i915_private *dev_priv = ppgtt->base.i915;
struct i915_ggtt *ggtt = &dev_priv->ggtt;
- bool retried = false;
int ret;
/* PPGTT PDEs reside in the GGTT and consists of 512 entries. The
if (ret)
return ret;
-alloc:
- ret = drm_mm_insert_node_in_range_generic(&ggtt->base.mm, &ppgtt->node,
- GEN6_PD_SIZE, GEN6_PD_ALIGN,
- I915_COLOR_UNEVICTABLE,
- 0, ggtt->base.total,
- DRM_MM_TOPDOWN);
- if (ret == -ENOSPC && !retried) {
- ret = i915_gem_evict_something(&ggtt->base,
- GEN6_PD_SIZE, GEN6_PD_ALIGN,
- I915_COLOR_UNEVICTABLE,
- 0, ggtt->base.total,
- 0);
- if (ret)
- goto err_out;
-
- retried = true;
- goto alloc;
- }
-
+ ret = i915_gem_gtt_insert(&ggtt->base, &ppgtt->node,
+ GEN6_PD_SIZE, GEN6_PD_ALIGN,
+ I915_COLOR_UNEVICTABLE,
+ 0, ggtt->base.total,
+ PIN_HIGH);
if (ret)
goto err_out;
-
if (ppgtt->node.start < ggtt->mappable_end)
DRM_DEBUG("Forced to use aperture for PDEs\n");
return ppgtt;
}
+void i915_ppgtt_close(struct i915_address_space *vm)
+{
+ struct list_head *phases[] = {
+ &vm->active_list,
+ &vm->inactive_list,
+ &vm->unbound_list,
+ NULL,
+ }, **phase;
+
+ GEM_BUG_ON(vm->closed);
+ vm->closed = true;
+
+ for (phase = phases; *phase; phase++) {
+ struct i915_vma *vma, *vn;
+
+ list_for_each_entry_safe(vma, vn, *phase, vm_link)
+ if (!i915_vma_is_closed(vma))
+ i915_vma_close(vma);
+ }
+}
+
void i915_ppgtt_release(struct kref *kref)
{
struct i915_hw_ppgtt *ppgtt =
POSTING_READ(RING_FAULT_REG(dev_priv->engine[RCS]));
}
-static void i915_ggtt_flush(struct drm_i915_private *dev_priv)
-{
- if (INTEL_INFO(dev_priv)->gen < 6) {
- intel_gtt_chipset_flush();
- } else {
- I915_WRITE(GFX_FLSH_CNTL_GEN6, GFX_FLSH_CNTL_EN);
- POSTING_READ(GFX_FLSH_CNTL_GEN6);
- }
-}
-
void i915_gem_suspend_gtt_mappings(struct drm_i915_private *dev_priv)
{
struct i915_ggtt *ggtt = &dev_priv->ggtt;
ggtt->base.clear_range(&ggtt->base, ggtt->base.start, ggtt->base.total);
- i915_ggtt_flush(dev_priv);
+ i915_ggtt_invalidate(dev_priv);
}
int i915_gem_gtt_prepare_pages(struct drm_i915_gem_object *obj,
enum i915_cache_level level,
u32 unused)
{
- struct drm_i915_private *dev_priv = vm->i915;
+ struct i915_ggtt *ggtt = i915_vm_to_ggtt(vm);
gen8_pte_t __iomem *pte =
- (gen8_pte_t __iomem *)dev_priv->ggtt.gsm +
- (offset >> PAGE_SHIFT);
+ (gen8_pte_t __iomem *)ggtt->gsm + (offset >> PAGE_SHIFT);
gen8_set_pte(pte, gen8_pte_encode(addr, level));
- I915_WRITE(GFX_FLSH_CNTL_GEN6, GFX_FLSH_CNTL_EN);
- POSTING_READ(GFX_FLSH_CNTL_GEN6);
+ ggtt->invalidate(vm->i915);
}
static void gen8_ggtt_insert_entries(struct i915_address_space *vm,
uint64_t start,
enum i915_cache_level level, u32 unused)
{
- struct drm_i915_private *dev_priv = vm->i915;
struct i915_ggtt *ggtt = i915_vm_to_ggtt(vm);
struct sgt_iter sgt_iter;
gen8_pte_t __iomem *gtt_entries;
* want to flush the TLBs only after we're certain all the PTE updates
* have finished.
*/
- I915_WRITE(GFX_FLSH_CNTL_GEN6, GFX_FLSH_CNTL_EN);
- POSTING_READ(GFX_FLSH_CNTL_GEN6);
+ ggtt->invalidate(vm->i915);
}
struct insert_entries {
enum i915_cache_level level,
u32 flags)
{
- struct drm_i915_private *dev_priv = vm->i915;
+ struct i915_ggtt *ggtt = i915_vm_to_ggtt(vm);
gen6_pte_t __iomem *pte =
- (gen6_pte_t __iomem *)dev_priv->ggtt.gsm +
- (offset >> PAGE_SHIFT);
+ (gen6_pte_t __iomem *)ggtt->gsm + (offset >> PAGE_SHIFT);
iowrite32(vm->pte_encode(addr, level, flags), pte);
- I915_WRITE(GFX_FLSH_CNTL_GEN6, GFX_FLSH_CNTL_EN);
- POSTING_READ(GFX_FLSH_CNTL_GEN6);
+ ggtt->invalidate(vm->i915);
}
/*
uint64_t start,
enum i915_cache_level level, u32 flags)
{
- struct drm_i915_private *dev_priv = vm->i915;
struct i915_ggtt *ggtt = i915_vm_to_ggtt(vm);
struct sgt_iter sgt_iter;
gen6_pte_t __iomem *gtt_entries;
* want to flush the TLBs only after we're certain all the PTE updates
* have finished.
*/
- I915_WRITE(GFX_FLSH_CNTL_GEN6, GFX_FLSH_CNTL_EN);
- POSTING_READ(GFX_FLSH_CNTL_GEN6);
+ ggtt->invalidate(vm->i915);
}
static void nop_clear_range(struct i915_address_space *vm,
u64 *end)
{
if (node->color != color)
- *start += 4096;
+ *start += I915_GTT_PAGE_SIZE;
node = list_next_entry(node, node_list);
if (node->allocated && node->color != color)
- *end -= 4096;
+ *end -= I915_GTT_PAGE_SIZE;
}
int i915_gem_init_ggtt(struct drm_i915_private *dev_priv)
/* Reserve a mappable slot for our lockless error capture */
ret = drm_mm_insert_node_in_range_generic(&ggtt->base.mm,
&ggtt->error_capture,
- 4096, 0,
+ PAGE_SIZE, 0,
I915_COLOR_UNEVICTABLE,
0, ggtt->mappable_end,
0, 0);
if (IS_CHERRYVIEW(dev_priv))
ggtt->base.insert_entries = gen8_ggtt_insert_entries__BKL;
+ ggtt->invalidate = gen6_ggtt_invalidate;
+
return ggtt_probe_common(ggtt, size);
}
ggtt->base.unbind_vma = ggtt_unbind_vma;
ggtt->base.cleanup = gen6_gmch_remove;
+ ggtt->invalidate = gen6_ggtt_invalidate;
+
if (HAS_EDRAM(dev_priv))
ggtt->base.pte_encode = iris_pte_encode;
else if (IS_HASWELL(dev_priv))
ggtt->base.unbind_vma = ggtt_unbind_vma;
ggtt->base.cleanup = i915_gmch_remove;
+ ggtt->invalidate = gmch_ggtt_invalidate;
+
if (unlikely(ggtt->do_idle_maps))
DRM_INFO("applying Ironlake quirks for intel_iommu\n");
return 0;
}
+void i915_ggtt_enable_guc(struct drm_i915_private *i915)
+{
+ i915->ggtt.invalidate = guc_ggtt_invalidate;
+}
+
+void i915_ggtt_disable_guc(struct drm_i915_private *i915)
+{
+ i915->ggtt.invalidate = gen6_ggtt_invalidate;
+}
+
void i915_gem_restore_gtt_mappings(struct drm_i915_private *dev_priv)
{
struct i915_ggtt *ggtt = &dev_priv->ggtt;
}
}
- i915_ggtt_flush(dev_priv);
-}
-
-struct i915_vma *
-i915_gem_obj_to_vma(struct drm_i915_gem_object *obj,
- struct i915_address_space *vm,
- const struct i915_ggtt_view *view)
-{
- struct rb_node *rb;
-
- rb = obj->vma_tree.rb_node;
- while (rb) {
- struct i915_vma *vma = rb_entry(rb, struct i915_vma, obj_node);
- long cmp;
-
- cmp = i915_vma_compare(vma, vm, view);
- if (cmp == 0)
- return vma;
-
- if (cmp < 0)
- rb = rb->rb_right;
- else
- rb = rb->rb_left;
- }
-
- return NULL;
-}
-
-struct i915_vma *
-i915_gem_obj_lookup_or_create_vma(struct drm_i915_gem_object *obj,
- struct i915_address_space *vm,
- const struct i915_ggtt_view *view)
-{
- struct i915_vma *vma;
-
- lockdep_assert_held(&obj->base.dev->struct_mutex);
- GEM_BUG_ON(view && !i915_is_ggtt(vm));
-
- vma = i915_gem_obj_to_vma(obj, vm, view);
- if (!vma) {
- vma = i915_vma_create(obj, vm, view);
- GEM_BUG_ON(vma != i915_gem_obj_to_vma(obj, vm, view));
- }
-
- GEM_BUG_ON(i915_vma_is_closed(vma));
- return vma;
+ i915_ggtt_invalidate(dev_priv);
}
static struct scatterlist *
{
struct sg_table *st;
struct scatterlist *sg, *iter;
- unsigned int count = view->params.partial.size;
+ unsigned int count = view->partial.size;
unsigned int offset;
int ret = -ENOMEM;
if (ret)
goto err_sg_alloc;
- iter = i915_gem_object_get_sg(obj,
- view->params.partial.offset,
- &offset);
+ iter = i915_gem_object_get_sg(obj, view->partial.offset, &offset);
GEM_BUG_ON(!iter);
sg = st->sgl;
vma->pages = vma->obj->mm.pages;
else if (vma->ggtt_view.type == I915_GGTT_VIEW_ROTATED)
vma->pages =
- intel_rotate_fb_obj_pages(&vma->ggtt_view.params.rotated, vma->obj);
+ intel_rotate_fb_obj_pages(&vma->ggtt_view.rotated,
+ vma->obj);
else if (vma->ggtt_view.type == I915_GGTT_VIEW_PARTIAL)
vma->pages = intel_partial_pages(&vma->ggtt_view, vma->obj);
else
return ret;
}
+/**
+ * i915_gem_gtt_reserve - reserve a node in an address_space (GTT)
+ * @vm: the &struct i915_address_space
+ * @node: the &struct drm_mm_node (typically i915_vma.mode)
+ * @size: how much space to allocate inside the GTT,
+ * must be #I915_GTT_PAGE_SIZE aligned
+ * @offset: where to insert inside the GTT,
+ * must be #I915_GTT_MIN_ALIGNMENT aligned, and the node
+ * (@offset + @size) must fit within the address space
+ * @color: color to apply to node, if this node is not from a VMA,
+ * color must be #I915_COLOR_UNEVICTABLE
+ * @flags: control search and eviction behaviour
+ *
+ * i915_gem_gtt_reserve() tries to insert the @node at the exact @offset inside
+ * the address space (using @size and @color). If the @node does not fit, it
+ * tries to evict any overlapping nodes from the GTT, including any
+ * neighbouring nodes if the colors do not match (to ensure guard pages between
+ * differing domains). See i915_gem_evict_for_node() for the gory details
+ * on the eviction algorithm. #PIN_NONBLOCK may used to prevent waiting on
+ * evicting active overlapping objects, and any overlapping node that is pinned
+ * or marked as unevictable will also result in failure.
+ *
+ * Returns: 0 on success, -ENOSPC if no suitable hole is found, -EINTR if
+ * asked to wait for eviction and interrupted.
+ */
+int i915_gem_gtt_reserve(struct i915_address_space *vm,
+ struct drm_mm_node *node,
+ u64 size, u64 offset, unsigned long color,
+ unsigned int flags)
+{
+ int err;
+
+ GEM_BUG_ON(!size);
+ GEM_BUG_ON(!IS_ALIGNED(size, I915_GTT_PAGE_SIZE));
+ GEM_BUG_ON(!IS_ALIGNED(offset, I915_GTT_MIN_ALIGNMENT));
+ GEM_BUG_ON(range_overflows(offset, size, vm->total));
+ GEM_BUG_ON(vm == &vm->i915->mm.aliasing_ppgtt->base);
+ GEM_BUG_ON(drm_mm_node_allocated(node));
+
+ node->size = size;
+ node->start = offset;
+ node->color = color;
+
+ err = drm_mm_reserve_node(&vm->mm, node);
+ if (err != -ENOSPC)
+ return err;
+
+ err = i915_gem_evict_for_node(vm, node, flags);
+ if (err == 0)
+ err = drm_mm_reserve_node(&vm->mm, node);
+
+ return err;
+}
+
+static u64 random_offset(u64 start, u64 end, u64 len, u64 align)
+{
+ u64 range, addr;
+
+ GEM_BUG_ON(range_overflows(start, len, end));
+ GEM_BUG_ON(round_up(start, align) > round_down(end - len, align));
+
+ range = round_down(end - len, align) - round_up(start, align);
+ if (range) {
+ if (sizeof(unsigned long) == sizeof(u64)) {
+ addr = get_random_long();
+ } else {
+ addr = get_random_int();
+ if (range > U32_MAX) {
+ addr <<= 32;
+ addr |= get_random_int();
+ }
+ }
+ div64_u64_rem(addr, range, &addr);
+ start += addr;
+ }
+
+ return round_up(start, align);
+}
+
+/**
+ * i915_gem_gtt_insert - insert a node into an address_space (GTT)
+ * @vm: the &struct i915_address_space
+ * @node: the &struct drm_mm_node (typically i915_vma.node)
+ * @size: how much space to allocate inside the GTT,
+ * must be #I915_GTT_PAGE_SIZE aligned
+ * @alignment: required alignment of starting offset, may be 0 but
+ * if specified, this must be a power-of-two and at least
+ * #I915_GTT_MIN_ALIGNMENT
+ * @color: color to apply to node
+ * @start: start of any range restriction inside GTT (0 for all),
+ * must be #I915_GTT_PAGE_SIZE aligned
+ * @end: end of any range restriction inside GTT (U64_MAX for all),
+ * must be #I915_GTT_PAGE_SIZE aligned if not U64_MAX
+ * @flags: control search and eviction behaviour
+ *
+ * i915_gem_gtt_insert() first searches for an available hole into which
+ * is can insert the node. The hole address is aligned to @alignment and
+ * its @size must then fit entirely within the [@start, @end] bounds. The
+ * nodes on either side of the hole must match @color, or else a guard page
+ * will be inserted between the two nodes (or the node evicted). If no
+ * suitable hole is found, first a victim is randomly selected and tested
+ * for eviction, otherwise then the LRU list of objects within the GTT
+ * is scanned to find the first set of replacement nodes to create the hole.
+ * Those old overlapping nodes are evicted from the GTT (and so must be
+ * rebound before any future use). Any node that is currently pinned cannot
+ * be evicted (see i915_vma_pin()). Similar if the node's VMA is currently
+ * active and #PIN_NONBLOCK is specified, that node is also skipped when
+ * searching for an eviction candidate. See i915_gem_evict_something() for
+ * the gory details on the eviction algorithm.
+ *
+ * Returns: 0 on success, -ENOSPC if no suitable hole is found, -EINTR if
+ * asked to wait for eviction and interrupted.
+ */
+int i915_gem_gtt_insert(struct i915_address_space *vm,
+ struct drm_mm_node *node,
+ u64 size, u64 alignment, unsigned long color,
+ u64 start, u64 end, unsigned int flags)
+{
+ u32 search_flag, alloc_flag;
+ u64 offset;
+ int err;
+
+ lockdep_assert_held(&vm->i915->drm.struct_mutex);
+ GEM_BUG_ON(!size);
+ GEM_BUG_ON(!IS_ALIGNED(size, I915_GTT_PAGE_SIZE));
+ GEM_BUG_ON(alignment && !is_power_of_2(alignment));
+ GEM_BUG_ON(alignment && !IS_ALIGNED(alignment, I915_GTT_MIN_ALIGNMENT));
+ GEM_BUG_ON(start >= end);
+ GEM_BUG_ON(start > 0 && !IS_ALIGNED(start, I915_GTT_PAGE_SIZE));
+ GEM_BUG_ON(end < U64_MAX && !IS_ALIGNED(end, I915_GTT_PAGE_SIZE));
+ GEM_BUG_ON(vm == &vm->i915->mm.aliasing_ppgtt->base);
+ GEM_BUG_ON(drm_mm_node_allocated(node));
+
+ if (unlikely(range_overflows(start, size, end)))
+ return -ENOSPC;
+
+ if (unlikely(round_up(start, alignment) > round_down(end - size, alignment)))
+ return -ENOSPC;
+
+ if (flags & PIN_HIGH) {
+ search_flag = DRM_MM_SEARCH_BELOW;
+ alloc_flag = DRM_MM_CREATE_TOP;
+ } else {
+ search_flag = DRM_MM_SEARCH_DEFAULT;
+ alloc_flag = DRM_MM_CREATE_DEFAULT;
+ }
+
+ /* We only allocate in PAGE_SIZE/GTT_PAGE_SIZE (4096) chunks,
+ * so we know that we always have a minimum alignment of 4096.
+ * The drm_mm range manager is optimised to return results
+ * with zero alignment, so where possible use the optimal
+ * path.
+ */
+ BUILD_BUG_ON(I915_GTT_MIN_ALIGNMENT > I915_GTT_PAGE_SIZE);
+ if (alignment <= I915_GTT_MIN_ALIGNMENT)
+ alignment = 0;
+
+ err = drm_mm_insert_node_in_range_generic(&vm->mm, node,
+ size, alignment, color,
+ start, end,
+ search_flag, alloc_flag);
+ if (err != -ENOSPC)
+ return err;
+
+ /* No free space, pick a slot at random.
+ *
+ * There is a pathological case here using a GTT shared between
+ * mmap and GPU (i.e. ggtt/aliasing_ppgtt but not full-ppgtt):
+ *
+ * |<-- 256 MiB aperture -->||<-- 1792 MiB unmappable -->|
+ * (64k objects) (448k objects)
+ *
+ * Now imagine that the eviction LRU is ordered top-down (just because
+ * pathology meets real life), and that we need to evict an object to
+ * make room inside the aperture. The eviction scan then has to walk
+ * the 448k list before it finds one within range. And now imagine that
+ * it has to search for a new hole between every byte inside the memcpy,
+ * for several simultaneous clients.
+ *
+ * On a full-ppgtt system, if we have run out of available space, there
+ * will be lots and lots of objects in the eviction list! Again,
+ * searching that LRU list may be slow if we are also applying any
+ * range restrictions (e.g. restriction to low 4GiB) and so, for
+ * simplicity and similarilty between different GTT, try the single
+ * random replacement first.
+ */
+ offset = random_offset(start, end,
+ size, alignment ?: I915_GTT_MIN_ALIGNMENT);
+ err = i915_gem_gtt_reserve(vm, node, size, offset, color, flags);
+ if (err != -ENOSPC)
+ return err;
+
+ /* Randomly selected placement is pinned, do a search */
+ err = i915_gem_evict_something(vm, size, alignment, color,
+ start, end, flags);
+ if (err)
+ return err;
+
+ search_flag = DRM_MM_SEARCH_DEFAULT;
+ return drm_mm_insert_node_in_range_generic(&vm->mm, node,
+ size, alignment, color,
+ start, end,
+ search_flag, alloc_flag);
+}
#include "i915_gem_timeline.h"
#include "i915_gem_request.h"
+#define I915_GTT_PAGE_SIZE 4096UL
+#define I915_GTT_MIN_ALIGNMENT I915_GTT_PAGE_SIZE
+
#define I915_FENCE_REG_NONE -1
#define I915_MAX_NUM_FENCES 32
/* 32 fences + sign bit for FENCE_REG_NONE */
struct sg_table;
-enum i915_ggtt_view_type {
- I915_GGTT_VIEW_NORMAL = 0,
- I915_GGTT_VIEW_ROTATED,
- I915_GGTT_VIEW_PARTIAL,
-};
-
struct intel_rotation_info {
- struct {
+ struct intel_rotation_plane_info {
/* tiles */
unsigned int width, height, stride, offset;
} plane[2];
+} __packed;
+
+static inline void assert_intel_rotation_info_is_packed(void)
+{
+ BUILD_BUG_ON(sizeof(struct intel_rotation_info) != 8*sizeof(unsigned int));
+}
+
+struct intel_partial_info {
+ u64 offset;
+ unsigned int size;
+} __packed;
+
+static inline void assert_intel_partial_info_is_packed(void)
+{
+ BUILD_BUG_ON(sizeof(struct intel_partial_info) != sizeof(u64) + sizeof(unsigned int));
+}
+
+enum i915_ggtt_view_type {
+ I915_GGTT_VIEW_NORMAL = 0,
+ I915_GGTT_VIEW_ROTATED = sizeof(struct intel_rotation_info),
+ I915_GGTT_VIEW_PARTIAL = sizeof(struct intel_partial_info),
};
+static inline void assert_i915_ggtt_view_type_is_unique(void)
+{
+ /* As we encode the size of each branch inside the union into its type,
+ * we have to be careful that each branch has a unique size.
+ */
+ switch ((enum i915_ggtt_view_type)0) {
+ case I915_GGTT_VIEW_NORMAL:
+ case I915_GGTT_VIEW_PARTIAL:
+ case I915_GGTT_VIEW_ROTATED:
+ /* gcc complains if these are identical cases */
+ break;
+ }
+}
+
struct i915_ggtt_view {
enum i915_ggtt_view_type type;
-
union {
- struct {
- u64 offset;
- unsigned int size;
- } partial;
+ /* Members need to contain no holes/padding */
+ struct intel_partial_info partial;
struct intel_rotation_info rotated;
- } params;
+ };
};
-extern const struct i915_ggtt_view i915_ggtt_view_normal;
-extern const struct i915_ggtt_view i915_ggtt_view_rotated;
-
enum i915_cache_level;
struct i915_vma;
/** "Graphics Stolen Memory" holds the global PTEs */
void __iomem *gsm;
+ void (*invalidate)(struct drm_i915_private *dev_priv);
bool do_idle_maps;
int i915_ggtt_probe_hw(struct drm_i915_private *dev_priv);
int i915_ggtt_init_hw(struct drm_i915_private *dev_priv);
int i915_ggtt_enable_hw(struct drm_i915_private *dev_priv);
+void i915_ggtt_enable_guc(struct drm_i915_private *i915);
+void i915_ggtt_disable_guc(struct drm_i915_private *i915);
int i915_gem_init_ggtt(struct drm_i915_private *dev_priv);
void i915_ggtt_cleanup_hw(struct drm_i915_private *dev_priv);
struct i915_hw_ppgtt *i915_ppgtt_create(struct drm_i915_private *dev_priv,
struct drm_i915_file_private *fpriv,
const char *name);
+void i915_ppgtt_close(struct i915_address_space *vm);
static inline void i915_ppgtt_get(struct i915_hw_ppgtt *ppgtt)
{
if (ppgtt)
void i915_gem_gtt_finish_pages(struct drm_i915_gem_object *obj,
struct sg_table *pages);
+int i915_gem_gtt_reserve(struct i915_address_space *vm,
+ struct drm_mm_node *node,
+ u64 size, u64 offset, unsigned long color,
+ unsigned int flags);
+
+int i915_gem_gtt_insert(struct i915_address_space *vm,
+ struct drm_mm_node *node,
+ u64 size, u64 alignment, unsigned long color,
+ u64 start, u64 end, unsigned int flags);
+
/* Flags used by pin/bind&friends. */
#define PIN_NONBLOCK BIT(0)
#define PIN_MAPPABLE BIT(1)
#define PIN_HIGH BIT(9)
#define PIN_OFFSET_BIAS BIT(10)
#define PIN_OFFSET_FIXED BIT(11)
-#define PIN_OFFSET_MASK (~4095)
+#define PIN_OFFSET_MASK (-I915_GTT_PAGE_SIZE)
#endif
*/
struct drm_i915_gem_object *
i915_gem_object_create_internal(struct drm_i915_private *i915,
- unsigned int size)
+ phys_addr_t size)
{
struct drm_i915_gem_object *obj;
+ GEM_BUG_ON(!size);
+ GEM_BUG_ON(!IS_ALIGNED(size, PAGE_SIZE));
+
+ if (overflows_type(size, obj->base.size))
+ return ERR_PTR(-E2BIG);
+
obj = i915_gem_object_alloc(i915);
if (!obj)
return ERR_PTR(-ENOMEM);
return obj->tiling_and_stride & STRIDE_MASK;
}
+static inline unsigned int
+i915_gem_tile_height(unsigned int tiling)
+{
+ GEM_BUG_ON(!tiling);
+ return tiling == I915_TILING_Y ? 32 : 8;
+}
+
+static inline unsigned int
+i915_gem_object_get_tile_height(struct drm_i915_gem_object *obj)
+{
+ return i915_gem_tile_height(i915_gem_object_get_tiling(obj));
+}
+
+static inline unsigned int
+i915_gem_object_get_tile_row_size(struct drm_i915_gem_object *obj)
+{
+ return (i915_gem_object_get_stride(obj) *
+ i915_gem_object_get_tile_height(obj));
+}
+
+int i915_gem_object_set_tiling(struct drm_i915_gem_object *obj,
+ unsigned int tiling, unsigned int stride);
+
static inline struct intel_engine_cs *
i915_gem_object_last_write_engine(struct drm_i915_gem_object *obj)
{
if (!rodata)
return 0;
- if (rodata->batch_items * 4 > 4096)
+ if (rodata->batch_items * 4 > PAGE_SIZE)
return -EINVAL;
so = kmalloc(sizeof(*so), GFP_KERNEL);
if (!so)
return -ENOMEM;
- obj = i915_gem_object_create_internal(engine->i915, 4096);
+ obj = i915_gem_object_create_internal(engine->i915, PAGE_SIZE);
if (IS_ERR(obj)) {
ret = PTR_ERR(obj);
goto err_free;
}
- so->vma = i915_vma_create(obj, &engine->i915->ggtt.base, NULL);
+ so->vma = i915_vma_instance(obj, &engine->i915->ggtt.base, NULL);
if (IS_ERR(so->vma)) {
ret = PTR_ERR(so->vma);
goto err_obj;
} while (tmp != req);
}
-static int i915_gem_check_wedge(struct drm_i915_private *dev_priv)
-{
- struct i915_gpu_error *error = &dev_priv->gpu_error;
-
- if (i915_terminally_wedged(error))
- return -EIO;
-
- if (i915_reset_in_progress(error)) {
- /* Non-interruptible callers can't handle -EAGAIN, hence return
- * -EIO unconditionally for these.
- */
- if (!dev_priv->mm.interruptible)
- return -EIO;
-
- return -EAGAIN;
- }
-
- return 0;
-}
-
static int i915_gem_init_global_seqno(struct drm_i915_private *i915, u32 seqno)
{
struct i915_gem_timeline *timeline = &i915->gt.global_timeline;
lockdep_assert_held(&dev_priv->drm.struct_mutex);
/* ABI: Before userspace accesses the GPU (e.g. execbuffer), report
- * EIO if the GPU is already wedged, or EAGAIN to drop the struct_mutex
- * and restart.
+ * EIO if the GPU is already wedged.
*/
- ret = i915_gem_check_wedge(dev_priv);
- if (ret)
- return ERR_PTR(ret);
+ if (i915_terminally_wedged(&dev_priv->gpu_error))
+ return ERR_PTR(-EIO);
/* Pinning the contexts may generate requests in order to acquire
* GGTT space, so do this first before we reserve a seqno for
lockdep_assert_held(&request->i915->drm.struct_mutex);
trace_i915_gem_request_add(request);
+ /* Make sure that no request gazumped us - if it was allocated after
+ * our i915_gem_request_alloc() and called __i915_add_request() before
+ * us, the timeline will hold its seqno which is later than ours.
+ */
+ GEM_BUG_ON(i915_seqno_passed(timeline->last_submitted_seqno,
+ request->fence.seqno));
+
/*
* To ensure that this call will not fail, space for its emissions
* should already have been reserved in the ring buffer. Let the ring
stolen_offset, gtt_offset, size);
/* KISS and expect everything to be page-aligned */
- if (WARN_ON(size == 0) || WARN_ON(size & 4095) ||
- WARN_ON(stolen_offset & 4095))
+ if (WARN_ON(size == 0) ||
+ WARN_ON(!IS_ALIGNED(size, I915_GTT_PAGE_SIZE)) ||
+ WARN_ON(!IS_ALIGNED(stolen_offset, I915_GTT_MIN_ALIGNMENT)))
return NULL;
stolen = kzalloc(sizeof(*stolen), GFP_KERNEL);
if (ret)
goto err;
- vma = i915_gem_obj_lookup_or_create_vma(obj, &ggtt->base, NULL);
+ vma = i915_vma_instance(obj, &ggtt->base, NULL);
if (IS_ERR(vma)) {
ret = PTR_ERR(vma);
goto err_pages;
* setting up the GTT space. The actual reservation will occur
* later.
*/
- vma->node.start = gtt_offset;
- vma->node.size = size;
-
- ret = drm_mm_reserve_node(&ggtt->base.mm, &vma->node);
+ ret = i915_gem_gtt_reserve(&ggtt->base, &vma->node,
+ size, gtt_offset, obj->cache_level,
+ 0);
if (ret) {
DRM_DEBUG_KMS("failed to allocate stolen GTT space\n");
goto err_pages;
}
+ GEM_BUG_ON(!drm_mm_node_allocated(&vma->node));
+
vma->pages = obj->mm.pages;
vma->flags |= I915_VMA_GLOBAL_BIND;
__i915_vma_set_map_and_fenceable(vma);
/**
* DOC: buffer object tiling
*
- * i915_gem_set_tiling() and i915_gem_get_tiling() is the userspace interface to
- * declare fence register requirements.
+ * i915_gem_set_tiling_ioctl() and i915_gem_get_tiling_ioctl() is the userspace
+ * interface to declare fence register requirements.
*
* In principle GEM doesn't care at all about the internal data layout of an
* object, and hence it also doesn't care about tiling or swizzling. There's two
* invovlement.
*/
+/**
+ * i915_gem_fence_size - required global GTT size for a fence
+ * @i915: i915 device
+ * @size: object size
+ * @tiling: tiling mode
+ * @stride: tiling stride
+ *
+ * Return the required global GTT size for a fence (view of a tiled object),
+ * taking into account potential fence register mapping.
+ */
+u32 i915_gem_fence_size(struct drm_i915_private *i915,
+ u32 size, unsigned int tiling, unsigned int stride)
+{
+ u32 ggtt_size;
+
+ GEM_BUG_ON(!size);
+
+ if (tiling == I915_TILING_NONE)
+ return size;
+
+ GEM_BUG_ON(!stride);
+
+ if (INTEL_GEN(i915) >= 4) {
+ stride *= i915_gem_tile_height(tiling);
+ GEM_BUG_ON(!IS_ALIGNED(stride, I965_FENCE_PAGE));
+ return roundup(size, stride);
+ }
+
+ /* Previous chips need a power-of-two fence region when tiling */
+ if (IS_GEN3(i915))
+ ggtt_size = 1024*1024;
+ else
+ ggtt_size = 512*1024;
+
+ while (ggtt_size < size)
+ ggtt_size <<= 1;
+
+ return ggtt_size;
+}
+
+/**
+ * i915_gem_fence_alignment - required global GTT alignment for a fence
+ * @i915: i915 device
+ * @size: object size
+ * @tiling: tiling mode
+ * @stride: tiling stride
+ *
+ * Return the required global GTT alignment for a fence (a view of a tiled
+ * object), taking into account potential fence register mapping.
+ */
+u32 i915_gem_fence_alignment(struct drm_i915_private *i915, u32 size,
+ unsigned int tiling, unsigned int stride)
+{
+ GEM_BUG_ON(!size);
+
+ /*
+ * Minimum alignment is 4k (GTT page size), but might be greater
+ * if a fence register is needed for the object.
+ */
+ if (tiling == I915_TILING_NONE)
+ return I915_GTT_MIN_ALIGNMENT;
+
+ if (INTEL_GEN(i915) >= 4)
+ return I965_FENCE_PAGE;
+
+ /*
+ * Previous chips need to be aligned to the size of the smallest
+ * fence register that can contain the object.
+ */
+ return i915_gem_fence_size(i915, size, tiling, stride);
+}
+
/* Check pitch constriants for all chips & tiling formats */
static bool
-i915_tiling_ok(struct drm_i915_private *dev_priv,
- int stride, int size, int tiling_mode)
+i915_tiling_ok(struct drm_i915_gem_object *obj,
+ unsigned int tiling, unsigned int stride)
{
- int tile_width;
+ struct drm_i915_private *i915 = to_i915(obj->base.dev);
+ unsigned int tile_width;
/* Linear is always fine */
- if (tiling_mode == I915_TILING_NONE)
+ if (tiling == I915_TILING_NONE)
return true;
- if (tiling_mode > I915_TILING_LAST)
+ if (tiling > I915_TILING_LAST)
return false;
- if (IS_GEN2(dev_priv) ||
- (tiling_mode == I915_TILING_Y && HAS_128_BYTE_Y_TILING(dev_priv)))
- tile_width = 128;
- else
- tile_width = 512;
-
/* check maximum stride & object size */
/* i965+ stores the end address of the gtt mapping in the fence
* reg, so dont bother to check the size */
- if (INTEL_GEN(dev_priv) >= 7) {
+ if (INTEL_GEN(i915) >= 7) {
if (stride / 128 > GEN7_FENCE_MAX_PITCH_VAL)
return false;
- } else if (INTEL_GEN(dev_priv) >= 4) {
+ } else if (INTEL_GEN(i915) >= 4) {
if (stride / 128 > I965_FENCE_MAX_PITCH_VAL)
return false;
} else {
if (stride > 8192)
return false;
- if (IS_GEN3(dev_priv)) {
- if (size > I830_FENCE_MAX_SIZE_VAL << 20)
+ if (IS_GEN3(i915)) {
+ if (obj->base.size > I830_FENCE_MAX_SIZE_VAL << 20)
return false;
} else {
- if (size > I830_FENCE_MAX_SIZE_VAL << 19)
+ if (obj->base.size > I830_FENCE_MAX_SIZE_VAL << 19)
return false;
}
}
- if (stride < tile_width)
+ if (IS_GEN2(i915) ||
+ (tiling == I915_TILING_Y && HAS_128_BYTE_Y_TILING(i915)))
+ tile_width = 128;
+ else
+ tile_width = 512;
+
+ if (!IS_ALIGNED(stride, tile_width))
return false;
/* 965+ just needs multiples of tile width */
- if (INTEL_GEN(dev_priv) >= 4) {
- if (stride & (tile_width - 1))
- return false;
+ if (INTEL_GEN(i915) >= 4)
return true;
- }
/* Pre-965 needs power of two tile widths */
- if (stride & (stride - 1))
- return false;
-
- return true;
+ return is_power_of_2(stride);
}
-static bool i915_vma_fence_prepare(struct i915_vma *vma, int tiling_mode)
+static bool i915_vma_fence_prepare(struct i915_vma *vma,
+ int tiling_mode, unsigned int stride)
{
- struct drm_i915_private *dev_priv = vma->vm->i915;
- u32 size;
+ struct drm_i915_private *i915 = vma->vm->i915;
+ u32 size, alignment;
if (!i915_vma_is_map_and_fenceable(vma))
return true;
- if (INTEL_GEN(dev_priv) == 3) {
- if (vma->node.start & ~I915_FENCE_START_MASK)
- return false;
- } else {
- if (vma->node.start & ~I830_FENCE_START_MASK)
- return false;
- }
-
- size = i915_gem_get_ggtt_size(dev_priv, vma->size, tiling_mode);
+ size = i915_gem_fence_size(i915, vma->size, tiling_mode, stride);
if (vma->node.size < size)
return false;
- if (vma->node.start & (size - 1))
+ alignment = i915_gem_fence_alignment(i915, vma->size, tiling_mode, stride);
+ if (!IS_ALIGNED(vma->node.start, alignment))
return false;
return true;
/* Make the current GTT allocation valid for the change in tiling. */
static int
-i915_gem_object_fence_prepare(struct drm_i915_gem_object *obj, int tiling_mode)
+i915_gem_object_fence_prepare(struct drm_i915_gem_object *obj,
+ int tiling_mode, unsigned int stride)
{
- struct drm_i915_private *dev_priv = to_i915(obj->base.dev);
struct i915_vma *vma;
int ret;
if (tiling_mode == I915_TILING_NONE)
return 0;
- if (INTEL_GEN(dev_priv) >= 4)
- return 0;
-
list_for_each_entry(vma, &obj->vma_list, obj_link) {
- if (i915_vma_fence_prepare(vma, tiling_mode))
+ if (!i915_vma_is_ggtt(vma))
+ break;
+
+ if (i915_vma_fence_prepare(vma, tiling_mode, stride))
continue;
ret = i915_vma_unbind(vma);
return 0;
}
+int
+i915_gem_object_set_tiling(struct drm_i915_gem_object *obj,
+ unsigned int tiling, unsigned int stride)
+{
+ struct drm_i915_private *i915 = to_i915(obj->base.dev);
+ struct i915_vma *vma;
+ int err;
+
+ /* Make sure we don't cross-contaminate obj->tiling_and_stride */
+ BUILD_BUG_ON(I915_TILING_LAST & STRIDE_MASK);
+
+ GEM_BUG_ON(!i915_tiling_ok(obj, tiling, stride));
+ GEM_BUG_ON(!stride ^ (tiling == I915_TILING_NONE));
+ lockdep_assert_held(&i915->drm.struct_mutex);
+
+ if ((tiling | stride) == obj->tiling_and_stride)
+ return 0;
+
+ if (obj->framebuffer_references)
+ return -EBUSY;
+
+ /* We need to rebind the object if its current allocation
+ * no longer meets the alignment restrictions for its new
+ * tiling mode. Otherwise we can just leave it alone, but
+ * need to ensure that any fence register is updated before
+ * the next fenced (either through the GTT or by the BLT unit
+ * on older GPUs) access.
+ *
+ * After updating the tiling parameters, we then flag whether
+ * we need to update an associated fence register. Note this
+ * has to also include the unfenced register the GPU uses
+ * whilst executing a fenced command for an untiled object.
+ */
+
+ err = i915_gem_object_fence_prepare(obj, tiling, stride);
+ if (err)
+ return err;
+
+ /* If the memory has unknown (i.e. varying) swizzling, we pin the
+ * pages to prevent them being swapped out and causing corruption
+ * due to the change in swizzling.
+ */
+ mutex_lock(&obj->mm.lock);
+ if (obj->mm.pages &&
+ obj->mm.madv == I915_MADV_WILLNEED &&
+ i915->quirks & QUIRK_PIN_SWIZZLED_PAGES) {
+ if (tiling == I915_TILING_NONE) {
+ GEM_BUG_ON(!obj->mm.quirked);
+ __i915_gem_object_unpin_pages(obj);
+ obj->mm.quirked = false;
+ }
+ if (!i915_gem_object_is_tiled(obj)) {
+ GEM_BUG_ON(!obj->mm.quirked);
+ __i915_gem_object_pin_pages(obj);
+ obj->mm.quirked = true;
+ }
+ }
+ mutex_unlock(&obj->mm.lock);
+
+ list_for_each_entry(vma, &obj->vma_list, obj_link) {
+ if (!i915_vma_is_ggtt(vma))
+ break;
+
+ vma->fence_size =
+ i915_gem_fence_size(i915, vma->size, tiling, stride);
+ vma->fence_alignment =
+ i915_gem_fence_alignment(i915,
+ vma->size, tiling, stride);
+
+ if (vma->fence)
+ vma->fence->dirty = true;
+ }
+
+ obj->tiling_and_stride = tiling | stride;
+
+ /* Force the fence to be reacquired for GTT access */
+ i915_gem_release_mmap(obj);
+
+ /* Try to preallocate memory required to save swizzling on put-pages */
+ if (i915_gem_object_needs_bit17_swizzle(obj)) {
+ if (!obj->bit_17) {
+ obj->bit_17 = kcalloc(BITS_TO_LONGS(obj->base.size >> PAGE_SHIFT),
+ sizeof(long), GFP_KERNEL);
+ }
+ } else {
+ kfree(obj->bit_17);
+ obj->bit_17 = NULL;
+ }
+
+ return 0;
+}
+
/**
- * i915_gem_set_tiling - IOCTL handler to set tiling mode
+ * i915_gem_set_tiling_ioctl - IOCTL handler to set tiling mode
* @dev: DRM device
* @data: data pointer for the ioctl
* @file: DRM file for the ioctl call
* Zero on success, negative errno on failure.
*/
int
-i915_gem_set_tiling(struct drm_device *dev, void *data,
- struct drm_file *file)
+i915_gem_set_tiling_ioctl(struct drm_device *dev, void *data,
+ struct drm_file *file)
{
struct drm_i915_gem_set_tiling *args = data;
- struct drm_i915_private *dev_priv = to_i915(dev);
struct drm_i915_gem_object *obj;
- int err = 0;
-
- /* Make sure we don't cross-contaminate obj->tiling_and_stride */
- BUILD_BUG_ON(I915_TILING_LAST & STRIDE_MASK);
+ int err;
obj = i915_gem_object_lookup(file, args->handle);
if (!obj)
return -ENOENT;
- if (!i915_tiling_ok(dev_priv,
- args->stride, obj->base.size, args->tiling_mode)) {
- i915_gem_object_put(obj);
- return -EINVAL;
- }
-
- mutex_lock(&dev->struct_mutex);
- if (obj->pin_display || obj->framebuffer_references) {
- err = -EBUSY;
+ if (!i915_tiling_ok(obj, args->tiling_mode, args->stride)) {
+ err = -EINVAL;
goto err;
}
args->stride = 0;
} else {
if (args->tiling_mode == I915_TILING_X)
- args->swizzle_mode = dev_priv->mm.bit_6_swizzle_x;
+ args->swizzle_mode = to_i915(dev)->mm.bit_6_swizzle_x;
else
- args->swizzle_mode = dev_priv->mm.bit_6_swizzle_y;
+ args->swizzle_mode = to_i915(dev)->mm.bit_6_swizzle_y;
/* Hide bit 17 swizzling from the user. This prevents old Mesa
* from aborting the application on sw fallbacks to bit 17,
}
}
- if (args->tiling_mode != i915_gem_object_get_tiling(obj) ||
- args->stride != i915_gem_object_get_stride(obj)) {
- /* We need to rebind the object if its current allocation
- * no longer meets the alignment restrictions for its new
- * tiling mode. Otherwise we can just leave it alone, but
- * need to ensure that any fence register is updated before
- * the next fenced (either through the GTT or by the BLT unit
- * on older GPUs) access.
- *
- * After updating the tiling parameters, we then flag whether
- * we need to update an associated fence register. Note this
- * has to also include the unfenced register the GPU uses
- * whilst executing a fenced command for an untiled object.
- */
+ err = mutex_lock_interruptible(&dev->struct_mutex);
+ if (err)
+ goto err;
- err = i915_gem_object_fence_prepare(obj, args->tiling_mode);
- if (!err) {
- struct i915_vma *vma;
-
- mutex_lock(&obj->mm.lock);
- if (obj->mm.pages &&
- obj->mm.madv == I915_MADV_WILLNEED &&
- dev_priv->quirks & QUIRK_PIN_SWIZZLED_PAGES) {
- if (args->tiling_mode == I915_TILING_NONE) {
- GEM_BUG_ON(!obj->mm.quirked);
- __i915_gem_object_unpin_pages(obj);
- obj->mm.quirked = false;
- }
- if (!i915_gem_object_is_tiled(obj)) {
- GEM_BUG_ON(!obj->mm.quirked);
- __i915_gem_object_pin_pages(obj);
- obj->mm.quirked = true;
- }
- }
- mutex_unlock(&obj->mm.lock);
-
- list_for_each_entry(vma, &obj->vma_list, obj_link) {
- if (!vma->fence)
- continue;
-
- vma->fence->dirty = true;
- }
- obj->tiling_and_stride =
- args->stride | args->tiling_mode;
-
- /* Force the fence to be reacquired for GTT access */
- i915_gem_release_mmap(obj);
- }
- }
- /* we have to maintain this existing ABI... */
+ err = i915_gem_object_set_tiling(obj, args->tiling_mode, args->stride);
+ mutex_unlock(&dev->struct_mutex);
+
+ /* We have to maintain this existing ABI... */
args->stride = i915_gem_object_get_stride(obj);
args->tiling_mode = i915_gem_object_get_tiling(obj);
- /* Try to preallocate memory required to save swizzling on put-pages */
- if (i915_gem_object_needs_bit17_swizzle(obj)) {
- if (obj->bit_17 == NULL) {
- obj->bit_17 = kcalloc(BITS_TO_LONGS(obj->base.size >> PAGE_SHIFT),
- sizeof(long), GFP_KERNEL);
- }
- } else {
- kfree(obj->bit_17);
- obj->bit_17 = NULL;
- }
-
err:
i915_gem_object_put(obj);
- mutex_unlock(&dev->struct_mutex);
-
return err;
}
/**
- * i915_gem_get_tiling - IOCTL handler to get tiling mode
+ * i915_gem_get_tiling_ioctl - IOCTL handler to get tiling mode
* @dev: DRM device
* @data: data pointer for the ioctl
* @file: DRM file for the ioctl call
* Zero on success, negative errno on failure.
*/
int
-i915_gem_get_tiling(struct drm_device *dev, void *data,
- struct drm_file *file)
+i915_gem_get_tiling_ioctl(struct drm_device *dev, void *data,
+ struct drm_file *file)
{
struct drm_i915_gem_get_tiling *args = data;
struct drm_i915_private *dev_priv = to_i915(dev);
e->pos += len;
}
+__printf(2, 0)
static void i915_error_vprintf(struct drm_i915_error_state_buf *e,
const char *f, va_list args)
{
#define DMA_ADDRESS_SPACE_GTT (8 << 16)
#define DMA_COPY_SIZE _MMIO(0xc310)
#define DMA_CTRL _MMIO(0xc314)
+#define HUC_UKERNEL (1<<9)
#define UOS_MOVE (1<<4)
#define START_DMA (1<<0)
#define DMA_GUC_WOPCM_OFFSET _MMIO(0xc340)
+#define HUC_LOADING_AGENT_VCR (0<<1)
+#define HUC_LOADING_AGENT_GUC (1<<1)
#define GUC_WOPCM_OFFSET_VALUE 0x80000 /* 512KB */
#define GUC_MAX_IDLE_COUNT _MMIO(0xC3E4)
+#define HUC_STATUS2 _MMIO(0xD3B0)
+#define HUC_FW_VERIFIED (1<<7)
+
/* Defines WOPCM space available to GuC firmware */
#define GUC_WOPCM_SIZE _MMIO(0xc050)
/* GuC addresses below GUC_WOPCM_TOP don't map through the GTT */
*
*/
#include <linux/circ_buf.h>
-#include <linux/debugfs.h>
-#include <linux/relay.h>
#include "i915_drv.h"
#include "intel_uc.h"
*/
/**
- * guc_allocate_vma() - Allocate a GGTT VMA for GuC usage
+ * intel_guc_allocate_vma() - Allocate a GGTT VMA for GuC usage
* @guc: the guc
* @size: size of area to allocate (both virtual space and memory)
*
*
* Return: A i915_vma if successful, otherwise an ERR_PTR.
*/
-static struct i915_vma *guc_allocate_vma(struct intel_guc *guc, u32 size)
+struct i915_vma *intel_guc_allocate_vma(struct intel_guc *guc, u32 size)
{
struct drm_i915_private *dev_priv = guc_to_i915(guc);
struct drm_i915_gem_object *obj;
if (IS_ERR(obj))
return ERR_CAST(obj);
- vma = i915_vma_create(obj, &dev_priv->ggtt.base, NULL);
+ vma = i915_vma_instance(obj, &dev_priv->ggtt.base, NULL);
if (IS_ERR(vma))
goto err;
goto err;
}
- /* Invalidate GuC TLB to let GuC take the latest updates to GTT. */
- I915_WRITE(GEN8_GTCR, GEN8_GTCR_INVALIDATE);
-
return vma;
err:
}
/* The first page is doorbell/proc_desc. Two followed pages are wq. */
- vma = guc_allocate_vma(guc, GUC_DB_SIZE + GUC_WQ_SIZE);
+ vma = intel_guc_allocate_vma(guc, GUC_DB_SIZE + GUC_WQ_SIZE);
if (IS_ERR(vma))
goto err;
return NULL;
}
-/*
- * Sub buffer switch callback. Called whenever relay has to switch to a new
- * sub buffer, relay stays on the same sub buffer if 0 is returned.
- */
-static int subbuf_start_callback(struct rchan_buf *buf,
- void *subbuf,
- void *prev_subbuf,
- size_t prev_padding)
-{
- /* Use no-overwrite mode by default, where relay will stop accepting
- * new data if there are no empty sub buffers left.
- * There is no strict synchronization enforced by relay between Consumer
- * and Producer. In overwrite mode, there is a possibility of getting
- * inconsistent/garbled data, the producer could be writing on to the
- * same sub buffer from which Consumer is reading. This can't be avoided
- * unless Consumer is fast enough and can always run in tandem with
- * Producer.
- */
- if (relay_buf_full(buf))
- return 0;
-
- return 1;
-}
-
-/*
- * file_create() callback. Creates relay file in debugfs.
- */
-static struct dentry *create_buf_file_callback(const char *filename,
- struct dentry *parent,
- umode_t mode,
- struct rchan_buf *buf,
- int *is_global)
-{
- struct dentry *buf_file;
-
- /* This to enable the use of a single buffer for the relay channel and
- * correspondingly have a single file exposed to User, through which
- * it can collect the logs in order without any post-processing.
- * Need to set 'is_global' even if parent is NULL for early logging.
- */
- *is_global = 1;
-
- if (!parent)
- return NULL;
-
- /* Not using the channel filename passed as an argument, since for each
- * channel relay appends the corresponding CPU number to the filename
- * passed in relay_open(). This should be fine as relay just needs a
- * dentry of the file associated with the channel buffer and that file's
- * name need not be same as the filename passed as an argument.
- */
- buf_file = debugfs_create_file("guc_log", mode,
- parent, buf, &relay_file_operations);
- return buf_file;
-}
-
-/*
- * file_remove() default callback. Removes relay file in debugfs.
- */
-static int remove_buf_file_callback(struct dentry *dentry)
-{
- debugfs_remove(dentry);
- return 0;
-}
-
-/* relay channel callbacks */
-static struct rchan_callbacks relay_callbacks = {
- .subbuf_start = subbuf_start_callback,
- .create_buf_file = create_buf_file_callback,
- .remove_buf_file = remove_buf_file_callback,
-};
-
-static void guc_log_remove_relay_file(struct intel_guc *guc)
-{
- relay_close(guc->log.relay_chan);
-}
-
-static int guc_log_create_relay_channel(struct intel_guc *guc)
-{
- struct drm_i915_private *dev_priv = guc_to_i915(guc);
- struct rchan *guc_log_relay_chan;
- size_t n_subbufs, subbuf_size;
-
- /* Keep the size of sub buffers same as shared log buffer */
- subbuf_size = guc->log.vma->obj->base.size;
-
- /* Store up to 8 snapshots, which is large enough to buffer sufficient
- * boot time logs and provides enough leeway to User, in terms of
- * latency, for consuming the logs from relay. Also doesn't take
- * up too much memory.
- */
- n_subbufs = 8;
-
- guc_log_relay_chan = relay_open(NULL, NULL, subbuf_size,
- n_subbufs, &relay_callbacks, dev_priv);
- if (!guc_log_relay_chan) {
- DRM_ERROR("Couldn't create relay chan for GuC logging\n");
- return -ENOMEM;
- }
-
- GEM_BUG_ON(guc_log_relay_chan->subbuf_size < subbuf_size);
- guc->log.relay_chan = guc_log_relay_chan;
- return 0;
-}
-
-static int guc_log_create_relay_file(struct intel_guc *guc)
-{
- struct drm_i915_private *dev_priv = guc_to_i915(guc);
- struct dentry *log_dir;
- int ret;
-
- /* For now create the log file in /sys/kernel/debug/dri/0 dir */
- log_dir = dev_priv->drm.primary->debugfs_root;
-
- /* If /sys/kernel/debug/dri/0 location do not exist, then debugfs is
- * not mounted and so can't create the relay file.
- * The relay API seems to fit well with debugfs only, for availing relay
- * there are 3 requirements which can be met for debugfs file only in a
- * straightforward/clean manner :-
- * i) Need the associated dentry pointer of the file, while opening the
- * relay channel.
- * ii) Should be able to use 'relay_file_operations' fops for the file.
- * iii) Set the 'i_private' field of file's inode to the pointer of
- * relay channel buffer.
- */
- if (!log_dir) {
- DRM_ERROR("Debugfs dir not available yet for GuC log file\n");
- return -ENODEV;
- }
-
- ret = relay_late_setup_files(guc->log.relay_chan, "guc_log", log_dir);
- if (ret) {
- DRM_ERROR("Couldn't associate relay chan with file %d\n", ret);
- return ret;
- }
-
- return 0;
-}
-
-static void guc_move_to_next_buf(struct intel_guc *guc)
-{
- /* Make sure the updates made in the sub buffer are visible when
- * Consumer sees the following update to offset inside the sub buffer.
- */
- smp_wmb();
-
- /* All data has been written, so now move the offset of sub buffer. */
- relay_reserve(guc->log.relay_chan, guc->log.vma->obj->base.size);
-
- /* Switch to the next sub buffer */
- relay_flush(guc->log.relay_chan);
-}
-
-static void *guc_get_write_buffer(struct intel_guc *guc)
-{
- if (!guc->log.relay_chan)
- return NULL;
-
- /* Just get the base address of a new sub buffer and copy data into it
- * ourselves. NULL will be returned in no-overwrite mode, if all sub
- * buffers are full. Could have used the relay_write() to indirectly
- * copy the data, but that would have been bit convoluted, as we need to
- * write to only certain locations inside a sub buffer which cannot be
- * done without using relay_reserve() along with relay_write(). So its
- * better to use relay_reserve() alone.
- */
- return relay_reserve(guc->log.relay_chan, 0);
-}
-
-static bool
-guc_check_log_buf_overflow(struct intel_guc *guc,
- enum guc_log_buffer_type type, unsigned int full_cnt)
-{
- unsigned int prev_full_cnt = guc->log.prev_overflow_count[type];
- bool overflow = false;
-
- if (full_cnt != prev_full_cnt) {
- overflow = true;
-
- guc->log.prev_overflow_count[type] = full_cnt;
- guc->log.total_overflow_count[type] += full_cnt - prev_full_cnt;
-
- if (full_cnt < prev_full_cnt) {
- /* buffer_full_cnt is a 4 bit counter */
- guc->log.total_overflow_count[type] += 16;
- }
- DRM_ERROR_RATELIMITED("GuC log buffer overflow\n");
- }
-
- return overflow;
-}
-
-static unsigned int guc_get_log_buffer_size(enum guc_log_buffer_type type)
-{
- switch (type) {
- case GUC_ISR_LOG_BUFFER:
- return (GUC_LOG_ISR_PAGES + 1) * PAGE_SIZE;
- case GUC_DPC_LOG_BUFFER:
- return (GUC_LOG_DPC_PAGES + 1) * PAGE_SIZE;
- case GUC_CRASH_DUMP_LOG_BUFFER:
- return (GUC_LOG_CRASH_PAGES + 1) * PAGE_SIZE;
- default:
- MISSING_CASE(type);
- }
-
- return 0;
-}
-
-static void guc_read_update_log_buffer(struct intel_guc *guc)
-{
- unsigned int buffer_size, read_offset, write_offset, bytes_to_copy, full_cnt;
- struct guc_log_buffer_state *log_buf_state, *log_buf_snapshot_state;
- struct guc_log_buffer_state log_buf_state_local;
- enum guc_log_buffer_type type;
- void *src_data, *dst_data;
- bool new_overflow;
-
- if (WARN_ON(!guc->log.buf_addr))
- return;
-
- /* Get the pointer to shared GuC log buffer */
- log_buf_state = src_data = guc->log.buf_addr;
-
- /* Get the pointer to local buffer to store the logs */
- log_buf_snapshot_state = dst_data = guc_get_write_buffer(guc);
-
- /* Actual logs are present from the 2nd page */
- src_data += PAGE_SIZE;
- dst_data += PAGE_SIZE;
-
- for (type = GUC_ISR_LOG_BUFFER; type < GUC_MAX_LOG_BUFFER; type++) {
- /* Make a copy of the state structure, inside GuC log buffer
- * (which is uncached mapped), on the stack to avoid reading
- * from it multiple times.
- */
- memcpy(&log_buf_state_local, log_buf_state,
- sizeof(struct guc_log_buffer_state));
- buffer_size = guc_get_log_buffer_size(type);
- read_offset = log_buf_state_local.read_ptr;
- write_offset = log_buf_state_local.sampled_write_ptr;
- full_cnt = log_buf_state_local.buffer_full_cnt;
-
- /* Bookkeeping stuff */
- guc->log.flush_count[type] += log_buf_state_local.flush_to_file;
- new_overflow = guc_check_log_buf_overflow(guc, type, full_cnt);
-
- /* Update the state of shared log buffer */
- log_buf_state->read_ptr = write_offset;
- log_buf_state->flush_to_file = 0;
- log_buf_state++;
-
- if (unlikely(!log_buf_snapshot_state))
- continue;
-
- /* First copy the state structure in snapshot buffer */
- memcpy(log_buf_snapshot_state, &log_buf_state_local,
- sizeof(struct guc_log_buffer_state));
-
- /* The write pointer could have been updated by GuC firmware,
- * after sending the flush interrupt to Host, for consistency
- * set write pointer value to same value of sampled_write_ptr
- * in the snapshot buffer.
- */
- log_buf_snapshot_state->write_ptr = write_offset;
- log_buf_snapshot_state++;
-
- /* Now copy the actual logs. */
- if (unlikely(new_overflow)) {
- /* copy the whole buffer in case of overflow */
- read_offset = 0;
- write_offset = buffer_size;
- } else if (unlikely((read_offset > buffer_size) ||
- (write_offset > buffer_size))) {
- DRM_ERROR("invalid log buffer state\n");
- /* copy whole buffer as offsets are unreliable */
- read_offset = 0;
- write_offset = buffer_size;
- }
-
- /* Just copy the newly written data */
- if (read_offset > write_offset) {
- i915_memcpy_from_wc(dst_data, src_data, write_offset);
- bytes_to_copy = buffer_size - read_offset;
- } else {
- bytes_to_copy = write_offset - read_offset;
- }
- i915_memcpy_from_wc(dst_data + read_offset,
- src_data + read_offset, bytes_to_copy);
-
- src_data += buffer_size;
- dst_data += buffer_size;
- }
-
- if (log_buf_snapshot_state)
- guc_move_to_next_buf(guc);
- else {
- /* Used rate limited to avoid deluge of messages, logs might be
- * getting consumed by User at a slow rate.
- */
- DRM_ERROR_RATELIMITED("no sub-buffer to capture logs\n");
- guc->log.capture_miss_count++;
- }
-}
-
-static void guc_capture_logs_work(struct work_struct *work)
-{
- struct drm_i915_private *dev_priv =
- container_of(work, struct drm_i915_private, guc.log.flush_work);
-
- i915_guc_capture_logs(dev_priv);
-}
-
-static void guc_log_cleanup(struct intel_guc *guc)
-{
- struct drm_i915_private *dev_priv = guc_to_i915(guc);
-
- lockdep_assert_held(&dev_priv->drm.struct_mutex);
-
- /* First disable the flush interrupt */
- gen9_disable_guc_interrupts(dev_priv);
- if (guc->log.flush_wq)
- destroy_workqueue(guc->log.flush_wq);
-
- guc->log.flush_wq = NULL;
-
- if (guc->log.relay_chan)
- guc_log_remove_relay_file(guc);
-
- guc->log.relay_chan = NULL;
-
- if (guc->log.buf_addr)
- i915_gem_object_unpin_map(guc->log.vma->obj);
-
- guc->log.buf_addr = NULL;
-}
-
-static int guc_log_create_extras(struct intel_guc *guc)
-{
- struct drm_i915_private *dev_priv = guc_to_i915(guc);
- void *vaddr;
- int ret;
-
- lockdep_assert_held(&dev_priv->drm.struct_mutex);
-
- /* Nothing to do */
- if (i915.guc_log_level < 0)
- return 0;
-
- if (!guc->log.buf_addr) {
- /* Create a WC (Uncached for read) vmalloc mapping of log
- * buffer pages, so that we can directly get the data
- * (up-to-date) from memory.
- */
- vaddr = i915_gem_object_pin_map(guc->log.vma->obj, I915_MAP_WC);
- if (IS_ERR(vaddr)) {
- ret = PTR_ERR(vaddr);
- DRM_ERROR("Couldn't map log buffer pages %d\n", ret);
- return ret;
- }
-
- guc->log.buf_addr = vaddr;
- }
-
- if (!guc->log.relay_chan) {
- /* Create a relay channel, so that we have buffers for storing
- * the GuC firmware logs, the channel will be linked with a file
- * later on when debugfs is registered.
- */
- ret = guc_log_create_relay_channel(guc);
- if (ret)
- return ret;
- }
-
- if (!guc->log.flush_wq) {
- INIT_WORK(&guc->log.flush_work, guc_capture_logs_work);
-
- /*
- * GuC log buffer flush work item has to do register access to
- * send the ack to GuC and this work item, if not synced before
- * suspend, can potentially get executed after the GFX device is
- * suspended.
- * By marking the WQ as freezable, we don't have to bother about
- * flushing of this work item from the suspend hooks, the pending
- * work item if any will be either executed before the suspend
- * or scheduled later on resume. This way the handling of work
- * item can be kept same between system suspend & rpm suspend.
- */
- guc->log.flush_wq = alloc_ordered_workqueue("i915-guc_log",
- WQ_HIGHPRI | WQ_FREEZABLE);
- if (guc->log.flush_wq == NULL) {
- DRM_ERROR("Couldn't allocate the wq for GuC logging\n");
- return -ENOMEM;
- }
- }
-
- return 0;
-}
-
-static void guc_log_create(struct intel_guc *guc)
-{
- struct i915_vma *vma;
- unsigned long offset;
- uint32_t size, flags;
-
- if (i915.guc_log_level > GUC_LOG_VERBOSITY_MAX)
- i915.guc_log_level = GUC_LOG_VERBOSITY_MAX;
-
- /* The first page is to save log buffer state. Allocate one
- * extra page for others in case for overlap */
- size = (1 + GUC_LOG_DPC_PAGES + 1 +
- GUC_LOG_ISR_PAGES + 1 +
- GUC_LOG_CRASH_PAGES + 1) << PAGE_SHIFT;
-
- vma = guc->log.vma;
- if (!vma) {
- /* We require SSE 4.1 for fast reads from the GuC log buffer and
- * it should be present on the chipsets supporting GuC based
- * submisssions.
- */
- if (WARN_ON(!i915_has_memcpy_from_wc())) {
- /* logging will not be enabled */
- i915.guc_log_level = -1;
- return;
- }
-
- vma = guc_allocate_vma(guc, size);
- if (IS_ERR(vma)) {
- /* logging will be off */
- i915.guc_log_level = -1;
- return;
- }
-
- guc->log.vma = vma;
-
- if (guc_log_create_extras(guc)) {
- guc_log_cleanup(guc);
- i915_vma_unpin_and_release(&guc->log.vma);
- i915.guc_log_level = -1;
- return;
- }
- }
-
- /* each allocated unit is a page */
- flags = GUC_LOG_VALID | GUC_LOG_NOTIFY_ON_HALF_FULL |
- (GUC_LOG_DPC_PAGES << GUC_LOG_DPC_SHIFT) |
- (GUC_LOG_ISR_PAGES << GUC_LOG_ISR_SHIFT) |
- (GUC_LOG_CRASH_PAGES << GUC_LOG_CRASH_SHIFT);
-
- offset = guc_ggtt_offset(vma) >> PAGE_SHIFT; /* in pages */
- guc->log.flags = (offset << GUC_LOG_BUF_ADDR_SHIFT) | flags;
-}
-
-static int guc_log_late_setup(struct intel_guc *guc)
-{
- struct drm_i915_private *dev_priv = guc_to_i915(guc);
- int ret;
-
- lockdep_assert_held(&dev_priv->drm.struct_mutex);
-
- if (i915.guc_log_level < 0)
- return -EINVAL;
-
- /* If log_level was set as -1 at boot time, then setup needed to
- * handle log buffer flush interrupts would not have been done yet,
- * so do that now.
- */
- ret = guc_log_create_extras(guc);
- if (ret)
- goto err;
-
- ret = guc_log_create_relay_file(guc);
- if (ret)
- goto err;
-
- return 0;
-err:
- guc_log_cleanup(guc);
- /* logging will remain off */
- i915.guc_log_level = -1;
- return ret;
-}
static void guc_policies_init(struct guc_policies *policies)
{
vma = guc->ads_vma;
if (!vma) {
- vma = guc_allocate_vma(guc, PAGE_ALIGN(size));
+ vma = intel_guc_allocate_vma(guc, PAGE_ALIGN(size));
if (IS_ERR(vma))
return;
if (guc->ctx_pool_vma)
return 0; /* already allocated */
- vma = guc_allocate_vma(guc, gemsize);
+ vma = intel_guc_allocate_vma(guc, gemsize);
if (IS_ERR(vma))
return PTR_ERR(vma);
guc->ctx_pool_vma = vma;
ida_init(&guc->ctx_ids);
- guc_log_create(guc);
+ intel_guc_log_create(guc);
guc_addon_create(guc);
guc->execbuf_client = guc_client_alloc(dev_priv,
struct i915_gem_context *ctx;
u32 data[3];
- if (guc->guc_fw.guc_fw_load_status != GUC_FIRMWARE_SUCCESS)
+ if (guc->fw.load_status != INTEL_UC_FIRMWARE_SUCCESS)
return 0;
gen9_disable_guc_interrupts(dev_priv);
struct i915_gem_context *ctx;
u32 data[3];
- if (guc->guc_fw.guc_fw_load_status != GUC_FIRMWARE_SUCCESS)
+ if (guc->fw.load_status != INTEL_UC_FIRMWARE_SUCCESS)
return 0;
if (i915.guc_log_level >= 0)
return intel_guc_send(guc, data, ARRAY_SIZE(data));
}
-void i915_guc_capture_logs(struct drm_i915_private *dev_priv)
-{
- guc_read_update_log_buffer(&dev_priv->guc);
-
- /* Generally device is expected to be active only at this
- * time, so get/put should be really quick.
- */
- intel_runtime_pm_get(dev_priv);
- intel_guc_log_flush_complete(&dev_priv->guc);
- intel_runtime_pm_put(dev_priv);
-}
-
-void i915_guc_flush_logs(struct drm_i915_private *dev_priv)
-{
- if (!i915.enable_guc_submission || (i915.guc_log_level < 0))
- return;
-
- /* First disable the interrupts, will be renabled afterwards */
- gen9_disable_guc_interrupts(dev_priv);
-
- /* Before initiating the forceful flush, wait for any pending/ongoing
- * flush to complete otherwise forceful flush may not actually happen.
- */
- flush_work(&dev_priv->guc.log.flush_work);
-
- /* Ask GuC to update the log buffer state */
- intel_guc_log_flush(&dev_priv->guc);
-
- /* GuC would have updated log buffer by now, so capture it */
- i915_guc_capture_logs(dev_priv);
-}
-
-void i915_guc_unregister(struct drm_i915_private *dev_priv)
-{
- if (!i915.enable_guc_submission)
- return;
- mutex_lock(&dev_priv->drm.struct_mutex);
- guc_log_cleanup(&dev_priv->guc);
- mutex_unlock(&dev_priv->drm.struct_mutex);
-}
-
-void i915_guc_register(struct drm_i915_private *dev_priv)
-{
- if (!i915.enable_guc_submission)
- return;
-
- mutex_lock(&dev_priv->drm.struct_mutex);
- guc_log_late_setup(&dev_priv->guc);
- mutex_unlock(&dev_priv->drm.struct_mutex);
-}
-
-int i915_guc_log_control(struct drm_i915_private *dev_priv, u64 control_val)
-{
- union guc_log_control log_param;
- int ret;
-
- log_param.value = control_val;
-
- if (log_param.verbosity < GUC_LOG_VERBOSITY_MIN ||
- log_param.verbosity > GUC_LOG_VERBOSITY_MAX)
- return -EINVAL;
-
- /* This combination doesn't make sense & won't have any effect */
- if (!log_param.logging_enabled && (i915.guc_log_level < 0))
- return 0;
-
- ret = intel_guc_log_control(&dev_priv->guc, log_param.value);
- if (ret < 0) {
- DRM_DEBUG_DRIVER("guc_logging_control action failed %d\n", ret);
- return ret;
- }
-
- i915.guc_log_level = log_param.verbosity;
-
- /* If log_level was set as -1 at boot time, then the relay channel file
- * wouldn't have been created by now and interrupts also would not have
- * been enabled.
- */
- if (!dev_priv->guc.log.relay_chan) {
- ret = guc_log_late_setup(&dev_priv->guc);
- if (!ret)
- gen9_enable_guc_interrupts(dev_priv);
- } else if (!log_param.logging_enabled) {
- /* Once logging is disabled, GuC won't generate logs & send an
- * interrupt. But there could be some data in the log buffer
- * which is yet to be captured. So request GuC to update the log
- * buffer state and then collect the left over logs.
- */
- i915_guc_flush_logs(dev_priv);
-
- /* As logging is disabled, update log level to reflect that */
- i915.guc_log_level = -1;
- } else {
- /* In case interrupts were disabled, enable them now */
- gen9_enable_guc_interrupts(dev_priv);
- }
-
- return ret;
-}
adj *= 2;
else /* CHV needs even encode values */
adj = IS_CHERRYVIEW(dev_priv) ? 2 : 1;
+
+ if (new_delay >= dev_priv->rps.max_freq_softlimit)
+ adj = 0;
/*
* For better performance, jump directly
* to RPe if we're below it.
adj *= 2;
else /* CHV needs even encode values */
adj = IS_CHERRYVIEW(dev_priv) ? -2 : -1;
+
+ if (new_delay <= dev_priv->rps.min_freq_softlimit)
+ adj = 0;
} else { /* unknown event */
adj = 0;
}
{
struct intel_pipe_crc *pipe_crc = &dev_priv->pipe_crc[pipe];
struct intel_pipe_crc_entry *entry;
+ struct intel_crtc *crtc = intel_get_crtc_for_pipe(dev_priv, pipe);
+ struct drm_driver *driver = dev_priv->drm.driver;
+ uint32_t crcs[5];
int head, tail;
spin_lock(&pipe_crc->lock);
+ if (pipe_crc->source) {
+ if (!pipe_crc->entries) {
+ spin_unlock(&pipe_crc->lock);
+ DRM_DEBUG_KMS("spurious interrupt\n");
+ return;
+ }
- if (!pipe_crc->entries) {
- spin_unlock(&pipe_crc->lock);
- DRM_DEBUG_KMS("spurious interrupt\n");
- return;
- }
-
- head = pipe_crc->head;
- tail = pipe_crc->tail;
+ head = pipe_crc->head;
+ tail = pipe_crc->tail;
- if (CIRC_SPACE(head, tail, INTEL_PIPE_CRC_ENTRIES_NR) < 1) {
- spin_unlock(&pipe_crc->lock);
- DRM_ERROR("CRC buffer overflowing\n");
- return;
- }
+ if (CIRC_SPACE(head, tail, INTEL_PIPE_CRC_ENTRIES_NR) < 1) {
+ spin_unlock(&pipe_crc->lock);
+ DRM_ERROR("CRC buffer overflowing\n");
+ return;
+ }
- entry = &pipe_crc->entries[head];
+ entry = &pipe_crc->entries[head];
- entry->frame = dev_priv->drm.driver->get_vblank_counter(&dev_priv->drm,
- pipe);
- entry->crc[0] = crc0;
- entry->crc[1] = crc1;
- entry->crc[2] = crc2;
- entry->crc[3] = crc3;
- entry->crc[4] = crc4;
+ entry->frame = driver->get_vblank_counter(&dev_priv->drm, pipe);
+ entry->crc[0] = crc0;
+ entry->crc[1] = crc1;
+ entry->crc[2] = crc2;
+ entry->crc[3] = crc3;
+ entry->crc[4] = crc4;
- head = (head + 1) & (INTEL_PIPE_CRC_ENTRIES_NR - 1);
- pipe_crc->head = head;
+ head = (head + 1) & (INTEL_PIPE_CRC_ENTRIES_NR - 1);
+ pipe_crc->head = head;
- spin_unlock(&pipe_crc->lock);
+ spin_unlock(&pipe_crc->lock);
- wake_up_interruptible(&pipe_crc->wq);
+ wake_up_interruptible(&pipe_crc->wq);
+ } else {
+ /*
+ * For some not yet identified reason, the first CRC is
+ * bonkers. So let's just wait for the next vblank and read
+ * out the buggy result.
+ *
+ * On CHV sometimes the second CRC is bonkers as well, so
+ * don't trust that one either.
+ */
+ if (pipe_crc->skipped == 0 ||
+ (IS_CHERRYVIEW(dev_priv) && pipe_crc->skipped == 1)) {
+ pipe_crc->skipped++;
+ spin_unlock(&pipe_crc->lock);
+ return;
+ }
+ spin_unlock(&pipe_crc->lock);
+ crcs[0] = crc0;
+ crcs[1] = crc1;
+ crcs[2] = crc2;
+ crcs[3] = crc3;
+ crcs[4] = crc4;
+ drm_crtc_add_crc_entry(&crtc->base, true,
+ drm_accurate_vblank_count(&crtc->base),
+ crcs);
+ }
}
#else
static inline void
* i915_handle_error - handle a gpu error
* @dev_priv: i915 device private
* @engine_mask: mask representing engines that are hung
+ * @fmt: Error message format string
+ *
* Do some basic checking of register state at error time and
* dump it to the syslog. Also call i915_capture_error_state() to make
* sure we get a record and make it available in debugfs. Fire a uevent
* so userspace knows something bad happened (should trigger collection
* of a ring dump etc.).
- * @fmt: Error message format string
*/
void i915_handle_error(struct drm_i915_private *dev_priv,
u32 engine_mask,
#define EDP_PSR_PERF_CNT_MASK 0xffffff
#define EDP_PSR_DEBUG_CTL _MMIO(dev_priv->psr_mmio_base + 0x60)
-#define EDP_PSR_DEBUG_MASK_LPSP (1<<27)
-#define EDP_PSR_DEBUG_MASK_MEMUP (1<<26)
-#define EDP_PSR_DEBUG_MASK_HPD (1<<25)
+#define EDP_PSR_DEBUG_MASK_MAX_SLEEP (1<<28)
+#define EDP_PSR_DEBUG_MASK_LPSP (1<<27)
+#define EDP_PSR_DEBUG_MASK_MEMUP (1<<26)
+#define EDP_PSR_DEBUG_MASK_HPD (1<<25)
+#define EDP_PSR_DEBUG_MASK_DISP_REG_WRITE (1<<16)
+#define EDP_PSR_DEBUG_EXIT_ON_PIXEL_UNDERRUN (1<<15)
#define EDP_PSR2_CTL _MMIO(0x6f900)
#define EDP_PSR2_ENABLE (1<<31)
#define EDP_PSR2_FRAME_BEFORE_SU_SHIFT 4
#define EDP_PSR2_FRAME_BEFORE_SU_MASK (0xf<<4)
#define EDP_PSR2_IDLE_MASK 0xf
+#define EDP_FRAMES_BEFORE_SU_ENTRY (1<<4)
+
+#define EDP_PSR2_STATUS_CTL _MMIO(0x6f940)
+#define EDP_PSR2_STATUS_STATE_MASK (0xf<<28)
+#define EDP_PSR2_STATUS_STATE_SHIFT 28
/* VGA port control */
#define ADPA _MMIO(0x61100)
#define BDW_DPRS_MASK_VBLANK_SRD (1 << 0)
#define CHICKEN_PIPESL_1(pipe) _MMIO_PIPE(pipe, _CHICKEN_PIPESL_1_A, _CHICKEN_PIPESL_1_B)
+#define CHICKEN_TRANS_A 0x420c0
+#define CHICKEN_TRANS_B 0x420c4
+#define CHICKEN_TRANS(trans) _MMIO_TRANS(trans, CHICKEN_TRANS_A, CHICKEN_TRANS_B)
+#define PSR2_VSC_ENABLE_PROG_HEADER (1<<12)
+#define PSR2_ADD_VERTICAL_LINE_COUNT (1<<15)
+
#define DISP_ARB_CTL _MMIO(0x45000)
#define DISP_FBC_MEMORY_WAKE (1<<31)
#define DISP_TILE_SURFACE_SWIZZLING (1<<13)
static inline void debug_fence_free(struct i915_sw_fence *fence)
{
debug_object_free(fence, &i915_sw_fence_debug_descr);
+ smp_wmb(); /* flush the change in state before reallocation */
}
static inline void debug_fence_assert(struct i915_sw_fence *fence)
if (I915_READ(VLV_COUNTER_CONTROL) & VLV_COUNT_RANGE_HIGH)
units <<= 8;
- } else if (IS_BROXTON(dev_priv)) {
+ } else if (IS_GEN9_LP(dev_priv)) {
units = 1;
div = 1200; /* 833.33ns */
}
TP_printk("dev=%d, vm=%p", __entry->dev, __entry->vm)
);
-TRACE_EVENT(i915_gem_evict_vma,
- TP_PROTO(struct i915_vma *vma, unsigned int flags),
- TP_ARGS(vma, flags),
+TRACE_EVENT(i915_gem_evict_node,
+ TP_PROTO(struct i915_address_space *vm, struct drm_mm_node *node, unsigned int flags),
+ TP_ARGS(vm, node, flags),
TP_STRUCT__entry(
__field(u32, dev)
),
TP_fast_assign(
- __entry->dev = vma->vm->i915->drm.primary->index;
- __entry->vm = vma->vm;
- __entry->start = vma->node.start;
- __entry->size = vma->node.size;
- __entry->color = vma->node.color;
+ __entry->dev = vm->i915->drm.primary->index;
+ __entry->vm = vm;
+ __entry->start = node->start;
+ __entry->size = node->size;
+ __entry->color = node->color;
__entry->flags = flags;
),
memset(&bl_info, 0, sizeof(bl_info));
}
-static int vgt_balloon_space(struct drm_mm *mm,
+static int vgt_balloon_space(struct i915_ggtt *ggtt,
struct drm_mm_node *node,
unsigned long start, unsigned long end)
{
unsigned long size = end - start;
- if (start == end)
+ if (start >= end)
return -EINVAL;
DRM_INFO("balloon space: range [ 0x%lx - 0x%lx ] %lu KiB.\n",
start, end, size / 1024);
-
- node->start = start;
- node->size = size;
-
- return drm_mm_reserve_node(mm, node);
+ return i915_gem_gtt_reserve(&ggtt->base, node,
+ size, start, I915_COLOR_UNEVICTABLE,
+ 0);
}
/**
/* Unmappable graphic memory ballooning */
if (unmappable_base > ggtt->mappable_end) {
- ret = vgt_balloon_space(&ggtt->base.mm,
- &bl_info.space[2],
- ggtt->mappable_end,
- unmappable_base);
+ ret = vgt_balloon_space(ggtt, &bl_info.space[2],
+ ggtt->mappable_end, unmappable_base);
if (ret)
goto err;
* because it is reserved to the guard page.
*/
if (unmappable_end < ggtt_end - PAGE_SIZE) {
- ret = vgt_balloon_space(&ggtt->base.mm,
- &bl_info.space[3],
- unmappable_end,
- ggtt_end - PAGE_SIZE);
+ ret = vgt_balloon_space(ggtt, &bl_info.space[3],
+ unmappable_end, ggtt_end - PAGE_SIZE);
if (ret)
goto err;
}
/* Mappable graphic memory ballooning */
if (mappable_base > ggtt->base.start) {
- ret = vgt_balloon_space(&ggtt->base.mm,
- &bl_info.space[0],
+ ret = vgt_balloon_space(ggtt, &bl_info.space[0],
ggtt->base.start, mappable_base);
if (ret)
}
if (mappable_end < ggtt->mappable_end) {
- ret = vgt_balloon_space(&ggtt->base.mm,
- &bl_info.space[1],
- mappable_end,
- ggtt->mappable_end);
+ ret = vgt_balloon_space(ggtt, &bl_info.space[1],
+ mappable_end, ggtt->mappable_end);
if (ret)
goto err;
if (i915_vma_is_active(vma))
return;
+ GEM_BUG_ON(!drm_mm_node_allocated(&vma->node));
list_move_tail(&vma->vm_link, &vma->vm->inactive_list);
if (unlikely(i915_vma_is_closed(vma) && !i915_vma_is_pinned(vma)))
WARN_ON(i915_vma_unbind(vma));
}
static struct i915_vma *
-__i915_vma_create(struct drm_i915_gem_object *obj,
- struct i915_address_space *vm,
- const struct i915_ggtt_view *view)
+vma_create(struct drm_i915_gem_object *obj,
+ struct i915_address_space *vm,
+ const struct i915_ggtt_view *view)
{
struct i915_vma *vma;
struct rb_node *rb, **p;
int i;
- GEM_BUG_ON(vm->closed);
-
- vma = kmem_cache_zalloc(to_i915(obj->base.dev)->vmas, GFP_KERNEL);
+ vma = kmem_cache_zalloc(vm->i915->vmas, GFP_KERNEL);
if (vma == NULL)
return ERR_PTR(-ENOMEM);
for (i = 0; i < ARRAY_SIZE(vma->last_read); i++)
init_request_active(&vma->last_read[i], i915_vma_retire);
init_request_active(&vma->last_fence, NULL);
- list_add(&vma->vm_link, &vm->unbound_list);
vma->vm = vm;
vma->obj = obj;
vma->size = obj->base.size;
+ vma->display_alignment = I915_GTT_MIN_ALIGNMENT;
- if (view) {
+ if (view && view->type != I915_GGTT_VIEW_NORMAL) {
vma->ggtt_view = *view;
if (view->type == I915_GGTT_VIEW_PARTIAL) {
GEM_BUG_ON(range_overflows_t(u64,
- view->params.partial.offset,
- view->params.partial.size,
+ view->partial.offset,
+ view->partial.size,
obj->base.size >> PAGE_SHIFT));
- vma->size = view->params.partial.size;
+ vma->size = view->partial.size;
vma->size <<= PAGE_SHIFT;
GEM_BUG_ON(vma->size >= obj->base.size);
} else if (view->type == I915_GGTT_VIEW_ROTATED) {
- vma->size =
- intel_rotation_info_size(&view->params.rotated);
+ vma->size = intel_rotation_info_size(&view->rotated);
vma->size <<= PAGE_SHIFT;
}
}
+ if (unlikely(vma->size > vm->total))
+ goto err_vma;
+
+ GEM_BUG_ON(!IS_ALIGNED(vma->size, I915_GTT_PAGE_SIZE));
+
if (i915_is_ggtt(vm)) {
+ if (unlikely(overflows_type(vma->size, u32)))
+ goto err_vma;
+
+ vma->fence_size = i915_gem_fence_size(vm->i915, vma->size,
+ i915_gem_object_get_tiling(obj),
+ i915_gem_object_get_stride(obj));
+ if (unlikely(vma->fence_size < vma->size || /* overflow */
+ vma->fence_size > vm->total))
+ goto err_vma;
+
+ GEM_BUG_ON(!IS_ALIGNED(vma->fence_size, I915_GTT_MIN_ALIGNMENT));
+
+ vma->fence_alignment = i915_gem_fence_alignment(vm->i915, vma->size,
+ i915_gem_object_get_tiling(obj),
+ i915_gem_object_get_stride(obj));
+ GEM_BUG_ON(!is_power_of_2(vma->fence_alignment));
+
vma->flags |= I915_VMA_GGTT;
list_add(&vma->obj_link, &obj->vma_list);
} else {
}
rb_link_node(&vma->obj_node, rb, p);
rb_insert_color(&vma->obj_node, &obj->vma_tree);
+ list_add(&vma->vm_link, &vm->unbound_list);
return vma;
+
+err_vma:
+ kmem_cache_free(vm->i915->vmas, vma);
+ return ERR_PTR(-E2BIG);
}
+static struct i915_vma *
+vma_lookup(struct drm_i915_gem_object *obj,
+ struct i915_address_space *vm,
+ const struct i915_ggtt_view *view)
+{
+ struct rb_node *rb;
+
+ rb = obj->vma_tree.rb_node;
+ while (rb) {
+ struct i915_vma *vma = rb_entry(rb, struct i915_vma, obj_node);
+ long cmp;
+
+ cmp = i915_vma_compare(vma, vm, view);
+ if (cmp == 0)
+ return vma;
+
+ if (cmp < 0)
+ rb = rb->rb_right;
+ else
+ rb = rb->rb_left;
+ }
+
+ return NULL;
+}
+
+/**
+ * i915_vma_instance - return the singleton instance of the VMA
+ * @obj: parent &struct drm_i915_gem_object to be mapped
+ * @vm: address space in which the mapping is located
+ * @view: additional mapping requirements
+ *
+ * i915_vma_instance() looks up an existing VMA of the @obj in the @vm with
+ * the same @view characteristics. If a match is not found, one is created.
+ * Once created, the VMA is kept until either the object is freed, or the
+ * address space is closed.
+ *
+ * Must be called with struct_mutex held.
+ *
+ * Returns the vma, or an error pointer.
+ */
struct i915_vma *
-i915_vma_create(struct drm_i915_gem_object *obj,
- struct i915_address_space *vm,
- const struct i915_ggtt_view *view)
+i915_vma_instance(struct drm_i915_gem_object *obj,
+ struct i915_address_space *vm,
+ const struct i915_ggtt_view *view)
{
+ struct i915_vma *vma;
+
lockdep_assert_held(&obj->base.dev->struct_mutex);
GEM_BUG_ON(view && !i915_is_ggtt(vm));
- GEM_BUG_ON(i915_gem_obj_to_vma(obj, vm, view));
+ GEM_BUG_ON(vm->closed);
+
+ vma = vma_lookup(obj, vm, view);
+ if (!vma)
+ vma = vma_create(obj, vm, view);
- return __i915_vma_create(obj, vm, view);
+ GEM_BUG_ON(!IS_ERR(vma) && i915_vma_is_closed(vma));
+ GEM_BUG_ON(!IS_ERR(vma) && i915_vma_compare(vma, vm, view));
+ GEM_BUG_ON(!IS_ERR(vma) && vma_lookup(obj, vm, view) != vma);
+ return vma;
}
/**
return ret;
}
+ trace_i915_vma_bind(vma, bind_flags);
ret = vma->vm->bind_vma(vma, cache_level, bind_flags);
if (ret)
return ret;
if (vma->node.size < size)
return true;
- if (alignment && vma->node.start & (alignment - 1))
+ GEM_BUG_ON(alignment && !is_power_of_2(alignment));
+ if (alignment && !IS_ALIGNED(vma->node.start, alignment))
return true;
if (flags & PIN_MAPPABLE && !i915_vma_is_map_and_fenceable(vma))
void __i915_vma_set_map_and_fenceable(struct i915_vma *vma)
{
- struct drm_i915_gem_object *obj = vma->obj;
- struct drm_i915_private *dev_priv = to_i915(obj->base.dev);
bool mappable, fenceable;
- u32 fence_size, fence_alignment;
-
- fence_size = i915_gem_get_ggtt_size(dev_priv,
- vma->size,
- i915_gem_object_get_tiling(obj));
- fence_alignment = i915_gem_get_ggtt_alignment(dev_priv,
- vma->size,
- i915_gem_object_get_tiling(obj),
- true);
-
- fenceable = (vma->node.size == fence_size &&
- (vma->node.start & (fence_alignment - 1)) == 0);
- mappable = (vma->node.start + fence_size <=
- dev_priv->ggtt.mappable_end);
+ GEM_BUG_ON(!i915_vma_is_ggtt(vma));
+ GEM_BUG_ON(!vma->fence_size);
/*
* Explicitly disable for rotated VMA since the display does not
* need the fence and the VMA is not accessible to other users.
*/
- if (mappable && fenceable &&
- vma->ggtt_view.type != I915_GGTT_VIEW_ROTATED)
+ if (vma->ggtt_view.type == I915_GGTT_VIEW_ROTATED)
+ return;
+
+ fenceable = (vma->node.size >= vma->fence_size &&
+ IS_ALIGNED(vma->node.start, vma->fence_alignment));
+
+ mappable = vma->node.start + vma->fence_size <= i915_vm_to_ggtt(vma->vm)->mappable_end;
+
+ if (mappable && fenceable)
vma->flags |= I915_VMA_CAN_FENCE;
else
vma->flags &= ~I915_VMA_CAN_FENCE;
GEM_BUG_ON(drm_mm_node_allocated(&vma->node));
size = max(size, vma->size);
- if (flags & PIN_MAPPABLE)
- size = i915_gem_get_ggtt_size(dev_priv, size,
- i915_gem_object_get_tiling(obj));
+ alignment = max(alignment, vma->display_alignment);
+ if (flags & PIN_MAPPABLE) {
+ size = max_t(typeof(size), size, vma->fence_size);
+ alignment = max_t(typeof(alignment),
+ alignment, vma->fence_alignment);
+ }
- alignment = max(max(alignment, vma->display_alignment),
- i915_gem_get_ggtt_alignment(dev_priv, size,
- i915_gem_object_get_tiling(obj),
- flags & PIN_MAPPABLE));
+ GEM_BUG_ON(!IS_ALIGNED(size, I915_GTT_PAGE_SIZE));
+ GEM_BUG_ON(!IS_ALIGNED(alignment, I915_GTT_MIN_ALIGNMENT));
+ GEM_BUG_ON(!is_power_of_2(alignment));
start = flags & PIN_OFFSET_BIAS ? flags & PIN_OFFSET_MASK : 0;
+ GEM_BUG_ON(!IS_ALIGNED(start, I915_GTT_PAGE_SIZE));
end = vma->vm->total;
if (flags & PIN_MAPPABLE)
end = min_t(u64, end, dev_priv->ggtt.mappable_end);
if (flags & PIN_ZONE_4G)
- end = min_t(u64, end, (1ULL << 32) - PAGE_SIZE);
+ end = min_t(u64, end, (1ULL << 32) - I915_GTT_PAGE_SIZE);
+ GEM_BUG_ON(!IS_ALIGNED(end, I915_GTT_PAGE_SIZE));
/* If binding the object/GGTT view requires more space than the entire
* aperture has, reject it early before evicting everything in a vain
if (flags & PIN_OFFSET_FIXED) {
u64 offset = flags & PIN_OFFSET_MASK;
- if (offset & (alignment - 1) ||
+ if (!IS_ALIGNED(offset, alignment) ||
range_overflows(offset, size, end)) {
ret = -EINVAL;
goto err_unpin;
}
- vma->node.start = offset;
- vma->node.size = size;
- vma->node.color = obj->cache_level;
- ret = drm_mm_reserve_node(&vma->vm->mm, &vma->node);
- if (ret) {
- ret = i915_gem_evict_for_vma(vma, flags);
- if (ret == 0)
- ret = drm_mm_reserve_node(&vma->vm->mm, &vma->node);
- if (ret)
- goto err_unpin;
- }
+ ret = i915_gem_gtt_reserve(vma->vm, &vma->node,
+ size, offset, obj->cache_level,
+ flags);
+ if (ret)
+ goto err_unpin;
} else {
- u32 search_flag, alloc_flag;
-
- if (flags & PIN_HIGH) {
- search_flag = DRM_MM_SEARCH_BELOW;
- alloc_flag = DRM_MM_CREATE_TOP;
- } else {
- search_flag = DRM_MM_SEARCH_DEFAULT;
- alloc_flag = DRM_MM_CREATE_DEFAULT;
- }
-
- /* We only allocate in PAGE_SIZE/GTT_PAGE_SIZE (4096) chunks,
- * so we know that we always have a minimum alignment of 4096.
- * The drm_mm range manager is optimised to return results
- * with zero alignment, so where possible use the optimal
- * path.
- */
- if (alignment <= 4096)
- alignment = 0;
-
-search_free:
- ret = drm_mm_insert_node_in_range_generic(&vma->vm->mm,
- &vma->node,
- size, alignment,
- obj->cache_level,
- start, end,
- search_flag,
- alloc_flag);
- if (ret) {
- ret = i915_gem_evict_something(vma->vm, size, alignment,
- obj->cache_level,
- start, end,
- flags);
- if (ret == 0)
- goto search_free;
-
+ ret = i915_gem_gtt_insert(vma->vm, &vma->node,
+ size, alignment, obj->cache_level,
+ start, end, flags);
+ if (ret)
goto err_unpin;
- }
GEM_BUG_ON(vma->node.start < start);
GEM_BUG_ON(vma->node.start + vma->node.size > end);
}
+ GEM_BUG_ON(!drm_mm_node_allocated(&vma->node));
GEM_BUG_ON(!i915_gem_valid_gtt_space(vma, obj->cache_level));
list_move_tail(&obj->global_link, &dev_priv->mm.bound_list);
if ((bound ^ vma->flags) & I915_VMA_GLOBAL_BIND)
__i915_vma_set_map_and_fenceable(vma);
+ GEM_BUG_ON(!drm_mm_node_allocated(&vma->node));
GEM_BUG_ON(i915_vma_misplaced(vma, size, alignment, flags));
return 0;
u64 size;
u64 display_alignment;
+ u32 fence_size;
+ u32 fence_alignment;
+
unsigned int flags;
/**
* How many users have pinned this object in GTT space. The following
};
struct i915_vma *
-i915_vma_create(struct drm_i915_gem_object *obj,
- struct i915_address_space *vm,
- const struct i915_ggtt_view *view);
+i915_vma_instance(struct drm_i915_gem_object *obj,
+ struct i915_address_space *vm,
+ const struct i915_ggtt_view *view);
void i915_vma_unpin_and_release(struct i915_vma **p_vma);
if (cmp)
return cmp;
+ BUILD_BUG_ON(I915_GGTT_VIEW_NORMAL != 0);
+ cmp = vma->ggtt_view.type;
if (!view)
- return vma->ggtt_view.type;
+ return cmp;
- if (vma->ggtt_view.type != view->type)
- return vma->ggtt_view.type - view->type;
+ cmp -= view->type;
+ if (cmp)
+ return cmp;
- return memcmp(&vma->ggtt_view.params,
- &view->params,
- sizeof(view->params));
+ /* ggtt_view.type also encodes its size so that we both distinguish
+ * different views using it as a "type" and also use a compact (no
+ * accessing of uninitialised padding bytes) memcmp without storing
+ * an extra parameter or adding more code.
+ *
+ * To ensure that the memcmp is valid for all branches of the union,
+ * even though the code looks like it is just comparing one branch,
+ * we assert above that all branches have the same address, and that
+ * each branch has a unique type/size.
+ */
+ BUILD_BUG_ON(I915_GGTT_VIEW_NORMAL >= I915_GGTT_VIEW_PARTIAL);
+ BUILD_BUG_ON(I915_GGTT_VIEW_PARTIAL >= I915_GGTT_VIEW_ROTATED);
+ BUILD_BUG_ON(offsetof(typeof(*view), rotated) !=
+ offsetof(typeof(*view), partial));
+ return memcmp(&vma->ggtt_view.partial, &view->partial, view->type);
}
int i915_vma_bind(struct i915_vma *vma, enum i915_cache_level cache_level,
/* Pin early to prevent the shrinker/eviction logic from destroying
* our vma as we insert and bind.
*/
- if (likely(((++vma->flags ^ flags) & I915_VMA_BIND_MASK) == 0))
+ if (likely(((++vma->flags ^ flags) & I915_VMA_BIND_MASK) == 0)) {
+ GEM_BUG_ON(!drm_mm_node_allocated(&vma->node));
+ GEM_BUG_ON(i915_vma_misplaced(vma, size, alignment, flags));
return 0;
+ }
return __i915_vma_do_pin(vma, size, alignment, flags);
}
__drm_atomic_helper_plane_duplicate_state(plane, state);
+ intel_state->vma = NULL;
+
return state;
}
intel_plane_destroy_state(struct drm_plane *plane,
struct drm_plane_state *state)
{
+ struct i915_vma *vma;
+
+ vma = fetch_and_zero(&to_intel_plane_state(state)->vma);
+
+ /*
+ * FIXME: Normally intel_cleanup_plane_fb handles destruction of vma.
+ * We currently don't clear all planes during driver unload, so we have
+ * to be able to unpin vma here for now.
+ *
+ * Normally this can only happen during unload when kmscon is disabled
+ * and userspace doesn't attempt to set a framebuffer at all.
+ */
+ if (vma) {
+ mutex_lock(&plane->dev->struct_mutex);
+ intel_unpin_fb_vma(vma);
+ mutex_unlock(&plane->dev->struct_mutex);
+ }
+
drm_atomic_helper_plane_destroy_state(plane, state);
}
* low-power state and comes back to normal.
*/
+#define I915_CSR_GLK "i915/glk_dmc_ver1_01.bin"
+MODULE_FIRMWARE(I915_CSR_GLK);
+#define GLK_CSR_VERSION_REQUIRED CSR_VERSION(1, 1)
+
#define I915_CSR_KBL "i915/kbl_dmc_ver1_01.bin"
MODULE_FIRMWARE(I915_CSR_KBL);
#define KBL_CSR_VERSION_REQUIRED CSR_VERSION(1, 1)
csr->version = css_header->version;
- if (IS_KABYLAKE(dev_priv)) {
+ if (IS_GEMINILAKE(dev_priv)) {
+ required_version = GLK_CSR_VERSION_REQUIRED;
+ } else if (IS_KABYLAKE(dev_priv)) {
required_version = KBL_CSR_VERSION_REQUIRED;
} else if (IS_SKYLAKE(dev_priv)) {
required_version = SKL_CSR_VERSION_REQUIRED;
if (!HAS_CSR(dev_priv))
return;
- if (IS_KABYLAKE(dev_priv))
+ if (IS_GEMINILAKE(dev_priv))
+ csr->fw_path = I915_CSR_GLK;
+ else if (IS_KABYLAKE(dev_priv))
csr->fw_path = I915_CSR_KBL;
else if (IS_SKYLAKE(dev_priv))
csr->fw_path = I915_CSR_SKL;
(IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv)) &&
hweight8(sseu->slice_mask) > 1;
sseu->has_subslice_pg =
- IS_BROXTON(dev_priv) && sseu_subslice_total(sseu) > 1;
+ IS_GEN9_LP(dev_priv) && sseu_subslice_total(sseu) > 1;
sseu->has_eu_pg = sseu->eu_per_subslice > 2;
if (IS_BROXTON(dev_priv)) {
const struct drm_framebuffer *fb,
unsigned int rotation)
{
+ view->type = I915_GGTT_VIEW_NORMAL;
if (drm_rotation_90_or_270(rotation)) {
- *view = i915_ggtt_view_rotated;
- view->params.rotated = to_intel_framebuffer(fb)->rot_info;
- } else {
- *view = i915_ggtt_view_normal;
+ view->type = I915_GGTT_VIEW_ROTATED;
+ view->rotated = to_intel_framebuffer(fb)->rot_info;
}
}
i915_vma_pin_fence(vma);
}
+ i915_vma_get(vma);
err:
intel_runtime_pm_put(dev_priv);
return vma;
}
-void intel_unpin_fb_obj(struct drm_framebuffer *fb, unsigned int rotation)
+void intel_unpin_fb_vma(struct i915_vma *vma)
{
- struct drm_i915_gem_object *obj = intel_fb_obj(fb);
- struct i915_ggtt_view view;
- struct i915_vma *vma;
-
- WARN_ON(!mutex_is_locked(&obj->base.dev->struct_mutex));
-
- intel_fill_fb_ggtt_view(&view, fb, rotation);
- vma = i915_gem_object_to_ggtt(obj, &view);
+ lockdep_assert_held(&vma->vm->i915->drm.struct_mutex);
i915_vma_unpin_fence(vma);
i915_gem_object_unpin_from_display_plane(vma);
+ i915_vma_put(vma);
}
static int intel_fb_pitch(const struct drm_framebuffer *fb, int plane,
struct drm_device *dev = intel_crtc->base.dev;
struct drm_i915_private *dev_priv = to_i915(dev);
struct drm_crtc *c;
- struct intel_crtc *i;
struct drm_i915_gem_object *obj;
struct drm_plane *primary = intel_crtc->base.primary;
struct drm_plane_state *plane_state = primary->state;
* an fb with another CRTC instead
*/
for_each_crtc(dev, c) {
- i = to_intel_crtc(c);
+ struct intel_plane_state *state;
if (c == &intel_crtc->base)
continue;
- if (!i->active)
+ if (!to_intel_crtc(c)->active)
continue;
- fb = c->primary->fb;
- if (!fb)
+ state = to_intel_plane_state(c->primary->state);
+ if (!state->vma)
continue;
- obj = intel_fb_obj(fb);
- if (i915_gem_object_ggtt_offset(obj, NULL) == plane_config->base) {
+ if (intel_plane_ggtt_offset(state) == plane_config->base) {
+ fb = c->primary->fb;
drm_framebuffer_reference(fb);
goto valid_fb;
}
* simplest solution is to just disable the primary plane now and
* pretend the BIOS never had it enabled.
*/
- to_intel_plane_state(plane_state)->base.visible = false;
+ plane_state->visible = false;
crtc_state->plane_mask &= ~(1 << drm_plane_index(primary));
intel_pre_disable_primary_noatomic(&intel_crtc->base);
intel_plane->disable_plane(primary, &intel_crtc->base);
return;
valid_fb:
+ mutex_lock(&dev->struct_mutex);
+ intel_state->vma =
+ intel_pin_and_fence_fb_obj(fb, primary->state->rotation);
+ mutex_unlock(&dev->struct_mutex);
+ if (IS_ERR(intel_state->vma)) {
+ DRM_ERROR("failed to pin boot fb on pipe %d: %li\n",
+ intel_crtc->pipe, PTR_ERR(intel_state->vma));
+
+ intel_state->vma = NULL;
+ drm_framebuffer_unreference(fb);
+ return;
+ }
+
plane_state->src_x = 0;
plane_state->src_y = 0;
plane_state->src_w = fb->width << 16;
unsigned int rotation = plane_state->base.rotation;
int ret;
+ if (!plane_state->base.visible)
+ return 0;
+
/* Rotate src coordinates to match rotated GTT view */
if (drm_rotation_90_or_270(rotation))
drm_rect_rotate(&plane_state->base.src,
I915_WRITE(DSPSTRIDE(plane), fb->pitches[0]);
if (INTEL_GEN(dev_priv) >= 4) {
I915_WRITE(DSPSURF(plane),
- intel_fb_gtt_offset(fb, rotation) +
+ intel_plane_ggtt_offset(plane_state) +
intel_crtc->dspaddr_offset);
I915_WRITE(DSPTILEOFF(plane), (y << 16) | x);
I915_WRITE(DSPLINOFF(plane), linear_offset);
} else {
I915_WRITE(DSPADDR(plane),
- intel_fb_gtt_offset(fb, rotation) +
+ intel_plane_ggtt_offset(plane_state) +
intel_crtc->dspaddr_offset);
}
POSTING_READ(reg);
I915_WRITE(DSPSTRIDE(plane), fb->pitches[0]);
I915_WRITE(DSPSURF(plane),
- intel_fb_gtt_offset(fb, rotation) +
+ intel_plane_ggtt_offset(plane_state) +
intel_crtc->dspaddr_offset);
if (IS_HASWELL(dev_priv) || IS_BROADWELL(dev_priv)) {
I915_WRITE(DSPOFFSET(plane), (y << 16) | x);
}
}
-u32 intel_fb_gtt_offset(struct drm_framebuffer *fb,
- unsigned int rotation)
-{
- struct drm_i915_gem_object *obj = intel_fb_obj(fb);
- struct i915_ggtt_view view;
- struct i915_vma *vma;
-
- intel_fill_fb_ggtt_view(&view, fb, rotation);
-
- vma = i915_gem_object_to_ggtt(obj, &view);
- if (WARN(!vma, "ggtt vma for display object not found! (view=%u)\n",
- view.type))
- return -1;
-
- return i915_ggtt_offset(vma);
-}
-
static void skl_detach_scaler(struct intel_crtc *intel_crtc, int id)
{
struct drm_device *dev = intel_crtc->base.dev;
}
I915_WRITE(PLANE_SURF(pipe, plane_id),
- intel_fb_gtt_offset(fb, rotation) + surf_addr);
+ intel_plane_ggtt_offset(plane_state) + surf_addr);
POSTING_READ(PLANE_SURF(pipe, plane_id));
}
state = drm_atomic_helper_duplicate_state(dev, ctx);
if (IS_ERR(state)) {
ret = PTR_ERR(state);
- state = NULL;
DRM_ERROR("Duplicating state failed with %i\n", ret);
- goto err;
+ return;
}
ret = drm_atomic_helper_disable_all(dev, ctx);
if (ret) {
DRM_ERROR("Suspending crtc's failed with %i\n", ret);
- goto err;
+ drm_atomic_state_put(state);
+ return;
}
dev_priv->modeset_restore_state = state;
state->acquire_ctx = ctx;
- return;
-
-err:
- drm_atomic_state_put(state);
}
void intel_finish_reset(struct drm_i915_private *dev_priv)
if (!intel_crtc->active)
return;
- if (to_intel_plane_state(crtc->primary->state)->base.visible) {
+ if (crtc->primary->state->visible) {
WARN_ON(intel_crtc->flip_work);
intel_pre_disable_primary_noatomic(crtc);
intel_crtc_disable_planes(crtc, 1 << drm_plane_index(crtc->primary));
- to_intel_plane_state(crtc->primary->state)->base.visible = false;
+ crtc->primary->state->visible = false;
}
state = drm_atomic_state_alloc(crtc->dev);
mode->type = DRM_MODE_TYPE_DRIVER;
mode->clock = pipe_config->base.adjusted_mode.crtc_clock;
- mode->flags |= pipe_config->base.adjusted_mode.flags;
mode->hsync = drm_mode_hsync(mode);
mode->vrefresh = drm_mode_vrefresh(mode);
}
old->restore_state = restore_state;
+ drm_atomic_state_put(state);
/* let the connector get through one full cycle before testing */
intel_wait_for_vblank(dev_priv, intel_crtc->pipe);
flush_work(&work->mmio_work);
mutex_lock(&dev->struct_mutex);
- intel_unpin_fb_obj(work->old_fb, primary->state->rotation);
+ intel_unpin_fb_vma(work->old_vma);
i915_gem_object_put(work->pending_flip_obj);
mutex_unlock(&dev->struct_mutex);
goto cleanup_pending;
}
- work->gtt_offset = intel_fb_gtt_offset(fb, primary->state->rotation);
- work->gtt_offset += intel_crtc->dspaddr_offset;
+ work->old_vma = to_intel_plane_state(primary->state)->vma;
+ to_intel_plane_state(primary->state)->vma = vma;
+
+ work->gtt_offset = i915_ggtt_offset(vma) + intel_crtc->dspaddr_offset;
work->rotation = crtc->primary->state->rotation;
/*
cleanup_request:
i915_add_request_no_flush(request);
cleanup_unpin:
- intel_unpin_fb_obj(fb, crtc->primary->state->rotation);
+ to_intel_plane_state(primary->state)->vma = work->old_vma;
+ intel_unpin_fb_vma(vma);
cleanup_pending:
atomic_dec(&intel_crtc->unpin_work_count);
unlock:
}
was_visible = old_plane_state->base.visible;
- visible = to_intel_plane_state(plane_state)->base.visible;
+ visible = plane_state->visible;
if (!was_crtc_enabled && WARN_ON(was_visible))
was_visible = false;
* only combine the results from all planes in the current place?
*/
if (!is_crtc_enabled)
- to_intel_plane_state(plane_state)->base.visible = visible = false;
+ plane_state->visible = visible = false;
if (!was_visible && !visible)
return 0;
.page_flip = intel_crtc_page_flip,
.atomic_duplicate_state = intel_crtc_duplicate_state,
.atomic_destroy_state = intel_crtc_destroy_state,
+ .set_crc_source = intel_crtc_set_crc_source,
};
/**
DRM_DEBUG_KMS("failed to pin object\n");
return PTR_ERR(vma);
}
+
+ to_intel_plane_state(new_state)->vma = vma;
}
return 0;
intel_cleanup_plane_fb(struct drm_plane *plane,
struct drm_plane_state *old_state)
{
- struct drm_i915_private *dev_priv = to_i915(plane->dev);
- struct intel_plane_state *old_intel_state;
- struct drm_i915_gem_object *old_obj = intel_fb_obj(old_state->fb);
- struct drm_i915_gem_object *obj = intel_fb_obj(plane->state->fb);
-
- old_intel_state = to_intel_plane_state(old_state);
-
- if (!obj && !old_obj)
- return;
+ struct i915_vma *vma;
- if (old_obj && (plane->type != DRM_PLANE_TYPE_CURSOR ||
- !INTEL_INFO(dev_priv)->cursor_needs_physical))
- intel_unpin_fb_obj(old_state->fb, old_state->rotation);
+ /* Should only be called after a successful intel_prepare_plane_fb()! */
+ vma = fetch_and_zero(&to_intel_plane_state(old_state)->vma);
+ if (vma)
+ intel_unpin_fb_vma(vma);
}
int
struct intel_plane *intel_plane = to_intel_plane(plane);
struct drm_framebuffer *old_fb;
struct drm_crtc_state *crtc_state = crtc->state;
+ struct i915_vma *old_vma;
/*
* When crtc is inactive or there is a modeset pending,
ret = PTR_ERR(vma);
goto out_unlock;
}
+
+ to_intel_plane_state(new_plane_state)->vma = vma;
}
old_fb = old_plane_state->fb;
+ old_vma = to_intel_plane_state(old_plane_state)->vma;
i915_gem_track_fb(intel_fb_obj(old_fb), intel_fb_obj(fb),
intel_plane->frontbuffer_bit);
*to_intel_plane_state(old_plane_state) = *to_intel_plane_state(new_plane_state);
new_plane_state->fence = NULL;
new_plane_state->fb = old_fb;
+ to_intel_plane_state(new_plane_state)->vma = old_vma;
intel_plane->update_plane(plane,
to_intel_crtc_state(crtc->state),
if (!obj)
addr = 0;
else if (!INTEL_INFO(dev_priv)->cursor_needs_physical)
- addr = i915_gem_object_ggtt_offset(obj, NULL);
+ addr = intel_plane_ggtt_offset(state);
else
addr = obj->phys_handle->busaddr;
* Temporarily change the plane mapping and disable everything
* ... */
plane = crtc->plane;
- to_intel_plane_state(crtc->base.primary->state)->base.visible = true;
+ crtc->base.primary->state->visible = true;
crtc->plane = !plane;
intel_crtc_disable_noatomic(&crtc->base);
crtc->plane = plane;
dev_priv->active_crtcs = 0;
for_each_intel_crtc(dev, crtc) {
- struct intel_crtc_state *crtc_state = crtc->config;
+ struct intel_crtc_state *crtc_state =
+ to_intel_crtc_state(crtc->base.state);
__drm_atomic_helper_crtc_destroy_state(&crtc_state->base);
memset(crtc_state, 0, sizeof(*crtc_state));
DRM_DEBUG_KMS("[CRTC:%d:%s] hw state readout: %s\n",
crtc->base.base.id, crtc->base.name,
- enableddisabled(crtc->active));
+ enableddisabled(crtc_state->base.active));
}
for (i = 0; i < dev_priv->num_shared_dpll; i++) {
&pll->state.hw_state);
pll->state.crtc_mask = 0;
for_each_intel_crtc(dev, crtc) {
- if (crtc->active && crtc->config->shared_dpll == pll)
+ struct intel_crtc_state *crtc_state =
+ to_intel_crtc_state(crtc->base.state);
+
+ if (crtc_state->base.active &&
+ crtc_state->shared_dpll == pll)
pll->state.crtc_mask |= 1 << crtc->pipe;
}
pll->active_mask = pll->state.crtc_mask;
pipe = 0;
if (encoder->get_hw_state(encoder, &pipe)) {
+ struct intel_crtc_state *crtc_state;
+
crtc = intel_get_crtc_for_pipe(dev_priv, pipe);
+ crtc_state = to_intel_crtc_state(crtc->base.state);
encoder->base.crtc = &crtc->base;
- crtc->config->output_types |= 1 << encoder->type;
- encoder->get_config(encoder, crtc->config);
+ crtc_state->output_types |= 1 << encoder->type;
+ encoder->get_config(encoder, crtc_state);
} else {
encoder->base.crtc = NULL;
}
}
for_each_intel_crtc(dev, crtc) {
+ struct intel_crtc_state *crtc_state =
+ to_intel_crtc_state(crtc->base.state);
int pixclk = 0;
- crtc->base.hwmode = crtc->config->base.adjusted_mode;
+ crtc->base.hwmode = crtc_state->base.adjusted_mode;
memset(&crtc->base.mode, 0, sizeof(crtc->base.mode));
- if (crtc->base.state->active) {
- intel_mode_from_pipe_config(&crtc->base.mode, crtc->config);
- intel_mode_from_pipe_config(&crtc->base.state->adjusted_mode, crtc->config);
+ if (crtc_state->base.active) {
+ intel_mode_from_pipe_config(&crtc->base.mode, crtc_state);
+ intel_mode_from_pipe_config(&crtc_state->base.adjusted_mode, crtc_state);
WARN_ON(drm_atomic_set_mode_for_crtc(crtc->base.state, &crtc->base.mode));
/*
* set a flag to indicate that a full recalculation is
* needed on the next commit.
*/
- crtc->base.state->mode.private_flags = I915_MODE_FLAG_INHERITED;
+ crtc_state->base.mode.private_flags = I915_MODE_FLAG_INHERITED;
if (INTEL_GEN(dev_priv) >= 9 || IS_BROADWELL(dev_priv))
- pixclk = ilk_pipe_pixel_rate(crtc->config);
+ pixclk = ilk_pipe_pixel_rate(crtc_state);
else if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv))
- pixclk = crtc->config->base.adjusted_mode.crtc_clock;
+ pixclk = crtc_state->base.adjusted_mode.crtc_clock;
else
WARN_ON(dev_priv->display.modeset_calc_cdclk);
/* pixel rate mustn't exceed 95% of cdclk with IPS on BDW */
- if (IS_BROADWELL(dev_priv) && crtc->config->ips_enabled)
+ if (IS_BROADWELL(dev_priv) && crtc_state->ips_enabled)
pixclk = DIV_ROUND_UP(pixclk * 100, 95);
drm_calc_timestamping_constants(&crtc->base, &crtc->base.hwmode);
dev_priv->min_pixclk[crtc->pipe] = pixclk;
- intel_pipe_config_sanity_check(dev_priv, crtc->config);
+ intel_pipe_config_sanity_check(dev_priv, crtc_state);
}
}
if (ret)
DRM_ERROR("Restoring old state failed with %i\n", ret);
- drm_atomic_state_put(state);
+ if (state)
+ drm_atomic_state_put(state);
}
void intel_modeset_gem_init(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = to_i915(dev);
- struct drm_crtc *c;
- struct drm_i915_gem_object *obj;
intel_init_gt_powersave(dev_priv);
intel_modeset_init_hw(dev);
intel_setup_overlay(dev_priv);
-
- /*
- * Make sure any fbs we allocated at startup are properly
- * pinned & fenced. When we do the allocation it's too early
- * for this.
- */
- for_each_crtc(dev, c) {
- struct i915_vma *vma;
-
- obj = intel_fb_obj(c->primary->fb);
- if (obj == NULL)
- continue;
-
- mutex_lock(&dev->struct_mutex);
- vma = intel_pin_and_fence_fb_obj(c->primary->fb,
- c->primary->state->rotation);
- mutex_unlock(&dev->struct_mutex);
- if (IS_ERR(vma)) {
- DRM_ERROR("failed to pin boot fb on pipe %d\n",
- to_intel_crtc(c)->pipe);
- drm_framebuffer_unreference(c->primary->fb);
- c->primary->fb = NULL;
- c->primary->crtc = c->primary->state->crtc = NULL;
- update_state_fb(c->primary);
- c->state->plane_mask &= ~(1 << drm_plane_index(c->primary));
- }
- }
}
int intel_connector_register(struct drm_connector *connector)
DP_LINK_STATUS_SIZE) == DP_LINK_STATUS_SIZE;
}
+static bool intel_dp_get_y_cord_status(struct intel_dp *intel_dp)
+{
+ uint8_t psr_caps = 0;
+
+ drm_dp_dpcd_readb(&intel_dp->aux, DP_PSR_CAPS, &psr_caps);
+ return psr_caps & DP_PSR2_SU_Y_COORDINATE_REQUIRED;
+}
+
+static bool intel_dp_get_colorimetry_status(struct intel_dp *intel_dp)
+{
+ uint8_t dprx = 0;
+
+ drm_dp_dpcd_readb(&intel_dp->aux,
+ DP_DPRX_FEATURE_ENUMERATION_LIST,
+ &dprx);
+ return dprx & DP_VSC_SDP_EXT_FOR_COLORIMETRY_SUPPORTED;
+}
+
+static bool intel_dp_get_alpm_status(struct intel_dp *intel_dp)
+{
+ uint8_t alpm_caps = 0;
+
+ drm_dp_dpcd_readb(&intel_dp->aux, DP_RECEIVER_ALPM_CAP, &alpm_caps);
+ return alpm_caps & DP_ALPM_CAP;
+}
+
/* These are source-specific values. */
uint8_t
intel_dp_voltage_max(struct intel_dp *intel_dp)
if (HAS_DDI(dev_priv)) {
signal_levels = ddi_signal_levels(intel_dp);
- if (IS_BROXTON(dev_priv))
+ if (IS_GEN9_LP(dev_priv))
signal_levels = 0;
else
mask = DDI_BUF_EMP_MASK;
dev_priv->psr.psr2_support = dev_priv->psr.aux_frame_sync;
DRM_DEBUG_KMS("PSR2 %s on sink",
dev_priv->psr.psr2_support ? "supported" : "not supported");
+
+ if (dev_priv->psr.psr2_support) {
+ dev_priv->psr.y_cord_support =
+ intel_dp_get_y_cord_status(intel_dp);
+ dev_priv->psr.colorimetry_support =
+ intel_dp_get_colorimetry_status(intel_dp);
+ dev_priv->psr.alpm =
+ intel_dp_get_alpm_status(intel_dp);
+ }
+
}
/* Read the eDP Display control capabilities registers */
struct intel_plane_state {
struct drm_plane_state base;
struct drm_rect clip;
+ struct i915_vma *vma;
struct {
u32 offset;
struct work_struct mmio_work;
struct drm_crtc *crtc;
+ struct i915_vma *old_vma;
struct drm_framebuffer *old_fb;
struct drm_i915_gem_object *pending_flip_obj;
struct drm_pending_vblank_event *event;
struct drm_modeset_acquire_ctx *ctx);
struct i915_vma *
intel_pin_and_fence_fb_obj(struct drm_framebuffer *fb, unsigned int rotation);
-void intel_unpin_fb_obj(struct drm_framebuffer *fb, unsigned int rotation);
+void intel_unpin_fb_vma(struct i915_vma *vma);
struct drm_framebuffer *
__intel_framebuffer_create(struct drm_device *dev,
struct drm_mode_fb_cmd2 *mode_cmd,
int skl_update_scaler_crtc(struct intel_crtc_state *crtc_state);
int skl_max_scale(struct intel_crtc *crtc, struct intel_crtc_state *crtc_state);
-u32 intel_fb_gtt_offset(struct drm_framebuffer *fb, unsigned int rotation);
+static inline u32 intel_plane_ggtt_offset(const struct intel_plane_state *state)
+{
+ return i915_ggtt_offset(state->vma);
+}
u32 skl_plane_ctl_format(uint32_t pixel_format);
u32 skl_plane_ctl_tiling(uint64_t fb_modifier);
/* intel_pipe_crc.c */
int intel_pipe_crc_create(struct drm_minor *minor);
void intel_pipe_crc_cleanup(struct drm_minor *minor);
+#ifdef CONFIG_DEBUG_FS
+int intel_crtc_set_crc_source(struct drm_crtc *crtc, const char *source_name,
+ size_t *values_cnt);
+#else
+#define intel_crtc_set_crc_source NULL
+#endif
extern const struct file_operations i915_display_crc_ctl_fops;
#endif /* __INTEL_DRV_H__ */
return PTR_ERR(obj);
}
- vma = i915_vma_create(obj, &engine->i915->ggtt.base, NULL);
+ vma = i915_vma_instance(obj, &engine->i915->ggtt.base, NULL);
if (IS_ERR(vma)) {
ret = PTR_ERR(vma);
goto err_unref;
if (IS_I945GM(dev_priv))
fbc_ctl |= FBC_CTL_C3_IDLE; /* 945 needs special SR handling */
fbc_ctl |= (cfb_pitch & 0xff) << FBC_CTL_STRIDE_SHIFT;
- fbc_ctl |= params->fb.fence_reg;
+ fbc_ctl |= params->vma->fence->id;
I915_WRITE(FBC_CONTROL, fbc_ctl);
}
else
dpfc_ctl |= DPFC_CTL_LIMIT_1X;
- if (params->fb.fence_reg != I915_FENCE_REG_NONE) {
- dpfc_ctl |= DPFC_CTL_FENCE_EN | params->fb.fence_reg;
+ if (params->vma->fence) {
+ dpfc_ctl |= DPFC_CTL_FENCE_EN | params->vma->fence->id;
I915_WRITE(DPFC_FENCE_YOFF, params->crtc.fence_y_offset);
} else {
I915_WRITE(DPFC_FENCE_YOFF, 0);
break;
}
- if (params->fb.fence_reg != I915_FENCE_REG_NONE) {
+ if (params->vma->fence) {
dpfc_ctl |= DPFC_CTL_FENCE_EN;
if (IS_GEN5(dev_priv))
- dpfc_ctl |= params->fb.fence_reg;
+ dpfc_ctl |= params->vma->fence->id;
if (IS_GEN6(dev_priv)) {
I915_WRITE(SNB_DPFC_CTL_SA,
- SNB_CPU_FENCE_ENABLE | params->fb.fence_reg);
+ SNB_CPU_FENCE_ENABLE |
+ params->vma->fence->id);
I915_WRITE(DPFC_CPU_FENCE_OFFSET,
params->crtc.fence_y_offset);
}
}
I915_WRITE(ILK_DPFC_FENCE_YOFF, params->crtc.fence_y_offset);
- I915_WRITE(ILK_FBC_RT_BASE, params->fb.ggtt_offset | ILK_FBC_RT_VALID);
+ I915_WRITE(ILK_FBC_RT_BASE,
+ i915_ggtt_offset(params->vma) | ILK_FBC_RT_VALID);
/* enable it... */
I915_WRITE(ILK_DPFC_CONTROL, dpfc_ctl | DPFC_CTL_EN);
break;
}
- if (params->fb.fence_reg != I915_FENCE_REG_NONE) {
+ if (params->vma->fence) {
dpfc_ctl |= IVB_DPFC_CTL_FENCE_EN;
I915_WRITE(SNB_DPFC_CTL_SA,
- SNB_CPU_FENCE_ENABLE | params->fb.fence_reg);
+ SNB_CPU_FENCE_ENABLE |
+ params->vma->fence->id);
I915_WRITE(DPFC_CPU_FENCE_OFFSET, params->crtc.fence_y_offset);
} else {
I915_WRITE(SNB_DPFC_CTL_SA,0);
return effective_w <= max_w && effective_h <= max_h;
}
-/* XXX replace me when we have VMA tracking for intel_plane_state */
-static int get_fence_id(struct drm_framebuffer *fb)
-{
- struct i915_vma *vma = i915_gem_object_to_ggtt(intel_fb_obj(fb), NULL);
-
- return vma && vma->fence ? vma->fence->id : I915_FENCE_REG_NONE;
-}
-
static void intel_fbc_update_state_cache(struct intel_crtc *crtc,
struct intel_crtc_state *crtc_state,
struct intel_plane_state *plane_state)
struct intel_fbc *fbc = &dev_priv->fbc;
struct intel_fbc_state_cache *cache = &fbc->state_cache;
struct drm_framebuffer *fb = plane_state->base.fb;
- struct drm_i915_gem_object *obj;
+
+ cache->vma = NULL;
cache->crtc.mode_flags = crtc_state->base.adjusted_mode.flags;
if (IS_HASWELL(dev_priv) || IS_BROADWELL(dev_priv))
if (!cache->plane.visible)
return;
- obj = intel_fb_obj(fb);
-
- /* FIXME: We lack the proper locking here, so only run this on the
- * platforms that need. */
- if (IS_GEN(dev_priv, 5, 6))
- cache->fb.ilk_ggtt_offset = i915_gem_object_ggtt_offset(obj, NULL);
cache->fb.format = fb->format;
cache->fb.stride = fb->pitches[0];
- cache->fb.fence_reg = get_fence_id(fb);
- cache->fb.tiling_mode = i915_gem_object_get_tiling(obj);
+
+ cache->vma = plane_state->vma;
}
static bool intel_fbc_can_activate(struct intel_crtc *crtc)
return false;
}
- if (!cache->plane.visible) {
+ if (!cache->vma) {
fbc->no_fbc_reason = "primary plane not visible";
return false;
}
* so have no fence associated with it) due to aperture constaints
* at the time of pinning.
*/
- if (cache->fb.tiling_mode != I915_TILING_X ||
- cache->fb.fence_reg == I915_FENCE_REG_NONE) {
+ if (!cache->vma->fence) {
fbc->no_fbc_reason = "framebuffer not tiled or fenced";
return false;
}
* zero. */
memset(params, 0, sizeof(*params));
+ params->vma = cache->vma;
+
params->crtc.pipe = crtc->pipe;
params->crtc.plane = crtc->plane;
params->crtc.fence_y_offset = get_crtc_fence_y_offset(crtc);
params->fb.format = cache->fb.format;
params->fb.stride = cache->fb.stride;
- params->fb.fence_reg = cache->fb.fence_reg;
params->cfb_size = intel_fbc_calculate_cfb_size(dev_priv, cache);
-
- params->fb.ggtt_offset = cache->fb.ilk_ggtt_offset;
}
static bool intel_fbc_reg_params_equal(struct intel_fbc_reg_params *params1,
for_each_intel_crtc(&dev_priv->drm, crtc)
if (intel_crtc_active(crtc) &&
- to_intel_plane_state(crtc->base.primary->state)->base.visible)
+ crtc->base.primary->state->visible)
dev_priv->fbc.visible_pipes_mask |= (1 << crtc->pipe);
}
out_destroy_fbi:
drm_fb_helper_release_fbi(helper);
out_unpin:
- intel_unpin_fb_obj(&ifbdev->fb->base, DRM_ROTATE_0);
+ intel_unpin_fb_vma(vma);
out_unlock:
mutex_unlock(&dev->struct_mutex);
return ret;
if (ifbdev->fb) {
mutex_lock(&ifbdev->helper.dev->struct_mutex);
- intel_unpin_fb_obj(&ifbdev->fb->base, DRM_ROTATE_0);
+ intel_unpin_fb_vma(ifbdev->vma);
mutex_unlock(&ifbdev->helper.dev->struct_mutex);
drm_framebuffer_remove(&ifbdev->fb->base);
{
struct intel_fbdev *ifbdev = to_i915(dev)->fbdev;
+ if (!ifbdev)
+ return;
+
ifbdev->cookie = async_schedule(intel_fbdev_initial_config, ifbdev);
}
* The GuC firmware layout looks like this:
*
* +-------------------------------+
- * | guc_css_header |
+ * | uc_css_header |
* | |
* | contains major/minor version |
* +-------------------------------+
* 3. Length info of each component can be found in header, in dwords.
* 4. Modulus and exponent key are not required by driver. They may not appear
* in fw. So driver will load a truncated firmware in this case.
+ *
+ * HuC firmware layout is same as GuC firmware.
+ *
+ * HuC firmware css header is different. However, the only difference is where
+ * the version information is saved. The uc_css_header is unified to support
+ * both. Driver should get HuC version from uc_css_header.huc_sw_version, while
+ * uc_css_header.guc_sw_version for GuC.
*/
-struct guc_css_header {
+struct uc_css_header {
uint32_t module_type;
/* header_size includes all non-uCode bits, including css_header, rsa
* key, modulus key and exponent data. */
char username[8];
char buildnumber[12];
- uint32_t device_id;
- uint32_t guc_sw_version;
+ union {
+ struct {
+ uint32_t branch_client_version;
+ uint32_t sw_version;
+ } guc;
+ struct {
+ uint32_t sw_version;
+ uint32_t reserved;
+ } huc;
+ };
uint32_t prod_preprod_fw;
uint32_t reserved[12];
uint32_t header_info;
INTEL_GUC_ACTION_ENTER_S_STATE = 0x501,
INTEL_GUC_ACTION_EXIT_S_STATE = 0x502,
INTEL_GUC_ACTION_SLPC_REQUEST = 0x3003,
+ INTEL_GUC_ACTION_AUTHENTICATE_HUC = 0x4000,
INTEL_GUC_ACTION_UK_LOG_ENABLE_LOGGING = 0x0E000,
INTEL_GUC_ACTION_LIMIT
};
* 512K. In order to exclude 0-512K address space from GGTT, all gfx objects
* used by GuC is pinned with PIN_OFFSET_BIAS along with size of WOPCM.
*
- * Firmware log:
- * Firmware log is enabled by setting i915.guc_log_level to non-negative level.
- * Log data is printed out via reading debugfs i915_guc_log_dump. Reading from
- * i915_guc_load_status will print out firmware loading status and scratch
- * registers value.
- *
*/
#define SKL_FW_MAJOR 6
MODULE_FIRMWARE(I915_KBL_GUC_UCODE);
/* User-friendly representation of an enum */
-const char *intel_guc_fw_status_repr(enum intel_guc_fw_status status)
+const char *intel_uc_fw_status_repr(enum intel_uc_fw_status status)
{
switch (status) {
- case GUC_FIRMWARE_FAIL:
+ case INTEL_UC_FIRMWARE_FAIL:
return "FAIL";
- case GUC_FIRMWARE_NONE:
+ case INTEL_UC_FIRMWARE_NONE:
return "NONE";
- case GUC_FIRMWARE_PENDING:
+ case INTEL_UC_FIRMWARE_PENDING:
return "PENDING";
- case GUC_FIRMWARE_SUCCESS:
+ case INTEL_UC_FIRMWARE_SUCCESS:
return "SUCCESS";
default:
return "UNKNOWN!";
static int guc_ucode_xfer_dma(struct drm_i915_private *dev_priv,
struct i915_vma *vma)
{
- struct intel_guc_fw *guc_fw = &dev_priv->guc.guc_fw;
+ struct intel_uc_fw *guc_fw = &dev_priv->guc.fw;
unsigned long offset;
struct sg_table *sg = vma->pages;
u32 status, rsa[UOS_RSA_SCRATCH_MAX_COUNT];
return ret;
}
-static u32 guc_wopcm_size(struct drm_i915_private *dev_priv)
+u32 intel_guc_wopcm_size(struct drm_i915_private *dev_priv)
{
u32 wopcm_size = GUC_WOPCM_TOP;
/* On BXT, the top of WOPCM is reserved for RC6 context */
- if (IS_BROXTON(dev_priv))
+ if (IS_GEN9_LP(dev_priv))
wopcm_size -= BXT_GUC_WOPCM_RC6_RESERVED;
return wopcm_size;
*/
static int guc_ucode_xfer(struct drm_i915_private *dev_priv)
{
- struct intel_guc_fw *guc_fw = &dev_priv->guc.guc_fw;
+ struct intel_uc_fw *guc_fw = &dev_priv->guc.fw;
struct i915_vma *vma;
int ret;
- ret = i915_gem_object_set_to_gtt_domain(guc_fw->guc_fw_obj, false);
+ ret = i915_gem_object_set_to_gtt_domain(guc_fw->obj, false);
if (ret) {
DRM_DEBUG_DRIVER("set-domain failed %d\n", ret);
return ret;
}
- vma = i915_gem_object_ggtt_pin(guc_fw->guc_fw_obj, NULL, 0, 0, 0);
+ vma = i915_gem_object_ggtt_pin(guc_fw->obj, NULL, 0, 0,
+ PIN_OFFSET_BIAS | GUC_WOPCM_TOP);
if (IS_ERR(vma)) {
DRM_DEBUG_DRIVER("pin failed %d\n", (int)PTR_ERR(vma));
return PTR_ERR(vma);
}
- /* Invalidate GuC TLB to let GuC take the latest updates to GTT. */
- I915_WRITE(GEN8_GTCR, GEN8_GTCR_INVALIDATE);
-
intel_uncore_forcewake_get(dev_priv, FORCEWAKE_ALL);
/* init WOPCM */
- I915_WRITE(GUC_WOPCM_SIZE, guc_wopcm_size(dev_priv));
+ I915_WRITE(GUC_WOPCM_SIZE, intel_guc_wopcm_size(dev_priv));
I915_WRITE(DMA_GUC_WOPCM_OFFSET, GUC_WOPCM_OFFSET_VALUE);
/* Enable MIA caching. GuC clock gating is disabled. */
if (IS_BXT_REVID(dev_priv, 0, BXT_REVID_B0))
I915_WRITE(GEN6_GFXPAUSE, 0x30FFF);
- if (IS_BROXTON(dev_priv))
+ if (IS_GEN9_LP(dev_priv))
I915_WRITE(GEN9LP_GT_PM_CONFIG, GT_DOORBELL_ENABLE);
else
I915_WRITE(GEN9_GT_PM_CONFIG, GT_DOORBELL_ENABLE);
*/
int intel_guc_setup(struct drm_i915_private *dev_priv)
{
- struct intel_guc_fw *guc_fw = &dev_priv->guc.guc_fw;
- const char *fw_path = guc_fw->guc_fw_path;
+ struct intel_uc_fw *guc_fw = &dev_priv->guc.fw;
+ const char *fw_path = guc_fw->path;
int retries, ret, err;
DRM_DEBUG_DRIVER("GuC fw status: path %s, fetch %s, load %s\n",
fw_path,
- intel_guc_fw_status_repr(guc_fw->guc_fw_fetch_status),
- intel_guc_fw_status_repr(guc_fw->guc_fw_load_status));
+ intel_uc_fw_status_repr(guc_fw->fetch_status),
+ intel_uc_fw_status_repr(guc_fw->load_status));
/* Loading forbidden, or no firmware to load? */
if (!i915.enable_guc_loading) {
}
/* Fetch failed, or already fetched but failed to load? */
- if (guc_fw->guc_fw_fetch_status != GUC_FIRMWARE_SUCCESS) {
+ if (guc_fw->fetch_status != INTEL_UC_FIRMWARE_SUCCESS) {
err = -EIO;
goto fail;
- } else if (guc_fw->guc_fw_load_status == GUC_FIRMWARE_FAIL) {
+ } else if (guc_fw->load_status == INTEL_UC_FIRMWARE_FAIL) {
err = -ENOEXEC;
goto fail;
}
guc_interrupts_release(dev_priv);
gen9_reset_guc_interrupts(dev_priv);
- guc_fw->guc_fw_load_status = GUC_FIRMWARE_PENDING;
+ /* We need to notify the guc whenever we change the GGTT */
+ i915_ggtt_enable_guc(dev_priv);
+
+ guc_fw->load_status = INTEL_UC_FIRMWARE_PENDING;
DRM_DEBUG_DRIVER("GuC fw status: fetch %s, load %s\n",
- intel_guc_fw_status_repr(guc_fw->guc_fw_fetch_status),
- intel_guc_fw_status_repr(guc_fw->guc_fw_load_status));
+ intel_uc_fw_status_repr(guc_fw->fetch_status),
+ intel_uc_fw_status_repr(guc_fw->load_status));
err = i915_guc_submission_init(dev_priv);
if (err)
if (err)
goto fail;
+ intel_huc_load(dev_priv);
err = guc_ucode_xfer(dev_priv);
if (!err)
break;
"retry %d more time(s)\n", err, retries);
}
- guc_fw->guc_fw_load_status = GUC_FIRMWARE_SUCCESS;
+ guc_fw->load_status = INTEL_UC_FIRMWARE_SUCCESS;
DRM_DEBUG_DRIVER("GuC fw status: fetch %s, load %s\n",
- intel_guc_fw_status_repr(guc_fw->guc_fw_fetch_status),
- intel_guc_fw_status_repr(guc_fw->guc_fw_load_status));
+ intel_uc_fw_status_repr(guc_fw->fetch_status),
+ intel_uc_fw_status_repr(guc_fw->load_status));
+
+ intel_guc_auth_huc(dev_priv);
if (i915.enable_guc_submission) {
if (i915.guc_log_level >= 0)
return 0;
fail:
- if (guc_fw->guc_fw_load_status == GUC_FIRMWARE_PENDING)
- guc_fw->guc_fw_load_status = GUC_FIRMWARE_FAIL;
+ if (guc_fw->load_status == INTEL_UC_FIRMWARE_PENDING)
+ guc_fw->load_status = INTEL_UC_FIRMWARE_FAIL;
guc_interrupts_release(dev_priv);
i915_guc_submission_disable(dev_priv);
i915_guc_submission_fini(dev_priv);
+ i915_ggtt_disable_guc(dev_priv);
/*
* We've failed to load the firmware :(
return ret;
}
-static void guc_fw_fetch(struct drm_i915_private *dev_priv,
- struct intel_guc_fw *guc_fw)
+void intel_uc_fw_fetch(struct drm_i915_private *dev_priv,
+ struct intel_uc_fw *uc_fw)
{
struct pci_dev *pdev = dev_priv->drm.pdev;
struct drm_i915_gem_object *obj;
const struct firmware *fw = NULL;
- struct guc_css_header *css;
+ struct uc_css_header *css;
size_t size;
int err;
- DRM_DEBUG_DRIVER("before requesting firmware: GuC fw fetch status %s\n",
- intel_guc_fw_status_repr(guc_fw->guc_fw_fetch_status));
+ DRM_DEBUG_DRIVER("before requesting firmware: uC fw fetch status %s\n",
+ intel_uc_fw_status_repr(uc_fw->fetch_status));
- err = request_firmware(&fw, guc_fw->guc_fw_path, &pdev->dev);
+ err = request_firmware(&fw, uc_fw->path, &pdev->dev);
if (err)
goto fail;
if (!fw)
goto fail;
- DRM_DEBUG_DRIVER("fetch GuC fw from %s succeeded, fw %p\n",
- guc_fw->guc_fw_path, fw);
+ DRM_DEBUG_DRIVER("fetch uC fw from %s succeeded, fw %p\n",
+ uc_fw->path, fw);
/* Check the size of the blob before examining buffer contents */
- if (fw->size < sizeof(struct guc_css_header)) {
+ if (fw->size < sizeof(struct uc_css_header)) {
DRM_NOTE("Firmware header is missing\n");
goto fail;
}
- css = (struct guc_css_header *)fw->data;
+ css = (struct uc_css_header *)fw->data;
/* Firmware bits always start from header */
- guc_fw->header_offset = 0;
- guc_fw->header_size = (css->header_size_dw - css->modulus_size_dw -
+ uc_fw->header_offset = 0;
+ uc_fw->header_size = (css->header_size_dw - css->modulus_size_dw -
css->key_size_dw - css->exponent_size_dw) * sizeof(u32);
- if (guc_fw->header_size != sizeof(struct guc_css_header)) {
+ if (uc_fw->header_size != sizeof(struct uc_css_header)) {
DRM_NOTE("CSS header definition mismatch\n");
goto fail;
}
/* then, uCode */
- guc_fw->ucode_offset = guc_fw->header_offset + guc_fw->header_size;
- guc_fw->ucode_size = (css->size_dw - css->header_size_dw) * sizeof(u32);
+ uc_fw->ucode_offset = uc_fw->header_offset + uc_fw->header_size;
+ uc_fw->ucode_size = (css->size_dw - css->header_size_dw) * sizeof(u32);
/* now RSA */
if (css->key_size_dw != UOS_RSA_SCRATCH_MAX_COUNT) {
DRM_NOTE("RSA key size is bad\n");
goto fail;
}
- guc_fw->rsa_offset = guc_fw->ucode_offset + guc_fw->ucode_size;
- guc_fw->rsa_size = css->key_size_dw * sizeof(u32);
+ uc_fw->rsa_offset = uc_fw->ucode_offset + uc_fw->ucode_size;
+ uc_fw->rsa_size = css->key_size_dw * sizeof(u32);
/* At least, it should have header, uCode and RSA. Size of all three. */
- size = guc_fw->header_size + guc_fw->ucode_size + guc_fw->rsa_size;
+ size = uc_fw->header_size + uc_fw->ucode_size + uc_fw->rsa_size;
if (fw->size < size) {
DRM_NOTE("Missing firmware components\n");
goto fail;
}
- /* Header and uCode will be loaded to WOPCM. Size of the two. */
- size = guc_fw->header_size + guc_fw->ucode_size;
- if (size > guc_wopcm_size(dev_priv)) {
- DRM_NOTE("Firmware is too large to fit in WOPCM\n");
- goto fail;
- }
-
/*
* The GuC firmware image has the version number embedded at a well-known
* offset within the firmware blob; note that major / minor version are
* TWO bytes each (i.e. u16), although all pointers and offsets are defined
* in terms of bytes (u8).
*/
- guc_fw->guc_fw_major_found = css->guc_sw_version >> 16;
- guc_fw->guc_fw_minor_found = css->guc_sw_version & 0xFFFF;
-
- if (guc_fw->guc_fw_major_found != guc_fw->guc_fw_major_wanted ||
- guc_fw->guc_fw_minor_found < guc_fw->guc_fw_minor_wanted) {
- DRM_NOTE("GuC firmware version %d.%d, required %d.%d\n",
- guc_fw->guc_fw_major_found, guc_fw->guc_fw_minor_found,
- guc_fw->guc_fw_major_wanted, guc_fw->guc_fw_minor_wanted);
+ switch (uc_fw->fw) {
+ case INTEL_UC_FW_TYPE_GUC:
+ /* Header and uCode will be loaded to WOPCM. Size of the two. */
+ size = uc_fw->header_size + uc_fw->ucode_size;
+
+ /* Top 32k of WOPCM is reserved (8K stack + 24k RC6 context). */
+ if (size > intel_guc_wopcm_size(dev_priv)) {
+ DRM_ERROR("Firmware is too large to fit in WOPCM\n");
+ goto fail;
+ }
+ uc_fw->major_ver_found = css->guc.sw_version >> 16;
+ uc_fw->minor_ver_found = css->guc.sw_version & 0xFFFF;
+ break;
+
+ case INTEL_UC_FW_TYPE_HUC:
+ uc_fw->major_ver_found = css->huc.sw_version >> 16;
+ uc_fw->minor_ver_found = css->huc.sw_version & 0xFFFF;
+ break;
+
+ default:
+ DRM_ERROR("Unknown firmware type %d\n", uc_fw->fw);
+ err = -ENOEXEC;
+ goto fail;
+ }
+
+ if (uc_fw->major_ver_found != uc_fw->major_ver_wanted ||
+ uc_fw->minor_ver_found < uc_fw->minor_ver_wanted) {
+ DRM_NOTE("uC firmware version %d.%d, required %d.%d\n",
+ uc_fw->major_ver_found, uc_fw->minor_ver_found,
+ uc_fw->major_ver_wanted, uc_fw->minor_ver_wanted);
err = -ENOEXEC;
goto fail;
}
DRM_DEBUG_DRIVER("firmware version %d.%d OK (minimum %d.%d)\n",
- guc_fw->guc_fw_major_found, guc_fw->guc_fw_minor_found,
- guc_fw->guc_fw_major_wanted, guc_fw->guc_fw_minor_wanted);
+ uc_fw->major_ver_found, uc_fw->minor_ver_found,
+ uc_fw->major_ver_wanted, uc_fw->minor_ver_wanted);
mutex_lock(&dev_priv->drm.struct_mutex);
obj = i915_gem_object_create_from_data(dev_priv, fw->data, fw->size);
goto fail;
}
- guc_fw->guc_fw_obj = obj;
- guc_fw->guc_fw_size = fw->size;
+ uc_fw->obj = obj;
+ uc_fw->size = fw->size;
- DRM_DEBUG_DRIVER("GuC fw fetch status SUCCESS, obj %p\n",
- guc_fw->guc_fw_obj);
+ DRM_DEBUG_DRIVER("uC fw fetch status SUCCESS, obj %p\n",
+ uc_fw->obj);
release_firmware(fw);
- guc_fw->guc_fw_fetch_status = GUC_FIRMWARE_SUCCESS;
+ uc_fw->fetch_status = INTEL_UC_FIRMWARE_SUCCESS;
return;
fail:
- DRM_WARN("Failed to fetch valid GuC firmware from %s (error %d)\n",
- guc_fw->guc_fw_path, err);
- DRM_DEBUG_DRIVER("GuC fw fetch status FAIL; err %d, fw %p, obj %p\n",
- err, fw, guc_fw->guc_fw_obj);
+ DRM_WARN("Failed to fetch valid uC firmware from %s (error %d)\n",
+ uc_fw->path, err);
+ DRM_DEBUG_DRIVER("uC fw fetch status FAIL; err %d, fw %p, obj %p\n",
+ err, fw, uc_fw->obj);
mutex_lock(&dev_priv->drm.struct_mutex);
- obj = guc_fw->guc_fw_obj;
+ obj = uc_fw->obj;
if (obj)
i915_gem_object_put(obj);
- guc_fw->guc_fw_obj = NULL;
+ uc_fw->obj = NULL;
mutex_unlock(&dev_priv->drm.struct_mutex);
release_firmware(fw); /* OK even if fw is NULL */
- guc_fw->guc_fw_fetch_status = GUC_FIRMWARE_FAIL;
+ uc_fw->fetch_status = INTEL_UC_FIRMWARE_FAIL;
}
/**
*/
void intel_guc_init(struct drm_i915_private *dev_priv)
{
- struct intel_guc_fw *guc_fw = &dev_priv->guc.guc_fw;
+ struct intel_uc_fw *guc_fw = &dev_priv->guc.fw;
const char *fw_path;
if (!HAS_GUC(dev_priv)) {
fw_path = NULL;
} else if (IS_SKYLAKE(dev_priv)) {
fw_path = I915_SKL_GUC_UCODE;
- guc_fw->guc_fw_major_wanted = SKL_FW_MAJOR;
- guc_fw->guc_fw_minor_wanted = SKL_FW_MINOR;
+ guc_fw->major_ver_wanted = SKL_FW_MAJOR;
+ guc_fw->minor_ver_wanted = SKL_FW_MINOR;
} else if (IS_BROXTON(dev_priv)) {
fw_path = I915_BXT_GUC_UCODE;
- guc_fw->guc_fw_major_wanted = BXT_FW_MAJOR;
- guc_fw->guc_fw_minor_wanted = BXT_FW_MINOR;
+ guc_fw->major_ver_wanted = BXT_FW_MAJOR;
+ guc_fw->minor_ver_wanted = BXT_FW_MINOR;
} else if (IS_KABYLAKE(dev_priv)) {
fw_path = I915_KBL_GUC_UCODE;
- guc_fw->guc_fw_major_wanted = KBL_FW_MAJOR;
- guc_fw->guc_fw_minor_wanted = KBL_FW_MINOR;
+ guc_fw->major_ver_wanted = KBL_FW_MAJOR;
+ guc_fw->minor_ver_wanted = KBL_FW_MINOR;
} else {
fw_path = ""; /* unknown device */
}
- guc_fw->guc_fw_path = fw_path;
- guc_fw->guc_fw_fetch_status = GUC_FIRMWARE_NONE;
- guc_fw->guc_fw_load_status = GUC_FIRMWARE_NONE;
+ guc_fw->path = fw_path;
+ guc_fw->fetch_status = INTEL_UC_FIRMWARE_NONE;
+ guc_fw->load_status = INTEL_UC_FIRMWARE_NONE;
/* Early (and silent) return if GuC loading is disabled */
if (!i915.enable_guc_loading)
if (*fw_path == '\0')
return;
- guc_fw->guc_fw_fetch_status = GUC_FIRMWARE_PENDING;
+ guc_fw->fetch_status = INTEL_UC_FIRMWARE_PENDING;
DRM_DEBUG_DRIVER("GuC firmware pending, path %s\n", fw_path);
- guc_fw_fetch(dev_priv, guc_fw);
+ intel_uc_fw_fetch(dev_priv, guc_fw);
/* status must now be FAIL or SUCCESS */
}
*/
void intel_guc_fini(struct drm_i915_private *dev_priv)
{
- struct intel_guc_fw *guc_fw = &dev_priv->guc.guc_fw;
+ struct intel_uc_fw *guc_fw = &dev_priv->guc.fw;
mutex_lock(&dev_priv->drm.struct_mutex);
guc_interrupts_release(dev_priv);
i915_guc_submission_disable(dev_priv);
i915_guc_submission_fini(dev_priv);
- if (guc_fw->guc_fw_obj)
- i915_gem_object_put(guc_fw->guc_fw_obj);
- guc_fw->guc_fw_obj = NULL;
+ if (guc_fw->obj)
+ i915_gem_object_put(guc_fw->obj);
+ guc_fw->obj = NULL;
mutex_unlock(&dev_priv->drm.struct_mutex);
- guc_fw->guc_fw_fetch_status = GUC_FIRMWARE_NONE;
+ guc_fw->fetch_status = INTEL_UC_FIRMWARE_NONE;
}
--- /dev/null
+/*
+ * Copyright © 2014-2017 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+#include <linux/debugfs.h>
+#include <linux/relay.h>
+#include "i915_drv.h"
+
+static void guc_log_capture_logs(struct intel_guc *guc);
+
+/**
+ * DOC: GuC firmware log
+ *
+ * Firmware log is enabled by setting i915.guc_log_level to non-negative level.
+ * Log data is printed out via reading debugfs i915_guc_log_dump. Reading from
+ * i915_guc_load_status will print out firmware loading status and scratch
+ * registers value.
+ *
+ */
+
+static int guc_log_flush_complete(struct intel_guc *guc)
+{
+ u32 action[] = {
+ INTEL_GUC_ACTION_LOG_BUFFER_FILE_FLUSH_COMPLETE
+ };
+
+ return intel_guc_send(guc, action, ARRAY_SIZE(action));
+}
+
+static int guc_log_flush(struct intel_guc *guc)
+{
+ u32 action[] = {
+ INTEL_GUC_ACTION_FORCE_LOG_BUFFER_FLUSH,
+ 0
+ };
+
+ return intel_guc_send(guc, action, ARRAY_SIZE(action));
+}
+
+static int guc_log_control(struct intel_guc *guc, u32 control_val)
+{
+ u32 action[] = {
+ INTEL_GUC_ACTION_UK_LOG_ENABLE_LOGGING,
+ control_val
+ };
+
+ return intel_guc_send(guc, action, ARRAY_SIZE(action));
+}
+
+
+/*
+ * Sub buffer switch callback. Called whenever relay has to switch to a new
+ * sub buffer, relay stays on the same sub buffer if 0 is returned.
+ */
+static int subbuf_start_callback(struct rchan_buf *buf,
+ void *subbuf,
+ void *prev_subbuf,
+ size_t prev_padding)
+{
+ /* Use no-overwrite mode by default, where relay will stop accepting
+ * new data if there are no empty sub buffers left.
+ * There is no strict synchronization enforced by relay between Consumer
+ * and Producer. In overwrite mode, there is a possibility of getting
+ * inconsistent/garbled data, the producer could be writing on to the
+ * same sub buffer from which Consumer is reading. This can't be avoided
+ * unless Consumer is fast enough and can always run in tandem with
+ * Producer.
+ */
+ if (relay_buf_full(buf))
+ return 0;
+
+ return 1;
+}
+
+/*
+ * file_create() callback. Creates relay file in debugfs.
+ */
+static struct dentry *create_buf_file_callback(const char *filename,
+ struct dentry *parent,
+ umode_t mode,
+ struct rchan_buf *buf,
+ int *is_global)
+{
+ struct dentry *buf_file;
+
+ /* This to enable the use of a single buffer for the relay channel and
+ * correspondingly have a single file exposed to User, through which
+ * it can collect the logs in order without any post-processing.
+ * Need to set 'is_global' even if parent is NULL for early logging.
+ */
+ *is_global = 1;
+
+ if (!parent)
+ return NULL;
+
+ /* Not using the channel filename passed as an argument, since for each
+ * channel relay appends the corresponding CPU number to the filename
+ * passed in relay_open(). This should be fine as relay just needs a
+ * dentry of the file associated with the channel buffer and that file's
+ * name need not be same as the filename passed as an argument.
+ */
+ buf_file = debugfs_create_file("guc_log", mode,
+ parent, buf, &relay_file_operations);
+ return buf_file;
+}
+
+/*
+ * file_remove() default callback. Removes relay file in debugfs.
+ */
+static int remove_buf_file_callback(struct dentry *dentry)
+{
+ debugfs_remove(dentry);
+ return 0;
+}
+
+/* relay channel callbacks */
+static struct rchan_callbacks relay_callbacks = {
+ .subbuf_start = subbuf_start_callback,
+ .create_buf_file = create_buf_file_callback,
+ .remove_buf_file = remove_buf_file_callback,
+};
+
+static void guc_log_remove_relay_file(struct intel_guc *guc)
+{
+ relay_close(guc->log.relay_chan);
+}
+
+static int guc_log_create_relay_channel(struct intel_guc *guc)
+{
+ struct drm_i915_private *dev_priv = guc_to_i915(guc);
+ struct rchan *guc_log_relay_chan;
+ size_t n_subbufs, subbuf_size;
+
+ /* Keep the size of sub buffers same as shared log buffer */
+ subbuf_size = guc->log.vma->obj->base.size;
+
+ /* Store up to 8 snapshots, which is large enough to buffer sufficient
+ * boot time logs and provides enough leeway to User, in terms of
+ * latency, for consuming the logs from relay. Also doesn't take
+ * up too much memory.
+ */
+ n_subbufs = 8;
+
+ guc_log_relay_chan = relay_open(NULL, NULL, subbuf_size,
+ n_subbufs, &relay_callbacks, dev_priv);
+ if (!guc_log_relay_chan) {
+ DRM_ERROR("Couldn't create relay chan for GuC logging\n");
+ return -ENOMEM;
+ }
+
+ GEM_BUG_ON(guc_log_relay_chan->subbuf_size < subbuf_size);
+ guc->log.relay_chan = guc_log_relay_chan;
+ return 0;
+}
+
+static int guc_log_create_relay_file(struct intel_guc *guc)
+{
+ struct drm_i915_private *dev_priv = guc_to_i915(guc);
+ struct dentry *log_dir;
+ int ret;
+
+ /* For now create the log file in /sys/kernel/debug/dri/0 dir */
+ log_dir = dev_priv->drm.primary->debugfs_root;
+
+ /* If /sys/kernel/debug/dri/0 location do not exist, then debugfs is
+ * not mounted and so can't create the relay file.
+ * The relay API seems to fit well with debugfs only, for availing relay
+ * there are 3 requirements which can be met for debugfs file only in a
+ * straightforward/clean manner :-
+ * i) Need the associated dentry pointer of the file, while opening the
+ * relay channel.
+ * ii) Should be able to use 'relay_file_operations' fops for the file.
+ * iii) Set the 'i_private' field of file's inode to the pointer of
+ * relay channel buffer.
+ */
+ if (!log_dir) {
+ DRM_ERROR("Debugfs dir not available yet for GuC log file\n");
+ return -ENODEV;
+ }
+
+ ret = relay_late_setup_files(guc->log.relay_chan, "guc_log", log_dir);
+ if (ret) {
+ DRM_ERROR("Couldn't associate relay chan with file %d\n", ret);
+ return ret;
+ }
+
+ return 0;
+}
+
+static void guc_move_to_next_buf(struct intel_guc *guc)
+{
+ /* Make sure the updates made in the sub buffer are visible when
+ * Consumer sees the following update to offset inside the sub buffer.
+ */
+ smp_wmb();
+
+ /* All data has been written, so now move the offset of sub buffer. */
+ relay_reserve(guc->log.relay_chan, guc->log.vma->obj->base.size);
+
+ /* Switch to the next sub buffer */
+ relay_flush(guc->log.relay_chan);
+}
+
+static void *guc_get_write_buffer(struct intel_guc *guc)
+{
+ if (!guc->log.relay_chan)
+ return NULL;
+
+ /* Just get the base address of a new sub buffer and copy data into it
+ * ourselves. NULL will be returned in no-overwrite mode, if all sub
+ * buffers are full. Could have used the relay_write() to indirectly
+ * copy the data, but that would have been bit convoluted, as we need to
+ * write to only certain locations inside a sub buffer which cannot be
+ * done without using relay_reserve() along with relay_write(). So its
+ * better to use relay_reserve() alone.
+ */
+ return relay_reserve(guc->log.relay_chan, 0);
+}
+
+static bool guc_check_log_buf_overflow(struct intel_guc *guc,
+ enum guc_log_buffer_type type,
+ unsigned int full_cnt)
+{
+ unsigned int prev_full_cnt = guc->log.prev_overflow_count[type];
+ bool overflow = false;
+
+ if (full_cnt != prev_full_cnt) {
+ overflow = true;
+
+ guc->log.prev_overflow_count[type] = full_cnt;
+ guc->log.total_overflow_count[type] += full_cnt - prev_full_cnt;
+
+ if (full_cnt < prev_full_cnt) {
+ /* buffer_full_cnt is a 4 bit counter */
+ guc->log.total_overflow_count[type] += 16;
+ }
+ DRM_ERROR_RATELIMITED("GuC log buffer overflow\n");
+ }
+
+ return overflow;
+}
+
+static unsigned int guc_get_log_buffer_size(enum guc_log_buffer_type type)
+{
+ switch (type) {
+ case GUC_ISR_LOG_BUFFER:
+ return (GUC_LOG_ISR_PAGES + 1) * PAGE_SIZE;
+ case GUC_DPC_LOG_BUFFER:
+ return (GUC_LOG_DPC_PAGES + 1) * PAGE_SIZE;
+ case GUC_CRASH_DUMP_LOG_BUFFER:
+ return (GUC_LOG_CRASH_PAGES + 1) * PAGE_SIZE;
+ default:
+ MISSING_CASE(type);
+ }
+
+ return 0;
+}
+
+static void guc_read_update_log_buffer(struct intel_guc *guc)
+{
+ unsigned int buffer_size, read_offset, write_offset, bytes_to_copy, full_cnt;
+ struct guc_log_buffer_state *log_buf_state, *log_buf_snapshot_state;
+ struct guc_log_buffer_state log_buf_state_local;
+ enum guc_log_buffer_type type;
+ void *src_data, *dst_data;
+ bool new_overflow;
+
+ if (WARN_ON(!guc->log.buf_addr))
+ return;
+
+ /* Get the pointer to shared GuC log buffer */
+ log_buf_state = src_data = guc->log.buf_addr;
+
+ /* Get the pointer to local buffer to store the logs */
+ log_buf_snapshot_state = dst_data = guc_get_write_buffer(guc);
+
+ /* Actual logs are present from the 2nd page */
+ src_data += PAGE_SIZE;
+ dst_data += PAGE_SIZE;
+
+ for (type = GUC_ISR_LOG_BUFFER; type < GUC_MAX_LOG_BUFFER; type++) {
+ /* Make a copy of the state structure, inside GuC log buffer
+ * (which is uncached mapped), on the stack to avoid reading
+ * from it multiple times.
+ */
+ memcpy(&log_buf_state_local, log_buf_state,
+ sizeof(struct guc_log_buffer_state));
+ buffer_size = guc_get_log_buffer_size(type);
+ read_offset = log_buf_state_local.read_ptr;
+ write_offset = log_buf_state_local.sampled_write_ptr;
+ full_cnt = log_buf_state_local.buffer_full_cnt;
+
+ /* Bookkeeping stuff */
+ guc->log.flush_count[type] += log_buf_state_local.flush_to_file;
+ new_overflow = guc_check_log_buf_overflow(guc, type, full_cnt);
+
+ /* Update the state of shared log buffer */
+ log_buf_state->read_ptr = write_offset;
+ log_buf_state->flush_to_file = 0;
+ log_buf_state++;
+
+ if (unlikely(!log_buf_snapshot_state))
+ continue;
+
+ /* First copy the state structure in snapshot buffer */
+ memcpy(log_buf_snapshot_state, &log_buf_state_local,
+ sizeof(struct guc_log_buffer_state));
+
+ /* The write pointer could have been updated by GuC firmware,
+ * after sending the flush interrupt to Host, for consistency
+ * set write pointer value to same value of sampled_write_ptr
+ * in the snapshot buffer.
+ */
+ log_buf_snapshot_state->write_ptr = write_offset;
+ log_buf_snapshot_state++;
+
+ /* Now copy the actual logs. */
+ if (unlikely(new_overflow)) {
+ /* copy the whole buffer in case of overflow */
+ read_offset = 0;
+ write_offset = buffer_size;
+ } else if (unlikely((read_offset > buffer_size) ||
+ (write_offset > buffer_size))) {
+ DRM_ERROR("invalid log buffer state\n");
+ /* copy whole buffer as offsets are unreliable */
+ read_offset = 0;
+ write_offset = buffer_size;
+ }
+
+ /* Just copy the newly written data */
+ if (read_offset > write_offset) {
+ i915_memcpy_from_wc(dst_data, src_data, write_offset);
+ bytes_to_copy = buffer_size - read_offset;
+ } else {
+ bytes_to_copy = write_offset - read_offset;
+ }
+ i915_memcpy_from_wc(dst_data + read_offset,
+ src_data + read_offset, bytes_to_copy);
+
+ src_data += buffer_size;
+ dst_data += buffer_size;
+ }
+
+ if (log_buf_snapshot_state)
+ guc_move_to_next_buf(guc);
+ else {
+ /* Used rate limited to avoid deluge of messages, logs might be
+ * getting consumed by User at a slow rate.
+ */
+ DRM_ERROR_RATELIMITED("no sub-buffer to capture logs\n");
+ guc->log.capture_miss_count++;
+ }
+}
+
+static void guc_log_cleanup(struct intel_guc *guc)
+{
+ struct drm_i915_private *dev_priv = guc_to_i915(guc);
+
+ lockdep_assert_held(&dev_priv->drm.struct_mutex);
+
+ /* First disable the flush interrupt */
+ gen9_disable_guc_interrupts(dev_priv);
+
+ if (guc->log.flush_wq)
+ destroy_workqueue(guc->log.flush_wq);
+
+ guc->log.flush_wq = NULL;
+
+ if (guc->log.relay_chan)
+ guc_log_remove_relay_file(guc);
+
+ guc->log.relay_chan = NULL;
+
+ if (guc->log.buf_addr)
+ i915_gem_object_unpin_map(guc->log.vma->obj);
+
+ guc->log.buf_addr = NULL;
+}
+
+static void capture_logs_work(struct work_struct *work)
+{
+ struct intel_guc *guc =
+ container_of(work, struct intel_guc, log.flush_work);
+
+ guc_log_capture_logs(guc);
+}
+
+static int guc_log_create_extras(struct intel_guc *guc)
+{
+ struct drm_i915_private *dev_priv = guc_to_i915(guc);
+ void *vaddr;
+ int ret;
+
+ lockdep_assert_held(&dev_priv->drm.struct_mutex);
+
+ /* Nothing to do */
+ if (i915.guc_log_level < 0)
+ return 0;
+
+ if (!guc->log.buf_addr) {
+ /* Create a WC (Uncached for read) vmalloc mapping of log
+ * buffer pages, so that we can directly get the data
+ * (up-to-date) from memory.
+ */
+ vaddr = i915_gem_object_pin_map(guc->log.vma->obj, I915_MAP_WC);
+ if (IS_ERR(vaddr)) {
+ ret = PTR_ERR(vaddr);
+ DRM_ERROR("Couldn't map log buffer pages %d\n", ret);
+ return ret;
+ }
+
+ guc->log.buf_addr = vaddr;
+ }
+
+ if (!guc->log.relay_chan) {
+ /* Create a relay channel, so that we have buffers for storing
+ * the GuC firmware logs, the channel will be linked with a file
+ * later on when debugfs is registered.
+ */
+ ret = guc_log_create_relay_channel(guc);
+ if (ret)
+ return ret;
+ }
+
+ if (!guc->log.flush_wq) {
+ INIT_WORK(&guc->log.flush_work, capture_logs_work);
+
+ /*
+ * GuC log buffer flush work item has to do register access to
+ * send the ack to GuC and this work item, if not synced before
+ * suspend, can potentially get executed after the GFX device is
+ * suspended.
+ * By marking the WQ as freezable, we don't have to bother about
+ * flushing of this work item from the suspend hooks, the pending
+ * work item if any will be either executed before the suspend
+ * or scheduled later on resume. This way the handling of work
+ * item can be kept same between system suspend & rpm suspend.
+ */
+ guc->log.flush_wq = alloc_ordered_workqueue("i915-guc_log",
+ WQ_HIGHPRI | WQ_FREEZABLE);
+ if (guc->log.flush_wq == NULL) {
+ DRM_ERROR("Couldn't allocate the wq for GuC logging\n");
+ return -ENOMEM;
+ }
+ }
+
+ return 0;
+}
+
+void intel_guc_log_create(struct intel_guc *guc)
+{
+ struct i915_vma *vma;
+ unsigned long offset;
+ uint32_t size, flags;
+
+ if (i915.guc_log_level > GUC_LOG_VERBOSITY_MAX)
+ i915.guc_log_level = GUC_LOG_VERBOSITY_MAX;
+
+ /* The first page is to save log buffer state. Allocate one
+ * extra page for others in case for overlap */
+ size = (1 + GUC_LOG_DPC_PAGES + 1 +
+ GUC_LOG_ISR_PAGES + 1 +
+ GUC_LOG_CRASH_PAGES + 1) << PAGE_SHIFT;
+
+ vma = guc->log.vma;
+ if (!vma) {
+ /* We require SSE 4.1 for fast reads from the GuC log buffer and
+ * it should be present on the chipsets supporting GuC based
+ * submisssions.
+ */
+ if (WARN_ON(!i915_has_memcpy_from_wc())) {
+ /* logging will not be enabled */
+ i915.guc_log_level = -1;
+ return;
+ }
+
+ vma = intel_guc_allocate_vma(guc, size);
+ if (IS_ERR(vma)) {
+ /* logging will be off */
+ i915.guc_log_level = -1;
+ return;
+ }
+
+ guc->log.vma = vma;
+
+ if (guc_log_create_extras(guc)) {
+ guc_log_cleanup(guc);
+ i915_vma_unpin_and_release(&guc->log.vma);
+ i915.guc_log_level = -1;
+ return;
+ }
+ }
+
+ /* each allocated unit is a page */
+ flags = GUC_LOG_VALID | GUC_LOG_NOTIFY_ON_HALF_FULL |
+ (GUC_LOG_DPC_PAGES << GUC_LOG_DPC_SHIFT) |
+ (GUC_LOG_ISR_PAGES << GUC_LOG_ISR_SHIFT) |
+ (GUC_LOG_CRASH_PAGES << GUC_LOG_CRASH_SHIFT);
+
+ offset = guc_ggtt_offset(vma) >> PAGE_SHIFT; /* in pages */
+ guc->log.flags = (offset << GUC_LOG_BUF_ADDR_SHIFT) | flags;
+}
+
+static int guc_log_late_setup(struct intel_guc *guc)
+{
+ struct drm_i915_private *dev_priv = guc_to_i915(guc);
+ int ret;
+
+ lockdep_assert_held(&dev_priv->drm.struct_mutex);
+
+ if (i915.guc_log_level < 0)
+ return -EINVAL;
+
+ /* If log_level was set as -1 at boot time, then setup needed to
+ * handle log buffer flush interrupts would not have been done yet,
+ * so do that now.
+ */
+ ret = guc_log_create_extras(guc);
+ if (ret)
+ goto err;
+
+ ret = guc_log_create_relay_file(guc);
+ if (ret)
+ goto err;
+
+ return 0;
+err:
+ guc_log_cleanup(guc);
+ /* logging will remain off */
+ i915.guc_log_level = -1;
+ return ret;
+}
+
+static void guc_log_capture_logs(struct intel_guc *guc)
+{
+ struct drm_i915_private *dev_priv = guc_to_i915(guc);
+
+ guc_read_update_log_buffer(guc);
+
+ /* Generally device is expected to be active only at this
+ * time, so get/put should be really quick.
+ */
+ intel_runtime_pm_get(dev_priv);
+ guc_log_flush_complete(guc);
+ intel_runtime_pm_put(dev_priv);
+}
+
+static void guc_flush_logs(struct intel_guc *guc)
+{
+ struct drm_i915_private *dev_priv = guc_to_i915(guc);
+
+ if (!i915.enable_guc_submission || (i915.guc_log_level < 0))
+ return;
+
+ /* First disable the interrupts, will be renabled afterwards */
+ gen9_disable_guc_interrupts(dev_priv);
+
+ /* Before initiating the forceful flush, wait for any pending/ongoing
+ * flush to complete otherwise forceful flush may not actually happen.
+ */
+ flush_work(&guc->log.flush_work);
+
+ /* Ask GuC to update the log buffer state */
+ guc_log_flush(guc);
+
+ /* GuC would have updated log buffer by now, so capture it */
+ guc_log_capture_logs(guc);
+}
+
+int i915_guc_log_control(struct drm_i915_private *dev_priv, u64 control_val)
+{
+ struct intel_guc *guc = &dev_priv->guc;
+
+ union guc_log_control log_param;
+ int ret;
+
+ log_param.value = control_val;
+
+ if (log_param.verbosity < GUC_LOG_VERBOSITY_MIN ||
+ log_param.verbosity > GUC_LOG_VERBOSITY_MAX)
+ return -EINVAL;
+
+ /* This combination doesn't make sense & won't have any effect */
+ if (!log_param.logging_enabled && (i915.guc_log_level < 0))
+ return 0;
+
+ ret = guc_log_control(guc, log_param.value);
+ if (ret < 0) {
+ DRM_DEBUG_DRIVER("guc_logging_control action failed %d\n", ret);
+ return ret;
+ }
+
+ i915.guc_log_level = log_param.verbosity;
+
+ /* If log_level was set as -1 at boot time, then the relay channel file
+ * wouldn't have been created by now and interrupts also would not have
+ * been enabled.
+ */
+ if (!dev_priv->guc.log.relay_chan) {
+ ret = guc_log_late_setup(guc);
+ if (!ret)
+ gen9_enable_guc_interrupts(dev_priv);
+ } else if (!log_param.logging_enabled) {
+ /* Once logging is disabled, GuC won't generate logs & send an
+ * interrupt. But there could be some data in the log buffer
+ * which is yet to be captured. So request GuC to update the log
+ * buffer state and then collect the left over logs.
+ */
+ guc_flush_logs(guc);
+
+ /* As logging is disabled, update log level to reflect that */
+ i915.guc_log_level = -1;
+ } else {
+ /* In case interrupts were disabled, enable them now */
+ gen9_enable_guc_interrupts(dev_priv);
+ }
+
+ return ret;
+}
+
+void i915_guc_log_register(struct drm_i915_private *dev_priv)
+{
+ if (!i915.enable_guc_submission)
+ return;
+
+ mutex_lock(&dev_priv->drm.struct_mutex);
+ guc_log_late_setup(&dev_priv->guc);
+ mutex_unlock(&dev_priv->drm.struct_mutex);
+}
+
+void i915_guc_log_unregister(struct drm_i915_private *dev_priv)
+{
+ if (!i915.enable_guc_submission)
+ return;
+
+ mutex_lock(&dev_priv->drm.struct_mutex);
+ guc_log_cleanup(&dev_priv->guc);
+ mutex_unlock(&dev_priv->drm.struct_mutex);
+}
/* Enable polling and queue hotplug re-enabling. */
if (hpd_disabled) {
- drm_kms_helper_poll_enable_locked(dev);
+ drm_kms_helper_poll_enable(dev);
mod_delayed_work(system_wq, &dev_priv->hotplug.reenable_work,
msecs_to_jiffies(HPD_STORM_REENABLE_DELAY));
}
}
if (enabled)
- drm_kms_helper_poll_enable_locked(dev);
+ drm_kms_helper_poll_enable(dev);
mutex_unlock(&dev->mode_config.mutex);
--- /dev/null
+/*
+ * Copyright © 2016-2017 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+#include <linux/firmware.h>
+#include "i915_drv.h"
+#include "intel_uc.h"
+
+/**
+ * DOC: HuC Firmware
+ *
+ * Motivation:
+ * GEN9 introduces a new dedicated firmware for usage in media HEVC (High
+ * Efficiency Video Coding) operations. Userspace can use the firmware
+ * capabilities by adding HuC specific commands to batch buffers.
+ *
+ * Implementation:
+ * The same firmware loader is used as the GuC. However, the actual
+ * loading to HW is deferred until GEM initialization is done.
+ *
+ * Note that HuC firmware loading must be done before GuC loading.
+ */
+
+#define BXT_HUC_FW_MAJOR 01
+#define BXT_HUC_FW_MINOR 07
+#define BXT_BLD_NUM 1398
+
+#define SKL_HUC_FW_MAJOR 01
+#define SKL_HUC_FW_MINOR 07
+#define SKL_BLD_NUM 1398
+
+#define KBL_HUC_FW_MAJOR 02
+#define KBL_HUC_FW_MINOR 00
+#define KBL_BLD_NUM 1810
+
+#define HUC_FW_PATH(platform, major, minor, bld_num) \
+ "i915/" __stringify(platform) "_huc_ver" __stringify(major) "_" \
+ __stringify(minor) "_" __stringify(bld_num) ".bin"
+
+#define I915_SKL_HUC_UCODE HUC_FW_PATH(skl, SKL_HUC_FW_MAJOR, \
+ SKL_HUC_FW_MINOR, SKL_BLD_NUM)
+MODULE_FIRMWARE(I915_SKL_HUC_UCODE);
+
+#define I915_BXT_HUC_UCODE HUC_FW_PATH(bxt, BXT_HUC_FW_MAJOR, \
+ BXT_HUC_FW_MINOR, BXT_BLD_NUM)
+MODULE_FIRMWARE(I915_BXT_HUC_UCODE);
+
+#define I915_KBL_HUC_UCODE HUC_FW_PATH(kbl, KBL_HUC_FW_MAJOR, \
+ KBL_HUC_FW_MINOR, KBL_BLD_NUM)
+MODULE_FIRMWARE(I915_KBL_HUC_UCODE);
+
+/**
+ * huc_ucode_xfer() - DMA's the firmware
+ * @dev_priv: the drm_i915_private device
+ *
+ * Transfer the firmware image to RAM for execution by the microcontroller.
+ *
+ * Return: 0 on success, non-zero on failure
+ */
+static int huc_ucode_xfer(struct drm_i915_private *dev_priv)
+{
+ struct intel_uc_fw *huc_fw = &dev_priv->huc.fw;
+ struct i915_vma *vma;
+ unsigned long offset = 0;
+ u32 size;
+ int ret;
+
+ ret = i915_gem_object_set_to_gtt_domain(huc_fw->obj, false);
+ if (ret) {
+ DRM_DEBUG_DRIVER("set-domain failed %d\n", ret);
+ return ret;
+ }
+
+ vma = i915_gem_object_ggtt_pin(huc_fw->obj, NULL, 0, 0,
+ PIN_OFFSET_BIAS | GUC_WOPCM_TOP);
+ if (IS_ERR(vma)) {
+ DRM_DEBUG_DRIVER("pin failed %d\n", (int)PTR_ERR(vma));
+ return PTR_ERR(vma);
+ }
+
+ intel_uncore_forcewake_get(dev_priv, FORCEWAKE_ALL);
+
+ /* init WOPCM */
+ I915_WRITE(GUC_WOPCM_SIZE, intel_guc_wopcm_size(dev_priv));
+ I915_WRITE(DMA_GUC_WOPCM_OFFSET, GUC_WOPCM_OFFSET_VALUE |
+ HUC_LOADING_AGENT_GUC);
+
+ /* Set the source address for the uCode */
+ offset = guc_ggtt_offset(vma) + huc_fw->header_offset;
+ I915_WRITE(DMA_ADDR_0_LOW, lower_32_bits(offset));
+ I915_WRITE(DMA_ADDR_0_HIGH, upper_32_bits(offset) & 0xFFFF);
+
+ /* Hardware doesn't look at destination address for HuC. Set it to 0,
+ * but still program the correct address space.
+ */
+ I915_WRITE(DMA_ADDR_1_LOW, 0);
+ I915_WRITE(DMA_ADDR_1_HIGH, DMA_ADDRESS_SPACE_WOPCM);
+
+ size = huc_fw->header_size + huc_fw->ucode_size;
+ I915_WRITE(DMA_COPY_SIZE, size);
+
+ /* Start the DMA */
+ I915_WRITE(DMA_CTRL, _MASKED_BIT_ENABLE(HUC_UKERNEL | START_DMA));
+
+ /* Wait for DMA to finish */
+ ret = wait_for((I915_READ(DMA_CTRL) & START_DMA) == 0, 100);
+
+ DRM_DEBUG_DRIVER("HuC DMA transfer wait over with ret %d\n", ret);
+
+ /* Disable the bits once DMA is over */
+ I915_WRITE(DMA_CTRL, _MASKED_BIT_DISABLE(HUC_UKERNEL));
+
+ intel_uncore_forcewake_put(dev_priv, FORCEWAKE_ALL);
+
+ /*
+ * We keep the object pages for reuse during resume. But we can unpin it
+ * now that DMA has completed, so it doesn't continue to take up space.
+ */
+ i915_vma_unpin(vma);
+
+ return ret;
+}
+
+/**
+ * intel_huc_init() - initiate HuC firmware loading request
+ * @dev_priv: the drm_i915_private device
+ *
+ * Called early during driver load, but after GEM is initialised. The loading
+ * will continue only when driver explicitly specify firmware name and version.
+ * All other cases are considered as INTEL_UC_FIRMWARE_NONE either because HW
+ * is not capable or driver yet support it. And there will be no error message
+ * for INTEL_UC_FIRMWARE_NONE cases.
+ *
+ * The DMA-copying to HW is done later when intel_huc_load() is called.
+ */
+void intel_huc_init(struct drm_i915_private *dev_priv)
+{
+ struct intel_huc *huc = &dev_priv->huc;
+ struct intel_uc_fw *huc_fw = &huc->fw;
+ const char *fw_path = NULL;
+
+ huc_fw->path = NULL;
+ huc_fw->fetch_status = INTEL_UC_FIRMWARE_NONE;
+ huc_fw->load_status = INTEL_UC_FIRMWARE_NONE;
+ huc_fw->fw = INTEL_UC_FW_TYPE_HUC;
+
+ if (!HAS_HUC_UCODE(dev_priv))
+ return;
+
+ if (IS_SKYLAKE(dev_priv)) {
+ fw_path = I915_SKL_HUC_UCODE;
+ huc_fw->major_ver_wanted = SKL_HUC_FW_MAJOR;
+ huc_fw->minor_ver_wanted = SKL_HUC_FW_MINOR;
+ } else if (IS_BROXTON(dev_priv)) {
+ fw_path = I915_BXT_HUC_UCODE;
+ huc_fw->major_ver_wanted = BXT_HUC_FW_MAJOR;
+ huc_fw->minor_ver_wanted = BXT_HUC_FW_MINOR;
+ } else if (IS_KABYLAKE(dev_priv)) {
+ fw_path = I915_KBL_HUC_UCODE;
+ huc_fw->major_ver_wanted = KBL_HUC_FW_MAJOR;
+ huc_fw->minor_ver_wanted = KBL_HUC_FW_MINOR;
+ }
+
+ huc_fw->path = fw_path;
+ huc_fw->fetch_status = INTEL_UC_FIRMWARE_PENDING;
+
+ DRM_DEBUG_DRIVER("HuC firmware pending, path %s\n", fw_path);
+
+ WARN(huc_fw->path == NULL, "HuC present but no fw path\n");
+
+ intel_uc_fw_fetch(dev_priv, huc_fw);
+}
+
+/**
+ * intel_huc_load() - load HuC uCode to device
+ * @dev_priv: the drm_i915_private device
+ *
+ * Called from guc_setup() during driver loading and also after a GPU reset.
+ * Be note that HuC loading must be done before GuC loading.
+ *
+ * The firmware image should have already been fetched into memory by the
+ * earlier call to intel_huc_init(), so here we need only check that
+ * is succeeded, and then transfer the image to the h/w.
+ *
+ * Return: non-zero code on error
+ */
+int intel_huc_load(struct drm_i915_private *dev_priv)
+{
+ struct intel_uc_fw *huc_fw = &dev_priv->huc.fw;
+ int err;
+
+ if (huc_fw->fetch_status == INTEL_UC_FIRMWARE_NONE)
+ return 0;
+
+ DRM_DEBUG_DRIVER("%s fw status: fetch %s, load %s\n",
+ huc_fw->path,
+ intel_uc_fw_status_repr(huc_fw->fetch_status),
+ intel_uc_fw_status_repr(huc_fw->load_status));
+
+ if (huc_fw->fetch_status == INTEL_UC_FIRMWARE_SUCCESS &&
+ huc_fw->load_status == INTEL_UC_FIRMWARE_FAIL)
+ return -ENOEXEC;
+
+ huc_fw->load_status = INTEL_UC_FIRMWARE_PENDING;
+
+ switch (huc_fw->fetch_status) {
+ case INTEL_UC_FIRMWARE_FAIL:
+ /* something went wrong :( */
+ err = -EIO;
+ goto fail;
+
+ case INTEL_UC_FIRMWARE_NONE:
+ case INTEL_UC_FIRMWARE_PENDING:
+ default:
+ /* "can't happen" */
+ WARN_ONCE(1, "HuC fw %s invalid fetch_status %s [%d]\n",
+ huc_fw->path,
+ intel_uc_fw_status_repr(huc_fw->fetch_status),
+ huc_fw->fetch_status);
+ err = -ENXIO;
+ goto fail;
+
+ case INTEL_UC_FIRMWARE_SUCCESS:
+ break;
+ }
+
+ err = huc_ucode_xfer(dev_priv);
+ if (err)
+ goto fail;
+
+ huc_fw->load_status = INTEL_UC_FIRMWARE_SUCCESS;
+
+ DRM_DEBUG_DRIVER("%s fw status: fetch %s, load %s\n",
+ huc_fw->path,
+ intel_uc_fw_status_repr(huc_fw->fetch_status),
+ intel_uc_fw_status_repr(huc_fw->load_status));
+
+ return 0;
+
+fail:
+ if (huc_fw->load_status == INTEL_UC_FIRMWARE_PENDING)
+ huc_fw->load_status = INTEL_UC_FIRMWARE_FAIL;
+
+ DRM_ERROR("Failed to complete HuC uCode load with ret %d\n", err);
+
+ return err;
+}
+
+/**
+ * intel_huc_fini() - clean up resources allocated for HuC
+ * @dev_priv: the drm_i915_private device
+ *
+ * Cleans up by releasing the huc firmware GEM obj.
+ */
+void intel_huc_fini(struct drm_i915_private *dev_priv)
+{
+ struct intel_uc_fw *huc_fw = &dev_priv->huc.fw;
+
+ mutex_lock(&dev_priv->drm.struct_mutex);
+ if (huc_fw->obj)
+ i915_gem_object_put(huc_fw->obj);
+ huc_fw->obj = NULL;
+ mutex_unlock(&dev_priv->drm.struct_mutex);
+
+ huc_fw->fetch_status = INTEL_UC_FIRMWARE_NONE;
+}
+
+/**
+ * intel_guc_auth_huc() - authenticate ucode
+ * @dev_priv: the drm_i915_device
+ *
+ * Triggers a HuC fw authentication request to the GuC via intel_guc_action_
+ * authenticate_huc interface.
+ */
+void intel_guc_auth_huc(struct drm_i915_private *dev_priv)
+{
+ struct intel_guc *guc = &dev_priv->guc;
+ struct intel_huc *huc = &dev_priv->huc;
+ struct i915_vma *vma;
+ int ret;
+ u32 data[2];
+
+ if (huc->fw.load_status != INTEL_UC_FIRMWARE_SUCCESS)
+ return;
+
+ vma = i915_gem_object_ggtt_pin(huc->fw.obj, NULL, 0, 0,
+ PIN_OFFSET_BIAS | GUC_WOPCM_TOP);
+ if (IS_ERR(vma)) {
+ DRM_ERROR("failed to pin huc fw object %d\n",
+ (int)PTR_ERR(vma));
+ return;
+ }
+
+ /* Specify auth action and where public signature is. */
+ data[0] = INTEL_GUC_ACTION_AUTHENTICATE_HUC;
+ data[1] = guc_ggtt_offset(vma) + huc->fw.rsa_offset;
+
+ ret = intel_guc_send(guc, data, ARRAY_SIZE(data));
+ if (ret) {
+ DRM_ERROR("HuC: GuC did not ack Auth request %d\n", ret);
+ goto out;
+ }
+
+ /* Check authentication status, it should be done by now */
+ ret = intel_wait_for_register(dev_priv,
+ HUC_STATUS2,
+ HUC_FW_VERIFIED,
+ HUC_FW_VERIFIED,
+ 50);
+
+ if (ret) {
+ DRM_ERROR("HuC: Authentication failed %d\n", ret);
+ goto out;
+ }
+
+out:
+ i915_vma_unpin(vma);
+}
+
ce->state->obj->mm.dirty = true;
- /* Invalidate GuC TLB. */
- if (i915.enable_guc_submission) {
- struct drm_i915_private *dev_priv = ctx->i915;
- I915_WRITE(GEN8_GTCR, GEN8_GTCR_INVALIDATE);
- }
-
i915_gem_context_get(ctx);
return 0;
uint32_t *batch,
uint32_t index)
{
- struct drm_i915_private *dev_priv = engine->i915;
uint32_t l3sqc4_flush = (0x40400000 | GEN8_LQSC_FLUSH_COHERENT_LINES);
- /*
- * WaDisableLSQCROPERFforOCL:kbl
- * This WA is implemented in skl_init_clock_gating() but since
- * this batch updates GEN8_L3SQCREG4 with default value we need to
- * set this bit here to retain the WA during flush.
- */
- if (IS_KBL_REVID(dev_priv, 0, KBL_REVID_E0))
- l3sqc4_flush |= GEN8_LQSC_RO_PERF_DIS;
-
wa_ctx_emit(batch, index, (MI_STORE_REGISTER_MEM_GEN8 |
MI_SRM_LRM_GLOBAL_GTT));
wa_ctx_emit_reg(batch, index, GEN8_L3SQCREG4);
if (IS_ERR(obj))
return PTR_ERR(obj);
- vma = i915_vma_create(obj, &engine->i915->ggtt.base, NULL);
+ vma = i915_vma_instance(obj, &engine->i915->ggtt.base, NULL);
if (IS_ERR(vma)) {
err = PTR_ERR(vma);
goto err;
engine->emit_breadcrumb = gen8_emit_breadcrumb_render;
engine->emit_breadcrumb_sz = gen8_emit_breadcrumb_render_sz;
- ret = intel_engine_create_scratch(engine, 4096);
+ ret = intel_engine_create_scratch(engine, PAGE_SIZE);
if (ret)
return ret;
ASSIGN_CTX_REG(reg_state, CTX_PDP0_LDW, GEN8_RING_PDP_LDW(engine, 0),
0);
- if (USES_FULL_48BIT_PPGTT(ppgtt->base.dev)) {
+ if (ppgtt && USES_FULL_48BIT_PPGTT(ppgtt->base.dev)) {
/* 64b PPGTT (48bit canonical)
* PDP0_DESCRIPTOR contains the base address to PML4 and
* other PDP Descriptors are ignored.
*/
ASSIGN_CTX_PML4(ppgtt, reg_state);
- } else {
- /* 32b PPGTT
- * PDP*_DESCRIPTOR contains the base address of space supported.
- * With dynamic page allocation, PDPs may not be allocated at
- * this point. Point the unallocated PDPs to the scratch page
- */
- execlists_update_context_pdps(ppgtt, reg_state);
}
if (engine->id == RCS) {
WARN_ON(ce->state);
- context_size = round_up(intel_lr_context_size(engine), 4096);
+ context_size = round_up(intel_lr_context_size(engine),
+ I915_GTT_PAGE_SIZE);
/* One extra page as the sharing data between driver and GuC */
context_size += PAGE_SIZE * LRC_PPHWSP_PN;
return PTR_ERR(ctx_obj);
}
- vma = i915_vma_create(ctx_obj, &ctx->i915->ggtt.base, NULL);
+ vma = i915_vma_instance(ctx_obj, &ctx->i915->ggtt.base, NULL);
if (IS_ERR(vma)) {
ret = PTR_ERR(vma);
goto error_deref_obj;
#include "intel_ringbuffer.h"
-#define GEN8_LR_CONTEXT_ALIGN 4096
+#define GEN8_LR_CONTEXT_ALIGN I915_GTT_MIN_ALIGNMENT
/* Execlists regs */
#define RING_ELSP(engine) _MMIO((engine)->mmio_base + 0x230)
if (ret != 0)
return ret;
- vma = i915_gem_object_pin_to_display_plane(new_bo, 0,
- &i915_ggtt_view_normal);
+ vma = i915_gem_object_pin_to_display_plane(new_bo, 0, NULL);
if (IS_ERR(vma))
return PTR_ERR(vma);
state = drm_atomic_state_alloc(dev);
if (!state) {
ret = -ENOMEM;
- goto out;
+ goto unlock;
}
state->acquire_ctx = drm_modeset_legacy_acquire_ctx(&crtc->base);
pipe_config = intel_atomic_get_crtc_state(state, crtc);
if (IS_ERR(pipe_config)) {
ret = PTR_ERR(pipe_config);
- goto out;
+ goto put_state;
}
pipe_config->pch_pfit.force_thru = enable;
pipe_config->base.connectors_changed = true;
ret = drm_atomic_commit(state);
-out:
+
+put_state:
+ drm_atomic_state_put(state);
+unlock:
WARN(ret, "Toggling workaround to %i returns %i\n", enable, ret);
drm_modeset_unlock_all(dev);
- drm_atomic_state_put(state);
}
static int ivb_pipe_crc_ctl_reg(struct drm_i915_private *dev_priv,
return 0;
}
+static int get_new_crc_ctl_reg(struct drm_i915_private *dev_priv,
+ enum pipe pipe,
+ enum intel_pipe_crc_source *source, u32 *val)
+{
+ if (IS_GEN2(dev_priv))
+ return i8xx_pipe_crc_ctl_reg(source, val);
+ else if (INTEL_GEN(dev_priv) < 5)
+ return i9xx_pipe_crc_ctl_reg(dev_priv, pipe, source, val);
+ else if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv))
+ return vlv_pipe_crc_ctl_reg(dev_priv, pipe, source, val);
+ else if (IS_GEN5(dev_priv) || IS_GEN6(dev_priv))
+ return ilk_pipe_crc_ctl_reg(source, val);
+ else
+ return ivb_pipe_crc_ctl_reg(dev_priv, pipe, source, val);
+}
+
static int pipe_crc_set_source(struct drm_i915_private *dev_priv,
enum pipe pipe,
enum intel_pipe_crc_source source)
return -EIO;
}
- if (IS_GEN2(dev_priv))
- ret = i8xx_pipe_crc_ctl_reg(&source, &val);
- else if (INTEL_GEN(dev_priv) < 5)
- ret = i9xx_pipe_crc_ctl_reg(dev_priv, pipe, &source, &val);
- else if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv))
- ret = vlv_pipe_crc_ctl_reg(dev_priv, pipe, &source, &val);
- else if (IS_GEN5(dev_priv) || IS_GEN6(dev_priv))
- ret = ilk_pipe_crc_ctl_reg(&source, &val);
- else
- ret = ivb_pipe_crc_ctl_reg(dev_priv, pipe, &source, &val);
-
+ ret = get_new_crc_ctl_reg(dev_priv, pipe, &source, &val);
if (ret != 0)
goto out;
POSTING_READ(PIPE_CRC_CTL(pipe));
/* real source -> none transition */
- if (source == INTEL_PIPE_CRC_SOURCE_NONE) {
+ if (!source) {
struct intel_pipe_crc_entry *entries;
struct intel_crtc *crtc = intel_get_crtc_for_pipe(dev_priv,
pipe);
{
int i;
+ if (!buf) {
+ *s = INTEL_PIPE_CRC_SOURCE_NONE;
+ return 0;
+ }
+
for (i = 0; i < ARRAY_SIZE(pipe_crc_sources); i++)
if (!strcmp(buf, pipe_crc_sources[i])) {
*s = i;
drm_debugfs_remove_files(info_list, 1, minor);
}
}
+
+int intel_crtc_set_crc_source(struct drm_crtc *crtc, const char *source_name,
+ size_t *values_cnt)
+{
+ struct drm_i915_private *dev_priv = crtc->dev->dev_private;
+ struct intel_pipe_crc *pipe_crc = &dev_priv->pipe_crc[crtc->index];
+ struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
+ enum intel_display_power_domain power_domain;
+ enum intel_pipe_crc_source source;
+ u32 val = 0; /* shut up gcc */
+ int ret = 0;
+
+ if (display_crc_ctl_parse_source(source_name, &source) < 0) {
+ DRM_DEBUG_DRIVER("unknown source %s\n", source_name);
+ return -EINVAL;
+ }
+
+ power_domain = POWER_DOMAIN_PIPE(crtc->index);
+ if (!intel_display_power_get_if_enabled(dev_priv, power_domain)) {
+ DRM_DEBUG_KMS("Trying to capture CRC while pipe is off\n");
+ return -EIO;
+ }
+
+ ret = get_new_crc_ctl_reg(dev_priv, crtc->index, &source, &val);
+ if (ret != 0)
+ goto out;
+
+ if (source) {
+ /*
+ * When IPS gets enabled, the pipe CRC changes. Since IPS gets
+ * enabled and disabled dynamically based on package C states,
+ * user space can't make reliable use of the CRCs, so let's just
+ * completely disable it.
+ */
+ hsw_disable_ips(intel_crtc);
+ }
+
+ I915_WRITE(PIPE_CRC_CTL(crtc->index), val);
+ POSTING_READ(PIPE_CRC_CTL(crtc->index));
+
+ if (!source) {
+ if (IS_G4X(dev_priv))
+ g4x_undo_pipe_scramble_reset(dev_priv, crtc->index);
+ else if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv))
+ vlv_undo_pipe_scramble_reset(dev_priv, crtc->index);
+ else if (IS_HASWELL(dev_priv) && crtc->index == PIPE_A)
+ hsw_trans_edp_pipe_A_crc_wa(dev_priv, false);
+
+ hsw_enable_ips(intel_crtc);
+ }
+
+ pipe_crc->skipped = 0;
+ *values_cnt = 5;
+
+out:
+ intel_display_power_put(dev_priv, power_domain);
+
+ return ret;
+}
static void skl_psr_setup_su_vsc(struct intel_dp *intel_dp)
{
struct edp_vsc_psr psr_vsc;
+ struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);
+ struct drm_device *dev = intel_dig_port->base.base.dev;
+ struct drm_i915_private *dev_priv = to_i915(dev);
/* Prepare VSC Header for SU as per EDP 1.4 spec, Table 6.11 */
memset(&psr_vsc, 0, sizeof(psr_vsc));
psr_vsc.sdp_header.HB0 = 0;
psr_vsc.sdp_header.HB1 = 0x7;
- psr_vsc.sdp_header.HB2 = 0x3;
- psr_vsc.sdp_header.HB3 = 0xb;
+ if (dev_priv->psr.colorimetry_support &&
+ dev_priv->psr.y_cord_support) {
+ psr_vsc.sdp_header.HB2 = 0x5;
+ psr_vsc.sdp_header.HB3 = 0x13;
+ } else if (dev_priv->psr.y_cord_support) {
+ psr_vsc.sdp_header.HB2 = 0x4;
+ psr_vsc.sdp_header.HB3 = 0xe;
+ } else {
+ psr_vsc.sdp_header.HB2 = 0x3;
+ psr_vsc.sdp_header.HB3 = 0xc;
+ }
+
intel_psr_write_vsc(intel_dp, &psr_vsc);
}
drm_dp_dpcd_writeb(&intel_dp->aux,
DP_SINK_DEVICE_AUX_FRAME_SYNC_CONF,
DP_AUX_FRAME_SYNC_ENABLE);
-
+ /* Enable ALPM at sink for psr2 */
+ if (dev_priv->psr.psr2_support && dev_priv->psr.alpm)
+ drm_dp_dpcd_writeb(&intel_dp->aux,
+ DP_RECEIVER_ALPM_CONFIG,
+ DP_ALPM_ENABLE);
if (dev_priv->psr.link_standby)
drm_dp_dpcd_writeb(&intel_dp->aux, DP_PSR_EN_CFG,
DP_PSR_ENABLE | DP_PSR_MAIN_LINK_ACTIVE);
VLV_EDP_PSR_ACTIVE_ENTRY);
}
-static void hsw_psr_enable_source(struct intel_dp *intel_dp)
+static void intel_enable_source_psr1(struct intel_dp *intel_dp)
{
struct intel_digital_port *dig_port = dp_to_dig_port(intel_dp);
struct drm_device *dev = dig_port->base.base.dev;
val |= EDP_PSR_TP1_TP2_SEL;
I915_WRITE(EDP_PSR_CTL, val);
+}
- if (!dev_priv->psr.psr2_support)
- return;
+static void intel_enable_source_psr2(struct intel_dp *intel_dp)
+{
+ struct intel_digital_port *dig_port = dp_to_dig_port(intel_dp);
+ struct drm_device *dev = dig_port->base.base.dev;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ /*
+ * Let's respect VBT in case VBT asks a higher idle_frame value.
+ * Let's use 6 as the minimum to cover all known cases including
+ * the off-by-one issue that HW has in some cases. Also there are
+ * cases where sink should be able to train
+ * with the 5 or 6 idle patterns.
+ */
+ uint32_t idle_frames = max(6, dev_priv->vbt.psr.idle_frames);
+ uint32_t val;
+
+ val = idle_frames << EDP_PSR_IDLE_FRAME_SHIFT;
/* FIXME: selective update is probably totally broken because it doesn't
* mesh at all with our frontbuffer tracking. And the hw alone isn't
* good enough. */
- val = EDP_PSR2_ENABLE | EDP_SU_TRACK_ENABLE;
+ val |= EDP_PSR2_ENABLE |
+ EDP_SU_TRACK_ENABLE |
+ EDP_FRAMES_BEFORE_SU_ENTRY;
if (dev_priv->vbt.psr.tp2_tp3_wakeup_time > 5)
val |= EDP_PSR2_TP2_TIME_2500;
I915_WRITE(EDP_PSR2_CTL, val);
}
+static void hsw_psr_enable_source(struct intel_dp *intel_dp)
+{
+ struct intel_digital_port *dig_port = dp_to_dig_port(intel_dp);
+ struct drm_device *dev = dig_port->base.base.dev;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+
+ /* psr1 and psr2 are mutually exclusive.*/
+ if (dev_priv->psr.psr2_support)
+ intel_enable_source_psr2(intel_dp);
+ else
+ intel_enable_source_psr1(intel_dp);
+}
+
static bool intel_psr_match_conditions(struct intel_dp *intel_dp)
{
struct intel_digital_port *dig_port = dp_to_dig_port(intel_dp);
return false;
}
+ /* PSR2 is restricted to work with panel resolutions upto 3200x2000 */
+ if (intel_crtc->config->pipe_src_w > 3200 ||
+ intel_crtc->config->pipe_src_h > 2000) {
+ dev_priv->psr.psr2_support = false;
+ return false;
+ }
+
+ /*
+ * FIXME:enable psr2 only for y-cordinate psr2 panels
+ * After gtc implementation , remove this restriction.
+ */
+ if (!dev_priv->psr.y_cord_support && dev_priv->psr.psr2_support) {
+ DRM_DEBUG_KMS("PSR2 disabled, panel does not support Y coordinate\n");
+ return false;
+ }
+
dev_priv->psr.source_ok = true;
return true;
}
struct drm_device *dev = intel_dig_port->base.base.dev;
struct drm_i915_private *dev_priv = to_i915(dev);
- WARN_ON(I915_READ(EDP_PSR_CTL) & EDP_PSR_ENABLE);
+ if (dev_priv->psr.psr2_support)
+ WARN_ON(I915_READ(EDP_PSR2_CTL) & EDP_PSR2_ENABLE);
+ else
+ WARN_ON(I915_READ(EDP_PSR_CTL) & EDP_PSR_ENABLE);
WARN_ON(dev_priv->psr.active);
lockdep_assert_held(&dev_priv->psr.lock);
struct drm_device *dev = intel_dig_port->base.base.dev;
struct drm_i915_private *dev_priv = to_i915(dev);
struct intel_crtc *crtc = to_intel_crtc(intel_dig_port->base.base.crtc);
+ enum transcoder cpu_transcoder = crtc->config->cpu_transcoder;
+ u32 chicken;
if (!HAS_PSR(dev_priv)) {
DRM_DEBUG_KMS("PSR not supported on this platform\n");
dev_priv->psr.busy_frontbuffer_bits = 0;
if (HAS_DDI(dev_priv)) {
- hsw_psr_setup_vsc(intel_dp);
-
if (dev_priv->psr.psr2_support) {
- /* PSR2 is restricted to work with panel resolutions upto 3200x2000 */
- if (crtc->config->pipe_src_w > 3200 ||
- crtc->config->pipe_src_h > 2000)
- dev_priv->psr.psr2_support = false;
- else
- skl_psr_setup_su_vsc(intel_dp);
+ skl_psr_setup_su_vsc(intel_dp);
+ chicken = PSR2_VSC_ENABLE_PROG_HEADER;
+ if (dev_priv->psr.y_cord_support)
+ chicken |= PSR2_ADD_VERTICAL_LINE_COUNT;
+ I915_WRITE(CHICKEN_TRANS(cpu_transcoder), chicken);
+ I915_WRITE(EDP_PSR_DEBUG_CTL,
+ EDP_PSR_DEBUG_MASK_MEMUP |
+ EDP_PSR_DEBUG_MASK_HPD |
+ EDP_PSR_DEBUG_MASK_LPSP |
+ EDP_PSR_DEBUG_MASK_MAX_SLEEP |
+ EDP_PSR_DEBUG_MASK_DISP_REG_WRITE);
+ } else {
+ /* set up vsc header for psr1 */
+ hsw_psr_setup_vsc(intel_dp);
+ /*
+ * Per Spec: Avoid continuous PSR exit by masking MEMUP
+ * and HPD. also mask LPSP to avoid dependency on other
+ * drivers that might block runtime_pm besides
+ * preventing other hw tracking issues now we can rely
+ * on frontbuffer tracking.
+ */
+ I915_WRITE(EDP_PSR_DEBUG_CTL,
+ EDP_PSR_DEBUG_MASK_MEMUP |
+ EDP_PSR_DEBUG_MASK_HPD |
+ EDP_PSR_DEBUG_MASK_LPSP);
}
- /*
- * Per Spec: Avoid continuous PSR exit by masking MEMUP and HPD.
- * Also mask LPSP to avoid dependency on other drivers that
- * might block runtime_pm besides preventing other hw tracking
- * issues now we can rely on frontbuffer tracking.
- */
- I915_WRITE(EDP_PSR_DEBUG_CTL, EDP_PSR_DEBUG_MASK_MEMUP |
- EDP_PSR_DEBUG_MASK_HPD | EDP_PSR_DEBUG_MASK_LPSP);
-
/* Enable PSR on the panel */
hsw_psr_enable_sink(intel_dp);
struct drm_i915_private *dev_priv = to_i915(dev);
if (dev_priv->psr.active) {
- I915_WRITE(EDP_PSR_CTL,
- I915_READ(EDP_PSR_CTL) & ~EDP_PSR_ENABLE);
+ i915_reg_t psr_ctl;
+ u32 psr_status_mask;
+
+ if (dev_priv->psr.aux_frame_sync)
+ drm_dp_dpcd_writeb(&intel_dp->aux,
+ DP_SINK_DEVICE_AUX_FRAME_SYNC_CONF,
+ 0);
+
+ if (dev_priv->psr.psr2_support) {
+ psr_ctl = EDP_PSR2_CTL;
+ psr_status_mask = EDP_PSR2_STATUS_STATE_MASK;
+
+ I915_WRITE(psr_ctl,
+ I915_READ(psr_ctl) &
+ ~(EDP_PSR2_ENABLE | EDP_SU_TRACK_ENABLE));
+
+ } else {
+ psr_ctl = EDP_PSR_STATUS_CTL;
+ psr_status_mask = EDP_PSR_STATUS_STATE_MASK;
+
+ I915_WRITE(psr_ctl,
+ I915_READ(psr_ctl) & ~EDP_PSR_ENABLE);
+ }
/* Wait till PSR is idle */
if (intel_wait_for_register(dev_priv,
- EDP_PSR_STATUS_CTL,
- EDP_PSR_STATUS_STATE_MASK,
- 0,
+ psr_ctl, psr_status_mask, 0,
2000))
DRM_ERROR("Timed out waiting for PSR Idle State\n");
dev_priv->psr.active = false;
} else {
- WARN_ON(I915_READ(EDP_PSR_CTL) & EDP_PSR_ENABLE);
+ if (dev_priv->psr.psr2_support)
+ WARN_ON(I915_READ(EDP_PSR2_CTL) & EDP_PSR2_ENABLE);
+ else
+ WARN_ON(I915_READ(EDP_PSR_CTL) & EDP_PSR_ENABLE);
}
}
* and be ready for re-enable.
*/
if (HAS_DDI(dev_priv)) {
- if (intel_wait_for_register(dev_priv,
- EDP_PSR_STATUS_CTL,
- EDP_PSR_STATUS_STATE_MASK,
- 0,
- 50)) {
- DRM_ERROR("Timed out waiting for PSR Idle for re-enable\n");
- return;
+ if (dev_priv->psr.psr2_support) {
+ if (intel_wait_for_register(dev_priv,
+ EDP_PSR2_STATUS_CTL,
+ EDP_PSR2_STATUS_STATE_MASK,
+ 0,
+ 50)) {
+ DRM_ERROR("Timed out waiting for PSR2 Idle for re-enable\n");
+ return;
+ }
+ } else {
+ if (intel_wait_for_register(dev_priv,
+ EDP_PSR_STATUS_CTL,
+ EDP_PSR_STATUS_STATE_MASK,
+ 0,
+ 50)) {
+ DRM_ERROR("Timed out waiting for PSR Idle for re-enable\n");
+ return;
+ }
}
} else {
if (intel_wait_for_register(dev_priv,
return;
if (HAS_DDI(dev_priv)) {
- val = I915_READ(EDP_PSR_CTL);
-
- WARN_ON(!(val & EDP_PSR_ENABLE));
-
- I915_WRITE(EDP_PSR_CTL, val & ~EDP_PSR_ENABLE);
+ if (dev_priv->psr.aux_frame_sync)
+ drm_dp_dpcd_writeb(&intel_dp->aux,
+ DP_SINK_DEVICE_AUX_FRAME_SYNC_CONF,
+ 0);
+ if (dev_priv->psr.psr2_support) {
+ val = I915_READ(EDP_PSR2_CTL);
+ WARN_ON(!(val & EDP_PSR2_ENABLE));
+ I915_WRITE(EDP_PSR2_CTL, val & ~EDP_PSR2_ENABLE);
+ } else {
+ val = I915_READ(EDP_PSR_CTL);
+ WARN_ON(!(val & EDP_PSR_ENABLE));
+ I915_WRITE(EDP_PSR_CTL, val & ~EDP_PSR_ENABLE);
+ }
} else {
val = I915_READ(VLV_PSRCTL(pipe));
WA_SET_BIT_MASKED(HDC_CHICKEN0,
HDC_FENCE_DEST_SLM_DISABLE);
- /* GEN8_L3SQCREG4 has a dependency with WA batch so any new changes
- * involving this register should also be added to WA batch as required.
- */
- if (IS_KBL_REVID(dev_priv, 0, KBL_REVID_E0))
- /* WaDisableLSQCROPERFforOCL:kbl */
- I915_WRITE(GEN8_L3SQCREG4, I915_READ(GEN8_L3SQCREG4) |
- GEN8_LQSC_RO_PERF_DIS);
-
/* WaToEnableHwFixForPushConstHWBug:kbl */
if (IS_KBL_REVID(dev_priv, KBL_REVID_C0, REVID_FOREVER))
WA_SET_BIT_MASKED(COMMON_SLICE_CHICKEN2,
void *vaddr;
int ret;
- obj = i915_gem_object_create_internal(engine->i915, 4096);
+ obj = i915_gem_object_create_internal(engine->i915, PAGE_SIZE);
if (IS_ERR(obj)) {
DRM_ERROR("Failed to allocate status page\n");
return PTR_ERR(obj);
if (ret)
goto err;
- vma = i915_vma_create(obj, &engine->i915->ggtt.base, NULL);
+ vma = i915_vma_instance(obj, &engine->i915->ggtt.base, NULL);
if (IS_ERR(vma)) {
ret = PTR_ERR(vma);
goto err;
engine->status_page.vma = vma;
engine->status_page.ggtt_offset = i915_ggtt_offset(vma);
- engine->status_page.page_addr = memset(vaddr, 0, 4096);
+ engine->status_page.page_addr = memset(vaddr, 0, PAGE_SIZE);
DRM_DEBUG_DRIVER("%s hws offset: 0x%08x\n",
engine->name, i915_ggtt_offset(vma));
/* mark ring buffers as read-only from GPU side by default */
obj->gt_ro = 1;
- vma = i915_vma_create(obj, &dev_priv->ggtt.base, NULL);
+ vma = i915_vma_instance(obj, &dev_priv->ggtt.base, NULL);
if (IS_ERR(vma))
goto err;
}
/* Ring wraparound at offset 0 sometimes hangs. No idea why. */
- ret = intel_ring_pin(ring, 4096);
+ ret = intel_ring_pin(ring, I915_GTT_PAGE_SIZE);
if (ret) {
intel_ring_free(ring);
goto error;
if (INTEL_GEN(dev_priv) >= 8 && !dev_priv->semaphore) {
struct i915_vma *vma;
- obj = i915_gem_object_create(dev_priv, 4096);
+ obj = i915_gem_object_create(dev_priv, PAGE_SIZE);
if (IS_ERR(obj))
goto err;
- vma = i915_vma_create(obj, &dev_priv->ggtt.base, NULL);
+ vma = i915_vma_instance(obj, &dev_priv->ggtt.base, NULL);
if (IS_ERR(vma))
goto err_obj;
return ret;
if (INTEL_GEN(dev_priv) >= 6) {
- ret = intel_engine_create_scratch(engine, 4096);
+ ret = intel_engine_create_scratch(engine, PAGE_SIZE);
if (ret)
return ret;
} else if (HAS_BROKEN_CS_TLB(dev_priv)) {
I915_WRITE(PLANE_CTL(pipe, plane_id), plane_ctl);
I915_WRITE(PLANE_SURF(pipe, plane_id),
- intel_fb_gtt_offset(fb, rotation) + surf_addr);
+ intel_plane_ggtt_offset(plane_state) + surf_addr);
POSTING_READ(PLANE_SURF(pipe, plane_id));
}
I915_WRITE(SPSIZE(pipe, plane_id), (crtc_h << 16) | crtc_w);
I915_WRITE(SPCNTR(pipe, plane_id), sprctl);
I915_WRITE(SPSURF(pipe, plane_id),
- intel_fb_gtt_offset(fb, rotation) + sprsurf_offset);
+ intel_plane_ggtt_offset(plane_state) + sprsurf_offset);
POSTING_READ(SPSURF(pipe, plane_id));
}
I915_WRITE(SPRSCALE(pipe), sprscale);
I915_WRITE(SPRCTL(pipe), sprctl);
I915_WRITE(SPRSURF(pipe),
- intel_fb_gtt_offset(fb, rotation) + sprsurf_offset);
+ intel_plane_ggtt_offset(plane_state) + sprsurf_offset);
POSTING_READ(SPRSURF(pipe));
}
I915_WRITE(DVSSCALE(pipe), dvsscale);
I915_WRITE(DVSCNTR(pipe), dvscntr);
I915_WRITE(DVSSURF(pipe),
- intel_fb_gtt_offset(fb, rotation) + dvssurf_offset);
+ intel_plane_ggtt_offset(plane_state) + dvssurf_offset);
POSTING_READ(DVSSURF(pipe));
}
return intel_guc_send(guc, action, ARRAY_SIZE(action));
}
-int intel_guc_log_flush_complete(struct intel_guc *guc)
-{
- u32 action[] = { INTEL_GUC_ACTION_LOG_BUFFER_FILE_FLUSH_COMPLETE };
-
- return intel_guc_send(guc, action, ARRAY_SIZE(action));
-}
-
-int intel_guc_log_flush(struct intel_guc *guc)
-{
- u32 action[] = {
- INTEL_GUC_ACTION_FORCE_LOG_BUFFER_FLUSH,
- 0
- };
-
- return intel_guc_send(guc, action, ARRAY_SIZE(action));
-}
-
-int intel_guc_log_control(struct intel_guc *guc, u32 control_val)
-{
- u32 action[] = {
- INTEL_GUC_ACTION_UK_LOG_ENABLE_LOGGING,
- control_val
- };
-
- return intel_guc_send(guc, action, ARRAY_SIZE(action));
-}
uint64_t submissions[I915_NUM_ENGINES];
};
-enum intel_guc_fw_status {
- GUC_FIRMWARE_FAIL = -1,
- GUC_FIRMWARE_NONE = 0,
- GUC_FIRMWARE_PENDING,
- GUC_FIRMWARE_SUCCESS
+enum intel_uc_fw_status {
+ INTEL_UC_FIRMWARE_FAIL = -1,
+ INTEL_UC_FIRMWARE_NONE = 0,
+ INTEL_UC_FIRMWARE_PENDING,
+ INTEL_UC_FIRMWARE_SUCCESS
+};
+
+enum intel_uc_fw_type {
+ INTEL_UC_FW_TYPE_GUC,
+ INTEL_UC_FW_TYPE_HUC
};
/*
* This structure encapsulates all the data needed during the process
* of fetching, caching, and loading the firmware image into the GuC.
*/
-struct intel_guc_fw {
- const char * guc_fw_path;
- size_t guc_fw_size;
- struct drm_i915_gem_object * guc_fw_obj;
- enum intel_guc_fw_status guc_fw_fetch_status;
- enum intel_guc_fw_status guc_fw_load_status;
-
- uint16_t guc_fw_major_wanted;
- uint16_t guc_fw_minor_wanted;
- uint16_t guc_fw_major_found;
- uint16_t guc_fw_minor_found;
-
+struct intel_uc_fw {
+ const char *path;
+ size_t size;
+ struct drm_i915_gem_object *obj;
+ enum intel_uc_fw_status fetch_status;
+ enum intel_uc_fw_status load_status;
+
+ uint16_t major_ver_wanted;
+ uint16_t minor_ver_wanted;
+ uint16_t major_ver_found;
+ uint16_t minor_ver_found;
+
+ enum intel_uc_fw_type fw;
uint32_t header_size;
uint32_t header_offset;
uint32_t rsa_size;
};
struct intel_guc {
- struct intel_guc_fw guc_fw;
+ struct intel_uc_fw fw;
struct intel_guc_log log;
/* intel_guc_recv interrupt related state */
struct mutex send_mutex;
};
+struct intel_huc {
+ /* Generic uC firmware management */
+ struct intel_uc_fw fw;
+
+ /* HuC-specific additions */
+};
+
/* intel_uc.c */
void intel_uc_init_early(struct drm_i915_private *dev_priv);
int intel_guc_send(struct intel_guc *guc, const u32 *action, u32 len);
int intel_guc_sample_forcewake(struct intel_guc *guc);
-int intel_guc_log_flush_complete(struct intel_guc *guc);
-int intel_guc_log_flush(struct intel_guc *guc);
-int intel_guc_log_control(struct intel_guc *guc, u32 control_val);
/* intel_guc_loader.c */
extern void intel_guc_init(struct drm_i915_private *dev_priv);
extern int intel_guc_setup(struct drm_i915_private *dev_priv);
extern void intel_guc_fini(struct drm_i915_private *dev_priv);
-extern const char *intel_guc_fw_status_repr(enum intel_guc_fw_status status);
+extern const char *intel_uc_fw_status_repr(enum intel_uc_fw_status status);
extern int intel_guc_suspend(struct drm_i915_private *dev_priv);
extern int intel_guc_resume(struct drm_i915_private *dev_priv);
+void intel_uc_fw_fetch(struct drm_i915_private *dev_priv,
+ struct intel_uc_fw *uc_fw);
+u32 intel_guc_wopcm_size(struct drm_i915_private *dev_priv);
/* i915_guc_submission.c */
int i915_guc_submission_init(struct drm_i915_private *dev_priv);
void i915_guc_wq_unreserve(struct drm_i915_gem_request *request);
void i915_guc_submission_disable(struct drm_i915_private *dev_priv);
void i915_guc_submission_fini(struct drm_i915_private *dev_priv);
-void i915_guc_capture_logs(struct drm_i915_private *dev_priv);
-void i915_guc_flush_logs(struct drm_i915_private *dev_priv);
-void i915_guc_register(struct drm_i915_private *dev_priv);
-void i915_guc_unregister(struct drm_i915_private *dev_priv);
+struct i915_vma *intel_guc_allocate_vma(struct intel_guc *guc, u32 size);
+
+/* intel_guc_log.c */
+void intel_guc_log_create(struct intel_guc *guc);
+void i915_guc_log_register(struct drm_i915_private *dev_priv);
+void i915_guc_log_unregister(struct drm_i915_private *dev_priv);
int i915_guc_log_control(struct drm_i915_private *dev_priv, u64 control_val);
static inline u32 guc_ggtt_offset(struct i915_vma *vma)
return offset;
}
+/* intel_huc.c */
+void intel_huc_init(struct drm_i915_private *dev_priv);
+void intel_huc_fini(struct drm_i915_private *dev_priv);
+int intel_huc_load(struct drm_i915_private *dev_priv);
+void intel_guc_auth_huc(struct drm_i915_private *dev_priv);
+
#endif
struct drm_crtc_state *crtc_state;
struct drm_rect clip = { 0, };
+ if (!state->crtc)
+ return 0;
+
crtc_state = drm_atomic_get_crtc_state(state->state, state->crtc);
if (IS_ERR(crtc_state))
return PTR_ERR(crtc_state);
* - TV Panel encoding via ENCT
*/
+/* HHI Registers */
+#define HHI_VDAC_CNTL0 0x2F4 /* 0xbd offset in data sheet */
+#define HHI_VDAC_CNTL1 0x2F8 /* 0xbe offset in data sheet */
+#define HHI_HDMI_PHY_CNTL0 0x3a0 /* 0xe8 offset in data sheet */
+
struct meson_cvbs_enci_mode meson_cvbs_enci_pal = {
.mode_tag = MESON_VENC_MODE_CVBS_PAL,
.hso_begin = 3,
void meson_venc_init(struct meson_drm *priv)
{
+ /* Disable CVBS VDAC */
+ regmap_write(priv->hhi, HHI_VDAC_CNTL0, 0);
+ regmap_write(priv->hhi, HHI_VDAC_CNTL1, 8);
+
+ /* Power Down Dacs */
+ writel_relaxed(0xff, priv->io_base + _REG(VENC_VDAC_SETTING));
+
+ /* Disable HDMI PHY */
+ regmap_write(priv->hhi, HHI_HDMI_PHY_CNTL0, 0);
+
+ /* Disable HDMI */
+ writel_bits_relaxed(0x3, 0,
+ priv->io_base + _REG(VPU_HDMI_SETTING));
+
/* Disable all encoders */
writel_relaxed(0, priv->io_base + _REG(ENCI_VIDEO_EN));
writel_relaxed(0, priv->io_base + _REG(ENCP_VIDEO_EN));
/* Disable CVBS VDAC */
regmap_write(priv->hhi, HHI_VDAC_CNTL0, 0);
- regmap_write(priv->hhi, HHI_VDAC_CNTL1, 0);
+ regmap_write(priv->hhi, HHI_VDAC_CNTL1, 8);
}
static void meson_venc_cvbs_encoder_enable(struct drm_encoder *encoder)
{ PCI_VENDOR_ID_MATROX, 0x533, PCI_ANY_ID, PCI_ANY_ID, 0, 0, G200_EH },
{ PCI_VENDOR_ID_MATROX, 0x534, PCI_ANY_ID, PCI_ANY_ID, 0, 0, G200_ER },
{ PCI_VENDOR_ID_MATROX, 0x536, PCI_ANY_ID, PCI_ANY_ID, 0, 0, G200_EW3 },
+ { PCI_VENDOR_ID_MATROX, 0x538, PCI_ANY_ID, PCI_ANY_ID, 0, 0, G200_EH3 },
{0,}
};
G200_WB,
G200_EV,
G200_EH,
+ G200_EH3,
G200_ER,
G200_EW3,
};
clock = 2;
break;
case G200_EH:
+ case G200_EH3:
case G200_ER:
data = 2;
clock = 1;
bool pll_locked = false;
m = n = p = 0;
- vcomax = 800000;
- vcomin = 400000;
- pllreffreq = 33333;
- delta = 0xffffffff;
+ if (mdev->type == G200_EH3) {
+ vcomax = 3000000;
+ vcomin = 1500000;
+ pllreffreq = 25000;
- for (testp = 16; testp > 0; testp >>= 1) {
- if (clock * testp > vcomax)
- continue;
- if (clock * testp < vcomin)
- continue;
+ delta = 0xffffffff;
- for (testm = 1; testm < 33; testm++) {
- for (testn = 17; testn < 257; testn++) {
- computed = (pllreffreq * testn) /
- (testm * testp);
+ testp = 0;
+
+ for (testm = 150; testm >= 6; testm--) {
+ if (clock * testm > vcomax)
+ continue;
+ if (clock * testm < vcomin)
+ continue;
+ for (testn = 120; testn >= 60; testn--) {
+ computed = (pllreffreq * testn) / testm;
if (computed > clock)
tmpdelta = computed - clock;
else
tmpdelta = clock - computed;
if (tmpdelta < delta) {
delta = tmpdelta;
- n = testn - 1;
- m = (testm - 1);
- p = testp - 1;
+ n = testn;
+ m = testm;
+ p = testp;
+ }
+ if (delta == 0)
+ break;
+ }
+ if (delta == 0)
+ break;
+ }
+ } else {
+
+ vcomax = 800000;
+ vcomin = 400000;
+ pllreffreq = 33333;
+
+ delta = 0xffffffff;
+
+ for (testp = 16; testp > 0; testp >>= 1) {
+ if (clock * testp > vcomax)
+ continue;
+ if (clock * testp < vcomin)
+ continue;
+
+ for (testm = 1; testm < 33; testm++) {
+ for (testn = 17; testn < 257; testn++) {
+ computed = (pllreffreq * testn) /
+ (testm * testp);
+ if (computed > clock)
+ tmpdelta = computed - clock;
+ else
+ tmpdelta = clock - computed;
+ if (tmpdelta < delta) {
+ delta = tmpdelta;
+ n = testn - 1;
+ m = (testm - 1);
+ p = testp - 1;
+ }
+ if ((clock * testp) >= 600000)
+ p |= 0x80;
}
- if ((clock * testp) >= 600000)
- p |= 0x80;
}
}
}
return mga_g200ev_set_plls(mdev, clock);
break;
case G200_EH:
+ case G200_EH3:
return mga_g200eh_set_plls(mdev, clock);
break;
case G200_ER:
option2 = 0x0000b000;
break;
case G200_EH:
+ case G200_EH3:
dacvalue[MGA1064_MISC_CTL] = MGA1064_MISC_CTL_VGA8 |
MGA1064_MISC_CTL_DAC_RAM_CS;
option = 0x00000120;
if ((mdev->type == G200_EV ||
mdev->type == G200_WB ||
mdev->type == G200_EH ||
- mdev->type == G200_EW3) &&
+ mdev->type == G200_EW3 ||
+ mdev->type == G200_EH3) &&
(i >= 0x44) && (i <= 0x4e))
continue;
void adreno_flush(struct msm_gpu *gpu)
{
struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu);
- uint32_t wptr = get_wptr(gpu->rb);
+ uint32_t wptr;
+
+ /*
+ * Mask wptr value that we calculate to fit in the HW range. This is
+ * to account for the possibility that the last command fit exactly into
+ * the ringbuffer and rb->next hasn't wrapped to zero yet
+ */
+ wptr = get_wptr(gpu->rb) & ((gpu->rb->size / 4) - 1);
/* ensure writes to ringbuffer have hit system memory: */
mb();
{
struct adreno_platform_config *config = pdev->dev.platform_data;
struct msm_gpu *gpu = &adreno_gpu->base;
- struct msm_mmu *mmu;
int ret;
adreno_gpu->funcs = funcs;
return ret;
}
- mmu = gpu->aspace->mmu;
- if (mmu) {
+ if (gpu->aspace && gpu->aspace->mmu) {
+ struct msm_mmu *mmu = gpu->aspace->mmu;
ret = mmu->funcs->attach(mmu, iommu_ports,
ARRAY_SIZE(iommu_ports));
if (ret)
static void mdp5_complete_commit(struct msm_kms *kms, struct drm_atomic_state *state)
{
- int i;
struct mdp5_kms *mdp5_kms = to_mdp5_kms(to_mdp_kms(kms));
- struct drm_plane *plane;
- struct drm_plane_state *plane_state;
-
- for_each_plane_in_state(state, plane, plane_state, i)
- mdp5_plane_complete_commit(plane, plane_state);
if (mdp5_kms->smp)
mdp5_smp_complete_commit(mdp5_kms->smp, &mdp5_kms->state->smp);
/* assigned by crtc blender */
enum mdp_mixer_stage_id stage;
-
- bool pending : 1;
};
#define to_mdp5_plane_state(x) \
container_of(x, struct mdp5_plane_state, base)
void mdp5_irq_domain_fini(struct mdp5_kms *mdp5_kms);
uint32_t mdp5_plane_get_flush(struct drm_plane *plane);
-void mdp5_plane_complete_commit(struct drm_plane *plane,
- struct drm_plane_state *state);
enum mdp5_pipe mdp5_plane_pipe(struct drm_plane *plane);
struct drm_plane *mdp5_plane_init(struct drm_device *dev, bool primary);
drm_printf(p, "\tzpos=%u\n", pstate->zpos);
drm_printf(p, "\talpha=%u\n", pstate->alpha);
drm_printf(p, "\tstage=%s\n", stage2name(pstate->stage));
- drm_printf(p, "\tpending=%u\n", pstate->pending);
}
static void mdp5_plane_reset(struct drm_plane *plane)
if (mdp5_state && mdp5_state->base.fb)
drm_framebuffer_reference(mdp5_state->base.fb);
- mdp5_state->pending = false;
-
return &mdp5_state->base;
}
DBG("%s: check (%d -> %d)", plane->name,
plane_enabled(old_state), plane_enabled(state));
- /* We don't allow faster-than-vblank updates.. if we did add this
- * some day, we would need to disallow in cases where hwpipe
- * changes
- */
- if (WARN_ON(to_mdp5_plane_state(old_state)->pending))
- return -EBUSY;
-
max_width = config->hw->lm.max_width << 16;
max_height = config->hw->lm.max_height << 16;
struct drm_plane_state *old_state)
{
struct drm_plane_state *state = plane->state;
- struct mdp5_plane_state *mdp5_state = to_mdp5_plane_state(state);
DBG("%s: update", plane->name);
- mdp5_state->pending = true;
-
if (plane_enabled(state)) {
int ret;
return pstate->hwpipe->flush_mask;
}
-/* called after vsync in thread context */
-void mdp5_plane_complete_commit(struct drm_plane *plane,
- struct drm_plane_state *state)
-{
- struct mdp5_plane_state *pstate = to_mdp5_plane_state(plane->state);
-
- pstate->pending = false;
-}
-
/* initialize plane */
struct drm_plane *mdp5_plane_init(struct drm_device *dev, bool primary)
{
WARN_ON(!mutex_is_locked(&dev->struct_mutex));
for (id = 0; id < ARRAY_SIZE(msm_obj->domain); id++) {
+ if (!priv->aspace[id])
+ continue;
msm_gem_unmap_vma(priv->aspace[id],
&msm_obj->domain[id], msm_obj->sgt);
}
pagefault_disable();
}
- if (submit_bo.flags & ~MSM_SUBMIT_BO_FLAGS) {
+ if ((submit_bo.flags & ~MSM_SUBMIT_BO_FLAGS) ||
+ !(submit_bo.flags & MSM_SUBMIT_BO_FLAGS)) {
DRM_ERROR("invalid flags: %x\n", submit_bo.flags);
ret = -EINVAL;
goto out_unlock;
{
uint32_t i, last_offset = 0;
uint32_t *ptr;
- int ret;
+ int ret = 0;
if (offset % 4) {
DRM_ERROR("non-aligned cmdstream buffer: %u\n", offset);
ret = copy_from_user(&submit_reloc, userptr, sizeof(submit_reloc));
if (ret)
- return -EFAULT;
+ goto out;
if (submit_reloc.submit_offset % 4) {
DRM_ERROR("non-aligned reloc offset: %u\n",
submit_reloc.submit_offset);
- return -EINVAL;
+ ret = -EINVAL;
+ goto out;
}
/* offset in dwords: */
if ((off >= (obj->base.size / 4)) ||
(off < last_offset)) {
DRM_ERROR("invalid offset %u at reloc %u\n", off, i);
- return -EINVAL;
+ ret = -EINVAL;
+ goto out;
}
ret = submit_bo(submit, submit_reloc.reloc_idx, NULL, &iova, &valid);
if (ret)
- return ret;
+ goto out;
if (valid)
continue;
last_offset = off;
}
+out:
msm_gem_put_vaddr_locked(&obj->base);
- return 0;
+ return ret;
}
static void submit_cleanup(struct msm_gem_submit *submit)
struct msm_ringbuffer *ring;
int ret;
- size = ALIGN(size, 4); /* size should be dword aligned */
+ if (WARN_ON(!is_power_of_2(size)))
+ return ERR_PTR(-EINVAL);
ring = kzalloc(sizeof(*ring), GFP_KERNEL);
if (!ring) {
dsicm_hw_reset(ddata);
if (ddata->use_dsi_backlight) {
- memset(&props, 0, sizeof(struct backlight_properties));
+ memset(&props, 0, sizeof(props));
props.max_brightness = 255;
props.type = BACKLIGHT_RAW;
module_spi_driver(acx565akm_driver);
+MODULE_ALIAS("spi:sony,acx565akm");
MODULE_AUTHOR("Nokia Corporation");
MODULE_DESCRIPTION("acx565akm LCD Driver");
MODULE_LICENSE("GPL");
return DISPC_IRQ_FRAMEDONEWB;
}
+void dispc_mgr_enable(enum omap_channel channel, bool enable)
+{
+ mgr_fld_write(channel, DISPC_MGR_FLD_ENABLE, enable);
+ /* flush posted write */
+ mgr_fld_read(channel, DISPC_MGR_FLD_ENABLE);
+}
+EXPORT_SYMBOL(dispc_mgr_enable);
+
+static bool dispc_mgr_is_enabled(enum omap_channel channel)
+{
+ return !!mgr_fld_read(channel, DISPC_MGR_FLD_ENABLE);
+}
+
bool dispc_mgr_go_busy(enum omap_channel channel)
{
return mgr_fld_read(channel, DISPC_MGR_FLD_GO) == 1;
}
EXPORT_SYMBOL(dispc_mgr_get_supported_outputs);
-void dispc_mgr_enable(enum omap_channel channel, bool enable)
-{
- mgr_fld_write(channel, DISPC_MGR_FLD_ENABLE, enable);
- /* flush posted write */
- mgr_fld_read(channel, DISPC_MGR_FLD_ENABLE);
-}
-EXPORT_SYMBOL(dispc_mgr_enable);
-
-bool dispc_mgr_is_enabled(enum omap_channel channel)
-{
- return !!mgr_fld_read(channel, DISPC_MGR_FLD_ENABLE);
-}
-EXPORT_SYMBOL(dispc_mgr_is_enabled);
-
void dispc_wb_enable(bool enable)
{
dispc_ovl_enable(OMAP_DSS_WB, enable);
wc = DIV_ROUND_UP(t->hact * t->bitspp, 8);
pps = DIV_ROUND_UP(wc + 6, t->ndl); /* pixel packet size */
- bl = t->hss + t->hsa + t->hse + t->hbp + t->hfront_porch;
+ bl = t->hss + t->hsa + t->hse + t->hbp + t->hfp;
tot = bl + pps;
#define TO_DSI_T(x) ((u32)div64_u64((u64)x * 1000000000llu, byteclk))
"%u/%u/%u/%u/%u/%u = %u + %u = %u\n",
str,
byteclk,
- t->hss, t->hsa, t->hse, t->hbp, pps, t->hfront_porch,
+ t->hss, t->hsa, t->hse, t->hbp, pps, t->hfp,
bl, pps, tot,
TO_DSI_T(t->hss),
TO_DSI_T(t->hsa),
TO_DSI_T(t->hse),
TO_DSI_T(t->hbp),
TO_DSI_T(pps),
- TO_DSI_T(t->hfront_porch),
+ TO_DSI_T(t->hfp),
TO_DSI_T(bl),
TO_DSI_T(pps),
int hact, bl, tot;
hact = vm->hactive;
- bl = vm->hsync_len + vm->hbp + vm->hfront_porch;
+ bl = vm->hsync_len + vm->hback_porch + vm->hfront_porch;
tot = hact + bl;
#define TO_DISPC_T(x) ((u32)div64_u64((u64)x * 1000000000llu, pck))
"%u/%u/%u/%u = %u + %u = %u\n",
str,
pck,
- vm->hsync_len, vm->hbp, hact, vm->hfront_porch,
+ vm->hsync_len, vm->hback_porch, hact, vm->hfront_porch,
bl, hact, tot,
TO_DISPC_T(vm->hsync_len),
- TO_DISPC_T(vm->hbp),
+ TO_DISPC_T(vm->hback_porch),
TO_DISPC_T(hact),
TO_DISPC_T(vm->hfront_porch),
TO_DISPC_T(bl),
dsi_tput = (u64)byteclk * t->ndl * 8;
pck = (u32)div64_u64(dsi_tput, t->bitspp);
dsi_hact = DIV_ROUND_UP(DIV_ROUND_UP(t->hact * t->bitspp, 8) + 6, t->ndl);
- dsi_htot = t->hss + t->hsa + t->hse + t->hbp + dsi_hact + t->hfront_porch;
+ dsi_htot = t->hss + t->hsa + t->hse + t->hbp + dsi_hact + t->hfp;
vm.pixelclock = pck;
vm.hsync_len = div64_u64((u64)(t->hsa + t->hse) * pck, byteclk);
- vm.hbp = div64_u64((u64)t->hbp * pck, byteclk);
- vm.hfront_porch = div64_u64((u64)t->hfront_porch * pck, byteclk);
+ vm.hback_porch = div64_u64((u64)t->hbp * pck, byteclk);
+ vm.hfront_porch = div64_u64((u64)t->hfp * pck, byteclk);
vm.hactive = t->hact;
print_dispc_vm(str, &vm);
static void __init omapdss_add_to_list(struct device_node *node, bool root)
{
- struct dss_conv_node *n = kmalloc(sizeof(struct dss_conv_node),
- GFP_KERNEL);
+ struct dss_conv_node *n = kmalloc(sizeof(*n), GFP_KERNEL);
if (n) {
n->node = node;
n->root = root;
void dispc_runtime_put(void);
void dispc_mgr_enable(enum omap_channel channel, bool enable);
-bool dispc_mgr_is_enabled(enum omap_channel channel);
u32 dispc_mgr_get_vsync_irq(enum omap_channel channel);
u32 dispc_mgr_get_framedone_irq(enum omap_channel channel);
u32 dispc_mgr_get_sync_lost_irq(enum omap_channel channel);
dssdrv->get_timings(dssdev, &t);
- if (memcmp(&vm, &t, sizeof(struct videomode)))
+ if (memcmp(&vm, &t, sizeof(vm)))
r = -EINVAL;
else
r = 0;
omap_dss_get_device(dssdev);
- omap_connector = kzalloc(sizeof(struct omap_connector), GFP_KERNEL);
+ omap_connector = kzalloc(sizeof(*omap_connector), GFP_KERNEL);
if (!omap_connector)
goto fail;
connector->interlace_allowed = 1;
connector->doublescan_allowed = 0;
- drm_connector_register(connector);
-
return connector;
fail:
struct videomode vm;
- struct omap_drm_irq vblank_irq;
- struct omap_drm_irq error_irq;
-
bool ignore_digit_sync_lost;
+ bool enabled;
bool pending;
wait_queue_head_t pending_wait;
+ struct drm_pending_vblank_event *event;
};
/* -----------------------------------------------------------------------------
* Helper Functions
*/
-uint32_t pipe2vbl(struct drm_crtc *crtc)
-{
- struct omap_crtc *omap_crtc = to_omap_crtc(crtc);
-
- return dispc_mgr_get_vsync_irq(omap_crtc->channel);
-}
-
struct videomode *omap_crtc_timings(struct drm_crtc *crtc)
{
struct omap_crtc *omap_crtc = to_omap_crtc(crtc);
return omap_crtc->channel;
}
+static bool omap_crtc_is_pending(struct drm_crtc *crtc)
+{
+ struct omap_crtc *omap_crtc = to_omap_crtc(crtc);
+ unsigned long flags;
+ bool pending;
+
+ spin_lock_irqsave(&crtc->dev->event_lock, flags);
+ pending = omap_crtc->pending;
+ spin_unlock_irqrestore(&crtc->dev->event_lock, flags);
+
+ return pending;
+}
+
int omap_crtc_wait_pending(struct drm_crtc *crtc)
{
struct omap_crtc *omap_crtc = to_omap_crtc(crtc);
* a single frame refresh even on slower displays.
*/
return wait_event_timeout(omap_crtc->pending_wait,
- !omap_crtc->pending,
+ !omap_crtc_is_pending(crtc),
msecs_to_jiffies(250));
}
u32 framedone_irq, vsync_irq;
int ret;
+ if (WARN_ON(omap_crtc->enabled == enable))
+ return;
+
if (omap_crtc_output[channel]->output_type == OMAP_DISPLAY_TYPE_HDMI) {
dispc_mgr_enable(channel, enable);
+ omap_crtc->enabled = enable;
return;
}
- if (dispc_mgr_is_enabled(channel) == enable)
- return;
-
if (omap_crtc->channel == OMAP_DSS_CHANNEL_DIGIT) {
/*
* Digit output produces some sync lost interrupts during the
}
dispc_mgr_enable(channel, enable);
+ omap_crtc->enabled = enable;
ret = omap_irq_wait(dev, wait, msecs_to_jiffies(100));
if (ret) {
* Setup, Flush and Page Flip
*/
-static void omap_crtc_complete_page_flip(struct drm_crtc *crtc)
+void omap_crtc_error_irq(struct drm_crtc *crtc, uint32_t irqstatus)
{
- struct drm_pending_vblank_event *event;
- struct drm_device *dev = crtc->dev;
- unsigned long flags;
-
- event = crtc->state->event;
-
- if (!event)
- return;
-
- spin_lock_irqsave(&dev->event_lock, flags);
- drm_crtc_send_vblank_event(crtc, event);
- spin_unlock_irqrestore(&dev->event_lock, flags);
-}
-
-static void omap_crtc_error_irq(struct omap_drm_irq *irq, uint32_t irqstatus)
-{
- struct omap_crtc *omap_crtc =
- container_of(irq, struct omap_crtc, error_irq);
+ struct omap_crtc *omap_crtc = to_omap_crtc(crtc);
if (omap_crtc->ignore_digit_sync_lost) {
irqstatus &= ~DISPC_IRQ_SYNC_LOST_DIGIT;
DRM_ERROR_RATELIMITED("%s: errors: %08x\n", omap_crtc->name, irqstatus);
}
-static void omap_crtc_vblank_irq(struct omap_drm_irq *irq, uint32_t irqstatus)
+void omap_crtc_vblank_irq(struct drm_crtc *crtc)
{
- struct omap_crtc *omap_crtc =
- container_of(irq, struct omap_crtc, vblank_irq);
- struct drm_device *dev = omap_crtc->base.dev;
+ struct omap_crtc *omap_crtc = to_omap_crtc(crtc);
+ bool pending;
- if (dispc_mgr_go_busy(omap_crtc->channel))
+ spin_lock(&crtc->dev->event_lock);
+ /*
+ * If the dispc is busy we're racing the flush operation. Try again on
+ * the next vblank interrupt.
+ */
+ if (dispc_mgr_go_busy(omap_crtc->channel)) {
+ spin_unlock(&crtc->dev->event_lock);
return;
+ }
- DBG("%s: apply done", omap_crtc->name);
-
- __omap_irq_unregister(dev, &omap_crtc->vblank_irq);
+ /* Send the vblank event if one has been requested. */
+ if (omap_crtc->event) {
+ drm_crtc_send_vblank_event(crtc, omap_crtc->event);
+ omap_crtc->event = NULL;
+ }
- rmb();
- WARN_ON(!omap_crtc->pending);
+ pending = omap_crtc->pending;
omap_crtc->pending = false;
- wmb();
+ spin_unlock(&crtc->dev->event_lock);
- /* wake up userspace */
- omap_crtc_complete_page_flip(&omap_crtc->base);
+ if (pending)
+ drm_crtc_vblank_put(crtc);
- /* wake up omap_atomic_complete */
+ /* Wake up omap_atomic_complete. */
wake_up(&omap_crtc->pending_wait);
+
+ DBG("%s: apply done", omap_crtc->name);
}
/* -----------------------------------------------------------------------------
DBG("%s", omap_crtc->name);
- WARN_ON(omap_crtc->vblank_irq.registered);
- omap_irq_unregister(crtc->dev, &omap_crtc->error_irq);
-
drm_crtc_cleanup(crtc);
kfree(omap_crtc);
static void omap_crtc_enable(struct drm_crtc *crtc)
{
struct omap_crtc *omap_crtc = to_omap_crtc(crtc);
+ int ret;
DBG("%s", omap_crtc->name);
- rmb();
+ spin_lock_irq(&crtc->dev->event_lock);
+ drm_crtc_vblank_on(crtc);
+ ret = drm_crtc_vblank_get(crtc);
+ WARN_ON(ret != 0);
+
WARN_ON(omap_crtc->pending);
omap_crtc->pending = true;
- wmb();
-
- omap_irq_register(crtc->dev, &omap_crtc->vblank_irq);
-
- drm_crtc_vblank_on(crtc);
+ spin_unlock_irq(&crtc->dev->event_lock);
}
static void omap_crtc_disable(struct drm_crtc *crtc)
}
static void omap_crtc_atomic_begin(struct drm_crtc *crtc,
- struct drm_crtc_state *old_crtc_state)
+ struct drm_crtc_state *old_crtc_state)
{
}
static void omap_crtc_atomic_flush(struct drm_crtc *crtc,
- struct drm_crtc_state *old_crtc_state)
+ struct drm_crtc_state *old_crtc_state)
{
struct omap_crtc *omap_crtc = to_omap_crtc(crtc);
-
- WARN_ON(omap_crtc->vblank_irq.registered);
+ int ret;
if (crtc->state->color_mgmt_changed) {
struct drm_color_lut *lut = NULL;
dispc_mgr_set_gamma(omap_crtc->channel, lut, length);
}
- if (dispc_mgr_is_enabled(omap_crtc->channel)) {
+ /*
+ * Only flush the CRTC if it is currently enabled. CRTCs that require a
+ * mode set are disabled prior plane updates and enabled afterwards.
+ * They are thus not active (regardless of what their CRTC core state
+ * reports) and the DRM core could thus call this function even though
+ * the CRTC is currently disabled. Do nothing in that case.
+ */
+ if (!omap_crtc->enabled)
+ return;
+
+ DBG("%s: GO", omap_crtc->name);
- DBG("%s: GO", omap_crtc->name);
+ ret = drm_crtc_vblank_get(crtc);
+ WARN_ON(ret != 0);
- rmb();
- WARN_ON(omap_crtc->pending);
- omap_crtc->pending = true;
- wmb();
+ spin_lock_irq(&crtc->dev->event_lock);
+ dispc_mgr_go(omap_crtc->channel);
- dispc_mgr_go(omap_crtc->channel);
- omap_irq_register(crtc->dev, &omap_crtc->vblank_irq);
- }
+ WARN_ON(omap_crtc->pending);
+ omap_crtc->pending = true;
+
+ if (crtc->state->event)
+ omap_crtc->event = crtc->state->event;
+ spin_unlock_irq(&crtc->dev->event_lock);
}
static bool omap_crtc_is_plane_prop(struct drm_crtc *crtc,
omap_crtc->channel = channel;
omap_crtc->name = channel_names[channel];
- omap_crtc->vblank_irq.irqmask = pipe2vbl(crtc);
- omap_crtc->vblank_irq.irq = omap_crtc_vblank_irq;
-
- omap_crtc->error_irq.irqmask =
- dispc_mgr_get_sync_lost_irq(channel);
- omap_crtc->error_irq.irq = omap_crtc_error_irq;
- omap_irq_register(dev, &omap_crtc->error_irq);
-
ret = drm_crtc_init_with_planes(dev, crtc, plane, NULL,
&omap_crtc_funcs, NULL);
if (ret < 0) {
int rows = (1 + area->y1 - area->y0);
int i = columns*rows;
- pat = alloc_dma(txn, sizeof(struct pat), &pat_pa);
+ pat = alloc_dma(txn, sizeof(*pat), &pat_pa);
if (txn->last_pat)
txn->last_pat->next_pa = (uint32_t)pat_pa;
/* alloc engines */
omap_dmm->engines = kcalloc(omap_dmm->num_engines,
- sizeof(struct refill_engine), GFP_KERNEL);
+ sizeof(*omap_dmm->engines), GFP_KERNEL);
if (!omap_dmm->engines) {
ret = -ENOMEM;
goto fail;
dispc_runtime_get();
drm_atomic_helper_commit_modeset_disables(dev, old_state);
- drm_atomic_helper_commit_planes(dev, old_state, 0);
+ drm_atomic_helper_commit_planes(dev, old_state,
+ DRM_PLANE_COMMIT_ACTIVE_ONLY);
drm_atomic_helper_commit_modeset_enables(dev, old_state);
omap_atomic_wait_for_completion(dev, old_state);
drm_mode_config_init(dev);
- omap_drm_irq_install(dev);
-
ret = omap_modeset_init_properties(dev);
if (ret < 0)
return ret;
drm_mode_config_reset(dev);
- return 0;
-}
+ omap_drm_irq_install(dev);
-static void omap_modeset_free(struct drm_device *dev)
-{
- drm_mode_config_cleanup(dev);
+ return 0;
}
/*
* drm driver funcs
*/
-/**
- * load - setup chip and create an initial config
- * @dev: DRM device
- * @flags: startup flags
- *
- * The driver load routine has to do several things:
- * - initialize the memory manager
- * - allocate initial config memory
- * - setup the DRM framebuffer with the allocated memory
- */
-static int dev_load(struct drm_device *dev, unsigned long flags)
-{
- struct omap_drm_platform_data *pdata = dev->dev->platform_data;
- struct omap_drm_private *priv;
- unsigned int i;
- int ret;
-
- DBG("load: dev=%p", dev);
-
- priv = kzalloc(sizeof(*priv), GFP_KERNEL);
- if (!priv)
- return -ENOMEM;
-
- priv->omaprev = pdata->omaprev;
-
- dev->dev_private = priv;
-
- priv->wq = alloc_ordered_workqueue("omapdrm", 0);
- init_waitqueue_head(&priv->commit.wait);
- spin_lock_init(&priv->commit.lock);
-
- spin_lock_init(&priv->list_lock);
- INIT_LIST_HEAD(&priv->obj_list);
-
- omap_gem_init(dev);
-
- ret = omap_modeset_init(dev);
- if (ret) {
- dev_err(dev->dev, "omap_modeset_init failed: ret=%d\n", ret);
- dev->dev_private = NULL;
- kfree(priv);
- return ret;
- }
-
- /* Initialize vblank handling, start with all CRTCs disabled. */
- ret = drm_vblank_init(dev, priv->num_crtcs);
- if (ret)
- dev_warn(dev->dev, "could not init vblank\n");
-
- for (i = 0; i < priv->num_crtcs; i++)
- drm_crtc_vblank_off(priv->crtcs[i]);
-
- priv->fbdev = omap_fbdev_init(dev);
-
- /* store off drm_device for use in pm ops */
- dev_set_drvdata(dev->dev, dev);
-
- drm_kms_helper_poll_init(dev);
-
- return 0;
-}
-
-static void dev_unload(struct drm_device *dev)
-{
- struct omap_drm_private *priv = dev->dev_private;
-
- DBG("unload: dev=%p", dev);
-
- drm_kms_helper_poll_fini(dev);
-
- if (priv->fbdev)
- omap_fbdev_free(dev);
-
- omap_modeset_free(dev);
- omap_gem_deinit(dev);
-
- destroy_workqueue(priv->wq);
-
- drm_vblank_cleanup(dev);
- omap_drm_irq_uninstall(dev);
-
- kfree(dev->dev_private);
- dev->dev_private = NULL;
-
- dev_set_drvdata(dev->dev, NULL);
-}
-
static int dev_open(struct drm_device *dev, struct drm_file *file)
{
file->driver_priv = NULL;
static struct drm_driver omap_drm_driver = {
.driver_features = DRIVER_MODESET | DRIVER_GEM | DRIVER_PRIME |
DRIVER_ATOMIC,
- .load = dev_load,
- .unload = dev_unload,
.open = dev_open,
.lastclose = dev_lastclose,
.get_vblank_counter = drm_vblank_no_hw_counter,
.patchlevel = DRIVER_PATCHLEVEL,
};
-static int pdev_probe(struct platform_device *device)
+static int pdev_probe(struct platform_device *pdev)
{
- int r;
+ struct omap_drm_platform_data *pdata = pdev->dev.platform_data;
+ struct omap_drm_private *priv;
+ struct drm_device *ddev;
+ unsigned int i;
+ int ret;
+
+ DBG("%s", pdev->name);
if (omapdss_is_initialized() == false)
return -EPROBE_DEFER;
omap_crtc_pre_init();
- r = omap_connect_dssdevs();
- if (r) {
- omap_crtc_pre_uninit();
- return r;
+ ret = omap_connect_dssdevs();
+ if (ret)
+ goto err_crtc_uninit;
+
+ /* Allocate and initialize the driver private structure. */
+ priv = kzalloc(sizeof(*priv), GFP_KERNEL);
+ if (!priv) {
+ ret = -ENOMEM;
+ goto err_disconnect_dssdevs;
}
- DBG("%s", device->name);
- return drm_platform_init(&omap_drm_driver, device);
+ priv->omaprev = pdata->omaprev;
+ priv->wq = alloc_ordered_workqueue("omapdrm", 0);
+
+ init_waitqueue_head(&priv->commit.wait);
+ spin_lock_init(&priv->commit.lock);
+ spin_lock_init(&priv->list_lock);
+ INIT_LIST_HEAD(&priv->obj_list);
+
+ /* Allocate and initialize the DRM device. */
+ ddev = drm_dev_alloc(&omap_drm_driver, &pdev->dev);
+ if (IS_ERR(ddev)) {
+ ret = PTR_ERR(ddev);
+ goto err_free_priv;
+ }
+
+ ddev->dev_private = priv;
+ platform_set_drvdata(pdev, ddev);
+
+ omap_gem_init(ddev);
+
+ ret = omap_modeset_init(ddev);
+ if (ret) {
+ dev_err(&pdev->dev, "omap_modeset_init failed: ret=%d\n", ret);
+ goto err_free_drm_dev;
+ }
+
+ /* Initialize vblank handling, start with all CRTCs disabled. */
+ ret = drm_vblank_init(ddev, priv->num_crtcs);
+ if (ret) {
+ dev_err(&pdev->dev, "could not init vblank\n");
+ goto err_cleanup_modeset;
+ }
+
+ for (i = 0; i < priv->num_crtcs; i++)
+ drm_crtc_vblank_off(priv->crtcs[i]);
+
+ priv->fbdev = omap_fbdev_init(ddev);
+
+ drm_kms_helper_poll_init(ddev);
+
+ /*
+ * Register the DRM device with the core and the connectors with
+ * sysfs.
+ */
+ ret = drm_dev_register(ddev, 0);
+ if (ret)
+ goto err_cleanup_helpers;
+
+ return 0;
+
+err_cleanup_helpers:
+ drm_kms_helper_poll_fini(ddev);
+ if (priv->fbdev)
+ omap_fbdev_free(ddev);
+err_cleanup_modeset:
+ drm_mode_config_cleanup(ddev);
+ omap_drm_irq_uninstall(ddev);
+err_free_drm_dev:
+ omap_gem_deinit(ddev);
+ drm_dev_unref(ddev);
+err_free_priv:
+ destroy_workqueue(priv->wq);
+ kfree(priv);
+err_disconnect_dssdevs:
+ omap_disconnect_dssdevs();
+err_crtc_uninit:
+ omap_crtc_pre_uninit();
+ return ret;
}
-static int pdev_remove(struct platform_device *device)
+static int pdev_remove(struct platform_device *pdev)
{
+ struct drm_device *ddev = platform_get_drvdata(pdev);
+ struct omap_drm_private *priv = ddev->dev_private;
+
DBG("");
- drm_put_dev(platform_get_drvdata(device));
+ drm_dev_unregister(ddev);
+
+ drm_kms_helper_poll_fini(ddev);
+
+ if (priv->fbdev)
+ omap_fbdev_free(ddev);
+
+ drm_mode_config_cleanup(ddev);
+
+ omap_drm_irq_uninstall(ddev);
+ omap_gem_deinit(ddev);
+
+ drm_dev_unref(ddev);
+
+ destroy_workqueue(priv->wq);
+ kfree(priv);
omap_disconnect_dssdevs();
omap_crtc_pre_uninit();
uint32_t src_w, src_h;
};
-/* For transiently registering for different DSS irqs that various parts
- * of the KMS code need during setup/configuration. We these are not
- * necessarily the same as what drm_vblank_get/put() are requesting, and
- * the hysteresis in drm_vblank_put() is not necessarily desirable for
- * internal housekeeping related irq usage.
- */
-struct omap_drm_irq {
- struct list_head node;
- uint32_t irqmask;
- bool registered;
- void (*irq)(struct omap_drm_irq *irq, uint32_t irqstatus);
-};
-
/* For KMS code that needs to wait for a certain # of IRQs:
*/
struct omap_irq_wait;
struct drm_property *zorder_prop;
/* irq handling: */
- struct list_head irq_list; /* list of omap_drm_irq */
- uint32_t vblank_mask; /* irq bits set for userspace vblank */
- struct omap_drm_irq error_handler;
+ spinlock_t wait_lock; /* protects the wait_list */
+ struct list_head wait_list; /* list of omap_irq_wait */
+ uint32_t irq_mask; /* enabled irqs in addition to wait_list */
/* atomic commit */
struct {
int omap_irq_enable_vblank(struct drm_device *dev, unsigned int pipe);
void omap_irq_disable_vblank(struct drm_device *dev, unsigned int pipe);
-void __omap_irq_register(struct drm_device *dev, struct omap_drm_irq *irq);
-void __omap_irq_unregister(struct drm_device *dev, struct omap_drm_irq *irq);
-void omap_irq_register(struct drm_device *dev, struct omap_drm_irq *irq);
-void omap_irq_unregister(struct drm_device *dev, struct omap_drm_irq *irq);
void omap_drm_irq_uninstall(struct drm_device *dev);
int omap_drm_irq_install(struct drm_device *dev);
struct drm_crtc *omap_crtc_init(struct drm_device *dev,
struct drm_plane *plane, enum omap_channel channel, int id);
int omap_crtc_wait_pending(struct drm_crtc *crtc);
+void omap_crtc_error_irq(struct drm_crtc *crtc, uint32_t irqstatus);
+void omap_crtc_vblank_irq(struct drm_crtc *crtc);
struct drm_plane *omap_plane_init(struct drm_device *dev,
int id, enum drm_plane_type type,
struct dma_buf *buffer);
/* map crtc to vblank mask */
-uint32_t pipe2vbl(struct drm_crtc *crtc);
struct omap_dss_device *omap_encoder_get_dssdev(struct drm_encoder *encoder);
-/* should these be made into common util helpers?
- */
-
-static inline int objects_lookup(
- struct drm_file *filp, uint32_t pixel_format,
- struct drm_gem_object **bos, const uint32_t *handles)
-{
- int i, n = drm_format_num_planes(pixel_format);
-
- for (i = 0; i < n; i++) {
- bos[i] = drm_gem_object_lookup(filp, handles[i]);
- if (!bos[i])
- goto fail;
-
- }
-
- return 0;
-
-fail:
- while (--i > 0)
- drm_gem_object_unreference_unlocked(bos[i]);
-
- return -ENOENT;
-}
-
#endif /* __OMAP_DRV_H__ */
dssdrv->get_timings(dssdev, &t);
- if (memcmp(vm, &t, sizeof(struct videomode)))
+ if (memcmp(vm, &t, sizeof(*vm)))
ret = -EINVAL;
else
ret = 0;
* framebuffer funcs
*/
-/* per-format info: */
-struct format {
+/* DSS to DRM formats mapping */
+static const struct {
enum omap_color_mode dss_format;
uint32_t pixel_format;
- struct {
- int stride_bpp; /* this times width is stride */
- int sub_y; /* sub-sample in y dimension */
- } planes[4];
- bool yuv;
-};
-
-static const struct format formats[] = {
+} formats[] = {
/* 16bpp [A]RGB: */
- { OMAP_DSS_COLOR_RGB16, DRM_FORMAT_RGB565, {{2, 1}}, false }, /* RGB16-565 */
- { OMAP_DSS_COLOR_RGB12U, DRM_FORMAT_RGBX4444, {{2, 1}}, false }, /* RGB12x-4444 */
- { OMAP_DSS_COLOR_RGBX16, DRM_FORMAT_XRGB4444, {{2, 1}}, false }, /* xRGB12-4444 */
- { OMAP_DSS_COLOR_RGBA16, DRM_FORMAT_RGBA4444, {{2, 1}}, false }, /* RGBA12-4444 */
- { OMAP_DSS_COLOR_ARGB16, DRM_FORMAT_ARGB4444, {{2, 1}}, false }, /* ARGB16-4444 */
- { OMAP_DSS_COLOR_XRGB16_1555, DRM_FORMAT_XRGB1555, {{2, 1}}, false }, /* xRGB15-1555 */
- { OMAP_DSS_COLOR_ARGB16_1555, DRM_FORMAT_ARGB1555, {{2, 1}}, false }, /* ARGB16-1555 */
+ { OMAP_DSS_COLOR_RGB16, DRM_FORMAT_RGB565 }, /* RGB16-565 */
+ { OMAP_DSS_COLOR_RGB12U, DRM_FORMAT_RGBX4444 }, /* RGB12x-4444 */
+ { OMAP_DSS_COLOR_RGBX16, DRM_FORMAT_XRGB4444 }, /* xRGB12-4444 */
+ { OMAP_DSS_COLOR_RGBA16, DRM_FORMAT_RGBA4444 }, /* RGBA12-4444 */
+ { OMAP_DSS_COLOR_ARGB16, DRM_FORMAT_ARGB4444 }, /* ARGB16-4444 */
+ { OMAP_DSS_COLOR_XRGB16_1555, DRM_FORMAT_XRGB1555 }, /* xRGB15-1555 */
+ { OMAP_DSS_COLOR_ARGB16_1555, DRM_FORMAT_ARGB1555 }, /* ARGB16-1555 */
/* 24bpp RGB: */
- { OMAP_DSS_COLOR_RGB24P, DRM_FORMAT_RGB888, {{3, 1}}, false }, /* RGB24-888 */
+ { OMAP_DSS_COLOR_RGB24P, DRM_FORMAT_RGB888 }, /* RGB24-888 */
/* 32bpp [A]RGB: */
- { OMAP_DSS_COLOR_RGBX32, DRM_FORMAT_RGBX8888, {{4, 1}}, false }, /* RGBx24-8888 */
- { OMAP_DSS_COLOR_RGB24U, DRM_FORMAT_XRGB8888, {{4, 1}}, false }, /* xRGB24-8888 */
- { OMAP_DSS_COLOR_RGBA32, DRM_FORMAT_RGBA8888, {{4, 1}}, false }, /* RGBA32-8888 */
- { OMAP_DSS_COLOR_ARGB32, DRM_FORMAT_ARGB8888, {{4, 1}}, false }, /* ARGB32-8888 */
+ { OMAP_DSS_COLOR_RGBX32, DRM_FORMAT_RGBX8888 }, /* RGBx24-8888 */
+ { OMAP_DSS_COLOR_RGB24U, DRM_FORMAT_XRGB8888 }, /* xRGB24-8888 */
+ { OMAP_DSS_COLOR_RGBA32, DRM_FORMAT_RGBA8888 }, /* RGBA32-8888 */
+ { OMAP_DSS_COLOR_ARGB32, DRM_FORMAT_ARGB8888 }, /* ARGB32-8888 */
/* YUV: */
- { OMAP_DSS_COLOR_NV12, DRM_FORMAT_NV12, {{1, 1}, {1, 2}}, true },
- { OMAP_DSS_COLOR_YUV2, DRM_FORMAT_YUYV, {{2, 1}}, true },
- { OMAP_DSS_COLOR_UYVY, DRM_FORMAT_UYVY, {{2, 1}}, true },
+ { OMAP_DSS_COLOR_NV12, DRM_FORMAT_NV12 },
+ { OMAP_DSS_COLOR_YUV2, DRM_FORMAT_YUYV },
+ { OMAP_DSS_COLOR_UYVY, DRM_FORMAT_UYVY },
};
/* convert from overlay's pixel formats bitmask to an array of fourcc's */
struct omap_framebuffer {
struct drm_framebuffer base;
int pin_count;
- const struct format *format;
- struct plane planes[4];
+ const struct drm_format_info *format;
+ enum omap_color_mode dss_format;
+ struct plane planes[2];
/* lock for pinning (pin_count and planes.paddr) */
struct mutex lock;
};
};
static uint32_t get_linear_addr(struct plane *plane,
- const struct format *format, int n, int x, int y)
+ const struct drm_format_info *format, int n, int x, int y)
{
uint32_t offset;
- offset = plane->offset +
- (x * format->planes[n].stride_bpp) +
- (y * plane->pitch / format->planes[n].sub_y);
+ offset = plane->offset
+ + (x * format->cpp[n] / (n == 0 ? 1 : format->hsub))
+ + (y * plane->pitch / (n == 0 ? 1 : format->vsub));
return plane->paddr + offset;
}
struct omap_drm_window *win, struct omap_overlay_info *info)
{
struct omap_framebuffer *omap_fb = to_omap_framebuffer(fb);
- const struct format *format = omap_fb->format;
+ const struct drm_format_info *format = omap_fb->format;
struct plane *plane = &omap_fb->planes[0];
uint32_t x, y, orient = 0;
- info->color_mode = format->dss_format;
+ info->color_mode = omap_fb->dss_format;
info->pos_x = win->crtc_x;
info->pos_y = win->crtc_y;
}
/* convert to pixels: */
- info->screen_width /= format->planes[0].stride_bpp;
+ info->screen_width /= format->cpp[0];
- if (format->dss_format == OMAP_DSS_COLOR_NV12) {
+ if (omap_fb->dss_format == OMAP_DSS_COLOR_NV12) {
plane = &omap_fb->planes[1];
if (info->rotation_type == OMAP_DSS_ROT_TILER) {
struct drm_framebuffer *omap_framebuffer_create(struct drm_device *dev,
struct drm_file *file, const struct drm_mode_fb_cmd2 *mode_cmd)
{
+ unsigned int num_planes = drm_format_num_planes(mode_cmd->pixel_format);
struct drm_gem_object *bos[4];
struct drm_framebuffer *fb;
- int ret;
+ int i;
- ret = objects_lookup(file, mode_cmd->pixel_format,
- bos, mode_cmd->handles);
- if (ret)
- return ERR_PTR(ret);
+ for (i = 0; i < num_planes; i++) {
+ bos[i] = drm_gem_object_lookup(file, mode_cmd->handles[i]);
+ if (!bos[i]) {
+ fb = ERR_PTR(-ENOENT);
+ goto error;
+ }
+ }
fb = omap_framebuffer_init(dev, mode_cmd, bos);
- if (IS_ERR(fb)) {
- int i, n = drm_format_num_planes(mode_cmd->pixel_format);
- for (i = 0; i < n; i++)
- drm_gem_object_unreference_unlocked(bos[i]);
- return fb;
- }
+ if (IS_ERR(fb))
+ goto error;
+
+ return fb;
+
+error:
+ while (--i > 0)
+ drm_gem_object_unreference_unlocked(bos[i]);
+
return fb;
}
struct drm_framebuffer *omap_framebuffer_init(struct drm_device *dev,
const struct drm_mode_fb_cmd2 *mode_cmd, struct drm_gem_object **bos)
{
+ const struct drm_format_info *format = NULL;
struct omap_framebuffer *omap_fb = NULL;
struct drm_framebuffer *fb = NULL;
- const struct format *format = NULL;
- int ret, i, n = drm_format_num_planes(mode_cmd->pixel_format);
+ enum omap_color_mode dss_format = 0;
+ unsigned int pitch = mode_cmd->pitches[0];
+ int ret, i;
DBG("create framebuffer: dev=%p, mode_cmd=%p (%dx%d@%4.4s)",
dev, mode_cmd, mode_cmd->width, mode_cmd->height,
(char *)&mode_cmd->pixel_format);
+ format = drm_format_info(mode_cmd->pixel_format);
+
for (i = 0; i < ARRAY_SIZE(formats); i++) {
if (formats[i].pixel_format == mode_cmd->pixel_format) {
- format = &formats[i];
+ dss_format = formats[i].dss_format;
break;
}
}
- if (!format) {
- dev_err(dev->dev, "unsupported pixel format: %4.4s\n",
- (char *)&mode_cmd->pixel_format);
+ if (!format || !dss_format) {
+ dev_dbg(dev->dev, "unsupported pixel format: %4.4s\n",
+ (char *)&mode_cmd->pixel_format);
ret = -EINVAL;
goto fail;
}
fb = &omap_fb->base;
omap_fb->format = format;
+ omap_fb->dss_format = dss_format;
mutex_init(&omap_fb->lock);
- for (i = 0; i < n; i++) {
- struct plane *plane = &omap_fb->planes[i];
- int size, pitch = mode_cmd->pitches[i];
-
- if (pitch < (mode_cmd->width * format->planes[i].stride_bpp)) {
- dev_err(dev->dev, "provided buffer pitch is too small! %d < %d\n",
- pitch, mode_cmd->width * format->planes[i].stride_bpp);
- ret = -EINVAL;
- goto fail;
- }
+ /*
+ * The code below assumes that no format use more than two planes, and
+ * that the two planes of multiplane formats need the same number of
+ * bytes per pixel.
+ */
+ if (format->num_planes == 2 && pitch != mode_cmd->pitches[1]) {
+ dev_dbg(dev->dev, "pitches differ between planes 0 and 1\n");
+ ret = -EINVAL;
+ goto fail;
+ }
- if (pitch % format->planes[i].stride_bpp != 0) {
- dev_err(dev->dev,
- "buffer pitch (%d bytes) is not a multiple of pixel size (%d bytes)\n",
- pitch, format->planes[i].stride_bpp);
- ret = -EINVAL;
- goto fail;
- }
+ if (pitch % format->cpp[0]) {
+ dev_dbg(dev->dev,
+ "buffer pitch (%u bytes) is not a multiple of pixel size (%u bytes)\n",
+ pitch, format->cpp[0]);
+ ret = -EINVAL;
+ goto fail;
+ }
- size = pitch * mode_cmd->height / format->planes[i].sub_y;
+ for (i = 0; i < format->num_planes; i++) {
+ struct plane *plane = &omap_fb->planes[i];
+ unsigned int vsub = i == 0 ? 1 : format->vsub;
+ unsigned int size;
- if (size > (omap_gem_mmap_size(bos[i]) - mode_cmd->offsets[i])) {
- dev_err(dev->dev, "provided buffer object is too small! %d < %d\n",
- bos[i]->size - mode_cmd->offsets[i], size);
- ret = -EINVAL;
- goto fail;
- }
+ size = pitch * mode_cmd->height / vsub;
- if (i > 0 && pitch != mode_cmd->pitches[i - 1]) {
- dev_err(dev->dev,
- "pitches are not the same between framebuffer planes %d != %d\n",
- pitch, mode_cmd->pitches[i - 1]);
+ if (size > omap_gem_mmap_size(bos[i]) - mode_cmd->offsets[i]) {
+ dev_dbg(dev->dev,
+ "provided buffer object is too small! %d < %d\n",
+ bos[i]->size - mode_cmd->offsets[i], size);
ret = -EINVAL;
goto fail;
}
#include "omap_drv.h"
-static DEFINE_SPINLOCK(list_lock);
-
-static void omap_irq_error_handler(struct omap_drm_irq *irq,
- uint32_t irqstatus)
-{
- DRM_ERROR("errors: %08x\n", irqstatus);
-}
+struct omap_irq_wait {
+ struct list_head node;
+ wait_queue_head_t wq;
+ uint32_t irqmask;
+ int count;
+};
-/* call with list_lock and dispc runtime held */
+/* call with wait_lock and dispc runtime held */
static void omap_irq_update(struct drm_device *dev)
{
struct omap_drm_private *priv = dev->dev_private;
- struct omap_drm_irq *irq;
- uint32_t irqmask = priv->vblank_mask;
+ struct omap_irq_wait *wait;
+ uint32_t irqmask = priv->irq_mask;
- assert_spin_locked(&list_lock);
+ assert_spin_locked(&priv->wait_lock);
- list_for_each_entry(irq, &priv->irq_list, node)
- irqmask |= irq->irqmask;
+ list_for_each_entry(wait, &priv->wait_list, node)
+ irqmask |= wait->irqmask;
DBG("irqmask=%08x", irqmask);
dispc_read_irqenable(); /* flush posted write */
}
-void __omap_irq_register(struct drm_device *dev, struct omap_drm_irq *irq)
-{
- struct omap_drm_private *priv = dev->dev_private;
- unsigned long flags;
-
- spin_lock_irqsave(&list_lock, flags);
-
- if (!WARN_ON(irq->registered)) {
- irq->registered = true;
- list_add(&irq->node, &priv->irq_list);
- omap_irq_update(dev);
- }
-
- spin_unlock_irqrestore(&list_lock, flags);
-}
-
-void omap_irq_register(struct drm_device *dev, struct omap_drm_irq *irq)
-{
- dispc_runtime_get();
-
- __omap_irq_register(dev, irq);
-
- dispc_runtime_put();
-}
-
-void __omap_irq_unregister(struct drm_device *dev, struct omap_drm_irq *irq)
+static void omap_irq_wait_handler(struct omap_irq_wait *wait)
{
- unsigned long flags;
-
- spin_lock_irqsave(&list_lock, flags);
-
- if (!WARN_ON(!irq->registered)) {
- irq->registered = false;
- list_del(&irq->node);
- omap_irq_update(dev);
- }
-
- spin_unlock_irqrestore(&list_lock, flags);
-}
-
-void omap_irq_unregister(struct drm_device *dev, struct omap_drm_irq *irq)
-{
- dispc_runtime_get();
-
- __omap_irq_unregister(dev, irq);
-
- dispc_runtime_put();
-}
-
-struct omap_irq_wait {
- struct omap_drm_irq irq;
- int count;
-};
-
-static DECLARE_WAIT_QUEUE_HEAD(wait_event);
-
-static void wait_irq(struct omap_drm_irq *irq, uint32_t irqstatus)
-{
- struct omap_irq_wait *wait =
- container_of(irq, struct omap_irq_wait, irq);
wait->count--;
- wake_up_all(&wait_event);
+ wake_up(&wait->wq);
}
struct omap_irq_wait * omap_irq_wait_init(struct drm_device *dev,
uint32_t irqmask, int count)
{
+ struct omap_drm_private *priv = dev->dev_private;
struct omap_irq_wait *wait = kzalloc(sizeof(*wait), GFP_KERNEL);
- wait->irq.irq = wait_irq;
- wait->irq.irqmask = irqmask;
+ unsigned long flags;
+
+ init_waitqueue_head(&wait->wq);
+ wait->irqmask = irqmask;
wait->count = count;
- omap_irq_register(dev, &wait->irq);
+
+ spin_lock_irqsave(&priv->wait_lock, flags);
+ list_add(&wait->node, &priv->wait_list);
+ omap_irq_update(dev);
+ spin_unlock_irqrestore(&priv->wait_lock, flags);
+
return wait;
}
int omap_irq_wait(struct drm_device *dev, struct omap_irq_wait *wait,
unsigned long timeout)
{
- int ret = wait_event_timeout(wait_event, (wait->count <= 0), timeout);
- omap_irq_unregister(dev, &wait->irq);
+ struct omap_drm_private *priv = dev->dev_private;
+ unsigned long flags;
+ int ret;
+
+ ret = wait_event_timeout(wait->wq, (wait->count <= 0), timeout);
+
+ spin_lock_irqsave(&priv->wait_lock, flags);
+ list_del(&wait->node);
+ omap_irq_update(dev);
+ spin_unlock_irqrestore(&priv->wait_lock, flags);
+
kfree(wait);
- if (ret == 0)
- return -1;
- return 0;
+
+ return ret == 0 ? -1 : 0;
}
/**
DBG("dev=%p, crtc=%u", dev, pipe);
- spin_lock_irqsave(&list_lock, flags);
- priv->vblank_mask |= pipe2vbl(crtc);
+ spin_lock_irqsave(&priv->wait_lock, flags);
+ priv->irq_mask |= dispc_mgr_get_vsync_irq(omap_crtc_channel(crtc));
omap_irq_update(dev);
- spin_unlock_irqrestore(&list_lock, flags);
+ spin_unlock_irqrestore(&priv->wait_lock, flags);
return 0;
}
DBG("dev=%p, crtc=%u", dev, pipe);
- spin_lock_irqsave(&list_lock, flags);
- priv->vblank_mask &= ~pipe2vbl(crtc);
+ spin_lock_irqsave(&priv->wait_lock, flags);
+ priv->irq_mask &= ~dispc_mgr_get_vsync_irq(omap_crtc_channel(crtc));
omap_irq_update(dev);
- spin_unlock_irqrestore(&list_lock, flags);
+ spin_unlock_irqrestore(&priv->wait_lock, flags);
+}
+
+static void omap_irq_fifo_underflow(struct omap_drm_private *priv,
+ u32 irqstatus)
+{
+ static DEFINE_RATELIMIT_STATE(_rs, DEFAULT_RATELIMIT_INTERVAL,
+ DEFAULT_RATELIMIT_BURST);
+ static const struct {
+ const char *name;
+ u32 mask;
+ } sources[] = {
+ { "gfx", DISPC_IRQ_GFX_FIFO_UNDERFLOW },
+ { "vid1", DISPC_IRQ_VID1_FIFO_UNDERFLOW },
+ { "vid2", DISPC_IRQ_VID2_FIFO_UNDERFLOW },
+ { "vid3", DISPC_IRQ_VID3_FIFO_UNDERFLOW },
+ };
+
+ const u32 mask = DISPC_IRQ_GFX_FIFO_UNDERFLOW
+ | DISPC_IRQ_VID1_FIFO_UNDERFLOW
+ | DISPC_IRQ_VID2_FIFO_UNDERFLOW
+ | DISPC_IRQ_VID3_FIFO_UNDERFLOW;
+ unsigned int i;
+
+ spin_lock(&priv->wait_lock);
+ irqstatus &= priv->irq_mask & mask;
+ spin_unlock(&priv->wait_lock);
+
+ if (!irqstatus)
+ return;
+
+ if (!__ratelimit(&_rs))
+ return;
+
+ DRM_ERROR("FIFO underflow on ");
+
+ for (i = 0; i < ARRAY_SIZE(sources); ++i) {
+ if (sources[i].mask & irqstatus)
+ pr_cont("%s ", sources[i].name);
+ }
+
+ pr_cont("(0x%08x)\n", irqstatus);
+}
+
+static void omap_irq_ocp_error_handler(u32 irqstatus)
+{
+ if (!(irqstatus & DISPC_IRQ_OCP_ERR))
+ return;
+
+ DRM_ERROR("OCP error\n");
}
static irqreturn_t omap_irq_handler(int irq, void *arg)
{
struct drm_device *dev = (struct drm_device *) arg;
struct omap_drm_private *priv = dev->dev_private;
- struct omap_drm_irq *handler, *n;
+ struct omap_irq_wait *wait, *n;
unsigned long flags;
unsigned int id;
u32 irqstatus;
for (id = 0; id < priv->num_crtcs; id++) {
struct drm_crtc *crtc = priv->crtcs[id];
+ enum omap_channel channel = omap_crtc_channel(crtc);
- if (irqstatus & pipe2vbl(crtc))
+ if (irqstatus & dispc_mgr_get_vsync_irq(channel)) {
drm_handle_vblank(dev, id);
+ omap_crtc_vblank_irq(crtc);
+ }
+
+ if (irqstatus & dispc_mgr_get_sync_lost_irq(channel))
+ omap_crtc_error_irq(crtc, irqstatus);
}
- spin_lock_irqsave(&list_lock, flags);
- list_for_each_entry_safe(handler, n, &priv->irq_list, node) {
- if (handler->irqmask & irqstatus) {
- spin_unlock_irqrestore(&list_lock, flags);
- handler->irq(handler, handler->irqmask & irqstatus);
- spin_lock_irqsave(&list_lock, flags);
- }
+ omap_irq_ocp_error_handler(irqstatus);
+ omap_irq_fifo_underflow(priv, irqstatus);
+
+ spin_lock_irqsave(&priv->wait_lock, flags);
+ list_for_each_entry_safe(wait, n, &priv->wait_list, node) {
+ if (wait->irqmask & irqstatus)
+ omap_irq_wait_handler(wait);
}
- spin_unlock_irqrestore(&list_lock, flags);
+ spin_unlock_irqrestore(&priv->wait_lock, flags);
return IRQ_HANDLED;
}
+static const u32 omap_underflow_irqs[] = {
+ [OMAP_DSS_GFX] = DISPC_IRQ_GFX_FIFO_UNDERFLOW,
+ [OMAP_DSS_VIDEO1] = DISPC_IRQ_VID1_FIFO_UNDERFLOW,
+ [OMAP_DSS_VIDEO2] = DISPC_IRQ_VID2_FIFO_UNDERFLOW,
+ [OMAP_DSS_VIDEO3] = DISPC_IRQ_VID3_FIFO_UNDERFLOW,
+};
+
/*
* We need a special version, instead of just using drm_irq_install(),
* because we need to register the irq via omapdss. Once omapdss and
int omap_drm_irq_install(struct drm_device *dev)
{
struct omap_drm_private *priv = dev->dev_private;
- struct omap_drm_irq *error_handler = &priv->error_handler;
+ unsigned int num_mgrs = dss_feat_get_num_mgrs();
+ unsigned int max_planes;
+ unsigned int i;
int ret;
- INIT_LIST_HEAD(&priv->irq_list);
+ spin_lock_init(&priv->wait_lock);
+ INIT_LIST_HEAD(&priv->wait_list);
+
+ priv->irq_mask = DISPC_IRQ_OCP_ERR;
+
+ max_planes = min(ARRAY_SIZE(priv->planes),
+ ARRAY_SIZE(omap_underflow_irqs));
+ for (i = 0; i < max_planes; ++i) {
+ if (priv->planes[i])
+ priv->irq_mask |= omap_underflow_irqs[i];
+ }
+
+ for (i = 0; i < num_mgrs; ++i)
+ priv->irq_mask |= dispc_mgr_get_sync_lost_irq(i);
dispc_runtime_get();
dispc_clear_irqstatus(0xffffffff);
if (ret < 0)
return ret;
- error_handler->irq = omap_irq_error_handler;
- error_handler->irqmask = DISPC_IRQ_OCP_ERR;
-
- /* for now ignore DISPC_IRQ_SYNC_LOST_DIGIT.. really I think
- * we just need to ignore it while enabling tv-out
- */
- error_handler->irqmask &= ~DISPC_IRQ_SYNC_LOST_DIGIT;
-
- omap_irq_register(dev, error_handler);
-
dev->irq_enabled = true;
return 0;
uint32_t nformats;
uint32_t formats[32];
-
- struct omap_drm_irq error_irq;
};
struct omap_plane_state {
DBG("%s", omap_plane->name);
- omap_irq_unregister(plane->dev, &omap_plane->error_irq);
-
drm_plane_cleanup(plane);
kfree(omap_plane);
.atomic_get_property = omap_plane_atomic_get_property,
};
-static void omap_plane_error_irq(struct omap_drm_irq *irq, uint32_t irqstatus)
-{
- struct omap_plane *omap_plane =
- container_of(irq, struct omap_plane, error_irq);
- DRM_ERROR_RATELIMITED("%s: errors: %08x\n", omap_plane->name,
- irqstatus);
-}
-
static const char *plane_names[] = {
[OMAP_DSS_GFX] = "gfx",
[OMAP_DSS_VIDEO1] = "vid1",
[OMAP_DSS_VIDEO3] = "vid3",
};
-static const uint32_t error_irqs[] = {
- [OMAP_DSS_GFX] = DISPC_IRQ_GFX_FIFO_UNDERFLOW,
- [OMAP_DSS_VIDEO1] = DISPC_IRQ_VID1_FIFO_UNDERFLOW,
- [OMAP_DSS_VIDEO2] = DISPC_IRQ_VID2_FIFO_UNDERFLOW,
- [OMAP_DSS_VIDEO3] = DISPC_IRQ_VID3_FIFO_UNDERFLOW,
-};
-
/* initialize plane */
struct drm_plane *omap_plane_init(struct drm_device *dev,
int id, enum drm_plane_type type,
plane = &omap_plane->base;
- omap_plane->error_irq.irqmask = error_irqs[id];
- omap_plane->error_irq.irq = omap_plane_error_irq;
- omap_irq_register(dev, &omap_plane->error_irq);
-
ret = drm_universal_plane_init(dev, plane, possible_crtcs,
&omap_plane_funcs, omap_plane->formats,
omap_plane->nformats, type, NULL);
return plane;
error:
- omap_irq_unregister(plane->dev, &omap_plane->error_irq);
kfree(omap_plane);
return NULL;
}
MODULE_FIRMWARE("radeon/tahiti_mc.bin");
MODULE_FIRMWARE("radeon/tahiti_rlc.bin");
MODULE_FIRMWARE("radeon/tahiti_smc.bin");
-MODULE_FIRMWARE("radeon/tahiti_k_smc.bin");
MODULE_FIRMWARE("radeon/PITCAIRN_pfp.bin");
MODULE_FIRMWARE("radeon/PITCAIRN_me.bin");
MODULE_FIRMWARE("radeon/hainan_rlc.bin");
MODULE_FIRMWARE("radeon/hainan_smc.bin");
MODULE_FIRMWARE("radeon/hainan_k_smc.bin");
+MODULE_FIRMWARE("radeon/banks_k_2_smc.bin");
+
+MODULE_FIRMWARE("radeon/si58_mc.bin");
static u32 si_get_cu_active_bitmap(struct radeon_device *rdev, u32 se, u32 sh);
static void si_pcie_gen3_enable(struct radeon_device *rdev);
int err;
int new_fw = 0;
bool new_smc = false;
+ bool si58_fw = false;
+ bool banks2_fw = false;
DRM_DEBUG("\n");
switch (rdev->family) {
case CHIP_TAHITI:
chip_name = "TAHITI";
- /* XXX: figure out which Tahitis need the new ucode */
- if (0)
- new_smc = true;
new_chip_name = "tahiti";
pfp_req_size = SI_PFP_UCODE_SIZE * 4;
me_req_size = SI_PM4_UCODE_SIZE * 4;
break;
case CHIP_PITCAIRN:
chip_name = "PITCAIRN";
- if ((rdev->pdev->revision == 0x81) ||
- (rdev->pdev->device == 0x6810) ||
- (rdev->pdev->device == 0x6811) ||
- (rdev->pdev->device == 0x6816) ||
- (rdev->pdev->device == 0x6817) ||
- (rdev->pdev->device == 0x6806))
+ if ((rdev->pdev->revision == 0x81) &&
+ ((rdev->pdev->device == 0x6810) ||
+ (rdev->pdev->device == 0x6811)))
new_smc = true;
new_chip_name = "pitcairn";
pfp_req_size = SI_PFP_UCODE_SIZE * 4;
break;
case CHIP_VERDE:
chip_name = "VERDE";
- if ((rdev->pdev->revision == 0x81) ||
- (rdev->pdev->revision == 0x83) ||
- (rdev->pdev->revision == 0x87) ||
- (rdev->pdev->device == 0x6820) ||
- (rdev->pdev->device == 0x6821) ||
- (rdev->pdev->device == 0x6822) ||
- (rdev->pdev->device == 0x6823) ||
- (rdev->pdev->device == 0x682A) ||
- (rdev->pdev->device == 0x682B))
+ if (((rdev->pdev->device == 0x6820) &&
+ ((rdev->pdev->revision == 0x81) ||
+ (rdev->pdev->revision == 0x83))) ||
+ ((rdev->pdev->device == 0x6821) &&
+ ((rdev->pdev->revision == 0x83) ||
+ (rdev->pdev->revision == 0x87))) ||
+ ((rdev->pdev->revision == 0x87) &&
+ ((rdev->pdev->device == 0x6823) ||
+ (rdev->pdev->device == 0x682b))))
new_smc = true;
new_chip_name = "verde";
pfp_req_size = SI_PFP_UCODE_SIZE * 4;
break;
case CHIP_OLAND:
chip_name = "OLAND";
- if ((rdev->pdev->revision == 0xC7) ||
- (rdev->pdev->revision == 0x80) ||
- (rdev->pdev->revision == 0x81) ||
- (rdev->pdev->revision == 0x83) ||
- (rdev->pdev->revision == 0x87) ||
- (rdev->pdev->device == 0x6604) ||
- (rdev->pdev->device == 0x6605))
+ if (((rdev->pdev->revision == 0x81) &&
+ ((rdev->pdev->device == 0x6600) ||
+ (rdev->pdev->device == 0x6604) ||
+ (rdev->pdev->device == 0x6605) ||
+ (rdev->pdev->device == 0x6610))) ||
+ ((rdev->pdev->revision == 0x83) &&
+ (rdev->pdev->device == 0x6610)))
new_smc = true;
new_chip_name = "oland";
pfp_req_size = SI_PFP_UCODE_SIZE * 4;
break;
case CHIP_HAINAN:
chip_name = "HAINAN";
- if ((rdev->pdev->revision == 0x81) ||
- (rdev->pdev->revision == 0x83) ||
- (rdev->pdev->revision == 0xC3) ||
- (rdev->pdev->device == 0x6664) ||
- (rdev->pdev->device == 0x6665) ||
- (rdev->pdev->device == 0x6667))
+ if (((rdev->pdev->revision == 0x81) &&
+ (rdev->pdev->device == 0x6660)) ||
+ ((rdev->pdev->revision == 0x83) &&
+ ((rdev->pdev->device == 0x6660) ||
+ (rdev->pdev->device == 0x6663) ||
+ (rdev->pdev->device == 0x6665) ||
+ (rdev->pdev->device == 0x6667))))
new_smc = true;
+ else if ((rdev->pdev->revision == 0xc3) &&
+ (rdev->pdev->device == 0x6665))
+ banks2_fw = true;
new_chip_name = "hainan";
pfp_req_size = SI_PFP_UCODE_SIZE * 4;
me_req_size = SI_PM4_UCODE_SIZE * 4;
default: BUG();
}
+ /* this memory configuration requires special firmware */
+ if (((RREG32(MC_SEQ_MISC0) & 0xff000000) >> 24) == 0x58)
+ si58_fw = true;
+
DRM_INFO("Loading %s Microcode\n", new_chip_name);
snprintf(fw_name, sizeof(fw_name), "radeon/%s_pfp.bin", new_chip_name);
}
}
- snprintf(fw_name, sizeof(fw_name), "radeon/%s_mc.bin", new_chip_name);
+ if (si58_fw)
+ snprintf(fw_name, sizeof(fw_name), "radeon/si58_mc.bin");
+ else
+ snprintf(fw_name, sizeof(fw_name), "radeon/%s_mc.bin", new_chip_name);
err = request_firmware(&rdev->mc_fw, fw_name, rdev->dev);
if (err) {
snprintf(fw_name, sizeof(fw_name), "radeon/%s_mc2.bin", chip_name);
}
}
- if (new_smc)
+ if (banks2_fw)
+ snprintf(fw_name, sizeof(fw_name), "radeon/banks_k_2_smc.bin");
+ else if (new_smc)
snprintf(fw_name, sizeof(fw_name), "radeon/%s_k_smc.bin", new_chip_name);
else
snprintf(fw_name, sizeof(fw_name), "radeon/%s_smc.bin", new_chip_name);
(rdev->pdev->device == 0x6817) ||
(rdev->pdev->device == 0x6806))
max_mclk = 120000;
- } else if (rdev->family == CHIP_VERDE) {
- if ((rdev->pdev->revision == 0x81) ||
- (rdev->pdev->revision == 0x83) ||
- (rdev->pdev->revision == 0x87) ||
- (rdev->pdev->device == 0x6820) ||
- (rdev->pdev->device == 0x6821) ||
- (rdev->pdev->device == 0x6822) ||
- (rdev->pdev->device == 0x6823) ||
- (rdev->pdev->device == 0x682A) ||
- (rdev->pdev->device == 0x682B)) {
- max_sclk = 75000;
- max_mclk = 80000;
- }
- } else if (rdev->family == CHIP_OLAND) {
- if ((rdev->pdev->revision == 0xC7) ||
- (rdev->pdev->revision == 0x80) ||
- (rdev->pdev->revision == 0x81) ||
- (rdev->pdev->revision == 0x83) ||
- (rdev->pdev->revision == 0x87) ||
- (rdev->pdev->device == 0x6604) ||
- (rdev->pdev->device == 0x6605)) {
- max_sclk = 75000;
- max_mclk = 80000;
- }
} else if (rdev->family == CHIP_HAINAN) {
if ((rdev->pdev->revision == 0x81) ||
(rdev->pdev->revision == 0x83) ||
(rdev->pdev->device == 0x6665) ||
(rdev->pdev->device == 0x6667)) {
max_sclk = 75000;
- max_mclk = 80000;
}
}
/* Apply dpm quirks */
sti_dvo.o \
sti_awg_utils.o \
sti_vtg.o \
- sti_vtac.o \
sti_hda.o \
sti_tvout.o \
sti_hqvdp.o \
{
struct sti_private *private = ddev->dev_private;
- if (!ddev->mode_config.num_connector)
- return;
-
- if (private->fbdev) {
- drm_fbdev_cma_hotplug_event(private->fbdev);
- return;
- }
-
- private->fbdev = drm_fbdev_cma_init(ddev, 32,
- ddev->mode_config.num_crtc,
- ddev->mode_config.num_connector);
- if (IS_ERR(private->fbdev))
- private->fbdev = NULL;
+ drm_fbdev_cma_hotplug_event(private->fbdev);
}
static const struct drm_mode_config_funcs sti_mode_config_funcs = {
static int sti_bind(struct device *dev)
{
struct drm_device *ddev;
+ struct sti_private *private;
+ struct drm_fbdev_cma *fbdev;
int ret;
ddev = drm_dev_alloc(&sti_driver, dev);
drm_mode_config_reset(ddev);
+ private = ddev->dev_private;
+ if (ddev->mode_config.num_connector) {
+ fbdev = drm_fbdev_cma_init(ddev, 32, ddev->mode_config.num_crtc,
+ ddev->mode_config.num_connector);
+ if (IS_ERR(fbdev)) {
+ DRM_DEBUG_DRIVER("Warning: fails to create fbdev\n");
+ fbdev = NULL;
+ }
+ private->fbdev = fbdev;
+ }
+
return 0;
err_register:
static struct platform_driver * const drivers[] = {
&sti_tvout_driver,
- &sti_vtac_driver,
&sti_hqvdp_driver,
&sti_hdmi_driver,
&sti_hda_driver,
};
extern struct platform_driver sti_tvout_driver;
-extern struct platform_driver sti_vtac_driver;
extern struct platform_driver sti_hqvdp_driver;
extern struct platform_driver sti_hdmi_driver;
extern struct platform_driver sti_hda_driver;
hdmi->enabled = false;
}
+/**
+ * sti_hdmi_audio_get_non_coherent_n() - get N parameter for non-coherent
+ * clocks. None-coherent clocks means that audio and TMDS clocks have not the
+ * same source (drifts between clocks). In this case assumption is that CTS is
+ * automatically calculated by hardware.
+ *
+ * @audio_fs: audio frame clock frequency in Hz
+ *
+ * Values computed are based on table described in HDMI specification 1.4b
+ *
+ * Returns n value.
+ */
+static int sti_hdmi_audio_get_non_coherent_n(unsigned int audio_fs)
+{
+ unsigned int n;
+
+ switch (audio_fs) {
+ case 32000:
+ n = 4096;
+ break;
+ case 44100:
+ n = 6272;
+ break;
+ case 48000:
+ n = 6144;
+ break;
+ case 88200:
+ n = 6272 * 2;
+ break;
+ case 96000:
+ n = 6144 * 2;
+ break;
+ case 176400:
+ n = 6272 * 4;
+ break;
+ case 192000:
+ n = 6144 * 4;
+ break;
+ default:
+ /* Not pre-defined, recommended value: 128 * fs / 1000 */
+ n = (audio_fs * 128) / 1000;
+ }
+
+ return n;
+}
+
+static int hdmi_audio_configure(struct sti_hdmi *hdmi)
+{
+ int audio_cfg, n;
+ struct hdmi_audio_params *params = &hdmi->audio;
+ struct hdmi_audio_infoframe *info = ¶ms->cea;
+
+ DRM_DEBUG_DRIVER("\n");
+
+ if (!hdmi->enabled)
+ return 0;
+
+ /* update N parameter */
+ n = sti_hdmi_audio_get_non_coherent_n(params->sample_rate);
+
+ DRM_DEBUG_DRIVER("Audio rate = %d Hz, TMDS clock = %d Hz, n = %d\n",
+ params->sample_rate, hdmi->mode.clock * 1000, n);
+ hdmi_write(hdmi, n, HDMI_AUDN);
+
+ /* update HDMI registers according to configuration */
+ audio_cfg = HDMI_AUD_CFG_SPDIF_DIV_2 | HDMI_AUD_CFG_DTS_INVALID |
+ HDMI_AUD_CFG_ONE_BIT_INVALID;
+
+ switch (info->channels) {
+ case 8:
+ audio_cfg |= HDMI_AUD_CFG_CH78_VALID;
+ case 6:
+ audio_cfg |= HDMI_AUD_CFG_CH56_VALID;
+ case 4:
+ audio_cfg |= HDMI_AUD_CFG_CH34_VALID | HDMI_AUD_CFG_8CH;
+ case 2:
+ audio_cfg |= HDMI_AUD_CFG_CH12_VALID;
+ break;
+ default:
+ DRM_ERROR("ERROR: Unsupported number of channels (%d)!\n",
+ info->channels);
+ return -EINVAL;
+ }
+
+ hdmi_write(hdmi, audio_cfg, HDMI_AUDIO_CFG);
+
+ return hdmi_audio_infoframe_config(hdmi);
+}
+
static void sti_hdmi_pre_enable(struct drm_bridge *bridge)
{
struct sti_hdmi *hdmi = bridge->driver_private;
if (hdmi_avi_infoframe_config(hdmi))
DRM_ERROR("Unable to configure AVI infoframe\n");
- /* Program AUDIO infoframe */
- if (hdmi_audio_infoframe_config(hdmi))
- DRM_ERROR("Unable to configure AUDIO infoframe\n");
+ if (hdmi->audio.enabled) {
+ if (hdmi_audio_configure(hdmi))
+ DRM_ERROR("Unable to configure audio\n");
+ } else {
+ hdmi_audio_infoframe_config(hdmi);
+ }
/* Program VS infoframe */
if (hdmi_vendor_infoframe_config(hdmi))
return NULL;
}
-/**
- * sti_hdmi_audio_get_non_coherent_n() - get N parameter for non-coherent
- * clocks. None-coherent clocks means that audio and TMDS clocks have not the
- * same source (drifts between clocks). In this case assumption is that CTS is
- * automatically calculated by hardware.
- *
- * @audio_fs: audio frame clock frequency in Hz
- *
- * Values computed are based on table described in HDMI specification 1.4b
- *
- * Returns n value.
- */
-static int sti_hdmi_audio_get_non_coherent_n(unsigned int audio_fs)
-{
- unsigned int n;
-
- switch (audio_fs) {
- case 32000:
- n = 4096;
- break;
- case 44100:
- n = 6272;
- break;
- case 48000:
- n = 6144;
- break;
- case 88200:
- n = 6272 * 2;
- break;
- case 96000:
- n = 6144 * 2;
- break;
- case 176400:
- n = 6272 * 4;
- break;
- case 192000:
- n = 6144 * 4;
- break;
- default:
- /* Not pre-defined, recommended value: 128 * fs / 1000 */
- n = (audio_fs * 128) / 1000;
- }
-
- return n;
-}
-
-static int hdmi_audio_configure(struct sti_hdmi *hdmi,
- struct hdmi_audio_params *params)
-{
- int audio_cfg, n;
- struct hdmi_audio_infoframe *info = ¶ms->cea;
-
- DRM_DEBUG_DRIVER("\n");
-
- if (!hdmi->enabled)
- return 0;
-
- /* update N parameter */
- n = sti_hdmi_audio_get_non_coherent_n(params->sample_rate);
-
- DRM_DEBUG_DRIVER("Audio rate = %d Hz, TMDS clock = %d Hz, n = %d\n",
- params->sample_rate, hdmi->mode.clock * 1000, n);
- hdmi_write(hdmi, n, HDMI_AUDN);
-
- /* update HDMI registers according to configuration */
- audio_cfg = HDMI_AUD_CFG_SPDIF_DIV_2 | HDMI_AUD_CFG_DTS_INVALID |
- HDMI_AUD_CFG_ONE_BIT_INVALID;
-
- switch (info->channels) {
- case 8:
- audio_cfg |= HDMI_AUD_CFG_CH78_VALID;
- case 6:
- audio_cfg |= HDMI_AUD_CFG_CH56_VALID;
- case 4:
- audio_cfg |= HDMI_AUD_CFG_CH34_VALID | HDMI_AUD_CFG_8CH;
- case 2:
- audio_cfg |= HDMI_AUD_CFG_CH12_VALID;
- break;
- default:
- DRM_ERROR("ERROR: Unsupported number of channels (%d)!\n",
- info->channels);
- return -EINVAL;
- }
-
- hdmi_write(hdmi, audio_cfg, HDMI_AUDIO_CFG);
-
- hdmi->audio = *params;
-
- return hdmi_audio_infoframe_config(hdmi);
-}
-
static void hdmi_audio_shutdown(struct device *dev, void *data)
{
struct sti_hdmi *hdmi = dev_get_drvdata(dev);
{
struct sti_hdmi *hdmi = dev_get_drvdata(dev);
int ret;
- struct hdmi_audio_params audio = {
- .sample_width = params->sample_width,
- .sample_rate = params->sample_rate,
- .cea = params->cea,
- };
DRM_DEBUG_DRIVER("\n");
- if (!hdmi->enabled)
- return 0;
-
if ((daifmt->fmt != HDMI_I2S) || daifmt->bit_clk_inv ||
daifmt->frame_clk_inv || daifmt->bit_clk_master ||
daifmt->frame_clk_master) {
return -EINVAL;
}
- audio.enabled = true;
+ hdmi->audio.sample_width = params->sample_width;
+ hdmi->audio.sample_rate = params->sample_rate;
+ hdmi->audio.cea = params->cea;
+
+ hdmi->audio.enabled = true;
- ret = hdmi_audio_configure(hdmi, &audio);
+ ret = hdmi_audio_configure(hdmi);
if (ret < 0)
return ret;
* @hqvdp_cmd_paddr: physical address of hqvdp_cmd
* @vtg: vtg for main data path
* @xp70_initialized: true if xp70 is already initialized
+ * @vtg_registered: true if registered to VTG
*/
struct sti_hqvdp {
struct device *dev;
u32 hqvdp_cmd_paddr;
struct sti_vtg *vtg;
bool xp70_initialized;
+ bool vtg_registered;
};
#define to_sti_hqvdp(x) container_of(x, struct sti_hqvdp, plane)
DRM_ERROR("XP70 could not revert to idle\n");
hqvdp->plane.status = STI_PLANE_DISABLED;
- hqvdp->xp70_initialized = false;
+ hqvdp->vtg_registered = false;
}
/**
return -EINVAL;
}
- if (!hqvdp->xp70_initialized) {
+ if (!hqvdp->xp70_initialized)
/* Start HQVDP XP70 coprocessor */
sti_hqvdp_start_xp70(hqvdp);
+ if (!hqvdp->vtg_registered) {
/* Prevent VTG shutdown */
if (clk_prepare_enable(hqvdp->clk_pix_main)) {
DRM_ERROR("Failed to prepare/enable pix main clk\n");
DRM_ERROR("Cannot register VTG notifier\n");
return -EINVAL;
}
+ hqvdp->vtg_registered = true;
}
DRM_DEBUG_KMS("CRTC:%d (%s) drm plane:%d (%s)\n",
if (!crtc || !fb)
return;
+ if ((oldstate->fb == state->fb) &&
+ (oldstate->crtc_x == state->crtc_x) &&
+ (oldstate->crtc_y == state->crtc_y) &&
+ (oldstate->crtc_w == state->crtc_w) &&
+ (oldstate->crtc_h == state->crtc_h) &&
+ (oldstate->src_x == state->src_x) &&
+ (oldstate->src_y == state->src_y) &&
+ (oldstate->src_w == state->src_w) &&
+ (oldstate->src_h == state->src_h)) {
+ /* No change since last update, do not post cmd */
+ DRM_DEBUG_DRIVER("No change, not posting cmd\n");
+ plane->status = STI_PLANE_UPDATED;
+ return;
+ }
+
mode = &crtc->mode;
dst_x = state->crtc_x;
dst_y = state->crtc_y;
fps->last_timestamp = now;
fps->last_frame_counter = fps->curr_frame_counter;
- fpks = (num_frames * 1000000) / ms_since_last;
- snprintf(plane->fps_info.fps_str, FPS_LENGTH, "%-6s @ %d.%.3d fps",
- sti_plane_to_str(plane), fpks / 1000, fpks % 1000);
+
+ if (plane->drm_plane.fb) {
+ fpks = (num_frames * 1000000) / ms_since_last;
+ snprintf(plane->fps_info.fps_str, FPS_LENGTH,
+ "%-8s %4dx%-4d %.4s @ %3d.%-3.3d fps (%s)",
+ plane->drm_plane.name,
+ plane->drm_plane.fb->width,
+ plane->drm_plane.fb->height,
+ (char *)&plane->drm_plane.fb->format->format,
+ fpks / 1000, fpks % 1000,
+ sti_plane_to_str(plane));
+ }
if (fps->curr_field_counter) {
/* Compute number of field updates */
fps->last_field_counter = fps->curr_field_counter;
fipks = (num_fields * 1000000) / ms_since_last;
snprintf(plane->fps_info.fips_str,
- FPS_LENGTH, " - %d.%.3d field/sec",
+ FPS_LENGTH, " - %3d.%-3.3d field/sec",
fipks / 1000, fipks % 1000);
} else {
plane->fps_info.fips_str[0] = '\0';
STI_PLANE_DISABLED,
};
-#define FPS_LENGTH 64
+#define FPS_LENGTH 128
struct sti_fps_info {
bool output;
unsigned int curr_frame_counter;
+++ /dev/null
-/*
- * Copyright (C) STMicroelectronics SA 2014
- * Author: Benjamin Gaignard <benjamin.gaignard@st.com> for STMicroelectronics.
- * License terms: GNU General Public License (GPL), version 2
- */
-
-#include <linux/clk.h>
-#include <linux/io.h>
-#include <linux/module.h>
-#include <linux/of.h>
-#include <linux/platform_device.h>
-
-#include <drm/drmP.h>
-
-#include "sti_drv.h"
-
-/* registers offset */
-#define VTAC_CONFIG 0x00
-#define VTAC_RX_FIFO_CONFIG 0x04
-#define VTAC_FIFO_CONFIG_VAL 0x04
-
-#define VTAC_SYS_CFG8521 0x824
-#define VTAC_SYS_CFG8522 0x828
-
-/* Number of phyts per pixel */
-#define VTAC_2_5_PPP 0x0005
-#define VTAC_3_PPP 0x0006
-#define VTAC_4_PPP 0x0008
-#define VTAC_5_PPP 0x000A
-#define VTAC_6_PPP 0x000C
-#define VTAC_13_PPP 0x001A
-#define VTAC_14_PPP 0x001C
-#define VTAC_15_PPP 0x001E
-#define VTAC_16_PPP 0x0020
-#define VTAC_17_PPP 0x0022
-#define VTAC_18_PPP 0x0024
-
-/* enable bits */
-#define VTAC_ENABLE 0x3003
-
-#define VTAC_TX_PHY_ENABLE_CLK_PHY BIT(0)
-#define VTAC_TX_PHY_ENABLE_CLK_DLL BIT(1)
-#define VTAC_TX_PHY_PLL_NOT_OSC_MODE BIT(3)
-#define VTAC_TX_PHY_RST_N_DLL_SWITCH BIT(4)
-#define VTAC_TX_PHY_PROG_N3 BIT(9)
-
-
-/**
- * VTAC mode structure
- *
- * @vid_in_width: Video Data Resolution
- * @phyts_width: Width of phyt buses(phyt low and phyt high).
- * @phyts_per_pixel: Number of phyts sent per pixel
- */
-struct sti_vtac_mode {
- u32 vid_in_width;
- u32 phyts_width;
- u32 phyts_per_pixel;
-};
-
-static const struct sti_vtac_mode vtac_mode_main = {
- .vid_in_width = 0x2,
- .phyts_width = 0x2,
- .phyts_per_pixel = VTAC_5_PPP,
-};
-static const struct sti_vtac_mode vtac_mode_aux = {
- .vid_in_width = 0x1,
- .phyts_width = 0x0,
- .phyts_per_pixel = VTAC_17_PPP,
-};
-
-/**
- * VTAC structure
- *
- * @dev: pointer to device structure
- * @regs: ioremapped registers for RX and TX devices
- * @phy_regs: phy registers for TX device
- * @clk: clock
- * @mode: main or auxillary configuration mode
- */
-struct sti_vtac {
- struct device *dev;
- void __iomem *regs;
- void __iomem *phy_regs;
- struct clk *clk;
- const struct sti_vtac_mode *mode;
-};
-
-static void sti_vtac_rx_set_config(struct sti_vtac *vtac)
-{
- u32 config;
-
- /* Enable VTAC clock */
- if (clk_prepare_enable(vtac->clk))
- DRM_ERROR("Failed to prepare/enable vtac_rx clock.\n");
-
- writel(VTAC_FIFO_CONFIG_VAL, vtac->regs + VTAC_RX_FIFO_CONFIG);
-
- config = VTAC_ENABLE;
- config |= vtac->mode->vid_in_width << 4;
- config |= vtac->mode->phyts_width << 16;
- config |= vtac->mode->phyts_per_pixel << 23;
- writel(config, vtac->regs + VTAC_CONFIG);
-}
-
-static void sti_vtac_tx_set_config(struct sti_vtac *vtac)
-{
- u32 phy_config;
- u32 config;
-
- /* Enable VTAC clock */
- if (clk_prepare_enable(vtac->clk))
- DRM_ERROR("Failed to prepare/enable vtac_tx clock.\n");
-
- /* Configure vtac phy */
- phy_config = 0x00000000;
- writel(phy_config, vtac->phy_regs + VTAC_SYS_CFG8522);
- phy_config = VTAC_TX_PHY_ENABLE_CLK_PHY;
- writel(phy_config, vtac->phy_regs + VTAC_SYS_CFG8521);
- phy_config = readl(vtac->phy_regs + VTAC_SYS_CFG8521);
- phy_config |= VTAC_TX_PHY_PROG_N3;
- writel(phy_config, vtac->phy_regs + VTAC_SYS_CFG8521);
- phy_config = readl(vtac->phy_regs + VTAC_SYS_CFG8521);
- phy_config |= VTAC_TX_PHY_ENABLE_CLK_DLL;
- writel(phy_config, vtac->phy_regs + VTAC_SYS_CFG8521);
- phy_config = readl(vtac->phy_regs + VTAC_SYS_CFG8521);
- phy_config |= VTAC_TX_PHY_RST_N_DLL_SWITCH;
- writel(phy_config, vtac->phy_regs + VTAC_SYS_CFG8521);
- phy_config = readl(vtac->phy_regs + VTAC_SYS_CFG8521);
- phy_config |= VTAC_TX_PHY_PLL_NOT_OSC_MODE;
- writel(phy_config, vtac->phy_regs + VTAC_SYS_CFG8521);
-
- /* Configure vtac tx */
- config = VTAC_ENABLE;
- config |= vtac->mode->vid_in_width << 4;
- config |= vtac->mode->phyts_width << 16;
- config |= vtac->mode->phyts_per_pixel << 23;
- writel(config, vtac->regs + VTAC_CONFIG);
-}
-
-static const struct of_device_id vtac_of_match[] = {
- {
- .compatible = "st,vtac-main",
- .data = &vtac_mode_main,
- }, {
- .compatible = "st,vtac-aux",
- .data = &vtac_mode_aux,
- }, {
- /* end node */
- }
-};
-MODULE_DEVICE_TABLE(of, vtac_of_match);
-
-static int sti_vtac_probe(struct platform_device *pdev)
-{
- struct device *dev = &pdev->dev;
- struct device_node *np = dev->of_node;
- const struct of_device_id *id;
- struct sti_vtac *vtac;
- struct resource *res;
-
- vtac = devm_kzalloc(dev, sizeof(*vtac), GFP_KERNEL);
- if (!vtac)
- return -ENOMEM;
-
- vtac->dev = dev;
-
- id = of_match_node(vtac_of_match, np);
- if (!id)
- return -ENOMEM;
-
- vtac->mode = id->data;
-
- res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- if (!res) {
- DRM_ERROR("Invalid resource\n");
- return -ENOMEM;
- }
- vtac->regs = devm_ioremap_resource(dev, res);
- if (IS_ERR(vtac->regs))
- return PTR_ERR(vtac->regs);
-
-
- vtac->clk = devm_clk_get(dev, "vtac");
- if (IS_ERR(vtac->clk)) {
- DRM_ERROR("Cannot get vtac clock\n");
- return PTR_ERR(vtac->clk);
- }
-
- res = platform_get_resource(pdev, IORESOURCE_MEM, 1);
- if (res) {
- vtac->phy_regs = devm_ioremap_nocache(dev, res->start,
- resource_size(res));
- sti_vtac_tx_set_config(vtac);
- } else {
-
- sti_vtac_rx_set_config(vtac);
- }
-
- platform_set_drvdata(pdev, vtac);
- DRM_INFO("%s %s\n", __func__, dev_name(vtac->dev));
-
- return 0;
-}
-
-static int sti_vtac_remove(struct platform_device *pdev)
-{
- return 0;
-}
-
-struct platform_driver sti_vtac_driver = {
- .driver = {
- .name = "sti-vtac",
- .owner = THIS_MODULE,
- .of_match_table = vtac_of_match,
- },
- .probe = sti_vtac_probe,
- .remove = sti_vtac_remove,
-};
-
-MODULE_AUTHOR("Benjamin Gaignard <benjamin.gaignard@st.com>");
-MODULE_DESCRIPTION("STMicroelectronics SoC DRM driver");
-MODULE_LICENSE("GPL");
return -ENOMEM;
}
vtg->regs = devm_ioremap_nocache(dev, res->start, resource_size(res));
+ if (!vtg->regs) {
+ DRM_ERROR("failed to remap I/O memory\n");
+ return -ENOMEM;
+ }
np = of_parse_phandle(pdev->dev.of_node, "st,slave", 0);
if (np) {
struct tilcdc_crtc *tilcdc_crtc = to_tilcdc_crtc(crtc);
struct drm_device *dev = crtc->dev;
struct tilcdc_drm_private *priv = dev->dev_private;
- uint32_t stat;
+ uint32_t stat, reg;
stat = tilcdc_read_irqstatus(dev);
tilcdc_clear_irqstatus(dev, stat);
dev_err_ratelimited(dev->dev, "%s(0x%08x): Sync lost",
__func__, stat);
tilcdc_crtc->frame_intact = false;
- if (tilcdc_crtc->sync_lost_count++ >
- SYNC_LOST_COUNT_LIMIT) {
- dev_err(dev->dev, "%s(0x%08x): Sync lost flood detected, recovering", __func__, stat);
- queue_work(system_wq, &tilcdc_crtc->recover_work);
- if (priv->rev == 1)
+ if (priv->rev == 1) {
+ reg = tilcdc_read(dev, LCDC_RASTER_CTRL_REG);
+ if (reg & LCDC_RASTER_ENABLE) {
tilcdc_clear(dev, LCDC_RASTER_CTRL_REG,
- LCDC_V1_SYNC_LOST_INT_ENA);
- else
+ LCDC_RASTER_ENABLE);
+ tilcdc_set(dev, LCDC_RASTER_CTRL_REG,
+ LCDC_RASTER_ENABLE);
+ }
+ } else {
+ if (tilcdc_crtc->sync_lost_count++ >
+ SYNC_LOST_COUNT_LIMIT) {
+ dev_err(dev->dev,
+ "%s(0x%08x): Sync lost flood detected, recovering",
+ __func__, stat);
+ queue_work(system_wq,
+ &tilcdc_crtc->recover_work);
tilcdc_write(dev, LCDC_INT_ENABLE_CLR_REG,
LCDC_SYNC_LOST);
- tilcdc_crtc->sync_lost_count = 0;
+ tilcdc_crtc->sync_lost_count = 0;
+ }
}
}
info->fbops = &virtio_gpufb_ops;
info->pixmap.flags = FB_PIXMAP_SYSTEM;
- info->screen_base = obj->vmap;
+ info->screen_buffer = obj->vmap;
info->screen_size = obj->gem_base.size;
drm_fb_helper_fill_fix(info, fb->pitches[0], fb->format->depth);
drm_fb_helper_fill_var(info, &vfbdev->helper,
select DRM_KMS_CMA_HELPER
select DRM_KMS_FB_HELPER
select DRM_KMS_HELPER
+ select SND_SOC_HDMI_CODEC if SND_SOC
+ select VIDEOMODE_HELPERS
help
Choose this option to enable DRM on ZTE ZX SoCs.
zx_drm_drv.o \
zx_hdmi.o \
zx_plane.o \
+ zx_tvenc.o \
zx_vou.o
obj-$(CONFIG_DRM_ZTE) += zxdrm.o
static struct platform_driver *drivers[] = {
&zx_crtc_driver,
&zx_hdmi_driver,
+ &zx_tvenc_driver,
&zx_drm_platform_driver,
};
extern struct platform_driver zx_crtc_driver;
extern struct platform_driver zx_hdmi_driver;
+extern struct platform_driver zx_tvenc_driver;
static inline u32 zx_readl(void __iomem *reg)
{
#include <drm/drm_of.h>
#include <drm/drmP.h>
+#include <sound/hdmi-codec.h>
+
#include "zx_hdmi_regs.h"
#include "zx_vou.h"
bool sink_is_hdmi;
bool sink_has_audio;
const struct vou_inf *inf;
+ struct platform_device *audio_pdev;
};
#define to_zx_hdmi(x) container_of(x, struct zx_hdmi, x)
-static const struct vou_inf vou_inf_hdmi = {
- .id = VOU_HDMI,
- .data_sel = VOU_YUV444,
- .clocks_en_bits = BIT(24) | BIT(18) | BIT(6),
- .clocks_sel_bits = BIT(13) | BIT(2),
-};
-
static inline u8 hdmi_readb(struct zx_hdmi *hdmi, u16 offset)
{
return readl_relaxed(hdmi->mmio + offset * 4);
zx_hdmi_hw_enable(hdmi);
- vou_inf_enable(hdmi->inf, encoder->crtc);
+ vou_inf_enable(VOU_HDMI, encoder->crtc);
}
static void zx_hdmi_encoder_disable(struct drm_encoder *encoder)
{
struct zx_hdmi *hdmi = to_zx_hdmi(encoder);
- vou_inf_disable(hdmi->inf, encoder->crtc);
+ vou_inf_disable(VOU_HDMI, encoder->crtc);
zx_hdmi_hw_disable(hdmi);
return IRQ_NONE;
}
+static int zx_hdmi_audio_startup(struct device *dev, void *data)
+{
+ struct zx_hdmi *hdmi = dev_get_drvdata(dev);
+ struct drm_encoder *encoder = &hdmi->encoder;
+
+ vou_inf_hdmi_audio_sel(encoder->crtc, VOU_HDMI_AUD_SPDIF);
+
+ return 0;
+}
+
+static void zx_hdmi_audio_shutdown(struct device *dev, void *data)
+{
+ struct zx_hdmi *hdmi = dev_get_drvdata(dev);
+
+ /* Disable audio input */
+ hdmi_writeb_mask(hdmi, AUD_EN, AUD_IN_EN, 0);
+}
+
+static inline int zx_hdmi_audio_get_n(unsigned int fs)
+{
+ unsigned int n;
+
+ if (fs && (fs % 44100) == 0)
+ n = 6272 * (fs / 44100);
+ else
+ n = fs * 128 / 1000;
+
+ return n;
+}
+
+static int zx_hdmi_audio_hw_params(struct device *dev,
+ void *data,
+ struct hdmi_codec_daifmt *daifmt,
+ struct hdmi_codec_params *params)
+{
+ struct zx_hdmi *hdmi = dev_get_drvdata(dev);
+ struct hdmi_audio_infoframe *cea = ¶ms->cea;
+ union hdmi_infoframe frame;
+ int n;
+
+ /* We only support spdif for now */
+ if (daifmt->fmt != HDMI_SPDIF) {
+ DRM_DEV_ERROR(hdmi->dev, "invalid daifmt %d\n", daifmt->fmt);
+ return -EINVAL;
+ }
+
+ switch (params->sample_width) {
+ case 16:
+ hdmi_writeb_mask(hdmi, TPI_AUD_CONFIG, SPDIF_SAMPLE_SIZE_MASK,
+ SPDIF_SAMPLE_SIZE_16BIT);
+ break;
+ case 20:
+ hdmi_writeb_mask(hdmi, TPI_AUD_CONFIG, SPDIF_SAMPLE_SIZE_MASK,
+ SPDIF_SAMPLE_SIZE_20BIT);
+ break;
+ case 24:
+ hdmi_writeb_mask(hdmi, TPI_AUD_CONFIG, SPDIF_SAMPLE_SIZE_MASK,
+ SPDIF_SAMPLE_SIZE_24BIT);
+ break;
+ default:
+ DRM_DEV_ERROR(hdmi->dev, "invalid sample width %d\n",
+ params->sample_width);
+ return -EINVAL;
+ }
+
+ /* CTS is calculated by hardware, and we only need to take care of N */
+ n = zx_hdmi_audio_get_n(params->sample_rate);
+ hdmi_writeb(hdmi, N_SVAL1, n & 0xff);
+ hdmi_writeb(hdmi, N_SVAL2, (n >> 8) & 0xff);
+ hdmi_writeb(hdmi, N_SVAL3, (n >> 16) & 0xf);
+
+ /* Enable spdif mode */
+ hdmi_writeb_mask(hdmi, AUD_MODE, SPDIF_EN, SPDIF_EN);
+
+ /* Enable audio input */
+ hdmi_writeb_mask(hdmi, AUD_EN, AUD_IN_EN, AUD_IN_EN);
+
+ memcpy(&frame.audio, cea, sizeof(*cea));
+
+ return zx_hdmi_infoframe_trans(hdmi, &frame, FSEL_AUDIO);
+}
+
+static int zx_hdmi_audio_digital_mute(struct device *dev, void *data,
+ bool enable)
+{
+ struct zx_hdmi *hdmi = dev_get_drvdata(dev);
+
+ if (enable)
+ hdmi_writeb_mask(hdmi, TPI_AUD_CONFIG, TPI_AUD_MUTE,
+ TPI_AUD_MUTE);
+ else
+ hdmi_writeb_mask(hdmi, TPI_AUD_CONFIG, TPI_AUD_MUTE, 0);
+
+ return 0;
+}
+
+static int zx_hdmi_audio_get_eld(struct device *dev, void *data,
+ uint8_t *buf, size_t len)
+{
+ struct zx_hdmi *hdmi = dev_get_drvdata(dev);
+ struct drm_connector *connector = &hdmi->connector;
+
+ memcpy(buf, connector->eld, min(sizeof(connector->eld), len));
+
+ return 0;
+}
+
+static const struct hdmi_codec_ops zx_hdmi_codec_ops = {
+ .audio_startup = zx_hdmi_audio_startup,
+ .hw_params = zx_hdmi_audio_hw_params,
+ .audio_shutdown = zx_hdmi_audio_shutdown,
+ .digital_mute = zx_hdmi_audio_digital_mute,
+ .get_eld = zx_hdmi_audio_get_eld,
+};
+
+static struct hdmi_codec_pdata zx_hdmi_codec_pdata = {
+ .ops = &zx_hdmi_codec_ops,
+ .spdif = 1,
+};
+
+static int zx_hdmi_audio_register(struct zx_hdmi *hdmi)
+{
+ struct platform_device *pdev;
+
+ pdev = platform_device_register_data(hdmi->dev, HDMI_CODEC_DRV_NAME,
+ PLATFORM_DEVID_AUTO,
+ &zx_hdmi_codec_pdata,
+ sizeof(zx_hdmi_codec_pdata));
+ if (IS_ERR(pdev))
+ return PTR_ERR(pdev);
+
+ hdmi->audio_pdev = pdev;
+
+ return 0;
+}
+
static int zx_hdmi_i2c_read(struct zx_hdmi *hdmi, struct i2c_msg *msg)
{
int len = msg->len;
hdmi->dev = dev;
hdmi->drm = drm;
- hdmi->inf = &vou_inf_hdmi;
dev_set_drvdata(dev, hdmi);
return ret;
}
+ ret = zx_hdmi_audio_register(hdmi);
+ if (ret) {
+ DRM_DEV_ERROR(dev, "failed to register audio: %d\n", ret);
+ return ret;
+ }
+
ret = zx_hdmi_register(drm, hdmi);
if (ret) {
DRM_DEV_ERROR(dev, "failed to register hdmi: %d\n", ret);
hdmi->connector.funcs->destroy(&hdmi->connector);
hdmi->encoder.funcs->destroy(&hdmi->encoder);
+
+ if (hdmi->audio_pdev)
+ platform_device_unregister(hdmi->audio_pdev);
}
static const struct component_ops zx_hdmi_component_ops = {
#define TPI_INFO_TRANS_RPT BIT(6)
#define TPI_DDC_MASTER_EN 0x06f8
#define HW_DDC_MASTER BIT(7)
+#define N_SVAL1 0xa03
+#define N_SVAL2 0xa04
+#define N_SVAL3 0xa05
+#define AUD_EN 0xa13
+#define AUD_IN_EN BIT(0)
+#define AUD_MODE 0xa14
+#define SPDIF_EN BIT(1)
+#define TPI_AUD_CONFIG 0xa62
+#define SPDIF_SAMPLE_SIZE_SHIFT 6
+#define SPDIF_SAMPLE_SIZE_MASK (0x3 << SPDIF_SAMPLE_SIZE_SHIFT)
+#define SPDIF_SAMPLE_SIZE_16BIT (0x1 << SPDIF_SAMPLE_SIZE_SHIFT)
+#define SPDIF_SAMPLE_SIZE_20BIT (0x2 << SPDIF_SAMPLE_SIZE_SHIFT)
+#define SPDIF_SAMPLE_SIZE_24BIT (0x3 << SPDIF_SAMPLE_SIZE_SHIFT)
+#define TPI_AUD_MUTE BIT(4)
#endif /* __ZX_HDMI_REGS_H__ */
#include "zx_plane_regs.h"
#include "zx_vou.h"
-struct zx_plane {
- struct drm_plane plane;
- void __iomem *layer;
- void __iomem *csc;
- void __iomem *hbsc;
- void __iomem *rsz;
-};
-
-#define to_zx_plane(plane) container_of(plane, struct zx_plane, plane)
-
static const uint32_t gl_formats[] = {
DRM_FORMAT_ARGB8888,
DRM_FORMAT_XRGB8888,
DRM_FORMAT_ARGB4444,
};
+static const uint32_t vl_formats[] = {
+ DRM_FORMAT_NV12, /* Semi-planar YUV420 */
+ DRM_FORMAT_YUV420, /* Planar YUV420 */
+ DRM_FORMAT_YUYV, /* Packed YUV422 */
+ DRM_FORMAT_YVYU,
+ DRM_FORMAT_UYVY,
+ DRM_FORMAT_VYUY,
+ DRM_FORMAT_YUV444, /* YUV444 8bit */
+ /*
+ * TODO: add formats below that HW supports:
+ * - YUV420 P010
+ * - YUV420 Hantro
+ * - YUV444 10bit
+ */
+};
+
+#define FRAC_16_16(mult, div) (((mult) << 16) / (div))
+
+static int zx_vl_plane_atomic_check(struct drm_plane *plane,
+ struct drm_plane_state *plane_state)
+{
+ struct drm_framebuffer *fb = plane_state->fb;
+ struct drm_crtc *crtc = plane_state->crtc;
+ struct drm_crtc_state *crtc_state;
+ struct drm_rect clip;
+ int min_scale = FRAC_16_16(1, 8);
+ int max_scale = FRAC_16_16(8, 1);
+
+ if (!crtc || !fb)
+ return 0;
+
+ crtc_state = drm_atomic_get_existing_crtc_state(plane_state->state,
+ crtc);
+ if (WARN_ON(!crtc_state))
+ return -EINVAL;
+
+ /* nothing to check when disabling or disabled */
+ if (!crtc_state->enable)
+ return 0;
+
+ /* plane must be enabled */
+ if (!plane_state->crtc)
+ return -EINVAL;
+
+ clip.x1 = 0;
+ clip.y1 = 0;
+ clip.x2 = crtc_state->adjusted_mode.hdisplay;
+ clip.y2 = crtc_state->adjusted_mode.vdisplay;
+
+ return drm_plane_helper_check_state(plane_state, &clip,
+ min_scale, max_scale,
+ true, true);
+}
+
+static int zx_vl_get_fmt(uint32_t format)
+{
+ switch (format) {
+ case DRM_FORMAT_NV12:
+ return VL_FMT_YUV420;
+ case DRM_FORMAT_YUV420:
+ return VL_YUV420_PLANAR | VL_FMT_YUV420;
+ case DRM_FORMAT_YUYV:
+ return VL_YUV422_YUYV | VL_FMT_YUV422;
+ case DRM_FORMAT_YVYU:
+ return VL_YUV422_YVYU | VL_FMT_YUV422;
+ case DRM_FORMAT_UYVY:
+ return VL_YUV422_UYVY | VL_FMT_YUV422;
+ case DRM_FORMAT_VYUY:
+ return VL_YUV422_VYUY | VL_FMT_YUV422;
+ case DRM_FORMAT_YUV444:
+ return VL_FMT_YUV444_8BIT;
+ default:
+ WARN_ONCE(1, "invalid pixel format %d\n", format);
+ return -EINVAL;
+ }
+}
+
+static inline void zx_vl_set_update(struct zx_plane *zplane)
+{
+ void __iomem *layer = zplane->layer;
+
+ zx_writel_mask(layer + VL_CTRL0, VL_UPDATE, VL_UPDATE);
+}
+
+static inline void zx_vl_rsz_set_update(struct zx_plane *zplane)
+{
+ zx_writel(zplane->rsz + RSZ_VL_ENABLE_CFG, 1);
+}
+
+static int zx_vl_rsz_get_fmt(uint32_t format)
+{
+ switch (format) {
+ case DRM_FORMAT_NV12:
+ case DRM_FORMAT_YUV420:
+ return RSZ_VL_FMT_YCBCR420;
+ case DRM_FORMAT_YUYV:
+ case DRM_FORMAT_YVYU:
+ case DRM_FORMAT_UYVY:
+ case DRM_FORMAT_VYUY:
+ return RSZ_VL_FMT_YCBCR422;
+ case DRM_FORMAT_YUV444:
+ return RSZ_VL_FMT_YCBCR444;
+ default:
+ WARN_ONCE(1, "invalid pixel format %d\n", format);
+ return -EINVAL;
+ }
+}
+
+static inline u32 rsz_step_value(u32 src, u32 dst)
+{
+ u32 val = 0;
+
+ if (src == dst)
+ val = 0;
+ else if (src < dst)
+ val = RSZ_PARA_STEP((src << 16) / dst);
+ else if (src > dst)
+ val = RSZ_DATA_STEP(src / dst) |
+ RSZ_PARA_STEP(((src << 16) / dst) & 0xffff);
+
+ return val;
+}
+
+static void zx_vl_rsz_setup(struct zx_plane *zplane, uint32_t format,
+ u32 src_w, u32 src_h, u32 dst_w, u32 dst_h)
+{
+ void __iomem *rsz = zplane->rsz;
+ u32 src_chroma_w = src_w;
+ u32 src_chroma_h = src_h;
+ u32 fmt;
+
+ /* Set up source and destination resolution */
+ zx_writel(rsz + RSZ_SRC_CFG, RSZ_VER(src_h - 1) | RSZ_HOR(src_w - 1));
+ zx_writel(rsz + RSZ_DEST_CFG, RSZ_VER(dst_h - 1) | RSZ_HOR(dst_w - 1));
+
+ /* Configure data format for VL RSZ */
+ fmt = zx_vl_rsz_get_fmt(format);
+ if (fmt >= 0)
+ zx_writel_mask(rsz + RSZ_VL_CTRL_CFG, RSZ_VL_FMT_MASK, fmt);
+
+ /* Calculate Chroma height and width */
+ if (fmt == RSZ_VL_FMT_YCBCR420) {
+ src_chroma_w = src_w >> 1;
+ src_chroma_h = src_h >> 1;
+ } else if (fmt == RSZ_VL_FMT_YCBCR422) {
+ src_chroma_w = src_w >> 1;
+ }
+
+ /* Set up Luma and Chroma step registers */
+ zx_writel(rsz + RSZ_VL_LUMA_HOR, rsz_step_value(src_w, dst_w));
+ zx_writel(rsz + RSZ_VL_LUMA_VER, rsz_step_value(src_h, dst_h));
+ zx_writel(rsz + RSZ_VL_CHROMA_HOR, rsz_step_value(src_chroma_w, dst_w));
+ zx_writel(rsz + RSZ_VL_CHROMA_VER, rsz_step_value(src_chroma_h, dst_h));
+
+ zx_vl_rsz_set_update(zplane);
+}
+
+static void zx_vl_plane_atomic_update(struct drm_plane *plane,
+ struct drm_plane_state *old_state)
+{
+ struct zx_plane *zplane = to_zx_plane(plane);
+ struct drm_plane_state *state = plane->state;
+ struct drm_framebuffer *fb = state->fb;
+ struct drm_rect *src = &state->src;
+ struct drm_rect *dst = &state->dst;
+ struct drm_gem_cma_object *cma_obj;
+ void __iomem *layer = zplane->layer;
+ void __iomem *hbsc = zplane->hbsc;
+ void __iomem *paddr_reg;
+ dma_addr_t paddr;
+ u32 src_x, src_y, src_w, src_h;
+ u32 dst_x, dst_y, dst_w, dst_h;
+ uint32_t format;
+ u32 fmt;
+ int num_planes;
+ int i;
+
+ if (!fb)
+ return;
+
+ format = fb->format->format;
+
+ src_x = src->x1 >> 16;
+ src_y = src->y1 >> 16;
+ src_w = drm_rect_width(src) >> 16;
+ src_h = drm_rect_height(src) >> 16;
+
+ dst_x = dst->x1;
+ dst_y = dst->y1;
+ dst_w = drm_rect_width(dst);
+ dst_h = drm_rect_height(dst);
+
+ /* Set up data address registers for Y, Cb and Cr planes */
+ num_planes = drm_format_num_planes(format);
+ paddr_reg = layer + VL_Y;
+ for (i = 0; i < num_planes; i++) {
+ cma_obj = drm_fb_cma_get_gem_obj(fb, i);
+ paddr = cma_obj->paddr + fb->offsets[i];
+ paddr += src_y * fb->pitches[i];
+ paddr += src_x * drm_format_plane_cpp(format, i);
+ zx_writel(paddr_reg, paddr);
+ paddr_reg += 4;
+ }
+
+ /* Set up source height/width register */
+ zx_writel(layer + VL_SRC_SIZE, GL_SRC_W(src_w) | GL_SRC_H(src_h));
+
+ /* Set up start position register */
+ zx_writel(layer + VL_POS_START, GL_POS_X(dst_x) | GL_POS_Y(dst_y));
+
+ /* Set up end position register */
+ zx_writel(layer + VL_POS_END,
+ GL_POS_X(dst_x + dst_w) | GL_POS_Y(dst_y + dst_h));
+
+ /* Strides of Cb and Cr planes should be identical */
+ zx_writel(layer + VL_STRIDE, LUMA_STRIDE(fb->pitches[0]) |
+ CHROMA_STRIDE(fb->pitches[1]));
+
+ /* Set up video layer data format */
+ fmt = zx_vl_get_fmt(format);
+ if (fmt >= 0)
+ zx_writel(layer + VL_CTRL1, fmt);
+
+ /* Always use scaler since it exists (set for not bypass) */
+ zx_writel_mask(layer + VL_CTRL2, VL_SCALER_BYPASS_MODE,
+ VL_SCALER_BYPASS_MODE);
+
+ zx_vl_rsz_setup(zplane, format, src_w, src_h, dst_w, dst_h);
+
+ /* Enable HBSC block */
+ zx_writel_mask(hbsc + HBSC_CTRL0, HBSC_CTRL_EN, HBSC_CTRL_EN);
+
+ zx_vou_layer_enable(plane);
+
+ zx_vl_set_update(zplane);
+}
+
+static void zx_plane_atomic_disable(struct drm_plane *plane,
+ struct drm_plane_state *old_state)
+{
+ struct zx_plane *zplane = to_zx_plane(plane);
+ void __iomem *hbsc = zplane->hbsc;
+
+ zx_vou_layer_disable(plane);
+
+ /* Disable HBSC block */
+ zx_writel_mask(hbsc + HBSC_CTRL0, HBSC_CTRL_EN, 0);
+}
+
+static const struct drm_plane_helper_funcs zx_vl_plane_helper_funcs = {
+ .atomic_check = zx_vl_plane_atomic_check,
+ .atomic_update = zx_vl_plane_atomic_update,
+ .atomic_disable = zx_plane_atomic_disable,
+};
+
static int zx_gl_plane_atomic_check(struct drm_plane *plane,
struct drm_plane_state *plane_state)
{
zx_writel(zplane->rsz + RSZ_ENABLE_CFG, 1);
}
-void zx_plane_set_update(struct drm_plane *plane)
-{
- struct zx_plane *zplane = to_zx_plane(plane);
-
- zx_gl_rsz_set_update(zplane);
- zx_gl_set_update(zplane);
-}
-
static void zx_gl_rsz_setup(struct zx_plane *zplane, u32 src_w, u32 src_h,
u32 dst_w, u32 dst_h)
{
/* Enable HBSC block */
zx_writel_mask(hbsc + HBSC_CTRL0, HBSC_CTRL_EN, HBSC_CTRL_EN);
+ zx_vou_layer_enable(plane);
+
zx_gl_set_update(zplane);
}
static const struct drm_plane_helper_funcs zx_gl_plane_helper_funcs = {
.atomic_check = zx_gl_plane_atomic_check,
.atomic_update = zx_gl_plane_atomic_update,
+ .atomic_disable = zx_plane_atomic_disable,
};
static void zx_plane_destroy(struct drm_plane *plane)
.atomic_destroy_state = drm_atomic_helper_plane_destroy_state,
};
+void zx_plane_set_update(struct drm_plane *plane)
+{
+ struct zx_plane *zplane = to_zx_plane(plane);
+
+ /* Do nothing if the plane is not enabled */
+ if (!plane->state->crtc)
+ return;
+
+ switch (plane->type) {
+ case DRM_PLANE_TYPE_PRIMARY:
+ zx_gl_rsz_set_update(zplane);
+ zx_gl_set_update(zplane);
+ break;
+ case DRM_PLANE_TYPE_OVERLAY:
+ zx_vl_rsz_set_update(zplane);
+ zx_vl_set_update(zplane);
+ break;
+ default:
+ WARN_ONCE(1, "unsupported plane type %d\n", plane->type);
+ }
+}
+
static void zx_plane_hbsc_init(struct zx_plane *zplane)
{
void __iomem *hbsc = zplane->hbsc;
zx_writel(hbsc + HBSC_THRESHOLD_COL3, (0x3c0 << 16) | 0x40);
}
-struct drm_plane *zx_plane_init(struct drm_device *drm, struct device *dev,
- struct zx_layer_data *data,
- enum drm_plane_type type)
+int zx_plane_init(struct drm_device *drm, struct zx_plane *zplane,
+ enum drm_plane_type type)
{
const struct drm_plane_helper_funcs *helper;
- struct zx_plane *zplane;
- struct drm_plane *plane;
+ struct drm_plane *plane = &zplane->plane;
+ struct device *dev = zplane->dev;
const uint32_t *formats;
unsigned int format_count;
int ret;
- zplane = devm_kzalloc(dev, sizeof(*zplane), GFP_KERNEL);
- if (!zplane)
- return ERR_PTR(-ENOMEM);
-
- plane = &zplane->plane;
-
- zplane->layer = data->layer;
- zplane->hbsc = data->hbsc;
- zplane->csc = data->csc;
- zplane->rsz = data->rsz;
-
zx_plane_hbsc_init(zplane);
switch (type) {
format_count = ARRAY_SIZE(gl_formats);
break;
case DRM_PLANE_TYPE_OVERLAY:
- /* TODO: add video layer (vl) support */
+ helper = &zx_vl_plane_helper_funcs;
+ formats = vl_formats;
+ format_count = ARRAY_SIZE(vl_formats);
break;
default:
- return ERR_PTR(-ENODEV);
+ return -ENODEV;
}
ret = drm_universal_plane_init(drm, plane, VOU_CRTC_MASK,
type, NULL);
if (ret) {
DRM_DEV_ERROR(dev, "failed to init universal plane: %d\n", ret);
- return ERR_PTR(ret);
+ return ret;
}
drm_plane_helper_add(plane, helper);
- return plane;
+ return 0;
}
#ifndef __ZX_PLANE_H__
#define __ZX_PLANE_H__
-struct zx_layer_data {
+struct zx_plane {
+ struct drm_plane plane;
+ struct device *dev;
void __iomem *layer;
void __iomem *csc;
void __iomem *hbsc;
void __iomem *rsz;
+ const struct vou_layer_bits *bits;
};
-struct drm_plane *zx_plane_init(struct drm_device *drm, struct device *dev,
- struct zx_layer_data *data,
- enum drm_plane_type type);
+#define to_zx_plane(plane) container_of(plane, struct zx_plane, plane)
+
+int zx_plane_init(struct drm_device *drm, struct zx_plane *zplane,
+ enum drm_plane_type type);
void zx_plane_set_update(struct drm_plane *plane);
#endif /* __ZX_PLANE_H__ */
#define GL_POS_X(x) (((x) << GL_POS_X_SHIFT) & GL_POS_X_MASK)
#define GL_POS_Y(x) (((x) << GL_POS_Y_SHIFT) & GL_POS_Y_MASK)
+/* VL registers */
+#define VL_CTRL0 0x00
+#define VL_UPDATE BIT(3)
+#define VL_CTRL1 0x04
+#define VL_YUV420_PLANAR BIT(5)
+#define VL_YUV422_SHIFT 3
+#define VL_YUV422_YUYV (0 << VL_YUV422_SHIFT)
+#define VL_YUV422_YVYU (1 << VL_YUV422_SHIFT)
+#define VL_YUV422_UYVY (2 << VL_YUV422_SHIFT)
+#define VL_YUV422_VYUY (3 << VL_YUV422_SHIFT)
+#define VL_FMT_YUV420 0
+#define VL_FMT_YUV422 1
+#define VL_FMT_YUV420_P010 2
+#define VL_FMT_YUV420_HANTRO 3
+#define VL_FMT_YUV444_8BIT 4
+#define VL_FMT_YUV444_10BIT 5
+#define VL_CTRL2 0x08
+#define VL_SCALER_BYPASS_MODE BIT(0)
+#define VL_STRIDE 0x0c
+#define LUMA_STRIDE_SHIFT 16
+#define LUMA_STRIDE_MASK (0xffff << LUMA_STRIDE_SHIFT)
+#define CHROMA_STRIDE_SHIFT 0
+#define CHROMA_STRIDE_MASK (0xffff << CHROMA_STRIDE_SHIFT)
+#define VL_SRC_SIZE 0x10
+#define VL_Y 0x14
+#define VL_POS_START 0x30
+#define VL_POS_END 0x34
+
+#define LUMA_STRIDE(x) (((x) << LUMA_STRIDE_SHIFT) & LUMA_STRIDE_MASK)
+#define CHROMA_STRIDE(x) (((x) << CHROMA_STRIDE_SHIFT) & CHROMA_STRIDE_MASK)
+
/* CSC registers */
#define CSC_CTRL0 0x30
#define CSC_COV_MODE_SHIFT 16
#define RSZ_DEST_CFG 0x04
#define RSZ_ENABLE_CFG 0x14
+#define RSZ_VL_LUMA_HOR 0x08
+#define RSZ_VL_LUMA_VER 0x0c
+#define RSZ_VL_CHROMA_HOR 0x10
+#define RSZ_VL_CHROMA_VER 0x14
+#define RSZ_VL_CTRL_CFG 0x18
+#define RSZ_VL_FMT_SHIFT 3
+#define RSZ_VL_FMT_MASK (0x3 << RSZ_VL_FMT_SHIFT)
+#define RSZ_VL_FMT_YCBCR420 (0x0 << RSZ_VL_FMT_SHIFT)
+#define RSZ_VL_FMT_YCBCR422 (0x1 << RSZ_VL_FMT_SHIFT)
+#define RSZ_VL_FMT_YCBCR444 (0x2 << RSZ_VL_FMT_SHIFT)
+#define RSZ_VL_ENABLE_CFG 0x1c
+
#define RSZ_VER_SHIFT 16
#define RSZ_VER_MASK (0xffff << RSZ_VER_SHIFT)
#define RSZ_HOR_SHIFT 0
#define RSZ_VER(x) (((x) << RSZ_VER_SHIFT) & RSZ_VER_MASK)
#define RSZ_HOR(x) (((x) << RSZ_HOR_SHIFT) & RSZ_HOR_MASK)
+#define RSZ_DATA_STEP_SHIFT 16
+#define RSZ_DATA_STEP_MASK (0xffff << RSZ_DATA_STEP_SHIFT)
+#define RSZ_PARA_STEP_SHIFT 0
+#define RSZ_PARA_STEP_MASK (0xffff << RSZ_PARA_STEP_SHIFT)
+
+#define RSZ_DATA_STEP(x) (((x) << RSZ_DATA_STEP_SHIFT) & RSZ_DATA_STEP_MASK)
+#define RSZ_PARA_STEP(x) (((x) << RSZ_PARA_STEP_SHIFT) & RSZ_PARA_STEP_MASK)
+
/* HBSC registers */
#define HBSC_SATURATION 0x00
#define HBSC_HUE 0x04
--- /dev/null
+/*
+ * Copyright 2017 Linaro Ltd.
+ * Copyright 2017 ZTE Corporation.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ */
+
+#include <linux/clk.h>
+#include <linux/component.h>
+#include <linux/mfd/syscon.h>
+#include <linux/regmap.h>
+
+#include <drm/drm_atomic_helper.h>
+#include <drm/drm_crtc_helper.h>
+#include <drm/drmP.h>
+
+#include "zx_drm_drv.h"
+#include "zx_tvenc_regs.h"
+#include "zx_vou.h"
+
+struct zx_tvenc_pwrctrl {
+ struct regmap *regmap;
+ u32 reg;
+ u32 mask;
+};
+
+struct zx_tvenc {
+ struct drm_connector connector;
+ struct drm_encoder encoder;
+ struct device *dev;
+ void __iomem *mmio;
+ const struct vou_inf *inf;
+ struct zx_tvenc_pwrctrl pwrctrl;
+};
+
+#define to_zx_tvenc(x) container_of(x, struct zx_tvenc, x)
+
+struct zx_tvenc_mode {
+ struct drm_display_mode mode;
+ u32 video_info;
+ u32 video_res;
+ u32 field1_param;
+ u32 field2_param;
+ u32 burst_line_odd1;
+ u32 burst_line_even1;
+ u32 burst_line_odd2;
+ u32 burst_line_even2;
+ u32 line_timing_param;
+ u32 weight_value;
+ u32 blank_black_level;
+ u32 burst_level;
+ u32 control_param;
+ u32 sub_carrier_phase1;
+ u32 phase_line_incr_cvbs;
+};
+
+/*
+ * The CRM cannot directly provide a suitable frequency, and we have to
+ * ask a multiplied rate from CRM and use the divider in VOU to get the
+ * desired one.
+ */
+#define TVENC_CLOCK_MULTIPLIER 4
+
+static const struct zx_tvenc_mode tvenc_mode_pal = {
+ .mode = {
+ .clock = 13500 * TVENC_CLOCK_MULTIPLIER,
+ .hdisplay = 720,
+ .hsync_start = 720 + 12,
+ .hsync_end = 720 + 12 + 2,
+ .htotal = 720 + 12 + 2 + 130,
+ .vdisplay = 576,
+ .vsync_start = 576 + 2,
+ .vsync_end = 576 + 2 + 2,
+ .vtotal = 576 + 2 + 2 + 20,
+ .flags = DRM_MODE_FLAG_NHSYNC | DRM_MODE_FLAG_NVSYNC |
+ DRM_MODE_FLAG_INTERLACE,
+ },
+ .video_info = 0x00040040,
+ .video_res = 0x05a9c760,
+ .field1_param = 0x0004d416,
+ .field2_param = 0x0009b94f,
+ .burst_line_odd1 = 0x0004d406,
+ .burst_line_even1 = 0x0009b53e,
+ .burst_line_odd2 = 0x0004d805,
+ .burst_line_even2 = 0x0009b93f,
+ .line_timing_param = 0x06a96fdf,
+ .weight_value = 0x00c188a0,
+ .blank_black_level = 0x0000fcfc,
+ .burst_level = 0x00001595,
+ .control_param = 0x00000001,
+ .sub_carrier_phase1 = 0x1504c566,
+ .phase_line_incr_cvbs = 0xc068db8c,
+};
+
+static const struct zx_tvenc_mode tvenc_mode_ntsc = {
+ .mode = {
+ .clock = 13500 * TVENC_CLOCK_MULTIPLIER,
+ .hdisplay = 720,
+ .hsync_start = 720 + 16,
+ .hsync_end = 720 + 16 + 2,
+ .htotal = 720 + 16 + 2 + 120,
+ .vdisplay = 480,
+ .vsync_start = 480 + 3,
+ .vsync_end = 480 + 3 + 2,
+ .vtotal = 480 + 3 + 2 + 17,
+ .flags = DRM_MODE_FLAG_NHSYNC | DRM_MODE_FLAG_NVSYNC |
+ DRM_MODE_FLAG_INTERLACE,
+ },
+ .video_info = 0x00040080,
+ .video_res = 0x05a8375a,
+ .field1_param = 0x00041817,
+ .field2_param = 0x0008351e,
+ .burst_line_odd1 = 0x00041006,
+ .burst_line_even1 = 0x0008290d,
+ .burst_line_odd2 = 0x00000000,
+ .burst_line_even2 = 0x00000000,
+ .line_timing_param = 0x06a8ef9e,
+ .weight_value = 0x00b68197,
+ .blank_black_level = 0x0000f0f0,
+ .burst_level = 0x0000009c,
+ .control_param = 0x00000001,
+ .sub_carrier_phase1 = 0x10f83e10,
+ .phase_line_incr_cvbs = 0x80000000,
+};
+
+static const struct zx_tvenc_mode *tvenc_modes[] = {
+ &tvenc_mode_pal,
+ &tvenc_mode_ntsc,
+};
+
+static const struct zx_tvenc_mode *
+zx_tvenc_find_zmode(struct drm_display_mode *mode)
+{
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(tvenc_modes); i++) {
+ const struct zx_tvenc_mode *zmode = tvenc_modes[i];
+
+ if (drm_mode_equal(mode, &zmode->mode))
+ return zmode;
+ }
+
+ return NULL;
+}
+
+static void zx_tvenc_encoder_mode_set(struct drm_encoder *encoder,
+ struct drm_display_mode *mode,
+ struct drm_display_mode *adj_mode)
+{
+ struct zx_tvenc *tvenc = to_zx_tvenc(encoder);
+ const struct zx_tvenc_mode *zmode;
+ struct vou_div_config configs[] = {
+ { VOU_DIV_INF, VOU_DIV_4 },
+ { VOU_DIV_TVENC, VOU_DIV_1 },
+ { VOU_DIV_LAYER, VOU_DIV_2 },
+ };
+
+ zx_vou_config_dividers(encoder->crtc, configs, ARRAY_SIZE(configs));
+
+ zmode = zx_tvenc_find_zmode(mode);
+ if (!zmode) {
+ DRM_DEV_ERROR(tvenc->dev, "failed to find zmode\n");
+ return;
+ }
+
+ zx_writel(tvenc->mmio + VENC_VIDEO_INFO, zmode->video_info);
+ zx_writel(tvenc->mmio + VENC_VIDEO_RES, zmode->video_res);
+ zx_writel(tvenc->mmio + VENC_FIELD1_PARAM, zmode->field1_param);
+ zx_writel(tvenc->mmio + VENC_FIELD2_PARAM, zmode->field2_param);
+ zx_writel(tvenc->mmio + VENC_LINE_O_1, zmode->burst_line_odd1);
+ zx_writel(tvenc->mmio + VENC_LINE_E_1, zmode->burst_line_even1);
+ zx_writel(tvenc->mmio + VENC_LINE_O_2, zmode->burst_line_odd2);
+ zx_writel(tvenc->mmio + VENC_LINE_E_2, zmode->burst_line_even2);
+ zx_writel(tvenc->mmio + VENC_LINE_TIMING_PARAM,
+ zmode->line_timing_param);
+ zx_writel(tvenc->mmio + VENC_WEIGHT_VALUE, zmode->weight_value);
+ zx_writel(tvenc->mmio + VENC_BLANK_BLACK_LEVEL,
+ zmode->blank_black_level);
+ zx_writel(tvenc->mmio + VENC_BURST_LEVEL, zmode->burst_level);
+ zx_writel(tvenc->mmio + VENC_CONTROL_PARAM, zmode->control_param);
+ zx_writel(tvenc->mmio + VENC_SUB_CARRIER_PHASE1,
+ zmode->sub_carrier_phase1);
+ zx_writel(tvenc->mmio + VENC_PHASE_LINE_INCR_CVBS,
+ zmode->phase_line_incr_cvbs);
+}
+
+static void zx_tvenc_encoder_enable(struct drm_encoder *encoder)
+{
+ struct zx_tvenc *tvenc = to_zx_tvenc(encoder);
+ struct zx_tvenc_pwrctrl *pwrctrl = &tvenc->pwrctrl;
+
+ /* Set bit to power up TVENC DAC */
+ regmap_update_bits(pwrctrl->regmap, pwrctrl->reg, pwrctrl->mask,
+ pwrctrl->mask);
+
+ vou_inf_enable(VOU_TV_ENC, encoder->crtc);
+
+ zx_writel(tvenc->mmio + VENC_ENABLE, 1);
+}
+
+static void zx_tvenc_encoder_disable(struct drm_encoder *encoder)
+{
+ struct zx_tvenc *tvenc = to_zx_tvenc(encoder);
+ struct zx_tvenc_pwrctrl *pwrctrl = &tvenc->pwrctrl;
+
+ zx_writel(tvenc->mmio + VENC_ENABLE, 0);
+
+ vou_inf_disable(VOU_TV_ENC, encoder->crtc);
+
+ /* Clear bit to power down TVENC DAC */
+ regmap_update_bits(pwrctrl->regmap, pwrctrl->reg, pwrctrl->mask, 0);
+}
+
+static const struct drm_encoder_helper_funcs zx_tvenc_encoder_helper_funcs = {
+ .enable = zx_tvenc_encoder_enable,
+ .disable = zx_tvenc_encoder_disable,
+ .mode_set = zx_tvenc_encoder_mode_set,
+};
+
+static const struct drm_encoder_funcs zx_tvenc_encoder_funcs = {
+ .destroy = drm_encoder_cleanup,
+};
+
+static int zx_tvenc_connector_get_modes(struct drm_connector *connector)
+{
+ struct zx_tvenc *tvenc = to_zx_tvenc(connector);
+ struct device *dev = tvenc->dev;
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(tvenc_modes); i++) {
+ const struct zx_tvenc_mode *zmode = tvenc_modes[i];
+ struct drm_display_mode *mode;
+
+ mode = drm_mode_duplicate(connector->dev, &zmode->mode);
+ if (!mode) {
+ DRM_DEV_ERROR(dev, "failed to duplicate drm mode\n");
+ continue;
+ }
+
+ drm_mode_set_name(mode);
+ drm_mode_probed_add(connector, mode);
+ }
+
+ return i;
+}
+
+static enum drm_mode_status
+zx_tvenc_connector_mode_valid(struct drm_connector *connector,
+ struct drm_display_mode *mode)
+{
+ struct zx_tvenc *tvenc = to_zx_tvenc(connector);
+ const struct zx_tvenc_mode *zmode;
+
+ zmode = zx_tvenc_find_zmode(mode);
+ if (!zmode) {
+ DRM_DEV_ERROR(tvenc->dev, "unsupported mode: %s\n", mode->name);
+ return MODE_NOMODE;
+ }
+
+ return MODE_OK;
+}
+
+static struct drm_connector_helper_funcs zx_tvenc_connector_helper_funcs = {
+ .get_modes = zx_tvenc_connector_get_modes,
+ .mode_valid = zx_tvenc_connector_mode_valid,
+};
+
+static const struct drm_connector_funcs zx_tvenc_connector_funcs = {
+ .dpms = drm_atomic_helper_connector_dpms,
+ .fill_modes = drm_helper_probe_single_connector_modes,
+ .destroy = drm_connector_cleanup,
+ .reset = drm_atomic_helper_connector_reset,
+ .atomic_duplicate_state = drm_atomic_helper_connector_duplicate_state,
+ .atomic_destroy_state = drm_atomic_helper_connector_destroy_state,
+};
+
+static int zx_tvenc_register(struct drm_device *drm, struct zx_tvenc *tvenc)
+{
+ struct drm_encoder *encoder = &tvenc->encoder;
+ struct drm_connector *connector = &tvenc->connector;
+
+ /*
+ * The tvenc is designed to use aux channel, as there is a deflicker
+ * block for the channel.
+ */
+ encoder->possible_crtcs = BIT(1);
+
+ drm_encoder_init(drm, encoder, &zx_tvenc_encoder_funcs,
+ DRM_MODE_ENCODER_TVDAC, NULL);
+ drm_encoder_helper_add(encoder, &zx_tvenc_encoder_helper_funcs);
+
+ connector->interlace_allowed = true;
+
+ drm_connector_init(drm, connector, &zx_tvenc_connector_funcs,
+ DRM_MODE_CONNECTOR_Composite);
+ drm_connector_helper_add(connector, &zx_tvenc_connector_helper_funcs);
+
+ drm_mode_connector_attach_encoder(connector, encoder);
+
+ return 0;
+}
+
+static int zx_tvenc_pwrctrl_init(struct zx_tvenc *tvenc)
+{
+ struct zx_tvenc_pwrctrl *pwrctrl = &tvenc->pwrctrl;
+ struct device *dev = tvenc->dev;
+ struct of_phandle_args out_args;
+ struct regmap *regmap;
+ int ret;
+
+ ret = of_parse_phandle_with_fixed_args(dev->of_node,
+ "zte,tvenc-power-control", 2, 0, &out_args);
+ if (ret)
+ return ret;
+
+ regmap = syscon_node_to_regmap(out_args.np);
+ if (IS_ERR(regmap)) {
+ ret = PTR_ERR(regmap);
+ goto out;
+ }
+
+ pwrctrl->regmap = regmap;
+ pwrctrl->reg = out_args.args[0];
+ pwrctrl->mask = out_args.args[1];
+
+out:
+ of_node_put(out_args.np);
+ return ret;
+}
+
+static int zx_tvenc_bind(struct device *dev, struct device *master, void *data)
+{
+ struct platform_device *pdev = to_platform_device(dev);
+ struct drm_device *drm = data;
+ struct resource *res;
+ struct zx_tvenc *tvenc;
+ int ret;
+
+ tvenc = devm_kzalloc(dev, sizeof(*tvenc), GFP_KERNEL);
+ if (!tvenc)
+ return -ENOMEM;
+
+ tvenc->dev = dev;
+ dev_set_drvdata(dev, tvenc);
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ tvenc->mmio = devm_ioremap_resource(dev, res);
+ if (IS_ERR(tvenc->mmio)) {
+ ret = PTR_ERR(tvenc->mmio);
+ DRM_DEV_ERROR(dev, "failed to remap tvenc region: %d\n", ret);
+ return ret;
+ }
+
+ ret = zx_tvenc_pwrctrl_init(tvenc);
+ if (ret) {
+ DRM_DEV_ERROR(dev, "failed to init power control: %d\n", ret);
+ return ret;
+ }
+
+ ret = zx_tvenc_register(drm, tvenc);
+ if (ret) {
+ DRM_DEV_ERROR(dev, "failed to register tvenc: %d\n", ret);
+ return ret;
+ }
+
+ return 0;
+}
+
+static void zx_tvenc_unbind(struct device *dev, struct device *master,
+ void *data)
+{
+ /* Nothing to do */
+}
+
+static const struct component_ops zx_tvenc_component_ops = {
+ .bind = zx_tvenc_bind,
+ .unbind = zx_tvenc_unbind,
+};
+
+static int zx_tvenc_probe(struct platform_device *pdev)
+{
+ return component_add(&pdev->dev, &zx_tvenc_component_ops);
+}
+
+static int zx_tvenc_remove(struct platform_device *pdev)
+{
+ component_del(&pdev->dev, &zx_tvenc_component_ops);
+ return 0;
+}
+
+static const struct of_device_id zx_tvenc_of_match[] = {
+ { .compatible = "zte,zx296718-tvenc", },
+ { /* end */ },
+};
+MODULE_DEVICE_TABLE(of, zx_tvenc_of_match);
+
+struct platform_driver zx_tvenc_driver = {
+ .probe = zx_tvenc_probe,
+ .remove = zx_tvenc_remove,
+ .driver = {
+ .name = "zx-tvenc",
+ .of_match_table = zx_tvenc_of_match,
+ },
+};
--- /dev/null
+/*
+ * Copyright 2017 Linaro Ltd.
+ * Copyright 2017 ZTE Corporation.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ */
+
+#ifndef __ZX_TVENC_REGS_H__
+#define __ZX_TVENC_REGS_H__
+
+#define VENC_VIDEO_INFO 0x04
+#define VENC_VIDEO_RES 0x08
+#define VENC_FIELD1_PARAM 0x10
+#define VENC_FIELD2_PARAM 0x14
+#define VENC_LINE_O_1 0x18
+#define VENC_LINE_E_1 0x1c
+#define VENC_LINE_O_2 0x20
+#define VENC_LINE_E_2 0x24
+#define VENC_LINE_TIMING_PARAM 0x28
+#define VENC_WEIGHT_VALUE 0x2c
+#define VENC_BLANK_BLACK_LEVEL 0x30
+#define VENC_BURST_LEVEL 0x34
+#define VENC_CONTROL_PARAM 0x3c
+#define VENC_SUB_CARRIER_PHASE1 0x40
+#define VENC_PHASE_LINE_INCR_CVBS 0x48
+#define VENC_ENABLE 0xa8
+
+#endif /* __ZX_TVENC_REGS_H__ */
u32 fir_active;
u32 fir_htiming;
u32 fir_vtiming;
+ u32 sec_vtiming;
u32 timing_shift;
u32 timing_pi_shift;
};
.fir_active = FIR_MAIN_ACTIVE,
.fir_htiming = FIR_MAIN_H_TIMING,
.fir_vtiming = FIR_MAIN_V_TIMING,
+ .sec_vtiming = SEC_MAIN_V_TIMING,
.timing_shift = TIMING_MAIN_SHIFT,
.timing_pi_shift = TIMING_MAIN_PI_SHIFT,
};
.fir_active = FIR_AUX_ACTIVE,
.fir_htiming = FIR_AUX_H_TIMING,
.fir_vtiming = FIR_AUX_V_TIMING,
+ .sec_vtiming = SEC_AUX_V_TIMING,
.timing_shift = TIMING_AUX_SHIFT,
.timing_pi_shift = TIMING_AUX_PI_SHIFT,
};
u32 polarity_shift;
u32 int_frame_mask;
u32 tc_enable;
- u32 gl_enable;
+ u32 sec_vactive_shift;
+ u32 sec_vactive_mask;
+ u32 interlace_select;
+ u32 pi_enable;
+ u32 div_vga_shift;
+ u32 div_pic_shift;
+ u32 div_tvenc_shift;
+ u32 div_hdmi_pnx_shift;
+ u32 div_hdmi_shift;
+ u32 div_inf_shift;
+ u32 div_layer_shift;
};
static const struct zx_crtc_bits main_crtc_bits = {
.polarity_shift = MAIN_POL_SHIFT,
.int_frame_mask = TIMING_INT_MAIN_FRAME,
.tc_enable = MAIN_TC_EN,
- .gl_enable = OSD_CTRL0_GL0_EN,
+ .sec_vactive_shift = SEC_VACT_MAIN_SHIFT,
+ .sec_vactive_mask = SEC_VACT_MAIN_MASK,
+ .interlace_select = MAIN_INTERLACE_SEL,
+ .pi_enable = MAIN_PI_EN,
+ .div_vga_shift = VGA_MAIN_DIV_SHIFT,
+ .div_pic_shift = PIC_MAIN_DIV_SHIFT,
+ .div_tvenc_shift = TVENC_MAIN_DIV_SHIFT,
+ .div_hdmi_pnx_shift = HDMI_MAIN_PNX_DIV_SHIFT,
+ .div_hdmi_shift = HDMI_MAIN_DIV_SHIFT,
+ .div_inf_shift = INF_MAIN_DIV_SHIFT,
+ .div_layer_shift = LAYER_MAIN_DIV_SHIFT,
};
static const struct zx_crtc_bits aux_crtc_bits = {
.polarity_shift = AUX_POL_SHIFT,
.int_frame_mask = TIMING_INT_AUX_FRAME,
.tc_enable = AUX_TC_EN,
- .gl_enable = OSD_CTRL0_GL1_EN,
+ .sec_vactive_shift = SEC_VACT_AUX_SHIFT,
+ .sec_vactive_mask = SEC_VACT_AUX_MASK,
+ .interlace_select = AUX_INTERLACE_SEL,
+ .pi_enable = AUX_PI_EN,
+ .div_vga_shift = VGA_AUX_DIV_SHIFT,
+ .div_pic_shift = PIC_AUX_DIV_SHIFT,
+ .div_tvenc_shift = TVENC_AUX_DIV_SHIFT,
+ .div_hdmi_pnx_shift = HDMI_AUX_PNX_DIV_SHIFT,
+ .div_hdmi_shift = HDMI_AUX_DIV_SHIFT,
+ .div_inf_shift = INF_AUX_DIV_SHIFT,
+ .div_layer_shift = LAYER_AUX_DIV_SHIFT,
};
struct zx_crtc {
#define to_zx_crtc(x) container_of(x, struct zx_crtc, crtc)
+struct vou_layer_bits {
+ u32 enable;
+ u32 chnsel;
+ u32 clksel;
+};
+
+static const struct vou_layer_bits zx_gl_bits[GL_NUM] = {
+ {
+ .enable = OSD_CTRL0_GL0_EN,
+ .chnsel = OSD_CTRL0_GL0_SEL,
+ .clksel = VOU_CLK_GL0_SEL,
+ }, {
+ .enable = OSD_CTRL0_GL1_EN,
+ .chnsel = OSD_CTRL0_GL1_SEL,
+ .clksel = VOU_CLK_GL1_SEL,
+ },
+};
+
+static const struct vou_layer_bits zx_vl_bits[VL_NUM] = {
+ {
+ .enable = OSD_CTRL0_VL0_EN,
+ .chnsel = OSD_CTRL0_VL0_SEL,
+ .clksel = VOU_CLK_VL0_SEL,
+ }, {
+ .enable = OSD_CTRL0_VL1_EN,
+ .chnsel = OSD_CTRL0_VL1_SEL,
+ .clksel = VOU_CLK_VL1_SEL,
+ }, {
+ .enable = OSD_CTRL0_VL2_EN,
+ .chnsel = OSD_CTRL0_VL2_SEL,
+ .clksel = VOU_CLK_VL2_SEL,
+ },
+};
+
struct zx_vou_hw {
struct device *dev;
void __iomem *osd;
struct zx_crtc *aux_crtc;
};
+enum vou_inf_data_sel {
+ VOU_YUV444 = 0,
+ VOU_RGB_101010 = 1,
+ VOU_RGB_888 = 2,
+ VOU_RGB_666 = 3,
+};
+
+struct vou_inf {
+ enum vou_inf_id id;
+ enum vou_inf_data_sel data_sel;
+ u32 clocks_en_bits;
+ u32 clocks_sel_bits;
+};
+
+static struct vou_inf vou_infs[] = {
+ [VOU_HDMI] = {
+ .data_sel = VOU_YUV444,
+ .clocks_en_bits = BIT(24) | BIT(18) | BIT(6),
+ .clocks_sel_bits = BIT(13) | BIT(2),
+ },
+ [VOU_TV_ENC] = {
+ .data_sel = VOU_YUV444,
+ .clocks_en_bits = BIT(15),
+ .clocks_sel_bits = BIT(11) | BIT(0),
+ },
+};
+
static inline struct zx_vou_hw *crtc_to_vou(struct drm_crtc *crtc)
{
struct zx_crtc *zcrtc = to_zx_crtc(crtc);
return zcrtc->vou;
}
-void vou_inf_enable(const struct vou_inf *inf, struct drm_crtc *crtc)
+void vou_inf_hdmi_audio_sel(struct drm_crtc *crtc,
+ enum vou_inf_hdmi_audio aud)
{
struct zx_crtc *zcrtc = to_zx_crtc(crtc);
struct zx_vou_hw *vou = zcrtc->vou;
+
+ zx_writel_mask(vou->vouctl + VOU_INF_HDMI_CTRL, VOU_HDMI_AUD_MASK, aud);
+}
+
+void vou_inf_enable(enum vou_inf_id id, struct drm_crtc *crtc)
+{
+ struct zx_crtc *zcrtc = to_zx_crtc(crtc);
+ struct zx_vou_hw *vou = zcrtc->vou;
+ struct vou_inf *inf = &vou_infs[id];
bool is_main = zcrtc->chn_type == VOU_CHN_MAIN;
- u32 data_sel_shift = inf->id << 1;
+ u32 data_sel_shift = id << 1;
/* Select data format */
zx_writel_mask(vou->vouctl + VOU_INF_DATA_SEL, 0x3 << data_sel_shift,
inf->data_sel << data_sel_shift);
/* Select channel */
- zx_writel_mask(vou->vouctl + VOU_INF_CH_SEL, 0x1 << inf->id,
- zcrtc->chn_type << inf->id);
+ zx_writel_mask(vou->vouctl + VOU_INF_CH_SEL, 0x1 << id,
+ zcrtc->chn_type << id);
/* Select interface clocks */
zx_writel_mask(vou->vouctl + VOU_CLK_SEL, inf->clocks_sel_bits,
inf->clocks_en_bits);
/* Enable the device */
- zx_writel_mask(vou->vouctl + VOU_INF_EN, 1 << inf->id, 1 << inf->id);
+ zx_writel_mask(vou->vouctl + VOU_INF_EN, 1 << id, 1 << id);
}
-void vou_inf_disable(const struct vou_inf *inf, struct drm_crtc *crtc)
+void vou_inf_disable(enum vou_inf_id id, struct drm_crtc *crtc)
{
struct zx_vou_hw *vou = crtc_to_vou(crtc);
+ struct vou_inf *inf = &vou_infs[id];
/* Disable the device */
- zx_writel_mask(vou->vouctl + VOU_INF_EN, 1 << inf->id, 0);
+ zx_writel_mask(vou->vouctl + VOU_INF_EN, 1 << id, 0);
/* Disable interface clocks */
zx_writel_mask(vou->vouctl + VOU_CLK_EN, inf->clocks_en_bits, 0);
}
+void zx_vou_config_dividers(struct drm_crtc *crtc,
+ struct vou_div_config *configs, int num)
+{
+ struct zx_crtc *zcrtc = to_zx_crtc(crtc);
+ struct zx_vou_hw *vou = zcrtc->vou;
+ const struct zx_crtc_bits *bits = zcrtc->bits;
+ int i;
+
+ /* Clear update flag bit */
+ zx_writel_mask(vou->vouctl + VOU_DIV_PARA, DIV_PARA_UPDATE, 0);
+
+ for (i = 0; i < num; i++) {
+ struct vou_div_config *cfg = configs + i;
+ u32 reg, shift;
+
+ switch (cfg->id) {
+ case VOU_DIV_VGA:
+ reg = VOU_CLK_SEL;
+ shift = bits->div_vga_shift;
+ break;
+ case VOU_DIV_PIC:
+ reg = VOU_CLK_SEL;
+ shift = bits->div_pic_shift;
+ break;
+ case VOU_DIV_TVENC:
+ reg = VOU_DIV_PARA;
+ shift = bits->div_tvenc_shift;
+ break;
+ case VOU_DIV_HDMI_PNX:
+ reg = VOU_DIV_PARA;
+ shift = bits->div_hdmi_pnx_shift;
+ break;
+ case VOU_DIV_HDMI:
+ reg = VOU_DIV_PARA;
+ shift = bits->div_hdmi_shift;
+ break;
+ case VOU_DIV_INF:
+ reg = VOU_DIV_PARA;
+ shift = bits->div_inf_shift;
+ break;
+ case VOU_DIV_LAYER:
+ reg = VOU_DIV_PARA;
+ shift = bits->div_layer_shift;
+ break;
+ default:
+ continue;
+ }
+
+ /* Each divider occupies 3 bits */
+ zx_writel_mask(vou->vouctl + reg, 0x7 << shift,
+ cfg->val << shift);
+ }
+
+ /* Set update flag bit to get dividers effected */
+ zx_writel_mask(vou->vouctl + VOU_DIV_PARA, DIV_PARA_UPDATE,
+ DIV_PARA_UPDATE);
+}
+
static inline void vou_chn_set_update(struct zx_crtc *zcrtc)
{
zx_writel(zcrtc->chnreg + CHN_UPDATE, 1);
static void zx_crtc_enable(struct drm_crtc *crtc)
{
struct drm_display_mode *mode = &crtc->state->adjusted_mode;
+ bool interlaced = mode->flags & DRM_MODE_FLAG_INTERLACE;
struct zx_crtc *zcrtc = to_zx_crtc(crtc);
struct zx_vou_hw *vou = zcrtc->vou;
const struct zx_crtc_regs *regs = zcrtc->regs;
const struct zx_crtc_bits *bits = zcrtc->bits;
struct videomode vm;
+ u32 scan_mask;
u32 pol = 0;
u32 val;
int ret;
drm_display_mode_to_videomode(mode, &vm);
/* Set up timing parameters */
- val = V_ACTIVE(vm.vactive - 1);
+ val = V_ACTIVE((interlaced ? vm.vactive / 2 : vm.vactive) - 1);
val |= H_ACTIVE(vm.hactive - 1);
zx_writel(vou->timing + regs->fir_active, val);
val |= FRONT_PORCH(vm.vfront_porch - 1);
zx_writel(vou->timing + regs->fir_vtiming, val);
+ if (interlaced) {
+ u32 shift = bits->sec_vactive_shift;
+ u32 mask = bits->sec_vactive_mask;
+
+ val = zx_readl(vou->timing + SEC_V_ACTIVE);
+ val &= ~mask;
+ val |= ((vm.vactive / 2 - 1) << shift) & mask;
+ zx_writel(vou->timing + SEC_V_ACTIVE, val);
+
+ val = SYNC_WIDE(vm.vsync_len - 1);
+ /*
+ * The vback_porch for the second field needs to shift one on
+ * the value for the first field.
+ */
+ val |= BACK_PORCH(vm.vback_porch);
+ val |= FRONT_PORCH(vm.vfront_porch - 1);
+ zx_writel(vou->timing + regs->sec_vtiming, val);
+ }
+
/* Set up polarities */
if (vm.flags & DISPLAY_FLAGS_VSYNC_LOW)
pol |= 1 << POL_VSYNC_SHIFT;
pol << bits->polarity_shift);
/* Setup SHIFT register by following what ZTE BSP does */
- zx_writel(vou->timing + regs->timing_shift, H_SHIFT_VAL);
+ val = H_SHIFT_VAL;
+ if (interlaced)
+ val |= V_SHIFT_VAL << 16;
+ zx_writel(vou->timing + regs->timing_shift, val);
zx_writel(vou->timing + regs->timing_pi_shift, H_PI_SHIFT_VAL);
+ /* Progressive or interlace scan select */
+ scan_mask = bits->interlace_select | bits->pi_enable;
+ zx_writel_mask(vou->timing + SCAN_CTRL, scan_mask,
+ interlaced ? scan_mask : 0);
+
/* Enable TIMING_CTRL */
zx_writel_mask(vou->timing + TIMING_TC_ENABLE, bits->tc_enable,
bits->tc_enable);
zx_writel_mask(zcrtc->chnreg + CHN_CTRL1, CHN_SCREEN_H_MASK,
vm.vactive << CHN_SCREEN_H_SHIFT);
+ /* Configure channel interlace buffer control */
+ zx_writel_mask(zcrtc->chnreg + CHN_INTERLACE_BUF_CTRL, CHN_INTERLACE_EN,
+ interlaced ? CHN_INTERLACE_EN : 0);
+
/* Update channel */
vou_chn_set_update(zcrtc);
/* Enable channel */
zx_writel_mask(zcrtc->chnreg + CHN_CTRL0, CHN_ENABLE, CHN_ENABLE);
- /* Enable Graphic Layer */
- zx_writel_mask(vou->osd + OSD_CTRL0, bits->gl_enable,
- bits->gl_enable);
-
drm_crtc_vblank_on(crtc);
ret = clk_set_rate(zcrtc->pixclk, mode->clock * 1000);
drm_crtc_vblank_off(crtc);
- /* Disable Graphic Layer */
- zx_writel_mask(vou->osd + OSD_CTRL0, bits->gl_enable, 0);
-
/* Disable channel */
zx_writel_mask(zcrtc->chnreg + CHN_CTRL0, CHN_ENABLE, 0);
enum vou_chn_type chn_type)
{
struct device *dev = vou->dev;
- struct zx_layer_data data;
+ struct zx_plane *zplane;
struct zx_crtc *zcrtc;
int ret;
zcrtc->vou = vou;
zcrtc->chn_type = chn_type;
+ zplane = devm_kzalloc(dev, sizeof(*zplane), GFP_KERNEL);
+ if (!zplane)
+ return -ENOMEM;
+
+ zplane->dev = dev;
+
if (chn_type == VOU_CHN_MAIN) {
- data.layer = vou->osd + MAIN_GL_OFFSET;
- data.csc = vou->osd + MAIN_CSC_OFFSET;
- data.hbsc = vou->osd + MAIN_HBSC_OFFSET;
- data.rsz = vou->otfppu + MAIN_RSZ_OFFSET;
+ zplane->layer = vou->osd + MAIN_GL_OFFSET;
+ zplane->csc = vou->osd + MAIN_CSC_OFFSET;
+ zplane->hbsc = vou->osd + MAIN_HBSC_OFFSET;
+ zplane->rsz = vou->otfppu + MAIN_RSZ_OFFSET;
+ zplane->bits = &zx_gl_bits[0];
zcrtc->chnreg = vou->osd + OSD_MAIN_CHN;
zcrtc->regs = &main_crtc_regs;
zcrtc->bits = &main_crtc_bits;
} else {
- data.layer = vou->osd + AUX_GL_OFFSET;
- data.csc = vou->osd + AUX_CSC_OFFSET;
- data.hbsc = vou->osd + AUX_HBSC_OFFSET;
- data.rsz = vou->otfppu + AUX_RSZ_OFFSET;
+ zplane->layer = vou->osd + AUX_GL_OFFSET;
+ zplane->csc = vou->osd + AUX_CSC_OFFSET;
+ zplane->hbsc = vou->osd + AUX_HBSC_OFFSET;
+ zplane->rsz = vou->otfppu + AUX_RSZ_OFFSET;
+ zplane->bits = &zx_gl_bits[1];
zcrtc->chnreg = vou->osd + OSD_AUX_CHN;
zcrtc->regs = &aux_crtc_regs;
zcrtc->bits = &aux_crtc_bits;
return ret;
}
- zcrtc->primary = zx_plane_init(drm, dev, &data, DRM_PLANE_TYPE_PRIMARY);
- if (IS_ERR(zcrtc->primary)) {
- ret = PTR_ERR(zcrtc->primary);
+ ret = zx_plane_init(drm, zplane, DRM_PLANE_TYPE_PRIMARY);
+ if (ret) {
DRM_DEV_ERROR(dev, "failed to init primary plane: %d\n", ret);
return ret;
}
+ zcrtc->primary = &zplane->plane;
+
ret = drm_crtc_init_with_planes(drm, &zcrtc->crtc, zcrtc->primary, NULL,
&zx_crtc_funcs, NULL);
if (ret) {
zcrtc->bits->int_frame_mask, 0);
}
+void zx_vou_layer_enable(struct drm_plane *plane)
+{
+ struct zx_crtc *zcrtc = to_zx_crtc(plane->state->crtc);
+ struct zx_vou_hw *vou = zcrtc->vou;
+ struct zx_plane *zplane = to_zx_plane(plane);
+ const struct vou_layer_bits *bits = zplane->bits;
+
+ if (zcrtc->chn_type == VOU_CHN_MAIN) {
+ zx_writel_mask(vou->osd + OSD_CTRL0, bits->chnsel, 0);
+ zx_writel_mask(vou->vouctl + VOU_CLK_SEL, bits->clksel, 0);
+ } else {
+ zx_writel_mask(vou->osd + OSD_CTRL0, bits->chnsel,
+ bits->chnsel);
+ zx_writel_mask(vou->vouctl + VOU_CLK_SEL, bits->clksel,
+ bits->clksel);
+ }
+
+ zx_writel_mask(vou->osd + OSD_CTRL0, bits->enable, bits->enable);
+}
+
+void zx_vou_layer_disable(struct drm_plane *plane)
+{
+ struct zx_crtc *zcrtc = to_zx_crtc(plane->crtc);
+ struct zx_vou_hw *vou = zcrtc->vou;
+ struct zx_plane *zplane = to_zx_plane(plane);
+ const struct vou_layer_bits *bits = zplane->bits;
+
+ zx_writel_mask(vou->osd + OSD_CTRL0, bits->enable, 0);
+}
+
+static void zx_overlay_init(struct drm_device *drm, struct zx_vou_hw *vou)
+{
+ struct device *dev = vou->dev;
+ struct zx_plane *zplane;
+ int i;
+ int ret;
+
+ /*
+ * VL0 has some quirks on scaling support which need special handling.
+ * Let's leave it out for now.
+ */
+ for (i = 1; i < VL_NUM; i++) {
+ zplane = devm_kzalloc(dev, sizeof(*zplane), GFP_KERNEL);
+ if (!zplane) {
+ DRM_DEV_ERROR(dev, "failed to allocate zplane %d\n", i);
+ return;
+ }
+
+ zplane->layer = vou->osd + OSD_VL_OFFSET(i);
+ zplane->hbsc = vou->osd + HBSC_VL_OFFSET(i);
+ zplane->rsz = vou->otfppu + RSZ_VL_OFFSET(i);
+ zplane->bits = &zx_vl_bits[i];
+
+ ret = zx_plane_init(drm, zplane, DRM_PLANE_TYPE_OVERLAY);
+ if (ret) {
+ DRM_DEV_ERROR(dev, "failed to init overlay %d\n", i);
+ continue;
+ }
+ }
+}
+
+static inline void zx_osd_int_update(struct zx_crtc *zcrtc)
+{
+ struct drm_crtc *crtc = &zcrtc->crtc;
+ struct drm_plane *plane;
+
+ vou_chn_set_update(zcrtc);
+
+ drm_for_each_plane_mask(plane, crtc->dev, crtc->state->plane_mask)
+ zx_plane_set_update(plane);
+}
+
static irqreturn_t vou_irq_handler(int irq, void *dev_id)
{
struct zx_vou_hw *vou = dev_id;
state = zx_readl(vou->osd + OSD_INT_STA);
zx_writel(vou->osd + OSD_INT_CLRSTA, state);
- if (state & OSD_INT_MAIN_UPT) {
- vou_chn_set_update(vou->main_crtc);
- zx_plane_set_update(vou->main_crtc->primary);
- }
+ if (state & OSD_INT_MAIN_UPT)
+ zx_osd_int_update(vou->main_crtc);
- if (state & OSD_INT_AUX_UPT) {
- vou_chn_set_update(vou->aux_crtc);
- zx_plane_set_update(vou->aux_crtc->primary);
- }
+ if (state & OSD_INT_AUX_UPT)
+ zx_osd_int_update(vou->aux_crtc);
if (state & OSD_INT_ERROR)
DRM_DEV_ERROR(vou->dev, "OSD ERROR: 0x%08x!\n", state);
static void vou_hw_init(struct zx_vou_hw *vou)
{
- /* Set GL0 to main channel and GL1 to aux channel */
- zx_writel_mask(vou->osd + OSD_CTRL0, OSD_CTRL0_GL0_SEL, 0);
- zx_writel_mask(vou->osd + OSD_CTRL0, OSD_CTRL0_GL1_SEL,
- OSD_CTRL0_GL1_SEL);
-
/* Release reset for all VOU modules */
zx_writel(vou->vouctl + VOU_SOFT_RST, ~0);
- /* Select main clock for GL0 and aux clock for GL1 module */
- zx_writel_mask(vou->vouctl + VOU_CLK_SEL, VOU_CLK_GL0_SEL, 0);
- zx_writel_mask(vou->vouctl + VOU_CLK_SEL, VOU_CLK_GL1_SEL,
- VOU_CLK_GL1_SEL);
-
/* Enable clock auto-gating for all VOU modules */
zx_writel(vou->vouctl + VOU_CLK_REQEN, ~0);
goto disable_ppu_clk;
}
+ zx_overlay_init(drm, vou);
+
return 0;
disable_ppu_clk:
VOU_VGA = 5,
};
-enum vou_inf_data_sel {
- VOU_YUV444 = 0,
- VOU_RGB_101010 = 1,
- VOU_RGB_888 = 2,
- VOU_RGB_666 = 3,
+enum vou_inf_hdmi_audio {
+ VOU_HDMI_AUD_SPDIF = BIT(0),
+ VOU_HDMI_AUD_I2S = BIT(1),
+ VOU_HDMI_AUD_DSD = BIT(2),
+ VOU_HDMI_AUD_HBR = BIT(3),
+ VOU_HDMI_AUD_PARALLEL = BIT(4),
};
-struct vou_inf {
- enum vou_inf_id id;
- enum vou_inf_data_sel data_sel;
- u32 clocks_en_bits;
- u32 clocks_sel_bits;
+void vou_inf_hdmi_audio_sel(struct drm_crtc *crtc,
+ enum vou_inf_hdmi_audio aud);
+void vou_inf_enable(enum vou_inf_id id, struct drm_crtc *crtc);
+void vou_inf_disable(enum vou_inf_id id, struct drm_crtc *crtc);
+
+enum vou_div_id {
+ VOU_DIV_VGA,
+ VOU_DIV_PIC,
+ VOU_DIV_TVENC,
+ VOU_DIV_HDMI_PNX,
+ VOU_DIV_HDMI,
+ VOU_DIV_INF,
+ VOU_DIV_LAYER,
+};
+
+enum vou_div_val {
+ VOU_DIV_1 = 0,
+ VOU_DIV_2 = 1,
+ VOU_DIV_4 = 3,
+ VOU_DIV_8 = 7,
};
-void vou_inf_enable(const struct vou_inf *inf, struct drm_crtc *crtc);
-void vou_inf_disable(const struct vou_inf *inf, struct drm_crtc *crtc);
+struct vou_div_config {
+ enum vou_div_id id;
+ enum vou_div_val val;
+};
+
+void zx_vou_config_dividers(struct drm_crtc *crtc,
+ struct vou_div_config *configs, int num);
int zx_vou_enable_vblank(struct drm_device *drm, unsigned int pipe);
void zx_vou_disable_vblank(struct drm_device *drm, unsigned int pipe);
+void zx_vou_layer_enable(struct drm_plane *plane);
+void zx_vou_layer_disable(struct drm_plane *plane);
+
#endif /* __ZX_VOU_H__ */
#define AUX_HBSC_OFFSET 0x860
#define AUX_RSZ_OFFSET 0x800
+#define OSD_VL0_OFFSET 0x040
+#define OSD_VL_OFFSET(i) (OSD_VL0_OFFSET + 0x050 * (i))
+
+#define HBSC_VL0_OFFSET 0x760
+#define HBSC_VL_OFFSET(i) (HBSC_VL0_OFFSET + 0x040 * (i))
+
+#define RSZ_VL1_U0 0xa00
+#define RSZ_VL_OFFSET(i) (RSZ_VL1_U0 + 0x200 * (i))
+
/* OSD (GPC_GLOBAL) registers */
#define OSD_INT_STA 0x04
#define OSD_INT_CLRSTA 0x08
)
#define OSD_INT_ENABLE (OSD_INT_ERROR | OSD_INT_AUX_UPT | OSD_INT_MAIN_UPT)
#define OSD_CTRL0 0x10
+#define OSD_CTRL0_VL0_EN BIT(13)
+#define OSD_CTRL0_VL0_SEL BIT(12)
+#define OSD_CTRL0_VL1_EN BIT(11)
+#define OSD_CTRL0_VL1_SEL BIT(10)
+#define OSD_CTRL0_VL2_EN BIT(9)
+#define OSD_CTRL0_VL2_SEL BIT(8)
#define OSD_CTRL0_GL0_EN BIT(7)
#define OSD_CTRL0_GL0_SEL BIT(6)
#define OSD_CTRL0_GL1_EN BIT(5)
#define CHN_SCREEN_H_SHIFT 5
#define CHN_SCREEN_H_MASK (0x1fff << CHN_SCREEN_H_SHIFT)
#define CHN_UPDATE 0x08
+#define CHN_INTERLACE_BUF_CTRL 0x24
+#define CHN_INTERLACE_EN BIT(2)
/* TIMING_CTRL registers */
#define TIMING_TC_ENABLE 0x04
#define TIMING_MAIN_SHIFT 0x2c
#define TIMING_AUX_SHIFT 0x30
#define H_SHIFT_VAL 0x0048
+#define V_SHIFT_VAL 0x0001
+#define SCAN_CTRL 0x34
+#define AUX_PI_EN BIT(19)
+#define MAIN_PI_EN BIT(18)
+#define AUX_INTERLACE_SEL BIT(1)
+#define MAIN_INTERLACE_SEL BIT(0)
+#define SEC_V_ACTIVE 0x38
+#define SEC_VACT_MAIN_SHIFT 0
+#define SEC_VACT_MAIN_MASK (0xffff << SEC_VACT_MAIN_SHIFT)
+#define SEC_VACT_AUX_SHIFT 16
+#define SEC_VACT_AUX_MASK (0xffff << SEC_VACT_AUX_SHIFT)
+#define SEC_MAIN_V_TIMING 0x3c
+#define SEC_AUX_V_TIMING 0x40
#define TIMING_MAIN_PI_SHIFT 0x68
#define TIMING_AUX_PI_SHIFT 0x6c
#define H_PI_SHIFT_VAL 0x000f
#define VOU_INF_DATA_SEL 0x08
#define VOU_SOFT_RST 0x14
#define VOU_CLK_SEL 0x18
+#define VGA_AUX_DIV_SHIFT 29
+#define VGA_MAIN_DIV_SHIFT 26
+#define PIC_MAIN_DIV_SHIFT 23
+#define PIC_AUX_DIV_SHIFT 20
+#define VOU_CLK_VL2_SEL BIT(8)
+#define VOU_CLK_VL1_SEL BIT(7)
+#define VOU_CLK_VL0_SEL BIT(6)
#define VOU_CLK_GL1_SEL BIT(5)
#define VOU_CLK_GL0_SEL BIT(4)
+#define VOU_DIV_PARA 0x1c
+#define DIV_PARA_UPDATE BIT(31)
+#define TVENC_AUX_DIV_SHIFT 28
+#define HDMI_AUX_PNX_DIV_SHIFT 25
+#define HDMI_MAIN_PNX_DIV_SHIFT 22
+#define HDMI_AUX_DIV_SHIFT 19
+#define HDMI_MAIN_DIV_SHIFT 16
+#define TVENC_MAIN_DIV_SHIFT 13
+#define INF_AUX_DIV_SHIFT 9
+#define INF_MAIN_DIV_SHIFT 6
+#define LAYER_AUX_DIV_SHIFT 3
+#define LAYER_MAIN_DIV_SHIFT 0
#define VOU_CLK_REQEN 0x20
#define VOU_CLK_EN 0x24
+#define VOU_INF_HDMI_CTRL 0x30
+#define VOU_HDMI_AUD_MASK 0x1f
/* OTFPPU_CTRL registers */
#define OTFPPU_RSZ_DATA_SOURCE 0x04
{ HID_USB_DEVICE(USB_VENDOR_ID_PANJIT, 0x0002) },
{ HID_USB_DEVICE(USB_VENDOR_ID_PANJIT, 0x0003) },
{ HID_USB_DEVICE(USB_VENDOR_ID_PANJIT, 0x0004) },
+ { HID_USB_DEVICE(USB_VENDOR_ID_PETZL, USB_DEVICE_ID_PETZL_HEADLAMP) },
{ HID_USB_DEVICE(USB_VENDOR_ID_PHILIPS, USB_DEVICE_ID_PHILIPS_IEEE802154_DONGLE) },
{ HID_USB_DEVICE(USB_VENDOR_ID_POWERCOM, USB_DEVICE_ID_POWERCOM_UPS) },
#if IS_ENABLED(CONFIG_MOUSE_SYNAPTICS_USB)
struct usb_interface *usbif = to_usb_interface(dev->parent);
struct usb_device *usbdev = interface_to_usbdev(usbif);
int brightness;
- char data[8];
+ char *data;
+
+ data = kmalloc(8, GFP_KERNEL);
+ if (!data)
+ return -ENOMEM;
ret = usb_control_msg(usbdev, usb_rcvctrlpipe(usbdev, 0),
K90_REQUEST_STATUS,
USB_DIR_IN | USB_TYPE_VENDOR |
USB_RECIP_DEVICE, 0, 0, data, 8,
USB_CTRL_SET_TIMEOUT);
- if (ret < 0) {
+ if (ret < 5) {
dev_warn(dev, "Failed to get K90 initial state (error %d).\n",
ret);
- return -EIO;
+ ret = -EIO;
+ goto out;
}
brightness = data[4];
if (brightness < 0 || brightness > 3) {
dev_warn(dev,
"Read invalid backlight brightness: %02hhx.\n",
data[4]);
- return -EIO;
+ ret = -EIO;
+ goto out;
}
- return brightness;
+ ret = brightness;
+out:
+ kfree(data);
+
+ return ret;
}
static enum led_brightness k90_record_led_get(struct led_classdev *led_cdev)
struct usb_interface *usbif = to_usb_interface(dev->parent);
struct usb_device *usbdev = interface_to_usbdev(usbif);
const char *macro_mode;
- char data[8];
+ char *data;
+
+ data = kmalloc(2, GFP_KERNEL);
+ if (!data)
+ return -ENOMEM;
ret = usb_control_msg(usbdev, usb_rcvctrlpipe(usbdev, 0),
K90_REQUEST_GET_MODE,
USB_DIR_IN | USB_TYPE_VENDOR |
USB_RECIP_DEVICE, 0, 0, data, 2,
USB_CTRL_SET_TIMEOUT);
- if (ret < 0) {
+ if (ret < 1) {
dev_warn(dev, "Failed to get K90 initial mode (error %d).\n",
ret);
- return -EIO;
+ ret = -EIO;
+ goto out;
}
switch (data[0]) {
default:
dev_warn(dev, "K90 in unknown mode: %02hhx.\n",
data[0]);
- return -EIO;
+ ret = -EIO;
+ goto out;
}
- return snprintf(buf, PAGE_SIZE, "%s\n", macro_mode);
+ ret = snprintf(buf, PAGE_SIZE, "%s\n", macro_mode);
+out:
+ kfree(data);
+
+ return ret;
}
static ssize_t k90_store_macro_mode(struct device *dev,
struct usb_interface *usbif = to_usb_interface(dev->parent);
struct usb_device *usbdev = interface_to_usbdev(usbif);
int current_profile;
- char data[8];
+ char *data;
+
+ data = kmalloc(8, GFP_KERNEL);
+ if (!data)
+ return -ENOMEM;
ret = usb_control_msg(usbdev, usb_rcvctrlpipe(usbdev, 0),
K90_REQUEST_STATUS,
USB_DIR_IN | USB_TYPE_VENDOR |
USB_RECIP_DEVICE, 0, 0, data, 8,
USB_CTRL_SET_TIMEOUT);
- if (ret < 0) {
+ if (ret < 8) {
dev_warn(dev, "Failed to get K90 initial state (error %d).\n",
ret);
- return -EIO;
+ ret = -EIO;
+ goto out;
}
current_profile = data[7];
if (current_profile < 1 || current_profile > 3) {
dev_warn(dev, "Read invalid current profile: %02hhx.\n",
data[7]);
- return -EIO;
+ ret = -EIO;
+ goto out;
}
- return snprintf(buf, PAGE_SIZE, "%d\n", current_profile);
+ ret = snprintf(buf, PAGE_SIZE, "%d\n", current_profile);
+out:
+ kfree(data);
+
+ return ret;
}
static ssize_t k90_store_current_profile(struct device *dev,
if (!(quirks & CP_RDESC_SWAPPED_MIN_MAX))
return rdesc;
+ if (*rsize < 4)
+ return rdesc;
+
for (i = 0; i < *rsize - 4; i++)
if (rdesc[i] == 0x29 && rdesc[i + 2] == 0x19) {
rdesc[i] = 0x19;
#define USB_VENDOR_ID_PETALYNX 0x18b1
#define USB_DEVICE_ID_PETALYNX_MAXTER_REMOTE 0x0037
+#define USB_VENDOR_ID_PETZL 0x2122
+#define USB_DEVICE_ID_PETZL_HEADLAMP 0x1234
+
#define USB_VENDOR_ID_PHILIPS 0x0471
#define USB_DEVICE_ID_PHILIPS_IEEE802154_DONGLE 0x0617
if (ret)
goto out_unlock;
+ /*
+ * The HID over I2C specification states that if a DEVICE needs time
+ * after the PWR_ON request, it should utilise CLOCK stretching.
+ * However, it has been observered that the Windows driver provides a
+ * 1ms sleep between the PWR_ON and RESET requests and that some devices
+ * rely on this.
+ */
+ usleep_range(1000, 5000);
+
i2c_hid_dbg(ihid, "resetting...\n");
ret = i2c_hid_command(client, &hid_reset_cmd, NULL, 0);
return retval;
}
+ if (wacom_wac->features.device_type & WACOM_DEVICETYPE_TOUCH)
+ wacom_wac->shared->touch = hdev;
+ else if (wacom_wac->features.device_type & WACOM_DEVICETYPE_PEN)
+ wacom_wac->shared->pen = hdev;
+
out:
mutex_unlock(&wacom_udev_list_lock);
return retval;
if (error)
goto fail;
- error = wacom_add_shared_data(hdev);
- if (error)
- goto fail;
-
/*
* Bamboo Pad has a generic hid handling for the Pen, and we switch it
* into debug mode for the touch part.
wacom_update_name(wacom, wireless ? " (WL)" : "");
- if (wacom_wac->features.device_type & WACOM_DEVICETYPE_TOUCH)
- wacom_wac->shared->touch = hdev;
- else if (wacom_wac->features.device_type & WACOM_DEVICETYPE_PEN)
- wacom_wac->shared->pen = hdev;
+ error = wacom_add_shared_data(hdev);
+ if (error)
+ goto fail;
if (!(features->device_type & WACOM_DEVICETYPE_WL_MONITOR) &&
(features->quirks & WACOM_QUIRK_BATTERY)) {
wacom_report_events(hdev, report);
+ /*
+ * Non-input reports may be sent prior to the device being
+ * completely initialized. Since only their events need
+ * to be processed, exit after 'wacom_report_events' has
+ * been called to prevent potential crashes in the report-
+ * processing functions.
+ */
+ if (report->type != HID_INPUT_REPORT)
+ return;
+
if (WACOM_PAD_FIELD(field)) {
wacom_wac_pad_battery_report(hdev, report);
if (wacom->wacom_wac.pad_input)
u8 command, int size, union i2c_smbus_data *data)
{
struct i2c_piix4_adapdata *adapdata = i2c_get_adapdata(adap);
+ unsigned short piix4_smba = adapdata->smba;
+ int retries = MAX_TIMEOUT;
+ int smbslvcnt;
u8 smba_en_lo;
u8 port;
int retval;
+ /* Request the SMBUS semaphore, avoid conflicts with the IMC */
+ smbslvcnt = inb_p(SMBSLVCNT);
+ do {
+ outb_p(smbslvcnt | 0x10, SMBSLVCNT);
+
+ /* Check the semaphore status */
+ smbslvcnt = inb_p(SMBSLVCNT);
+ if (smbslvcnt & 0x10)
+ break;
+
+ usleep_range(1000, 2000);
+ } while (--retries);
+ /* SMBus is still owned by the IMC, we give up */
+ if (!retries)
+ return -EBUSY;
+
mutex_lock(&piix4_mutex_sb800);
outb_p(piix4_port_sel_sb800, SB800_PIIX4_SMB_IDX);
mutex_unlock(&piix4_mutex_sb800);
+ /* Release the semaphore */
+ outb_p(smbslvcnt | 0x20, SMBSLVCNT);
+
return retval;
}
if (!client->irq) {
int irq = -ENOENT;
- if (dev->of_node) {
+ if (client->flags & I2C_CLIENT_HOST_NOTIFY) {
+ dev_dbg(dev, "Using Host Notify IRQ\n");
+ irq = i2c_smbus_host_notify_to_irq(client);
+ } else if (dev->of_node) {
irq = of_irq_get_byname(dev->of_node, "irq");
if (irq == -EINVAL || irq == -ENODATA)
irq = of_irq_get(dev->of_node, 0);
}
if (irq == -EPROBE_DEFER)
return irq;
- /*
- * ACPI and OF did not find any useful IRQ, try to see
- * if Host Notify can be used.
- */
- if (irq < 0) {
- dev_dbg(dev, "Using Host Notify IRQ\n");
- irq = i2c_smbus_host_notify_to_irq(client);
- }
+
if (irq < 0)
irq = 0;
if (i2c_check_addr_validity(addr, info.flags)) {
dev_err(&adap->dev, "of_i2c: invalid addr=%x on %s\n",
- info.addr, node->full_name);
+ addr, node->full_name);
return ERR_PTR(-EINVAL);
}
info.of_node = of_node_get(node);
info.archdata = &dev_ad;
+ if (of_property_read_bool(node, "host-notify"))
+ info.flags |= I2C_CLIENT_HOST_NOTIFY;
+
if (of_get_property(node, "wakeup-source", NULL))
info.flags |= I2C_CLIENT_WAKE;
int ret;
if (!client || !slave_cb) {
- WARN(1, "insufficent data\n");
+ WARN(1, "insufficient data\n");
return -EINVAL;
}
unsigned long arg)
{
struct i2c_smbus_ioctl_data data_arg;
- union i2c_smbus_data temp;
+ union i2c_smbus_data temp = {};
int datasize, res;
if (copy_from_user(&data_arg,
#include <linux/sched.h>
#include <linux/slab.h>
#include <linux/mm.h>
-#include <linux/miscdevice.h>
#include <linux/module.h>
#include <linux/poll.h>
#include <linux/init.h>
input_dev->name = xpad->name;
input_dev->phys = xpad->phys;
usb_to_input_id(xpad->udev, &input_dev->id);
+
+ if (xpad->xtype == XTYPE_XBOX360W) {
+ /* x360w controllers and the receiver have different ids */
+ input_dev->id.product = 0x02a1;
+ }
+
input_dev->dev.parent = &xpad->intf->dev;
input_set_drvdata(input_dev, xpad);
MODULE_DEVICE_TABLE(i2c, adxl34x_id);
-#ifdef CONFIG_OF
static const struct of_device_id adxl34x_of_id[] = {
/*
* The ADXL346 is backward-compatible with the ADXL345. Differences are
};
MODULE_DEVICE_TABLE(of, adxl34x_of_id);
-#endif
static struct i2c_driver adxl34x_driver = {
.driver = {
.name = "adxl34x",
.pm = &adxl34x_i2c_pm,
- .of_match_table = of_match_ptr(adxl34x_of_id),
+ .of_match_table = adxl34x_of_id,
},
.probe = adxl34x_i2c_probe,
.remove = adxl34x_i2c_remove,
(_b[1] & 0x7F) \
)
-#define SS4_TS_Y_V2(_b) (s8)( \
+#define SS4_TS_Y_V2(_b) -(s8)( \
((_b[3] & 0x01) << 7) | \
(_b[2] & 0x7F) \
)
* after soft reset, we should wait for 1 ms
* before the device becomes operational
*/
-#define SOFT_RESET_DELAY_MS 3
+#define SOFT_RESET_DELAY_US 3000
/* and after hard reset, we should wait for max 500ms */
#define HARD_RESET_DELAY_MS 500
if (ret) {
dev_err(&client->dev, "Unable to reset device\n");
} else {
- msleep(SOFT_RESET_DELAY_MS);
+ usleep_range(SOFT_RESET_DELAY_US, SOFT_RESET_DELAY_US + 100);
ret = synaptics_i2c_config(client);
if (ret)
dev_err(&client->dev, "Unable to config device\n");
config RMI4_F03
bool "RMI4 Function 03 (PS2 Guest)"
- depends on RMI4_CORE && SERIO
+ depends on RMI4_CORE
+ depends on SERIO=y || RMI4_CORE=SERIO
help
Say Y here if you want to add support for RMI4 function 03.
DMI_MATCH(DMI_PRODUCT_VERSION, "Rev 1"),
},
},
+ {
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR, "PEGATRON CORPORATION"),
+ DMI_MATCH(DMI_PRODUCT_NAME, "C15B"),
+ },
+ },
{ }
};
case QUEUE_HEADER_NORMAL:
report_count = ts->buf[FW_HDR_COUNT];
- if (report_count > 3) {
+ if (report_count == 0 || report_count > 3) {
dev_err(&client->dev,
- "too large report count: %*ph\n",
+ "bad report count: %*ph\n",
HEADER_SIZE, ts->buf);
break;
}
int is_new);
struct md_cluster_info;
+/* change UNSUPPORTED_MDDEV_FLAGS for each array type if new flag is added */
enum mddev_flags {
MD_ARRAY_FIRST_USE, /* First use of array, needs initialization */
MD_CLOSING, /* If set, we are closing the array, do not open
{
return mddev->cluster_info && mddev->bitmap_info.nodes > 1;
}
+
+/* clear unsupported mddev_flags */
+static inline void mddev_clear_unsupported_flags(struct mddev *mddev,
+ unsigned long unsupported_flags)
+{
+ mddev->flags &= ~unsupported_flags;
+}
#endif /* _MD_MD_H */
#include "raid0.h"
#include "raid5.h"
+#define UNSUPPORTED_MDDEV_FLAGS \
+ ((1L << MD_HAS_JOURNAL) | \
+ (1L << MD_JOURNAL_CLEAN) | \
+ (1L << MD_FAILFAST_SUPPORTED))
+
static int raid0_congested(struct mddev *mddev, int bits)
{
struct r0conf *conf = mddev->private;
mddev->delta_disks = -1;
/* make sure it will be not marked as dirty */
mddev->recovery_cp = MaxSector;
- clear_bit(MD_HAS_JOURNAL, &mddev->flags);
- clear_bit(MD_JOURNAL_CLEAN, &mddev->flags);
+ mddev_clear_unsupported_flags(mddev, UNSUPPORTED_MDDEV_FLAGS);
create_strip_zones(mddev, &priv_conf);
mddev->degraded = 0;
/* make sure it will be not marked as dirty */
mddev->recovery_cp = MaxSector;
- clear_bit(MD_FAILFAST_SUPPORTED, &mddev->flags);
+ mddev_clear_unsupported_flags(mddev, UNSUPPORTED_MDDEV_FLAGS);
create_strip_zones(mddev, &priv_conf);
return priv_conf;
mddev->raid_disks = 1;
/* make sure it will be not marked as dirty */
mddev->recovery_cp = MaxSector;
- clear_bit(MD_FAILFAST_SUPPORTED, &mddev->flags);
+ mddev_clear_unsupported_flags(mddev, UNSUPPORTED_MDDEV_FLAGS);
create_strip_zones(mddev, &priv_conf);
return priv_conf;
#include "raid1.h"
#include "bitmap.h"
+#define UNSUPPORTED_MDDEV_FLAGS \
+ ((1L << MD_HAS_JOURNAL) | \
+ (1L << MD_JOURNAL_CLEAN))
+
/*
* Number of guaranteed r1bios in case of extreme VM load:
*/
kfree(plug);
}
-static void raid1_make_request(struct mddev *mddev, struct bio * bio)
+static void raid1_read_request(struct mddev *mddev, struct bio *bio,
+ struct r1bio *r1_bio)
{
struct r1conf *conf = mddev->private;
struct raid1_info *mirror;
- struct r1bio *r1_bio;
struct bio *read_bio;
+ struct bitmap *bitmap = mddev->bitmap;
+ const int op = bio_op(bio);
+ const unsigned long do_sync = (bio->bi_opf & REQ_SYNC);
+ int sectors_handled;
+ int max_sectors;
+ int rdisk;
+
+ wait_barrier(conf, bio);
+
+read_again:
+ rdisk = read_balance(conf, r1_bio, &max_sectors);
+
+ if (rdisk < 0) {
+ /* couldn't find anywhere to read from */
+ raid_end_bio_io(r1_bio);
+ return;
+ }
+ mirror = conf->mirrors + rdisk;
+
+ if (test_bit(WriteMostly, &mirror->rdev->flags) &&
+ bitmap) {
+ /*
+ * Reading from a write-mostly device must take care not to
+ * over-take any writes that are 'behind'
+ */
+ raid1_log(mddev, "wait behind writes");
+ wait_event(bitmap->behind_wait,
+ atomic_read(&bitmap->behind_writes) == 0);
+ }
+ r1_bio->read_disk = rdisk;
+ r1_bio->start_next_window = 0;
+
+ read_bio = bio_clone_mddev(bio, GFP_NOIO, mddev);
+ bio_trim(read_bio, r1_bio->sector - bio->bi_iter.bi_sector,
+ max_sectors);
+
+ r1_bio->bios[rdisk] = read_bio;
+
+ read_bio->bi_iter.bi_sector = r1_bio->sector +
+ mirror->rdev->data_offset;
+ read_bio->bi_bdev = mirror->rdev->bdev;
+ read_bio->bi_end_io = raid1_end_read_request;
+ bio_set_op_attrs(read_bio, op, do_sync);
+ if (test_bit(FailFast, &mirror->rdev->flags) &&
+ test_bit(R1BIO_FailFast, &r1_bio->state))
+ read_bio->bi_opf |= MD_FAILFAST;
+ read_bio->bi_private = r1_bio;
+
+ if (mddev->gendisk)
+ trace_block_bio_remap(bdev_get_queue(read_bio->bi_bdev),
+ read_bio, disk_devt(mddev->gendisk),
+ r1_bio->sector);
+
+ if (max_sectors < r1_bio->sectors) {
+ /*
+ * could not read all from this device, so we will need another
+ * r1_bio.
+ */
+ sectors_handled = (r1_bio->sector + max_sectors
+ - bio->bi_iter.bi_sector);
+ r1_bio->sectors = max_sectors;
+ spin_lock_irq(&conf->device_lock);
+ if (bio->bi_phys_segments == 0)
+ bio->bi_phys_segments = 2;
+ else
+ bio->bi_phys_segments++;
+ spin_unlock_irq(&conf->device_lock);
+
+ /*
+ * Cannot call generic_make_request directly as that will be
+ * queued in __make_request and subsequent mempool_alloc might
+ * block waiting for it. So hand bio over to raid1d.
+ */
+ reschedule_retry(r1_bio);
+
+ r1_bio = mempool_alloc(conf->r1bio_pool, GFP_NOIO);
+
+ r1_bio->master_bio = bio;
+ r1_bio->sectors = bio_sectors(bio) - sectors_handled;
+ r1_bio->state = 0;
+ r1_bio->mddev = mddev;
+ r1_bio->sector = bio->bi_iter.bi_sector + sectors_handled;
+ goto read_again;
+ } else
+ generic_make_request(read_bio);
+}
+
+static void raid1_write_request(struct mddev *mddev, struct bio *bio,
+ struct r1bio *r1_bio)
+{
+ struct r1conf *conf = mddev->private;
int i, disks;
- struct bitmap *bitmap;
+ struct bitmap *bitmap = mddev->bitmap;
unsigned long flags;
const int op = bio_op(bio);
- const int rw = bio_data_dir(bio);
const unsigned long do_sync = (bio->bi_opf & REQ_SYNC);
const unsigned long do_flush_fua = (bio->bi_opf &
(REQ_PREFLUSH | REQ_FUA));
md_write_start(mddev, bio); /* wait on superblock update early */
- if (bio_data_dir(bio) == WRITE &&
- ((bio_end_sector(bio) > mddev->suspend_lo &&
+ if ((bio_end_sector(bio) > mddev->suspend_lo &&
bio->bi_iter.bi_sector < mddev->suspend_hi) ||
(mddev_is_clustered(mddev) &&
md_cluster_ops->area_resyncing(mddev, WRITE,
- bio->bi_iter.bi_sector, bio_end_sector(bio))))) {
- /* As the suspend_* range is controlled by
- * userspace, we want an interruptible
- * wait.
+ bio->bi_iter.bi_sector, bio_end_sector(bio)))) {
+
+ /*
+ * As the suspend_* range is controlled by userspace, we want
+ * an interruptible wait.
*/
DEFINE_WAIT(w);
for (;;) {
bio->bi_iter.bi_sector >= mddev->suspend_hi ||
(mddev_is_clustered(mddev) &&
!md_cluster_ops->area_resyncing(mddev, WRITE,
- bio->bi_iter.bi_sector, bio_end_sector(bio))))
+ bio->bi_iter.bi_sector,
+ bio_end_sector(bio))))
break;
schedule();
}
finish_wait(&conf->wait_barrier, &w);
}
-
start_next_window = wait_barrier(conf, bio);
- bitmap = mddev->bitmap;
-
- /*
- * make_request() can abort the operation when read-ahead is being
- * used and no empty request is available.
- *
- */
- r1_bio = mempool_alloc(conf->r1bio_pool, GFP_NOIO);
-
- r1_bio->master_bio = bio;
- r1_bio->sectors = bio_sectors(bio);
- r1_bio->state = 0;
- r1_bio->mddev = mddev;
- r1_bio->sector = bio->bi_iter.bi_sector;
-
- /* We might need to issue multiple reads to different
- * devices if there are bad blocks around, so we keep
- * track of the number of reads in bio->bi_phys_segments.
- * If this is 0, there is only one r1_bio and no locking
- * will be needed when requests complete. If it is
- * non-zero, then it is the number of not-completed requests.
- */
- bio->bi_phys_segments = 0;
- bio_clear_flag(bio, BIO_SEG_VALID);
-
- if (rw == READ) {
- /*
- * read balancing logic:
- */
- int rdisk;
-
-read_again:
- rdisk = read_balance(conf, r1_bio, &max_sectors);
-
- if (rdisk < 0) {
- /* couldn't find anywhere to read from */
- raid_end_bio_io(r1_bio);
- return;
- }
- mirror = conf->mirrors + rdisk;
-
- if (test_bit(WriteMostly, &mirror->rdev->flags) &&
- bitmap) {
- /* Reading from a write-mostly device must
- * take care not to over-take any writes
- * that are 'behind'
- */
- raid1_log(mddev, "wait behind writes");
- wait_event(bitmap->behind_wait,
- atomic_read(&bitmap->behind_writes) == 0);
- }
- r1_bio->read_disk = rdisk;
- r1_bio->start_next_window = 0;
-
- read_bio = bio_clone_mddev(bio, GFP_NOIO, mddev);
- bio_trim(read_bio, r1_bio->sector - bio->bi_iter.bi_sector,
- max_sectors);
-
- r1_bio->bios[rdisk] = read_bio;
-
- read_bio->bi_iter.bi_sector = r1_bio->sector +
- mirror->rdev->data_offset;
- read_bio->bi_bdev = mirror->rdev->bdev;
- read_bio->bi_end_io = raid1_end_read_request;
- bio_set_op_attrs(read_bio, op, do_sync);
- if (test_bit(FailFast, &mirror->rdev->flags) &&
- test_bit(R1BIO_FailFast, &r1_bio->state))
- read_bio->bi_opf |= MD_FAILFAST;
- read_bio->bi_private = r1_bio;
-
- if (mddev->gendisk)
- trace_block_bio_remap(bdev_get_queue(read_bio->bi_bdev),
- read_bio, disk_devt(mddev->gendisk),
- r1_bio->sector);
-
- if (max_sectors < r1_bio->sectors) {
- /* could not read all from this device, so we will
- * need another r1_bio.
- */
-
- sectors_handled = (r1_bio->sector + max_sectors
- - bio->bi_iter.bi_sector);
- r1_bio->sectors = max_sectors;
- spin_lock_irq(&conf->device_lock);
- if (bio->bi_phys_segments == 0)
- bio->bi_phys_segments = 2;
- else
- bio->bi_phys_segments++;
- spin_unlock_irq(&conf->device_lock);
- /* Cannot call generic_make_request directly
- * as that will be queued in __make_request
- * and subsequent mempool_alloc might block waiting
- * for it. So hand bio over to raid1d.
- */
- reschedule_retry(r1_bio);
-
- r1_bio = mempool_alloc(conf->r1bio_pool, GFP_NOIO);
-
- r1_bio->master_bio = bio;
- r1_bio->sectors = bio_sectors(bio) - sectors_handled;
- r1_bio->state = 0;
- r1_bio->mddev = mddev;
- r1_bio->sector = bio->bi_iter.bi_sector +
- sectors_handled;
- goto read_again;
- } else
- generic_make_request(read_bio);
- return;
- }
-
- /*
- * WRITE:
- */
if (conf->pending_count >= max_queued_requests) {
md_wakeup_thread(mddev->thread);
raid1_log(mddev, "wait queued");
int bad_sectors;
int is_bad;
- is_bad = is_badblock(rdev, r1_bio->sector,
- max_sectors,
+ is_bad = is_badblock(rdev, r1_bio->sector, max_sectors,
&first_bad, &bad_sectors);
if (is_bad < 0) {
/* mustn't write here until the bad block is
continue;
mbio = bio_clone_mddev(bio, GFP_NOIO, mddev);
- bio_trim(mbio, r1_bio->sector - bio->bi_iter.bi_sector, max_sectors);
+ bio_trim(mbio, r1_bio->sector - bio->bi_iter.bi_sector,
+ max_sectors);
if (first_clone) {
/* do behind I/O ?
wake_up(&conf->wait_barrier);
}
+static void raid1_make_request(struct mddev *mddev, struct bio *bio)
+{
+ struct r1conf *conf = mddev->private;
+ struct r1bio *r1_bio;
+
+ /*
+ * make_request() can abort the operation when read-ahead is being
+ * used and no empty request is available.
+ *
+ */
+ r1_bio = mempool_alloc(conf->r1bio_pool, GFP_NOIO);
+
+ r1_bio->master_bio = bio;
+ r1_bio->sectors = bio_sectors(bio);
+ r1_bio->state = 0;
+ r1_bio->mddev = mddev;
+ r1_bio->sector = bio->bi_iter.bi_sector;
+
+ /*
+ * We might need to issue multiple reads to different devices if there
+ * are bad blocks around, so we keep track of the number of reads in
+ * bio->bi_phys_segments. If this is 0, there is only one r1_bio and
+ * no locking will be needed when requests complete. If it is
+ * non-zero, then it is the number of not-completed requests.
+ */
+ bio->bi_phys_segments = 0;
+ bio_clear_flag(bio, BIO_SEG_VALID);
+
+ if (bio_data_dir(bio) == READ)
+ raid1_read_request(mddev, bio, r1_bio);
+ else
+ raid1_write_request(mddev, bio, r1_bio);
+}
+
static void raid1_status(struct seq_file *seq, struct mddev *mddev)
{
struct r1conf *conf = mddev->private;
if (!IS_ERR(conf)) {
/* Array must appear to be quiesced */
conf->array_frozen = 1;
- clear_bit(MD_HAS_JOURNAL, &mddev->flags);
- clear_bit(MD_JOURNAL_CLEAN, &mddev->flags);
+ mddev_clear_unsupported_flags(mddev,
+ UNSUPPORTED_MDDEV_FLAGS);
}
return conf;
}
kfree(plug);
}
-static void __make_request(struct mddev *mddev, struct bio *bio)
+static void raid10_read_request(struct mddev *mddev, struct bio *bio,
+ struct r10bio *r10_bio)
{
struct r10conf *conf = mddev->private;
- struct r10bio *r10_bio;
struct bio *read_bio;
+ const int op = bio_op(bio);
+ const unsigned long do_sync = (bio->bi_opf & REQ_SYNC);
+ int sectors_handled;
+ int max_sectors;
+ sector_t sectors;
+ struct md_rdev *rdev;
+ int slot;
+
+ /*
+ * Register the new request and wait if the reconstruction
+ * thread has put up a bar for new requests.
+ * Continue immediately if no resync is active currently.
+ */
+ wait_barrier(conf);
+
+ sectors = bio_sectors(bio);
+ while (test_bit(MD_RECOVERY_RESHAPE, &mddev->recovery) &&
+ bio->bi_iter.bi_sector < conf->reshape_progress &&
+ bio->bi_iter.bi_sector + sectors > conf->reshape_progress) {
+ /*
+ * IO spans the reshape position. Need to wait for reshape to
+ * pass
+ */
+ raid10_log(conf->mddev, "wait reshape");
+ allow_barrier(conf);
+ wait_event(conf->wait_barrier,
+ conf->reshape_progress <= bio->bi_iter.bi_sector ||
+ conf->reshape_progress >= bio->bi_iter.bi_sector +
+ sectors);
+ wait_barrier(conf);
+ }
+
+read_again:
+ rdev = read_balance(conf, r10_bio, &max_sectors);
+ if (!rdev) {
+ raid_end_bio_io(r10_bio);
+ return;
+ }
+ slot = r10_bio->read_slot;
+
+ read_bio = bio_clone_mddev(bio, GFP_NOIO, mddev);
+ bio_trim(read_bio, r10_bio->sector - bio->bi_iter.bi_sector,
+ max_sectors);
+
+ r10_bio->devs[slot].bio = read_bio;
+ r10_bio->devs[slot].rdev = rdev;
+
+ read_bio->bi_iter.bi_sector = r10_bio->devs[slot].addr +
+ choose_data_offset(r10_bio, rdev);
+ read_bio->bi_bdev = rdev->bdev;
+ read_bio->bi_end_io = raid10_end_read_request;
+ bio_set_op_attrs(read_bio, op, do_sync);
+ if (test_bit(FailFast, &rdev->flags) &&
+ test_bit(R10BIO_FailFast, &r10_bio->state))
+ read_bio->bi_opf |= MD_FAILFAST;
+ read_bio->bi_private = r10_bio;
+
+ if (mddev->gendisk)
+ trace_block_bio_remap(bdev_get_queue(read_bio->bi_bdev),
+ read_bio, disk_devt(mddev->gendisk),
+ r10_bio->sector);
+ if (max_sectors < r10_bio->sectors) {
+ /*
+ * Could not read all from this device, so we will need another
+ * r10_bio.
+ */
+ sectors_handled = (r10_bio->sector + max_sectors
+ - bio->bi_iter.bi_sector);
+ r10_bio->sectors = max_sectors;
+ spin_lock_irq(&conf->device_lock);
+ if (bio->bi_phys_segments == 0)
+ bio->bi_phys_segments = 2;
+ else
+ bio->bi_phys_segments++;
+ spin_unlock_irq(&conf->device_lock);
+ /*
+ * Cannot call generic_make_request directly as that will be
+ * queued in __generic_make_request and subsequent
+ * mempool_alloc might block waiting for it. so hand bio over
+ * to raid10d.
+ */
+ reschedule_retry(r10_bio);
+
+ r10_bio = mempool_alloc(conf->r10bio_pool, GFP_NOIO);
+
+ r10_bio->master_bio = bio;
+ r10_bio->sectors = bio_sectors(bio) - sectors_handled;
+ r10_bio->state = 0;
+ r10_bio->mddev = mddev;
+ r10_bio->sector = bio->bi_iter.bi_sector + sectors_handled;
+ goto read_again;
+ } else
+ generic_make_request(read_bio);
+ return;
+}
+
+static void raid10_write_request(struct mddev *mddev, struct bio *bio,
+ struct r10bio *r10_bio)
+{
+ struct r10conf *conf = mddev->private;
int i;
const int op = bio_op(bio);
- const int rw = bio_data_dir(bio);
const unsigned long do_sync = (bio->bi_opf & REQ_SYNC);
const unsigned long do_fua = (bio->bi_opf & REQ_FUA);
unsigned long flags;
struct md_rdev *blocked_rdev;
struct blk_plug_cb *cb;
struct raid10_plug_cb *plug = NULL;
+ sector_t sectors;
int sectors_handled;
int max_sectors;
- int sectors;
md_write_start(mddev, bio);
while (test_bit(MD_RECOVERY_RESHAPE, &mddev->recovery) &&
bio->bi_iter.bi_sector < conf->reshape_progress &&
bio->bi_iter.bi_sector + sectors > conf->reshape_progress) {
- /* IO spans the reshape position. Need to wait for
- * reshape to pass
+ /*
+ * IO spans the reshape position. Need to wait for reshape to
+ * pass
*/
raid10_log(conf->mddev, "wait reshape");
allow_barrier(conf);
sectors);
wait_barrier(conf);
}
+
if (test_bit(MD_RECOVERY_RESHAPE, &mddev->recovery) &&
- bio_data_dir(bio) == WRITE &&
(mddev->reshape_backwards
? (bio->bi_iter.bi_sector < conf->reshape_safe &&
bio->bi_iter.bi_sector + sectors > conf->reshape_progress)
conf->reshape_safe = mddev->reshape_position;
}
- r10_bio = mempool_alloc(conf->r10bio_pool, GFP_NOIO);
-
- r10_bio->master_bio = bio;
- r10_bio->sectors = sectors;
-
- r10_bio->mddev = mddev;
- r10_bio->sector = bio->bi_iter.bi_sector;
- r10_bio->state = 0;
-
- /* We might need to issue multiple reads to different
- * devices if there are bad blocks around, so we keep
- * track of the number of reads in bio->bi_phys_segments.
- * If this is 0, there is only one r10_bio and no locking
- * will be needed when the request completes. If it is
- * non-zero, then it is the number of not-completed requests.
- */
- bio->bi_phys_segments = 0;
- bio_clear_flag(bio, BIO_SEG_VALID);
-
- if (rw == READ) {
- /*
- * read balancing logic:
- */
- struct md_rdev *rdev;
- int slot;
-
-read_again:
- rdev = read_balance(conf, r10_bio, &max_sectors);
- if (!rdev) {
- raid_end_bio_io(r10_bio);
- return;
- }
- slot = r10_bio->read_slot;
-
- read_bio = bio_clone_mddev(bio, GFP_NOIO, mddev);
- bio_trim(read_bio, r10_bio->sector - bio->bi_iter.bi_sector,
- max_sectors);
-
- r10_bio->devs[slot].bio = read_bio;
- r10_bio->devs[slot].rdev = rdev;
-
- read_bio->bi_iter.bi_sector = r10_bio->devs[slot].addr +
- choose_data_offset(r10_bio, rdev);
- read_bio->bi_bdev = rdev->bdev;
- read_bio->bi_end_io = raid10_end_read_request;
- bio_set_op_attrs(read_bio, op, do_sync);
- if (test_bit(FailFast, &rdev->flags) &&
- test_bit(R10BIO_FailFast, &r10_bio->state))
- read_bio->bi_opf |= MD_FAILFAST;
- read_bio->bi_private = r10_bio;
-
- if (mddev->gendisk)
- trace_block_bio_remap(bdev_get_queue(read_bio->bi_bdev),
- read_bio, disk_devt(mddev->gendisk),
- r10_bio->sector);
- if (max_sectors < r10_bio->sectors) {
- /* Could not read all from this device, so we will
- * need another r10_bio.
- */
- sectors_handled = (r10_bio->sector + max_sectors
- - bio->bi_iter.bi_sector);
- r10_bio->sectors = max_sectors;
- spin_lock_irq(&conf->device_lock);
- if (bio->bi_phys_segments == 0)
- bio->bi_phys_segments = 2;
- else
- bio->bi_phys_segments++;
- spin_unlock_irq(&conf->device_lock);
- /* Cannot call generic_make_request directly
- * as that will be queued in __generic_make_request
- * and subsequent mempool_alloc might block
- * waiting for it. so hand bio over to raid10d.
- */
- reschedule_retry(r10_bio);
-
- r10_bio = mempool_alloc(conf->r10bio_pool, GFP_NOIO);
-
- r10_bio->master_bio = bio;
- r10_bio->sectors = bio_sectors(bio) - sectors_handled;
- r10_bio->state = 0;
- r10_bio->mddev = mddev;
- r10_bio->sector = bio->bi_iter.bi_sector +
- sectors_handled;
- goto read_again;
- } else
- generic_make_request(read_bio);
- return;
- }
-
- /*
- * WRITE:
- */
if (conf->pending_count >= max_queued_requests) {
md_wakeup_thread(mddev->thread);
raid10_log(mddev, "wait queued");
int bad_sectors;
int is_bad;
- is_bad = is_badblock(rdev, dev_sector,
- max_sectors,
+ is_bad = is_badblock(rdev, dev_sector, max_sectors,
&first_bad, &bad_sectors);
if (is_bad < 0) {
/* Mustn't write here until the bad block
r10_bio->devs[i].bio = mbio;
mbio->bi_iter.bi_sector = (r10_bio->devs[i].addr+
- choose_data_offset(r10_bio,
- rdev));
+ choose_data_offset(r10_bio, rdev));
mbio->bi_bdev = rdev->bdev;
mbio->bi_end_io = raid10_end_write_request;
bio_set_op_attrs(mbio, op, do_sync | do_fua);
r10_bio->devs[i].repl_bio = mbio;
mbio->bi_iter.bi_sector = (r10_bio->devs[i].addr +
- choose_data_offset(
- r10_bio, rdev));
+ choose_data_offset(r10_bio, rdev));
mbio->bi_bdev = rdev->bdev;
mbio->bi_end_io = raid10_end_write_request;
bio_set_op_attrs(mbio, op, do_sync | do_fua);
one_write_done(r10_bio);
}
+static void __make_request(struct mddev *mddev, struct bio *bio)
+{
+ struct r10conf *conf = mddev->private;
+ struct r10bio *r10_bio;
+
+ r10_bio = mempool_alloc(conf->r10bio_pool, GFP_NOIO);
+
+ r10_bio->master_bio = bio;
+ r10_bio->sectors = bio_sectors(bio);
+
+ r10_bio->mddev = mddev;
+ r10_bio->sector = bio->bi_iter.bi_sector;
+ r10_bio->state = 0;
+
+ /*
+ * We might need to issue multiple reads to different devices if there
+ * are bad blocks around, so we keep track of the number of reads in
+ * bio->bi_phys_segments. If this is 0, there is only one r10_bio and
+ * no locking will be needed when the request completes. If it is
+ * non-zero, then it is the number of not-completed requests.
+ */
+ bio->bi_phys_segments = 0;
+ bio_clear_flag(bio, BIO_SEG_VALID);
+
+ if (bio_data_dir(bio) == READ)
+ raid10_read_request(mddev, bio, r10_bio);
+ else
+ raid10_write_request(mddev, bio, r10_bio);
+}
+
static void raid10_make_request(struct mddev *mddev, struct bio *bio)
{
struct r10conf *conf = mddev->private;
static struct stripe_head *
r5c_recovery_alloc_stripe(struct r5conf *conf,
- sector_t stripe_sect,
- sector_t log_start)
+ sector_t stripe_sect)
{
struct stripe_head *sh;
return NULL; /* no more stripe available */
r5l_recovery_reset_stripe(sh);
- sh->log_start = log_start;
return sh;
}
stripe_sect);
if (!sh) {
- sh = r5c_recovery_alloc_stripe(conf, stripe_sect, ctx->pos);
+ sh = r5c_recovery_alloc_stripe(conf, stripe_sect);
/*
* cannot get stripe from raid5_get_active_stripe
* try replay some stripes
r5c_recovery_replay_stripes(
cached_stripe_list, ctx);
sh = r5c_recovery_alloc_stripe(
- conf, stripe_sect, ctx->pos);
+ conf, stripe_sect);
}
if (!sh) {
pr_debug("md/raid:%s: Increasing stripe cache size to %d to recovery data on journal.\n",
conf->min_nr_stripes * 2);
raid5_set_cache_size(mddev,
conf->min_nr_stripes * 2);
- sh = r5c_recovery_alloc_stripe(
- conf, stripe_sect, ctx->pos);
+ sh = r5c_recovery_alloc_stripe(conf,
+ stripe_sect);
}
if (!sh) {
pr_err("md/raid:%s: Cannot get enough stripes due to memory pressure. Recovery failed.\n",
if (!test_bit(STRIPE_R5C_CACHING, &sh->state) &&
test_bit(R5_Wantwrite, &sh->dev[sh->pd_idx].flags)) {
r5l_recovery_replay_one_stripe(conf, sh, ctx);
- sh->log_start = ctx->pos;
list_move_tail(&sh->lru, cached_stripe_list);
}
r5l_recovery_load_data(log, sh, ctx, payload,
set_bit(R5_UPTODATE, &dev->flags);
}
}
- list_add_tail(&sh->r5c, &log->stripe_in_journal_list);
- atomic_inc(&log->stripe_in_journal_count);
}
/*
struct stripe_head *sh, *next;
struct mddev *mddev = log->rdev->mddev;
struct page *page;
+ sector_t next_checkpoint = MaxSector;
page = alloc_page(GFP_KERNEL);
if (!page) {
return -ENOMEM;
}
+ WARN_ON(list_empty(&ctx->cached_list));
+
list_for_each_entry_safe(sh, next, &ctx->cached_list, lru) {
struct r5l_meta_block *mb;
int i;
sync_page_io(log->rdev, ctx->pos, PAGE_SIZE, page,
REQ_OP_WRITE, REQ_FUA, false);
sh->log_start = ctx->pos;
+ list_add_tail(&sh->r5c, &log->stripe_in_journal_list);
+ atomic_inc(&log->stripe_in_journal_count);
ctx->pos = write_pos;
ctx->seq += 1;
-
+ next_checkpoint = sh->log_start;
list_del_init(&sh->lru);
raid5_release_stripe(sh);
}
+ log->next_checkpoint = next_checkpoint;
__free_page(page);
return 0;
}
struct r5l_recovery_ctx ctx;
int ret;
sector_t pos;
- struct stripe_head *sh;
ctx.pos = log->last_checkpoint;
ctx.seq = log->last_cp_seq;
log->next_checkpoint = ctx.pos;
r5l_log_write_empty_meta_block(log, ctx.pos, ctx.seq++);
ctx.pos = r5l_ring_add(log, ctx.pos, BLOCK_SECTORS);
- } else {
- sh = list_last_entry(&ctx.cached_list, struct stripe_head, lru);
- log->next_checkpoint = sh->log_start;
}
if ((ctx.data_only_stripes == 0) && (ctx.data_parity_stripes == 0))
pr_debug("md/raid:%s: starting from clean shutdown\n",
mdname(mddev));
else {
- pr_debug("md/raid:%s: recoverying %d data-only stripes and %d data-parity stripes\n",
+ pr_debug("md/raid:%s: recovering %d data-only stripes and %d data-parity stripes\n",
mdname(mddev), ctx.data_only_stripes,
ctx.data_parity_stripes);
if (do_wakeup)
wake_up(&conf->wait_for_overlap);
- if (conf->log->r5c_journal_mode == R5C_JOURNAL_MODE_WRITE_THROUGH)
- return;
-
spin_lock_irq(&conf->log->stripe_in_journal_lock);
list_del_init(&sh->r5c);
spin_unlock_irq(&conf->log->stripe_in_journal_lock);
spin_lock_init(&log->stripe_in_journal_lock);
atomic_set(&log->stripe_in_journal_count, 0);
+ rcu_assign_pointer(conf->log, log);
+
if (r5l_load_log(log))
goto error;
- rcu_assign_pointer(conf->log, log);
set_bit(MD_HAS_JOURNAL, &conf->mddev->flags);
return 0;
error:
+ rcu_assign_pointer(conf->log, NULL);
md_unregister_thread(&log->reclaim_thread);
reclaim_thread:
mempool_destroy(log->meta_pool);
#include "raid0.h"
#include "bitmap.h"
+#define UNSUPPORTED_MDDEV_FLAGS (1L << MD_FAILFAST_SUPPORTED)
+
#define cpu_to_group(cpu) cpu_to_node(cpu)
#define ANY_GROUP NUMA_NO_NODE
mddev->new_chunk_sectors = chunksect;
ret = setup_conf(mddev);
- if (!IS_ERR_VALUE(ret))
- clear_bit(MD_FAILFAST_SUPPORTED, &mddev->flags);
+ if (!IS_ERR(ret))
+ mddev_clear_unsupported_flags(mddev,
+ UNSUPPORTED_MDDEV_FLAGS);
return ret;
}
struct ms_id_register id_reg;
if (!(*mrq)) {
- memstick_init_req(&card->current_mrq, MS_TPC_READ_REG, NULL,
+ memstick_init_req(&card->current_mrq, MS_TPC_READ_REG, &id_reg,
sizeof(struct ms_id_register));
*mrq = &card->current_mrq;
return 0;
{
int ret;
+ if (!cldev->bus->hbm_f_os_supported)
+ return;
+
ret = mei_cldev_enable(cldev);
if (ret)
return;
dev->hbm_f_ev_supported);
pos += scnprintf(buf + pos, bufsz - pos, "\tFA: %01d\n",
dev->hbm_f_fa_supported);
+ pos += scnprintf(buf + pos, bufsz - pos, "\tOS: %01d\n",
+ dev->hbm_f_os_supported);
}
pos += scnprintf(buf + pos, bufsz - pos, "pg: %s, %s\n",
/* Fixed Address Client Support */
if (dev->version.major_version >= HBM_MAJOR_VERSION_FA)
dev->hbm_f_fa_supported = 1;
+
+ /* OS ver message Support */
+ if (dev->version.major_version >= HBM_MAJOR_VERSION_OS)
+ dev->hbm_f_os_supported = 1;
}
/**
#define HBM_MINOR_VERSION_FA 0
#define HBM_MAJOR_VERSION_FA 2
+/*
+ * MEI version with OS ver message support
+ */
+#define HBM_MINOR_VERSION_OS 0
+#define HBM_MAJOR_VERSION_OS 2
+
/* Host bus message command opcode */
#define MEI_HBM_CMD_OP_MSK 0x7f
/* Host bus message command RESPONSE */
* @hbm_f_ev_supported : hbm feature event notification
* @hbm_f_fa_supported : hbm feature fixed address client
* @hbm_f_ie_supported : hbm feature immediate reply to enum request
+ * @hbm_f_os_supported : hbm feature support OS ver message
*
* @me_clients_rwsem: rw lock over me_clients list
* @me_clients : list of FW clients
unsigned int hbm_f_ev_supported:1;
unsigned int hbm_f_fa_supported:1;
unsigned int hbm_f_ie_supported:1;
+ unsigned int hbm_f_os_supported:1;
struct rw_semaphore me_clients_rwsem;
struct list_head me_clients;
}
} while (busy);
- if (host->ops->card_busy && send_status)
- return mmc_switch_status(card);
-
return 0;
}
if (!use_busy_signal)
goto out;
- /* Switch to new timing before poll and check switch status. */
- if (timing)
- mmc_set_timing(host, timing);
-
/*If SPI or used HW busy detection above, then we don't need to poll. */
if (((host->caps & MMC_CAP_WAIT_WHILE_BUSY) && use_r1b_resp) ||
- mmc_host_is_spi(host)) {
- if (send_status)
- err = mmc_switch_status(card);
+ mmc_host_is_spi(host))
goto out_tim;
- }
/* Let's try to poll to find out when the command is completed. */
err = mmc_poll_for_busy(card, timeout_ms, send_status, retry_crc_err);
+ if (err)
+ goto out;
out_tim:
- if (err && timing)
- mmc_set_timing(host, old_timing);
+ /* Switch to new timing before check switch status. */
+ if (timing)
+ mmc_set_timing(host, timing);
+
+ if (send_status) {
+ err = mmc_switch_status(card);
+ if (err && timing)
+ mmc_set_timing(host, old_timing);
+ }
out:
mmc_retune_release(host);
{
struct meson_host *host = dev_id;
struct mmc_request *mrq;
- struct mmc_command *cmd = host->cmd;
+ struct mmc_command *cmd;
u32 irq_en, status, raw_status;
irqreturn_t ret = IRQ_HANDLED;
if (WARN_ON(!host))
return IRQ_NONE;
+ cmd = host->cmd;
+
mrq = host->mrq;
if (WARN_ON(!mrq))
int ret = IRQ_HANDLED;
if (WARN_ON(!mrq))
- ret = IRQ_NONE;
+ return IRQ_NONE;
if (WARN_ON(!cmd))
- ret = IRQ_NONE;
+ return IRQ_NONE;
data = cmd->data;
if (data) {
cmd0 = BF_SSP(cmd->opcode, CMD0_CMD);
cmd1 = cmd->arg;
+ if (cmd->opcode == MMC_STOP_TRANSMISSION)
+ cmd0 |= BM_SSP_CMD0_APPEND_8CYC;
+
if (host->sdio_irq_en) {
ctrl0 |= BM_SSP_CTRL0_SDIO_IRQ_CHECK;
cmd0 |= BM_SSP_CMD0_CONT_CLKING_EN | BM_SSP_CMD0_SLOW_CLKING_EN;
ssp->base + HW_SSP_BLOCK_SIZE);
}
- if ((cmd->opcode == MMC_STOP_TRANSMISSION) ||
- (cmd->opcode == SD_IO_RW_EXTENDED))
+ if (cmd->opcode == SD_IO_RW_EXTENDED)
cmd0 |= BM_SSP_CMD0_APPEND_8CYC;
cmd1 = cmd->arg;
/* Power on the SDHCI controller and its children */
acpi_device_fix_up_power(device);
list_for_each_entry(child, &device->children, node)
- acpi_device_fix_up_power(child);
+ if (child->status.present && child->status.enabled)
+ acpi_device_fix_up_power(child);
if (acpi_bus_get_status(device) || !device->status.present)
return -ENODEV;
config MTD_NAND_OXNAS
tristate "NAND Flash support for Oxford Semiconductor SoC"
+ depends on HAS_IOMEM
help
This enables the NAND flash controller on Oxford Semiconductor SoCs.
Flexible Static Memory Controller (FSMC)
config MTD_NAND_XWAY
- tristate "Support for NAND on Lantiq XWAY SoC"
+ bool "Support for NAND on Lantiq XWAY SoC"
depends on LANTIQ && SOC_TYPE_XWAY
help
Enables support for NAND Flash chips on Lantiq XWAY SoCs. NAND is attached
init_completion(&host->comp_controller);
host->irq = platform_get_irq(pdev, 0);
- if ((host->irq < 0) || (host->irq >= NR_IRQS)) {
+ if (host->irq < 0) {
dev_err(&pdev->dev, "failed to get platform irq\n");
res = -EINVAL;
goto err_exit3;
if (IS_ERR(nfc->pbus_base))
return PTR_ERR(nfc->pbus_base);
+ writel_relaxed(MODE_RAW, nfc->pbus_base + PBUS_PAD_MODE);
+
clk = clk_get(&pdev->dev, NULL);
if (IS_ERR(clk))
return PTR_ERR(clk);
- nfc->chan = dma_request_chan(&pdev->dev, "nfc_sbox");
+ nfc->chan = dma_request_chan(&pdev->dev, "rxtx");
if (IS_ERR(nfc->chan))
return PTR_ERR(nfc->chan);
{ .compatible = "lantiq,nand-xway" },
{},
};
-MODULE_DEVICE_TABLE(of, xway_nand_match);
static struct platform_driver xway_nand_driver = {
.probe = xway_nand_probe,
},
};
-module_platform_driver(xway_nand_driver);
-
-MODULE_LICENSE("GPL");
+builtin_platform_driver(xway_nand_driver);
*/
static int ipddp_create(struct ipddp_route *new_rt)
{
- struct ipddp_route *rt = kmalloc(sizeof(*rt), GFP_KERNEL);
+ struct ipddp_route *rt = kzalloc(sizeof(*rt), GFP_KERNEL);
if (rt == NULL)
return -ENOMEM;
if (addr == BRCM_PSEUDO_PHY_ADDR && priv->indir_phy_mask & BIT(addr))
return bcm_sf2_sw_indir_rw(priv, 1, addr, regnum, 0);
else
- return mdiobus_read(priv->master_mii_bus, addr, regnum);
+ return mdiobus_read_nested(priv->master_mii_bus, addr, regnum);
}
static int bcm_sf2_sw_mdio_write(struct mii_bus *bus, int addr, int regnum,
if (addr == BRCM_PSEUDO_PHY_ADDR && priv->indir_phy_mask & BIT(addr))
bcm_sf2_sw_indir_rw(priv, 0, addr, regnum, val);
else
- mdiobus_write(priv->master_mii_bus, addr, regnum, val);
+ mdiobus_write_nested(priv->master_mii_bus, addr, regnum, val);
return 0;
}
const char *reg_names[BCM_SF2_REGS_NUM] = BCM_SF2_REGS_NAME;
struct device_node *dn = pdev->dev.of_node;
struct b53_platform_data *pdata;
+ struct dsa_switch_ops *ops;
struct bcm_sf2_priv *priv;
struct b53_device *dev;
struct dsa_switch *ds;
if (!priv)
return -ENOMEM;
+ ops = devm_kzalloc(&pdev->dev, sizeof(*ops), GFP_KERNEL);
+ if (!ops)
+ return -ENOMEM;
+
dev = b53_switch_alloc(&pdev->dev, &bcm_sf2_io_ops, priv);
if (!dev)
return -ENOMEM;
ds = dev->ds;
/* Override the parts that are non-standard wrt. normal b53 devices */
+ memcpy(ops, ds->ops, sizeof(*ops));
+ ds->ops = ops;
ds->ops->get_tag_protocol = bcm_sf2_sw_get_tag_protocol;
ds->ops->setup = bcm_sf2_sw_setup;
ds->ops->get_phy_flags = bcm_sf2_sw_get_phy_flags;
}
}
+isr_done:
/* If there is not a separate AN irq, handle it here */
if (pdata->dev_irq == pdata->an_irq)
pdata->phy_if.an_isr(irq, pdata);
if (pdata->vdata->i2c_support && (pdata->dev_irq == pdata->i2c_irq))
pdata->i2c_if.i2c_isr(irq, pdata);
-isr_done:
return IRQ_HANDLED;
}
unsigned int c_index, last_c_index, last_tx_cn, num_tx_cbs;
unsigned int pkts_compl = 0, bytes_compl = 0;
struct bcm_sysport_cb *cb;
- struct netdev_queue *txq;
u32 hw_ind;
- txq = netdev_get_tx_queue(ndev, ring->index);
-
/* Compute how many descriptors have been processed since last call */
hw_ind = tdma_readl(priv, TDMA_DESC_RING_PROD_CONS_INDEX(ring->index));
c_index = (hw_ind >> RING_CONS_INDEX_SHIFT) & RING_CONS_INDEX_MASK;
ring->c_index = c_index;
- if (netif_tx_queue_stopped(txq) && pkts_compl)
- netif_tx_wake_queue(txq);
-
netif_dbg(priv, tx_done, ndev,
"ring=%d c_index=%d pkts_compl=%d, bytes_compl=%d\n",
ring->index, ring->c_index, pkts_compl, bytes_compl);
static unsigned int bcm_sysport_tx_reclaim(struct bcm_sysport_priv *priv,
struct bcm_sysport_tx_ring *ring)
{
+ struct netdev_queue *txq;
unsigned int released;
unsigned long flags;
+ txq = netdev_get_tx_queue(priv->netdev, ring->index);
+
spin_lock_irqsave(&ring->lock, flags);
released = __bcm_sysport_tx_reclaim(priv, ring);
+ if (released)
+ netif_tx_wake_queue(txq);
+
spin_unlock_irqrestore(&ring->lock, flags);
return released;
}
+/* Locked version of the per-ring TX reclaim, but does not wake the queue */
+static void bcm_sysport_tx_clean(struct bcm_sysport_priv *priv,
+ struct bcm_sysport_tx_ring *ring)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&ring->lock, flags);
+ __bcm_sysport_tx_reclaim(priv, ring);
+ spin_unlock_irqrestore(&ring->lock, flags);
+}
+
static int bcm_sysport_tx_poll(struct napi_struct *napi, int budget)
{
struct bcm_sysport_tx_ring *ring =
napi_disable(&ring->napi);
netif_napi_del(&ring->napi);
- bcm_sysport_tx_reclaim(priv, ring);
+ bcm_sysport_tx_clean(priv, ring);
kfree(ring->cbs);
ring->cbs = NULL;
tg3_mem_rx_release(tp);
tg3_mem_tx_release(tp);
+ /* Protect tg3_get_stats64() from reading freed tp->hw_stats. */
+ tg3_full_lock(tp, 0);
if (tp->hw_stats) {
dma_free_coherent(&tp->pdev->dev, sizeof(struct tg3_hw_stats),
tp->hw_stats, tp->stats_mapping);
tp->hw_stats = NULL;
}
+ tg3_full_unlock(tp);
}
/*
struct bgx {
u8 bgx_id;
struct lmac lmac[MAX_LMAC_PER_BGX];
- int lmac_count;
+ u8 lmac_count;
u8 max_lmac;
+ u8 acpi_lmac_idx;
void __iomem *reg_base;
struct pci_dev *pdev;
bool is_dlm;
if (acpi_bus_get_device(handle, &adev))
goto out;
- acpi_get_mac_address(dev, adev, bgx->lmac[bgx->lmac_count].mac);
+ acpi_get_mac_address(dev, adev, bgx->lmac[bgx->acpi_lmac_idx].mac);
- SET_NETDEV_DEV(&bgx->lmac[bgx->lmac_count].netdev, dev);
+ SET_NETDEV_DEV(&bgx->lmac[bgx->acpi_lmac_idx].netdev, dev);
- bgx->lmac[bgx->lmac_count].lmacid = bgx->lmac_count;
+ bgx->lmac[bgx->acpi_lmac_idx].lmacid = bgx->acpi_lmac_idx;
+ bgx->acpi_lmac_idx++; /* move to next LMAC */
out:
- bgx->lmac_count++;
return AE_OK;
}
err:
mutex_unlock(&adapter->mcc_lock);
- if (status == MCC_STATUS_UNAUTHORIZED_REQUEST)
+ if (base_status(status) == MCC_STATUS_UNAUTHORIZED_REQUEST)
status = -EPERM;
return status;
/* Check if mac has already been added as part of uc-list */
for (i = 0; i < adapter->uc_macs; i++) {
- if (ether_addr_equal((u8 *)&adapter->uc_list[i * ETH_ALEN],
- mac)) {
+ if (ether_addr_equal(adapter->uc_list[i].mac, mac)) {
/* mac already added, skip addition */
adapter->pmac_id[0] = adapter->pmac_id[i + 1];
return 0;
if (ether_addr_equal(addr->sa_data, adapter->dev_mac))
return 0;
+ /* BE3 VFs without FILTMGMT privilege are not allowed to set its MAC
+ * address
+ */
+ if (BEx_chip(adapter) && be_virtfn(adapter) &&
+ !check_privilege(adapter, BE_PRIV_FILTMGMT))
+ return -EPERM;
+
/* if device is not running, copy MAC to netdev->dev_addr */
if (!netif_running(netdev))
goto done;
static int be_uc_mac_add(struct be_adapter *adapter, int uc_idx)
{
- if (ether_addr_equal((u8 *)&adapter->uc_list[uc_idx * ETH_ALEN],
- adapter->dev_mac)) {
+ if (ether_addr_equal(adapter->uc_list[uc_idx].mac, adapter->dev_mac)) {
adapter->pmac_id[uc_idx + 1] = adapter->pmac_id[0];
return 0;
}
- return be_cmd_pmac_add(adapter,
- (u8 *)&adapter->uc_list[uc_idx * ETH_ALEN],
+ return be_cmd_pmac_add(adapter, adapter->uc_list[uc_idx].mac,
adapter->if_handle,
&adapter->pmac_id[uc_idx + 1], 0);
}
}
if (adapter->update_uc_list) {
- i = 1; /* First slot is claimed by the Primary MAC */
-
/* cache the uc-list in adapter array */
+ i = 0;
netdev_for_each_uc_addr(ha, netdev) {
ether_addr_copy(adapter->uc_list[i].mac, ha->addr);
i++;
static void be_disable_if_filters(struct be_adapter *adapter)
{
- be_dev_mac_del(adapter, adapter->pmac_id[0]);
+ /* Don't delete MAC on BE3 VFs without FILTMGMT privilege */
+ if (!BEx_chip(adapter) || !be_virtfn(adapter) ||
+ check_privilege(adapter, BE_PRIV_FILTMGMT))
+ be_dev_mac_del(adapter, adapter->pmac_id[0]);
+
be_clear_uc_list(adapter);
be_clear_mc_list(adapter);
if (status)
return status;
- /* For BE3 VFs, the PF programs the initial MAC address */
- if (!(BEx_chip(adapter) && be_virtfn(adapter))) {
+ /* Don't add MAC on BE3 VFs without FILTMGMT privilege */
+ if (!BEx_chip(adapter) || !be_virtfn(adapter) ||
+ check_privilege(adapter, BE_PRIV_FILTMGMT)) {
status = be_dev_mac_add(adapter, adapter->netdev->dev_addr);
if (status)
return status;
PAGE_SIZE,
DMA_FROM_DEVICE,
DMA_ATTR_SKIP_CPU_SYNC);
- __page_frag_drain(buffer_info->page, 0,
- buffer_info->pagecnt_bias);
+ __page_frag_cache_drain(buffer_info->page,
+ buffer_info->pagecnt_bias);
buffer_info->page = NULL;
}
dma_unmap_page_attrs(rx_ring->dev, rx_buffer->dma,
PAGE_SIZE, DMA_FROM_DEVICE,
DMA_ATTR_SKIP_CPU_SYNC);
- __page_frag_drain(page, 0, rx_buffer->pagecnt_bias);
+ __page_frag_cache_drain(page, rx_buffer->pagecnt_bias);
}
/* clear contents of rx_buffer */
{
struct mlx4_cq *cq;
+ rcu_read_lock();
cq = radix_tree_lookup(&mlx4_priv(dev)->cq_table.tree,
cqn & (dev->caps.num_cqs - 1));
+ rcu_read_unlock();
+
if (!cq) {
mlx4_dbg(dev, "Completion event for bogus CQ %08x\n", cqn);
return;
}
+ /* Acessing the CQ outside of rcu_read_lock is safe, because
+ * the CQ is freed only after interrupt handling is completed.
+ */
++cq->arm_sn;
cq->comp(cq);
struct mlx4_cq_table *cq_table = &mlx4_priv(dev)->cq_table;
struct mlx4_cq *cq;
- spin_lock(&cq_table->lock);
-
+ rcu_read_lock();
cq = radix_tree_lookup(&cq_table->tree, cqn & (dev->caps.num_cqs - 1));
- if (cq)
- atomic_inc(&cq->refcount);
-
- spin_unlock(&cq_table->lock);
+ rcu_read_unlock();
if (!cq) {
- mlx4_warn(dev, "Async event for bogus CQ %08x\n", cqn);
+ mlx4_dbg(dev, "Async event for bogus CQ %08x\n", cqn);
return;
}
+ /* Acessing the CQ outside of rcu_read_lock is safe, because
+ * the CQ is freed only after interrupt handling is completed.
+ */
cq->event(cq, event_type);
-
- if (atomic_dec_and_test(&cq->refcount))
- complete(&cq->free);
}
static int mlx4_SW2HW_CQ(struct mlx4_dev *dev, struct mlx4_cmd_mailbox *mailbox,
if (err)
return err;
- spin_lock_irq(&cq_table->lock);
+ spin_lock(&cq_table->lock);
err = radix_tree_insert(&cq_table->tree, cq->cqn, cq);
- spin_unlock_irq(&cq_table->lock);
+ spin_unlock(&cq_table->lock);
if (err)
goto err_icm;
return 0;
err_radix:
- spin_lock_irq(&cq_table->lock);
+ spin_lock(&cq_table->lock);
radix_tree_delete(&cq_table->tree, cq->cqn);
- spin_unlock_irq(&cq_table->lock);
+ spin_unlock(&cq_table->lock);
err_icm:
mlx4_cq_free_icm(dev, cq->cqn);
if (err)
mlx4_warn(dev, "HW2SW_CQ failed (%d) for CQN %06x\n", err, cq->cqn);
+ spin_lock(&cq_table->lock);
+ radix_tree_delete(&cq_table->tree, cq->cqn);
+ spin_unlock(&cq_table->lock);
+
synchronize_irq(priv->eq_table.eq[MLX4_CQ_TO_EQ_VECTOR(cq->vector)].irq);
if (priv->eq_table.eq[MLX4_CQ_TO_EQ_VECTOR(cq->vector)].irq !=
priv->eq_table.eq[MLX4_EQ_ASYNC].irq)
synchronize_irq(priv->eq_table.eq[MLX4_EQ_ASYNC].irq);
- spin_lock_irq(&cq_table->lock);
- radix_tree_delete(&cq_table->tree, cq->cqn);
- spin_unlock_irq(&cq_table->lock);
-
if (atomic_dec_and_test(&cq->refcount))
complete(&cq->free);
wait_for_completion(&cq->free);
/* Process all completions if exist to prevent
* the queues freezing if they are full
*/
- for (i = 0; i < priv->rx_ring_num; i++)
+ for (i = 0; i < priv->rx_ring_num; i++) {
+ local_bh_disable();
napi_schedule(&priv->rx_cq[i]->napi);
+ local_bh_enable();
+ }
netif_tx_start_all_queues(dev);
netif_device_attach(dev);
if (priv->tx_ring_num[TX_XDP] &&
!mlx4_en_check_xdp_mtu(dev, new_mtu))
- return -ENOTSUPP;
+ return -EOPNOTSUPP;
dev->mtu = new_mtu;
break;
case MLX4_EVENT_TYPE_SRQ_LIMIT:
- mlx4_dbg(dev, "%s: MLX4_EVENT_TYPE_SRQ_LIMIT\n",
- __func__);
+ mlx4_dbg(dev, "%s: MLX4_EVENT_TYPE_SRQ_LIMIT. srq_no=0x%x, eq 0x%x\n",
+ __func__, be32_to_cpu(eqe->event.srq.srqn),
+ eq->eqn);
case MLX4_EVENT_TYPE_SRQ_CATAS_ERROR:
if (mlx4_is_master(dev)) {
/* forward only to slave owning the SRQ */
eq->eqn, eq->cons_index, ret);
break;
}
- mlx4_warn(dev, "%s: slave:%d, srq_no:0x%x, event: %02x(%02x)\n",
- __func__, slave,
- be32_to_cpu(eqe->event.srq.srqn),
- eqe->type, eqe->subtype);
+ if (eqe->type ==
+ MLX4_EVENT_TYPE_SRQ_CATAS_ERROR)
+ mlx4_warn(dev, "%s: slave:%d, srq_no:0x%x, event: %02x(%02x)\n",
+ __func__, slave,
+ be32_to_cpu(eqe->event.srq.srqn),
+ eqe->type, eqe->subtype);
if (!ret && slave != dev->caps.function) {
- mlx4_warn(dev, "%s: sending event %02x(%02x) to slave:%d\n",
- __func__, eqe->type,
- eqe->subtype, slave);
+ if (eqe->type ==
+ MLX4_EVENT_TYPE_SRQ_CATAS_ERROR)
+ mlx4_warn(dev, "%s: sending event %02x(%02x) to slave:%d\n",
+ __func__, eqe->type,
+ eqe->subtype, slave);
mlx4_slave_event(dev, slave, eqe);
break;
}
put_res(dev, slave, srqn, RES_SRQ);
qp->srq = srq;
}
+
+ /* Save param3 for dynamic changes from VST back to VGT */
+ qp->param3 = qpc->param3;
put_res(dev, slave, rcqn, RES_CQ);
put_res(dev, slave, mtt_base, RES_MTT);
res_end_move(dev, slave, RES_QP, qpn);
int qpn = vhcr->in_modifier & 0x7fffff;
struct res_qp *qp;
u8 orig_sched_queue;
- __be32 orig_param3 = qpc->param3;
u8 orig_vlan_control = qpc->pri_path.vlan_control;
u8 orig_fvl_rx = qpc->pri_path.fvl_rx;
u8 orig_pri_path_fl = qpc->pri_path.fl;
*/
if (!err) {
qp->sched_queue = orig_sched_queue;
- qp->param3 = orig_param3;
qp->vlan_control = orig_vlan_control;
qp->fvl_rx = orig_fvl_rx;
qp->pri_path_fl = orig_pri_path_fl;
static void mlx5e_nic_cleanup(struct mlx5e_priv *priv)
{
- struct mlx5_core_dev *mdev = priv->mdev;
- struct mlx5_eswitch *esw = mdev->priv.eswitch;
-
mlx5e_vxlan_cleanup(priv);
- if (MLX5_CAP_GEN(mdev, vport_group_manager))
- mlx5_eswitch_unregister_vport_rep(esw, 0);
-
if (priv->xdp_prog)
bpf_prog_put(priv->xdp_prog);
}
static void mlx5e_nic_disable(struct mlx5e_priv *priv)
{
+ struct mlx5_core_dev *mdev = priv->mdev;
+ struct mlx5_eswitch *esw = mdev->priv.eswitch;
+
queue_work(priv->wq, &priv->set_rx_mode_work);
+ if (MLX5_CAP_GEN(mdev, vport_group_manager))
+ mlx5_eswitch_unregister_vport_rep(esw, 0);
mlx5e_disable_async_events(priv);
- mlx5_lag_remove(priv->mdev);
+ mlx5_lag_remove(mdev);
}
static const struct mlx5e_profile mlx5e_nic_profile = {
switch (am->tune_state) {
case MLX5E_AM_PARKING_ON_TOP:
case MLX5E_AM_PARKING_TIRED:
- WARN_ONCE(true, "mlx5e_am_on_top: PARKING\n");
return true;
case MLX5E_AM_GOING_RIGHT:
return (am->steps_left > 1) && (am->steps_right == 1);
switch (am->tune_state) {
case MLX5E_AM_PARKING_ON_TOP:
case MLX5E_AM_PARKING_TIRED:
- WARN_ONCE(true, "mlx5e_am_turn: PARKING\n");
break;
case MLX5E_AM_GOING_RIGHT:
am->tune_state = MLX5E_AM_GOING_LEFT;
switch (am->tune_state) {
case MLX5E_AM_PARKING_ON_TOP:
case MLX5E_AM_PARKING_TIRED:
- WARN_ONCE(true, "mlx5e_am_step: PARKING\n");
break;
case MLX5E_AM_GOING_RIGHT:
if (am->profile_ix == (MLX5E_PARAMS_AM_NUM_PROFILES - 1))
u32 delta_us = ktime_us_delta(end->time, start->time);
unsigned int npkts = end->pkt_ctr - start->pkt_ctr;
- if (!delta_us) {
- WARN_ONCE(true, "mlx5e_am_calc_stats: delta_us=0\n");
+ if (!delta_us)
return;
- }
curr_stats->ppms = (npkts * USEC_PER_MSEC) / delta_us;
curr_stats->epms = (MLX5E_AM_NEVENTS * USEC_PER_MSEC) / delta_us;
}
}
+/* we get here also when setting rule to the FW failed, etc. It means that the
+ * flow rule itself might not exist, but some offloading related to the actions
+ * should be cleaned.
+ */
static void mlx5e_tc_del_flow(struct mlx5e_priv *priv,
struct mlx5e_tc_flow *flow)
{
struct mlx5_eswitch *esw = priv->mdev->priv.eswitch;
struct mlx5_fc *counter = NULL;
- counter = mlx5_flow_rule_counter(flow->rule);
-
- mlx5_del_flow_rules(flow->rule);
+ if (!IS_ERR(flow->rule)) {
+ counter = mlx5_flow_rule_counter(flow->rule);
+ mlx5_del_flow_rules(flow->rule);
+ mlx5_fc_destroy(priv->mdev, counter);
+ }
if (esw && esw->mode == SRIOV_OFFLOADS) {
mlx5_eswitch_del_vlan_action(esw, flow->attr);
mlx5e_detach_encap(priv, flow);
}
- mlx5_fc_destroy(priv->mdev, counter);
-
if (!mlx5e_tc_num_filters(priv) && (priv->fs.tc.t)) {
mlx5_destroy_flow_table(priv->fs.tc.t);
priv->fs.tc.t = NULL;
void *headers_v = MLX5_ADDR_OF(fte_match_param, spec->match_value,
outer_headers);
+ struct flow_dissector_key_control *enc_control =
+ skb_flow_dissector_target(f->dissector,
+ FLOW_DISSECTOR_KEY_ENC_CONTROL,
+ f->key);
+
if (dissector_uses_key(f->dissector, FLOW_DISSECTOR_KEY_ENC_PORTS)) {
struct flow_dissector_key_ports *key =
skb_flow_dissector_target(f->dissector,
/* Full udp dst port must be given */
if (memchr_inv(&mask->dst, 0xff, sizeof(mask->dst)))
- return -EOPNOTSUPP;
-
- /* udp src port isn't supported */
- if (memchr_inv(&mask->src, 0, sizeof(mask->src)))
- return -EOPNOTSUPP;
+ goto vxlan_match_offload_err;
if (mlx5e_vxlan_lookup_port(priv, be16_to_cpu(key->dst)) &&
MLX5_CAP_ESW(priv->mdev, vxlan_encap_decap))
parse_vxlan_attr(spec, f);
- else
+ else {
+ netdev_warn(priv->netdev,
+ "%d isn't an offloaded vxlan udp dport\n", be16_to_cpu(key->dst));
return -EOPNOTSUPP;
+ }
MLX5_SET(fte_match_set_lyr_2_4, headers_c,
udp_dport, ntohs(mask->dst));
MLX5_SET(fte_match_set_lyr_2_4, headers_v,
udp_dport, ntohs(key->dst));
+ MLX5_SET(fte_match_set_lyr_2_4, headers_c,
+ udp_sport, ntohs(mask->src));
+ MLX5_SET(fte_match_set_lyr_2_4, headers_v,
+ udp_sport, ntohs(key->src));
} else { /* udp dst port must be given */
- return -EOPNOTSUPP;
+vxlan_match_offload_err:
+ netdev_warn(priv->netdev,
+ "IP tunnel decap offload supported only for vxlan, must set UDP dport\n");
+ return -EOPNOTSUPP;
}
- if (dissector_uses_key(f->dissector, FLOW_DISSECTOR_KEY_ENC_IPV4_ADDRS)) {
+ if (enc_control->addr_type == FLOW_DISSECTOR_KEY_IPV4_ADDRS) {
struct flow_dissector_key_ipv4_addrs *key =
skb_flow_dissector_target(f->dissector,
FLOW_DISSECTOR_KEY_ENC_IPV4_ADDRS,
MLX5_SET(fte_match_set_lyr_2_4, headers_v,
dst_ipv4_dst_ipv6.ipv4_layout.ipv4,
ntohl(key->dst));
- }
- MLX5_SET_TO_ONES(fte_match_set_lyr_2_4, headers_c, ethertype);
- MLX5_SET(fte_match_set_lyr_2_4, headers_v, ethertype, ETH_P_IP);
+ MLX5_SET_TO_ONES(fte_match_set_lyr_2_4, headers_c, ethertype);
+ MLX5_SET(fte_match_set_lyr_2_4, headers_v, ethertype, ETH_P_IP);
+ }
/* Enforce DMAC when offloading incoming tunneled flows.
* Flow counters require a match on the DMAC.
if (parse_tunnel_attr(priv, spec, f))
return -EOPNOTSUPP;
break;
+ case FLOW_DISSECTOR_KEY_IPV6_ADDRS:
+ netdev_warn(priv->netdev,
+ "IPv6 tunnel decap offload isn't supported\n");
default:
return -EOPNOTSUPP;
}
MLX5_SET(fte_match_set_lyr_2_4, headers_c, frag, 1);
MLX5_SET(fte_match_set_lyr_2_4, headers_v, frag,
key->flags & FLOW_DIS_IS_FRAGMENT);
+
+ /* the HW doesn't need L3 inline to match on frag=no */
+ if (key->flags & FLOW_DIS_IS_FRAGMENT)
+ *min_inline = MLX5_INLINE_MODE_IP;
}
}
int ttl;
#if IS_ENABLED(CONFIG_INET)
+ int ret;
+
rt = ip_route_output_key(dev_net(mirred_dev), fl4);
- if (IS_ERR(rt)) {
- pr_warn("%s: no route to %pI4\n", __func__, &fl4->daddr);
- return -EOPNOTSUPP;
- }
+ ret = PTR_ERR_OR_ZERO(rt);
+ if (ret)
+ return ret;
#else
return -EOPNOTSUPP;
#endif
if (!switchdev_port_same_parent_id(priv->netdev, rt->dst.dev)) {
- pr_warn("%s: Can't offload the flow, netdevices aren't on the same HW e-switch\n",
- __func__);
+ pr_warn("%s: can't offload, devices not on same HW e-switch\n", __func__);
ip_rt_put(rt);
return -EOPNOTSUPP;
}
struct net_device **out_dev)
{
int max_encap_size = MLX5_CAP_ESW(priv->mdev, max_encap_header_size);
+ struct neighbour *n = NULL;
struct flowi4 fl4 = {};
- struct neighbour *n;
char *encap_header;
int encap_size;
__be32 saddr;
e->out_dev = *out_dev;
if (!(n->nud_state & NUD_VALID)) {
- err = -ENOTSUPP;
+ pr_warn("%s: can't offload, neighbour to %pI4 invalid\n", __func__, &fl4.daddr);
+ err = -EOPNOTSUPP;
goto out;
}
err = mlx5_encap_alloc(priv->mdev, e->tunnel_type,
encap_size, encap_header, &e->encap_id);
out:
+ if (err && n)
+ neigh_release(n);
kfree(encap_header);
return err;
}
int tunnel_type;
int err;
- /* udp dst port must be given */
+ /* udp dst port must be set */
if (!memchr_inv(&key->tp_dst, 0, sizeof(key->tp_dst)))
+ goto vxlan_encap_offload_err;
+
+ /* setting udp src port isn't supported */
+ if (memchr_inv(&key->tp_src, 0, sizeof(key->tp_src))) {
+vxlan_encap_offload_err:
+ netdev_warn(priv->netdev,
+ "must set udp dst port and not set udp src port\n");
return -EOPNOTSUPP;
+ }
if (mlx5e_vxlan_lookup_port(priv, be16_to_cpu(key->tp_dst)) &&
MLX5_CAP_ESW(priv->mdev, vxlan_encap_decap)) {
info.tun_id = tunnel_id_to_key32(key->tun_id);
tunnel_type = MLX5_HEADER_TYPE_VXLAN;
} else {
+ netdev_warn(priv->netdev,
+ "%d isn't an offloaded vxlan udp dport\n", be16_to_cpu(key->tp_dst));
return -EOPNOTSUPP;
}
case AF_INET:
info.daddr = key->u.ipv4.dst;
break;
+ case AF_INET6:
+ netdev_warn(priv->netdev,
+ "IPv6 tunnel encap offload isn't supported\n");
default:
return -EOPNOTSUPP;
}
if (IS_ERR(flow->rule)) {
err = PTR_ERR(flow->rule);
- goto err_free;
+ goto err_del_rule;
}
err = rhashtable_insert_fast(&tc->ht, &flow->node,
goto out;
err_del_rule:
- mlx5_del_flow_rules(flow->rule);
+ mlx5e_tc_del_flow(priv, flow);
err_free:
kfree(flow);
{
int err = 0;
- mlx5_drain_health_wq(dev);
+ if (cleanup)
+ mlx5_drain_health_wq(dev);
mutex_lock(&dev->intf_state_mutex);
if (test_bit(MLX5_INTERFACE_STATE_DOWN, &dev->intf_state)) {
mlx5_enter_error_state(dev);
mlx5_unload_one(dev, priv, false);
- /* In case of kernel call save the pci state */
+ /* In case of kernel call save the pci state and drain the health wq */
if (state) {
pci_save_state(pdev);
+ mlx5_drain_health_wq(dev);
mlx5_pci_disable_device(dev);
}
/* pci_eqe_cmd_token
* Command completion event - token
*/
-MLXSW_ITEM32(pci, eqe, cmd_token, 0x08, 16, 16);
+MLXSW_ITEM32(pci, eqe, cmd_token, 0x00, 16, 16);
/* pci_eqe_cmd_status
* Command completion event - status
*/
-MLXSW_ITEM32(pci, eqe, cmd_status, 0x08, 0, 8);
+MLXSW_ITEM32(pci, eqe, cmd_status, 0x00, 0, 8);
/* pci_eqe_cmd_out_param_h
* Command completion event - output parameter - higher part
*/
-MLXSW_ITEM32(pci, eqe, cmd_out_param_h, 0x0C, 0, 32);
+MLXSW_ITEM32(pci, eqe, cmd_out_param_h, 0x04, 0, 32);
/* pci_eqe_cmd_out_param_l
* Command completion event - output parameter - lower part
*/
-MLXSW_ITEM32(pci, eqe, cmd_out_param_l, 0x10, 0, 32);
+MLXSW_ITEM32(pci, eqe, cmd_out_param_l, 0x08, 0, 32);
#endif
dev_kfree_skb_any(skb_orig);
return NETDEV_TX_OK;
}
+ dev_consume_skb_any(skb_orig);
}
if (eth_skb_pad(skb)) {
dev_kfree_skb_any(skb_orig);
return NETDEV_TX_OK;
}
+ dev_consume_skb_any(skb_orig);
}
mlxsw_sx_txhdr_construct(skb, &tx_info);
/* TX header is consumed by HW on the way so we shouldn't count its
else
adpt->phydev = mdiobus_get_phy(mii_bus, phy_addr);
+ /* of_phy_find_device() claims a reference to the phydev,
+ * so we do that here manually as well. When the driver
+ * later unloads, it can unilaterally drop the reference
+ * without worrying about ACPI vs DT.
+ */
+ if (adpt->phydev)
+ get_device(&adpt->phydev->mdio.dev);
} else {
struct device_node *phy_np;
err_undo_napi:
netif_napi_del(&adpt->rx_q.napi);
err_undo_mdiobus:
- if (!has_acpi_companion(&pdev->dev))
- put_device(&adpt->phydev->mdio.dev);
+ put_device(&adpt->phydev->mdio.dev);
mdiobus_unregister(adpt->mii_bus);
err_undo_clocks:
emac_clks_teardown(adpt);
emac_clks_teardown(adpt);
- if (!has_acpi_companion(&pdev->dev))
- put_device(&adpt->phydev->mdio.dev);
+ put_device(&adpt->phydev->mdio.dev);
mdiobus_unregister(adpt->mii_bus);
free_netdev(netdev);
enum rtl_rx_desc_bit {
/* Rx private */
PID1 = (1 << 18), /* Protocol ID bit 1/2 */
- PID0 = (1 << 17), /* Protocol ID bit 2/2 */
+ PID0 = (1 << 17), /* Protocol ID bit 0/2 */
#define RxProtoUDP (PID1)
#define RxProtoTCP (PID0)
/* Receive error message handling */
priv->rx_over_errors = priv->stats[RAVB_BE].rx_over_errors;
priv->rx_over_errors += priv->stats[RAVB_NC].rx_over_errors;
- if (priv->rx_over_errors != ndev->stats.rx_over_errors) {
+ if (priv->rx_over_errors != ndev->stats.rx_over_errors)
ndev->stats.rx_over_errors = priv->rx_over_errors;
- netif_err(priv, rx_err, ndev, "Receive Descriptor Empty\n");
- }
- if (priv->rx_fifo_errors != ndev->stats.rx_fifo_errors) {
+ if (priv->rx_fifo_errors != ndev->stats.rx_fifo_errors)
ndev->stats.rx_fifo_errors = priv->rx_fifo_errors;
- netif_err(priv, rx_err, ndev, "Receive FIFO Overflow\n");
- }
out:
return budget - quota;
}
buffer = PTR_ALIGN(priv->tx_align[q], DPTR_ALIGN) +
entry / NUM_TX_DESC * DPTR_ALIGN;
len = PTR_ALIGN(skb->data, DPTR_ALIGN) - skb->data;
+ /* Zero length DMA descriptors are problematic as they seem to
+ * terminate DMA transfers. Avoid them by simply using a length of
+ * DPTR_ALIGN (4) when skb data is aligned to DPTR_ALIGN.
+ *
+ * As skb is guaranteed to have at least ETH_ZLEN (60) bytes of
+ * data by the call to skb_put_padto() above this is safe with
+ * respect to both the length of the first DMA descriptor (len)
+ * overflowing the available data and the length of the second DMA
+ * descriptor (skb->len - len) being negative.
+ */
+ if (len == 0)
+ len = DPTR_ALIGN;
+
memcpy(buffer, skb->data, len);
dma_addr = dma_map_single(ndev->dev.parent, buffer, len, DMA_TO_DEVICE);
if (dma_mapping_error(ndev->dev.parent, dma_addr))
.rpadir_value = 2 << 16,
.no_trimd = 1,
.no_ade = 1,
+ .hw_crc = 1,
.tsu = 1,
.select_mii = 1,
.shift_rd0 = 1,
.ecsr_value = ECSR_ICD | ECSR_MPD,
.ecsipr_value = ECSIPR_LCHNGIP | ECSIPR_ICDIP | ECSIPR_MPDIP,
- .eesipr_value = DMAC_M_RFRMER | DMAC_M_ECI | 0x003fffff,
+ .eesipr_value = DMAC_M_RFRMER | DMAC_M_ECI | 0x003f07ff,
.tx_check = EESR_TC1 | EESR_FTC,
.eesr_err_check = EESR_TWB1 | EESR_TWB | EESR_TABT | EESR_RABT |
.ecsr_value = ECSR_ICD | ECSR_MPD,
.ecsipr_value = ECSIPR_LCHNGIP | ECSIPR_ICDIP | ECSIPR_MPDIP,
- .eesipr_value = DMAC_M_RFRMER | DMAC_M_ECI | 0x003fffff,
+ .eesipr_value = DMAC_M_RFRMER | DMAC_M_ECI | 0x003f07ff,
.tx_check = EESR_TC1 | EESR_FTC,
.eesr_err_check = EESR_TWB1 | EESR_TWB | EESR_TABT | EESR_RABT |
ndev->max_mtu = JUMBO_LEN;
else
ndev->max_mtu = SKB_MAX_HEAD(NET_SKB_PAD + NET_IP_ALIGN);
- if (priv->plat->maxmtu < ndev->max_mtu)
+ /* Will not overwrite ndev->max_mtu if plat->maxmtu > ndev->max_mtu
+ * as well as plat->maxmtu < ndev->min_mtu which is a invalid range.
+ */
+ if ((priv->plat->maxmtu < ndev->max_mtu) &&
+ (priv->plat->maxmtu >= ndev->min_mtu))
ndev->max_mtu = priv->plat->maxmtu;
+ else if (priv->plat->maxmtu < ndev->min_mtu)
+ dev_warn(priv->device,
+ "%s: warning: maxmtu having invalid value (%d)\n",
+ __func__, priv->plat->maxmtu);
if (flow_ctrl)
priv->flow_ctrl = FLOW_AUTO; /* RX/TX pause on */
*/
if ((priv->synopsys_id >= DWMAC_CORE_3_50) && (!priv->plat->riwt_off)) {
priv->use_riwt = 1;
- netdev_info(priv->dev, "Enable RX Mitigation via HW Watchdog Timer\n");
+ dev_info(priv->device,
+ "Enable RX Mitigation via HW Watchdog Timer\n");
}
netif_napi_add(ndev, &priv->napi, stmmac_poll, 64);
/* MDIO bus Registration */
ret = stmmac_mdio_register(ndev);
if (ret < 0) {
- netdev_err(priv->dev,
- "%s: MDIO bus (id: %d) registration failed",
- __func__, priv->plat->bus_id);
+ dev_err(priv->device,
+ "%s: MDIO bus (id: %d) registration failed",
+ __func__, priv->plat->bus_id);
goto error_mdio_register;
}
}
ret = register_netdev(ndev);
if (ret) {
- netdev_err(priv->dev, "%s: ERROR %i registering the device\n",
- __func__, ret);
+ dev_err(priv->device, "%s: ERROR %i registering the device\n",
+ __func__, ret);
goto error_netdev_register;
}
/* Set default value for unicast filter entries */
plat->unicast_filter_entries = 1;
+
+ /* Set the maxmtu to a default of JUMBO_LEN */
+ plat->maxmtu = JUMBO_LEN;
}
static int quark_default_data(struct plat_stmmacenet_data *plat,
/* Set default value for unicast filter entries */
plat->unicast_filter_entries = 1;
+ /* Set the maxmtu to a default of JUMBO_LEN */
+ plat->maxmtu = JUMBO_LEN;
+
return 0;
}
goto fail_alloc;
}
-#warning FIXME: unhardcode gpio&reset bits
+ /* FIXME: unhardcode gpio&reset bits */
ar7_gpio_disable(26);
ar7_gpio_disable(27);
ar7_device_reset(AR7_RESET_BIT_CPMAC_LO);
* policy filters on the host). Deliver these via the VF
* interface in the guest.
*/
+ rcu_read_lock();
vf_netdev = rcu_dereference(net_device_ctx->vf_netdev);
if (vf_netdev && (vf_netdev->flags & IFF_UP))
net = vf_netdev;
skb = netvsc_alloc_recv_skb(net, packet, csum_info, *data, vlan_tci);
if (unlikely(!skb)) {
++net->stats.rx_dropped;
+ rcu_read_unlock();
return NVSP_STAT_FAIL;
}
* TODO - use NAPI?
*/
netif_rx(skb);
+ rcu_read_unlock();
return 0;
}
/* Reset */
if (gpio_is_valid(rstn)) {
udelay(1);
- gpio_set_value(rstn, 0);
+ gpio_set_value_cansleep(rstn, 0);
udelay(1);
- gpio_set_value(rstn, 1);
+ gpio_set_value_cansleep(rstn, 1);
usleep_range(120, 240);
}
{
struct usb_device *usb_dev = atusb->usb_dev;
int ret;
+ uint8_t *buffer;
uint8_t value;
+ buffer = kmalloc(1, GFP_KERNEL);
+ if (!buffer)
+ return -ENOMEM;
+
dev_dbg(&usb_dev->dev, "atusb: reg = 0x%x\n", reg);
ret = atusb_control_msg(atusb, usb_rcvctrlpipe(usb_dev, 0),
ATUSB_REG_READ, ATUSB_REQ_FROM_DEV,
- 0, reg, &value, 1, 1000);
- return ret >= 0 ? value : ret;
+ 0, reg, buffer, 1, 1000);
+
+ if (ret >= 0) {
+ value = buffer[0];
+ kfree(buffer);
+ return value;
+ } else {
+ kfree(buffer);
+ return ret;
+ }
}
static int atusb_write_subreg(struct atusb *atusb, uint8_t reg, uint8_t mask,
atusb_set_frame_retries(struct ieee802154_hw *hw, s8 retries)
{
struct atusb *atusb = hw->priv;
- struct device *dev = &atusb->usb_dev->dev;
-
- if (atusb->fw_ver_maj == 0 && atusb->fw_ver_min < 3) {
- dev_info(dev, "Automatic frame retransmission is only available from "
- "firmware version 0.3. Please update if you want this feature.");
- return -EINVAL;
- }
return atusb_write_subreg(atusb, SR_MAX_FRAME_RETRIES, retries);
}
static int atusb_get_and_show_revision(struct atusb *atusb)
{
struct usb_device *usb_dev = atusb->usb_dev;
- unsigned char buffer[3];
+ unsigned char *buffer;
int ret;
+ buffer = kmalloc(3, GFP_KERNEL);
+ if (!buffer)
+ return -ENOMEM;
+
/* Get a couple of the ATMega Firmware values */
ret = atusb_control_msg(atusb, usb_rcvctrlpipe(usb_dev, 0),
ATUSB_ID, ATUSB_REQ_FROM_DEV, 0, 0,
dev_info(&usb_dev->dev, "Please update to version 0.2 or newer");
}
+ kfree(buffer);
return ret;
}
static int atusb_get_and_show_build(struct atusb *atusb)
{
struct usb_device *usb_dev = atusb->usb_dev;
- char build[ATUSB_BUILD_SIZE + 1];
+ char *build;
int ret;
+ build = kmalloc(ATUSB_BUILD_SIZE + 1, GFP_KERNEL);
+ if (!build)
+ return -ENOMEM;
+
ret = atusb_control_msg(atusb, usb_rcvctrlpipe(usb_dev, 0),
ATUSB_BUILD, ATUSB_REQ_FROM_DEV, 0, 0,
build, ATUSB_BUILD_SIZE, 1000);
dev_info(&usb_dev->dev, "Firmware: build %s\n", build);
}
+ kfree(build);
return ret;
}
static int atusb_set_extended_addr(struct atusb *atusb)
{
struct usb_device *usb_dev = atusb->usb_dev;
- unsigned char buffer[IEEE802154_EXTENDED_ADDR_LEN];
+ unsigned char *buffer;
__le64 extended_addr;
u64 addr;
int ret;
return 0;
}
+ buffer = kmalloc(IEEE802154_EXTENDED_ADDR_LEN, GFP_KERNEL);
+ if (!buffer)
+ return -ENOMEM;
+
/* Firmware is new enough so we fetch the address from EEPROM */
ret = atusb_control_msg(atusb, usb_rcvctrlpipe(usb_dev, 0),
ATUSB_EUI64_READ, ATUSB_REQ_FROM_DEV, 0, 0,
buffer, IEEE802154_EXTENDED_ADDR_LEN, 1000);
- if (ret < 0)
- dev_err(&usb_dev->dev, "failed to fetch extended address\n");
+ if (ret < 0) {
+ dev_err(&usb_dev->dev, "failed to fetch extended address, random address set\n");
+ ieee802154_random_extended_addr(&atusb->hw->phy->perm_extended_addr);
+ kfree(buffer);
+ return ret;
+ }
memcpy(&extended_addr, buffer, IEEE802154_EXTENDED_ADDR_LEN);
/* Check if read address is not empty and the unicast bit is set correctly */
&addr);
}
+ kfree(buffer);
return ret;
}
hw->parent = &usb_dev->dev;
hw->flags = IEEE802154_HW_TX_OMIT_CKSUM | IEEE802154_HW_AFILT |
- IEEE802154_HW_PROMISCUOUS | IEEE802154_HW_CSMA_PARAMS |
- IEEE802154_HW_FRAME_RETRIES;
+ IEEE802154_HW_PROMISCUOUS | IEEE802154_HW_CSMA_PARAMS;
hw->phy->flags = WPAN_PHY_FLAG_TXPOWER | WPAN_PHY_FLAG_CCA_ED_LEVEL |
WPAN_PHY_FLAG_CCA_MODE;
atusb_get_and_show_build(atusb);
atusb_set_extended_addr(atusb);
+ if (atusb->fw_ver_maj >= 0 && atusb->fw_ver_min >= 3)
+ hw->flags |= IEEE802154_HW_FRAME_RETRIES;
+
ret = atusb_get_and_clear_error(atusb);
if (ret) {
dev_err(&atusb->usb_dev->dev,
config MESON_GXL_PHY
tristate "Amlogic Meson GXL Internal PHY"
+ depends on ARCH_MESON || COMPILE_TEST
---help---
Currently has a driver for the Amlogic Meson GXL Internal PHY
#define MII_DP83867_MICR 0x12
#define MII_DP83867_ISR 0x13
#define DP83867_CTRL 0x1f
+#define DP83867_CFG3 0x1e
/* Extended Registers */
#define DP83867_RGMIICTL 0x0032
micr_status |=
(MII_DP83867_MICR_AN_ERR_INT_EN |
MII_DP83867_MICR_SPEED_CHNG_INT_EN |
+ MII_DP83867_MICR_AUTONEG_COMP_INT_EN |
+ MII_DP83867_MICR_LINK_STS_CHNG_INT_EN |
MII_DP83867_MICR_DUP_MODE_CHNG_INT_EN |
MII_DP83867_MICR_SLEEP_MODE_CHNG_INT_EN);
ret = of_property_read_u32(of_node, "ti,rx-internal-delay",
&dp83867->rx_id_delay);
- if (ret)
+ if (ret &&
+ (phydev->interface == PHY_INTERFACE_MODE_RGMII_ID ||
+ phydev->interface == PHY_INTERFACE_MODE_RGMII_RXID))
return ret;
ret = of_property_read_u32(of_node, "ti,tx-internal-delay",
&dp83867->tx_id_delay);
- if (ret)
+ if (ret &&
+ (phydev->interface == PHY_INTERFACE_MODE_RGMII_ID ||
+ phydev->interface == PHY_INTERFACE_MODE_RGMII_TXID))
return ret;
return of_property_read_u32(of_node, "ti,fifo-depth",
}
}
+ /* Enable Interrupt output INT_OE in CFG3 register */
+ if (phy_interrupt_is_valid(phydev)) {
+ val = phy_read(phydev, DP83867_CFG3);
+ val |= BIT(7);
+ phy_write(phydev, DP83867_CFG3, val);
+ }
+
return 0;
}
int err;
/* Check the fiber mode first */
- if (phydev->supported & SUPPORTED_FIBRE) {
+ if (phydev->supported & SUPPORTED_FIBRE &&
+ phydev->interface != PHY_INTERFACE_MODE_SGMII) {
err = phy_write(phydev, MII_MARVELL_PHY_PAGE, MII_M1111_FIBER);
if (err < 0)
goto error;
if (old_link != phydev->link)
phydev->state = PHY_CHANGELINK;
}
+ /*
+ * Failsafe: check that nobody set phydev->link=0 between two
+ * poll cycles, otherwise we won't leave RUNNING state as long
+ * as link remains down.
+ */
+ if (!phydev->link && phydev->state == PHY_RUNNING) {
+ phydev->state = PHY_CHANGELINK;
+ phydev_err(phydev, "no link in PHY_RUNNING\n");
+ }
break;
case PHY_CHANGELINK:
err = phy_read_status(phydev);
u8 checksum = CHECKSUM_NONE;
u32 opts2, opts3;
- if (tp->version == RTL_VER_01 || tp->version == RTL_VER_02)
+ if (!(tp->netdev->features & NETIF_F_RXCSUM))
goto return_result;
opts2 = le32_to_cpu(rx_desc->opts2);
return false;
}
-static int rtl8152_suspend(struct usb_interface *intf, pm_message_t message)
+static int rtl8152_rumtime_suspend(struct r8152 *tp)
{
- struct r8152 *tp = usb_get_intfdata(intf);
struct net_device *netdev = tp->netdev;
int ret = 0;
- mutex_lock(&tp->control);
+ if (netif_running(netdev) && test_bit(WORK_ENABLE, &tp->flags)) {
+ u32 rcr = 0;
- if (PMSG_IS_AUTO(message)) {
- if (netif_running(netdev) && delay_autosuspend(tp)) {
+ if (delay_autosuspend(tp)) {
ret = -EBUSY;
goto out1;
}
- set_bit(SELECTIVE_SUSPEND, &tp->flags);
- } else {
- netif_device_detach(netdev);
+ if (netif_carrier_ok(netdev)) {
+ u32 ocp_data;
+
+ rcr = ocp_read_dword(tp, MCU_TYPE_PLA, PLA_RCR);
+ ocp_data = rcr & ~RCR_ACPT_ALL;
+ ocp_write_dword(tp, MCU_TYPE_PLA, PLA_RCR, ocp_data);
+ rxdy_gated_en(tp, true);
+ ocp_data = ocp_read_byte(tp, MCU_TYPE_PLA,
+ PLA_OOB_CTRL);
+ if (!(ocp_data & RXFIFO_EMPTY)) {
+ rxdy_gated_en(tp, false);
+ ocp_write_dword(tp, MCU_TYPE_PLA, PLA_RCR, rcr);
+ ret = -EBUSY;
+ goto out1;
+ }
+ }
+
+ clear_bit(WORK_ENABLE, &tp->flags);
+ usb_kill_urb(tp->intr_urb);
+
+ tp->rtl_ops.autosuspend_en(tp, true);
+
+ if (netif_carrier_ok(netdev)) {
+ napi_disable(&tp->napi);
+ rtl_stop_rx(tp);
+ rxdy_gated_en(tp, false);
+ ocp_write_dword(tp, MCU_TYPE_PLA, PLA_RCR, rcr);
+ napi_enable(&tp->napi);
+ }
}
+ set_bit(SELECTIVE_SUSPEND, &tp->flags);
+
+out1:
+ return ret;
+}
+
+static int rtl8152_system_suspend(struct r8152 *tp)
+{
+ struct net_device *netdev = tp->netdev;
+ int ret = 0;
+
+ netif_device_detach(netdev);
+
if (netif_running(netdev) && test_bit(WORK_ENABLE, &tp->flags)) {
clear_bit(WORK_ENABLE, &tp->flags);
usb_kill_urb(tp->intr_urb);
napi_disable(&tp->napi);
- if (test_bit(SELECTIVE_SUSPEND, &tp->flags)) {
- rtl_stop_rx(tp);
- tp->rtl_ops.autosuspend_en(tp, true);
- } else {
- cancel_delayed_work_sync(&tp->schedule);
- tp->rtl_ops.down(tp);
- }
+ cancel_delayed_work_sync(&tp->schedule);
+ tp->rtl_ops.down(tp);
napi_enable(&tp->napi);
}
-out1:
+
+ return ret;
+}
+
+static int rtl8152_suspend(struct usb_interface *intf, pm_message_t message)
+{
+ struct r8152 *tp = usb_get_intfdata(intf);
+ int ret;
+
+ mutex_lock(&tp->control);
+
+ if (PMSG_IS_AUTO(message))
+ ret = rtl8152_rumtime_suspend(tp);
+ else
+ ret = rtl8152_system_suspend(tp);
+
mutex_unlock(&tp->control);
return ret;
NETIF_F_HIGHDMA | NETIF_F_FRAGLIST |
NETIF_F_IPV6_CSUM | NETIF_F_TSO6;
+ if (tp->version == RTL_VER_01) {
+ netdev->features &= ~NETIF_F_RXCSUM;
+ netdev->hw_features &= ~NETIF_F_RXCSUM;
+ }
+
netdev->ethtool_ops = &ops;
netif_set_gso_max_size(netdev, RTL_LIMITED_TSO_SIZE);
.flowi4_iif = LOOPBACK_IFINDEX,
.flowi4_tos = RT_TOS(ip4h->tos),
.flowi4_flags = FLOWI_FLAG_ANYSRC | FLOWI_FLAG_SKIP_NH_OIF,
+ .flowi4_proto = ip4h->protocol,
.daddr = ip4h->daddr,
+ .saddr = ip4h->saddr,
};
struct net *net = dev_net(vrf_dev);
struct rtable *rt;
return -EINVAL;
vrf->tb_id = nla_get_u32(data[IFLA_VRF_TABLE]);
+ if (vrf->tb_id == RT_TABLE_UNSPEC)
+ return -EINVAL;
dev->priv_flags |= IFF_L3MDEV_MASTER;
static struct rtable *vxlan_get_route(struct vxlan_dev *vxlan, struct net_device *dev,
struct vxlan_sock *sock4,
struct sk_buff *skb, int oif, u8 tos,
- __be32 daddr, __be32 *saddr,
+ __be32 daddr, __be32 *saddr, __be16 dport, __be16 sport,
struct dst_cache *dst_cache,
const struct ip_tunnel_info *info)
{
fl4.flowi4_proto = IPPROTO_UDP;
fl4.daddr = daddr;
fl4.saddr = *saddr;
+ fl4.fl4_dport = dport;
+ fl4.fl4_sport = sport;
rt = ip_route_output_key(vxlan->net, &fl4);
if (likely(!IS_ERR(rt))) {
__be32 label,
const struct in6_addr *daddr,
struct in6_addr *saddr,
+ __be16 dport, __be16 sport,
struct dst_cache *dst_cache,
const struct ip_tunnel_info *info)
{
fl6.flowlabel = ip6_make_flowinfo(RT_TOS(tos), label);
fl6.flowi6_mark = skb->mark;
fl6.flowi6_proto = IPPROTO_UDP;
+ fl6.fl6_dport = dport;
+ fl6.fl6_sport = sport;
err = ipv6_stub->ipv6_dst_lookup(vxlan->net,
sock6->sock->sk,
rdst ? rdst->remote_ifindex : 0, tos,
dst->sin.sin_addr.s_addr,
&src->sin.sin_addr.s_addr,
+ dst_port, src_port,
dst_cache, info);
if (IS_ERR(rt)) {
err = PTR_ERR(rt);
rdst ? rdst->remote_ifindex : 0, tos,
label, &dst->sin6.sin6_addr,
&src->sin6.sin6_addr,
+ dst_port, src_port,
dst_cache, info);
if (IS_ERR(ndst)) {
err = PTR_ERR(ndst);
rt = vxlan_get_route(vxlan, dev, sock4, skb, 0, info->key.tos,
info->key.u.ipv4.dst,
- &info->key.u.ipv4.src, NULL, info);
+ &info->key.u.ipv4.src, dport, sport, NULL, info);
if (IS_ERR(rt))
return PTR_ERR(rt);
ip_rt_put(rt);
ndst = vxlan6_get_route(vxlan, dev, sock6, skb, 0, info->key.tos,
info->key.label, &info->key.u.ipv6.dst,
- &info->key.u.ipv6.src, NULL, info);
+ &info->key.u.ipv6.src, dport, sport, NULL, info);
if (IS_ERR(ndst))
return PTR_ERR(ndst);
dst_release(ndst);
/********************************************************************/
int orinoco_mic_init(struct orinoco_private *priv)
{
- priv->tx_tfm_mic = crypto_alloc_ahash("michael_mic", 0,
+ priv->tx_tfm_mic = crypto_alloc_shash("michael_mic", 0,
CRYPTO_ALG_ASYNC);
if (IS_ERR(priv->tx_tfm_mic)) {
printk(KERN_DEBUG "orinoco_mic_init: could not allocate "
return -ENOMEM;
}
- priv->rx_tfm_mic = crypto_alloc_ahash("michael_mic", 0,
+ priv->rx_tfm_mic = crypto_alloc_shash("michael_mic", 0,
CRYPTO_ALG_ASYNC);
if (IS_ERR(priv->rx_tfm_mic)) {
printk(KERN_DEBUG "orinoco_mic_init: could not allocate "
void orinoco_mic_free(struct orinoco_private *priv)
{
if (priv->tx_tfm_mic)
- crypto_free_ahash(priv->tx_tfm_mic);
+ crypto_free_shash(priv->tx_tfm_mic);
if (priv->rx_tfm_mic)
- crypto_free_ahash(priv->rx_tfm_mic);
+ crypto_free_shash(priv->rx_tfm_mic);
}
-int orinoco_mic(struct crypto_ahash *tfm_michael, u8 *key,
+int orinoco_mic(struct crypto_shash *tfm_michael, u8 *key,
u8 *da, u8 *sa, u8 priority,
u8 *data, size_t data_len, u8 *mic)
{
- AHASH_REQUEST_ON_STACK(req, tfm_michael);
- struct scatterlist sg[2];
+ SHASH_DESC_ON_STACK(desc, tfm_michael);
u8 hdr[ETH_HLEN + 2]; /* size of header + padding */
int err;
hdr[ETH_ALEN * 2 + 2] = 0;
hdr[ETH_ALEN * 2 + 3] = 0;
- /* Use scatter gather to MIC header and data in one go */
- sg_init_table(sg, 2);
- sg_set_buf(&sg[0], hdr, sizeof(hdr));
- sg_set_buf(&sg[1], data, data_len);
+ desc->tfm = tfm_michael;
+ desc->flags = 0;
- if (crypto_ahash_setkey(tfm_michael, key, MIC_KEYLEN))
- return -1;
+ err = crypto_shash_setkey(tfm_michael, key, MIC_KEYLEN);
+ if (err)
+ return err;
+
+ err = crypto_shash_init(desc);
+ if (err)
+ return err;
+
+ err = crypto_shash_update(desc, hdr, sizeof(hdr));
+ if (err)
+ return err;
+
+ err = crypto_shash_update(desc, data, data_len);
+ if (err)
+ return err;
+
+ err = crypto_shash_final(desc, mic);
+ shash_desc_zero(desc);
- ahash_request_set_tfm(req, tfm_michael);
- ahash_request_set_callback(req, 0, NULL, NULL);
- ahash_request_set_crypt(req, sg, mic, data_len + sizeof(hdr));
- err = crypto_ahash_digest(req);
- ahash_request_zero(req);
return err;
}
#define _ORINOCO_MIC_H_
#include <linux/types.h>
+#include <crypto/hash.h>
#define MICHAEL_MIC_LEN 8
int orinoco_mic_init(struct orinoco_private *priv);
void orinoco_mic_free(struct orinoco_private *priv);
-int orinoco_mic(struct crypto_ahash *tfm_michael, u8 *key,
+int orinoco_mic(struct crypto_shash *tfm_michael, u8 *key,
u8 *da, u8 *sa, u8 priority,
u8 *data, size_t data_len, u8 *mic);
u8 *wpa_ie;
int wpa_ie_len;
- struct crypto_ahash *rx_tfm_mic;
- struct crypto_ahash *tx_tfm_mic;
+ struct crypto_shash *rx_tfm_mic;
+ struct crypto_shash *tx_tfm_mic;
unsigned int wpa_enabled:1;
unsigned int tkip_cm_active:1;
return -ENOMEM;
}
rtlpriv = hw->priv;
+ rtlpriv->hw = hw;
rtlpriv->usb_data = kzalloc(RTL_USB_MAX_RX_COUNT * sizeof(u32),
GFP_KERNEL);
if (!rtlpriv->usb_data)
{
resource_size_t allocated = 0, available = 0;
struct nd_region *nd_region = to_nd_region(dev->parent);
+ struct nd_namespace_common *ndns = to_ndns(dev);
struct nd_mapping *nd_mapping;
struct nvdimm_drvdata *ndd;
struct nd_label_id label_id;
u8 *uuid = NULL;
int rc, i;
- if (dev->driver || to_ndns(dev)->claim)
+ if (dev->driver || ndns->claim)
return -EBUSY;
if (is_namespace_pmem(dev)) {
nd_namespace_pmem_set_resource(nd_region, nspm,
val * nd_region->ndr_mappings);
- } else if (is_namespace_blk(dev)) {
- struct nd_namespace_blk *nsblk = to_nd_namespace_blk(dev);
-
- /*
- * Try to delete the namespace if we deleted all of its
- * allocation, this is not the seed device for the
- * region, and it is not actively claimed by a btt
- * instance.
- */
- if (val == 0 && nd_region->ns_seed != dev
- && !nsblk->common.claim)
- nd_device_unregister(dev, ND_ASYNC);
}
+ /*
+ * Try to delete the namespace if we deleted all of its
+ * allocation, this is not the seed device for the region, and
+ * it is not actively claimed by a btt instance.
+ */
+ if (val == 0 && nd_region->ns_seed != dev && !ndns->claim)
+ nd_device_unregister(dev, ND_ASYNC);
+
return rc;
}
rc = memcpy_from_pmem(mem + off, pmem_addr, len);
kunmap_atomic(mem);
- return rc;
+ if (rc)
+ return -EIO;
+ return 0;
}
static int pmem_do_bvec(struct pmem_device *pmem, struct page *page,
if (ret)
return ret;
- /* Checking for ctrl->tagset is a trick to avoid sleeping on module
- * load, since we only need the quirk on reset_controller. Notice
- * that the HGST device needs this delay only in firmware activation
- * procedure; unfortunately we have no (easy) way to verify this.
- */
- if ((ctrl->quirks & NVME_QUIRK_DELAY_BEFORE_CHK_RDY) && ctrl->tagset)
+ if (ctrl->quirks & NVME_QUIRK_DELAY_BEFORE_CHK_RDY)
msleep(NVME_QUIRK_DELAY_AMOUNT);
return nvme_wait_ready(ctrl, cap, false);
struct nvme_fc_fcp_op *op)
{
struct nvmefc_fcp_req *freq = &op->fcp_req;
- u32 map_len = nvme_map_len(rq);
enum dma_data_direction dir;
int ret;
freq->sg_cnt = 0;
- if (!map_len)
+ if (!blk_rq_payload_bytes(rq))
return 0;
freq->sg_table.sgl = freq->first_sgl;
if (ret)
return ret;
- data_len = nvme_map_len(rq);
+ data_len = blk_rq_payload_bytes(rq);
if (data_len)
io_dir = ((rq_data_dir(rq) == WRITE) ?
NVMEFC_FCP_WRITE : NVMEFC_FCP_READ);
return (sector >> (ns->lba_shift - 9));
}
-static inline unsigned nvme_map_len(struct request *rq)
-{
- if (req_op(rq) == REQ_OP_DISCARD)
- return sizeof(struct nvme_dsm_range);
- else
- return blk_rq_bytes(rq);
-}
-
static inline void nvme_cleanup_cmd(struct request *req)
{
if (req->rq_flags & RQF_SPECIAL_PAYLOAD) {
return (__le64 **)(iod->sg + blk_rq_nr_phys_segments(req));
}
-static int nvme_init_iod(struct request *rq, unsigned size,
- struct nvme_dev *dev)
+static int nvme_init_iod(struct request *rq, struct nvme_dev *dev)
{
struct nvme_iod *iod = blk_mq_rq_to_pdu(rq);
int nseg = blk_rq_nr_phys_segments(rq);
+ unsigned int size = blk_rq_payload_bytes(rq);
if (nseg > NVME_INT_PAGES || size > NVME_INT_BYTES(dev)) {
iod->sg = kmalloc(nvme_iod_alloc_size(dev, size, nseg), GFP_ATOMIC);
}
#endif
-static bool nvme_setup_prps(struct nvme_dev *dev, struct request *req,
- int total_len)
+static bool nvme_setup_prps(struct nvme_dev *dev, struct request *req)
{
struct nvme_iod *iod = blk_mq_rq_to_pdu(req);
struct dma_pool *pool;
- int length = total_len;
+ int length = blk_rq_payload_bytes(req);
struct scatterlist *sg = iod->sg;
int dma_len = sg_dma_len(sg);
u64 dma_addr = sg_dma_address(sg);
}
static int nvme_map_data(struct nvme_dev *dev, struct request *req,
- unsigned size, struct nvme_command *cmnd)
+ struct nvme_command *cmnd)
{
struct nvme_iod *iod = blk_mq_rq_to_pdu(req);
struct request_queue *q = req->q;
DMA_ATTR_NO_WARN))
goto out;
- if (!nvme_setup_prps(dev, req, size))
+ if (!nvme_setup_prps(dev, req))
goto out_unmap;
ret = BLK_MQ_RQ_QUEUE_ERROR;
struct nvme_dev *dev = nvmeq->dev;
struct request *req = bd->rq;
struct nvme_command cmnd;
- unsigned map_len;
int ret = BLK_MQ_RQ_QUEUE_OK;
/*
if (ret != BLK_MQ_RQ_QUEUE_OK)
return ret;
- map_len = nvme_map_len(req);
- ret = nvme_init_iod(req, map_len, dev);
+ ret = nvme_init_iod(req, dev);
if (ret != BLK_MQ_RQ_QUEUE_OK)
goto out_free_cmd;
if (blk_rq_nr_phys_segments(req))
- ret = nvme_map_data(dev, req, map_len, &cmnd);
+ ret = nvme_map_data(dev, req, &cmnd);
if (ret != BLK_MQ_RQ_QUEUE_OK)
goto out_cleanup_iod;
}
static int nvme_rdma_map_data(struct nvme_rdma_queue *queue,
- struct request *rq, unsigned int map_len,
- struct nvme_command *c)
+ struct request *rq, struct nvme_command *c)
{
struct nvme_rdma_request *req = blk_mq_rq_to_pdu(rq);
struct nvme_rdma_device *dev = queue->device;
}
if (count == 1) {
- if (rq_data_dir(rq) == WRITE &&
- map_len <= nvme_rdma_inline_data_size(queue) &&
- nvme_rdma_queue_idx(queue))
+ if (rq_data_dir(rq) == WRITE && nvme_rdma_queue_idx(queue) &&
+ blk_rq_payload_bytes(rq) <=
+ nvme_rdma_inline_data_size(queue))
return nvme_rdma_map_sg_inline(queue, req, c);
if (dev->pd->flags & IB_PD_UNSAFE_GLOBAL_RKEY)
struct request *rq)
{
if (unlikely(!test_bit(NVME_RDMA_Q_LIVE, &queue->flags))) {
- struct nvme_command *cmd = (struct nvme_command *)rq->cmd;
+ struct nvme_command *cmd = nvme_req(rq)->cmd;
if (rq->cmd_type != REQ_TYPE_DRV_PRIV ||
cmd->common.opcode != nvme_fabrics_command ||
struct nvme_command *c = sqe->data;
bool flush = false;
struct ib_device *dev;
- unsigned int map_len;
int ret;
WARN_ON_ONCE(rq->tag < 0);
blk_mq_start_request(rq);
- map_len = nvme_map_len(rq);
- ret = nvme_rdma_map_data(queue, rq, map_len, c);
+ ret = nvme_rdma_map_data(queue, rq, c);
if (ret < 0) {
dev_err(queue->ctrl->ctrl.device,
"Failed to map data (%d)\n", ret);
rc = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "pci/xgene:online",
xgene_msi_hwirq_alloc, NULL);
- if (rc)
+ if (rc < 0)
goto err_cpuhp;
pci_xgene_online = rc;
rc = cpuhp_setup_state(CPUHP_PCI_XGENE_DEAD, "pci/xgene:dead", NULL,
{
u32 val;
- /* get iATU unroll support */
- pp->iatu_unroll_enabled = dw_pcie_iatu_unroll_enabled(pp);
- dev_dbg(pp->dev, "iATU unroll: %s\n",
- pp->iatu_unroll_enabled ? "enabled" : "disabled");
-
/* set the number of lanes */
val = dw_pcie_readl_rc(pp, PCIE_PORT_LINK_CONTROL);
val &= ~PORT_LINK_MODE_MASK;
* we should not program the ATU here.
*/
if (!pp->ops->rd_other_conf) {
+ /* get iATU unroll support */
+ pp->iatu_unroll_enabled = dw_pcie_iatu_unroll_enabled(pp);
+ dev_dbg(pp->dev, "iATU unroll: %s\n",
+ pp->iatu_unroll_enabled ? "enabled" : "disabled");
+
dw_pcie_prog_outbound_atu(pp, PCIE_ATU_REGION_INDEX0,
PCIE_ATU_TYPE_MEM, pp->mem_base,
pp->mem_bus_addr, pp->mem_size);
pos = pci_find_capability(pdev, PCI_CAP_ID_EXP);
if (!pos)
return;
+
pdev->pcie_cap = pos;
pci_read_config_word(pdev, pos + PCI_EXP_FLAGS, ®16);
pdev->pcie_flags_reg = reg16;
pdev->pcie_mpss = reg16 & PCI_EXP_DEVCAP_PAYLOAD;
/*
- * A Root Port is always the upstream end of a Link. No PCIe
- * component has two Links. Two Links are connected by a Switch
- * that has a Port on each Link and internal logic to connect the
- * two Ports.
+ * A Root Port or a PCI-to-PCIe bridge is always the upstream end
+ * of a Link. No PCIe component has two Links. Two Links are
+ * connected by a Switch that has a Port on each Link and internal
+ * logic to connect the two Ports.
*/
type = pci_pcie_type(pdev);
- if (type == PCI_EXP_TYPE_ROOT_PORT)
+ if (type == PCI_EXP_TYPE_ROOT_PORT ||
+ type == PCI_EXP_TYPE_PCIE_BRIDGE)
pdev->has_secondary_link = 1;
else if (type == PCI_EXP_TYPE_UPSTREAM ||
type == PCI_EXP_TYPE_DOWNSTREAM) {
enum pin_config_param param = pinconf_to_config_param(*config);
void __iomem *conf_reg = byt_gpio_reg(vg, offset, BYT_CONF0_REG);
void __iomem *val_reg = byt_gpio_reg(vg, offset, BYT_VAL_REG);
+ void __iomem *db_reg = byt_gpio_reg(vg, offset, BYT_DEBOUNCE_REG);
unsigned long flags;
u32 conf, pull, val, debounce;
u16 arg = 0;
return -EINVAL;
raw_spin_lock_irqsave(&vg->lock, flags);
- debounce = readl(byt_gpio_reg(vg, offset, BYT_DEBOUNCE_REG));
+ debounce = readl(db_reg);
raw_spin_unlock_irqrestore(&vg->lock, flags);
switch (debounce & BYT_DEBOUNCE_PULSE_MASK) {
unsigned int param, arg;
void __iomem *conf_reg = byt_gpio_reg(vg, offset, BYT_CONF0_REG);
void __iomem *val_reg = byt_gpio_reg(vg, offset, BYT_VAL_REG);
+ void __iomem *db_reg = byt_gpio_reg(vg, offset, BYT_DEBOUNCE_REG);
unsigned long flags;
u32 conf, val, debounce;
int i, ret = 0;
break;
case PIN_CONFIG_INPUT_DEBOUNCE:
- debounce = readl(byt_gpio_reg(vg, offset,
- BYT_DEBOUNCE_REG));
- conf &= ~BYT_DEBOUNCE_PULSE_MASK;
+ debounce = readl(db_reg);
+ debounce &= ~BYT_DEBOUNCE_PULSE_MASK;
switch (arg) {
+ case 0:
+ conf &= BYT_DEBOUNCE_EN;
+ break;
case 375:
- conf |= BYT_DEBOUNCE_PULSE_375US;
+ debounce |= BYT_DEBOUNCE_PULSE_375US;
break;
case 750:
- conf |= BYT_DEBOUNCE_PULSE_750US;
+ debounce |= BYT_DEBOUNCE_PULSE_750US;
break;
case 1500:
- conf |= BYT_DEBOUNCE_PULSE_1500US;
+ debounce |= BYT_DEBOUNCE_PULSE_1500US;
break;
case 3000:
- conf |= BYT_DEBOUNCE_PULSE_3MS;
+ debounce |= BYT_DEBOUNCE_PULSE_3MS;
break;
case 6000:
- conf |= BYT_DEBOUNCE_PULSE_6MS;
+ debounce |= BYT_DEBOUNCE_PULSE_6MS;
break;
case 12000:
- conf |= BYT_DEBOUNCE_PULSE_12MS;
+ debounce |= BYT_DEBOUNCE_PULSE_12MS;
break;
case 24000:
- conf |= BYT_DEBOUNCE_PULSE_24MS;
+ debounce |= BYT_DEBOUNCE_PULSE_24MS;
break;
default:
ret = -EINVAL;
}
+ if (!ret)
+ writel(debounce, db_reg);
break;
default:
ret = -ENOTSUPP;
static void byt_gpio_irq_init_hw(struct byt_gpio *vg)
{
+ struct gpio_chip *gc = &vg->chip;
+ struct device *dev = &vg->pdev->dev;
void __iomem *reg;
u32 base, value;
int i;
}
value = readl(reg);
- if ((value & BYT_PIN_MUX) == byt_get_gpio_mux(vg, i) &&
- !(value & BYT_DIRECT_IRQ_EN)) {
+ if (value & BYT_DIRECT_IRQ_EN) {
+ clear_bit(i, gc->irq_valid_mask);
+ dev_dbg(dev, "excluding GPIO %d from IRQ domain\n", i);
+ } else if ((value & BYT_PIN_MUX) == byt_get_gpio_mux(vg, i)) {
byt_gpio_clear_triggering(vg, i);
- dev_dbg(&vg->pdev->dev, "disabling GPIO %d\n", i);
+ dev_dbg(dev, "disabling GPIO %d\n", i);
}
}
gc->can_sleep = false;
gc->parent = &vg->pdev->dev;
gc->ngpio = vg->soc_data->npins;
+ gc->irq_need_valid_mask = true;
#ifdef CONFIG_PM_SLEEP
vg->saved_context = devm_kcalloc(&vg->pdev->dev, gc->ngpio,
#define BXT_PAD_OWN 0x020
#define BXT_HOSTSW_OWN 0x080
-#define BXT_PADCFGLOCK 0x090
+#define BXT_PADCFGLOCK 0x060
#define BXT_GPI_IE 0x110
#define BXT_COMMUNITY(s, e) \
return 0;
}
+static void __intel_gpio_set_direction(void __iomem *padcfg0, bool input)
+{
+ u32 value;
+
+ value = readl(padcfg0);
+ if (input) {
+ value &= ~PADCFG0_GPIORXDIS;
+ value |= PADCFG0_GPIOTXDIS;
+ } else {
+ value &= ~PADCFG0_GPIOTXDIS;
+ value |= PADCFG0_GPIORXDIS;
+ }
+ writel(value, padcfg0);
+}
+
static int intel_gpio_request_enable(struct pinctrl_dev *pctldev,
struct pinctrl_gpio_range *range,
unsigned pin)
/* Disable SCI/SMI/NMI generation */
value &= ~(PADCFG0_GPIROUTIOXAPIC | PADCFG0_GPIROUTSCI);
value &= ~(PADCFG0_GPIROUTSMI | PADCFG0_GPIROUTNMI);
- /* Disable TX buffer and enable RX (this will be input) */
- value &= ~PADCFG0_GPIORXDIS;
- value |= PADCFG0_GPIOTXDIS;
writel(value, padcfg0);
+ /* Disable TX buffer and enable RX (this will be input) */
+ __intel_gpio_set_direction(padcfg0, true);
+
raw_spin_unlock_irqrestore(&pctrl->lock, flags);
return 0;
struct intel_pinctrl *pctrl = pinctrl_dev_get_drvdata(pctldev);
void __iomem *padcfg0;
unsigned long flags;
- u32 value;
raw_spin_lock_irqsave(&pctrl->lock, flags);
padcfg0 = intel_get_padcfg(pctrl, pin, PADCFG0);
-
- value = readl(padcfg0);
- if (input)
- value |= PADCFG0_GPIOTXDIS;
- else
- value &= ~PADCFG0_GPIOTXDIS;
- writel(value, padcfg0);
+ __intel_gpio_set_direction(padcfg0, input);
raw_spin_unlock_irqrestore(&pctrl->lock, flags);
static const unsigned int uart_rx_ao_a_pins[] = { PIN(GPIOAO_1, 0) };
static const unsigned int uart_cts_ao_a_pins[] = { PIN(GPIOAO_2, 0) };
static const unsigned int uart_rts_ao_a_pins[] = { PIN(GPIOAO_3, 0) };
-static const unsigned int uart_tx_ao_b_pins[] = { PIN(GPIOAO_0, 0) };
-static const unsigned int uart_rx_ao_b_pins[] = { PIN(GPIOAO_1, 0),
- PIN(GPIOAO_5, 0) };
+static const unsigned int uart_tx_ao_b_pins[] = { PIN(GPIOAO_4, 0) };
+static const unsigned int uart_rx_ao_b_pins[] = { PIN(GPIOAO_5, 0) };
static const unsigned int uart_cts_ao_b_pins[] = { PIN(GPIOAO_2, 0) };
static const unsigned int uart_rts_ao_b_pins[] = { PIN(GPIOAO_3, 0) };
GPIO_GROUP(GPIOAO_13, 0),
/* bank AO */
- GROUP(uart_tx_ao_b, 0, 26),
+ GROUP(uart_tx_ao_b, 0, 24),
GROUP(uart_rx_ao_b, 0, 25),
GROUP(uart_tx_ao_a, 0, 12),
GROUP(uart_rx_ao_a, 0, 11),
static const unsigned int uart_rx_ao_a_pins[] = { PIN(GPIOAO_1, 0) };
static const unsigned int uart_cts_ao_a_pins[] = { PIN(GPIOAO_2, 0) };
static const unsigned int uart_rts_ao_a_pins[] = { PIN(GPIOAO_3, 0) };
-static const unsigned int uart_tx_ao_b_pins[] = { PIN(GPIOAO_0, 0) };
-static const unsigned int uart_rx_ao_b_pins[] = { PIN(GPIOAO_1, 0),
- PIN(GPIOAO_5, 0) };
+static const unsigned int uart_tx_ao_b_pins[] = { PIN(GPIOAO_4, 0) };
+static const unsigned int uart_rx_ao_b_pins[] = { PIN(GPIOAO_5, 0) };
static const unsigned int uart_cts_ao_b_pins[] = { PIN(GPIOAO_2, 0) };
static const unsigned int uart_rts_ao_b_pins[] = { PIN(GPIOAO_3, 0) };
GPIO_GROUP(GPIOAO_9, 0),
/* bank AO */
- GROUP(uart_tx_ao_b, 0, 26),
+ GROUP(uart_tx_ao_b, 0, 24),
GROUP(uart_rx_ao_b, 0, 25),
GROUP(uart_tx_ao_a, 0, 12),
GROUP(uart_rx_ao_a, 0, 11),
i = 128;
pin_num = AMD_GPIO_PINS_BANK2 + i;
break;
+ default:
+ return;
}
for (; i < pin_num; i++) {
0, 0, 0, 0};
static const unsigned ether_rmii_pins[] = {30, 31, 32, 33, 34, 35, 36, 37, 39,
41, 42, 45};
-static const int ether_rmii_muxvals[] = {1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1};
+static const int ether_rmii_muxvals[] = {0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1};
static const unsigned i2c0_pins[] = {63, 64};
static const int i2c0_muxvals[] = {0, 0};
static const unsigned i2c1_pins[] = {65, 66};
case 8:
case 7:
case 6:
+ case 1:
ideapad_input_report(priv, vpc_bit);
break;
case 5:
input_set_capability(input, EV_KEY, KEY_POWER);
- error = request_threaded_irq(irq, NULL, mfld_pb_isr, 0,
+ error = request_threaded_irq(irq, NULL, mfld_pb_isr, IRQF_ONESHOT,
DRIVER_NAME, input);
if (error) {
dev_err(&pdev->dev, "Unable to request irq %d for mfld power"
return 0;
fail_platform_mux_register:
- for (i--; i > 0 ; i--)
+ while (--i >= 0)
platform_device_unregister(priv->pdev_mux[i]);
platform_device_unregister(priv->pdev_i2c);
fail_alloc:
static int s3_wmi_check_platform_device(struct device *dev, void *data)
{
- struct acpi_device *adev, *ts_adev;
+ struct acpi_device *adev, *ts_adev = NULL;
acpi_handle handle;
acpi_status status;
return 0;
}
-#ifdef CONFIG_PM
-static int s3_wmi_resume(struct device *dev)
+static int __maybe_unused s3_wmi_resume(struct device *dev)
{
s3_wmi_send_lid_state();
return 0;
}
-#endif
static SIMPLE_DEV_PM_OPS(s3_wmi_pm, NULL, s3_wmi_resume);
static struct platform_driver s3_wmi_driver = {
goto unwind_vring_allocations;
}
- /* track the rvdevs list reference */
- kref_get(&rvdev->refcount);
-
list_add_tail(&rvdev->node, &rproc->rvdevs);
rproc_add_subdev(rproc, &rvdev->subdev,
/*
* Create a copy of the resource table. When a virtio device starts
* and calls vring_new_virtqueue() the address of the allocated vring
- * will be stored in the table_ptr. Before the device is started,
- * table_ptr will be copied into device memory.
+ * will be stored in the cached_table. Before the device is started,
+ * cached_table will be copied into device memory.
*/
- rproc->table_ptr = kmemdup(table, tablesz, GFP_KERNEL);
- if (!rproc->table_ptr)
+ rproc->cached_table = kmemdup(table, tablesz, GFP_KERNEL);
+ if (!rproc->cached_table)
goto clean_up;
+ rproc->table_ptr = rproc->cached_table;
+
/* reset max_notifyid */
rproc->max_notifyid = -1;
}
/*
- * The starting device has been given the rproc->table_ptr as the
+ * The starting device has been given the rproc->cached_table as the
* resource table. The address of the vring along with the other
- * allocated resources (carveouts etc) is stored in table_ptr.
+ * allocated resources (carveouts etc) is stored in cached_table.
* In order to pass this information to the remote device we must copy
* this information to device memory. We also update the table_ptr so
* that any subsequent changes will be applied to the loaded version.
*/
loaded_table = rproc_find_loaded_rsc_table(rproc, fw);
- if (loaded_table)
- memcpy(loaded_table, rproc->table_ptr, tablesz);
+ if (loaded_table) {
+ memcpy(loaded_table, rproc->cached_table, tablesz);
+ rproc->table_ptr = loaded_table;
+ }
/* power up the remote processor */
ret = rproc->ops->start(rproc);
clean_up_resources:
rproc_resource_cleanup(rproc);
clean_up:
- kfree(rproc->table_ptr);
+ kfree(rproc->cached_table);
+ rproc->cached_table = NULL;
rproc->table_ptr = NULL;
rproc_disable_iommu(rproc);
rproc_disable_iommu(rproc);
/* Free the copy of the resource table */
- kfree(rproc->table_ptr);
+ kfree(rproc->cached_table);
+ rproc->cached_table = NULL;
rproc->table_ptr = NULL;
/* if in crash state, unlock crash handler */
struct device *dev = &rpdev->dev;
int ret;
- dev_set_name(&rpdev->dev, "%s:%s",
- dev_name(dev->parent), rpdev->id.name);
+ dev_set_name(&rpdev->dev, "%s.%s.%d.%d", dev_name(dev->parent),
+ rpdev->id.name, rpdev->src, rpdev->dst);
rpdev->dev.bus = &rpmsg_bus;
rpdev->dev.release = rpmsg_release_device;
#define CCW_CMD_WRITE_CONF 0x21
#define CCW_CMD_WRITE_STATUS 0x31
#define CCW_CMD_READ_VQ_CONF 0x32
+#define CCW_CMD_READ_STATUS 0x72
#define CCW_CMD_SET_IND_ADAPTER 0x73
#define CCW_CMD_SET_VIRTIO_REV 0x83
#define VIRTIO_CCW_DOING_SET_CONF_IND 0x04000000
#define VIRTIO_CCW_DOING_SET_IND_ADAPTER 0x08000000
#define VIRTIO_CCW_DOING_SET_VIRTIO_REV 0x10000000
+#define VIRTIO_CCW_DOING_READ_STATUS 0x20000000
#define VIRTIO_CCW_INTPARM_MASK 0xffff0000
static struct virtio_ccw_device *to_vc_device(struct virtio_device *vdev)
* This may happen on device detach.
*/
if (ret && (ret != -ENODEV))
- dev_warn(&vq->vdev->dev, "Error %d while deleting queue %d",
+ dev_warn(&vq->vdev->dev, "Error %d while deleting queue %d\n",
ret, index);
vring_del_virtqueue(vq);
static u8 virtio_ccw_get_status(struct virtio_device *vdev)
{
struct virtio_ccw_device *vcdev = to_vc_device(vdev);
+ u8 old_status = *vcdev->status;
+ struct ccw1 *ccw;
+
+ if (vcdev->revision < 1)
+ return *vcdev->status;
+
+ ccw = kzalloc(sizeof(*ccw), GFP_DMA | GFP_KERNEL);
+ if (!ccw)
+ return old_status;
+
+ ccw->cmd_code = CCW_CMD_READ_STATUS;
+ ccw->flags = 0;
+ ccw->count = sizeof(*vcdev->status);
+ ccw->cda = (__u32)(unsigned long)vcdev->status;
+ ccw_io_helper(vcdev, ccw, VIRTIO_CCW_DOING_READ_STATUS);
+/*
+ * If the channel program failed (should only happen if the device
+ * was hotunplugged, and then we clean up via the machine check
+ * handler anyway), vcdev->status was not overwritten and we just
+ * return the old status, which is fine.
+*/
+ kfree(ccw);
return *vcdev->status;
}
kfree(ccw);
}
-static struct virtio_config_ops virtio_ccw_config_ops = {
+static const struct virtio_config_ops virtio_ccw_config_ops = {
.get_features = virtio_ccw_get_features,
.finalize_features = virtio_ccw_finalize_features,
.get = virtio_ccw_get_config,
case VIRTIO_CCW_DOING_READ_CONFIG:
case VIRTIO_CCW_DOING_WRITE_CONFIG:
case VIRTIO_CCW_DOING_WRITE_STATUS:
+ case VIRTIO_CCW_DOING_READ_STATUS:
case VIRTIO_CCW_DOING_SET_VQ:
case VIRTIO_CCW_DOING_SET_IND:
case VIRTIO_CCW_DOING_SET_CONF_IND:
u32 bfi_image_cb_size, bfi_image_ct_size, bfi_image_ct2_size;
u32 *bfi_image_cb, *bfi_image_ct, *bfi_image_ct2;
-#define BFAD_FW_FILE_CB "cbfw-3.2.3.0.bin"
-#define BFAD_FW_FILE_CT "ctfw-3.2.3.0.bin"
-#define BFAD_FW_FILE_CT2 "ct2fw-3.2.3.0.bin"
+#define BFAD_FW_FILE_CB "cbfw-3.2.5.1.bin"
+#define BFAD_FW_FILE_CT "ctfw-3.2.5.1.bin"
+#define BFAD_FW_FILE_CT2 "ct2fw-3.2.5.1.bin"
static u32 *bfad_load_fwimg(struct pci_dev *pdev);
static void bfad_free_fwimg(void);
struct bfad_fcxp *drv_fcxp;
struct bfa_fcs_lport_s *fcs_port;
struct bfa_fcs_rport_s *fcs_rport;
- struct fc_bsg_request *bsg_request = bsg_request;
+ struct fc_bsg_request *bsg_request = job->request;
struct fc_bsg_reply *bsg_reply = job->reply;
uint32_t command_type = bsg_request->msgcode;
unsigned long flags;
#ifdef BFA_DRIVER_VERSION
#define BFAD_DRIVER_VERSION BFA_DRIVER_VERSION
#else
-#define BFAD_DRIVER_VERSION "3.2.25.0"
+#define BFAD_DRIVER_VERSION "3.2.25.1"
#endif
#define BFAD_PROTO_NAME FCPI_NAME
struct completion *remove_wait; /* device remove thread blocks */
atomic_t in_flight; /* io counter */
+ bool internal_reset_inprogress;
u32 _reserved; /* fill hole */
unsigned long state_flags; /* protected by host lock */
enum fnic_state state;
unsigned long wait_host_tmo;
struct Scsi_Host *shost = sc->device->host;
struct fc_lport *lp = shost_priv(shost);
+ struct fnic *fnic = lport_priv(lp);
+ unsigned long flags;
+
+ spin_lock_irqsave(&fnic->fnic_lock, flags);
+ if (fnic->internal_reset_inprogress == 0) {
+ fnic->internal_reset_inprogress = 1;
+ } else {
+ spin_unlock_irqrestore(&fnic->fnic_lock, flags);
+ FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host,
+ "host reset in progress skipping another host reset\n");
+ return SUCCESS;
+ }
+ spin_unlock_irqrestore(&fnic->fnic_lock, flags);
/*
* If fnic_reset is successful, wait for fabric login to complete
}
}
+ spin_lock_irqsave(&fnic->fnic_lock, flags);
+ fnic->internal_reset_inprogress = 0;
+ spin_unlock_irqrestore(&fnic->fnic_lock, flags);
return ret;
}
#define INITIAL_SRP_LIMIT 800
#define DEFAULT_MAX_SECTORS 256
+#define MAX_TXU 1024 * 1024
static uint max_vdma_size = MAX_H_COPY_RDMA;
}
info = dma_alloc_coherent(&vscsi->dma_dev->dev, sizeof(*info), &token,
- GFP_KERNEL);
+ GFP_ATOMIC);
if (!info) {
dev_err(&vscsi->dev, "bad dma_alloc_coherent %p\n",
iue->target);
info->mad_version = cpu_to_be32(MAD_VERSION_1);
info->os_type = cpu_to_be32(LINUX);
memset(&info->port_max_txu[0], 0, sizeof(info->port_max_txu));
- info->port_max_txu[0] = cpu_to_be32(128 * PAGE_SIZE);
+ info->port_max_txu[0] = cpu_to_be32(MAX_TXU);
dma_wmb();
rc = h_copy_rdma(sizeof(*info), vscsi->dds.window[LOCAL].liobn,
}
cap = dma_alloc_coherent(&vscsi->dma_dev->dev, olen, &token,
- GFP_KERNEL);
+ GFP_ATOMIC);
if (!cap) {
dev_err(&vscsi->dev, "bad dma_alloc_coherent %p\n",
iue->target);
1, 1);
if (rc) {
pr_err("srp_transfer_data() failed: %d\n", rc);
- return -EAGAIN;
+ return -EIO;
}
/*
* We now tell TCM to add this WRITE CDB directly into the TCM storage
} else {
buf_ptr1 = (struct lpfc_dmabuf *) elsiocb->context2;
lpfc_els_free_data(phba, buf_ptr1);
+ elsiocb->context2 = NULL;
}
}
if (elsiocb->context3) {
buf_ptr = (struct lpfc_dmabuf *) elsiocb->context3;
lpfc_els_free_bpl(phba, buf_ptr);
+ elsiocb->context3 = NULL;
}
lpfc_sli_release_iocbq(phba, elsiocb);
return 0;
free_vfi_bmask:
kfree(phba->sli4_hba.vfi_bmask);
+ phba->sli4_hba.vfi_bmask = NULL;
free_xri_ids:
kfree(phba->sli4_hba.xri_ids);
+ phba->sli4_hba.xri_ids = NULL;
free_xri_bmask:
kfree(phba->sli4_hba.xri_bmask);
+ phba->sli4_hba.xri_bmask = NULL;
free_vpi_ids:
kfree(phba->vpi_ids);
+ phba->vpi_ids = NULL;
free_vpi_bmask:
kfree(phba->vpi_bmask);
+ phba->vpi_bmask = NULL;
free_rpi_ids:
kfree(phba->sli4_hba.rpi_ids);
+ phba->sli4_hba.rpi_ids = NULL;
free_rpi_bmask:
kfree(phba->sli4_hba.rpi_bmask);
+ phba->sli4_hba.rpi_bmask = NULL;
err_exit:
return rc;
}
* @eedp_enable: eedp support enable bit
* @eedp_type: 0(type_1), 1(type_2), 2(type_3)
* @eedp_block_length: block size
+ * @ata_command_pending: SATL passthrough outstanding for device
*/
struct MPT3SAS_DEVICE {
struct MPT3SAS_TARGET *sas_target;
u8 ignore_delay_remove;
/* Iopriority Command Handling */
u8 ncq_prio_enable;
+ /*
+ * Bug workaround for SATL handling: the mpt2/3sas firmware
+ * doesn't return BUSY or TASK_SET_FULL for subsequent
+ * commands while a SATL pass through is in operation as the
+ * spec requires, it simply does nothing with them until the
+ * pass through completes, causing them possibly to timeout if
+ * the passthrough is a long executing command (like format or
+ * secure erase). This variable allows us to do the right
+ * thing while a SATL command is pending.
+ */
+ unsigned long ata_command_pending;
};
}
}
-static inline bool ata_12_16_cmd(struct scsi_cmnd *scmd)
+static int _scsih_set_satl_pending(struct scsi_cmnd *scmd, bool pending)
{
- return (scmd->cmnd[0] == ATA_12 || scmd->cmnd[0] == ATA_16);
+ struct MPT3SAS_DEVICE *priv = scmd->device->hostdata;
+
+ if (scmd->cmnd[0] != ATA_12 && scmd->cmnd[0] != ATA_16)
+ return 0;
+
+ if (pending)
+ return test_and_set_bit(0, &priv->ata_command_pending);
+
+ clear_bit(0, &priv->ata_command_pending);
+ return 0;
}
/**
if (!scmd)
continue;
count++;
- if (ata_12_16_cmd(scmd))
- scsi_internal_device_unblock(scmd->device,
- SDEV_RUNNING);
+ _scsih_set_satl_pending(scmd, false);
mpt3sas_base_free_smid(ioc, smid);
scsi_dma_unmap(scmd);
if (ioc->pci_error_recovery)
if (ioc->logging_level & MPT_DEBUG_SCSI)
scsi_print_command(scmd);
- /*
- * Lock the device for any subsequent command until command is
- * done.
- */
- if (ata_12_16_cmd(scmd))
- scsi_internal_device_block(scmd->device);
-
sas_device_priv_data = scmd->device->hostdata;
if (!sas_device_priv_data || !sas_device_priv_data->sas_target) {
scmd->result = DID_NO_CONNECT << 16;
return 0;
}
+ /*
+ * Bug work around for firmware SATL handling. The loop
+ * is based on atomic operations and ensures consistency
+ * since we're lockless at this point
+ */
+ do {
+ if (test_bit(0, &sas_device_priv_data->ata_command_pending)) {
+ scmd->result = SAM_STAT_BUSY;
+ scmd->scsi_done(scmd);
+ return 0;
+ }
+ } while (_scsih_set_satl_pending(scmd, true));
+
sas_target_priv_data = sas_device_priv_data->sas_target;
/* invalid device handle */
if (scmd == NULL)
return 1;
- if (ata_12_16_cmd(scmd))
- scsi_internal_device_unblock(scmd->device, SDEV_RUNNING);
+ _scsih_set_satl_pending(scmd, false);
mpi_request = mpt3sas_base_get_msg_frame(ioc, smid);
config QEDI
tristate "QLogic QEDI 25/40/100Gb iSCSI Initiator Driver Support"
- depends on PCI && SCSI
+ depends on PCI && SCSI && UIO
depends on QED
select SCSI_ISCSI_ATTRS
select QED_LL2
struct qla_hw_data *ha = vha->hw;
ssize_t rval = 0;
+ mutex_lock(&ha->optrom_mutex);
+
if (ha->optrom_state != QLA_SREADING)
- return 0;
+ goto out;
- mutex_lock(&ha->optrom_mutex);
rval = memory_read_from_buffer(buf, count, &off, ha->optrom_buffer,
ha->optrom_region_size);
+
+out:
mutex_unlock(&ha->optrom_mutex);
return rval;
struct device, kobj)));
struct qla_hw_data *ha = vha->hw;
- if (ha->optrom_state != QLA_SWRITING)
+ mutex_lock(&ha->optrom_mutex);
+
+ if (ha->optrom_state != QLA_SWRITING) {
+ mutex_unlock(&ha->optrom_mutex);
return -EINVAL;
- if (off > ha->optrom_region_size)
+ }
+ if (off > ha->optrom_region_size) {
+ mutex_unlock(&ha->optrom_mutex);
return -ERANGE;
+ }
if (off + count > ha->optrom_region_size)
count = ha->optrom_region_size - off;
- mutex_lock(&ha->optrom_mutex);
memcpy(&ha->optrom_buffer[off], buf, count);
mutex_unlock(&ha->optrom_mutex);
struct scsi_qla_host *vha = shost_priv(dev_to_shost(container_of(kobj,
struct device, kobj)));
int type;
- int rval = 0;
port_id_t did;
type = simple_strtol(buf, NULL, 10);
ql_log(ql_log_info, vha, 0x70e4, "%s: %d\n", __func__, type);
- rval = qla24xx_els_dcmd_iocb(vha, ELS_DCMD_LOGO, did);
+ qla24xx_els_dcmd_iocb(vha, ELS_DCMD_LOGO, did);
return count;
}
struct atio {
uint8_t entry_type; /* Entry type. */
uint8_t entry_count; /* Entry count. */
- uint8_t data[58];
+ __le16 attr_n_length;
+ uint8_t data[56];
uint32_t signature;
#define ATIO_PROCESSED 0xDEADDEAD /* Signature */
};
#define QLA_MSIX_FW_MODE(m) (((m) & (BIT_7|BIT_8|BIT_9)) >> 7)
#define QLA_MSIX_FW_MODE_1(m) (QLA_MSIX_FW_MODE(m) == 1)
-#define QLA_MSIX_DEFAULT 0x00
+#define QLA_BASE_VECTORS 2 /* default + RSP */
#define QLA_MSIX_RSP_Q 0x01
#define QLA_ATIO_VECTOR 0x02
#define QLA_MSIX_QPAIR_MULTIQ_RSP_Q 0x03
uint16_t entry;
char name[30];
void *handle;
- struct irq_affinity_notify irq_notify;
int cpuid;
};
/* Wait for soft-reset to complete. */
RD_REG_DWORD(®->ctrl_status);
- for (cnt = 0; cnt < 6000000; cnt++) {
+ for (cnt = 0; cnt < 60; cnt++) {
barrier();
if ((RD_REG_DWORD(®->ctrl_status) &
CSRX_ISP_SOFT_RESET) == 0)
RD_REG_DWORD(®->hccr);
RD_REG_WORD(®->mailbox0);
- for (cnt = 6000000; RD_REG_WORD(®->mailbox0) != 0 &&
+ for (cnt = 60; RD_REG_WORD(®->mailbox0) != 0 &&
rval == QLA_SUCCESS; cnt--) {
barrier();
if (cnt)
static void qla2x00_status_cont_entry(struct rsp_que *, sts_cont_entry_t *);
static void qla2x00_error_entry(scsi_qla_host_t *, struct rsp_que *,
sts_entry_t *);
-static void qla_irq_affinity_notify(struct irq_affinity_notify *,
- const cpumask_t *);
-static void qla_irq_affinity_release(struct kref *);
-
/**
* qla2100_intr_handler() - Process interrupts for the ISP2100 and ISP2200.
if (pkt->entry_status & RF_BUSY)
res = DID_BUS_BUSY << 16;
+ if (pkt->entry_type == NOTIFY_ACK_TYPE &&
+ pkt->handle == QLA_TGT_SKIP_HANDLE)
+ return;
+
sp = qla2x00_get_sp_from_handle(vha, func, req, pkt);
if (sp) {
sp->done(ha, sp, res);
if (!vha->flags.online)
return;
- if (rsp->msix && rsp->msix->cpuid != smp_processor_id()) {
- /* if kernel does not notify qla of IRQ's CPU change,
- * then set it here.
- */
- rsp->msix->cpuid = smp_processor_id();
- ha->tgt.rspq_vector_cpuid = rsp->msix->cpuid;
- }
-
while (rsp->ring_ptr->signature != RESPONSE_PROCESSED) {
pkt = (struct sts_entry_24xx *)rsp->ring_ptr;
static int
qla24xx_enable_msix(struct qla_hw_data *ha, struct rsp_que *rsp)
{
-#define MIN_MSIX_COUNT 2
int i, ret;
struct qla_msix_entry *qentry;
scsi_qla_host_t *vha = pci_get_drvdata(ha->pdev);
+ struct irq_affinity desc = {
+ .pre_vectors = QLA_BASE_VECTORS,
+ };
+
+ if (QLA_TGT_MODE_ENABLED() && IS_ATIO_MSIX_CAPABLE(ha))
+ desc.pre_vectors++;
+
+ ret = pci_alloc_irq_vectors_affinity(ha->pdev, QLA_BASE_VECTORS,
+ ha->msix_count, PCI_IRQ_MSIX | PCI_IRQ_AFFINITY,
+ &desc);
- ret = pci_alloc_irq_vectors(ha->pdev, MIN_MSIX_COUNT, ha->msix_count,
- PCI_IRQ_MSIX | PCI_IRQ_AFFINITY);
if (ret < 0) {
ql_log(ql_log_fatal, vha, 0x00c7,
"MSI-X: Failed to enable support, "
qentry->have_irq = 0;
qentry->in_use = 0;
qentry->handle = NULL;
- qentry->irq_notify.notify = qla_irq_affinity_notify;
- qentry->irq_notify.release = qla_irq_affinity_release;
- qentry->cpuid = -1;
}
/* Enable MSI-X vectors for the base queue */
- for (i = 0; i < (QLA_MSIX_RSP_Q + 1); i++) {
+ for (i = 0; i < QLA_BASE_VECTORS; i++) {
qentry = &ha->msix_entries[i];
qentry->handle = rsp;
rsp->msix = qentry;
goto msix_register_fail;
qentry->have_irq = 1;
qentry->in_use = 1;
-
- /* Register for CPU affinity notification. */
- irq_set_affinity_notifier(qentry->vector, &qentry->irq_notify);
-
- /* Schedule work (ie. trigger a notification) to read cpu
- * mask for this specific irq.
- * kref_get is required because
- * irq_affinity_notify() will do
- * kref_put().
- */
- kref_get(&qentry->irq_notify.kref);
- schedule_work(&qentry->irq_notify.work);
}
/*
msix->handle = qpair;
return ret;
}
-
-
-/* irq_set_affinity/irqbalance will trigger notification of cpu mask update */
-static void qla_irq_affinity_notify(struct irq_affinity_notify *notify,
- const cpumask_t *mask)
-{
- struct qla_msix_entry *e =
- container_of(notify, struct qla_msix_entry, irq_notify);
- struct qla_hw_data *ha;
- struct scsi_qla_host *base_vha;
- struct rsp_que *rsp = e->handle;
-
- /* user is recommended to set mask to just 1 cpu */
- e->cpuid = cpumask_first(mask);
-
- ha = rsp->hw;
- base_vha = pci_get_drvdata(ha->pdev);
-
- ql_dbg(ql_dbg_init, base_vha, 0xffff,
- "%s: host %ld : vector %d cpu %d \n", __func__,
- base_vha->host_no, e->vector, e->cpuid);
-
- if (e->have_irq) {
- if ((IS_QLA83XX(ha) || IS_QLA27XX(ha)) &&
- (e->entry == QLA83XX_RSPQ_MSIX_ENTRY_NUMBER)) {
- ha->tgt.rspq_vector_cpuid = e->cpuid;
- ql_dbg(ql_dbg_init, base_vha, 0xffff,
- "%s: host%ld: rspq vector %d cpu %d runtime change\n",
- __func__, base_vha->host_no, e->vector, e->cpuid);
- }
- }
-}
-
-static void qla_irq_affinity_release(struct kref *ref)
-{
- struct irq_affinity_notify *notify =
- container_of(ref, struct irq_affinity_notify, kref);
- struct qla_msix_entry *e =
- container_of(notify, struct qla_msix_entry, irq_notify);
- struct rsp_que *rsp = e->handle;
- struct scsi_qla_host *base_vha = pci_get_drvdata(rsp->hw->pdev);
-
- ql_dbg(ql_dbg_init, base_vha, 0xffff,
- "%s: host%ld: vector %d cpu %d\n", __func__,
- base_vha->host_no, e->vector, e->cpuid);
-}
#include <linux/delay.h>
#include <linux/gfp.h>
-struct rom_cmd {
+static struct rom_cmd {
uint16_t cmd;
} rom_cmds[] = {
{ MBC_LOAD_RAM },
return QLA_FUNCTION_TIMEOUT;
}
- /* if PCI error, then avoid mbx processing.*/
- if (test_bit(PCI_ERR, &base_vha->dpc_flags)) {
+ /* if PCI error, then avoid mbx processing.*/
+ if (test_bit(PCI_ERR, &base_vha->dpc_flags)) {
ql_log(ql_log_warn, vha, 0x1191,
"PCI error, exiting.\n");
return QLA_FUNCTION_TIMEOUT;
- }
+ }
reg = ha->iobase;
io_lock_on = base_vha->flags.init_done;
}
} else {
- uint16_t mb0;
- uint32_t ictrl;
+ uint16_t mb[8];
+ uint32_t ictrl, host_status, hccr;
uint16_t w;
if (IS_FWI2_CAPABLE(ha)) {
- mb0 = RD_REG_WORD(®->isp24.mailbox0);
+ mb[0] = RD_REG_WORD(®->isp24.mailbox0);
+ mb[1] = RD_REG_WORD(®->isp24.mailbox1);
+ mb[2] = RD_REG_WORD(®->isp24.mailbox2);
+ mb[3] = RD_REG_WORD(®->isp24.mailbox3);
+ mb[7] = RD_REG_WORD(®->isp24.mailbox7);
ictrl = RD_REG_DWORD(®->isp24.ictrl);
+ host_status = RD_REG_DWORD(®->isp24.host_status);
+ hccr = RD_REG_DWORD(®->isp24.hccr);
+
+ ql_log(ql_log_warn, vha, 0x1119,
+ "MBX Command timeout for cmd %x, iocontrol=%x jiffies=%lx "
+ "mb[0-3]=[0x%x 0x%x 0x%x 0x%x] mb7 0x%x host_status 0x%x hccr 0x%x\n",
+ command, ictrl, jiffies, mb[0], mb[1], mb[2], mb[3],
+ mb[7], host_status, hccr);
+
} else {
- mb0 = RD_MAILBOX_REG(ha, ®->isp, 0);
+ mb[0] = RD_MAILBOX_REG(ha, ®->isp, 0);
ictrl = RD_REG_WORD(®->isp.ictrl);
+ ql_dbg(ql_dbg_mbx + ql_dbg_buffer, vha, 0x1119,
+ "MBX Command timeout for cmd %x, iocontrol=%x jiffies=%lx "
+ "mb[0]=0x%x\n", command, ictrl, jiffies, mb[0]);
}
- ql_dbg(ql_dbg_mbx + ql_dbg_buffer, vha, 0x1119,
- "MBX Command timeout for cmd %x, iocontrol=%x jiffies=%lx "
- "mb[0]=0x%x\n", command, ictrl, jiffies, mb0);
ql_dump_regs(ql_dbg_mbx + ql_dbg_buffer, vha, 0x1019);
/* Capture FW dump only, if PCI device active */
mbx_cmd_t mc;
mbx_cmd_t *mcp = &mc;
struct qla_hw_data *ha = vha->hw;
- int configured_count;
ql_dbg(ql_dbg_mbx + ql_dbg_verbose, vha, 0x111a,
"Entered %s.\n", __func__);
/*EMPTY*/
ql_dbg(ql_dbg_mbx, vha, 0x111b, "Failed=%x.\n", rval);
} else {
- configured_count = mcp->mb[11];
ql_dbg(ql_dbg_mbx + ql_dbg_verbose, vha, 0x118c,
"Done %s.\n", __func__);
}
(crb_addr_xform[QLA82XX_HW_PX_MAP_CRB_##name] = \
QLA82XX_HW_CRB_HUB_AGT_ADR_##name << 20)
+const int MD_MIU_TEST_AGT_RDDATA[] = {
+ 0x410000A8, 0x410000AC,
+ 0x410000B8, 0x410000BC
+};
+
static void qla82xx_crb_addr_transform_setup(void)
{
qla82xx_crb_addr_transform(XDMA);
#define MD_MIU_TEST_AGT_ADDR_LO 0x41000094
#define MD_MIU_TEST_AGT_ADDR_HI 0x41000098
-static const int MD_MIU_TEST_AGT_RDDATA[] = { 0x410000A8, 0x410000AC,
- 0x410000B8, 0x410000BC };
+extern const int MD_MIU_TEST_AGT_RDDATA[4];
#define CRB_NIU_XG_PAUSE_CTL_P0 0x1
#define CRB_NIU_XG_PAUSE_CTL_P1 0x8
#define TIMEOUT_100_MS 100
+static const uint32_t qla8044_reg_tbl[] = {
+ QLA8044_PEG_HALT_STATUS1,
+ QLA8044_PEG_HALT_STATUS2,
+ QLA8044_PEG_ALIVE_COUNTER,
+ QLA8044_CRB_DRV_ACTIVE,
+ QLA8044_CRB_DEV_STATE,
+ QLA8044_CRB_DRV_STATE,
+ QLA8044_CRB_DRV_SCRATCH,
+ QLA8044_CRB_DEV_PART_INFO1,
+ QLA8044_CRB_IDC_VER_MAJOR,
+ QLA8044_FW_VER_MAJOR,
+ QLA8044_FW_VER_MINOR,
+ QLA8044_FW_VER_SUB,
+ QLA8044_CMDPEG_STATE,
+ QLA8044_ASIC_TEMP,
+};
+
/* 8044 Flash Read/Write functions */
uint32_t
qla8044_rd_reg(struct qla_hw_data *ha, ulong addr)
#define CRB_CMDPEG_CHECK_RETRY_COUNT 60
#define CRB_CMDPEG_CHECK_DELAY 500
-static const uint32_t qla8044_reg_tbl[] = {
- QLA8044_PEG_HALT_STATUS1,
- QLA8044_PEG_HALT_STATUS2,
- QLA8044_PEG_ALIVE_COUNTER,
- QLA8044_CRB_DRV_ACTIVE,
- QLA8044_CRB_DEV_STATE,
- QLA8044_CRB_DRV_STATE,
- QLA8044_CRB_DRV_SCRATCH,
- QLA8044_CRB_DEV_PART_INFO1,
- QLA8044_CRB_IDC_VER_MAJOR,
- QLA8044_FW_VER_MAJOR,
- QLA8044_FW_VER_MINOR,
- QLA8044_FW_VER_SUB,
- QLA8044_CMDPEG_STATE,
- QLA8044_ASIC_TEMP,
-};
-
/* MiniDump Structures */
/* Driver_code is for driver to write some info about the entry
continue;
rsp = ha->rsp_q_map[cnt];
- clear_bit(cnt, ha->req_qid_map);
+ clear_bit(cnt, ha->rsp_qid_map);
ha->rsp_q_map[cnt] = NULL;
spin_unlock_irqrestore(&ha->hardware_lock, flags);
qla2x00_free_rsp_que(ha, rsp);
sizeof(struct ct6_dsd), 0,
SLAB_HWCACHE_ALIGN, NULL);
if (!ctx_cachep)
- goto fail_free_gid_list;
+ goto fail_free_srb_mempool;
}
ha->ctx_mempool = mempool_create_slab_pool(SRB_MIN_REQ,
ctx_cachep);
ha->loop_id_map = kzalloc(BITS_TO_LONGS(LOOPID_MAP_SIZE) * sizeof(long),
GFP_KERNEL);
if (!ha->loop_id_map)
- goto fail_async_pd;
+ goto fail_loop_id_map;
else {
qla2x00_set_reserved_loop_ids(ha);
ql_dbg_pci(ql_dbg_init, ha->pdev, 0x0123,
return 0;
+fail_loop_id_map:
+ dma_pool_free(ha->s_dma_pool, ha->async_pd, ha->async_pd_dma);
fail_async_pd:
dma_pool_free(ha->s_dma_pool, ha->ex_init_cb, ha->ex_init_cb_dma);
fail_ex_init_cb:
dma_pool_free(ha->s_dma_pool, ha->ms_iocb, ha->ms_iocb_dma);
ha->ms_iocb = NULL;
ha->ms_iocb_dma = 0;
+
+ if (ha->sns_cmd)
+ dma_free_coherent(&ha->pdev->dev, sizeof(struct sns_cmd_pkt),
+ ha->sns_cmd, ha->sns_cmd_dma);
fail_dma_pool:
if (IS_QLA82XX(ha) || ql2xenabledif) {
dma_pool_destroy(ha->fcp_cmnd_dma_pool);
kfree(ha->nvram);
ha->nvram = NULL;
fail_free_ctx_mempool:
- mempool_destroy(ha->ctx_mempool);
+ if (ha->ctx_mempool)
+ mempool_destroy(ha->ctx_mempool);
ha->ctx_mempool = NULL;
fail_free_srb_mempool:
- mempool_destroy(ha->srb_mempool);
+ if (ha->srb_mempool)
+ mempool_destroy(ha->srb_mempool);
ha->srb_mempool = NULL;
fail_free_gid_list:
dma_free_coherent(&ha->pdev->dev, qla2x00_gid_list_size(ha),
{
struct qla_hw_data *ha = vha->hw;
struct qla_tgt_sess *sess = NULL;
- uint32_t unpacked_lun, lun = 0;
uint16_t loop_id;
int res = 0;
struct imm_ntfy_from_isp *n = (struct imm_ntfy_from_isp *)iocb;
- struct atio_from_isp *a = (struct atio_from_isp *)iocb;
unsigned long flags;
loop_id = le16_to_cpu(n->u.isp24.nport_handle);
"loop_id %d)\n", vha->host_no, sess, sess->port_name,
mcmd, loop_id);
- lun = a->u.isp24.fcp_cmnd.lun;
- unpacked_lun = scsilun_to_int((struct scsi_lun *)&lun);
-
- return qlt_issue_task_mgmt(sess, unpacked_lun, mcmd,
- iocb, QLA24XX_MGMT_SEND_NACK);
+ return qlt_issue_task_mgmt(sess, 0, mcmd, iocb, QLA24XX_MGMT_SEND_NACK);
}
/* ha->tgt.sess_lock supposed to be held on entry */
pkt->entry_type = NOTIFY_ACK_TYPE;
pkt->entry_count = 1;
- pkt->handle = QLA_TGT_SKIP_HANDLE | CTIO_COMPLETION_HANDLE_MARK;
+ pkt->handle = QLA_TGT_SKIP_HANDLE;
nack = (struct nack_to_isp *)pkt;
nack->ox_id = ntfy->ox_id;
#if 0 /* Todo */
if (rc == -ENOMEM)
qlt_alloc_qfull_cmd(vha, imm, 0, 0);
+#else
+ if (rc) {
+ }
#endif
goto done;
}
if (!vha->flags.online)
return;
- while (ha->tgt.atio_ring_ptr->signature != ATIO_PROCESSED) {
+ while ((ha->tgt.atio_ring_ptr->signature != ATIO_PROCESSED) ||
+ fcpcmd_is_corrupted(ha->tgt.atio_ring_ptr)) {
pkt = (struct atio_from_isp *)ha->tgt.atio_ring_ptr;
cnt = pkt->u.raw.entry_count;
- qlt_24xx_atio_pkt_all_vps(vha, (struct atio_from_isp *)pkt,
- ha_locked);
+ if (unlikely(fcpcmd_is_corrupted(ha->tgt.atio_ring_ptr))) {
+ /*
+ * This packet is corrupted. The header + payload
+ * can not be trusted. There is no point in passing
+ * it further up.
+ */
+ ql_log(ql_log_warn, vha, 0xffff,
+ "corrupted fcp frame SID[%3phN] OXID[%04x] EXCG[%x] %64phN\n",
+ pkt->u.isp24.fcp_hdr.s_id,
+ be16_to_cpu(pkt->u.isp24.fcp_hdr.ox_id),
+ le32_to_cpu(pkt->u.isp24.exchange_addr), pkt);
+
+ adjust_corrupted_atio(pkt);
+ qlt_send_term_exchange(vha, NULL, pkt, ha_locked, 0);
+ } else {
+ qlt_24xx_atio_pkt_all_vps(vha,
+ (struct atio_from_isp *)pkt, ha_locked);
+ }
for (i = 0; i < cnt; i++) {
ha->tgt.atio_ring_index++;
/* Disable Full Login after LIP */
nv->host_p &= cpu_to_le32(~BIT_10);
+
+ /*
+ * clear BIT 15 explicitly as we have seen at least
+ * a couple of instances where this was set and this
+ * was causing the firmware to not be initialized.
+ */
+ nv->firmware_options_1 &= cpu_to_le32(~BIT_15);
/* Enable target PRLI control */
nv->firmware_options_2 |= cpu_to_le32(BIT_14);
} else {
return;
}
- /* out-of-order frames reassembly */
- nv->firmware_options_3 |= BIT_6|BIT_9;
-
if (ha->tgt.enable_class_2) {
if (vha->flags.init_done)
fc_host_supported_classes(vha->host) =
/* Disable ini mode, if requested */
if (!qla_ini_mode_enabled(vha))
nv->firmware_options_1 |= cpu_to_le32(BIT_5);
-
/* Disable Full Login after LIP */
nv->firmware_options_1 &= cpu_to_le32(~BIT_13);
/* Enable initial LIP */
nv->firmware_options_1 &= cpu_to_le32(~BIT_9);
+ /*
+ * clear BIT 15 explicitly as we have seen at
+ * least a couple of instances where this was set
+ * and this was causing the firmware to not be
+ * initialized.
+ */
+ nv->firmware_options_1 &= cpu_to_le32(~BIT_15);
if (ql2xtgt_tape_enable)
/* Enable FC tape support */
nv->firmware_options_2 |= cpu_to_le32(BIT_12);
return;
}
- /* out-of-order frames reassembly */
- nv->firmware_options_3 |= BIT_6|BIT_9;
-
if (ha->tgt.enable_class_2) {
if (vha->flags.init_done)
fc_host_supported_classes(vha->host) =
struct {
uint8_t entry_type; /* Entry type. */
uint8_t entry_count; /* Entry count. */
- uint8_t data[58];
+ __le16 attr_n_length;
+#define FCP_CMD_LENGTH_MASK 0x0fff
+#define FCP_CMD_LENGTH_MIN 0x38
+ uint8_t data[56];
uint32_t signature;
#define ATIO_PROCESSED 0xDEADDEAD /* Signature */
} raw;
} u;
} __packed;
+static inline int fcpcmd_is_corrupted(struct atio *atio)
+{
+ if (atio->entry_type == ATIO_TYPE7 &&
+ (le16_to_cpu(atio->attr_n_length & FCP_CMD_LENGTH_MASK) <
+ FCP_CMD_LENGTH_MIN))
+ return 1;
+ else
+ return 0;
+}
+
+/* adjust corrupted atio so we won't trip over the same entry again. */
+static inline void adjust_corrupted_atio(struct atio_from_isp *atio)
+{
+ atio->u.raw.attr_n_length = cpu_to_le16(FCP_CMD_LENGTH_MIN);
+ atio->u.isp24.fcp_cmnd.add_cdb_len = 0;
+}
+
#define CTIO_TYPE7 0x12 /* Continue target I/O entry (for 24xx) */
/*
count++;
}
}
+ } else if (QLA_TGT_MODE_ENABLED() &&
+ ent->t263.queue_type == T263_QUEUE_TYPE_ATIO) {
+ struct qla_hw_data *ha = vha->hw;
+ struct atio *atr = ha->tgt.atio_ring;
+
+ if (atr || !buf) {
+ length = ha->tgt.atio_q_length;
+ qla27xx_insert16(0, buf, len);
+ qla27xx_insert16(length, buf, len);
+ qla27xx_insertbuf(atr, length * sizeof(*atr), buf, len);
+ count++;
+ }
} else {
ql_dbg(ql_dbg_misc, vha, 0xd026,
"%s: unknown queue %x\n", __func__, ent->t263.queue_type);
count++;
}
}
+ } else if (QLA_TGT_MODE_ENABLED() &&
+ ent->t274.queue_type == T274_QUEUE_TYPE_ATIO_SHAD) {
+ struct qla_hw_data *ha = vha->hw;
+ struct atio *atr = ha->tgt.atio_ring_ptr;
+
+ if (atr || !buf) {
+ qla27xx_insert16(0, buf, len);
+ qla27xx_insert16(1, buf, len);
+ qla27xx_insert32(ha->tgt.atio_q_in ?
+ readl(ha->tgt.atio_q_in) : 0, buf, len);
+ count++;
+ }
} else {
ql_dbg(ql_dbg_misc, vha, 0xd02f,
"%s: unknown queue %x\n", __func__, ent->t274.queue_type);
{
return sprintf(page,
"TCM QLOGIC QLA2XXX NPIV capable fabric module %s on %s/%s on "
- UTS_RELEASE"\n", TCM_QLA2XXX_VERSION, utsname()->sysname,
+ UTS_RELEASE"\n", QLA2XXX_VERSION, utsname()->sysname,
utsname()->machine);
}
int ret;
pr_debug("TCM QLOGIC QLA2XXX fabric module %s on %s/%s on "
- UTS_RELEASE"\n", TCM_QLA2XXX_VERSION, utsname()->sysname,
+ UTS_RELEASE"\n", QLA2XXX_VERSION, utsname()->sysname,
utsname()->machine);
ret = target_register_template(&tcm_qla2xxx_ops);
#include <target/target_core_base.h>
#include <linux/btree.h>
-#define TCM_QLA2XXX_VERSION "v0.1"
/* length of ASCII WWPNs including pad */
#define TCM_QLA2XXX_NAMELEN 32
/*
count = blk_rq_map_sg(req->q, req, sdb->table.sgl);
BUG_ON(count > sdb->table.nents);
sdb->table.nents = count;
- sdb->length = blk_rq_bytes(req);
+ sdb->length = blk_rq_payload_bytes(req);
return BLKPREP_OK;
}
* request queue.
*/
if (q->mq_ops) {
- blk_mq_stop_hw_queues(q);
+ blk_mq_quiesce_queue(q);
} else {
spin_lock_irqsave(q->queue_lock, flags);
blk_stop_queue(q);
struct bio *bio = rq->bio;
sector_t sector = blk_rq_pos(rq);
unsigned int nr_sectors = blk_rq_sectors(rq);
- unsigned int nr_bytes = blk_rq_bytes(rq);
int ret;
if (sdkp->device->no_write_same)
cmd->transfersize = sdp->sector_size;
cmd->allowed = SD_MAX_RETRIES;
-
- /*
- * For WRITE_SAME the data transferred in the DATA IN buffer is
- * different from the amount of data actually written to the target.
- *
- * We set up __data_len to the amount of data transferred from the
- * DATA IN buffer so that blk_rq_map_sg set up the proper S/G list
- * to transfer a single sector of data first, but then reset it to
- * the amount of data to be written right after so that the I/O path
- * knows how much to actually write.
- */
- rq->__data_len = sdp->sector_size;
- ret = scsi_init_io(cmd);
- rq->__data_len = nr_bytes;
- return ret;
+ return scsi_init_io(cmd);
}
static int sd_setup_flush_cmnd(struct scsi_cmnd *cmd)
if (sdp->broken_fua) {
sd_first_printk(KERN_NOTICE, sdkp, "Disabling FUA\n");
sdkp->DPOFUA = 0;
- } else if (sdkp->DPOFUA && !sdkp->device->use_10_for_rw) {
+ } else if (sdkp->DPOFUA && !sdkp->device->use_10_for_rw &&
+ !sdkp->device->use_16_for_rw) {
sd_first_printk(KERN_NOTICE, sdkp,
"Uses READ/WRITE(6), disabling FUA\n");
sdkp->DPOFUA = 0;
queue_flag_clear_unlocked(QUEUE_FLAG_ADD_RANDOM, q);
}
- sdkp->zoned = (buffer[8] >> 4) & 3;
- if (sdkp->zoned == 1)
- q->limits.zoned = BLK_ZONED_HA;
- else if (sdkp->device->type == TYPE_ZBC)
+ if (sdkp->device->type == TYPE_ZBC) {
+ /* Host-managed */
q->limits.zoned = BLK_ZONED_HM;
- else
- q->limits.zoned = BLK_ZONED_NONE;
+ } else {
+ sdkp->zoned = (buffer[8] >> 4) & 3;
+ if (sdkp->zoned == 1)
+ /* Host-aware */
+ q->limits.zoned = BLK_ZONED_HA;
+ else
+ /*
+ * Treat drive-managed devices as
+ * regular block devices.
+ */
+ q->limits.zoned = BLK_ZONED_NONE;
+ }
if (blk_queue_is_zoned(q) && sdkp->first_scan)
sd_printk(KERN_NOTICE, sdkp, "Host-%s zoned block device\n",
q->limits.zoned == BLK_ZONED_HM ? "managed" : "aware");
ses_enclosure_data_process(edev, to_scsi_device(edev->edev.parent), 0);
- if (scsi_is_sas_rphy(&sdev->sdev_gendev))
+ if (scsi_is_sas_rphy(sdev->sdev_target->dev.parent))
efd.addr = sas_get_address(sdev);
if (efd.addr) {
if (!pool) {
SNIC_HOST_ERR(shost, "dflt sgl pool creation failed\n");
+ ret = -ENOMEM;
goto err_free_res;
}
if (!pool) {
SNIC_HOST_ERR(shost, "max sgl pool creation failed\n");
+ ret = -ENOMEM;
goto err_free_dflt_sgl_pool;
}
if (!pool) {
SNIC_HOST_ERR(shost, "snic tmreq info pool creation failed.\n");
+ ret = -ENOMEM;
goto err_free_max_sgl_pool;
}
if (IS_ERR(task)) {
dev_err(dev, "can't create rproc_boot thread\n");
+ ret = PTR_ERR(task);
goto err_put_rproc;
}
config SPI_FSL_DSPI
tristate "Freescale DSPI controller"
select REGMAP_MMIO
+ depends on HAS_DMA
depends on SOC_VF610 || SOC_LS1021A || ARCH_LAYERSCAPE || COMPILE_TEST
help
This enables support for the Freescale DSPI controller in master
struct spi_master *master;
struct a3700_spi *spi;
u32 num_cs = 0;
- int ret = 0;
+ int irq, ret = 0;
master = spi_alloc_master(dev, sizeof(*spi));
if (!master) {
master->unprepare_message = a3700_spi_unprepare_message;
master->set_cs = a3700_spi_set_cs;
master->flags = SPI_MASTER_HALF_DUPLEX;
- master->mode_bits |= (SPI_RX_DUAL | SPI_RX_DUAL |
+ master->mode_bits |= (SPI_RX_DUAL | SPI_TX_DUAL |
SPI_RX_QUAD | SPI_TX_QUAD);
platform_set_drvdata(pdev, master);
goto error;
}
- spi->irq = platform_get_irq(pdev, 0);
- if (spi->irq < 0) {
- dev_err(dev, "could not get irq: %d\n", spi->irq);
+ irq = platform_get_irq(pdev, 0);
+ if (irq < 0) {
+ dev_err(dev, "could not get irq: %d\n", irq);
ret = -ENXIO;
goto error;
}
+ spi->irq = irq;
init_completion(&spi->done);
SPI_ENGINE_VERSION_MAJOR(version),
SPI_ENGINE_VERSION_MINOR(version),
SPI_ENGINE_VERSION_PATCH(version));
- return -ENODEV;
+ ret = -ENODEV;
+ goto err_put_master;
}
spi_engine->clk = devm_clk_get(&pdev->dev, "s_axi_aclk");
buf = t->rx_buf;
t->rx_dma = dma_map_single(&spi->dev, buf,
t->len, DMA_FROM_DEVICE);
- if (!t->rx_dma) {
+ if (dma_mapping_error(&spi->dev, !t->rx_dma)) {
ret = -EFAULT;
goto err_rx_map;
}
buf = (void *)t->tx_buf;
t->tx_dma = dma_map_single(&spi->dev, buf,
t->len, DMA_TO_DEVICE);
- if (!t->tx_dma) {
+ if (dma_mapping_error(&spi->dev, t->tx_dma)) {
ret = -EFAULT;
goto err_tx_map;
}
static void mid_spi_dma_stop(struct dw_spi *dws)
{
if (test_bit(TX_BUSY, &dws->dma_chan_busy)) {
- dmaengine_terminate_all(dws->txchan);
+ dmaengine_terminate_sync(dws->txchan);
clear_bit(TX_BUSY, &dws->dma_chan_busy);
}
if (test_bit(RX_BUSY, &dws->dma_chan_busy)) {
- dmaengine_terminate_all(dws->rxchan);
+ dmaengine_terminate_sync(dws->rxchan);
clear_bit(RX_BUSY, &dws->dma_chan_busy);
}
}
static int dw_spi_debugfs_init(struct dw_spi *dws)
{
- dws->debugfs = debugfs_create_dir("dw_spi", NULL);
+ char name[128];
+
+ snprintf(name, 128, "dw_spi-%s", dev_name(&dws->master->dev));
+ dws->debugfs = debugfs_create_dir(name, NULL);
if (!dws->debugfs)
return -ENOMEM;
pxa2xx_spi_write(drv_data, SSCR1, tmp);
tmp = SSCR0_SCR(2) | SSCR0_Motorola | SSCR0_DataSize(8);
pxa2xx_spi_write(drv_data, SSCR0, tmp);
+ break;
default:
tmp = SSCR1_RxTresh(RX_THRESH_DFLT) |
SSCR1_TxTresh(TX_THRESH_DFLT);
};
static const struct of_device_id sh_msiof_match[] = {
- { .compatible = "renesas,sh-msiof", .data = &sh_data },
{ .compatible = "renesas,sh-mobile-msiof", .data = &sh_data },
{ .compatible = "renesas,msiof-r8a7790", .data = &r8a779x_data },
{ .compatible = "renesas,msiof-r8a7791", .data = &r8a779x_data },
{ .compatible = "renesas,msiof-r8a7792", .data = &r8a779x_data },
{ .compatible = "renesas,msiof-r8a7793", .data = &r8a779x_data },
{ .compatible = "renesas,msiof-r8a7794", .data = &r8a779x_data },
+ { .compatible = "renesas,rcar-gen2-msiof", .data = &r8a779x_data },
{ .compatible = "renesas,msiof-r8a7796", .data = &r8a779x_data },
+ { .compatible = "renesas,rcar-gen3-msiof", .data = &r8a779x_data },
+ { .compatible = "renesas,sh-msiof", .data = &sh_data }, /* Deprecated */
{},
};
MODULE_DEVICE_TABLE(of, sh_msiof_match);
case TCM_LOGICAL_BLOCK_APP_TAG_CHECK_FAILED:
case TCM_LOGICAL_BLOCK_REF_TAG_CHECK_FAILED:
case TCM_COPY_TARGET_DEVICE_NOT_REACHABLE:
+ case TCM_TOO_MANY_TARGET_DESCS:
+ case TCM_UNSUPPORTED_TARGET_DESC_TYPE_CODE:
+ case TCM_TOO_MANY_SEGMENT_DESCS:
+ case TCM_UNSUPPORTED_SEGMENT_DESC_TYPE_CODE:
break;
case TCM_OUT_OF_RESOURCES:
sense_reason = TCM_LOGICAL_UNIT_COMMUNICATION_FAILURE;
.key = ILLEGAL_REQUEST,
.asc = 0x26, /* INVALID FIELD IN PARAMETER LIST */
},
+ [TCM_TOO_MANY_TARGET_DESCS] = {
+ .key = ILLEGAL_REQUEST,
+ .asc = 0x26,
+ .ascq = 0x06, /* TOO MANY TARGET DESCRIPTORS */
+ },
+ [TCM_UNSUPPORTED_TARGET_DESC_TYPE_CODE] = {
+ .key = ILLEGAL_REQUEST,
+ .asc = 0x26,
+ .ascq = 0x07, /* UNSUPPORTED TARGET DESCRIPTOR TYPE CODE */
+ },
+ [TCM_TOO_MANY_SEGMENT_DESCS] = {
+ .key = ILLEGAL_REQUEST,
+ .asc = 0x26,
+ .ascq = 0x08, /* TOO MANY SEGMENT DESCRIPTORS */
+ },
+ [TCM_UNSUPPORTED_SEGMENT_DESC_TYPE_CODE] = {
+ .key = ILLEGAL_REQUEST,
+ .asc = 0x26,
+ .ascq = 0x09, /* UNSUPPORTED SEGMENT DESCRIPTOR TYPE CODE */
+ },
[TCM_PARAMETER_LIST_LENGTH_ERROR] = {
.key = ILLEGAL_REQUEST,
.asc = 0x1a, /* PARAMETER LIST LENGTH ERROR */
return 0;
}
-static int target_xcopy_locate_se_dev_e4(struct se_cmd *se_cmd, struct xcopy_op *xop,
- bool src)
+static int target_xcopy_locate_se_dev_e4(const unsigned char *dev_wwn,
+ struct se_device **found_dev)
{
struct se_device *se_dev;
- unsigned char tmp_dev_wwn[XCOPY_NAA_IEEE_REGEX_LEN], *dev_wwn;
+ unsigned char tmp_dev_wwn[XCOPY_NAA_IEEE_REGEX_LEN];
int rc;
- if (src)
- dev_wwn = &xop->dst_tid_wwn[0];
- else
- dev_wwn = &xop->src_tid_wwn[0];
-
mutex_lock(&g_device_mutex);
list_for_each_entry(se_dev, &g_device_list, g_dev_node) {
if (rc != 0)
continue;
- if (src) {
- xop->dst_dev = se_dev;
- pr_debug("XCOPY 0xe4: Setting xop->dst_dev: %p from located"
- " se_dev\n", xop->dst_dev);
- } else {
- xop->src_dev = se_dev;
- pr_debug("XCOPY 0xe4: Setting xop->src_dev: %p from located"
- " se_dev\n", xop->src_dev);
- }
+ *found_dev = se_dev;
+ pr_debug("XCOPY 0xe4: located se_dev: %p\n", se_dev);
rc = target_depend_item(&se_dev->dev_group.cg_item);
if (rc != 0) {
}
static int target_xcopy_parse_tiddesc_e4(struct se_cmd *se_cmd, struct xcopy_op *xop,
- unsigned char *p, bool src)
+ unsigned char *p, unsigned short cscd_index)
{
unsigned char *desc = p;
unsigned short ript;
return -EINVAL;
}
- if (src) {
+ if (cscd_index != xop->stdi && cscd_index != xop->dtdi) {
+ pr_debug("XCOPY 0xe4: ignoring CSCD entry %d - neither src nor "
+ "dest\n", cscd_index);
+ return 0;
+ }
+
+ if (cscd_index == xop->stdi) {
memcpy(&xop->src_tid_wwn[0], &desc[8], XCOPY_NAA_IEEE_REGEX_LEN);
/*
* Determine if the source designator matches the local device
pr_debug("XCOPY 0xe4: Set xop->src_dev %p from source"
" received xop\n", xop->src_dev);
}
- } else {
+ }
+
+ if (cscd_index == xop->dtdi) {
memcpy(&xop->dst_tid_wwn[0], &desc[8], XCOPY_NAA_IEEE_REGEX_LEN);
/*
- * Determine if the destination designator matches the local device
+ * Determine if the destination designator matches the local
+ * device. If @cscd_index corresponds to both source (stdi) and
+ * destination (dtdi), or dtdi comes after stdi, then
+ * XCOL_DEST_RECV_OP wins.
*/
if (!memcmp(&xop->local_dev_wwn[0], &xop->dst_tid_wwn[0],
XCOPY_NAA_IEEE_REGEX_LEN)) {
{
struct se_device *local_dev = se_cmd->se_dev;
unsigned char *desc = p;
- int offset = tdll % XCOPY_TARGET_DESC_LEN, rc, ret = 0;
+ int offset = tdll % XCOPY_TARGET_DESC_LEN, rc;
+ unsigned short cscd_index = 0;
unsigned short start = 0;
- bool src = true;
*sense_ret = TCM_INVALID_PARAMETER_LIST;
if (offset != 0) {
pr_err("XCOPY target descriptor list length is not"
" multiple of %d\n", XCOPY_TARGET_DESC_LEN);
+ *sense_ret = TCM_UNSUPPORTED_TARGET_DESC_TYPE_CODE;
return -EINVAL;
}
- if (tdll > 64) {
+ if (tdll > RCR_OP_MAX_TARGET_DESC_COUNT * XCOPY_TARGET_DESC_LEN) {
pr_err("XCOPY target descriptor supports a maximum"
" two src/dest descriptors, tdll: %hu too large..\n", tdll);
+ /* spc4r37 6.4.3.4 CSCD DESCRIPTOR LIST LENGTH field */
+ *sense_ret = TCM_TOO_MANY_TARGET_DESCS;
return -EINVAL;
}
/*
while (start < tdll) {
/*
- * Check target descriptor identification with 0xE4 type with
- * use VPD 0x83 WWPN matching ..
+ * Check target descriptor identification with 0xE4 type, and
+ * compare the current index with the CSCD descriptor IDs in
+ * the segment descriptor. Use VPD 0x83 WWPN matching ..
*/
switch (desc[0]) {
case 0xe4:
rc = target_xcopy_parse_tiddesc_e4(se_cmd, xop,
- &desc[0], src);
+ &desc[0], cscd_index);
if (rc != 0)
goto out;
- /*
- * Assume target descriptors are in source -> destination order..
- */
- if (src)
- src = false;
- else
- src = true;
start += XCOPY_TARGET_DESC_LEN;
desc += XCOPY_TARGET_DESC_LEN;
- ret++;
+ cscd_index++;
break;
default:
pr_err("XCOPY unsupported descriptor type code:"
" 0x%02x\n", desc[0]);
+ *sense_ret = TCM_UNSUPPORTED_TARGET_DESC_TYPE_CODE;
goto out;
}
}
- if (xop->op_origin == XCOL_SOURCE_RECV_OP)
- rc = target_xcopy_locate_se_dev_e4(se_cmd, xop, true);
- else
- rc = target_xcopy_locate_se_dev_e4(se_cmd, xop, false);
+ switch (xop->op_origin) {
+ case XCOL_SOURCE_RECV_OP:
+ rc = target_xcopy_locate_se_dev_e4(xop->dst_tid_wwn,
+ &xop->dst_dev);
+ break;
+ case XCOL_DEST_RECV_OP:
+ rc = target_xcopy_locate_se_dev_e4(xop->src_tid_wwn,
+ &xop->src_dev);
+ break;
+ default:
+ pr_err("XCOPY CSCD descriptor IDs not found in CSCD list - "
+ "stdi: %hu dtdi: %hu\n", xop->stdi, xop->dtdi);
+ rc = -EINVAL;
+ break;
+ }
/*
* If a matching IEEE NAA 0x83 descriptor for the requested device
* is not located on this node, return COPY_ABORTED with ASQ/ASQC
pr_debug("XCOPY TGT desc: Dest dev: %p NAA IEEE WWN: 0x%16phN\n",
xop->dst_dev, &xop->dst_tid_wwn[0]);
- return ret;
+ return cscd_index;
out:
return -EINVAL;
xop->stdi = get_unaligned_be16(&desc[4]);
xop->dtdi = get_unaligned_be16(&desc[6]);
+
+ if (xop->stdi > XCOPY_CSCD_DESC_ID_LIST_OFF_MAX ||
+ xop->dtdi > XCOPY_CSCD_DESC_ID_LIST_OFF_MAX) {
+ pr_err("XCOPY segment desc 0x02: unsupported CSCD ID > 0x%x; stdi: %hu dtdi: %hu\n",
+ XCOPY_CSCD_DESC_ID_LIST_OFF_MAX, xop->stdi, xop->dtdi);
+ return -EINVAL;
+ }
+
pr_debug("XCOPY seg desc 0x02: desc_len: %hu stdi: %hu dtdi: %hu, DC: %d\n",
desc_len, xop->stdi, xop->dtdi, dc);
static int target_xcopy_parse_segment_descriptors(struct se_cmd *se_cmd,
struct xcopy_op *xop, unsigned char *p,
- unsigned int sdll)
+ unsigned int sdll, sense_reason_t *sense_ret)
{
unsigned char *desc = p;
unsigned int start = 0;
int offset = sdll % XCOPY_SEGMENT_DESC_LEN, rc, ret = 0;
+ *sense_ret = TCM_INVALID_PARAMETER_LIST;
+
if (offset != 0) {
pr_err("XCOPY segment descriptor list length is not"
" multiple of %d\n", XCOPY_SEGMENT_DESC_LEN);
+ *sense_ret = TCM_UNSUPPORTED_SEGMENT_DESC_TYPE_CODE;
+ return -EINVAL;
+ }
+ if (sdll > RCR_OP_MAX_SG_DESC_COUNT * XCOPY_SEGMENT_DESC_LEN) {
+ pr_err("XCOPY supports %u segment descriptor(s), sdll: %u too"
+ " large..\n", RCR_OP_MAX_SG_DESC_COUNT, sdll);
+ /* spc4r37 6.4.3.5 SEGMENT DESCRIPTOR LIST LENGTH field */
+ *sense_ret = TCM_TOO_MANY_SEGMENT_DESCS;
return -EINVAL;
}
default:
pr_err("XCOPY unsupported segment descriptor"
"type: 0x%02x\n", desc[0]);
+ *sense_ret = TCM_UNSUPPORTED_SEGMENT_DESC_TYPE_CODE;
goto out;
}
}
return TCM_UNSUPPORTED_SCSI_OPCODE;
}
+ if (se_cmd->data_length == 0) {
+ target_complete_cmd(se_cmd, SAM_STAT_GOOD);
+ return TCM_NO_SENSE;
+ }
+ if (se_cmd->data_length < XCOPY_HDR_LEN) {
+ pr_err("XCOPY parameter truncation: length %u < hdr_len %u\n",
+ se_cmd->data_length, XCOPY_HDR_LEN);
+ return TCM_PARAMETER_LIST_LENGTH_ERROR;
+ }
+
xop = kzalloc(sizeof(struct xcopy_op), GFP_KERNEL);
if (!xop) {
pr_err("Unable to allocate xcopy_op\n");
*/
tdll = get_unaligned_be16(&p[2]);
sdll = get_unaligned_be32(&p[8]);
+ if (tdll + sdll > RCR_OP_MAX_DESC_LIST_LEN) {
+ pr_err("XCOPY descriptor list length %u exceeds maximum %u\n",
+ tdll + sdll, RCR_OP_MAX_DESC_LIST_LEN);
+ ret = TCM_PARAMETER_LIST_LENGTH_ERROR;
+ goto out;
+ }
inline_dl = get_unaligned_be32(&p[12]);
if (inline_dl != 0) {
goto out;
}
+ if (se_cmd->data_length < (XCOPY_HDR_LEN + tdll + sdll + inline_dl)) {
+ pr_err("XCOPY parameter truncation: data length %u too small "
+ "for tdll: %hu sdll: %u inline_dl: %u\n",
+ se_cmd->data_length, tdll, sdll, inline_dl);
+ ret = TCM_PARAMETER_LIST_LENGTH_ERROR;
+ goto out;
+ }
+
pr_debug("Processing XCOPY with list_id: 0x%02x list_id_usage: 0x%02x"
" tdll: %hu sdll: %u inline_dl: %u\n", list_id, list_id_usage,
tdll, sdll, inline_dl);
+ /*
+ * skip over the target descriptors until segment descriptors
+ * have been passed - CSCD ids are needed to determine src and dest.
+ */
+ seg_desc = &p[16] + tdll;
+
+ rc = target_xcopy_parse_segment_descriptors(se_cmd, xop, seg_desc,
+ sdll, &ret);
+ if (rc <= 0)
+ goto out;
+
+ pr_debug("XCOPY: Processed %d segment descriptors, length: %u\n", rc,
+ rc * XCOPY_SEGMENT_DESC_LEN);
+
rc = target_xcopy_parse_target_descriptors(se_cmd, xop, &p[16], tdll, &ret);
if (rc <= 0)
goto out;
pr_debug("XCOPY: Processed %d target descriptors, length: %u\n", rc,
rc * XCOPY_TARGET_DESC_LEN);
- seg_desc = &p[16];
- seg_desc += (rc * XCOPY_TARGET_DESC_LEN);
-
- rc = target_xcopy_parse_segment_descriptors(se_cmd, xop, seg_desc, sdll);
- if (rc <= 0) {
- xcopy_pt_undepend_remotedev(xop);
- goto out;
- }
transport_kunmap_data_sg(se_cmd);
- pr_debug("XCOPY: Processed %d segment descriptors, length: %u\n", rc,
- rc * XCOPY_SEGMENT_DESC_LEN);
INIT_WORK(&xop->xop_work, target_xcopy_do_work);
queue_work(xcopy_wq, &xop->xop_work);
return TCM_NO_SENSE;
#include <target/target_core_base.h>
+#define XCOPY_HDR_LEN 16
#define XCOPY_TARGET_DESC_LEN 32
#define XCOPY_SEGMENT_DESC_LEN 28
#define XCOPY_NAA_IEEE_REGEX_LEN 16
#define XCOPY_MAX_SECTORS 1024
+/*
+ * SPC4r37 6.4.6.1
+ * Table 150 — CSCD descriptor ID values
+ */
+#define XCOPY_CSCD_DESC_ID_LIST_OFF_MAX 0x07FF
+
enum xcopy_origin_list {
XCOL_SOURCE_RECV_OP = 0x01,
XCOL_DEST_RECV_OP = 0x02,
void (*control)(void __iomem *reg, bool on);
/* Per-sensor methods */
- int (*get_temp)(struct chip_tsadc_table table,
+ int (*get_temp)(const struct chip_tsadc_table *table,
int chn, void __iomem *reg, int *temp);
- void (*set_alarm_temp)(struct chip_tsadc_table table,
- int chn, void __iomem *reg, int temp);
- void (*set_tshut_temp)(struct chip_tsadc_table table,
- int chn, void __iomem *reg, int temp);
+ int (*set_alarm_temp)(const struct chip_tsadc_table *table,
+ int chn, void __iomem *reg, int temp);
+ int (*set_tshut_temp)(const struct chip_tsadc_table *table,
+ int chn, void __iomem *reg, int temp);
void (*set_tshut_mode)(int chn, void __iomem *reg, enum tshut_mode m);
/* Per-table methods */
{3452, 115000},
{3437, 120000},
{3421, 125000},
+ {0, 125000},
};
static const struct tsadc_table rk3368_code_table[] = {
{TSADCV3_DATA_MASK, 125000},
};
-static u32 rk_tsadcv2_temp_to_code(struct chip_tsadc_table table,
+static u32 rk_tsadcv2_temp_to_code(const struct chip_tsadc_table *table,
int temp)
{
int high, low, mid;
- u32 error = 0;
+ unsigned long num;
+ unsigned int denom;
+ u32 error = table->data_mask;
low = 0;
- high = table.length - 1;
+ high = (table->length - 1) - 1; /* ignore the last check for table */
mid = (high + low) / 2;
/* Return mask code data when the temp is over table range */
- if (temp < table.id[low].temp || temp > table.id[high].temp) {
- error = table.data_mask;
+ if (temp < table->id[low].temp || temp > table->id[high].temp)
goto exit;
- }
while (low <= high) {
- if (temp == table.id[mid].temp)
- return table.id[mid].code;
- else if (temp < table.id[mid].temp)
+ if (temp == table->id[mid].temp)
+ return table->id[mid].code;
+ else if (temp < table->id[mid].temp)
high = mid - 1;
else
low = mid + 1;
mid = (low + high) / 2;
}
+ /*
+ * The conversion code granularity provided by the table. Let's
+ * assume that the relationship between temperature and
+ * analog value between 2 table entries is linear and interpolate
+ * to produce less granular result.
+ */
+ num = abs(table->id[mid + 1].code - table->id[mid].code);
+ num *= temp - table->id[mid].temp;
+ denom = table->id[mid + 1].temp - table->id[mid].temp;
+
+ switch (table->mode) {
+ case ADC_DECREMENT:
+ return table->id[mid].code - (num / denom);
+ case ADC_INCREMENT:
+ return table->id[mid].code + (num / denom);
+ default:
+ pr_err("%s: unknown table mode: %d\n", __func__, table->mode);
+ return error;
+ }
+
exit:
- pr_err("Invalid the conversion, error=%d\n", error);
+ pr_err("%s: invalid temperature, temp=%d error=%d\n",
+ __func__, temp, error);
return error;
}
-static int rk_tsadcv2_code_to_temp(struct chip_tsadc_table table, u32 code,
- int *temp)
+static int rk_tsadcv2_code_to_temp(const struct chip_tsadc_table *table,
+ u32 code, int *temp)
{
unsigned int low = 1;
- unsigned int high = table.length - 1;
+ unsigned int high = table->length - 1;
unsigned int mid = (low + high) / 2;
unsigned int num;
unsigned long denom;
- WARN_ON(table.length < 2);
+ WARN_ON(table->length < 2);
- switch (table.mode) {
+ switch (table->mode) {
case ADC_DECREMENT:
- code &= table.data_mask;
- if (code < table.id[high].code)
+ code &= table->data_mask;
+ if (code <= table->id[high].code)
return -EAGAIN; /* Incorrect reading */
while (low <= high) {
- if (code >= table.id[mid].code &&
- code < table.id[mid - 1].code)
+ if (code >= table->id[mid].code &&
+ code < table->id[mid - 1].code)
break;
- else if (code < table.id[mid].code)
+ else if (code < table->id[mid].code)
low = mid + 1;
else
high = mid - 1;
}
break;
case ADC_INCREMENT:
- code &= table.data_mask;
- if (code < table.id[low].code)
+ code &= table->data_mask;
+ if (code < table->id[low].code)
return -EAGAIN; /* Incorrect reading */
while (low <= high) {
- if (code <= table.id[mid].code &&
- code > table.id[mid - 1].code)
+ if (code <= table->id[mid].code &&
+ code > table->id[mid - 1].code)
break;
- else if (code > table.id[mid].code)
+ else if (code > table->id[mid].code)
low = mid + 1;
else
high = mid - 1;
}
break;
default:
- pr_err("Invalid the conversion table\n");
+ pr_err("%s: unknown table mode: %d\n", __func__, table->mode);
+ return -EINVAL;
}
/*
* temperature between 2 table entries is linear and interpolate
* to produce less granular result.
*/
- num = table.id[mid].temp - table.id[mid - 1].temp;
- num *= abs(table.id[mid - 1].code - code);
- denom = abs(table.id[mid - 1].code - table.id[mid].code);
- *temp = table.id[mid - 1].temp + (num / denom);
+ num = table->id[mid].temp - table->id[mid - 1].temp;
+ num *= abs(table->id[mid - 1].code - code);
+ denom = abs(table->id[mid - 1].code - table->id[mid].code);
+ *temp = table->id[mid - 1].temp + (num / denom);
return 0;
}
writel_relaxed(val, regs + TSADCV2_AUTO_CON);
}
-static int rk_tsadcv2_get_temp(struct chip_tsadc_table table,
+static int rk_tsadcv2_get_temp(const struct chip_tsadc_table *table,
int chn, void __iomem *regs, int *temp)
{
u32 val;
return rk_tsadcv2_code_to_temp(table, val, temp);
}
-static void rk_tsadcv2_alarm_temp(struct chip_tsadc_table table,
- int chn, void __iomem *regs, int temp)
+static int rk_tsadcv2_alarm_temp(const struct chip_tsadc_table *table,
+ int chn, void __iomem *regs, int temp)
{
- u32 alarm_value, int_en;
+ u32 alarm_value;
+ u32 int_en, int_clr;
+
+ /*
+ * In some cases, some sensors didn't need the trip points, the
+ * set_trips will pass {-INT_MAX, INT_MAX} to trigger tsadc alarm
+ * in the end, ignore this case and disable the high temperature
+ * interrupt.
+ */
+ if (temp == INT_MAX) {
+ int_clr = readl_relaxed(regs + TSADCV2_INT_EN);
+ int_clr &= ~TSADCV2_INT_SRC_EN(chn);
+ writel_relaxed(int_clr, regs + TSADCV2_INT_EN);
+ return 0;
+ }
/* Make sure the value is valid */
alarm_value = rk_tsadcv2_temp_to_code(table, temp);
- if (alarm_value == table.data_mask)
- return;
+ if (alarm_value == table->data_mask)
+ return -ERANGE;
- writel_relaxed(alarm_value & table.data_mask,
+ writel_relaxed(alarm_value & table->data_mask,
regs + TSADCV2_COMP_INT(chn));
int_en = readl_relaxed(regs + TSADCV2_INT_EN);
int_en |= TSADCV2_INT_SRC_EN(chn);
writel_relaxed(int_en, regs + TSADCV2_INT_EN);
+
+ return 0;
}
-static void rk_tsadcv2_tshut_temp(struct chip_tsadc_table table,
- int chn, void __iomem *regs, int temp)
+static int rk_tsadcv2_tshut_temp(const struct chip_tsadc_table *table,
+ int chn, void __iomem *regs, int temp)
{
u32 tshut_value, val;
/* Make sure the value is valid */
tshut_value = rk_tsadcv2_temp_to_code(table, temp);
- if (tshut_value == table.data_mask)
- return;
+ if (tshut_value == table->data_mask)
+ return -ERANGE;
writel_relaxed(tshut_value, regs + TSADCV2_COMP_SHUT(chn));
/* TSHUT will be valid */
val = readl_relaxed(regs + TSADCV2_AUTO_CON);
writel_relaxed(val | TSADCV2_AUTO_SRC_EN(chn), regs + TSADCV2_AUTO_CON);
+
+ return 0;
}
static void rk_tsadcv2_tshut_mode(int chn, void __iomem *regs,
dev_dbg(&thermal->pdev->dev, "%s: sensor %d: low: %d, high %d\n",
__func__, sensor->id, low, high);
- tsadc->set_alarm_temp(tsadc->table,
- sensor->id, thermal->regs, high);
-
- return 0;
+ return tsadc->set_alarm_temp(&tsadc->table,
+ sensor->id, thermal->regs, high);
}
static int rockchip_thermal_get_temp(void *_sensor, int *out_temp)
const struct rockchip_tsadc_chip *tsadc = sensor->thermal->chip;
int retval;
- retval = tsadc->get_temp(tsadc->table,
+ retval = tsadc->get_temp(&tsadc->table,
sensor->id, thermal->regs, out_temp);
dev_dbg(&thermal->pdev->dev, "sensor %d - temp: %d, retval: %d\n",
sensor->id, *out_temp, retval);
int error;
tsadc->set_tshut_mode(id, thermal->regs, thermal->tshut_mode);
- tsadc->set_tshut_temp(tsadc->table, id, thermal->regs,
+
+ error = tsadc->set_tshut_temp(&tsadc->table, id, thermal->regs,
thermal->tshut_temp);
+ if (error)
+ dev_err(&pdev->dev, "%s: invalid tshut=%d, error=%d\n",
+ __func__, thermal->tshut_temp, error);
sensor->thermal = thermal;
sensor->id = id;
thermal->chip->set_tshut_mode(id, thermal->regs,
thermal->tshut_mode);
- thermal->chip->set_tshut_temp(thermal->chip->table,
+
+ error = thermal->chip->set_tshut_temp(&thermal->chip->table,
id, thermal->regs,
thermal->tshut_temp);
+ if (error)
+ dev_err(&pdev->dev, "%s: invalid tshut=%d, error=%d\n",
+ __func__, thermal->tshut_temp, error);
}
thermal->chip->control(thermal->regs, true);
if (!strncmp(dev_name(dev), "thermal_zone",
sizeof("thermal_zone") - 1)) {
tz = to_thermal_zone(dev);
+ kfree(tz->trip_type_attrs);
+ kfree(tz->trip_temp_attrs);
+ kfree(tz->trip_hyst_attrs);
+ kfree(tz->trips_attribute_group.attrs);
+ kfree(tz->device.groups);
kfree(tz);
} else if (!strncmp(dev_name(dev), "cooling_device",
sizeof("cooling_device") - 1)) {
thermal_zone_device_set_polling(tz, 0);
- kfree(tz->trip_type_attrs);
- kfree(tz->trip_temp_attrs);
- kfree(tz->trip_hyst_attrs);
- kfree(tz->trips_attribute_group.attrs);
thermal_set_governor(tz, NULL);
thermal_remove_hwmon_sysfs(tz);
idr_destroy(&tz->idr);
mutex_destroy(&tz->lock);
device_unregister(&tz->device);
- kfree(tz->device.groups);
}
EXPORT_SYMBOL_GPL(thermal_zone_device_unregister);
static DEFINE_MUTEX(thermal_hwmon_list_lock);
-static ssize_t
-name_show(struct device *dev, struct device_attribute *attr, char *buf)
-{
- struct thermal_hwmon_device *hwmon = dev_get_drvdata(dev);
- return sprintf(buf, "%s\n", hwmon->type);
-}
-static DEVICE_ATTR_RO(name);
-
static ssize_t
temp_input_show(struct device *dev, struct device_attribute *attr, char *buf)
{
INIT_LIST_HEAD(&hwmon->tz_list);
strlcpy(hwmon->type, tz->type, THERMAL_NAME_LENGTH);
- hwmon->device = hwmon_device_register(NULL);
+ hwmon->device = hwmon_device_register_with_info(NULL, hwmon->type,
+ hwmon, NULL, NULL);
if (IS_ERR(hwmon->device)) {
result = PTR_ERR(hwmon->device);
goto free_mem;
}
- dev_set_drvdata(hwmon->device, hwmon);
- result = device_create_file(hwmon->device, &dev_attr_name);
- if (result)
- goto free_mem;
register_sys_interface:
temp = kzalloc(sizeof(*temp), GFP_KERNEL);
free_temp_mem:
kfree(temp);
unregister_name:
- if (new_hwmon_device) {
- device_remove_file(hwmon->device, &dev_attr_name);
+ if (new_hwmon_device)
hwmon_device_unregister(hwmon->device);
- }
free_mem:
if (new_hwmon_device)
kfree(hwmon);
list_del(&hwmon->node);
mutex_unlock(&thermal_hwmon_list_lock);
- device_remove_file(hwmon->device, &dev_attr_name);
hwmon_device_unregister(hwmon->device);
kfree(hwmon);
}
.device = uart_console_device,
.setup = univ8250_console_setup,
.match = univ8250_console_match,
- .flags = CON_PRINTBUFFER | CON_ANYTIME | CON_CONSDEV,
+ .flags = CON_PRINTBUFFER | CON_ANYTIME,
.index = -1,
.data = &serial8250_reg,
};
static void serial8250_io_resume(struct pci_dev *dev)
{
struct serial_private *priv = pci_get_drvdata(dev);
- const struct pciserial_board *board;
+ struct serial_private *new;
if (!priv)
return;
- board = priv->board;
- kfree(priv);
- priv = pciserial_init_ports(dev, board);
-
- if (!IS_ERR(priv)) {
- pci_set_drvdata(dev, priv);
+ new = pciserial_init_ports(dev, priv->board);
+ if (!IS_ERR(new)) {
+ pci_set_drvdata(dev, new);
+ kfree(priv);
}
}
* Enable previously disabled RX interrupts.
*/
if (!(p->port.rs485.flags & SER_RS485_RX_DURING_TX)) {
- serial8250_clear_fifos(p);
+ serial8250_clear_and_reinit_fifos(p);
p->ier |= UART_IER_RLSI | UART_IER_RDI;
serial_port_out(&p->port, UART_IER, p->ier);
/* disable PDC transmit */
atmel_uart_writel(port, ATMEL_PDC_PTCR, ATMEL_PDC_TXTDIS);
}
+
+ /*
+ * Disable the transmitter.
+ * This is mandatory when DMA is used, otherwise the DMA buffer
+ * is fully transmitted.
+ */
+ atmel_uart_writel(port, ATMEL_US_CR, ATMEL_US_TXDIS);
+
/* Disable interrupts */
atmel_uart_writel(port, ATMEL_US_IDR, atmel_port->tx_done_mask);
/* Enable interrupts */
atmel_uart_writel(port, ATMEL_US_IER, atmel_port->tx_done_mask);
+
+ /* re-enable the transmitter */
+ atmel_uart_writel(port, ATMEL_US_CR, ATMEL_US_TXEN);
}
/*
*/
if (!uart_circ_empty(xmit))
atmel_tasklet_schedule(atmel_port, &atmel_port->tasklet_tx);
+ else if ((port->rs485.flags & SER_RS485_ENABLED) &&
+ !(port->rs485.flags & SER_RS485_RX_DURING_TX)) {
+ /* DMA done, stop TX, start RX for RS485 */
+ atmel_start_rx(port);
+ }
spin_unlock_irqrestore(&port->lock, flags);
}
desc->callback = atmel_complete_tx_dma;
desc->callback_param = atmel_port;
atmel_port->cookie_tx = dmaengine_submit(desc);
-
- } else {
- if (port->rs485.flags & SER_RS485_ENABLED) {
- /* DMA done, stop TX, start RX for RS485 */
- atmel_start_rx(port);
- }
}
if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
{
.flags = INPUT_DEVICE_ID_MATCH_EVBIT |
INPUT_DEVICE_ID_MATCH_KEYBIT,
- .evbit = { BIT_MASK(EV_KEY) },
- .keybit = { BIT_MASK(KEY_LEFTALT) },
+ .evbit = { [BIT_WORD(EV_KEY)] = BIT_MASK(EV_KEY) },
+ .keybit = { [BIT_WORD(KEY_LEFTALT)] = BIT_MASK(KEY_LEFTALT) },
},
{ },
};
/* Gadget parameters */
bool g_dma;
bool g_dma_desc;
- u16 g_rx_fifo_size;
- u16 g_np_tx_fifo_size;
+ u32 g_rx_fifo_size;
+ u32 g_np_tx_fifo_size;
u32 g_tx_fifo_size[MAX_EPS_CHANNELS];
};
/* keep other bits untouched (so e.g. forced modes are not lost) */
usbcfg = dwc2_readl(hsotg->regs + GUSBCFG);
usbcfg &= ~(GUSBCFG_TOUTCAL_MASK | GUSBCFG_PHYIF16 | GUSBCFG_SRPCAP |
- GUSBCFG_HNPCAP);
+ GUSBCFG_HNPCAP | GUSBCFG_USBTRDTIM_MASK);
if (hsotg->params.phy_type == DWC2_PHY_TYPE_PARAM_FS &&
(hsotg->params.speed == DWC2_SPEED_PARAM_FULL ||
__func__, epctrl, epctrl_reg);
/* Allocate DMA descriptor chain for non-ctrl endpoints */
- if (using_desc_dma(hsotg)) {
- hs_ep->desc_list = dma_alloc_coherent(hsotg->dev,
+ if (using_desc_dma(hsotg) && !hs_ep->desc_list) {
+ hs_ep->desc_list = dmam_alloc_coherent(hsotg->dev,
MAX_DMA_DESC_NUM_GENERIC *
sizeof(struct dwc2_dma_desc),
&hs_ep->desc_list_dma, GFP_ATOMIC);
error2:
if (ret && using_desc_dma(hsotg) && hs_ep->desc_list) {
- dma_free_coherent(hsotg->dev, MAX_DMA_DESC_NUM_GENERIC *
+ dmam_free_coherent(hsotg->dev, MAX_DMA_DESC_NUM_GENERIC *
sizeof(struct dwc2_dma_desc),
hs_ep->desc_list, hs_ep->desc_list_dma);
hs_ep->desc_list = NULL;
return -EINVAL;
}
- /* Remove DMA memory allocated for non-control Endpoints */
- if (using_desc_dma(hsotg)) {
- dma_free_coherent(hsotg->dev, MAX_DMA_DESC_NUM_GENERIC *
- sizeof(struct dwc2_dma_desc),
- hs_ep->desc_list, hs_ep->desc_list_dma);
- hs_ep->desc_list = NULL;
- }
-
epctrl_reg = dir_in ? DIEPCTL(index) : DOEPCTL(index);
spin_lock_irqsave(&hsotg->lock, flags);
/* keep other bits untouched (so e.g. forced modes are not lost) */
usbcfg = dwc2_readl(hsotg->regs + GUSBCFG);
usbcfg &= ~(GUSBCFG_TOUTCAL_MASK | GUSBCFG_PHYIF16 | GUSBCFG_SRPCAP |
- GUSBCFG_HNPCAP);
+ GUSBCFG_HNPCAP | GUSBCFG_USBTRDTIM_MASK);
/* set the PLL on, remove the HNP/SRP and set the PHY */
trdtim = (hsotg->phyif == GUSBCFG_PHYIF8) ? 9 : 5;
if (!HCD_HW_ACCESSIBLE(hcd))
goto unlock;
+ if (hsotg->op_state == OTG_STATE_B_PERIPHERAL)
+ goto unlock;
+
if (!hsotg->params.hibernation)
goto skip_power_saving;
{
#ifdef VERBOSE_DEBUG
struct dwc2_hsotg *hsotg = dwc2_hcd_to_hsotg(hcd);
- char *pipetype;
- char *speed;
+ char *pipetype = NULL;
+ char *speed = NULL;
dev_vdbg(hsotg->dev, "%s, urb %p\n", fn_name, urb);
dev_vdbg(hsotg->dev, " Device address: %d\n",
}
/**
- * dwc2_set_param_u16() - Set a u16 parameter
+ * dwc2_set_param_u32() - Set a u32 parameter
*
* See dwc2_set_param().
*/
-static void dwc2_set_param_u16(struct dwc2_hsotg *hsotg, u16 *param,
+static void dwc2_set_param_u32(struct dwc2_hsotg *hsotg, u32 *param,
bool lookup, char *property, u16 legacy,
u16 def, u16 min, u16 max)
{
dwc2_set_param(hsotg, param, lookup, property,
- legacy, def, min, max, 2);
+ legacy, def, min, max, 4);
}
/**
* auto-detect if the hardware does not support the
* default.
*/
- dwc2_set_param_u16(hsotg, &p->g_rx_fifo_size,
+ dwc2_set_param_u32(hsotg, &p->g_rx_fifo_size,
true, "g-rx-fifo-size", 2048,
hw->rx_fifo_size,
16, hw->rx_fifo_size);
- dwc2_set_param_u16(hsotg, &p->g_np_tx_fifo_size,
+ dwc2_set_param_u32(hsotg, &p->g_np_tx_fifo_size,
true, "g-np-tx-fifo-size", 1024,
hw->dev_nperio_tx_fifo_size,
16, hw->dev_nperio_tx_fifo_size);
exynos->axius_clk = devm_clk_get(dev, "usbdrd30_axius_clk");
if (IS_ERR(exynos->axius_clk)) {
dev_err(dev, "no AXI UpScaler clk specified\n");
- return -ENODEV;
+ ret = -ENODEV;
+ goto axius_clk_err;
}
clk_prepare_enable(exynos->axius_clk);
} else {
regulator_disable(exynos->vdd33);
err2:
clk_disable_unprepare(exynos->axius_clk);
+axius_clk_err:
clk_disable_unprepare(exynos->susp_clk);
clk_disable_unprepare(exynos->clk);
return ret;
cdev->os_desc_req->buf = kmalloc(4096, GFP_KERNEL);
if (!cdev->os_desc_req->buf) {
ret = -ENOMEM;
- kfree(cdev->os_desc_req);
+ usb_ep_free_request(ep0, cdev->os_desc_req);
goto end;
}
cdev->os_desc_req->context = cdev;
unsigned long flags;
spin_lock_irqsave(&func->ffs->eps_lock, flags);
- do {
+ while (count--) {
/* pending requests get nuked */
if (likely(ep->ep))
usb_ep_disable(ep->ep);
__ffs_epfile_read_buffer_free(epfile);
++epfile;
}
- } while (--count);
+ }
spin_unlock_irqrestore(&func->ffs->eps_lock, flags);
}
int ret = 0;
spin_lock_irqsave(&func->ffs->eps_lock, flags);
- do {
+ while(count--) {
struct usb_endpoint_descriptor *ds;
int desc_idx;
++ep;
++epfile;
- } while (--count);
+ }
spin_unlock_irqrestore(&func->ffs->eps_lock, flags);
return ret;
/* cleanup after autoconfig */
spin_lock_irqsave(&func->ffs->eps_lock, flags);
- do {
+ while (count--) {
if (ep->ep && ep->req)
usb_ep_free_request(ep->ep, ep->req);
ep->req = NULL;
++ep;
- } while (--count);
+ }
spin_unlock_irqrestore(&func->ffs->eps_lock, flags);
kfree(func->eps);
func->eps = NULL;
dev_err(&pdev->dev, "of_probe: name error(%d)\n", ret);
goto err;
}
- ep->ep.name = kasprintf(GFP_KERNEL, "ep%d", ep->index);
+ sprintf(ep->name, "ep%d", ep->index);
+ ep->ep.name = ep->name;
ep->ep_regs = udc->regs + USBA_EPT_BASE(i);
ep->dma_regs = udc->regs + USBA_DMA_BASE(i);
void __iomem *ep_regs;
void __iomem *dma_regs;
void __iomem *fifo;
+ char name[8];
struct usb_ep ep;
struct usba_udc *udc;
return -ENODEV;
/* Try to set 64-bit DMA first */
- if (WARN_ON(!pdev->dev.dma_mask))
+ if (!pdev->dev.dma_mask)
/* Platform did not initialize dma_mask */
ret = dma_coerce_mask_and_coherent(&pdev->dev,
DMA_BIT_MASK(64));
spin_lock_irqsave(&xhci->lock, flags);
ep->stop_cmds_pending--;
- if (xhci->xhc_state & XHCI_STATE_REMOVING) {
- spin_unlock_irqrestore(&xhci->lock, flags);
- return;
- }
- if (xhci->xhc_state & XHCI_STATE_DYING) {
- xhci_dbg_trace(xhci, trace_xhci_dbg_cancel_urb,
- "Stop EP timer ran, but another timer marked "
- "xHCI as DYING, exiting.");
- spin_unlock_irqrestore(&xhci->lock, flags);
- return;
- }
if (!(ep->stop_cmds_pending == 0 && (ep->ep_state & EP_HALT_PENDING))) {
xhci_dbg_trace(xhci, trace_xhci_dbg_cancel_urb,
"Stop EP timer ran, but no command pending, "
xhci_urb_free_priv(urb_priv);
return ret;
}
- if ((xhci->xhc_state & XHCI_STATE_DYING) ||
- (xhci->xhc_state & XHCI_STATE_HALTED)) {
- xhci_dbg_trace(xhci, trace_xhci_dbg_cancel_urb,
- "Ep 0x%x: URB %p to be canceled on "
- "non-responsive xHCI host.",
- urb->ep->desc.bEndpointAddress, urb);
- /* Let the stop endpoint command watchdog timer (which set this
- * state) finish cleaning up the endpoint TD lists. We must
- * have caught it in the middle of dropping a lock and giving
- * back an URB.
- */
- goto done;
- }
ep_index = xhci_get_endpoint_index(&urb->ep->desc);
ep = &xhci->devs[urb->dev->slot_id]->eps[ep_index];
unsigned i;
seq_printf(s, "MUSB (M)HDRC Register Dump\n");
+ pm_runtime_get_sync(musb->controller);
for (i = 0; i < ARRAY_SIZE(musb_regmap); i++) {
switch (musb_regmap[i].size) {
}
}
+ pm_runtime_mark_last_busy(musb->controller);
+ pm_runtime_put_autosuspend(musb->controller);
return 0;
}
struct musb *musb = s->private;
unsigned test;
+ pm_runtime_get_sync(musb->controller);
test = musb_readb(musb->mregs, MUSB_TESTMODE);
+ pm_runtime_mark_last_busy(musb->controller);
+ pm_runtime_put_autosuspend(musb->controller);
if (test & MUSB_TEST_FORCE_HOST)
seq_printf(s, "force host\n");
u8 test;
char buf[18];
+ pm_runtime_get_sync(musb->controller);
test = musb_readb(musb->mregs, MUSB_TESTMODE);
if (test) {
dev_err(musb->controller, "Error: test mode is already set. "
"Please do USB Bus Reset to start a new test.\n");
- return count;
+ goto ret;
}
memset(buf, 0x00, sizeof(buf));
musb_writeb(musb->mregs, MUSB_TESTMODE, test);
+ret:
+ pm_runtime_mark_last_busy(musb->controller);
+ pm_runtime_put_autosuspend(musb->controller);
return count;
}
switch (musb->xceiv->otg->state) {
case OTG_STATE_A_HOST:
case OTG_STATE_A_WAIT_BCON:
+ pm_runtime_get_sync(musb->controller);
+
reg = musb_readb(musb->mregs, MUSB_DEVCTL);
connect = reg & MUSB_DEVCTL_SESSION ? 1 : 0;
+
+ pm_runtime_mark_last_busy(musb->controller);
+ pm_runtime_put_autosuspend(musb->controller);
break;
default:
connect = -1;
if (copy_from_user(&buf, ubuf, min_t(size_t, sizeof(buf) - 1, count)))
return -EFAULT;
+ pm_runtime_get_sync(musb->controller);
if (!strncmp(buf, "0", 1)) {
switch (musb->xceiv->otg->state) {
case OTG_STATE_A_HOST:
}
}
+ pm_runtime_mark_last_busy(musb->controller);
+ pm_runtime_put_autosuspend(musb->controller);
return count;
}
unsigned baud_rate; /* set baud rate */
u8 line_control; /* set line control value RTS/DTR */
u8 line_status; /* active status of modem control inputs */
+ u8 lcr;
};
static void ch341_set_termios(struct tty_struct *tty,
r = usb_control_msg(dev, usb_sndctrlpipe(dev, 0), request,
USB_TYPE_VENDOR | USB_RECIP_DEVICE | USB_DIR_OUT,
value, index, NULL, 0, DEFAULT_TIMEOUT);
+ if (r < 0)
+ dev_err(&dev->dev, "failed to send control message: %d\n", r);
return r;
}
r = usb_control_msg(dev, usb_rcvctrlpipe(dev, 0), request,
USB_TYPE_VENDOR | USB_RECIP_DEVICE | USB_DIR_IN,
value, index, buf, bufsize, DEFAULT_TIMEOUT);
- return r;
+ if (r < bufsize) {
+ if (r >= 0) {
+ dev_err(&dev->dev,
+ "short control message received (%d < %u)\n",
+ r, bufsize);
+ r = -EIO;
+ }
+
+ dev_err(&dev->dev, "failed to receive control message: %d\n",
+ r);
+ return r;
+ }
+
+ return 0;
}
-static int ch341_init_set_baudrate(struct usb_device *dev,
- struct ch341_private *priv, unsigned ctrl)
+static int ch341_set_baudrate_lcr(struct usb_device *dev,
+ struct ch341_private *priv, u8 lcr)
{
short a;
int r;
factor = 0x10000 - factor;
a = (factor & 0xff00) | divisor;
- /* 0x9c is "enable SFR_UART Control register and timer" */
- r = ch341_control_out(dev, CH341_REQ_SERIAL_INIT,
- 0x9c | (ctrl << 8), a | 0x80);
+ /*
+ * CH341A buffers data until a full endpoint-size packet (32 bytes)
+ * has been received unless bit 7 is set.
+ */
+ a |= BIT(7);
+
+ r = ch341_control_out(dev, CH341_REQ_WRITE_REG, 0x1312, a);
+ if (r)
+ return r;
+
+ r = ch341_control_out(dev, CH341_REQ_WRITE_REG, 0x2518, lcr);
+ if (r)
+ return r;
return r;
}
static int ch341_get_status(struct usb_device *dev, struct ch341_private *priv)
{
+ const unsigned int size = 2;
char *buffer;
int r;
- const unsigned size = 8;
unsigned long flags;
buffer = kmalloc(size, GFP_KERNEL);
if (r < 0)
goto out;
- /* setup the private status if available */
- if (r == 2) {
- r = 0;
- spin_lock_irqsave(&priv->lock, flags);
- priv->line_status = (~(*buffer)) & CH341_BITS_MODEM_STAT;
- spin_unlock_irqrestore(&priv->lock, flags);
- } else
- r = -EPROTO;
+ spin_lock_irqsave(&priv->lock, flags);
+ priv->line_status = (~(*buffer)) & CH341_BITS_MODEM_STAT;
+ spin_unlock_irqrestore(&priv->lock, flags);
out: kfree(buffer);
return r;
static int ch341_configure(struct usb_device *dev, struct ch341_private *priv)
{
+ const unsigned int size = 2;
char *buffer;
int r;
- const unsigned size = 8;
buffer = kmalloc(size, GFP_KERNEL);
if (!buffer)
if (r < 0)
goto out;
- r = ch341_init_set_baudrate(dev, priv, 0);
+ r = ch341_set_baudrate_lcr(dev, priv, priv->lcr);
if (r < 0)
goto out;
spin_lock_init(&priv->lock);
priv->baud_rate = DEFAULT_BAUD_RATE;
- priv->line_control = CH341_BIT_RTS | CH341_BIT_DTR;
r = ch341_configure(port->serial->dev, priv);
if (r < 0)
r = ch341_configure(serial->dev, priv);
if (r)
- goto out;
+ return r;
if (tty)
ch341_set_termios(tty, port, NULL);
if (r) {
dev_err(&port->dev, "%s - failed to submit interrupt urb: %d\n",
__func__, r);
- goto out;
+ return r;
}
r = usb_serial_generic_open(tty, port);
+ if (r)
+ goto err_kill_interrupt_urb;
+
+ return 0;
+
+err_kill_interrupt_urb:
+ usb_kill_urb(port->interrupt_in_urb);
-out: return r;
+ return r;
}
/* Old_termios contains the original termios settings and
baud_rate = tty_get_baud_rate(tty);
- priv->baud_rate = baud_rate;
ctrl = CH341_LCR_ENABLE_RX | CH341_LCR_ENABLE_TX;
switch (C_CSIZE(tty)) {
ctrl |= CH341_LCR_STOP_BITS_2;
if (baud_rate) {
- spin_lock_irqsave(&priv->lock, flags);
- priv->line_control |= (CH341_BIT_DTR | CH341_BIT_RTS);
- spin_unlock_irqrestore(&priv->lock, flags);
- r = ch341_init_set_baudrate(port->serial->dev, priv, ctrl);
+ priv->baud_rate = baud_rate;
+
+ r = ch341_set_baudrate_lcr(port->serial->dev, priv, ctrl);
if (r < 0 && old_termios) {
priv->baud_rate = tty_termios_baud_rate(old_termios);
tty_termios_copy_hw(&tty->termios, old_termios);
+ } else if (r == 0) {
+ priv->lcr = ctrl;
}
- } else {
- spin_lock_irqsave(&priv->lock, flags);
- priv->line_control &= ~(CH341_BIT_DTR | CH341_BIT_RTS);
- spin_unlock_irqrestore(&priv->lock, flags);
}
- ch341_set_handshake(port->serial->dev, priv->line_control);
+ spin_lock_irqsave(&priv->lock, flags);
+ if (C_BAUD(tty) == B0)
+ priv->line_control &= ~(CH341_BIT_DTR | CH341_BIT_RTS);
+ else if (old_termios && (old_termios->c_cflag & CBAUD) == B0)
+ priv->line_control |= (CH341_BIT_DTR | CH341_BIT_RTS);
+ spin_unlock_irqrestore(&priv->lock, flags);
+ ch341_set_handshake(port->serial->dev, priv->line_control);
}
static void ch341_break_ctl(struct tty_struct *tty, int break_state)
static int ch341_reset_resume(struct usb_serial *serial)
{
- struct ch341_private *priv;
-
- priv = usb_get_serial_port_data(serial->port[0]);
+ struct usb_serial_port *port = serial->port[0];
+ struct ch341_private *priv = usb_get_serial_port_data(port);
+ int ret;
/* reconfigure ch341 serial port after bus-reset */
ch341_configure(serial->dev, priv);
- return 0;
+ if (tty_port_initialized(&port->port)) {
+ ret = usb_submit_urb(port->interrupt_in_urb, GFP_NOIO);
+ if (ret) {
+ dev_err(&port->dev, "failed to submit interrupt urb: %d\n",
+ ret);
+ return ret;
+ }
+ }
+
+ return usb_serial_generic_resume(serial);
}
static struct usb_serial_driver ch341_device = {
status_buf, KLSI_STATUSBUF_LEN,
10000
);
- if (rc < 0)
- dev_err(&port->dev, "Reading line status failed (error = %d)\n",
- rc);
- else {
+ if (rc != KLSI_STATUSBUF_LEN) {
+ dev_err(&port->dev, "reading line status failed: %d\n", rc);
+ if (rc >= 0)
+ rc = -EIO;
+ } else {
status = get_unaligned_le16(status_buf);
dev_info(&port->serial->dev->dev, "read status %x %x\n",
struct scatterlist sg[4], sg_dst;
void *dst_buf;
size_t dst_size;
- const u8 bzero[16] = { 0 };
u8 iv[crypto_skcipher_ivsize(tfm_cbc)];
size_t zero_padding;
sg_set_buf(&sg[1], &scratch->b1, sizeof(scratch->b1));
sg_set_buf(&sg[2], b, blen);
/* 0 if well behaved :) */
- sg_set_buf(&sg[3], bzero, zero_padding);
+ sg_set_page(&sg[3], ZERO_PAGE(0), zero_padding, 0);
sg_init_one(&sg_dst, dst_buf, dst_size);
skcipher_request_set_tfm(req, tfm_cbc);
#include <linux/uaccess.h>
#include <linux/vfio.h>
#include <linux/workqueue.h>
-#include <linux/pid_namespace.h>
#include <linux/mdev.h>
#include <linux/notifier.h>
unsigned long *pfn_base, bool do_accounting)
{
unsigned long limit;
- bool lock_cap = ns_capable(task_active_pid_ns(dma->task)->user_ns,
- CAP_IPC_LOCK);
+ bool lock_cap = has_capability(dma->task, CAP_IPC_LOCK);
struct mm_struct *mm;
int ret;
bool rsvd;
struct iov_iter out_iter, in_iter, prot_iter, data_iter;
u64 tag;
u32 exp_data_len, data_direction;
- unsigned out, in;
+ unsigned int out = 0, in = 0;
int head, ret, prot_bytes;
size_t req_size, rsp_size = sizeof(struct virtio_scsi_cmd_resp);
size_t out_size, in_size;
NULL,
};
-static struct target_core_fabric_ops vhost_scsi_ops = {
+static const struct target_core_fabric_ops vhost_scsi_ops = {
.module = THIS_MODULE,
.name = "vhost",
.get_fabric_name = vhost_scsi_get_fabric_name,
static int vhost_vsock_start(struct vhost_vsock *vsock)
{
+ struct vhost_virtqueue *vq;
size_t i;
int ret;
goto err;
for (i = 0; i < ARRAY_SIZE(vsock->vqs); i++) {
- struct vhost_virtqueue *vq = &vsock->vqs[i];
+ vq = &vsock->vqs[i];
mutex_lock(&vq->mutex);
if (!vhost_vq_access_ok(vq)) {
ret = -EFAULT;
- mutex_unlock(&vq->mutex);
goto err_vq;
}
if (!vq->private_data) {
vq->private_data = vsock;
- vhost_vq_init_access(vq);
+ ret = vhost_vq_init_access(vq);
+ if (ret)
+ goto err_vq;
}
mutex_unlock(&vq->mutex);
return 0;
err_vq:
+ vq->private_data = NULL;
+ mutex_unlock(&vq->mutex);
+
for (i = 0; i < ARRAY_SIZE(vsock->vqs); i++) {
- struct vhost_virtqueue *vq = &vsock->vqs[i];
+ vq = &vsock->vqs[i];
mutex_lock(&vq->mutex);
vq->private_data = NULL;
int fb_copy_cmap(const struct fb_cmap *from, struct fb_cmap *to)
{
- int tooff = 0, fromoff = 0;
- int size;
+ unsigned int tooff = 0, fromoff = 0;
+ size_t size;
if (to->start > from->start)
fromoff = to->start - from->start;
else
tooff = from->start - to->start;
- size = to->len - tooff;
- if (size > (int) (from->len - fromoff))
- size = from->len - fromoff;
- if (size <= 0)
+ if (fromoff >= from->len || tooff >= to->len)
+ return -EINVAL;
+
+ size = min_t(size_t, to->len - tooff, from->len - fromoff);
+ if (size == 0)
return -EINVAL;
size *= sizeof(u16);
int fb_cmap_to_user(const struct fb_cmap *from, struct fb_cmap_user *to)
{
- int tooff = 0, fromoff = 0;
- int size;
+ unsigned int tooff = 0, fromoff = 0;
+ size_t size;
if (to->start > from->start)
fromoff = to->start - from->start;
else
tooff = from->start - to->start;
- size = to->len - tooff;
- if (size > (int) (from->len - fromoff))
- size = from->len - fromoff;
- if (size <= 0)
+ if (fromoff >= from->len || tooff >= to->len)
+ return -EINVAL;
+
+ size = min_t(size_t, to->len - tooff, from->len - fromoff);
+ if (size == 0)
return -EINVAL;
size *= sizeof(u16);
#define pr_fmt(fmt) "virtio-mmio: " fmt
#include <linux/acpi.h>
+#include <linux/dma-mapping.h>
#include <linux/highmem.h>
#include <linux/interrupt.h>
#include <linux/io.h>
struct virtio_mmio_device *vm_dev;
struct resource *mem;
unsigned long magic;
+ int rc;
mem = platform_get_resource(pdev, IORESOURCE_MEM, 0);
if (!mem)
}
vm_dev->vdev.id.vendor = readl(vm_dev->base + VIRTIO_MMIO_VENDOR_ID);
- if (vm_dev->version == 1)
+ if (vm_dev->version == 1) {
writel(PAGE_SIZE, vm_dev->base + VIRTIO_MMIO_GUEST_PAGE_SIZE);
+ rc = dma_set_mask(&pdev->dev, DMA_BIT_MASK(64));
+ /*
+ * In the legacy case, ensure our coherently-allocated virtio
+ * ring will be at an address expressable as a 32-bit PFN.
+ */
+ if (!rc)
+ dma_set_coherent_mask(&pdev->dev,
+ DMA_BIT_MASK(32 + PAGE_SHIFT));
+ } else {
+ rc = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64));
+ }
+ if (rc)
+ rc = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32));
+ if (rc)
+ dev_warn(&pdev->dev, "Failed to enable 64-bit or 32-bit DMA. Trying to continue, but this might not work.\n");
+
platform_set_drvdata(pdev, vm_dev);
return register_virtio_device(&vm_dev->vdev);
if (xen_domain())
return true;
+ /*
+ * On ARM-based machines, the DMA ops will do the right thing,
+ * so always use them with legacy devices.
+ */
+ if (IS_ENABLED(CONFIG_ARM) || IS_ENABLED(CONFIG_ARM64))
+ return !virtio_has_feature(vdev, VIRTIO_F_VERSION_1);
+
return false;
}
vme_bound = ioread32(bridge->base + CA91CX42_VSI_BD[i]);
pci_offset = ioread32(bridge->base + CA91CX42_VSI_TO[i]);
- *pci_base = (dma_addr_t)vme_base + pci_offset;
+ *pci_base = (dma_addr_t)*vme_base + pci_offset;
*size = (unsigned long long)((vme_bound - *vme_base) + granularity);
*enabled = 0;
static unsigned long platform_mmio;
static unsigned long platform_mmio_alloc;
static unsigned long platform_mmiolen;
+static uint64_t callback_via;
static unsigned long alloc_xen_mmio(unsigned long len)
{
return addr;
}
+static uint64_t get_callback_via(struct pci_dev *pdev)
+{
+ u8 pin;
+ int irq;
+
+ irq = pdev->irq;
+ if (irq < 16)
+ return irq; /* ISA IRQ */
+
+ pin = pdev->pin;
+
+ /* We don't know the GSI. Specify the PCI INTx line instead. */
+ return ((uint64_t)0x01 << HVM_CALLBACK_VIA_TYPE_SHIFT) | /* PCI INTx identifier */
+ ((uint64_t)pci_domain_nr(pdev->bus) << 32) |
+ ((uint64_t)pdev->bus->number << 16) |
+ ((uint64_t)(pdev->devfn & 0xff) << 8) |
+ ((uint64_t)(pin - 1) & 3);
+}
+
+static irqreturn_t do_hvm_evtchn_intr(int irq, void *dev_id)
+{
+ xen_hvm_evtchn_do_upcall();
+ return IRQ_HANDLED;
+}
+
+static int xen_allocate_irq(struct pci_dev *pdev)
+{
+ return request_irq(pdev->irq, do_hvm_evtchn_intr,
+ IRQF_NOBALANCING | IRQF_TRIGGER_RISING,
+ "xen-platform-pci", pdev);
+}
+
+static int platform_pci_resume(struct pci_dev *pdev)
+{
+ int err;
+ if (!xen_pv_domain())
+ return 0;
+ err = xen_set_callback_via(callback_via);
+ if (err) {
+ dev_err(&pdev->dev, "platform_pci_resume failure!\n");
+ return err;
+ }
+ return 0;
+}
+
static int platform_pci_probe(struct pci_dev *pdev,
const struct pci_device_id *ent)
{
platform_mmio = mmio_addr;
platform_mmiolen = mmio_len;
+ /*
+ * Xen HVM guests always use the vector callback mechanism.
+ * L1 Dom0 in a nested Xen environment is a PV guest inside in an
+ * HVM environment. It needs the platform-pci driver to get
+ * notifications from L0 Xen, but it cannot use the vector callback
+ * as it is not exported by L1 Xen.
+ */
+ if (xen_pv_domain()) {
+ ret = xen_allocate_irq(pdev);
+ if (ret) {
+ dev_warn(&pdev->dev, "request_irq failed err=%d\n", ret);
+ goto out;
+ }
+ callback_via = get_callback_via(pdev);
+ ret = xen_set_callback_via(callback_via);
+ if (ret) {
+ dev_warn(&pdev->dev, "Unable to set the evtchn callback "
+ "err=%d\n", ret);
+ goto out;
+ }
+ }
+
max_nr_gframes = gnttab_max_grant_frames();
grant_frames = alloc_xen_mmio(PAGE_SIZE * max_nr_gframes);
ret = gnttab_setup_auto_xlat_frames(grant_frames);
.name = DRV_NAME,
.probe = platform_pci_probe,
.id_table = platform_pci_tbl,
+#ifdef CONFIG_PM
+ .resume_early = platform_pci_resume,
+#endif
};
builtin_pci_driver(platform_driver);
bool "Direct Access (DAX) support"
depends on MMU
depends on !(ARM || MIPS || SPARC)
+ select FS_IOMAP
help
Direct Access (DAX) can be used on memory-backed block devices.
If the block device supports DAX and the filesystem supports DAX,
* Tell lockdep we inherited freeze protection from submission
* thread.
*/
- __sb_writers_acquired(file_inode(file)->i_sb, SB_FREEZE_WRITE);
+ if (S_ISREG(file_inode(file)->i_mode))
+ __sb_writers_acquired(file_inode(file)->i_sb, SB_FREEZE_WRITE);
file_end_write(file);
}
* by telling it the lock got released so that it doesn't
* complain about held lock when we return to userspace.
*/
- __sb_writers_release(file_inode(file)->i_sb, SB_FREEZE_WRITE);
+ if (S_ISREG(file_inode(file)->i_mode))
+ __sb_writers_release(file_inode(file)->i_sb, SB_FREEZE_WRITE);
}
kfree(iovec);
return ret;
goto end_coredump;
}
}
+ dump_truncate(cprm);
if (!elf_core_write_extra_data(cprm))
goto end_coredump;
unsigned long flags;
while (1) {
+ void *wtag;
+
spin_lock_irqsave(lock, flags);
if (list_empty(list))
break;
spin_unlock_irqrestore(lock, flags);
/*
- * we don't want to call the ordered free functions
- * with the lock held though
+ * We don't want to call the ordered free functions with the
+ * lock held though. Save the work as tag for the trace event,
+ * because the callback could free the structure.
*/
+ wtag = work;
work->ordered_free(work);
- trace_btrfs_all_work_done(work);
+ trace_btrfs_all_work_done(wq->fs_info, wtag);
}
spin_unlock_irqrestore(lock, flags);
}
static void normal_work_helper(struct btrfs_work *work)
{
struct __btrfs_workqueue *wq;
+ void *wtag;
int need_order = 0;
/*
if (work->ordered_func)
need_order = 1;
wq = work->wq;
+ /* Safe for tracepoints in case work gets freed by the callback */
+ wtag = work;
trace_btrfs_work_sched(work);
thresh_exec_hook(wq);
run_ordered_work(wq);
}
if (!need_order)
- trace_btrfs_all_work_done(work);
+ trace_btrfs_all_work_done(wq->fs_info, wtag);
}
void btrfs_init_work(struct btrfs_work *work, btrfs_work_func_t uniq_func,
if (ref && ref->seq &&
btrfs_check_delayed_seq(fs_info, delayed_refs, ref->seq)) {
spin_unlock(&locked_ref->lock);
- btrfs_delayed_ref_unlock(locked_ref);
spin_lock(&delayed_refs->lock);
locked_ref->processing = 0;
delayed_refs->num_heads_ready++;
spin_unlock(&delayed_refs->lock);
+ btrfs_delayed_ref_unlock(locked_ref);
locked_ref = NULL;
cond_resched();
count++;
*/
if (must_insert_reserved)
locked_ref->must_insert_reserved = 1;
+ spin_lock(&delayed_refs->lock);
locked_ref->processing = 0;
+ delayed_refs->num_heads_ready++;
+ spin_unlock(&delayed_refs->lock);
btrfs_debug(fs_info,
"run_delayed_extent_op returned %d",
ret);
spin_unlock(&cluster->refill_lock);
- down_read(&used_bg->data_rwsem);
+ /* We should only have one-level nested. */
+ down_read_nested(&used_bg->data_rwsem, SINGLE_DEPTH_NESTING);
spin_lock(&cluster->refill_lock);
if (used_bg == cluster->block_group)
write_unlock(&em_tree->lock);
out:
- trace_btrfs_get_extent(root, em);
+ trace_btrfs_get_extent(root, inode, em);
btrfs_free_path(path);
if (trans) {
* within our reservation, otherwise we need to adjust our inode
* counter appropriately.
*/
- if (dio_data->outstanding_extents) {
+ if (dio_data->outstanding_extents >= num_extents) {
dio_data->outstanding_extents -= num_extents;
} else {
+ /*
+ * If dio write length has been split due to no large enough
+ * contiguous space, we need to compensate our inode counter
+ * appropriately.
+ */
+ u64 num_needed = num_extents - dio_data->outstanding_extents;
+
spin_lock(&BTRFS_I(inode)->lock);
- BTRFS_I(inode)->outstanding_extents += num_extents;
+ BTRFS_I(inode)->outstanding_extents += num_needed;
spin_unlock(&BTRFS_I(inode)->lock);
}
}
*/
#define LOG_INODE_ALL 0
#define LOG_INODE_EXISTS 1
+#define LOG_OTHER_INODE 2
/*
* directory trouble cases
if (S_ISDIR(inode->i_mode) ||
(!test_bit(BTRFS_INODE_NEEDS_FULL_SYNC,
&BTRFS_I(inode)->runtime_flags) &&
- inode_only == LOG_INODE_EXISTS))
+ inode_only >= LOG_INODE_EXISTS))
max_key.type = BTRFS_XATTR_ITEM_KEY;
else
max_key.type = (u8)-1;
return ret;
}
- mutex_lock(&BTRFS_I(inode)->log_mutex);
+ if (inode_only == LOG_OTHER_INODE) {
+ inode_only = LOG_INODE_EXISTS;
+ mutex_lock_nested(&BTRFS_I(inode)->log_mutex,
+ SINGLE_DEPTH_NESTING);
+ } else {
+ mutex_lock(&BTRFS_I(inode)->log_mutex);
+ }
/*
* a brute force approach to making sure we get the most uptodate
* unpin it.
*/
err = btrfs_log_inode(trans, root, other_inode,
- LOG_INODE_EXISTS,
+ LOG_OTHER_INODE,
0, LLONG_MAX, ctx);
iput(other_inode);
if (err)
out:
btrfs_free_path(path);
- if (ret)
- btrfs_warn(fs_info, "btrfs_uuid_tree_iterate failed %d", ret);
- return 0;
+ return ret;
}
dout(" head snapc %p has %d dirty pages\n",
snapc, ci->i_wrbuffer_ref_head);
if (truncate_size)
- *truncate_size = capsnap->truncate_size;
+ *truncate_size = ci->i_truncate_size;
if (truncate_seq)
- *truncate_seq = capsnap->truncate_seq;
+ *truncate_seq = ci->i_truncate_seq;
}
spin_unlock(&ci->i_ceph_lock);
return snapc;
add_wait_queue(&ci->i_cap_wq, &wait);
while (!try_get_cap_refs(ci, need, want, endoff,
- true, &_got, &err))
+ true, &_got, &err)) {
+ if (signal_pending(current)) {
+ ret = -ERESTARTSYS;
+ break;
+ }
wait_woken(&wait, TASK_INTERRUPTIBLE, MAX_SCHEDULE_TIMEOUT);
+ }
remove_wait_queue(&ci->i_cap_wq, &wait);
struct ceph_mds_client *mdsc =
ceph_sb_to_client(dir->i_sb)->mdsc;
struct ceph_mds_request *req;
- int op, mask, err;
+ int op, err;
+ u32 mask;
if (flags & LOOKUP_RCU)
return -ECHILD;
mask = CEPH_STAT_CAP_INODE | CEPH_CAP_AUTH_SHARED;
if (ceph_security_xattr_wanted(dir))
mask |= CEPH_CAP_XATTR_SHARED;
- req->r_args.getattr.mask = mask;
+ req->r_args.getattr.mask = cpu_to_le32(mask);
err = ceph_mdsc_do_request(mdsc, NULL, req);
switch (err) {
{
struct ceph_frag_tree_split *ls = (struct ceph_frag_tree_split*)l;
struct ceph_frag_tree_split *rs = (struct ceph_frag_tree_split*)r;
- return ceph_frag_compare(ls->frag, rs->frag);
+ return ceph_frag_compare(le32_to_cpu(ls->frag),
+ le32_to_cpu(rs->frag));
}
static bool is_frag_child(u32 f, struct ceph_inode_frag *frag)
struct ceph_mds_reply_info_parsed *info,
u64 features)
{
- if (info->head->op == CEPH_MDS_OP_GETFILELOCK)
+ u32 op = le32_to_cpu(info->head->op);
+
+ if (op == CEPH_MDS_OP_GETFILELOCK)
return parse_reply_info_filelock(p, end, info, features);
- else if (info->head->op == CEPH_MDS_OP_READDIR ||
- info->head->op == CEPH_MDS_OP_LSSNAP)
+ else if (op == CEPH_MDS_OP_READDIR || op == CEPH_MDS_OP_LSSNAP)
return parse_reply_info_dir(p, end, info, features);
- else if (info->head->op == CEPH_MDS_OP_CREATE)
+ else if (op == CEPH_MDS_OP_CREATE)
return parse_reply_info_create(p, end, info, features);
else
return -EIO;
dout("do_request mdsmap err %d\n", err);
goto finish;
}
+ if (mdsc->mdsmap->m_epoch == 0) {
+ dout("do_request no mdsmap, waiting for map\n");
+ list_add(&req->r_wait, &mdsc->waiting_for_map);
+ goto finish;
+ }
if (!(mdsc->fsc->mount_options->flags &
CEPH_MOUNT_OPT_MOUNTWAIT) &&
!ceph_mdsmap_is_cluster_available(mdsc->mdsmap)) {
return mod ? dump_skip(cprm, align - mod) : 1;
}
EXPORT_SYMBOL(dump_align);
+
+/*
+ * Ensures that file size is big enough to contain the current file
+ * postion. This prevents gdb from complaining about a truncated file
+ * if the last "write" to the file was dump_skip.
+ */
+void dump_truncate(struct coredump_params *cprm)
+{
+ struct file *file = cprm->file;
+ loff_t offset;
+
+ if (file->f_op->llseek && file->f_op->llseek != no_llseek) {
+ offset = file->f_op->llseek(file, 0, SEEK_CUR);
+ if (i_size_read(file->f_mapping->host) < offset)
+ do_truncate(file->f_path.dentry, offset, 0, file);
+ }
+}
+EXPORT_SYMBOL(dump_truncate);
pgoff_t index, unsigned long pfn)
{
struct vm_area_struct *vma;
- pte_t *ptep;
- pte_t pte;
+ pte_t pte, *ptep = NULL;
+ pmd_t *pmdp = NULL;
spinlock_t *ptl;
bool changed;
address = pgoff_address(index, vma);
changed = false;
- if (follow_pte(vma->vm_mm, address, &ptep, &ptl))
+ if (follow_pte_pmd(vma->vm_mm, address, &ptep, &pmdp, &ptl))
continue;
- if (pfn != pte_pfn(*ptep))
- goto unlock;
- if (!pte_dirty(*ptep) && !pte_write(*ptep))
- goto unlock;
- flush_cache_page(vma, address, pfn);
- pte = ptep_clear_flush(vma, address, ptep);
- pte = pte_wrprotect(pte);
- pte = pte_mkclean(pte);
- set_pte_at(vma->vm_mm, address, ptep, pte);
- changed = true;
-unlock:
- pte_unmap_unlock(ptep, ptl);
+ if (pmdp) {
+#ifdef CONFIG_FS_DAX_PMD
+ pmd_t pmd;
+
+ if (pfn != pmd_pfn(*pmdp))
+ goto unlock_pmd;
+ if (!pmd_dirty(*pmdp) && !pmd_write(*pmdp))
+ goto unlock_pmd;
+
+ flush_cache_page(vma, address, pfn);
+ pmd = pmdp_huge_clear_flush(vma, address, pmdp);
+ pmd = pmd_wrprotect(pmd);
+ pmd = pmd_mkclean(pmd);
+ set_pmd_at(vma->vm_mm, address, pmdp, pmd);
+ changed = true;
+unlock_pmd:
+ spin_unlock(ptl);
+#endif
+ } else {
+ if (pfn != pte_pfn(*ptep))
+ goto unlock_pte;
+ if (!pte_dirty(*ptep) && !pte_write(*ptep))
+ goto unlock_pte;
+
+ flush_cache_page(vma, address, pfn);
+ pte = ptep_clear_flush(vma, address, ptep);
+ pte = pte_wrprotect(pte);
+ pte = pte_mkclean(pte);
+ set_pte_at(vma->vm_mm, address, ptep, pte);
+ changed = true;
+unlock_pte:
+ pte_unmap_unlock(ptep, ptl);
+ }
if (changed)
mmu_notifier_invalidate_page(vma->vm_mm, address);
}
EXPORT_SYMBOL_GPL(__dax_zero_page_range);
-#ifdef CONFIG_FS_IOMAP
static sector_t dax_iomap_sector(struct iomap *iomap, loff_t pos)
{
return iomap->blkno + (((pos & PAGE_MASK) - iomap->offset) >> 9);
}
EXPORT_SYMBOL_GPL(dax_iomap_pmd_fault);
#endif /* CONFIG_FS_DAX_PMD */
-#endif /* CONFIG_FS_IOMAP */
}
spin_lock(&dentry->d_lock);
if (!d_unlinked(dentry)) {
- dentry->d_flags |= DCACHE_MOUNTED;
- ret = 0;
+ ret = -EBUSY;
+ if (!d_mountpoint(dentry)) {
+ dentry->d_flags |= DCACHE_MOUNTED;
+ ret = 0;
+ }
}
spin_unlock(&dentry->d_lock);
out:
struct buffer_head *map_bh)
{
const unsigned blkbits = sdio->blkbits;
+ const unsigned i_blkbits = blkbits + sdio->blkfactor;
int ret = 0;
while (sdio->block_in_file < sdio->final_block_in_request) {
clean_bdev_aliases(
map_bh->b_bdev,
map_bh->b_blocknr,
- map_bh->b_size >> blkbits);
+ map_bh->b_size >> i_blkbits);
}
if (!sdio->blkfactor)
config EXT2_FS
tristate "Second extended fs support"
- select FS_IOMAP if FS_DAX
help
Ext2 is a standard Linux file system for hard disks.
select CRC16
select CRYPTO
select CRYPTO_CRC32C
- select FS_IOMAP if FS_DAX
help
This is the next generation of the ext3 filesystem.
}
sector = SECTOR_FROM_BLOCK(blkstart);
- if (sector & (bdev_zone_size(bdev) - 1) ||
- nr_sects != bdev_zone_size(bdev)) {
+ if (sector & (bdev_zone_sectors(bdev) - 1) ||
+ nr_sects != bdev_zone_sectors(bdev)) {
f2fs_msg(sbi->sb, KERN_INFO,
"(%d) %s: Unaligned discard attempted (block %x + %x)",
devi, sbi->s_ndevs ? FDEV(devi).path: "",
return 0;
if (sbi->blocks_per_blkz && sbi->blocks_per_blkz !=
- SECTOR_TO_BLOCK(bdev_zone_size(bdev)))
+ SECTOR_TO_BLOCK(bdev_zone_sectors(bdev)))
return -EINVAL;
- sbi->blocks_per_blkz = SECTOR_TO_BLOCK(bdev_zone_size(bdev));
+ sbi->blocks_per_blkz = SECTOR_TO_BLOCK(bdev_zone_sectors(bdev));
if (sbi->log_blocks_per_blkz && sbi->log_blocks_per_blkz !=
__ilog2_u32(sbi->blocks_per_blkz))
return -EINVAL;
sbi->log_blocks_per_blkz = __ilog2_u32(sbi->blocks_per_blkz);
FDEV(devi).nr_blkz = SECTOR_TO_BLOCK(nr_sectors) >>
sbi->log_blocks_per_blkz;
- if (nr_sectors & (bdev_zone_size(bdev) - 1))
+ if (nr_sectors & (bdev_zone_sectors(bdev) - 1))
FDEV(devi).nr_blkz++;
FDEV(devi).blkz_type = kmalloc(FDEV(devi).nr_blkz, GFP_KERNEL);
struct fuse_req *req;
req = list_entry(head->next, struct fuse_req, list);
req->out.h.error = -ECONNABORTED;
- clear_bit(FR_PENDING, &req->flags);
clear_bit(FR_SENT, &req->flags);
list_del_init(&req->list);
request_end(fc, req);
spin_lock(&fiq->waitq.lock);
fiq->connected = 0;
list_splice_init(&fiq->pending, &to_end2);
+ list_for_each_entry(req, &to_end2, list)
+ clear_bit(FR_PENDING, &req->flags);
while (forget_pending(fiq))
kfree(dequeue_forget(fiq, 1, NULL));
wake_up_all_locked(&fiq->waitq);
if (sec || nsec) {
struct timespec64 ts = {
sec,
- max_t(u32, nsec, NSEC_PER_SEC - 1)
+ min_t(u32, nsec, NSEC_PER_SEC - 1)
};
return get_jiffies_64() + timespec64_to_jiffies(&ts);
struct inode *root;
struct qstr d_name = QSTR_INIT(name, strlen(name));
- s = sget(fs_type, NULL, set_anon_super, MS_NOUSER, NULL);
+ s = sget_userns(fs_type, NULL, set_anon_super, MS_KERNMOUNT|MS_NOUSER,
+ &init_user_ns, NULL);
if (IS_ERR(s))
return ERR_CAST(s);
return NULL;
}
-static struct mountpoint *new_mountpoint(struct dentry *dentry)
+static struct mountpoint *get_mountpoint(struct dentry *dentry)
{
- struct hlist_head *chain = mp_hash(dentry);
- struct mountpoint *mp;
+ struct mountpoint *mp, *new = NULL;
int ret;
- mp = kmalloc(sizeof(struct mountpoint), GFP_KERNEL);
- if (!mp)
+ if (d_mountpoint(dentry)) {
+mountpoint:
+ read_seqlock_excl(&mount_lock);
+ mp = lookup_mountpoint(dentry);
+ read_sequnlock_excl(&mount_lock);
+ if (mp)
+ goto done;
+ }
+
+ if (!new)
+ new = kmalloc(sizeof(struct mountpoint), GFP_KERNEL);
+ if (!new)
return ERR_PTR(-ENOMEM);
+
+ /* Exactly one processes may set d_mounted */
ret = d_set_mounted(dentry);
- if (ret) {
- kfree(mp);
- return ERR_PTR(ret);
- }
- mp->m_dentry = dentry;
- mp->m_count = 1;
- hlist_add_head(&mp->m_hash, chain);
- INIT_HLIST_HEAD(&mp->m_list);
+ /* Someone else set d_mounted? */
+ if (ret == -EBUSY)
+ goto mountpoint;
+
+ /* The dentry is not available as a mountpoint? */
+ mp = ERR_PTR(ret);
+ if (ret)
+ goto done;
+
+ /* Add the new mountpoint to the hash table */
+ read_seqlock_excl(&mount_lock);
+ new->m_dentry = dentry;
+ new->m_count = 1;
+ hlist_add_head(&new->m_hash, mp_hash(dentry));
+ INIT_HLIST_HEAD(&new->m_list);
+ read_sequnlock_excl(&mount_lock);
+
+ mp = new;
+ new = NULL;
+done:
+ kfree(new);
return mp;
}
struct mount *mnt;
namespace_lock();
+ lock_mount_hash();
mp = lookup_mountpoint(dentry);
if (IS_ERR_OR_NULL(mp))
goto out_unlock;
- lock_mount_hash();
event++;
while (!hlist_empty(&mp->m_list)) {
mnt = hlist_entry(mp->m_list.first, struct mount, mnt_mp_list);
}
else umount_tree(mnt, UMOUNT_CONNECTED);
}
- unlock_mount_hash();
put_mountpoint(mp);
out_unlock:
+ unlock_mount_hash();
namespace_unlock();
}
namespace_lock();
mnt = lookup_mnt(path);
if (likely(!mnt)) {
- struct mountpoint *mp = lookup_mountpoint(dentry);
- if (!mp)
- mp = new_mountpoint(dentry);
+ struct mountpoint *mp = get_mountpoint(dentry);
if (IS_ERR(mp)) {
namespace_unlock();
inode_unlock(dentry->d_inode);
static void unlock_mount(struct mountpoint *where)
{
struct dentry *dentry = where->m_dentry;
+
+ read_seqlock_excl(&mount_lock);
put_mountpoint(where);
+ read_sequnlock_excl(&mount_lock);
+
namespace_unlock();
inode_unlock(dentry->d_inode);
}
touch_mnt_namespace(current->nsproxy->mnt_ns);
/* A moved mount should not expire automatically */
list_del_init(&new_mnt->mnt_expire);
+ put_mountpoint(root_mp);
unlock_mount_hash();
chroot_fs_refs(&root, &new);
- put_mountpoint(root_mp);
error = 0;
out4:
unlock_mount(old_mp);
#include <linux/mm.h>
#include <linux/delay.h>
#include <linux/errno.h>
-#include <linux/file.h>
#include <linux/string.h>
#include <linux/ratelimit.h>
#include <linux/printk.h>
return nfs4_call_sync_sequence(clnt, server, msg, args, res);
}
-static void update_changeattr(struct inode *dir, struct nfs4_change_info *cinfo)
+static void update_changeattr(struct inode *dir, struct nfs4_change_info *cinfo,
+ unsigned long timestamp)
{
struct nfs_inode *nfsi = NFS_I(dir);
NFS_INO_INVALID_ACL;
}
dir->i_version = cinfo->after;
+ nfsi->read_cache_jiffies = timestamp;
nfsi->attr_gencount = nfs_inc_attr_generation_counter();
nfs_fscache_invalidate(dir);
spin_unlock(&dir->i_lock);
nfs_fattr_map_and_free_names(server, &data->f_attr);
if (o_arg->open_flags & O_CREAT) {
- update_changeattr(dir, &o_res->cinfo);
if (o_arg->open_flags & O_EXCL)
data->file_created = 1;
else if (o_res->cinfo.before != o_res->cinfo.after)
data->file_created = 1;
+ if (data->file_created || dir->i_version != o_res->cinfo.after)
+ update_changeattr(dir, &o_res->cinfo,
+ o_res->f_attr->time_start);
}
if ((o_res->rflags & NFS4_OPEN_RESULT_LOCKTYPE_POSIX) == 0)
server->caps &= ~NFS_CAP_POSIX_LOCK;
.rpc_argp = &args,
.rpc_resp = &res,
};
+ unsigned long timestamp = jiffies;
int status;
status = nfs4_call_sync(server->client, server, &msg, &args.seq_args, &res.seq_res, 1);
if (status == 0)
- update_changeattr(dir, &res.cinfo);
+ update_changeattr(dir, &res.cinfo, timestamp);
return status;
}
if (nfs4_async_handle_error(task, res->server, NULL,
&data->timeout) == -EAGAIN)
return 0;
- update_changeattr(dir, &res->cinfo);
+ if (task->tk_status == 0)
+ update_changeattr(dir, &res->cinfo, res->dir_attr->time_start);
return 1;
}
if (nfs4_async_handle_error(task, res->server, NULL, &data->timeout) == -EAGAIN)
return 0;
- update_changeattr(old_dir, &res->old_cinfo);
- update_changeattr(new_dir, &res->new_cinfo);
+ if (task->tk_status == 0) {
+ update_changeattr(old_dir, &res->old_cinfo, res->old_fattr->time_start);
+ if (new_dir != old_dir)
+ update_changeattr(new_dir, &res->new_cinfo, res->new_fattr->time_start);
+ }
return 1;
}
status = nfs4_call_sync(server->client, server, &msg, &arg.seq_args, &res.seq_res, 1);
if (!status) {
- update_changeattr(dir, &res.cinfo);
+ update_changeattr(dir, &res.cinfo, res.fattr->time_start);
status = nfs_post_op_update_inode(inode, res.fattr);
if (!status)
nfs_setsecurity(inode, res.fattr, res.label);
int status = nfs4_call_sync(NFS_SERVER(dir)->client, NFS_SERVER(dir), &data->msg,
&data->arg.seq_args, &data->res.seq_res, 1);
if (status == 0) {
- update_changeattr(dir, &data->res.dir_cinfo);
+ update_changeattr(dir, &data->res.dir_cinfo,
+ data->res.fattr->time_start);
status = nfs_instantiate(dentry, data->res.fh, data->res.fattr, data->res.label);
}
return status;
p->server = server;
atomic_inc(&lsp->ls_count);
p->ctx = get_nfs_open_context(ctx);
- get_file(fl->fl_file);
memcpy(&p->fl, fl, sizeof(p->fl));
return p;
out_free_seqid:
nfs_free_seqid(data->arg.lock_seqid);
nfs4_put_lock_state(data->lsp);
put_nfs_open_context(data->ctx);
- fput(data->fl.fl_file);
kfree(data);
dprintk("%s: done!\n", __func__);
}
break;
case -NFS4ERR_STALE_CLIENTID:
set_bit(NFS4CLNT_LEASE_EXPIRED, &clp->cl_state);
- nfs4_state_clear_reclaim_reboot(clp);
nfs4_state_start_reclaim_reboot(clp);
break;
case -NFS4ERR_EXPIRED:
p++; /* to be backfilled later */
if (bmval0 & FATTR4_WORD0_SUPPORTED_ATTRS) {
- u32 *supp = nfsd_suppattrs[minorversion];
+ u32 supp[3];
+
+ memcpy(supp, nfsd_suppattrs[minorversion], sizeof(supp));
if (!IS_POSIXACL(dentry->d_inode))
supp[0] &= ~FATTR4_WORD0_ACL;
mlog(ML_BASTS, "lockres %s, level %d => %d\n", lockres->l_name,
lockres->l_level, new_level);
+ /*
+ * On DLM_LKF_VALBLK, fsdlm behaves differently with o2cb. It always
+ * expects DLM_LKF_VALBLK being set if the LKB has LVB, so that
+ * we can recover correctly from node failure. Otherwise, we may get
+ * invalid LVB in LKB, but without DLM_SBF_VALNOTVALID being set.
+ */
+ if (!ocfs2_is_o2cb_active() &&
+ lockres->l_ops->flags & LOCK_TYPE_USES_LVB)
+ lvb = 1;
+
if (lvb)
dlm_flags |= DLM_LKF_VALBLK;
*/
static struct ocfs2_stack_plugin *active_stack;
+inline int ocfs2_is_o2cb_active(void)
+{
+ return !strcmp(active_stack->sp_name, OCFS2_STACK_PLUGIN_O2CB);
+}
+EXPORT_SYMBOL_GPL(ocfs2_is_o2cb_active);
+
static struct ocfs2_stack_plugin *ocfs2_stack_lookup(const char *name)
{
struct ocfs2_stack_plugin *p;
int ocfs2_stack_glue_register(struct ocfs2_stack_plugin *plugin);
void ocfs2_stack_glue_unregister(struct ocfs2_stack_plugin *plugin);
+/* In ocfs2_downconvert_lock(), we need to know which stack we are using */
+int ocfs2_is_o2cb_active(void);
+
extern struct kset *ocfs2_kset;
#endif /* STACKGLUE_H */
static int ovl_lookup_layer(struct dentry *base, struct ovl_lookup_data *d,
struct dentry **ret)
{
- const char *s = d->name.name;
+ /* Counting down from the end, since the prefix can change */
+ size_t rem = d->name.len - 1;
struct dentry *dentry = NULL;
int err;
- if (*s != '/')
+ if (d->name.name[0] != '/')
return ovl_lookup_single(base, d, d->name.name, d->name.len,
0, "", ret);
- while (*s++ == '/' && !IS_ERR_OR_NULL(base) && d_can_lookup(base)) {
+ while (!IS_ERR_OR_NULL(base) && d_can_lookup(base)) {
+ const char *s = d->name.name + d->name.len - rem;
const char *next = strchrnul(s, '/');
- size_t slen = strlen(s);
+ size_t thislen = next - s;
+ bool end = !next[0];
- if (WARN_ON(slen > d->name.len) ||
- WARN_ON(strcmp(d->name.name + d->name.len - slen, s)))
+ /* Verify we did not go off the rails */
+ if (WARN_ON(s[-1] != '/'))
return -EIO;
- err = ovl_lookup_single(base, d, s, next - s,
- d->name.len - slen, next, &base);
+ err = ovl_lookup_single(base, d, s, thislen,
+ d->name.len - rem, next, &base);
dput(dentry);
if (err)
return err;
dentry = base;
- s = next;
+ if (end)
+ break;
+
+ rem -= thislen + 1;
+
+ if (WARN_ON(rem >= d->name.len))
+ return -EIO;
}
*ret = dentry;
return 0;
int error;
if (type == ACL_TYPE_ACCESS) {
- error = posix_acl_equiv_mode(acl, &inode->i_mode);
- if (error < 0)
- return 0;
- if (error == 0)
- acl = NULL;
+ error = posix_acl_update_mode(inode,
+ &inode->i_mode, &acl);
+ if (error)
+ return error;
}
inode->i_ctime = current_time(inode);
iter.tgid += 1, iter = next_tgid(ns, iter)) {
char name[PROC_NUMBUF];
int len;
+
+ cond_resched();
if (!has_pid_permissions(ns, iter.task, 2))
continue;
ctl_dir = container_of(head, struct ctl_dir, header);
if (!dir_emit_dots(file, ctx))
- return 0;
+ goto out;
pos = 2;
break;
}
}
+out:
sysctl_head_finish(head);
return 0;
}
#include <linux/highmem.h>
#include <linux/pagemap.h>
#include <linux/uaccess.h>
+#include <linux/major.h>
#include "internal.h"
static struct kmem_cache *romfs_inode_cachep;
static int romfs_statfs(struct dentry *dentry, struct kstatfs *buf)
{
struct super_block *sb = dentry->d_sb;
- u64 id = huge_encode_dev(sb->s_bdev->bd_dev);
+ u64 id = 0;
+
+ /* When calling huge_encode_dev(),
+ * use sb->s_bdev->bd_dev when,
+ * - CONFIG_ROMFS_ON_BLOCK defined
+ * use sb->s_dev when,
+ * - CONFIG_ROMFS_ON_BLOCK undefined and
+ * - CONFIG_ROMFS_ON_MTD defined
+ * leave id as 0 when,
+ * - CONFIG_ROMFS_ON_BLOCK undefined and
+ * - CONFIG_ROMFS_ON_MTD undefined
+ */
+ if (sb->s_bdev)
+ id = huge_encode_dev(sb->s_bdev->bd_dev);
+ else if (sb->s_dev)
+ id = huge_encode_dev(sb->s_dev);
buf->f_type = ROMFS_MAGIC;
buf->f_namelen = ROMFS_MAXFN;
sb->s_flags |= MS_RDONLY | MS_NOATIME;
sb->s_op = &romfs_super_ops;
+#ifdef CONFIG_ROMFS_ON_MTD
+ /* Use same dev ID from the underlying mtdblock device */
+ if (sb->s_mtd)
+ sb->s_dev = MKDEV(MTD_BLOCK_MAJOR, sb->s_mtd->index);
+#endif
/* read the image superblock and check it */
rsb = kmalloc(512, GFP_KERNEL);
if (!rsb)
config UBIFS_FS_ENCRYPTION
bool "UBIFS Encryption"
- depends on UBIFS_FS
+ depends on UBIFS_FS && BLOCK
select FS_ENCRYPTION
default n
help
dbg_gen("dent '%pd', mode %#hx in dir ino %lu",
dentry, mode, dir->i_ino);
- if (ubifs_crypt_is_encrypted(dir)) {
- err = fscrypt_get_encryption_info(dir);
- if (err)
- return err;
-
- if (!fscrypt_has_encryption_key(dir)) {
- return -EPERM;
- }
- }
-
err = fscrypt_setup_filename(dir, &dentry->d_name, 0, &nm);
if (err)
return err;
ubifs_assert(inode_is_locked(dir));
ubifs_assert(inode_is_locked(inode));
- if (ubifs_crypt_is_encrypted(dir)) {
- if (!fscrypt_has_permitted_context(dir, inode))
- return -EPERM;
-
- err = fscrypt_get_encryption_info(inode);
- if (err)
- return err;
-
- if (!fscrypt_has_encryption_key(inode))
- return -EPERM;
- }
+ if (ubifs_crypt_is_encrypted(dir) &&
+ !fscrypt_has_permitted_context(dir, inode))
+ return -EPERM;
err = fscrypt_setup_filename(dir, &dentry->d_name, 0, &nm);
if (err)
if (err)
return err;
- if (ubifs_crypt_is_encrypted(dir)) {
- err = fscrypt_get_encryption_info(dir);
- if (err)
- goto out_budg;
-
- if (!fscrypt_has_encryption_key(dir)) {
- err = -EPERM;
- goto out_budg;
- }
- }
-
err = fscrypt_setup_filename(dir, &dentry->d_name, 0, &nm);
if (err)
goto out_budg;
return err;
}
- if (ubifs_crypt_is_encrypted(dir)) {
- err = fscrypt_get_encryption_info(dir);
- if (err)
- goto out_budg;
-
- if (!fscrypt_has_encryption_key(dir)) {
- err = -EPERM;
- goto out_budg;
- }
- }
-
err = fscrypt_setup_filename(dir, &dentry->d_name, 0, &nm);
if (err)
goto out_budg;
goto out_inode;
}
- err = fscrypt_get_encryption_info(inode);
- if (err) {
- kfree(sd);
- goto out_inode;
- }
-
- if (!fscrypt_has_encryption_key(inode)) {
- kfree(sd);
- err = -EPERM;
- goto out_inode;
- }
-
ostr.name = sd->encrypted_path;
ostr.len = disk_link.len;
case FS_IOC32_SETFLAGS:
cmd = FS_IOC_SETFLAGS;
break;
+ case FS_IOC_SET_ENCRYPTION_POLICY:
+ case FS_IOC_GET_ENCRYPTION_POLICY:
+ break;
default:
return -ENOIOCTLCMD;
}
} else {
data->compr_size = 0;
+ out_len = compr_len;
}
dlen = UBIFS_DATA_NODE_SZ + out_len;
dn->compr_type = cpu_to_le16(compr_type);
dn->size = cpu_to_le32(*new_len);
*new_len = UBIFS_DATA_NODE_SZ + out_len;
+ err = 0;
out:
kfree(buf);
return err;
#include <linux/slab.h>
#include "ubifs.h"
+static int try_read_node(const struct ubifs_info *c, void *buf, int type,
+ int len, int lnum, int offs);
+static int fallible_read_node(struct ubifs_info *c, const union ubifs_key *key,
+ struct ubifs_zbranch *zbr, void *node);
+
/*
* Returned codes of 'matches_name()' and 'fallible_matches_name()' functions.
* @NAME_LESS: name corresponding to the first argument is less than second
return 0;
}
- err = ubifs_tnc_read_node(c, zbr, node);
+ if (c->replaying) {
+ err = fallible_read_node(c, &zbr->key, zbr, node);
+ /*
+ * When the node was not found, return -ENOENT, 0 otherwise.
+ * Negative return codes stay as-is.
+ */
+ if (err == 0)
+ err = -ENOENT;
+ else if (err == 1)
+ err = 0;
+ } else {
+ err = ubifs_tnc_read_node(c, zbr, node);
+ }
if (err)
return err;
if (fname_len(nm) > 0) {
if (err) {
/* Handle collisions */
- err = resolve_collision(c, key, &znode, &n, nm);
+ if (c->replaying)
+ err = fallible_resolve_collision(c, key, &znode, &n,
+ nm, 0);
+ else
+ err = resolve_collision(c, key, &znode, &n, nm);
dbg_tnc("rc returned %d, znode %p, n %d",
err, znode, n);
if (unlikely(err < 0))
struct uffd_msg msg;
wait_queue_t wq;
struct userfaultfd_ctx *ctx;
+ bool waken;
};
struct userfaultfd_wake_range {
if (len && (start > uwq->msg.arg.pagefault.address ||
start + len <= uwq->msg.arg.pagefault.address))
goto out;
+ WRITE_ONCE(uwq->waken, true);
+ /*
+ * The implicit smp_mb__before_spinlock in try_to_wake_up()
+ * renders uwq->waken visible to other CPUs before the task is
+ * waken.
+ */
ret = wake_up_state(wq->private, mode);
if (ret)
/*
struct userfaultfd_wait_queue uwq;
int ret;
bool must_wait, return_to_userland;
+ long blocking_state;
BUG_ON(!rwsem_is_locked(&mm->mmap_sem));
uwq.wq.private = current;
uwq.msg = userfault_msg(vmf->address, vmf->flags, reason);
uwq.ctx = ctx;
+ uwq.waken = false;
return_to_userland =
(vmf->flags & (FAULT_FLAG_USER|FAULT_FLAG_KILLABLE)) ==
(FAULT_FLAG_USER|FAULT_FLAG_KILLABLE);
+ blocking_state = return_to_userland ? TASK_INTERRUPTIBLE :
+ TASK_KILLABLE;
spin_lock(&ctx->fault_pending_wqh.lock);
/*
* following the spin_unlock to happen before the list_add in
* __add_wait_queue.
*/
- set_current_state(return_to_userland ? TASK_INTERRUPTIBLE :
- TASK_KILLABLE);
+ set_current_state(blocking_state);
spin_unlock(&ctx->fault_pending_wqh.lock);
must_wait = userfaultfd_must_wait(ctx, vmf->address, vmf->flags,
wake_up_poll(&ctx->fd_wqh, POLLIN);
schedule();
ret |= VM_FAULT_MAJOR;
+
+ /*
+ * False wakeups can orginate even from rwsem before
+ * up_read() however userfaults will wait either for a
+ * targeted wakeup on the specific uwq waitqueue from
+ * wake_userfault() or for signals or for uffd
+ * release.
+ */
+ while (!READ_ONCE(uwq.waken)) {
+ /*
+ * This needs the full smp_store_mb()
+ * guarantee as the state write must be
+ * visible to other CPUs before reading
+ * uwq.waken from other CPUs.
+ */
+ set_current_state(blocking_state);
+ if (READ_ONCE(uwq.waken) ||
+ READ_ONCE(ctx->released) ||
+ (return_to_userland ? signal_pending(current) :
+ fatal_signal_pending(current)))
+ break;
+ schedule();
+ }
}
__set_current_state(TASK_RUNNING);
xfs_alloc_set_aside(
struct xfs_mount *mp)
{
- unsigned int blocks;
-
- blocks = 4 + (mp->m_sb.sb_agcount * XFS_ALLOC_AGFL_RESERVE);
- return blocks;
+ return mp->m_sb.sb_agcount * (XFS_ALLOC_AGFL_RESERVE + 4);
}
/*
return;
ASSERT(rlen >= args->minlen && rlen <= args->maxlen);
ASSERT(rlen % args->prod == args->mod);
+ ASSERT(args->pag->pagf_freeblks + args->pag->pagf_flcount >=
+ rlen + args->minleft);
args->len = rlen;
}
-/*
- * Fix up length if there is too little space left in the a.g.
- * Return 1 if ok, 0 if too little, should give up.
- */
-STATIC int
-xfs_alloc_fix_minleft(
- xfs_alloc_arg_t *args) /* allocation argument structure */
-{
- xfs_agf_t *agf; /* a.g. freelist header */
- int diff; /* free space difference */
-
- if (args->minleft == 0)
- return 1;
- agf = XFS_BUF_TO_AGF(args->agbp);
- diff = be32_to_cpu(agf->agf_freeblks)
- - args->len - args->minleft;
- if (diff >= 0)
- return 1;
- args->len += diff; /* shrink the allocated space */
- /* casts to (int) catch length underflows */
- if ((int)args->len >= (int)args->minlen)
- return 1;
- args->agbno = NULLAGBLOCK;
- return 0;
-}
-
/*
* Update the two btrees, logically removing from freespace the extent
* starting at rbno, rlen blocks. The extent is contained within the
xfs_alloc_arg_t *args) /* argument structure for allocation */
{
int error=0;
- xfs_extlen_t reservation;
- xfs_extlen_t oldmax;
ASSERT(args->minlen > 0);
ASSERT(args->maxlen > 0);
ASSERT(args->mod < args->prod);
ASSERT(args->alignment > 0);
- /*
- * Clamp maxlen to the amount of free space minus any reservations
- * that have been made.
- */
- oldmax = args->maxlen;
- reservation = xfs_ag_resv_needed(args->pag, args->resv);
- if (args->maxlen > args->pag->pagf_freeblks - reservation)
- args->maxlen = args->pag->pagf_freeblks - reservation;
- if (args->maxlen == 0) {
- args->agbno = NULLAGBLOCK;
- args->maxlen = oldmax;
- return 0;
- }
-
/*
* Branch to correct routine based on the type.
*/
/* NOTREACHED */
}
- args->maxlen = oldmax;
-
if (error || args->agbno == NULLAGBLOCK)
return error;
args->len = XFS_AGBLOCK_MIN(tend, args->agbno + args->maxlen)
- args->agbno;
xfs_alloc_fix_len(args);
- if (!xfs_alloc_fix_minleft(args))
- goto not_found;
-
ASSERT(args->agbno + args->len <= tend);
/*
XFS_WANT_CORRUPTED_GOTO(args->mp, i == 1, error0);
ASSERT(ltbno + ltlen <= be32_to_cpu(XFS_BUF_TO_AGF(args->agbp)->agf_length));
args->len = blen;
- if (!xfs_alloc_fix_minleft(args)) {
- xfs_btree_del_cursor(cnt_cur, XFS_BTREE_NOERROR);
- trace_xfs_alloc_near_nominleft(args);
- return 0;
- }
- blen = args->len;
+
/*
* We are allocating starting at bnew for blen blocks.
*/
*/
args->len = XFS_EXTLEN_MIN(ltlena, args->maxlen);
xfs_alloc_fix_len(args);
- if (!xfs_alloc_fix_minleft(args)) {
- trace_xfs_alloc_near_nominleft(args);
- xfs_btree_del_cursor(bno_cur_lt, XFS_BTREE_NOERROR);
- xfs_btree_del_cursor(cnt_cur, XFS_BTREE_NOERROR);
- return 0;
- }
rlen = args->len;
(void)xfs_alloc_compute_diff(args->agbno, rlen, args->alignment,
args->datatype, ltbnoa, ltlena, <new);
}
xfs_alloc_fix_len(args);
- if (!xfs_alloc_fix_minleft(args))
- goto out_nominleft;
rlen = args->len;
XFS_WANT_CORRUPTED_GOTO(args->mp, rlen <= flen, error0);
/*
int flags)
{
struct xfs_perag *pag = args->pag;
- xfs_extlen_t longest;
+ xfs_extlen_t alloc_len, longest;
xfs_extlen_t reservation; /* blocks that are still reserved */
int available;
reservation = xfs_ag_resv_needed(pag, args->resv);
/* do we have enough contiguous free space for the allocation? */
+ alloc_len = args->minlen + (args->alignment - 1) + args->minalignslop;
longest = xfs_alloc_longest_free_extent(args->mp, pag, min_free,
reservation);
- if ((args->minlen + args->alignment + args->minalignslop - 1) > longest)
+ if (longest < alloc_len)
return false;
/* do we have enough free space remaining for the allocation? */
available = (int)(pag->pagf_freeblks + pag->pagf_flcount -
- reservation - min_free - args->total);
- if (available < (int)args->minleft || available <= 0)
+ reservation - min_free - args->minleft);
+ if (available < (int)max(args->total, alloc_len))
return false;
+ /*
+ * Clamp maxlen to the amount of free space available for the actual
+ * extent allocation.
+ */
+ if (available < (int)args->maxlen && !(flags & XFS_ALLOC_FLAG_CHECK)) {
+ args->maxlen = available;
+ ASSERT(args->maxlen > 0);
+ ASSERT(args->maxlen >= args->minlen);
+ }
+
return true;
}
}
need = xfs_alloc_min_freelist(mp, pag);
- if (!xfs_alloc_space_available(args, need, flags))
+ if (!xfs_alloc_space_available(args, need, flags |
+ XFS_ALLOC_FLAG_CHECK))
goto out_agbp_relse;
/*
xfs_agblock_t agsize; /* allocation group size */
int error;
int flags; /* XFS_ALLOC_FLAG_... locking flags */
- xfs_extlen_t minleft;/* minimum left value, temp copy */
xfs_mount_t *mp; /* mount structure pointer */
xfs_agnumber_t sagno; /* starting allocation group number */
xfs_alloctype_t type; /* input allocation type */
int bump_rotor = 0;
- int no_min = 0;
xfs_agnumber_t rotorstep = xfs_rotorstep; /* inode32 agf stepper */
mp = args->mp;
trace_xfs_alloc_vextent_badargs(args);
return 0;
}
- minleft = args->minleft;
switch (type) {
case XFS_ALLOCTYPE_THIS_AG:
*/
args->agno = XFS_FSB_TO_AGNO(mp, args->fsbno);
args->pag = xfs_perag_get(mp, args->agno);
- args->minleft = 0;
error = xfs_alloc_fix_freelist(args, 0);
- args->minleft = minleft;
if (error) {
trace_xfs_alloc_vextent_nofix(args);
goto error0;
*/
for (;;) {
args->pag = xfs_perag_get(mp, args->agno);
- if (no_min) args->minleft = 0;
error = xfs_alloc_fix_freelist(args, flags);
- args->minleft = minleft;
if (error) {
trace_xfs_alloc_vextent_nofix(args);
goto error0;
* or switch to non-trylock mode.
*/
if (args->agno == sagno) {
- if (no_min == 1) {
+ if (flags == 0) {
args->agbno = NULLAGBLOCK;
trace_xfs_alloc_vextent_allfailed(args);
break;
}
- if (flags == 0) {
- no_min = 1;
- } else {
- flags = 0;
- if (type == XFS_ALLOCTYPE_START_BNO) {
- args->agbno = XFS_FSB_TO_AGBNO(mp,
- args->fsbno);
- args->type = XFS_ALLOCTYPE_NEAR_BNO;
- }
+
+ flags = 0;
+ if (type == XFS_ALLOCTYPE_START_BNO) {
+ args->agbno = XFS_FSB_TO_AGBNO(mp,
+ args->fsbno);
+ args->type = XFS_ALLOCTYPE_NEAR_BNO;
}
}
xfs_perag_put(args->pag);
#define XFS_ALLOC_FLAG_FREEING 0x00000002 /* indicate caller is freeing extents*/
#define XFS_ALLOC_FLAG_NORMAP 0x00000004 /* don't modify the rmapbt */
#define XFS_ALLOC_FLAG_NOSHRINK 0x00000008 /* don't shrink the freelist */
-
+#define XFS_ALLOC_FLAG_CHECK 0x00000010 /* test only, don't modify args */
/*
* Argument structure for xfs_alloc routines.
args.fsbno = 0;
args.type = XFS_ALLOCTYPE_FIRST_AG;
args.total = ap->minlen;
- args.minleft = 0;
if ((error = xfs_alloc_vextent(&args)))
return error;
ap->dfops->dop_low = true;
if (error)
return error;
- if (bma->dfops->dop_low)
- bma->minleft = 0;
if (bma->cur)
bma->cur->bc_private.b.firstblock = *bma->firstblock;
if (bma->blkno == NULLFSBLOCK)
if (args.fsbno == NULLFSBLOCK && args.minleft) {
/*
* Could not find an AG with enough free space to satisfy
- * a full btree split. Try again without minleft and if
+ * a full btree split. Try again and if
* successful activate the lowspace algorithm.
*/
args.fsbno = 0;
args.type = XFS_ALLOCTYPE_FIRST_AG;
- args.minleft = 0;
error = xfs_alloc_vextent(&args);
if (error)
goto error0;
struct xfs_name xfs_name_dotdot = { (unsigned char *)"..", 2, XFS_DIR3_FT_DIR };
/*
- * @mode, if set, indicates that the type field needs to be set up.
- * This uses the transformation from file mode to DT_* as defined in linux/fs.h
- * for file type specification. This will be propagated into the directory
- * structure if appropriate for the given operation and filesystem config.
+ * Convert inode mode to directory entry filetype
*/
-const unsigned char xfs_mode_to_ftype[S_IFMT >> S_SHIFT] = {
- [0] = XFS_DIR3_FT_UNKNOWN,
- [S_IFREG >> S_SHIFT] = XFS_DIR3_FT_REG_FILE,
- [S_IFDIR >> S_SHIFT] = XFS_DIR3_FT_DIR,
- [S_IFCHR >> S_SHIFT] = XFS_DIR3_FT_CHRDEV,
- [S_IFBLK >> S_SHIFT] = XFS_DIR3_FT_BLKDEV,
- [S_IFIFO >> S_SHIFT] = XFS_DIR3_FT_FIFO,
- [S_IFSOCK >> S_SHIFT] = XFS_DIR3_FT_SOCK,
- [S_IFLNK >> S_SHIFT] = XFS_DIR3_FT_SYMLINK,
-};
+unsigned char xfs_mode_to_ftype(int mode)
+{
+ switch (mode & S_IFMT) {
+ case S_IFREG:
+ return XFS_DIR3_FT_REG_FILE;
+ case S_IFDIR:
+ return XFS_DIR3_FT_DIR;
+ case S_IFCHR:
+ return XFS_DIR3_FT_CHRDEV;
+ case S_IFBLK:
+ return XFS_DIR3_FT_BLKDEV;
+ case S_IFIFO:
+ return XFS_DIR3_FT_FIFO;
+ case S_IFSOCK:
+ return XFS_DIR3_FT_SOCK;
+ case S_IFLNK:
+ return XFS_DIR3_FT_SYMLINK;
+ default:
+ return XFS_DIR3_FT_UNKNOWN;
+ }
+}
/*
* ASCII case-insensitive (ie. A-Z) support for directories that was
if ((rval = xfs_bmap_last_offset(args->dp, &last, XFS_DATA_FORK)))
return rval;
rval = XFS_FSB_TO_B(args->dp->i_mount, last) == args->geo->blksize;
- ASSERT(rval == 0 || args->dp->i_d.di_size == args->geo->blksize);
+ if (rval != 0 && args->dp->i_d.di_size != args->geo->blksize)
+ return -EFSCORRUPTED;
*vp = rval;
return 0;
}
#ifndef __XFS_DIR2_H__
#define __XFS_DIR2_H__
+#include "xfs_da_format.h"
+#include "xfs_da_btree.h"
+
struct xfs_defer_ops;
struct xfs_da_args;
struct xfs_inode;
extern struct xfs_name xfs_name_dotdot;
/*
- * directory filetype conversion tables.
+ * Convert inode mode to directory entry filetype
*/
-#define S_SHIFT 12
-extern const unsigned char xfs_mode_to_ftype[];
+extern unsigned char xfs_mode_to_ftype(int mode);
/*
* directory operations vector for encode/decode routines
#include "xfs_icache.h"
#include "xfs_trans.h"
#include "xfs_ialloc.h"
+#include "xfs_dir2.h"
/*
* Check that none of the inode's in the buffer have a next
xfs_ino_t ino,
struct xfs_dinode *dip)
{
+ uint16_t mode;
uint16_t flags;
uint64_t flags2;
if (be64_to_cpu(dip->di_size) & (1ULL << 63))
return false;
- /* No zero-length symlinks. */
- if (S_ISLNK(be16_to_cpu(dip->di_mode)) && dip->di_size == 0)
+ mode = be16_to_cpu(dip->di_mode);
+ if (mode && xfs_mode_to_ftype(mode) == XFS_DIR3_FT_UNKNOWN)
+ return false;
+
+ /* No zero-length symlinks/dirs. */
+ if ((S_ISLNK(mode) || S_ISDIR(mode)) && dip->di_size == 0)
return false;
/* only version 3 or greater inodes are extensively verified here */
* block_invalidatepage() can send pages that are still marked dirty
* but otherwise have invalidated buffers.
*
- * We've historically freed buffers on the latter. Instead, quietly
- * filter out all dirty pages to avoid spurious buffer state warnings.
- * This can likely be removed once shrink_active_list() is fixed.
+ * We want to release the latter to avoid unnecessary buildup of the
+ * LRU, skip the former and warn if we've left any lingering
+ * delalloc/unwritten buffers on clean pages. Skip pages with delalloc
+ * or unwritten buffers and warn if the page is not dirty. Otherwise
+ * try to release the buffers.
*/
- if (PageDirty(page))
- return 0;
-
xfs_count_page_state(page, &delalloc, &unwritten);
- if (WARN_ON_ONCE(delalloc))
+ if (delalloc) {
+ WARN_ON_ONCE(!PageDirty(page));
return 0;
- if (WARN_ON_ONCE(unwritten))
+ }
+ if (unwritten) {
+ WARN_ON_ONCE(!PageDirty(page));
return 0;
+ }
return try_to_free_buffers(page);
}
/* Simple advance */
next_id = *id + 1;
+ /* If we'd wrap past the max ID, stop */
+ if (next_id < *id)
+ return -ENOENT;
+
/* If new ID is within the current chunk, advancing it sufficed */
if (next_id % mp->m_quotainfo->qi_dqperchunk) {
*id = next_id;
static void
xfs_dentry_to_name(
+ struct xfs_name *namep,
+ struct dentry *dentry)
+{
+ namep->name = dentry->d_name.name;
+ namep->len = dentry->d_name.len;
+ namep->type = XFS_DIR3_FT_UNKNOWN;
+}
+
+static int
+xfs_dentry_mode_to_name(
struct xfs_name *namep,
struct dentry *dentry,
int mode)
{
namep->name = dentry->d_name.name;
namep->len = dentry->d_name.len;
- namep->type = xfs_mode_to_ftype[(mode & S_IFMT) >> S_SHIFT];
+ namep->type = xfs_mode_to_ftype(mode);
+
+ if (unlikely(namep->type == XFS_DIR3_FT_UNKNOWN))
+ return -EFSCORRUPTED;
+
+ return 0;
}
STATIC void
* xfs_init_security we must back out.
* ENOSPC can hit here, among other things.
*/
- xfs_dentry_to_name(&teardown, dentry, 0);
+ xfs_dentry_to_name(&teardown, dentry);
xfs_remove(XFS_I(dir), &teardown, XFS_I(inode));
}
if (error)
return error;
+ /* Verify mode is valid also for tmpfile case */
+ error = xfs_dentry_mode_to_name(&name, dentry, mode);
+ if (unlikely(error))
+ goto out_free_acl;
+
if (!tmpfile) {
- xfs_dentry_to_name(&name, dentry, mode);
error = xfs_create(XFS_I(dir), &name, mode, rdev, &ip);
} else {
error = xfs_create_tmpfile(XFS_I(dir), dentry, mode, &ip);
if (dentry->d_name.len >= MAXNAMELEN)
return ERR_PTR(-ENAMETOOLONG);
- xfs_dentry_to_name(&name, dentry, 0);
+ xfs_dentry_to_name(&name, dentry);
error = xfs_lookup(XFS_I(dir), &name, &cip, NULL);
if (unlikely(error)) {
if (unlikely(error != -ENOENT))
if (dentry->d_name.len >= MAXNAMELEN)
return ERR_PTR(-ENAMETOOLONG);
- xfs_dentry_to_name(&xname, dentry, 0);
+ xfs_dentry_to_name(&xname, dentry);
error = xfs_lookup(XFS_I(dir), &xname, &ip, &ci_name);
if (unlikely(error)) {
if (unlikely(error != -ENOENT))
struct xfs_name name;
int error;
- xfs_dentry_to_name(&name, dentry, inode->i_mode);
+ error = xfs_dentry_mode_to_name(&name, dentry, inode->i_mode);
+ if (unlikely(error))
+ return error;
error = xfs_link(XFS_I(dir), XFS_I(inode), &name);
if (unlikely(error))
struct xfs_name name;
int error;
- xfs_dentry_to_name(&name, dentry, 0);
+ xfs_dentry_to_name(&name, dentry);
error = xfs_remove(XFS_I(dir), &name, XFS_I(d_inode(dentry)));
if (error)
mode = S_IFLNK |
(irix_symlink_mode ? 0777 & ~current_umask() : S_IRWXUGO);
- xfs_dentry_to_name(&name, dentry, mode);
+ error = xfs_dentry_mode_to_name(&name, dentry, mode);
+ if (unlikely(error))
+ goto out;
error = xfs_symlink(XFS_I(dir), &name, symname, mode, &cip);
if (unlikely(error))
{
struct inode *new_inode = d_inode(ndentry);
int omode = 0;
+ int error;
struct xfs_name oname;
struct xfs_name nname;
if (flags & RENAME_EXCHANGE)
omode = d_inode(ndentry)->i_mode;
- xfs_dentry_to_name(&oname, odentry, omode);
- xfs_dentry_to_name(&nname, ndentry, d_inode(odentry)->i_mode);
+ error = xfs_dentry_mode_to_name(&oname, odentry, omode);
+ if (omode && unlikely(error))
+ return error;
+
+ error = xfs_dentry_mode_to_name(&nname, ndentry,
+ d_inode(odentry)->i_mode);
+ if (unlikely(error))
+ return error;
return xfs_rename(XFS_I(odir), &oname, XFS_I(d_inode(odentry)),
XFS_I(ndir), &nname,
}
#define ASSERT_ALWAYS(expr) \
- (unlikely(expr) ? (void)0 : assfail(#expr, __FILE__, __LINE__))
+ (likely(expr) ? (void)0 : assfail(#expr, __FILE__, __LINE__))
#ifdef DEBUG
#define ASSERT(expr) \
- (unlikely(expr) ? (void)0 : assfail(#expr, __FILE__, __LINE__))
+ (likely(expr) ? (void)0 : assfail(#expr, __FILE__, __LINE__))
#ifndef STATIC
# define STATIC noinline
#ifdef XFS_WARN
#define ASSERT(expr) \
- (unlikely(expr) ? (void)0 : asswarn(#expr, __FILE__, __LINE__))
+ (likely(expr) ? (void)0 : asswarn(#expr, __FILE__, __LINE__))
#ifndef STATIC
# define STATIC static noinline
xfs_mount_t *mp,
uint flags)
{
- int error;
-
trace_xfs_log_force(mp, 0, _RET_IP_);
- error = _xfs_log_force(mp, flags, NULL);
- if (error)
- xfs_warn(mp, "%s: error %d returned.", __func__, error);
+ _xfs_log_force(mp, flags, NULL);
}
/*
xfs_lsn_t lsn,
uint flags)
{
- int error;
-
trace_xfs_log_force(mp, lsn, _RET_IP_);
- error = _xfs_log_force_lsn(mp, lsn, flags, NULL);
- if (error)
- xfs_warn(mp, "%s: error %d returned.", __func__, error);
+ _xfs_log_force_lsn(mp, lsn, flags, NULL);
}
/*
extern void drm_kms_helper_poll_disable(struct drm_device *dev);
extern void drm_kms_helper_poll_enable(struct drm_device *dev);
-extern void drm_kms_helper_poll_enable_locked(struct drm_device *dev);
#endif
# define DP_PSR_SETUP_TIME_0 (6 << 1)
# define DP_PSR_SETUP_TIME_MASK (7 << 1)
# define DP_PSR_SETUP_TIME_SHIFT 1
-
+# define DP_PSR2_SU_Y_COORDINATE_REQUIRED (1 << 4) /* eDP 1.4a */
+# define DP_PSR2_SU_GRANULARITY_REQUIRED (1 << 5) /* eDP 1.4b */
/*
* 0x80-0x8f describe downstream port capabilities, but there are two layouts
* based on whether DP_DETAILED_CAP_INFO_AVAILABLE was set. If it was not,
#define DP_RECEIVER_ALPM_STATUS 0x200b /* eDP 1.4 */
# define DP_ALPM_LOCK_TIMEOUT_ERROR (1 << 0)
+#define DP_DPRX_FEATURE_ENUMERATION_LIST 0x2210 /* DP 1.3 */
+# define DP_GTC_CAP (1 << 0) /* DP 1.3 */
+# define DP_SST_SPLIT_SDP_CAP (1 << 1) /* DP 1.4 */
+# define DP_AV_SYNC_CAP (1 << 2) /* DP 1.3 */
+# define DP_VSC_SDP_EXT_FOR_COLORIMETRY_SUPPORTED (1 << 3) /* DP 1.3 */
+# define DP_VSC_EXT_VESA_SDP_SUPPORTED (1 << 4) /* DP 1.4 */
+# define DP_VSC_EXT_VESA_SDP_CHAINING_SUPPORTED (1 << 5) /* DP 1.4 */
+# define DP_VSC_EXT_CEA_SDP_SUPPORTED (1 << 6) /* DP 1.4 */
+# define DP_VSC_EXT_CEA_SDP_CHAINING_SUPPORTED (1 << 7) /* DP 1.4 */
+
/* DP 1.2 Sideband message defines */
/* peer device type - DP 1.2a Table 2-92 */
#define DP_PEER_DEVICE_NONE 0x0
void kvm_timer_vcpu_put(struct kvm_vcpu *vcpu);
+void kvm_timer_init_vhe(void);
#endif
}
}
-static inline unsigned int blk_queue_zone_size(struct request_queue *q)
+static inline unsigned int blk_queue_zone_sectors(struct request_queue *q)
{
return blk_queue_is_zoned(q) ? q->limits.chunk_sectors : 0;
}
return blk_rq_cur_bytes(rq) >> 9;
}
+/*
+ * Some commands like WRITE SAME have a payload or data transfer size which
+ * is different from the size of the request. Any driver that supports such
+ * commands using the RQF_SPECIAL_PAYLOAD flag needs to use this helper to
+ * calculate the data transfer size.
+ */
+static inline unsigned int blk_rq_payload_bytes(struct request *rq)
+{
+ if (rq->rq_flags & RQF_SPECIAL_PAYLOAD)
+ return rq->special_vec.bv_len;
+ return blk_rq_bytes(rq);
+}
+
static inline unsigned int blk_queue_get_max_sectors(struct request_queue *q,
int op)
{
return false;
}
-static inline unsigned int bdev_zone_size(struct block_device *bdev)
+static inline unsigned int bdev_zone_sectors(struct block_device *bdev)
{
struct request_queue *q = bdev_get_queue(bdev);
if (q)
- return blk_queue_zone_size(q);
+ return blk_queue_zone_sectors(q);
return 0;
}
u64 bpf_get_stackid(u64 r1, u64 r2, u64 r3, u64 r4, u64 r5);
bool bpf_prog_array_compatible(struct bpf_array *array, const struct bpf_prog *fp);
-int bpf_prog_calc_digest(struct bpf_prog *fp);
+int bpf_prog_calc_tag(struct bpf_prog *fp);
const struct bpf_func_proto *bpf_get_trace_printk_proto(void);
extern int dump_skip(struct coredump_params *cprm, size_t nr);
extern int dump_emit(struct coredump_params *cprm, const void *addr, int nr);
extern int dump_align(struct coredump_params *cprm, int align);
+extern void dump_truncate(struct coredump_params *cprm);
#ifdef CONFIG_COREDUMP
extern void do_coredump(const siginfo_t *siginfo);
#else
CPUHP_ZCOMP_PREPARE,
CPUHP_TIMERS_DEAD,
CPUHP_MIPS_SOC_PREPARE,
+ CPUHP_BP_PREPARE_DYN,
+ CPUHP_BP_PREPARE_DYN_END = CPUHP_BP_PREPARE_DYN + 20,
CPUHP_BRINGUP_CPU,
CPUHP_AP_IDLE_DEAD,
CPUHP_AP_OFFLINE,
#define EFI_PAGE_SHIFT 12
#define EFI_PAGE_SIZE (1UL << EFI_PAGE_SHIFT)
+#define EFI_PAGES_MAX (U64_MAX >> EFI_PAGE_SHIFT)
typedef struct {
u32 type;
#endif
extern void __iomem *efi_lookup_mapped_addr(u64 phys_addr);
+extern phys_addr_t __init efi_memmap_alloc(unsigned int num_entries);
extern int __init efi_memmap_init_early(struct efi_memory_map_data *data);
extern int __init efi_memmap_init_late(phys_addr_t addr, unsigned long size);
extern void __init efi_memmap_unmap(void);
/* BPF program can access up to 512 bytes of stack space. */
#define MAX_BPF_STACK 512
+#define BPF_TAG_SIZE 8
+
/* Helper macros for filter block array initializers. */
/* ALU ops on registers, bpf_add|sub|...: dst_reg += src_reg */
kmemcheck_bitfield_end(meta);
enum bpf_prog_type type; /* Type of BPF program */
u32 len; /* Number of filter blocks */
- u32 digest[SHA_DIGEST_WORDS]; /* Program digest */
+ u8 tag[BPF_TAG_SIZE];
struct bpf_prog_aux *aux; /* Auxiliary fields */
struct sock_fprog_kern *orig_prog; /* Original BPF program */
unsigned int (*bpf_func)(const void *ctx,
return prog->len * sizeof(struct bpf_insn);
}
-static inline u32 bpf_prog_digest_scratch_size(const struct bpf_prog *prog)
+static inline u32 bpf_prog_tag_scratch_size(const struct bpf_prog *prog)
{
return round_up(bpf_prog_insn_size(prog) +
sizeof(__be64) + 1, SHA_MESSAGE_BYTES);
#define ___GFP_ACCOUNT 0x100000u
#define ___GFP_NOTRACK 0x200000u
#define ___GFP_DIRECT_RECLAIM 0x400000u
-#define ___GFP_OTHER_NODE 0x800000u
-#define ___GFP_WRITE 0x1000000u
-#define ___GFP_KSWAPD_RECLAIM 0x2000000u
+#define ___GFP_WRITE 0x800000u
+#define ___GFP_KSWAPD_RECLAIM 0x1000000u
/* If the above are modified, __GFP_BITS_SHIFT may need updating */
/*
* __GFP_NOTRACK_FALSE_POSITIVE is an alias of __GFP_NOTRACK. It's a means of
* distinguishing in the source between false positives and allocations that
* cannot be supported (e.g. page tables).
- *
- * __GFP_OTHER_NODE is for allocations that are on a remote node but that
- * should not be accounted for as a remote allocation in vmstat. A
- * typical user would be khugepaged collapsing a huge page on a remote
- * node.
*/
#define __GFP_COLD ((__force gfp_t)___GFP_COLD)
#define __GFP_NOWARN ((__force gfp_t)___GFP_NOWARN)
#define __GFP_ZERO ((__force gfp_t)___GFP_ZERO)
#define __GFP_NOTRACK ((__force gfp_t)___GFP_NOTRACK)
#define __GFP_NOTRACK_FALSE_POSITIVE (__GFP_NOTRACK)
-#define __GFP_OTHER_NODE ((__force gfp_t)___GFP_OTHER_NODE)
/* Room for N __GFP_FOO bits */
-#define __GFP_BITS_SHIFT 26
+#define __GFP_BITS_SHIFT 25
#define __GFP_BITS_MASK ((__force gfp_t)((1 << __GFP_BITS_SHIFT) - 1))
/*
extern void free_hot_cold_page_list(struct list_head *list, bool cold);
struct page_frag_cache;
-extern void __page_frag_drain(struct page *page, unsigned int order,
- unsigned int count);
-extern void *__alloc_page_frag(struct page_frag_cache *nc,
- unsigned int fragsz, gfp_t gfp_mask);
-extern void __free_page_frag(void *addr);
+extern void __page_frag_cache_drain(struct page *page, unsigned int count);
+extern void *page_frag_alloc(struct page_frag_cache *nc,
+ unsigned int fragsz, gfp_t gfp_mask);
+extern void page_frag_free(void *addr);
#define __free_page(page) __free_pages((page), 0)
#define free_page(addr) free_pages((addr), 0)
struct irq_chip *irqchip,
int parent_irq);
-int _gpiochip_irqchip_add(struct gpio_chip *gpiochip,
+int gpiochip_irqchip_add_key(struct gpio_chip *gpiochip,
+ struct irq_chip *irqchip,
+ unsigned int first_irq,
+ irq_flow_handler_t handler,
+ unsigned int type,
+ bool nested,
+ struct lock_class_key *lock_key);
+
+#ifdef CONFIG_LOCKDEP
+
+/*
+ * Lockdep requires that each irqchip instance be created with a
+ * unique key so as to avoid unnecessary warnings. This upfront
+ * boilerplate static inlines provides such a key for each
+ * unique instance.
+ */
+static inline int gpiochip_irqchip_add(struct gpio_chip *gpiochip,
+ struct irq_chip *irqchip,
+ unsigned int first_irq,
+ irq_flow_handler_t handler,
+ unsigned int type)
+{
+ static struct lock_class_key key;
+
+ return gpiochip_irqchip_add_key(gpiochip, irqchip, first_irq,
+ handler, type, false, &key);
+}
+
+static inline int gpiochip_irqchip_add_nested(struct gpio_chip *gpiochip,
struct irq_chip *irqchip,
unsigned int first_irq,
irq_flow_handler_t handler,
- unsigned int type,
- bool nested,
- struct lock_class_key *lock_key);
+ unsigned int type)
+{
+
+ static struct lock_class_key key;
+
+ return gpiochip_irqchip_add_key(gpiochip, irqchip, first_irq,
+ handler, type, true, &key);
+}
+#else
+static inline int gpiochip_irqchip_add(struct gpio_chip *gpiochip,
+ struct irq_chip *irqchip,
+ unsigned int first_irq,
+ irq_flow_handler_t handler,
+ unsigned int type)
+{
+ return gpiochip_irqchip_add_key(gpiochip, irqchip, first_irq,
+ handler, type, false, NULL);
+}
-/* FIXME: I assume threaded IRQchips do not have the lockdep problem */
static inline int gpiochip_irqchip_add_nested(struct gpio_chip *gpiochip,
struct irq_chip *irqchip,
unsigned int first_irq,
irq_flow_handler_t handler,
unsigned int type)
{
- return _gpiochip_irqchip_add(gpiochip, irqchip, first_irq,
- handler, type, true, NULL);
+ return gpiochip_irqchip_add_key(gpiochip, irqchip, first_irq,
+ handler, type, true, NULL);
}
-
-#ifdef CONFIG_LOCKDEP
-#define gpiochip_irqchip_add(...) \
-( \
- ({ \
- static struct lock_class_key _key; \
- _gpiochip_irqchip_add(__VA_ARGS__, false, &_key); \
- }) \
-)
-#else
-#define gpiochip_irqchip_add(...) \
- _gpiochip_irqchip_add(__VA_ARGS__, false, NULL)
-#endif
+#endif /* CONFIG_LOCKDEP */
#endif /* CONFIG_GPIOLIB_IRQCHIP */
#define I2C_CLIENT_TEN 0x10 /* we have a ten bit chip address */
/* Must equal I2C_M_TEN below */
#define I2C_CLIENT_SLAVE 0x20 /* we are the slave */
+#define I2C_CLIENT_HOST_NOTIFY 0x40 /* We want to use I2C host notify */
#define I2C_CLIENT_WAKE 0x80 /* for board_info; true iff can wake */
#define I2C_CLIENT_SCCB 0x9000 /* Use Omnivision SCCB protocol */
/* Must match I2C_M_STOP|IGNORE_NAK */
#ifdef HAVE_JUMP_LABEL
extern void static_key_slow_dec_deferred(struct static_key_deferred *key);
+extern void static_key_deferred_flush(struct static_key_deferred *key);
extern void
jump_label_rate_limit(struct static_key_deferred *key, unsigned long rl);
STATIC_KEY_CHECK_USE();
static_key_slow_dec(&key->key);
}
+static inline void static_key_deferred_flush(struct static_key_deferred *key)
+{
+ STATIC_KEY_CHECK_USE();
+}
static inline void
jump_label_rate_limit(struct static_key_deferred *key,
unsigned long rl)
#define TAINT_FLAGS_COUNT 16
struct taint_flag {
- char true; /* character printed when tainted */
- char false; /* character printed when not tainted */
+ char c_true; /* character printed when tainted */
+ char c_false; /* character printed when not tainted */
bool module; /* also show as a per-module taint flag */
};
*/
struct mem_cgroup_per_node {
struct lruvec lruvec;
- unsigned long lru_size[NR_LRU_LISTS];
+ unsigned long lru_zone_size[MAX_NR_ZONES][NR_LRU_LISTS];
struct mem_cgroup_reclaim_iter iter[DEF_PRIORITY + 1];
int mem_cgroup_select_victim_node(struct mem_cgroup *memcg);
void mem_cgroup_update_lru_size(struct lruvec *lruvec, enum lru_list lru,
- int nr_pages);
+ int zid, int nr_pages);
unsigned long mem_cgroup_node_nr_lru_pages(struct mem_cgroup *memcg,
int nid, unsigned int lru_mask);
unsigned long mem_cgroup_get_lru_size(struct lruvec *lruvec, enum lru_list lru)
{
struct mem_cgroup_per_node *mz;
+ unsigned long nr_pages = 0;
+ int zid;
mz = container_of(lruvec, struct mem_cgroup_per_node, lruvec);
- return mz->lru_size[lru];
+ for (zid = 0; zid < MAX_NR_ZONES; zid++)
+ nr_pages += mz->lru_zone_size[zid][lru];
+ return nr_pages;
+}
+
+static inline
+unsigned long mem_cgroup_get_zone_lru_size(struct lruvec *lruvec,
+ enum lru_list lru, int zone_idx)
+{
+ struct mem_cgroup_per_node *mz;
+
+ mz = container_of(lruvec, struct mem_cgroup_per_node, lruvec);
+ return mz->lru_zone_size[zone_idx][lru];
}
void mem_cgroup_handle_over_high(void);
{
return 0;
}
+static inline
+unsigned long mem_cgroup_get_zone_lru_size(struct lruvec *lruvec,
+ enum lru_list lru, int zone_idx)
+{
+ return 0;
+}
static inline unsigned long
mem_cgroup_node_nr_lru_pages(struct mem_cgroup *memcg,
unsigned long map_offset);
extern struct page *sparse_decode_mem_map(unsigned long coded_mem_map,
unsigned long pnum);
-extern int zone_can_shift(unsigned long pfn, unsigned long nr_pages,
- enum zone_type target);
+extern bool zone_can_shift(unsigned long pfn, unsigned long nr_pages,
+ enum zone_type target, int *zone_shift);
#endif /* __LINUX_MEMORY_HOTPLUG_H */
struct vm_area_struct *vma);
void unmap_mapping_range(struct address_space *mapping,
loff_t const holebegin, loff_t const holelen, int even_cows);
-int follow_pte(struct mm_struct *mm, unsigned long address, pte_t **ptepp,
- spinlock_t **ptlp);
+int follow_pte_pmd(struct mm_struct *mm, unsigned long address,
+ pte_t **ptepp, pmd_t **pmdpp, spinlock_t **ptlp);
int follow_pfn(struct vm_area_struct *vma, unsigned long address,
unsigned long *pfn);
int follow_phys(struct vm_area_struct *vma, unsigned long address,
{
__update_lru_size(lruvec, lru, zid, nr_pages);
#ifdef CONFIG_MEMCG
- mem_cgroup_update_lru_size(lruvec, lru, nr_pages);
+ mem_cgroup_update_lru_size(lruvec, lru, zid, nr_pages);
#endif
}
* @zonelist - The zonelist to search for a suitable zone
* @highest_zoneidx - The zone index of the highest zone to return
* @nodes - An optional nodemask to filter the zonelist with
- * @zone - The first suitable zone found is returned via this parameter
+ * @return - Zoneref pointer for the first suitable zone found (see below)
*
* This function returns the first zone at or below a given zone index that is
* within the allowed nodemask. The zoneref returned is a cursor that can be
* used to iterate the zonelist with next_zones_zonelist by advancing it by
* one before calling.
+ *
+ * When no eligible zone is found, zoneref->zone is NULL (zoneref itself is
+ * never NULL). This may happen either genuinely, or due to concurrent nodemask
+ * update due to cpuset modification.
*/
static inline struct zoneref *first_zones_zonelist(struct zonelist *zonelist,
enum zone_type highest_zoneidx,
return NAPI_GRO_CB(skb)->frag0_len < hlen;
}
+static inline void skb_gro_frag0_invalidate(struct sk_buff *skb)
+{
+ NAPI_GRO_CB(skb)->frag0 = NULL;
+ NAPI_GRO_CB(skb)->frag0_len = 0;
+}
+
static inline void *skb_gro_header_slow(struct sk_buff *skb, unsigned int hlen,
unsigned int offset)
{
if (!pskb_may_pull(skb, hlen))
return NULL;
- NAPI_GRO_CB(skb)->frag0 = NULL;
- NAPI_GRO_CB(skb)->frag0_len = 0;
+ skb_gro_frag0_invalidate(skb);
return skb->data + offset;
}
extern int watchdog_thresh;
extern unsigned long watchdog_enabled;
extern unsigned long *watchdog_cpumask_bits;
+extern atomic_t watchdog_park_in_progress;
#ifdef CONFIG_SMP
extern int sysctl_softlockup_all_cpu_backtrace;
extern int sysctl_hardlockup_all_cpu_backtrace;
extern void perf_event_disable_local(struct perf_event *event);
extern void perf_event_disable_inatomic(struct perf_event *event);
extern void perf_event_task_tick(void);
+extern int perf_event_account_interrupt(struct perf_event *event);
#else /* !CONFIG_PERF_EVENTS: */
static inline void *
perf_aux_output_begin(struct perf_output_handle *handle,
#error "Unknown RCU implementation specified to kernel configuration"
#endif
+#define RCU_SCHEDULER_INACTIVE 0
+#define RCU_SCHEDULER_INIT 1
+#define RCU_SCHEDULER_RUNNING 2
+
/*
* init_rcu_head_on_stack()/destroy_rcu_head_on_stack() are needed for dynamic
* initialization and destruction of rcu_head on the stack. rcu_head structures
* @crash_comp: completion used to sync crash handler and the rproc reload
* @recovery_disabled: flag that state if recovery was disabled
* @max_notifyid: largest allocated notify id.
- * @table_ptr: our copy of the resource table
+ * @table_ptr: pointer to the resource table in effect
+ * @cached_table: copy of the resource table
* @has_iommu: flag to indicate if remote processor is behind an MMU
*/
struct rproc {
bool recovery_disabled;
int max_notifyid;
struct resource_table *table_ptr;
+ struct resource_table *cached_table;
bool has_iommu;
bool auto_boot;
};
#define SIGNAL_UNKILLABLE 0x00000040 /* for init: ignore fatal signals */
+#define SIGNAL_STOP_MASK (SIGNAL_CLD_MASK | SIGNAL_STOP_STOPPED | \
+ SIGNAL_STOP_CONTINUED)
+
+static inline void signal_set_stop_flags(struct signal_struct *sig,
+ unsigned int flags)
+{
+ WARN_ON(sig->flags & (SIGNAL_GROUP_EXIT|SIGNAL_GROUP_COREDUMP));
+ sig->flags = (sig->flags & ~SIGNAL_STOP_MASK) | flags;
+}
+
/* If true, all threads except ->group_exit_task have pending SIGKILL */
static inline int signal_group_exit(const struct signal_struct *sig)
{
static inline void skb_free_frag(void *addr)
{
- __free_page_frag(addr);
+ page_frag_free(addr);
}
void *napi_alloc_frag(unsigned int fragsz);
* (PAGE_SIZE*2). Larger requests are passed to the page allocator.
*/
#define KMALLOC_SHIFT_HIGH (PAGE_SHIFT + 1)
-#define KMALLOC_SHIFT_MAX (MAX_ORDER + PAGE_SHIFT)
+#define KMALLOC_SHIFT_MAX (MAX_ORDER + PAGE_SHIFT - 1)
#ifndef KMALLOC_SHIFT_LOW
#define KMALLOC_SHIFT_LOW 3
#endif
* be allocated from the same page.
*/
#define KMALLOC_SHIFT_HIGH PAGE_SHIFT
-#define KMALLOC_SHIFT_MAX 30
+#define KMALLOC_SHIFT_MAX (MAX_ORDER + PAGE_SHIFT - 1)
#ifndef KMALLOC_SHIFT_LOW
#define KMALLOC_SHIFT_LOW 3
#endif
#define XPT_LISTENER 10 /* listening endpoint */
#define XPT_CACHE_AUTH 11 /* cache auth info */
#define XPT_LOCAL 12 /* connection from loopback interface */
+#define XPT_KILL_TEMP 13 /* call xpo_kill_temp_xprt before closing */
struct svc_serv *xpt_server; /* service for transport */
atomic_t xpt_reserved; /* space on outq that is rsvd */
SWP_FILE = (1 << 7), /* set after swap_activate success */
SWP_AREA_DISCARD = (1 << 8), /* single-time swap area discards */
SWP_PAGE_DISCARD = (1 << 9), /* freed swap page-cluster discards */
+ SWP_STABLE_WRITES = (1 << 10), /* no overwrite PG_writeback pages */
/* add others here before... */
- SWP_SCANNING = (1 << 10), /* refcount in scan_swap_map */
+ SWP_SCANNING = (1 << 11), /* refcount in scan_swap_map */
};
#define SWAP_CLUSTER_MAX 32UL
/* TCP Fast Open Cookie as stored in memory */
struct tcp_fastopen_cookie {
+ union {
+ u8 val[TCP_FASTOPEN_COOKIE_MAX];
+#if IS_ENABLED(CONFIG_IPV6)
+ struct in6_addr addr;
+#endif
+ };
s8 len;
- u8 val[TCP_FASTOPEN_COOKIE_MAX];
bool exp; /* In RFC6994 experimental option format */
};
#ifndef _LINUX_TIMERFD_H
#define _LINUX_TIMERFD_H
-/* For O_CLOEXEC and O_NONBLOCK */
-#include <linux/fcntl.h>
-
-/* For _IO helpers */
-#include <linux/ioctl.h>
-
-/*
- * CAREFUL: Check include/asm-generic/fcntl.h when defining
- * new flags, since they might collide with O_* ones. We want
- * to re-use O_* flags that couldn't possibly have a meaning
- * from eventfd, in order to leave a free define-space for
- * shared O_* flags.
- */
-#define TFD_TIMER_ABSTIME (1 << 0)
-#define TFD_TIMER_CANCEL_ON_SET (1 << 1)
-#define TFD_CLOEXEC O_CLOEXEC
-#define TFD_NONBLOCK O_NONBLOCK
+#include <uapi/linux/timerfd.h>
#define TFD_SHARED_FCNTL_FLAGS (TFD_CLOEXEC | TFD_NONBLOCK)
/* Flags for timerfd_create. */
/* Flags for timerfd_settime. */
#define TFD_SETTIME_FLAGS (TFD_TIMER_ABSTIME | TFD_TIMER_CANCEL_ON_SET)
-#define TFD_IOC_SET_TICKS _IOW('T', 0, u64)
-
#endif /* _LINUX_TIMERFD_H */
/**
* fc_set_wwpn() - Set the World Wide Port Name of a local port
* @lport: The local port whose WWPN is to be set
- * @wwnn: The new WWPN
+ * @wwpn: The new WWPN
*/
-static inline void fc_set_wwpn(struct fc_lport *lport, u64 wwnn)
+static inline void fc_set_wwpn(struct fc_lport *lport, u64 wwpn)
{
- lport->wwpn = wwnn;
+ lport->wwpn = wwpn;
}
/**
HDMI_AC97,
HDMI_SPDIF,
} fmt;
- int bit_clk_inv:1;
- int frame_clk_inv:1;
- int bit_clk_master:1;
- int frame_clk_master:1;
+ unsigned int bit_clk_inv:1;
+ unsigned int frame_clk_inv:1;
+ unsigned int bit_clk_master:1;
+ unsigned int frame_clk_master:1;
};
/*
unsigned int suspended:1; /* is in suspend PM state */
struct list_head list;
+ struct list_head card_aux_list; /* for auxiliary bound components */
struct list_head card_list;
struct snd_soc_dai_driver *dai_drv;
*/
struct snd_soc_aux_dev *aux_dev;
int num_aux_devs;
+ struct list_head aux_comp_list;
const struct snd_kcontrol_new *controls;
int num_controls;
INIT_LIST_HEAD(&card->widgets);
INIT_LIST_HEAD(&card->paths);
INIT_LIST_HEAD(&card->dapm_list);
+ INIT_LIST_HEAD(&card->aux_comp_list);
INIT_LIST_HEAD(&card->component_dev_list);
}
TCM_LOGICAL_BLOCK_APP_TAG_CHECK_FAILED = R(0x16),
TCM_LOGICAL_BLOCK_REF_TAG_CHECK_FAILED = R(0x17),
TCM_COPY_TARGET_DEVICE_NOT_REACHABLE = R(0x18),
+ TCM_TOO_MANY_TARGET_DESCS = R(0x19),
+ TCM_UNSUPPORTED_TARGET_DESC_TYPE_CODE = R(0x1a),
+ TCM_TOO_MANY_SEGMENT_DESCS = R(0x1b),
+ TCM_UNSUPPORTED_SEGMENT_DESC_TYPE_CODE = R(0x1c),
#undef R
};
BTRFS_I(inode)->root->root_key.objectid;
),
- TP_printk_btrfs("root = %llu(%s), gen = %llu, ino = %lu, blocks = %llu, "
- "disk_i_size = %llu, last_trans = %llu, logged_trans = %llu",
+ TP_printk_btrfs("root=%llu(%s) gen=%llu ino=%lu blocks=%llu "
+ "disk_i_size=%llu last_trans=%llu logged_trans=%llu",
show_root_type(__entry->root_objectid),
(unsigned long long)__entry->generation,
(unsigned long)__entry->ino,
TRACE_EVENT_CONDITION(btrfs_get_extent,
- TP_PROTO(struct btrfs_root *root, struct extent_map *map),
+ TP_PROTO(struct btrfs_root *root, struct inode *inode,
+ struct extent_map *map),
- TP_ARGS(root, map),
+ TP_ARGS(root, inode, map),
TP_CONDITION(map),
TP_STRUCT__entry_btrfs(
__field( u64, root_objectid )
+ __field( u64, ino )
__field( u64, start )
__field( u64, len )
__field( u64, orig_start )
TP_fast_assign_btrfs(root->fs_info,
__entry->root_objectid = root->root_key.objectid;
- __entry->start = map->start;
+ __entry->ino = btrfs_ino(inode);
+ __entry->start = map->start;
__entry->len = map->len;
__entry->orig_start = map->orig_start;
__entry->block_start = map->block_start;
__entry->compress_type = map->compress_type;
),
- TP_printk_btrfs("root = %llu(%s), start = %llu, len = %llu, "
- "orig_start = %llu, block_start = %llu(%s), "
- "block_len = %llu, flags = %s, refs = %u, "
- "compress_type = %u",
+ TP_printk_btrfs("root=%llu(%s) ino=%llu start=%llu len=%llu "
+ "orig_start=%llu block_start=%llu(%s) "
+ "block_len=%llu flags=%s refs=%u "
+ "compress_type=%u",
show_root_type(__entry->root_objectid),
+ (unsigned long long)__entry->ino,
(unsigned long long)__entry->start,
(unsigned long long)__entry->len,
(unsigned long long)__entry->orig_start,
__field( int, compress_type )
__field( int, refs )
__field( u64, root_objectid )
+ __field( u64, truncated_len )
),
TP_fast_assign_btrfs(btrfs_sb(inode->i_sb),
__entry->refs = atomic_read(&ordered->refs);
__entry->root_objectid =
BTRFS_I(inode)->root->root_key.objectid;
+ __entry->truncated_len = ordered->truncated_len;
),
- TP_printk_btrfs("root = %llu(%s), ino = %llu, file_offset = %llu, "
- "start = %llu, len = %llu, disk_len = %llu, "
- "bytes_left = %llu, flags = %s, compress_type = %d, "
- "refs = %d",
+ TP_printk_btrfs("root=%llu(%s) ino=%llu file_offset=%llu "
+ "start=%llu len=%llu disk_len=%llu "
+ "truncated_len=%llu "
+ "bytes_left=%llu flags=%s compress_type=%d "
+ "refs=%d",
show_root_type(__entry->root_objectid),
(unsigned long long)__entry->ino,
(unsigned long long)__entry->file_offset,
(unsigned long long)__entry->start,
(unsigned long long)__entry->len,
(unsigned long long)__entry->disk_len,
+ (unsigned long long)__entry->truncated_len,
(unsigned long long)__entry->bytes_left,
show_ordered_flags(__entry->flags),
__entry->compress_type, __entry->refs)
BTRFS_I(inode)->root->root_key.objectid;
),
- TP_printk_btrfs("root = %llu(%s), ino = %lu, page_index = %lu, "
- "nr_to_write = %ld, pages_skipped = %ld, range_start = %llu, "
- "range_end = %llu, for_kupdate = %d, "
- "for_reclaim = %d, range_cyclic = %d, writeback_index = %lu",
+ TP_printk_btrfs("root=%llu(%s) ino=%lu page_index=%lu "
+ "nr_to_write=%ld pages_skipped=%ld range_start=%llu "
+ "range_end=%llu for_kupdate=%d "
+ "for_reclaim=%d range_cyclic=%d writeback_index=%lu",
show_root_type(__entry->root_objectid),
(unsigned long)__entry->ino, __entry->index,
__entry->nr_to_write, __entry->pages_skipped,
BTRFS_I(page->mapping->host)->root->root_key.objectid;
),
- TP_printk_btrfs("root = %llu(%s), ino = %lu, page_index = %lu, start = %llu, "
- "end = %llu, uptodate = %d",
+ TP_printk_btrfs("root=%llu(%s) ino=%lu page_index=%lu start=%llu "
+ "end=%llu uptodate=%d",
show_root_type(__entry->root_objectid),
(unsigned long)__entry->ino, (unsigned long)__entry->index,
(unsigned long long)__entry->start,
BTRFS_I(inode)->root->root_key.objectid;
),
- TP_printk_btrfs("root = %llu(%s), ino = %ld, parent = %ld, datasync = %d",
+ TP_printk_btrfs("root=%llu(%s) ino=%ld parent=%ld datasync=%d",
show_root_type(__entry->root_objectid),
(unsigned long)__entry->ino, (unsigned long)__entry->parent,
__entry->datasync)
__entry->create = create;
),
- TP_printk("%pU: block_group offset = %llu, size = %llu, "
- "flags = %llu(%s), bytes_used = %llu, bytes_super = %llu, "
- "create = %d", __entry->fsid,
+ TP_printk("%pU: block_group offset=%llu size=%llu "
+ "flags=%llu(%s) bytes_used=%llu bytes_super=%llu "
+ "create=%d", __entry->fsid,
(unsigned long long)__entry->offset,
(unsigned long long)__entry->size,
(unsigned long long)__entry->flags,
__entry->seq = ref->seq;
),
- TP_printk_btrfs("bytenr = %llu, num_bytes = %llu, action = %s, "
- "parent = %llu(%s), ref_root = %llu(%s), level = %d, "
- "type = %s, seq = %llu",
+ TP_printk_btrfs("bytenr=%llu num_bytes=%llu action=%s "
+ "parent=%llu(%s) ref_root=%llu(%s) level=%d "
+ "type=%s seq=%llu",
(unsigned long long)__entry->bytenr,
(unsigned long long)__entry->num_bytes,
show_ref_action(__entry->action),
__entry->seq = ref->seq;
),
- TP_printk_btrfs("bytenr = %llu, num_bytes = %llu, action = %s, "
- "parent = %llu(%s), ref_root = %llu(%s), owner = %llu, "
- "offset = %llu, type = %s, seq = %llu",
+ TP_printk_btrfs("bytenr=%llu num_bytes=%llu action=%s "
+ "parent=%llu(%s) ref_root=%llu(%s) owner=%llu "
+ "offset=%llu type=%s seq=%llu",
(unsigned long long)__entry->bytenr,
(unsigned long long)__entry->num_bytes,
show_ref_action(__entry->action),
__entry->is_data = head_ref->is_data;
),
- TP_printk_btrfs("bytenr = %llu, num_bytes = %llu, action = %s, is_data = %d",
+ TP_printk_btrfs("bytenr=%llu num_bytes=%llu action=%s is_data=%d",
(unsigned long long)__entry->bytenr,
(unsigned long long)__entry->num_bytes,
show_ref_action(__entry->action),
__entry->root_objectid = fs_info->chunk_root->root_key.objectid;
),
- TP_printk_btrfs("root = %llu(%s), offset = %llu, size = %llu, "
- "num_stripes = %d, sub_stripes = %d, type = %s",
+ TP_printk_btrfs("root=%llu(%s) offset=%llu size=%llu "
+ "num_stripes=%d sub_stripes=%d type=%s",
show_root_type(__entry->root_objectid),
(unsigned long long)__entry->offset,
(unsigned long long)__entry->size,
__entry->cow_level = btrfs_header_level(cow);
),
- TP_printk_btrfs("root = %llu(%s), refs = %d, orig_buf = %llu "
- "(orig_level = %d), cow_buf = %llu (cow_level = %d)",
+ TP_printk_btrfs("root=%llu(%s) refs=%d orig_buf=%llu "
+ "(orig_level=%d) cow_buf=%llu (cow_level=%d)",
show_root_type(__entry->root_objectid),
__entry->refs,
(unsigned long long)__entry->buf_start,
__assign_str(reason, reason)
),
- TP_printk("%pU: %s: flush = %d(%s), flags = %llu(%s), bytes = %llu",
+ TP_printk("%pU: %s: flush=%d(%s) flags=%llu(%s) bytes=%llu",
__entry->fsid, __get_str(reason), __entry->flush,
show_flush_action(__entry->flush),
(unsigned long long)__entry->flags,
__entry->ret = ret;
),
- TP_printk("%pU: state = %d(%s), flags = %llu(%s), num_bytes = %llu, "
- "orig_bytes = %llu, ret = %d", __entry->fsid, __entry->state,
+ TP_printk("%pU: state=%d(%s) flags=%llu(%s) num_bytes=%llu "
+ "orig_bytes=%llu ret=%d", __entry->fsid, __entry->state,
show_flush_state(__entry->state),
(unsigned long long)__entry->flags,
__print_flags((unsigned long)__entry->flags, "|",
__entry->len = len;
),
- TP_printk_btrfs("root = %llu(%s), start = %llu, len = %llu",
+ TP_printk_btrfs("root=%llu(%s) start=%llu len=%llu",
show_root_type(BTRFS_EXTENT_TREE_OBJECTID),
(unsigned long long)__entry->start,
(unsigned long long)__entry->len)
__entry->data = data;
),
- TP_printk_btrfs("root = %Lu(%s), len = %Lu, empty_size = %Lu, flags = %Lu(%s)",
+ TP_printk_btrfs("root=%Lu(%s) len=%Lu empty_size=%Lu flags=%Lu(%s)",
show_root_type(BTRFS_EXTENT_TREE_OBJECTID),
__entry->num_bytes, __entry->empty_size, __entry->data,
__print_flags((unsigned long)__entry->data, "|",
__entry->len = len;
),
- TP_printk_btrfs("root = %Lu(%s), block_group = %Lu, flags = %Lu(%s), "
- "start = %Lu, len = %Lu",
+ TP_printk_btrfs("root=%Lu(%s) block_group=%Lu flags=%Lu(%s) "
+ "start=%Lu len=%Lu",
show_root_type(BTRFS_EXTENT_TREE_OBJECTID),
__entry->bg_objectid,
__entry->flags, __print_flags((unsigned long)__entry->flags,
__entry->min_bytes = min_bytes;
),
- TP_printk_btrfs("block_group = %Lu, flags = %Lu(%s), start = %Lu, len = %Lu,"
- " empty_size = %Lu, min_bytes = %Lu", __entry->bg_objectid,
+ TP_printk_btrfs("block_group=%Lu flags=%Lu(%s) start=%Lu len=%Lu "
+ "empty_size=%Lu min_bytes=%Lu", __entry->bg_objectid,
__entry->flags,
__print_flags((unsigned long)__entry->flags, "|",
BTRFS_GROUP_FLAGS), __entry->start,
__entry->bg_objectid = block_group->key.objectid;
),
- TP_printk_btrfs("block_group = %Lu", __entry->bg_objectid)
+ TP_printk_btrfs("block_group=%Lu", __entry->bg_objectid)
);
TRACE_EVENT(btrfs_setup_cluster,
__entry->bitmap = bitmap;
),
- TP_printk_btrfs("block_group = %Lu, flags = %Lu(%s), window_start = %Lu, "
- "size = %Lu, max_size = %Lu, bitmap = %d",
+ TP_printk_btrfs("block_group=%Lu flags=%Lu(%s) window_start=%Lu "
+ "size=%Lu max_size=%Lu bitmap=%d",
__entry->bg_objectid,
__entry->flags,
__print_flags((unsigned long)__entry->flags, "|",
__entry->ip = IP
),
- TP_printk("state=%p; mask = %s; caller = %pS", __entry->state,
+ TP_printk("state=%p mask=%s caller=%pS", __entry->state,
show_gfp_flags(__entry->mask), (void *)__entry->ip)
);
__entry->ip = IP
),
- TP_printk(" state=%p; caller = %pS", __entry->state,
+ TP_printk("state=%p caller=%pS", __entry->state,
(void *)__entry->ip)
);
__entry->normal_work = &work->normal_work;
),
- TP_printk_btrfs("work=%p (normal_work=%p), wq=%p, func=%pf, ordered_func=%p,"
- " ordered_free=%p",
+ TP_printk_btrfs("work=%p (normal_work=%p) wq=%p func=%pf ordered_func=%p "
+ "ordered_free=%p",
__entry->work, __entry->normal_work, __entry->wq,
__entry->func, __entry->ordered_func, __entry->ordered_free)
);
-/* For situiations that the work is freed */
+/*
+ * For situiations when the work is freed, we pass fs_info and a tag that that
+ * matches address of the work structure so it can be paired with the
+ * scheduling event.
+ */
DECLARE_EVENT_CLASS(btrfs__work__done,
- TP_PROTO(struct btrfs_work *work),
+ TP_PROTO(struct btrfs_fs_info *fs_info, void *wtag),
- TP_ARGS(work),
+ TP_ARGS(fs_info, wtag),
TP_STRUCT__entry_btrfs(
- __field( void *, work )
+ __field( void *, wtag )
),
- TP_fast_assign_btrfs(btrfs_work_owner(work),
- __entry->work = work;
+ TP_fast_assign_btrfs(fs_info,
+ __entry->wtag = wtag;
),
- TP_printk_btrfs("work->%p", __entry->work)
+ TP_printk_btrfs("work->%p", __entry->wtag)
);
DEFINE_EVENT(btrfs__work, btrfs_work_queued,
DEFINE_EVENT(btrfs__work__done, btrfs_all_work_done,
- TP_PROTO(struct btrfs_work *work),
+ TP_PROTO(struct btrfs_fs_info *fs_info, void *wtag),
- TP_ARGS(work)
+ TP_ARGS(fs_info, wtag)
);
DEFINE_EVENT(btrfs__work, btrfs_ordered_sched,
__entry->high = high;
),
- TP_printk_btrfs("name=%s%s, wq=%p", __get_str(name),
+ TP_printk_btrfs("name=%s%s wq=%p", __get_str(name),
__print_flags(__entry->high, "",
{(WQ_HIGHPRI), "-high"}),
__entry->wq)
__entry->free_reserved = free_reserved;
),
- TP_printk_btrfs("rootid=%llu, ino=%lu, free_reserved=%llu",
+ TP_printk_btrfs("rootid=%llu ino=%lu free_reserved=%llu",
__entry->rootid, __entry->ino, __entry->free_reserved)
);
__entry->op = op;
),
- TP_printk_btrfs("root=%llu, ino=%lu, start=%llu, len=%llu, reserved=%llu, op=%s",
+ TP_printk_btrfs("root=%llu ino=%lu start=%llu len=%llu reserved=%llu op=%s",
__entry->rootid, __entry->ino, __entry->start, __entry->len,
__entry->reserved,
__print_flags((unsigned long)__entry->op, "",
__entry->reserved = reserved;
),
- TP_printk_btrfs("root=%llu, reserved=%llu, op=free",
+ TP_printk_btrfs("root=%llu reserved=%llu op=free",
__entry->ref_root, __entry->reserved)
);
__entry->num_bytes = rec->num_bytes;
),
- TP_printk_btrfs("bytenr = %llu, num_bytes = %llu",
+ TP_printk_btrfs("bytenr=%llu num_bytes=%llu",
(unsigned long long)__entry->bytenr,
(unsigned long long)__entry->num_bytes)
);
__entry->nr_new_roots = nr_new_roots;
),
- TP_printk_btrfs("bytenr = %llu, num_bytes = %llu, nr_old_roots = %llu, "
- "nr_new_roots = %llu",
+ TP_printk_btrfs("bytenr=%llu num_bytes=%llu nr_old_roots=%llu "
+ "nr_new_roots=%llu",
__entry->bytenr,
__entry->num_bytes,
__entry->nr_old_roots,
__entry->cur_new_count = cur_new_count;
),
- TP_printk_btrfs("qgid = %llu, cur_old_count = %llu, cur_new_count = %llu",
+ TP_printk_btrfs("qgid=%llu cur_old_count=%llu cur_new_count=%llu",
__entry->qgid,
__entry->cur_old_count,
__entry->cur_new_count)
{(unsigned long)__GFP_WRITE, "__GFP_WRITE"}, \
{(unsigned long)__GFP_RECLAIM, "__GFP_RECLAIM"}, \
{(unsigned long)__GFP_DIRECT_RECLAIM, "__GFP_DIRECT_RECLAIM"},\
- {(unsigned long)__GFP_KSWAPD_RECLAIM, "__GFP_KSWAPD_RECLAIM"},\
- {(unsigned long)__GFP_OTHER_NODE, "__GFP_OTHER_NODE"} \
+ {(unsigned long)__GFP_KSWAPD_RECLAIM, "__GFP_KSWAPD_RECLAIM"}\
#define show_gfp_flags(flags) \
(flags) ? __print_flags(flags, "|", \
header-y += i915_drm.h
header-y += mga_drm.h
header-y += nouveau_drm.h
+header-y += omap_drm.h
header-y += qxl_drm.h
header-y += r128_drm.h
header-y += radeon_drm.h
* priorities and the driver will attempt to execute batches in priority order.
*/
#define I915_PARAM_HAS_SCHEDULER 41
+#define I915_PARAM_HUC_STATUS 42
typedef struct drm_i915_getparam {
__s32 param;
header-y += termios.h
header-y += thermal.h
header-y += time.h
+header-y += timerfd.h
header-y += times.h
header-y += timex.h
header-y += tiocl.h
*
* @NL80211_ATTR_OPMODE_NOTIF: Operating mode field from Operating Mode
* Notification Element based on association request when used with
- * %NL80211_CMD_NEW_STATION; u8 attribute.
+ * %NL80211_CMD_NEW_STATION or %NL80211_CMD_SET_STATION (only when
+ * %NL80211_FEATURE_FULL_AP_CLIENT_STATE is supported, or with TDLS);
+ * u8 attribute.
*
* @NL80211_ATTR_VENDOR_ID: The vendor ID, either a 24-bit OUI or, if
* %NL80211_VENDOR_ID_IS_LINUX is set, a special Linux ID (not used yet)
TCA_BPF_NAME,
TCA_BPF_FLAGS,
TCA_BPF_FLAGS_GEN,
- TCA_BPF_DIGEST,
+ TCA_BPF_TAG,
__TCA_BPF_MAX,
};
TCA_ACT_BPF_FD,
TCA_ACT_BPF_NAME,
TCA_ACT_BPF_PAD,
- TCA_ACT_BPF_DIGEST,
+ TCA_ACT_BPF_TAG,
__TCA_ACT_BPF_MAX,
};
#define TCA_ACT_BPF_MAX (__TCA_ACT_BPF_MAX - 1)
--- /dev/null
+/*
+ * include/linux/timerfd.h
+ *
+ * Copyright (C) 2007 Davide Libenzi <davidel@xmailserver.org>
+ *
+ */
+
+#ifndef _UAPI_LINUX_TIMERFD_H
+#define _UAPI_LINUX_TIMERFD_H
+
+#include <linux/types.h>
+
+/* For O_CLOEXEC and O_NONBLOCK */
+#include <linux/fcntl.h>
+
+/* For _IO helpers */
+#include <linux/ioctl.h>
+
+/*
+ * CAREFUL: Check include/asm-generic/fcntl.h when defining
+ * new flags, since they might collide with O_* ones. We want
+ * to re-use O_* flags that couldn't possibly have a meaning
+ * from eventfd, in order to leave a free define-space for
+ * shared O_* flags.
+ *
+ * Also make sure to update the masks in include/linux/timerfd.h
+ * when adding new flags.
+ */
+#define TFD_TIMER_ABSTIME (1 << 0)
+#define TFD_TIMER_CANCEL_ON_SET (1 << 1)
+#define TFD_CLOEXEC O_CLOEXEC
+#define TFD_NONBLOCK O_NONBLOCK
+
+#define TFD_IOC_SET_TICKS _IOW('T', 0, __u64)
+
+#endif /* _UAPI_LINUX_TIMERFD_H */
Say N.
+config SOCK_CGROUP_DATA
+ bool
+ default n
+
endif # CGROUPS
config CHECKPOINT_RESTORE
}
rcu_read_lock();
- sem_lock(sma, sops, nsops);
+ locknum = sem_lock(sma, sops, nsops);
if (!ipc_valid_object(&sma->sem_perm))
goto out_unlock_free;
attr->value_size == 0 || attr->map_flags)
return ERR_PTR(-EINVAL);
- if (attr->value_size >= 1 << (KMALLOC_SHIFT_MAX - 1))
+ if (attr->value_size > KMALLOC_MAX_SIZE)
/* if value_size is bigger, the user space won't be able to
* access the elements.
*/
vfree(fp);
}
-int bpf_prog_calc_digest(struct bpf_prog *fp)
+int bpf_prog_calc_tag(struct bpf_prog *fp)
{
const u32 bits_offset = SHA_MESSAGE_BYTES - sizeof(__be64);
- u32 raw_size = bpf_prog_digest_scratch_size(fp);
+ u32 raw_size = bpf_prog_tag_scratch_size(fp);
+ u32 digest[SHA_DIGEST_WORDS];
u32 ws[SHA_WORKSPACE_WORDS];
u32 i, bsize, psize, blocks;
struct bpf_insn *dst;
if (!raw)
return -ENOMEM;
- sha_init(fp->digest);
+ sha_init(digest);
memset(ws, 0, sizeof(ws));
/* We need to take out the map fd for the digest calculation
*bits = cpu_to_be64((psize - 1) << 3);
while (blocks--) {
- sha_transform(fp->digest, todo, ws);
+ sha_transform(digest, todo, ws);
todo += SHA_MESSAGE_BYTES;
}
- result = (__force __be32 *)fp->digest;
+ result = (__force __be32 *)digest;
for (i = 0; i < SHA_DIGEST_WORDS; i++)
- result[i] = cpu_to_be32(fp->digest[i]);
+ result[i] = cpu_to_be32(digest[i]);
+ memcpy(fp->tag, result, sizeof(fp->tag));
vfree(raw);
return 0;
*/
goto free_htab;
- if (htab->map.value_size >= (1 << (KMALLOC_SHIFT_MAX - 1)) -
+ if (htab->map.value_size >= KMALLOC_MAX_SIZE -
MAX_BPF_STACK - sizeof(struct htab_elem))
/* if value_size is bigger, the user space won't be able to
* access the elements via bpf syscall. This check also makes
static void bpf_prog_show_fdinfo(struct seq_file *m, struct file *filp)
{
const struct bpf_prog *prog = filp->private_data;
- char prog_digest[sizeof(prog->digest) * 2 + 1] = { };
+ char prog_tag[sizeof(prog->tag) * 2 + 1] = { };
- bin2hex(prog_digest, prog->digest, sizeof(prog->digest));
+ bin2hex(prog_tag, prog->tag, sizeof(prog->tag));
seq_printf(m,
"prog_type:\t%u\n"
"prog_jited:\t%u\n"
- "prog_digest:\t%s\n"
+ "prog_tag:\t%s\n"
"memlock:\t%llu\n",
prog->type,
prog->jited,
- prog_digest,
+ prog_tag,
prog->pages * 1ULL << PAGE_SHIFT);
}
#endif
int insn_cnt = env->prog->len;
int i, j, err;
- err = bpf_prog_calc_digest(env->prog);
+ err = bpf_prog_calc_tag(env->prog);
if (err)
return err;
{
return has_ns_capability(t, &init_user_ns, cap);
}
+EXPORT_SYMBOL(has_capability);
/**
* has_ns_capability_noaudit - Does a task have a capability (unaudited)
{
struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
int prev_state, ret = 0;
- bool hasdied = false;
if (num_online_cpus() == 1)
return -EBUSY;
cpuhp_kick_ap_work(cpu);
}
- hasdied = prev_state != st->state && st->state == CPUHP_OFFLINE;
out:
cpu_hotplug_done();
return ret;
*/
static int cpuhp_reserve_state(enum cpuhp_state state)
{
- enum cpuhp_state i;
+ enum cpuhp_state i, end;
+ struct cpuhp_step *step;
- for (i = CPUHP_AP_ONLINE_DYN; i <= CPUHP_AP_ONLINE_DYN_END; i++) {
- if (!cpuhp_ap_states[i].name)
+ switch (state) {
+ case CPUHP_AP_ONLINE_DYN:
+ step = cpuhp_ap_states + CPUHP_AP_ONLINE_DYN;
+ end = CPUHP_AP_ONLINE_DYN_END;
+ break;
+ case CPUHP_BP_PREPARE_DYN:
+ step = cpuhp_bp_states + CPUHP_BP_PREPARE_DYN;
+ end = CPUHP_BP_PREPARE_DYN_END;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ for (i = state; i <= end; i++, step++) {
+ if (!step->name)
return i;
}
WARN(1, "No more dynamic states available for CPU hotplug\n");
mutex_lock(&cpuhp_state_mutex);
- if (state == CPUHP_AP_ONLINE_DYN) {
+ if (state == CPUHP_AP_ONLINE_DYN || state == CPUHP_BP_PREPARE_DYN) {
ret = cpuhp_reserve_state(state);
if (ret < 0)
goto out;
struct perf_event_context *ctx = event->ctx;
struct perf_cpu_context *cpuctx = __get_cpu_context(ctx);
struct perf_event_context *task_ctx = cpuctx->task_ctx;
- bool activate = true;
+ bool reprogram = true;
int ret = 0;
raw_spin_lock(&cpuctx->ctx.lock);
raw_spin_lock(&ctx->lock);
task_ctx = ctx;
- /* If we're on the wrong CPU, try again */
- if (task_cpu(ctx->task) != smp_processor_id()) {
- ret = -ESRCH;
- goto unlock;
- }
+ reprogram = (ctx->task == current);
/*
- * If we're on the right CPU, see if the task we target is
- * current, if not we don't have to activate the ctx, a future
- * context switch will do that for us.
+ * If the task is running, it must be running on this CPU,
+ * otherwise we cannot reprogram things.
+ *
+ * If its not running, we don't care, ctx->lock will
+ * serialize against it becoming runnable.
*/
- if (ctx->task != current)
- activate = false;
- else
- WARN_ON_ONCE(cpuctx->task_ctx && cpuctx->task_ctx != ctx);
+ if (task_curr(ctx->task) && !reprogram) {
+ ret = -ESRCH;
+ goto unlock;
+ }
+ WARN_ON_ONCE(reprogram && cpuctx->task_ctx && cpuctx->task_ctx != ctx);
} else if (task_ctx) {
raw_spin_lock(&task_ctx->lock);
}
- if (activate) {
+ if (reprogram) {
ctx_sched_out(ctx, cpuctx, EVENT_TIME);
add_event_to_ctx(event, ctx);
ctx_resched(cpuctx, task_ctx);
/*
* Installing events is tricky because we cannot rely on ctx->is_active
* to be set in case this is the nr_events 0 -> 1 transition.
+ *
+ * Instead we use task_curr(), which tells us if the task is running.
+ * However, since we use task_curr() outside of rq::lock, we can race
+ * against the actual state. This means the result can be wrong.
+ *
+ * If we get a false positive, we retry, this is harmless.
+ *
+ * If we get a false negative, things are complicated. If we are after
+ * perf_event_context_sched_in() ctx::lock will serialize us, and the
+ * value must be correct. If we're before, it doesn't matter since
+ * perf_event_context_sched_in() will program the counter.
+ *
+ * However, this hinges on the remote context switch having observed
+ * our task->perf_event_ctxp[] store, such that it will in fact take
+ * ctx::lock in perf_event_context_sched_in().
+ *
+ * We do this by task_function_call(), if the IPI fails to hit the task
+ * we know any future context switch of task must see the
+ * perf_event_ctpx[] store.
*/
-again:
+
/*
- * Cannot use task_function_call() because we need to run on the task's
- * CPU regardless of whether its current or not.
+ * This smp_mb() orders the task->perf_event_ctxp[] store with the
+ * task_cpu() load, such that if the IPI then does not find the task
+ * running, a future context switch of that task must observe the
+ * store.
*/
- if (!cpu_function_call(task_cpu(task), __perf_install_in_context, event))
+ smp_mb();
+again:
+ if (!task_function_call(task, __perf_install_in_context, event))
return;
raw_spin_lock_irq(&ctx->lock);
raw_spin_unlock_irq(&ctx->lock);
return;
}
- raw_spin_unlock_irq(&ctx->lock);
/*
- * Since !ctx->is_active doesn't mean anything, we must IPI
- * unconditionally.
+ * If the task is not running, ctx->lock will avoid it becoming so,
+ * thus we can safely install the event.
*/
- goto again;
+ if (task_curr(task)) {
+ raw_spin_unlock_irq(&ctx->lock);
+ goto again;
+ }
+ add_event_to_ctx(event, ctx);
+ raw_spin_unlock_irq(&ctx->lock);
}
/*
perf_output_end(&handle);
}
-/*
- * Generic event overflow handling, sampling.
- */
-
-static int __perf_event_overflow(struct perf_event *event,
- int throttle, struct perf_sample_data *data,
- struct pt_regs *regs)
+static int
+__perf_event_account_interrupt(struct perf_event *event, int throttle)
{
- int events = atomic_read(&event->event_limit);
struct hw_perf_event *hwc = &event->hw;
- u64 seq;
int ret = 0;
-
- /*
- * Non-sampling counters might still use the PMI to fold short
- * hardware counters, ignore those.
- */
- if (unlikely(!is_sampling_event(event)))
- return 0;
+ u64 seq;
seq = __this_cpu_read(perf_throttled_seq);
if (seq != hwc->interrupts_seq) {
perf_adjust_period(event, delta, hwc->last_period, true);
}
+ return ret;
+}
+
+int perf_event_account_interrupt(struct perf_event *event)
+{
+ return __perf_event_account_interrupt(event, 1);
+}
+
+/*
+ * Generic event overflow handling, sampling.
+ */
+
+static int __perf_event_overflow(struct perf_event *event,
+ int throttle, struct perf_sample_data *data,
+ struct pt_regs *regs)
+{
+ int events = atomic_read(&event->event_limit);
+ int ret = 0;
+
+ /*
+ * Non-sampling counters might still use the PMI to fold short
+ * hardware counters, ignore those.
+ */
+ if (unlikely(!is_sampling_event(event)))
+ return 0;
+
+ ret = __perf_event_account_interrupt(event, throttle);
+
/*
* XXX event_limit might not quite work as expected on inherited
* events
return 0;
}
+/*
+ * Variation on perf_event_ctx_lock_nested(), except we take two context
+ * mutexes.
+ */
+static struct perf_event_context *
+__perf_event_ctx_lock_double(struct perf_event *group_leader,
+ struct perf_event_context *ctx)
+{
+ struct perf_event_context *gctx;
+
+again:
+ rcu_read_lock();
+ gctx = READ_ONCE(group_leader->ctx);
+ if (!atomic_inc_not_zero(&gctx->refcount)) {
+ rcu_read_unlock();
+ goto again;
+ }
+ rcu_read_unlock();
+
+ mutex_lock_double(&gctx->mutex, &ctx->mutex);
+
+ if (group_leader->ctx != gctx) {
+ mutex_unlock(&ctx->mutex);
+ mutex_unlock(&gctx->mutex);
+ put_ctx(gctx);
+ goto again;
+ }
+
+ return gctx;
+}
+
/**
* sys_perf_event_open - open a performance event, associate it to a task/cpu
*
}
if (move_group) {
- gctx = group_leader->ctx;
- mutex_lock_double(&gctx->mutex, &ctx->mutex);
+ gctx = __perf_event_ctx_lock_double(group_leader, ctx);
+
if (gctx->task == TASK_TOMBSTONE) {
err = -ESRCH;
goto err_locked;
}
+
+ /*
+ * Check if we raced against another sys_perf_event_open() call
+ * moving the software group underneath us.
+ */
+ if (!(group_leader->group_caps & PERF_EV_CAP_SOFTWARE)) {
+ /*
+ * If someone moved the group out from under us, check
+ * if this new event wound up on the same ctx, if so
+ * its the regular !move_group case, otherwise fail.
+ */
+ if (gctx != ctx) {
+ err = -EINVAL;
+ goto err_locked;
+ } else {
+ perf_event_ctx_unlock(group_leader, gctx);
+ move_group = 0;
+ }
+ }
} else {
mutex_lock(&ctx->mutex);
}
perf_unpin_context(ctx);
if (move_group)
- mutex_unlock(&gctx->mutex);
+ perf_event_ctx_unlock(group_leader, gctx);
mutex_unlock(&ctx->mutex);
if (task) {
err_locked:
if (move_group)
- mutex_unlock(&gctx->mutex);
+ perf_event_ctx_unlock(group_leader, gctx);
mutex_unlock(&ctx->mutex);
/* err_file: */
fput(event_file);
}
EXPORT_SYMBOL_GPL(static_key_slow_dec_deferred);
+void static_key_deferred_flush(struct static_key_deferred *key)
+{
+ STATIC_KEY_CHECK_USE();
+ flush_delayed_work(&key->work);
+}
+EXPORT_SYMBOL_GPL(static_key_deferred_flush);
+
void jump_label_rate_limit(struct static_key_deferred *key,
unsigned long rl)
{
/* pages are dead and unused, undo the arch mapping */
align_start = res->start & ~(SECTION_SIZE - 1);
align_size = ALIGN(resource_size(res), SECTION_SIZE);
+ mem_hotplug_begin();
arch_remove_memory(align_start, align_size);
+ mem_hotplug_done();
untrack_pfn(NULL, PHYS_PFN(align_start), align_size);
pgmap_radix_release(res);
dev_WARN_ONCE(dev, pgmap->altmap && pgmap->altmap->alloc,
if (error)
goto err_pfn_remap;
+ mem_hotplug_begin();
error = arch_add_memory(nid, align_start, align_size, true);
+ mem_hotplug_done();
if (error)
goto err_add_memory;
for (i = 0; i < TAINT_FLAGS_COUNT; i++) {
if (taint_flags[i].module && test_bit(i, &mod->taints))
- buf[l++] = taint_flags[i].true;
+ buf[l++] = taint_flags[i].c_true;
}
return l;
* Delay timeout seconds before rebooting the machine.
* We can't use the "normal" timers since we just panicked.
*/
- pr_emerg("Rebooting in %d seconds..", panic_timeout);
+ pr_emerg("Rebooting in %d seconds..\n", panic_timeout);
for (i = 0; i < panic_timeout * 1000; i += PANIC_TIMER_STEP) {
touch_nmi_watchdog();
for (i = 0; i < TAINT_FLAGS_COUNT; i++) {
const struct taint_flag *t = &taint_flags[i];
*s++ = test_bit(i, &tainted_mask) ?
- t->true : t->false;
+ t->c_true : t->c_false;
}
*s = 0;
} else
static void delayed_free_pidns(struct rcu_head *p)
{
- kmem_cache_free(pid_ns_cachep,
- container_of(p, struct pid_namespace, rcu));
+ struct pid_namespace *ns = container_of(p, struct pid_namespace, rcu);
+
+ dec_pid_namespaces(ns->ucounts);
+ put_user_ns(ns->user_ns);
+
+ kmem_cache_free(pid_ns_cachep, ns);
}
static void destroy_pid_namespace(struct pid_namespace *ns)
ns_free_inum(&ns->ns);
for (i = 0; i < PIDMAP_ENTRIES; i++)
kfree(ns->pidmap[i].page);
- dec_pid_namespaces(ns->ucounts);
- put_user_ns(ns->user_ns);
call_rcu(&ns->rcu, delayed_free_pidns);
}
#define TPS(x) tracepoint_string(x)
void rcu_early_boot_tests(void);
+void rcu_test_sync_prims(void);
/*
* This function really isn't for public consumption, but RCU is special in
* benefits of doing might_sleep() to reduce latency.)
*
* Cool, huh? (Due to Josh Triplett.)
- *
- * But we want to make this a static inline later. The cond_resched()
- * currently makes this problematic.
*/
void synchronize_sched(void)
{
lock_is_held(&rcu_lock_map) ||
lock_is_held(&rcu_sched_lock_map),
"Illegal synchronize_sched() in RCU read-side critical section");
- cond_resched();
}
EXPORT_SYMBOL_GPL(synchronize_sched);
/*
* During boot, we forgive RCU lockdep issues. After this function is
- * invoked, we start taking RCU lockdep issues seriously.
+ * invoked, we start taking RCU lockdep issues seriously. Note that unlike
+ * Tree RCU, Tiny RCU transitions directly from RCU_SCHEDULER_INACTIVE
+ * to RCU_SCHEDULER_RUNNING, skipping the RCU_SCHEDULER_INIT stage.
+ * The reason for this is that Tiny RCU does not need kthreads, so does
+ * not have to care about the fact that the scheduler is half-initialized
+ * at a certain phase of the boot process.
*/
void __init rcu_scheduler_starting(void)
{
WARN_ON(nr_context_switches() > 0);
- rcu_scheduler_active = 1;
+ rcu_scheduler_active = RCU_SCHEDULER_RUNNING;
}
#endif /* #ifdef CONFIG_DEBUG_LOCK_ALLOC */
int sysctl_panic_on_rcu_stall __read_mostly;
/*
- * The rcu_scheduler_active variable transitions from zero to one just
- * before the first task is spawned. So when this variable is zero, RCU
- * can assume that there is but one task, allowing RCU to (for example)
+ * The rcu_scheduler_active variable is initialized to the value
+ * RCU_SCHEDULER_INACTIVE and transitions RCU_SCHEDULER_INIT just before the
+ * first task is spawned. So when this variable is RCU_SCHEDULER_INACTIVE,
+ * RCU can assume that there is but one task, allowing RCU to (for example)
* optimize synchronize_rcu() to a simple barrier(). When this variable
- * is one, RCU must actually do all the hard work required to detect real
- * grace periods. This variable is also used to suppress boot-time false
- * positives from lockdep-RCU error checking.
+ * is RCU_SCHEDULER_INIT, RCU must actually do all the hard work required
+ * to detect real grace periods. This variable is also used to suppress
+ * boot-time false positives from lockdep-RCU error checking. Finally, it
+ * transitions from RCU_SCHEDULER_INIT to RCU_SCHEDULER_RUNNING after RCU
+ * is fully initialized, including all of its kthreads having been spawned.
*/
int rcu_scheduler_active __read_mostly;
EXPORT_SYMBOL_GPL(rcu_scheduler_active);
early_initcall(rcu_spawn_gp_kthread);
/*
- * This function is invoked towards the end of the scheduler's initialization
- * process. Before this is called, the idle task might contain
- * RCU read-side critical sections (during which time, this idle
- * task is booting the system). After this function is called, the
- * idle tasks are prohibited from containing RCU read-side critical
- * sections. This function also enables RCU lockdep checking.
+ * This function is invoked towards the end of the scheduler's
+ * initialization process. Before this is called, the idle task might
+ * contain synchronous grace-period primitives (during which time, this idle
+ * task is booting the system, and such primitives are no-ops). After this
+ * function is called, any synchronous grace-period primitives are run as
+ * expedited, with the requesting task driving the grace period forward.
+ * A later core_initcall() rcu_exp_runtime_mode() will switch to full
+ * runtime RCU functionality.
*/
void rcu_scheduler_starting(void)
{
WARN_ON(num_online_cpus() != 1);
WARN_ON(nr_context_switches() > 0);
- rcu_scheduler_active = 1;
+ rcu_test_sync_prims();
+ rcu_scheduler_active = RCU_SCHEDULER_INIT;
+ rcu_test_sync_prims();
}
/*
struct work_struct rew_work;
};
+/*
+ * Common code to drive an expedited grace period forward, used by
+ * workqueues and mid-boot-time tasks.
+ */
+static void rcu_exp_sel_wait_wake(struct rcu_state *rsp,
+ smp_call_func_t func, unsigned long s)
+{
+ /* Initialize the rcu_node tree in preparation for the wait. */
+ sync_rcu_exp_select_cpus(rsp, func);
+
+ /* Wait and clean up, including waking everyone. */
+ rcu_exp_wait_wake(rsp, s);
+}
+
/*
* Work-queue handler to drive an expedited grace period forward.
*/
{
struct rcu_exp_work *rewp;
- /* Initialize the rcu_node tree in preparation for the wait. */
rewp = container_of(wp, struct rcu_exp_work, rew_work);
- sync_rcu_exp_select_cpus(rewp->rew_rsp, rewp->rew_func);
-
- /* Wait and clean up, including waking everyone. */
- rcu_exp_wait_wake(rewp->rew_rsp, rewp->rew_s);
+ rcu_exp_sel_wait_wake(rewp->rew_rsp, rewp->rew_func, rewp->rew_s);
}
/*
if (exp_funnel_lock(rsp, s))
return; /* Someone else did our work for us. */
- /* Marshall arguments and schedule the expedited grace period. */
- rew.rew_func = func;
- rew.rew_rsp = rsp;
- rew.rew_s = s;
- INIT_WORK_ONSTACK(&rew.rew_work, wait_rcu_exp_gp);
- schedule_work(&rew.rew_work);
+ /* Ensure that load happens before action based on it. */
+ if (unlikely(rcu_scheduler_active == RCU_SCHEDULER_INIT)) {
+ /* Direct call during scheduler init and early_initcalls(). */
+ rcu_exp_sel_wait_wake(rsp, func, s);
+ } else {
+ /* Marshall arguments & schedule the expedited grace period. */
+ rew.rew_func = func;
+ rew.rew_rsp = rsp;
+ rew.rew_s = s;
+ INIT_WORK_ONSTACK(&rew.rew_work, wait_rcu_exp_gp);
+ schedule_work(&rew.rew_work);
+ }
/* Wait for expedited grace period to complete. */
rdp = per_cpu_ptr(rsp->rda, raw_smp_processor_id());
{
struct rcu_state *rsp = rcu_state_p;
+ if (rcu_scheduler_active == RCU_SCHEDULER_INACTIVE)
+ return;
_synchronize_rcu_expedited(rsp, sync_rcu_exp_handler);
}
EXPORT_SYMBOL_GPL(synchronize_rcu_expedited);
EXPORT_SYMBOL_GPL(synchronize_rcu_expedited);
#endif /* #else #ifdef CONFIG_PREEMPT_RCU */
+
+/*
+ * Switch to run-time mode once Tree RCU has fully initialized.
+ */
+static int __init rcu_exp_runtime_mode(void)
+{
+ rcu_test_sync_prims();
+ rcu_scheduler_active = RCU_SCHEDULER_RUNNING;
+ rcu_test_sync_prims();
+ return 0;
+}
+core_initcall(rcu_exp_runtime_mode);
lock_is_held(&rcu_lock_map) ||
lock_is_held(&rcu_sched_lock_map),
"Illegal synchronize_rcu() in RCU read-side critical section");
- if (!rcu_scheduler_active)
+ if (rcu_scheduler_active == RCU_SCHEDULER_INACTIVE)
return;
if (rcu_gp_is_expedited())
synchronize_rcu_expedited();
* Should expedited grace-period primitives always fall back to their
* non-expedited counterparts? Intended for use within RCU. Note
* that if the user specifies both rcu_expedited and rcu_normal, then
- * rcu_normal wins.
+ * rcu_normal wins. (Except during the time period during boot from
+ * when the first task is spawned until the rcu_exp_runtime_mode()
+ * core_initcall() is invoked, at which point everything is expedited.)
*/
bool rcu_gp_is_normal(void)
{
- return READ_ONCE(rcu_normal);
+ return READ_ONCE(rcu_normal) &&
+ rcu_scheduler_active != RCU_SCHEDULER_INIT;
}
EXPORT_SYMBOL_GPL(rcu_gp_is_normal);
/*
* Should normal grace-period primitives be expedited? Intended for
* use within RCU. Note that this function takes the rcu_expedited
- * sysfs/boot variable into account as well as the rcu_expedite_gp()
- * nesting. So looping on rcu_unexpedite_gp() until rcu_gp_is_expedited()
- * returns false is a -really- bad idea.
+ * sysfs/boot variable and rcu_scheduler_active into account as well
+ * as the rcu_expedite_gp() nesting. So looping on rcu_unexpedite_gp()
+ * until rcu_gp_is_expedited() returns false is a -really- bad idea.
*/
bool rcu_gp_is_expedited(void)
{
- return rcu_expedited || atomic_read(&rcu_expedited_nesting);
+ return rcu_expedited || atomic_read(&rcu_expedited_nesting) ||
+ rcu_scheduler_active == RCU_SCHEDULER_INIT;
}
EXPORT_SYMBOL_GPL(rcu_gp_is_expedited);
int notrace debug_lockdep_rcu_enabled(void)
{
- return rcu_scheduler_active && debug_locks &&
+ return rcu_scheduler_active != RCU_SCHEDULER_INACTIVE && debug_locks &&
current->lockdep_recursion == 0;
}
EXPORT_SYMBOL_GPL(debug_lockdep_rcu_enabled);
void synchronize_rcu_tasks(void)
{
/* Complain if the scheduler has not started. */
- RCU_LOCKDEP_WARN(!rcu_scheduler_active,
+ RCU_LOCKDEP_WARN(rcu_scheduler_active == RCU_SCHEDULER_INACTIVE,
"synchronize_rcu_tasks called too soon");
/* Wait for the grace period. */
#endif /* #ifdef CONFIG_TASKS_RCU */
+/*
+ * Test each non-SRCU synchronous grace-period wait API. This is
+ * useful just after a change in mode for these primitives, and
+ * during early boot.
+ */
+void rcu_test_sync_prims(void)
+{
+ if (!IS_ENABLED(CONFIG_PROVE_RCU))
+ return;
+ synchronize_rcu();
+ synchronize_rcu_bh();
+ synchronize_sched();
+ synchronize_rcu_expedited();
+ synchronize_rcu_bh_expedited();
+ synchronize_sched_expedited();
+}
+
#ifdef CONFIG_PROVE_RCU
/*
early_boot_test_call_rcu_bh();
if (rcu_self_test_sched)
early_boot_test_call_rcu_sched();
+ rcu_test_sync_prims();
}
static int rcu_verify_early_boot_tests(void)
* fresh group stop. Read comment in do_signal_stop() for details.
*/
if (!sig->group_stop_count && !(sig->flags & SIGNAL_STOP_STOPPED)) {
- sig->flags = SIGNAL_STOP_STOPPED;
+ signal_set_stop_flags(sig, SIGNAL_STOP_STOPPED);
return true;
}
return false;
* will take ->siglock, notice SIGNAL_CLD_MASK, and
* notify its parent. See get_signal_to_deliver().
*/
- signal->flags = why | SIGNAL_STOP_CONTINUED;
+ signal_set_stop_flags(signal, why | SIGNAL_STOP_CONTINUED);
signal->group_stop_count = 0;
signal->group_exit_code = 0;
}
break;
if (neg)
continue;
+ val = convmul * val / convdiv;
if ((min && val < *min) || (max && val > *max))
continue;
*i = val;
tick = expires;
/* Skip reprogram of event if its not changed */
- if (ts->tick_stopped && (expires == dev->next_event))
+ if (ts->tick_stopped && (expires == ts->next_tick))
goto out;
/*
trace_tick_stop(1, TICK_DEP_MASK_NONE);
}
+ ts->next_tick = tick;
+
/*
* If the expiration time == KTIME_MAX, then we simply stop
* the tick timer.
else
tick_program_event(tick, 1);
out:
- /* Update the estimated sleep length */
+ /*
+ * Update the estimated sleep length until the next timer
+ * (not only the tick).
+ */
ts->sleep_length = ktime_sub(dev->next_event, now);
return tick;
}
* timer is modified for nohz sleeps. This is necessary
* to resume the tick timer operation in the timeline
* when the CPU returns from nohz sleep.
+ * @next_tick: Next tick to be fired when in dynticks mode.
* @tick_stopped: Indicator that the idle tick has been stopped
* @idle_jiffies: jiffies at the entry to idle for idle time accounting
* @idle_calls: Total number of idle calls
unsigned long check_clocks;
enum tick_nohz_mode nohz_mode;
ktime_t last_tick;
+ ktime_t next_tick;
int inidle;
int tick_stopped;
unsigned long idle_jiffies;
struct hlist_head *hashent = ucounts_hashentry(ns, uid);
struct ucounts *ucounts, *new;
- spin_lock(&ucounts_lock);
+ spin_lock_irq(&ucounts_lock);
ucounts = find_ucounts(ns, uid, hashent);
if (!ucounts) {
- spin_unlock(&ucounts_lock);
+ spin_unlock_irq(&ucounts_lock);
new = kzalloc(sizeof(*new), GFP_KERNEL);
if (!new)
new->uid = uid;
atomic_set(&new->count, 0);
- spin_lock(&ucounts_lock);
+ spin_lock_irq(&ucounts_lock);
ucounts = find_ucounts(ns, uid, hashent);
if (ucounts) {
kfree(new);
}
if (!atomic_add_unless(&ucounts->count, 1, INT_MAX))
ucounts = NULL;
- spin_unlock(&ucounts_lock);
+ spin_unlock_irq(&ucounts_lock);
return ucounts;
}
static void put_ucounts(struct ucounts *ucounts)
{
+ unsigned long flags;
+
if (atomic_dec_and_test(&ucounts->count)) {
- spin_lock(&ucounts_lock);
+ spin_lock_irqsave(&ucounts_lock, flags);
hlist_del_init(&ucounts->node);
- spin_unlock(&ucounts_lock);
+ spin_unlock_irqrestore(&ucounts_lock, flags);
kfree(ucounts);
}
#define for_each_watchdog_cpu(cpu) \
for_each_cpu_and((cpu), cpu_online_mask, &watchdog_cpumask)
+atomic_t watchdog_park_in_progress = ATOMIC_INIT(0);
+
/*
* The 'watchdog_running' variable is set to 1 when the watchdog threads
* are registered/started and is set to 0 when the watchdog threads are
int duration;
int softlockup_all_cpu_backtrace = sysctl_softlockup_all_cpu_backtrace;
+ if (atomic_read(&watchdog_park_in_progress) != 0)
+ return HRTIMER_NORESTART;
+
/* kick the hardlockup detector */
watchdog_interrupt_count();
{
int cpu, ret = 0;
+ atomic_set(&watchdog_park_in_progress, 1);
+
for_each_watchdog_cpu(cpu) {
ret = kthread_park(per_cpu(softlockup_watchdog, cpu));
if (ret)
break;
}
+ atomic_set(&watchdog_park_in_progress, 0);
+
return ret;
}
/* Ensure the watchdog never gets throttled */
event->hw.interrupts = 0;
+ if (atomic_read(&watchdog_park_in_progress) != 0)
+ return;
+
if (__this_cpu_read(watchdog_nmi_touch) == true) {
__this_cpu_write(watchdog_nmi_touch, false);
return;
config DEBUG_INFO_SPLIT
bool "Produce split debuginfo in .dwo files"
- depends on DEBUG_INFO
+ depends on DEBUG_INFO && !FRV
help
Generate debug info into separate .dwo files. This significantly
reduces the build directory size for builds with DEBUG_INFO,
return err;
}
-EXPORT_SYMBOL_GPL(ioremap_page_range);
}
EXPORT_SYMBOL(iov_iter_copy_from_user_atomic);
+static inline void pipe_truncate(struct iov_iter *i)
+{
+ struct pipe_inode_info *pipe = i->pipe;
+ if (pipe->nrbufs) {
+ size_t off = i->iov_offset;
+ int idx = i->idx;
+ int nrbufs = (idx - pipe->curbuf) & (pipe->buffers - 1);
+ if (off) {
+ pipe->bufs[idx].len = off - pipe->bufs[idx].offset;
+ idx = next_idx(idx, pipe);
+ nrbufs++;
+ }
+ while (pipe->nrbufs > nrbufs) {
+ pipe_buf_release(pipe, &pipe->bufs[idx]);
+ idx = next_idx(idx, pipe);
+ pipe->nrbufs--;
+ }
+ }
+}
+
static void pipe_advance(struct iov_iter *i, size_t size)
{
struct pipe_inode_info *pipe = i->pipe;
- struct pipe_buffer *buf;
- int idx = i->idx;
- size_t off = i->iov_offset, orig_sz;
-
if (unlikely(i->count < size))
size = i->count;
- orig_sz = size;
-
if (size) {
+ struct pipe_buffer *buf;
+ size_t off = i->iov_offset, left = size;
+ int idx = i->idx;
if (off) /* make it relative to the beginning of buffer */
- size += off - pipe->bufs[idx].offset;
+ left += off - pipe->bufs[idx].offset;
while (1) {
buf = &pipe->bufs[idx];
- if (size <= buf->len)
+ if (left <= buf->len)
break;
- size -= buf->len;
+ left -= buf->len;
idx = next_idx(idx, pipe);
}
- buf->len = size;
i->idx = idx;
- off = i->iov_offset = buf->offset + size;
- }
- if (off)
- idx = next_idx(idx, pipe);
- if (pipe->nrbufs) {
- int unused = (pipe->curbuf + pipe->nrbufs) & (pipe->buffers - 1);
- /* [curbuf,unused) is in use. Free [idx,unused) */
- while (idx != unused) {
- pipe_buf_release(pipe, &pipe->bufs[idx]);
- idx = next_idx(idx, pipe);
- pipe->nrbufs--;
- }
+ i->iov_offset = buf->offset + left;
}
- i->count -= orig_sz;
+ i->count -= size;
+ /* ... and discard everything past that point */
+ pipe_truncate(i);
}
void iov_iter_advance(struct iov_iter *i, size_t size)
size_t count)
{
BUG_ON(direction != ITER_PIPE);
+ WARN_ON(pipe->nrbufs == pipe->buffers);
i->type = direction;
i->pipe = pipe;
i->idx = (pipe->curbuf + pipe->nrbufs) & (pipe->buffers - 1);
struct radix_tree_node *old = child;
offset = child->offset + 1;
child = child->parent;
- WARN_ON_ONCE(!list_empty(&node->private_list));
+ WARN_ON_ONCE(!list_empty(&old->private_list));
radix_tree_node_free(old);
if (old == entry_to_node(node))
return;
: 1UL << (BITS_PER_LONG - IO_TLB_SHIFT);
/*
- * For mappings greater than a page, we limit the stride (and
- * hence alignment) to a page size.
+ * For mappings greater than or equal to a page, we limit the stride
+ * (and hence alignment) to a page size.
*/
nslots = ALIGN(size, 1 << IO_TLB_SHIFT) >> IO_TLB_SHIFT;
- if (size > PAGE_SIZE)
+ if (size >= PAGE_SIZE)
stride = (1 << (PAGE_SHIFT - IO_TLB_SHIFT));
else
stride = 1;
dax_radix_locked_entry(0, RADIX_DAX_EMPTY));
/* Wakeup waiters for exceptional entry lock */
dax_wake_mapping_entry_waiter(mapping, page->index, p,
- false);
+ true);
}
}
__radix_tree_replace(&mapping->page_tree, node, slot, page,
assert_spin_locked(pmd_lockptr(mm, pmd));
+ /*
+ * When we COW a devmap PMD entry, we split it into PTEs, so we should
+ * not be in this function with `flags & FOLL_COW` set.
+ */
+ WARN_ONCE(flags & FOLL_COW, "mm: In follow_devmap_pmd with FOLL_COW set");
+
if (flags & FOLL_WRITE && !pmd_write(*pmd))
return NULL;
{
pmd_t entry;
unsigned long haddr;
+ bool write = vmf->flags & FAULT_FLAG_WRITE;
vmf->ptl = pmd_lock(vmf->vma->vm_mm, vmf->pmd);
if (unlikely(!pmd_same(*vmf->pmd, orig_pmd)))
goto unlock;
entry = pmd_mkyoung(orig_pmd);
+ if (write)
+ entry = pmd_mkdirty(entry);
haddr = vmf->address & HPAGE_PMD_MASK;
- if (pmdp_set_access_flags(vmf->vma, haddr, vmf->pmd, entry,
- vmf->flags & FAULT_FLAG_WRITE))
+ if (pmdp_set_access_flags(vmf->vma, haddr, vmf->pmd, entry, write))
update_mmu_cache_pmd(vmf->vma, vmf->address, vmf->pmd);
unlock:
}
for (i = 0; i < HPAGE_PMD_NR; i++) {
- pages[i] = alloc_page_vma_node(GFP_HIGHUSER_MOVABLE |
- __GFP_OTHER_NODE, vma,
+ pages[i] = alloc_page_vma_node(GFP_HIGHUSER_MOVABLE, vma,
vmf->address, page_to_nid(page));
if (unlikely(!pages[i] ||
mem_cgroup_try_charge(pages[i], vma->vm_mm,
return ret;
}
+/*
+ * FOLL_FORCE can write to even unwritable pmd's, but only
+ * after we've gone through a COW cycle and they are dirty.
+ */
+static inline bool can_follow_write_pmd(pmd_t pmd, unsigned int flags)
+{
+ return pmd_write(pmd) ||
+ ((flags & FOLL_FORCE) && (flags & FOLL_COW) && pmd_dirty(pmd));
+}
+
struct page *follow_trans_huge_pmd(struct vm_area_struct *vma,
unsigned long addr,
pmd_t *pmd,
assert_spin_locked(pmd_lockptr(mm, pmd));
- if (flags & FOLL_WRITE && !pmd_write(*pmd))
+ if (flags & FOLL_WRITE && !can_follow_write_pmd(*pmd, flags))
goto out;
/* Avoid dumping huge zero page */
}
/*
- * When releasing a hugetlb pool reservation, any surplus pages that were
- * allocated to satisfy the reservation must be explicitly freed if they were
- * never used.
- * Called with hugetlb_lock held.
+ * This routine has two main purposes:
+ * 1) Decrement the reservation count (resv_huge_pages) by the value passed
+ * in unused_resv_pages. This corresponds to the prior adjustments made
+ * to the associated reservation map.
+ * 2) Free any unused surplus pages that may have been allocated to satisfy
+ * the reservation. As many as unused_resv_pages may be freed.
+ *
+ * Called with hugetlb_lock held. However, the lock could be dropped (and
+ * reacquired) during calls to cond_resched_lock. Whenever dropping the lock,
+ * we must make sure nobody else can claim pages we are in the process of
+ * freeing. Do this by ensuring resv_huge_page always is greater than the
+ * number of huge pages we plan to free when dropping the lock.
*/
static void return_unused_surplus_pages(struct hstate *h,
unsigned long unused_resv_pages)
{
unsigned long nr_pages;
- /* Uncommit the reservation */
- h->resv_huge_pages -= unused_resv_pages;
-
/* Cannot return gigantic pages currently */
if (hstate_is_gigantic(h))
- return;
+ goto out;
+ /*
+ * Part (or even all) of the reservation could have been backed
+ * by pre-allocated pages. Only free surplus pages.
+ */
nr_pages = min(unused_resv_pages, h->surplus_huge_pages);
/*
* when the nodes with surplus pages have no free pages.
* free_pool_huge_page() will balance the the freed pages across the
* on-line nodes with memory and will handle the hstate accounting.
+ *
+ * Note that we decrement resv_huge_pages as we free the pages. If
+ * we drop the lock, resv_huge_pages will still be sufficiently large
+ * to cover subsequent pages we may free.
*/
while (nr_pages--) {
+ h->resv_huge_pages--;
+ unused_resv_pages--;
if (!free_pool_huge_page(h, &node_states[N_MEMORY], 1))
- break;
+ goto out;
cond_resched_lock(&hugetlb_lock);
}
+
+out:
+ /* Fully uncommit the reservation */
+ h->resv_huge_pages -= unused_resv_pages;
}
VM_BUG_ON(address & ~HPAGE_PMD_MASK);
/* Only allocate from the target node */
- gfp = alloc_hugepage_khugepaged_gfpmask() | __GFP_OTHER_NODE | __GFP_THISNODE;
+ gfp = alloc_hugepage_khugepaged_gfpmask() | __GFP_THISNODE;
/*
* Before allocating the hugepage, release the mmap_sem read lock.
struct vm_area_struct *vma;
unsigned long addr;
pmd_t *pmd, _pmd;
- bool deposited = false;
i_mmap_lock_write(mapping);
vma_interval_tree_foreach(vma, &mapping->i_mmap, pgoff, pgoff) {
spinlock_t *ptl = pmd_lock(vma->vm_mm, pmd);
/* assume page table is clear */
_pmd = pmdp_collapse_flush(vma, addr, pmd);
- /*
- * now deposit the pgtable for arch that need it
- * otherwise free it.
- */
- if (arch_needs_pgtable_deposit()) {
- /*
- * The deposit should be visibile only after
- * collapse is seen by others.
- */
- smp_wmb();
- pgtable_trans_huge_deposit(vma->vm_mm, pmd,
- pmd_pgtable(_pmd));
- deposited = true;
- }
spin_unlock(ptl);
up_write(&vma->vm_mm->mmap_sem);
- if (!deposited) {
- atomic_long_dec(&vma->vm_mm->nr_ptes);
- pte_free(vma->vm_mm, pmd_pgtable(_pmd));
- }
+ atomic_long_dec(&vma->vm_mm->nr_ptes);
+ pte_free(vma->vm_mm, pmd_pgtable(_pmd));
}
}
i_mmap_unlock_write(mapping);
VM_BUG_ON(start & (HPAGE_PMD_NR - 1));
/* Only allocate from the target node */
- gfp = alloc_hugepage_khugepaged_gfpmask() |
- __GFP_OTHER_NODE | __GFP_THISNODE;
+ gfp = alloc_hugepage_khugepaged_gfpmask() | __GFP_THISNODE;
new_page = khugepaged_alloc_page(hpage, gfp, node);
if (!new_page) {
unsigned long mem_cgroup_node_nr_lru_pages(struct mem_cgroup *memcg,
int nid, unsigned int lru_mask)
{
+ struct lruvec *lruvec = mem_cgroup_lruvec(NODE_DATA(nid), memcg);
unsigned long nr = 0;
- struct mem_cgroup_per_node *mz;
enum lru_list lru;
VM_BUG_ON((unsigned)nid >= nr_node_ids);
for_each_lru(lru) {
if (!(BIT(lru) & lru_mask))
continue;
- mz = mem_cgroup_nodeinfo(memcg, nid);
- nr += mz->lru_size[lru];
+ nr += mem_cgroup_get_lru_size(lruvec, lru);
}
return nr;
}
* mem_cgroup_update_lru_size - account for adding or removing an lru page
* @lruvec: mem_cgroup per zone lru vector
* @lru: index of lru list the page is sitting on
+ * @zid: zone id of the accounted pages
* @nr_pages: positive when adding or negative when removing
*
* This function must be called under lru_lock, just before a page is added
* so as to allow it to check that lru_size 0 is consistent with list_empty).
*/
void mem_cgroup_update_lru_size(struct lruvec *lruvec, enum lru_list lru,
- int nr_pages)
+ int zid, int nr_pages)
{
struct mem_cgroup_per_node *mz;
unsigned long *lru_size;
long size;
- bool empty;
if (mem_cgroup_disabled())
return;
mz = container_of(lruvec, struct mem_cgroup_per_node, lruvec);
- lru_size = mz->lru_size + lru;
- empty = list_empty(lruvec->lists + lru);
+ lru_size = &mz->lru_zone_size[zid][lru];
if (nr_pages < 0)
*lru_size += nr_pages;
size = *lru_size;
- if (WARN_ONCE(size < 0 || empty != !size,
- "%s(%p, %d, %d): lru_size %ld but %sempty\n",
- __func__, lruvec, lru, nr_pages, size, empty ? "" : "not ")) {
+ if (WARN_ONCE(size < 0,
+ "%s(%p, %d, %d): lru_size %ld\n",
+ __func__, lruvec, lru, nr_pages, size)) {
VM_BUG_ON(1);
*lru_size = 0;
}
return ret;
}
- /* Try charges one by one with reclaim */
+ /* Try charges one by one with reclaim, but do not retry */
while (count--) {
- ret = try_charge(mc.to, GFP_KERNEL & ~__GFP_NORETRY, 1);
+ ret = try_charge(mc.to, GFP_KERNEL | __GFP_NORETRY, 1);
if (ret)
return ret;
mc.precharge++;
}
#endif /* __PAGETABLE_PMD_FOLDED */
-static int __follow_pte(struct mm_struct *mm, unsigned long address,
- pte_t **ptepp, spinlock_t **ptlp)
+static int __follow_pte_pmd(struct mm_struct *mm, unsigned long address,
+ pte_t **ptepp, pmd_t **pmdpp, spinlock_t **ptlp)
{
pgd_t *pgd;
pud_t *pud;
pmd = pmd_offset(pud, address);
VM_BUG_ON(pmd_trans_huge(*pmd));
- if (pmd_none(*pmd) || unlikely(pmd_bad(*pmd)))
- goto out;
- /* We cannot handle huge page PFN maps. Luckily they don't exist. */
- if (pmd_huge(*pmd))
+ if (pmd_huge(*pmd)) {
+ if (!pmdpp)
+ goto out;
+
+ *ptlp = pmd_lock(mm, pmd);
+ if (pmd_huge(*pmd)) {
+ *pmdpp = pmd;
+ return 0;
+ }
+ spin_unlock(*ptlp);
+ }
+
+ if (pmd_none(*pmd) || unlikely(pmd_bad(*pmd)))
goto out;
ptep = pte_offset_map_lock(mm, pmd, address, ptlp);
return -EINVAL;
}
-int follow_pte(struct mm_struct *mm, unsigned long address, pte_t **ptepp,
- spinlock_t **ptlp)
+static inline int follow_pte(struct mm_struct *mm, unsigned long address,
+ pte_t **ptepp, spinlock_t **ptlp)
+{
+ int res;
+
+ /* (void) is needed to make gcc happy */
+ (void) __cond_lock(*ptlp,
+ !(res = __follow_pte_pmd(mm, address, ptepp, NULL,
+ ptlp)));
+ return res;
+}
+
+int follow_pte_pmd(struct mm_struct *mm, unsigned long address,
+ pte_t **ptepp, pmd_t **pmdpp, spinlock_t **ptlp)
{
int res;
/* (void) is needed to make gcc happy */
(void) __cond_lock(*ptlp,
- !(res = __follow_pte(mm, address, ptepp, ptlp)));
+ !(res = __follow_pte_pmd(mm, address, ptepp, pmdpp,
+ ptlp)));
return res;
}
+EXPORT_SYMBOL(follow_pte_pmd);
/**
* follow_pfn - look up PFN at a user virtual address
node_set_state(node, N_MEMORY);
}
-int zone_can_shift(unsigned long pfn, unsigned long nr_pages,
- enum zone_type target)
+bool zone_can_shift(unsigned long pfn, unsigned long nr_pages,
+ enum zone_type target, int *zone_shift)
{
struct zone *zone = page_zone(pfn_to_page(pfn));
enum zone_type idx = zone_idx(zone);
int i;
+ *zone_shift = 0;
+
if (idx < target) {
/* pages must be at end of current zone */
if (pfn + nr_pages != zone_end_pfn(zone))
- return 0;
+ return false;
/* no zones in use between current zone and target */
for (i = idx + 1; i < target; i++)
if (zone_is_initialized(zone - idx + i))
- return 0;
+ return false;
}
if (target < idx) {
/* pages must be at beginning of current zone */
if (pfn != zone->zone_start_pfn)
- return 0;
+ return false;
/* no zones in use between current zone and target */
for (i = target + 1; i < idx; i++)
if (zone_is_initialized(zone - idx + i))
- return 0;
+ return false;
}
- return target - idx;
+ *zone_shift = target - idx;
+ return true;
}
/* Must be protected by mem_hotplug_begin() */
!can_online_high_movable(zone))
return -EINVAL;
- if (online_type == MMOP_ONLINE_KERNEL)
- zone_shift = zone_can_shift(pfn, nr_pages, ZONE_NORMAL);
- else if (online_type == MMOP_ONLINE_MOVABLE)
- zone_shift = zone_can_shift(pfn, nr_pages, ZONE_MOVABLE);
+ if (online_type == MMOP_ONLINE_KERNEL) {
+ if (!zone_can_shift(pfn, nr_pages, ZONE_NORMAL, &zone_shift))
+ return -EINVAL;
+ } else if (online_type == MMOP_ONLINE_MOVABLE) {
+ if (!zone_can_shift(pfn, nr_pages, ZONE_MOVABLE, &zone_shift))
+ return -EINVAL;
+ }
zone = move_pfn_range(zone_shift, pfn, pfn + nr_pages);
if (!zone)
nmask = policy_nodemask(gfp, pol);
zl = policy_zonelist(gfp, pol, node);
- mpol_cond_put(pol);
page = __alloc_pages_nodemask(gfp, order, zl, nmask);
+ mpol_cond_put(pol);
out:
if (unlikely(!page && read_mems_allowed_retry(cpuset_mems_cookie)))
goto retry_cpuset;
#endif
for (page = start_page; page <= end_page;) {
- /* Make sure we are not inadvertently changing nodes */
- VM_BUG_ON_PAGE(page_to_nid(page) != zone_to_nid(zone), page);
-
if (!pfn_valid_within(page_to_pfn(page))) {
page++;
continue;
}
+ /* Make sure we are not inadvertently changing nodes */
+ VM_BUG_ON_PAGE(page_to_nid(page) != zone_to_nid(zone), page);
+
if (!PageBuddy(page)) {
page++;
continue;
* Update NUMA hit/miss statistics
*
* Must be called with interrupts disabled.
- *
- * When __GFP_OTHER_NODE is set assume the node of the preferred
- * zone is the local node. This is useful for daemons who allocate
- * memory on behalf of other processes.
*/
-static inline void zone_statistics(struct zone *preferred_zone, struct zone *z,
- gfp_t flags)
+static inline void zone_statistics(struct zone *preferred_zone, struct zone *z)
{
#ifdef CONFIG_NUMA
- int local_nid = numa_node_id();
enum zone_stat_item local_stat = NUMA_LOCAL;
- if (unlikely(flags & __GFP_OTHER_NODE)) {
+ if (z->node != numa_node_id())
local_stat = NUMA_OTHER;
- local_nid = preferred_zone->node;
- }
- if (z->node == local_nid) {
+ if (z->node == preferred_zone->node)
__inc_zone_state(z, NUMA_HIT);
- __inc_zone_state(z, local_stat);
- } else {
+ else {
__inc_zone_state(z, NUMA_MISS);
__inc_zone_state(preferred_zone, NUMA_FOREIGN);
}
+ __inc_zone_state(z, local_stat);
#endif
}
}
__count_zid_vm_events(PGALLOC, page_zonenum(page), 1 << order);
- zone_statistics(preferred_zone, zone, gfp_flags);
+ zone_statistics(preferred_zone, zone);
local_irq_restore(flags);
VM_BUG_ON_PAGE(bad_range(zone, page), page);
struct page *page = NULL;
unsigned int alloc_flags;
unsigned long did_some_progress;
- enum compact_priority compact_priority = DEF_COMPACT_PRIORITY;
+ enum compact_priority compact_priority;
enum compact_result compact_result;
- int compaction_retries = 0;
- int no_progress_loops = 0;
+ int compaction_retries;
+ int no_progress_loops;
unsigned long alloc_start = jiffies;
unsigned int stall_timeout = 10 * HZ;
+ unsigned int cpuset_mems_cookie;
/*
* In the slowpath, we sanity check order to avoid ever trying to
(__GFP_ATOMIC|__GFP_DIRECT_RECLAIM)))
gfp_mask &= ~__GFP_ATOMIC;
+retry_cpuset:
+ compaction_retries = 0;
+ no_progress_loops = 0;
+ compact_priority = DEF_COMPACT_PRIORITY;
+ cpuset_mems_cookie = read_mems_allowed_begin();
+ /*
+ * We need to recalculate the starting point for the zonelist iterator
+ * because we might have used different nodemask in the fast path, or
+ * there was a cpuset modification and we are retrying - otherwise we
+ * could end up iterating over non-eligible zones endlessly.
+ */
+ ac->preferred_zoneref = first_zones_zonelist(ac->zonelist,
+ ac->high_zoneidx, ac->nodemask);
+ if (!ac->preferred_zoneref->zone)
+ goto nopage;
+
+
/*
* The fast path uses conservative alloc_flags to succeed only until
* kswapd needs to be woken up, and to avoid the cost of setting up
&compaction_retries))
goto retry;
+ /*
+ * It's possible we raced with cpuset update so the OOM would be
+ * premature (see below the nopage: label for full explanation).
+ */
+ if (read_mems_allowed_retry(cpuset_mems_cookie))
+ goto retry_cpuset;
+
/* Reclaim has failed us, start killing things */
page = __alloc_pages_may_oom(gfp_mask, order, ac, &did_some_progress);
if (page)
}
nopage:
+ /*
+ * When updating a task's mems_allowed or mempolicy nodemask, it is
+ * possible to race with parallel threads in such a way that our
+ * allocation can fail while the mask is being updated. If we are about
+ * to fail, check if the cpuset changed during allocation and if so,
+ * retry.
+ */
+ if (read_mems_allowed_retry(cpuset_mems_cookie))
+ goto retry_cpuset;
+
warn_alloc(gfp_mask,
"page allocation failure: order:%u", order);
got_pg:
struct zonelist *zonelist, nodemask_t *nodemask)
{
struct page *page;
- unsigned int cpuset_mems_cookie;
unsigned int alloc_flags = ALLOC_WMARK_LOW;
gfp_t alloc_mask = gfp_mask; /* The gfp_t that was actually used for allocation */
struct alloc_context ac = {
if (IS_ENABLED(CONFIG_CMA) && ac.migratetype == MIGRATE_MOVABLE)
alloc_flags |= ALLOC_CMA;
-retry_cpuset:
- cpuset_mems_cookie = read_mems_allowed_begin();
-
/* Dirty zone balancing only done in the fast path */
ac.spread_dirty_pages = (gfp_mask & __GFP_WRITE);
*/
ac.preferred_zoneref = first_zones_zonelist(ac.zonelist,
ac.high_zoneidx, ac.nodemask);
- if (!ac.preferred_zoneref) {
+ if (!ac.preferred_zoneref->zone) {
page = NULL;
+ /*
+ * This might be due to race with cpuset_current_mems_allowed
+ * update, so make sure we retry with original nodemask in the
+ * slow path.
+ */
goto no_zone;
}
if (likely(page))
goto out;
+no_zone:
/*
* Runtime PM, block IO and its error handling path can deadlock
* because I/O on the device might not complete.
* Restore the original nodemask if it was potentially replaced with
* &cpuset_current_mems_allowed to optimize the fast-path attempt.
*/
- if (cpusets_enabled())
+ if (unlikely(ac.nodemask != nodemask))
ac.nodemask = nodemask;
- page = __alloc_pages_slowpath(alloc_mask, order, &ac);
-no_zone:
- /*
- * When updating a task's mems_allowed, it is possible to race with
- * parallel threads in such a way that an allocation can fail while
- * the mask is being updated. If a page allocation is about to fail,
- * check if the cpuset changed during allocation and if so, retry.
- */
- if (unlikely(!page && read_mems_allowed_retry(cpuset_mems_cookie))) {
- alloc_mask = gfp_mask;
- goto retry_cpuset;
- }
+ page = __alloc_pages_slowpath(alloc_mask, order, &ac);
out:
if (memcg_kmem_enabled() && (gfp_mask & __GFP_ACCOUNT) && page &&
* drivers to provide a backing region of memory for use as either an
* sk_buff->head, or to be used in the "frags" portion of skb_shared_info.
*/
-static struct page *__page_frag_refill(struct page_frag_cache *nc,
- gfp_t gfp_mask)
+static struct page *__page_frag_cache_refill(struct page_frag_cache *nc,
+ gfp_t gfp_mask)
{
struct page *page = NULL;
gfp_t gfp = gfp_mask;
return page;
}
-void __page_frag_drain(struct page *page, unsigned int order,
- unsigned int count)
+void __page_frag_cache_drain(struct page *page, unsigned int count)
{
VM_BUG_ON_PAGE(page_ref_count(page) == 0, page);
if (page_ref_sub_and_test(page, count)) {
+ unsigned int order = compound_order(page);
+
if (order == 0)
free_hot_cold_page(page, false);
else
__free_pages_ok(page, order);
}
}
-EXPORT_SYMBOL(__page_frag_drain);
+EXPORT_SYMBOL(__page_frag_cache_drain);
-void *__alloc_page_frag(struct page_frag_cache *nc,
- unsigned int fragsz, gfp_t gfp_mask)
+void *page_frag_alloc(struct page_frag_cache *nc,
+ unsigned int fragsz, gfp_t gfp_mask)
{
unsigned int size = PAGE_SIZE;
struct page *page;
if (unlikely(!nc->va)) {
refill:
- page = __page_frag_refill(nc, gfp_mask);
+ page = __page_frag_cache_refill(nc, gfp_mask);
if (!page)
return NULL;
return nc->va + offset;
}
-EXPORT_SYMBOL(__alloc_page_frag);
+EXPORT_SYMBOL(page_frag_alloc);
/*
* Frees a page fragment allocated out of either a compound or order 0 page.
*/
-void __free_page_frag(void *addr)
+void page_frag_free(void *addr)
{
struct page *page = virt_to_head_page(addr);
if (unlikely(put_page_testzero(page)))
__free_pages_ok(page, compound_order(page));
}
-EXPORT_SYMBOL(__free_page_frag);
+EXPORT_SYMBOL(page_frag_free);
static void *make_alloc_exact(unsigned long addr, unsigned int order,
size_t size)
.zone = page_zone(pfn_to_page(start)),
.mode = MIGRATE_SYNC,
.ignore_skip_hint = true,
+ .gfp_mask = GFP_KERNEL,
};
INIT_LIST_HEAD(&cc.migratepages);
unsigned int pos;
unsigned int *list;
unsigned int count;
- unsigned int rand;
};
struct rnd_state rnd_state;
};
} else {
state->list = cachep->random_seq;
state->count = count;
- state->pos = 0;
- state->rand = rand;
+ state->pos = rand % count;
ret = true;
}
return ret;
/* Get the next entry on the list and randomize it using a random shift */
static freelist_idx_t next_random_slot(union freelist_init_state *state)
{
- return (state->list[state->pos++] + state->rand) % state->count;
+ if (state->pos >= state->count)
+ state->pos = 0;
+ return state->list[state->pos++];
}
/* Swap two freelist entries */
return 1;
}
-static void print_section(char *text, u8 *addr, unsigned int length)
+static void print_section(char *level, char *text, u8 *addr,
+ unsigned int length)
{
metadata_access_enable();
- print_hex_dump(KERN_ERR, text, DUMP_PREFIX_ADDRESS, 16, 1, addr,
+ print_hex_dump(level, text, DUMP_PREFIX_ADDRESS, 16, 1, addr,
length, 1);
metadata_access_disable();
}
p, p - addr, get_freepointer(s, p));
if (s->flags & SLAB_RED_ZONE)
- print_section("Redzone ", p - s->red_left_pad, s->red_left_pad);
+ print_section(KERN_ERR, "Redzone ", p - s->red_left_pad,
+ s->red_left_pad);
else if (p > addr + 16)
- print_section("Bytes b4 ", p - 16, 16);
+ print_section(KERN_ERR, "Bytes b4 ", p - 16, 16);
- print_section("Object ", p, min_t(unsigned long, s->object_size,
- PAGE_SIZE));
+ print_section(KERN_ERR, "Object ", p,
+ min_t(unsigned long, s->object_size, PAGE_SIZE));
if (s->flags & SLAB_RED_ZONE)
- print_section("Redzone ", p + s->object_size,
+ print_section(KERN_ERR, "Redzone ", p + s->object_size,
s->inuse - s->object_size);
if (s->offset)
if (off != size_from_object(s))
/* Beginning of the filler is the free pointer */
- print_section("Padding ", p + off, size_from_object(s) - off);
+ print_section(KERN_ERR, "Padding ", p + off,
+ size_from_object(s) - off);
dump_stack();
}
end--;
slab_err(s, page, "Padding overwritten. 0x%p-0x%p", fault, end - 1);
- print_section("Padding ", end - remainder, remainder);
+ print_section(KERN_ERR, "Padding ", end - remainder, remainder);
restore_bytes(s, "slab padding", POISON_INUSE, end - remainder, end);
return 0;
page->freelist);
if (!alloc)
- print_section("Object ", (void *)object,
+ print_section(KERN_INFO, "Object ", (void *)object,
s->object_size);
dump_stack();
count = page_trans_huge_mapcount(page, total_mapcount);
if (count <= 1 && PageSwapCache(page)) {
count += page_swapcount(page);
- if (count == 1 && !PageWriteback(page)) {
+ if (count != 1)
+ goto out;
+ if (!PageWriteback(page)) {
delete_from_swap_cache(page);
SetPageDirty(page);
+ } else {
+ swp_entry_t entry;
+ struct swap_info_struct *p;
+
+ entry.val = page_private(page);
+ p = swap_info_get(entry);
+ if (p->flags & SWP_STABLE_WRITES) {
+ spin_unlock(&p->lock);
+ return false;
+ }
+ spin_unlock(&p->lock);
}
}
+out:
return count <= 1;
}
error = -ENOMEM;
goto bad_swap;
}
+
+ if (bdi_cap_stable_pages_required(inode_to_bdi(inode)))
+ p->flags |= SWP_STABLE_WRITES;
+
if (p->bdev && blk_queue_nonrot(bdev_get_queue(p->bdev))) {
int cpu;
return node_page_state(lruvec_pgdat(lruvec), NR_LRU_BASE + lru);
}
+unsigned long lruvec_zone_lru_size(struct lruvec *lruvec, enum lru_list lru,
+ int zone_idx)
+{
+ if (!mem_cgroup_disabled())
+ return mem_cgroup_get_zone_lru_size(lruvec, lru, zone_idx);
+
+ return zone_page_state(&lruvec_pgdat(lruvec)->node_zones[zone_idx],
+ NR_ZONE_LRU_BASE + lru);
+}
+
/*
* Add a shrinker callback to be called from the vm.
*/
* be complete before mem_cgroup_update_lru_size due to a santity check.
*/
static __always_inline void update_lru_sizes(struct lruvec *lruvec,
- enum lru_list lru, unsigned long *nr_zone_taken,
- unsigned long nr_taken)
+ enum lru_list lru, unsigned long *nr_zone_taken)
{
int zid;
continue;
__update_lru_size(lruvec, lru, zid, -nr_zone_taken[zid]);
- }
-
#ifdef CONFIG_MEMCG
- mem_cgroup_update_lru_size(lruvec, lru, -nr_taken);
+ mem_cgroup_update_lru_size(lruvec, lru, zid, -nr_zone_taken[zid]);
#endif
+ }
+
}
/*
*nr_scanned = scan;
trace_mm_vmscan_lru_isolate(sc->reclaim_idx, sc->order, nr_to_scan, scan,
nr_taken, mode, is_file_lru(lru));
- update_lru_sizes(lruvec, lru, nr_zone_taken, nr_taken);
+ update_lru_sizes(lruvec, lru, nr_zone_taken);
return nr_taken;
}
if (!managed_zone(zone))
continue;
- inactive_zone = zone_page_state(zone,
- NR_ZONE_LRU_BASE + (file * LRU_FILE));
- active_zone = zone_page_state(zone,
- NR_ZONE_LRU_BASE + (file * LRU_FILE) + LRU_ACTIVE);
+ inactive_zone = lruvec_zone_lru_size(lruvec, file * LRU_FILE, zid);
+ active_zone = lruvec_zone_lru_size(lruvec, (file * LRU_FILE) + LRU_ACTIVE, zid);
inactive -= min(inactive, inactive_zone);
active -= min(active, active_zone);
config HWBM
bool
-config SOCK_CGROUP_DATA
- bool
- default n
-
config CGROUP_NET_PRIO
bool "Network priority cgroup"
depends on CGROUPS
{
ax25_clear_queues(ax25);
- if (!sock_flag(ax25->sk, SOCK_DESTROY))
+ if (!ax25->sk || !sock_flag(ax25->sk, SOCK_DESTROY))
ax25_stop_heartbeat(ax25);
ax25_stop_t1timer(ax25);
ax25_stop_t2timer(ax25);
br_nf_hook_thresh(NF_BR_PRE_ROUTING,
net, sk, skb, skb->dev,
NULL,
- br_nf_pre_routing_finish);
+ br_nf_pre_routing_finish_bridge);
return 0;
}
ether_addr_copy(eth_hdr(skb)->h_dest, dev->dev_addr);
SKCIPHER_REQUEST_ON_STACK(req, key->tfm);
struct sg_table sgt;
struct scatterlist prealloc_sg;
- char iv[AES_BLOCK_SIZE];
+ char iv[AES_BLOCK_SIZE] __aligned(8);
int pad_byte = AES_BLOCK_SIZE - (in_len & (AES_BLOCK_SIZE - 1));
int crypt_len = encrypt ? in_len + pad_byte : in_len;
int ret;
pinfo->nr_frags &&
!PageHighMem(skb_frag_page(frag0))) {
NAPI_GRO_CB(skb)->frag0 = skb_frag_address(frag0);
- NAPI_GRO_CB(skb)->frag0_len = skb_frag_size(frag0);
+ NAPI_GRO_CB(skb)->frag0_len = min_t(unsigned int,
+ skb_frag_size(frag0),
+ skb->end - skb->tail);
}
}
* The function will try to retrieve a be32 entity at
* offset poff
*/
-__be16 skb_flow_get_be16(const struct sk_buff *skb, int poff, void *data,
- int hlen)
+static __be16 skb_flow_get_be16(const struct sk_buff *skb, int poff,
+ void *data, int hlen)
{
__be16 *u, _u;
local_irq_save(flags);
nc = this_cpu_ptr(&netdev_alloc_cache);
- data = __alloc_page_frag(nc, fragsz, gfp_mask);
+ data = page_frag_alloc(nc, fragsz, gfp_mask);
local_irq_restore(flags);
return data;
}
{
struct napi_alloc_cache *nc = this_cpu_ptr(&napi_alloc_cache);
- return __alloc_page_frag(&nc->page, fragsz, gfp_mask);
+ return page_frag_alloc(&nc->page, fragsz, gfp_mask);
}
void *napi_alloc_frag(unsigned int fragsz)
local_irq_save(flags);
nc = this_cpu_ptr(&netdev_alloc_cache);
- data = __alloc_page_frag(nc, len, gfp_mask);
+ data = page_frag_alloc(nc, len, gfp_mask);
pfmemalloc = nc->pfmemalloc;
local_irq_restore(flags);
if (sk_memalloc_socks())
gfp_mask |= __GFP_MEMALLOC;
- data = __alloc_page_frag(&nc->page, len, gfp_mask);
+ data = page_frag_alloc(&nc->page, len, gfp_mask);
if (unlikely(!data))
return NULL;
"sk_lock-AF_RXRPC" , "sk_lock-AF_ISDN" , "sk_lock-AF_PHONET" ,
"sk_lock-AF_IEEE802154", "sk_lock-AF_CAIF" , "sk_lock-AF_ALG" ,
"sk_lock-AF_NFC" , "sk_lock-AF_VSOCK" , "sk_lock-AF_KCM" ,
- "sk_lock-AF_MAX"
+ "sk_lock-AF_QIPCRTR", "sk_lock-AF_MAX"
};
static const char *const af_family_slock_key_strings[AF_MAX+1] = {
"slock-AF_UNSPEC", "slock-AF_UNIX" , "slock-AF_INET" ,
"slock-AF_RXRPC" , "slock-AF_ISDN" , "slock-AF_PHONET" ,
"slock-AF_IEEE802154", "slock-AF_CAIF" , "slock-AF_ALG" ,
"slock-AF_NFC" , "slock-AF_VSOCK" ,"slock-AF_KCM" ,
- "slock-AF_MAX"
+ "slock-AF_QIPCRTR", "slock-AF_MAX"
};
static const char *const af_family_clock_key_strings[AF_MAX+1] = {
"clock-AF_UNSPEC", "clock-AF_UNIX" , "clock-AF_INET" ,
"clock-AF_RXRPC" , "clock-AF_ISDN" , "clock-AF_PHONET" ,
"clock-AF_IEEE802154", "clock-AF_CAIF" , "clock-AF_ALG" ,
"clock-AF_NFC" , "clock-AF_VSOCK" , "clock-AF_KCM" ,
- "clock-AF_MAX"
+ "clock-AF_QIPCRTR", "clock-AF_MAX"
};
/*
return err;
}
- err = dsa_cpu_port_ethtool_setup(dst->ds[0]);
- if (err)
- return err;
+ if (dst->ds[0]) {
+ err = dsa_cpu_port_ethtool_setup(dst->ds[0]);
+ if (err)
+ return err;
+ }
/* If we use a tagging format that doesn't have an ethertype
* field, make sure that all packets from this point on get
dsa_ds_unapply(dst, ds);
}
- dsa_cpu_port_ethtool_restore(dst->ds[0]);
+ if (dst->ds[0])
+ dsa_cpu_port_ethtool_restore(dst->ds[0]);
pr_info("DSA: tree %d unapplied\n", dst->tree);
dst->applied = false;
nla_put_u32(skb, RTA_FLOW, fi->fib_nh[0].nh_tclassid))
goto nla_put_failure;
#endif
- if (fi->fib_nh->nh_lwtstate)
- lwtunnel_fill_encap(skb, fi->fib_nh->nh_lwtstate);
+ if (fi->fib_nh->nh_lwtstate &&
+ lwtunnel_fill_encap(skb, fi->fib_nh->nh_lwtstate) < 0)
+ goto nla_put_failure;
}
#ifdef CONFIG_IP_ROUTE_MULTIPATH
if (fi->fib_nhs > 1) {
nla_put_u32(skb, RTA_FLOW, nh->nh_tclassid))
goto nla_put_failure;
#endif
- if (nh->nh_lwtstate)
- lwtunnel_fill_encap(skb, nh->nh_lwtstate);
+ if (nh->nh_lwtstate &&
+ lwtunnel_fill_encap(skb, nh->nh_lwtstate) < 0)
+ goto nla_put_failure;
+
/* length of rtnetlink header + attributes */
rtnh->rtnh_len = nlmsg_get_pos(skb) - (void *) rtnh;
} endfor_nexthops(fi);
void fib_select_path(struct net *net, struct fib_result *res,
struct flowi4 *fl4, int mp_hash)
{
+ bool oif_check;
+
+ oif_check = (fl4->flowi4_oif == 0 ||
+ fl4->flowi4_flags & FLOWI_FLAG_SKIP_NH_OIF);
+
#ifdef CONFIG_IP_ROUTE_MULTIPATH
- if (res->fi->fib_nhs > 1 && fl4->flowi4_oif == 0) {
+ if (res->fi->fib_nhs > 1 && oif_check) {
if (mp_hash < 0)
mp_hash = get_hash_from_flowi4(fl4) >> 1;
#endif
if (!res->prefixlen &&
res->table->tb_num_default > 1 &&
- res->type == RTN_UNICAST && !fl4->flowi4_oif)
+ res->type == RTN_UNICAST && oif_check)
fib_select_default(fl4, res);
if (!fl4->saddr)
rcu_read_lock_bh();
c = __clusterip_config_find(net, clusterip);
if (c) {
- if (unlikely(!atomic_inc_not_zero(&c->refcount)))
+ if (!c->pde || unlikely(!atomic_inc_not_zero(&c->refcount)))
c = NULL;
else if (entry)
atomic_inc(&c->entries);
static struct clusterip_config *
clusterip_config_init(const struct ipt_clusterip_tgt_info *i, __be32 ip,
- struct net_device *dev)
+ struct net_device *dev)
{
+ struct net *net = dev_net(dev);
struct clusterip_config *c;
- struct clusterip_net *cn = net_generic(dev_net(dev), clusterip_net_id);
+ struct clusterip_net *cn = net_generic(net, clusterip_net_id);
c = kzalloc(sizeof(*c), GFP_ATOMIC);
if (!c)
- return NULL;
+ return ERR_PTR(-ENOMEM);
c->dev = dev;
c->clusterip = ip;
atomic_set(&c->refcount, 1);
atomic_set(&c->entries, 1);
+ spin_lock_bh(&cn->lock);
+ if (__clusterip_config_find(net, ip)) {
+ spin_unlock_bh(&cn->lock);
+ kfree(c);
+
+ return ERR_PTR(-EBUSY);
+ }
+
+ list_add_rcu(&c->list, &cn->configs);
+ spin_unlock_bh(&cn->lock);
+
#ifdef CONFIG_PROC_FS
{
char buffer[16];
cn->procdir,
&clusterip_proc_fops, c);
if (!c->pde) {
+ spin_lock_bh(&cn->lock);
+ list_del_rcu(&c->list);
+ spin_unlock_bh(&cn->lock);
kfree(c);
- return NULL;
+
+ return ERR_PTR(-ENOMEM);
}
}
#endif
- spin_lock_bh(&cn->lock);
- list_add_rcu(&c->list, &cn->configs);
- spin_unlock_bh(&cn->lock);
-
return c;
}
config = clusterip_config_init(cipinfo,
e->ip.dst.s_addr, dev);
- if (!config) {
+ if (IS_ERR(config)) {
dev_put(dev);
- return -ENOMEM;
+ return PTR_ERR(config);
}
dev_mc_add(config->dev, config->clustermac);
}
r->rtm_dst_len = 32;
r->rtm_src_len = 0;
r->rtm_tos = fl4->flowi4_tos;
- r->rtm_table = table_id;
+ r->rtm_table = table_id < 256 ? table_id : RT_TABLE_COMPAT;
if (nla_put_u32(skb, RTA_TABLE, table_id))
goto nla_put_failure;
r->rtm_type = rt->rt_type;
.data = &init_net.ipv4.sysctl_tcp_notsent_lowat,
.maxlen = sizeof(unsigned int),
.mode = 0644,
- .proc_handler = proc_dointvec,
+ .proc_handler = proc_douintvec,
},
{
.procname = "tcp_tw_reuse",
struct tcp_fastopen_cookie tmp;
if (__tcp_fastopen_cookie_gen(&ip6h->saddr, &tmp)) {
- struct in6_addr *buf = (struct in6_addr *) tmp.val;
+ struct in6_addr *buf = &tmp.addr;
int i;
for (i = 0; i < 4; i++)
return ret;
}
-EXPORT_SYMBOL_GPL(tcp_peer_is_proven);
void tcp_fetch_timewait_stamp(struct sock *sk, struct dst_entry *dst)
{
ops = rcu_dereference(inet6_offloads[proto]);
if (!ops || !ops->callbacks.gro_receive) {
__pskb_pull(skb, skb_gro_offset(skb));
+ skb_gro_frag0_invalidate(skb);
proto = ipv6_gso_pull_exthdrs(skb, proto);
skb_gro_pull(skb, -skb_transport_offset(skb));
skb_reset_transport_header(skb);
t->parms.name);
goto tx_err_dst_release;
}
- mtu = dst_mtu(dst) - psh_hlen;
+ mtu = dst_mtu(dst) - psh_hlen - t->tun_hlen;
if (encap_limit >= 0) {
max_headroom += 8;
mtu -= 8;
mtu = IPV6_MIN_MTU;
if (skb_dst(skb) && !t->parms.collect_md)
skb_dst(skb)->ops->update_pmtu(skb_dst(skb), NULL, skb, mtu);
- if (skb->len > mtu && !skb_is_gso(skb)) {
+ if (skb->len - t->tun_hlen > mtu && !skb_is_gso(skb)) {
*pmtu = mtu;
err = -EMSGSIZE;
goto tx_err_dst_release;
struct vti6_net *ip6n = net_generic(net, vti6_net_id);
int err;
+ dev->rtnl_link_ops = &vti6_link_ops;
err = register_netdevice(dev);
if (err < 0)
goto out;
strcpy(t->parms.name, dev->name);
- dev->rtnl_link_ops = &vti6_link_ops;
dev_hold(dev);
vti6_tnl_link(ip6n, t);
static void mld_ifc_timer_expire(unsigned long data);
static void mld_ifc_event(struct inet6_dev *idev);
static void mld_add_delrec(struct inet6_dev *idev, struct ifmcaddr6 *pmc);
-static void mld_del_delrec(struct inet6_dev *idev, const struct in6_addr *addr);
+static void mld_del_delrec(struct inet6_dev *idev, struct ifmcaddr6 *pmc);
static void mld_clear_delrec(struct inet6_dev *idev);
static bool mld_in_v1_mode(const struct inet6_dev *idev);
static int sf_setstate(struct ifmcaddr6 *pmc);
dev_mc_del(dev, buf);
}
- if (mc->mca_flags & MAF_NOREPORT)
- goto done;
spin_unlock_bh(&mc->mca_lock);
+ if (mc->mca_flags & MAF_NOREPORT)
+ return;
if (!mc->idev->dead)
igmp6_leave_group(mc);
spin_lock_bh(&mc->mca_lock);
if (del_timer(&mc->mca_timer))
atomic_dec(&mc->mca_refcnt);
-done:
- ip6_mc_clear_src(mc);
spin_unlock_bh(&mc->mca_lock);
}
spin_unlock_bh(&idev->mc_lock);
}
-static void mld_del_delrec(struct inet6_dev *idev, const struct in6_addr *pmca)
+static void mld_del_delrec(struct inet6_dev *idev, struct ifmcaddr6 *im)
{
struct ifmcaddr6 *pmc, *pmc_prev;
- struct ip6_sf_list *psf, *psf_next;
+ struct ip6_sf_list *psf;
+ struct in6_addr *pmca = &im->mca_addr;
spin_lock_bh(&idev->mc_lock);
pmc_prev = NULL;
}
spin_unlock_bh(&idev->mc_lock);
+ spin_lock_bh(&im->mca_lock);
if (pmc) {
- for (psf = pmc->mca_tomb; psf; psf = psf_next) {
- psf_next = psf->sf_next;
- kfree(psf);
+ im->idev = pmc->idev;
+ im->mca_crcount = idev->mc_qrv;
+ im->mca_sfmode = pmc->mca_sfmode;
+ if (pmc->mca_sfmode == MCAST_INCLUDE) {
+ im->mca_tomb = pmc->mca_tomb;
+ im->mca_sources = pmc->mca_sources;
+ for (psf = im->mca_sources; psf; psf = psf->sf_next)
+ psf->sf_crcount = im->mca_crcount;
}
in6_dev_put(pmc->idev);
- kfree(pmc);
}
+ spin_unlock_bh(&im->mca_lock);
}
static void mld_clear_delrec(struct inet6_dev *idev)
mca_get(mc);
write_unlock_bh(&idev->lock);
- mld_del_delrec(idev, &mc->mca_addr);
+ mld_del_delrec(idev, mc);
igmp6_group_added(mc);
ma_put(mc);
return 0;
write_unlock_bh(&idev->lock);
igmp6_group_dropped(ma);
+ ip6_mc_clear_src(ma);
ma_put(ma);
return 0;
/* Withdraw multicast list */
read_lock_bh(&idev->lock);
- mld_ifc_stop_timer(idev);
- mld_gq_stop_timer(idev);
- mld_dad_stop_timer(idev);
for (i = idev->mc_list; i; i = i->next)
igmp6_group_dropped(i);
- read_unlock_bh(&idev->lock);
- mld_clear_delrec(idev);
+ /* Should stop timer after group drop. or we will
+ * start timer again in mld_ifc_event()
+ */
+ mld_ifc_stop_timer(idev);
+ mld_gq_stop_timer(idev);
+ mld_dad_stop_timer(idev);
+ read_unlock_bh(&idev->lock);
}
static void ipv6_mc_reset(struct inet6_dev *idev)
read_lock_bh(&idev->lock);
ipv6_mc_reset(idev);
- for (i = idev->mc_list; i; i = i->next)
+ for (i = idev->mc_list; i; i = i->next) {
+ mld_del_delrec(idev, i);
igmp6_group_added(i);
+ }
read_unlock_bh(&idev->lock);
}
/* Deactivate timers */
ipv6_mc_down(idev);
+ mld_clear_delrec(idev);
/* Delete all-nodes address. */
/* We cannot call ipv6_dev_mc_dec() directly, our caller in
write_lock_bh(&idev->lock);
while ((i = idev->mc_list) != NULL) {
idev->mc_list = i->next;
- write_unlock_bh(&idev->lock);
- igmp6_group_dropped(i);
+ write_unlock_bh(&idev->lock);
ma_put(i);
-
write_lock_bh(&idev->lock);
}
write_unlock_bh(&idev->lock);
struct fib6_node *fn;
/* Get the "current" route for this destination and
- * check if the redirect has come from approriate router.
+ * check if the redirect has come from appropriate router.
*
* RFC 4861 specifies that redirects should only be
* accepted if they come from the nexthop to the target.
old MTU is the lowest MTU in the path, update the route PMTU
to reflect the increase. In this case if the other nodes' MTU
also have the lowest MTU, TOO BIG MESSAGE will be lead to
- PMTU discouvery.
+ PMTU discovery.
*/
if (rt->dst.dev == arg->dev &&
dst_metric_raw(&rt->dst, RTAX_MTU) &&
if (nla_put_u8(skb, RTA_PREF, IPV6_EXTRACT_PREF(rt->rt6i_flags)))
goto nla_put_failure;
- lwtunnel_fill_encap(skb, rt->dst.lwtstate);
+ if (lwtunnel_fill_encap(skb, rt->dst.lwtstate) < 0)
+ goto nla_put_failure;
nlmsg_end(skb, nlh);
return 0;
*p_tfm = tfm;
}
- p_tfm = this_cpu_ptr(algo->tfms);
+ p_tfm = raw_cpu_ptr(algo->tfms);
tfm = *p_tfm;
shsize = sizeof(*shash) + crypto_shash_descsize(tfm);
slwt = seg6_lwt_lwtunnel(orig_dst->lwtstate);
#ifdef CONFIG_DST_CACHE
+ preempt_disable();
dst = dst_cache_get(&slwt->cache);
+ preempt_enable();
#endif
if (unlikely(!dst)) {
}
#ifdef CONFIG_DST_CACHE
+ preempt_disable();
dst_cache_set_ip6(&slwt->cache, dst, &fl6.saddr);
+ preempt_enable();
#endif
}
{
struct sock *sk = sock->sk;
struct iucv_sock *iucv = iucv_sk(sk);
- size_t headroom, linear;
+ size_t headroom = 0;
+ size_t linear;
struct sk_buff *skb;
struct iucv_message txmsg = {0};
struct cmsghdr *cmsg;
* this is fine for SOCK_SEQPACKET (unless we want to support
* segmented records using the MSG_EOR flag), but
* for SOCK_STREAM we might want to improve it in future */
- headroom = (iucv->transport == AF_IUCV_TRANS_HIPER)
- ? sizeof(struct af_iucv_trans_hdr) + ETH_HLEN : 0;
- if (headroom + len < PAGE_SIZE) {
+ if (iucv->transport == AF_IUCV_TRANS_HIPER) {
+ headroom = sizeof(struct af_iucv_trans_hdr) + ETH_HLEN;
linear = len;
} else {
- /* In nonlinear "classic" iucv skb,
- * reserve space for iucv_array
- */
- if (iucv->transport != AF_IUCV_TRANS_HIPER)
- headroom += sizeof(struct iucv_array) *
- (MAX_SKB_FRAGS + 1);
- linear = PAGE_SIZE - headroom;
+ if (len < PAGE_SIZE) {
+ linear = len;
+ } else {
+ /* In nonlinear "classic" iucv skb,
+ * reserve space for iucv_array
+ */
+ headroom = sizeof(struct iucv_array) *
+ (MAX_SKB_FRAGS + 1);
+ linear = PAGE_SIZE - headroom;
+ }
}
skb = sock_alloc_send_pskb(sk, headroom + linear, len - linear,
noblock, &err, 0);
!(sta->sdata->bss && sta->sdata->bss == sdata->bss))
continue;
- if (!sta->uploaded || !test_sta_flag(sta, WLAN_STA_ASSOC))
- continue;
-
max_bw = max(max_bw, ieee80211_get_sta_bw(&sta->sta));
}
rcu_read_unlock();
* Copyright (c) 2006 Jiri Benc <jbenc@suse.cz>
* Copyright 2008, Johannes Berg <johannes@sipsolutions.net>
* Copyright 2013-2014 Intel Mobile Communications GmbH
+ * Copyright (c) 2016 Intel Deutschland GmbH
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 as
} else if (ieee80211_is_action(mgmt->frame_control) &&
mgmt->u.action.category == WLAN_CATEGORY_VHT) {
switch (mgmt->u.action.u.vht_group_notif.action_code) {
+ case WLAN_VHT_ACTION_OPMODE_NOTIF: {
+ struct ieee80211_rx_status *status;
+ enum nl80211_band band;
+ u8 opmode;
+
+ status = IEEE80211_SKB_RXCB(skb);
+ band = status->band;
+ opmode = mgmt->u.action.u.vht_opmode_notif.operating_mode;
+
+ mutex_lock(&local->sta_mtx);
+ sta = sta_info_get_bss(sdata, mgmt->sa);
+
+ if (sta)
+ ieee80211_vht_handle_opmode(sdata, sta,
+ opmode,
+ band);
+
+ mutex_unlock(&local->sta_mtx);
+ break;
+ }
case WLAN_VHT_ACTION_GROUPID_MGMT:
ieee80211_process_mu_groups(sdata, mgmt);
break;
supp_ht = supp_ht || sband->ht_cap.ht_supported;
supp_vht = supp_vht || sband->vht_cap.vht_supported;
- if (sband->ht_cap.ht_supported)
- local->rx_chains =
- max(ieee80211_mcs_to_chains(&sband->ht_cap.mcs),
- local->rx_chains);
+ if (!sband->ht_cap.ht_supported)
+ continue;
/* TODO: consider VHT for RX chains, hopefully it's the same */
+ local->rx_chains =
+ max(ieee80211_mcs_to_chains(&sband->ht_cap.mcs),
+ local->rx_chains);
+
+ /* no need to mask, SM_PS_DISABLED has all bits set */
+ sband->ht_cap.cap |= WLAN_HT_CAP_SM_PS_DISABLED <<
+ IEEE80211_HT_CAP_SM_PS_SHIFT;
}
/* if low-level driver supports AP, we also support VLAN */
ieee80211_sta_set_rx_nss(sta);
+ ieee80211_recalc_min_chandef(sta->sdata);
+
if (!ref)
return;
if (!ifmsh->mshcfg.dot11MeshForwarding)
goto out;
- fwd_skb = skb_copy_expand(skb, local->tx_headroom, 0, GFP_ATOMIC);
+ fwd_skb = skb_copy_expand(skb, local->tx_headroom +
+ sdata->encrypt_headroom, 0, GFP_ATOMIC);
if (!fwd_skb) {
net_info_ratelimited("%s: failed to clone mesh frame\n",
sdata->name);
switch (mgmt->u.action.u.vht_opmode_notif.action_code) {
case WLAN_VHT_ACTION_OPMODE_NOTIF: {
- u8 opmode;
-
/* verify opmode is present */
if (len < IEEE80211_MIN_ACTION_SIZE + 2)
goto invalid;
-
- opmode = mgmt->u.action.u.vht_opmode_notif.operating_mode;
-
- ieee80211_vht_handle_opmode(rx->sdata, rx->sta,
- opmode, status->band);
- goto handled;
+ goto queue;
}
case WLAN_VHT_ACTION_GROUPID_MGMT: {
if (len < IEEE80211_MIN_ACTION_SIZE + 25)
u64_stats_update_end(&stats->syncp);
if (fast_rx->internal_forward) {
- struct sta_info *dsta = sta_info_get(rx->sdata, skb->data);
+ struct sk_buff *xmit_skb = NULL;
+ bool multicast = is_multicast_ether_addr(skb->data);
+
+ if (multicast) {
+ xmit_skb = skb_copy(skb, GFP_ATOMIC);
+ } else if (sta_info_get(rx->sdata, skb->data)) {
+ xmit_skb = skb;
+ skb = NULL;
+ }
- if (dsta) {
+ if (xmit_skb) {
/*
* Send to wireless media and increase priority by 256
* to keep the received priority instead of
* reclassifying the frame (see cfg80211_classify8021d).
*/
- skb->priority += 256;
- skb->protocol = htons(ETH_P_802_3);
- skb_reset_network_header(skb);
- skb_reset_mac_header(skb);
- dev_queue_xmit(skb);
- return true;
+ xmit_skb->priority += 256;
+ xmit_skb->protocol = htons(ETH_P_802_3);
+ skb_reset_network_header(xmit_skb);
+ skb_reset_mac_header(xmit_skb);
+ dev_queue_xmit(xmit_skb);
}
+
+ if (!skb)
+ return true;
}
/* deliver to local stack */
/* This will evaluate to 1, 3, 5 or 7. */
for (ac = IEEE80211_AC_VO; ac < IEEE80211_NUM_ACS; ac++)
- if (ignored_acs & BIT(ac))
- continue;
+ if (!(ignored_acs & ieee80211_ac_to_qos_mask[ac]))
+ break;
tid = 7 - 2 * ac;
ieee80211_send_null_response(sta, tid, reason, true, false);
static struct txq_info *ieee80211_get_txq(struct ieee80211_local *local,
struct ieee80211_vif *vif,
- struct ieee80211_sta *pubsta,
+ struct sta_info *sta,
struct sk_buff *skb)
{
struct ieee80211_hdr *hdr = (struct ieee80211_hdr *) skb->data;
if (!ieee80211_is_data(hdr->frame_control))
return NULL;
- if (pubsta) {
+ if (sta) {
u8 tid = skb->priority & IEEE80211_QOS_CTL_TID_MASK;
- txq = pubsta->txq[tid];
+ if (!sta->uploaded)
+ return NULL;
+
+ txq = sta->sta.txq[tid];
} else if (vif) {
txq = vif->txq;
}
struct fq *fq = &local->fq;
struct ieee80211_vif *vif;
struct txq_info *txqi;
- struct ieee80211_sta *pubsta;
if (!local->ops->wake_tx_queue ||
sdata->vif.type == NL80211_IFTYPE_MONITOR)
return false;
- if (sta && sta->uploaded)
- pubsta = &sta->sta;
- else
- pubsta = NULL;
-
if (sdata->vif.type == NL80211_IFTYPE_AP_VLAN)
sdata = container_of(sdata->bss,
struct ieee80211_sub_if_data, u.ap);
vif = &sdata->vif;
- txqi = ieee80211_get_txq(local, vif, pubsta, skb);
+ txqi = ieee80211_get_txq(local, vif, sta, skb);
if (!txqi)
return false;
u32 changed = __ieee80211_vht_handle_opmode(sdata, sta, opmode, band);
- if (changed > 0)
+ if (changed > 0) {
+ ieee80211_recalc_min_chandef(sdata);
rate_control_rate_update(local, sband, sta, changed);
+ }
}
void ieee80211_get_vht_mask_from_cap(__le16 vht_cap,
* is called on error from nf_tables_newrule().
*/
expr = nft_expr_first(rule);
- while (expr->ops && expr != nft_expr_last(rule)) {
+ while (expr != nft_expr_last(rule) && expr->ops) {
nf_tables_expr_destroy(ctx, expr);
expr = nft_expr_next(expr);
}
return 0;
}
+static int nft_payload_csum_inet(struct sk_buff *skb, const u32 *src,
+ __wsum fsum, __wsum tsum, int csum_offset)
+{
+ __sum16 sum;
+
+ if (skb_copy_bits(skb, csum_offset, &sum, sizeof(sum)) < 0)
+ return -1;
+
+ nft_csum_replace(&sum, fsum, tsum);
+ if (!skb_make_writable(skb, csum_offset + sizeof(sum)) ||
+ skb_store_bits(skb, csum_offset, &sum, sizeof(sum)) < 0)
+ return -1;
+
+ return 0;
+}
+
static void nft_payload_set_eval(const struct nft_expr *expr,
struct nft_regs *regs,
const struct nft_pktinfo *pkt)
const u32 *src = ®s->data[priv->sreg];
int offset, csum_offset;
__wsum fsum, tsum;
- __sum16 sum;
switch (priv->base) {
case NFT_PAYLOAD_LL_HEADER:
csum_offset = offset + priv->csum_offset;
offset += priv->offset;
- if (priv->csum_type == NFT_PAYLOAD_CSUM_INET &&
+ if ((priv->csum_type == NFT_PAYLOAD_CSUM_INET || priv->csum_flags) &&
(priv->base != NFT_PAYLOAD_TRANSPORT_HEADER ||
skb->ip_summed != CHECKSUM_PARTIAL)) {
- if (skb_copy_bits(skb, csum_offset, &sum, sizeof(sum)) < 0)
- goto err;
-
fsum = skb_checksum(skb, offset, priv->len, 0);
tsum = csum_partial(src, priv->len, 0);
- nft_csum_replace(&sum, fsum, tsum);
- if (!skb_make_writable(skb, csum_offset + sizeof(sum)) ||
- skb_store_bits(skb, csum_offset, &sum, sizeof(sum)) < 0)
+ if (priv->csum_type == NFT_PAYLOAD_CSUM_INET &&
+ nft_payload_csum_inet(skb, src, fsum, tsum, csum_offset))
goto err;
if (priv->csum_flags &&
if (priv->queues_total > 1) {
if (priv->flags & NFT_QUEUE_FLAG_CPU_FANOUT) {
- int cpu = smp_processor_id();
+ int cpu = raw_smp_processor_id();
queue = priv->queuenum + cpu % priv->queues_total;
} else {
static int nft_quota_do_dump(struct sk_buff *skb, struct nft_quota *priv,
bool reset)
{
+ u64 consumed, consumed_cap;
u32 flags = priv->flags;
- u64 consumed;
-
- if (reset) {
- consumed = atomic64_xchg(&priv->consumed, 0);
- if (test_and_clear_bit(NFT_QUOTA_DEPLETED_BIT, &priv->flags))
- flags |= NFT_QUOTA_F_DEPLETED;
- } else {
- consumed = atomic64_read(&priv->consumed);
- }
/* Since we inconditionally increment consumed quota for each packet
* that we see, don't go over the quota boundary in what we send to
* userspace.
*/
- if (consumed > priv->quota)
- consumed = priv->quota;
+ consumed = atomic64_read(&priv->consumed);
+ if (consumed >= priv->quota) {
+ consumed_cap = priv->quota;
+ flags |= NFT_QUOTA_F_DEPLETED;
+ } else {
+ consumed_cap = consumed;
+ }
if (nla_put_be64(skb, NFTA_QUOTA_BYTES, cpu_to_be64(priv->quota),
NFTA_QUOTA_PAD) ||
- nla_put_be64(skb, NFTA_QUOTA_CONSUMED, cpu_to_be64(consumed),
+ nla_put_be64(skb, NFTA_QUOTA_CONSUMED, cpu_to_be64(consumed_cap),
NFTA_QUOTA_PAD) ||
nla_put_be32(skb, NFTA_QUOTA_FLAGS, htonl(flags)))
goto nla_put_failure;
+
+ if (reset) {
+ atomic64_sub(consumed, &priv->consumed);
+ clear_bit(NFT_QUOTA_DEPLETED_BIT, &priv->flags);
+ }
return 0;
nla_put_failure:
printk(KERN_INFO "NetLabel: Initializing\n");
printk(KERN_INFO "NetLabel: domain hash size = %u\n",
(1 << NETLBL_DOMHSH_BITSIZE));
- printk(KERN_INFO "NetLabel: protocols ="
- " UNLABELED"
- " CIPSOv4"
- "\n");
+ printk(KERN_INFO "NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO\n");
ret_val = netlbl_domhsh_init(NETLBL_DOMHSH_BITSIZE);
if (ret_val != 0)
int hooknum, nh_off, err = NF_ACCEPT;
nh_off = skb_network_offset(skb);
- skb_pull(skb, nh_off);
+ skb_pull_rcsum(skb, nh_off);
/* See HOOK2MANIP(). */
if (maniptype == NF_NAT_MANIP_SRC)
err = nf_nat_packet(ct, ctinfo, hooknum, skb);
push:
skb_push(skb, nh_off);
+ skb_postpush_rcsum(skb, skb->data, nh_off);
return err;
}
/* The conntrack module expects to be working at L3. */
nh_ofs = skb_network_offset(skb);
- skb_pull(skb, nh_ofs);
+ skb_pull_rcsum(skb, nh_ofs);
if (key->ip.frag != OVS_FRAG_TYPE_NONE) {
err = handle_fragments(net, key, info->zone.id, skb);
err = ovs_ct_lookup(net, key, info, skb);
skb_push(skb, nh_ofs);
+ skb_postpush_rcsum(skb, skb->data, nh_ofs);
if (err)
kfree_skb(skb);
return err;
const int pkt_len = 20;
struct qrtr_hdr *hdr;
struct sk_buff *skb;
- u32 *buf;
+ __le32 *buf;
skb = alloc_skb(QRTR_HDR_SIZE + pkt_len, GFP_KERNEL);
if (!skb)
hdr->dst_node_id = cpu_to_le32(dst_node);
hdr->dst_port_id = cpu_to_le32(QRTR_PORT_CTRL);
- buf = (u32 *)skb_put(skb, pkt_len);
+ buf = (__le32 *)skb_put(skb, pkt_len);
memset(buf, 0, pkt_len);
buf[0] = cpu_to_le32(QRTR_TYPE_RESUME_TX);
buf[1] = cpu_to_le32(src_node);
goto err;
}
act->order = i;
- if (event == RTM_GETACTION)
- act->tcfa_refcnt++;
list_add_tail(&act->list, &actions);
}
return ret;
}
err:
- tcf_action_destroy(&actions, 0);
+ if (event != RTM_GETACTION)
+ tcf_action_destroy(&actions, 0);
return ret;
}
nla_put_string(skb, TCA_ACT_BPF_NAME, prog->bpf_name))
return -EMSGSIZE;
- nla = nla_reserve(skb, TCA_ACT_BPF_DIGEST,
- sizeof(prog->filter->digest));
+ nla = nla_reserve(skb, TCA_ACT_BPF_TAG, sizeof(prog->filter->tag));
if (nla == NULL)
return -EMSGSIZE;
- memcpy(nla_data(nla), prog->filter->digest, nla_len(nla));
+ memcpy(nla_data(nla), prog->filter->tag, nla_len(nla));
return 0;
}
nla_put_string(skb, TCA_BPF_NAME, prog->bpf_name))
return -EMSGSIZE;
- nla = nla_reserve(skb, TCA_BPF_DIGEST, sizeof(prog->filter->digest));
+ nla = nla_reserve(skb, TCA_BPF_TAG, sizeof(prog->filter->tag));
if (nla == NULL)
return -EMSGSIZE;
- memcpy(nla_data(nla), prog->filter->digest, nla_len(nla));
+ memcpy(nla_data(nla), prog->filter->tag, nla_len(nla));
return 0;
}
(new_transport->state == SCTP_PF)))
new_transport = asoc->peer.active_path;
if (new_transport->state == SCTP_UNCONFIRMED) {
- WARN_ONCE(1, "Atempt to send packet on unconfirmed path.");
+ WARN_ONCE(1, "Attempt to send packet on unconfirmed path.");
sctp_chunk_fail(chunk, 0);
sctp_chunk_free(chunk);
continue;
return used;
}
-int sockfs_setattr(struct dentry *dentry, struct iattr *iattr)
+static int sockfs_setattr(struct dentry *dentry, struct iattr *iattr)
{
int err = simple_setattr(dentry, iattr);
case RPC_GSS_PROC_DESTROY:
if (gss_write_verf(rqstp, rsci->mechctx, gc->gc_seq))
goto auth_err;
- rsci->h.expiry_time = get_seconds();
+ rsci->h.expiry_time = seconds_since_boot();
set_bit(CACHE_NEGATIVE, &rsci->h.flags);
if (resv->iov_len + 4 > PAGE_SIZE)
goto drop;
if (test_bit(XPT_CLOSE, &xprt->xpt_flags)) {
dprintk("svc_recv: found XPT_CLOSE\n");
+ if (test_and_clear_bit(XPT_KILL_TEMP, &xprt->xpt_flags))
+ xprt->xpt_ops->xpo_kill_temp_xprt(xprt);
svc_delete_xprt(xprt);
/* Leave XPT_BUSY set on the dead xprt: */
goto out;
le = to_be_closed.next;
list_del_init(le);
xprt = list_entry(le, struct svc_xprt, xpt_list);
- dprintk("svc_age_temp_xprts_now: closing %p\n", xprt);
- xprt->xpt_ops->xpo_kill_temp_xprt(xprt);
- svc_close_xprt(xprt);
+ set_bit(XPT_CLOSE, &xprt->xpt_flags);
+ set_bit(XPT_KILL_TEMP, &xprt->xpt_flags);
+ dprintk("svc_age_temp_xprts_now: queuing xprt %p for closing\n",
+ xprt);
+ svc_xprt_enqueue(xprt);
}
}
EXPORT_SYMBOL_GPL(svc_age_temp_xprts_now);
atomic_inc(&rdma_stat_read);
return ret;
err:
- ib_dma_unmap_sg(xprt->sc_cm_id->device,
- frmr->sg, frmr->sg_nents, frmr->direction);
svc_rdma_put_context(ctxt, 0);
svc_rdma_put_frmr(xprt, frmr);
return ret;
/* Send response, if necessary */
if (respond && (mtyp == DSC_REQ_MSG)) {
- rskb = tipc_buf_acquire(MAX_H_SIZE);
+ rskb = tipc_buf_acquire(MAX_H_SIZE, GFP_ATOMIC);
if (!rskb)
return;
tipc_disc_init_msg(net, rskb, DSC_RESP_MSG, bearer);
req = kmalloc(sizeof(*req), GFP_ATOMIC);
if (!req)
return -ENOMEM;
- req->buf = tipc_buf_acquire(MAX_H_SIZE);
+ req->buf = tipc_buf_acquire(MAX_H_SIZE, GFP_ATOMIC);
if (!req->buf) {
kfree(req);
return -ENOMEM;
msg_set_seqno(hdr, seqno++);
pktlen = msg_size(hdr);
msg_set_size(&tnlhdr, pktlen + INT_H_SIZE);
- tnlskb = tipc_buf_acquire(pktlen + INT_H_SIZE);
+ tnlskb = tipc_buf_acquire(pktlen + INT_H_SIZE, GFP_ATOMIC);
if (!tnlskb) {
pr_warn("%sunable to send packet\n", link_co_err);
return;
* NOTE: Headroom is reserved to allow prepending of a data link header.
* There may also be unrequested tailroom present at the buffer's end.
*/
-struct sk_buff *tipc_buf_acquire(u32 size)
+struct sk_buff *tipc_buf_acquire(u32 size, gfp_t gfp)
{
struct sk_buff *skb;
unsigned int buf_size = (BUF_HEADROOM + size + 3) & ~3u;
- skb = alloc_skb_fclone(buf_size, GFP_ATOMIC);
+ skb = alloc_skb_fclone(buf_size, gfp);
if (skb) {
skb_reserve(skb, BUF_HEADROOM);
skb_put(skb, size);
struct tipc_msg *msg;
struct sk_buff *buf;
- buf = tipc_buf_acquire(hdr_sz + data_sz);
+ buf = tipc_buf_acquire(hdr_sz + data_sz, GFP_ATOMIC);
if (unlikely(!buf))
return NULL;
/* No fragmentation needed? */
if (likely(msz <= pktmax)) {
- skb = tipc_buf_acquire(msz);
+ skb = tipc_buf_acquire(msz, GFP_KERNEL);
if (unlikely(!skb))
return -ENOMEM;
skb_orphan(skb);
msg_set_importance(&pkthdr, msg_importance(mhdr));
/* Prepare first fragment */
- skb = tipc_buf_acquire(pktmax);
+ skb = tipc_buf_acquire(pktmax, GFP_KERNEL);
if (!skb)
return -ENOMEM;
skb_orphan(skb);
pktsz = drem + INT_H_SIZE;
else
pktsz = pktmax;
- skb = tipc_buf_acquire(pktsz);
+ skb = tipc_buf_acquire(pktsz, GFP_KERNEL);
if (!skb) {
rc = -ENOMEM;
goto error;
if (msz > (max / 2))
return false;
- _skb = tipc_buf_acquire(max);
+ _skb = tipc_buf_acquire(max, GFP_ATOMIC);
if (!_skb)
return false;
/* Never return SHORT header; expand by replacing buffer if necessary */
if (msg_short(hdr)) {
- *skb = tipc_buf_acquire(BASIC_H_SIZE + dlen);
+ *skb = tipc_buf_acquire(BASIC_H_SIZE + dlen, GFP_ATOMIC);
if (!*skb)
goto exit;
memcpy((*skb)->data + BASIC_H_SIZE, msg_data(hdr), dlen);
return (msg_user(hdr) == LINK_PROTOCOL) && (msg_type(hdr) == RESET_MSG);
}
-struct sk_buff *tipc_buf_acquire(u32 size);
+struct sk_buff *tipc_buf_acquire(u32 size, gfp_t gfp);
bool tipc_msg_validate(struct sk_buff *skb);
bool tipc_msg_reverse(u32 own_addr, struct sk_buff **skb, int err);
void tipc_msg_init(u32 own_addr, struct tipc_msg *m, u32 user, u32 type,
u32 dest)
{
struct tipc_net *tn = net_generic(net, tipc_net_id);
- struct sk_buff *buf = tipc_buf_acquire(INT_H_SIZE + size);
+ struct sk_buff *buf = tipc_buf_acquire(INT_H_SIZE + size, GFP_ATOMIC);
struct tipc_msg *msg;
if (buf != NULL) {
break;
}
+ /*
+ * Older kernel versions ignored this attribute entirely, so don't
+ * reject attempts to update it but mark it as unused instead so the
+ * driver won't look at the data.
+ */
+ if (statype != CFG80211_STA_AP_CLIENT_UNASSOC &&
+ statype != CFG80211_STA_TDLS_PEER_SETUP)
+ params->opmode_notif_used = false;
+
return 0;
}
EXPORT_SYMBOL(cfg80211_check_station_change);
params.local_pm = pm;
}
+ if (info->attrs[NL80211_ATTR_OPMODE_NOTIF]) {
+ params.opmode_notif_used = true;
+ params.opmode_notif =
+ nla_get_u8(info->attrs[NL80211_ATTR_OPMODE_NOTIF]);
+ }
+
/* Include parameters for TDLS peer (will check later) */
err = nl80211_set_station_tdls(info, ¶ms);
if (err)
list_for_each_entry_rcu(rdev, &cfg80211_rdev_list, list) {
bool schedule_destroy_work = false;
- bool schedule_scan_stop = false;
struct cfg80211_sched_scan_request *sched_scan_req =
rcu_dereference(rdev->sched_scan_req);
if (sched_scan_req && notify->portid &&
- sched_scan_req->owner_nlportid == notify->portid)
- schedule_scan_stop = true;
+ sched_scan_req->owner_nlportid == notify->portid) {
+ sched_scan_req->owner_nlportid = 0;
+
+ if (rdev->ops->sched_scan_stop &&
+ rdev->wiphy.flags & WIPHY_FLAG_SUPPORTS_SCHED_SCAN)
+ schedule_work(&rdev->sched_scan_stop_wk);
+ }
list_for_each_entry_rcu(wdev, &rdev->wiphy.wdev_list, list) {
cfg80211_mlme_unregister_socket(wdev, notify->portid);
spin_unlock(&rdev->destroy_list_lock);
schedule_work(&rdev->destroy_work);
}
- } else if (schedule_scan_stop) {
- sched_scan_req->owner_nlportid = 0;
-
- if (rdev->ops->sched_scan_stop &&
- rdev->wiphy.flags & WIPHY_FLAG_SUPPORTS_SCHED_SCAN)
- schedule_work(&rdev->sched_scan_stop_wk);
}
}
#include <unistd.h>
#include <string.h>
#include <errno.h>
-#include <net/ethernet.h>
+#include <linux/if_ether.h>
#include <net/if.h>
#include <linux/if_packet.h>
#include <arpa/inet.h>
#include <string.h>
#include <fcntl.h>
#include <poll.h>
-#include <sys/ioctl.h>
#include <linux/perf_event.h>
#include <linux/bpf.h>
#include <errno.h>
{
unsigned int size = 0;
struct mdev_state *mdev_state;
- int bar_index;
+ u32 bar_index;
if (!mdev)
return -EINVAL;
if (!mdev_state)
return -EINVAL;
- mutex_lock(&mdev_state->ops_lock);
bar_index = region_info->index;
+ if (bar_index >= VFIO_PCI_NUM_REGIONS)
+ return -EINVAL;
+
+ mutex_lock(&mdev_state->ops_lock);
switch (bar_index) {
case VFIO_PCI_CONFIG_REGION_INDEX:
memcpy(&mdev_state->dev_info, &info, sizeof(info));
- return copy_to_user((void __user *)arg, &info, minsz);
+ if (copy_to_user((void __user *)arg, &info, minsz))
+ return -EFAULT;
+
+ return 0;
}
case VFIO_DEVICE_GET_REGION_INFO:
{
if (ret)
return ret;
- return copy_to_user((void __user *)arg, &info, minsz);
+ if (copy_to_user((void __user *)arg, &info, minsz))
+ return -EFAULT;
+
+ return 0;
}
case VFIO_DEVICE_GET_IRQ_INFO:
if (ret)
return ret;
- if (info.count == -1)
- return -EINVAL;
+ if (copy_to_user((void __user *)arg, &info, minsz))
+ return -EFAULT;
- return copy_to_user((void __user *)arg, &info, minsz);
+ return 0;
}
case VFIO_DEVICE_SET_IRQS:
{
}
EXPORT_SYMBOL_GPL(snd_hda_get_conn_index);
-
-/* return DEVLIST_LEN parameter of the given widget */
-static unsigned int get_num_devices(struct hda_codec *codec, hda_nid_t nid)
+/**
+ * snd_hda_get_num_devices - get DEVLIST_LEN parameter of the given widget
+ * @codec: the HDA codec
+ * @nid: NID of the pin to parse
+ *
+ * Get the device entry number on the given widget. This is a feature of
+ * DP MST audio. Each pin can have several device entries in it.
+ */
+unsigned int snd_hda_get_num_devices(struct hda_codec *codec, hda_nid_t nid)
{
unsigned int wcaps = get_wcaps(codec, nid);
unsigned int parm;
parm = 0;
return parm & AC_DEV_LIST_LEN_MASK;
}
+EXPORT_SYMBOL_GPL(snd_hda_get_num_devices);
/**
* snd_hda_get_devices - copy device list without cache
unsigned int parm;
int i, dev_len, devices;
- parm = get_num_devices(codec, nid);
+ parm = snd_hda_get_num_devices(codec, nid);
if (!parm) /* not multi-stream capable */
return 0;
return devices;
}
+/**
+ * snd_hda_get_dev_select - get device entry select on the pin
+ * @codec: the HDA codec
+ * @nid: NID of the pin to get device entry select
+ *
+ * Get the devcie entry select on the pin. Return the device entry
+ * id selected on the pin. Return 0 means the first device entry
+ * is selected or MST is not supported.
+ */
+int snd_hda_get_dev_select(struct hda_codec *codec, hda_nid_t nid)
+{
+ /* not support dp_mst will always return 0, using first dev_entry */
+ if (!codec->dp_mst)
+ return 0;
+
+ return snd_hda_codec_read(codec, nid, 0, AC_VERB_GET_DEVICE_SEL, 0);
+}
+EXPORT_SYMBOL_GPL(snd_hda_get_dev_select);
+
+/**
+ * snd_hda_set_dev_select - set device entry select on the pin
+ * @codec: the HDA codec
+ * @nid: NID of the pin to set device entry select
+ * @dev_id: device entry id to be set
+ *
+ * Set the device entry select on the pin nid.
+ */
+int snd_hda_set_dev_select(struct hda_codec *codec, hda_nid_t nid, int dev_id)
+{
+ int ret, num_devices;
+
+ /* not support dp_mst will always return 0, using first dev_entry */
+ if (!codec->dp_mst)
+ return 0;
+
+ /* AC_PAR_DEVLIST_LEN is 0 based. */
+ num_devices = snd_hda_get_num_devices(codec, nid) + 1;
+ /* If Device List Length is 0 (num_device = 1),
+ * the pin is not multi stream capable.
+ * Do nothing in this case.
+ */
+ if (num_devices == 1)
+ return 0;
+
+ /* Behavior of setting index being equal to or greater than
+ * Device List Length is not predictable
+ */
+ if (num_devices <= dev_id)
+ return -EINVAL;
+
+ ret = snd_hda_codec_write(codec, nid, 0,
+ AC_VERB_SET_DEVICE_SEL, dev_id);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(snd_hda_set_dev_select);
+
/*
* read widget caps for each widget and store in cache
*/
pin->nid = nid;
pin->cfg = snd_hda_codec_read(codec, nid, 0,
AC_VERB_GET_CONFIG_DEFAULT, 0);
+ /*
+ * all device entries are the same widget control so far
+ * fixme: if any codec is different, need fix here
+ */
pin->ctrl = snd_hda_codec_read(codec, nid, 0,
AC_VERB_GET_PIN_WIDGET_CONTROL,
0);
const hda_nid_t *list);
int snd_hda_get_conn_index(struct hda_codec *codec, hda_nid_t mux,
hda_nid_t nid, int recursive);
+unsigned int snd_hda_get_num_devices(struct hda_codec *codec, hda_nid_t nid);
int snd_hda_get_devices(struct hda_codec *codec, hda_nid_t nid,
u8 *dev_list, int max_devices);
+int snd_hda_get_dev_select(struct hda_codec *codec, hda_nid_t nid);
+int snd_hda_set_dev_select(struct hda_codec *codec, hda_nid_t nid, int dev_id);
struct hda_verb {
hda_nid_t nid;
struct hdmi_spec_per_pin {
hda_nid_t pin_nid;
+ int dev_id;
/* pin idx, different device entries on the same pin use the same idx */
int pin_nid_idx;
int num_mux_nids;
struct snd_array cvts; /* struct hdmi_spec_per_cvt */
hda_nid_t cvt_nids[4]; /* only for haswell fix */
+ /*
+ * num_pins is the number of virtual pins
+ * for example, there are 3 pins, and each pin
+ * has 4 device entries, then the num_pins is 12
+ */
int num_pins;
+ /*
+ * num_nids is the number of real pins
+ * In the above example, num_nids is 3
+ */
+ int num_nids;
+ /*
+ * dev_num is the number of device entries
+ * on each pin.
+ * In the above example, dev_num is 4
+ */
+ int dev_num;
struct snd_array pins; /* struct hdmi_spec_per_pin */
struct hdmi_pcm pcm_rec[16];
struct mutex pcm_lock;
/* obtain hda_pcm object assigned to idx */
#define get_pcm_rec(spec, idx) (get_hdmi_pcm(spec, idx)->pcm)
-static int pin_nid_to_pin_index(struct hda_codec *codec, hda_nid_t pin_nid)
+static int pin_id_to_pin_index(struct hda_codec *codec,
+ hda_nid_t pin_nid, int dev_id)
{
struct hdmi_spec *spec = codec->spec;
int pin_idx;
+ struct hdmi_spec_per_pin *per_pin;
+
+ /*
+ * (dev_id == -1) means it is NON-MST pin
+ * return the first virtual pin on this port
+ */
+ if (dev_id == -1)
+ dev_id = 0;
- for (pin_idx = 0; pin_idx < spec->num_pins; pin_idx++)
- if (get_pin(spec, pin_idx)->pin_nid == pin_nid)
+ for (pin_idx = 0; pin_idx < spec->num_pins; pin_idx++) {
+ per_pin = get_pin(spec, pin_idx);
+ if ((per_pin->pin_nid == pin_nid) &&
+ (per_pin->dev_id == dev_id))
return pin_idx;
+ }
codec_warn(codec, "HDMI: pin nid %d not registered\n", pin_nid);
return -EINVAL;
static bool hdmi_present_sense(struct hdmi_spec_per_pin *per_pin, int repoll);
-static void check_presence_and_report(struct hda_codec *codec, hda_nid_t nid)
+static void check_presence_and_report(struct hda_codec *codec, hda_nid_t nid,
+ int dev_id)
{
struct hdmi_spec *spec = codec->spec;
- int pin_idx = pin_nid_to_pin_index(codec, nid);
+ int pin_idx = pin_id_to_pin_index(codec, nid, dev_id);
if (pin_idx < 0)
return;
static void jack_callback(struct hda_codec *codec,
struct hda_jack_callback *jack)
{
- check_presence_and_report(codec, jack->nid);
+ /* hda_jack don't support DP MST */
+ check_presence_and_report(codec, jack->nid, 0);
}
static void hdmi_intrinsic_event(struct hda_codec *codec, unsigned int res)
struct hda_jack_tbl *jack;
int dev_entry = (res & AC_UNSOL_RES_DE) >> AC_UNSOL_RES_DE_SHIFT;
+ /*
+ * assume DP MST uses dyn_pcm_assign and acomp and
+ * never comes here
+ * if DP MST supports unsol event, below code need
+ * consider dev_entry
+ */
jack = snd_hda_jack_tbl_get_from_tag(codec, tag);
if (!jack)
return;
codec->addr, jack->nid, dev_entry, !!(res & AC_UNSOL_RES_IA),
!!(res & AC_UNSOL_RES_PD), !!(res & AC_UNSOL_RES_ELDV));
- check_presence_and_report(codec, jack->nid);
+ /* hda_jack don't support DP MST */
+ check_presence_and_report(codec, jack->nid, 0);
}
static void hdmi_non_intrinsic_event(struct hda_codec *codec, unsigned int res)
* by any other pins.
*/
static void intel_not_share_assigned_cvt(struct hda_codec *codec,
- hda_nid_t pin_nid, int mux_idx)
+ hda_nid_t pin_nid,
+ int dev_id, int mux_idx)
{
struct hdmi_spec *spec = codec->spec;
hda_nid_t nid;
int cvt_idx, curr;
struct hdmi_spec_per_cvt *per_cvt;
+ struct hdmi_spec_per_pin *per_pin;
+ int pin_idx;
+
+ /* configure the pins connections */
+ for (pin_idx = 0; pin_idx < spec->num_pins; pin_idx++) {
+ int dev_id_saved;
+ int dev_num;
- /* configure all pins, including "no physical connection" ones */
- for_each_hda_codec_node(nid, codec) {
- unsigned int wid_caps = get_wcaps(codec, nid);
- unsigned int wid_type = get_wcaps_type(wid_caps);
+ per_pin = get_pin(spec, pin_idx);
+ /*
+ * pin not connected to monitor
+ * no need to operate on it
+ */
+ if (!per_pin->pcm)
+ continue;
- if (wid_type != AC_WID_PIN)
+ if ((per_pin->pin_nid == pin_nid) &&
+ (per_pin->dev_id == dev_id))
continue;
- if (nid == pin_nid)
+ /*
+ * if per_pin->dev_id >= dev_num,
+ * snd_hda_get_dev_select() will fail,
+ * and the following operation is unpredictable.
+ * So skip this situation.
+ */
+ dev_num = snd_hda_get_num_devices(codec, per_pin->pin_nid) + 1;
+ if (per_pin->dev_id >= dev_num)
continue;
+ nid = per_pin->pin_nid;
+
+ /*
+ * Calling this function should not impact
+ * on the device entry selection
+ * So let's save the dev id for each pin,
+ * and restore it when return
+ */
+ dev_id_saved = snd_hda_get_dev_select(codec, nid);
+ snd_hda_set_dev_select(codec, nid, per_pin->dev_id);
curr = snd_hda_codec_read(codec, nid, 0,
AC_VERB_GET_CONNECT_SEL, 0);
- if (curr != mux_idx)
+ if (curr != mux_idx) {
+ snd_hda_set_dev_select(codec, nid, dev_id_saved);
continue;
+ }
+
/* choose an unassigned converter. The conveters in the
* connection list are in the same order as in the codec.
break;
}
}
+ snd_hda_set_dev_select(codec, nid, dev_id_saved);
}
}
/* A wrapper of intel_not_share_asigned_cvt() */
static void intel_not_share_assigned_cvt_nid(struct hda_codec *codec,
- hda_nid_t pin_nid, hda_nid_t cvt_nid)
+ hda_nid_t pin_nid, int dev_id, hda_nid_t cvt_nid)
{
int mux_idx;
struct hdmi_spec *spec = codec->spec;
*/
mux_idx = intel_cvt_id_to_mux_idx(spec, cvt_nid);
if (mux_idx >= 0)
- intel_not_share_assigned_cvt(codec, pin_nid, mux_idx);
+ intel_not_share_assigned_cvt(codec, pin_nid, dev_id, mux_idx);
}
/* skeleton caller of pin_cvt_fixup ops */
per_pin->cvt_nid = per_cvt->cvt_nid;
hinfo->nid = per_cvt->cvt_nid;
+ snd_hda_set_dev_select(codec, per_pin->pin_nid, per_pin->dev_id);
snd_hda_codec_write_cache(codec, per_pin->pin_nid, 0,
AC_VERB_SET_CONNECT_SEL,
per_pin->mux_idx);
return -EINVAL;
}
+ /* all the device entries on the same pin have the same conn list */
per_pin->num_mux_nids = snd_hda_get_connections(codec, pin_nid,
per_pin->mux_nids,
HDA_MAX_CONNECTIONS);
return per_pin->pin_nid_idx;
/* have a second try; check the "reserved area" over num_pins */
- for (i = spec->num_pins; i < spec->pcm_used; i++) {
+ for (i = spec->num_nids; i < spec->pcm_used; i++) {
if (!test_bit(i, &spec->pcm_bitmap))
return i;
}
/* the last try; check the empty slots in pins */
- for (i = 0; i < spec->num_pins; i++) {
+ for (i = 0; i < spec->num_nids; i++) {
if (!test_bit(i, &spec->pcm_bitmap))
return i;
}
per_pin->cvt_nid = hinfo->nid;
mux_idx = hdmi_get_pin_cvt_mux(spec, per_pin, hinfo->nid);
- if (mux_idx < per_pin->num_mux_nids)
+ if (mux_idx < per_pin->num_mux_nids) {
+ snd_hda_set_dev_select(codec, per_pin->pin_nid,
+ per_pin->dev_id);
snd_hda_codec_write_cache(codec, per_pin->pin_nid, 0,
AC_VERB_SET_CONNECT_SEL,
mux_idx);
+ }
snd_hda_spdif_ctls_assign(codec, per_pin->pcm_idx, hinfo->nid);
non_pcm = check_non_pcm_per_cvt(codec, hinfo->nid);
if (per_pin->pcm_idx >= 0 && spec->dyn_pcm_assign)
jack = spec->pcm_rec[per_pin->pcm_idx].jack;
else if (!spec->dyn_pcm_assign) {
+ /*
+ * jack tbl doesn't support DP MST
+ * DP MST will use dyn_pcm_assign,
+ * so DP MST will never come here
+ */
jack_tbl = snd_hda_jack_tbl_get(codec, per_pin->pin_nid);
if (jack_tbl)
jack = jack_tbl->jack;
mutex_lock(&per_pin->lock);
eld->monitor_present = false;
- size = snd_hdac_acomp_get_eld(&codec->core, per_pin->pin_nid, -1,
- &eld->monitor_present, eld->eld_buffer,
- ELD_MAX_SIZE);
+ size = snd_hdac_acomp_get_eld(&codec->core, per_pin->pin_nid,
+ per_pin->dev_id, &eld->monitor_present,
+ eld->eld_buffer, ELD_MAX_SIZE);
if (size > 0) {
size = min(size, ELD_MAX_SIZE);
if (snd_hdmi_parse_eld(codec, &eld->info,
int pin_idx;
struct hdmi_spec_per_pin *per_pin;
int err;
+ int dev_num, i;
caps = snd_hda_query_pin_caps(codec, pin_nid);
if (!(caps & (AC_PINCAP_HDMI | AC_PINCAP_DP)))
return 0;
+ /*
+ * For DP MST audio, Configuration Default is the same for
+ * all device entries on the same pin
+ */
config = snd_hda_codec_get_pincfg(codec, pin_nid);
if (get_defcfg_connect(config) == AC_JACK_PORT_NONE)
return 0;
- if (is_haswell_plus(codec))
- intel_haswell_fixup_connect_list(codec, pin_nid);
-
- pin_idx = spec->num_pins;
- per_pin = snd_array_new(&spec->pins);
- if (!per_pin)
- return -ENOMEM;
-
- per_pin->pin_nid = pin_nid;
- per_pin->non_pcm = false;
- if (spec->dyn_pcm_assign)
- per_pin->pcm_idx = -1;
- else {
- per_pin->pcm = get_hdmi_pcm(spec, pin_idx);
- per_pin->pcm_idx = pin_idx;
+ /*
+ * To simplify the implementation, malloc all
+ * the virtual pins in the initialization statically
+ */
+ if (is_haswell_plus(codec)) {
+ /*
+ * On Intel platforms, device entries number is
+ * changed dynamically. If there is a DP MST
+ * hub connected, the device entries number is 3.
+ * Otherwise, it is 1.
+ * Here we manually set dev_num to 3, so that
+ * we can initialize all the device entries when
+ * bootup statically.
+ */
+ dev_num = 3;
+ spec->dev_num = 3;
+ } else if (spec->dyn_pcm_assign && codec->dp_mst) {
+ dev_num = snd_hda_get_num_devices(codec, pin_nid) + 1;
+ /*
+ * spec->dev_num is the maxinum number of device entries
+ * among all the pins
+ */
+ spec->dev_num = (spec->dev_num > dev_num) ?
+ spec->dev_num : dev_num;
+ } else {
+ /*
+ * If the platform doesn't support DP MST,
+ * manually set dev_num to 1. This means
+ * the pin has only one device entry.
+ */
+ dev_num = 1;
+ spec->dev_num = 1;
}
- per_pin->pin_nid_idx = pin_idx;
- err = hdmi_read_pin_conn(codec, pin_idx);
- if (err < 0)
- return err;
+ for (i = 0; i < dev_num; i++) {
+ pin_idx = spec->num_pins;
+ per_pin = snd_array_new(&spec->pins);
- spec->num_pins++;
+ if (!per_pin)
+ return -ENOMEM;
+
+ if (spec->dyn_pcm_assign) {
+ per_pin->pcm = NULL;
+ per_pin->pcm_idx = -1;
+ } else {
+ per_pin->pcm = get_hdmi_pcm(spec, pin_idx);
+ per_pin->pcm_idx = pin_idx;
+ }
+ per_pin->pin_nid = pin_nid;
+ per_pin->pin_nid_idx = spec->num_nids;
+ per_pin->dev_id = i;
+ per_pin->non_pcm = false;
+ snd_hda_set_dev_select(codec, pin_nid, i);
+ if (is_haswell_plus(codec))
+ intel_haswell_fixup_connect_list(codec, pin_nid);
+ err = hdmi_read_pin_conn(codec, pin_idx);
+ if (err < 0)
+ return err;
+ spec->num_pins++;
+ }
+ spec->num_nids++;
return 0;
}
/* Call sync_audio_rate to set the N/CTS/M manually if necessary */
/* Todo: add DP1.2 MST audio support later */
if (codec_has_acomp(codec))
- snd_hdac_sync_audio_rate(&codec->core, pin_nid, -1,
+ snd_hdac_sync_audio_rate(&codec->core, pin_nid, per_pin->dev_id,
runtime->rate);
non_pcm = check_non_pcm_per_cvt(codec, cvt_nid);
pinctl | PIN_OUT);
}
+ /* snd_hda_set_dev_select() has been called before */
err = spec->ops.setup_stream(codec, cvt_nid, pin_nid,
stream_tag, format);
mutex_unlock(&spec->pcm_lock);
static int generic_hdmi_build_pcms(struct hda_codec *codec)
{
struct hdmi_spec *spec = codec->spec;
- int pin_idx;
+ int idx;
- for (pin_idx = 0; pin_idx < spec->num_pins; pin_idx++) {
+ /*
+ * for non-mst mode, pcm number is the same as before
+ * for DP MST mode, pcm number is (nid number + dev_num - 1)
+ * dev_num is the device entry number in a pin
+ *
+ */
+ for (idx = 0; idx < spec->num_nids + spec->dev_num - 1; idx++) {
struct hda_pcm *info;
struct hda_pcm_stream *pstr;
- info = snd_hda_codec_pcm_new(codec, "HDMI %d", pin_idx);
+ info = snd_hda_codec_pcm_new(codec, "HDMI %d", idx);
if (!info)
return -ENOMEM;
- spec->pcm_rec[pin_idx].pcm = info;
+ spec->pcm_rec[idx].pcm = info;
spec->pcm_used++;
info->pcm_type = HDA_PCM_TYPE_HDMI;
info->own_chmap = true;
pstr = &info->stream[SNDRV_PCM_STREAM_PLAYBACK];
pstr->substreams = 1;
pstr->ops = generic_ops;
+ /* pcm number is less than 16 */
+ if (spec->pcm_used >= 16)
+ break;
/* other pstr fields are set in open */
}
for (pin_idx = 0; pin_idx < spec->num_pins; pin_idx++) {
struct hdmi_spec_per_pin *per_pin = get_pin(spec, pin_idx);
hda_nid_t pin_nid = per_pin->pin_nid;
+ int dev_id = per_pin->dev_id;
+ snd_hda_set_dev_select(codec, pin_nid, dev_id);
hdmi_init_pin(codec, pin_nid);
if (!codec_has_acomp(codec))
snd_hda_jack_detect_enable_callback(codec, pin_nid,
return -ENOMEM;
spec->ops = generic_standard_hdmi_ops;
+ spec->dev_num = 1; /* initialize to 1 */
mutex_init(&spec->pcm_lock);
snd_hdac_register_chmap_ops(&codec->core, &spec->chmap);
{
struct hda_codec *codec = audio_ptr;
int pin_nid;
+ int dev_id = pipe;
/* we assume only from port-B to port-D */
if (port < 1 || port > 3)
return;
snd_hdac_i915_set_bclk(&codec->bus->core);
- check_presence_and_report(codec, pin_nid);
+ check_presence_and_report(codec, pin_nid, dev_id);
}
/* register i915 component pin_eld_notify callback */
hda_nid_t cvt_nid)
{
if (per_pin) {
+ snd_hda_set_dev_select(codec, per_pin->pin_nid,
+ per_pin->dev_id);
intel_verify_pin_cvt_connect(codec, per_pin);
intel_not_share_assigned_cvt(codec, per_pin->pin_nid,
- per_pin->mux_idx);
+ per_pin->dev_id, per_pin->mux_idx);
} else {
- intel_not_share_assigned_cvt_nid(codec, 0, cvt_nid);
+ intel_not_share_assigned_cvt_nid(codec, 0, 0, cvt_nid);
}
}
if (err < 0)
return err;
spec = codec->spec;
+ codec->dp_mst = true;
+ spec->dyn_pcm_assign = true;
intel_haswell_enable_all_pins(codec, true);
intel_haswell_fixup_enable_dp12(codec);
codec->core.link_power_control = 1;
codec->patch_ops.set_power_state = haswell_set_power_state;
- codec->dp_mst = true;
codec->depop_delay = 0;
codec->auto_runtime_pm = 1;
nau8825_xtalk_backup(nau8825);
/* Config IIS as master to output signal by codec */
regmap_update_bits(nau8825->regmap, NAU8825_REG_I2S_PCM_CTRL2,
- NAU8825_I2S_MS_MASK | NAU8825_I2S_DRV_MASK |
+ NAU8825_I2S_MS_MASK | NAU8825_I2S_LRC_DIV_MASK |
NAU8825_I2S_BLK_DIV_MASK, NAU8825_I2S_MS_MASTER |
- (0x2 << NAU8825_I2S_DRV_SFT) | 0x1);
+ (0x2 << NAU8825_I2S_LRC_DIV_SFT) | 0x1);
/* Ramp up headphone volume to 0dB to get better performance and
* avoid pop noise in headphone.
*/
NAU8825_IRQ_RMS_EN, NAU8825_IRQ_RMS_EN);
/* Recover default value for IIS */
regmap_update_bits(nau8825->regmap, NAU8825_REG_I2S_PCM_CTRL2,
- NAU8825_I2S_MS_MASK | NAU8825_I2S_DRV_MASK |
+ NAU8825_I2S_MS_MASK | NAU8825_I2S_LRC_DIV_MASK |
NAU8825_I2S_BLK_DIV_MASK, NAU8825_I2S_MS_SLAVE);
/* Restore value of specific register for cross talk */
nau8825_xtalk_restore(nau8825);
NAU8825_FLL_INTEGER_MASK, fll_param->fll_int);
/* FLL pre-scaler */
regmap_update_bits(nau8825->regmap, NAU8825_REG_FLL4,
- NAU8825_FLL_REF_DIV_MASK, fll_param->clk_ref_div);
+ NAU8825_FLL_REF_DIV_MASK,
+ fll_param->clk_ref_div << NAU8825_FLL_REF_DIV_SFT);
/* select divided VCO input */
regmap_update_bits(nau8825->regmap, NAU8825_REG_FLL5,
NAU8825_FLL_CLK_SW_MASK, NAU8825_FLL_CLK_SW_REF);
#define NAU8825_FLL_CLK_SRC_FS (0x3 << NAU8825_FLL_CLK_SRC_SFT)
/* FLL4 (0x07) */
-#define NAU8825_FLL_REF_DIV_MASK (0x3 << 10)
+#define NAU8825_FLL_REF_DIV_SFT 10
+#define NAU8825_FLL_REF_DIV_MASK (0x3 << NAU8825_FLL_REF_DIV_SFT)
/* FLL5 (0x08) */
#define NAU8825_FLL_PDB_DAC_EN (0x1 << 15)
/* I2S_PCM_CTRL2 (0x1d) */
#define NAU8825_I2S_TRISTATE (1 << 15) /* 0 - normal mode, 1 - Hi-Z output */
-#define NAU8825_I2S_DRV_SFT 12
-#define NAU8825_I2S_DRV_MASK (0x3 << NAU8825_I2S_DRV_SFT)
+#define NAU8825_I2S_LRC_DIV_SFT 12
+#define NAU8825_I2S_LRC_DIV_MASK (0x3 << NAU8825_I2S_LRC_DIV_SFT)
#define NAU8825_I2S_MS_SFT 3
#define NAU8825_I2S_MS_MASK (1 << NAU8825_I2S_MS_SFT)
#define NAU8825_I2S_MS_MASTER (1 << NAU8825_I2S_MS_SFT)
}
}
+ regmap_update_bits(rt5645->regmap, RT5645_ADDA_CLK1,
+ RT5645_I2S_PD1_MASK, RT5645_I2S_PD1_2);
+
if (rt5645->pdata.jd_invert) {
regmap_update_bits(rt5645->regmap, RT5645_IRQ_CTRL2,
RT5645_JD_1_1_MASK, RT5645_JD_1_1_INV);
{ 108, 0x00 }, { 109, 0x00 },
};
+static bool aic3x_volatile_reg(struct device *dev, unsigned int reg)
+{
+ switch (reg) {
+ case AIC3X_RESET:
+ return true;
+ default:
+ return false;
+ }
+}
+
static const struct regmap_config aic3x_regmap = {
.reg_bits = 8,
.val_bits = 8,
.max_register = DAC_ICC_ADJ,
.reg_defaults = aic3x_reg,
.num_reg_defaults = ARRAY_SIZE(aic3x_reg),
+
+ .volatile_reg = aic3x_volatile_reg,
+
.cache_type = REGCACHE_RBTREE,
};
const struct wmfw_region *region;
const struct wm_adsp_region *mem;
const char *region_name;
- char *file, *text;
+ char *file, *text = NULL;
struct wm_adsp_buf *buf;
unsigned int reg;
int regions = 0;
regions, le32_to_cpu(region->len), offset,
region_name);
+ if ((pos + le32_to_cpu(region->len) + sizeof(*region)) >
+ firmware->size) {
+ adsp_err(dsp,
+ "%s.%d: %s region len %d bytes exceeds file length %zu\n",
+ file, regions, region_name,
+ le32_to_cpu(region->len), firmware->size);
+ ret = -EINVAL;
+ goto out_fw;
+ }
+
if (text) {
memcpy(text, region->data, le32_to_cpu(region->len));
adsp_info(dsp, "%s: %s\n", file, text);
kfree(text);
+ text = NULL;
}
if (reg) {
regmap_async_complete(regmap);
wm_adsp_buf_free(&buf_list);
release_firmware(firmware);
+ kfree(text);
out:
kfree(file);
}
if (reg) {
+ if ((pos + le32_to_cpu(blk->len) + sizeof(*blk)) >
+ firmware->size) {
+ adsp_err(dsp,
+ "%s.%d: %s region len %d bytes exceeds file length %zu\n",
+ file, blocks, region_name,
+ le32_to_cpu(blk->len),
+ firmware->size);
+ ret = -EINVAL;
+ goto out_fw;
+ }
+
buf = wm_adsp_buf_alloc(blk->data,
le32_to_cpu(blk->len),
&buf_list);
}
if (!pdata) {
- ret = devm_snd_dmaengine_pcm_register(&pdev->dev, NULL, 0);
- if (ret == -EPROBE_DEFER) {
- dev_err(&pdev->dev,
- "failed to register PCM, deferring probe\n");
- return ret;
- } else if (ret) {
- dev_err(&pdev->dev,
- "Could not register DMA PCM: %d\n"
- "falling back to PIO mode\n", ret);
+ if (irq >= 0) {
ret = dw_pcm_register(pdev);
- if (ret) {
- dev_err(&pdev->dev,
- "Could not register PIO PCM: %d\n",
+ dev->use_pio = true;
+ } else {
+ ret = devm_snd_dmaengine_pcm_register(&pdev->dev, NULL,
+ 0);
+ dev->use_pio = false;
+ }
+
+ if (ret) {
+ dev_err(&pdev->dev, "could not register pcm: %d\n",
ret);
- goto err_clk_disable;
- }
+ goto err_clk_disable;
}
}
* @dbg_stats: Debugging statistics
*
* @soc: SoC specific data
+ *
+ * @fifo_watermark: the FIFO watermark setting. Notifies DMA when
+ * there are @fifo_watermark or fewer words in TX fifo or
+ * @fifo_watermark or more empty words in RX fifo.
+ * @dma_maxburst: max number of words to transfer in one go. So far,
+ * this is always the same as fifo_watermark.
*/
struct fsl_ssi_private {
struct regmap *regs;
const struct fsl_ssi_soc_data *soc;
struct device *dev;
+
+ u32 fifo_watermark;
+ u32 dma_maxburst;
};
/*
regmap_write(regs, CCSR_SSI_SRCR, srcr);
regmap_write(regs, CCSR_SSI_SCR, scr);
- /*
- * Set the watermark for transmit FIFI 0 and receive FIFO 0. We don't
- * use FIFO 1. We program the transmit water to signal a DMA transfer
- * if there are only two (or fewer) elements left in the FIFO. Two
- * elements equals one frame (left channel, right channel). This value,
- * however, depends on the depth of the transmit buffer.
- *
- * We set the watermark on the same level as the DMA burstsize. For
- * fiq it is probably better to use the biggest possible watermark
- * size.
- */
- if (ssi_private->use_dma)
- wm = ssi_private->fifo_depth - 2;
- else
- wm = ssi_private->fifo_depth;
+ wm = ssi_private->fifo_watermark;
regmap_write(regs, CCSR_SSI_SFCSR,
CCSR_SSI_SFCSR_TFWM0(wm) | CCSR_SSI_SFCSR_RFWM0(wm) |
dev_dbg(&pdev->dev, "could not get baud clock: %ld\n",
PTR_ERR(ssi_private->baudclk));
- /*
- * We have burstsize be "fifo_depth - 2" to match the SSI
- * watermark setting in fsl_ssi_startup().
- */
- ssi_private->dma_params_tx.maxburst = ssi_private->fifo_depth - 2;
- ssi_private->dma_params_rx.maxburst = ssi_private->fifo_depth - 2;
+ ssi_private->dma_params_tx.maxburst = ssi_private->dma_maxburst;
+ ssi_private->dma_params_rx.maxburst = ssi_private->dma_maxburst;
ssi_private->dma_params_tx.addr = ssi_private->ssi_phys + CCSR_SSI_STX0;
ssi_private->dma_params_rx.addr = ssi_private->ssi_phys + CCSR_SSI_SRX0;
/* Older 8610 DTs didn't have the fifo-depth property */
ssi_private->fifo_depth = 8;
+ /*
+ * Set the watermark for transmit FIFO 0 and receive FIFO 0. We don't
+ * use FIFO 1 but set the watermark appropriately nontheless.
+ * We program the transmit water to signal a DMA transfer
+ * if there are N elements left in the FIFO. For chips with 15-deep
+ * FIFOs, set watermark to 8. This allows the SSI to operate at a
+ * high data rate without channel slipping. Behavior is unchanged
+ * for the older chips with a fifo depth of only 8. A value of 4
+ * might be appropriate for the older chips, but is left at
+ * fifo_depth-2 until sombody has a chance to test.
+ *
+ * We set the watermark on the same level as the DMA burstsize. For
+ * fiq it is probably better to use the biggest possible watermark
+ * size.
+ */
+ switch (ssi_private->fifo_depth) {
+ case 15:
+ /*
+ * 2 samples is not enough when running at high data
+ * rates (like 48kHz @ 16 bits/channel, 16 channels)
+ * 8 seems to split things evenly and leave enough time
+ * for the DMA to fill the FIFO before it's over/under
+ * run.
+ */
+ ssi_private->fifo_watermark = 8;
+ ssi_private->dma_maxburst = 8;
+ break;
+ case 8:
+ default:
+ /*
+ * maintain old behavior for older chips.
+ * Keeping it the same because I don't have an older
+ * board to test with.
+ * I suspect this could be changed to be something to
+ * leave some more space in the fifo.
+ */
+ ssi_private->fifo_watermark = ssi_private->fifo_depth - 2;
+ ssi_private->dma_maxburst = ssi_private->fifo_depth - 2;
+ break;
+ }
+
dev_set_drvdata(&pdev->dev, ssi_private);
if (ssi_private->soc->imx) {
* for Jack detection and button press
*/
ret = snd_soc_dai_set_sysclk(codec_dai, RT5640_SCLK_S_RCCLK,
- 0,
+ 48000 * 512,
SND_SOC_CLOCK_IN);
if (!ret) {
if ((byt_rt5640_quirk & BYT_RT5640_MCLK_EN) && priv->mclk)
if ((byt_rt5640_quirk & BYT_RT5640_MCLK_EN) && (is_valleyview())) {
priv->mclk = devm_clk_get(&pdev->dev, "pmc_plt_clk_3");
if (IS_ERR(priv->mclk)) {
+ ret_val = PTR_ERR(priv->mclk);
+
dev_err(&pdev->dev,
- "Failed to get MCLK from pmc_plt_clk_3: %ld\n",
- PTR_ERR(priv->mclk));
- return PTR_ERR(priv->mclk);
+ "Failed to get MCLK from pmc_plt_clk_3: %d\n",
+ ret_val);
+
+ /*
+ * Fall back to bit clock usage for -ENOENT (clock not
+ * available likely due to missing dependencies), bail
+ * for all other errors, including -EPROBE_DEFER
+ */
+ if (ret_val != -ENOENT)
+ return ret_val;
+ byt_rt5640_quirk &= ~BYT_RT5640_MCLK_EN;
}
}
snd_pcm_set_sync(substream);
mconfig = skl_tplg_fe_get_cpr_module(dai, substream->stream);
+ if (!mconfig)
+ return -EINVAL;
+
skl_tplg_d0i3_get(skl, mconfig->d0i3_caps);
return 0;
void skl_sst_dsp_cleanup(struct device *dev, struct skl_sst *ctx)
{
+
+ if (ctx->dsp->fw)
+ release_firmware(ctx->dsp->fw);
skl_clear_module_table(ctx->dsp);
skl_freeup_uuid_list(ctx);
skl_ipc_free(&ctx->ipc);
return -ENOMEM;
ret = snd_ctl_add(card, kctrl);
- if (ret < 0) {
- snd_ctl_free_one(kctrl);
+ if (ret < 0)
return ret;
- }
cfg->update = update;
cfg->card = card;
component->init = aux_dev->init;
component->auxiliary = 1;
+ list_add(&component->card_aux_list, &card->aux_comp_list);
return 0;
static int soc_probe_aux_devices(struct snd_soc_card *card)
{
- struct snd_soc_component *comp;
+ struct snd_soc_component *comp, *tmp;
int order;
int ret;
for (order = SND_SOC_COMP_ORDER_FIRST; order <= SND_SOC_COMP_ORDER_LAST;
order++) {
- list_for_each_entry(comp, &card->component_dev_list, card_list) {
- if (!comp->auxiliary)
- continue;
-
+ list_for_each_entry_safe(comp, tmp, &card->aux_comp_list,
+ card_aux_list) {
if (comp->driver->probe_order == order) {
ret = soc_probe_component(card, comp);
if (ret < 0) {
comp->name, ret);
return ret;
}
+ list_del(&comp->card_aux_list);
}
}
}
break;
case SNDRV_PCM_TRIGGER_STOP:
case SNDRV_PCM_TRIGGER_SUSPEND:
- case SNDRV_PCM_TRIGGER_PAUSE_PUSH:
fe->dpcm[stream].state = SND_SOC_DPCM_STATE_STOP;
break;
+ case SNDRV_PCM_TRIGGER_PAUSE_PUSH:
+ fe->dpcm[stream].state = SND_SOC_DPCM_STATE_PAUSED;
+ break;
}
out:
== SND_SOC_TPLG_TYPE_MIXER)
kfree(kcontrol->tlv.p);
- snd_ctl_remove(card, kcontrol);
-
/* Private value is used as struct soc_mixer_control
* for volume mixers or soc_bytes_ext for bytes
* controls.
*/
kfree((void *)kcontrol->private_value);
+ snd_ctl_remove(card, kcontrol);
}
kfree(w->kcontrol_news);
}
case USB_ID(0x045E, 0x076F): /* MS Lifecam HD-6000 */
case USB_ID(0x045E, 0x0772): /* MS Lifecam Studio */
case USB_ID(0x045E, 0x0779): /* MS Lifecam HD-3000 */
+ case USB_ID(0x047F, 0x02F7): /* Plantronics BT-600 */
case USB_ID(0x047F, 0x0415): /* Plantronics BT-300 */
case USB_ID(0x047F, 0xAA05): /* Plantronics DA45 */
case USB_ID(0x04D8, 0xFEEA): /* Benchmark DAC1 Pre */
else
err = get_arg(p, opt, flags, (const char **)opt->value);
+ if (opt->set)
+ *(bool *)opt->set = true;
+
/* PARSE_OPT_NOEMPTY: Allow NULL but disallow empty string. */
if (opt->flags & PARSE_OPT_NOEMPTY) {
const char *val = *(const char **)opt->value;
{ .type = OPTION_STRING, .short_name = (s), .long_name = (l), \
.value = check_vtype(v, const char **), (a), .help = (h), \
.flags = PARSE_OPT_OPTARG, .defval = (intptr_t)(d) }
+#define OPT_STRING_OPTARG_SET(s, l, v, os, a, h, d) \
+ { .type = OPTION_STRING, .short_name = (s), .long_name = (l), \
+ .value = check_vtype(v, const char **), (a), .help = (h), \
+ .flags = PARSE_OPT_OPTARG, .defval = (intptr_t)(d), \
+ .set = check_vtype(os, bool *)}
#define OPT_STRING_NOEMPTY(s, l, v, a, h) { .type = OPTION_STRING, .short_name = (s), .long_name = (l), .value = check_vtype(v, const char **), (a), .help = (h), .flags = PARSE_OPT_NOEMPTY}
#define OPT_DATE(s, l, v, h) \
{ .type = OPTION_CALLBACK, .short_name = (s), .long_name = (l), .value = (v), .argh = "time", .help = (h), .callback = parse_opt_approxidate_cb }
trace_seq_printf(s, "%lld ", val);
if (pevent_get_field_val(s, event, "prev_prio", record, &val, 0) == 0)
- trace_seq_printf(s, "[%lld] ", val);
+ trace_seq_printf(s, "[%d] ", (int) val);
if (pevent_get_field_val(s, event, "prev_state", record, &val, 0) == 0)
write_state(s, val);
trace_seq_printf(s, "%lld", val);
if (pevent_get_field_val(s, event, "next_prio", record, &val, 0) == 0)
- trace_seq_printf(s, " [%lld]", val);
+ trace_seq_printf(s, " [%d]", (int) val);
return 0;
}
particular perf.data snapshot should be kept or not.
Implies --timestamp-filename, --no-buildid and --no-buildid-cache.
+The reason for the latter two is to reduce the data file switching
+overhead. You can still switch them on with:
+
+ --switch-output --no-no-buildid --no-no-buildid-cache
--dry-run::
Parse options then exit. --dry-run can be used to detect errors in cmdline
$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/attr'; \
$(INSTALL) tests/attr/* '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/attr'
-install-bin: install-tools install-tests
+install-bin: install-tools install-tests install-traceevent-plugins
-install: install-bin try-install-man install-traceevent-plugins
+install: install-bin try-install-man
install-python_ext:
$(PYTHON_WORD) util/setup.py --quiet install --root='/$(DESTDIR_SQ)'
{ "__GFP_RECLAIM", "R" },
{ "__GFP_DIRECT_RECLAIM", "DR" },
{ "__GFP_KSWAPD_RECLAIM", "KR" },
- { "__GFP_OTHER_NODE", "ON" },
};
static size_t max_gfp_len;
* perf_evlist__prepare_workload, etc instead of fork+exec'in 'perf record',
* using pipes, etc.
*/
-struct option __record_options[] = {
+static struct option __record_options[] = {
OPT_CALLBACK('e', "event", &record.evlist, "event",
"event selector. use 'perf list' to list available events",
parse_events_option),
* overhead. Still generate buildid if they are required
* explicitly using
*
- * perf record --signal-trigger --no-no-buildid \
+ * perf record --switch-output --no-no-buildid \
* --no-no-buildid-cache
*
* Following code equals to:
u64 skipped_samples;
const char *time_str;
struct perf_time_interval ptime;
+ struct perf_time_interval hist_time;
};
/* per thread run time data */
timehist_print_sample(sched, sample, &al, thread, t);
out:
+ if (sched->hist_time.start == 0 && t >= ptime->start)
+ sched->hist_time.start = t;
+ if (ptime->end == 0 || t <= ptime->end)
+ sched->hist_time.end = t;
+
if (tr) {
/* time of this sched_switch event becomes last time task seen */
tr->last_time = sample->time;
struct thread *t;
struct thread_runtime *r;
int i;
+ u64 hist_time = sched->hist_time.end - sched->hist_time.start;
memset(&totals, 0, sizeof(totals));
totals.sched_count += r->run_stats.n;
printf(" CPU %2d idle for ", i);
print_sched_time(r->total_run_time, 6);
- printf(" msec\n");
+ printf(" msec (%6.2f%%)\n", 100.0 * r->total_run_time / hist_time);
} else
printf(" CPU %2d idle entire time window\n", i);
}
printf("\n"
" Total number of unique tasks: %" PRIu64 "\n"
- "Total number of context switches: %" PRIu64 "\n"
- " Total run time (msec): ",
+ "Total number of context switches: %" PRIu64 "\n",
totals.task_count, totals.sched_count);
+ printf(" Total run time (msec): ");
print_sched_time(totals.total_run_time, 2);
printf("\n");
+
+ printf(" Total scheduling time (msec): ");
+ print_sched_time(hist_time, 2);
+ printf(" (x %d)\n", sched->max_cpu);
}
typedef int (*sched_handler)(struct perf_tool *tool,
/* A file path -- this is an offline module */
if (module && strchr(module, '/'))
- return machine__findnew_module_map(host_machine, 0, module);
+ return dso__new_map(module);
if (!module)
module = "kernel";
if (strncmp(pos->dso->short_name + 1, module,
pos->dso->short_name_len - 2) == 0 &&
module[pos->dso->short_name_len - 2] == '\0') {
+ map__get(pos);
return pos;
}
}
return kernel_get_module_map(target);
}
-static void put_target_map(struct map *map, bool user)
-{
- if (map && user) {
- /* Only the user map needs to be released */
- map__put(map);
- }
-}
-
-
static int convert_exec_to_group(const char *exec, char **result)
{
char *ptr1, *ptr2, *exec_copy;
return true;
}
-/*
- * NOTE:
- * '.gnu.linkonce.this_module' section of kernel module elf directly
- * maps to 'struct module' from linux/module.h. This section contains
- * actual module name which will be used by kernel after loading it.
- * But, we cannot use 'struct module' here since linux/module.h is not
- * exposed to user-space. Offset of 'name' has remained same from long
- * time, so hardcoding it here.
- */
-#ifdef __LP64__
-#define MOD_NAME_OFFSET 24
-#else
-#define MOD_NAME_OFFSET 12
-#endif
-
/*
* @module can be module name of module file path. In case of path,
* inspect elf and find out what is actual module name.
Elf_Data *data;
Elf_Scn *sec;
char *mod_name = NULL;
+ int name_offset;
fd = open(module, O_RDONLY);
if (fd < 0)
if (!data || !data->d_buf)
goto ret_err;
- mod_name = strdup((char *)data->d_buf + MOD_NAME_OFFSET);
+ /*
+ * NOTE:
+ * '.gnu.linkonce.this_module' section of kernel module elf directly
+ * maps to 'struct module' from linux/module.h. This section contains
+ * actual module name which will be used by kernel after loading it.
+ * But, we cannot use 'struct module' here since linux/module.h is not
+ * exposed to user-space. Offset of 'name' has remained same from long
+ * time, so hardcoding it here.
+ */
+ if (ehdr.e_ident[EI_CLASS] == ELFCLASS32)
+ name_offset = 12;
+ else /* expect ELFCLASS64 by default */
+ name_offset = 24;
+
+ mod_name = strdup((char *)data->d_buf + name_offset);
ret_err:
elf_end(elf);
}
out:
- put_target_map(map, uprobes);
+ map__put(map);
return ret;
}
return ret ? : -ENOENT;
}
+/* Adjust symbol name and address */
+static int post_process_probe_trace_point(struct probe_trace_point *tp,
+ struct map *map, unsigned long offs)
+{
+ struct symbol *sym;
+ u64 addr = tp->address + tp->offset - offs;
+
+ sym = map__find_symbol(map, addr);
+ if (!sym)
+ return -ENOENT;
+
+ if (strcmp(sym->name, tp->symbol)) {
+ /* If we have no realname, use symbol for it */
+ if (!tp->realname)
+ tp->realname = tp->symbol;
+ else
+ free(tp->symbol);
+ tp->symbol = strdup(sym->name);
+ if (!tp->symbol)
+ return -ENOMEM;
+ }
+ tp->offset = addr - sym->start;
+ tp->address -= offs;
+
+ return 0;
+}
+
+/*
+ * Rename DWARF symbols to ELF symbols -- gcc sometimes optimizes functions
+ * and generate new symbols with suffixes such as .constprop.N or .isra.N
+ * etc. Since those symbols are not recorded in DWARF, we have to find
+ * correct generated symbols from offline ELF binary.
+ * For online kernel or uprobes we don't need this because those are
+ * rebased on _text, or already a section relative address.
+ */
+static int
+post_process_offline_probe_trace_events(struct probe_trace_event *tevs,
+ int ntevs, const char *pathname)
+{
+ struct map *map;
+ unsigned long stext = 0;
+ int i, ret = 0;
+
+ /* Prepare a map for offline binary */
+ map = dso__new_map(pathname);
+ if (!map || get_text_start_address(pathname, &stext) < 0) {
+ pr_warning("Failed to get ELF symbols for %s\n", pathname);
+ return -EINVAL;
+ }
+
+ for (i = 0; i < ntevs; i++) {
+ ret = post_process_probe_trace_point(&tevs[i].point,
+ map, stext);
+ if (ret < 0)
+ break;
+ }
+ map__put(map);
+
+ return ret;
+}
+
static int add_exec_to_probe_trace_events(struct probe_trace_event *tevs,
int ntevs, const char *exec)
{
return ret;
}
-static int add_module_to_probe_trace_events(struct probe_trace_event *tevs,
- int ntevs, const char *module)
+static int
+post_process_module_probe_trace_events(struct probe_trace_event *tevs,
+ int ntevs, const char *module,
+ struct debuginfo *dinfo)
{
+ Dwarf_Addr text_offs = 0;
int i, ret = 0;
char *mod_name = NULL;
+ struct map *map;
if (!module)
return 0;
- mod_name = find_module_name(module);
+ map = get_target_map(module, false);
+ if (!map || debuginfo__get_text_offset(dinfo, &text_offs, true) < 0) {
+ pr_warning("Failed to get ELF symbols for %s\n", module);
+ return -EINVAL;
+ }
+ mod_name = find_module_name(module);
for (i = 0; i < ntevs; i++) {
+ ret = post_process_probe_trace_point(&tevs[i].point,
+ map, (unsigned long)text_offs);
+ if (ret < 0)
+ break;
tevs[i].point.module =
strdup(mod_name ? mod_name : module);
if (!tevs[i].point.module) {
}
free(mod_name);
+ map__put(map);
+
return ret;
}
/* Skip post process if the target is an offline kernel */
if (symbol_conf.ignore_vmlinux_buildid)
- return 0;
+ return post_process_offline_probe_trace_events(tevs, ntevs,
+ symbol_conf.vmlinux_name);
reloc_sym = kernel_get_ref_reloc_sym();
if (!reloc_sym) {
static int post_process_probe_trace_events(struct perf_probe_event *pev,
struct probe_trace_event *tevs,
int ntevs, const char *module,
- bool uprobe)
+ bool uprobe, struct debuginfo *dinfo)
{
int ret;
ret = add_exec_to_probe_trace_events(tevs, ntevs, module);
else if (module)
/* Currently ref_reloc_sym based probe is not for drivers */
- ret = add_module_to_probe_trace_events(tevs, ntevs, module);
+ ret = post_process_module_probe_trace_events(tevs, ntevs,
+ module, dinfo);
else
ret = post_process_kernel_probe_trace_events(tevs, ntevs);
}
}
- debuginfo__delete(dinfo);
-
if (ntevs > 0) { /* Succeeded to find trace events */
pr_debug("Found %d probe_trace_events.\n", ntevs);
ret = post_process_probe_trace_events(pev, *tevs, ntevs,
- pev->target, pev->uprobes);
+ pev->target, pev->uprobes, dinfo);
if (ret < 0 || ret == ntevs) {
+ pr_debug("Post processing failed or all events are skipped. (%d)\n", ret);
clear_probe_trace_events(*tevs, ntevs);
zfree(tevs);
+ ntevs = 0;
}
- if (ret != ntevs)
- return ret < 0 ? ret : ntevs;
- ntevs = 0;
- /* Fall through */
}
+ debuginfo__delete(dinfo);
+
if (ntevs == 0) { /* No error but failed to find probe point. */
pr_warning("Probe point '%s' not found.\n",
synthesize_perf_probe_point(&pev->point));
return -ENOENT;
- }
- /* Error path : ntevs < 0 */
- pr_debug("An error occurred in debuginfo analysis (%d).\n", ntevs);
- if (ntevs < 0) {
+ } else if (ntevs < 0) {
+ /* Error path : ntevs < 0 */
+ pr_debug("An error occurred in debuginfo analysis (%d).\n", ntevs);
if (ntevs == -EBADF)
pr_warning("Warning: No dwarf info found in the vmlinux - "
"please rebuild kernel with CONFIG_DEBUG_INFO=y.\n");
}
out:
- put_target_map(map, pev->uprobes);
+ map__put(map);
free(syms);
return ret;
return ret;
/* Get a symbol map */
- if (user)
- map = dso__new_map(target);
- else
- map = kernel_get_module_map(target);
+ map = get_target_map(target, user);
if (!map) {
pr_err("Failed to get a map for %s\n", (target) ? : "kernel");
return -EINVAL;
}
end:
- if (user) {
- map__put(map);
- }
+ map__put(map);
exit_probe_symbol_maps();
return ret;
}
/* For the kernel module, we need a special code to get a DIE */
-static int debuginfo__get_text_offset(struct debuginfo *dbg, Dwarf_Addr *offs)
+int debuginfo__get_text_offset(struct debuginfo *dbg, Dwarf_Addr *offs,
+ bool adjust_offset)
{
int n, i;
Elf32_Word shndx;
if (!shdr)
return -ENOENT;
*offs = shdr->sh_addr;
+ if (adjust_offset)
+ *offs -= shdr->sh_offset;
}
}
return 0;
Dwarf_Addr _addr = 0, baseaddr = 0;
const char *fname = NULL, *func = NULL, *basefunc = NULL, *tmp;
int baseline = 0, lineno = 0, ret = 0;
- bool reloc = false;
-retry:
+ /* We always need to relocate the address for aranges */
+ if (debuginfo__get_text_offset(dbg, &baseaddr, false) == 0)
+ addr += baseaddr;
/* Find cu die */
if (!dwarf_addrdie(dbg->dbg, (Dwarf_Addr)addr, &cudie)) {
- if (!reloc && debuginfo__get_text_offset(dbg, &baseaddr) == 0) {
- addr += baseaddr;
- reloc = true;
- goto retry;
- }
pr_warning("Failed to find debug information for address %lx\n",
addr);
ret = -EINVAL;
int debuginfo__find_probe_point(struct debuginfo *dbg, unsigned long addr,
struct perf_probe_point *ppt);
+int debuginfo__get_text_offset(struct debuginfo *dbg, Dwarf_Addr *offs,
+ bool adjust_offset);
+
/* Find a line range */
int debuginfo__find_line_range(struct debuginfo *dbg, struct line_range *lr);
break;
} else {
int n = namesz + descsz;
+
+ if (n > (int)sizeof(bf)) {
+ n = sizeof(bf);
+ pr_debug("%s: truncating reading of build id in sysfs file %s: n_namesz=%u, n_descsz=%u.\n",
+ __func__, filename, nhdr.n_namesz, nhdr.n_descsz);
+ }
if (read(fd, bf, n) != n)
break;
}
done;
@# Ask all targets to emit their test scripts
- echo "#!/bin/bash" > $(ALL_SCRIPT)
+ echo "#!/bin/sh" > $(ALL_SCRIPT)
echo "cd \$$(dirname \$$0)" >> $(ALL_SCRIPT)
echo "ROOT=\$$PWD" >> $(ALL_SCRIPT)
-#!/bin/bash
+#!/bin/sh
SRC_TREE=../../../../
-#!/bin/bash
+#!/bin/sh
echo "--------------------"
echo "running socket test"
FAIL_IF(ebb_event_enable(&event));
- mtspr(SPRN_PMC1, pmc_sample_period(sample_period));
+ mtspr(SPRN_PMC2, pmc_sample_period(sample_period));
mtspr(SPRN_PMC5, 0);
mtspr(SPRN_PMC6, 0);
tracing_off();
close_test_fds();
- printf("test %2d PASSED (itertation %d)\n", test_nr, iteration_nr);
+ printf("test %2d PASSED (iteration %d)\n", test_nr, iteration_nr);
dprintf1("======================\n\n");
}
iteration_nr++;
#define VMEXIT_CYCLES 500
#define VMENTRY_CYCLES 500
+#elif defined(__s390x__)
+static inline void wait_cycles(unsigned long long cycles)
+{
+ asm volatile("0: brctg %0,0b" : : "d" (cycles));
+}
+
+/* tweak me */
+#define VMEXIT_CYCLES 200
+#define VMENTRY_CYCLES 200
+
#else
static inline void wait_cycles(unsigned long long cycles)
{
/* Is there a portable way to do this? */
#if defined(__x86_64__) || defined(__i386__)
#define cpu_relax() asm ("rep; nop" ::: "memory")
+#elif defined(__s390x__)
+#define cpu_relax() barrier()
#else
#define cpu_relax() assert(0)
#endif
#!/bin/sh
+CPUS_ONLINE=$(lscpu --online -p=cpu|grep -v -e '#')
#use last CPU for host. Why not the first?
#many devices tend to use cpu0 by default so
#it tends to be busier
-HOST_AFFINITY=$(lscpu -p=cpu | tail -1)
+HOST_AFFINITY=$(echo "${CPUS_ONLINE}"|tail -n 1)
#run command on all cpus
-for cpu in $(seq 0 $HOST_AFFINITY)
+for cpu in $CPUS_ONLINE
do
#Don't run guest and host on same CPU
#It actually works ok if using signalling
#include <clocksource/arm_arch_timer.h>
#include <asm/arch_timer.h>
+#include <asm/kvm_hyp.h>
#include <kvm/arm_vgic.h>
#include <kvm/arm_arch_timer.h>
struct kvm_vcpu *vcpu;
vcpu = container_of(work, struct kvm_vcpu, arch.timer_cpu.expired);
- vcpu->arch.timer_cpu.armed = false;
-
- WARN_ON(!kvm_timer_should_fire(vcpu));
/*
* If the vcpu is blocked we want to wake it up so that it will see
{
kvm->arch.timer.cntvoff = kvm_phys_timer_read();
}
+
+/*
+ * On VHE system, we only need to configure trap on physical timer and counter
+ * accesses in EL0 and EL1 once, not for every world switch.
+ * The host kernel runs at EL2 with HCR_EL2.TGE == 1,
+ * and this makes those bits have no effect for the host kernel execution.
+ */
+void kvm_timer_init_vhe(void)
+{
+ /* When HCR_EL2.E2H ==1, EL1PCEN and EL1PCTEN are shifted by 10 */
+ u32 cnthctl_shift = 10;
+ u64 val;
+
+ /*
+ * Disallow physical timer access for the guest.
+ * Physical counter access is allowed.
+ */
+ val = read_sysreg(cnthctl_el2);
+ val &= ~(CNTHCTL_EL1PCEN << cnthctl_shift);
+ val |= (CNTHCTL_EL1PCTEN << cnthctl_shift);
+ write_sysreg(val, cnthctl_el2);
+}
/* Disable the virtual timer */
write_sysreg_el0(0, cntv_ctl);
- /* Allow physical timer/counter access for the host */
- val = read_sysreg(cnthctl_el2);
- val |= CNTHCTL_EL1PCTEN | CNTHCTL_EL1PCEN;
- write_sysreg(val, cnthctl_el2);
+ /*
+ * We don't need to do this for VHE since the host kernel runs in EL2
+ * with HCR_EL2.TGE ==1, which makes those bits have no impact.
+ */
+ if (!has_vhe()) {
+ /* Allow physical timer/counter access for the host */
+ val = read_sysreg(cnthctl_el2);
+ val |= CNTHCTL_EL1PCTEN | CNTHCTL_EL1PCEN;
+ write_sysreg(val, cnthctl_el2);
+ }
/* Clear cntvoff for the host */
write_sysreg(0, cntvoff_el2);
struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
u64 val;
- /*
- * Disallow physical timer access for the guest
- * Physical counter access is allowed
- */
- val = read_sysreg(cnthctl_el2);
- val &= ~CNTHCTL_EL1PCEN;
- val |= CNTHCTL_EL1PCTEN;
- write_sysreg(val, cnthctl_el2);
+ /* Those bits are already configured at boot on VHE-system */
+ if (!has_vhe()) {
+ /*
+ * Disallow physical timer access for the guest
+ * Physical counter access is allowed
+ */
+ val = read_sysreg(cnthctl_el2);
+ val &= ~CNTHCTL_EL1PCEN;
+ val |= CNTHCTL_EL1PCTEN;
+ write_sysreg(val, cnthctl_el2);
+ }
if (timer->enabled) {
write_sysreg(kvm->arch.timer.cntvoff, cntvoff_el2);
{
struct vgic_dist *dist = &kvm->arch.vgic;
- mutex_lock(&kvm->lock);
-
dist->ready = false;
dist->initialized = false;
kfree(dist->spis);
dist->nr_spis = 0;
-
- mutex_unlock(&kvm->lock);
}
void kvm_vgic_vcpu_destroy(struct kvm_vcpu *vcpu)
INIT_LIST_HEAD(&vgic_cpu->ap_list_head);
}
-void kvm_vgic_destroy(struct kvm *kvm)
+/* To be called with kvm->lock held */
+static void __kvm_vgic_destroy(struct kvm *kvm)
{
struct kvm_vcpu *vcpu;
int i;
kvm_vgic_vcpu_destroy(vcpu);
}
+void kvm_vgic_destroy(struct kvm *kvm)
+{
+ mutex_lock(&kvm->lock);
+ __kvm_vgic_destroy(kvm);
+ mutex_unlock(&kvm->lock);
+}
+
/**
* vgic_lazy_init: Lazy init is only allowed if the GIC exposed to the guest
* is a GICv2. A GICv3 must be explicitly initialized by the guest using the
ret = vgic_v2_map_resources(kvm);
else
ret = vgic_v3_map_resources(kvm);
+
+ if (ret)
+ __kvm_vgic_destroy(kvm);
+
out:
mutex_unlock(&kvm->lock);
return ret;
dist->ready = true;
out:
- if (ret)
- kvm_vgic_destroy(kvm);
return ret;
}
dist->ready = true;
out:
- if (ret)
- kvm_vgic_destroy(kvm);
return ret;
}
mutex_lock(&lock);
list_for_each_entry(tmp, &consumers, node) {
- if (tmp->token == consumer->token) {
+ if (tmp->token == consumer->token || tmp == consumer) {
mutex_unlock(&lock);
module_put(THIS_MODULE);
return -EBUSY;
mutex_lock(&lock);
list_for_each_entry(tmp, &consumers, node) {
- if (tmp->token != consumer->token)
+ if (tmp != consumer)
continue;
list_for_each_entry(producer, &producers, node) {
projects / linux-block.git / commitdiff